mac安裝apache flink
本文使用mac平臺專屬工具homebrew安裝apache flink,並簡單啟動一個本地叢集(local cluster)。
安裝flink
mac上可以直接使用homebrew的命令$ brew install apache-flink安裝flink:
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> New Formulae
curl-openssl gitmoji kubespy qalculate-gtk
fx healpix maven@3.5
==> Updated Formulae
openssl ✔ libphonenumber
sqlite ✔ libpq
abcm2ps libpqxx
abyss libqalculate
......
......
.....
ibhttpserver yq
libimagequant z3
liblcf zimg
libomp
==> Renamed Formulae
gutenberg -> zola mat -> mat2
==> Deleted Formulae
maven@3.0 maven@3.1
==> Downloading https://www.apache.org/dyn/closer.lua?path=flink/flink-1.6.2/fli
==> Downloading from http://mirror.bit.edu.cn/apache/flink/flink-1.6.2/flink-1.6
######################################################################## 100.0%
? /usr/local/Cellar/apache-flink/1.6.2: 111 files, 324MB, built in 4 minutes 46 seconds
安裝完成以後使用$ flink --version命令檢視版本，終端輸出：
Version: 1.6.2, Commit ID: 3456ad0
flink專案結構
flink 1.6.2專案資料夾目錄結構如下：
.
├── INSTALL_RECEIPT.json
├── LICENSE
├── NOTICE
├── README.txt
├── bin
│ └── flink
└── libexec
├── bin
│ ├── config.sh
│ ├── flink
│ ├── flink-console.sh
│ ├── flink-daemon.sh
│ ├── historyserver.sh
│ ├── jobmanager.sh
│ ├── mesos-appmaster-job.sh
│ ├── mesos-appmaster.sh
│ ├── mesos-taskmanager.sh
│ ├── pyflink-stream.sh
│ ├── pyflink.sh
│ ├── sql-client.sh
│ ├── standalone-job.sh
│ ├── start-cluster.sh
│ ├── start-scala-shell.sh
│ ├── start-zookeeper-quorum.sh
│ ├── stop-cluster.sh
│ ├── stop-zookeeper-quorum.sh
│ ├── taskmanager.sh
│ ├── yarn-session.sh
│ └── zookeeper.sh
├── conf
│ ├── flink-conf.yaml
│ ├── log4j-cli.properties
│ ├── log4j-console.properties
│ ├── log4j-yarn-session.properties
│ ├── log4j.properties
│ ├── logback-console.xml
│ ├── logback-yarn.xml
│ ├── logback.xml
│ ├── masters
│ ├── slaves
│ ├── sql-client-defaults.yaml
│ └── zoo.cfg
├── examples
│ ├── batch
│ │ ├── ConnectedComponents.jar
│ │ ├── DistCp.jar
│ │ ├── EnumTriangles.jar
│ │ ├── KMeans.jar
│ │ ├── PageRank.jar
│ │ ├── TransitiveClosure.jar
│ │ ├── WebLogAnalysis.jar
│ │ └── WordCount.jar
│ ├── gelly
│ │ └── flink-gelly-examples_2.11-1.6.2.jar
│ ├── python
│ │ ├── batch
│ │ │ ├── TPCHQuery10.py
│ │ │ ├── TPCHQuery3.py
│ │ │ ├── TriangleEnumeration.py
│ │ │ ├── WebLogAnalysis.py
│ │ │ ├── WordCount.py
│ │ │ └── __init__.py
│ │ └── streaming
│ │ ├── fibonacci.py
│ │ └── word_count.py
│ └── streaming
│ ├── IncrementalLearning.jar
│ ├── Iteration.jar
│ ├── Kafka010Example.jar
│ ├── SessionWindowing.jar
│ ├── SocketWindowWordCount.jar
│ ├── StateMachineExample.jar
│ ├── TopSpeedWindowing.jar
│ ├── Twitter.jar
│ ├── WindowJoin.jar
│ └── WordCount.jar
├── lib
│ ├── flink-dist_2.11-1.6.2.jar
│ ├── flink-python_2.11-1.6.2.jar
│ ├── flink-shaded-hadoop2-uber-1.6.2.jar
│ ├── log4j-1.2.17.jar
│ └── slf4j-log4j12-1.7.7.jar
├── libexec
│ ├── config.sh
│ ├── flink
│ ├── flink-console.sh
│ ├── flink-daemon.sh
│ ├── historyserver.sh
│ ├── jobmanager.sh
│ ├── mesos-appmaster-job.sh
│ ├── mesos-appmaster.sh
│ ├── mesos-taskmanager.sh
│ ├── pyflink-stream.sh
│ ├── pyflink.sh
│ ├── sql-client.sh
│ ├── standalone-job.sh
│ ├── start-cluster.sh
│ ├── start-scala-shell.sh
│ ├── start-zookeeper-quorum.sh
│ ├── stop-cluster.sh
│ ├── stop-zookeeper-quorum.sh
│ ├── taskmanager.sh
│ ├── yarn-session.sh
│ └── zookeeper.sh
├── log
│ └── flink-chenxin-client-MacBookPro.log
└── opt
├── flink-cep-scala_2.11-1.6.2.jar
├── flink-cep_2.11-1.6.2.jar
├── flink-gelly-scala_2.11-1.6.2.jar
├── flink-gelly_2.11-1.6.2.jar
├── flink-metrics-datadog-1.6.2.jar
├── flink-metrics-dropwizard-1.6.2.jar
├── flink-metrics-ganglia-1.6.2.jar
├── flink-metrics-graphite-1.6.2.jar
├── flink-metrics-prometheus-1.6.2.jar
├── flink-metrics-slf4j-1.6.2.jar
├── flink-metrics-statsd-1.6.2.jar
├── flink-ml_2.11-1.6.2.jar
├── flink-queryable-state-runtime_2.11-1.6.2.jar
├── flink-s3-fs-hadoop-1.6.2.jar
├── flink-s3-fs-presto-1.6.2.jar
├── flink-sql-client-1.6.2.jar
├── flink-streaming-python_2.11-1.6.2.jar
├── flink-swift-fs-hadoop-1.6.2.jar
└── flink-table_2.11-1.6.2.jar
15 directories, 111 files
啟動本地叢集
執行命令$ cd /usr/local/cellar/apache-flink/1.6.2/libexec進入libexec資料夾，執行命令$ ./bin/start-cluster.sh啟動本地叢集(local cluster),終端輸出
Starting cluster.
Starting standalonesession daemon on host MacBookPro.
Starting taskexecutor daemon on host MacBookPro.
這些資訊太粗略了，去/usr/local/cellar/apache-flink/1.6.2/libexec/log下檢視flink-chenxin-standalonesession-0-MacBookPro.log，日誌如下：
2018-11-28 21:33:41,089 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --------------------------------------------------------------------------------
2018-11-28 21:33:41,091 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting StandaloneSessionClusterEntrypoint (Version: 1.6.2, Rev:3456ad0, Date:17.10.2018 @ 18:46:46 GMT)
2018-11-28 21:33:41,091 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - OS current user: chenxin
2018-11-28 21:33:41,384 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-11-28 21:33:41,451 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Current Hadoop/Kerberos user: chenxin
2018-11-28 21:33:41,451 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.8/25.92-b14
2018-11-28 21:33:41,451 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Maximum heap size: 981 MiBytes
2018-11-28 21:33:41,451 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JAVA_HOME: /Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home
2018-11-28 21:33:41,453 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Hadoop version: 2.7.5
2018-11-28 21:33:41,453 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM Options:
2018-11-28 21:33:41,453 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xms1024m
2018-11-28 21:33:41,453 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xmx1024m
2018-11-28 21:33:41,453 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Dlog.file=/usr/local/Cellar/apache-flink/1.6.2/libexec/log/flink-chenxin-standalonesession-0-MacBookPro.log
2018-11-28 21:33:41,453 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Dlog4j.configuration=file:/usr/local/Cellar/apache-flink/1.6.2/libexec/conf/log4j.properties
2018-11-28 21:33:41,453 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Dlogback.configurationFile=file:/usr/local/Cellar/apache-flink/1.6.2/libexec/conf/logback.xml
2018-11-28 21:33:41,453 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Program Arguments:
2018-11-28 21:33:41,454 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --configDir
2018-11-28 21:33:41,454 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - /usr/local/Cellar/apache-flink/1.6.2/libexec/conf
2018-11-28 21:33:41,454 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --executionMode
2018-11-28 21:33:41,454 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - cluster
2018-11-28 21:33:41,454 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Classpath: /usr/local/Cellar/apache-flink/1.6.2/libexec/lib/flink-python_2.11-1.6.2.jar:/usr/local/Cellar/apache-flink/1.6.2/libexec/lib/flink-shaded-hadoop2-uber-1.6.2.jar:/usr/local/Cellar/apache-flink/1.6.2/libexec/lib/log4j-1.2.17.jar:/usr/local/Cellar/apache-flink/1.6.2/libexec/lib/slf4j-log4j12-1.7.7.jar:/usr/local/Cellar/apache-flink/1.6.2/libexec/lib/flink-dist_2.11-1.6.2.jar:::
2018-11-28 21:33:41,454 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --------------------------------------------------------------------------------
2018-11-28 21:33:41,455 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Registered UNIX signal handlers for [TERM, HUP, INT]
2018-11-28 21:33:41,466 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.address, localhost
2018-11-28 21:33:41,467 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.rpc.port, 6123
2018-11-28 21:33:41,467 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.heap.size, 1024m
2018-11-28 21:33:41,467 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.heap.size, 1024m
2018-11-28 21:33:41,467 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: taskmanager.numberOfTaskSlots, 1
2018-11-28 21:33:41,467 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: parallelism.default, 1
2018-11-28 21:33:41,467 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: rest.port, 8081
2018-11-28 21:33:41,482 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting StandaloneSessionClusterEntrypoint.
2018-11-28 21:33:41,482 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Install default filesystem.
2018-11-28 21:33:41,524 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Install security context.
2018-11-28 21:33:41,558 INFO org.apache.flink.runtime.security.modules.HadoopModule - Hadoop user set to chenxin (auth:SIMPLE)
2018-11-28 21:33:41,586 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Initializing cluster services.
2018-11-28 21:33:41,609 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Trying to start actor system at localhost:6123
2018-11-28 21:33:42,114 INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
2018-11-28 21:33:42,171 INFO akka.remote.Remoting - Starting remoting
2018-11-28 21:33:42,354 INFO akka.remote.Remoting - Remoting started; listening on addresses :[akka.tcp://flink@localhost:6123]
2018-11-28 21:33:42,363 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Actor system started at akka.tcp://flink@localhost:6123
2018-11-28 21:33:42,380 WARN org.apache.flink.configuration.Configuration - Config uses deprecated configuration key 'jobmanager.rpc.address' instead of proper key 'rest.address'
2018-11-28 21:33:42,388 INFO org.apache.flink.runtime.blob.BlobServer - Created BLOB server storage directory /var/folders/pz/1sq2sn2922d0bzrpycsht1d40000gn/T/blobStore-a34dc063-20a2-4f91-9853-d8852f0e6229
2018-11-28 21:33:42,392 INFO org.apache.flink.runtime.blob.BlobServer - Started BLOB server at 0.0.0.0:58832 - max concurrent requests: 50 - max backlog: 1000
2018-11-28 21:33:42,411 INFO org.apache.flink.runtime.metrics.MetricRegistryImpl - No metrics reporter configured, no metrics will be exposed/reported.
2018-11-28 21:33:42,416 INFO org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore - Initializing FileArchivedExecutionGraphStore: Storage directory /var/folders/pz/1sq2sn2922d0bzrpycsht1d40000gn/T/executionGraphStore-d014dd84-3370-482c-bf5d-a46269ce75a8, expiration time 3600000, maximum cache size 52428800 bytes.
2018-11-28 21:33:42,446 INFO org.apache.flink.runtime.blob.TransientBlobCache - Created BLOB cache storage directory /var/folders/pz/1sq2sn2922d0bzrpycsht1d40000gn/T/blobStore-2a864ac9-2728-4b96-9398-721b756c399c
2018-11-28 21:33:42,454 WARN org.apache.flink.configuration.Configuration - Config uses deprecated configuration key 'jobmanager.rpc.address' instead of proper key 'rest.address'
2018-11-28 21:33:42,457 WARN org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Upload directory /var/folders/pz/1sq2sn2922d0bzrpycsht1d40000gn/T/flink-web-572c3581-6395-4a0f-b0b6-673945ce72c8/flink-web-upload does not exist, or has been deleted externally. Previously uploaded files are no longer available.
2018-11-28 21:33:42,458 INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Created directory /var/folders/pz/1sq2sn2922d0bzrpycsht1d40000gn/T/flink-web-572c3581-6395-4a0f-b0b6-673945ce72c8/flink-web-upload for file uploads.
2018-11-28 21:33:42,463 INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Starting rest endpoint.
2018-11-28 21:33:42,702 INFO org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined location of main cluster component log file: /usr/local/Cellar/apache-flink/1.6.2/libexec/log/flink-chenxin-standalonesession-0-MacBookPro.log
2018-11-28 21:33:42,702 INFO org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined location of main cluster component stdout file: /usr/local/Cellar/apache-flink/1.6.2/libexec/log/flink-chenxin-standalonesession-0-MacBookPro.out
2018-11-28 21:33:42,850 INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Rest endpoint listening at localhost:8081
2018-11-28 21:33:42,850 INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - http://localhost:8081 was granted leadership with leaderSessionID=00000000-0000-0000-0000-000000000000
2018-11-28 21:33:42,850 INFO org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Web frontend listening at http://localhost:8081.
2018-11-28 21:33:42,927 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Starting RPC endpoint for org.apache.flink.runtime.resourcemanager.StandaloneResourceManager at akka://flink/user/resourcemanager .
2018-11-28 21:33:42,951 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/dispatcher .
2018-11-28 21:33:42,971 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - ResourceManager akka.tcp://flink@localhost:6123/user/resourcemanager was granted leadership with fencing token 00000000000000000000000000000000
2018-11-28 21:33:42,972 INFO org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager - Starting the SlotManager.
2018-11-28 21:33:42,990 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Dispatcher akka.tcp://flink@localhost:6123/user/dispatcher was granted leadership with fencing token 00000000-0000-0000-0000-000000000000
2018-11-28 21:33:42,992 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Recovering all persisted jobs.
2018-11-28 21:33:43,445 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - Registering TaskManager with ResourceID a1fc84e9fc2f53dad7998bb834eb15cd (akka.tcp://flink@macbookpro:58833/user/taskmanager_0) at ResourceManager
檢視任務頁面
如果在上一步正常啟動，那麼可以在瀏覽器中訪問http://localhost:8081/，會展示如下頁面：
1.png
點選左側的
Running Jobs會展示如下頁面：
2.png
點選左側的
Task Managers會展示如下頁面：
3.png
點選左側的
Job Manager會展示如下頁面：
4.png
點選左側的
Submit new Job會展示如下頁面：
5.png
可以在這裡提交job,一般一個job就是一個jar包。
參考
相關文章
- Mac下安裝apache waf模組modsecurityMacApache
- Apache安裝Apache
- 【高可用HA】Apache (2) —— Mac下安裝多個Apache Tomcat例項ApacheMacTomcat
- apache flink 簡介Apache
- 【高可用HA】Apache (1) —— Mac下安裝Apache Httpd到自定義路徑（非/etc/apache2）ApacheMachttpd
- 下載安裝ApacheApache
- apache編譯安裝Apache編譯
- Apache Ranger安裝部署ApacheRanger
- apache httpd安裝 配置Apachehttpd
- apache tomcat 安裝ApacheTomcat
- MAC 安裝 apache ab 壓力測試工具以及遇到的坑MacApache
- Mac下Apache+MySQL+PHP開發環境安裝過程MacApacheMySqlPHP開發環境
- Apache Flink核心技術Apache
- Flink 1.7.0 安裝、配置與使用
- 安裝apache及介紹Apache
- Linux下安裝ApacheLinuxApache
- 離線安裝 Apache AmbariApache
- 伺服器安裝Linux中安裝 Apache伺服器LinuxApache
- ffmpeg安裝之mac安裝Mac
- mac 安裝opencv homebrew安裝MacOpenCV
- Docker安裝flink及避坑指南Docker
- Apache-Flink深度解析-TableAPIApacheAPI
- Apache-Flink深度解析-概述Apache
- Eta中的Apache Flink示例Apache
- apache安裝和調整效能安裝實踐(轉)Apache
- CentOS 下編譯安裝 apacheCentOS編譯Apache
- PHP安裝及與apache整合PHPApache
- Apache安裝部署及工作模式Apache模式
- CentOS 6.4 編譯安裝 apacheCentOS編譯Apache
- 安裝apache+php記錄ApachePHP
- CentOS下Apache安裝總結CentOSApache
- apache、php、mysql安裝過程ApachePHPMySql
- 指令碼安裝apache+php指令碼ApachePHP
- apache 2.0.47 安裝筆記(轉)Apache筆記
- Apache2.2.21安裝圖解Apache圖解
- Mac Redis安裝與解除安裝MacRedis
- Mac 安裝 NginxMacNginx
- Mac 安裝 JDKMacJDK