Cassandra系列-3-叢集搭建
5. 搭建叢集
5.1 建立使用者,目錄
--logon as root
mkdir -p /nosql/cassandra
groupadd nosqlg
useradd nosql -g nosqlg
chown -R nosql:nosqlg /nosql/cassandra
chmod -R 755 /nosql/cassandra
passwd nosql
5.2 複製JRE,複製Cassandra
ftp jre,cassandra軟體到/nosql/cassandra/目錄下:
[root@bocnet95 cassandra]# tar -zxf jre-7u4-linux-i586[1].tar.gz
[root@bocnet95 cassandra]# tar -zxf apache-cassandra-1.0.10-bin.tar.gz
chown -R nosql:nosqlg /nosql/cassandra
chmod -R 755 /nosql/cassandra
5.3 配置使用者.bash_profile
vi .bash_profile
-----------add following lines
JAVA_HOME=/nosql/cassandra/jre1.7.0_04
export JAVA_HOME
PATH=$JAVA_HOME/bin:$PATH:$HOME/bin
export PATH
------------
. ./.bash_profile
[nosql@bocnet95 ~]$ java -version
java version "1.7.0_04"
Java(TM) SE Runtime Environment (build 1.7.0_04-b20)
Java HotSpot(TM) Client VM (build 23.0-b21, mixed mode)
5.4 配置Cassandra
5.4.1配置cassandra.yaml
[nosql@rhel5 conf]$vi cassandra.yaml
----------------------------------------------------------
# directories where Cassandra should store data on disk.
data_file_directories:
- /nosql/cassandra/data
# commit log
commitlog_directory: /nosql/cassandra/commitlog
# saved caches
saved_caches_directory: /nosql/cassandra/saved_caches
cluster_name: 'MyCluster'
# TCP port, for commands and data
storage_port: 7000
# port for Thrift to listen for clients on
rpc_port: 9160
seeds: "22.11.97.96,22.11.97.96"
listen_address: 22.11.97.96
rpc_address: 22.11.97.96
----------------------------------------------------------
同時修改節點1上cassandra.yaml
seeds: "22.11.97.96,22.11.97.95"
listen_address: 22.11.97.95
rpc_address: 22.11.97.95
5.4.2配置log4j-server.properties
[nosql@rhel5 conf]$ vi log4j-server.properties
----------------------------------------------------------
# Edit the next line to point to your logs directory
log4j.appender.R.File=/nosql/cassandra/system.log
----------------------------------------------------------
5.4.3配置cassandra-env.sh
[nosql@rhel5 conf]$ vi cassandra-env.sh
MAX_HEAP_SIZE根據系統實體記憶體設定;HEAP_NEWSIZE設定為MAX_HEAP_SIZE/4;這兩個引數會自動設定,可以去掉註釋指定具體值.
----------------------------------------------------------
MAX_HEAP_SIZE="400M"
HEAP_NEWSIZE="100M"
----------------------------------------------------------
5.5 啟動Cassandra新增節點
在叢集內已經有節點啟動的情況下，啟動：
[nosql@rhel5 bin]$ ./cassandra
本節點system.log:
INFO [GossipStage:1] 2015-06-29 09:59:33,869 Gossiper.java (line 838) Node /22.11.97.96 is now part of the cluster
INFO [GossipStage:1] 2015-06-29 09:59:33,870 Gossiper.java (line 804) InetAddress /22.11.97.96 is now UP
...
INFO [FlushWriter:1] 2015-06-29 09:59:34,124 Memtable.java (line 283) Completed flushing /nosql/cassandra/data/system/LocationInfo-hd-13-Data.db (89 bytes)
INFO [CompactionExecutor:4] 2015-06-29 09:59:34,225 CompactionTask.java (line 220) Compacted to [/nosql/cassandra/data/system/LocationInfo-hd-14-Data.db,]. 749 to 343 (~45% of original) bytes for 3 keys at 0.000997MB/s. Time: 328ms.
INFO [MigrationStage:1] 2015-06-29 09:59:35,229 Migration.java (line 119) Applying migration e2998350-e468-11e3-0000-242d50cf1fff Add keyspace: test, rep strategy:NetworkTopologyStrategy{}, durable_writes: true
INFO [MigrationStage:1] 2015-06-29 09:59:35,231 ColumnFamilyStore.java (line 705) Enqueuing flush of serialized/live bytes, 1 ops)
...
INFO [MigrationStage:1] 2015-06-29 09:59:35,509 Migration.java (line 119) Applying migration 060ae350-e46a-11e3-0000-242d50cf1fff Add column family: org.apache.cassandra.config.CFMetaData@10ab6fa[cfId=1000,ksName=test,cfName=sessioninfo,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@72fc8c,mergeShardsChance=0.1,keyAlias=,column_metadata={},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={},bloomFilterFpChance=]
...
INFO [MigrationStage:1] 2015-06-29 09:59:35,776 Migration.java (line 119) Applying migration 37900280-e478-11e3-0000-242d50cf1fff Add column family: org.apache.cassandra.config.CFMetaData@1e2a2c0[cfId=1001,ksName=test,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.UTF8Type,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@2b2bd1,mergeShardsChance=0.1,keyAlias=,column_metadata={java.nio.HeapByteBuffer[pos=0 lim=10 cap=10]=ColumnDefinition{name=62697274685f64617465, validator=org.apache.cassandra.db.marshal.LongType, index_type=KEYS, index_name='users_birth_date_idx'}, java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]=ColumnDefinition{name=66756c6c5f6e616d65, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=null, index_name='null'}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={},bloomFilterFpChance=]
...
INFO [MigrationStage:1] 2015-06-29 09:59:36,043 SecondaryIndexManager.java (line 183) Creating new index : ColumnDefinition{name=62697274685f64617465, validator=org.apache.cassandra.db.marshal.LongType, index_type=KEYS, index_name='users_birth_date_idx'}
INFO [Creating index: users.users_birth_date_idx] 2015-06-29 09:59:36,060 SecondaryIndex.java (line 150) Submitting index build of users.users_birth_date_idx for data in
INFO [MigrationStage:1] 2015-06-29 09:59:36,064 Migration.java (line 119) Applying migration 23e610b0-e47a-11e3-0000-242d50cf1fff Update column family to org.apache.cassandra.config.CFMetaData@1b83ca2[cfId=1001,ksName=test,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.UTF8Type,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@94e6ee,mergeShardsChance=0.1,keyAlias=java.nio.HeapByteBuffer[pos=0 lim=3 cap=3],column_metadata={java.nio.HeapByteBuffer[pos=0 lim=10 cap=10]=ColumnDefinition{name=62697274685f64617465, validator=org.apache.cassandra.db.marshal.LongType, index_type=KEYS, index_name='users_birth_date_idx'}, java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]=ColumnDefinition{name=66756c6c5f6e616d65, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=null, index_name='null'}, java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=7374617465, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=KEYS, index_name='users_state_idx'}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={},bloomFilterFpChance=]
啟動比較早的節點system.log:
INFO [GossipStage:1] 2014-05-27 10:15:35,336 Gossiper.java (line 838) Node /22.11.97.95 is now part of the cluster
INFO [GossipStage:1] 2014-05-27 10:15:35,336 Gossiper.java (line 804) InetAddress /22.11.97.95 is now UP
INFO [GossipStage:1] 2014-05-27 10:15:35,349 ColumnFamilyStore.java (line 705) Enqueuing flush of serialized/live bytes, 1 ops)
INFO [FlushWriter:2] 2014-05-27 10:15:35,350 Memtable.java (line 246) Writing serialized/live bytes, 1 ops)
INFO [FlushWriter:2] 2014-05-27 10:15:35,359 Memtable.java (line 283) Completed flushing /nosql/cassandra/data/system/LocationInfo-hd-30-Data.db (89 bytes)
[nosql@rhel5 bin]$ ./nodetool -h rhel5 ring
Address DC Rack Status State Load Owns Token
159495876378727078405590336465058029385
22.11.97.95 datacenter1 rack1 Up Normal 36.02 KB 76.93% 120248406591972837751075008114949334676
22.11.97.96 datacenter1 rack1 Up Normal 92.22 KB 23.07% 159495876378727078405590336465058029385
5.6 客戶端驗證
--查詢資料
--插入，跨節點查詢
[nosql@bocnet95 bin]$ ./cassandra-cli -h 22.11.97.95 -p 9160
Connected to: "MyCluster" on 22.11.97.95/9160
Welcome to Cassandra CLI version 1.0.10
Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.
[default@unknown] use test;
Authenticated to keyspace: test
[default@test] get users[utf8('userc')];
Returned 0 results.
Elapsed time: 99 msec(s).
[default@test] get users[utf8('usera')];
=> (column=birth_date, value=1975, timestamp=1401069384452000)
=> (column=full_name, value=Brandon Sanderson, timestamp=1401069384437000)
=> (column=state, value=FL, timestamp=1401069751547000)
Returned 3 results.
Elapsed time: 40 msec(s).
[default@test] set users[utf8('userc')][utf8('full_name')]='Howard Tayler';
Value inserted.
Elapsed time: 57 msec(s).
[default@test] set users[utf8('userc')][utf8('birth_date')]=1968;
Value inserted.
Elapsed time: 4 msec(s).
[default@test] get users[utf8('userc')];
=> (column=birth_date, value=1968, timestamp=1435545861556000)
=> (column=full_name, value=Howard Tayler, timestamp=1435545861495000)
Returned 2 results.
Elapsed time: 6 msec(s).
在節點1查詢：
[default@test] get users[utf8('userc')];
=> (column=birth_date, value=1968, timestamp=1435545861556000)
=> (column=full_name, value=Howard Tayler, timestamp=1435545861495000)
Returned 2 results.
Elapsed time: 15 msec(s).
5.7 停止本節點服務
[nosql@bocnet95 bin]$ ps -ef|grep cass
nosql 13402 1 0 09:59 pts/2 00:00:02 /nosql/cassandra/jre1.7.0_04/bin/java -ea -javaagent:./../lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms400M -Xmx400M -Xmn100M -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dlog4j.configuration=log4j-server.properties -Dlog4j.defaultInitOverride=true -cp ./../conf:./../build/classes/main:./../build/classes/thrift:./../lib/antlr-3.2.jar:./../lib/apache-cassandra-1.0.10.jar:./../lib/apache-cassandra-clientutil-1.0.10.jar:./../lib/apache-cassandra-thrift-1.0.10.jar:./../lib/avro-1.4.0-fixes.jar:./../lib/avro-1.4.0-sources-fixes.jar:./../lib/commons-cli-1.1.jar:./../lib/commons-codec-1.2.jar:./../lib/commons-lang-2.4.jar:./../lib/compress-lzf-0.8.4.jar:./../lib/concurrentlinkedhashmap-lru-1.2.jar:./../lib/guava-r08.jar:./../lib/high-scale-lib-1.1.2.jar:./../lib/jackson-core-asl-1.4.0.jar:./../lib/jackson-mapper-asl-1.4.0.jar:./../lib/jamm-0.2.5.jar:./../lib/jline-0.9.94.jar:./../lib/json-simple-1.1.jar:./../lib/libthrift-0.6.jar:./../lib/log4j-1.2.16.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/slf4j-api-1.6.1.jar:./../lib/slf4j-log4j12-1.6.1.jar:./../lib/snakeyaml-1.6.jar:./../lib/snappy-java-1.0.4.1.jar org.apache.cassandra.thrift.CassandraDaemon
nosql 20083 18237 0 10:46 pts/2 00:00:00 grep cass
[nosql@bocnet95 bin]$ kill -9 13402
[nosql@bocnet95 conf]$ ../bin/nodetool -h bocnet95 ring
Error connection to remote JMX agent!
java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: bocnet95; nested exception is:
java.net.ConnectException: 拒絕連線]
at javax.management.remote.rmi.RMIConnector.connect(Unknown Source)
at javax.management.remote.JMXConnectorFactory.connect(Unknown Source)
at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:141)
at org.apache.cassandra.tools.NodeProbe.
at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:623)
Caused by: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: bocnet95; nested exception is:
java.net.ConnectException: 拒絕連線]
at com.sun.jndi.rmi.registry.RegistryContext.lookup(Unknown Source)
at com.sun.jndi.toolkit.url.GenericURLContext.lookup(Unknown Source)
at javax.naming.InitialContext.lookup(Unknown Source)
at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(Unknown Source)
at javax.management.remote.rmi.RMIConnector.findRMIServer(Unknown Source)
... 5 more
Caused by: java.rmi.ConnectException: Connection refused to host: bocnet95; nested exception is:
java.net.ConnectException: 拒絕連線
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(Unknown Source)
at sun.rmi.transport.tcp.TCPChannel.createConnection(Unknown Source)
at sun.rmi.transport.tcp.TCPChannel.newConnection(Unknown Source)
at sun.rmi.server.UnicastRef.newCall(Unknown Source)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
... 10 more
Caused by: java.net.ConnectException: 拒絕連線
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.
at java.net.Socket.
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(Unknown Source)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(Unknown Source)
... 15 more
--連線叢集記憶體活的節點可以查詢到叢集狀態
[nosql@bocnet95 conf]$ ../bin/nodetool -h rhel5 ring
Address DC Rack Status State Load Owns Token
159495876378727078405590336465058029385
22.11.97.95 datacenter1 rack1 Down Normal 82.99 KB 76.93% 120248406591972837751075008114949334676
22.11.97.96 datacenter1 rack1 Up Normal 92.22 KB 23.07% 159495876378727078405590336465058029385
--從本節點連線到叢集記憶體活的節點:
[nosql@bocnet95 bin]$ ./cassandra-cli -h 22.11.97.96 -p 9160
Connected to: "MyCluster" on 22.11.97.96/9160
Welcome to Cassandra CLI version 1.0.10
Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.
[default@unknown] use test;
--對於僅由存活節點儲存的資料,查詢報錯:
[default@test] get users[utf8('usera')];
=> (column=birth_date, value=1975, timestamp=1401069384452000)
=> (column=full_name, value=Brandon Sanderson, timestamp=1401069384437000)
=> (column=state, value=FL, timestamp=1401069751547000)
Returned 3 results.
Elapsed time: 25 msec(s).
--對於僅由失敗節點儲存的資料,查詢報錯:
[default@test] get users[utf8('userc')];
null
UnavailableException()
at org.apache.cassandra.thrift.Cassandra$get_slice_result.read(Cassandra.java:7764)
at org.apache.cassandra.thrift.Cassandra$Client.recv_get_slice(Cassandra.java:570)
at org.apache.cassandra.thrift.Cassandra$Client.get_slice(Cassandra.java:542)
at org.apache.cassandra.cli.CliClient.doSlice(CliClient.java:467)
at org.apache.cassandra.cli.CliClient.executeGet(CliClient.java:603)
at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:209)
at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:220)
at org.apache.cassandra.cli.CliMain.main(CliMain.java:348)
--在節點1上查詢：
[default@test] get users[utf8('userc')];
null
UnavailableException()
at org.apache.cassandra.thrift.Cassandra$get_slice_result.read(Cassandra.java:7764)
at org.apache.cassandra.thrift.Cassandra$Client.recv_get_slice(Cassandra.java:570)
at org.apache.cassandra.thrift.Cassandra$Client.get_slice(Cassandra.java:542)
at org.apache.cassandra.cli.CliClient.doSlice(CliClient.java:467)
at org.apache.cassandra.cli.CliClient.executeGet(CliClient.java:603)
at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:209)
at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:220)
at org.apache.cassandra.cli.CliMain.main(CliMain.java:348)
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/18922393/viewspace-731634/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- Cassandra安裝及分散式叢集搭建分散式
- Redis系列：搭建Redis叢集(叢集模式)Redis模式
- Elasticsearch使用系列-Docker搭建Elasticsearch叢集ElasticsearchDocker
- 微服務Consul系列之叢集搭建微服務
- 搭建zookeeper叢集（偽叢集）
- redhat安裝雙節點cassandra叢集Redhat
- 叢集搭建
- zookeeper叢集及kafka叢集搭建Kafka
- redis叢集搭建Redis
- nacos 叢集搭建
- kafka叢集搭建Kafka
- Ambari叢集搭建
- 搭建ELK叢集
- RabbitMQ叢集搭建MQ
- HBASE叢集搭建
- zookeeper 叢集搭建
- mysql叢集搭建MySql
- Zookeeper叢集搭建
- 搭建 Redis 叢集Redis
- 搭建 zookeeper 叢集
- Elasticsearch 叢集搭建Elasticsearch
- mysql 叢集搭建MySql
- Storm叢集搭建ORM
- kafka系列二：多節點分散式叢集搭建Kafka分散式
- redis系列：叢集Redis
- 從0到1搭建spark叢集---企業叢集搭建Spark
- 在多節點的叢集上執行Cassandra
- linux下搭建ZooKeeper叢集（偽叢集）Linux
- Kafka學習之（五）搭建kafka叢集之Zookeeper叢集搭建Kafka
- 【環境搭建】RocketMQ叢集搭建MQ
- 【greenplum】greenplum叢集搭建
- ElasticSearch 7.8.1叢集搭建Elasticsearch
- MongoDB 分片叢集搭建MongoDB
- Docker Swarm 叢集搭建DockerSwarm
- Redis（5.0） 叢集搭建Redis
- MySQL MGR 叢集搭建MySql
- nacos叢集開箱搭建
- redis叢集的搭建Redis