乾貨！ClickHouse 24.x 叢集部署(去zookeeper方案)文末附看板

StarsL發表於2024-11-03

主機配置

3臺 12核 24G 2T儲存的伺服器部署去zookeeper模式的ClickHouse 24.X叢集。

Ubuntu（3臺伺服器都要執行安裝）

sudo apt-get install -y apt-transport-https ca-certificates curl gnupg
curl -fsSL 'https://packages.clickhouse.com/rpm/lts/repodata/repomd.xml.key' | sudo gpg --dearmor -o /usr/share/keyrings/clickhouse-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/clickhouse-keyring.gpg] https://packages.clickhouse.com/deb stable main" | sudo tee \
    /etc/apt/sources.list.d/clickhouse.list
sudo apt-get update
sudo apt-get install -y clickhouse-server clickhouse-client

# 注意安裝時會提示，配置好default使用者的密碼。

CentOS（3臺伺服器都要執行安裝）

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://packages.clickhouse.com/rpm/clickhouse.repo
sudo yum install -y clickhouse-server clickhouse-client

# 注意安裝時會提示，配置好default使用者的密碼。

配置密碼

如果安裝的時候沒有設定密碼，可以使用以下方式配置default使用者的密碼。

# 生成密碼(返回的第一行是明文，第二行是密文)
PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-'

# 以下是部分配置檔案請參考修改。
# vi /opt/clickhouse/etc/clickhouse-server/users.d/users.xml
<?xml version="1.0"?>
<clickhouse replace="true">
...
    <users>
        <default>
            <password remove='1' />
            <password_sha256_hex>【填寫生成的密碼密文】</password_sha256_hex>
            <access_management>1</access_management>
            <profile>default</profile>
            <networks>
...
</clickhouse>

修改/etc/hosts（3臺伺服器）

10.7.0.104      logs-clickhouse-0001
10.7.0.203      logs-clickhouse-0002
10.7.0.153      logs-clickhouse-0003

叢集最佳化與配置

/etc/clickhouse-server/config.d

  • custom.xml（3臺伺服器）

    最佳化效能的配置

<clickhouse>
    <timezone>Asia/Shanghai</timezone>
    <listen_host>0.0.0.0</listen_host>

    <max_connections>40960</max_connections>

    <max_concurrent_queries>20000</max_concurrent_queries>

    <max_thread_pool_size>20000</max_thread_pool_size>
    <background_pool_size>64</background_pool_size>
    <background_distributed_schedule_pool_size>64</background_distributed_schedule_pool_size>

    <max_table_size_to_drop>0</max_table_size_to_drop>
    <max_partition_size_to_drop>0</max_partition_size_to_drop>
</clickhouse>

  • clusters.xml（3臺伺服器）

    儲存結構化（非全文搜尋）日誌使用，3臺伺服器組成叢集，3分片0副本配置：

<clickhouse>
    <remote_servers>
        <!--這是叢集的名稱-->
        <opslogsch>
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <!--這是host配置的主機名-->
                    <host>logs-clickhouse-0001</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>【填寫密碼明文】</password>
                </replica>
            </shard>
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>logs-clickhouse-0002</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>【填寫密碼明文】</password>
                </replica>
            </shard>
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>logs-clickhouse-0003</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>【填寫密碼明文】</password>
                </replica>
            </shard>
        <!--這是叢集的名稱-->
        </opslogsch>
    </remote_servers>
</clickhouse>

  • clickhouse-server中已經整合了clickhouse-keeper，直接啟動clickhouse-server即可，所以不用再安裝zookeeper。官方建議在獨立的節點上執行 clickhouse-keeper，如果需要獨立節點安裝可以使用以下命令：

    sudo apt-get install -y clickhouse-keeper || sudo yum install -y clickhouse-keeper
sudo systemctl enable clickhouse-keeper
sudo systemctl start clickhouse-keeper

  • keeper.xml（3臺伺服器）

<clickhouse>
    <keeper_server>
        <tcp_port>9181</tcp_port>
        <!--以下行id每臺伺服器不能重複-->
        <server_id>1</server_id>
        <log_storage_path>/var/lib/clickhouse/coordination/log</log_storage_path>
        <snapshot_storage_path>/var/lib/clickhouse/coordination/snapshots</snapshot_storage_path>
    
        <coordination_settings>
            <operation_timeout_ms>10000</operation_timeout_ms>
            <session_timeout_ms>30000</session_timeout_ms>
            <raft_logs_level>warning</raft_logs_level>
        </coordination_settings>
    
        <raft_configuration>
            <server>
                <id>1</id>
                <hostname>logs-clickhouse-0001</hostname>
                <port>9444</port>
            </server>
            <server>
                <id>2</id>
                <hostname>logs-clickhouse-0002</hostname>
                <port>9444</port>
            </server>
            <server>
                <id>3</id>
                <hostname>logs-clickhouse-0003</hostname>
                <port>9444</port>
            </server>
        </raft_configuration>
    </keeper_server>
    <zookeeper>
        <node>
            <host>logs-clickhouse-0001</host>
            <port>9181</port>
        </node>
        <node>
            <host>logs-clickhouse-0002</host>
            <port>9181</port>
        </node>
        <node>
            <host>logs-clickhouse-0003</host>
            <port>9181</port>
        </node>
    </zookeeper>
</clickhouse>

/etc/clickhouse-server/users.d

  • custom.xml（3臺伺服器）
<clickhouse>
    <profiles>
        <default>
            <max_partitions_per_insert_block>3000</max_partitions_per_insert_block>         
        </default>
    </profiles>
</clickhouse>

啟動ClickHouse（3臺伺服器）

sudo systemctl enable clickhouse-keeper
sudo systemctl start clickhouse-keeper
sudo systemctl status clickhouse-keeper

檢查叢集狀態（任意伺服器執行）

clickhouse-server status
clickhouse-client --password
select * from system.clusters

測試叢集（任意伺服器執行）

# 在任意伺服器登入clickhouse
clickhouse-client --password
# 以下各語句加上 ON CLUSTER opslogsch 會在所有叢集伺服器同時執行
# 建立資料庫
CREATE DATABASE test ON CLUSTER opslogsch;
# 建立本地表
CREATE TABLE test.test_local ON CLUSTER opslogsch
(
    `id` Int32,
    `aa` String,
    `bb` String 
)
ENGINE = MergeTree PRIMARY KEY id;
# 建立分散式表
CREATE TABLE test.test_all ON CLUSTER opslogsch as test.test_local ENGINE = Distributed(opslogsch,test,test_local,rand());
# 寫本地表
INSERT INTO test.test_local (id,aa,bb)values(1,'a1','b1');
INSERT INTO test.test_local (id,aa,bb)values(1,'a2','b2');
INSERT INTO test.test_local (id,aa,bb)values(1,'a3','b3');
# 寫分散式表
INSERT INTO test.test_all (id,aa,bb)values(1,'x1','x1');
# 查分散式表
SELECT * from test.test_all
# 刪庫
drop DATABASE test ON CLUSTER opslogsch;

推薦:基於CH的nginx日誌分析看板：
你還用ES存請求日誌？CH+Vector打造最強Grafana日誌分析看板

在這裡插入圖片描述### 監控看板
Grafana增加ClickHouse資料來源後即可直接匯入CH監控看板
在這裡插入圖片描述

相關文章