cgroup、資源池、使用者的關係..涉及到GaussDB(DWS)的資源設定

华为云开发者联盟發表於2024-03-29

本文分享自華為雲社群《GaussDB(DWS)的cgroup、資源池、使用者的關係》,作者: nullptr_。

1. 前言

本文主要展示了DWS中cgroup、資源池、使用者之間的關係,從而對DWS的資源設定情況有個初步瞭解。

2. 相關物件建立指令碼

gs_ssh -c "gs_cgroup -cS ClassN1 -G wn1"
gs_ssh -c "gs_cgroup -cS ClassN1 -G wn2"
gs_ssh -c "gs_cgroup -cS ClassN2 -G wn3"
gs_ssh -c "gs_cgroup -cS ClassG1 -G wg1_1"
gs_ssh -c "gs_cgroup -cS ClassG1 -G wg1_2"
gs_ssh -c "gs_cgroup -cS ClassG2 -G wg2_1"
gs_ssh -c "gs_cgroup -cS ClassG2 -G wg2_2"

#建立資源池
gsql -d postgres -p 6000 -c  "create resource pool respool_1 with (control_group = 'ClassN1:wn1');"
gsql -d postgres -p 6000 -c  "create resource pool respool_2 with (control_group = 'ClassN1:wn2');"
gsql -d postgres -p 6000 -c  "create resource pool respool_3 with (control_group = 'ClassN2:wn3');"
gsql -d postgres -p 6000 -c  "create resource pool respool_4 with (control_group = 'ClassN2:wn3');"
gsql -d postgres -p 6000 -c  "create resource pool respool_grp_1 with (control_group = 'ClassG1');"
gsql -d postgres -p 6000 -c  "create resource pool respool_g1_job_1 with (control_group = 'ClassG1:wg1_1');"
gsql -d postgres -p 6000 -c  "create resource pool respool_g1_job_2 with (control_group = 'ClassG1:wg1_2');"
gsql -d postgres -p 6000 -c  "create resource pool respool_grp_2 with (control_group = 'ClassG2');"
gsql -d postgres -p 6000 -c  "create resource pool respool_g2_job_1 with (control_group = 'ClassG2:wg2_1');"
gsql -d postgres -p 6000 -c  "create resource pool respool_g2_job_2 with (control_group = 'ClassG2:wg2_2');"

#建立租戶,建立使用者
gsql -d postgres -p 6000 -c  "CREATE USER user_1 RESOURCE POOL 'respool_1' PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_2 RESOURCE POOL 'respool_2' PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_3 RESOURCE POOL 'respool_3' PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_4 PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_5 PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_grp_1 RESOURCE POOL 'respool_grp_1' PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_g1_job_1 RESOURCE POOL 'respool_g1_job_1' USER GROUP 'user_grp_1' PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_g1_job_2 RESOURCE POOL 'respool_g1_job_2' USER GROUP 'user_grp_1' PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_grp_2 RESOURCE POOL 'respool_grp_2' PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_g2_job_1 RESOURCE POOL 'respool_g2_job_1' USER GROUP 'user_grp_2' PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_g2_job_2 RESOURCE POOL 'respool_g2_job_2' USER GROUP 'user_grp_2' PASSWORD 'Gauss_ab1' ;"
gsql -d postgres -p 6000 -c  "CREATE USER user_grp_3 RESOURCE POOL 'respool_grp_1' PASSWORD 'Gauss_ab1' ;"

3. cgroup

GaussDB(DWS)資源負載管理的核心是資源池,而配置資源池首先要在環境中實現控制組Cgroups的設定。
Class控制組為資料庫業務執行所在的頂層控制組,叢集部署時會自動生成預設子Class控制組“DefaultClass”。DefaultClass的Medium控制組會含有系統觸發的作業在執行,該控制組不允許進行資源修改,且執行在該控制組上的作業不受資源管理的控制,所以推薦建立新的子Class及其Workload控制組來設定資源比例。

3.1 執行指令碼之後cgroup分佈情況如下

per910mas@xx:~> gs_cgroup -p

Top Group information is listed:
GID:   0 Type: Top    Percent(%): 1000( 50) Name: Root                  Cores: 0-103
GID:   1 Type: Top    Percent(%):  833( 83) Name: Gaussdb:per910mas     Cores: 0-103
GID:   2 Type: Top    Percent(%):  333( 40) Name: Backend               Cores: 0-103
GID:   3 Type: Top    Percent(%):  499( 60) Name: Class                 Cores: 0-103

Backend Group information is listed:
GID:   4 Type: BAKWD  Name: DefaultBackend   TopGID:   2 Percent(%): 266(80) Cores: 0-103
GID:   5 Type: BAKWD  Name: Vacuum           TopGID:   2 Percent(%):  66(20) Cores: 0-103

Class Group information is listed:
GID:  20 Type: CLASS  Name: DefaultClass     TopGID:   3 Percent(%):  99(20) MaxLevel: 1 RemPCT: 100 Cores: 0-103
GID:  21 Type: CLASS  Name: ClassN1          TopGID:   3 Percent(%):  99(20) MaxLevel: 3 RemPCT:  60 Cores: 0-103
GID:  22 Type: CLASS  Name: ClassN2          TopGID:   3 Percent(%):  99(20) MaxLevel: 2 RemPCT:  80 Cores: 0-103
GID:  23 Type: CLASS  Name: ClassG1          TopGID:   3 Percent(%):  99(20) MaxLevel: 3 RemPCT:  60 Cores: 0-103
GID:  24 Type: CLASS  Name: ClassG2          TopGID:   3 Percent(%):  99(20) MaxLevel: 3 RemPCT:  60 Cores: 0-103

Workload Group information is listed:
GID:  86 Type: DEFWD  Name: wn1:2            ClsGID:  21 Percent(%):  19(20) WDLevel:  2 Cores: 0-103
GID:  87 Type: DEFWD  Name: wn2:3            ClsGID:  21 Percent(%):  19(20) WDLevel:  3 Cores: 0-103
GID:  89 Type: DEFWD  Name: wn3:2            ClsGID:  22 Percent(%):  19(20) WDLevel:  2 Cores: 0-103
GID:  91 Type: DEFWD  Name: wg1_1:2          ClsGID:  23 Percent(%):  19(20) WDLevel:  2 Cores: 0-103
GID:  92 Type: DEFWD  Name: wg1_2:3          ClsGID:  23 Percent(%):  19(20) WDLevel:  3 Cores: 0-103
GID:  94 Type: DEFWD  Name: wg2_1:2          ClsGID:  24 Percent(%):  19(20) WDLevel:  2 Cores: 0-103
GID:  95 Type: DEFWD  Name: wg2_2:3          ClsGID:  24 Percent(%):  19(20) WDLevel:  3 Cores: 0-103

CM Group information is listed:

Timeshare Group information is listed:
GID: 724 Type: TSWD   Name: Low              Rate: 1
GID: 725 Type: TSWD   Name: Medium           Rate: 2
GID: 726 Type: TSWD   Name: High             Rate: 4
GID: 727 Type: TSWD   Name: Rush             Rate: 8

系統資源限制分為配額限額。預設情況下為配額

  • 配額:配額是一種比較靈活的控制方式,例如wn1:2的配額為20%,在正常情況下組內資源使用可以超過20%,如果在資源繁忙的情況下(使用率為100%)則資源嚴格按照配額的大小進行限制
  • 限額:限額則直接限制CPU使用的核數的範圍。
  • 配額&限額:則在CPU核數範圍內限制配額比例

4. 資源池

4.1 資源池分佈情況如下

postgres=# select oid,* from pg_resource_pool;
    oid     |   respool_name   | mem_percent | cpu_affinity |    control_group    | active_statements | max_dop | memory_limit |  parentid  | io_limits | io_priority |  nodegroup   | is_foreign | short_acc | except_rule | weight
------------+------------------+-------------+--------------+---------------------+-------------------+---------+--------------+------------+-----------+-------------+--------------+------------+-----------+-------------+--------
         10 | default_pool     |           0 |           -1 | DefaultClass:Medium |                -1 |      -1 | default      |          0 |         0 | None        | installation | f          | t         | None        |     -1
 2147585814 | respool_1        |           0 |           -1 | ClassN1:wn1         |                10 |      -1 | default      |          0 |         0 | None        | installation | f          | t         | None        |     -1
 2147585815 | respool_2        |           0 |           -1 | ClassN1:wn2         |                10 |      -1 | default      |          0 |         0 | None        | installation | f          | t         | None        |     -1
 2147585816 | respool_3        |           0 |           -1 | ClassN2:wn3         |                10 |      -1 | default      |          0 |         0 | None        | installation | f          | t         | None        |     -1
 2147585817 | respool_grp_1    |          20 |           -1 | ClassG1             |                10 |      -1 | default      |          0 |         0 | None        | installation | f          | t         | None        |     -1
 2147585818 | respool_g1_job_1 |          20 |           -1 | ClassG1:wg1_1       |                10 |      -1 | default      | 2147585817 |         0 | None        | installation | f          | t         | None        |     -1
 2147585819 | respool_g1_job_2 |          20 |           -1 | ClassG1:wg1_2       |                10 |      -1 | default      | 2147585817 |         0 | None        | installation | f          | t         | None        |     -1
 2147585820 | respool_grp_2    |          20 |           -1 | ClassG2             |                10 |      -1 | default      |          0 |         0 | None        | installation | f          | t         | None        |     -1
 2147585821 | respool_g2_job_1 |          20 |           -1 | ClassG2:wg2_1       |                10 |      -1 | default      | 2147585820 |         0 | None        | installation | f          | t         | None        |     -1
 2147585822 | respool_g2_job_2 |          20 |           -1 | ClassG2:wg2_2       |                10 |      -1 | default      | 2147585820 |         0 | None        | installation | f          | t         | None        |     -1
 2147586195 | respool_4        |           0 |           -1 | ClassN2:wn3         |                10 |      -1 | default      |          0 |         0 | None        | installation | f          | t         | None        |     -1
(11 rows)

4.1.1 組資源池限制

per910mas@xx:~> gsql -d postgres -p 6000 -c  "create resource pool respool_grp_3 with (control_group = 'ClassG1');"
ERROR:  resource pool with control_group ClassG1 has been existed in the two-layer resource pool list

4.1.2 業務資源池

資源池的記憶體資源計算mem_percent需要按照層級進行比例計算

4.1.3 預設資源池

如果開啟了資源管理功能,則系統會預設建立一個資源池default_pool。當一個會話或者使用者沒有指定關聯的資源池時,都會被預設關聯到default_pool。default_pool預設繫結DefaultClass:Medium控制組,同時併發和記憶體預設不管控,default_pool支援引數修改,但關聯default_pool的作業會受到max_active_statements全域性併發限制。當管理員執行運維操作不需要進行管控時,需要在執行SQL前執行SET session_respool=‘root’;將資源池切換至運維佇列,此時作業將不受控。

5. 使用者

5.1 使用者分佈情況

postgres=# select * from pg_user;、
    usename    |  usesysid  | usecreatedb | usesuper | usecatupd | userepl |  passwd  | valbegin | valuntil |     respool      |   parent   | spacelimit | useconfig | nodegroup | tempspacelimit | spillspacelimit
---------------+------------+-------------+----------+-----------+---------+----------+----------+----------+------------------+------------+------------+-----------+-----------+----------------+-----------------
 per910mas     |         10 | t           | t        | t         | t       | ******** |          |          | default_pool     |          0 |            |           |           |                |
 u1            | 2147558961 | f           | f        | f         | f       | ******** |          |          | default_pool     |          0 |            |           |           |                |
 user_1        | 2147585823 | f           | f        | f         | f       | ******** |          |          | respool_1        |          0 |            |           |           |                |
 user_2        | 2147585827 | f           | f        | f         | f       | ******** |          |          | respool_2        |          0 |            |           |           |                |
 user_3        | 2147585831 | f           | f        | f         | f       | ******** |          |          | respool_3        |          0 |            |           |           |                |
 user_4        | 2147585835 | f           | f        | f         | f       | ******** |          |          | default_pool     |          0 |            |           |           |                |
 user_5        | 2147585839 | f           | f        | f         | f       | ******** |          |          | default_pool     |          0 |            |           |           |                |
 user_grp_1    | 2147585843 | f           | f        | f         | f       | ******** |          |          | respool_grp_1    |          0 |            |           |           |                |
 user_g1_job_1 | 2147585847 | f           | f        | f         | f       | ******** |          |          | respool_g1_job_1 | 2147585843 |            |           |           |                |
 user_g1_job_2 | 2147585851 | f           | f        | f         | f       | ******** |          |          | respool_g1_job_2 | 2147585843 |            |           |           |                |
 user_grp_2    | 2147585855 | f           | f        | f         | f       | ******** |          |          | respool_grp_2    |          0 |            |           |           |                |
 user_g2_job_1 | 2147585859 | f           | f        | f         | f       | ******** |          |          | respool_g2_job_1 | 2147585855 |            |           |           |                |
 user_g2_job_2 | 2147585863 | f           | f        | f         | f       | ******** |          |          | respool_g2_job_2 | 2147585855 |            |           |           |                |
 user_grp_3    | 2147586254 | f           | f        | f         | f       | ******** |          |          | respool_grp_1    |          0 |            |           |           |                |
(14 rows)

5.1.2 多租戶場景

  • 業務使用者共享組使用者的資源,組使用者共享其所在資源池的資源。
  • 業務使用者必須掛在到組使用者下,且層級必須與資源池層級一一對應

點選關注,第一時間瞭解華為雲新鮮技術~

相關文章