ceph-crush map區分ssd和hdd資源池
環境
-
3塊SSD
-
6塊HDD
# ceph -s cluster: id: b4c125fd-60ab-41ce-b51c-88833089a3ad health: HEALTH_OK services: mon: 3 daemons, quorum node1,node2,node3 (age 47m) mgr: node1(active, since 56m), standbys: node2, node3 osd: 9 osds: 9 up (since 44m), 9 in (since 44m) data: pools: 1 pools, 1 pgs objects: 0 objects, 0 B usage: 9.1 GiB used, 30 TiB / 30 TiB avail pgs: 1 active+clean # ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 29.83720 root default -3 8.73286 host node1 0 hdd 3.63869 osd.0 up 1.00000 1.00000 1 hdd 3.63869 osd.1 up 1.00000 1.00000 6 ssd 1.45549 osd.6 up 1.00000 1.00000 -5 12.37148 host node2 2 hdd 5.45799 osd.2 up 1.00000 1.00000 3 hdd 5.45799 osd.3 up 1.00000 1.00000 7 ssd 1.45549 osd.7 up 1.00000 1.00000 -7 8.73286 host node3 4 hdd 3.63869 osd.4 up 1.00000 1.00000 5 hdd 3.63869 osd.5 up 1.00000 1.00000 8 ssd 1.45549 osd.8 up 1.00000 1.00000
需求
根據crush map將ssd和hdd分為兩個資源池，iops高的放在ssd的pool，不高的放在hdd的pool
部署
- 檢視crush map分佈：
# ceph osd crush tree
ID CLASS WEIGHT TYPE NAME
-1 29.83720 root default
-3 8.73286 host node1
0 hdd 3.63869 osd.0
1 hdd 3.63869 osd.1
6 ssd 1.45549 osd.6
-5 12.37148 host node2
2 hdd 5.45799 osd.2
3 hdd 5.45799 osd.3
7 ssd 1.45549 osd.7
-7 8.73286 host node3
4 hdd 3.63869 osd.4
5 hdd 3.63869 osd.5
8 ssd 1.45549 osd.8
-
檢視詳細crush規則
# ceph osd crush dump { "devices": [ { "id": 0, "name": "osd.0", "class": "hdd" }, { "id": 1, "name": "osd.1", "class": "hdd" }, { "id": 2, "name": "osd.2", "class": "hdd" }, { "id": 3, "name": "osd.3", "class": "hdd" }, { "id": 4, "name": "osd.4", "class": "hdd" }, { "id": 5, "name": "osd.5", "class": "hdd" }, { "id": 6, "name": "osd.6", "class": "ssd" }, { "id": 7, "name": "osd.7", "class": "ssd" }, { "id": 8, "name": "osd.8", "class": "ssd" } ], "types": [ { "type_id": 0, "name": "osd" }, { "type_id": 1, "name": "host" }, { "type_id": 2, "name": "chassis" }, { "type_id": 3, "name": "rack" }, { "type_id": 4, "name": "row" }, { "type_id": 5, "name": "pdu" }, { "type_id": 6, "name": "pod" }, { "type_id": 7, "name": "room" }, { "type_id": 8, "name": "datacenter" }, { "type_id": 9, "name": "zone" }, { "type_id": 10, "name": "region" }, { "type_id": 11, "name": "root" } ], "buckets": [ { "id": -1, "name": "default", "type_id": 11, "type_name": "root", "weight": 1955411, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": -3, "weight": 572317, "pos": 0 }, { "id": -5, "weight": 810777, "pos": 1 }, { "id": -7, "weight": 572317, "pos": 2 } ] }, { "id": -2, "name": "default~hdd", "type_id": 11, "type_name": "root", "weight": 1669250, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": -4, "weight": 476930, "pos": 0 }, { "id": -6, "weight": 715390, "pos": 1 }, { "id": -8, "weight": 476930, "pos": 2 } ] }, { "id": -3, "name": "node1", "type_id": 1, "type_name": "host", "weight": 572317, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": 0, "weight": 238465, "pos": 0 }, { "id": 1, "weight": 238465, "pos": 1 }, { "id": 6, "weight": 95387, "pos": 2 } ] }, { "id": -4, "name": "node1~hdd", "type_id": 1, "type_name": "host", "weight": 476930, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": 0, "weight": 238465, "pos": 0 }, { "id": 1, "weight": 238465, "pos": 1 } ] }, { "id": -5, "name": "node2", "type_id": 1, "type_name": "host", "weight": 810777, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": 2, "weight": 357695, "pos": 0 }, { "id": 3, "weight": 357695, "pos": 1 }, { "id": 7, "weight": 95387, "pos": 2 } ] }, { "id": -6, "name": "node2~hdd", "type_id": 1, "type_name": "host", "weight": 715390, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": 2, "weight": 357695, "pos": 0 }, { "id": 3, "weight": 357695, "pos": 1 } ] }, { "id": -7, "name": "node3", "type_id": 1, "type_name": "host", "weight": 572317, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": 4, "weight": 238465, "pos": 0 }, { "id": 5, "weight": 238465, "pos": 1 }, { "id": 8, "weight": 95387, "pos": 2 } ] }, { "id": -8, "name": "node3~hdd", "type_id": 1, "type_name": "host", "weight": 476930, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": 4, "weight": 238465, "pos": 0 }, { "id": 5, "weight": 238465, "pos": 1 } ] }, { "id": -9, "name": "node1~ssd", "type_id": 1, "type_name": "host", "weight": 95387, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": 6, "weight": 95387, "pos": 0 } ] }, { "id": -10, "name": "node2~ssd", "type_id": 1, "type_name": "host", "weight": 95387, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": 7, "weight": 95387, "pos": 0 } ] }, { "id": -11, "name": "node3~ssd", "type_id": 1, "type_name": "host", "weight": 95387, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": 8, "weight": 95387, "pos": 0 } ] }, { "id": -12, "name": "default~ssd", "type_id": 11, "type_name": "root", "weight": 286161, "alg": "straw2", "hash": "rjenkins1", "items": [ { "id": -9, "weight": 95387, "pos": 0 }, { "id": -10, "weight": 95387, "pos": 1 }, { "id": -11, "weight": 95387, "pos": 2 } ] } ], "rules": [ { "rule_id": 0, "rule_name": "replicated_rule", "ruleset": 0, "type": 1, "min_size": 1, "max_size": 10, "steps": [ { "op": "take", "item": -1, "item_name": "default" }, { "op": "chooseleaf_firstn", "num": 0, "type": "host" }, { "op": "emit" } ] } ], "tunables": { "choose_local_tries": 0, "choose_local_fallback_tries": 0, "choose_total_tries": 50, "chooseleaf_descend_once": 1, "chooseleaf_vary_r": 1, "chooseleaf_stable": 1, "straw_calc_version": 1, "allowed_bucket_algs": 54, "profile": "jewel", "optimal_tunables": 1, "legacy_tunables": 0, "minimum_required_version": "jewel", "require_feature_tunables": 1, "require_feature_tunables2": 1, "has_v2_rules": 0, "require_feature_tunables3": 1, "has_v3_rules": 0, "has_v4_buckets": 1, "require_feature_tunables5": 1, "has_v5_rules": 0 }, "choose_args": {} } -
檢視當前pool寫入osd的規則
# ceph osd crush rule ls replicated_rule
手動編輯crush規則
-
匯出crushmap規則
# ceph osd getcrushmap -o crushmap20200922.bin 19 -
編譯匯出的crushmap檔案
# crushtool -d crushmap20200922.bin -o crushmap20200922.txt [root@node1 ceph]# ll total 8 -rw-r--r-- 1 root root 1326 Sep 22 23:06 crushmap20200922.bin -rw-r--r-- 1 root root 1935 Sep 22 23:07 crushmap20200922.txt -
修改規則
-
新增三個buckets來定義ssd host 並且將ssd host從原來的hdd host中刪除 並且將以前host順便改名
# 原來的host ... # buckets 定義osd怎麼存放的 host node1 { id -3 # do not change unnecessarily id -4 class hdd # do not change unnecessarily id -9 class ssd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.0 weight 3.639 item osd.1 weight 3.639 item osd.6 weight 1.455 } host node2 { id -5 # do not change unnecessarily id -6 class hdd # do not change unnecessarily id -10 class ssd # do not change unnecessarily # weight 12.371 alg straw2 hash 0 # rjenkins1 item osd.2 weight 5.458 item osd.3 weight 5.458 item osd.7 weight 1.455 } host node3 { id -7 # do not change unnecessarily id -8 class hdd # do not change unnecessarily id -11 class ssd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.4 weight 3.639 item osd.5 weight 3.639 item osd.8 weight 1.455 } ... # 修改後的host ... # buckets 定義osd怎麼存放的 host node1-hdd { id -3 # do not change unnecessarily id -4 class hdd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.0 weight 3.639 item osd.1 weight 3.639 } host node2-hdd { id -5 # do not change unnecessarily id -6 class hdd # do not change unnecessarily # weight 12.371 alg straw2 hash 0 # rjenkins1 item osd.2 weight 5.458 item osd.3 weight 5.458 } host node3-hdd { id -7 # do not change unnecessarily id -8 class hdd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.4 weight 3.639 item osd.5 weight 3.639 } host node1-ssd { id -9 class ssd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.6 weight 1.455 } host node2-ssd { id -10 class ssd # do not change unnecessarily # weight 12.371 alg straw2 hash 0 # rjenkins1 item osd.7 weight 1.455 } host node3-ssd { id -11 class ssd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.8 weight 1.455 } ... -
定義一個上層規則去呼叫新加的三個host，並且將以前的root default改為root hdd 並且調節權重，權重就是所有osd容量大小相加，比如單盤1.8TB，系統容量為1.635，權重為1.63539
#之前的default呼叫 ... root default { id -1 # do not change unnecessarily id -2 class hdd # do not change unnecessarily id -12 class ssd # do not change unnecessarily # weight 29.837 alg straw2 hash 0 # rjenkins1 item node1 weight 8.733 item node2 weight 12.371 item node3 weight 8.733 } ... #修改後的呼叫 ... root hdd { id -1 # do not change unnecessarily id -2 class hdd # do not change unnecessarily # weight 29.837 alg straw2 hash 0 # rjenkins1 item node1-hdd weight 7.278 item node2-hdd weight 10.916 item node3-hdd weight 7.278 } root ssd { # weight 29.837 alg straw2 hash 0 # rjenkins1 item node1-ssd weight 1.456 item node2-ssd weight 1.456 item node3-ssd weight 1.456 } ... -
最後定義一個rule規則關聯到上層規則,並且將以前的replicated_rule改為hdd_rule
#之前的rule ... rule replicated_rule { id 0 type replicated min_size 1 max_size 10 step take default step chooseleaf firstn 0 type host step emit } ... #修改後rule ... rule hdd_rule { id 0 type replicated min_size 1 max_size 10 step take hdd step chooseleaf firstn 0 type host step emit } rule ssd_rule { id 1 type replicated min_size 1 max_size 10 step take ssd step chooseleaf firstn 0 type host step emit } ... -
修改完成後的規則
# begin crush map tunable choose_local_tries 0 tunable choose_local_fallback_tries 0 tunable choose_total_tries 50 tunable chooseleaf_descend_once 1 tunable chooseleaf_vary_r 1 tunable chooseleaf_stable 1 tunable straw_calc_version 1 tunable allowed_bucket_algs 54 # devices 定義osd資訊 device 0 osd.0 class hdd device 1 osd.1 class hdd device 2 osd.2 class hdd device 3 osd.3 class hdd device 4 osd.4 class hdd device 5 osd.5 class hdd device 6 osd.6 class ssd device 7 osd.7 class ssd device 8 osd.8 class ssd # types 定義陣列組織型別 osd 主機 機櫃 機房等等 type 0 osd type 1 host type 2 chassis type 3 rack type 4 row type 5 pdu type 6 pod type 7 room type 8 datacenter type 9 zone type 10 region type 11 root # buckets 定義osd怎麼存放的 host node1-hdd { id -3 # do not change unnecessarily id -4 class hdd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.0 weight 3.639 item osd.1 weight 3.639 } host node2-hdd { id -5 # do not change unnecessarily id -6 class hdd # do not change unnecessarily # weight 12.371 alg straw2 hash 0 # rjenkins1 item osd.2 weight 5.458 item osd.3 weight 5.458 } host node3-hdd { id -7 # do not change unnecessarily id -8 class hdd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.4 weight 3.639 item osd.5 weight 3.639 } host node1-ssd { id -9 class ssd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.6 weight 1.455 } host node2-ssd { id -10 class ssd # do not change unnecessarily # weight 12.371 alg straw2 hash 0 # rjenkins1 item osd.7 weight 1.455 } host node3-ssd { id -11 class ssd # do not change unnecessarily # weight 8.733 alg straw2 hash 0 # rjenkins1 item osd.8 weight 1.455 } root hdd { id -1 # do not change unnecessarily id -2 class hdd # do not change unnecessarily # weight 29.837 alg straw2 hash 0 # rjenkins1 item node1-hdd weight 7.278 item node2-hdd weight 10.916 item node3-hdd weight 7.278 } root ssd { # weight 29.837 alg straw2 hash 0 # rjenkins1 item node1-ssd weight 1.456 item node2-ssd weight 1.456 item node3-ssd weight 1.456 } # rules 最終的root規則 pool應用到哪個 rule hdd_rule { id 0 type replicated min_size 1 max_size 10 step take hdd step chooseleaf firstn 0 type host step emit } rule ssd_rule { id 1 type replicated min_size 1 max_size 10 step take ssd step chooseleaf firstn 0 type host step emit } # end crush map -
將修改完的規則編譯為bin檔案
# crushtool -c crushmap20200922.txt -o crushmap20200922-new.bin # ll total 12 -rw-r--r-- 1 root root 1326 Sep 22 23:06 crushmap20200922.bin -rw-r--r-- 1 root root 2113 Sep 22 23:33 crushmap20200922-new.bin -rw-r--r-- 1 root root 2516 Sep 22 23:33 crushmap20200922.txt -
應用新的規則
# ceph osd setcrushmap -i crushmap20200922-new.bin 20 -
應用後新老規則對比，已經可以區分出ssd和hdd規則
# 應用前 # ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 29.83720 root default -3 8.73286 host node1 0 hdd 3.63869 osd.0 up 1.00000 1.00000 1 hdd 3.63869 osd.1 up 1.00000 1.00000 6 ssd 1.45549 osd.6 up 1.00000 1.00000 -5 12.37148 host node2 2 hdd 5.45799 osd.2 up 1.00000 1.00000 3 hdd 5.45799 osd.3 up 1.00000 1.00000 7 ssd 1.45549 osd.7 up 1.00000 1.00000 -7 8.73286 host node3 4 hdd 3.63869 osd.4 up 1.00000 1.00000 5 hdd 3.63869 osd.5 up 1.00000 1.00000 8 ssd 1.45549 osd.8 up 1.00000 1.00000 # 應用後 可以看出有兩個root規則，分為hdd和ssd ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -15 4.36798 root ssd -12 1.45599 host node1-ssd 6 ssd 1.45499 osd.6 up 1.00000 1.00000 -13 1.45599 host node2-ssd 7 ssd 1.45499 osd.7 up 1.00000 1.00000 -14 1.45599 host node3-ssd 8 ssd 1.45499 osd.8 up 1.00000 1.00000 -1 25.47200 root hdd -3 7.27800 host node1-hdd 0 hdd 3.63899 osd.0 up 1.00000 1.00000 1 hdd 3.63899 osd.1 up 1.00000 1.00000 -5 10.91600 host node2-hdd 2 hdd 5.45799 osd.2 up 1.00000 1.00000 3 hdd 5.45799 osd.3 up 1.00000 1.00000 -7 7.27800 host node3-hdd 4 hdd 3.63899 osd.4 up 1.00000 1.00000 5 hdd 3.63899 osd.5 up 1.00000 1.00000 -
驗證，將之前的ceph-demo資源池的規則改為ssd的
# ceph osd pool get ceph-demo crush_rule crush_rule: hdd_rule # ceph osd pool set ceph-demo crush_rule ssd_rule set pool 2 crush_rule to ssd_rule # ceph osd pool get ceph-demo crush_rule crush_rule: ssd_rule #建立一個100G的rbd # rbd create ceph-demo/rbd-demo.img --size 100G # rbd info ceph-demo/rbd-demo.img rbd image 'rbd-demo.img': size 100 GiB in 25600 objects order 22 (4 MiB objects) snapshot_count: 0 id: 39a814402607 block_name_prefix: rbd_data.39a814402607 format: 2 features: layering op_features: flags: create_timestamp: Tue Sep 22 23:42:55 2020 access_timestamp: Tue Sep 22 23:42:55 2020 modify_timestamp: Tue Sep 22 23:42:55 2020 #將ceph-demo設定為rbd型別： # ceph osd pool application enable ceph-demo rbd # 掛載到本地 # rbd map ceph-demo/rbd-demo.img /dev/rbd0 ]# mkfs.xfs /dev/rbd0 meta-data=/dev/rbd0 isize=512 agcount=16, agsize=1638400 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=0, sparse=0 data = bsize=4096 blocks=26214400, imaxpct=25 = sunit=1024 swidth=1024 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=12800, version=2 = sectsz=512 sunit=8 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [root@node1 ceph]# mkdir /mnt/rdb-demo [root@node1 ceph]# mount /dev/rbd0 /mnt/rdb-demo #最後測速測一把 寫入1.2G 符合ssd # time dd if=/dev/zero of=test.dbf bs=8k count=300000 300000+0 records in 300000+0 records out 2457600000 bytes (2.5 GB) copied, 2.06784 s, 1.2 GB/s real 0m2.072s user 0m0.318s sys 0m1.742s #同時也可以看到檔案落在三個ssd上 # ceph osd map ceph-demo rbd-demo.img osdmap e81 pool 'ceph-demo' (2) object 'rbd-demo.img' -> pg 2.7d92bf55 (2.15) -> up ([6,8,7], p6) acting ([6,8,7], p6) -
禁用掉禁用在增加或者修改osd的時候自動分配curshmap，否在在重啟服務或者新增osd的時候 會有大量的pg遷移。出現事故,在配置檔案中新增osd crush update on start = false 新增了這個引數後，後面加了osd後 需要手動將osd新增到curshmap規則裡 重新應用
# vim ceph.conf ... [osd] osd crush update on start = false ... #推送到其他節點 # ceph-deploy --overwrite-conf config push node1 node2 node3 [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (2.0.1): /usr/bin/ceph-deploy --overwrite-conf config push node1 node2 node3 [ceph_deploy.cli][INFO ] ceph-deploy options: [ceph_deploy.cli][INFO ] username : None [ceph_deploy.cli][INFO ] verbose : False [ceph_deploy.cli][INFO ] overwrite_conf : True [ceph_deploy.cli][INFO ] subcommand : push [ceph_deploy.cli][INFO ] quiet : False [ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f1e65ed7b00> [ceph_deploy.cli][INFO ] cluster : ceph [ceph_deploy.cli][INFO ] client : ['node1', 'node2', 'node3'] [ceph_deploy.cli][INFO ] func : <function config at 0x7f1e667c0c08> [ceph_deploy.cli][INFO ] ceph_conf : None [ceph_deploy.cli][INFO ] default_release : False [ceph_deploy.config][DEBUG ] Pushing config to node1 [node1][DEBUG ] connected to host: node1 [node1][DEBUG ] detect platform information from remote host [node1][DEBUG ] detect machine type [node1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph_deploy.config][DEBUG ] Pushing config to node2 [node2][DEBUG ] connected to host: node2 [node2][DEBUG ] detect platform information from remote host [node2][DEBUG ] detect machine type [node2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph_deploy.config][DEBUG ] Pushing config to node3 [node3][DEBUG ] connected to host: node3 [node3][DEBUG ] detect platform information from remote host [node3][DEBUG ] detect machine type [node3][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf #重啟osd 三臺節點全部重啟 # systemctl restart ceph-osd.target注意事項
-
在擴容和刪除osd還有編輯curshmap的時候最好備份一個
-
初始化就規劃好curshmap的規則，否則應用過程中改變的話，就會有大量的pg遷移
-
重啟服務的時候就會自動修改curshmap，所以需要備份，也可以禁用在增加或者修改osd的時候自動分配curshmap
-
-
相關文章
- IDC：HDD和SSD市場雖受疫情衝擊 但需求強勁
- 西部資料關閉機械硬碟工廠 SSD取代HDD已成真？硬碟
- 如何在 macOS Monterey(SSD 或 HDD)中刪除 Mac 分割槽Mac
- Wells Fargo：資料顯示企業級SSD硬碟價格仍是HDD 10倍以上Go硬碟
- Linux下判斷磁碟是SSD還是HDD的3種方法Linux
- JavaScript map和reduce的區別JavaScript
- 怎麼判斷Win10系統中硬碟是SSD還是HDDWin10硬碟
- for、for...in、for...of、forEach和map的區別
- Spring系列 之資料來源的配置 資料庫 資料來源 連線池的區別Spring資料庫
- 【MySQL】自定義資料庫連線池和開源資料庫連線池的使用MySql資料庫
- Rxjs map, mergeMap 和 switchMap 的區別和聯絡JS
- Win10系統SSD+HDD雙硬碟電腦卡頓的解決方法Win10硬碟
- OceanBase學習之路41|如何檢視資源池的分佈情況？
- JavaScript中的new map()和new set()使用詳細（new map()和new set()的區別）JavaScript
- 分散式系統硬體資源池原理和接入實踐分散式
- Set和Map資料結構。資料結構
- 資料探勘和資料提取該怎麼區分？
- solaris11怎麼擴充套件資源池？solaris11擴充套件資源池的方法套件
- 人力資源管理系統和oa的區別？
- openGauss 資源池化效能最佳化
- 如何降低 Oracle 容災加固資源池的總成本和複雜度？Oracle複雜度
- map和multimap
- Set 和Map
- App Tamer for Mac(節省電池和CPU電源)APPMac
- openGauss gs_collector適配資源池化
- 分析一個簡單的goroutine資源池Go
- $.each()、$.map()區別淺談
- 資料訪問連線池和執行緒池執行緒
- c++ map和unordered_map比較C++
- 如何區分 Connection、Thread和SessionthreadSession
- 【區分】Typescript 中 interface 和 typeTypeScript
- 區分slice,splice和split方法
- python map和lambdaPython
- go map 和 sliceGo
- OceanBase學習之路38|如何合併資源池？
- 第 15 期 多路複用資源池元件剖析元件
- HikariPool原始碼（三）資源池動態伸縮原始碼
- python中小資料池和編碼Python