milvus-backup安裝部署

slnngk發表於2024-08-15

環境:
OS:Centos 7
milvus:2.4.6 standalone 部署
milvus-backup:0.49

1.官網:
https://milvus.io/docs/milvus_backup_cli.md

2.下載地址：
https://github.com/zilliztech/milvus-backup/releases

3.解壓壓縮包
[root@host135 milvus_backup]#mkdir -p /opt/milvus_backup ##軟體部署目錄
[root@host135 milvus_backup]#mv milvus-backup_Linux_x86_64.tar.gz /opt/milvus_backup/
[root@host135 milvus_backup]#cd /opt/milvus_backup
[root@host135 milvus_backup]#tar -zxvf milvus-backup_Linux_x86_64.tar.gz

4.準備配置檔案
mkdir -p /opt/milvus_backup/conf
cd /opt/milvus_backup/conf
wget https://raw.githubusercontent.com/zilliztech/milvus-backup/master/configs/backup.yaml

若milvus啟用了密碼認證的,需要修改如下地方(紅色標註,其他地方預設不需要修改):

[root@host135 backup]# more /opt/milvus_backup/conf/backup.yaml
# Configures the system log output.
log:
  level: info # Only supports debug, info, warn, error, panic, or fatal. Default 'info'.
  console: true # whether print log to console
  file:
    rootPath: "logs/backup.log"

http:
  simpleResponse: true

# milvus proxy address, compatible to milvus.yaml
milvus:
  address: localhost
  port: 19530
  authorizationEnabled: true
  # tls mode values [0, 1, 2]
  # 0 is close, 1 is one-way authentication, 2 is two-way authentication.
  tlsMode: 0
  user: "root"
  password: "Milvus"

# Related configuration of minio, which is responsible for data persistence for Milvus.
minio:
  # cloudProvider: "minio" # deprecated use storageType instead
  storageType: "minio" # support storage type: local, minio, s3, aws, gcp, ali(aliyun), azure, tc(tencent)
  
  address: localhost # Address of MinIO/S3
  port: 9000   # Port of MinIO/S3
  accessKeyID: minioadmin  # accessKeyID of MinIO/S3
  secretAccessKey: minioadmin # MinIO/S3 encryption string
  useSSL: false # Access to MinIO/S3 with SSL
  useIAM: false
  iamEndpoint: ""
  
  bucketName: "a-bucket" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
  rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance

  # only for azure
  backupAccessKeyID: minioadmin  # accessKeyID of MinIO/S3
  backupSecretAccessKey: minioadmin # MinIO/S3 encryption string
  
  backupBucketName: "a-bucket" # Bucket name to store backup data. Backup data will store to backupBucketName/backupRo
otPath
  backupRootPath: "backup" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPath

backup:
  maxSegmentGroupSize: 2G

  parallelism: 
    # collection level parallelism to backup
    backupCollection: 4
    # thread pool to copy data. reduce it if blocks your storage's network bandwidth
    copydata: 128
    # Collection level parallelism to restore
    restoreCollection: 2
  
  # keep temporary files during restore, only use to debug 
  keepTempFiles: false
  
  # Pause GC during backup through Milvus Http API. 
  gcPause:
    enable: true
    seconds: 7200
    address: http://localhost:9091

bucketName: "a-bucket" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance

這兩個引數是在minio中定義的,可以登入檢視

http://192.168.1.135:9001/

賬號密碼:minioadmin/minioadmin

6.執行備份
備份整個庫,這裡備份db_test
/opt/milvus_backup/milvus-backup create -n bak_db_test_20240815 -d db_test --config=/opt/milvus_backup/conf/backup.yaml

備份檔案儲存路徑為minio的路徑下
[root@host135 backup]# pwd
/home/middle/milvus/volumes/minio/a-bucket/backup

7.恢復
恢復前可以刪除目前的表
恢復整個庫,需要帶上 --restore_index=true 才能載入資料
/opt/milvus_backup/milvus-backup restore -n bak_db_test_20240815 -d db_test --restore_index=true --config=/opt/milvus_backup/conf/backup.yaml

說明:
1.針對某個庫的備份,如db_test,-d會備份整個庫下的表,但是恢復的時候也只能恢復整個庫,指定-c,--database_collections 引數都不管用
2.針對default庫的備份,-d default備份後,恢復的時候可以只指定-c引數恢復某個表;

相關文章