一 Harbor簡介
Harbor支援高階特性，如使用者管理、訪問控制、活動監視和例項之間的複製。
Project Harbor是一個開源的可信雲本地註冊專案，用於儲存、標記和掃描上下文。
Harbor擴充套件了開源Docker分發版，增加了使用者通常需要的功能，如安全、身份和管理。
Harbor是由VMWare在Docker Registry的基礎之上進行了二次封裝，加進去了很多額外程式，而且提供了一個非常漂亮的web介面。
1 Harbor的功能
harbor除了具有基礎的拉取和推送映象功能外，還有以下功能。
1.1 效率
搭建了組織內部的私有容器registry服務，可以減少訪問公共registry服務的網路需求。
1.2 映象複製
可在例項之間複製映象。
1.3 審計
所有對registry服務進行的操作均會被記錄，便於日後審計。
1.4 管理介面
有友好、易用的圖形管理介面，還可以進行映象的查詢和搜尋。
1.5 映象遠端複製
映象可以在多個映象私庫之間複製、同步。
1.6 REST API
提供了REST API，用於更加方便地與外系統整合。
1.7 映象掃描
準確地說，這是安全工具Clair的功能，透過整合Clair的功能可對儲存在harbor上的映象漏洞進行警示。
1.8 國際化
支援多種語言實時切換
1.9 訪問控制
提供了基於角色的訪問控制，使用者和映象私庫之間透過專案進行關聯，使用者對專案的映象可根據角色設定不同的訪問許可權，還可以整合企業目前擁有的使用者管理系統，例如：AD和LDAP.
2 主機初始化
2.1 配置CentOS主機yum倉庫
下載並安裝阿里雲yum源
wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
或
curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo2.2 配置kubernetes的yum倉庫
cat << END > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
END2.3 配置docker yum源
新增阿里的 docker映象源，使用其中的 containerd 軟體
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@harbor yum.repos.d]# ll
total 12
drwxr-xr-x 2 root root 187 May 14 10:50 bak
-rw-r--r-- 1 root root 2523 Aug 4 2022 CentOS-Base.repo
-rw-r--r-- 1 root root 2081 May 14 03:00 docker-ce.repo
-rw-r--r-- 1 root root 276 May 14 10:51 kubernetes.repo執行以下命令生成快取
yum clean all && yum makecache2.4 主機名配置
cat > /etc/hosts <<EOF
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.6 k8s-master01
192.168.1.7 k8s-slave01
192.168.1.8 k8s-slave02
192.168.1.100 harbor.image.com
EOF3 安裝docker
yum install -y docker-ce注意：在設定Docker配置檔案時，需要將harbor訪問地址，設定在Docker的Http訪問白名單中，不然本伺服器的Docker會拒絕訪問私有Harbor服務(推送、拉取映象)。
3.1 私有映象地址
由於harbor採用的是htp服務，而docker倉庫預設採用htps，如果對私服採用docker login、pull、push等命令操作非htps的docker regsitry的時就會報錯，所以需要在docker的配置檔案中將harbor私服地址配置到insecure-registry作為不安全的地址執行訪問。
私有映象地址的配置，如下所示：
mkdir -p /etc/docker
cat << EOF > /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors": ["https://kn0t2bca.mirror.aliyuncs.com"],
"insecure-registries": ["192.168.1.100"]
}
EOF載入配置使修改的配置檔案生效systemctl daemon-reload
3.2 重啟docker
systemctl restart docker && systemctl enable docker
4 安裝docker compose
4.1 介紹docker compose
docker-Compose（這裡使用版本號：v2.26.0）是用於定義和執行多個Docker容器（程式）的工具。透過 Compose 我們可以使用 YML檔案來配置應用程式需要的所有服務。然後使用一個命令，就可以從 YML 檔案配置中建立並啟動所有服務。
4.2 下載docker compose
cd /opt/
curl -SL https://github.com/docker/compose/releases/download/v2.26.0/docker-compose-linux-x86_64 -o docker-compose-linux-x864-v2.26.04.3 配置環境
cd /opt/
cp docker-compose-linux-x864-v2.26.0 /usr/local/bin/docker-compose4.4 許可權配置
設定docker-compose的路徑許可權chmod +x /usr/local/bin/docker-compose
4.5 檢查版本
[root@Harbor ~]# docker-compose -v
Docker Compose version v2.26.0
[root@Harbor ~]# 5 Harbor部署
必須優先部署docker環境，才進行harbor的安裝。
5.1 下載harbor
wget https://github.com/goharbor/harbor/releases/download/v2.9.4/harbor-offline-installer-v2.9.4.tgz5.2 建立家目錄
mkdir -p /opt/harbor5.3 解壓安裝包
tar zxf harbor-offline-installer-v2.9.4.tgz 5.4 修改配置檔案
cd /opt/harbor/harbor/
cp harbor.yml.tmpl harbor.yml修改的地方有：
hostname: 192.168.1.100
註釋掉協議：
https:
修改登陸admin的密碼和資料庫的密碼。
上圖中4個箭頭分別對應4個需要修改的部分
（1）訪問IP（務必改成當前伺服器的IP或域名，不可用localhost或127.0.0.1）
（2）http訪問埠（如果這伺服器是專用於私有映象倉庫的，可設port為80，透過IP即可訪問harbor服務）
（3）初次安裝harbor時的admin賬號密碼（啟動服務後，登入並修改即可，之後就會以修改後的密碼為準），預設初始密碼：harbor2024devops
（4）修改資料庫的密碼，注意記錄新密碼的值，預設資料庫密碼：harbor2024DevOps
5.5 修改後的配置檔案
[root@harbor harbor]# egrep -v "#|^$" harbor.yml
hostname: 192.168.1.100
http:
port: 80
harbor_admin_password: harbor2024devops
database:
password: harbor2024DevOps
max_idle_conns: 100
max_open_conns: 900
conn_max_lifetime: 5m
conn_max_idle_time: 0
data_volume: /data
trivy:
ignore_unfixed: false
skip_update: false
skip_java_db_update: false
offline_scan: false
security_check: vuln
insecure: false
jobservice:
max_job_workers: 10
job_loggers:
- STD_OUTPUT
- FILE
notification:
webhook_job_max_retry: 3
log:
level: info
local:
rotate_count: 50
rotate_size: 200M
location: /var/log/harbor
_version: 2.9.0
proxy:
http_proxy:
https_proxy:
no_proxy:
components:
- core
- jobservice
- trivy
upload_purging:
enabled: true
age: 168h
interval: 24h
dryrun: false
cache:
enabled: false
expire_hours: 24
[root@harbor harbor]# 5.6 檢查環境
檢查相關元件的映象列表，如果本地沒有相關映象，則會自動聯網下載。沒網的話會報一堆依賴找不到，這裡已提前全部下載。
執行命令：
./prepare
#檢查過程；
[root@harbor harbor]# ./prepare
prepare base dir is set to /opt/harbor/harbor
Unable to find image 'goharbor/prepare:v2.9.4' locally
v2.9.4: Pulling from goharbor/prepare
5c70ea440659: Pull complete
97b717f3829c: Pull complete
8e3e2393f1f5: Pull complete
922fd4ce5cf9: Pull complete
6d5dc2e2a2f6: Pull complete
2e06411d314c: Pull complete
b129f0ef1377: Pull complete
4dcbfb4c87cd: Pull complete
792e707a745f: Pull complete
c25b0082aebd: Pull complete
Digest: sha256:8525fb155c3471a624b9e736a1ab26298e574450f045a96efd85862e8a873711
Status: Downloaded newer image for goharbor/prepare:v2.9.4
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /data/secret/keys/secretkey
Successfully called func: create_root_cert
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[root@harbor harbor]# 檢查環境時，這裡是聯網狀態，所以會自動下載該映象：goharbor/prepare:v2.9.4，如果是公司內網，需要提前匯入該映象，才能執行環境的檢查，生成docker-compose.yml。
5.7 檢視依賴的映象
以下harbor依賴的映象，聯網情況下執行安裝會自動下載，如果是內網環境，需要提前下載，然後上傳至部署的離線環境中，匯入即可。
cd /opt/harbor/harbor/
[root@harbor harbor]# cat docker-compose.yml |grep image|uniq
image: goharbor/harbor-log:v2.9.4
image: goharbor/registry-photon:v2.9.4
image: goharbor/harbor-registryctl:v2.9.4
image: goharbor/harbor-db:v2.9.4
image: goharbor/harbor-core:v2.9.4
image: goharbor/harbor-portal:v2.9.4
image: goharbor/harbor-jobservice:v2.9.4
image: goharbor/redis-photon:v2.9.4
image: goharbor/nginx-photon:v2.9.4
[root@harbor harbor]# 5.8 執行安裝指令碼
[root@harbor harbor]# ./install.sh
[Step 0]: checking if docker is installed ...
Note: docker version: 26.1.2
[Step 1]: checking docker-compose is installed ...
Note: Docker Compose version v2.27.0
[Step 2]: loading Harbor images ...
1ffa30c514e7: Loading layer [==================================================>] 115.6MB/115.6MB
Loaded image: goharbor/nginx-photon:v2.9.4
fe9d27457645: Loading layer [==================================================>] 9.117MB/9.117MB
8a1bd330387a: Loading layer [==================================================>] 4.096kB/4.096kB
de5aec39cfef: Loading layer [==================================================>] 3.072kB/3.072kB
b2dc5f9b0adc: Loading layer [==================================================>] 209.9MB/209.9MB
867f6775e77c: Loading layer [==================================================>] 17.45MB/17.45MB
3d23e7d1c061: Loading layer [==================================================>] 228.1MB/228.1MB
Loaded image: goharbor/trivy-adapter-photon:v2.9.4
8e8d9c5fab4c: Loading layer [==================================================>] 115.6MB/115.6MB
8c2100da7b2f: Loading layer [==================================================>] 6.46MB/6.46MB
beb75a4af778: Loading layer [==================================================>] 245.8kB/245.8kB
8b93536ed20c: Loading layer [==================================================>] 1.233MB/1.233MB
Loaded image: goharbor/harbor-portal:v2.9.4
303f42b51cc2: Loading layer [==================================================>] 11.61MB/11.61MB
f6d69dd1d0ad: Loading layer [==================================================>] 3.584kB/3.584kB
8dbc774dc94d: Loading layer [==================================================>] 2.56kB/2.56kB
baba5716726a: Loading layer [==================================================>] 58.75MB/58.75MB
5b67417f435d: Loading layer [==================================================>] 5.632kB/5.632kB
31a7f7742aaa: Loading layer [==================================================>] 122.4kB/122.4kB
a9bc112e775d: Loading layer [==================================================>] 80.38kB/80.38kB
3c10f76aad84: Loading layer [==================================================>] 59.74MB/59.74MB
6c5eb2684647: Loading layer [==================================================>] 2.56kB/2.56kB
Loaded image: goharbor/harbor-core:v2.9.4
807b050bbb83: Loading layer [==================================================>] 125.3MB/125.3MB
00b81cc4331d: Loading layer [==================================================>] 3.584kB/3.584kB
73d4601a8fa0: Loading layer [==================================================>] 3.072kB/3.072kB
873b556fed34: Loading layer [==================================================>] 2.56kB/2.56kB
d34eaff1dda9: Loading layer [==================================================>] 3.072kB/3.072kB
875416323a99: Loading layer [==================================================>] 3.584kB/3.584kB
04668820172e: Loading layer [==================================================>] 20.48kB/20.48kB
Loaded image: goharbor/harbor-log:v2.9.4
fc9708739fba: Loading layer [==================================================>] 11.61MB/11.61MB
5fbf23947d53: Loading layer [==================================================>] 3.584kB/3.584kB
8980553d20e3: Loading layer [==================================================>] 2.56kB/2.56kB
394a06e3fead: Loading layer [==================================================>] 44.64MB/44.64MB
0b32832b0417: Loading layer [==================================================>] 45.43MB/45.43MB
Loaded image: goharbor/harbor-jobservice:v2.9.4
649c2fb761b1: Loading layer [==================================================>] 11.61MB/11.61MB
dee3ec490421: Loading layer [==================================================>] 27.58MB/27.58MB
ce1ae31f2333: Loading layer [==================================================>] 4.608kB/4.608kB
67b43b0afe41: Loading layer [==================================================>] 28.37MB/28.37MB
Loaded image: goharbor/harbor-exporter:v2.9.4
Loaded image: goharbor/prepare:v2.9.4
9680f9b537e4: Loading layer [==================================================>] 16.08MB/16.08MB
5c80f459c1fa: Loading layer [==================================================>] 173.8MB/173.8MB
b4a8872dec5a: Loading layer [==================================================>] 25.53MB/25.53MB
273925008692: Loading layer [==================================================>] 18.3MB/18.3MB
61756a3742aa: Loading layer [==================================================>] 5.12kB/5.12kB
6aef2ba5f411: Loading layer [==================================================>] 6.144kB/6.144kB
06b8309f6264: Loading layer [==================================================>] 3.072kB/3.072kB
2378215d8cc7: Loading layer [==================================================>] 2.048kB/2.048kB
61639698f424: Loading layer [==================================================>] 2.56kB/2.56kB
11138989c354: Loading layer [==================================================>] 7.68kB/7.68kB
Loaded image: goharbor/harbor-db:v2.9.4
5476b1e98825: Loading layer [==================================================>] 8.603MB/8.603MB
21306bf47b19: Loading layer [==================================================>] 4.096kB/4.096kB
b6a29409bcb0: Loading layer [==================================================>] 17.39MB/17.39MB
f923c3b9c6b2: Loading layer [==================================================>] 3.072kB/3.072kB
b65b25d422a3: Loading layer [==================================================>] 32.69MB/32.69MB
f33443527772: Loading layer [==================================================>] 50.88MB/50.88MB
Loaded image: goharbor/harbor-registryctl:v2.9.4
297dfef001b5: Loading layer [==================================================>] 16.08MB/16.08MB
628e609117e3: Loading layer [==================================================>] 110.6MB/110.6MB
d3164a9024b3: Loading layer [==================================================>] 3.072kB/3.072kB
3bbb99540044: Loading layer [==================================================>] 59.9kB/59.9kB
0b8ee78ceda9: Loading layer [==================================================>] 61.95kB/61.95kB
Loaded image: goharbor/redis-photon:v2.9.4
a4efec05118e: Loading layer [==================================================>] 8.603MB/8.603MB
fc5a28753b96: Loading layer [==================================================>] 4.096kB/4.096kB
e7ebcb1d0d37: Loading layer [==================================================>] 3.072kB/3.072kB
f5a739f25b3b: Loading layer [==================================================>] 17.39MB/17.39MB
3bdc4fd383a7: Loading layer [==================================================>] 18.19MB/18.19MB
Loaded image: goharbor/registry-photon:v2.9.4
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /opt/harbor/harbor
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
Clearing the configuration file: /config/portal/nginx.conf
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/passwd
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /data/secret/keys/secretkey
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
Note: stopping existing Harbor instance ...
WARN[0000] /opt/harbor/harbor/docker-compose.yml: `version` is obsolete
[Step 5]: starting Harbor ...
WARN[0000] /opt/harbor/harbor/docker-compose.yml: `version` is obsolete
[+] Running 10/10
✔ Network harbor_harbor Created 0.1s
✔ Container harbor-log Started 0.3s
✔ Container harbor-portal Started 0.9s
✔ Container registry Started 0.9s
✔ Container registryctl Started 1.0s
✔ Container harbor-db Started 0.8s
✔ Container redis Started 0.9s
✔ Container harbor-core Started 1.2s
✔ Container nginx Started 1.7s
✔ Container harbor-jobservice Started 1.8s
✔ ----Harbor has been installed and started successfully.----
[root@harbor harbor]# 上述安裝過程介紹：前面是檢查docker和docker-compose部署情況，以及映象包的獲取情況。環境檢查完全沒問題後，建立一些配置檔案，最後部署成功之後自動啟動harbor及相關元件。
5.9 檢視harbor web埠
[root@harbor ~]# ss -antl|grep 80
LISTEN 0 128 *:80 *:*
LISTEN 0 128 :::80 :::*
[root@harbor ~]# 5.10 配置自啟動
新增啟動服務檔案，設定好之後記得重啟，檢測效果。
cat << END > /lib/systemd/system/harbor.service
[Unit]
Description=Harbor
After=docker.service systemd-networkd.service systemd-resolved.service
Requires=docker.service
Documentation=http://github.com/vmware/harbor
[Service]
Type=simple
Restart=on-failure
RestartSec=60
ExecStart=/usr/local/bin/docker-compose -f /opt/harbor/harbor/docker-compose.yml up
ExecStop=/usr/local/bin/docker-compose -f /opt/harbor/harbor/docker-compose.yml stop
[Install]
WantedBy=multi-user.target
END更新配置資訊，更新 ExecStart 與 ExecStop 的值，前半段是伺服器安裝 docker-compose的位置，後半段是 harbor的docker-compose檔案的位置。其中，/usr/local/bin/docker-compose 的位置需要根據安裝的docker-compose的位置自行更新，要在對應的位置找到該檔案才行，否則不生效。
注意：其中的 RestartSec 的值記得設定長一些，太短的話容易因重啟時服務沒啟動全，導致部分服務不能正常啟動，還需要事後手動啟動。
5.11 重啟harbor服務
# 載入配置檔案的更新
systemctl daemon-reload && systemctl enable harbor
# 開機自啟
systemctl enable harbor
# 啟動服務（已經啟動就不用執行了）
systemctl start harbor
# 檢查狀態
systemctl status harbor
# 檢查狀態
systemctl stop harbor使用Harbor的注意以下幾點：
在客戶端上傳映象時一定要記得執行docker login進行使用者認證，否則無法直接push。
在客戶端使用的時候如果不是用的https則必須要在客戶端的/etc/docker/daemon.json配置檔案中配置insecure-registries引數。
資料存放路徑應在配置檔案中配置到一個容量比較充足的共享儲存中。
Harbor是使用docker-compose命令來管理的，如果需要停止Harbor也應用 docker-compose stop 來停止，其他引數請--help。
到此，harbor映象倉庫部署完成，進入下一步使用及測試驗證功能。
6 登陸 Harbor web介面
#登陸地址：
http://192.168.1.100/
#登陸賬號和密碼
admin/harbor2024devops
6.1 進入harbor主頁面
輸入登陸賬號和密碼，進入下一步
6.2 新增開發測試使用者
#新使用者和密碼
testimage/Himage123#@！
可以進一步設定為管理員，這裡就不做操作啦。
6.3 建立公有倉庫
新增專案和設定專案成員許可權，映象都需要根據專案名做分類索引進行存放，新建一個專案，點選專案->新建專案。將專案設定為公開，所有使用者都可以讀取，如果設定了私有專案，只有登陸了相關使用者才可以操作，預設是私有專案。
6.3.1 映象專案配置解釋
1）專案名稱：nc_public，也可以叫它公有映象倉庫名稱。
2）訪問級別：當專案設為公開後，任何人都有此專案下映象的讀許可權。命令列使用者不需要“docker login”就可以拉取此專案下的映象。
3）專案配額限制：專案可以使用的最大邏輯空間，如果您想要不設定上限，配置值為‘-1’.
4）映象代理：開啟此項，以使得該專案成為目標倉庫的映象代理，僅支援DockerHub, Docker Registry, Harbor, Aws ECR, AzureACR, Quay, Google GcR,JFrog Artifactory.和GithubGHCR 型別的倉庫。
這裡就不配置映象代理了，以後有機會用到，在更新分享。
6.3.2 配置普通使用者
新增使用者 testimage，角色是開發者，管理員對專案擁有所有操作許可權，push/pull/delete/複製策略等操作，開發人員只能push/pull，訪客只能pull。點選專案名稱 nc_public 就會跳轉彈窗下圖，把之前建立好的使用者名稱稱 testimage新增 角色配置成‘開發者’即可。
新增完成後專案擁有兩個專案成員，分別是管理員admin和開發者 testimage，可以點選其他操作修改成員的許可權或者移除成員。
6.4 公有倉庫測試
6.4.1 推送映象
上傳映象需要在docker主機進行操作，對於私有專案必須登入Harbor私服才能進行pull和push操作，push到Harbor的映象的命名必須按照如下規則。
{Harbor地址}:{埠}/{Harbor專案名}/{自定義映象名}:{自定義tag}下一步測試登入私服：
登入成功，進行映象推送至 harbor 的nc_public倉庫中。
6.4.2 獲取映象
在k8s的工作節點匯入映象，以 NGINX 為例：
[root@k8s-slave01 ~]# docker load -i nginx-latest.tar
08249ce7456a: Loading layer [==================================================>] 83.9MB/83.9MB
d5b40e80384b: Loading layer [==================================================>] 62.01MB/62.01MB
b2f82de68e0d: Loading layer [==================================================>] 3.072kB/3.072kB
41451f050aa8: Loading layer [==================================================>] 4.096kB/4.096kB
44193d3f4ea2: Loading layer [==================================================>] 3.584kB/3.584kB
e7344f8a29a3: Loading layer [==================================================>] 7.168kB/7.168kB
Loaded image: nginx:latest
[root@k8s-slave01 ~]# docker images|grep nginx
nginx latest 55f4b40fe486 23 months ago 142MB
[root@k8s-slave01 ~]# 6.4.3 修改映象標籤
將獲取到的映象，先使用docker tag建立一個新的映象並且重新命名為harbor指定的 ip:host/專案名格式，然後直接push。
將匯入預設版本nginx映象為例：
在專案中標記映象：
#模板
docker tag SOURCE_IMAGE[:TAG] 192.168.1.100/nc_public/REPOSITORY[:TAG]
#執行的命令列
docker tag nginx:latest 192.168.1.100/nc_public/nginx:v1.0.0
[root@k8s-slave01 ~]# docker tag nginx:latest 192.168.1.100/nc_public/nginx:v1.0.0
[root@k8s-slave01 ~]# docker images|grep nginx
nginx latest 55f4b40fe486 23 months ago 142MB
192.168.1.100/nc_public/nginx v1.0.0 55f4b40fe486 23 months ago 142MB
[root@k8s-slave01 ~]# 6.4.4 push推送映象
Docker 推送命令，推送映象到當前專案：
#模板
docker push 192.168.1.100/nc_public/REPOSITORY[:TAG]
#執行的命令列
docker push 192.168.1.100/nc_public/nginx:v1.0.0直接執行推送命令，檢視過程發現沒有推送成功，因為我在執行這一步的時候，沒有提前進行"docker login"登陸到harbor 所以報了為授權錯誤，該工作節點沒有許可權推送映象到nc_public/nginx專案路徑下，具體報錯如下。
6.4.4.1 解決授權問題
在k8s的工作節點上，透過開發者使用者 testimage 登陸到harbor映象倉庫，在進行推送。
登陸成功，執行推送命令，發現映象已經push成功：
[root@k8s-slave01 ~]# docker push 192.168.1.100/nc_public/nginx:v1.0.0
The push refers to repository [192.168.1.100/nc_public/nginx]
e7344f8a29a3: Pushed
44193d3f4ea2: Pushed
41451f050aa8: Pushed
b2f82de68e0d: Pushed
d5b40e80384b: Pushed
08249ce7456a: Pushed
v1.0.0: digest: sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1 size: 1570
[root@k8s-slave01 ~]# 6.4.5 驗證映象推送
登陸或重新整理harbor介面，檢視工作節點k8s-slave01推送的NGINX映象是否上傳至192.168.1.100/nc_public/專案路徑下，如果有存在，則表示映象推送成功，反之失敗。近一步點進去可以檢視映象的標籤資訊，點選操作可以對映象進行復制摘要、新增標籤、複製映象、刪除映象等操作。
如圖所示，NGINX映象已經存放在192.168.1.100/nc_public/專案路徑下，說明我在工作節點k8s-slave01使用開發者使用者 testimage 推送映象已經成功，也可以使用該使用者登陸進行檢視。當前介面是以 testimage 使用者（開發者）進行登入的，因此沒有刪除許可權，也沒有左側的系統管理欄。
6.4.6 pull映象
下一步在另一臺工作節點k8s-slave02主機上從私服拉去映象，確保這臺機器和私服ip互通，並且將私服地址新增到docker的不安全倉庫地址配置檔案中，直接使用docker pull命令拉取私服映象。
在工作節點拉取鏡：
k8s-slave02:
docker pull 192.168.1.100/nc_public/nginx:v1.0.0
#執行命令，可以直接拉取映象：
[root@k8s-slave02 ~]# docker pull 192.168.1.100/nc_public/nginx:v1.0.0
v1.0.0: Pulling from nc_public/nginx
b85a868b505f: Pull complete
f4407ba1f103: Pull complete
4a7307612456: Pull complete
935cecace2a0: Pull complete
8f46223e4234: Pull complete
fe0ef4c895f5: Pull complete
Digest: sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1
Status: Downloaded newer image for 192.168.1.100/nc_public/nginx:v1.0.0
192.168.1.100/nc_public/nginx:v1.0.0
#映象拉取成功
[root@k8s-slave02 ~]# docker images|grep nginx
192.168.1.100/nc_public/nginx v1.0.0 55f4b40fe486 23 months ago 142MB
[root@k8s-slave02 ~]# 6.4.6.1 執行公有映象
以此映象在k8s-slave02工作節點（192.168.1.8）啟動容器，並訪問測試：
192.168.1.100/nc_public/nginx:v1.0.0
docker run -itd -p 8088:80 192.168.1.100/nc_public/nginx:v1.0.0
[root@k8s-slave02 ~]# docker images|grep nginx
192.168.1.100/nc_public/nginx v1.0.0 55f4b40fe486 23 months ago 142MB
[root@k8s-slave02 ~]#
[root@k8s-slave02 ~]# docker run -itd -p 8088:80 192.168.1.100/nc_public/nginx:v1.0.0
a8fc741cff8ac673bbce865f56b8bf4503cc839c28368053775f60edf9187b77
[root@k8s-slave02 ~]#
[root@k8s-slave02 ~]# docker ps -a|grep nginx
a8fc741cff8a 192.168.1.100/nc_public/nginx:v1.0.0 "/docker-entrypoint.…" 4 minutes ago Up 4 minutes 0.0.0.0:8088->80/tcp gallant_bose
[root@k8s-slave02 ~]# 6.4.6.2 訪問nginx-web
http://192.168.1.8:8088/
服務訪問正常，說明我的公共映象的推送和拉取整個環節都形成完整的閉環，則表示功能沒問題。
6.4.7 Harbor伺服器上的測試
#先打標籤，在執行 “docker login”的情況下執行push
docker tag nginx:latest 192.168.1.100/nc_public/nginx:v1.0.1
docker push 192.168.1.100/nc_public/nginx:v1.0.1
[root@harbor ~]# docker push 192.168.1.100/nc_public/nginx:v1.0.1
The push refers to repository [192.168.1.100/nc_public/nginx]
e7344f8a29a3: Pushed
44193d3f4ea2: Pushed
41451f050aa8: Pushed
b2f82de68e0d: Pushed
d5b40e80384b: Pushed
08249ce7456a: Pushed
v1.0.1: digest: sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1 size: 1570
[root@harbor ~]#
#如果退出“docker login”，執行“docker push”就會失敗報錯如下：
[root@harbor ~]# docker logout 192.168.1.100
Removing login credentials for 192.168.1.100
[root@harbor ~]#
[root@harbor ~]# docker push 192.168.1.100/nc_public/nginx:v1.0.2
The push refers to repository [192.168.1.100/nc_public/nginx]
e7344f8a29a3: Layer already exists
44193d3f4ea2: Layer already exists
41451f050aa8: Layer already exists
b2f82de68e0d: Layer already exists
d5b40e80384b: Layer already exists
08249ce7456a: Layer already exists
unauthorized: unauthorized to access repository: nc_public/nginx, action: push: unauthorized to access repository: nc_public/nginx, action: push
[root@harbor ~]#
[root@harbor ~]# docker login 192.168.1.100 -u testimage
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[root@harbor ~]#
[root@harbor ~]# docker push 192.168.1.100/nc_public/nginx:v1.0.2
The push refers to repository [192.168.1.100/nc_public/nginx]
e7344f8a29a3: Layer already exists
44193d3f4ea2: Layer already exists
41451f050aa8: Layer already exists
b2f82de68e0d: Layer already exists
d5b40e80384b: Layer already exists
08249ce7456a: Layer already exists
v1.0.2: digest: sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1 size: 1570
[root@harbor ~]# 6.4.7.1 檢視映象上傳
192.168.1.100/nc_public/nginx:v1.0.1 映象存在，則表示上傳成功。
6.4.7.2 拉取映象並執行
#先把harbor上原有的映象刪除，在驗證拉取上傳的nginx:v1.0.1映象
[root@harbor ~]# docker rmi 55f4b40fe486 -f
Untagged: 192.168.1.100/nc_public/nginx:v1.0.1
Untagged: 192.168.1.100/nc_public/nginx@sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1
Untagged: nginx:latest
Deleted: sha256:55f4b40fe486a5b734b46bb7bf28f52fa31426bf23be068c8e7b19e58d9b8deb
Deleted: sha256:5f58fed9b4d8e6c09cdc42eed6de6df7a7e35b40d92c98f30f8ecad4960fb7a0
Deleted: sha256:8bb72c1d014292ebf1ae348a77624c536e766757356c6dbb0de75122a94b445d
Deleted: sha256:cc9ac0adbded956d924bcf6c26ffbc93ea070019be1437d204b530a033ff4b16
Deleted: sha256:30f210588f35917f0edb5a2465db7ad60e4ef3b6ac74fe155474e14e6f0995c5
Deleted: sha256:5ecd5431cf49a2a11115844de1e7b23b9535be8789add9ab50973867db5f7d36
Deleted: sha256:08249ce7456a1c0613eafe868aed936a284ed9f1d6144f7d2d08c514974a2af9
[root@harbor ~]#
#在不執行 “docker login”的情況下執行pull
docker pull 192.168.1.100/nc_public/nginx:v1.0.1
[root@harbor ~]# docker pull 192.168.1.100/nc_public/nginx:v1.0.1
v1.0.1: Pulling from nc_public/nginx
b85a868b505f: Pull complete
f4407ba1f103: Pull complete
4a7307612456: Pull complete
935cecace2a0: Pull complete
8f46223e4234: Pull complete
fe0ef4c895f5: Pull complete
Digest: sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1
Status: Downloaded newer image for 192.168.1.100/nc_public/nginx:v1.0.1
192.168.1.100/nc_public/nginx:v1.0.1
[root@harbor ~]# docker images|grep nginx
192.168.1.100/nc_public/nginx v1.0.1 55f4b40fe486 23 months ago 142MB
[root@harbor ~]#
#執行nginx映象
docker run -itd -p 8089:80 192.168.1.100/nc_public/nginx:v1.0.1
[root@harbor ~]# docker run -itd -p 8089:80 192.168.1.100/nc_public/nginx:v1.0.1
c76090c2fa2dfe488611af0821a6c24f742d88be948e24a0c6c9c19e5a7c36a1
[root@harbor ~]#
#訪問nginx-web映象
curl -I http://192.168.1.100:8089/
[root@k8s-slave02 ~]# curl -I http://192.168.1.100:8089/
HTTP/1.1 200 OK
Server: nginx/1.23.0
Date: Wed, 29 May 2024 20:11:43 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 21 Jun 2022 14:25:37 GMT
Connection: keep-alive
ETag: "62b1d4e1-267"
Accept-Ranges: bytes
[root@k8s-slave02 ~]# 經過一番測試，在harbor伺服器上執行，對於公開專案不需要執行“docker login”就可以訪問到私服的使用者pull拉取，最後執行服務，能正常訪問，則滿足功能需求。
6.5 私有倉庫測試
對於私有專案需要“docker login”才能進行操作，先建立一個私有倉庫 nc_private 進行驗證。登陸harborWEB主頁面，單擊專案，選擇新建專案如下：
6.5.1 新增新成員
6.5.2 用普通使用者push映象
驗證使用普通使用者進行tag和push，是否報錯未授權訪問私有倉庫。
6.5.2.1 Harbor主機上測試
在harbor伺服器上測試，先不登陸普通使用者 testimage ，檢驗映象是否順利推送至私有倉庫 nc_private。
#docker推送命令模板
docker tag SOURCE_IMAGE[:TAG] 192.168.1.100/nc_private/REPOSITORY[:TAG]
docker push 192.168.1.100/nc_private/REPOSITORY[:TAG]
#先打標籤
docker tag nginx:latest 192.168.1.100/nc_private/nginx:v1.0.1
#在執行推送
docker push 192.168.1.100/nc_private/nginx:v1.0.1
[root@harbor ~]# docker tag nginx:latest 192.168.1.100/nc_private/nginx:v1.0.1
[root@harbor ~]# docker images|grep nginx
goharbor/nginx-photon v2.9.4 7d707a6be242 6 weeks ago 153MB
192.168.1.100/nc_private/nginx v1.0.1 55f4b40fe486 23 months ago 142MB
nginx latest 55f4b40fe486 23 months ago 142MB
[root@harbor ~]#
#推送報許可權錯誤問題
[root@harbor ~]# docker push 192.168.1.100/nc_private/nginx:v1.0.1
The push refers to repository [192.168.1.100/nc_private/nginx]
e7344f8a29a3: Preparing
44193d3f4ea2: Preparing
41451f050aa8: Preparing
b2f82de68e0d: Preparing
d5b40e80384b: Preparing
08249ce7456a: Waiting
unauthorized: unauthorized to access repository: nc_private/nginx, action: push: unauthorized to access repository: nc_private/nginx, action: push
[root@harbor ~]#
#解決問題，執行“docker login” 登陸使用者，在執行 push
[root@harbor ~]# docker push 192.168.1.100/nc_private/nginx:v1.0.1
The push refers to repository [192.168.1.100/nc_private/nginx]
e7344f8a29a3: Mounted from nc_public/nginx
44193d3f4ea2: Mounted from nc_public/nginx
41451f050aa8: Mounted from nc_public/nginx
b2f82de68e0d: Mounted from nc_public/nginx
d5b40e80384b: Mounted from nc_public/nginx
08249ce7456a: Mounted from nc_public/nginx
v1.0.1: digest: sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1 size: 1570
[root@harbor ~]# 如圖映象已經推送成功：
6.5.2.2 拉取映象
在harbor主機上拉取映象，具體操作如下：
#先刪除映象
[root@harbor ~]# docker rmi -f 55f4b40fe486
Untagged: 192.168.1.100/nc_private/nginx:v1.0.1
Untagged: 192.168.1.100/nc_private/nginx@sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1
Untagged: nginx:latest
Deleted: sha256:55f4b40fe486a5b734b46bb7bf28f52fa31426bf23be068c8e7b19e58d9b8deb
[root@harbor ~]#
#在執行拉取映象命令
docker pull 192.168.1.100/nc_private/nginx:v1.0.1
#不執行登陸，就會報以下錯誤
[root@harbor ~]# docker pull 192.168.1.100/nc_private/nginx:v1.0.1
Error response from daemon: unauthorized: unauthorized to access repository: nc_private/nginx, action: pull: unauthorized to access repository: nc_private/nginx, action: pull
[root@harbor ~]#
#執行登陸解決該問題
[root@harbor ~]# docker pull 192.168.1.100/nc_private/nginx:v1.0.1
v1.0.1: Pulling from nc_private/nginx
b85a868b505f: Already exists
f4407ba1f103: Already exists
4a7307612456: Already exists
935cecace2a0: Already exists
8f46223e4234: Already exists
fe0ef4c895f5: Already exists
Digest: sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1
Status: Downloaded newer image for 192.168.1.100/nc_private/nginx:v1.0.1
192.168.1.100/nc_private/nginx:v1.0.1
[root@harbor ~]#
[root@harbor ~]#
[root@harbor ~]# docker images|grep nginx
goharbor/nginx-photon v2.9.4 7d707a6be242 6 weeks ago 153MB
192.168.1.100/nc_private/nginx v1.0.1 55f4b40fe486 23 months ago 142MB
[root@harbor ~]# 6.5.2.3 執行映象
在harbor主機上執行上一步拉取的映象，執行命令如下。
#docker執行命令
docker run -itd -p 8099:80 192.168.1.100/nc_private/nginx:v1.0.1
[root@harbor ~]# docker ps -all
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e9c3e6cdf7c1 192.168.1.100/nc_private/nginx:v1.0.1 "/docker-entrypoint.…" 10 seconds ago Up 8 seconds 0.0.0.0:8099->80/tcp, :::8099->80/tcp upbeat_benz
[root@harbor ~]#
#訪問測試
curl -I http://192.168.1.100:8099/
[root@k8s-slave01 ~]# curl -I http://192.168.1.100:8099/
HTTP/1.1 200 OK
Server: nginx/1.23.0
Date: Sun, 02 Jun 2024 18:24:53 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 21 Jun 2022 14:25:37 GMT
Connection: keep-alive
ETag: "62b1d4e1-267"
Accept-Ranges: bytes
[root@k8s-slave01 ~]# 
6.5.3 在工作節點測試
在k8s-slave01工作節點測試，驗證使用普通使用者進行tag和push以及pull映象，是否報錯未授權訪問私有倉庫。
#檢查工作節點是否有nginx映象，這裡沒有，那先測試在不登陸使用者的情況下，執行pull拉取之前在harbor推送的nginx:v1.0.1映象
[root@k8s-slave01 ~]# docker images|grep nginx
[root@k8s-slave01 ~]#
#執行命令
docker pull 192.168.1.100/nc_private/nginx:v1.0.1
[root@k8s-slave01 ~]# docker pull 192.168.1.100/nc_private/nginx:v1.0.1
Error response from daemon: unauthorized: unauthorized to access repository: nc_private/nginx, action: pull: unauthorized to access repository: nc_private/nginx, action: pull
[root@k8s-slave01 ~]#
#這裡報錯了，大致意思，就是沒有許可權訪問私有倉庫地址
#解決方法：登陸使用者testimage
[root@k8s-slave01 ~]# docker login 192.168.1.100 -u testimage
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[root@k8s-slave01 ~]#
#再次執行，pull映象拉取
[root@k8s-slave01 ~]# docker pull 192.168.1.100/nc_private/nginx:v1.0.1
v1.0.1: Pulling from nc_private/nginx
b85a868b505f: Pull complete
f4407ba1f103: Pull complete
4a7307612456: Pull complete
935cecace2a0: Pull complete
8f46223e4234: Pull complete
fe0ef4c895f5: Pull complete
Digest: sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1
Status: Downloaded newer image for 192.168.1.100/nc_private/nginx:v1.0.1
192.168.1.100/nc_private/nginx:v1.0.1
#映象拉取成功
[root@k8s-slave01 ~]# docker images|grep nginx
192.168.1.100/nc_private/nginx v1.0.1 55f4b40fe486 23 months ago 142MB
[root@k8s-slave01 ~]#
#退出 docker login
[root@k8s-slave01 ~]# docker logout 192.168.1.100
Removing login credentials for 192.168.1.100
[root@k8s-slave01 ~]#
#修改標籤
[root@k8s-slave01 ~]# docker tag 192.168.1.100/nc_private/nginx:v1.0.1 192.168.1.100/nc_private/nginx:v1.0.2
[root@k8s-slave01 ~]# docker images|grep nginx
192.168.1.100/nc_private/nginx v1.0.1 55f4b40fe486 23 months ago 142MB
192.168.1.100/nc_private/nginx v1.0.2 55f4b40fe486 23 months ago 142MB
[root@k8s-slave01 ~]#
#推送映象 nginx:v1.0.2
[root@k8s-slave01 ~]# docker push 192.168.1.100/nc_private/nginx:v1.0.2
The push refers to repository [192.168.1.100/nc_private/nginx]
e7344f8a29a3: Preparing
44193d3f4ea2: Preparing
41451f050aa8: Preparing
b2f82de68e0d: Preparing
d5b40e80384b: Preparing
08249ce7456a: Waiting
unauthorized: unauthorized to access repository: nc_private/nginx, action: push: unauthorized to access repository: nc_private/nginx, action: push
[root@k8s-slave01 ~]#
#果然推送失敗了，沒有登陸docker login，是沒有許可權訪問私有映象倉庫nc_private的
#再次執行登陸
docker login 192.168.1.100 -u testimage
#登陸成功之後，執行push命令
[root@k8s-slave01 ~]# docker push 192.168.1.100/nc_private/nginx:v1.0.2
The push refers to repository [192.168.1.100/nc_private/nginx]
e7344f8a29a3: Layer already exists
44193d3f4ea2: Layer already exists
41451f050aa8: Layer already exists
b2f82de68e0d: Layer already exists
d5b40e80384b: Layer already exists
08249ce7456a: Layer already exists
v1.0.2: digest: sha256:3536d368b898eef291fb1f6d184a95f8bc1a6f863c48457395aab859fda354d1 size: 1570
[root@k8s-slave01 ~]# 
這一次nginx:v1.0.2映象推送成功了。
6.6 部署總結