目錄
- 涉及內容:
- 一、CICD伺服器環境搭建
- 1、docker 環境安裝
- （1）、拉取映象，啟動並設定開機自啟
- （2）、配置docker加速器
- 2、安裝並配置GitLab
- （1）、建立共享卷目錄
- （2）、建立 gitlab 容器
- (3)、關閉容器修改配置檔案
- （4）、修改完配置檔案之後。直接啟動容器
- （5）、相關的git命令（針對已存在的資料夾）
- 3、安裝配置遠端映象倉庫harbor
- （1）、首先需要設定selinux、防火牆
- （2）、安裝並啟動docker並安裝docker-compose，關於docker-compose，這裡不用瞭解太多，一個輕量的docker編排工具
- （3）、解壓harbor 安裝包：harbor-offline-installer-v2.0.6.tgz，匯入相關映象
- （4）、修改配置檔案
- （5）、harbor.yml：設定IP和使用者名稱密碼
- （6）、
./prepare && ./install.sh - （7）、檢視相關的映象
- （8）、訪問測試
- 4、CI伺服器的docker配置
- （1）、修改配置檔案
- （2）、push一個映象，可以在私倉的web頁面檢視
- 5、安裝配置jenkins
- （1）、映象jenkins拉取
- （2）、建立共享卷，修改所屬組和使用者,和容器裡相同
- （3）、建立建立 jenkins 容器
- （4）、更換國內清華大學映象,Jenkins下載外掛特別慢，更換國內的清華源的映象地址會快不少
- （5）、重啟docker，獲取登入密匙
- （6）、安裝 docker 外掛
- （7）、jenkins 安全設定
- （8）、下載kubectl客戶端工具
- 複製 kubeconfig 檔案
- 複製證書和k8s叢集客戶端工具到jenkins容器內
- （9）、kubectl命令測試
- 1、docker 環境安裝
- 二 、hexo部落格系統CICD實戰
- 1、k8s叢集中配置hexo生產環境高可用
- （1）、deployments建立
- （2）、service建立
- 2、k8s叢集配置私倉地址
- 3、jenkins配置CICD流程
- 4、配置 gitlab 和 jenkins 的聯動
- 5、編寫Dockerfile檔案，更新程式碼測試
- 6、訪問hexo部落格系統
- 1、k8s叢集中配置hexo生產環境高可用
本篇文章參考 山河已無恙大佬的文章：（持續整合部署Hexo部落格Demo）
涉及內容:
-
Gitlab+Jenkins+Docker+Harbor+K8S叢集的CICD搭建教程 -
在搭建好的
CICD平臺上持續整合部署hexo部落格系統 -
其中
Gitlab+Jenkins+Harbor都是透過容器化部署 -
篇幅有限，關於CD環境
k8s叢集這裡用之前部署好的，並且已經做了kubeconfig證書 -
下面為涉及到的機器：
| 用到的機器 | ip |
|---|---|
| 客戶機 | 本地物理機 |
| Gitlab+Jenkins+Docker | 192.168.112.10 |
| docker映象倉庫:harbor | 192.168.112.20 |
| k8s叢集-master節點 | 192.168.112.30 |
| k8s叢集-node節點 | 192.168.112.40 |
| k8s叢集-node節點 | 192.168.112.50 |
| 拓撲圖 |
|---|
這裡客戶機用本地的IDE持續編碼，然後push程式碼到gitlab，gitlab中的web鉤子觸發jenkins中配置好的構建觸發器，透過shell命令拉取gitlab倉庫中的程式碼，然後透過拉取的應用原始碼和Dockerfile檔案來構建應用映象，構建完成後將應用映象push到harbor私有映象倉庫，然後透過shell命令的方式在jenkins中用kubelet客戶端將映象從私有倉庫拉取到k8s叢集並更新其deploy中的映象,預設deploy更新副本的方式為滾動更新，整個流程中，只有客戶機push程式碼是手手動的方式，其他全是自動 |
 |
一、CICD伺服器環境搭建
CI即為持續整合(Continue Integration,簡稱CI)，用通俗的話講，就是持續的整合版本庫程式碼編譯後製作應用映象。建立有效的持續整合環境可以減少開發過程中一些不必要的問題、提高程式碼質量、快速迭代等,
常用的工具和平臺有:
Jenkins:基於Java開發的一種持續整合工具,用於監控持續重複的工作,旨在提供一個開放易用的軟體平臺,使軟體的持續整合變成可能。
Bamboo: 是一個企業級商用軟體,可以部署在大規模生產環境中。
CD即持續交付Continuous Delivery和持續部署Continuous Deployment，用通俗的話說，即可以持續的部署到生產環境給客戶使用，這裡分為兩個階段，持續交付我理解為滿足上線條件的過程，但是沒有上線，持續部署，即為上線應用的過程
關於CD環境，我們使用以前搭建好的K8s叢集，K8s叢集可以實現應用的健康檢測，動態擴容，滾動更新等優點，關於K8s叢集的搭建，小夥伴可以看看我的其他文章
我們來搭建CI伺服器:操作伺服器： jenkins：192.168.112.10
1、docker 環境安裝
（1）、拉取映象，啟動並設定開機自啟
[root@jenkins ~]# systemctl start docker
[root@jenkins ~]# systemctl enable docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
（2）、配置docker加速器
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://2tefyfv7.mirror.aliyuncs.com"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
2、安裝並配置GitLab
GitLab是一個基於Git的版本控制平臺，,提供了Git倉庫管理、程式碼審查、問題跟蹤、活動反饋和wiki，當然同時也提供了
[root@jenkins ~]# docker pull beginor/gitlab-ce
（1）、建立共享卷目錄
[root@jenkins ~]# mkdir -p /data/gitlab/etc/ /data/gitlab/log/ /data/gitlab/data
[root@jenkins ~]# chmod 777 /data/gitlab/etc/ /data/gitlab/log/ /data/gitlab/data/
（2）、建立 gitlab 容器
[root@jenkins ~]# docker run -itd --name=gitlab --restart=always --privileged=true -p 8443:443 -p 80:80 -p 222:22 -v /data/gitlab/etc:/etc/gitlab -v /data/gitlab/log:/var/log/gitlab -v /data/gitlab/data:/var/opt/gitlab beginor/gitlab-ce
805eb9eac8367c53a8d458fec17649e3b3b206f3dc74c99c7a037a41dd9e8ca6
[root@jenkins ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
805eb9eac836 beginor/gitlab-ce "/assets/wrapper" 20 seconds ago Up 19 seconds (health: starting) 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:222->22/tcp, :::222->22/tcp, 0.0.0.0:8443->443/tcp, :::8443->443/tcp gitlab
切記:這裡的埠要設定成80，要不push專案會提示沒有報錯，如果宿主機埠被佔用，需要把這個埠騰出來
(3)、關閉容器修改配置檔案
[root@jenkins ~]# docker stop gitlab
gitlab
external_url 'http://192.168.112.10'
[root@jenkins ~]# cat /data/gitlab/etc/gitlab.rb |grep external_url
##! For more details on configuring external_url see:
# external_url 'GENERATED_EXTERNAL_URL'
# registry_external_url 'https://registry.gitlab.example.com'
# pages_external_url "http://pages.example.com/"
# gitlab_pages['artifacts_server_url'] = nil # Defaults to external_url + '/api/v4'
# mattermost_external_url 'http://mattermost.example.com'
[root@jenkins ~]# sed -i "/external_url 'GENERATED_EXTERNAL_URL'/a external_url\t'http://192.168.112.10' " /data/gitlab/etc/gitlab.rb
[root@jenkins ~]# cat /data/gitlab/etc/gitlab.rb |grep external_url
##! For more details on configuring external_url see:
# external_url 'GENERATED_EXTERNAL_URL'
external_url 'http://192.168.112.10'
# registry_external_url 'https://registry.gitlab.example.com'
# pages_external_url "http://pages.example.com/"
# gitlab_pages['artifacts_server_url'] = nil # Defaults to external_url + '/api/v4'
# mattermost_external_url 'http://mattermost.example.com'
gitlab_rails[‘gitlab_ssh_host’] = '192.168.112.10'
[root@jenkins ~]# cat /data/gitlab/etc/gitlab.rb |grep gitlab_ssh_host
# gitlab_rails['gitlab_ssh_host'] = 'ssh.host_example.com'
[root@jenkins ~]# sed -i "/gitlab_ssh_host/a gitlab_rails['gitlab_ssh_host'] = '192.168.112.10' " /data/gitlab/etc/gitlab.rb
[root@jenkins ~]# cat /data/gitlab/etc/gitlab.rb |grep gitlab_ssh_host # gitlab_rails['gitlab_ssh_host'] = 'ssh.host_example.com'
gitlab_rails['gitlab_ssh_host'] = '192.168.112.10'
gitlab_rails[gitlab_shell_ssh_port] = 222
[root@jenkins ~]# cat /data/gitlab/etc/gitlab.rb | grep gitlab_shell_ssh
# gitlab_rails['gitlab_shell_ssh_port'] = 22
[root@jenkins ~]# sed -i "/gitlab_shell_ssh_port/a gitlab_rails['gitlab_shell_ssh_port'] = 222" /data/gitlab/etc/gitlab.rb
[root@jenkins ~]# cat /data/gitlab/etc/gitlab.rb | grep gitlab_shell_ssh # gitlab_rails['gitlab_shell_ssh_port'] = 22
gitlab_rails['gitlab_shell_ssh_port'] = 222
[root@jenkins ~]# vim /data/gitlab/data/gitlab-rails/etc/gitlab.yml
## GitLab settings
gitlab:
## Web server settings (note: host is the FQDN, do not include http://)
host: 192.168.112.10
port: 80
https: false
（4）、修改完配置檔案之後。直接啟動容器
[root@jenkins ~]# docker start gitlab
gitlab
[root@jenkins ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
805eb9eac836 beginor/gitlab-ce "/assets/wrapper" 21 minutes ago Up 7 seconds (health: starting) 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:222->22/tcp, :::222->22/tcp, 0.0.0.0:8443->443/tcp, :::8443->443/tcp gitlab
| Gitlab |
|---|
在宿主機所在的物理機訪問，http://192.168.112.10/ ，會自動跳轉到修改密碼(root使用者),如果密碼設定的沒有滿足一定的複雜性，則會報500，需要從新設定 |
 |
 |
| 登入進入儀表盤 |
 |
 |
 |
| 然後我們簡單測試一下，push一個專案上去,會提示輸入使用者密碼,這裡的專案是一個基於hexo的部落格系統 |
 |
| 專案成功上傳Gitlab |
 |
 |
（5）、相關的git命令（針對已存在的資料夾）
cd existing_folder
git init
git remote add origin http://192.168.112.10/root/hexo-gitlab-blog.git
git add .
git commit -m "Initial commit"
git push -u origin master
3、安裝配置遠端映象倉庫harbor
下面我們要配置私有的docker映象倉庫，用到的機器為：
操作伺服器: harbor:192.168.112.20
這裡倉庫我們選擇harbor，因為有web頁面，當然也可以使用 registry
harbor的配置
| harbor的安裝使用步驟 |
|---|
| 安裝並啟動docker並安裝docker-compose |
| 上傳harbor的離線包 |
| 匯入harbor的映象 |
| 編輯harbor.yml |
| 修改hostname 為自己的主機名,不用證書需要註釋掉https |
| harbor_admin_password 登入密碼 |
| 安裝compose |
| 執行指令碼 ./install.sh |
| 在瀏覽器裡輸入IP訪問 |
| docker login IP --家目錄下會有一個.docker資料夾 |
下面我們開始安裝
（1）、首先需要設定selinux、防火牆
[root@harbor ~]# getenforce
Disabled
[root@harbor ~]# systemctl disable firewalld.service --now
（2）、安裝並啟動docker並安裝docker-compose，關於docker-compose，這裡不用瞭解太多，一個輕量的docker編排工具
yum install -y docker-ce
yum install -y docker-compose
（3）、解壓harbor 安裝包：harbor-offline-installer-v2.0.6.tgz，匯入相關映象
harbor安裝包：harbor
[root@harbor ~]# ls
aliyun.sh anaconda-ks.cfg harbor-offline-installer-v2.0.6.tgz
[root@harbor ~]# tar -zxvf harbor-offline-installer-v2.0.6.tgz
harbor/harbor.v2.0.6.tar.gz
harbor/prepare
harbor/LICENSE
harbor/install.sh
harbor/common.sh
harbor/harbor.yml.tmpl
[root@harbor ~]# docker load -i harbor/harbor.v2.0.6.tar.gz
（4）、修改配置檔案
[root@harbor ~]# cd harbor/
[root@harbor harbor]# ls
common.sh harbor.v2.0.6.tar.gz harbor.yml.tmpl install.sh LICENSE prepare
[root@harbor harbor]# cp harbor.yml.tmpl harbor.yml
[root@harbor harbor]# ls
common.sh harbor.yml install.sh prepare
harbor.v2.0.6.tar.gz harbor.yml.tmpl LICENSE
[root@harbor harbor]# vim harbor.yml
（5）、harbor.yml：設定IP和使用者名稱密碼
# Configuration file of Harbor
# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: 192.168.112.20
# http related config
http:
# port for http, default is 80. If https enabled, this port will redirect to https port
port: 80
# https related config
#https:
# https port for harbor, default is 443
# port: 443
# The path of cert and key files for nginx
# certificate: /your/certificate/path
# private_key: /your/private/key/path
# # Uncomment following will enable tls communication between all harbor components
# internal_tls:
# # set enabled to true means internal tls is enabled
# enabled: true
# # put your cert and key files on dir
# dir: /etc/harbor/tls/internal
# Uncomment external_url if you want to enable external proxy
# And when it enabled the hostname will no longer used
# external_url: https://reg.mydomain.com:8433
# The initial password of Harbor admin
# It only works in first time to install harbor
# Remember Change the admin password from UI after launching Harbor.
harbor_admin_password: Harbor12345
（6）、./prepare && ./install.sh
[root@harbor harbor]# ./prepare
prepare base dir is set to /root/harbor
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /data/secret/keys/secretkey
Successfully called func: create_root_cert
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[root@harbor harbor]# ./install.sh
[Step 0]: checking if docker is installed ...
Note: docker version: 25.0.4
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 1.18.0
[Step 2]: loading Harbor images ...
Loaded image: goharbor/notary-server-photon:v2.0.6
Loaded image: goharbor/clair-photon:v2.0.6
Loaded image: goharbor/clair-adapter-photon:v2.0.6
Loaded image: goharbor/harbor-portal:v2.0.6
Loaded image: goharbor/harbor-core:v2.0.6
Loaded image: goharbor/harbor-db:v2.0.6
Loaded image: goharbor/harbor-jobservice:v2.0.6
Loaded image: goharbor/redis-photon:v2.0.6
Loaded image: goharbor/notary-signer-photon:v2.0.6
Loaded image: goharbor/harbor-log:v2.0.6
Loaded image: goharbor/harbor-registryctl:v2.0.6
Loaded image: goharbor/trivy-adapter-photon:v2.0.6
Loaded image: goharbor/chartmuseum-photon:v2.0.6
Loaded image: goharbor/prepare:v2.0.6
Loaded image: goharbor/nginx-photon:v2.0.6
Loaded image: goharbor/registry-photon:v2.0.6
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /root/harbor
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/passwd
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Creating harbor-log ... done
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
Creating registry ... done
Creating harbor-core ... done
Creating network "harbor_harbor" with the default driver
Creating nginx ... done
Creating harbor-db ...
Creating redis ...
Creating registryctl ...
Creating registry ...
Creating harbor-portal ...
Creating harbor-core ...
Creating nginx ...
Creating harbor-jobservice ...
✔ ----Harbor has been installed and started successfully.----
（7）、檢視相關的映象
[root@harbor harbor]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9572b7a8d0a8 goharbor/harbor-jobservice:v2.0.6 "/harbor/entrypoint.…" 5 minutes ago Up 5 minutes (healthy) harbor-jobservice
83b679a70258 goharbor/nginx-photon:v2.0.6 "nginx -g 'daemon of…" 5 minutes ago Up 5 minutes (healthy) 0.0.0.0:80->8080/tcp, :::80->8080/tcp nginx
e7c53195c856 goharbor/harbor-core:v2.0.6 "/harbor/entrypoint.…" 5 minutes ago Up 5 minutes (healthy) harbor-core
37884d3bb185 goharbor/registry-photon:v2.0.6 "/home/harbor/entryp…" 5 minutes ago Up 5 minutes (healthy) 5000/tcp registry
d4de74c6b397 goharbor/harbor-portal:v2.0.6 "nginx -g 'daemon of…" 5 minutes ago Up 5 minutes (healthy) 8080/tcp harbor-portal
3459fba85f4c goharbor/harbor-db:v2.0.6 "/docker-entrypoint.…" 5 minutes ago Up 5 minutes (healthy) 5432/tcp harbor-db
febab24100f4 goharbor/redis-photon:v2.0.6 "redis-server /etc/r…" 5 minutes ago Up 5 minutes (healthy) 6379/tcp redis
8b6f3d626464 goharbor/harbor-registryctl:v2.0.6 "/home/harbor/start.…" 5 minutes ago Up 5 minutes (healthy) registryctl
52a51aae1c1b goharbor/harbor-log:v2.0.6 "/bin/sh -c /usr/loc…" 5 minutes ago Up 5 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log
（8）、訪問測試
| harbor |
|---|
 |
 |
4、CI伺服器的docker配置
這裡因為我們要在192.168.112.10(CI伺服器)上push映象到192.168.112.20(私倉)，所有需要修改CI伺服器上的Docker配置。新增倉庫地址
操作伺服器： jenkins：192.168.112.10
（1）、修改配置檔案
[root@jenkins ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://2tefyfv7.mirror.aliyuncs.com"]
}
[root@jenkins ~]# vim /etc/docker/daemon.json
修改後的配置檔案
[root@jenkins ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://2tefyfv7.mirror.aliyuncs.com"],
"insecure-registries": ["192.168.112.20"]
}
載入使其生效
[root@jenkins ~]# systemctl daemon-reload
[root@jenkins ~]# systemctl restart docker
CI機器簡單測試一下
[root@jenkins ~]# docker login 192.168.112.20
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[root@jenkins ~]# docker tag busybox 192.168.112.20/library/busybox
[root@jenkins ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
192.168.112.20/library/busybox latest beae173ccac6 2 years ago 1.24MB
busybox latest beae173ccac6 2 years ago 1.24MB
beginor/gitlab-ce latest 5595d4ff803e 5 years ago 1.5GB
[root@jenkins ~]# docker push 192.168.112.20/library/busybox
Using default tag: latest
The push refers to repository [192.168.112.20/library/busybox]
01fd6df81c8e: Mounted from library/bysybox
latest: digest: sha256:62ffc2ed7554e4c6d360bce40bbcf196573dd27c4ce080641a2c59867e732dee size: 527
（2）、push一個映象，可以在私倉的web頁面檢視
| harbor |
|---|
 |
到這裡。我們配置了映象倉庫
5、安裝配置jenkins
操作伺服器： jenkins：192.168.112.10
（1）、映象jenkins拉取
[root@jenkins ~]# docker pull jenkins/jenkins:latest
latest: Pulling from jenkins/jenkins
0e29546d541c: Pull complete
11bbb8c402a7: Pull complete
cf91f018150b: Pull complete
a98e88c6f0f0: Pull complete
f67fc70d671a: Pull complete
edbe48067464: Pull complete
fa23ca93dd6b: Pull complete
00159d993c13: Pull complete
f28fb40a17cf: Pull complete
071d309df04b: Pull complete
78599f36e494: Pull complete
896a32d969fb: Pull complete
3f1a51ea9f7f: Pull complete
26e724f0bfad: Pull complete
b377e1ae1384: Pull complete
d3cdbe7e8b9f: Pull complete
f3b40ebc3458: Pull complete
Digest: sha256:c3fa8e7f70d1e873ea6aa87040c557aa53e6707eb1d5ecace7f6884a87588ac8
Status: Downloaded newer image for jenkins/jenkins:latest
docker.io/jenkins/jenkins:latest
（2）、建立共享卷，修改所屬組和使用者,和容器裡相同
這裡為什麼要改成 1000，是因為容器裡是以 jenkins 使用者的身份去讀寫資料，而在容器裡jenkins 的 uid 是 1000
[root@jenkins ~]# mkdir /jenkins
[root@jenkins ~]# chown 1000:1000 /jenkins
# 這裡為什麼要改成 1000，是因為容器裡是以 jenkins 使用者的身份去讀寫資料，而在容器裡jenkins 的 uid 是 1000
（3）、建立建立 jenkins 容器
[root@jenkins ~]# docker run -dit -p 8080:8080 -p 50000:50000 --name jenkins --privileged=true --restart=always -v /jenkins:/var/jenkins_home jenkins/jenkins:latest
f250456a77abeb916eb36781eafd8c17e3aad8ec26d5f6e006df4956d234f445
[root@jenkins ~]# docker ps | grep jenkins
f250456a77ab jenkins/jenkins:latest "/sbin/tini -- /usr/…" 17 seconds ago Up 16 seconds 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:50000->50000/tcp, :::50000->50000/tcp jenkins
| 訪問jenkins |
|---|
 |
| 因為要修改 jenkins 的配置，所以此時關閉 jenkins 容器 |
[root@jenkins ~]# docker stop jenkins
jenkins
（4）、更換國內清華大學映象,Jenkins下載外掛特別慢，更換國內的清華源的映象地址會快不少
[root@jenkins jenkins]# cat /jenkins/hudson.model.UpdateCenter.xml
<?xml version='1.1' encoding='UTF-8'?>
<sites>
<site>
<id>default</id>
<url>https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json</url>
</site>
</sites>[root@jenkins jenkins]# sed -i 's#updates.jenkins.io/update-center.json#mirrors.nghua.edu.cn/jenkins/updates/update-center.json#g ' /jenkins/hudson.model.UpdateCenter.xml
[root@jenkins jenkins]# cat /jenkins/hudson.model.UpdateCenter.xml <?xml version='1.1' encoding='UTF-8'?>
<sites>
<site>
<id>default</id>
<url>https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json</url>
</site>
</sites>
"http://www.google.com/" 替換為 "http://www.baidu.com/"
[root@jenkins jenkins]# yum install -y jq
[root@jenkins jenkins]# cat /jenkins/updates/default.json | jq '.connectionCheckUrl'
"https://www.google.com/"
[root@jenkins jenkins]# cat /jenkins/updates/default.json | jq 'keys'
[
"connectionCheckUrl",
"core",
"deprecations",
"generationTimestamp",
"id",
"plugins",
"signature",
"updateCenterVersion",
"warnings"
]
[root@jenkins jenkins]# sed -i s#http://www.google.com/#http://www.baidu.com/#g /jenkins/updates/default.json
替換後檢視
[root@jenkins jenkins]# cat /jenkins/updates/default.json | jq '.connectionCheckUrl'
"https://www.baidu.com/"
[root@jenkins jenkins]# cat /jenkins/updates/default.json | jq 'keys' [
"connectionCheckUrl",
"core",
"deprecations",
"generationTimestamp",
"id",
"plugins",
"signature",
"updateCenterVersion",
"warnings"
]
（5）、重啟docker，獲取登入密匙
[root@jenkins jenkins]# docker start jenkins
jenkins
[root@jenkins jenkins]# cat /jenkins/secret
secret.key secret.key.not-so-secret secrets/
[root@jenkins jenkins]# cat /jenkins/secrets/initialAdminPassword
f54e4a2c7dd249ce9f7d4f15121005d8
需要修改jenkins繫結的docker的啟動引數，ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock
vim /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock
修改映象庫啟動引數後需要重啟docker
[root@jenkins jenkins]# systemctl daemon-reload
[root@jenkins jenkins]# systemctl restart docker
（6）、安裝 docker 外掛
| jenkins相關配置，這裡的配置照著圖片就好，需要配置一個docker叢集供jenkins來根據Dockerfile構建映象並push到私倉，這裡docker叢集即為CI伺服器的docker |
|---|
 |
 |
 |
 |
 |
依此點選Manage Jenkins->Manage Plugins->AVAILABLE->Search 搜尋docker、docker-build-step |
 |
 |
 |
 |
 |
修改映象庫啟動引數，ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock |
 |
 |
| 關聯docker和jenkins |
 |
（7）、jenkins 安全設定
後面 gitlab 要和 jenkins 進行聯動，所以必須要需要對 jenkins 的安全做一些設定，依次點選 系統管理-全域性安全配置-授權策略，勾選"匿名使用者具有可讀許可權"
 |
 |
新增 JVM 執行引數 -Dhudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true 執行跨站請求訪問
[root@jenkins jenkins]# docker exec -u root -it jenkins /bin/bash
（8）、下載kubectl客戶端工具
這裡的話我們要透過jenkins上的kubectl客戶端連線k8s,所以我們需要安裝一個k8s的客戶端kubectl，下載k8s客戶端
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
yum install kubelet-1.22.2 kubeadm-1.22.2 kubectl-1.22.2 -y
systemctl enable kubelet && systemctl start kubelet
複製 kubeconfig 檔案
然後複製kubeconfig 證書,k8s叢集(一主兩從)中檢視證書位置/etc/kubernetes/admin.conf
[root@jenkins ~]# scp root@192.168.112.30:/etc/kubernetes/admin.conf .
The authenticity of host '192.168.112.30 (192.168.112.30)' can't be established.
ECDSA key fingerprint is SHA256:d5XrT2DNJojgq53QBNjVvg8JwuYbQyctCh2Bi2l2f0E.
ECDSA key fingerprint is MD5:96:8c:ec:78:63:de:7a:b2:3c:85:8d:5b:9f:f4:94:e8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.112.30' (ECDSA) to the list of known hosts.
root@192.168.112.30's password:
admin.conf 100% 5634 3.6MB/s 00:00
複製證書和k8s叢集客戶端工具到jenkins容器內
[root@jenkins ~]# docker cp admin.conf jenkins:/
Successfully copied 7.68kB to jenkins:/
[root@jenkins ~]# docker cp kubectl jenkins:/
Successfully copied 21.5MB to jenkins:/
（9）、kubectl命令測試
./kubectl --kubeconfig=admin.conf get pods -A
發現沒有許可權，這裡我們為了方便，直接賦予叢集中的
cluster-admin角色
kubectl create clusterrolebinding test --clusterrole=cluster-admin --user=jenkins
命令測試沒有問題
./kubectl --kubeconfig=admin.conf get pods -A
二 、hexo部落格系統CICD實戰
1、k8s叢集中配置hexo生產環境高可用
我們要部署Nginx來執行hexo部落格系統，hexo編譯完後為一堆靜態檔案，所以我們需要建立一個svc和一個deploy，使用SVC提供服務，使用deploy提供服務能力,使用Nginx+hexo的靜態檔案構成的映象
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginxdep
spec:
replicas: 2
selector:
matchLabels:
app: nginx
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: blog
name: web
resources:
requests:
cpu: 100m
restartPolicy: Always
（1）、deployments建立
這裡我們先用一個Nginx映象來代替hexo部落格的映象
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl apply -f nginx.yaml
deployment.apps/nginxdep created
檢視deployments和pod
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl get deployments.apps | grep nginxdep
nginxdep 2/2 2 2 109s
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl get pods -o wide | grep web
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl get pods -o wide | grep nginxdep
nginxdep-645bf755b9-2w8jv 1/1 Running 0 2m22s 10.244.171.164 vms82.liruilongs.github.io <none> <none>
nginxdep-645bf755b9-jfqxj 1/1 Running 0 2m22s 10.244.171.157 vms82.liruilongs.github.io <none> <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$
（2）、service建立
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl expose deploy nginxdep --port=8888 --target-port=80 --type=NodePort
service/nginxdep exposed
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl get svc -o wide | grep nginxdep
nginxdep NodePort 10.106.217.50 <none> 8888:31964/TCP 16s app=nginx
訪問測試沒有問題，之後我們配置好jenkins上的觸發器，直接替換就OK
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$curl 127.0.0.1:31964
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$
2、k8s叢集配置私倉地址
我們透過kubectl set命令更新deploy的映象時，獲取的映象是透過私倉獲取的，所以需要在啟動引數新增私倉地址
ExecStart=/usr/bin/dockerd --insecure-registry 192.168.26.56 -H fd:// --containerd=/run/containerd/containerd.sock
這裡所有的節點都需要設定後重啟docker
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$vim /usr/lib/systemd/system/docker.service
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$systemctl daemon-reload ;systemctl restart docker &
[1] 23273
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$ssh root@192.168.26.82
Last login: Sun Jan 16 06:09:07 2022 from 192.168.26.1
┌──[root@vms82.liruilongs.github.io]-[~]
└─$vim /usr/lib/systemd/system/docker.service
┌──[root@vms82.liruilongs.github.io]-[~]
└─$systemctl daemon-reload ;systemctl restart docker &
[1] 26843
┌──[root@vms82.liruilongs.github.io]-[~]
└─$exit
登出
Connection to 192.168.26.82 closed.
3、jenkins配置CICD流程
訪問jenkins，接下來才是重點，我們要的jenkins上配置整個CICD流程，從而實現自動化
| 訪問jenkins，接下來才是重點，我們要的jenkins上配置整個CICD流程，從而實現自動化 |
|---|
 |
 |
 |
這裡的Token我們設定為：4bf636c8214b7ff0a0fb，同時需要記住訪問方式：JENKINS_URL/job/liruilong-cicd/build?token=TOKEN_NAME |
| 構建觸發器選擇shell構建：克隆程式碼 |
 |
 |
| 選擇映象構建 |
 |
| 構建映象並push私倉 |
 |
| 這裡切記需要新增私倉的認證資訊，即上面設定的使用者名稱和密碼 |
 |
| 選擇shell構建，更新映象 |
|
 |
相關的文字資訊
cd ~
rm -rf blog
git clone http://192.168.26.55/root/blog.git
/var/jenkins_home/blog/
192.168.26.56/library/blog:${BUILD_NUMBER}
export KUBECONFIG=/kc1;
/kubectl set image deployment/nginxdep *="192.168.26.56/library/blog:${BUILD_NUMBER}" -n kube-system
4、配置 gitlab 和 jenkins 的聯動
| 訪問gitlab配置聯動 |
|---|
 |
 |
 |
 |
 |
 |
| 點選增加web鉤子 |
| /view/all/job/liruilong-cicd/build?token= |
 |
到這裡,聯動已經配置完成
5、編寫Dockerfile檔案，更新程式碼測試
下面我們編譯一下hexo，生成public的一個資料夾,然後上傳gitlab
PS F:\blogger> hexo g
.....
PS F:\blogger> git add .\public\
PS F:\blogger> git commit -m "編譯程式碼"
PS F:\blogger> git push
同時需要編寫Dockerfile檔案來建立映象
FROM docker.io/library/nginx:latest
MAINTAINER liruilong
ADD ./public/ /usr/share/nginx/html/
EXPOSE 80
CMD ["nginx", "-g","daemon off;"]
12345
PS F:\blogger> git add .
PS F:\blogger> git commit -m "Dockcerfile檔案編寫"
[master 217e0ed] Dockcerfile檔案編寫
1 file changed, 1 deletion(-)
PS F:\blogger> git push
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 307 bytes | 307.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
To http://192.168.26.55/root/blog.git
6690612..217e0ed master -> master
PS F:\blogger>
| jenkins輸出 |
|---|
 |
Started by remote host 192.168.26.1
Running as SYSTEM
Building in workspace /var/jenkins_home/workspace/liruilong-cicd
[liruilong-cicd] $ /bin/sh -xe /tmp/jenkins6108687102523328796.sh
+ cd /var/jenkins_home
+ rm -rf blog
+ git clone http://192.168.26.55/root/blog.git
Cloning into 'blog'...
Docker Build
Docker Build: building image at path /var/jenkins_home/blog
Step 1/5 : FROM docker.io/library/nginx:latest
---> f8f4ffc8092c
Step 2/5 : MAINTAINER liruilong
---> Running in e341b5562b64
Removing intermediate container e341b5562b64
---> 4e9f5aa47ab5
Step 3/5 : ADD ./public/ /usr/share/nginx/html/
---> 3956cff32507
Step 4/5 : EXPOSE 80
---> Running in b4c27124989d
Removing intermediate container b4c27124989d
---> ba9d1764d764
Step 5/5 : CMD ["nginx", "-g","daemon off;"]
---> Running in 61dca01a4883
Removing intermediate container 61dca01a4883
---> 2aadc5732a60
Successfully built 2aadc5732a60
Tagging built image with 192.168.26.56/library/blog:41
Docker Build Response : 2aadc5732a60
Pushing [192.168.26.56/library/blog:41]
The push refers to repository [192.168.26.56/library/blog]
89570901cdea: Preparing
65e1ea1dc98c: Preparing
88891187bdd7: Preparing
6e109f6c2f99: Preparing
0772cb25d5ca: Preparing
525950111558: Preparing
476baebdfbf7: Preparing
525950111558: Waiting
476baebdfbf7: Waiting
88891187bdd7: Layer already exists
6e109f6c2f99: Layer already exists
65e1ea1dc98c: Layer already exists
0772cb25d5ca: Layer already exists
89570901cdea: Pushing [> ] 301.6kB/28.75MB
89570901cdea: Pushing [==> ] 1.193MB/28.75MB
476baebdfbf7: Layer already exists
525950111558: Layer already exists
89570901cdea: Pushing [======> ] 3.917MB/28.75MB
89570901cdea: Pushing [==========> ] 5.996MB/28.75MB
89570901cdea: Pushing [==============> ] 8.097MB/28.75MB
89570901cdea: Pushing [==================> ] 10.76MB/28.75MB
89570901cdea: Pushing [=====================> ] 12.57MB/28.75MB
89570901cdea: Pushing [========================> ] 13.8MB/28.75MB
89570901cdea: Pushing [=========================> ] 14.71MB/28.75MB
89570901cdea: Pushing [===========================> ] 15.59MB/28.75MB
89570901cdea: Pushing [=============================> ] 16.79MB/28.75MB
89570901cdea: Pushing [===============================> ] 18.27MB/28.75MB
89570901cdea: Pushing [=================================> ] 19.45MB/28.75MB
89570901cdea: Pushing [===================================> ] 20.34MB/28.75MB
89570901cdea: Pushing [=====================================> ] 21.55MB/28.75MB
89570901cdea: Pushing [=======================================> ] 22.44MB/28.75MB
89570901cdea: Pushing [=========================================> ] 23.64MB/28.75MB
89570901cdea: Pushing [==========================================> ] 24.52MB/28.75MB
89570901cdea: Pushing [============================================> ] 25.42MB/28.75MB
89570901cdea: Pushing [==============================================> ] 26.61MB/28.75MB
89570901cdea: Pushing [===============================================> ] 27.19MB/28.75MB
89570901cdea: Pushing [=================================================> ] 28.69MB/28.75MB
89570901cdea: Pushing [==================================================>] 29.32MB
89570901cdea: Pushed
41: digest: sha256:c90b64945a8d063f7bcdcc39f00f91b6d83acafcd6b2ec6aba5b070474bafc37 size: 1782
Cleaning local images [2aadc5732a60]
Docker Build Done
[liruilong-cicd] $ /bin/sh -xe /tmp/jenkins246013519648603221.sh
+ export KUBECONFIG=/kc1
+ KUBECONFIG=/kc1
+ /kubectl set image deployment/nginxdep '*=192.168.26.56/library/blog:41' -n kube-system
deployment.apps/nginxdep image updated
Finished: SUCCESS
6、訪問hexo部落格系統
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl get deployments.apps | grep nginxdep
nginxdep 2/2 2 2 30h
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl get pods -o wide | grep nginxdep
nginxdep-bddfd9b5f-94d88 1/1 Running 0 110s 10.244.171.142 vms82.liruilongs.github.io <none> <none>
nginxdep-bddfd9b5f-z57qc 1/1 Running 0 35m 10.244.171.177 vms82.liruilongs.github.io <none> <none>
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl get svc -o wide | grep nginxdep
nginxdep NodePort 10.106.217.50 <none> 8888:31964/TCP 30h app=nginx
┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create]
└─$kubectl describe pods nginxdep-bddfd9b5f-94d88
Name: nginxdep-bddfd9b5f-94d88
Namespace: kube-system
Priority: 0
Node: vms82.liruilongs.github.io/192.168.26.82
Start Time: Fri, 04 Feb 2022 03:11:14 +0800
Labels: app=nginx
pod-template-hash=bddfd9b5f
Annotations: cni.projectcalico.org/podIP: 10.244.171.142/32
cni.projectcalico.org/podIPs: 10.244.171.142/32
Status: Running
IP: 10.244.171.142
IPs:
IP: 10.244.171.142
Controlled By: ReplicaSet/nginxdep-bddfd9b5f
Containers:
web:
Container ID: docker://669f48cb626d5067f40bb1aaa378268a7ee9879488b0b298a86271957c162316
Image: 192.168.26.56/library/blog:41
Image ID: docker-pullable://192.168.26.56/library/blog@sha256:c90b64945a8d063f7bcdcc39f00f91b6d83acafcd6b2ec6aba5b070474bafc37
Port: <none>
Host Port: <none>
State: Running
Started: Fri, 04 Feb 2022 03:11:15 +0800
Ready: True
Restart Count: 0
Requests:
cpu: 100m
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-trn5n (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-trn5n:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 4m10s default-scheduler Successfully assigned kube-system/nginxdep-bddfd9b5f-94d88 to vms82.liruilongs.github.io
Normal Pulling 4m9s kubelet Pulling image "192.168.26.56/library/blog:41"
Normal Pulled 4m9s kubelet Successfully pulled image "192.168.26.56/library/blog:41" in 67.814838ms
Normal Created 4m9s kubelet Created container web
Normal Started 4m9s kubelet Started container web
| 訪問hexo部落格系統 |
|---|
 |