傳統部署和k8s部署區別
通常使用傳統的部署的時候,我們一個web專案,網站的搭建,往往使用的如下的一種整體架構,可能有的公司在某一環節使用的東西是不一樣,但是大體的框架流程是都是差不多的
使用k8s部署,便於彈性伸縮,節約資源,釋出週期快,整體框架如下
環境
節點名 | IP | 軟體版本 | 硬體 | 網路 | 說明 |
---|---|---|---|---|---|
K8s-master | 192.168.43.190 | list 裡面都有 | 2C4G | Nat,內網 | 測試環境 |
K8s-node1 | 192.168.43.120 | list 裡面都有 | 2C4G | Nat,內網 | 測試環境 |
K8s-node2 | 192.168.43.9 | list 裡面都有 | 2C4G | Nat,內網 | 測試環境 |
K8s-harbor | 192.168.43.129 | list 裡面都有 | 2C4G | Nat,內網 | 測試環境 |
安裝執行harbor(http方式)
安裝docker
# 安裝一些必要的系統工具
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
# 新增軟體源資訊
# docker 官方源
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# 阿里雲源
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# 安裝前可以先更新 yum 快取:
sudo yum makecache fast
# CentOS7安裝 Docker-ce
yum -y install docker-ce # CentOS 中安裝
apt-get install docker-ce # Ubuntu 中安裝
pacman -S docker # Arch 中安裝
emerge --ask docker # Gentoo 中安裝
# 如果想安裝特定版本的Docker-ce版本,先列出repo中可用版本,然後選擇安裝
yum list docker-ce --showduplicates |sort -r
Loading mirror speeds from cached hostfile
Loaded plugins: fastestmirror
Installed Packages
docker-ce.x86_64 3:19.03.4-3.el7 docker-ce-stable
docker-ce.x86_64 3:19.03.4-3.el7 @docker-ce-stable
docker-ce.x86_64 3:19.03.3-3.el7 docker-ce-stable
docker-ce.x86_64 3:19.03.2-3.el7 docker-ce-stable
docker-ce.x86_64 3:19.03.1-3.el7 docker-ce-stable
yum install docker-ce-<VERSION STRING>
# 選擇安裝 docker-ce-18.06.1.ce
yum install docker-ce-18.06.1.ce -y
# Docker映象加速
# 沒有啟動/etc/docker目錄不存在,需要自己建立,docker啟動也會自己建立
# 為了期望我們的映象下載快一點,應該定義一個映象加速器,加速器在國內
mkdir /etc/docker
vim /etc/docker/daemon.json
{
"registry-mirrors": ["https://registry.docker-cn.com"]
}
# 啟動Docker後臺服務
systemctl start docker && systemctl enable docker
systemctl daemon-reload # 守護程式重啟
# 通過執行hello-world映象,驗證是否正確安裝了docker,或者通過檢視版本
docker run hello-world
docker version
Client: Docker Engine - Community
Version: 19.03.4
API version: 1.40
Go version: go1.12.10
Git commit: 9013bf583a
Built: Fri Oct 18 15:52:22 2019
OS/Arch: linux/amd64
Experimental: false
https請看我下面專門寫的文章
https://www.cnblogs.com/you-men/p/13121835.html
Harbor 可幫助使用者迅速搭建企業級的 Registry 服務, 它提供了管理圖形介面, 基於角色的訪問控制 ( Role Based Access Control), 映象遠端複製 (同步), AD/LDAP 整合, 以及審計日誌等企業使用者需求的功能, 同時還原生支援中文, 深受中國使用者的喜愛;
安裝harbor
注意
安裝harbor之前需要安裝docker
是
VMware
公司開源了企業級Registry
專案, 其的目標是幫助使用者迅速搭建一個企業級的Docker registry
服務。由於 Harbor 是基於 Docker Registry V2 版本,所以 docker 版本必須
>=1.10.0
docker-compose>=1.6.0
下載最新版 Docker Compose
curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
下載最新版Docker Harbor
wget https://github.com/goharbor/harbor/releases/download/v1.10.0-rc1/harbor-offline-installer-v1.10.0-rc1.tgz
# 對二進位制檔案應用可執行許可權:
sudo chmod +x /usr/local/bin/docker-compose
# 測試是否安裝成功
docker-compose --version
# 按照上面給的docker harbor地址,下載離線安裝包
tar xvf harbor-offline-installer-v1.8.1.tgz -C /usr/local/
vim /usr/local/harbor/harbor.yml
hostname: 47.92.24.137
# 執行安裝指令碼
./install.sh
[Step 0]: checking installation environment ...
Note: docker version: 19.03.4
Note: docker-compose version: 1.22.0
[Step 1]: loading Harbor images ...
Loaded image: goharbor/harbor-core:v1.8.1
Loaded image: goharbor/harbor-registryctl:v1.8.1
Loaded image: goharbor/redis-photon:v1.8.1
Loaded image: goharbor/notary-server-photon:v0.6.1-v1.8.1
Loaded image: goharbor/chartmuseum-photon:v0.8.1-v1.8.1
Loaded image: goharbor/harbor-db:v1.8.1
Loaded image: goharbor/harbor-jobservice:v1.8.1
Loaded image: goharbor/nginx-photon:v1.8.1
Loaded image: goharbor/registry-photon:v2.7.1-patch-2819-v1.8.1
Loaded image: goharbor/harbor-migrator:v1.8.1
Loaded image: goharbor/prepare:v1.8.1
Loaded image: goharbor/harbor-portal:v1.8.1
Loaded image: goharbor/harbor-log:v1.8.1
Loaded image: goharbor/notary-signer-photon:v0.6.1-v1.8.1
Loaded image: goharbor/clair-photon:v2.0.8-v1.8.1
[Step 2]: preparing environment ...
prepare base dir is set to /usr/local/harbor
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /secret/keys/secretkey
Generated certificate, key file:/secret/core/private_key.pem, cert file:/secret/registry/root.crt
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[Step 3]: starting Harbor ...
✔ ----Harbor has been installed and started successfully.----
Now you should be able to visit the admin portal at http://47.92.24.137.
For more details, please visit https://github.com/goharbor/harbor
接下來我們可以直接瀏覽器訪問配置檔案定義的IP或者域名加埠
預設使用者密碼: admin/Harbor12345
修改harbor埠
# 因為harbor預設埠是80,而大多數時候是不希望使用80埠,修改方法如下
# vim harbor.yml
# 找到port選項修改埠,然後執行./install 就會使用配置檔案埠
# 還有一種情況就是更改已有harbor的配置
vim docker-compose.yml
dns_search: .
ports:
- 99:80
auth:
token:
issuer: harbor-token-issuer
realm: http://47.92.24.137:99/service/token
rootcertbundle: /etc/registry/root.crt
service: harbor-registry
docker-compose down -v
docker-compose up -d
使用harbor
為了體現出效果,建議使用非harbor的另一臺機器
# 映象推送
docker login 47.92.24.137:99 -u admin -p Harbor12345
vim /etc/docker/daemon.json
{
"insecure-registries":["192.168.43.129"]
}
systemctl daemon-reload
systemctl restart docker
# 因為docker預設使用的是https協議,而搭建harbor是http提供服務的,
# 所以要配置可信任,或者強制docker login和docker push 走http的80埠,而不是443埠.
docker tag daocloud.io/library/nginx:latest 192.168.43.129/library/nginx:latest
docker push 192.168.43.129/library/nginx:latest
PHP部署專案流程
當我們把專案遷移到K8S平臺上時,首先我們需要了解的是整個部署的流程,按照這個流程部署,才能避免出現問題,也方便大家理解
製作映象
使用Dockerfile製作映象,把應用程式、執行環境、檔案系統一起打包成一個映象,然後推送到Harbor映象倉庫中 首先在k8s的master節點進行操作
[root@k8s-master ]# git clone https://github.com/zhangdongdong7/php-demo.git
[root@k8s-master ]# cd php-demo
[root@k8s-master php-demo]# ls
deployment.yaml ingress.yaml mysql.yaml namespace.yaml README.md service.yaml wordpress
使用wordpress建立一個部落格網站,開啟wordpress,編寫Dockerfile構建映象,然後推送到一個harbor映象倉庫中,可以看前面章節,harbor映象的搭建,這裡是使用的harbor映象倉庫地址為192.168.43.129
[root@k8s-master php-demo]cd wordpress
[root@k8s-master wordpress]# vim Dockerfile
FROM lizhenliang/nginx-php:latest
MAINTAINER www.ctnrs.com
ADD . /usr/local/nginx/html
[root@k8s-master wordpress]docker login 192.168.43.129
[root@k8s-master wordpress] docker build -t 192.168.43.129/library/php-demo:latest .
[root@k8s-master wordpress] docker push 192.168.43.129/library/php-demo:latest
建立控制器管理Pod
回到php-demo目錄編寫yaml,首先部署一個test的命令空間
[root@k8s-master wordpress]# cd ../
[root@k8s-master php-demo]# vim namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: test
[root@k8s-master php-demo]# kubectl apply -f namespace.yaml
kubectl get ns
NAME STATUS AGE
default Active 5h59m
kube-node-lease Active 5h59m
kube-public Active 5h59m
kube-system Active 5h59m
kubernetes-dashboard Active 5h18m
test Active 3s
建立認證
kubectl create secret docker-registry regsecret --docker-server=192.168.43.129 --docker-username=admin --docker-password=Harbor12345 -n test
配置deployment控制器
編寫deployment.yaml控制器,這裡需要把image進行修改成剛才推送到Harbor映象倉庫中的地址
[root@k8s-master php-demo]# vim deployment.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: php-demo
namespace: test
spec:
replicas: 2
selector:
matchLabels:
project: www
app: php-demo
template:
metadata:
labels:
project: www
app: php-demo
spec:
imagePullSecrets:
- name: registry-pull-secret
containers:
- name: nginx
image: 192.168.43.129/library/php-demo:latest
imagePullPolicy: Always
ports:
- containerPort: 80
name: web
protocol: TCP
resources:
requests:
cpu: 0.5
memory: 256Mi
limits:
cpu: 1
memory: 1Gi
resources:
requests:
cpu: 0.5
memory: 256Mi
limits:
cpu: 1
memory: 1Gi
livenessProbe:
httpGet:
path: /status.php
port: 80
Pod資料持久化
因為是一個靜態的網站,基本不需要做持久化,直接把程式碼打包到映象中
暴露應用
建立一個service來暴露應用,直接使用的了ingress控制器的方式暴露應用了
[root@k8s-master php-demo]# vim service.yaml
apiVersion: v1
kind: Service
metadata:
name: php-demo
namespace: test
spec:
selector:
project: www
app: php-demo
ports:
- name: web
port: 80
targetPort: 80
[root@k8s-master php-demo]# kubectl apply -f service.yaml
[root@k8s-master php-demo]# kubectl apply -f deployment.yaml
[root@master php-demo]# kubectl get pods,svc -n test -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/php-demo-65bc56fdb8-grklk 1/1 Running 0 15s 10.244.0.11 master <none> <none>
pod/php-demo-65bc56fdb8-td6nv 0/1 Running 0 15s 10.244.2.7 node2 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/php-demo ClusterIP 10.0.0.221 <none> 80/TCP 4m16s app=php-demo,project=www
建立ingress對外發布應用
編寫yaml,首先建立ingress控制器,建立ingress,最後可以檢視pod,svc,ingress的狀態,全部都正常可以開始下一步,如果有異常可以使用kubectl describe命令檢視日誌進行排錯
[root@k8s-master java-demo]# kubectl apply -f mandatory.yaml
[root@k8s-master php-demo]# vim ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: php-demo
namespace: test
spec:
rules:
- host: php.ctnrs.com
http:
paths:
- path: /
backend:
serviceName: php-demo
servicePort: 80
[root@k8s-master php-demo]# kubectl apply -f ingress.yaml
kubectl get pods,svc,ingress -n test -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/php-demo-65bc56fdb8-grklk 1/1 Running 0 2m31s 10.244.0.11 master <none> <none>
pod/php-demo-65bc56fdb8-td6nv 1/1 Running 0 2m31s 10.244.2.7 node2 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/php-demo ClusterIP 10.0.0.221 <none> 80/TCP 6m32s app=php-demo,project=www
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.extensions/php-demo <none> php.ctnrs.com 80 60s
可以在叢集之外找一個資料庫,也可以在harbor映象倉庫安裝
docker run -d -p 3306:3306 -e MYSQL_ROOT_PASSWORD=123456 daocloud.io/library/mysql:5.7.5
docker exec -it mysql:5.7 /bin/bash
mysql -uroot -p$MYSQL_ROOT_PASSWORD
create database wp;
grant all on youmen.* TO 'youmen'@'%' IDENTIFIED BY 'zhoujian20';
# 我們可以進入已經執行的pod修改下資料庫ip,最好構建映象時候就修改好
# 此處就不演示了,我們能訪問到錯誤頁面說明服務訪問暴露是沒有問題的
繫結hosts,訪問域名驗證
windows系統,hosts檔案地址:C:\Windows\System32\drivers\etc
Mac系統sudo vi /private/etc/hosts 編輯hosts檔案,在底部加入域名和ip,用於解析
這個ip地址為node節點ip地址 加入如下命令,然後儲存
在瀏覽器中,輸入php.ctnrs.com,會跳轉到初始化設定介面,設定對應的賬號,然後安裝,登入,然後就可以編輯文章釋出了,一個簡單的WordPress的php網站搭建完成
Java專案部署流程
製作映象
使用Dockerfile製作映象,把應用程式、執行環境、檔案系統一起打包成一個映象,然後推送到Harbor映象倉庫中
首先在k8s的master節點進行操作
[root@k8s-master ]# git clone https://github.com/zhangdongdong7/java-demo.git
[root@k8s-master java-demo]# cd java-demo
[root@k8s-master java-demo]# ls
deployment.yaml ingress.yaml mysql.yaml README.md tomcat-java-demo-master.zipdeploy.yml mandatory.yaml namespace.yaml service.yaml
[root@k8s-master java-demo]# unzip tomcat-java-demo-master.zip
[root@k8s-master java-demo]# cd tomcat-java-demo-master/
安裝環境
[root@k8s-master tomcat-java-demo-master]# yum install java-1.8.0-openjdk maven -y
編譯構建
如果maven構建慢可以使用阿里源
vim /etc/maven/settings.xml,大概在(159-164行),更換為如下程式碼
[root@k8s-master tomcat-java-demo-master]# vim /etc/maven/settings.xml
...
<mirror>
<id>central</id>
<mirrorOf>central</mirrorOf>
<name>aliyun maven</name>
<url>https://maven.aliyun.com/repository/public</url>
</mirror>
...
[root@k8s-master tomcat-java-demo-master]# ls
db Dockerfile LICENSE pom.xml README.md src target
# 這一次我們提前修改好資料庫配置再生成映象
vim src/main/resources/application.yml
在tomcat目錄下建立映象
[root@k8s-master tomcat-java-demo-master]# docker login 192.168.73.136
Authenticating with existing credentials…WARNING!
Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning.
Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin
Succeeded
[root@k8s-master tomcat-java-demo-master]# docker build -t 192.168.43.129/library/java-demo:latest .
[root@k8s-master tomcat-java-demo-master]# docker push 192.168.73.136/test/java-demo:latest
回到上一級java-demo目錄中
[root@k8s-master tomcat-java-demo-master]# cd ../
[root@k8s-master java-demo]# ls
db deploy.yml mandatory.yaml namespace.yaml service.yaml tomcat-java-demo-master.zip
deployment.yaml ingress.yaml mysql.yaml README.md tomcat-java-demo-master
[root@k8s-master java-demo]#
建立一個test的命令空間
[root@k8s-master java-demo]# cat namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: test namespace.yaml
[root@k8s-master java-demo]# kubectl apply -f namespace.yaml
建立控制器管理Pod
編寫deployment.yaml,建立pods,這裡需要把image進行修改成剛才推送到Harbor映象倉庫中的地址
[root@k8s-master java-demo]# vim deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-java-demo
namespace: test
spec:
replicas: 2
selector:
matchLabels:
project: www
app: java-demo
template:
metadata:
labels:
project: www
app: java-demo
spec:
imagePullSecrets:
- name: registry-pull-secret
containers:
- name: tomcat
image: 192.168.73.136/test/java-demo:latest
imagePullPolicy: Always
ports:
- containerPort: 8080
name: web
protocol: TCP
resources:
requests:
cpu: 0.25
memory: 1Gi
limits:
cpu: 1
memory: 2Gi
livenessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 20
readinessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 60
[root@k8s-master java-demo]# kubectl adpply -f deployment.yaml
Pod資料持久化
這裡演示的是一個靜態的web網站,基本不需要做持久化,直接把程式碼打包到了映象中
暴露應用
建立一個service來暴露應用,直接使用的了ingress控制器的方式暴露應用了
[root@k8s-master java-demo]# cat service.yaml
apiVersion: v1
kind: Service
metadata:
name: tomcat-java-demo
namespace: test
spec:
selector:
project: www
app: java-demo
ports:
- name: web
port: 80
targetPort: 8080
[root@k8s-master java-demo]# kubectl apply -f service.yaml
建立ingress對外發布應用
編寫yaml,因為剛才php專案建立過ingress控制器,因此可以不用建立,直接建立ingress,最後可以檢視pod,svc,ingress的狀態,全部都正常可以開始下一步,如果有異常可以使用kubectl describe命令檢視日誌進行排錯
[root@k8s-master java-demo]# cat ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tomcat-java-demo
namespace: test
spec:
rules:
- host: java.ctnrs.com
http:
paths:
- path: /
backend:
serviceName: tomcat-java-demo
servicePort: 80
[root@k8s-master java-demo]# kubectl apply -f ingress.yaml
[root@k8s-master java-demo]# kubectl get pod,svc,ingress -n test -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/php-demo-66d9c64968-4r4vn 1/1 Running 0 24h 10.244.1.73 k8s-node01 <none> <none>
pod/php-demo-66d9c64968-8zw9s 1/1 Running 0 24h 10.244.2.43 k8s-node02 <none> <none>
pod/tomcat-java-demo-5f4f64dd4b-tcmtv 1/1 Running 0 24h 10.244.2.42 k8s-node02 <none> <none>
pod/tomcat-java-demo-5f4f64dd4b-vvx5x 1/1 Running 0 24h 10.244.1.72 k8s-node01 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/php-demo NodePort 10.1.136.96 <none> 80:32625/TCP 24h app=php-demo,project=www
service/tomcat-java-demo ClusterIP 10.1.198.15 <none> 80/TCP 24h app=java-demo,project=www
NAME HOSTS ADDRESS PORTS AGE
ingress.extensions/php-demo php.ctnrs.com 80 24h
ingress.extensions/tomcat-java-demo java.ctnrs.com 80 24h
繫結本機hosts,訪問域名驗證
windows系統,hosts檔案地址:C:\Windows\System32\drivers\etc
Mac系統sudo vi /private/etc/hosts 編輯hosts檔案,在底部加入域名和ip,用於解析,這裡的ip是node的ip地址 加入如下命令,然後儲存
192.168.43.120 java.ctnrs.com