陷入jenkins自動化部署的坑

原文地址

背景

Jenkins官網

Build great things at any scale The leading open source automation server, Jenkins provides hundreds of plugins to support building, deploying and automating any project.

其是一款開源自動化部署伺服器，由java編寫，目的是為了持久整合。

具體步驟

• 專案和遠端倉庫
  • 將最新的程式碼推送到遠端倉庫
• 遠端倉庫和Jenkins
  • 定時獲取遠端倉庫上最新的完整專案下載到本地
• Jenkins與遠端伺服器
  • 通過jenkins上傳到遠端伺服器

安裝

先新增其Debian軟體包，然後更新儲存庫,最後使用儲存庫apt-get安裝jenkins。

安裝執行環境

安裝jdk

sudo apt-get install openjdk-8-jdk
複製程式碼

安裝完成如下

update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jconsole to provide /usr/bin/jconsole (jconsole) in auto mode
Processing triggers for libc-bin (2.23-0ubuntu10) ...
Processing triggers for systemd (229-4ubuntu21.2) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for ca-certificates (20170717~16.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...

done.
done.
複製程式碼

儲存庫金鑰新增到系統

wget -q -O - https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -
複製程式碼

出現ok，新增成功

Debian包儲存庫地址新增到伺服器sources.list

echo deb http://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list
複製程式碼

新增成功如下

deb http://pkg.jenkins.io/debian-stable binary/
複製程式碼

更新儲存庫

sudo apt-get update
複製程式碼

更新成功如下

Ign:17 http://pkg.jenkins.io/debian-stable binary/ InRelease                   
Get:18 http://pkg.jenkins.io/debian-stable binary/ Release [2042 B]
Get:19 http://pkg.jenkins.io/debian-stable binary/ Release.gpg [181 B]
Ign:20 https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/4.0 InRelease
Hit:21 https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/4.0 Release
Get:23 http://pkg.jenkins.io/debian-stable binary/ Packages [14.7 kB]
Fetched 6813 kB in 3s (2187 kB/s)     
Reading package lists... Done
複製程式碼

安裝jenkins

sudo apt-get install jenkins
複製程式碼

安裝成功如下

perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
locale: Cannot set LC_ALL to default locale: No such file or directory
Selecting previously unselected package daemon.
(Reading database ... 136881 files and directories currently installed.)
Preparing to unpack .../daemon_0.6.4-1_amd64.deb ...
Unpacking daemon (0.6.4-1) ...
Selecting previously unselected package jenkins.
Preparing to unpack .../jenkins_2.164.2_all.deb ...
Unpacking jenkins (2.164.2) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu21.2) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up daemon (0.6.4-1) ...
Setting up jenkins (2.164.2) ...
Processing triggers for systemd (229-4ubuntu21.2) ...
Processing triggers for ureadahead (0.100.0-19) ...
複製程式碼

修改預設埠8080

• 修改 /etc/init.d/jenkins 指令碼

sudo vim /etc/init.d/jenkins
複製程式碼

修改$HTTP_PORT改成所需的埠

  # Verify that the jenkins port is not already in use, winstone does not exit
    # even for BindException
    check_tcp_port "http" "$HTTP_PORT" "1314" "$HTTP_HOST" "0.0.0.0" || return 2

    # If the var MAXOPENFILES is enabled in /etc/default/jenkins then set the max open files to the
    # proper value

複製程式碼

• 修改 /etc/default/jenkins 檔案

sudo vim  /etc/default/jenkins
複製程式碼

修改HTTP_PORT改成所需的埠

ration, build records,
#   that sort of things.
#
#   If commented out, the value from the OS is inherited,  which is normally 022 (as of Ubuntu 12.04,
#   by default umask comes from pam_umask(8) and /etc/login.defs

# UMASK=027

# port for HTTP connector (default 8080; disable with -1)
HTTP_PORT=1314


# servlet context, important if you want to use apache proxying
PREFIX=/$NAME

# arguments to pass to jenkins.
# --javahome=$JAVA_HOME
# --httpListenAddress=$HTTP_HOST (default 0.0.0.0)
# --httpPort=$HTTP_PORT (default 8080; disable with -1)
# --httpsPort=$HTTP_PORT
# --argumentsRealm.passwd.$ADMIN_USER=[password]
     
複製程式碼

• 重啟伺服器

sudo systemctl restart jenkins
複製程式碼

改完後重啟出現bug

Warning: jenkins.service changed on disk. Run 'systemctl daemon-reload' to reload units.
複製程式碼

解決方法：

systemctl daemon-reload
複製程式碼

systemctl start jenkins
複製程式碼

新埠是1314

啟動jenkins

sudo systemctl start jenkins
複製程式碼

• 用下面命令測試或者直接ip+埠或者0.0.0.0:8080訪問jenkins

sudo systemctl status jenkins
複製程式碼

• 成功如下：

● jenkins.service - LSB: Start Jenkins at boot time
   Loaded: loaded (/etc/init.d/jenkins; bad; vendor preset: enabled)
   Active: active (exited) since 四 2019-04-18 09:00:28 CST; 8h ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1136 ExecStart=/etc/init.d/jenkins start (code=exited, status=0/SUCCE

4月 18 09:00:24 devue-System-Product-Name systemd[1]: Starting LSB: Start Jenkin
4月 18 09:00:27 devue-System-Product-Name jenkins[1136]: Correct java version fo
4月 18 09:00:27 devue-System-Product-Name jenkins[1136]:  * Starting Jenkins Aut
4月 18 09:00:27 devue-System-Product-Name su[1511]: Successful su for jenkins by
4月 18 09:00:27 devue-System-Product-Name su[1511]: + ??? root:jenkins
4月 18 09:00:27 devue-System-Product-Name su[1511]: pam_unix(su:session): sessio
4月 18 09:00:28 devue-System-Product-Name jenkins[1136]:    ...done.
4月 18 09:00:28 devue-System-Product-Name systemd[1]: Started LSB: Start Jenkins
複製程式碼

• jenkins預設埠就是8080，需要解鎖jenkins

• 將拷貝的密碼填入,點選繼續

sudo cat /var/lib/jenkins/secrets/initialAdminPassword
複製程式碼

• 點選安裝建議的外掛

• 開始安裝，等的時間會比較久

• 安裝完成，會提示設定管理使用者。可以選擇跳過，但是密碼未知。最好還是建立使用者。

• 建立好使用者後，進入例項配置

• 歡迎來到jenkins皮膚，開始我們的表演

jenkins無法登入或者空白頁面

解決方案：

vim /var/lib/jenkins/config.xml
複製程式碼

在其中版本資訊後加入

<authorizationStrategy class="hudson.security.AuthorizationStrategy$Unsecured"/>
  <securityRealm class="hudson.security.SecurityRealm$None"/>
複製程式碼

重啟伺服器即可

sudo systemctl restart jenkins
複製程式碼

連線github

在jenkin上的操作：安裝相關外掛

• 安裝外掛'Publish Over SSH',連線遠端伺服器的外掛。

下圖是已經安裝後的

• 安裝外掛'GitHub Integration Plugin',GitHub整合外掛

在github上的操作：配置webhook

• github專案中點選Settings選項卡
  • 點選webhook選單項
    • 新增webhook

      

      • Payload URL中http://+jenkins部署的ip和埠號+/github-webhook/
      • Content type中選擇application/json
      • Which events would you like to trigger this webhook?選擇just the push event
      • 選擇Active
      • 點選Update webhook

        

jenkins拉取github上vue程式碼在本地啟動

• 新建任務
  • 任務名隨意
  • 選擇構建一個自由風格的軟體專案，最後確定

• 繫結github專案

• 繫結專案的下載連結

  

  • 選擇Git
  • 在Repositories中的Repository URL填入專案下載連結(http)
  • Credentials中新增身份資訊
    • 在型別中選擇Username with password
    • 使用者名稱和密碼就是github的賬號和密碼，最後確定

      

  • Branches to build選擇部署的分支(*/分支名)

• 構建觸發器

  • 選擇GitHub hook trigger for GITScm polling

    

• 構建

  • 執行shell
    • 命令

    cd /var/lib/jenkins/workspace/vue 
    npm install chromedriver --chromedriver_cdnurl=http://cdn.npm.taobao.org/dist/chromedriver 
    npm install
    npm run dev
    複製程式碼

    

    • 命令不能缺，否則包下不完整

        npm install chromedriver --chromedriver_cdnurl=http://cdn.npm.taobao.org/dist/chromedriver 
    複製程式碼

• 儲存
  任務建立完成，jenkins大功告成

jenkins拉取github上vue程式碼在遠端伺服器啟動

• 連線遠端伺服器
  • 系統管理->系統設定->Publish over SSH
    • Passphrase:輸入jenkins的密碼
    • Key:jenkins私鑰
    • Name:伺服器ip名字
    • Hostname:伺服器ip
    • Username:伺服器中的使用者名稱
    • Remote Directory:專案地址
    • 點選Test Configuration,出現success，那麼連線成功

      

  • 遠端伺服器
    • 設定公鑰

獲取公鑰和私鑰

su jenkins
ssh-keygen -t rsa
複製程式碼

判斷是否生成公鑰和私鑰

ls -l /var/lib/jenkins/.ssh/
複製程式碼

jenkins@devue-System-Product-Name:/home/devue$ ls -l /var/lib/jenkins/.ssh/
total 8
-rw------- 1 jenkins jenkins 1675 4月  19 16:26 id_rsa
-rw-r--r-- 1 jenkins jenkins  415 4月  19 16:26 id_rsa.pub
複製程式碼

拷貝公鑰

cd /var/lib/jenkins/.ssh/
cat id_rsa.pub >> authorized_keys
chmod 600 authorized_keys
複製程式碼

將公鑰放進遠端伺服器 authorized_keys

  vim ~/.ssh/authorized_keys 
複製程式碼

• 立即構建 傳輸失敗

  

  卡在這裡，經過的大佬指點迷津，謝謝啦

連線gitlab

原始碼管理

Repository URL必須要http請求

構建

安裝Gitlab Hook

如果沒有安裝Gitlab Hook和gitlab上增加webhook的話，會報錯

• gitlab專案側邊欄中Settings-Integrations增加webhook

  

• 新增Gitlab Hook外掛

  

jenkins輪詢gitlab(必須要是管理員身份)

jenkins想要執行下一個構建任務的時候，是必須等上一個任務完成的（沒有勾選併發執行任務）

由於npm run dev，所以在定時構建的時候，並沒有收到理想效果。 需求是：維護程式，定時執行 步驟如下：

• 加上pm2構建專案，pm2入口

• 構建觸發器，定時構建和輪詢SCM二選一

  

• 構建中的執行shell為

cd /var/lib/jenkins/workspace/ceres-cms-vue
npm install chromedriver --chromedriver_cdnurl=http://cdn.npm.taobao.org/dist/chromedriver
npm install
npm run build
npm run pm2
pm2 list
複製程式碼

輪詢

• 定時構建：無論有無最新程式碼，都按時構建
• 輪詢SCM：只要程式碼有更新，都會構建

---

• 構建語法說明：

  • 首先格式為：* * * * *（五個星）

選項	意思
第一個*表示分鐘	取值0~59
第二個*表示小時	取值0~23
第三個*表示一個月的第幾天	取值1~31
第四個*表示第幾月	取值1~12
第五個*表示一週中的第幾天	取值0~7，其中0和7代表的都是週日

• 使用舉例（不加H為時刻之前）：

選項	意思
每隔1分鐘構建一次	H/1 * * * *
每隔1小時構建一次	H H/1 * * *
每月1號構建一次	H H 1 * *

• 定時構建和輪詢SCM使用互不衝突，具體如何組合，需要根據專案情況合理配置；

測試

• 本地push程式碼到github

  

• 點選立即構建

  

• 構建執行狀態

  

• 點選#13,進入工程詳情

  

• 控制檯輸出結果

執行中控制檯輸出
Started by user unknown or anonymous
Started by user unknown or anonymous
Started by user unknown or anonymous
Building in workspace /var/lib/jenkins/workspace/vue
using credential 12dc8386-52e8-4c57-b667-bd8d263626cd
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/qiufeihong2018/vuepress-app.git # timeout=10
Fetching upstream changes from https://github.com/qiufeihong2018/vuepress-app.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials 
 > git fetch --tags --progress https://github.com/qiufeihong2018/vuepress-app.git +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision ab0c6b6de9b810dcd9fd107c6329d1e782054976 (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ab0c6b6de9b810dcd9fd107c6329d1e782054976
Commit message: "Merge branch 'master' of github.com:qiufeihong2018/vuepress-app"
 > git rev-list --no-walk 9b7e2475ffaef9a60cc38cec1c660d0f9d8dc490 # timeout=10
[vue] $ /bin/sh -xe /tmp/jenkins5471132310334499324.sh
+ cd /var/lib/jenkins/workspace/vue
+ npm install chromedriver --chromedriver_cdnurl=http://cdn.npm.taobao.org/dist/chromedriver

> chromedriver@73.0.0 install /var/lib/jenkins/workspace/vue/node_modules/chromedriver
> node install.js

ChromeDriver binary exists. Validating...
ChromeDriver 73.0.3683.20 (8e2b610813e167eee3619ac4ce6e42e3ec622017)

ChromeDriver is already available at '/tmp/73.0.3683.20/chromedriver/chromedriver'.
Copying to target path /var/lib/jenkins/workspace/vue/node_modules/chromedriver/lib/chromedriver
Fixing file permissions
Done. ChromeDriver binary available at /var/lib/jenkins/workspace/vue/node_modules/chromedriver/lib/chromedriver/chromedriver
npm WARN vuepress-app@1.0.0 No repository field.
npm WARN vuepress-app@1.0.0 scripts['server'] should probably be scripts['start'].
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.7 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.7: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

+ chromedriver@73.0.0
updated 1 package and audited 14738 packages in 13.642s
found 15 vulnerabilities (1 low, 7 moderate, 7 high)
  run `npm audit fix` to fix them, or `npm audit` for details
+ npm install
npm WARN vuepress-app@1.0.0 No repository field.
npm WARN vuepress-app@1.0.0 scripts['server'] should probably be scripts['start'].
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.7 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.7: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

audited 14738 packages in 8.37s
found 15 vulnerabilities (1 low, 7 moderate, 7 high)
  run `npm audit fix` to fix them, or `npm audit` for details
+ npm run dev

> vuepress-app@1.0.0 dev /var/lib/jenkins/workspace/vue
> vuepress dev docs


 WAIT  Extracting site metadata...
<button @click.ctrl="onClick">A</button>
<button @click.ctrl.exact="onCtrlClick">A</button>
<button @click.exact="onClick">A</button>
 TIP  override.styl has been split into 2 APIs, we recommend you upgrade to continue.
      See: https://v0.vuepress.vuejs.org/default-theme-config/#simple-css-override
[2:52:53 PM] Compiling Client
[2:52:56 PM] Compiled Client in 4s
c
 DONE  [14:52:56] Build 854bb5 finished in 3536 ms! 

> VuePress dev server listening at http://localhost:7777/
c[2:52:57 PM] Compiling Client
[2:52:57 PM] Compiled Client in 204ms
c
 DONE  [14:52:57] Build 056d13 finished in 208 ms! (http://localhost:7777/)

複製程式碼

• 專案啟動成功
• 修改提交後，一鍵立即構建，就可以將最新提交的程式碼執行起來

解決構建完成後自動殺掉衍生程式的問題

• 修改BUILD_ID

jenkins預設會在構建完成後殺掉構建過程中shell命令觸發的衍生程式。jenkins根據BUILD_ID識別某個程式是否為構建過程的衍生程式，故修改BUILD_ID後，jenkins就無法識別是否為衍生程式，則此程式能在後臺保留執行

OLD_BUILD_ID=$BUILD_ID
echo $OLD_BUILD_ID
BUILD_ID=dontKillMe
cd /var/lib/jenkins/workspace/ceres-cms-vue
npm install chromedriver --chromedriver_cdnurl=http://cdn.npm.taobao.org/dist/chromedriver
npm install
npm run build
npm run pm2
pm2 list
BUILD_ID=$OLD_BUILD_ID
echo $BUILD_ID
複製程式碼

殺掉改變id的衍生程式

netstat -lntp
kill -g {id}
複製程式碼

參考文獻

如何在Ubuntu 16.04上安裝Jenkins

Jenkins解除安裝方法（Windows/Linux/MacOS）

jenkins 忘記admin使用者賬號密碼

手把手教你搭建Jenkins+Github持續整合環境

Integrate with GitHub: build after each commit (Get started with Jenkins, part 13)

實戰筆記：Jenkins打造強大的前端自動化工作流

教你用Vue、GitLab、Jenkins、Nginx實現自動打包釋出上線

Jenkins+github 前端自動化部署

Jenkins自動化構建vue專案然後釋出到遠端伺服器

Jenkins自動化釋出前端程式碼VUE (配置模式)

jenkins配置publish over ssh遇到的問題

jenkins使用publishover ssh外掛連線應用機器時，報Message Auth fail的問題

Jenkins+git+webhook自動觸發部署和測試任務

Jenkins定時構建和輪詢SCM設定說明

第四十一章 微服務CICD（3）- jenkins + gitlab + webhooks + publish-over-ssh（1）

jenkins配置許可權不對導致無法登陸或者空白頁面解決辦法

jenkins解決構建完成後自動殺掉衍生程式