03 . Jenkins構建之程式碼掃描

men發表於2020-06-16

Sonar簡介

Sonar 是一個用於程式碼質量管理的開放平臺。通過外掛機制，Sonar可以整合不同的測試工具，程式碼分析工具，以及持續整合工具。與持續整合工具（例如 Hudson/Jenkins 等）不同，Sonar 並不是簡單地把不同的程式碼檢查工具結果（例如 FindBugs，PMD 等）直接顯示在 Web 頁面上，而是通過不同的外掛對這些結果進行再加工處理，通過量化的方式度量程式碼質量的變化，從而可以方便地對不同規模和種類的工程進行程式碼質量管理。

在對其他工具的支援方面，Sonar 不僅提供了對 IDE 的支援，可以在 Eclipse和 IntelliJ IDEA 這些工具裡聯機檢視結果；同時 Sonar 還對大量的持續整合工具提供了介面支援，可以很方便地在持續整合中使用 Sonar。

此外，Sonar 的外掛還可以對 Java 以外的其他程式語言提供支援，對國際化以及報告文件化也有良好的支援

程式碼質量測試

程式碼質量七宗罪

# 編碼規範：是否遵守了編碼規範，遵循了最佳實踐。
# 潛在的 BUG：可能在最壞情況下出現問題的程式碼，以及存在安全漏洞的程式碼。
# 文件和註釋：過少（缺少必要資訊）、過多（沒有資訊量）、過時的文件或註釋。
# 重複程式碼：違反了 Don’t Repeat Yourself 原則。
# 複雜度：程式碼結構太複雜（如圈複雜度高），難以理解、測試和維護。
# 測試覆蓋率：編寫單元測試，特別是針對複雜程式碼的測試覆蓋是否足夠。
# 設計與架構：是否高內聚、低耦合，依賴最少。

Sonar部署

Sonar的相關下載和文件可以在下面的連結中找到：http://www.sonarqube.org/downloads/。需要注意最新版的Sonar需要至少JDK 1.8及以上版本。

Sonar的功能就是來檢查程式碼是否有BUG。除了檢查程式碼是否有bug還有其他的功能，比如說：你的程式碼註釋率是多少，程式碼有一些建議，編寫語法的建議。所以我們叫質量管理.

準備java環境和sonaqube包

# jdk下載地址
# http://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html
# 解除安裝系統自帶的java環境
[root@jenkins-8 git-2.9.5]# rpm -qa |grep jdk
[root@jenkins-8 git-2.9.5]# java
bash: java: 未找到命令

# 解壓安裝jdk
tar xvf jdk-8u151-linux-x64.tar.gz -C /usr/local/
cd /usr/local/
mv jdk1.8.0_151/ jdk
# 修改/etc/bashrc配置檔案，末尾加入下面一行環境變數.
tail -2 /etc/bashrc
JAVA_HOME=/usr/local/jdk
export PATH=$PATH:$JAVA_HOME/bin

source /etc/bashrc
java -version        # 儘量不要用openjdk
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)

# sonar下載地址
# http://www.sonarqube.org/downloads/
# 最新版的Sonar需要至少JDK1.8以上版本, 建議用國外網路下載,此處我使用5.6
https://binaries.sonarsource.com/Distribution/sonarqube/

unzip sonarqube-6.5.zip 
mv sonarqube-6.5 /usr/local/
ln -s /usr/local/sonarqube-6.5/ /usr/local/sonarqube

配置資料庫

wget https://dev.mysql.com/get/mysql80-community-release-el7-3.noarch.rpm

rpm -ivh mysql57-community-release-el7-10.noarch.rpm 
yum install -y mysql-community-server
  
systemctl start  mysqld.service

# 修改Mysql密碼下面有三種辦法
# 1.剛安裝好的mysql，可以從/var/log/mysqld.log獲取臨時密碼
grep  "password"  /var/log/mysqld.log   
        [root@mysql ~]# mysql -uroot -p
    Enter password:
    mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'ZHOUjian.22';

# 2.mysqladmin -uroot -p password "Baidu.123.com"
    Enter password:

# 3.實驗環境不知道root密碼操作方法如下
sed -i '/\[mysqld]/ a skip-grant-tables' /etc/my.cnf
systemctl restart mysqld
mysql <<EOF
        update mysql.user set authentication_string='' where user='root' and Host='localhost';
        flush privileges;
EOF
sed -i '/skip-grant/d' /etc/my.cnf
systemctl restart mysqld
mysqladmin -uroot -p password "ZHOUjian.20"
Enter password:              # 此處回車一下即可

CREATE DATABASE sonar CHARACTER SET utf8 COLLATE utf8_general_ci;
 GRANT ALL ON sonar.* TO sonar@"%" IDENTIFIED BY "ZHOUjian.20";

flush privileges;

配置啟動Sonar

cd /usr/local/sonarqube/conf/
vim sonar.properties 
sonar.jdbc.username=sonar
sonar.jdbc.password=123456
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance&useSSL=false
sonar.web.host=0.0.0.0
sonar.web.port=9000

# 啟動sonar
/usr/local/sonarqube/bin/linux-x86-64/sonar.sh start
Starting SonarQube...
SonarQube is already running.

# 驗證埠
ss -atnp |grep 9000
LISTEN     0      25           *:9000                     *:*                   users:(("java",pid=18028,fd=77))

訪問web介面

預設admin/admin

安裝中文支援

/usr/local/sonarqube/extensions/plugins/ #外掛本地路徑安裝中文外掛：

mv sonar-l10n-zh-plugin-1.11.jar /usr/local/sonarqube/extensions/plugins/

administration-system-update center-available，在後面的搜尋框搜尋外掛名稱，然後點 install 安裝：

或在插件目錄 /usr/local/sonar/extensions/plugins 執行以下命令：

 wget https://github.com/SonarQubeCommunity/sonar-l10n-zh/releases/download/sonar-l10n-zhplugin-1.11/sonar-l10n-zh-plugin-1.11.jar
 
 # 重啟服務生效
 /usr/local/sonarqube/bin/linux-x86-64/sonar.sh restart
 
# 重新整理頁面即可看到中文了

安裝外掛

可以安裝各種語言外掛

php,java,python

Sonar-scaner掃描器部署使用

Sonar-scaner掃描

sonarqube 通過呼叫掃描器 sonar-scanner 進行程式碼質量分析，即掃描器的具體工作就是掃描程式碼：

下載地址：http://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner

unzip sonar-scanner-2.6.1.zip
mv sonar-scanner-2.6.1 /usr/local/
ln -s /usr/local/sonar-scanner-2.6.1/ /usr/local/sonar-scanner
cd /usr/local/sonar-scanner/

grep "^[a-Z]" conf/sonar-scanner.properties 
sonar.host.url=http://149.129.38.117:9000
sonar.sourceEncoding=UTF-8
sonar.jdbc.username=sonar
sonar.jdbc.password=ZHOUjian.21
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&amp;characterEncoding=utf8

準備測試程式碼

unzip sonar-examples-master.zip 
mv sonar-examples-master/ /usr/local/src/
cd /usr/local/src/sonar-examples-master/

cat projects/languages/python/python-sonar-runner/sonar-project.properties 
# Required metadata
sonar.projectKey=org.sonarqube:python-simple-sonar-scanner
sonar.projectName=Python :: Simple Project : SonarQube Scanner   # 專案名稱,會顯示在儀表盤
sonar.projectVersion=1.0

# Comma-separated paths to directories with sources (required)
sonar.sources=src		# 程式碼目錄
# Language
sonar.language=py 	# 語言格式
# Encoding of the source files
sonar.sourceEncoding=UTF-8

執行掃描

# 注意看我當前目錄
[root@jenkins python-sonar-runner]# pwd
/usr/local/src/sonar-examples-master/projects/languages/python/python-sonar-runner


# 手動在當前專案目錄執行掃描,一下是掃描過程中資訊
#  sonar-project.propertie 每個專案都要有
/usr/local/sonar-scanner/bin/sonar-scanner 
INFO: Scanner configuration file: /usr/local/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /usr/local/src/sonar-examples-master/projects/languages/python/python-sonar-runner/sonar-project.properties
INFO: SonarQube Scanner 2.6.1
INFO: Java 1.8.0_151 Oracle Corporation (64-bit)
INFO: Linux 3.10.0-514.26.2.el7.x86_64 amd64
INFO: User cache: /root/.sonar/cache
INFO: Load global settings
INFO: Load global settings (done) | time=252ms
WARN: Property 'sonar.jdbc.url' is not supported any more. It will be ignored. There is no longer any DB connection to the SQ database.
WARN: Property 'sonar.jdbc.username' is not supported any more. It will be ignored. There is no longer any DB connection to the SQ database.
WARN: Property 'sonar.jdbc.password' is not supported any more. It will be ignored. There is no longer any DB connection to the SQ database.
INFO: User cache: /root/.sonar/cache
INFO: Load plugins index
INFO: Load plugins index (done) | time=30ms
INFO: Download sonar-csharp-plugin-5.10.1.1411.jar
INFO: Download sonar-python-plugin-1.8.0.1496.jar
INFO: Download sonar-java-plugin-4.12.0.11033.jar
INFO: Download sonar-l10n-zh-plugin-1.11.jar
INFO: Plugin [l10nzh] defines 'l10nen' as base plugin. This metadata can be removed from manifest of l10n plugins since version 5.2.
INFO: Download sonar-flex-plugin-2.3.jar
INFO: Download sonar-scm-git-plugin-1.2.jar
INFO: Download sonar-xml-plugin-1.4.3.1027.jar
INFO: Download sonar-php-plugin-2.10.0.2087.jar
INFO: Download sonar-scm-svn-plugin-1.5.0.715.jar
INFO: Download sonar-javascript-plugin-3.1.1.5128.jar
INFO: SonarQube server 6.5.0
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Process project properties
INFO: Load project repositories
INFO: Load project repositories (done) | time=199ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=106ms
INFO: Load active rules
INFO: Load active rules (done) | time=1965ms
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=202ms
WARN: SCM provider autodetection failed. No SCM provider claims to support this project. Please use sonar.scm.provider to define SCM of your project.
INFO: Publish mode
INFO: Project key: org.sonarqube:python-simple-sonar-scanner
INFO: -------------  Scan Python :: Simple Project : SonarQube Scanner
INFO: Load server rules
INFO: Load server rules (done) | time=277ms
INFO: Language is forced to py
INFO: Base dir: /usr/local/src/sonar-examples-master/projects/languages/python/python-sonar-runner
INFO: Working dir: /usr/local/src/sonar-examples-master/projects/languages/python/python-sonar-runner/.sonar
INFO: Source paths: src
INFO: Source encoding: UTF-8, default locale: en_US
INFO: Index files
INFO: 9 files indexed
INFO: Quality profile for py: Sonar way
INFO: Sensor PythonXUnitSensor [python]
INFO: Sensor PythonXUnitSensor [python] (done) | time=115ms
INFO: Sensor Python Squid Sensor [python]
INFO: Python unit test coverage
INFO: Python integration test coverage
INFO: Python overall test coverage
INFO: Sensor Python Squid Sensor [python] (done) | time=1378ms
INFO: Sensor SonarJavaXmlFileSensor [java]
INFO: Sensor SonarJavaXmlFileSensor [java] (done) | time=0ms
INFO: Sensor Analyzer for "php.ini" files [php]
INFO: Sensor Analyzer for "php.ini" files [php] (done) | time=13ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=27ms
INFO: Sensor CPD Block Indexer
INFO: Sensor CPD Block Indexer (done) | time=157ms
INFO: No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.
INFO: 5 files had no CPD blocks
INFO: Calculating CPD for 4 files
INFO: CPD calculation finished
INFO: Analysis report generated in 141ms, dir size=54 KB
INFO: Analysis reports compressed in 26ms, zip size=27 KB
INFO: Analysis report uploaded in 782ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://149.129.38.117:9000/dashboard/index/org.sonarqube:python-simple-sonar-scanner
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://149.129.38.117:9000/api/ce/task?id=AXK4TU1CIpzWVToFffwU
INFO: Task total time: 20.514 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 38.804s
INFO: Final Memory: 43M/105M
INFO: ------------------------------------------------------------------------
# 接下來我們到sonar儀表看檢視一下

Jenkins關聯到SonarQube

jenkins安裝sonar外掛

Jenkins安裝請看上一篇文章

要想讓Jenkins關聯到sonarqube需要先安裝外掛,在jenkins外掛安裝SonarQubePlugin,其次配置SonarQube server.

jenkins配置sonar

jenkins關聯到sonar

配置掃描

[root@jenkins ~]# cat /usr/local/src/sonar-examples-master/projects/languages/python/python-sonar-runner/sonar-project.properties
# Required metadata
sonar.projectKey=org.sonarqube:python-simple-sonar-scanner
sonar.projectName=Python :: Simple Project : SonarQube Scanner
sonar.projectVersion=1.0

# Comma-separated paths to directories with sources (required)
sonar.sources=src

# Language
sonar.language=py

# Encoding of the source files
sonar.sourceEncoding=UTF-8

持續整合工具之jenkins+sonarqube做程式碼掃描
2020-10-17
Jenkins
微服務測試之靜態程式碼掃描
2019-02-22
微服務
CentOS 7 jenkins 程式碼構建同步
2019-12-17
CentOSJenkins
CentOS7 jenkins 程式碼構建同步
2019-12-17
CentOSJenkins
iOS開發之掃描二維碼
2017-12-13
iOS
隨行付微服務測試之靜態程式碼掃描
2018-11-29
微服務
Jenkins + Gitee 實現程式碼自動化構建
2020-10-16
JenkinsGitee
jenkins如何構建github上的程式碼(ci cd)
2020-10-11
JenkinsGithub
BCSphere入門教程03：掃描周圍裝置
2014-07-23
iOS 使用CIDetector掃描相簿二維碼、原生掃描
2016-04-25
iOSIDE
SonarQube學習（三）- 專案程式碼掃描
2021-01-03
使用Python的Socket模組構建一個UDP掃描工具
2015-06-24
PythonUDP
iOS二維碼掃描
2016-01-14
iOS
掃描技術和掃描工具
2017-11-30
Rust 程式碼質量和漏洞掃描工具 - Reddit
2022-05-02
Rust
iOS中二維碼掃描
2017-09-05
iOS
Pythonpyclamad病毒掃描與目錄病毒掃描指令碼（轉載）
2018-01-19
Python指令碼
新型的按鍵掃描程式
2015-01-29
C++原始碼單詞掃描程式（詞法分析）
2020-10-16
C++原始碼詞法分析
iOS 掃描二維碼/條形碼
2018-12-10
iOS
03《構建之法》閱讀筆記03
2018-01-04
筆記
AWVS掃描器掃描web漏洞操作
2018-08-06
Web
TWAIN掃描識別控制元件：Web應用程式的掃描器SDK
2021-03-23
AI控制元件Web
win10系統掃描器提示掃描不到掃描器如何解決
2019-01-07
Win10
掃描二維碼登入思路
2019-05-20
安卓自定義二維碼掃描
2018-01-23
安卓
IOS 使用 ZbarSDK 二維碼掃描
2015-01-28
iOS
Jenkins和maven自動化構建java程式
2016-02-16
JenkinsMavenJava
【Lua篇】靜態程式碼掃描分析（一）初步介紹
2021-08-14
掃描王 for Mac專業圖片掃描工具
2020-12-04
Mac
Nmap掃描教程之基礎掃描詳解
2015-06-29
微信小程式掃描普通二維碼開啟小程式的方法
2021-12-03
微信小程式
Win10系統下掃描器程式無法掃描的解決方法
2018-10-05
Win10
Oracle中存取資料掃描Table及索引的方式(全表掃描，索引掃描等)
2009-05-27
Oracle索引
MySQL中的全表掃描和索引樹掃描
2022-05-14
MySql索引
Mac Jenkins 構建 Android App 時動態設定程式碼引數
2021-12-16
MacJenkinsAndroidAPP
Gitlab利用Webhook實現Push程式碼後的jenkins自動構建
2018-06-12
GitlabWebHookJenkins
Jenkins + GitHub 自動構建
2018-09-17
JenkinsGithub