如何極狐GitLab Runner 使用特權身份執行

极狐GitLab發表於2024-03-25

本文作者：徐曉偉

GitLab 是一個全球知名的一體化 DevOps 平臺，很多人都透過私有化部署 GitLab 來進行原始碼託管。極狐GitLab 是 GitLab 在中國的發行版，專門為中國程式設計師服務。可以一鍵式部署極狐GitLab。

本文主要講述瞭如何使用極狐GitLab Runner 使用特權身份執行。

問題

安裝/升級時，提示內容一下：docker in docker 需要 特權身份 執行，如果已經設定了特權身份執行，則不會提示下方極狐GitLab Runner 警告，但是還是無法使用 docker in docker，會出現下一步的錯誤

[root@anolis-7-9 ~]# helm upgrade -n gitlab-test --install my-gitlab gitlab/gitlab -f my-gitlab.yaml --timeout 600s --version 7.7.0
Release "my-gitlab" has been upgraded. Happy Helming!
NAME: my-gitlab
LAST DEPLOYED: Sat Dec 23 21:20:46 2023
NAMESPACE: gitlab-test
STATUS: deployed
REVISION: 28
NOTES:
=== CRITICAL
The following charts are included for evaluation purposes only. They will not be supported by GitLab Support
for production workloads. Use Cloud Native Hybrid deployments for production. For more information visit
https://docs.gitlab.com/charts/installation/index.html#use-the-reference-architectures.
- PostgreSQL
- Redis
- Gitaly
- MinIO

=== NOTICE
The minimum required version of PostgreSQL is now 13. See https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/doc/installation/upgrade.md for more details.

=== NOTICE
You've installed GitLab Runner without the ability to use 'docker in docker'.
The GitLab Runner chart (gitlab/gitlab-runner) is deployed without the `privileged` flag by default for security purposes. This can be changed by setting `gitlab-runner.runners.privileged` to `true`. Before doing so, please read the GitLab Runner chart's documentation on why we
chose not to enable this by default. See https://docs.gitlab.com/runner/install/kubernetes.html#running-docker-in-docker-containers-with-gitlab-runners
[root@anolis-7-9 ~]#

如果沒有設定 特權身份 執行，可能會遇見下方異常

[root@anolis-7-9 ~]# kubectl -n gitlab-test get pod | grep runner
my-gitlab-gitlab-runner-6bf49f49db-7jn8w 1/1 Running 4 (53m ago)    22h
runner-q5jcztox-project-4-concurrent-0-ltmz9a7f 2/3 Error 0 37s
[root@anolis-7-9 ~]#

[root@anolis-7-9 ~]# kubectl -n gitlab-test logs -f runner-q5jcztox-project-4-concurrent-0-ltmz9a7f svc-0
Certificate request self-signature ok
subject=CN = docker:dind server
/certs/server/cert.pem: OK
Certificate request self-signature ok
subject=CN = docker:dind client
/certs/client/cert.pem: OK
ip: can't find device 'nf_tables'
nf_tables 74274 0
nfnetlink 14519 4 ip_set,nf_tables,nf_conntrack_netlink
modprobe: can't change directory to '/lib/modules': No such file or directory
ip: can't find device 'ip_tables'
ip_tables 27126 4 iptable_raw,iptable_mangle,iptable_nat,iptable_filter
modprobe: can't change directory to '/lib/modules': No such file or directory
mount: permission denied (are you root?)
Could not mount /sys/kernel/security.
AppArmor detection and --privileged mode might break.
mount: permission denied (are you root?)
[root@anolis-7-9 ~]#

解決辦法

匯出 helm gitlab 配置

# 將已配置的值匯出到檔案中
helm -n gitlab-test get values my-gitlab > my-gitlab.yaml

檢視 gitlab runner 預設配置

# 此處為節選，不同版本可能會存在差異，請以 https://artifacthub.io/packages/helm/gitlab/gitlab?modal=values 中的配置為準
gitlab-runner:
  runners:
    config: |
      [[runners]]
        [runners.kubernetes]
        image = "ubuntu:22.04"
        {{- if .Values.global.minio.enabled }}
        [runners.cache]
          Type = "s3"
          Path = "gitlab-runner"
          Shared = true
          [runners.cache.s3]
            ServerAddress = {{ include "gitlab-runner.cache-tpl.s3ServerAddress" . }}
            BucketName = "runner-cache"
            BucketLocation = "us-east-1"
            Insecure = false
        {{ end }}

修改配置如下

gitlab-runner:
  runners:
    config: |
      [[runners]]
        [runners.kubernetes]
        
        # pod 使用特權身份執行
        privileged = true
        
        image = "ubuntu:22.04"
        {{- if .Values.global.minio.enabled }}
        [runners.cache]
          Type = "s3"
          Path = "gitlab-runner"
          Shared = true
          [runners.cache.s3]
            ServerAddress = {{ include "gitlab-runner.cache-tpl.s3ServerAddress" . }}
            BucketName = "runner-cache"
            BucketLocation = "us-east-1"
            Insecure = false
        {{ end }}

更新配置

# GitLab Runner 使用特權身份執行
helm upgrade -n gitlab-test --install my-gitlab gitlab/gitlab --timeout 600s -f my-gitlab.yaml --set gitlab-runner.runners.privileged=true --version 7.7.0

等待所有 gitlab-runner 舊 pod 刪除完成，新 pod 正常執行時，重試流水線，即可使用 docker in docker

[root@anolis-7-9 ~]# kubectl -n gitlab-test get pod | grep runner
my-gitlab-gitlab-runner-5f6ff5994c-wdw5l             1/1     Running     0                115m
runner-yr5wzqmq-project-4-concurrent-0-idibutkf      3/3     Running     0                4s
[root@anolis-7-9 ~]#

注意此處檢視的是 svc-0 的日誌，即：services docker 的日誌，多個 services 時按順序排序

[root@anolis-7-9 ~]# kubectl -n gitlab-test logs -f runner-yr5wzqmq-project-4-concurrent-0-idibutkf svc-0 
time="2023-12-23T16:34:27.467258283Z" level=info msg="Starting up"
time="2023-12-23T16:34:27.469102439Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
time="2023-12-23T16:34:27.469335776Z" level=warning msg="Binding to IP address without --tlsverify is insecure and gives root access on this machine to everyone who has access to your network." host="tcp://0.0.0.0:2375"
time="2023-12-23T16:34:27.469359429Z" level=warning msg="Binding to an IP address, even on localhost, can also give access to scripts run in a browser. Be safe out there!" host="tcp://0.0.0.0:2375"
time="2023-12-23T16:34:28.469505651Z" level=warning msg="Binding to an IP address without --tlsverify is deprecated. Startup is intentionally being slowed down to show this message" host="tcp://0.0.0.0:2375"
time="2023-12-23T16:34:28.469545042Z" level=warning msg="Please consider generating tls certificates with client validation to prevent exposing unauthenticated root access to your network" host="tcp://0.0.0.0:2375"
time="2023-12-23T16:34:28.469555494Z" level=warning msg="You can override this by explicitly specifying '--tls=false' or '--tlsverify=false'" host="tcp://0.0.0.0:2375"
time="2023-12-23T16:34:28.469568946Z" level=warning msg="Support for listening on TCP without authentication or explicit intent to run without authentication will be removed in the next release" host="tcp://0.0.0.0:2375"
time="2023-12-23T16:34:43.473007148Z" level=info msg="libcontainerd: started new containerd process" pid=33
time="2023-12-23T16:34:43.473101488Z" level=info msg="parsed scheme: "unix"" module=grpc
time="2023-12-23T16:34:43.473114333Z" level=info msg="scheme "unix" not registered, fallback to default scheme" module=grpc
time="2023-12-23T16:34:43.473147363Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2023-12-23T16:34:43.473181211Z" level=info msg="ClientConn switching balancer to "pick_first"" module=grpc
time="2023-12-23T16:34:43Z" level=warning msg="deprecated version : `1`, please switch to version `2`"
time="2023-12-23T16:34:43.515744080Z" level=info msg="starting containerd" revision=212e8b6fa2f44b9c21b2798135fc6fb7c53efc16 version=v1.6.4
time="2023-12-23T16:34:43.531630020Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1
time="2023-12-23T16:34:43.531804565Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1
time="2023-12-23T16:34:43.538312017Z" level=info msg="skip loading plugin "io.containerd.snapshotter.v1.aufs"..." error="aufs is not supported (modprobe aufs failed: exit status 1 "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n"): skip plugin" type=io.containerd.snapshotter.v1
time="2023-12-23T16:34:43.538412286Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1
time="2023-12-23T16:34:43.538731958Z" level=info msg="skip loading plugin "io.containerd.snapshotter.v1.btrfs"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs (xfs) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2023-12-23T16:34:43.538767621Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.devmapper"..." type=io.containerd.snapshotter.v1
time="2023-12-23T16:34:43.538782676Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
time="2023-12-23T16:34:43.538792101Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1
time="2023-12-23T16:34:43.538972652Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1
time="2023-12-23T16:34:43.539265161Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1
time="2023-12-23T16:34:43.539441863Z" level=info msg="skip loading plugin "io.containerd.snapshotter.v1.zfs"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2023-12-23T16:34:43.539459872Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1
time="2023-12-23T16:34:43.539554275Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
time="2023-12-23T16:34:43.539571921Z" level=info msg="metadata content store policy set" policy=shared
time="2023-12-23T16:34:43.541205021Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1
time="2023-12-23T16:34:43.541252637Z" level=info msg="loading plugin "io.containerd.event.v1.exchange"..." type=io.containerd.event.v1
time="2023-12-23T16:34:43.541272679Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1
time="2023-12-23T16:34:43.541407221Z" level=info msg="loading plugin "io.containerd.service.v1.introspection-service"..." type=io.containerd.service.v1
time="2023-12-23T16:34:43.541450967Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1
time="2023-12-23T16:34:43.541468124Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1
time="2023-12-23T16:34:43.541488780Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1
time="2023-12-23T16:34:43.541510847Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1
time="2023-12-23T16:34:43.543022895Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1
time="2023-12-23T16:34:43.543105358Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1
time="2023-12-23T16:34:43.543146490Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1
time="2023-12-23T16:34:43.543314215Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1
time="2023-12-23T16:34:43.543713683Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2
time="2023-12-23T16:34:43.543917845Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1
time="2023-12-23T16:34:43.544701904Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1
time="2023-12-23T16:34:43.544758132Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.544778482Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1
time="2023-12-23T16:34:43.544870050Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.544905675Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.544964281Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.544992194Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.545007550Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.545027513Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.545056889Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.545075653Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.545095993Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1
time="2023-12-23T16:34:43.545466830Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.545564033Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.545604555Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1
time="2023-12-23T16:34:43.545630506Z" level=info msg="loading plugin "io.containerd.tracing.processor.v1.otlp"..." type=io.containerd.tracing.processor.v1
time="2023-12-23T16:34:43.545670690Z" level=info msg="skip loading plugin "io.containerd.tracing.processor.v1.otlp"..." error="no OpenTelemetry endpoint: skip plugin" type=io.containerd.tracing.processor.v1
time="2023-12-23T16:34:43.545688192Z" level=info msg="loading plugin "io.containerd.internal.v1.tracing"..." type=io.containerd.internal.v1
time="2023-12-23T16:34:43.545726357Z" level=error msg="failed to initialize a tracing processor "otlp"" error="no OpenTelemetry endpoint: skip plugin"
time="2023-12-23T16:34:43.546085654Z" level=info msg=serving... address=/var/run/docker/containerd/containerd-debug.sock
time="2023-12-23T16:34:43.546185861Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
time="2023-12-23T16:34:43.546250880Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock
time="2023-12-23T16:34:43.546308828Z" level=info msg="containerd successfully booted in 0.031780s"
time="2023-12-23T16:34:43.553104238Z" level=info msg="parsed scheme: "unix"" module=grpc
time="2023-12-23T16:34:43.553132141Z" level=info msg="scheme "unix" not registered, fallback to default scheme" module=grpc
time="2023-12-23T16:34:43.553169135Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2023-12-23T16:34:43.553189870Z" level=info msg="ClientConn switching balancer to "pick_first"" module=grpc
time="2023-12-23T16:34:43.555532743Z" level=info msg="parsed scheme: "unix"" module=grpc
time="2023-12-23T16:34:43.555571303Z" level=info msg="scheme "unix" not registered, fallback to default scheme" module=grpc
time="2023-12-23T16:34:43.555596960Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2023-12-23T16:34:43.555618644Z" level=info msg="ClientConn switching balancer to "pick_first"" module=grpc
time="2023-12-23T16:34:43.574453031Z" level=info msg="Loading containers: start."
time="2023-12-23T16:34:43.584406471Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge                151336  1 br_netfilter\nstp                    12976  1 bridge\nllc                    14552  2 bridge,stp\nip: can't find device 'br_netfilter'\nbr_netfilter           22256  0 \nbridge                151336  1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
time="2023-12-23T16:34:43.631967390Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
time="2023-12-23T16:34:43.662502145Z" level=info msg="Loading containers: done."
time="2023-12-23T16:34:43.688080250Z" level=info msg="Docker daemon" commit=f756502 graphdriver(s)=overlay2 version=20.10.16
time="2023-12-23T16:34:43.689334788Z" level=info msg="Daemon has completed initialization"
time="2023-12-23T16:34:43.707018978Z" level=info msg="API listen on /var/run/docker.sock"
time="2023-12-23T16:34:43.711636430Z" level=info msg="API listen on [::]:2375"
^C
[root@anolis-7-9 ~]#

流水線也能正常執行

更多關於極狐GitLab 的最佳實踐，請搜尋關注【極狐GitLab】公眾號或者登入極狐GitLab 官網 https://gitlab.cn 進行學習。

極狐GitLab Runner 新增極狐GitLab 域名 host
2024-03-01
Gitlab
如何使用極狐GitLab Runner 修改日誌大小限制
2024-03-22
Gitlab
如何在CentOS 安裝極狐GitLab Runner？
2024-03-08
CentOSGitlab
如何配置極狐GitLab Runner Cache 快取
2024-03-13
Gitlab快取
如何配置極狐GitLab Runner 幫助映象
2024-03-19
Gitlab
如何部署極狐GitLab Runner Cache 快取配置？
2024-03-07
Gitlab快取
如何在Anolis 8 安裝極狐GitLab Runner
2024-03-12
Gitlab
如何在極狐GitLab Runner Job 新增域名 host？
2024-03-19
Gitlab
極狐GitLab Runner 信任域名證書
2024-03-04
Gitlab
如何給極狐GitLab Runner 新增 MinIO 域名 host
2024-03-21
Gitlab
如何在極狐GitLab Runner 新增信任快取域名證書
2024-03-21
Gitlab快取
極狐GitLab Runner Kubernetes（k8s）配置
2024-03-06
GitlabK8S
極狐GitLab怎麼樣？如何利用極狐GitLab CI提高生產力？
2021-07-19
Gitlab
如何使用極狐GitLab 自定義 Pages 根域名
2024-03-27
Gitlab
極狐GitLab 如何配置多個 LDAP？
2024-02-19
GitlabLDA
如何配置極狐GitLab Maven 倉庫
2024-03-08
GitlabMaven
開個腦洞，帶你寫一個自己的極狐GitLab CI Runner
2022-12-29
Gitlab
如何在極狐GitLab 配置郵件功能
2024-03-29
Gitlab
如何配置極狐GitLab Docker 容器映象倉庫
2024-03-07
GitlabDocker
如何依賴機器人安裝極狐GitLab
2024-03-14
機器人Gitlab
如何啟動極狐GitLab 靜態網站Pages功能
2024-03-26
Gitlab網站
如何在極狐GitLab 自定義 Pages 域名、SSL/TLS 證書
2024-03-28
GitlabTLS
使用gitlab runner 進行CI（四）：使用Gitlab Page託管專案文件
2021-11-07
Gitlab
使用 KubeSphere 和極狐GitLab 打造雲原生持續交付系統
2022-06-01
Gitlab
極狐GitLab小課堂|如何利用DevOps來償還技術負債？
2021-08-21
Gitlabdev
極狐GitLab怎麼樣？幾大解決方案解密來了
2021-08-14
Gitlab解密
極狐GitLab|雲原生時代，關於GitOps你知道多少？
2021-08-28
Gitlab
GitLab CI持續整合-GitLab Runner
2019-03-17
Gitlab
Gitlab自動部署之四：使用Gitlab-runner部署Gitlab的專案
2019-04-19
Gitlab
如何以管理員身份執行cmd
2017-06-25
如何使用VSCode中的Code Runner外掛執行golang程式碼
2019-05-16
VSCodeGolang
docker環境下使用gitlab,gitlab-runner 為 NetCore 持續整合
2019-07-27
DockerGitlabNetCore
極狐GitLab好用嗎|五種方式降低“開放式DevOps平臺”成本
2021-06-26
Gitlabdev
支援“中國芯”，極狐GitLab助力AI晶片企業DevOps體系建設
2021-09-11
GitlabAI晶片dev
Gitlab自動部署之二：安裝GITLAB-RUNNER
2019-04-19
Gitlab
K8s 部署 Gitlab CI Runner
2021-08-22
K8SGitlab
GitLab Runner部署(kubernetes環境)
2020-12-28
Gitlab
gitlab-runner 中的 Docker-in-Docker
2023-05-17
GitlabDocker

如何極狐GitLab Runner 使用特權身份執行

問題

解決辦法

相關文章