009.Ansible模板管理 Jinja2

夢中淚發表於2020-05-02

一 Jinja2簡介

Jinja2是基於python的模板引擎。

假設說現在我們需要一次性在10臺主機上安裝redis,這個通過playbook現在已經很容易實現。預設情況下,所有的redis安裝完成之後,我們可以統一為其分發配置檔案。這個時候就面臨一個問題,這些redis需要監聽的地址各不相同,我們也不可能為每一個redis單獨寫一個配置檔案。因為這些配置檔案中,絕大部分的配置其實都是相同的。這個時候最好的方式其實就是用一個通用的配置檔案來解決所有的問題。將所有需要修改的地方使用變數替換

二 模板使用

playbook使用template模組來實現模板檔案的分發,其用法與copy模組基本相同,唯一的區別是,copy模組會將原檔案原封不動的複製到被控端,而template會將原檔案複製到被控端,並且使用變數的值將檔案中的變數替換以生成完整的配置檔案。

2.1 redis模板配置

建立一個模板目錄

[root@node1 ansible]# mkdir template

為了方便區分,模板檔案最好使用.j2結尾,就知道是模板檔案,在複製時需要使用template模組

[root@node1 ansible]# vim template/redis.conf.j2

daemonize yes
pidfile /var/run/redis.pid
port 6379
logfile "/var/log/redis/redis.log"
dbfilename dump.rdb
dir /data/redis

maxmemory {{redismem }}

bind {{ ansible_ens33.ipv4.address }} 127.0.0.1

timeout 300
loglevel notice

databases 16
save 900 1
save 300 10
save 60 10000

rdbcompression yes

maxclients 10000
appendonly yes
appendfilename appendonly.aof
appendfsync everysec

[root@node1 ansible]# vim redis_config.yml 

- hosts: all
  tasks:
    - name: set redis-server
      set_fact: redismem="{{ ansible_memtotal_mb/2|int }}"
    - name: install redis
      yum:
        name: redis
        state: present
    - name: ensure sest direectory exists
      file:
        path: "{{ item }}"
        state: directory
        mode: 0755
        recurse: yes
        owner: redis
        group: redis
      with_items:
        - "/var/log/redis"
        - "/data/redis"
    - name: cp redis.conf to /etc
      template:
        src: template/redis.conf.j2
        dest: /etc/redis.conf
        mode: 0755
      notify: restart redis
    - name: start redis
      systemd:
        name: redis
        state: restarted
  handlers:
    - name: restart redis
      systemd:
        name: redis
        state: restarted

關於template模組的更多引數說明:

  • backup:如果原目標檔案存在,則先備份目標檔案
  • dest:目標檔案路徑
  • force:是否強制覆蓋,預設為yes
  • group:目標檔案屬組
  • mode:目標檔案的許可權
  • owner:目標檔案屬主
  • src:源模板檔案路徑
  • validate:在複製之前通過命令驗證目標檔案,如果驗證通過則複製

執行

[root@node1 ansible]# ansible-playbook redis_config.yml

PLAY [all] ************************************************************************************************************************************

TASK [set redis-server] ***********************************************************************************************************************
ok: [demo4.example.com]
ok: [demo5.example.com]
ok: [demo1.example.com]
ok: [demo2.example.com]
ok: [demo3.example.com]

TASK [install redis] **************************************************************************************************************************
ok: [demo5.example.com]
ok: [demo2.example.com]
ok: [demo3.example.com]
ok: [demo1.example.com]
ok: [demo4.example.com]

TASK [ensure sest direectory exists] **********************************************************************************************************
changed: [demo1.example.com] => (item=/var/log/redis)
changed: [demo5.example.com] => (item=/var/log/redis)
changed: [demo2.example.com] => (item=/var/log/redis)
changed: [demo3.example.com] => (item=/var/log/redis)
changed: [demo4.example.com] => (item=/var/log/redis)
changed: [demo5.example.com] => (item=/data/redis)
changed: [demo2.example.com] => (item=/data/redis)
changed: [demo1.example.com] => (item=/data/redis)
changed: [demo3.example.com] => (item=/data/redis)
changed: [demo4.example.com] => (item=/data/redis)

TASK [cp redis.conf to /etc] ******************************************************************************************************************
ok: [demo1.example.com]
ok: [demo4.example.com]
ok: [demo5.example.com]
ok: [demo3.example.com]
ok: [demo2.example.com]

TASK [start redis] ****************************************************************************************************************************
changed: [demo5.example.com]
changed: [demo1.example.com]
changed: [demo4.example.com]
changed: [demo2.example.com]
changed: [demo3.example.com]

PLAY RECAP ************************************************************************************************************************************
demo1.example.com          : ok=5    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
demo2.example.com          : ok=5    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
demo3.example.com          : ok=5    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
demo4.example.com          : ok=5    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
demo5.example.com          : ok=5    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

[root@node1 ansible]# ansible all -m shell -a "cat /etc/redis.conf|grep bind"

demo2.example.com | CHANGED | rc=0 >>
bind 192.168.132.132 127.0.0.1
demo1.example.com | CHANGED | rc=0 >>
bind 192.168.132.131 127.0.0.1
demo3.example.com | CHANGED | rc=0 >>
bind 192.168.132.133 127.0.0.1
demo5.example.com | CHANGED | rc=0 >>
bind 192.168.132.135 127.0.0.1
demo4.example.com | CHANGED | rc=0 >>
bind 192.168.132.134 127.0.0.1

使用條件判斷

2.2 條件語句

在上面的示例中,我們直接取了被控節點的ens33網路卡的ip作為其監聽地址。那麼假如有些機器的網路卡是bond0,這種做法就會報錯。這個時候我們就需要在模板檔案中定義條件語句如下:

[root@node1 ansible]# cat template/redis.conf.j2

daemonize yes
pidfile /var/run/redis.pid
port 6379
logfile "/var/log/redis/redis.log"
dbfilename dump.rdb
dir /data/redis

maxmemory {{redismem }}
{% if ansible_bond0 is defined  %}
bind {{ ansible_bind0.ipv4.address }} 127.0.0.1
{% elif ansible_ens33 is defined %}
bind {{ ansible_ens33.ipv4.address }} 127.0.0.1
{% else %}
bind 0.0.0.0
{% endif %}
timeout 300
loglevel notice

databases 16
save 900 1
save 300 10
save 60 10000

rdbcompression yes

maxclients 10000
appendonly yes
appendfilename appendonly.aof
appendfsync everysec
You have new mail in /var/spool/mail/root

讓redis主從角色都可以使用該檔案:

配置主從條件

[root@node1 ansible]# vim inventory 

[redis]
demo3.example.com
demo4.example.com  masterip=demo3.example.com

模板檔案

[root@node1 ansible]# vim template/redis.conf.j2

daemonize yes
pidfile /var/run/redis.pid
port 6379
logfile "/var/log/redis/redis.log"
dbfilename dump.rdb
dir /data/redis

maxmemory {{redismem }}
{% if ansible_bond0 is defined  %}
bind {{ ansible_bind0.ipv4.address }} 127.0.0.1
{% elif ansible_ens33 is defined %}
bind {{ ansible_ens33.ipv4.address }} 127.0.0.1
{% else %}
bind 0.0.0.0
{% endif %}

{% if masterip is defined %}
slaveof {{ masterip }} {{ materport|default(6379) }}
{% endif %}
timeout 300
loglevel notice

databases 16
save 900 1
save 300 10
save 60 10000

rdbcompression yes

maxclients 10000
appendonly yes
appendfilename appendonly.aof
appendfsync everysec

[root@node1 ansible]# vim redis_config.yml

- hosts: redis
  tasks:
    - name: set redis-server
      set_fact: redismem="{{ ansible_memtotal_mb/2|int }}"
    - name: install redis
      yum:
        name: redis
        state: present
    - name: ensure sest direectory exists
      file:
        path: "{{ item }}"
        state: directory
        mode: 0755
        recurse: yes
        owner: redis
        group: redis
      with_items:
        - "/var/log/redis"
        - "/data/redis"
    - name: cp redis.conf to /etc
      template:
        src: template/redis.conf.j2
        dest: /etc/redis.conf
        mode: 0755
      notify: restart redis
    - name: start redis
      systemd:
        name: redis
        state: restarted
  handlers:
    - name: restart redis
      systemd:
        name: redis
        state: restarted

節點檢視

[root@node4 ~]# redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:demo3.example.com
master_port:6379
master_link_status:up
master_last_io_seconds_ago:10
master_sync_in_progress:0
slave_repl_offset:57
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
[root@node3 ~]# redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.132.134,port=6379,state=online,offset=421,lag=0
master_repl_offset:421
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:420

2.3 jinj2的迴圈語句

現在把proxy主機組中的主機作為代理伺服器,安裝nginx做反向代理,將請求轉發至後面的兩臺webserver,即webserver組的伺服器。

[root@node1 ansible]# vim inventory 

[webserver]
demo1.example.com
demo2.example.com
demo3.example.com

[proxy]
demo5.example.com

[redis]
demo3.example.com
demo4.example.com  masterip=demo3.example.com

[root@node1 ansible]# vim systeminit.yml

- hosts: all
  tasks:
    - name: ipatbles flush filter
      iptables:
        chain: "{{ item }}"
        flush: yes
      with_items: ['INPUT','FORWARD','OUTPUT']

[root@node1 ansible]# ansible-playbook systeminit.yml

部署httpd

[root@node1 ansible]# vim config_httpd.yml

- hosts: webserver
  tasks: 
    - name: install httpd
      yum:
        name: httpd
        state: present
    - name: start httpd
      systemd:
        name: httpd
        state: started
        enabled: yes
        daemon_reload: yes

[root@node1 ansible]# ansible-playbook config_httpd.yml

配置nginxproxy

[root@node1 ansible]# vim config_proxy.yml

- name: gather facts   #這裡需要配置快取,觸發setup,把facts引數快取到本地,否則在下面獲取到的fact將是nginx proxy的fact值,就不會有結果
  gather_facts: False
  hosts: webserver
  tasks: 
    - name: gather facts
      setup: 
- name: Configue Nginx
  hosts: proxy
  tasks: 
    - name: install nginx
      yum:
        name: nginx
        state: present
    - name: copy nginx.conf to dest
      template:
        src: template/nginx.conf.j2
        dest: /etc/nginx/nginx.conf
      notify: reload nginx
    - name: start nginx
      systemd:
        name: nginx
        enabled: yes
        daemon_reload: yes
  handlers:
    - name: reload nginx
      systemd:
        name: nginx
        state: reloaded

[root@node1 ansible]# vim  template/nginx.conf.j2 

user nginx;
worker_processes {{ ansible_processor_vcpus }};
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
include /usr/share/nginx/modules/*.conf;
events {
    worker_connections 65535;
    use epoll;
}
http {
    map $http_x_forwarded_for $clientRealIP {
        "" $remote_addr;
        ~^(?P<firstAddr>[0-9\.]+),?.*$ $firstAddr;
    }
    log_format  real_ip '{ "datetime": "$time_local", '
                        '"remote_addr": "$remote_addr", '
                        '"source_addr": "$clientRealIP", '
                        '"x_forwarded_for": "$http_x_forwarded_for", '
                        '"request": "$request_uri", '
                        '"status": "$status", '
                        '"request_method": "$request_method", '
                        '"request_length": "$request_length", '
                        '"body_bytes_sent": "$body_bytes_sent", '
                        '"request_time": "$request_time", '
                        '"http_referrer": "$http_referer", '
                        '"user_agent": "$http_user_agent", '
                        '"upstream_addr": "$upstream_addr", '
                        '"upstream_status": "$upstream_status", '
                        '"upstream_http_header": "$upstream_http_host",'
                        '"upstream_response_time": "$upstream_response_time", '
                        '"x-req-id": "$http_x_request_id", '
                        '"servername": "$host"'
                        ' }';
    access_log  /var/log/nginx/access.log  real_ip;
    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;
    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;
    include /etc/nginx/conf.d/*.conf;

    upstream web {
    {% for host in groups['webserver'] %}
        {% if hostvars[host]['ansible_bond0']['ipv4']['address'] is defined %}
        server {{ hostvars[host]['ansible_bond0']['ipv4']['address'] }}:80;
        {% elif hostvars[host]['ansible_ens33']['ipv4']['address'] is defined %}
        server {{ hostvars[host]['ansible_ens33']['ipv4']['address'] }}:80;
        {% endif %}
    {% endfor %}
    }
    server {
        listen       80 default_server;
        server_name  _; 
        location / { 
            proxy_pass http://web;
        }   
    }   
}  

執行驗證

[root@node1 ansible]# ansible-playbook config_proxy.yml

[root@node5 ~]# vim /etc/nginx/nginx.conf

user nginx;
worker_processes 4;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
include /usr/share/nginx/modules/*.conf;
events {
    worker_connections 65535;
    use epoll;
}
http {
    map $http_x_forwarded_for $clientRealIP {
        "" $remote_addr;
        ~^(?P<firstAddr>[0-9\.]+),?.*$ $firstAddr;
    }
    log_format  real_ip '{ "datetime": "$time_local", '
                        '"remote_addr": "$remote_addr", '
                        '"source_addr": "$clientRealIP", '
                        '"x_forwarded_for": "$http_x_forwarded_for", '
                        '"request": "$request_uri", '
                        '"status": "$status", '
                        '"request_method": "$request_method", '
                        '"request_length": "$request_length", '
                        '"body_bytes_sent": "$body_bytes_sent", '
                        '"request_time": "$request_time", '
                        '"http_referrer": "$http_referer", '
                        '"user_agent": "$http_user_agent", '
                        '"upstream_addr": "$upstream_addr", '
                        '"upstream_status": "$upstream_status", '
                        '"upstream_http_header": "$upstream_http_host",'
                        '"upstream_response_time": "$upstream_response_time", '
                        '"x-req-id": "$http_x_request_id", '
                        '"servername": "$host"'
                        ' }';
    access_log  /var/log/nginx/access.log  real_ip;
    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;
    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;
    include /etc/nginx/conf.d/*.conf;

    upstream web {
                    server 192.168.132.131:80;
                            server 192.168.132.132:80;
                            server 192.168.132.133:80;
                }
    server {
        listen       80 default_server;
        server_name  _;
        location / {
            proxy_pass http://web;
        }
    }
}

域名解析服務bind的配置檔案 named.conf的jinja2模板示例:

[root@node1 ansible]# vim inventory 

[dnsmaster]
demo2.example.com
demo3.example.com

[dnsslave]
demo4.example.com
demo5.example.com

[root@node1 ansible]# vim config_dns.yml

- hosts: dnsmaster,dnsslave
  tasks:
    - template:
        src: template/named.conf.j2
        dest: /tmp/named.conf                  

[root@node1 ansible]# vim template/named.conf.j2

options {

listen-on port 53 {
127.0.0.1;
{% for ip in ansible_all_ipv4_addresses %}
{{ ip }};
{% endfor %}
};

listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
};

zone "." IN {
type hint;
file "named.ca";
};

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

{% if 'dnsmaster' in group_names %}    #設定變數,屬於這個組設為master
{% set zone_type = 'master' %}
{% set zone_dir = 'data' %}
{% else %}
{% set zone_type = 'slave' %}          #否則設為salve
{% set zone_dir = 'slaves' %}
{% endif %}

zone "internal.example.com" IN {
type {{ zone_type }};
file "{{ zone_dir }}/internal.example.com";   #引用變數
{% if 'dnsmaster' not in group_names %}
masters { 192.168.2.2; };
{% endif %}
};

執行anslibe檢視主從

node2和node3

node4和node5

三 Jinja2過濾器

3.1  default過濾器

例如上一個redis案例

{% if masterip is defined %}
slaveof {{ masterip }} {{ materport|default(6379) }}
{% endif %}

 另一個示例

- hosts:
  gather_facts: false
  vars:
    - path: /tmp/test
      mode: 0400
    - path: /tmp/foo
    - path: /tmp/bar
  tasks:
    - file:
        dest: {{item}}
        state: touch
        mode: {{ item.mode|default(omit) }}   #如果存在設定,不存在忽略
      with_items: '{{ paths }}'

3.2 字串相關過濾器

  • upper:將所有字串轉換為大寫
  • |ower:將所有字宇符串轉換為小寫
  • capitalize:將字串的首字母大寫,其他字母小寫
  • reverse:將宇符串倒序排列
  • first:返回字串的第一個宇符
  • last:返回字串的最後一個字元
  • trim:將宇符串開頭和結尾的空格去掉
  • center(30):將宇符串放在中間,並且字串兩邊用空格補齊30位
  • length:返回字串的長度,與 count等價
  • |ist:將宇符串轉換為列表
  • shuffle:list將宇符串轉換為列表,但是順序排列, shuffle同樣將宇符串轉換為列表,但是會隨機打亂宇符串順序

3.3 數字相關操作

  • int:將對應的值轉換為整數
  • float:好對應的值轉換為浮點數
  • abs:獲取絕對值
  • round:小數點四捨五入
  • randon:從一個給定的範圍中獲取隨機值
- hosts: demo2.example.com
  gather_facts: no
  vars:
    testnum: -1
  tasks:
    - debug:
        msg: "{{ 8+('8'|int) }}"
    - debug:
        msg: "{{ 'a'|int(default=6) }}"
    - debug:
        msg: "{{ '8'|float }}"
    - debug:
        msg: "{{ testnum|abs }}"
    - debug:
        msg: "{{ 12.5|round }}"
    - debug:
        msg: "{{ 3.1415926|round(5) }}"
    - debug:
        #從0到100隨即返回一個數字
        msg: "{{ 100|random }}"
    - debug:
        #從5到10中隨機返回一個數字
        msg: "{{ 10|random(start=5) }}"
    - debug:
        #從4到15隨機返回一個數字,步長為3
        #返回的隨機數這隻可能是:4 7 10 13中的一個
        msg: "{{ 15|random(start=5,step=3) }}"
    - debug:
        #從0到15隨機返回一個數字,步長為4
        msg: "{{ 15|random(step=4) }}"

執行結果

TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "16"
}
TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "6"
}
TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "8.0"
}
TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "1"
}
TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "13.0"
}
TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "3.14159"
}
TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "11"
}
TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "7"
}
TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "11"
}
TASK [debug] **************************************************************************************************************************************
ok: [demo2.example.com] => {
    "msg": "0"
}

3.4 列表過濾器

  • length:返回列表長度
  • first:返回列表的第一個值
  • last:返回列表的最後一個值
  • min:返回列表中最小的值
  • max:返回列表中最大的值
  • sort:重新排列列表,預設為升序排列, sort(reverse=true)為降序
  • sum:返回皺教寧非巢狀列表中所有數字的和I
  • flatten:如果列表中包含列表,則 flatten可拉平巢狀的列表 levels引數可用於指定被拉平的層級
  • join:將列表中的元素合併為一個字串
  • random:從列表中隨機返回一個元素
  • shuffle
  • upper
  • lower
  • union:將兩個列表合併,如果元素有重複,則只留下一個
  • intersect:獲取兩個列表的交集
  • difference:獲取存在於第一個列表中,但不存在於第二個列表中的元素
  • symmetric difference:取出兩個列表中各自獨立的元素,如果重複則只留一個

3.5 應用於檔案路徑的過濾器

  • basename:返回檔案路徑中的檔名部分
  • dirname:返回檔案路徑中的目錄部分
  • expanduser:將檔案路徑中的~替換為使用者目錄
  • realpath:處理符號連結後的檔案實際路徑

示例:

- name: test basename
  hosts: test
  vars:
    homepage: /usr/share/nginx/html/index.html
  tasks:
    - name: copy homepage
      copy:
        src: files/index.html
        dest: {{ homepage }}

改寫

- name: test basename
  hosts: test
  vars:
    homepage: /usr/share/nginx/html/index.html
  tasks:
    - name: copy homepage
      copy:
        src: files/{{ homepage | basename }}
        dest: {{ homepage }}

3.6 自定義過濾器

舉個簡單的例子,現在有一個playbook如下:

- name: test filter
  hosts: demo2.example.com
  vars:
    domains: ["www.example.com","example.com"]
  tasks:
    - template:
        src: template/test.conf.j2
        dest: /tmp/test.conf

template/test.conf.j2如下:

hosts = [{{ domains | join(',') }}]

執行playbook後,在目標機上的test.conf如下:

[root@node1 ansible]# ansible demo2.example.com  -m shell -a "cat /tmp/test.conf"

demo2.example.com | CHANGED | rc=0 >>
hosts = [www.example.com,example.com]

現在如果希望目標機上的test.conf檔案返回結果如下:

hosts = ["www.example.com","example.com"]

沒有現成的過濾器來幫我們做這件事情。我們可以自己簡單寫一個surround_by_quote.py內容如下:

我們需要開啟ansible.cfg的配置項:

filter_plugins     = /etc/ansible/plugins/filter

[root@node1 ansible]# mkdir -p /etc/ansible/plugins/filter

[root@node1 ansible]# vim /etc/ansible/plugins/filter/surround_by_quote.py

#!/usr/bin/env python
def surround_by_quote(a_list):
#  return ['"%s"' % an_element for an_element in a_list]    #這個是下面的簡寫,python語法
  lst = []
  for index in a_list:
    lst.append('"%s"' %index)
  return lst
class FilterModule(object):
  def filters(self):
    return {'surround_by_quote': surround_by_quote}

將剛剛編寫的程式碼檔案放入/etc/ansible/plugins/filter目錄下,然後修改templates/test.conf.j2如下:

hosts = [{{ domains |surround_by_quote|join(',') }}]

執行檢視

[root@node1 ansible]# ansible demo2.example.com  -m shell -a "cat /tmp/test.conf"


博主宣告:本文的內容來源主要來自譽天教育晏威老師,由本人實驗完成操作驗證,需要的博友請聯絡譽天教育(http://www.yutianedu.com/),獲得官方同意或者晏老師(https://www.cnblogs.com/breezey/)本人同意即可轉載,謝謝!

相關文章