Nginx+Lua 入門

Able張發表於2020-10-28

OpenResty 官網：http://openresty.org/

OpenResty 是一個nginx和它的各種三方模組的一個打包而成的軟體平臺。最重要的一點是它將lua/luajit打包了進來，使得我們可以使用lua指令碼來進行web的開發。有了lua，我們可以藉助於nginx的非同步非阻塞的功能，達到使用 lua 非同步併發訪問後端的 MySQL, PostgreSQL, Memcached, Redis等等服務。特別是特有的 ngx.location.capture_multi 功能讓人印象深刻，其可以達到極大的減少瀏覽器的http連線數量，並且可以非同步併發的訪問後臺 Java/PHP/Python 等等介面。OpenResty 架構的web可以輕鬆超越Node.js的效能，並且對後端語言沒有限制，你可以使用Java/PHP/Python等等各種語言。OpenResty(nginx+lua)可以替代node.js的前端渲染的功能。

OpenResty (aka. ngx_openresty) is a full-fledged web application server by bundling the standard Nginx core, lots of 3rd-party Nginx modules, as well as most of their external dependencies.

By taking advantage of various well-designed Nginx modules, OpenResty effectively turns the nginx server into a powerful web app server, in which the web developers can use the Lua programming language to script various existing nginx C modules and Lua modules and construct extremely high-performance web applications that are capable to handle 10K+ connections.

OpenResty aims to run your server-side web app completely in the Nginx server, leveraging Nginx's event model to do non-blocking I/O not only with the HTTP clients, but also with remote backends like MySQL, PostgreSQL, Memcached, and Redis.

1. 安裝OpenResty

先安裝依賴：yum install readline-devel pcre-devel openssl-devel gcc

解壓： tar zxvf ngx_openresty-1.9.3.1.tar.gz

建立一個軟連線：ln -s ngx_openresty-1.9.3.1 openresty

進入目錄：cd openresty

編譯：

./configure \
             --with-cc-opt="-I/usr/local/include" \
             --with-ld-opt="-L/usr/local/lib" \
             --prefix=/opt/openresty 

... ...
Configuration summary
  + using system PCRE library
  + using system OpenSSL library
  + md5: using OpenSSL library
  + sha1: using OpenSSL library
  + using system zlib library

  nginx path prefix: "/opt/openresty/nginx"
  nginx binary file: "/opt/openresty/nginx/sbin/nginx"
  nginx configuration prefix: "/opt/openresty/nginx/conf"
  nginx configuration file: "/opt/openresty/nginx/conf/nginx.conf"
  nginx pid file: "/opt/openresty/nginx/logs/nginx.pid"
  nginx error log file: "/opt/openresty/nginx/logs/error.log"
  nginx http access log file: "/opt/openresty/nginx/logs/access.log"
  nginx http client request body temporary files: "client_body_temp"
  nginx http proxy temporary files: "proxy_temp"
  nginx http fastcgi temporary files: "fastcgi_temp"
  nginx http uwsgi temporary files: "uwsgi_temp"
  nginx http scgi temporary files: "scgi_temp"

其中 --prefix=/opt/openresty 指定了安裝目錄，不指定的話預設會安裝到 /usr/local/openresty 目錄下。

編譯安裝： make && make install

[root@localhost src]# cd /opt/openresty/
[root@localhost openresty]# ls
bin  luajit  lualib  nginx

可以看到 /opt/openresty 目錄下四個資料夾，其中包括了 luajit，nginx。

啟動openresty: /opt/openresty/nginx/sbin/nginx -c /opt/openresty/nginx/conf/nginx.conf -p /opt/openresty/nginx/

[root@localhost src]# ps -elf|grep nginx
1 S root      2076     1  0  80   0 - 34999 -      21:24 ?        00:00:00 nginx: master process /opt/openresty/nginx/sbin/nginx -c /opt/openresty/nginx/conf/nginx.conf -p /opt/openresty/nginx/
5 S nobody    2077  2076  0  80   0 - 35045 -      21:24 ?        00:00:00 nginx: worker process                                    
0 S root      2079  1678  0  80   0 -  1088 -      21:24 pts/1    00:00:00 grep nginx

驗證可以訪問： curl 127.0.0.1

2. content_by_lua 和 content_by_lua_file

nginx 如何嵌入 lua 指令碼。方法就是在nginx的配置檔案nginx.conf 中使用 content_by_lua 或者 cotent_by_lua_file 指令：

1) content_by_lua 一般在很簡單的lua指令碼時使用：

複製程式碼

        location /lua {
                set $test "hello, world.";
                content_by_lua '
                        ngx.header.content_type = "text/plain";
                        ngx.say(ngx.var.test);
                ';
        }

訪問 http://localhost/lua 可以看到輸出到頁面的 hello, world.

2）cotent_by_lua_file 適應於複雜的 lua 指令碼，專門放入一個檔案中：

        location /lua2 {
            #lua_code_cache off;
            content_by_lua_file lua/hello.lua;
        }

路徑相對於 /opt/openresty/nginx

[root@localhost lua]# pwd
/opt/openresty/nginx/lua
[root@localhost lua]# cat hello.lua
ngx.say('hello ngx_lua!!!!');

本例子中 hello.lua 只包含一句： ngx.say('hello ngx_lua!!!!');

訪問 /lua2 :

[root@localhost lua]# curl localhost/lua
hello ngx_lua!!!!

可以看到訪問成功。

在 nginx.conf 檔案的 server {.. ...} 中加入 lua_code_cache off; 可以方便除錯lua指令碼，修改lua指令碼之後，不需要 reload nginx.

openresty 中的 nginx 嵌入 luajit 的原理：

每一個nginx的程式中都嵌入了一個 luajit的虛擬機器，來執行lua指令碼。nginx將lua指令碼的執行交給了luajit vm.

3. ngx_lua 的指令和 API

上面我們說到 nginx 嵌入 lua 指令碼可以使用 content_by_lua 和 content_by_lua_file，它們其實是指令(Directives)，類似的指令還有很多，

具體參見：https://www.nginx.com/resources/wiki/modules/lua/#directives

這些指令都是 nginx 訪問 lua 指令碼的入口。

ngx_lua API：

指令是 nginx 訪問 lua 指令碼的入口。那麼lua指令碼如何呼叫nginx中的函式呢？就是通過 ngx_lua 的API 。

具體介紹參見：https://www.nginx.com/resources/wiki/modules/lua/#nginx-api-for-lua

The various *_by_lua and *_by_lua_file configuration directives serve as gateways to the Lua API within the nginx.conf file. The NGINX Lua API described below can only be called within the user Lua code run in the context of these configuration directives.

The API is exposed to Lua in the form of two standard packages ngx and ndk. These packages are in the default global scope within ngx_lua and are always available within ngx_lua directives.

其實nginx和Lua的互動開發主要就是指令和API，當然還有lua指令碼的語法。指令是nginx訪問lua的入口，API是lua呼叫nginx的函式，lua是指令碼程式語言。

指令其實很簡單，所以主要就是熟悉ngx_lua的 API 和Lua語法。

4. lua 訪問 redis

lua-resty-redis 模組：https://github.com/openresty/lua-resty-redis （有文件可以參考）

在nginx.conf中加入：

        location /redis_test{
            content_by_lua_file lua/redis_test.lua;
        }

redis_test.lua 內容:

[root@localhost lua]# cat redis_test.lua
local redis = require "resty.redis"
local red = redis:new()

red:set_timeout(1000)

local ok, err = red:connect("127.0.0.1", 6379)
if not ok then
        ngx.say("failed to connect: ", err)
        return
end

ngx.say("set result: ", ok)

local res, err = red:get("dog")
if not res then
        ngx.say("failed to get doy: ", err)
        return
end

if res == ngx.null then
        ngx.say("dog not found.")
        return
end

ngx.say("dog: ", res)

[root@localhost lua]#

訪問：

[root@localhost lua]# curl localhost/redis_test
set result: 1
dog: an animal
[root@localhost lua]#

我們看到訪問成功。

5. lua 訪問mysql

openresty的mysql模組：lua-resty-mysql :https://github.com/openresty/lua-resty-mysql（有文件可以參考）

在nginx.conf加入如下配置：

        location /mysql_test {
            content_by_lua_file lua/mysql_test.lua;
        }

mysql_test.lua指令碼內容：

複製程式碼

[root@localhost lua]# pwd
/opt/openresty/nginx/lua
[root@localhost lua]# cat mysql_test.lua
local mysql = require "resty.mysql"
local db, err = mysql:new()

if not db then
        ngx.say("failed to instantiate mysql: ", err)
        return
end

db:set_timeout(1000)

local ok, err, errno, sqlstate = db:connect{
        host = "127.0.0.1",
        port = 3306,
        database = "ngx_lua",
        user = "root",
        password="digdeep",
        max_packet_size = 1024 * 1024
}

if not ok then
        ngx.say("failed to connect: ", err, ": ", errno, " ", sqlstate)
        return
end

ngx.say("connected to mysql.")

local res, err, errno, sqlstate = db:query("drop table if exists cats")
if not res then
        ngx.say("bad result: ", err, ": ", errno, ": ", sqlstate, ".")
        return
end

res, err, errno, sqlstate = db:query("create table cats " .. "(id int not null primary key auto_increment, "
                                        .. "name varchar(30))")
if not res then
        ngx.say("bad result: ", err, ": ", errno, ": ", sqlstate, ".")
        return
end

ngx.say("table cats created.")

res, err, errno, sqlstate = db:query("insert into cats(name) " .. "values (\'Bob\'),(\'\'),(null)")
if not res then
        ngx.say("bad request: ", err, ": ", errno, ": ", sqlstate, ".")
        return
end

ngx.say(res.affected_rows, " rows inserted into table cats ", "(last insert id: ", res.insert_id, ")")

res, err, errno, sqlstate = db:query("select * from cats order by id asc", 10)
if not res then
        ngx.say("bad result ", err, ": ", errno, ": ", sqlstate, ".")
        return
end

local cjson = require "cjson"
ngx.say("result: ", cjson.encode(res))

local ok, err = db:set_keepalive(1000, 100)
if not ok then
        ngx.say("failed to set keepalive: ", err)
        return
end

測試：

[root@localhost lua]# curl localhost/mysql_test
connected to mysql.
table cats created.
3 rows inserted into table cats (last insert id: 1)
result: [{"name":"Bob","id":1},{"name":"","id":2},{"name":null,"id":3}]

測試通過。

5. lua 的 capture 和 capture_multi(子查詢)

capture_multi 是 openresty 一個十分強大的功能。它能極大的減少前端瀏覽器傳送的http請求的數量，突破了瀏覽器對於同一個伺服器併發請求數量的限制，因為他可以將前端的多個http請求減少為只要一個http請求到nginx，然後nginx使用capture_multi特性，對後端發起多個非同步併發請求，然後統一將結果返回給前端。下面看一個例子：

首先在nginx.conf中加入下面的 location 配置，並且配置好 nginx 訪問 php 的配置：

        location /capture {
            content_by_lua_file lua/capture.lua;
            #access_by_lua_file lua/capture.lua;
        }

        location ~ \.php$ {
            root           html;
            fastcgi_pass   127.0.0.1:9000;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
            include        fastcgi_params;

capture.lua 的程式碼如下：

[root@localhost lua]# pwd
/opt/openresty/nginx/lua
[root@localhost lua]# cat capture.lua
local res1,res2,res3,res4 = ngx.location.capture_multi{
        {"/mysql_test", {args="t=1&id=1"}},
        {"/redis_test", {args="t=2&id=2"}},
        {"/lua", {args="t=3&id=3"}},
        {"/index.php", {args="t=3&id=3"}},
}

ngx.header.content_type="text/plain"
ngx.say(res1.body)
ngx.say(res2.body)
ngx.say(res3.body)
ngx.say(res4.truncated)
ngx.say(res4.status)
ngx.say(res4.header["Set-Cookie"])

--ngx.say(res4.body)

index.php 程式碼：

[root@localhost html]# pwd
/opt/openresty/nginx/html
[root@localhost html]# cat index.php
<?php
        echo phpinfo();
?>

訪問：

[root@localhost html]# curl localhost/capture
connected to mysql.
table cats created.
3 rows inserted into table cats (last insert id: 1)
result: [{"name":"Bob","id":1},{"name":"","id":2},{"name":null,"id":3}]

set result: 1
dog: an animal

hello ngx_lua!!!!

false
200
nil

可以看到訪問成功了。/mysql_test，/redis_test, /lua, /index.php 四個請求的結果都輸出了。

注意：

ngx.location.capture_multi{... ...} 中的多個非同步併發請求可以是 nginx.conf 中配置的 location(比如 /mysql_test, /redis_test, /lua)，也可以不是 location配置的路徑，比如 index.php 就不是。index.php 就是一個簡單的後臺php 指令碼。當然也可以是一個 java 實現的後臺介面。

6. openresty的快取 lua_shared_dict

定義一個快取：

在nginx的配置檔案 nginx.conf 的 http 端下面加入指令：

lua_shared_dict ngx_cache 128m;

就定義了一個名稱為 ngx_cache 大小為128m的記憶體用於快取，注意該快取是所有nginx work process所共享的。

在lua指令碼中訪問快取：

local ngx_cache = ngx.shared.ngx_cache
local value = ngx_cache:get(key)

local succ, err, forcible = ngx_cache:set(key, value, exptime)

下面測試一下，首先在 nginx.conf的server端中加入：

        location /cache {
            content_by_lua_file lua/cache.lua;
        }

然後編寫 cache.lua 指令碼：

複製程式碼

[root@localhost lua]# cat cache.lua
local redis = require "resty.redis"
local red = redis:new()

function set_to_cache(key, value, exptime)
        if not exptime then
                exptime = 0
        end
        local ngx_cache = ngx.shared.ngx_cache
        local succ, err, forcible = ngx_cache:set(key, value, exptime)
        return succ
end

function get_from_cache(key)
        local ngx_cache = ngx.shared.ngx_cache;
        local value = ngx_cache:get(key)
        if not value then
                value = get_from_redis(key)
                set_to_cache(key, value)
                return value
        end

        ngx.say("get from cache.")
        return value
end

function get_from_redis(key)
        red:set_timeout(1000)

        local ok, err = red:connect("127.0.0.1", 6379)
        if not ok then
                ngx.say("failed to connect: ", err)
                return
        end

        local res, err = red:get(key)
        if not res then
                ngx.say("failed to get doy: ", err)
                return ngx.null
        end

        ngx.say("get from redis.")
        return res
end

function set_to_redis(key, value)
        red:set_timeout(1000)
        local ok, err = red:connect("127.0.0.1", 6379)
        if not ok then
                ngx.say("failed to connect: ", err)
                return
        end

        local ok, err = red:set(key, value)
        if not ok then
                ngx.say("failed to set to redis: ", err)
                return
        end
        return ok
end

set_to_redis('dog', "Bob")
local rs = get_from_cache('dog')
ngx.say(rs)

測試：

[root@localhost ~]# curl localhost/cache
get from redis.
Bob
[root@localhost ~]# curl localhost/cache
get from cache.
Bob
[root@localhost ~]# curl localhost/cache
get from cache.
Bob

第一次從 redis中獲取，以後每次都從cache中獲取。

可以使用 ab 測試一下rps(Requests per second):

 ab -n 1000 -c 100 -k http://127.0.0.1/cache

7. 解決快取失效風暴 lua-resty-lock

快取失效風暴是指快取因為時間過期而失效時，會導致所有的請求都去訪問後臺的redis或者mysql，而導致CPU效能即刻增長的現象。所以關鍵是當快取失效時，用lock保證只有一個執行緒去訪問後臺的redis或者mysql，然後更新快取。需要使用到 lua-resty-lock 模組的加鎖、解鎖功能。

lua-resty-lock 文件：https://github.com/openresty/lua-resty-lock

首先在nginx.conf 的 http 端下面加入指令：

lua_shared_dict ngx_cache 128m;     # cache
lua_shared_dict cache_lock 100k;    # lock for cache

然後在nginx.conf的server端中加入：

        location /cache_lock {
            content_by_lua_file lua/cache_lock.lua;
        }

cache_lock.lua程式碼：

[root@localhost lua]# cat cache_lock.lua
local redis = require "resty.redis"
local red = redis:new()
local resty_lock = require "resty.lock"
local ngx_cache = ngx.shared.ngx_cache

function set_to_cache(key, value, exptime)
        if not exptime then
                exptime = 0
        end
        local succ, err, forcible = ngx_cache:set(key, value, exptime)
        return succ
end

function get_from_cache(key)
        local ngx_cache = ngx.shared.ngx_cache;
        local value = ngx_cache:get(key)
        if not value then       -- cache miss
                local lock = resty_lock:new("cache_lock")
                local elapsed, err = lock:lock(key)
                if not elapsed then
                        return fail("failed to acquire the lock: ", err)
                end

                value = get_from_redis(key)
                if not value then
                        local ok, err = lock:unlock()
                        if not ok then
                                return fail("failed to unlock: ", err)
                        end
                        ngx.say("no value found")
                        return
                end

                local ok, err = ngx_cache:set(key, value, 1)
                if not ok then
                        local ok, err = lock:unlock()
                        if not ok then
                                return fail("failed to unlock: ", err)
                        end
                        return faile("failed to update ngx_cache: ", err)
                end

                local ok, err = lock:unlock()
                if not ok then
                        return faile("failed to unlock: ", err)
                end

                return value
        end

        ngx.say("get from cache.")
        return value
end

function get_from_redis(key)
        red:set_timeout(1000)

        local ok, err = red:connect("127.0.0.1", 6379)
        if not ok then
                ngx.say("failed to connect: ", err)
                return
        end

        local res, err = red:get(key)
        if not res then
                ngx.say("failed to get doy: ", err)
                return ngx.null
        end

        ngx.say("get from redis.")
        return res
end

function set_to_redis(key, value)
        red:set_timeout(1000)
        local ok, err = red:connect("127.0.0.1", 6379)
        if not ok then
                ngx.say("failed to connect: ", err)
                return
        end

        local ok, err = red:set(key, value)
        if not ok then
                ngx.say("failed to set to redis: ", err)
                return
        end
        return ok
end

set_to_redis('dog', "Bob")
local rs = get_from_cache('dog')
ngx.say(rs)

測試：

[root@localhost lua]# curl localhost/cache_lock
get from cache.
Bob
[root@localhost lua]# curl localhost/cache_lock
get from cache.
Bob

7. openresty 執行階段

nginx的執行階段分成了很多個階段,所以第三方模組就可以在某個適當的階段加入一些處理。openresty進行了簡化成了7個階段：

7個階段的執行順序如下：

set_by_lua: 流程分支判斷，判斷變數初始哈

rewrite_by_lua: 用lua指令碼實現nginx rewrite

access_by_lua: ip准入，是否能合法性訪問，防火牆

content_by_lua: 記憶體生成

header_filter_by_lua：過濾http頭資訊，增加頭資訊

body_filter_by_lua: 內容大小寫，內容加密

log_by_lua: 本地/遠端記錄日誌

但是其實我們可以只用 content_by_lua，所有功能都在該階段完成，也是可以的。

Nginx+Lua開發環境搭建
2017-05-24
Nginx開發環境
安裝Nginx+Lua開發環境
2019-04-13
Nginx開發環境
Nginx+lua 實現呼叫.so檔案
2019-04-13
Nginx
入門入門入門 MySQL命名行
2020-12-15
MySql
如何入CTF的“門”？——所謂入門就是入門
2020-12-21
何入CTF的“門”？——所謂入門就是入門
2020-11-04
scala 從入門到入門+
2019-02-16
makefile從入門到入門
2020-10-06
ACM入門之新手入門
2012-07-20
ACM
【小入門】react極簡入門
2019-07-22
React
gRPC（二）入門：Protobuf入門
2022-11-09
RPC
《Flutter 入門經典》之“Flutter 入門 ”
2021-02-02
Flutter
新手入門，webpack入門詳細教程
2018-11-15
Web
Android入門教程 | RecyclerView使用入門
2021-11-07
AndroidView
linux新手入門――shell入門(轉)
2007-08-10
Linux
MyBatis從入門到精通(一)：MyBatis入門
2019-06-28
MyBatis
SqlSugar ORM 入門到精通【一】入門篇
2018-02-21
SqlSugarORM
Storm入門指南第二章入門
2015-11-18
ORM
VUE入門
2020-06-18
Vue
MyBatis 入門
2020-02-16
MyBatis
CSS 入門
2020-07-08
CSS
JavaScript 入門
2020-07-07
JavaScript
Nginx 入門
2020-08-10
Nginx
RabbitMQ入門
2020-08-24
MQ
GitHub入門
2020-04-05
Github
Redis 入門
2019-12-23
Redis
PostgreSQL 入門
2019-11-20
SQL
Prometheus 入門
2019-07-30
Prometheus
VuePress 入門
2019-02-21
Vue
Tableau入門
2019-02-28
WebSocket 入門
2019-03-27
Web
Webpack 入門
2019-03-27
Web
UML入門
2019-04-04
nlp入門
2019-03-05
gulp入門
2019-03-04
Dubbo 入門
2019-04-21
AVFoundation 入門
2019-04-19
Nginx入門
2019-07-03
Nginx

Nginx+Lua 入門

相關文章