011-openresty的maxminddb外掛

這是堅持技術寫作計劃（含翻譯）的第11篇，定個小目標999，每週最少2篇。

本文主要介紹我之前基於openresty寫的maxminddb的解析外掛 -- anjia0532/lua-resty-maxminddb (已開源)。主要用途是根據ip獲取地理位置。國內精確度不如國內ipip.net ，但是勝在免費。在精確度要求不高的場景，還是可以用的。

如果要用ipip.net的lua庫，可以參考官方的 ipipdotnet/ipdb-luajit

前提條件

OpenResty

# import our GPG key:
wget -qO - https://openresty.org/package/pubkey.gpg | sudo apt-key add -

# for installing the add-apt-repository command
# (you can remove this package and its dependencies later):
sudo apt-get -y install software-properties-common

# add the our official APT repository:
sudo add-apt-repository -y "deb http://openresty.org/package/ubuntu $(lsb_release -sc) main"

# to update the APT index:
sudo apt-get update

sudo apt-get install openresty
複製程式碼

maxmind/libmaxminddb && maxmind/geoipupdate

sudo add-apt-repository ppa:maxmind/ppa
sudo apt update
sudo apt install libmaxminddb0 libmaxminddb-dev mmdb-bin geoipupdate
複製程式碼

配置 geoipupdate

sudo tee /etc/GeoIP.conf <<-'EOF'
# The following AccountID and LicenseKey are required placeholders.
# For geoipupdate versions earlier than 2.5.0, use UserId here instead of AccountID.
AccountID 0
LicenseKey 000000000000

# Include one or more of the following edition IDs:
# * GeoLite2-City - GeoLite 2 City
# * GeoLite2-Country - GeoLite2 Country
# For geoipupdate versions earlier than 2.5.0, use ProductIds here instead of EditionIDs.
EditionIDs GeoLite2-City GeoLite2-Country
EOF

sudo /usr/local/bin/geoipupdate
複製程式碼

安裝和使用 lua-resty-maxminddb

安裝

opm get anjia0532/lua-resty-maxminddb
複製程式碼

配置openresty

  local cjson = require 'cjson'
  local geo = require 'resty.maxminddb'
  if not geo.initted() then
      geo.init("/path/to/GeoLite2-City.mmdb")
  end
  local res,err = geo.lookup(ngx.var.arg_ip or ngx.var.remote_addr) --support ipv6 e.g. 2001:4860:0:1001::3004:ef68

  if not res then
      ngx.log(ngx.ERR,'failed to lookup by ip ,reason:',err)
  end

  ngx.say("full :",cjson.encode(res))
  if ngx.var.arg_node then
     ngx.say("node name:",ngx.var.arg_node," ,value:", cjson.encode(res[ngx.var.arg_node] or {}))
  end
複製程式碼

測試

#ipv4
curl localhost/ip=114.114.114.114&node=city
#ipv6
#curl localhost/ip=2001:4860:0:1001::3004:ef68&node=country
full :{"city":{"geoname_id":1799962,"names":{"en":"Nanjing","ru":"Нанкин","fr":"Nankin","pt-BR":"Nanquim","zh-CN":"南京","es":"Nankín","de":"Nanjing","ja":"南京市"}},"subdivisions":[{"geoname_id":1806260,"names":{"en":"Jiangsu","fr":"Province de Jiangsu","zh-CN":"江蘇省"},"iso_code":"32"}],"country":{"geoname_id":1814991,"names":{"en":"China","ru":"Китай","fr":"Chine","pt-BR":"China","zh-CN":"中國","es":"China","de":"China","ja":"中國"},"iso_code":"CN"},"registered_country":{"geoname_id":1814991,"names":{"en":"China","ru":"Китай","fr":"Chine","pt-BR":"China","zh-CN":"中國","es":"China","de":"China","ja":"中國"},"iso_code":"CN"},"location":{"time_zone":"Asia\/Shanghai","longitude":118.7778,"accuracy_radius":50,"latitude":32.0617},"continent":{"geoname_id":6255147,"names":{"en":"Asia","ru":"Азия","fr":"Asie","pt-BR":"Ásia","zh-CN":"亞洲","es":"Asia","de":"Asien","ja":"アジア"},"code":"AS"}}
node name:city ,value:{"geoname_id":1799962,"names":{"en":"Nanjing","ru":"Нанкин","fr":"Nankin","pt-BR":"Nanquim","zh-CN":"南京","es":"Nankín","de":"Nanjing","ja":"南京市"}}
複製程式碼

格式化一下

full: {
    "city": {
        "geoname_id": 1799962,
        "names": {
            "en": "Nanjing",
            "ru": "Нанкин",
            "fr": "Nankin",
            "pt-BR": "Nanquim",
            "zh-CN": "南京",
            "es": "Nankín",
            "de": "Nanjing",
            "ja": "南京市"
        }
    },
    "subdivisions": [{
            "geoname_id": 1806260,
            "names": {
                "en": "Jiangsu",
                "fr": "Province de Jiangsu",
                "zh-CN": "江蘇省"
            },
            "iso_code": "32"
        }
    ],
    "country": {
        "geoname_id": 1814991,
        "names": {
            "en": "China",
            "ru": "Китай",
            "fr": "Chine",
            "pt-BR": "China",
            "zh-CN": "中國",
            "es": "China",
            "de": "China",
            "ja": "中國"
        },
        "iso_code": "CN"
    },
    "registered_country": {
        "geoname_id": 1814991,
        "names": {
            "en": "China",
            "ru": "Китай",
            "fr": "Chine",
            "pt-BR": "China",
            "zh-CN": "中國",
            "es": "China",
            "de": "China",
            "ja": "中國"
        },
        "iso_code": "CN"
    },
    "location": {
        "time_zone": "Asia\/Shanghai",
        "longitude": 118.7778,
        "accuracy_radius": 50,
        "latitude": 32.0617
    },
    "continent": {
        "geoname_id": 6255147,
        "names": {
            "en": "Asia",
            "ru": "Азия",
            "fr": "Asie",
            "pt-BR": "Ásia",
            "zh-CN": "亞洲",
            "es": "Asia",
            "de": "Asien",
            "ja": "アジア"
        },
        "code": "AS"
    }
}
node name: city, value: {
    "geoname_id": 1799962,
    "names": {
        "en": "Nanjing",
        "ru": "Нанкин",
        "fr": "Nankin",
        "pt-BR": "Nanquim",
        "zh-CN": "南京",
        "es": "Nankín",
        "de": "Nanjing",
        "ja": "南京市"
    }
}
複製程式碼

壓測 & 效能

事先安裝好 wrk

sudo tee /tmp/wrk.lua <<-'EOF'
wrk.method = "GET";
wrk.body = "";

logfile = io.open("wrk.log", "w");

request = function()
ip = tostring(math.random(1, 255)).."."..tostring(math.random(1, 255)).."."..tostring(math.random(1, 255)).."."..tostring(math.random(1, 255))
path = "/?ip=" .. ip
return wrk.format(nil, path)
end

response = function(status,header,body)
logfile:write("\nbody:" .. body .. "\n-----------------");
end
EOF

sudo wrk -t50 -c200 -d120s -s /tmp/wrk.lua --latency http://127.0.0.1
複製程式碼

參考資料

• OpenResty® Linux Packages
• Automatic Updates for GeoIP2 and GeoIP Legacy Databases
• maxmind/libmaxminddb
• maxmind/geoipupdate
• wg/wrk#wiki
• MMDB_free_entry_data_list (entry_data_list=0x23) at maxminddb.c:1860 #9

招聘小廣告

山東濟南的小夥伴歡迎投簡歷啊 加入我們 , 一起搞事情。

長期招聘，Java程式設計師，大資料工程師，運維工程師，前端工程師。