kafka-ngx_kafka_module

森林森發表於2020-11-16

使用Kafka做日誌收集。

一、需要收集的資訊：

1、使用者ID（user_id）

2、時間（act_time）

3、操作（action，可以是：點選：click，收藏：job_collect，投簡歷：cv_send，上傳簡歷：cv_upload）

4、對方企業編碼（job_code）

git clone https://github.com/edenhill/librdkafka
cd librdkafka
./configure
make
sudo make install

下載模組

git clone https://github.com/brg-liuwei/ngx_kafka_module

# cd /path/to/nginx

./configure --add-module=/usr/local/zookeeper/ngx_kafka_module
make
sudo make install

啟動nginx


./nginx start

loading shared libraries: librdkafka.so.1: cannot open shared object file: No such file or directory

執行下面的操作
解決辦法：
載入so庫：
執行命令：

echo "/usr/local/lib" >> /etc/ld.so.conf

/etc/ld.so.conf這個是Linux上的檔案，裡面放的東西是指定Linux在啟動時要載入的檔案。

然後再執行下面的命令使修改生效：

ldconfig

修改nginx

cd /usr/local/nginx/conf
vim nginx.conf

	kafka;

    kafka_broker_list 192.168.181.141:9092 192.168.181.142:9092 192.168.181.144:9092; # host:port ...

    server {

        # some other configs

        location = /log {
            # optional directive: kafka_partition [<partition-num> | auto]
            #
            # kafka_partition auto; # default value
            # kafka_partition 0;
            # kafka_partition 1;

            add_header 'Access-Control-Allow-Origin' $http_origin;
			add_header 'Access-Control-Allow-Credentials' 'true';
			add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
			add_header 'Access-Control-Allow-Headers' 'DNT,web-token,app-token,Authorization,Accept,Origin,Keep-Alive,User-Agent,X-Mx-ReqToken,X-Data-Type,X-Auth-Token,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
			add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
			if ($request_method = 'OPTIONS') {
            add_header 'Access-Control-Max-Age' 1728000;
            add_header 'Content-Type' 'text/plain; charset=utf-8';
            add_header 'Content-Length' 0;
            return 204;
          }
          kafka_topic monitor_logs;
        }
    }

html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>index</title>
    <style>
        button{
            width: 150px;
            height: 80px;
        }
    </style>

    <script src="https://apps.bdimg.com/libs/jquery/2.1.4/jquery.min.js"></script>
</head>
<body>

<button id="login">登入</button>

<button id="store">收藏簡歷</button>

<button id="deliver">投簡歷</button>

<button id="view">檢視簡歷詳情</button>

<button id="upload">上傳簡歷</button>

</body>
<script>


    function clickEvent(id,message) {
        $("#"+id).click(function () {
            $.ajax({
                url: "/log",
                contentType:"application/json",
                dataType: "JSON",
                data:{
                    type:id,
                    desc:message,
                    time:Date.parse(new Date())
                },
                type:"POST",
                success: function (result) {
                    alert(message+"成功");
                }
            });
        });
    }
    clickEvent("login","user login system");
    clickEvent("store","user store resume");
    clickEvent("deliver","user deliver resume");
    clickEvent("view","user view resume");
    clickEvent("upload","user upload resume");

</script>

</html>