聊聊jesque在redis中的資料結構

codecraft發表於2017-11-18
Redis資料結構

序

本文主要介紹一下jesque在redis的儲存結構

示例配置

namespace=demo
listen-queue=demoqueue

啟動時的reids物件

127.0.0.1:6379> keys *
1) "demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started"
2) "demo:worker:Mars.local:3820-1:JAVA_DYNAMIC_QUEUES,demoqueue:started"
3) "demo:worker:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue:started"
4) "demo:workers"
5) "demo:worker:Mars.local:3820-2:JAVA_DYNAMIC_QUEUES,demoqueue:started"
6) "demo:worker:Mars.local:3820-4:JAVA_DYNAMIC_QUEUES,demoqueue:started"
127.0.0.1:6379> type demo:workers
set
127.0.0.1:6379> smembers demo:workers
1) "Mars.local:3820-4:JAVA_DYNAMIC_QUEUES,demoqueue"
2) "Mars.local:3820-2:JAVA_DYNAMIC_QUEUES,demoqueue"
3) "Mars.local:3820-1:JAVA_DYNAMIC_QUEUES,demoqueue"
4) "Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue"
5) "Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue"

namespace:workers

set型別的資料介面，儲存該namespace下的workers的名稱

namespace:worker:host:queueType,listen-queue:state

這個是標記每個namespace的worker的狀態，是個string型別，儲存事件產生時間

127.0.0.1:6379> type demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started
string
127.0.0.1:6379> get demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started
"2017-11-18T15:35:24.913+0800"

新增job

127.0.0.1:6379> keys *
1) "demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started"
2) "demo:queue:demoqueue"
3) "demo:worker:Mars.local:3820-1:JAVA_DYNAMIC_QUEUES,demoqueue:started"
4) "demo:worker:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue:started"
5) "demo:queues"
6) "demo:workers"
7) "demo:worker:Mars.local:3820-2:JAVA_DYNAMIC_QUEUES,demoqueue:started"
8) "demo:worker:Mars.local:3820-4:JAVA_DYNAMIC_QUEUES,demoqueue:started"

可以發現新增了幾個queue相關的物件
demo:queue:demoqueue這個是個鎖資訊

namespace:queues

是一個set結構，儲存佇列資訊

127.0.0.1:6379> smembers demo:queues
1) "demoqueue"

處理完job

127.0.0.1:6379> keys *
1) "demo:worker:Mars.local:3820-1:JAVA_DYNAMIC_QUEUES,demoqueue:started"
2) "demo:worker:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue:started"
3) "demo:queues"
4) "demo:stat:processed"
5) "demo:stat:processed:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue"
6) "demo:worker:Mars.local:3820-3:JAVA_DYNAMIC_QUEUES,demoqueue:started"
7) "demo:workers"
8) "demo:worker:Mars.local:3820-4:JAVA_DYNAMIC_QUEUES,demoqueue:started"
9) "demo:worker:Mars.local:3820-2:JAVA_DYNAMIC_QUEUES,demoqueue:started"

可以發現新增了兩個stat相關的物件

namespace:stat:processed

是string型別，統計處理過的job的數量

127.0.0.1:6379> type demo:stat:processed
string
127.0.0.1:6379> get demo:stat:processed
"1"

namespace:stat:processed:host:queueType,listen-queue

儲存每個host處理每個queue的數量

127.0.0.1:6379> type demo:stat:processed:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue
string
127.0.0.1:6379> get demo:stat:processed:Mars.local:3820-0:JAVA_DYNAMIC_QUEUES,demoqueue
"1"

關閉worker的時候

127.0.0.1:6379> keys *
1) "demo:queues"
2) "demo:stat:processed"

就只剩下兩個資訊

job執行失敗的時候

127.0.0.1:6379> keys *
 1) "demo:stat:failed"
 2) "demo:failed"
 3) "demo:queues"
 4) "demo:stat:processed"
 5) "demo:worker:Mars.local:3947-2:JAVA_DYNAMIC_QUEUES,demoqueue:started"
 6) "demo:worker:Mars.local:3947-3:JAVA_DYNAMIC_QUEUES,demoqueue:started"
 7) "demo:worker:Mars.local:3947-1:JAVA_DYNAMIC_QUEUES,demoqueue:started"
 8) "demo:worker:Mars.local:3947-4:JAVA_DYNAMIC_QUEUES,demoqueue:started"
 9) "demo:worker:Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue:started"
10) "demo:workers"
11) "demo:stat:failed:Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue"

可以發現多了三個failed相關的物件

namespace:failed

是一個list型別

127.0.0.1:6379> type demo:failed
list
127.0.0.1:6379> lrange demo:failed 0 -1
1) "{"worker":"Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue","queue":"demoqueue","payload":{"class":"com.example.demo.job.DemoJob","args":["111",1510991413785],"vars":null,"jobId":"c4c0289e-5d13-4bc3-8499-f3e35c5956a6"},"backtrace":["\tat com.example.demo.job.DemoJob.run(DemoJob.java:30)","\tat cn.codecraft.jesque.component.RedisPooledWorkerImpl.execute(RedisPooledWorkerImpl.java:656)","\tat cn.codecraft.jesque.component.RedisPooledWorkerImpl.process(RedisPooledWorkerImpl.java:611)","\tat cn.codecraft.jesque.component.RedisPooledWorkerImpl.poll(RedisPooledWorkerImpl.java:420)","\tat cn.codecraft.jesque.component.RedisPooledWorkerImpl.run(RedisPooledWorkerImpl.java:171)","\tat java.lang.Thread.run(Thread.java:745)"],"error":"demo ex","exception":"java.lang.RuntimeException","failed_at":"2017-11-18T07:50:33.921+0000","retried_at":null}"

儲存了具體的job資訊及異常

namespace:stat:failed

儲存失敗job的計數

127.0.0.1:6379> type demo:stat:failed
string
127.0.0.1:6379> get demo:stat:failed
"1"

namespace:stat:failed:host:queueType,listen-queue

儲存每個host每個queue處理失敗的數量

127.0.0.1:6379> type demo:stat:failed:Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue
string
127.0.0.1:6379> get demo:stat:failed:Mars.local:3947-0:JAVA_DYNAMIC_QUEUES,demoqueue
"1"

相關文章