Storm-原始碼分析-timer(backtype.storm.timer)

寒凝雪發表於2017-05-02
原文網址 : https://flycode.co/archives/191210
ORM原始碼

mk-timer

timer是基於PriorityQueue實現的(和PriorityBlockingQueue區別, 在於沒有阻塞機制, 不是執行緒安全的), 優先順序佇列是堆資料結構的典型應用 
預設情況下, 按照自然順序(其實就是預設comparator的定義), 最小的元素排在堆頭 
當然也可以自己重新實現comparator介面, 比如timer就用reify重新實現了comparator介面

整個過程其實比較簡單, 開個timer-thread, 不斷check PriorityQueue裡面時間最小的timer是否已經可以觸發 
如果可以, 就poll出來, 呼叫callback, 並sleep, 都很好理解

唯一需要說的是, 這裡使用Semaphore, 
訊號量和lock相似, 都是用於互斥 
不同在於, 訊號量模擬資源管理, 所以不同於lock的排他, 訊號量可以接收多個aquire(取決於配置) 
另外一個比較大的區別, lock是解鈴還須繫鈴人, 誰鎖誰解, 而訊號量無所謂, 任何執行緒都可以呼叫release, 或acquire 
這裡使用訊號量, 是用於在cancel-timer時, 等待timer-thread結束

(defn cancel-timer [timer]
  (check-active! timer)
  (locking (:lock timer)
    (reset! (:active timer) false)
    (.interrupt (:timer-thread timer)))
  (.acquire (:cancel-notifier timer)))

因為cancel的過程就是將active置false, 然後就是呼叫acquire等待訊號量cancel-notifier被釋放 
而timer-thread線上程結束前, 會release這個訊號量

 

(defnk mk-timer [:kill-fn (fn [& _] )]
  (let [queue (PriorityQueue. 10
                              (reify Comparator
                                (compare [this o1 o2]
                                  (- (first o1) (first o2))
                                  )
                                (equals [this obj]
                                  true
                                  )))
        active (atom true) ;;標誌位
        lock (Object.)     ;;建立lock物件, 由於PriorityQueue非執行緒安全, 所以使用locking來保證同時只有一個執行緒訪問queue
        notifier (Semaphore. 0) ;;建立訊號量, 初始為0
        timer-thread (Thread.
                      (fn []
                        (while @active
                          (try
                            ;;peek讀但不從queue中取出, 先讀出time看看, 符合條件再取出 
                            (let [[time-secs _ _ :as elem] (locking lock (.peek queue))]
                              (if (and elem (>= (current-time-secs) time-secs)) 
                                ;;無法保證恰好, 只要當前時間>=time-secs, 就可以執行, 可想而知對於afn必須不能耗時, 否則會影響其他timer
                                ;; imperative to not run the function inside the timer lock
                                ;; otherwise, it`s possible to deadlock if function deals with other locks
                                ;; (like the submit lock)
                                (let [afn (locking lock (second (.poll queue)))]  ;;poll從queue中取出
                                  (afn))    ;;真正執行timer中的callback
                                (Time/sleep 1000)
                                ))
                            (catch Throwable t
                              ;; because the interrupted exception can be wrapped in a runtimeexception
                              (when-not (exception-cause? InterruptedException t)
                                (kill-fn t)
                                (reset! active false)
                                (throw t))
                              )))
                        (.release notifier)))]
    (.setDaemon timer-thread true)
    (.setPriority timer-thread Thread/MAX_PRIORITY)
    (.start timer-thread)
    {:timer-thread timer-thread
     :queue queue
     :active active
     :lock lock
     :cancel-notifier notifier}))

 

schedule

schedule其實就是往PriorityQueue裡面插入timer

對於迴圈schdule, 就是在timer的callback裡面, 再次schedule

(defnk schedule [timer delay-secs afn :check-active true]
  (when check-active (check-active! timer))
  (let [id (uuid)
        ^PriorityQueue queue (:queue timer)]
    (locking (:lock timer)
      (.add queue [(+ (current-time-secs) delay-secs) afn id])
      )))

(defn schedule-recurring [timer delay-secs recur-secs afn]
  (schedule timer
            delay-secs
            (fn this []
              (afn)
              (schedule timer recur-secs this :check-active false)) ; this avoids a race condition with cancel-timer
            ))

 

使用例子

Supervisor中的使用例子, 定期的呼叫hb函式更新supervisor的hb 
在mk-timer時, 傳入的kill-fn callback, 會在timer-thread發生exception的時候被呼叫

:timer (mk-timer :kill-fn (fn [t]
                            (log-error t "Error when processing event")
                            (halt-process! 20 "Error when processing an event")
                            ))


(schedule-recurring (:timer supervisor)
                        0
                        (conf SUPERVISOR-HEARTBEAT-FREQUENCY-SECS)
                        heartbeat-fn)

本文章摘自部落格園，原文釋出日期：2013-07-02


相關文章