Storm-原始碼分析-hook(backtype.storm.hooks)

寒凝雪發表於2017-05-02
ORM原始碼Hook

task hook

在某些task事件發生時, 如果使用者希望執行一些額外的邏輯, 就需要使用hook

當前定義如下事件, emit, cleanup, spoutAck……

使用者只需要開發實現ITaskHook的類, 並將類名配置到(storm-conf TOPOLOGY-AUTO-TASK-HOOKS)

系統會在這些事件發生時, 自動呼叫所有註冊的hook中的相應的functions

 

public interface ITaskHook {
    void prepare(Map conf, TopologyContext context);
    void cleanup();
    void emit(EmitInfo info);
    void spoutAck(SpoutAckInfo info);
    void spoutFail(SpoutFailInfo info);
    void boltExecute(BoltExecuteInfo info);
    void boltAck(BoltAckInfo info);
    void boltFail(BoltFailInfo info);
}


public class EmitInfo {
    public List<Object> values;
    public String stream;
    public int taskId;
    public Collection<Integer> outTasks;
    
    public EmitInfo(List<Object> values, String stream, int taskId, Collection<Integer> outTasks) {
        this.values = values;
        this.stream = stream;
        this.taskId = taskId;
        this.outTasks = outTasks;
    }
}

 

1. add hook

在mk-task的時候, 會從storm-conf配置裡面讀出hooks的class names 
建立hook物件, 加入到TopologyContext的_hooks中

(defn mk-task [executor-data task-id]
    (doseq [klass (storm-conf TOPOLOGY-AUTO-TASK-HOOKS)]
      (.addTaskHook ^TopologyContext (:user-context task-data) (-> klass Class/forName .newInstance)))
)
public class TopologyContext extends WorkerTopologyContext implements IMetricsContext {
    private List<ITaskHook> _hooks = new ArrayList<ITaskHook>();
    
    public void addTaskHook(ITaskHook hook) {
        hook.prepare(_stormConf, this);
        _hooks.add(hook);
    }
    
    public Collection<ITaskHook> getHooks() {
        return _hooks;
    }
}

 

2. apply hook

當發生相應的事件時, 呼叫事先註冊的hooks

下面的例子是在emit時, 呼叫相應的hooks 
apply-hooks巨集實現也很簡單, 從topology context中取出hooks列表, 對每個hook呼叫emit(EmitInfo)

(apply-hooks user-context .emit (EmitInfo. values stream task-id out-tasks))


(defmacro apply-hooks [topology-context method-sym info-form]
  (let [hook-sym (with-meta (gensym "hook") {:tag `backtype.storm.hooks.ITaskHook})]
    `(let [hooks# (get-context-hooks ~topology-context)]
       (when-not (hooks-empty? hooks#)
         (let [info# ~info-form]
           (fast-list-iter [~hook-sym hooks#]
             (~method-sym ~hook-sym info#)
             ))))))

本文章摘自部落格園，原文釋出日期：2013-07-30


相關文章