分散式協調服務ZooKeeper的典型應用

Zollty發表於2016-12-31

Zookeeper典型應用


參考官方文件:

http://zookeeper.apache.org/doc/current/recipes.html


Barriers(障礙)

Distributed systems use barriers to block processing of a set of nodes until a condition is met at which time all the nodes are allowed to proceed.

Double Barriers(雙屏障)

Double barriers enable clients to synchronize the beginning and the end of a computation. When enough processes have joined the barrier, processes start their computation and leave the barrier once they have finished. 


Distributed Queues(分散式佇列)


Distributed Locks(分散式鎖,排它鎖)

At any snapshot in time no two clients think they hold the same lock.

Shared Locks(共享鎖,又稱為讀鎖,可以檢視,但無法修改和刪除的一種資料鎖)

Recoverable Shared Locks(可撤銷的共享鎖)


Two-phased Commit(兩階段提交協議)

A two-phase commit protocol is an algorithm that lets all clients in a distributed system agree either to commit a transaction or abort.

兩階段提交協議可以保證資料的強一致性,許多分散式關係型資料管理系統採用此協議來完成分散式事務。它是協調所有分散式原子事務參與者,並決定提交或取消(回滾)的分散式演算法。

在兩階段提交協議中,系統一般包含兩類機器(或節點):一類為協調者(coordinator),通常一個系統中只有一個;另一類為事務參與者(participants,cohorts或workers),一般包含多個,在資料儲存系統中可以理解為資料副本的個數。協議中假設每個節點都會記錄寫前日誌(write-ahead log)並永續性儲存,即使節點發生故障日誌也不會丟失。協議中同時假設節點不會發生永久性故障而且任意兩個節點都可以互相通訊。


當事務的最後一步完成之後,協調器執行協議,參與者根據本地事務能夠成功完成回覆同意提交事務或者回滾事務。


顧名思義,兩階段提交協議由兩個階段組成。在正常的執行下,這兩個階段的執行過程如下所述:


階段1:請求階段(commit-request phase,或稱表決階段,voting phase)


在請求階段,協調者將通知事務參與者準備提交或取消事務,然後進入表決過程。在表決過程中,參與者將告知協調者自己的決策:同意(事務參與者本地作業執行成功)或取消(本地作業執行故障)。


階段2:提交階段(commit phase)


在該階段,協調者將基於第一個階段的投票結果進行決策:提交或取消。當且僅當所有的參與者同意提交事務協調者才通知所有的參與者提交事務,否則協調者將通知所有的參與者取消事務。參與者在接收到協調者發來的訊息後將執行響應的操作。


Leader Election(Leader 選舉)


具體應用可以使用 Curator 的實現

具體用法參見官方文件:

http://curator.apache.org/curator-recipes/index.html

下面列出它的所有功能:

Curator implements all of the recipes listed on the ZooKeeper recipes doc (except two phase commit). Click on the recipe name below for detailed documentation. NOTE: Most Curator recipes will autocreate parent nodes of paths given to the recipe as CreateMode.CONTAINER. Also, see Tech Note 7 regarding "Curator Recipes Own Their ZNode/Paths".

Elections
Leader Latch - In distributed computing, leader election is the process of designating a single process as the organizer of some task distributed among several computers (nodes). Before the task is begun, all network nodes are unaware which node will serve as the "leader," or coordinator, of the task. After a leader election algorithm has been run, however, each node throughout the network recognizes a particular, unique node as the task leader.
Leader Election - Initial Curator leader election recipe.
Locks
Shared Reentrant Lock - Fully distributed locks that are globally synchronous, meaning at any snapshot in time no two clients think they hold the same lock.
Shared Lock - Similar to Shared Reentrant Lock but not reentrant.
Shared Reentrant Read Write Lock - A re-entrant read/write mutex that works across JVMs. A read write lock maintains a pair of associated locks, one for read-only operations and one for writing. The read lock may be held simultaneously by multiple reader processes, so long as there are no writers. The write lock is exclusive.
Shared Semaphore - A counting semaphore that works across JVMs. All processes in all JVMs that use the same lock path will achieve an inter-process limited set of leases. Further, this semaphore is mostly "fair" - each user will get a lease in the order requested (from ZK's point of view).
Multi Shared Lock - A container that manages multiple locks as a single entity. When acquire() is called, all the locks are acquired. If that fails, any paths that were acquired are released. Similarly, when release() is called, all locks are released (failures are ignored).
Barriers
Barrier - Distributed systems use barriers to block processing of a set of nodes until a condition is met at which time all the nodes are allowed to proceed.
Double Barrier - Double barriers enable clients to synchronize the beginning and the end of a computation. When enough processes have joined the barrier, processes start their computation and leave the barrier once they have finished.
Counters
Shared Counter - Manages a shared integer. All clients watching the same path will have the up-to-date value of the shared integer (considering ZK's normal consistency guarantees).
Distributed Atomic Long - A counter that attempts atomic increments. It first tries using optimistic locking. If that fails, an optional InterProcessMutex is taken. For both optimistic and mutex, a retry policy is used to retry the increment.
Caches
Path Cache - A Path Cache is used to watch a ZNode. Whenever a child is added, updated or removed, the Path Cache will change its state to contain the current set of children, the children's data and the children's state. Path caches in the Curator Framework are provided by the PathChildrenCache class. Changes to the path are passed to registered PathChildrenCacheListener instances.
Node Cache - A utility that attempts to keep the data from a node locally cached. This class will watch the node, respond to update/create/delete events, pull down the data, etc. You can register a listener that will get notified when changes occur.
Tree Cache - A utility that attempts to keep all data from all children of a ZK path locally cached. This class will watch the ZK path, respond to update/create/delete events, pull down the data, etc. You can register a listener that will get notified when changes occur.
Nodes
Persistent Ephemeral Node - An ephemeral node that attempts to stay present in ZooKeeper, even through connection and session interruptions.
Group Member - Group membership management. Adds this instance into a group and keeps a cache of members in the group.
Queues
Distributed Queue - An implementation of the Distributed Queue ZK recipe. Items put into the queue are guaranteed to be ordered (by means of ZK's PERSISTENTSEQUENTIAL node). If a single consumer takes items out of the queue, they will be ordered FIFO. If ordering is important, use a LeaderSelector to nominate a single consumer.
Distributed Id Queue - A version of DistributedQueue that allows IDs to be associated with queue items. Items can then be removed from the queue if needed.
Distributed Priority Queue - An implementation of the Distributed Priority Queue ZK recipe.
Distributed Delay Queue - An implementation of a Distributed Delay Queue.
Simple Distributed Queue - A drop-in replacement for the DistributedQueue that comes with the ZK distribution.


相關文章