本節主要通過幾個例子來介紹貪心策略，主要包括揹包問題、哈夫曼編碼和最小生成樹
貪心演算法顧名思義就是每次都貪心地選擇當前最好的那個(區域性最優解)，不去考慮以後的情況，而且選擇了就不能夠“反悔”了，如果原問題滿足貪心選擇性質和最優子結構，那麼最後得到的解就是最優解。貪心演算法和其他的演算法比較有明顯的區別，動態規劃每次都是綜合所有子問題的解得到當前的最優解(全域性最優解)，而不是貪心地選擇；回溯法是嘗試選擇一條路，如果選擇錯了的話可以“反悔”，也就是回過頭來重新選擇其他的試試。
這個演算法想必大家也都很熟悉了，我覺得貪心法總是比較容易想到，但是很難證明它是正確的，所有對於一類問題，條件稍有不同也許就不能使用貪心策略了。這一節採用類似上節的形式，記錄下原書中的一些重點難點內容
[果然貪心我領悟的不夠，很多問題我貌似都講不到點子上，大家將就著看下]
1.匹配問題 matching problem (maximum-weight matching problem)
問題是這樣的，有一群人打算一起跳探戈，跳之前要進行分組，一個男人和一個女人成為一組，而且任意一個異性組合都會一個相應的匹配值(compatibility)，目標是求使得匹配值之和達到最大的分組方式。
To be on the safe side, just let me emphasize that this greedy solution would not work in general, with an arbitrary set of weights. The distinct powers of two are key here.
一般情況下，如果匹配值是任意值的話，這個問題使用貪心法是不行的！但是如果匹配值都是2的整數冪的話，那麼貪心法就能解決這個問題了！[這點我不明白，這是此題的一個重點，避免誤導，我附上原文，不解釋了，如果讀者有明白了的希望能留言告知，嘿嘿]
In this case (or the bipartite case, for that matter), greed won’t work in general. However, by some freak coincidence, all the compatibility numbers happen to be distinct powers of two. Now, what happens?
Let’s first consider what a greedy algorithm would look like here and then see why it yields an optimal result. We’ll be building a solution piece by piece—let the pieces be pairs and a partial solution be a set of pairs. Such a partial solution is valid only if no person in it participates in two (or more) of its pairs. The algorithm will then be roughly as follows:
- List potential pairs, sorted by decreasing compatibility.
- Pick the first unused pair from the list.
- Is anyone in the pair already occupied? If so, discard it; otherwise, use it.
- Are there any more pairs on the list? If so, go to 2.
As you’ll see later, this is rather similar to Kruskal’s algorithm for minimum spanning trees (although that works regardless of the edge weights). It also is a rather prototypical greedy algorithm. Its correctness is another matter. Using distinct powers of two is sort of cheating, because it would make virtually any greedy algorithm work; that is, you’d get an optimal result as long as you could get a valid solution at all. Even though it’s cheating (see Exercise 7-3), it illustrates the central idea here: making the greedy choice is safe. Using the most compatible of the remaining couples will always be at least as good as any other choice.
貪心解決的思路大致如下：首先列舉出所有可能的組合，然後將它們按照匹配值進行降序排序，接著按順序從中選擇前面沒有使用過而且人物沒有在前面出現過的組合，遍歷完整個序列就得到了匹配值之和最大的分組方式。
[原書關於穩定婚姻的擴充套件知識 EAGER SUITORS AND STABLE MARRIAGES]
There is, in fact, one classical matching problem that can be solved (sort of) greedily: the stable marriage problem. The idea is that each person in a group has preferences about whom he or she would like to marry. We’d like to see everyone married, and we’d like the marriages to be stable, meaning that there is no man who prefers a woman outside his marriage who also prefers him. (To keep things simple, we disregard same-sex marriages and polygamy here.)
There’s a simple algorithm for solving this problem, designed by David Gale and Lloyd Shapley. The formulation is quite gender-conservative but will certainly also work if the gender roles are reversed. The algorithm runs for a number of rounds, until there are no unengaged men left. Each round consists of two steps:
- Each unengaged man proposes to his favorite of the women he has not yet asked.
- Each woman is (provisionally) engaged to her favorite suitor and rejects the rest.
This can be viewed as greedy in that we consider only the available favorites (both of the men and women) right now. You might object that it’s only sort of greedy in that we don’t lock in and go straight for marriage; the women are allowed to break their engagement if a more interesting suitor comes along. Even so, once a man has been rejected, he has been rejected for good, which means that we’re guaranteed progress.
To show that this is an optimal and correct algorithm, we need to know that everyone gets married and that the marriages are stable. Once a woman is engaged, she stays engaged (although she may replace her fiancé). There is no way we can get stuck with an unmarried pair, because at some point the man would have proposed to the woman, and she would have (provisionally) accepted his proposal.
How do we know the marriages are stable? Let’s say Scarlett and Stuart are both married but not to each other. Is it possible they secretly prefer each other to their current spouses? No: if so, Stuart would already have proposed to her. If she accepted that proposal, she must later have found someone she liked better; if she rejected it, she would already have a preferable mate.
Although this problem may seem silly and trivial, it is not. For example, it is used for admission to some colleges and to allocate medical students to hospital jobs. There have, in fact, been written entire books (such as those by Donald Knuth and by Dan Gusfield and Robert W. Irwing) devoted to the problem and its variations.
2.揹包問題
這個問題大家很熟悉了，而且該問題的變種很多，常見的有整數揹包和部分揹包問題。問題大致是這樣的，假設現在我們要裝一些物品到一個書包裡，每樣物品都有一定的重量w和價值v，但是呢，這個書包承重量有限，所以我們要進行決策，如何選擇物品才能使得最終的價值最大呢？整數揹包是說一個物品要麼拿要麼不拿，比如茶杯或者檯燈等等，而部分揹包問題是說一個物品你可以拿其中的一部分，比如一袋子蘋果放不下可以只裝半袋子蘋果。[更加複雜的版本是說每個物品都有一定的體積，同時書包還有體積的限制等等]
很顯然，部分揹包問題是可以用貪心法來求解的，我們計算每個物品的單位重量的價值，然後將它們降序排序，接著開始拿物品，只要裝得下全部的該類物品那麼就全裝進去，如果不能全部裝下就裝部分進去直到書包載重量滿了為止，這種策略肯定是正確的。
但是，整數揹包問題就不能用貪心策略了。整數揹包問題還可以分成兩種：一種是每類物品數量都是有限的(bounded)，比如只有3個茶杯和2個檯燈；還有一種是數量無限的(unbounded)，也就是你想要多少有多少，這兩種都不能使用貪心策略。0-1揹包問題是典型的第一種整數揹包問題，看下演算法導論上的這個例子就明白了，在(b)中，雖然物品1單位重量的價值最大，但是任何包含物品1的選擇都沒有超過選擇物品2和物品3得到的最優解220；而(c)中能達到最大的價值是240。
整數揹包問題還沒有能夠在多項式時間內解決它的演算法，下一節我們介紹的動態規劃能夠解決0-1揹包問題，但是是一個偽多項式時間複雜度。[實際時間複雜度是O(nw)，n是物品數目，w是書包載重量，嚴格意義上說這不是一個多項式時間複雜度]
There are two important cases of the integer knapsack problem—the bounded and unbounded cases. The bounded case assumes we have a fixed number of objects in each category,4 and the unbounded case lets us use as many as we want. Sadly, greed won’t work in either case. In fact, these are both unsolved problems, in the sense that no polynomial algorithms are known to solve them. There is hope, however. As you’ll see in the next chapter, we can use dynamic programming to solve the problems in pseudopolynomial time, which may be good enough in many important cases. Also, for the unbounded case, it turns out that the greedy approach ain’t half bad! Or, rather, it’s at least half good, meaning that we’ll never get less than half the optimum value. And with a slight modification, you can get as good results for the bounded version, too. This concept of greedy approximation is discussed in more detail in Chapter 11.
3.哈夫曼編碼
這個問題原始是用來實現一個可變長度的編碼問題，但可以總結成這樣一個問題，假設我們有很多的葉子節點，每個節點都有一個權值w(可以是任何有意義的數值，比如它出現的概率)，我們要用這些葉子節點構造一棵樹，那麼每個葉子節點就有一個深度d，我們的目標是使得所有葉子節點的權值與深度的乘積之和Σwidi最小。
很自然的一個想法就是，對於權值大的葉子節點我們讓它的深度小些(更加靠近根節點)，權值小的讓它的深度相對大些，這樣的話我們自然就會想著每次取當前權值最小的兩個節點將它們組合出一個父節點，一直這樣組合下去直到只有一個節點即根節點為止。如下圖所示的示例
程式碼實現比較簡單，使用了heapq模組，樹結構是用list來儲存的，有意思的是其中zip函式的使用，其中統計函式count作為zip函式的引數，詳情見python docs
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
from heapq import heapify, heappush, heappop from itertools import count def huffman(seq, frq): num = count() trees = list(zip(frq, num, seq)) # num ensures valid ordering heapify(trees) # A min-heap based on freq while len(trees) > 1: # Until all are combined fa, _, a = heappop(trees) # Get the two smallest trees fb, _, b = heappop(trees) n = next(num) heappush(trees, (fa+fb, n, [a, b])) # Combine and re-add them # print trees return trees[0][-1] seq = "abcdefghi" frq = [4, 5, 6, 9, 11, 12, 15, 16, 20] print huffman(seq, frq) # [['i', [['a', 'b'], 'e']], [['f', 'g'], [['c', 'd'], 'h']]] |
現在我們考慮另外一個問題，合併檔案問題，假設我們將大小為 m 和大小為 n 的兩個檔案合併在一起需要 m+n 的時間，現在給定一些檔案，求一個最優的合併策略使得所需要的時間最小。
如果我們將上面哈夫曼樹中的葉子節點看成是檔案，兩個檔案合併得到的大檔案就是樹中的內部節點，假設每個節點上都有一個值表示該檔案的大小，合併得到的大檔案上的值是合併的兩個檔案的值之和，那我們的目標是就是使得內部節點的和最小的合併方案，因為葉子節點的大小是固定的，所以實際上也就是使得所有節點的和最小的合併方案！
consider how each leaf contributes to the sum over all nodes: the leaf weight occurs as a summand once in each of its ancestor nodes—which means that the sum is exactly the same! That is, sum(weight(node) for node in nodes) is exactly the same as sum(depth(leaf)*weight(leaf) for leaf in leaves).
細想也就有了一個葉子節點的所有祖先節點們都有一份該葉子節點的值包含在裡面，也就是說所有葉子節點的深度與它的值的乘積之和就是所有節點的值之和！可以看下下面的示例圖，最終我們知道哈夫曼樹就是這個問題的解決方案。
[哈夫曼樹問題的一個擴充套件就是最優二叉搜尋樹問題，後者可以用動態規劃演算法來求解，感興趣的話可以閱讀演算法導論中動態規劃部分內容]
4.最小生成樹
最小生成樹是圖中的重要演算法，主要有兩個大家耳熟能詳的Kruskal和Prim演算法，兩個演算法都是基於貪心策略，不過略有不同。
[如果對最小生成樹問題的歷史感興趣的話作者推薦看這篇論文“On the History of the Minimum Spanning Tree Problem,” by Graham and Hell]
不瞭解Kruskal或者Prim演算法的童鞋可以參考演算法導論的示例圖理解下面的內容
Kruskal演算法
Prim演算法
連通無向圖G的生成樹是指包含它所有頂點但是部分邊的子圖，假設每條邊都有一個權值，那麼權值之和最小的生成樹就是最小生成樹，它不一定是唯一的。如果圖G是非連通的，那麼它就沒有生成樹。
前面我們在介紹遍歷的時候也得到過生成樹，那裡我們是一個頂點一個頂點進行遍歷，下面我們通過每次新增一條邊來得到最小生成樹，而且每次我們貪心地選擇剩下的邊中權值最小的那條邊，但是要保證不能形成環！
那怎麼判斷是否會出現環呢？
假設我們要考慮是否新增邊(u,v)，一個最直接的想法就是遍歷已生成的樹，看是否能夠從 u 到 v，如果能，那麼就捨棄這條邊繼續考慮後面的邊，否則就新增這條邊。很顯然，採用遍歷的方式太費時了。
再假設我們用一個集合來儲存我們已經生成的樹中的節點，如果我們要考慮是否新增邊(u,v)，那麼我們就看下集合中這兩個節點是否都存在，如果都存在的話說明這條邊加進來的話會形成環。這麼做可以在常數時間內確定是否會形成環，但是…它是錯誤的！除非我們每次新增一條邊之後得到的區域性解一直都只有一棵樹才對，如果之前加入的節點 u 和節點 v 在不同的分支上的話，上面的判斷不能確定新增這條邊之後會形成環！[後面的Prim演算法採用的策略就能保證區域性解一直都是一棵樹]
下面我們可以試著讓每個加入的節點都知道自己處在哪個分支上，而且我們可以用分支中的某一個節點作為該分支的“代表”，該分支中的所有節點都指向這個“代表”，顯然我們接下來會遇到分支合併的問題。如果兩個分支因為某條邊的加入而連通了，那麼它們就要合併了，那怎麼合併呢？我們讓兩個分支中的所有節點都指向同一個“代表”就行了，但是這是一個線性時間的操作，我們可以做得更快！假設我們改變下策略，讓每個節點指向另一個節點(這個節點不一定是分支的“代表”)，如果我們順著指向鏈一直找，就肯定能找到“代表”，因為“代表”是自己指向自己的。這樣的話，如果兩個分支要合併，只需要讓其中的一個分支的“代表”指向另一個分支的“代表”就行啦！這就是一個常數時間的操作。
基於上面的思路我們就有了下面的實現
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
#A Naïve Implementation of Kruskal’s Algorithm def naive_find(C, u): # Find component rep. while C[u] != u: # Rep. would point to itself u = C[u] return u def naive_union(C, u, v): u = naive_find(C, u) # Find both reps v = naive_find(C, v) C[u] = v # Make one refer to the other def naive_kruskal(G): E = [(G[u][v],u,v) for u in G for v in G[u]] T = set() # Empty partial solution C = {u:u for u in G} # Component reps for _, u, v in sorted(E): # Edges, sorted by weight if naive_find(C, u) != naive_find(C, v): T.add((u, v)) # Different reps? Use it! naive_union(C, u, v) # Combine components return T G = { 0: {1:1, 2:3, 3:4}, 1: {2:5}, 2: {3:2}, 3: set() } print list(naive_kruskal(G)) #[(0, 1), (2, 3), (0, 2)] |
從上面的分析我們可以看到，雖然合併時修改指向的操作是常數時間的，但是通過指向鏈的方式找到“代表”所花的時間是線性的，而這裡還可以做些改進。
首先，在合併(union)的時候我們讓“小”分支指向“大”分支，這樣平衡了之後平均查詢時間肯定有所下降，那麼怎麼確定分支的“大小”呢？這個可以用平衡樹的方式來思考，假設我們給每個節點都設定一個權重(rank or weight)，其實重要的還是“代表”的權重，如果要合併的兩個分支的“代表”的權重相等的話，在將“小”分支指向“大”分支之後，還要將“大”分支的權重加1。
其次，在查詢(find)的時候我們一邊查詢一邊修正經過的點的指向，讓它直接指向“代表”，這個怎麼做到呢？使用遞迴就行了，因為遞迴在找到了之後會回溯，回溯的時候就可以設定其他節點的“代表”了，這個叫做path compression技術，是Kruskal演算法常用的一個技巧。
基於上面的改進就有了下面優化的Kruskal演算法
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
#Kruskal’s Algorithm def find(C, u): if C[u] != u: C[u] = find(C, C[u]) # Path compression return C[u] def union(C, R, u, v): u, v = find(C, u), find(C, v) if R[u] > R[v]: # Union by rank C[v] = u else: C[u] = v if R[u] == R[v]: # A tie: Move v up a level R[v] += 1 def kruskal(G): E = [(G[u][v],u,v) for u in G for v in G[u]] T = set() C, R = {u:u for u in G}, {u:0 for u in G} # Comp. reps and ranks for _, u, v in sorted(E): if find(C, u) != find(C, v): T.add((u, v)) union(C, R, u, v) return T G = { 0: {1:1, 2:3, 3:4}, 1: {2:5}, 2: {3:2}, 3: set() } print list(kruskal(G)) #[(0, 1), (2, 3), (0, 2)] |
接下來就是Prim演算法了，它其實就是我們前面介紹的traversal演算法中的一種，不同點是它對待辦事項(to-do list，即前面提到的“邊緣節點”，也就是我們已經包含的這些節點能夠直接到達的那些節點)進行了一定的排序，我們在實現BFS時使用的是雙端佇列deque，此時我們只要把它改成一個優先佇列(priority queue)就行了，這裡選用heapq模組中的堆heap。
Prim演算法不斷地新增新的邊(也可以說是一個新的頂點)，一旦我們加入了一條新的邊，可能會導致某些原來的邊緣節點到生成樹的距離更加近了，所以我們要更新一下它們的距離值，然後重新調整下排序，那怎麼修改距離值呢？我們可以先找到原來的那個節點，然後再修改它的距離值接著重新調整堆，但是這麼做實在是太麻煩了！這裡有一個巧妙的技巧就是直接向堆中插入新的距離值的節點！為什麼可以呢？因為插入的新節點B的距離值比原來的節點A的距離值小，那麼Prim演算法新增頂點的時候肯定是先彈出堆中的節點B，後面如果彈出節點A的話，因為這個節點已經新增進入了，直接忽略就行了，也就是說我們這麼做不僅很簡單，而且並沒有把原來的問題搞砸了。下面是作者給出的詳細解釋，總共三點，第三點是重複的新增不會影響演算法的漸近時間複雜度
• We’re using a priority queue, so if a node has been added multiple times, by the time we remove one of its entries, it will be the one with the lowest weight (at that time), which is the one we want.
• We make sure we don’t add the same node to our traversal tree more than once. This can be ensured by a constant-time membership check. Therefore, all but one of the queue entries for any given node will be discarded.
• The multiple additions won’t affect asymptotic running time
[重新新增一次權值減小了的節點就相當於是鬆弛(或者說是隱含了鬆弛操作在裡面)，Re-adding a node with a lower weight is equivalent to a relaxation，這兩種方式是可以相互交換的，後面圖演算法中作者在實現Dijkstra演算法時使用的是relax，那其實我們還可以實現帶relex的Prim和不帶relax的Dijkstra]
根據上面的分析就有了下面的Prim演算法實現
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
from heapq import heappop, heappush def prim(G, s): P, Q = {}, [(0, None, s)] while Q: _, p, u = heappop(Q) if u in P: continue P[u] = p for v, w in G[u].items(): heappush(Q, (w, u, v)) #weight, predecessor node, node return P G = { 0: {1:1, 2:3, 3:4}, 1: {0:1, 2:5}, 2: {0:3, 1:5, 3:2}, 3: {2:2, 0:4} } print prim(G, 0) # {0: None, 1: 0, 2: 0, 3: 2} |
[擴充套件知識，另一個角度來看最小生成樹 A SLIGHTLY DIFFERENT PERSPECTIVE]
In their historical overview of minimum spanning tree algorithms, Ronald L. Graham and Pavol Hell outline three algorithms that they consider especially important and that have played a central role in the history of the problem. The first two are the algorithms that are commonly attributed to Kruskal and Prim (although the second one was originally formulated by Vojtěch Jarník in 1930), while the third is the one initially described by Borůvka. Graham and Hell succinctly explain the algorithms as follows. A partial solution is a spanning forest, consisting of a set of fragments (components, trees). Initially, each node is a fragment. In each iteration, edges are added, joining fragments, until we have a spanning tree.
Algorithm 1: Add a shortest edge that joins two different fragments.
Algorithm 2: Add a shortest edge that joins the fragment containing the root to another fragment.
Algorithm 3: For every fragment, add the shortest edge that joins it to another fragment.
For algorithm 2, the root is chosen arbitrarily at the beginning. For algorithm 3, it is assumed that all edge weights are different to ensure that no cycles can occur. As you can see, all three algorithms are based on the same fundamental fact—that the shortest edge over a cut is safe. Also, in order to implement them efficiently, you need to be able to find shortest edges, detect whether two nodes belong to the same fragment, and so forth (as explained for algorithms 1 and 2 in the main text). Still, these brief explanations can be useful as a memory aid or to get the bird’s-eye perspective on what’s going on.
5.Greed Works. But When?
還是老話題，貪心演算法真的很好，有時候也比較容易想到，但是它什麼時候是正確的呢？
針對這個問題，作者提出了些建議和方法[都比較難翻譯和理解，感興趣還是閱讀原文較好]
(1)Keeping Up with the Best
This is what Kleinberg and Tardos (in Algorithm Design) call staying ahead. The idea is to show that as you build your solution, one step at a time, the greedy algorithm will always have gotten at least as far as a hypothetical optimal algorithm would have. Once you reach the finish line, you’ve shown that greed is optimal.
(2)No Worse Than Perfect
This is a technique I used in showing the greedy choice property for Huffman’s algorithm. It involves showing that you can transform a hypothetical optimal solution to the greedy one, without reducing the quality. Kleinberg and Tardos call this an exchange argument.
(3)Staying Safe
This is where we started: to make sure a greedy algorithm is correct, we must make sure each greedy step along the way is safe. One way of doing this is the two-part approach of showing (1) the greedy choice property, that is, that a greedy choice is compatible with optimality, and (2) optimal substructure, that is, that the remaining subproblem is a smaller instance that must also be solved optimally.
[擴充套件知識：演算法導論中還介紹了貪心演算法的內在原理，也就是擬陣，貪心演算法一般都是求這個擬陣的最大獨立子集，方法就是從一個空的獨立子集開始，從一個已經經過排序的序列中依次取出一個元素，嘗試新增到獨立子集中，如果新元素加入之後的集合仍然是一個獨立子集的話那就加入進去，這樣就形成了一個更大的獨立子集，待遍歷完整個序列時我們就得到最大的獨立子集。擬陣的內容比較難，感興趣不妨閱讀下演算法導論然後證明一兩道練習題挑戰下，嘻嘻]
用Python程式碼來形容上面的過程就是
|
1 2 3 4 5 6 7 |
#貪心演算法的框架 [擬陣的思想] def greedy(E, S, w): T = [] # Emtpy, partial solution for e in sorted(E, key=w): # Greedily consider elements TT = T + [e] # Tentative solution if TT in S: T = TT # Is it valid? Use it! return T |