作者|白松

1、 新增類，把每個超步傳送的訊息量大小寫入Hadoop的Counter中。在org.apache.giraph.counters包下新建GiraphMessages類，來統計訊息量。

原始碼如下：

```
package org.apache.giraph.counters;

import java.util.Iterator;
import java.util.Map;

import org.apache.hadoop.mapreduce.Mapper.Context;
import com.google.common.collect.Maps;

/**
 * Hadoop Counters in group "Giraph Messages" for counting every superstep
 * message count.
 */
public class GiraphMessages extends HadoopCountersBase {
	/** Counter group name for the giraph Messages */
	public static final String GROUP_NAME = "Giraph Messages";

/** Singleton instance for everyone to use */
	private static GiraphMessages INSTANCE;

/** superstep time in msec */
	private final Map superstepMessages;

private GiraphMessages(Context context) {
		super(context, GROUP_NAME);
		superstepMessages = Maps.newHashMap();
	}

/**
	 * Instantiate with Hadoop Context.
	 * 
	 * @param context
	 *            Hadoop Context to use.
	 */
	public static void init(Context context) {
		INSTANCE = new GiraphMessages(context);
	}

/**
	 * Get singleton instance.
	 * 
	 * @return singleton GiraphTimers instance.
	 */
	public static GiraphMessages getInstance() {
		return INSTANCE;
	}

/**
	 * Get counter for superstep messages
	 * 
	 * @param superstep
	 * @return
	 */
	public GiraphHadoopCounter getSuperstepMessages(long superstep) {
		GiraphHadoopCounter counter = superstepMessages.get(superstep);
		if (counter == null) {
			String counterPrefix = "Superstep- " + superstep+" ";
			counter = getCounter(counterPrefix);
			superstepMessages.put(superstep, counter);
		}
		return counter;
	}

@Override
	public Iterator iterator() {
		return superstepMessages.values().iterator();
	}
}
```

2、在BspServiceMaster類中新增統計功能。Master在每次同步時候，會聚集每個Worker傳送的訊息量大小（求和），儲存於GlobalStats中。因此只需要在每次同步後，從GlobalStats物件中取出總的通訊量大小，然後寫入GiraphMessages中。格式為<superstep-number>，實際儲存於上步GiraphMessages類中定義的Map<long giraphhadoopcounter=""> superstepMessages 物件中。 在BspServiceMaster的構造方法中，最後面追加一行程式碼，對GiraphMessages進行初始化。

```
GiraphMessages.init(context);
```

在BspServiceMaster類的SuperstepState coordinateSuperstep()方法中，新增記錄功能。片段程式碼如下：

```
……
// If the master is halted or all the vertices voted to halt and there
// are no more messages in the system, stop the computation
GlobalStats globalStats = aggregateWorkerStats(getSuperstep());

LOG.info("D-globalStats: "+globalStats+"\n\n");
//新增下面語句。從第0個超步起開始記錄。
if(getSuperstep() != INPUT_SUPERSTEP) {
    GiraphMessages.getInstance().getSuperstepMessages(getSuperstep()).increment(globalStats.getMessageCount());
}
……
```

3、實驗結果如下：

![1.png](https://upload-images.jianshu.io/upload_images/19094728-cbb8fe7d9d47099d.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

完！</long></superstep-number>

Giraph原始碼分析（七）—— 新增訊息統計功能

相關文章