探險新型序列化工具MessagePack

MessagePack是最近一個比較熱門與Json做比較的序列化工具，它的優點是相比於json，序列化速度更快和序列化之後的位元組陣列更小，正如它的官網https://msgpack.org/賣的廣告所說

It’s like JSON.
but fast and small.

下面我們以三個方面來對MessagePack做一個初步的探險

一.What is MessagePack

1.以官網的闡述來表明：

MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it’s faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.
中文翻譯：
MessagePack是一種有效的二進位制序列化格式。 它允許您在多種語言（如JSON）之間交換資料。 但它更快更小。 小整數被編碼為單個位元組，典型的短字串除了字串本身外只需要一個額外的位元組。

可以看出它序列化出來能比json更小的原因是，整數會被編碼成一個位元組，另外json的字串的冒號和引號也會被以一種特殊的方式進行壓縮，下面的圖片可以看出來MessagePack壓縮之後的效果：

from:http://indiegamr.com/cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json/

2.驗證Message的序列化速度和內容是否更優於JSON

package msgpack;

import com.alibaba.fastjson.JSON;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.msgpack.jackson.dataformat.MessagePackFactory;

import java.io.IOException;

public class MsgPackMain {
    public static void main(String[] args) throws IOException {
        Person person = new Person("andrew", "24", "man");
        //fastjson
        long jsonStartTime = System.currentTimeMillis();
        String jsonStr = JSON.toJSONString(person);
        System.out.println("json花費時間:" + (System.currentTimeMillis() - jsonStartTime));
        System.out.println("json byteArr size is" + jsonStr.getBytes("UTF-8").length);

        //msgPack use jackson
        ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
        long msgPackStartTime = System.currentTimeMillis();
        byte[] msgPackByteArr = objectMapper.writeValueAsBytes(person);
        System.out.println("msgPack花費時間:" + (System.currentTimeMillis() - msgPackStartTime));
        System.out.println("msgPack byteArr size is :" + msgPackByteArr.length);
    }
}

測試結果：

json花費時間:199
json byteArr size is40
msgPack花費時間:72
msgPack byteArr size is :28

從測試結果可以看出msgPack明顯在速度和size方面都遠勝於json，而且由於有jackson的依賴庫，所以使用和替換jackson-json都非常簡單

二.How to use

下面我們來做一些簡單的demo：
引入需要依賴的maven庫:

 <dependency>
      <groupId>org.msgpack</groupId>
      <artifactId>jackson-dataformat-msgpack</artifactId>
      <version>${msgpack.version}</version>
</dependency>

jackson包裝的msgPack，使用非常簡單，下面來封裝簡單的MsgPackUtil

package msgpack;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.msgpack.jackson.dataformat.MessagePackFactory;

import java.io.IOException;
import java.util.List;
import java.util.Map;

public class MsgPackUtil {

    private static final ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());

    public static <T> T readObj(byte[] byteArr, Class<T> clazz) throws IOException {
        return objectMapper.readValue(byteArr, clazz);
    }

    public static <T> byte[] writeObj(T t) throws JsonProcessingException {
        return objectMapper.writeValueAsBytes(t);
    }

    public static <T> List<T> readList(byte[] listByteArr, TypeReference<List<T>> typeReference) throws IOException {
        return objectMapper.readValue(listByteArr, typeReference);
    }

    public static <T,V> Map<T, V> readMap(byte[] mapByteArr, TypeReference<Map<T, V>> typeReference) throws IOException {
        return objectMapper.readValue(mapByteArr, typeReference);
    }
}

demo程式碼:

private static void msgPackJackSonDemo() throws IOException {
        //Object
        Person person = new Person("andrew", "24", "man");
        byte[] personByteArr = MsgPackUtil.writeObj(person);
        System.out.println("ObjByteArr size is :" + personByteArr.length);
        //使用readValue時候，需要空的構造方法
        System.out.println(MsgPackUtil.readObj(personByteArr, Person.class));

        //list
        List<String> testList = new ArrayList<>(3);
        testList.add("hello");
        testList.add("msgPack");
        testList.add("java");
        byte[] listByteArr = MsgPackUtil.writeObj(testList);
        System.out.println("listByteArr size is :" + listByteArr.length);
        //readList只支援POJO和基本型別List
        System.out.println(JSON.toJSONString(MsgPackUtil.readList(listByteArr, new TypeReference<List<String>>() {})));

        //map
        Map<String, String> testMap = new HashMap<>(3);
        testMap.put("hello", "hello");
        testMap.put("msgPack", "msgPack");
        testMap.put("java", "java");
        byte[] mapByteArr = MsgPackUtil.writeObj(testMap);
        System.out.println("mapByteArr size is :" + mapByteArr.length);
        System.out.println(JSON.toJSONString(MsgPackUtil.readMap(mapByteArr, new TypeReference<Map<String, String>>() {})));
    }

三.使用場景

1.適用於將資料儲存在類似於mc和redis這類的nosql資料庫中，因為資料可以序列化的更小更快，那麼在網路上的傳輸消耗會減少，同時存在redis裡面的記憶體佔用也會減少。
2.適用於分散式框架裡面多機之間的網路傳輸，生產上的實現：fluentd https://github.com/fluent/fluentd
3.適用於前後端資料的互動，當前後端完全分離之後，後端可以使用MsgPack代替json以獲取更快的速度

四.引用的相關資料：

http://indiegamr.com/cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json/
https://msgpack.org/
https://github.com/msgpack/msgpack-java/blob/develop/msgpack-jackson/README.md

                                                                                written by 黃文嶽
                                                                                2018.09.02