Protocol Buffer序列化Java框架-Protostuff

Silence_L發表於2021-11-29
ProtocolJava框架

瞭解Protocol Buffer

首先要知道什麼是Protocol Buffer，在程式設計過程中，當涉及資料交換時，我們往往需要將物件進行序列化然後再傳輸。常見的序列化的格式有JSON，XML等，這些格式雖然可讀性較好，但佔用的空間大小並不是最優的。基於此，Google建立了一種名叫Protocol Buffer的序列化格式，它與JSON，XML相比可讀性較差，但佔用的空間也會更小，在一些對於速度要求比較高的場景中較為常用。

Java序列化Protocol Buffer框架—ProtoStuff

Google對於Protocol Buffer提供了多種語言的實現方法：Java，C++，go和python。但我們在使用時仍然需要去編寫可讀性不高的.proto檔案，然後使用Google提供的實現方法編譯成對應的語言，這就提高了我們使用Protocol Buffer的門檻。因此ProtoStuff就誕生了，通過ProtoStuff這個框架，我們能直接將物件通過Protocol Buffer這種序列化的方式轉成對應的位元組，極大地降低了我們使用Protocol Buffer的使用成本。

例項

首先我們新建一個maven專案，然後新增ProtoStuff的依賴，其中Objenesis是一個用來例項化一個特定類的新物件的Java庫。通過該庫，我們能在不呼叫建構函式的情況下例項化一個類的物件。

<dependency>
	<groupId>com.dyuproject.protostuff</groupId>
    <artifactId>protostuff-core</artifactId>
    <version>${protostuff.version}</version>
</dependency>

<dependency>
	<groupId>com.dyuproject.protostuff</groupId>
    <artifactId>protostuff-runtime</artifactId>
    <version>${protostuff.version}</version>
</dependency>

<!-- Objenesis -->
<dependency>
	<groupId>org.objenesis</groupId>
    <artifactId>objenesis</artifactId>
    <version>${objenesis.version}</version>
</dependency>
<!-- Lombok -->
<dependency>
	<groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>${lombok.version}</version>
</dependency>

然後我們建立兩個POJO來進行序列化的測試

@Data
@Builder
public Class Goods {
    
    private Integer num;
    private String name;
    private Double price;
    
}

@Data
@Builder
public Class Repository {
    
    private String name;
    private String location;
    private List<Goods> goodsList;
    
}

再之後編寫Protocol Buffer序列化的工具類

public Class SerializationUtil {
    
    private static Map<Class<?>, Schema<?>> cacheSchema = new ConcurrentHashMap();
    private static Objenesis objenesis = new ObjenesisStd(true);
    
    /**
    * 序列化（物件 -> 位元組陣列）
    *
    */
    public static <T> byte[] serialize(T obj) {
        Class<T> cls = (Class<T>) obj.getClass();
        LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
        try {
            Schema<T> schema = getSchema(cls);
            return ProtobufIOUtil.toByteArray(obj, schema, buffer);
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        } finally {
            buffer.clear();
        }
    }
    
    /**
    * 反序列化（位元組陣列 -> 物件）
    *
    */
    public static <T> T deserilize(byte[] data, Class<T> cls) {
        try {
            T message = objenesis.newInstance(cls);
            Schema<T> schema = getSchema(cls);
            ProtobufIOUtil.mergeFrom(data, message, schema);
            return message;
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }
    
    @SuppressWarnings("unchecked")
    private static <T> Schema<T> getSchema(Class<T> cls) {
        Schema<T> schema = (Schema<T>) cacheSchema.get(cls);
        if (schema == null) {
            schema = RuntimeSchema.createFrom(cls);
            cacheSchema.put(cls, schema);
        }
        return schema;
    }
}

最後編寫測試類來對序列化工具類進行測試

public Class Test {
    public static void main(String[] args) {
        Goods phone = Goods.builder().num(10).name("phone").price(1999.99).build();
        Goods water = Goods.builder().num(100).name("water").price(1.00).build();
        Repository repository = Repository.builder().name("Taobao").location("china").goodsList(Arrays.asList(phone, water)).build();
        byte[] data = SerializationUtil.serialize(repository);
        System.out.println("序列化結果：" + Arrays.toString(data));
        Repository result = SerializationUtil.deserilize(data, Repository.class);
        System.out.println("反序列化結果：" + result);
    }
}

輸出結果：

序列化結果：[10, 6, 84, 97, 111, 98, 97, 111, 18, 5, 99, 104, 105, 110, 97, 26, 18, 8, 10, 18, 5, 112, 104, 111, 110, 101, 25, 41, 92, -113, -62, -11, 63, -97, 64, 26, 18, 8, 100, 18, 5, 119, 97, 116, 101, 114, 25, 0, 0, 0, 0, 0, 0, -16, 63]
反序列化結果：Repository(name=Taobao, location=china, goodsList=[Goods(num=10, name=phone, price=1999.99), Goods(num=100, name=water, price=1.0)])

與JSON的對比

首先匯入JSON處理的依賴，這裡我們使用jackson來對JSON進行處理

<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>${jackson.version}</version>
</dependency>

之後修改測試類

public class Test {
    public static void main(String[] args) throws IOException {
        Goods phone = Goods.builder().num(10).name("phone").price(1999.99).build();
        Goods water = Goods.builder().num(100).name("water").price(1.00).build();
        Repository repository = Repository.builder().name("Taobao").location("china").goodsList(Arrays.asList(phone, water)).build();

        byte[] protobufData = SerializationUtil.serialize(repository);
        System.out.println("ProtoBuf序列化結果：" + Arrays.toString(protobufData));
        Repository protobufResult = SerializationUtil.deserilize(protobufData, Repository.class);
        System.out.println("ProtoBuf反序列化結果：" + protobufResult);

        ObjectMapper mapper = new ObjectMapper();
        byte[] jsonData = mapper.writeValueAsBytes(repository);
        System.out.println("JSON序列化結果：" + Arrays.toString(jsonData));
        Repository jsonResult = mapper.readValue(jsonData, Repository.class);
        System.out.println("JSON序列化結果：" + jsonResult);

        System.out.println();
        System.out.println("ProtoBuf序列化後字串結果：" + new String(protobufData, StandardCharsets.UTF_8));
        System.out.println("JSON序列化後字串結果：" + new String(jsonData, StandardCharsets.UTF_8));

        System.out.println();
        System.out.println("ProtoBuf序列化長度：" + protobufData.length);
        System.out.println("JSON序列化長度：" + jsonData.length);
    }
}

輸出結果：

ProtoBuf序列化結果：[10, 6, 84, 97, 111, 98, 97, 111, 18, 5, 99, 104, 105, 110, 97, 26, 18, 8, 10, 18, 5, 112, 104, 111, 110, 101, 25, 41, 92, -113, -62, -11, 63, -97, 64, 26, 18, 8, 100, 18, 5, 119, 97, 116, 101, 114, 25, 0, 0, 0, 0, 0, 0, -16, 63]
ProtoBuf反序列化結果：Repository(name=Taobao, location=china, goodsList=[Goods(num=10, name=phone, price=1999.99), Goods(num=100, name=water, price=1.0)])
JSON序列化結果：[123, 34, 110, 97, 109, 101, 34, 58, 34, 84, 97, 111, 98, 97, 111, 34, 44, 34, 108, 111, 99, 97, 116, 105, 111, 110, 34, 58, 34, 99, 104, 105, 110, 97, 34, 44, 34, 103, 111, 111, 100, 115, 76, 105, 115, 116, 34, 58, 91, 123, 34, 110, 117, 109, 34, 58, 49, 48, 44, 34, 110, 97, 109, 101, 34, 58, 34, 112, 104, 111, 110, 101, 34, 44, 34, 112, 114, 105, 99, 101, 34, 58, 49, 57, 57, 57, 46, 57, 57, 125, 44, 123, 34, 110, 117, 109, 34, 58, 49, 48, 48, 44, 34, 110, 97, 109, 101, 34, 58, 34, 119, 97, 116, 101, 114, 34, 44, 34, 112, 114, 105, 99, 101, 34, 58, 49, 46, 48, 125, 93, 125]
JSON序列化結果：Repository(name=Taobao, location=china, goodsList=[Goods(num=10, name=phone, price=1999.99), Goods(num=100, name=water, price=1.0)])

ProtoBuf序列化後字串結果：
Taobaochina
phone)\���?�@dwater �?
JSON序列化後字串結果：{"name":"Taobao","location":"china","goodsList":[{"num":10,"name":"phone","price":1999.99},{"num":100,"name":"water","price":1.0}]}

ProtoBuf序列化長度：55
JSON序列化長度：131

從結果來看在可讀性上顯然JSON更加易讀，ProtoBuf序列化後再轉為字串甚至會亂碼，但在長度上則顯然ProtoBuf更佔優勢，JSON的長度比ProtoBuf多了一倍多。

⚠️：在使用Jackson進行JSON反序列化時我們需要對我們的POJO類新增有參和無參構造，即新增@NoArgsConstructor 和@AllArgsConstructor 這兩個註解，否則會丟擲如下異常：

Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of com.xxx.xxx.Repository (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
at [Source: (byte[])"{"name":"Taobao","location":"china","goodsList":[{"num":10,"name":"phone","price":1999.99},{"num":100,"name":"water","price":1.0}]}"; line: 1, column: 2]
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1764)
at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:400)
at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1209)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1400)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3609)
at com.silence.rpc.test.Test.main(Test.java:31)

原因是因為@Builder並不會新增無參構造，而Jackson的反序列化需要無參構造，因為在反序列化的時候，會先初始化物件，此時預設呼叫的是無參函式，然後再進行賦值，故此我們需要新增@NoArgsConstructor ，如果只新增這個註解，又會導致缺少有參構造，因此我們還需要新增@AllArgsConstructor 。

相關文章