CyberRT_recorder原始碼解讀以及record解析

辰令發表於2024-05-04
原始碼

cyberRT訊息解析

 Rosbag的包 的解析工具
        即只透過bag包資料直接反序列化，不用新增訊息的descriptor相關資訊。

CyberRecord解析工具

        需要新增訊息的descriptor相關資訊，進行解析   	
    	https://zhuanlan.zhihu.com/p/499516617
    	https://github.com/daohu527/record_msg/tree/main/modules/drivers/lidar/proto	
  ###https://apollo.baidu.com/community/article/148	
  cyber_launch主要用來啟動cyber模組，
  其中一個launch檔案可以有一個或者多個module，
     每個module 包含一個dag檔案，而一個dag檔案則對應一個或者多個components。
  launch檔案中有幾個module則會啟動幾個程序，每個程序有單獨的記憶體空間，比如靜態變數等都不會共享。
   一個dag檔案中可以有一個或者多個components，
   一個components對應一個協程。協程中的靜態變數是共享的，並且全域性唯一。
   
   DAG依賴配置檔案 (例如common.dag)中配置

cyber_recorder工具

01.record 結構   
    cyber/proto/record.proto 
    cyber/proto/proto_desc.proto
    cyber/record/file/section.h 
	
02.record 讀寫
    cyber/record/file/record_file_base.h 
	cyber/record/file/record_file_reader
	cyber/record/file/record_file_writer
    cyber/record/[header_build record_base]
	cyber/record/record_reader
	cyber/record/record_viewer
	cyber/record/record_writer	
	
03.record-python讀寫
    cyber/python/cyber_py3/record.py 
    cyber/python/cyber_py3/examples/py_record.py
	
04.cyber_recorder 工具 tools  cyber_recorder  
    cyber/tools/cyber_recorder/main.cc	
    cyber/tools/cyber_recorder/recorder.cc

改寫

1.cyber_record/common.py包含了section的定義以及 header的一些引數 
 section.h
  struct Section {
     proto::SectionType type;
     int64_t size;
   };
 header_builder.h
    private:
    static const uint32_t MAJOR_VERSION_ = 1;
    static const uint32_t MINOR_VERSION_ = 0;
    static const proto::CompressType COMPRESS_TYPE_ = proto::CompressType::COMPRESS_NONE;
    static const uint64_t CHUNK_INTERVAL_ = 20 * 1000 * 1000 * 1000ULL;    // 20s
    static const uint64_t SEGMENT_INTERVAL_ = 60 * 1000 * 1000 * 1000ULL;  // 60s
    static const uint64_t CHUNK_RAW_SIZE_ = 16 * 1024 * 1024ULL;           // 16MB
    static const uint64_t SEGMENT_RAW_SIZE_ = 2048 * 1024 * 1024ULL;       // 2GB
 header_build.cc
    proto::Header header;
    header.set_major_version(MAJOR_VERSION_);
    header.set_minor_version(MINOR_VERSION_);
    header.set_compress(COMPRESS_TYPE_);
    header.set_chunk_interval(CHUNK_INTERVAL_);
    header.set_segment_interval(SEGMENT_INTERVAL_);	
    header.set_chunk_raw_size(CHUNK_RAW_SIZE_);
    header.set_segment_raw_size(SEGMENT_RAW_SIZE_);		
 GetHeader GetHeaderWithSegmentParams GetHeaderWithChunkParams
apollo是整個chunk序列化

proto

proto2
   each class is __metaclass__ = reflection.GeneratedProtocolMessageType

proto3
 Python is a little different — 
  the Python compiler generates a module with a static descriptor of each message type in your .proto, 
  which is then used with a metaclass to create the necessary Python data access class at runtime
  
  When you run the protocol buffer compiler on a .proto 
    that uses an enum, the generated code will have a corresponding enum for Java, Kotlin, or C++, 
    or a special EnumDescriptor class for Python
	  that’s used to create a set of symbolic constants with integer values 
	  in the runtime-generated class
	  
In Python, the package directive is ignored, 
  since Python modules are organized according to their location in the file system.	

In Python, it is still strongly recommended to specify the package for the .proto file, 
as otherwise it may lead to naming conflicts 
 in descriptors and make the proto not portable for other languages.	  


protobuf2
   SerializeToString(): serializes the message and returns it as a string.
     	   Note that the bytes are binary, not text; we only use the str type as a convenient container.
   ParseFromString(data)
python3
序列化的時候， python3 使用 protobuf3 的介面變了。
      encode_to_bytes  替換了 SerializeToString
      parse_from_bytes 替換了  ParseFromString 	  
說明
   ParseFromString是一種方法-它不返回任何內容，而是填充self已解析的內容
   
from google.protobuf import message_factory, descriptor_pb2, descriptor_pool   
file_desc_proto = descriptor_pb2.FileDescriptorProto()
file_desc_proto.ParseFromString(proto_desc.desc)

參考

https://apollo.baidu.com/docs/apollo/latest/recorder_8cc_source.html
 cyber_recorder報文錄製簡介 https://cloud.tencent.com/developer/article/2348431
https://protobuf.dev/programming-guides/proto3/	
https://googleapis.dev/python/protobuf/latest/google/protobuf/message.html#google.protobuf.message.Message	
https://googleapis.dev/python/protobuf/latest/google/protobuf/message.html#google.protobuf.message.Message	
 https://protobuf.dev/getting-started/pythontutorial/

相關文章