如果您最近有學習 gRPC 如何使用的打算,那麼您第一步其實應該學習 Protocol Buffers。
廢話不多說,我們開始學習:
一、定義訊息
1、首先看一個簡單的例子:
1 syntax = "proto3"; 2 3 message SearchRequest { 4 string query = 1; 5 int32 page_number = 2; 6 int32 result_per_page = 3; 7 }
第一行 syntax 用於宣告版本,如果不宣告則預設使用版本2。
第三行 message 用於宣告訊息結構體。
第四到第六行每個欄位後面都有一個數值,用於標識訊息在二進位制格式中的位置。值從1到15採用1個位元組編碼。
值從16到2047採用兩個位元組編碼,最小的值是1,最大的值是2的29次方減1,另外19000到19999你也不可以使用,這些是保留值。
1 message SearchRequest { 2 string query = 1; 3 int32 page_number = 2; 4 int32 result_per_page = 3; 5 } 6 7 message SearchResponse { 8 ... 9 }
另外,多個訊息型別是可以定義在一個以 .proto 為字尾的檔案的,如果你想要新增註釋的話,同註釋程式碼一樣。
值得注意的是,如果您要修改訊息型別,比如刪除欄位,或者註釋欄位,那麼以後有可能有人在你刪除欄位的位置新增了一個新欄位,並使用原刪除欄位相同的數值編號。如果此時有別的服務還在使用老版本的話,那麼會導致資料被破壞。此時你可以使用 reserved 關鍵字來預佔編號或者預佔欄位名稱,如下所示:
message Foo { reserved 2, 15, 9 to 11, 40 to max; reserved "foo", "bar"; }
另外,在同一個 reserved 語句中不能同時預佔編號和欄位名稱。
2、Protocol Buffers 同樣支援列舉與結構巢狀。
enum Corpus { CORPUS_UNSPECIFIED = 0; CORPUS_UNIVERSAL = 1; CORPUS_WEB = 2; CORPUS_IMAGES = 3; CORPUS_LOCAL = 4; CORPUS_NEWS = 5; CORPUS_PRODUCTS = 6; CORPUS_VIDEO = 7; } message SearchRequest { string query = 1; int32 page_number = 2; int32 result_per_page = 3; Corpus corpus = 4; }
列舉的第一個值為0,另外也支援鍵值對型別 map<key_type, value_type> map_field = N;
3、再來看另外一個語句
message SearchResponse { repeated Result results = 1; } message Result { optional string url = 1; string title = 2; repeated string snippets = 3; }
optional 關鍵字
字面意思是可選的意思,具體protobuf裡面怎麼處理這個欄位呢,就是protobuf處理的時候另外加了一個bool的變數,用來標記這個optional欄位是否有值,傳送方在傳送的時候,如果這個欄位有值,那麼就給bool變數標記為true,否則就標記為false,接收方在收到這個欄位的同時,也會收到傳送方同時傳送的bool變數,拿著bool變數就知道這個欄位是否有值了,這就是option的意思。
repeated 關鍵字
字面意思大概是重複的意思,其實protobuf處理這個欄位的時候,也是optional欄位一樣,另外加了一個count計數變數,用於標明這個欄位有多少個,這樣傳送方傳送的時候,同時傳送了count計數變數和這個欄位的起始地址,接收方在接受到資料之後,按照count來解析對應的資料即可。
4、匯入
如果你想匯入別的 proto 檔案裡的訊息型別,同樣也可以使用 import 匯入:
import "myproject/other_protos.proto";
預設情況下,只能使用直接匯入的 .proto 檔案中的定義。然而,有時您可能需要將 .proto 檔案移動到新的位置。您可以在舊位置放置一個佔位符的.proto檔案,使用 import public 將所有匯入轉發到新位置,而不是直接移動 .proto 檔案並在一次更改中更新所有呼叫站點。
5、包
您可以向.proto檔案新增可選的包說明符,以防止協議訊息型別之間的名稱衝突。
package foo.bar; message Open { ... }
message Foo { ... foo.bar.Open open = 1; ... }
為了生成Go程式碼,可以為提供編譯後的輸出路徑。
option go_package = "example.com/project/protos/fizz";
二、定義服務
如果您想在RPC(遠端過程呼叫)系統中使用您的訊息型別,您可以在 .proto 檔案中定義RPC服務介面,protocol buffer compiler將用您選擇的語言生成服務介面程式碼。因此,如果你想用一個方法定義一個RPC服務,它接受你的SearchRequest並返回一個SearchResponse,你可以在你的 .proto 檔案中定義它,如下所示:
service SearchService {
rpc Search(SearchRequest) returns (SearchResponse);
}
三、編譯 protocol buffer
現在執行編譯器,指定源目錄(應用程式原始碼所在的目錄-如果不提供值,則使用當前目錄)和目標目錄(您希望生成的程式碼存放的目錄;通常與$SRC_DIR相同),以及你的 .proto 的路徑。在這種情況下,您將呼叫:
protoc -I=$SRC_DIR --go_out=$DST_DIR $SRC_DIR/addressbook.proto
假如您輸入以下命令進行編譯,並且 proto 檔案使用了 option go_package = "/path2";
protoc --go_out=./path1 ./test.proto
那麼輸出目錄就在當前目錄的 /path1/paht2 目錄下。
舉個例子:
syntax = "proto3"; package test; option go_package = "order/service"; message SearchRequest { string requestParam1 = 1; string requestParam2 = 2; int32 requestParam3 = 3; } message SearchResponse { int32 code = 1; string msg = 2; } service SearchService { rpc SearchOrder(SearchRequest) returns (SearchResponse); }
使用如下命令進行編譯:
protoc --go_out=./gen ./test.proto
編譯之後的檔案在 ./gen/order/service/ 目錄下,檔名為 test.pb.go:
// Code generated by protoc-gen-go. DO NOT EDIT. // versions: // protoc-gen-go v1.28.1 // protoc v4.22.0 // source: test.proto package service import ( protoreflect "google.golang.org/protobuf/reflect/protoreflect" protoimpl "google.golang.org/protobuf/runtime/protoimpl" reflect "reflect" sync "sync" ) const ( // Verify that this generated code is sufficiently up-to-date. _ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion) // Verify that runtime/protoimpl is sufficiently up-to-date. _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20) ) type SearchRequest struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields RequestParam1 string `protobuf:"bytes,1,opt,name=requestParam1,proto3" json:"requestParam1,omitempty"` RequestParam2 string `protobuf:"bytes,2,opt,name=requestParam2,proto3" json:"requestParam2,omitempty"` RequestParam3 int32 `protobuf:"varint,3,opt,name=requestParam3,proto3" json:"requestParam3,omitempty"` } func (x *SearchRequest) Reset() { *x = SearchRequest{} if protoimpl.UnsafeEnabled { mi := &file_test_proto_msgTypes[0] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } } func (x *SearchRequest) String() string { return protoimpl.X.MessageStringOf(x) } func (*SearchRequest) ProtoMessage() {} func (x *SearchRequest) ProtoReflect() protoreflect.Message { mi := &file_test_proto_msgTypes[0] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) } return ms } return mi.MessageOf(x) } // Deprecated: Use SearchRequest.ProtoReflect.Descriptor instead. func (*SearchRequest) Descriptor() ([]byte, []int) { return file_test_proto_rawDescGZIP(), []int{0} } func (x *SearchRequest) GetRequestParam1() string { if x != nil { return x.RequestParam1 } return "" } func (x *SearchRequest) GetRequestParam2() string { if x != nil { return x.RequestParam2 } return "" } func (x *SearchRequest) GetRequestParam3() int32 { if x != nil { return x.RequestParam3 } return 0 } type SearchResponse struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields Code int32 `protobuf:"varint,1,opt,name=code,proto3" json:"code,omitempty"` Msg string `protobuf:"bytes,2,opt,name=msg,proto3" json:"msg,omitempty"` } func (x *SearchResponse) Reset() { *x = SearchResponse{} if protoimpl.UnsafeEnabled { mi := &file_test_proto_msgTypes[1] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } } func (x *SearchResponse) String() string { return protoimpl.X.MessageStringOf(x) } func (*SearchResponse) ProtoMessage() {} func (x *SearchResponse) ProtoReflect() protoreflect.Message { mi := &file_test_proto_msgTypes[1] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) } return ms } return mi.MessageOf(x) } // Deprecated: Use SearchResponse.ProtoReflect.Descriptor instead. func (*SearchResponse) Descriptor() ([]byte, []int) { return file_test_proto_rawDescGZIP(), []int{1} } func (x *SearchResponse) GetCode() int32 { if x != nil { return x.Code } return 0 } func (x *SearchResponse) GetMsg() string { if x != nil { return x.Msg } return "" } var File_test_proto protoreflect.FileDescriptor var file_test_proto_rawDesc = []byte{ 0x0a, 0x0a, 0x74, 0x65, 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x04, 0x74, 0x65, 0x73, 0x74, 0x22, 0x81, 0x01, 0x0a, 0x0d, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x24, 0x0a, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x31, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x31, 0x12, 0x24, 0x0a, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x32, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x32, 0x12, 0x24, 0x0a, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x33, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x33, 0x22, 0x36, 0x0a, 0x0e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x63, 0x6f, 0x64, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x05, 0x52, 0x04, 0x63, 0x6f, 0x64, 0x65, 0x12, 0x10, 0x0a, 0x03, 0x6d, 0x73, 0x67, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6d, 0x73, 0x67, 0x32, 0x49, 0x0a, 0x0d, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x38, 0x0a, 0x0b, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x4f, 0x72, 0x64, 0x65, 0x72, 0x12, 0x13, 0x2e, 0x74, 0x65, 0x73, 0x74, 0x2e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x14, 0x2e, 0x74, 0x65, 0x73, 0x74, 0x2e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x0f, 0x5a, 0x0d, 0x6f, 0x72, 0x64, 0x65, 0x72, 0x2f, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( file_test_proto_rawDescOnce sync.Once file_test_proto_rawDescData = file_test_proto_rawDesc ) func file_test_proto_rawDescGZIP() []byte { file_test_proto_rawDescOnce.Do(func() { file_test_proto_rawDescData = protoimpl.X.CompressGZIP(file_test_proto_rawDescData) }) return file_test_proto_rawDescData } var file_test_proto_msgTypes = make([]protoimpl.MessageInfo, 2) var file_test_proto_goTypes = []interface{}{ (*SearchRequest)(nil), // 0: test.SearchRequest (*SearchResponse)(nil), // 1: test.SearchResponse } var file_test_proto_depIdxs = []int32{ 0, // 0: test.SearchService.SearchOrder:input_type -> test.SearchRequest 1, // 1: test.SearchService.SearchOrder:output_type -> test.SearchResponse 1, // [1:2] is the sub-list for method output_type 0, // [0:1] is the sub-list for method input_type 0, // [0:0] is the sub-list for extension type_name 0, // [0:0] is the sub-list for extension extendee 0, // [0:0] is the sub-list for field type_name } func init() { file_test_proto_init() } func file_test_proto_init() { if File_test_proto != nil { return } if !protoimpl.UnsafeEnabled { file_test_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*SearchRequest); i { case 0: return &v.state case 1: return &v.sizeCache case 2: return &v.unknownFields default: return nil } } file_test_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*SearchResponse); i { case 0: return &v.state case 1: return &v.sizeCache case 2: return &v.unknownFields default: return nil } } } type x struct{} out := protoimpl.TypeBuilder{ File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: file_test_proto_rawDesc, NumEnums: 0, NumMessages: 2, NumExtensions: 0, NumServices: 1, }, GoTypes: file_test_proto_goTypes, DependencyIndexes: file_test_proto_depIdxs, MessageInfos: file_test_proto_msgTypes, }.Build() File_test_proto = out.File file_test_proto_rawDesc = nil file_test_proto_goTypes = nil file_test_proto_depIdxs = nil }
如果你要使用的訊息結構的話,可以看如下示例:
package main import ( "fmt" "github.com/golang/protobuf/proto" pb "study.com/study-user/api/proto/gen/order/service" ) func main() { request := &pb.SearchRequest{ RequestParam1: "aaa", RequestParam2: "bbb", RequestParam3: 0, } fmt.Printf("編碼前: %v \n", request) marshal, err := proto.Marshal(request) fmt.Printf("編碼後: %v \n", marshal) if err == nil { newRequest := &pb.SearchRequest{} proto.Unmarshal(marshal, newRequest) fmt.Printf("解碼後: %v \n", newRequest) } }
細心的你可能發現,我沒有呼叫函式,畢竟也沒有定義函式,你就算用 pb. 後面也找不到函式名,可以先看看gRPC,這裡會告訴你答案。
四、gRPC介紹
當你來到gRPC官網Go語言時,他會要求你提前準備如下:
1、安裝 Go
3、安裝編譯外掛
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28 $ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2
4、配置 Protocol Buffer 環境變數
$ export PATH="$PATH:$(go env GOPATH)/bin"
5、下載測試用例
git clone -b v1.53.0 --depth 1 https://github.com/grpc/grpc-go
6、執行示例
go run greeter_server/main.go
go run greeter_client/main.go
6、生成 gRPC程式碼
$ protoc --go_out=. --go_opt=paths=source_relative \ --go-grpc_out=. --go-grpc_opt=paths=source_relative \ helloworld/helloworld.proto
這個我們拿之前的例子來舉例,go_opt 選項你可以在原始檔中使用 option go_package 來替代。
protoc --go_out=./gen --go-grpc_out=./gen ./test.proto
這時可以看到生成了兩個檔案:test.pb.go、test_grpc.pb.go,我們來看一下生成的 grpc 程式碼:
// Code generated by protoc-gen-go-grpc. DO NOT EDIT. // versions: // - protoc-gen-go-grpc v1.2.0 // - protoc v4.22.0 // source: test.proto package service import ( context "context" grpc "google.golang.org/grpc" codes "google.golang.org/grpc/codes" status "google.golang.org/grpc/status" ) // This is a compile-time assertion to ensure that this generated file // is compatible with the grpc package it is being compiled against. // Requires gRPC-Go v1.32.0 or later. const _ = grpc.SupportPackageIsVersion7 // SearchServiceClient is the client API for SearchService service. // // For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream. type SearchServiceClient interface { SearchOrder(ctx context.Context, in *SearchRequest, opts ...grpc.CallOption) (*SearchResponse, error) } type searchServiceClient struct { cc grpc.ClientConnInterface } func NewSearchServiceClient(cc grpc.ClientConnInterface) SearchServiceClient { return &searchServiceClient{cc} } func (c *searchServiceClient) SearchOrder(ctx context.Context, in *SearchRequest, opts ...grpc.CallOption) (*SearchResponse, error) { out := new(SearchResponse) err := c.cc.Invoke(ctx, "/test.SearchService/SearchOrder", in, out, opts...) if err != nil { return nil, err } return out, nil } // SearchServiceServer is the server API for SearchService service. // All implementations must embed UnimplementedSearchServiceServer // for forward compatibility type SearchServiceServer interface { SearchOrder(context.Context, *SearchRequest) (*SearchResponse, error) mustEmbedUnimplementedSearchServiceServer() } // UnimplementedSearchServiceServer must be embedded to have forward compatible implementations. type UnimplementedSearchServiceServer struct { } func (UnimplementedSearchServiceServer) SearchOrder(context.Context, *SearchRequest) (*SearchResponse, error) { return nil, status.Errorf(codes.Unimplemented, "method SearchOrder not implemented") } func (UnimplementedSearchServiceServer) mustEmbedUnimplementedSearchServiceServer() {} // UnsafeSearchServiceServer may be embedded to opt out of forward compatibility for this service. // Use of this interface is not recommended, as added methods to SearchServiceServer will // result in compilation errors. type UnsafeSearchServiceServer interface { mustEmbedUnimplementedSearchServiceServer() } func RegisterSearchServiceServer(s grpc.ServiceRegistrar, srv SearchServiceServer) { s.RegisterService(&SearchService_ServiceDesc, srv) } func _SearchService_SearchOrder_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(SearchRequest) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(SearchServiceServer).SearchOrder(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/test.SearchService/SearchOrder", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(SearchServiceServer).SearchOrder(ctx, req.(*SearchRequest)) } return interceptor(ctx, in, info, handler) } // SearchService_ServiceDesc is the grpc.ServiceDesc for SearchService service. // It's only intended for direct use with grpc.RegisterService, // and not to be introspected or modified (even as a copy) var SearchService_ServiceDesc = grpc.ServiceDesc{ ServiceName: "test.SearchService", HandlerType: (*SearchServiceServer)(nil), Methods: []grpc.MethodDesc{ { MethodName: "SearchOrder", Handler: _SearchService_SearchOrder_Handler, }, }, Streams: []grpc.StreamDesc{}, Metadata: "test.proto", }
我們重點關注如下:
// SearchServiceServer is the server API for SearchService service. // All implementations must embed UnimplementedSearchServiceServer // for forward compatibility type SearchServiceServer interface { SearchOrder(context.Context, *SearchRequest) (*SearchResponse, error) mustEmbedUnimplementedSearchServiceServer() }
SearchServiceServer 是 SearchService 服務的 api,所有的實現必須組合 mustEmbedUnimplementedSearchServiceServer 為了向前相容。
7、官方helloworld使用示例
proto 檔案:
syntax = "proto3"; option go_package = "google.golang.org/grpc/examples/helloworld/helloworld"; option java_multiple_files = true; option java_package = "io.grpc.examples.helloworld"; option java_outer_classname = "HelloWorldProto"; package helloworld; // The greeting service definition. service Greeter { // Sends a greeting rpc SayHello (HelloRequest) returns (HelloReply) {} } // The request message containing the user's name. message HelloRequest { string name = 1; } // The response message containing the greetings message HelloReply { string message = 1; }
服務端:
package main import ( "context" "flag" "fmt" "log" "net" "google.golang.org/grpc" pb "google.golang.org/grpc/examples/helloworld/helloworld" ) var ( port = flag.Int("port", 50051, "The server port") ) // server is used to implement helloworld.GreeterServer. type server struct { pb.UnimplementedGreeterServer } // SayHello implements helloworld.GreeterServer func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) { log.Printf("Received: %v", in.GetName()) return &pb.HelloReply{Message: "Hello " + in.GetName()}, nil } func main() { flag.Parse() lis, err := net.Listen("tcp", fmt.Sprintf(":%d", *port)) if err != nil { log.Fatalf("failed to listen: %v", err) } s := grpc.NewServer() pb.RegisterGreeterServer(s, &server{}) log.Printf("server listening at %v", lis.Addr()) if err := s.Serve(lis); err != nil { log.Fatalf("failed to serve: %v", err) } }
官方使用 server 組合了 pb.UnimplementedGreeterServer,然後 server 實現了 SayHello 的方法。
在 main 方法裡對某個埠號進行監聽,並註冊了 SayHello 服務,因為只有一個 rpc 方法,然後服務端進行服務。
客戶端:
package main import ( "context" "flag" "log" "time" "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" pb "google.golang.org/grpc/examples/helloworld/helloworld" ) const ( defaultName = "world" ) var ( addr = flag.String("addr", "localhost:50051", "the address to connect to") name = flag.String("name", defaultName, "Name to greet") ) func main() { flag.Parse() // Set up a connection to the server. conn, err := grpc.Dial(*addr, grpc.WithTransportCredentials(insecure.NewCredentials())) if err != nil { log.Fatalf("did not connect: %v", err) } defer conn.Close() c := pb.NewGreeterClient(conn) // Contact the server and print out its response. ctx, cancel := context.WithTimeout(context.Background(), time.Second) defer cancel() r, err := c.SayHello(ctx, &pb.HelloRequest{Name: *name}) if err != nil { log.Fatalf("could not greet: %v", err) } log.Printf("Greeting: %s", r.GetMessage()) }
客戶端連線服務端gRPC服務地址,呼叫服務端的 SayHello 方法,拿到結果後返回。
如果想要詳細瞭解的話,可以看看 gRPC 官方文件哈,如果有幫助可以幫忙點個贊哈!