akka-grpc - 基於akka-http和akka-streams的scala gRPC開發工具

雪川大蟲發表於2020-08-23
RPCHTTP

 關於grpc，在前面的scalaPB討論裡已經做了詳細的介紹：google gRPC是一種全新的RPC框架，在開源前一直是google內部使用的整合工具。gRPC支援通過http/2實現protobuf格式資料交換。protobuf即protocol buffer，是google發明的一套全新的序列化傳輸協議serialization-protocol，是二進位制編碼binary-encoded的，相對java-object，XML，Json等在空間上佔有優勢，所以資料傳輸效率更高。由於gRPC支援http/2協議，可以實現雙向通訊duplex-communication，解決了獨立request/response互動模式在軟體程式設計中的諸多侷限。這是在系統整合程式設計方面相對akka-http佔優的一個亮點。protobuf格式資料可以很方便的轉換成 json格式資料，支援對外部系統的的開放協議資料交換。這也是一些人決定選擇gRPC作為大型系統微服務整合開發工具的主要原因。更重要的是：用protobuf和gRPC進行client/server互動不涉及任何http物件包括httprequest,httpresponse，很容易上手使用，而且又有在google等大公司內部的成功使用經驗，用起來會更加放心。

雖然gRPC基於http/2協議在網路通訊效率和模式上有了很大的提升，但近段時間對gRPC使用的調研主要還是衝著protobuf來的。在http/1應用中對二進位制檔案的傳輸交換有諸多限制和不便，特別是效率方面的問題。在protobuf這種序列化模式中對任何型別的資料格式都一視同仁，可以很方便的實現圖片等檔案的上傳下載。另一個原因是：http/2並不是一種普及的協議，並不適合作為一個開放資料平臺的連線協議。再就是：雖然gRPC是基於http協議上的，但對於HttpRequest的呼叫卻非常不便，需要通過interceptor來實現，不但麻煩而且有門檻。

實際上，在使用scalaPB的過程中一直在關注akka-grpc的發展，直到v1.01釋出。這是一個正式版本，相信不會在模式、風格、語法上再有大的改變，應該值得試著使用了。

對akka-http使用者來說，akka-grpc具有很大吸引（相對其它gRPC開放工具），因為它是基於akka-http的，看看下面grpc服務端的介面：

    // Bind service handler servers to localhost:8080/8081
    val binding = Http().bindAndHandleAsync(
      serviceHandlers,
      interface = host,
      port = port,
      connectionContext = HttpConnectionContext())

    // report successful binding
    binding.foreach { binding => println(s"gRPC server bound to: ${binding.localAddress}") }

    binding

上面這段程式碼不就是akka-http裡的嗎。那麼可以想象得到如果需要支援http+rpc混合模式的應用，akka-grpc將會發揮很大作用，這也是akka-http下一步的發展趨勢。

至於akka-grpc基於akka-streams的特性，我並沒有感到太大的興趣。如上所述，我們的目標是實現一種開放資料平臺的終端接入介面。akka-streams恰恰把總體方案限制在了內部系統整合模式，因為服務端客戶端就是akka-streams的兩端，是內部系統整合的場景。也許，在開發一套內部IT系統的過程中akka-grpc可以很趁手。

與scalaPB一樣，akka-grpc也是通過編譯IDL（.proto）檔案用相應的外掛（plugin）產生相關的scala類和服務函式程式碼。實際上akka-grpc產生程式碼的plugin還是採用scalaPB的外掛，這個過程已經在scalaPB系列部落格裡詳細介紹過了。

gRPC支援下面四種互動協議：

1、Unary：獨立的一對client-request/server-response，是我們常用的http互動模式

2、Server-Streaming：client發出一個request後從server端接收一串多個response

3、Client-Streaming：client向server傳送一串多個request後從server接收一個response

4、Bidirectional-Streaming：還是由client首先傳送request啟動連線，然後在這個連線上client/server雙方可以不斷互動資訊。

在akka-grpc的官網上有很好的示範例子。我在例子的基礎上增加了身份驗證使用的示範。資料型別和服務函式用IDL定義的.proto檔案內容如下：

syntax = "proto3";

//#options
option java_multiple_files = true;
//option java_package = "learn.akka.grpc";
//option java_outer_classname = "GreeterProto";

package learn.akka.grpc;
//#options

//#services
////////////////////////////////////// The greeting service definition.
service GreeterService {
  //////////////////////
  // Sends a greeting //
  ////////*****/////////
  //      HELLO       //
  ////////*****/////////
  rpc SayHello (HelloRequest) returns (HelloReply) {}

  // Comment spanning
  // on several lines
  rpc ItKeepsTalking (stream HelloRequest) returns (HelloReply) {}

  /*
   * C style comments
   */
  rpc ItKeepsReplying (HelloRequest) returns (stream HelloReply) {}

  /* C style comments
   * on several lines
   * with non-empty heading/trailing line */
  rpc StreamHellos (stream HelloRequest) returns (stream HelloReply) {}
}
service AuthService {
  rpc GetAuthToken(UCredential) returns (JWToken) {}
}
//#services

//#messages
// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings
message HelloReply {
  string message = 1;
}

message UCredential {
  string userid = 1;
  string password = 2;
}

message JWToken {
  string jwt = 1;
}
//#messages

注意：檔案裡增加了AuthService服務，函式和型別分別為：GetAuthToken, UCredential, JWToken

身份驗證需要通過HttpHeader實現，所以產生的函式樣板裡必須提供使用HttpHeader的引數。這個可以通過在built.sbt裡增加akkaGrpcCodeGeneratorSettings += "server_power_apis"就能做到。下面是在產生的sayHello函式款式基礎上的實現示範程式碼：

  override def sayHello(in: HelloRequest,header: Metadata): Future[HelloReply] = {
    if (isAuthenticated(header)) {
      println(s"sayHello to ${in.name}")
      FastFuture.successful(HelloReply(s"Hello, ${in.name}"))
    } else {
      println(s"${in.name} Unauthorized!")
      FastFuture.successful(HelloReply(s"Unauthorized! ${in.name}"))
    }
  }

header是客戶端提交的HttpRequest，如下：

   //#with-metadata
    def singleRequestReply(jwt: String): Unit = {
      sys.log.info("Performing request")
      val reply = client.sayHello().addHeader("Bearer", jwt).invoke(HelloRequest("Alice"))
      println(s"got single reply: ${Await.result(reply, 5.seconds).message}")
    }

客戶端向服務端提供了JWT header 作為身份憑證。

上面提到，雖然http/2推出已經不短時間了，但尚未得到普及性的認可。即使是低點版本的websocket，也只能在一小撮專業的應用中得到使用。所以，akka-grpc並沒有提供對OAuth2規範身份驗證的支援。在這個例子裡我們就只能進行基本的身份證明（如店號、機器號等），但身份驗證過程的安全性就不做任何加密操作了。首先，在IDL檔案裡增加對AuthService的描述，如下：

service AuthService {
  rpc GetAuthToken(UCredential) returns (JWToken) {}
}

message UCredential {
  string userid = 1;
  string password = 2;
}

message JWToken {
  string jwt = 1;
}

現在需要在系統裡提供兩套服務：authService和greeterService, 如下：

class gRPCServer(host: String, port: Int) {

  def run(system: ActorSystem[_]): Future[Http.ServerBinding] = {

    implicit val classic = system.toClassic
    implicit val ec: ExecutionContext = system.executionContext
    val greeterService: PartialFunction[HttpRequest, Future[HttpResponse]] =
      learn.akka.grpc.GreeterServicePowerApiHandler.partial(new GreeterServices(system))
    val authService: PartialFunction[HttpRequest, Future[HttpResponse]] =
      learn.akka.grpc.AuthServicePowerApiHandler.partial(new AuthServices())

    val serviceHandlers: HttpRequest => Future[HttpResponse] =
      ServiceHandler.concatOrNotFound(greeterService, authService)

    // Bind service handler servers to localhost:8080/8081
    val binding = Http().bindAndHandleAsync(
      serviceHandlers,
      interface = host,
      port = port,
      connectionContext = HttpConnectionContext())

    // report successful binding
    binding.foreach { binding => println(s"gRPC server bound to: ${binding.localAddress}") }

    binding

    //#server
  }
}

獲取身份憑證jwt示範如下：

客戶端：
    val authenticator = AuthServiceClient(clientSettings)

    val futJwt = authenticator.getAuthToken(UCredential("9013","123456"))
    val jwt = Await.result(futJwt,5.seconds).jwt
    println(s"got jwt: ${jwt}")

服務端：

class AuthServices() extends AuthServicePowerApi {
  //#service-request-reply
  override def getAuthToken(in: UCredential, header: Metadata): Future[JWToken] =
    FastFuture.successful(JWToken("jwtstring"))
}

服務端的身份驗證示範：

  override def sayHello(in: HelloRequest,header: Metadata): Future[HelloReply] = {
    if (isAuthenticated(header)) {
      println(s"sayHello to ${in.name}")
      FastFuture.successful(HelloReply(s"Hello, ${in.name}"))
    } else {
      println(s"${in.name} Unauthorized!")
      FastFuture.successful(HelloReply(s"Unauthorized! ${in.name}"))
    }
  }

  private def isAuthenticated(metadata: Metadata): Boolean = {
    metadata.getText("bearer") match {
      case Some(t) => t == "jwtstring"
      case _ => false
    }
  }

好了，下面是本次示範的原始碼：

project/plugins.sbt

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.9")
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.9.2")
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.3.15")
addSbtPlugin("com.lightbend.akka.grpc" % "sbt-akka-grpc" % "1.0.1")
addSbtPlugin("com.lightbend.sbt" % "sbt-javaagent" % "0.1.5")

build.sbt

val AkkaVersion = "2.6.8"
val AkkaPersistenceCassandraVersion = "1.0.1"
val AkkaHttpVersion = "10.1.12"
val elastic4sVersion = "7.6.0"
lazy val akkaGrpcVersion = "1.0.1"
enablePlugins(AkkaGrpcPlugin)
lazy val `learn-akka-grpc` = project
  .in(file("."))
  .settings(
    organization := "com.datatech",
    version := "1.0",
    scalaVersion := "2.13.2",
    scalacOptions in Compile ++= Seq("-deprecation", "-feature", "-unchecked", "-Xlog-reflective-calls", "-Xlint"),
    javacOptions in Compile ++= Seq("-Xlint:unchecked", "-Xlint:deprecation"),
    libraryDependencies ++= Seq(
      "com.typesafe.akka" %% "akka-actor-typed" % AkkaVersion,
      "com.typesafe.akka" %% "akka-stream" % AkkaVersion,
      "com.typesafe.akka" %% "akka-discovery" % AkkaVersion,
      "com.typesafe.akka" %% "akka-cluster-sharding-typed" % AkkaVersion,
      "com.typesafe.akka" %% "akka-persistence-typed" % AkkaVersion,
      "com.typesafe.akka" %% "akka-persistence-query" % AkkaVersion,
      "com.typesafe.akka" %% "akka-serialization-jackson" % AkkaVersion,
      "com.typesafe.akka" %% "akka-persistence-cassandra" % AkkaPersistenceCassandraVersion,
      "com.typesafe.akka" %% "akka-http" % AkkaHttpVersion,
      "com.typesafe.akka" %% "akka-http-spray-json" % AkkaHttpVersion,
      "com.typesafe.akka" %% "akka-slf4j" % AkkaVersion,
      "com.datastax.cassandra" % "cassandra-driver-core" % "3.6.0",
      "com.datastax.cassandra" % "cassandra-driver-extras" % "3.6.0",
      "com.sksamuel.elastic4s" %% "elastic4s-client-esjava" % elastic4sVersion,
      "ch.qos.logback" % "logback-classic" % "1.2.3",
      "com.pauldijou" %% "jwt-core" % "3.0.1",
      "org.json4s" %% "json4s-native" % "3.7.0-M4")
  )

enablePlugins(JavaAppPackaging)
akkaGrpcCodeGeneratorSettings += "server_power_apis"

gRPCServer.scala

package learn.akka.grpc.demo
import akka.actor.typed._
import akka.actor.typed.scaladsl._
import akka.actor.typed.scaladsl.adapter._
import akka.http.scaladsl._
import com.typesafe.config.ConfigFactory
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import scala.concurrent._
import akka.grpc.scaladsl.ServiceHandler
//#import

//#server
object gRPCServer {

  def main(args: Array[String]): Unit = {
    val hostPat = "(.*):(.*)".r

    val (host, port) = args(0) match {
      case hostPat(h, p) => (h, p)
      case _ => ("localhost", "8080")
    }

    // important to enable HTTP/2 in ActorSystem's config
    val conf = ConfigFactory.parseString("akka.http.server.preview.enable-http2 = on")
      .withFallback(ConfigFactory.defaultApplication())
    val system = ActorSystem[Nothing](Behaviors.empty, "grpc-server", conf)
    new gRPCServer(host,port.toInt).run(system)
  }
}

class gRPCServer(host: String, port: Int) {

  def run(system: ActorSystem[_]): Future[Http.ServerBinding] = {

    implicit val classic = system.toClassic
    implicit val ec: ExecutionContext = system.executionContext
    val greeterService: PartialFunction[HttpRequest, Future[HttpResponse]] =
      learn.akka.grpc.GreeterServicePowerApiHandler.partial(new GreeterServices(system))
    val authService: PartialFunction[HttpRequest, Future[HttpResponse]] =
      learn.akka.grpc.AuthServicePowerApiHandler.partial(new AuthServices())

    val serviceHandlers: HttpRequest => Future[HttpResponse] =
      ServiceHandler.concatOrNotFound(greeterService, authService)

    // Bind service handler servers to localhost:8080/8081
    val binding = Http().bindAndHandleAsync(
      serviceHandlers,
      interface = host,
      port = port,
      connectionContext = HttpConnectionContext())

    // report successful binding
    binding.foreach { binding => println(s"gRPC server bound to: ${binding.localAddress}") }

    binding

    //#server
  }
}

GreeterServices.scala

package learn.akka.grpc.demo
import akka.NotUsed
import akka.actor.typed.ActorSystem
import akka.http.scaladsl.util.FastFuture
import akka.grpc.scaladsl._
import akka.stream.scaladsl.Sink
import akka.stream.scaladsl.Source
import learn.akka.grpc._

import scala.concurrent.{ExecutionContext, Future}

//#import

//#service-request-reply
//#service-stream
class GreeterServices(system: ActorSystem[_]) extends GreeterServicePowerApi {
  private implicit val sys: ActorSystem[_] = system
  private implicit val ec: ExecutionContext = system.executionContext

  override def sayHello(in: HelloRequest,header: Metadata): Future[HelloReply] = {
    if (isAuthenticated(header)) {
      println(s"sayHello to ${in.name}")
      FastFuture.successful(HelloReply(s"Hello, ${in.name}"))
    } else {
      println(s"${in.name} Unauthorized!")
      FastFuture.successful(HelloReply(s"Unauthorized! ${in.name}"))
    }
  }

  override def itKeepsTalking(in: Source[HelloRequest, NotUsed],header: Metadata): Future[HelloReply] = {
    if (isAuthenticated(header)) {
      println(s"sayHello to in stream...")
      in.runWith(Sink.seq).map(elements => HelloReply(s"Hello, ${elements.map(_.name).mkString(", ")}"))
    } else {
      println(s"Unauthorized!")
      in.runWith(Sink.seq).map(elements => HelloReply(s"Unauthorized, ${elements.map(_.name).mkString(", ")}"))
    }
  }

  override def itKeepsReplying(in: HelloRequest,header: Metadata): Source[HelloReply, NotUsed] = {
    if (isAuthenticated(header)) {
      println(s"sayHello to ${in.name} with stream of chars...")
      Source(s"Hello, ${in.name}".toList).map(character => HelloReply(character.toString))
    } else {
      println(s"${in.name} Unauthorized!")
      Source(s"Unauthorized, ${in.name}".toList).map(character => HelloReply(character.toString))
    }
  }

  override def streamHellos(in: Source[HelloRequest, NotUsed],header: Metadata): Source[HelloReply, NotUsed] = {
    if (isAuthenticated(header)) {
      println(s"sayHello to stream...")
      in.map(request => HelloReply(s"Hello, ${request.name}"))
    } else {
      println(s"Unauthorized!")
      in.map(request => HelloReply(s"Unauthorized, ${request.name}"))
    }

  }

  private def isAuthenticated(metadata: Metadata): Boolean = {
    metadata.getText("bearer") match {
      case Some(t) => t == "jwtstring"
      case _ => false
    }
  }
}
//#service-stream
//#service-request-reply

AuthServices.scala

package learn.akka.grpc.demo
import akka.http.scaladsl.util.FastFuture
import akka.grpc.scaladsl._
import learn.akka.grpc._

import scala.concurrent.Future

class AuthServices() extends AuthServicePowerApi {
  //#service-request-reply
  override def getAuthToken(in: UCredential, header: Metadata): Future[JWToken] =
    FastFuture.successful(JWToken("jwtstring"))
}

AuthServiceClient.scala

package learn.akka.grpc.demo
import scala.concurrent.Await
import scala.concurrent.Future
import scala.concurrent.duration._
import scala.util.Try

import akka.Done
import akka.NotUsed
import akka.actor.ActorSystem
import akka.grpc.GrpcClientSettings
import akka.stream.scaladsl.Source

import learn.akka.grpc._


object AuthGreeterClient {
  def main(args: Array[String]): Unit = {
    implicit val sys = ActorSystem("grpc-client")
    implicit val ec = sys.dispatcher

    val clientSettings = GrpcClientSettings.fromConfig(GreeterService.name)
    val client = GreeterServiceClient(clientSettings)
    val authenticator = AuthServiceClient(clientSettings)

    val futJwt = authenticator.getAuthToken(UCredential("9013","123456"))
    val jwt = Await.result(futJwt,5.seconds).jwt
    println(s"got jwt: ${jwt}")

    singleRequestReply(jwt)
    streamingRequest(jwt)
    streamingReply(jwt)
    streamingRequestReply(jwt)

    sys.scheduler.scheduleAtFixedRate(1.second, 1.second)(new Runnable {
      override def run(): Unit = Try(singleRequestReply(jwt))
    })

    //#with-metadata
    def singleRequestReply(jwt: String): Unit = {
      sys.log.info("Performing request")
      val reply = client.sayHello().addHeader("Bearer", jwt).invoke(HelloRequest("Alice"))
      println(s"got single reply: ${Await.result(reply, 5.seconds).message}")
    }
    //#with-metadata

    def streamingRequest(jwt: String): Unit = {
      val requests = List("Alice", "Bob", "Peter").map(HelloRequest(_))
      val reply = client.itKeepsTalking().addHeader("Bearer", jwt).invoke(Source(requests))
      println(s"got single reply for streaming requests: ${Await.result(reply, 5.seconds).message}")
    }

    def streamingReply(jwt: String): Unit = {
      val responseStream = client.itKeepsReplying().addHeader("Bearer", jwt).invoke(HelloRequest("Alice"))
      val done: Future[Done] =
        responseStream.runForeach(reply => println(s"got streaming reply: ${reply.message}"))
      Await.ready(done, 1.minute)
    }

    def streamingRequestReply(jwt: String): Unit = {
      val requestStream: Source[HelloRequest, NotUsed] =
        Source
          .tick(100.millis, 1.second, "tick")
          .zipWithIndex
          .map { case (_, i) => i }
          .map(i => HelloRequest(s"Alice-$i"))
          .take(10)
          .mapMaterializedValue(_ => NotUsed)

      val responseStream: Source[HelloReply, NotUsed] =
        client.streamHellos().addHeader("Bearer", jwt).invoke(requestStream)
      val done: Future[Done] =
        responseStream.runForeach(reply => println(s"got streaming reply: ${reply.message}"))
      Await.ready(done, 1.minute)
    }
  }
}

 

相關文章