我又造了个轮子:GrpcGateway

  • 我又造了个轮子:GrpcGateway已关闭评论
  • 16 次浏览
  • A+
所属分类:.NET技术
摘要

我个人对GRPC是比较感兴趣的,最近在玩通过前端调用GRPC。通过前端调用GRPC业界有两种方式:GRPC Web和GRPC JSON转码。

我个人对GRPC是比较感兴趣的,最近在玩通过前端调用GRPC。通过前端调用GRPC业界有两种方式:GRPC Web和GRPC JSON转码。

 

GRPC Web

通过JS或者Blazor WASM调用GRPC,微软在这方面做的还是很好的,从.NET Core3.0之后就提供了两种实现GRPC Web的方式(Grpc.AspNetCore.Web与Envoy)。我在之前的一篇里也写过如何通过Blazor WASM调用GRPC Web。

GRPC JSON

通过Restful api调用一个代理服务,代理服务将数据转发到GRPC Server就是GRPC JSON。微软从.NET7开始也正式提供了GRPC JSON转码的方式。

为什么要造轮子

既然有了GRPC Web与GRPC Json,那我为啥还要再造这么一个轮子?

原因是有位同行看了如何通过Blazor WASM调用GRPC Web 这篇文章后,告诉我微信小程序目前没办法通过这种方式调用GRPC。我当时觉得很奇怪,微信小程序也属于前端,为啥不能调用GRPC呢?

GRPC Web+小程序遇到的问题

只是听说还不能确认,要自己试一试,于是我用GRPC Web的方式让小程序调用GRPC,首先需要生成GRPC JS Client代码:

protoc.exe -I=. test.proto --js_out=import_style=commonjs:.grpcjs --plugin=protoc-gen-grpc=.protoc-gen-grpc-web.exe --grpc-web_out=import_style=commonjs,mode=grpcwebtext:.grpcjs

然后将生成的代码引入小程序端,发现确实有问题,微信小程序编译后无法正常识别GRPC的namespace,会报以下错误:

proto is not defined

去查了下原因,应该是因为小程序目前不支持protobuf序列化。然后我通过一种取巧的方式手动在生成的GRPC JS中添加了proto变量

var proto = {}

再次尝试,虽然proto能找到,但是又找不到其他对象,并且最主要的是GRPC JS Client是通过proto工具生成的,每次生成手动定义proto变量也不现实。

 

GRPC Web+小程序遇到问题总结:

  1. 小程序目前不支持protobuf序列化
  2. 手动修改生成的GRPC JS Client不友好

既然小程序通过GRPC Web方式调用GRPC失败,那还有GRPC Json。

GRPC JSON+Envoy+小程序遇到的问题

我使用了Envoy来充当restful代理,调用GRPC。我在之前有一篇通过Envoy JSON代理GRPC的帖子。按这个帖子来了一遍。

计划通过docker-compose方式运行GRPC Server和Envoy代理。

既然用GRPC,那肯定用http2/http2,在docker里运行.net core必然需要证书,没有证书就自己搞一个自签证书。

openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 365 -out server.cer

openssl pkcs12 -export -in server.cer -inkey server.key -out server.pfx

证书有了,在GRPC里配置https

    builder.WebHost.ConfigureKestrel(o =>     {          o.ListenAnyIP(1111, p =>         {             p.Protocols = HttpProtocols.Http2;             p.UseHttps("/app/server.pfx", "123456");         });     });

然后就开始配置envoy

首先生成grpc proto描述符

protoc.exe -I=.  --descriptor_set_out=.test.pb --include_imports .test.proto  --proto_path=.

然后定义envoy配置文件

admin:   address:     socket_address: {address: 0.0.0.0, port_value: 9901}  static_resources:   listeners:   - name: listener1     address:       socket_address: {address: 0.0.0.0, port_value: 10000}     filter_chains:     - filters:       - name: envoy.filters.network.http_connection_manager         typed_config:           "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager           stat_prefix: grpc_json           codec_type: AUTO           route_config:             name: local_route             virtual_hosts:             - name: local_service               domains: ["*"]               routes:               - match: {prefix: "/test"}                 route:                   cluster: grpc           http_filters:           - name: envoy.filters.http.grpc_json_transcoder             typed_config:               "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder               proto_descriptor: "/etc/envoy/test.pb"               services: ["test"]               print_options:                 add_whitespace: true                 always_print_primitive_fields: true                 always_print_enums_as_ints: false                 preserve_proto_field_names: false               auto_mapping: true           - name: envoy.filters.http.router             typed_config:              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router   clusters:   - name: grpc     type: static     connect_timeout: 15s     lb_policy: ROUND_ROBIN     dns_lookup_family: V4_ONLY     typed_extension_protocol_options:       envoy.extensions.upstreams.http.v3.HttpProtocolOptions:         "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions         explicit_http_config:           http2_protocol_options: {}     load_assignment:       cluster_name: grpc       endpoints:       - lb_endpoints:         - endpoint:             address:               socket_address:                 address: 某ip                 port_value: 1111

下面就定义envoy的dockerfile,主要是信任自签证书

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.  FROM envoyproxy/envoy-dev:e834c24e061b710348ffd72016d5d1069698b4ff  COPY ["server.crt","/usr/local/share/ca-certificates/"]  RUN ["update-ca-certificates"]

最后就是定义docker-compsoe.yaml

version: '3.4'  services:   myenvoy:     image: myenvoy     container_name: myenvoy     command: "-c /etc/envoy/envoy.yaml  --log-level debug"     build:       context: .       dockerfile: GrpcServer/DockerfileEnvoy     volumes:      - "grpcpbs/:/etc/envoy/"      - "grpcpbs/logs:/logs"     ports:       - "9901:9901"       - "10000:10000"     depends_on:       - grpcserver     networks:     - mynetwork   grpcserver:     image: grpcserver     container_name: grpcserver     networks:     - mynetwork     build:       context: .       dockerfile: GrpcServer/Dockerfile     ports:       - "1111:1111"  networks:   mynetwork:

最后通过docker-compsoe up -d运行,但是postman调用的时候,envoy与grpcserver的通信连接成功了,但是数据传输时总是被 connection reset,去github上找原因也没找到。至此grpc json+envoy又失败了。

GRPC JSON+Envoy+小程序遇到问题总结:

  1. 数据传输时connection 被莫名reset

既然envoy走不通不行,那就自己造一个吧。

开始造轮子

GRPC JSON的形式,原理就是通过一个web api接收restful请求,将请求数据转发到GRPC Server。

首先创建一个web api命名为GrpcGateway,并引入proto文件,生成grpc client代码

  <ItemGroup>     <PackageReference Include="Google.Protobuf" Version="3.20.1" />     <PackageReference Include="Grpc.Net.Client" Version="2.46.0" />     <PackageReference Include="Grpc.Tools" Version="2.46.1">       <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>       <PrivateAssets>all</PrivateAssets>     </PackageReference>     <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="6.0.0" />   </ItemGroup>   <ItemGroup>     <Protobuf Include="..*.proto" GrpcServices="Client" />   </ItemGroup>

然后创建一个控制器去接受restful请求,而grpc client可采用反射来创建。

 [ApiController]     [Route("[controller]")]     public class ProcessGrpcRequestController : ControllerBase     {         private readonly ILogger<ProcessGrpcRequestController> _logger;         private readonly Func<string, ClientBase> _getGrpcClient;         public ProcessGrpcRequestController(ILogger<ProcessGrpcRequestController> logger, Func<string, ClientBase> getGrpcClient)         {             _logger = logger;             _getGrpcClient = getGrpcClient;         }          /// <summary>         /// 调用grpc         /// </summary>         /// <param name="serviceName">Grpc Service Name 从proto文件中查询</param>         /// <param name="method">Grpc Method Name 从proto文件中查询</param>         /// <returns></returns>         [HttpPost("serviceName/{serviceName}/method/{method}")]         public async Task<IActionResult> ProcessAsync(string serviceName, string method)         {             try             {                 if (string.IsNullOrEmpty(serviceName))                 {                     return BadRequest("serviceName不能为空");                 }                 if (string.IsNullOrEmpty(method))                 {                     return BadRequest("method不能为空");                 }                  using var sr = new StreamReader(Request.Body, leaveOpen: true, encoding: Encoding.UTF8);                 var paramJson = await sr.ReadToEndAsync();                 if (string.IsNullOrEmpty(paramJson))                 {                     return BadRequest("参数不能为空");                 }                 var client = _getGrpcClient(serviceName);                 if (client == null)                 {                     return NotFound();                 }                  Type t = client.GetType();                 var processMethod = t.GetMethods().Where(e => e.Name == method).FirstOrDefault();                 if (processMethod == null)                 {                     return NotFound();                 }                  var parameters = processMethod.GetParameters();                 if (parameters == null)                 {                     return NotFound();                 }                  var param = JsonConvert.DeserializeObject(paramJson, parameters[0].ParameterType);                 if (param == null)                 {                     return BadRequest("参数不能为空");                 }                 var pt = param.GetType();                 var headers = new Metadata();                  if (Request.Headers.Keys.Contains("Authorization"))                 {                     headers.Add("Authorization", Request.Headers["Authorization"]);                 }                  var result = processMethod.Invoke(client, new object[] { param, headers, null, null });                  return Ok(result);             }             catch(Exception ex) when (             ex.InnerException !=null && ex.InnerException !=null && ex.InnerException is RpcException &&              ((ex.InnerException as RpcException).StatusCode == Grpc.Core.StatusCode.Unauthenticated ||              ((ex.InnerException as RpcException).StatusCode == Grpc.Core.StatusCode.PermissionDenied)))             {                 _logger.LogError(ex, ex.ToString());                 return Unauthorized();             }             catch (Exception ex)             {                 _logger.LogError(ex, ex.Message);                 return BadRequest(ex.ToString());             }           }     }

然后注入动态反射创建grpc client的方法

            services.AddScoped(p => {                 Func<string, ClientBase> func = serviceName =>                 {                     var channel = GrpcChannel.ForAddress(grpcServerAddress);                      var parentClassName = $"{serviceName}";                      var assembly = Assembly.Load("你的dll名字");                     var parentType = assembly.GetType(parentClassName);                      var clientType=  parentType.GetNestedType($"{serviceName}Client");                     if (clientType == null)                     {                         throw new Exception($"serviceName:{serviceName}不存在");                     }                     var client = Activator.CreateInstance(clientType, new object[] { channel });                     return (ClientBase)client;                 };                 return func;             });

然后定义grpc gateway dockerfile ,最主要需要信任证书

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.  FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base WORKDIR /app EXPOSE 16666  FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build WORKDIR /src COPY ["MyGateway/MyGateway.csproj", "MyGateway/"] COPY . . WORKDIR "/src/MyGateway"  FROM build AS publish RUN dotnet publish "MyGateway.csproj" -c Release -o /app/publish  FROM base AS final COPY ["server.crt","/usr/local/share/ca-certificates/"] RUN ["update-ca-certificates"] WORKDIR /app COPY --from=publish /app/publish . ENTRYPOINT ["dotnet", "MyGateway.dll"]

 

最后通过定义docker-compose

version: '3.4'  services:   mygateway:     image: mygateway     container_name: mygateway     networks:     - mynetwork     build:       context: .       dockerfile: MyGateway/Dockerfile     ports:       - "2222:2222"   grpcserver:     image: grpcserver     container_name: grpcserver     networks:     - mynetwork     build:       context: .       dockerfile: GrpcServer/Dockerfile     ports:       - "1111:1111"  networks:   mynetwork:

通过docker-compsoe up -d 启动

通过postman调用,看到200状态码,终于成功了,最后试了下小程序也能通过这种方式调用后端GRPC了,整个人都舒服了...

我又造了个轮子:GrpcGateway