grpc-transcode

Description

The grpc-transcode Plugin converts between HTTP and gRPC requests.

APISIX takes in an HTTP request, transcodes it and forwards it to a gRPC service, gets the response and returns it back to the client in HTTP format.

Attributes

NameTypeRequiredDefaultDescription
proto_idstring/integerTrueid of the the proto content.
servicestringTrueName of the gRPC service.
methodstringTrueMethod name of the gRPC service.
deadlinenumberFalse0Deadline for the gRPC service in ms.
pb_optionarray[string(pb_option_def)]Falseprotobuf options.

Options for pb_option

TypeValid values
enum as resultenum_as_name, enum_as_value
int64 as resultint64_as_number, int64_as_string, int64_as_hexstring
default valuesauto_default_values, no_default_values, use_default_values, use_default_metatable
hooksenable_hooks, disable_hooks

Enabling the Plugin

Before enabling the Plugin, you have to add the content of your .proto or .pb files to APISIX.

You can use the /admin/proto/id endpoint and add the contents of the file to the content field:

  1. curl http://127.0.0.1:9080/apisix/admin/proto/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
  2. {
  3. "content" : "syntax = \"proto3\";
  4. package helloworld;
  5. service Greeter {
  6. rpc SayHello (HelloRequest) returns (HelloReply) {}
  7. }
  8. message HelloRequest {
  9. string name = 1;
  10. }
  11. message HelloReply {
  12. string message = 1;
  13. }"
  14. }'

If your proto file contains imports, or if you want to combine multiple proto files, you can generate a .pb file and use it in APISIX.

For example, if we have a file called proto/helloworld.proto which imports another proto file:

  1. syntax = "proto3";
  2. package helloworld;
  3. import "proto/import.proto";
  4. ...

We first generate a .pb file from the proto files:

  1. protoc --include_imports --descriptor_set_out=proto.pb proto/helloworld.proto

The output binary file, proto.pb will contain both helloworld.proto and import.proto.

We can now use the content of proto.pb in the content field of the API request.

As the content of the proto is binary, we encode it in base64 using this Python script:

upload_pb.py

  1. #!/usr/bin/env python
  2. # coding: utf-8
  3. import base64
  4. import sys
  5. # sudo pip install requests
  6. import requests
  7. if len(sys.argv) <= 1:
  8. print("bad argument")
  9. sys.exit(1)
  10. with open(sys.argv[1], 'rb') as f:
  11. content = base64.b64encode(f.read())
  12. id = sys.argv[2]
  13. api_key = "edd1c9f034335f136f87ad84b625c8f1" # use a different API key
  14. reqParam = {
  15. "content": content,
  16. }
  17. resp = requests.put("http://127.0.0.1:9080/apisix/admin/proto/" + id, json=reqParam, headers={
  18. "X-API-KEY": api_key,
  19. })
  20. print(resp.status_code)
  21. print(resp.text)

This script will take in a .pb file and the id to create, encodes the content of the proto to base64, and calls the Admin API with this encoded content.

To run the script:

  1. chmod +x ./upload_pb.pb
  1. ./upload_pb.py proto.pb 1

Response:

  1. # 200
  2. # {"node":{"value":{"create_time":1643879753,"update_time":1643883085,"content":"CmgKEnByb3RvL2ltcG9ydC5wcm90bxIDcGtnIhoKBFVzZXISEgoEbmFtZRgBIAEoCVIEbmFtZSIeCghSZXNwb25zZRISCgRib2R5GAEgASgJUgRib2R5QglaBy4vcHJvdG9iBnByb3RvMwq9AQoPcHJvdG8vc3JjLnByb3RvEgpoZWxsb3dvcmxkGhJwcm90by9pbXBvcnQucHJvdG8iPAoHUmVxdWVzdBIdCgR1c2VyGAEgASgLMgkucGtnLlVzZXJSBHVzZXISEgoEYm9keRgCIAEoCVIEYm9keTI5CgpUZXN0SW1wb3J0EisKA1J1bhITLmhlbGxvd29ybGQuUmVxdWVzdBoNLnBrZy5SZXNwb25zZSIAQglaBy4vcHJvdG9iBnByb3RvMw=="},"key":"\/apisix\/proto\/1"},"action":"set"}

Now, we can enable the grpc-transcode Plugin to a specific Route:

  1. curl http://127.0.0.1:9080/apisix/admin/routes/111 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
  2. {
  3. "methods": ["GET"],
  4. "uri": "/grpctest",
  5. "plugins": {
  6. "grpc-transcode": {
  7. "proto_id": "1",
  8. "service": "helloworld.Greeter",
  9. "method": "SayHello"
  10. }
  11. },
  12. "upstream": {
  13. "scheme": "grpc",
  14. "type": "roundrobin",
  15. "nodes": {
  16. "127.0.0.1:50051": 1
  17. }
  18. }
  19. }'
grpc-transcode - 图1note

The Upstream service used here should be a gRPC service. Note that the scheme is set to grpc.

You can use the grpc_server_example for testing.

Example usage

Once you configured the Plugin as mentioned above, you can make a request to APISIX to get a response back from the gRPC service (through APISIX):

  1. curl -i http://127.0.0.1:9080/grpctest?name=world

Response:

  1. HTTP/1.1 200 OK
  2. Date: Fri, 16 Aug 2019 11:55:36 GMT
  3. Content-Type: application/json
  4. Transfer-Encoding: chunked
  5. Connection: keep-alive
  6. Server: APISIX web server
  7. Proxy-Connection: keep-alive
  8. {"message":"Hello world"}

You can also configure the pb_option as shown below:

  1. curl http://127.0.0.1:9080/apisix/admin/routes/23 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
  2. {
  3. "methods": ["GET"],
  4. "uri": "/zeebe/WorkflowInstanceCreate",
  5. "plugins": {
  6. "grpc-transcode": {
  7. "proto_id": "1",
  8. "service": "gateway_protocol.Gateway",
  9. "method": "CreateWorkflowInstance",
  10. "pb_option":["int64_as_string"]
  11. }
  12. },
  13. "upstream": {
  14. "scheme": "grpc",
  15. "type": "roundrobin",
  16. "nodes": {
  17. "127.0.0.1:26500": 1
  18. }
  19. }
  20. }'

Now if you check the configured Route:

  1. curl -i "http://127.0.0.1:9080/zeebe/WorkflowInstanceCreate?bpmnProcessId=order-process&version=1&variables=\{\"orderId\":\"7\",\"ordervalue\":99\}"
  1. HTTP/1.1 200 OK
  2. Date: Wed, 13 Nov 2019 03:38:27 GMT
  3. Content-Type: application/json
  4. Transfer-Encoding: chunked
  5. Connection: keep-alive
  6. grpc-encoding: identity
  7. grpc-accept-encoding: gzip
  8. Server: APISIX web server
  9. Trailer: grpc-status
  10. Trailer: grpc-message
  11. {"workflowKey":"#2251799813685260","workflowInstanceKey":"#2251799813688013","bpmnProcessId":"order-process","version":1}

Disable Plugin

To disable the grpc-transcode Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect.

  1. curl http://127.0.0.1:9080/apisix/admin/routes/111 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
  2. {
  3. "uri": "/grpctest",
  4. "plugins": {},
  5. "upstream": {
  6. "scheme": "grpc",
  7. "type": "roundrobin",
  8. "nodes": {
  9. "127.0.0.1:50051": 1
  10. }
  11. }
  12. }'