The tutorial shows how Envoy’s External authorization filter can be used with OPA as an authorization service to enforce security policies over API requests received by Envoy. The tutorial also covers examples of authoring custom policies over the HTTP request body.

Prerequisites

This tutorial requires Kubernetes 1.14 or later. To run the tutorial locally, we recommend using minikube in version v1.0+ with Kubernetes 1.14+.

Steps

1. Start Minikube

  1. minikube start

2. Create ConfigMap containing configuration for Envoy

The Envoy configuration below defines an external authorization filter envoy.ext_authz for a gRPC authorization server.

Save the configuration as envoy.yaml:

  1. static_resources:
  2. listeners:
  3. - address:
  4. socket_address:
  5. address: 0.0.0.0
  6. port_value: 8000
  7. filter_chains:
  8. - filters:
  9. - name: envoy.http_connection_manager
  10. typed_config:
  11. "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
  12. codec_type: auto
  13. stat_prefix: ingress_http
  14. route_config:
  15. name: local_route
  16. virtual_hosts:
  17. - name: backend
  18. domains:
  19. - "*"
  20. routes:
  21. - match:
  22. prefix: "/"
  23. route:
  24. cluster: service
  25. http_filters:
  26. - name: envoy.ext_authz
  27. typed_config:
  28. "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
  29. transport_api_version: V3
  30. with_request_body:
  31. max_request_bytes: 8192
  32. allow_partial_message: true
  33. failure_mode_allow: false
  34. grpc_service:
  35. google_grpc:
  36. target_uri: 127.0.0.1:9191
  37. stat_prefix: ext_authz
  38. timeout: 0.5s
  39. - name: envoy.filters.http.router
  40. clusters:
  41. - name: service
  42. connect_timeout: 0.25s
  43. type: strict_dns
  44. lb_policy: round_robin
  45. load_assignment:
  46. cluster_name: service
  47. endpoints:
  48. - lb_endpoints:
  49. - endpoint:
  50. address:
  51. socket_address:
  52. address: 127.0.0.1
  53. port_value: 8080
  54. admin:
  55. access_log_path: "/dev/null"
  56. address:
  57. socket_address:
  58. address: 0.0.0.0
  59. port_value: 8001
  60. layered_runtime:
  61. layers:
  62. - name: static_layer_0
  63. static_layer:
  64. envoy:
  65. resource_limits:
  66. listener:
  67. example_listener_name:
  68. connection_limit: 10000
  69. overload:
  70. global_downstream_max_connections: 50000

Create the ConfigMap:

  1. kubectl create configmap proxy-config --from-file envoy.yaml

3. Define a OPA policy

The following OPA policy restricts access to the /people endpoint exposed by our sample app:

  • Alice is granted a guest role and can perform a GET request to /people.
  • Bob is granted an admin role and can perform a GET and POST request to /people.

The policy also restricts an admin user, in this case bob from creating an employee with the same firstname as himself.

policy.rego

  1. package envoy.authz
  2. import input.attributes.request.http as http_request
  3. default allow = false
  4. allow {
  5. is_token_valid
  6. action_allowed
  7. }
  8. is_token_valid {
  9. token.valid
  10. now := time.now_ns() / 1000000000
  11. token.payload.nbf <= now
  12. now < token.payload.exp
  13. }
  14. action_allowed {
  15. http_request.method == "GET"
  16. token.payload.role == "guest"
  17. glob.match("/people", ["/"], http_request.path)
  18. }
  19. action_allowed {
  20. http_request.method == "GET"
  21. token.payload.role == "admin"
  22. glob.match("/people", ["/"], http_request.path)
  23. }
  24. action_allowed {
  25. http_request.method == "POST"
  26. token.payload.role == "admin"
  27. glob.match("/people", ["/"], http_request.path)
  28. lower(input.parsed_body.firstname) != base64url.decode(token.payload.sub)
  29. }
  30. token := {"valid": valid, "payload": payload} {
  31. [_, encoded] := split(http_request.headers.authorization, " ")
  32. [valid, _, payload] := io.jwt.decode_verify(encoded, {"secret": "secret"})
  33. }

Store the policy in Kubernetes as a Secret.

  1. kubectl create secret generic opa-policy --from-file policy.rego

In the next step, OPA is configured to query for the data.envoy.authz.allow decision. If the response is true the operation is allowed, otherwise the operation is denied. Sample input received by OPA is shown below:

  1. data.envoy.authz.allow
  1. {
  2. "attributes": {
  3. "request": {
  4. "http": {
  5. "method": "GET",
  6. "path": "/people",
  7. "headers": {
  8. "authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiZ3Vlc3QiLCJzdWIiOiJZV3hwWTJVPSIsIm5iZiI6MTUxNDg1MTEzOSwiZXhwIjoxNjQxMDgxNTM5fQ.K5DnnbbIOspRbpCr2IKXE9cPVatGOCBrBQobQmBmaeU"
  9. }
  10. }
  11. }
  12. }
  13. }

With the input value above, the answer is:

  1. false

An example of the complete input received by OPA can be seen here.

In typical deployments the policy would either be built into the OPA container image or it would fetched dynamically via the Bundle API. ConfigMaps are used in this tutorial for test purposes.

4. Create App Deployment with OPA and Envoy sidecars

Our deployment contains a sample Go app which provides information about employees in a company. It exposes a /people endpoint to get and create employees. More information can on the app be found here.

OPA is started with a configuration that sets the listening address of Envoy External Authorization gRPC server and specifies the name of the policy decision to query. More information on the configuration options can be found here.

Save the deployment as deployment.yaml:

  1. kind: Deployment
  2. apiVersion: apps/v1
  3. metadata:
  4. name: example-app
  5. labels:
  6. app: example-app
  7. spec:
  8. replicas: 1
  9. selector:
  10. matchLabels:
  11. app: example-app
  12. template:
  13. metadata:
  14. labels:
  15. app: example-app
  16. spec:
  17. initContainers:
  18. - name: proxy-init
  19. image: openpolicyagent/proxy_init:v5
  20. # Configure the iptables bootstrap script to redirect traffic to the
  21. # Envoy proxy on port 8000, specify that Envoy will be running as user
  22. # 1111, and that we want to exclude port 8282 from the proxy for the
  23. # OPA health checks. These values must match up with the configuration
  24. # defined below for the "envoy" and "opa" containers.
  25. args: ["-p", "8000", "-u", "1111", "-w", "8282"]
  26. securityContext:
  27. capabilities:
  28. add:
  29. - NET_ADMIN
  30. runAsNonRoot: false
  31. runAsUser: 0
  32. containers:
  33. - name: app
  34. image: openpolicyagent/demo-test-server:v1
  35. ports:
  36. - containerPort: 8080
  37. - name: envoy
  38. image: envoyproxy/envoy:v1.17.0
  39. volumeMounts:
  40. - readOnly: true
  41. mountPath: /config
  42. name: proxy-config
  43. args:
  44. - "envoy"
  45. - "--config-path"
  46. - "/config/envoy.yaml"
  47. env:
  48. - name: ENVOY_UID
  49. value: "1111"
  50. - name: opa
  51. # Note: openpolicyagent/opa:latest-envoy is created by retagging
  52. # the latest released image of OPA-Envoy.
  53. image: openpolicyagent/opa:0.29.4-envoy
  54. volumeMounts:
  55. - readOnly: true
  56. mountPath: /policy
  57. name: opa-policy
  58. args:
  59. - "run"
  60. - "--server"
  61. - "--addr=localhost:8181"
  62. - "--diagnostic-addr=0.0.0.0:8282"
  63. - "--set=plugins.envoy_ext_authz_grpc.addr=:9191"
  64. - "--set=plugins.envoy_ext_authz_grpc.path=envoy/authz/allow"
  65. - "--set=decision_logs.console=true"
  66. - "--ignore=.*"
  67. - "/policy/policy.rego"
  68. livenessProbe:
  69. httpGet:
  70. path: /health?plugins
  71. scheme: HTTP
  72. port: 8282
  73. initialDelaySeconds: 5
  74. periodSeconds: 5
  75. readinessProbe:
  76. httpGet:
  77. path: /health?plugins
  78. scheme: HTTP
  79. port: 8282
  80. initialDelaySeconds: 5
  81. periodSeconds: 5
  82. volumes:
  83. - name: proxy-config
  84. configMap:
  85. name: proxy-config
  86. - name: opa-policy
  87. secret:
  88. secretName: opa-policy
  1. kubectl apply -f deployment.yaml

Check that the Pod shows 3/3 containers READY the STATUS as Running:

  1. kubectl get pod
  2. NAME READY STATUS RESTARTS AGE
  3. example-app-67c644b9cb-bbqgh 3/3 Running 0 8s

The proxy-init container installs iptables rules to redirect all container traffic through the Envoy proxy sidecar. More information can be found here.

5. Create a Service to expose HTTP server

In a second terminal, start a minikube tunnel to allow for use of the LoadBalancer service type.

  1. minikube tunnel

In the first terminal, create a LoadBalancer service for the deployment.

  1. kubectl expose deployment example-app --type=LoadBalancer --name=example-app-service --port=8080

Check that the Service shows an EXTERNAL-IP:

  1. kubectl get service example-app-service
  2. NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  3. example-app-service LoadBalancer 10.109.64.199 10.109.64.199 8080:32170/TCP 5s

Set the SERVICE_URL environment variable to the service’s IP/port.

minikube:

  1. export SERVICE_HOST=$(kubectl get service example-app-service -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
  2. export SERVICE_URL=$SERVICE_HOST:8080
  3. echo $SERVICE_URL

minikube (example):

  1. 10.109.64.199:8080

6. Exercise the OPA policy

For convenience, we’ll want to store Alice’s and Bob’s tokens in environment variables.

  1. export ALICE_TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiZ3Vlc3QiLCJzdWIiOiJZV3hwWTJVPSIsIm5iZiI6MTUxNDg1MTEzOSwiZXhwIjoxNjQxMDgxNTM5fQ.K5DnnbbIOspRbpCr2IKXE9cPVatGOCBrBQobQmBmaeU"
  2. export BOB_TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiYWRtaW4iLCJzdWIiOiJZbTlpIiwibmJmIjoxNTE0ODUxMTM5LCJleHAiOjE2NDEwODE1Mzl9.WCxNAveAVAdRCmkpIObOTaSd0AJRECY2Ch2Qdic3kU8"

Check that Alice can get employees but cannot create one.

  1. curl -i -H "Authorization: Bearer $ALICE_TOKEN" http://$SERVICE_URL/people
  2. curl -i -H "Authorization: Bearer $ALICE_TOKEN" -d '{"firstname":"Charlie", "lastname":"OPA"}' -H "Content-Type: application/json" -X POST http://$SERVICE_URL/people

Check that Bob can get employees and also create one.

  1. curl -i -H "Authorization: Bearer $BOB_TOKEN" http://$SERVICE_URL/people
  2. curl -i -H "Authorization: Bearer $BOB_TOKEN" -d '{"firstname":"Charlie", "lastname":"Opa"}' -H "Content-Type: application/json" -X POST http://$SERVICE_URL/people

Check that Bob cannot create an employee with the same firstname as himself.

  1. curl -i -H "Authorization: Bearer $BOB_TOKEN" -d '{"firstname":"Bob", "lastname":"Rego"}' -H "Content-Type: application/json" -X POST http://$SERVICE_URL/people

Wrap Up

Congratulations for finishing the tutorial !

This tutorial showed how to use OPA as an External authorization service to enforce custom policies by leveraging Envoy’s External authorization filter.

This tutorial also showed a sample OPA policy that returns a boolean decision to indicate whether a request should be allowed or not.

Envoy’s external authorization filter allows optional response headers and body to be sent to the downstream client or upstream. An example of a rule that returns an object that not only indicates if a request is allowed or not but also provides optional response headers, body and HTTP status that can be sent to the downstream client or upstream can be seen here.