Proxy Template

This policy allows to configure low-level Envoy resources directly in those situations where Kuma-native policies do not expose the Envoy functionality we are looking for.

Please open a new issue on GitHub describing what missing functionality couldn’t be found as a Kuma-native policy and we will make sure to prioritize it in the roadmap for future versions of Kuma.

Specifically by using the ProxyTemplate policy we can provide custom definitions of:

• Listeners
• Clusters
• ClusterLoadAssignments
• RouteConfigurations

The custom definitions will either complement or replace the resources that Kuma generates automatically.

Usage

By default Kuma uses the following default ProxyTemplate resource for every data plane proxy (kuma-dp, which embeds Envoy) that is being added to a Mesh. With a custom ProxyTemplate resource it is possible to extend or replace the default Envoy configuration that Kuma provides to every data plane proxy.

The default ProxyTemplate resource that by default Kuma applies to every data plane proxy looks like:

apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  namespace: default
  name: custom-template-1
spec:
  selectors:
    - match:
        service: '*'
  conf:
    # `imports` allows us to reuse the dataplane configuration that Kuma
    # generates automatically and add more customizations on top of it
    imports:
      # `default-proxy` is a reference name for the default
      # data plane proxy configuration generated by Kuma
      - default-proxy

type: ProxyTemplate
mesh: default
name: custom-template-1
selectors:
  - match:
      service: '*'
conf:
  # `imports` allows us to reuse the dataplane configuration that Kuma
  # generates automatically and add more customizations on top of it
  imports:
    # `default-proxy` is a reference name for the default
    # data plane proxy configuration generated by Kuma
    - default-proxy

In order to customize the configuration of a particular data plane proxy (or a group of data plane proxies), we can create a ProxyTemplate resource like:

apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  namespace: default
  name: custom-template-1
spec:
  selectors:
    - match:
        service: backend
  conf:
    # `imports` allows us to reuse the dataplane configuration that Kuma
    # generates automatically and add more customizations on top of it
    imports:
      # `default-proxy` is a reference name for the default
      # data plane proxy configuration generated by Kuma
      - default-proxy
    # `resources` defines a list of raw Envoy resources that will either
    # complement or replace the auto-generated ones
    resources:
      - name: localhost:9901
        version: v1
        # `resource` is a raw Envoy resource provided either in YAML or 
        # in JSON format
        resource: |
          '@type': type.googleapis.com/envoy.api.v2.Cluster
          connectTimeout: 5s
          name: localhost:9901
          ...

We will apply the configuration with kubectl apply -f [..].

type: ProxyTemplate
mesh: default
name: custom-template-1
selectors:
  - match:
      service: backend
conf:
  # `imports` allows us to reuse the data plane configuration that Kuma
  # generates automatically and add more customizations on top of it
  imports:
    # `default-proxy` is a reference name for the default
    # data plane proxy configuration generated by Kuma
    - default-proxy
  # `resources` defines a list of raw Envoy resources that will either
  # complement or replace the auto-generated ones
  resources:
    - name: localhost:9901
      version: v1
      # `resource` is a raw Envoy resource provided either in YAML or 
      # in JSON format
      resource: |
        '@type': type.googleapis.com/envoy.api.v2.Cluster
        connectTimeout: 5s
        name: localhost:9901
        ...

We will apply the configuration with kumactl apply -f [..] or via the HTTP API.

In the examples described above, please note that:

1. The selectors object allows us to determine what data plane proxies will be targeted by the ProxyTemplate resource (accordingly to the Kuma Tags specified).
2. The imports object allows us to reuse the configuration that Kuma generates automatically so that it can be extended by our own custom configuration.
3. The resources object allow us to provide the raw Envoy resource definitions that will either complement or replace the auto-generated ones.

The only available canned configuration that can be used inside the imports section is called default-proxy.

At runtime, whenever kuma-cp generates the configuration for a given data plane proxy, it will proceed as follows:

1. Kuma will search for all the ProxyTemplates resources that have been defined in the specified Mesh.
2. Then, it will load in memory those ProxyTemplates resources whose selectors match either an inbound or a gateway definition of any data plane proxy accordingly to the Kuma Tags selected.
3. Every matching ProxyTemplate will be then ranked. The ProxyTemplate resource with the highest ranking will be used to generate the configuration for that specific data plane proxy (or proxies).
4. If the ProxyTemplate resource specifies an imports object, these resource will be generated first.
5. If a ProxyTemplate defines a resources object, their definition will be copied “as is” and they will replace any auto-generated resource with the same name.

By defining resources in a ProxyTemplate you can:

• Add new resources in addition to those auto-generated by the imports object.
• Replace resources auto-generated from the imports spefification by creating new ones with the same name.

It is not possible to patch or delete resources that have been auto-generated from the configuration specified in the imports object. This limitation may be removed in the past if enough users require this feature, please open a new issue on GitHub if you would like this limitation to be removed.

Below an example of a ProxyTemplate resource:

apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  namespace: default
  name: custom-template-1
spec:
  selectors:
    - match:
        service: backend
  conf:
    imports:
      - default-proxy
    resources:
      - name: localhost:9901
        version: v1
        resource: |
          '@type': type.googleapis.com/envoy.api.v2.Cluster
          ...
      - name: inbound:0.0.0.0:4040
        version: v1
        resource: |
          '@type': type.googleapis.com/envoy.api.v2.Listener
          ...

We will apply the configuration with kubectl apply -f [..].

type: ProxyTemplate
mesh: default
name: custom-template-1
selectors:
  - match:
      service: backend   
conf:
  imports:
    - default-proxy
  resources:
    - name: localhost:9901
      version: v1
      resource: |
        '@type': type.googleapis.com/envoy.api.v2.Cluster
        ...
    - name: inbound:0.0.0.0:4040
      version: v1
      resource: |
        '@type': type.googleapis.com/envoy.api.v2.Listener
        ...

We will apply the configuration with kumactl apply -f [..] or via the HTTP API.

Example

Here we will show a more complete example of ProxyTemplate. Let’s assume that we want to apply a new ProxyTemplate policy that will configure any matching data plane proxy (captured by the selectors specified) to proxy requests to the internal Envoy “Admin API” in addition to the default Kuma behavior:

apiVersion: kuma.io/v1alpha1
kind: ProxyTemplate
mesh: default
metadata:
  namespace: default
  name: custom-template-1
spec:
  selectors:
    - match:
        service: backend
  conf:
    imports:
      - default-proxy
    resources:
      - name: localhost:9901
        version: v1
        resource: |
          '@type': type.googleapis.com/envoy.api.v2.Cluster
          connectTimeout: 5s
          name: localhost:9901
          loadAssignment:
            clusterName: localhost:9901
            endpoints:
            - lbEndpoints:
              - endpoint:
                  address:
                    socketAddress:
                      address: 127.0.0.1
                      # port the Envoy Admin API will be available on
                      # can be configured via `--admin-port` option
                      # of `kuma-dp`
                      portValue: 9901
          type: STATIC
      - name: inbound:0.0.0.0:4040
        version: v1
        resource: |
          '@type': type.googleapis.com/envoy.api.v2.Listener
          name: inbound:0.0.0.0:4040
          address:
            socket_address:
              address: 0.0.0.0
              port_value: 4040
          filter_chains:
          - filters:
            - name: envoy.http_connection_manager
              config:
                route_config:
                  virtual_hosts:
                  - routes:
                    - match:
                        # only proxy requests to the `/stats` endpoint
                        # of the Envoy Admin API
                        prefix: /stats
                      route:
                        cluster: localhost:9901
                    domains:
                    - '*'
                    name: envoy_admin
                codec_type: AUTO
                http_filters:
                  name: envoy.router
                stat_prefix: envoy_stats

We will apply the configuration with kubectl apply -f [..].

type: ProxyTemplate
mesh: default
name: custom-template-1
selectors:
  - match:
      service: backend 
conf:
  imports:
    - default-proxy
  resources:
    - name: localhost:9901
      version: v1
      resource: |
        '@type': type.googleapis.com/envoy.api.v2.Cluster
        connectTimeout: 5s
        name: localhost:9901
        loadAssignment:
          clusterName: localhost:9901
          endpoints:
          - lbEndpoints:
            - endpoint:
                address:
                  socketAddress:
                    address: 127.0.0.1
                    # port the Envoy Admin API will be available on
                    # can be configured via `--admin-port` option
                    # of `kuma-dp`
                    portValue: 9901
        type: STATIC
    - name: inbound:0.0.0.0:4040
      version: v1
      resource: |
        '@type': type.googleapis.com/envoy.api.v2.Listener
        name: inbound:0.0.0.0:4040
        address:
          socket_address:
            address: 0.0.0.0
            port_value: 4040
        filter_chains:
        - filters:
          - name: envoy.http_connection_manager
            config:
              route_config:
                virtual_hosts:
                - routes:
                  - match:
                      # only proxy requests to the `/stats` endpoint
                      # of the Envoy Admin API
                      prefix: /stats
                    route:
                      cluster: localhost:9901
                  domains:
                  - '*'
                  name: envoy_admin
              codec_type: AUTO
              http_filters:
                name: envoy.router
              stat_prefix: envoy_stats

We will apply the configuration with kumactl apply -f [..] or via the HTTP API.