GEP-718: Rework forwardTo segment in routes

  • Issue: #718
  • Status: Standard

Motivation

Complications due to the serviceName shortcut

Within RouteForwardTo (and HTTPRouteForwardTo by extension) there exists two mechanisms to specify the upstream network endpoint where the traffic should be forwarded to: - ServiceName: the name of the Kubernetes Service - BackendRef: a LocalObjectReference to a resource, this is an extension point in the API

While working on #685, it came to light that: 1. BackendRef itself could be used to point to a Kubernetes Service 2. With GEP-709, there will be a need to introduce a new namespace field which will enable use-cases to forward traffic to a services and backends located in different namespaces

While (1) can be fixed with more validation and documentation, it only solves for the specific case. (2) requires adding two fields: serviceNamespace and BackendRef.Namespace - something we would like to avoid since the serviceName field was introduced only as a shortcut and adding more service-specific fields defeats the original purpose.

These problems are in addition to the original problem that #685 attempts to solve: clarifying port requirements when a port is required or not. We have been updating documentation to tackle such corner-cases but that continues to result in more and more elaborations. Some excerpts from our existing documentation:

  1.     // ServiceName refers to the name of the Service to mirror matched requests
  2.     // to. When specified, this takes the place of BackendRef. If both
  3.     // BackendRef and ServiceName are specified, ServiceName will be given
  4.     // precedence.
  6.     // BackendRef is a local object reference to mirror matched requests to. If
  7.     // both BackendRef and ServiceName are specified, ServiceName will be given
  8.     // precedence.
  10.     // Weight specifies the proportion of HTTP requests forwarded to the backend
  11.     // referenced by the ServiceName or BackendRef field. This is computed as
  13.     // Port specifies the destination port number to use for the
  14.     // backend referenced by the ServiceName or BackendRef field.
  15.     // If unspecified, the destination port in the request is used
  16.     // when forwarding to a backendRef or serviceName.

Simplifying RouteForwardTo

RouteForwardTo is not the best of the names. Using a noun instead of a verb would help with in-person communication and documentation. Since we are already considering dropping the special case of service as a backend, we have an opportunity to further simplify RouteForwardTo for better UX.

Proposal

  • Remove the ServiceName field from RouteForwardTo
  • Introduce Service as a default for BackendRef.Kind
  • Pull fields from RouteForwardTo.BackendRef into RouteForwardTo itself
  • Rename RouteForwardTo to BackendRef
  • Rename Route.Spec.Rules[].ForwardTo[] to Route.Spec.Rules[].BackendRefs[] in all routes
  • Apply the same to HTTP types and filters

API

The updated BackendRef (formerly RouteForwardTo) struct will be: (HTTP version has been omitted for brevity)

  1. // BackendRef defines how and where a request should be forwarded from the Gateway.
  2. type BackendRef struct {
  3.     // Group is the group of the backend resource.
  4.     //
  5.     // +kubebuilder:validation:MinLength=1
  6.     // +kubebuilder:validation:MaxLength=253
  7.     Group string `json:"group"`
  9.     // Kind is kind of the backend resource.
  10.     //
  11.     // Support: Core (Kubernetes Service)
  12.     // Support: Implementation-specific (any other resource)
  13.     //
  14.     // +optional
  15.     // +kubebuilder:validation:MinLength=1
  16.     // +kubebuilder:validation:MaxLength=253
  17.     // +kubebuilder:default="service"
  18.     Kind *string `json:"kind"`
  20.     // Name is the name of the backend resource to forward matched requests to.
  21.     //
  22.     // If the referent cannot be found, the rule is not included in the route.
  23.     // The controller should raise the "ResolvedRefs" condition on the Gateway
  24.     // with the "DegradedRoutes" reason. The gateway status for this route should
  25.     // be updated with a condition that describes the error more specifically.
  26.     //
  27.     // +kubebuilder:validation:MinLength=1
  28.     // +kubebuilder:validation:MaxLength=253
  29.     Name string `json:"name"`
  31.     // Port specifies the destination port number to use for the
  32.     // backend resource.
  33.     // This field is required when the backend is a Kubernetes Service.
  34.     //
  35.     // Support: Core
  36.     //
  37.     // +optional
  38.     Port *PortNumber `json:"port,omitempty"`
  40.     // Weight specifies the proportion of HTTP requests forwarded to the backend
  41.     // This is computed as weight/(sum of all weights in this Backends list).
  42.     // For non-zero values, there may be some epsilon from the exact proportion
  43.     // defined here depending on the precision an implementation supports. Weight
  44.     // is not a percentage and the sum of weights does not need to equal 100.
  45.     //
  46.     // If only one backend is specified and it has a weight greater than 0, 100%
  47.     // of the traffic is forwarded to that backend. If weight is set to 0, no
  48.     // traffic should be forwarded for this entry. If unspecified, weight
  49.     // defaults to 1.
  50.     //
  51.     // Support: Extended
  52.     //
  53.     // +optional
  54.     // +kubebuilder:default=1
  55.     // +kubebuilder:validation:Minimum=0
  56.     // +kubebuilder:validation:Maximum=1000000
  57.     Weight *int32 `json:"weight,omitempty"`
  58. }

A summarized diff for the changes being proposed:

  1. diff --git a/apis/v1alpha1/shared_types.go b/apis/v1alpha1/shared_types.go
  2. index 458145f..de720cd 100644
  3. --- a/apis/v1alpha1/shared_types.go
  4. +++ b/apis/v1alpha1/shared_types.go
  5. @@ -94,51 +94,39 @@ type GatewayReference struct {
  6.     Namespace string `json:"namespace"`
  7.  }
  9. -// RouteForwardTo defines how a Route should forward a request.
  10. -type RouteForwardTo struct {
  11. -   // ServiceName refers to the name of the Service to forward matched requests
  12. -   // to. When specified, this takes the place of BackendRef. If both
  13. -   // BackendRef and ServiceName are specified, ServiceName will be given
  14. -   // precedence.
  15. +// BackendRef defines how and where a request should be forwarded from the Gateway.
  16. +type BackendRef struct {
  17. +   // Name is the name of the backend resource to forward matched requests to.
  18.     //
  19.     // If the referent cannot be found, the rule is not included in the route.
  20.     // The controller should raise the "ResolvedRefs" condition on the Gateway
  21.     // with the "DegradedRoutes" reason. The gateway status for this route should
  22.     // be updated with a condition that describes the error more specifically.
  23.     //
  24. -   // The protocol to use is defined using AppProtocol field (introduced in
  25. -   // Kubernetes 1.18) in the Service resource. In the absence of the
  26. -   // AppProtocol field a `networking.x-k8s.io/app-protocol` annotation on the
  27. -   // BackendPolicy resource may be used to define the protocol. If the
  28. -   // AppProtocol field is available, this annotation should not be used. The
  29. -   // AppProtocol field, when populated, takes precedence over the annotation
  30. -   // in the BackendPolicy resource. For custom backends, it is encouraged to
  31. -   // add a semantically-equivalent field in the Custom Resource Definition.
  32. +   // +kubebuilder:validation:MinLength=1
  33. +   // +kubebuilder:validation:MaxLength=253
  34. +   Name string `json:"name"`
  35. +
  36. +   // Kind is kind of the backend resource.
  37.     //
  38. -   // Support: Core
  39. +   // Support: Core (Kubernetes Service)
  40. +   // Support: Custom (any other resource)
  41.     //
  42.     // +optional
  43. +   // +kubebuilder:validation:MinLength=1
  44.     // +kubebuilder:validation:MaxLength=253
  45. -   ServiceName *string `json:"serviceName,omitempty"`
  46. +   // +kubebuilder:default="service"
  47. +   Kind *string `json:"kind"`
  49. -   // BackendRef is a reference to a backend to forward matched requests to. If
  50. -   // both BackendRef and ServiceName are specified, ServiceName will be given
  51. -   // precedence.
  52. -   //
  53. -   // If the referent cannot be found, the rule is not included in the route.
  54. -   // The controller should raise the "ResolvedRefs" condition on the Gateway
  55. -   // with the "DegradedRoutes" reason. The gateway status for this route should
  56. -   // be updated with a condition that describes the error more specifically.
  57. +   // Group is the group of the backend resource.
  58.     //
  59. -   // Support: Custom
  60. -   //
  61. -   // +optional
  62. -   BackendRef *LocalObjectReference `json:"backendRef,omitempty"`
  63. +   // +kubebuilder:validation:MinLength=1
  64. +   // +kubebuilder:validation:MaxLength=253
  65. +   Group string `json:"group"`
  67.     // Port specifies the destination port number to use for the
  68. -   // backend referenced by the ServiceName or BackendRef field.
  69. -   // If unspecified, the destination port in the request is used
  70. -   // when forwarding to a backendRef or serviceName.
  71. +   // backend resource.
  72. +   // This field is required when the backend is a Kubernetes Service.
  73.     //
  74.     // Support: Core
  75.     //
  76. @@ -146,11 +134,10 @@ type RouteForwardTo struct {
  77.     Port *PortNumber `json:"port,omitempty"`
  79.     // Weight specifies the proportion of HTTP requests forwarded to the backend
  80. -   // referenced by the ServiceName or BackendRef field. This is computed as
  81. -   // weight/(sum of all weights in this ForwardTo list). For non-zero values,
  82. -   // there may be some epsilon from the exact proportion defined here
  83. -   // depending on the precision an implementation supports. Weight is not a
  84. -   // percentage and the sum of weights does not need to equal 100.
  85. +   // This is computed as weight/(sum of all weights in this Backends list).
  86. +   // For non-zero values, there may be some epsilon from the exact proportion
  87. +   // defined here depending on the precision an implementation supports. Weight
  88. +   // is not a percentage and the sum of weights does not need to equal 100.
  89.     //
  90.     // If only one backend is specified and it has a weight greater than 0, 100%
  91.     // of the traffic is forwarded to that backend. If weight is set to 0, no

Examples

For Kubernetes Services, an updated forwardTo section will read as follows:

  1. ...
  2. backendRefs:
  3. - name: foo-service-v1
  4.   port: 80
  5.   weight: 80
  6. - name: foo-service-canary
  7.   port: 80
  8.   weight: 20
  9. ...

Here, the kind field is omitted as it will be injected as a default.

For custom backends, the API will look like the following:

  1. ...
  2. backendRefs:
  3. - name: foo-v1
  4.   kind: server
  5.   group: networking.acme.io
  6.   port: 80
  7.   weight: 80
  8. - name: foo-v1-canary
  9.   kind: server
  10.   group: networking.acme.io
  11.   port: 80
  12.   weight: 20
  13. ...

For completeness, here is an example of HTTPRoute:

  1. apiVersion: gateway.networking.k8s.io/v1alpha2
  2. kind: HTTPRoute
  3. metadata:
  4.   name: bar-route
  5.   labels:
  6.     gateway: prod-web-gw
  7. spec:
  8.   hostnames:
  9.   - "bar.example.com"
  10.   rules:
  11.   - matches:
  12.     - headers:
  13.         type: Exact
  14.         values:
  15.           env: canary
  16.     backendRefs:
  17.     - name: foo-service-v1
  18.       port: 80
  19.       weight: 80
  20.     - name: foo-service-canary
  21.       port: 80
  22.       weight: 20

Alternatives considered

Rename serviceName to backendName

As suggested in this comment, we could buy the best of both the worlds by introducing backendName:

  1. forwardTo:
  2. - backendName: foo
  3.   port: 80
  4.     kind: <defaults to Service>

While this saves one line of YAML (backendRef:), it could be potentially violating the Naming of the reference field API convention. Most of our object references are of the form XRef.

Concerns resolved

A concern was raised around flattening of object reference fields i.e. including port and weight field alongside object reference is by k8s API conventions or not. This is not a problem and has been confirmed with API maintainers (comment).

Out of scope

N/A

References

Existing documentation: - RouteForwardTo - HTTPRouteForwardTo - #685 - Comment thread that spawned this GEP - #578