GEP-709: Cross Namespace References from Routes

  • Issue: #709
  • Status: Standard

Note

This resource was originally named “ReferencePolicy”. It was renamed to “ReferenceGrant” to avoid any confusion with policy attachment.

TLDR

This GEP attempts to enable cross namespace forwarding from Routes and provide a way to simplify adding Route inclusion (Routes including other Routes) in the future. These are closely related concepts that can be solved with a new ReferenceGrant resource that enables app admins to describe where they trust references from.

Motivation/User Journeys/Background

This GEP keeps same namespace references simple while enabling the following capabilities for cross namespace references:

  1. Retaining full control of Gateway and Routes in an infra namespace, while targeting apps in different namespaces.
  2. Traffic splitting between Services in different namespaces.
  3. Mesh overrides to target Services in different namespaces. (For more info, see GEP #713)

ReferenceGrant

Anytime we allow crossing a namespace boundary, we need to be very cautious. In the past, we’ve seen that forwarding traffic across namespace boundaries is a desired feature, but without the kinds of safeguards proposed here, vulnerabilities can emerge.

To ensure that Gateway API is able to safely provide this functionality, we need to enforce a handshake mechanism that requires resources in both namespaces to agree to this reference. To accomplish that, a new ReferenceGrant resource should be introduced.

Reference Policy

With this model, Routes would be able to directly reference Routes and Services in other namespaces. These references would only be considered valid if a ReferenceGrant in the target namespace explicitly allowed it.

The following example shows how a HTTPRoute in namespace foo could reference a Service in namespace bar. In this example a ReferenceGrant in the bar namespace explicitly allows references to Services from HTTPRoutes in the foo namespace.

  1. kind: HTTPRoute
  2. metadata:
  3.   name: foo
  4.   namespace: foo
  5. spec:
  6.   rules:
  7.   - matches:
  8.     - path: /bar
  9.     forwardTo:
  10.       backend:
  11.       - name: bar
  12.         namespace: bar
  13. ---
  14. kind: ReferenceGrant
  15. metadata:
  16.   name: bar
  17.   namespace: bar
  18. spec:
  19.   from:
  20.   - group: networking.gateway.k8s.io
  21.     kind: HTTPRoute
  22.     namespace: foo
  23.   to:
  24.   - group: core
  25.     kind: Service

API

This proposed API is fairly straightforward, but comes with a few notable decisions:

  1. Each ReferenceGrant only supports a single From and To section. Additional trust relationships can be modeled with additional ReferenceGrant resources.
  2. Resource names are intentionally excluded from this policy for simplicity and because they rarely provide any meaningful protection. A user that is able to write to resources of a certain kind within a namespace can always rename resources or change the structure of the resources to match a given policy.
  3. A single Namespace is allowed per “From” struct. Although a selector would be more powerful it may encourage unnecessarily insecure configuration.
  1. // ReferenceGrant identifies kinds of resources in other namespaces that are
  2. // trusted to reference the specified kinds of resources in the local namespace.
  3. // Each ReferenceGrant can be used to represent a unique trust relationship.
  4. // Additional ReferenceGrants can be used to add to the set of trusted
  5. // sources of inbound references for the namespace they are defined within.
  6. type ReferenceGrant struct {
  7.     metav1.TypeMeta   `json:",inline"`
  8.     metav1.ObjectMeta `json:"metadata,omitempty"`
  10.     // Spec defines the desired state of ReferenceGrant.
  11.     Spec ReferenceGrantSpec `json:"spec,omitempty"`
  12. }
  15. // ReferenceGrantSpec identifies a cross namespace relationship that is trusted
  16. // for Gateway API.
  17. type ReferenceGrantSpec struct {
  18.     // From describes the trusted namespaces and kinds that can reference the
  19.     // resources described in "To". Each entry in this list must be considered
  20.     // to be an additional place that references can be valid from, or to put
  21.     // this another way, entries must be combined using OR.
  22.     //
  23.     // Support: Core
  24.     //
  25.     // +kubebuilder:validation:MinItems=1
  26.     From []ReferenceGrantFrom `json:"from"`
  28.     // To describes the resources that may be referenced by the resources
  29.     // described in "From". Each entry in this list must be considered to be an
  30.     // additional place that references can be valid to, or to put this another
  31.     // way, entries must be combined using OR.
  32.     //
  33.     // Support: Core
  34.     //
  35.     // +kubebuilder:validation:MinItems=1
  36.     To []ReferenceGrantTo `json:"to"`
  37. }
  39. // ReferenceGrantFrom describes trusted namespaces and kinds.
  40. type ReferenceGrantFrom struct {
  41.     // Group is the group of the referrent.
  42.     //
  43.     // Support: Core
  44.     //
  45.     // +kubebuilder:validation:MinLength=1
  46.     // +kubebuilder:validation:MaxLength=253
  47.     Group string `json:"group"`
  49.     // Kind is the kind of the referrent. Although implementations may support
  50.     // additional resources, the following Route types are part of the "Core"
  51.     // support level for this field:
  52.     //
  53.     // * HTTPRoute
  54.     // * TCPRoute
  55.     // * TLSRoute
  56.     // * UDPRoute
  57.     //
  58.     // +kubebuilder:validation:MinLength=1
  59.     // +kubebuilder:validation:MaxLength=253
  60.     Kind string `json:"kind"`
  62.     // Namespace is the namespace of the referrent.
  63.     //
  64.     // Support: Core
  65.     //
  66.     // +kubebuilder:validation:MinLength=1
  67.     // +kubebuilder:validation:MaxLength=253
  68.     Namespace string `json:"namespace,omitempty"`
  69. }
  71. // ReferenceGrantTo describes what Kinds are allowed as targets of the
  72. // references.
  73. type ReferenceGrantTo struct {
  74.     // Group is the group of the referrent.
  75.     //
  76.     // Support: Core
  77.     //
  78.     // +kubebuilder:validation:MinLength=1
  79.     // +kubebuilder:validation:MaxLength=253
  80.     Group string `json:"group"`
  82.     // Kind is the kind of the referrent. Although implementations may support
  83.     // additional resources, the following types are part of the "Core"
  84.     // support level for this field:
  85.     //
  86.     // * Service
  87.     // * HTTPRoute
  88.     // * TCPRoute
  89.     // * TLSRoute
  90.     // * UDPRoute
  91.     //
  92.     // +kubebuilder:validation:MinLength=1
  93.     // +kubebuilder:validation:MaxLength=253
  94.     Kind string `json:"kind"`
  95. }

Benefits

  • Conceptually similar to NetworkPolicy.
  • A separate resource enables admins to restrict who can allow cross namespace references.
  • Provides consistent way to control references to any resource from a Route.
  • Can be extended in the future for additional use cases.
  • A single ReferenceGrant resource can be used for a namespace in place of separate handshake config on each Service or Route resource.

Exceptions

There are some situations where it MAY be acceptable to ignore ReferenceGrant in favor of some other security mechanism. This MAY only be done if other mechanisms like NetworkPolicy can effectively limit cross-namespace references by the implementation.

An implementation choosing to make this exception MUST clearly document that ReferenceGrant is not honored by their implementations and detail which alternative safeguards are available. Note that this is unlikely to apply to ingress implementations of the API and will not apply to all mesh implementations.

For an example of the risks involved in cross-namespace references, refer to CVE-2021-25740. Implementations of this API need to be very careful to avoid confused deputy attacks. ReferenceGrant provides a safeguard for that. Exceptions MUST only be made by implementations that are absolutely certain that other equally effective safeguards are in place.

ForwardTo

To enable cross-namespace forwarding, we’ll need to add an optional namespace field to the ForwardTo BackendRef struct.

  1. type BackendRef struct {
  2.     // ...
  4.     // Namespace is the namespace of the backend. When unspecified, the local
  5.     // namespace is inferred.
  6.     //
  7.     // Support: Core
  8.     //
  9.     // +kubebuilder:validation:MinLength=1
  10.     // +kubebuilder:validation:MaxLength=253
  11.     // +optional
  12.     Namespace *string `json:"namespace,omitempty"`
  13. }

Alternatives

Inline Config

Instead of ReferenceGrant, it is possible to represent these relationships inline. Inline

  1. kind: HTTPRoute
  2. metadata:
  3.   name: foo
  4.   namespace: foo
  5. spec:
  6.   rules:
  7.   - matches:
  8.     - path: /bar
  9.     forwardTo:
  10.       backend:
  11.       - name: bar
  12.         namespace: bar
  13. ---
  14. kind: Service
  15. metadata:
  16.   name: baz
  17.   namespace: baz
  18.   annotations:
  19.     gateway.networking.k8s.io/accept-forwarding-from: bar

Although this requires less YAML for the simple case, it is less flexible. Annotations have real limitations and don’t provide any room for RBAC differentiation. Although it’s possible that we could eventually add a proper field to the Service API to represent this, it would be impossible to add this concept to all potential backend types.

Out of scope

  • Although closely related, this GEP does not attempt to improve the Gateway->Route relationship. That will instead be covered by a future GEP.
  • Although this GEP explores how ReferenceGrant could enable Route inclusion, the details of that feature will be left for a future GEP.

References

GitHub Issues:

Docs: