Securing and Exposing the Registry

Securing and Exposing the Registry

Overview

By default, the OKD registry is secured during cluster installation so that it serves traffic via TLS. A passthrough route is also created by default to expose the service externally.

If for any reason your registry has not been secured or exposed, see the following sections for steps on how to manually do so.

Manually Securing the Registry

To manually secure the registry to serve traffic via TLS:

Deploy the registry.

Fetch the service IP and port of the registry:

$ oc get svc/docker-registry
NAME              LABELS                                    SELECTOR                  IP(S)            PORT(S)
docker-registry   docker-registry=default                   docker-registry=default   172.30.124.220   5000/TCP

You can use an existing server certificate, or create a key and server certificate valid for specified IPs and host names, signed by a specified CA. To create a server certificate for the registry service IP and the docker-registry.default.svc.cluster.local host name, run the following command from the first master listed in the Ansible host inventory file, by default /etc/ansible/hosts:

$ oc adm ca create-server-cert \
    --signer-cert=/etc/origin/master/ca.crt \
    --signer-key=/etc/origin/master/ca.key \
    --signer-serial=/etc/origin/master/ca.serial.txt \
    --hostnames='docker-registry.default.svc.cluster.local,docker-registry.default.svc,172.30.124.220' \
    --cert=/etc/secrets/registry.crt \
    --key=/etc/secrets/registry.key

If the router will be exposed externally, add the public route host name in the --hostnames flag:

--hostnames='mydocker-registry.example.com,docker-registry.default.svc.cluster.local,172.30.124.220 \

See Redeploying Registry and Router Certificates for additional details on updating the default certificate so that the route is externally accessible.

The oc adm ca create-server-cert command generates a certificate that is valid for two years. This can be altered with the —expire-days option, but for security reasons, it is recommended to not make it greater than this value.

Create the secret for the registry certificates:

$ oc create secret generic registry-certificates \
    --from-file=/etc/secrets/registry.crt \
    --from-file=/etc/secrets/registry.key

Add the secret to the registry pod’s service accounts (including the default service account):

$ oc secrets link registry registry-certificates
$ oc secrets link default  registry-certificates

Limiting secrets to only the service accounts that reference them is disabled by default. This means that if serviceAccountConfig.limitSecretReferences is set to false (the default setting) in the master configuration file, linking secrets to a service is not required.

Pause the docker-registry service:
```
$ oc rollout pause dc/docker-registry
```

Add the secret volume to the registry deployment configuration:

$ oc set volume dc/docker-registry --add --type=secret \
    --secret-name=registry-certificates -m /etc/secrets

Enable TLS by adding the following environment variables to the registry deployment configuration:
```
$ oc set env dc/docker-registry \
    REGISTRY_HTTP_TLS_CERTIFICATE=/etc/secrets/registry.crt \
    REGISTRY_HTTP_TLS_KEY=/etc/secrets/registry.key
```
See the Configuring a registry section of the Docker documentation for more information.

Update the scheme used for the registry’s liveness probe from HTTP to HTTPS:

$ oc patch dc/docker-registry -p '{"spec": {"template": {"spec": {"containers":[{
    "name":"registry",
    "livenessProbe":  {"httpGet": {"scheme":"HTTPS"}}
  }]}}}}'

If your registry was initially deployed on OKD 1.1.2 or later, update the scheme used for the registry’s readiness probe from HTTP to HTTPS:

$ oc patch dc/docker-registry -p '{"spec": {"template": {"spec": {"containers":[{
   "name":"registry",
   "readinessProbe":  {"httpGet": {"scheme":"HTTPS"}}
 }]}}}}'

Resume the docker-registry service:
```
$ oc rollout resume dc/docker-registry
```
Validate the registry is running in TLS mode. Wait until the latest docker-registry deployment completes and verify the Docker logs for the registry container. You should find an entry for listening on :5000, tls.
```
$ oc logs dc/docker-registry | grep tls
time="2015-05-27T05:05:53Z" level=info msg="listening on :5000, tls" instance.id=deeba528-c478-41f5-b751-dc48e4935fc2
```

Copy the CA certificate to the Docker certificates directory. This must be done on all nodes in the cluster:

$ dcertsdir=/etc/docker/certs.d
$ destdir_addr=$dcertsdir/172.30.124.220:5000
$ destdir_name=$dcertsdir/docker-registry.default.svc.cluster.local:5000
$ sudo mkdir -p $destdir_addr $destdir_name
$ sudo cp ca.crt $destdir_addr    (1)
$ sudo cp ca.crt $destdir_name

1	The *ca.crt* file is a copy of */etc/origin/master/ca.crt* on the master.

When using authentication, some versions of docker also require you to configure your cluster to trust the certificate at the OS level.
1. Copy the certificate:
```
$ cp /etc/origin/master/ca.crt /etc/pki/ca-trust/source/anchors/myregistrydomain.com.crt
```
2. Run:
```
$ update-ca-trust enable
```

Remove the --insecure-registry option only for this particular registry in the /etc/sysconfig/docker file. Then, reload the daemon and restart the docker service to reflect this configuration change:
```
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker
```

Validate the docker client connection. Running docker push to the registry or docker pull from the registry should succeed. Make sure you have logged into the registry.

$ docker tag|push <registry/image> <internal_registry/project/image>

For example:

$ docker pull busybox
$ docker tag docker.io/busybox 172.30.124.220:5000/openshift/busybox
$ docker push 172.30.124.220:5000/openshift/busybox
...
cf2616975b4a: Image successfully pushed
Digest: sha256:3662dd821983bc4326bee12caec61367e7fb6f6a3ee547cbaff98f77403cab55

Manually Exposing a Secure Registry

Instead of logging in to the OKD registry from within the OKD cluster, you can gain external access to it by first securing the registry and then exposing it with a route. This allows you to log in to the registry from outside the cluster using the route address, and to tag and push images using the route host.

Each of the following prerequisite steps are performed by default during a typical cluster installation. If they have not been, perform them manually:

A passthrough route should have been created by default for the registry during the initial cluster installation:

Verify whether the route exists:
```
$ oc get route/docker-registry -o yaml
apiVersion: v1
kind: Route
metadata:
  name: docker-registry
spec:
  host: <host> (1)
  to:
    kind: Service
    name: docker-registry (2)
  tls:
    termination: passthrough (3)
```
1 The host for your route. You must be able to resolve this name externally via DNS to the router’s IP address.
2 The service name for your registry.
3 Specifies this route as a passthrough route.

Re-encrypt routes are also supported for exposing the secure registry.

If it does not exist, create the route via the oc create route passthrough command, specifying the registry as the route’s service. By default, the name of the created route is the same as the service name:

Get the docker-registry service details:

$ oc get svc
NAME              CLUSTER_IP       EXTERNAL_IP   PORT(S)                 SELECTOR                  AGE
docker-registry   172.30.69.167    <none>        5000/TCP                docker-registry=default   4h
kubernetes        172.30.0.1       <none>        443/TCP,53/UDP,53/TCP   <none>                    4h
router            172.30.172.132   <none>        80/TCP                  router=router             4h

Create the route:

$ oc create route passthrough    \
    --service=docker-registry    \(1)
    --hostname=<host>
route "docker-registry" created     (2)

1	Specifies the registry as the route’s service.
2	The route name is identical to the service name.

Next, you must trust the certificates being used for the registry on your host system to allow the host to push and pull images. The certificates referenced were created when you secured your registry.
```
$ sudo mkdir -p /etc/docker/certs.d/<host>
$ sudo cp <ca_certificate_file> /etc/docker/certs.d/<host>
$ sudo systemctl restart docker
```
Log in to the registry using the information from securing the registry. However, this time point to the host name used in the route rather than your service IP. When logging in to a secured and exposed registry, make sure you specify the registry in the docker login command:
```
# docker login -e user@company.com \
    -u f83j5h6 \
    -p Ju1PeM47R0B92Lk3AZp-bWJSck2F7aGCiZ66aFGZrs2 \
    <host>
```

You can now tag and push images using the route host. For example, to tag and push a busybox image in a project called test:

$ oc get imagestreams -n test
NAME      DOCKER REPO   TAGS      UPDATED
$ docker pull busybox
$ docker tag busybox <host>/test/busybox
$ docker push <host>/test/busybox
The push refers to a repository [<host>/test/busybox] (len: 1)
8c2e06607696: Image already exists
6ce2e90b0bc7: Image successfully pushed
cf2616975b4a: Image successfully pushed
Digest: sha256:6c7e676d76921031532d7d9c0394d0da7c2906f4cb4c049904c4031147d8ca31
$ docker pull <host>/test/busybox
latest: Pulling from <host>/test/busybox
cf2616975b4a: Already exists
6ce2e90b0bc7: Already exists
8c2e06607696: Already exists
Digest: sha256:6c7e676d76921031532d7d9c0394d0da7c2906f4cb4c049904c4031147d8ca31
Status: Image is up to date for <host>/test/busybox:latest
$ oc get imagestreams -n test
NAME      DOCKER REPO                       TAGS      UPDATED
busybox   172.30.11.215:5000/test/busybox   latest    2 seconds ago

Your image streams will have the IP address and port of the registry service, not the route name and port. See oc get imagestreams for details.

Manually Exposing a Non-Secure Registry

Instead of securing the registry in order to expose the registry, you can simply expose a non-secure registry for non-production OKD environments. This allows you to have an external route to the registry without using SSL certificates.

Only non-production environments should expose a non-secure registry to external access.

To expose a non-secure registry:

Expose the registry:

# oc expose service docker-registry --hostname=<hostname> -n default

This creates the following JSON file:

apiVersion: v1
kind: Route
metadata:
  creationTimestamp: null
  labels:
    docker-registry: default
  name: docker-registry
spec:
  host: registry.example.com
  port:
    targetPort: "5000"
  to:
    kind: Service
    name: docker-registry
status: {}

Verify that the route has been created successfully:

# oc get route
NAME              HOST/PORT                    PATH      SERVICE           LABELS                    INSECURE POLICY   TLS TERMINATION
docker-registry   registry.example.com            docker-registry   docker-registry=default

Check the health of the registry:
```
$ curl -v http://registry.example.com/healthz
```
Expect an HTTP 200/OK message.

After exposing the registry, update your /etc/sysconfig/docker file by adding the port number to the OPTIONS entry. For example:
```
OPTIONS='--selinux-enabled --insecure-registry=172.30.0.0/16 --insecure-registry registry.example.com:80'
```
The above options should be added on the client from which you are trying to log in.
Also, ensure that Docker is running on the client.

When logging in to the non-secured and exposed registry, make sure you specify the registry in the docker login command. For example:

# docker login -e user@company.com \
    -u f83j5h6 \
    -p Ju1PeM47R0B92Lk3AZp-bWJSck2F7aGCiZ66aFGZrs2 \
    <host>

1	The host for your route. You must be able to resolve this name externally via DNS to the router’s IP address.
2	The service name for your registry.
3	Specifies this route as a passthrough route.