Configure Kubeflow Fairing with Access to GCP

Configuring your Kubeflow Fairing development environment to access Kubeflow on GKE

This guide describes how to configure your development environment with accessto Google Cloud Platform (GCP), so you can use Kubeflow Fairing to train ordeploy a model on Kubeflow on Google Kubernetes Engine (GKE).

If you have not installed Kubeflow Fairing, follow the guide to installingKubeflow Fairing before continuing.

Using Kubeflow Fairing with Kubeflow notebooks

The standard Kubeflow notebook images include Kubeflow Fairing and comepreconfigured to run training jobs on your Kubeflow cluster. No additionalconfiguration is required.

If your Kubeflow notebook server was built from a custom Jupyter Docker image,follow the instruction in this guide to configure your notebooks environmentwith access to your Kubeflow environment.

Install and configure the Google Cloud SDK

Follow these instructions to set up the Google Cloud SDK in your localdevelopment environment.

  • To check if you have the Google Cloud SDK installed, run the followingcommand:
  1. which gcloud

The response should be something like this:

  1. /usr/bin/gcloud

If you do not have the Google Cloud SDK installed, follow the guide toinstalling the Google Cloud SDK.

  • Use gcloud to set a default project.
  1. export PROJECT_ID=<your-project-id>
  2. gcloud config set project $PROJECT_ID
  • Kubeflow Fairing needs a service account to make API calls to GCP. Therecommended way to provide Fairing with access to thisservice account is to set the GOOGLE_APPLICATION_CREDENTIALS environmentvariable. To check for the GOOGLE_APPLICATION_CREDENTIALS environmentvariable, run the following command:
  1. ls "$GOOGLE_APPLICATION_CREDENTIALS"

The response should be something like this:

  1. /.../.../key.json

If you do not have a service account, then create one and grant it accessto the required roles.

  1. export SA_NAME=<your-sa-name>
  2. gcloud iam service-accounts create $SA_NAME
  3. gcloud projects add-iam-policy-binding $PROJECT_ID \ 
  4.     --member serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com \
  5.     --role 'roles/editor'

Create a key for your service account.

  1. gcloud iam service-accounts keys create ~/key.json \
  2.     --iam-account $SA_NAME@$PROJECT_ID.iam.gserviceaccount.com

Create the GOOGLE_APPLICATION_CREDENTIALS environment variable.

  1. export GOOGLE_APPLICATION_CREDENTIALS=~/key.json

Configure Docker with access to Container Registry

Authorize Docker to access your GCP Container Registry.

  1. gcloud auth configure-docker

Configure access to your Kubeflow cluster

Use the following instructions to update your kubeconfig with credentialsand endpoint information for your Kubeflow cluster. If you do not have aKubeflow cluster, follow the guide to deploying Kubeflow onGKE to set one up.

  • To find your cluster’s name, run the following command to list theclusters in your project:
  1. gcloud container clusters list
  • Update the following command with your cluster’s name and GCP zone. Then,run the command to update your kubeconfig to provide it with credentialsto access this cluster.
  1. export CLUSTER_NAME=kubeflow
  2. export ZONE=us-central1-a
  3. gcloud container clusters get-credentials $CLUSTER_NAME --region $ZONE

Next steps