Use Karmada-search to experience multi-cloud search

This guide will cover:

  • Install karmada-search component in the Karmada control plane.
  • Cache Deployment resources across multiple clusters.
  • Use OpenSearch to graphically retrieve Kubernetes resources.

Prerequisites

Before installing karmada-search, you must install Karmada control plane first. To start up Karmada, you can refer to here. If you just want to try Karmada, we recommend building a development environment by hack/local-up-karmada.sh.

  1. git clone https://github.com/karmada-io/karmada
  2. cd karmada
  3. hack/local-up-karmada.sh

If you use hack/local-up-karmada.sh, karmada-search has been already installed.

If you install Karmada by Helm, you can choose any of the following ways to install karmada-search:

  • Install karmada-search together in host mode
  1. helm upgrade --install karmada -n karmada-system --create-namespace --dependency-update \
  2.      --cleanup-on-fail ./charts/karmada \
  3.      --set components={"search"}
  • Install karmada-search separately in component mode

Edit values.yaml for karmada-search

  1. installMode: "component"
  2. components: [
  3.   "search"
  4. ]
  5. ...

Execute command (switch to the root directory of the repo, and sets the current-context in a kubeconfig file)

  1. kubectl config use-context host
  2. helm install karmada -n karmada-system ./charts/karmada

In addition, karmadactl supports one-click installation for karmada-search.

  1. karmadactl addons enable karmada-search

For more details, you can refer to karmadactl addons instruction.

Cache Deployment resources across clusters

In the following steps, we are going to cache Deployment resources across member clusters. Now we have propagated a nginx Deployment to member1 and member2 according to the example.

1. Create ResourceRegistry that will cache deployment across target clusters

  1. ### deployment-search.yaml
  2. apiVersion: search.karmada.io/v1alpha1
  3. kind: ResourceRegistry
  4. metadata:
  5.   name: deployment-search
  6. spec:
  7.   targetCluster:
  8.     clusterNames:
  9.       - member1
  10.       - member2
  11.   resourceSelectors:
  12.     - apiVersion: apps/v1
  13.       kind: Deployment
  1. kubectl --kubeconfig $HOME/.kube/karmada.config --context karmada-apiserver create -f deployment-search.yaml

2. Test using Kubernetes API

You can obtain deployment resources from member1 and member2 by the following command.

  1. kubectl --kubeconfig $HOME/.kube/karmada.config --context karmada-apiserver get --raw /apis/search.karmada.io/v1alpha1/search/cache/apis/apps/v1/deployments

The output is similar to(irrelevant deployments and fields omitted):

  1. {
  2.   "kind": "List",
  3.   "apiVersion": "apps/v1",
  4.   "metadata": {},
  5.   "items": [{
  6.     "apiVersion": "apps/v1",
  7.     "kind": "Deployment",
  8.     "metadata": {
  9.       "annotations": {
  10.         "deployment.kubernetes.io/revision": "1",
  11.         "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"nginx\"},\"name\":\"nginx\",\"namespace\":\"default\"},\"spec\":{\"replicas\":2,\"selector\":{\"matchLabels\":{\"app\":\"nginx\"}},\"template\":{\"metadata\":{\"labels\":{\"app\":\"nginx\"}},\"spec\":{\"containers\":[{\"image\":\"nginx\",\"name\":\"nginx\"}]}}}}\n",
  12.         "resource.karmada.io/cached-from-cluster": "member1",
  13.         "resourcebinding.karmada.io/name": "nginx-deployment",
  14.         "resourcebinding.karmada.io/namespace": "default",
  15.         "resourcetemplate.karmada.io/uid": "b46d2736-78d8-47db-b589-6e819139ba33"
  16.       },
  17.       "creationTimestamp": "2022-11-18T08:34:28Z",
  18.       "generation": 1,
  19.       "labels": {
  20.         "app": "nginx",
  21.         "propagationpolicy.karmada.io/name": "nginx-propagation",
  22.         "propagationpolicy.karmada.io/namespace": "default",
  23.         "resourcebinding.karmada.io/key": "687f7fb96f",
  24.         "work.karmada.io/name": "nginx-687f7fb96f",
  25.         "work.karmada.io/namespace": "karmada-es-member1"
  26.       }
  27.     }
  28.   }, {
  29.     "apiVersion": "apps/v1",
  30.     "kind": "Deployment",
  31.     "metadata": {
  32.       "annotations": {
  33.         "deployment.kubernetes.io/revision": "1",
  34.         "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"nginx\"},\"name\":\"nginx\",\"namespace\":\"default\"},\"spec\":{\"replicas\":2,\"selector\":{\"matchLabels\":{\"app\":\"nginx\"}},\"template\":{\"metadata\":{\"labels\":{\"app\":\"nginx\"}},\"spec\":{\"containers\":[{\"image\":\"nginx\",\"name\":\"nginx\"}]}}}}\n",
  35.         "resource.karmada.io/cached-from-cluster": "member2",
  36.         "resourcebinding.karmada.io/name": "nginx-deployment",
  37.         "resourcebinding.karmada.io/namespace": "default",
  38.         "resourcetemplate.karmada.io/uid": "e785db97-4d17-4871-99be-6d629c556b89"
  39.       },
  40.       "creationTimestamp": "2022-11-21T02:23:26Z",
  41.       "generation": 1,
  42.       "labels": {
  43.         "app": "nginx",
  44.         "propagationpolicy.karmada.io/name": "nginx-propagation",
  45.         "propagationpolicy.karmada.io/namespace": "default",
  46.         "resourcebinding.karmada.io/key": "687f7fb96f",
  47.         "work.karmada.io/name": "nginx-687f7fb96f",
  48.         "work.karmada.io/namespace": "karmada-es-member2"
  49.       }
  50.     }
  51.   }]
  52. }

Use OpenSearch to graphically retrieve Kubernetes resources

The karmada-search also supports syncing cached resources to backend stores like Elasticsearch or OpenSearch. By leveraging the search engine, you can perform full-text searches with all desired features, by field, and by indice; rank results by score, sort results by field, and aggregate results.

The following is an example about using OpenSearch to graphically retrieve Kubernetes resources.

1. Deploy OpenSearch and OpenSearch Dashboard

Use the following script to deploy OpenSearch and OpenSearch dashboard.

  1. ./hack/deploy-karmada-opensearch.sh $HOME/.kube/karmada.config karmada-host

Verify the installation:

  1. kubectl --kubeconfig $HOME/.kube/karmada.config --context karmada-host get po -A

The output is similar to:

  1. NAMESPACE            NAME                                                   READY   STATUS    RESTARTS   AGE
  2. karmada-system       karmada-opensearch-77454fbcf5-7rpvz                    1/1     Running   0          155m
  3. karmada-system       karmada-opensearch-dashboards-596bf4d9dd-n9429         1/1     Running   0          156m
  4. ...

2. Update ResourceRegistry with backendStore

  1. ### deployment-search.yaml
  2. apiVersion: search.karmada.io/v1alpha1
  3. kind: ResourceRegistry
  4. metadata:
  5.   name: deployment-search
  6. spec:
  7.   backendStore:
  8.     openSearch:
  9.       addresses:
  10.         - http://karmada-opensearch.karmada-system.svc:9200
  11.   targetCluster:
  12.     clusterNames:
  13.       - member1
  14.       - member2
  15.   resourceSelectors:
  16.     - apiVersion: apps/v1
  17.       kind: Deployment
  1. kubectl --kubeconfig $HOME/.kube/karmada.config --context karmada-apiserver apply -f deployment-search.yaml

3. Expose the service of dashboard

You need to expose the web service to the host port so that you can visit the dashboard by HTTP requests.

  1. kubectl --kubeconfig $HOME/.kube/karmada.config --context karmada-host port-forward svc/karmada-opensearch-dashboards 5601:5601 -nkarmada-system --address=0.0.0.0

4. Visit the dashboard

Visit OpenSearch dashboard(http://NodeIP:5601):

opensearch

Now data of Deployment across member1 and member2 has been uploaded to OpenSearch. You can try multi-cloud search yourself by leveraging the search engine.