Workspace

Workspace is the entrypoint for all objects in JinaD. It primarily represents 4 pieces.

  1. Docker Image

    All objects created by JinaD are containerized. Workspace is responsible for building the base image. You can customize each image with the help of a .jinad file and a requirements.txt file.

  2. Docker Network

    Workspace is also responsible for managing a private bridge network for all child containers. This provides network isolation from other workspaces, while allowing all containers inside the same workspace to communicate.

  3. Local work directory

    All the files used to manage a Jina object or created by a Jina object are stored here. This directory is exposed to all child containers. These can be:

    • config files (e.g.- Flow/Executor YAML, Python modules etc),

    • data written by your Executor

    • .jinad file

    • requirements.txt file

  4. Special files

    • .jinad is an optional file defining how the base image is built. Following arguments are supported.

      1. build = default                 ; NOTE: devel/default, (gpu: to be added).
      2. python = 3.7                    ; NOTE: 3.7, 3.8, 3.9 allowed.
      3. jina = 2.0.rc7                  ; NOTE: to be added.
      4. ports = 45678                   ; NOTE: comma separated ports to be mapped.
      5. run = "python app.py 45678"     ; NOTE: command to start a Jina project on remote.

      You can also deploy an end-to-end Jina project on remote using the following steps.

      • Include a .jinad file with run set to your default entrypoint (e.g. - python app.py)

      • Upload all your files including .jinad during workspace creation.

      • This will deploy a custom container with your project

    • requirements.txt defines all python packages to be installed using pip in the base image.

      1. annoy
      2. torch>=1.8.0
      3. tensorflow

Create a workspace (redoc)

Create a directory (say awesome_project) on local which has all your files (yaml, py_modules, requirements.txt , .jinad etc.)

  1. from daemon.clients import JinaDClient
  3. client = JinaDClient(host=HOST, port=PORT)
  4. my_workspace_id = client.workspaces.create(paths=['path_to_awesome_project'])
  1.      [email protected][I]:uploading 3 file(s): flow.yml, requirements.txt, .jinad
  2.   🌏 36f9d7f70145 DaemonWorker1 INFO  ---> 70578df55b1c
  3.   🌏 36f9d7f70145 DaemonWorker1 INFO Step 4/7 : ARG PIP_REQUIREMENTS
  4.   🌏 36f9d7f70145 DaemonWorker1 INFO  ---> Running in e1588f87b32c
  5.   🌏 36f9d7f70145 DaemonWorker1 INFO Removing intermediate container e1588f87b32c
  6.   🌏 36f9d7f70145 DaemonWorker1 INFO  ---> 9f715ea59f8a
  7.   🌏 36f9d7f70145 DaemonWorker1 INFO Step 5/7 : RUN if [ -n "$PIP_REQUIREMENTS" ]; then         echo "Installing
  8.   ${PIP_REQUIREMENTS}";         for package in ${PIP_REQUIREMENTS}; do             pip install "${package}";         done;     fi
  9.   🌏 36f9d7f70145 DaemonWorker1 INFO  ---> Running in e9018896b366
  10.   🌏 36f9d7f70145 DaemonWorker1 INFO Installing tinydb sklearn
  11.   🌏 36f9d7f70145 DaemonWorker1 INFO Collecting tinydb
  12.   🌏 36f9d7f70145 DaemonWorker1 INFO   Downloading tinydb-4.5.1-py3-none-any.whl (23 kB)
  13.   🌏 36f9d7f70145 DaemonWorker1 INFO Requirement already satisfied: typing-extensions<4.0.0,>=3.10.0 in
  14.   /usr/local/lib/python3.7/site-packages (from tinydb) (3.10.0.0)
  15.   🌏 36f9d7f70145 DaemonWorker1 INFO Installing collected packages: tinydb
  16.   🌏 36f9d7f70145 DaemonWorker1 INFO Successfully installed tinydb-4.5.1
  17.   🌏 36f9d7f70145 DaemonWorker1 WARNING WARNING: Running pip as the 'root' user can result in broken permissions and conflicting
  18.   behaviour with the system package manager. It is recommended to use a virtual environment instead:
  19.   https://pip.pypa.io/warnings/venv
  20.   🌏 36f9d7f70145 DaemonWorker1 INFO Collecting sklearn
  21.   🌏 36f9d7f70145 DaemonWorker1 INFO   Downloading sklearn-0.0.tar.gz (1.1 kB)
  22.   🌏 36f9d7f70145 DaemonWorker1 INFO Collecting scikit-learn
  23.   🌏 36f9d7f70145 DaemonWorker1 INFO   Downloading scikit_learn-0.24.2-cp37-cp37m-manylinux2010_x86_64.whl (22.3 MB)
  24.   🌏 36f9d7f70145 DaemonWorker1 INFO Collecting joblib>=0.11
  25.   🌏 36f9d7f70145 DaemonWorker1 INFO   Downloading joblib-1.0.1-py3-none-any.whl (303 kB)
  26.   🌏 36f9d7f70145 DaemonWorker1 INFO Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib/python3.7/site-packages (from
  27.   scikit-learn->sklearn) (1.21.1)
  28.   🌏 36f9d7f70145 DaemonWorker1 INFO Collecting scipy>=0.19.1
  29.   🌏 36f9d7f70145 DaemonWorker1 INFO   Downloading scipy-1.7.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (28.5 MB)
  30.   🌏 36f9d7f70145 DaemonWorker1 INFO Collecting threadpoolctl>=2.0.0
  31.   🌏 36f9d7f70145 DaemonWorker1 INFO   Downloading threadpoolctl-2.2.0-py3-none-any.whl (12 kB)
  32.   🌏 36f9d7f70145 DaemonWorker1 INFO Building wheels for collected packages: sklearn
  33.   🌏 36f9d7f70145 DaemonWorker1 INFO   Building wheel for sklearn (setup.py): started
  34.   🌏 36f9d7f70145 DaemonWorker1 INFO   Building wheel for sklearn (setup.py): finished with status 'done'
  35.   🌏 36f9d7f70145 DaemonWorker1 INFO   Created wheel for sklearn: filename=sklearn-0.0-py2.py3-none-any.whl size=1309 sha256=ac85019415e0eeebf468e2f71c43d8ff9b78131eaaccce89e34bb5ba8a2473ca
  36.   🌏 36f9d7f70145 DaemonWorker1 INFO Successfully built sklearn
  37.   🌏 36f9d7f70145 DaemonWorker1 INFO Installing collected packages: threadpoolctl, scipy, joblib, scikit-learn, sklearn
  38.   🌏 36f9d7f70145 DaemonWorker1 INFO Successfully installed joblib-1.0.1 scikit-learn-0.24.2 scipy-1.7.0 sklearn-0.0 threadpoolctl-2.2.0
  39.   🌏 36f9d7f70145 DaemonWorker1 WARNING WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to
  40.   use a virtual environment instead: https://pip.pypa.io/warnings/venv
  41.   🌎  Workspace: Creating...    [email protected][I]:jworkspace-480ec0d8-ea02-4adb-8e02-04cd27962863 created successfully

Get details of a workspace (redoc)

  1. from daemon.clients import JinaDClient
  3. client = JinaDClient(host=HOST, port=PORT)
  4. client.workspaces.get(my_workspace_id)
  1. {
  2.   "time_created": "2021-07-26T17:31:29.326049",
  3.   "state": "ACTIVE",
  4.   "metadata": {
  5.     "image_id": "97b0cb4860",
  6.     "image_name": "jworkspace:480ec0d8-ea02-4adb-8e02-04cd27962863",
  7.     "network": "8dcd21b98a",
  8.     "workdir": "/tmp/jinad/jworkspace-480ec0d8-ea02-4adb-8e02-04cd27962863",
  9.     "container_id": None,
  10.     "managed_objects": []
  11.   },
  12.   "arguments": {
  13.     "files": ["flow.yml", "requirements.txt", ".jinad"],
  14.     "jinad": {
  15.       "build": "default",
  16.       "dockerfile": "/usr/local/lib/python3.7/site-packages/daemon/Dockerfiles/default.Dockerfile"
  17.     },
  18.     "requirements": "tinydb sklearn"
  19.   }
  20. }

List all workspaces (redoc)

  1. client.workspaces.list()
  1. {
  2.   "jworkspace-2b017b8f-19af-4d78-9364-6404447d91ac": {
  3.     ...
  4.   },
  5.   "jworkspace-8fec6449-2824-4913-9c06-3d0ec1314674": {
  6.     ...
  7.   },
  8.   "jworkspace-41dbe23a-9ecd-4e84-8df2-8dd6295a55b4": {
  9.     ...
  10.   },
  11.   "jworkspace-0cc90166-5ce2-4702-9d30-0ff8f3598a9f": {
  12.     ...
  13.   },
  14.   "jworkspace-be53f490-549a-4335-831a-5fb13a1de754": {
  15.     ...
  16.   },
  17.   "jworkspace-48319ab9-6c36-4e2d-b687-dd0ab498cb4f": {
  18.     ...
  19.   }
  20. }

Delete a workspace (redoc)

  1. success_deleted = client.workspaces.delete(id=workspace_id)
  2. assert success_deleted