Local Install via source for Mac

Instructions for using Vitess on your macOS machine for testing purposes

This guide covers installing Vitess locally for testing purposes, from pre-compiled binaries. We will launch multiple copies of mysqld, so it is recommended to have greater than 4GB RAM, as well as 20GB of available disk space.

A pure homebrew setup is also available.

Install Brew

For the purposes of installing software you will need to have brew installed. This will also install curl and git which will also be needed:

  1. $ curl https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh > brew-install.sh
  3. $ bash brew-install.sh

Install MySQL and etcd

Once brew is installed you will need to install some dependencies for Vitess. Vitess supports the databases listed here:

  1. $ brew install automake go mysql mysql-client etcd

When MySQL installs with brew it will startup, you will want to shut this process down, as Vitess will be managing the startup and shutdown of MySQL:

  1. $ brew services stop mysql

Install Node 16.13.0+ (required to run VTAdmin)

  1. $ brew install nvm
  2. $ nvm install --lts 16.13.0
  3. $ nvm use 16.13.0

See the vtadmin README for more details.

PATH Settings

With the tools you’ve just installed via brew, you will next update your PATH variable so your shell knows where to find the binaries:

  1. $ echo “export PATH=${PATH}:/opt/homebrew/opt/mysql-client/bin:/opt/homebrew/opt/mysql/bin:~/Github/vitess/bin:/Users/jason/go/bin:​​/opt/homebrew/bin” >> ~/.zshrc
  2. $ source ~/.zshrc

If you’re using bash for your shell you’ll have to update the paths in .bash_profile or .bashrc instead. Mac does not read .bashrc by default:

  1. $ echo “export PATH=${PATH}:/opt/homebrew/opt/mysql-client/bin:/opt/homebrew/opt/mysql/bin:~/Github/vitess/bin:/Users/jason/go/bin:/opt/homebrew/bin” >> ~/.bash_profile
  2. $ source ~/.bash_profile

System Check

Before going further, you should check to confirm your shell has access to go, mysql, and mysqld. If versions are not returned when you run the following commands you should check that the programs are installed and the path is correct for your shell:

  1. $ mysqld --version
  2. $ mysql --version
  3. $ go version
  4. $ etcd --version
  5. $ node --version
  6. $ npm --version

Install Vitess

With everything now in place you can clone and build Vitess.

  1. $ git clone https://github.com/vitessio/vitess.git
  2. $ cd vitess
  3. $ make build

It will take some time for Vitess to build. Once it completes you should see a bin folder which will hold the Vitess binaries. You will need to add this folder to your PATH variable as well:

  1. $ cd bin
  2. $ echo "$(printf 'export PATH="${PATH}:'; echo "$(pwd)\"")" >> ~/.zshrc
  3. $ source ~/.zshrc

If you are using bash this will need to be your .bash_profile or .bashrc file instead:

  1. $ cd bin
  2. $ echo "$(printf 'export PATH="${PATH}:'; echo "$(pwd)\"")" >> ~/.bash_profile
  3. $ source ~/.bash_profile

You are now ready to start your first cluster! Open a new terminal window to ensure your .bashrc file changes take effect.

Start a Single Keyspace Cluster

You are now ready to stand up your first Vitess cluster, using the example scripts provided in the source code. Assuming you are still in the bin directory you will need to navigate to the sample files:

  1. $ cd ../examples/local/

From here you can startup the cluster and source the env file which will help set environment variables used when working with this local cluster:

  1. $ ./101_initial_cluster.sh
  2. $ source env.sh

You should see an output similar to the following:

  1. ~/my-vitess-example> ./101_initial_cluster.sh
  2. $ ./101_initial_cluster.sh 
  3. add /vitess/global
  4. add /vitess/zone1
  5. add zone1 CellInfo
  6. Created cell: zone1
  7. etcd start done...
  8. Starting vtctld...
  9. vtctld is running!
  10. Starting MySQL for tablet zone1-0000000100...
  11. Starting vttablet for zone1-0000000100...
  12. HTTP/1.1 200 OK
  13. Date: Thu, 01 Sep 2022 12:49:50 GMT
  14. Content-Type: text/html; charset=utf-8
  16. Starting MySQL for tablet zone1-0000000101...
  17. Starting vttablet for zone1-0000000101...
  18. HTTP/1.1 200 OK
  19. Date: Thu, 01 Sep 2022 12:49:55 GMT
  20. Content-Type: text/html; charset=utf-8
  22. Starting MySQL for tablet zone1-0000000102...
  23. Starting vttablet for zone1-0000000102...
  24. HTTP/1.1 200 OK
  25. Date: Thu, 01 Sep 2022 12:50:00 GMT
  26. Content-Type: text/html; charset=utf-8
  28. {
  29.   "keyspace": {
  30.     "served_froms": [],
  31.     "keyspace_type": 0,
  32.     "base_keyspace": "",
  33.     "snapshot_time": null,
  34.     "durability_policy": "semi_sync",
  35.     "throttler_config": null,
  36.     "sidecar_db_name": "_vt"
  37.   }
  38. }
  39. vtorc is running!
  40.   - UI: http://localhost:16000
  41.   - Logs: /Users/manangupta/vitess/vtdataroot/tmp/vtorc.out
  42.   - PID: 74088
  44. zone1-0000000100 commerce 0 primary localhost:15100 localhost:17100 [] 2022-09-23T05:48:52Z
  46. New VSchema object:
  47. {
  48.   "sharded": false,
  49.   "vindexes": {},
  50.   "tables": {
  51.     "corder": {
  52.       "type": "",
  53.       "column_vindexes": [],
  54.       "auto_increment": null,
  55.       "columns": [],
  56.       "pinned": "",
  57.       "column_list_authoritative": false
  58.     },
  59.     "customer": {
  60.       "type": "",
  61.       "column_vindexes": [],
  62.       "auto_increment": null,
  63.       "columns": [],
  64.       "pinned": "",
  65.       "column_list_authoritative": false
  66.     },
  67.     "product": {
  68.       "type": "",
  69.       "column_vindexes": [],
  70.       "auto_increment": null,
  71.       "columns": [],
  72.       "pinned": "",
  73.       "column_list_authoritative": false
  74.     }
  75.   },
  76.   "require_explicit_routing": false
  77. }
  78. If this is not what you expected, check the input data (as JSON parsing will skip unexpected fields).
  79. Waiting for vtgate to be up...
  80. vtgate is up!
  81. Access vtgate at http://Manans-MacBook-Pro.local:15001/debug/status
  82. vtadmin-api is running!
  83.   - API: http://localhost:14200
  84.   - Logs: /Users/manangupta/vitess/vtdataroot/tmp/vtadmin-api.out
  85.   - PID: 74039
  87. > vtadmin@0.1.0 build
  88. > vite build
  90. vite v4.2.1 building for production...
  91. transforming (1218) src/icons/alertFail.svgUse of eval in "node_modules/@protobufjs/inquire/index.js" is strongly discouraged as it poses security risks and may cause issues with minification.
  92. ✓ 1231 modules transformed.
  93. build/assets/chevronUp-3d6782a5.svg              0.18 kB
  94. build/assets/chevronDown-02f94e73.svg            0.19 kB
  95. build/assets/download-8ef290b4.svg               0.21 kB
  96. build/assets/delete-a9184ef9.svg                 0.23 kB
  97. build/assets/info-2617ee9d.svg                   0.34 kB
  98. build/assets/circleAdd-cfd7e5db.svg              0.35 kB
  99. build/assets/alertFail-8056b6e4.svg              0.35 kB
  100. build/assets/checkSuccess-f8fd1dbb.svg           0.36 kB
  101. build/assets/search-3261bac7.svg                 0.41 kB
  102. build/assets/question-a67b2492.svg               0.46 kB
  103. build/assets/runQuery-edfab4ed.svg               0.49 kB
  104. build/assets/open-405dd348.svg                   0.49 kB
  105. build/index.html                                 0.90 kB
  106. build/assets/bug-5b6edb54.svg                    0.99 kB
  107. build/assets/topology-0032b65e.svg               1.62 kB
  108. build/assets/NotoMono-Regular-41fd7ccc.ttf     107.85 kB
  109. build/assets/NotoSans-Regular-c8cff31f.ttf     313.14 kB
  110. build/assets/NotoSans-SemiBold-43207822.ttf    313.72 kB
  111. build/assets/NotoSans-Bold-c6a598dd.ttf        313.79 kB
  112. build/assets/index-ef40fbc9.css                 87.78 kB │ gzip:  15.02 kB
  113. build/assets/index-4ddb52ed.js               2,811.88 kB │ gzip: 492.59 kB
  115. (!) Some chunks are larger than 500 kBs after minification. Consider:
  116. - Using dynamic import() to code-split the application
  117. - Use build.rollupOptions.output.manualChunks to improve chunking: https://rollupjs.org/configuration-options/#output-manualchunks
  118. - Adjust chunk size limit for this warning via build.chunkSizeWarningLimit.
  119. ✓ built in 10.85s
  121. vtadmin-web is running!
  122.   - Browser: http://localhost:14201
  123.   - Logs: /Users/manangupta/vitess/vtdataroot/tmp/vtadmin-web.out
  124.   - PID: 74070

If you encounter any errors, such as ports already in use, you can kill the processes and start over. Ensure you’re in the vitess/examples/local directory, then issue the statement to kill the processes and remove the data directory. This data directory vtdataroot will get recreated when you run the 101_initial_cluster.sh startup script again.

  1. user@computer:~/Github/vitess/examples/local$ pwd
  2. /home/user/Github/vitess/examples/local
  4. user@computer:~/Github/vitess/examples/local$ pkill -9 -f '(vtdataroot|VTDATAROOT|vitess|vtadmin)'
  5. etcd killed (pid 224091)
  6. vtctld killed (pid 224154)
  7. mysqld_safe killed (pid 224247)
  8. mysqld killed (pid 224716)
  9. vttablet killed (pid 224764)
  10. mysqld_safe killed (pid 224897)
  11. mysqld killed (pid 225364)
  12. vttablet killed (pid 225413)
  13. mysqld_safe killed (pid 225529)
  14. mysqld killed (pid 225995)
  15. vttablet killed (pid 226045)
  16. vtgate killed (pid 226204)
  17. vtadmin killed (pid 226391)
  18. vtorc killed (pid 226397)
  20. user@computer:~/Github/vitess/examples/local$ rm -rf ./vtdataroot

Connect to your cluster

You should now be able to connect to the VTGate server that was started in 101_initial_cluster.sh:

  1. $ mysql -P 15306 -u root --protocol tcp

You can also now browse and administer your new Vitess cluster using the VTAdmin UI at the following URL:

  1. http://localhost:14201

VTOrc is also setup as part of the initialization. You can look at its user-interface at:

  1. http://localhost:16000

Summary

In this example, we deployed a single unsharded keyspace named commerce. Unsharded keyspaces have a single shard named 0. The following schema reflects a common ecommerce scenario that was created by the script:

  1. create table product (
  2.   sku varbinary(128),
  3.   description varbinary(128),
  4.   price bigint,
  5.   primary key(sku)
  6. );
  7. create table customer (
  8.   customer_id bigint not null auto_increment,
  9.   email varbinary(128),
  10.   primary key(customer_id)
  11. );
  12. create table corder (
  13.   order_id bigint not null auto_increment,
  14.   customer_id bigint,
  15.   sku varbinary(128),
  16.   price bigint,
  17.   primary key(order_id)
  18. );

The schema has been simplified to include only those fields that are significant to the example:

  • The product table contains the product information for all of the products.
  • The customer table has a customer_id that has an auto_increment. A typical customer table would have a lot more columns, and sometimes additional detail tables.
  • The corder table (named so because order is an SQL reserved word) has an order_id auto-increment column. It also has foreign keys into customer(customer_id) and product(sku).

Next Steps

You can now proceed with MoveTables.

Or alternatively, once you are finished with the local examples or if you would like to start over, you can clean up by running the 401_teardown script:

  1. $ ./401_teardown.sh
  2. $ rm -rf ./vtdataroot

Sometimes you will still need to manually kill processes if there are errors in the environment:

  1. $ pkill -9 -f ./vtdataroot
  2. $ rm -rf ./vtdataroot