Setting Up Development Environment for FE using IntelliJ IDEA

1. Environmental Preparation

  • Git
  • JDK1.8+
  • IntelliJ IDEA
  • Maven (Optional, IDEA shipped embedded Maven3)

  1. Git clone codebase from https://github.com/apache/incubator-doris.git

  2. Use IntelliJ IDEA to open the code root directory

  3. If your are only interested in FE module, and for some reason you can’t or don’t want to compile full thirdparty libraries, the minimum tool required for FE module is thrift, so you can manually install thrift and copy or create a link of the executable thrift command to ./thirdparty/installed/bin.

    1. Doris build against `thrift` 0.13.0 ( note : `Doris` 0.15 and later version build against `thrift` 0.13.0 , the previous version is still `thrift` 0.9.3)   
    3. Windows: 
    4.    1. Download：`http://archive.apache.org/dist/thrift/0.13.0/thrift-0.13.0.exe`
    5.    2. Copy：copy the file to `./thirdparty/installed/bin`
    7. MacOS: 
    8.    1. Download：`brew install thrift@0.13.0`
    9.    2. Establish soft connection： 
    10.      `mkdir -p ./thirdparty/installed/bin`
    11.      `ln -s /opt/homebrew/Cellar/thrift@0.13.0/0.13.0/bin/thrift ./thirdparty/installed/bin/thrift`
    13. Note：The error that the version cannot be found may be reported when MacOS execute `brew install thrift@0.13.0`. The solution is execute at the terminal as follows:
    14.    1. `brew tap-new $USER/local-tap`
    15.    2. `brew extract --version='0.13.0' thrift $USER/local-tap`
    16.    3. `brew install thrift@0.13.0`
    17. Reference link: `https://gist.github.com/tonydeng/02e571f273d6cce4230dc8d5f394493c`

  4. Go to ./fe folder and run the following maven command to generate sources.

    1. mvn generate-sources

    If fails, run following command.

    1. mvn clean install -DskipTests

    You can also use IDE embedded GUI tools to run maven command to generate sources

Setting Up dev env for FE - IntelliJ IDEA - 图1

If you are developing on the OS which lack of support to run shell script and make such as Windows, a workround here is generate codes in Linux and copy them back. Using Docker should also be an option.

2. Debug

  1. Import ./fe into IDEA

  2. Follow the picture to create the folders (The directory may exist in the new version. If it exists, skip it, otherwise create it.)

Setting Up dev env for FE - IntelliJ IDEA - 图2

  1. Build ui project , and copy files from directory ui/dist into directory webroot ( you can skip this step , if you don’t need Doris UI )

3. Custom FE configuration

Copy below content into conf/fe.conf and tune it to fit your environment.

  1. # Licensed to the Apache Software Foundation (ASF) under one
  2. # or more contributor license agreements.  See the NOTICE file
  3. # distributed with this work for additional information
  4. # regarding copyright ownership.  The ASF licenses this file
  5. # to you under the Apache License, Version 2.0 (the
  6. # "License"); you may not use this file except in compliance
  7. # with the License.  You may obtain a copy of the License at
  8. #
  9. #   http://www.apache.org/licenses/LICENSE-2.0
  10. #
  11. # Unless required by applicable law or agreed to in writing,
  12. # software distributed under the License is distributed on an
  13. # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  14. # KIND, either express or implied.  See the License for the
  15. # specific language governing permissions and limitations
  16. # under the License.
  18. #####################################################################
  19. ## The uppercase properties are read and exported by bin/start_fe.sh.
  20. ## To see all Frontend configurations,
  21. ## see fe/src/org/apache/doris/common/Config.java
  22. #####################################################################
  24. # the output dir of stderr and stdout 
  25. LOG_DIR = ${DORIS_HOME}/log
  27. DATE = `date +%Y%m%d-%H%M%S`
  28. JAVA_OPTS="-Xmx2048m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"
  30. # For jdk 9+, this JAVA_OPTS will be used as default JVM options
  31. JAVA_OPTS_FOR_JDK_9="-Xmx4096m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$DATE:time"
  33. ##
  34. ## the lowercase properties are read by main program.
  35. ##
  37. # INFO, WARN, ERROR, FATAL
  38. sys_log_level = INFO
  40. # store metadata, create it if it is not exist.
  41. # Default value is ${DORIS_HOME}/doris-meta
  42. # meta_dir = ${DORIS_HOME}/doris-meta
  44. http_port = 8030
  45. rpc_port = 9020
  46. query_port = 9030
  47. edit_log_port = 9010
  48. mysql_service_nio_enabled = true
  50. # Choose one if there are more than one ip except loopback address. 
  51. # Note that there should at most one ip match this list.
  52. # If no ip match this rule, will choose one randomly.
  53. # use CIDR format, e.g. 10.10.10.0/24
  54. # Default value is empty.
  55. # priority_networks = 10.10.10.0/24;192.168.0.0/16
  57. # Advanced configurations 
  58. # log_roll_size_mb = 1024
  59. # sys_log_dir = ${DORIS_HOME}/log
  60. # sys_log_roll_num = 10
  61. # sys_log_verbose_modules = 
  62. # audit_log_dir = ${DORIS_HOME}/log
  63. # audit_log_modules = slow_query, query
  64. # audit_log_roll_num = 10
  65. # meta_delay_toleration_second = 10
  66. # qe_max_connection = 1024
  67. # max_conn_per_user = 100
  68. # qe_query_timeout_second = 300
  69. # qe_slow_log_ms = 5000

4. Setting Environment Variables

Follow the picture to set runtime Environment Variables in IDEA

Setting Up dev env for FE - IntelliJ IDEA - 图3

5. Start FE

Having fun with Doris FE!