Appendix 1: Configuration Parameters

Before starting to use IoTDB, you need to config the configuration files first. For your convenience, we have already set the default config in the files.

In total, we provide users three kinds of configurations module:

  • environment configuration file (iotdb-env.bat, iotdb-env.sh). The default configuration file for the environment configuration item. Users can configure the relevant system configuration items of JAVA-JVM in the file.

  • system configuration file (iotdb-engine.properties).

    • iotdb-engine.properties: The default configuration file for the IoTDB engine layer configuration item. Users can configure the IoTDB engine related parameters in the file, such as JDBC service listening port (rpc_port), unsequence data storage directory (unsequence_data_dir), etc. What’s more, Users can configure the information about the TsFile, such as the data size written to the disk per time(group_size_in_byte).
  • log configuration file (logback.xml)

The configuration files of the three configuration items are located in the IoTDB installation directory: $IOTDB_HOME/conf folder.

Hot Modification Configuration

For the convenience of users, IoTDB server provides users with hot modification function, that is, modifying some configuration parameters in iotdb engine. Properties during the system operation and applying them to the system immediately. In the parameters described below, these parameters whose way of Effective is trigger support hot modification.

Trigger way: The client sends the command load configuration to the IoTDB server. See Chapter 4 for the usage of the client.

IoTDB Environment Configuration File

The environment configuration file is mainly used to configure the Java environment related parameters when IoTDB Server is running, such as JVM related configuration. This part of the configuration is passed to the JVM when the IoTDB Server starts. Users can view the contents of the environment configuration file by viewing the iotdb-env.sh (or iotdb-env.bat) file.

The detail of each variables are as follows:

  • MAX_HEAP_SIZE
NameMAX_HEAP_SIZE
DescriptionThe maximum heap memory size that IoTDB can use at startup.
TypeString
DefaultOn Linux or MacOS, the default is one quarter of the memory. On Windows, the default value for 32-bit systems is 512M, and the default for 64-bit systems is 2G.
EffectiveAfter restart system
  • HEAP_NEWSIZE
NameHEAP_NEWSIZE
DescriptionThe minimum heap memory size that IoTDB can use at startup.
TypeString
DefaultOn Linux or MacOS, the default is min{cores * 100M, one quarter of MAX_HEAP_SIZE}. On Windows, the default value for 32-bit systems is 512M, and the default for 64-bit systems is 2G.
EffectiveAfter restart system
  • JMX_LOCAL
NameJMX_LOCAL
DescriptionJMX monitoring mode, configured as yes to allow only local monitoring, no to allow remote monitoring
TypeEnum String: “true”, “false”
Defaulttrue
EffectiveAfter restart system
  • JMX_PORT
NameJMX_PORT
DescriptionJMX listening port. Please confirm that the port is not a system reserved port and is not occupied
TypeShort Int: [0,65535]
Default31999
EffectiveAfter restart system
  • JMX_IP
NameJMX_IP
DescriptionJMX listening address. Only take effect if JMX_LOCAL=false. 0.0.0.0 is never allowed
TypeString
Default127.0.0.1
EffectiveAfter restart system

JMX Authorization

We STRONGLY RECOMMENDED you CHANGE the PASSWORD for the JMX remote connection.

The user and passwords are in ${IOTDB_CONF}/conf/jmx.password.

The permission definitions are in ${IOTDB_CONF}/conf/jmx.access.

IoTDB System Configuration File

File Layer

  • compressor
Namecompressor
DescriptionData compression method
TypeEnum String : “UNCOMPRESSED”, “SNAPPY”
DefaultUNCOMPRESSED
EffectiveTrigger
  • group_size_in_byte
Namegroup_size_in_byte
DescriptionThe data size written to the disk per time
TypeInt32
Default134217728
EffectiveTrigger
  • page_size_in_byte
Namepage_size_in_byte
DescriptionThe maximum size of a single page written in memory when each column in memory is written (in bytes)
TypeInt32
Default65536
EffectiveTrigger
  • max_number_of_points_in_page
Namemax_number_of_points_in_page
DescriptionThe maximum number of data points (timestamps - valued groups) contained in a page
TypeInt32
Default1048576
EffectiveTrigger
  • max_degree_of_index_node
Namemax_degree_of_index_node
DescriptionThe maximum degree of the metadata index tree (that is, the max number of each node’s children)
TypeInt32
Default256
EffectiveOnly allowed to be modified in first start up
  • max_string_length
Namemax_string_length
DescriptionThe maximum length of a single string (number of character)
TypeInt32
Default128
EffectiveTrigger
  • time_series_data_type
Nametime_series_data_type
DescriptionTimestamp data type
TypeEnum String: “INT32”, “INT64”
DefaultInt64
EffectiveTrigger
  • time_encoder
Nametime_encoder
DescriptionEncoding type of time column
TypeEnum String: “TS_2DIFF”,“PLAIN”,“RLE”
DefaultTS_2DIFF
EffectiveTrigger
  • value_encoder
Namevalue_encoder
DescriptionEncoding type of value column
TypeEnum String: “TS_2DIFF”,“PLAIN”,“RLE”
DefaultPLAIN
EffectiveTrigger
  • float_precision
Namefloat_precision
DescriptionThe precision of the floating point number.(The number of digits after the decimal point)
TypeInt32
DefaultThe default is 2 digits. Note: The 32-bit floating point number has a decimal precision of 7 bits, and the 64-bit floating point number has a decimal precision of 15 bits. If the setting is out of the range, it will have no practical significance.
EffectiveTrigger
  • bloomFilterErrorRate
NamebloomFilterErrorRate
DescriptionThe false positive rate of bloom filter in each TsFile. Bloom filter checks whether a given time series is in the tsfile before loading metadata. This can improve the performance of loading metadata and skip the tsfile that doesn’t contain specified time series. If you want to learn more about its mechanism, you can refer to: wiki page of bloom filterConfig Manual - 图1 (opens new window).
Typefloat, (0, 1)
Default0.05
EffectiveAfter restart system

Engine Layer

  • rpc_address
Namerpc_address
DescriptionThe jdbc service listens on the address.
TypeString
Default“0.0.0.0”
EffectiveAfter restart system
  • rpc_port
Namerpc_port
DescriptionThe jdbc service listens on the port. Please confirm that the port is not a system reserved port and is not occupied.
TypeShort Int : [0,65535]
Default6667
EffectiveAfter restart system
  • rpc_thrift_compression_enable
Namerpc_thrift_compression_enable
DescriptionWhether enable thrift’s compression (using GZIP).
TypeBoolean
Defaultfalse
EffectiveAfter restart system
  • rpc_advanced_compression_enable
Namerpc_advanced_compression_enable
DescriptionWhether enable thrift’s advanced compression.
TypeBoolean
Defaultfalse
EffectiveAfter restart system
  • time_zone
Nametime_zone
DescriptionThe time zone in which the server is located, the default is Beijing time (+8)
TypeTime Zone String
Default+08:00
EffectiveTrigger
  • base_dir
Namebase_dir
DescriptionThe IoTDB system folder. It is recommended to use an absolute path.
TypeString
Defaultdata
EffectiveAfter restart system
  • data_dirs
Namedata_dirs
DescriptionThe directories of data files. Multiple directories are separated by comma. The starting directory of the relative path is related to the operating system. It is recommended to use an absolute path. If the path does not exist, the system will automatically create it.
TypeString[]
Defaultdata/data
EffectiveTrigger
  • wal_dir
Namewal_dir
DescriptionWrite Ahead Log storage path. It is recommended to use an absolute path.
TypeString
Defaultdata/wal
EffectiveAfter restart system
  • enable_wal
Nameenable_wal
DescriptionWhether to enable the pre-write log. The default value is true(enabled), and false means closed.
TypeBool
Defaulttrue
EffectiveTrigger
  • enable_mem_control
Nameenable_mem_control
Descriptionenable memory control to avoid OOM
TypeBool
Defaulttrue
EffectiveAfter restart system
  • memtable_size_threshold
Namememtable_size_threshold
Descriptionmax memtable size
TypeLong
Default1073741824
Effectivewhen enable_mem_control is false & After restart system
  • avg_series_point_number_threshold
Nameavg_series_point_number_threshold
Descriptionmax average number of point of each series in memtable
TypeInt32
Default10000
EffectiveAfter restart system
  • tsfile_size_threshold
Nametsfile_size_threshold
Descriptionmax tsfile size
TypeLong
Default536870912
EffectiveAfter restart system
  • enable_partition
Nameenable_partition
DescriptionWhether enable time partition for data, if disabled, all data belongs to partition 0
TypeBool
Defaultfalse
EffectiveOnly allowed to be modified in first start up
  • partition_interval
Namepartition_interval
DescriptionTime range for dividing storage group, time series data will be divided into groups by this time range
TypeInt64
Default604800
EffectiveOnly allowed to be modified in first start up
  • concurrent_writing_time_partition
Nameconcurrent_writing_time_partition
DescriptionThis config decides how many time partitions in a storage group can be inserted concurrently
For example, your partitionInterval is 86400 and you want to insert data in 5 different days,
TypeInt32
Default1
EffectiveAfter restart system
  • multi_dir_strategy
Namemulti_dir_strategy
DescriptionIoTDB’s strategy for selecting directories for TsFile in tsfile_dir. You can use a simple class name or a full name of the class. The system provides the following three strategies:
1. SequenceStrategy: IoTDB selects the directory from tsfile_dir in order, traverses all the directories in tsfile_dir in turn, and keeps counting;
2. MaxDiskUsableSpaceFirstStrategy: IoTDB first selects the directory with the largest free disk space in tsfile_dir;
3. MinFolderOccupiedSpaceFirstStrategy: IoTDB prefers the directory with the least space used in tsfile_dir;
4. UserDfineStrategyPackage (user-defined policy)
You can complete a user-defined policy in the following ways:
1. Inherit the cn.edu.tsinghua.iotdb.conf.directories.strategy.DirectoryStrategy class and implement its own Strategy method;
2. Fill in the configuration class with the full class name of the implemented class (package name plus class name, UserDfineStrategyPackage);
3. Add the jar file to the project.
TypeString
DefaultMaxDiskUsableSpaceFirstStrategy
EffectiveTrigger
  • tsfile_size_threshold
Nametsfile_size_threshold
DescriptionWhen a TsFile size on the disk exceeds this threshold, the TsFile is closed and open a new TsFile to accept data writes. The unit is byte and the default value is 2G.
TypeInt64
Default536870912
EffectiveAfter restart system
  • tag_attribute_total_size
Nametag_attribute_total_size
DescriptionThe maximum persistence size of tags and attributes of each time series.
TypeInt32
Default700
EffectiveOnly allowed to be modified in first start up
  • enable_partial_insert
Nameenable_partial_insert
DescriptionWhether continue to write other measurements if some measurements are failed in one insertion.
TypeBool
Defaulttrue
EffectiveAfter restart system
  • mtree_snapshot_interval
Namemtree_snapshot_interval
DescriptionThe least interval line numbers of mlog.txt when creating a checkpoint and saving snapshot of MTree. Unit: line numbers
TypeInt32
Default100000
EffectiveAfter restart system
  • mlog_buffer_size
Namemlog_buffer_size
Descriptionsize of log buffer in each metadata operation plan(in byte)
TypeInt32
Default1048576
EffectiveAfter restart system
  • sync_mlog_period_in_ms
Namesync_mlog_period_in_ms
DescriptionThe cycle when metadata log is periodically forced to be written to disk(in milliseconds). If force_mlog_period_in_ms = 0 it means force metadata log to be written to disk after each refreshment
TypeInt64
Default100
EffectiveAfter restart system
  • flush_wal_threshold
Nameflush_wal_threshold
DescriptionAfter the WAL reaches this value, it is flushed to disk, and it is possible to lose at most flush_wal_threshold operations.
TypeInt32
Default10000
EffectiveTrigger
  • force_wal_period_in_ms
Nameforce_wal_period_in_ms
DescriptionThe period during which the log is periodically forced to flush to disk(in milliseconds)
TypeInt32
Default10
EffectiveTrigger
  • fetch_size
Namefetch_size
DescriptionThe amount of data read each time in batch (the number of data strips, that is, the number of different timestamps.)
TypeInt32
Default10000
EffectiveAfter restart system
  • merge_concurrent_threads
Namemerge_concurrent_threads
DescriptionTHe max threads which can be used when unsequence data is merged. The larger it is, the more IO and CPU cost. The smaller the value, the more the disk is occupied when the unsequence data is too large, the reading will be slower.
TypeInt32
Default0
EffectiveAfter restart system
  • enable_stat_monitor
Nameenable_stat_monitor
DescriptionWhether to enable background statistics
TypeBoolean
Defaultfalse
EffectiveAfter restart system
  • concurrent_flush_thread
Nameconcurrent_flush_thread
DescriptionThe thread number used to perform the operation when IoTDB writes data in memory to disk. If the value is less than or equal to 0, then the number of CPU cores installed on the machine is used. The default is 0.
TypeInt32
Default0
EffectiveAfter restart system
  • tsfile_storage_fs
Nametsfile_storage_fs
DescriptionThe storage file system of Tsfile and related data files. Currently LOCAL file system and HDFS are supported.
TypeString
DefaultLOCAL
EffectiveOnly allowed to be modified in first start up
  • core_site_path
Namecore_site_path
DescriptionAbsolute file path of core-site.xml if Tsfile and related data files are stored in HDFS.
TypeString
Default/etc/hadoop/conf/core-site.xml
EffectiveAfter restart system
  • hdfs_site_path
Namehdfs_site_path
DescriptionAbsolute file path of hdfs-site.xml if Tsfile and related data files are stored in HDFS.
TypeString
Default/etc/hadoop/conf/hdfs-site.xml
EffectiveAfter restart system
  • hdfs_ip
Namehdfs_ip
DescriptionIP of HDFS if Tsfile and related data files are stored in HDFS. If there are more than one hdfs_ip in configuration, Hadoop HA is used.
TypeString
Defaultlocalhost
EffectiveAfter restart system
  • hdfs_port
Namehdfs_port
DescriptionPort of HDFS if Tsfile and related data files are stored in HDFS
TypeString
Default9000
EffectiveAfter restart system
  • dfs_nameservices
Namehdfs_nameservices
DescriptionNameservices of HDFS HA if using Hadoop HA
TypeString
Defaulthdfsnamespace
EffectiveAfter restart system
  • dfs_ha_namenodes
Namehdfs_ha_namenodes
DescriptionNamenodes under DFS nameservices of HDFS HA if using Hadoop HA
TypeString
Defaultnn1,nn2
EffectiveAfter restart system
  • dfs_ha_automatic_failover_enabled
Namedfs_ha_automatic_failover_enabled
DescriptionWhether using automatic failover if using Hadoop HA
TypeBoolean
Defaulttrue
EffectiveAfter restart system
  • dfs_client_failover_proxy_provider
Namedfs_client_failover_proxy_provider
DescriptionProxy provider if using Hadoop HA and enabling automatic failover
TypeString
Defaultorg.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
EffectiveAfter restart system
  • hdfs_use_kerberos
Namehdfs_use_kerberos
DescriptionWhether use kerberos to authenticate hdfs
TypeString
Defaultfalse
EffectiveAfter restart system
  • kerberos_keytab_file_path
Namekerberos_keytab_file_path
DescriptionFull path of kerberos keytab file
TypeString
Default/path
EffectiveAfter restart system
  • kerberos_principal
Namekerberos_principal
DescriptionKerberos pricipal
TypeString
Defaultyour principal
EffectiveAfter restart system
  • authorizer_provider_class
Nameauthorizer_provider_class
Descriptionthe class name of the authorization service
TypeString
Defaultorg.apache.iotdb.db.auth.authorizer.LocalFileAuthorizer
EffectiveAfter restart system
Other available valuesorg.apache.iotdb.db.auth.authorizer.OpenIdAuthorizer
  • openID_url
NameopenID_url
Descriptionthe openID server if OpenIdAuthorizer is enabled
TypeString (a http url)
Defaultno
EffectiveAfter restart system
  • thrift_max_frame_size
Namethrift_max_frame_size
Descriptionthe max bytes in a RPC request/response
Typelong
Default67108864 (should >= 8 1024 1024)
EffectiveAfter restart system

Automatic Schema Creation and Type Inference

  • enable_auto_create_schema
Nameenable_auto_create_schema
Descriptionwhether auto create the time series when a non-existed time series data comes
Typetrue or false
Defaulttrue
EffectiveAfter restart system
  • default_storage_group_level
Namedefault_storage_group_level
DescriptionStorage group level when creating schema automatically is enabled. For example, if we receives a data point from root.sg0.d1.s2, we will set root.sg0 as the storage group if storage group level is 1. (root is level 0)
Typeinteger
Default1
EffectiveAfter restart system
  • boolean_string_infer_type
Nameboolean_string_infer_type
DescriptionTo which type the values “true” and “false” should be reslved
TypeBOOLEAN or TEXT
DefaultBOOLEAN
EffectiveAfter restart system
  • integer_string_infer_type
Nameinteger_string_infer_type
DescriptionTo which type an integer string like “67” in a query should be resolved
TypeINT32, INT64, DOUBLE, FLOAT or TEXT
DefaultDOUBLE
EffectiveAfter restart system
  • nan_string_infer_type
Namenan_string_infer_type
DescriptionTo which type the value NaN in a query should be resolved
TypeDOUBLE, FLOAT or TEXT
DefaultFLOAT
EffectiveAfter restart system
  • floating_string_infer_type
Namefloating_string_infer_type
DescriptionTo which type a floating number string like “6.7” in a query should be resolved
TypeDOUBLE, FLOAT or TEXT
DefaultFLOAT
EffectiveAfter restart system
  • enable_partition
Nameenable_partition
Descriptionwhether enable data partition. If disabled, all data belongs to partition 0
TypeBOOLEAN
Defaultfalse
EffectiveAfter restart system
  • partition_interval
Namepartition_interval
Descriptiontime range for partitioning data inside each storage group, the unit is second
TypeLONG
Default604800
EffectiveAfter restart system

Enable GC log

GC log is off by default. For performance tuning, you may want to collect the GC info.

To enable GC log, just add a parameter “printgc” when you start the server.

  1. nohup sbin/start-server.sh printgc >/dev/null 2>&1 &

Or

  1. sbin\start-server.bat printgc

GC log is stored at IOTDB_HOME/logs/gc.log. There will be at most 10 gc.log.* files and each one can reach to 10MB.