ClickHouse release v20.1

ClickHouse release v20.1.2.4, 2020-01-22

Backward Incompatible Change

• Make the setting merge_tree_uniform_read_distribution obsolete. The server still recognizes this setting but it has no effect. #8308 (alexey-milovidov)
• Changed return type of the function greatCircleDistance to Float32 because now the result of calculation is Float32. #7993 (alexey-milovidov)
• Now it’s expected that query parameters are represented in “escaped” format. For example, to pass string a<tab>b you have to write a\tb or a\<tab>b and respectively, a%5Ctb or a%5C%09b in URL. This is needed to add the possibility to pass NULL as \N. This fixes #7488. #8517 (alexey-milovidov)
• Enable use_minimalistic_part_header_in_zookeeper setting for ReplicatedMergeTree by default. This will significantly reduce amount of data stored in ZooKeeper. This setting is supported since version 19.1 and we already use it in production in multiple services without any issues for more than half a year. Disable this setting if you have a chance to downgrade to versions older than 19.1. #6850 (alexey-milovidov)
• Data skipping indices are production ready and enabled by default. The settings allow_experimental_data_skipping_indices, allow_experimental_cross_to_join_conversion and allow_experimental_multiple_joins_emulation are now obsolete and do nothing. #7974 (alexey-milovidov)
• Add new ANY JOIN logic for StorageJoin consistent with JOIN operation. To upgrade without changes in behaviour you need add SETTINGS any_join_distinct_right_table_keys = 1 to Engine Join tables metadata or recreate these tables after upgrade. #8400 (Artem Zuikov)
• Require server to be restarted to apply the changes in logging configuration. This is a temporary workaround to avoid the bug where the server logs to a deleted log file (see #8696). #8707 (Alexander Kuzmenkov)

New Feature

• Added information about part paths to system.merges. #8043 (Vladimir Chebotarev)
• Add ability to execute SYSTEM RELOAD DICTIONARY query in ON CLUSTER mode. #8288 (Guillaume Tassery)
• Add ability to execute CREATE DICTIONARY queries in ON CLUSTER mode. #8163 (alesapin)
• Now user’s profile in users.xml can inherit multiple profiles. #8343 (Mikhail f. Shiryaev)
• Added system.stack_trace table that allows to look at stack traces of all server threads. This is useful for developers to introspect server state. This fixes #7576. #8344 (alexey-milovidov)
• Add DateTime64 datatype with configurable sub-second precision. #7170 (Vasily Nemkov)
• Add table function clusterAllReplicas which allows to query all the nodes in the cluster. #8493 (kiran sunkari)
• Add aggregate function categoricalInformationValue which calculates the information value of a discrete feature. #8117 (hcz)
• Speed up parsing of data files in CSV, TSV and JSONEachRow format by doing it in parallel. #7780 (Alexander Kuzmenkov)
• Add function bankerRound which performs banker’s rounding. #8112 (hcz)
• Support more languages in embedded dictionary for region names: ‘ru’, ‘en’, ‘ua’, ‘uk’, ‘by’, ‘kz’, ‘tr’, ‘de’, ‘uz’, ‘lv’, ‘lt’, ‘et’, ‘pt’, ‘he’, ‘vi’. #8189 (alexey-milovidov)
• Improvements in consistency of ANY JOIN logic. Now t1 ANY LEFT JOIN t2 equals t2 ANY RIGHT JOIN t1. #7665 (Artem Zuikov)
• Add setting any_join_distinct_right_table_keys which enables old behaviour for ANY INNER JOIN. #7665 (Artem Zuikov)
• Add new SEMI and ANTI JOIN. Old ANY INNER JOIN behaviour now available as SEMI LEFT JOIN. #7665 (Artem Zuikov)
• Added Distributed format for File engine and file table function which allows to read from .bin files generated by asynchronous inserts into Distributed table. #8535 (Nikolai Kochetov)
• Add optional reset column argument for runningAccumulate which allows to reset aggregation results for each new key value. #8326 (Sergey Kononenko)
• Add ability to use ClickHouse as Prometheus endpoint. #7900 (vdimir)
• Add section <remote_url_allow_hosts> in config.xml which restricts allowed hosts for remote table engines and table functions URL, S3, HDFS. #7154 (Mikhail Korotov)
• Added function greatCircleAngle which calculates the distance on a sphere in degrees. #8105 (alexey-milovidov)
• Changed Earth radius to be consistent with H3 library. #8105 (alexey-milovidov)
• Added JSONCompactEachRow and JSONCompactEachRowWithNamesAndTypes formats for input and output. #7841 (Mikhail Korotov)
• Added feature for file-related table engines and table functions (File, S3, URL, HDFS) which allows to read and write gzip files based on additional engine parameter or file extension. #7840 (Andrey Bodrov)
• Added the randomASCII(length) function, generating a string with a random set of ASCII printable characters. #8401 (BayoNet)
• Added function JSONExtractArrayRaw which returns an array on unparsed json array elements from JSON string. #8081 (Oleg Matrokhin)
• Add arrayZip function which allows to combine multiple arrays of equal lengths into one array of tuples. #8149 (Winter Zhang)
• Add ability to move data between disks according to configured TTL-expressions for *MergeTree table engines family. #8140 (Vladimir Chebotarev)
• Added new aggregate function avgWeighted which allows to calculate weighted average. #7898 (Andrey Bodrov)
• Now parallel parsing is enabled by default for TSV, TSKV, CSV and JSONEachRow formats. #7894 (Nikita Mikhaylov)
• Add several geo functions from H3 library: h3GetResolution, h3EdgeAngle, h3EdgeLength, h3IsValid and h3kRing. #8034 (Konstantin Malanchev)
• Added support for brotli (br) compression in file-related storages and table functions. This fixes #8156. #8526 (alexey-milovidov)
• Add groupBit* functions for the SimpleAggregationFunction type. #8485 (Guillaume Tassery)

Bug Fix

• Fix rename of tables with Distributed engine. Fixes issue #7868. #8306 (tavplubix)
• Now dictionaries support EXPRESSION for attributes in arbitrary string in non-ClickHouse SQL dialect. #8098 (alesapin)
• Fix broken INSERT SELECT FROM mysql(...) query. This fixes #8070 and #7960. #8234 (tavplubix)
• Fix error “Mismatch column sizes” when inserting default Tuple from JSONEachRow. This fixes #5653. #8606 (tavplubix)
• Now an exception will be thrown in case of using WITH TIES alongside LIMIT BY. Also add ability to use TOP with LIMIT BY. This fixes #7472. #7637 (Nikita Mikhaylov)
• Fix unintendent dependency from fresh glibc version in clickhouse-odbc-bridge binary. #8046 (Amos Bird)
• Fix bug in check function of *MergeTree engines family. Now it doesn’t fail in case when we have equal amount of rows in last granule and last mark (non-final). #8047 (alesapin)
• Fix insert into Enum* columns after ALTER query, when underlying numeric type is equal to table specified type. This fixes #7836. #7908 (Anton Popov)
• Allowed non-constant negative “size” argument for function substring. It was not allowed by mistake. This fixes #4832. #7703 (alexey-milovidov)
• Fix parsing bug when wrong number of arguments passed to (O|J)DBC table engine. #7709 (alesapin)
• Using command name of the running clickhouse process when sending logs to syslog. In previous versions, empty string was used instead of command name. #8460 (Michael Nacharov)
• Fix check of allowed hosts for localhost. This PR fixes the solution provided in #8241. #8342 (Vitaly Baranov)
• Fix rare crash in argMin and argMax functions for long string arguments, when result is used in runningAccumulate function. This fixes #8325 #8341 (dinosaur)
• Fix memory overcommit for tables with Buffer engine. #8345 (Azat Khuzhin)
• Fixed potential bug in functions that can take NULL as one of the arguments and return non-NULL. #8196 (alexey-milovidov)
• Better metrics calculations in thread pool for background processes for MergeTree table engines. #8194 (Vladimir Chebotarev)
• Fix function IN inside WHERE statement when row-level table filter is present. Fixes #6687 #8357 (Ivan)
• Now an exception is thrown if the integral value is not parsed completely for settings values. #7678 (Mikhail Korotov)
• Fix exception when aggregate function is used in query to distributed table with more than two local shards. #8164 (小路)
• Now bloom filter can handle zero length arrays and doesn’t perform redundant calculations. #8242 (achimbab)
• Fixed checking if a client host is allowed by matching the client host to host_regexp specified in users.xml. #8241 (Vitaly Baranov)
• Relax ambiguous column check that leads to false positives in multiple JOIN ON section. #8385 (Artem Zuikov)
• Fixed possible server crash (std::terminate) when the server cannot send or write data in JSON or XML format with values of String data type (that require UTF-8 validation) or when compressing result data with Brotli algorithm or in some other rare cases. This fixes #7603 #8384 (alexey-milovidov)
• Fix race condition in StorageDistributedDirectoryMonitor found by CI. This fixes #8364. #8383 (Nikolai Kochetov)
• Now background merges in *MergeTree table engines family preserve storage policy volume order more accurately. #8549 (Vladimir Chebotarev)
• Now table engine Kafka works properly with Native format. This fixes #6731 #7337 #8003. #8016 (filimonov)
• Fixed formats with headers (like CSVWithNames) which were throwing exception about EOF for table engine Kafka. #8016 (filimonov)
• Fixed a bug with making set from subquery in right part of IN section. This fixes #5767 and #2542. #7755 (Nikita Mikhaylov)
• Fix possible crash while reading from storage File. #7756 (Nikolai Kochetov)
• Fixed reading of the files in Parquet format containing columns of type list. #8334 (maxulan)
• Fix error Not found column for distributed queries with PREWHERE condition dependent on sampling key if max_parallel_replicas > 1. #7913 (Nikolai Kochetov)
• Fix error Not found column if query used PREWHERE dependent on table’s alias and the result set was empty because of primary key condition. #7911 (Nikolai Kochetov)
• Fixed return type for functions rand and randConstant in case of Nullable argument. Now functions always return UInt32 and never Nullable(UInt32). #8204 (Nikolai Kochetov)
• Disabled predicate push-down for WITH FILL expression. This fixes #7784. #7789 (Winter Zhang)
• Fixed incorrect count() result for SummingMergeTree when FINAL section is used. #3280 #7786 (Nikita Mikhaylov)
• Fix possible incorrect result for constant functions from remote servers. It happened for queries with functions like version(), uptime(), etc. which returns different constant values for different servers. This fixes #7666. #7689 (Nikolai Kochetov)
• Fix complicated bug in push-down predicate optimization which leads to wrong results. This fixes a lot of issues on push-down predicate optimization. #8503 (Winter Zhang)
• Fix crash in CREATE TABLE .. AS dictionary query. #8508 (Azat Khuzhin)
• Several improvements ClickHouse grammar in .g4 file. #8294 (taiyang-li)
• Fix bug that leads to crashes in JOINs with tables with engine Join. This fixes #7556 #8254 #7915 #8100. #8298 (Artem Zuikov)
• Fix redundant dictionaries reload on CREATE DATABASE. #7916 (Azat Khuzhin)
• Limit maximum number of streams for read from StorageFile and StorageHDFS. Fixes https://github.com/ClickHouse/ClickHouse/issues/7650. #7981 (alesapin)
• Fix bug in ALTER ... MODIFY ... CODEC query, when user specify both default expression and codec. Fixes 8593. #8614 (alesapin)
• Fix error in background merge of columns with SimpleAggregateFunction(LowCardinality) type. #8613 (Nikolai Kochetov)
• Fixed type check in function toDateTime64. #8375 (Vasily Nemkov)
• Now server do not crash on LEFT or FULL JOIN with and Join engine and unsupported join_use_nulls settings. #8479 (Artem Zuikov)
• Now DROP DICTIONARY IF EXISTS db.dict query doesn’t throw exception if db doesn’t exist. #8185 (Vitaly Baranov)
• Fix possible crashes in table functions (file, mysql, remote) caused by usage of reference to removed IStorage object. Fix incorrect parsing of columns specified at insertion into table function. #7762 (tavplubix)
• Ensure network be up before starting clickhouse-server. This fixes #7507. #8570 (Zhichang Yu)
• Fix timeouts handling for secure connections, so queries doesn’t hang indefenitely. This fixes #8126. #8128 (alexey-milovidov)
• Fix clickhouse-copier‘s redundant contention between concurrent workers. #7816 (Ding Xiang Fei)
• Now mutations doesn’t skip attached parts, even if their mutation version were larger than current mutation version. #7812 (Zhichang Yu) #8250 (alesapin)
• Ignore redundant copies of *MergeTree data parts after move to another disk and server restart. #7810 (Vladimir Chebotarev)
• Fix crash in FULL JOIN with LowCardinality in JOIN key. #8252 (Artem Zuikov)
• Forbidden to use column name more than once in insert query like INSERT INTO tbl (x, y, x). This fixes #5465, #7681. #7685 (alesapin)
• Added fallback for detection the number of physical CPU cores for unknown CPUs (using the number of logical CPU cores). This fixes #5239. #7726 (alexey-milovidov)
• Fix There's no column error for materialized and alias columns. #8210 (Artem Zuikov)
• Fixed sever crash when EXISTS query was used without TABLE or DICTIONARY qualifier. Just like EXISTS t. This fixes #8172. This bug was introduced in version 19.17. #8213 (alexey-milovidov)
• Fix rare bug with error "Sizes of columns doesn't match" that might appear when using SimpleAggregateFunction column. #7790 (Boris Granveaud)
• Fix bug where user with empty allow_databases got access to all databases (and same for allow_dictionaries). #7793 (DeifyTheGod)
• Fix client crash when server already disconnected from client. #8071 (Azat Khuzhin)
• Fix ORDER BY behaviour in case of sorting by primary key prefix and non primary key suffix. #7759 (Anton Popov)
• Check if qualified column present in the table. This fixes #6836. #7758 (Artem Zuikov)
• Fixed behavior with ALTER MOVE ran immediately after merge finish moves superpart of specified. Fixes #8103. #8104 (Vladimir Chebotarev)
• Fix possible server crash while using UNION with different number of columns. Fixes #7279. #7929 (Nikolai Kochetov)
• Fix size of result substring for function substr with negative size. #8589 (Nikolai Kochetov)
• Now server does not execute part mutation in MergeTree if there are not enough free threads in background pool. #8588 (tavplubix)
• Fix a minor typo on formatting UNION ALL AST. #7999 (litao91)
• Fixed incorrect bloom filter results for negative numbers. This fixes #8317. #8566 (Winter Zhang)
• Fixed potential buffer overflow in decompress. Malicious user can pass fabricated compressed data that will cause read after buffer. This issue was found by Eldar Zaitov from Yandex information security team. #8404 (alexey-milovidov)
• Fix incorrect result because of integers overflow in arrayIntersect. #7777 (Nikolai Kochetov)
• Now OPTIMIZE TABLE query will not wait for offline replicas to perform the operation. #8314 (javi santana)
• Fixed ALTER TTL parser for Replicated*MergeTree tables. #8318 (Vladimir Chebotarev)
• Fix communication between server and client, so server read temporary tables info after query failure. #8084 (Azat Khuzhin)
• Fix bitmapAnd function error when intersecting an aggregated bitmap and a scalar bitmap. #8082 (Yue Huang)
• Refine the definition of ZXid according to the ZooKeeper Programmer’s Guide which fixes bug in clickhouse-cluster-copier. #8088 (Ding Xiang Fei)
• odbc table function now respects external_table_functions_use_nulls setting. #7506 (Vasily Nemkov)
• Fixed bug that lead to a rare data race. #8143 (Alexander Kazakov)
• Now SYSTEM RELOAD DICTIONARY reloads a dictionary completely, ignoring update_field. This fixes #7440. #8037 (Vitaly Baranov)
• Add ability to check if dictionary exists in create query. #8032 (alesapin)
• Fix Float* parsing in Values format. This fixes #7817. #7870 (tavplubix)
• Fix crash when we cannot reserve space in some background operations of *MergeTree table engines family. #7873 (Vladimir Chebotarev)
• Fix crash of merge operation when table contains SimpleAggregateFunction(LowCardinality) column. This fixes #8515. #8522 (Azat Khuzhin)
• Restore support of all ICU locales and add the ability to apply collations for constant expressions. Also add language name to system.collations table. #8051 (alesapin)
• Fix bug when external dictionaries with zero minimal lifetime (LIFETIME(MIN 0 MAX N), LIFETIME(N)) don’t update in background. #7983 (alesapin)
• Fix crash when external dictionary with ClickHouse source has subquery in query. #8351 (Nikolai Kochetov)
• Fix incorrect parsing of file extension in table with engine URL. This fixes #8157. #8419 (Andrey Bodrov)
• Fix CHECK TABLE query for *MergeTree tables without key. Fixes #7543. #7979 (alesapin)
• Fixed conversion of Float64 to MySQL type. #8079 (Yuriy Baranov)
• Now if table was not completely dropped because of server crash, server will try to restore and load it. #8176 (tavplubix)
• Fixed crash in table function file while inserting into file that doesn’t exist. Now in this case file would be created and then insert would be processed. #8177 (Olga Khvostikova)
• Fix rare deadlock which can happen when trace_log is in enabled. #7838 (filimonov)
• Add ability to work with different types besides Date in RangeHashed external dictionary created from DDL query. Fixes 7899. #8275 (alesapin)
• Fixes crash when now64() is called with result of another function. #8270 (Vasily Nemkov)
• Fixed bug with detecting client IP for connections through mysql wire protocol. #7743 (Dmitry Muzyka)
• Fix empty array handling in arraySplit function. This fixes #7708. #7747 (hcz)
• Fixed the issue when pid-file of another running clickhouse-server may be deleted. #8487 (Weiqing Xu)
• Fix dictionary reload if it has invalidate_query, which stopped updates and some exception on previous update tries. #8029 (alesapin)
• Fixed error in function arrayReduce that may lead to “double free” and error in aggregate function combinator Resample that may lead to memory leak. Added aggregate function aggThrow. This function can be used for testing purposes. #8446 (alexey-milovidov)

Improvement

• Improved logging when working with S3 table engine. #8251 (Grigory Pervakov)
• Printed help message when no arguments are passed when calling clickhouse-local. This fixes #5335. #8230 (Andrey Nagorny)
• Add setting mutations_sync which allows to wait ALTER UPDATE/DELETE queries synchronously. #8237 (alesapin)
• Allow to set up relative user_files_path in config.xml (in the way similar to format_schema_path). #7632 (hcz)
• Add exception for illegal types for conversion functions with -OrZero postfix. #7880 (Andrey Konyaev)
• Simplify format of the header of data sending to a shard in a distributed query. #8044 (Vitaly Baranov)
• Live View table engine refactoring. #8519 (vzakaznikov)
• Add additional checks for external dictionaries created from DDL-queries. #8127 (alesapin)
• Fix error Column ... already exists while using FINAL and SAMPLE together, e.g. select count() from table final sample 1/2. Fixes #5186. #7907 (Nikolai Kochetov)
• Now table the first argument of joinGet function can be table indentifier. #7707 (Amos Bird)
• Allow using MaterializedView with subqueries above Kafka tables. #8197 (filimonov)
• Now background moves between disks run it the seprate thread pool. #7670 (Vladimir Chebotarev)
• SYSTEM RELOAD DICTIONARY now executes synchronously. #8240 (Vitaly Baranov)
• Stack traces now display physical addresses (offsets in object file) instead of virtual memory addresses (where the object file was loaded). That allows the use of addr2line when binary is position independent and ASLR is active. This fixes #8360. #8387 (alexey-milovidov)
• Support new syntax for row-level security filters: <table name='table_name'>…</table>. Fixes #5779. #8381 (Ivan)
• Now cityHash function can work with Decimal and UUID types. Fixes #5184. #7693 (Mikhail Korotov)
• Removed fixed index granularity (it was 1024) from system logs because it’s obsolete after implementation of adaptive granularity. #7698 (alexey-milovidov)
• Enabled MySQL compatibility server when ClickHouse is compiled without SSL. #7852 (Yuriy Baranov)
• Now server checksums distributed batches, which gives more verbose errors in case of corrupted data in batch. #7914 (Azat Khuzhin)
• Support DROP DATABASE, DETACH TABLE, DROP TABLE and ATTACH TABLE for MySQL database engine. #8202 (Winter Zhang)
• Add authentication in S3 table function and table engine. #7623 (Vladimir Chebotarev)
• Added check for extra parts of MergeTree at different disks, in order to not allow to miss data parts at undefined disks. #8118 (Vladimir Chebotarev)
• Enable SSL support for Mac client and server. #8297 (Ivan)
• Now ClickHouse can work as MySQL federated server (see https://dev.mysql.com/doc/refman/5.7/en/federated-create-server.html). #7717 (Maxim Fedotov)
• clickhouse-client now only enable bracketed-paste when multiquery is on and multiline is off. This fixes (#7757)[https://github.com/ClickHouse/ClickHouse/issues/7757]. #7761 (Amos Bird)
• Support Array(Decimal) in if function. #7721 (Artem Zuikov)
• Support Decimals in arrayDifference, arrayCumSum and arrayCumSumNegative functions. #7724 (Artem Zuikov)
• Added lifetime column to system.dictionaries table. #6820 #7727 (kekekekule)
• Improved check for existing parts on different disks for *MergeTree table engines. Addresses #7660. #8440 (Vladimir Chebotarev)
• Integration with AWS SDK for S3 interactions which allows to use all S3 features out of the box. #8011 (Pavel Kovalenko)
• Added support for subqueries in Live View tables. #7792 (vzakaznikov)
• Check for using Date or DateTime column from TTL expressions was removed. #7920 (Vladimir Chebotarev)
• Information about disk was added to system.detached_parts table. #7833 (Vladimir Chebotarev)
• Now settings max_(table|partition)_size_to_drop can be changed without a restart. #7779 (Grigory Pervakov)
• Slightly better usability of error messages. Ask user not to remove the lines below Stack trace:. #7897 (alexey-milovidov)
• Better reading messages from Kafka engine in various formats after #7935. #8035 (Ivan)
• Better compatibility with MySQL clients which don’t support sha2_password auth plugin. #8036 (Yuriy Baranov)
• Support more column types in MySQL compatibility server. #7975 (Yuriy Baranov)
• Implement ORDER BY optimization for Merge, Buffer and Materilized View storages with underlying MergeTree tables. #8130 (Anton Popov)
• Now we always use POSIX implementation of getrandom to have better compatibility with old kernels (< 3.17). #7940 (Amos Bird)
• Better check for valid destination in a move TTL rule. #8410 (Vladimir Chebotarev)
• Better checks for broken insert batches for Distributed table engine. #7933 (Azat Khuzhin)
• Add column with array of parts name which mutations must process in future to system.mutations table. #8179 (alesapin)
• Parallel merge sort optimization for processors. #8552 (Nikolai Kochetov)
• The settings mark_cache_min_lifetime is now obsolete and does nothing. In previous versions, mark cache can grow in memory larger than mark_cache_size to accomodate data within mark_cache_min_lifetime seconds. That was leading to confusion and higher memory usage than expected, that is especially bad on memory constrained systems. If you will see performance degradation after installing this release, you should increase the mark_cache_size. #8484 (alexey-milovidov)
• Preparation to use tid everywhere. This is needed for #7477. #8276 (alexey-milovidov)

Performance Improvement

• Performance optimizations in processors pipeline. #7988 (Nikolai Kochetov)
• Non-blocking updates of expired keys in cache dictionaries (with permission to read old ones). #8303 (Nikita Mikhaylov)
• Compile ClickHouse without -fno-omit-frame-pointer globally to spare one more register. #8097 (Amos Bird)
• Speedup greatCircleDistance function and add performance tests for it. #7307 (Olga Khvostikova)
• Improved performance of function roundDown. #8465 (alexey-milovidov)
• Improved performance of max, min, argMin, argMax for DateTime64 data type. #8199 (Vasily Nemkov)
• Improved performance of sorting without a limit or with big limit and external sorting. #8545 (alexey-milovidov)
• Improved performance of formatting floating point numbers up to 6 times. #8542 (alexey-milovidov)
• Improved performance of modulo function. #7750 (Amos Bird)
• Optimized ORDER BY and merging with single column key. #8335 (alexey-milovidov)
• Better implementation for arrayReduce, -Array and -State combinators. #7710 (Amos Bird)
• Now PREWHERE should be optimized to be at least as efficient as WHERE. #7769 (Amos Bird)
• Improve the way round and roundBankers handling negative numbers. #8229 (hcz)
• Improved decoding performance of DoubleDelta and Gorilla codecs by roughly 30-40%. This fixes #7082. #8019 (Vasily Nemkov)
• Improved performance of base64 related functions. #8444 (alexey-milovidov)
• Added a function geoDistance. It is similar to greatCircleDistance but uses approximation to WGS-84 ellipsoid model. The performance of both functions are near the same. #8086 (alexey-milovidov)
• Faster min and max aggregation functions for Decimal data type. #8144 (Artem Zuikov)
• Vectorize processing arrayReduce. #7608 (Amos Bird)
• if chains are now optimized as multiIf. #8355 (kamalov-ruslan)
• Fix performance regression of Kafka table engine introduced in 19.15. This fixes #7261. #7935 (filimonov)
• Removed “pie” code generation that gcc from Debian packages occasionally brings by default. #8483 (alexey-milovidov)
• Parallel parsing data formats #6553 (Nikita Mikhaylov)
• Enable optimized parser of Values with expressions by default (input_format_values_deduce_templates_of_expressions=1). #8231 (tavplubix)

Build/Testing/Packaging Improvement

• Build fixes for ARM and in minimal mode. #8304 (proller)
• Add coverage file flush for clickhouse-server when std::atexit is not called. Also slightly improved logging in stateless tests with coverage. #8267 (alesapin)
• Update LLVM library in contrib. Avoid using LLVM from OS packages. #8258 (alexey-milovidov)
• Make bundled curl build fully quiet. #8232 #8203 (Pavel Kovalenko)
• Fix some MemorySanitizer warnings. #8235 (Alexander Kuzmenkov)
• Use add_warning and no_warning macros in CMakeLists.txt. #8604 (Ivan)
• Add support of Minio S3 Compatible object (https://min.io/) for better integration tests. #7863 #7875 (Pavel Kovalenko)
• Imported libc headers to contrib. It allows to make builds more consistent across various systems (only for x86_64-linux-gnu). #5773 (alexey-milovidov)
• Remove -fPIC from some libraries. #8464 (alexey-milovidov)
• Clean CMakeLists.txt for curl. See https://github.com/ClickHouse/ClickHouse/pull/8011#issuecomment-569478910 #8459 (alexey-milovidov)
• Silent warnings in CapNProto library. #8220 (alexey-milovidov)
• Add performance tests for short string optimized hash tables. #7679 (Amos Bird)
• Now ClickHouse will build on AArch64 even if MADV_FREE is not available. This fixes #8027. #8243 (Amos Bird)
• Update zlib-ng to fix memory sanitizer problems. #7182 #8206 (Alexander Kuzmenkov)
• Enable internal MySQL library on non-Linux system, because usage of OS packages is very fragile and usually doesn’t work at all. This fixes #5765. #8426 (alexey-milovidov)
• Fixed build on some systems after enabling libc++. This supersedes #8374. #8380 (alexey-milovidov)
• Make Field methods more type-safe to find more errors. #7386 #8209 (Alexander Kuzmenkov)
• Added missing files to the libc-headers submodule. #8507 (alexey-milovidov)
• Fix wrong JSON quoting in performance test output. #8497 (Nikolai Kochetov)
• Now stack trace is displayed for std::exception and Poco::Exception. In previous versions it was available only for DB::Exception. This improves diagnostics. #8501 (alexey-milovidov)
• Porting clock_gettime and clock_nanosleep for fresh glibc versions. #8054 (Amos Bird)
• Enable part_log in example config for developers. #8609 (alexey-milovidov)
• Fix async nature of reload in 01036_no_superfluous_dict_reload_on_create_database*. #8111 (Azat Khuzhin)
• Fixed codec performance tests. #8615 (Vasily Nemkov)
• Add install scripts for .tgz build and documentation for them. #8612 #8591 (alesapin)
• Removed old ZSTD test (it was created in year 2016 to reproduce the bug that pre 1.0 version of ZSTD has had). This fixes #8618. #8619 (alexey-milovidov)
• Fixed build on Mac OS Catalina. #8600 (meo)
• Increased number of rows in codec performance tests to make results noticeable. #8574 (Vasily Nemkov)
• In debug builds, treat LOGICAL_ERROR exceptions as assertion failures, so that they are easier to notice. #8475 (Alexander Kuzmenkov)
• Make formats-related performance test more deterministic. #8477 (alexey-milovidov)
• Update lz4 to fix a MemorySanitizer failure. #8181 (Alexander Kuzmenkov)
• Suppress a known MemorySanitizer false positive in exception handling. #8182 (Alexander Kuzmenkov)
• Update gcc and g++ to version 9 in build/docker/build.sh #7766 (TLightSky)
• Add performance test case to test that PREWHERE is worse than WHERE. #7768 (Amos Bird)
• Progress towards fixing one flacky test. #8621 (alexey-milovidov)
• Avoid MemorySanitizer report for data from libunwind. #8539 (alexey-milovidov)
• Updated libc++ to the latest version. #8324 (alexey-milovidov)
• Build ICU library from sources. This fixes #6460. #8219 (alexey-milovidov)
• Switched from libressl to openssl. ClickHouse should support TLS 1.3 and SNI after this change. This fixes #8171. #8218 (alexey-milovidov)
• Fixed UBSan report when using chacha20_poly1305 from SSL (happens on connect to https://yandex.ru/). #8214 (alexey-milovidov)
• Fix mode of default password file for .deb linux distros. #8075 (proller)
• Improved expression for getting clickhouse-server PID in clickhouse-test. #8063 (Alexander Kazakov)
• Updated contrib/googletest to v1.10.0. #8587 (Alexander Burmak)
• Fixed ThreadSaninitizer report in base64 library. Also updated this library to the latest version, but it doesn’t matter. This fixes #8397. #8403 (alexey-milovidov)
• Fix 00600_replace_running_query for processors. #8272 (Nikolai Kochetov)
• Remove support for tcmalloc to make CMakeLists.txt simpler. #8310 (alexey-milovidov)
• Release gcc builds now use libc++ instead of libstdc++. Recently libc++ was used only with clang. This will improve consistency of build configurations and portability. #8311 (alexey-milovidov)
• Enable ICU library for build with MemorySanitizer. #8222 (alexey-milovidov)
• Suppress warnings from CapNProto library. #8224 (alexey-milovidov)
• Removed special cases of code for tcmalloc, because it’s no longer supported. #8225 (alexey-milovidov)
• In CI coverage task, kill the server gracefully to allow it to save the coverage report. This fixes incomplete coverage reports we’ve been seeing lately. #8142 (alesapin)
• Performance tests for all codecs against Float64 and UInt64 values. #8349 (Vasily Nemkov)
• termcap is very much deprecated and lead to various problems (f.g. missing “up” cap and echoing ^J instead of multi line) . Favor terminfo or bundled ncurses. #7737 (Amos Bird)
• Fix test_storage_s3 integration test. #7734 (Nikolai Kochetov)
• Support StorageFile(<format>, null) to insert block into given format file without actually write to disk. This is required for performance tests. #8455 (Amos Bird)
• Added argument --print-time to functional tests which prints execution time per test. #8001 (Nikolai Kochetov)
• Added asserts to KeyCondition while evaluating RPN. This will fix warning from gcc-9. #8279 (alexey-milovidov)
• Dump cmake options in CI builds. #8273 (Alexander Kuzmenkov)
• Don’t generate debug info for some fat libraries. #8271 (alexey-milovidov)
• Make log_to_console.xml always log to stderr, regardless of is it interactive or not. #8395 (Alexander Kuzmenkov)
• Removed some unused features from clickhouse-performance-test tool. #8555 (alexey-milovidov)
• Now we will also search for lld-X with corresponding clang-X version. #8092 (alesapin)
• Parquet build improvement. #8421 (maxulan)
• More GCC warnings #8221 (kreuzerkrieg)
• Package for Arch Linux now allows to run ClickHouse server, and not only client. #8534 (Vladimir Chebotarev)
• Fix test with processors. Tiny performance fixes. #7672 (Nikolai Kochetov)
• Update contrib/protobuf. #8256 (Matwey V. Kornilov)
• In preparation of switching to c++20 as a new year celebration. “May the C++ force be with ClickHouse.” #8447 (Amos Bird)

Experimental Feature

• Added experimental setting min_bytes_to_use_mmap_io. It allows to read big files without copying data from kernel to userspace. The setting is disabled by default. Recommended threshold is about 64 MB, because mmap/munmap is slow. #8520 (alexey-milovidov)
• Reworked quotas as a part of access control system. Added new table system.quotas, new functions currentQuota, currentQuotaKey, new SQL syntax CREATE QUOTA, ALTER QUOTA, DROP QUOTA, SHOW QUOTA. #7257 (Vitaly Baranov)
• Allow skipping unknown settings with warnings instead of throwing exceptions. #7653 (Vitaly Baranov)
• Reworked row policies as a part of access control system. Added new table system.row_policies, new function currentRowPolicies(), new SQL syntax CREATE POLICY, ALTER POLICY, DROP POLICY, SHOW CREATE POLICY, SHOW POLICIES. #7808 (Vitaly Baranov)

Security Fix

• Fixed the possibility of reading directories structure in tables with File table engine. This fixes #8536. #8537 (alexey-milovidov)

ClickHouse release v19.17

ClickHouse release v19.17.6.36, 2019-12-27

Bug Fix

• Fixed potential buffer overflow in decompress. Malicious user can pass fabricated compressed data that could cause read after buffer. This issue was found by Eldar Zaitov from Yandex information security team. #8404 (alexey-milovidov)
• Fixed possible server crash (std::terminate) when the server cannot send or write data in JSON or XML format with values of String data type (that require UTF-8 validation) or when compressing result data with Brotli algorithm or in some other rare cases. #8384 (alexey-milovidov)
• Fixed dictionaries with source from a clickhouse VIEW, now reading such dictionaries doesn’t cause the error There is no query. #8351 (Nikolai Kochetov)
• Fixed checking if a client host is allowed by host_regexp specified in users.xml. #8241, #8342 (Vitaly Baranov)
• RENAME TABLE for a distributed table now renames the folder containing inserted data before sending to shards. This fixes an issue with successive renames tableA->tableB, tableC->tableA. #8306 (tavplubix)
• range_hashed external dictionaries created by DDL queries now allow ranges of arbitrary numeric types. #8275 (alesapin)
• Fixed INSERT INTO table SELECT ... FROM mysql(...) table function. #8234 (tavplubix)
• Fixed segfault in INSERT INTO TABLE FUNCTION file() while inserting into a file which doesn’t exist. Now in this case file would be created and then insert would be processed. #8177 (Olga Khvostikova)
• Fixed bitmapAnd error when intersecting an aggregated bitmap and a scalar bitmap. #8082 (Yue Huang)
• Fixed segfault when EXISTS query was used without TABLE or DICTIONARY qualifier, just like EXISTS t. #8213 (alexey-milovidov)
• Fixed return type for functions rand and randConstant in case of nullable argument. Now functions always return UInt32 and never Nullable(UInt32). #8204 (Nikolai Kochetov)
• Fixed DROP DICTIONARY IF EXISTS db.dict, now it doesn’t throw exception if db doesn’t exist. #8185 (Vitaly Baranov)
• If a table wasn’t completely dropped because of server crash, the server will try to restore and load it #8176 (tavplubix)
• Fixed a trivial count query for a distributed table if there are more than two shard local table. #8164 (小路)
• Fixed bug that lead to a data race in DB::BlockStreamProfileInfo::calculateRowsBeforeLimit() #8143 (Alexander Kazakov)
• Fixed ALTER table MOVE part executed immediately after merging the specified part, which could cause moving a part which the specified part merged into. Now it correctly moves the specified part. #8104 (Vladimir Chebotarev)
• Expressions for dictionaries can be specified as strings now. This is useful for calculation of attributes while extracting data from non-ClickHouse sources because it allows to use non-ClickHouse syntax for those expressions. #8098 (alesapin)
• Fixed a very rare race in clickhouse-copier because of an overflow in ZXid. #8088 (Ding Xiang Fei)
• Fixed the bug when after the query failed (due to “Too many simultaneous queries” for example) it would not read external tables info, and the
  next request would interpret this info as the beginning of the next query causing an error like Unknown packet from client. #8084 (Azat Khuzhin)
• Avoid null dereference after “Unknown packet X from server” #8071 (Azat Khuzhin)
• Restore support of all ICU locales, add the ability to apply collations for constant expressions and add language name to system.collations table. #8051 (alesapin)
• Number of streams for read from StorageFile and StorageHDFS is now limited, to avoid exceeding the memory limit. #7981 (alesapin)
• Fixed CHECK TABLE query for *MergeTree tables without key. #7979 (alesapin)
• Removed the mutation number from a part name in case there were no mutations. This removing improved the compatibility with older versions. #8250 (alesapin)
• Fixed the bug that mutations are skipped for some attached parts due to their data_version are larger than the table mutation version. #7812 (Zhichang Yu)
• Allow starting the server with redundant copies of parts after moving them to another device. #7810 (Vladimir Chebotarev)
• Fixed the error “Sizes of columns doesn’t match” that might appear when using aggregate function columns. #7790 (Boris Granveaud)
• Now an exception will be thrown in case of using WITH TIES alongside LIMIT BY. And now it’s possible to use TOP with LIMIT BY. #7637 (Nikita Mikhaylov)
• Fix dictionary reload if it has invalidate_query, which stopped updates and some exception on previous update tries. #8029 (alesapin)

ClickHouse release v19.17.4.11, 2019-11-22

Backward Incompatible Change

• Using column instead of AST to store scalar subquery results for better performance. Setting enable_scalar_subquery_optimization was added in 19.17 and it was enabled by default. It leads to errors like this during upgrade to 19.17.2 or 19.17.3 from previous versions. This setting was disabled by default in 19.17.4, to make possible upgrading from 19.16 and older versions without errors. #7392 (Amos Bird)

New Feature

• Add the ability to create dictionaries with DDL queries. #7360 (alesapin)
• Make bloom_filter type of index supporting LowCardinality and Nullable #7363 #7561 (Nikolai Kochetov)
• Add function isValidJSON to check that passed string is a valid json. #5910 #7293 (Vdimir)
• Implement arrayCompact function #7328 (Memo)
• Created function hex for Decimal numbers. It works like hex(reinterpretAsString()), but doesn’t delete last zero bytes. #7355 (Mikhail Korotov)
• Add arrayFill and arrayReverseFill functions, which replace elements by other elements in front/back of them in the array. #7380 (hcz)
• Add CRC32IEEE()/CRC64() support #7480 (Azat Khuzhin)
• Implement char function similar to one in mysql #7486 (sundyli)
• Add bitmapTransform function. It transforms an array of values in a bitmap to another array of values, the result is a new bitmap #7598 (Zhichang Yu)
• Implemented javaHashUTF16LE() function #7651 (achimbab)
• Add _shard_num virtual column for the Distributed engine #7624 (Azat Khuzhin)

Experimental Feature

• Support for processors (new query execution pipeline) in MergeTree. #7181 (Nikolai Kochetov)

Bug Fix

• Fix incorrect float parsing in Values #7817 #7870 (tavplubix)
• Fix rare deadlock which can happen when trace_log is enabled. #7838 (filimonov)
• Prevent message duplication when producing Kafka table has any MVs selecting from it #7265 (Ivan)
• Support for Array(LowCardinality(Nullable(String))) in IN. Resolves #7364 #7366 (achimbab)
• Add handling of SQL_TINYINT and SQL_BIGINT, and fix handling of SQL_FLOAT data source types in ODBC Bridge. #7491 (Denis Glazachev)
• Fix aggregation (avg and quantiles) over empty decimal columns #7431 (Andrey Konyaev)
• Fix INSERT into Distributed with MATERIALIZED columns #7377 (Azat Khuzhin)
• Make MOVE PARTITION work if some parts of partition are already on destination disk or volume #7434 (Vladimir Chebotarev)
• Fixed bug with hardlinks failing to be created during mutations in ReplicatedMergeTree in multi-disk configurations. #7558 (Vladimir Chebotarev)
• Fixed a bug with a mutation on a MergeTree when whole part remains unchanged and best space is being found on another disk #7602 (Vladimir Chebotarev)
• Fixed bug with keep_free_space_ratio not being read from disks configuration #7645 (Vladimir Chebotarev)
• Fix bug with table contains only Tuple columns or columns with complex paths. Fixes 7541. #7545 (alesapin)
• Do not account memory for Buffer engine in max_memory_usage limit #7552 (Azat Khuzhin)
• Fix final mark usage in MergeTree tables ordered by tuple(). In rare cases it could lead to Can't adjust last granule error while select. #7639 (Anton Popov)
• Fix bug in mutations that have predicate with actions that require context (for example functions for json), which may lead to crashes or strange exceptions. #7664 (alesapin)
• Fix mismatch of database and table names escaping in data/ and shadow/ directories #7575 (Alexander Burmak)
• Support duplicated keys in RIGHT|FULL JOINs, e.g. ON t.x = u.x AND t.x = u.y. Fix crash in this case. #7586 (Artem Zuikov)
• Fix Not found column <expression> in block when joining on expression with RIGHT or FULL JOIN. #7641 (Artem Zuikov)
• One more attempt to fix infinite loop in PrettySpace format #7591 (Olga Khvostikova)
• Fix bug in concat function when all arguments were FixedString of the same size. #7635 (alesapin)
• Fixed exception in case of using 1 argument while defining S3, URL and HDFS storages. #7618 (Vladimir Chebotarev)
• Fix scope of the InterpreterSelectQuery for views with query #7601 (Azat Khuzhin)

Improvement

• Nullable columns recognized and NULL-values handled correctly by ODBC-bridge #7402 (Vasily Nemkov)
• Write current batch for distributed send atomically #7600 (Azat Khuzhin)
• Throw an exception if we cannot detect table for column name in query. #7358 (Artem Zuikov)
• Add merge_max_block_size setting to MergeTreeSettings #7412 (Artem Zuikov)
• Queries with HAVING and without GROUP BY assume group by constant. So, SELECT 1 HAVING 1 now returns a result. #7496 (Amos Bird)
• Support parsing (X,) as tuple similar to python. #7501, #7562 (Amos Bird)
• Make range function behaviors almost like pythonic one. #7518 (sundyli)
• Add constraints columns to table system.settings #7553 (Vitaly Baranov)
• Better Null format for tcp handler, so that it’s possible to use select ignore(<expression>) from table format Null for perf measure via clickhouse-client #7606 (Amos Bird)
• Queries like CREATE TABLE ... AS (SELECT (1, 2)) are parsed correctly #7542 (hcz)

Performance Improvement

• The performance of aggregation over short string keys is improved. #6243 (Alexander Kuzmenkov, Amos Bird)
• Run another pass of syntax/expression analysis to get potential optimizations after constant predicates are folded. #7497 (Amos Bird)
• Use storage meta info to evaluate trivial SELECT count() FROM table; #7510 (Amos Bird, alexey-milovidov)
• Vectorize processing arrayReduce similar to Aggregator addBatch. #7608 (Amos Bird)
• Minor improvements in performance of Kafka consumption #7475 (Ivan)

Build/Testing/Packaging Improvement

• Add support for cross-compiling to the CPU architecture AARCH64. Refactor packager script. #7370 #7539 (Ivan)
• Unpack darwin-x86_64 and linux-aarch64 toolchains into mounted Docker volume when building packages #7534 (Ivan)
• Update Docker Image for Binary Packager #7474 (Ivan)
• Fixed compile errors on MacOS Catalina #7585 (Ernest Poletaev)
• Some refactoring in query analysis logic: split complex class into several simple ones. #7454 (Artem Zuikov)
• Fix build without submodules #7295 (proller)
• Better add_globs in CMake files #7418 (Amos Bird)
• Remove hardcoded paths in unwind target #7460 (Konstantin Podshumok)
• Allow to use mysql format without ssl #7524 (proller)

Other

• Added ANTLR4 grammar for ClickHouse SQL dialect #7595 #7596 (alexey-milovidov)

ClickHouse release v19.16

ClickHouse release v19.16.2.2, 2019-10-30

Backward Incompatible Change

• Add missing arity validation for count/counIf.
  #7095
  #7298 (Vdimir)
• Remove legacy asterisk_left_columns_only setting (it was disabled by default).
  #7335 (Artem
  Zuikov)
• Format strings for Template data format are now specified in files.
  #7118
  (tavplubix)

New Feature

• Introduce uniqCombined64() to calculate cardinality greater than UINT_MAX.
  #7213,
  #7222 (Azat
  Khuzhin)
• Support Bloom filter indexes on Array columns.
  #6984
  (achimbab)
• Add a function getMacro(name) that returns String with the value of corresponding <macros>
  from server configuration. #7240
  (alexey-milovidov)
• Set two configuration options for a dictionary based on an HTTP source: credentials and
  http-headers. #7092 (Guillaume
  Tassery)
• Add a new ProfileEvent Merge that counts the number of launched background merges.
  #7093 (Mikhail
  Korotov)
• Add fullHostName function that returns a fully qualified domain name.
  #7263
  #7291 (sundyli)
• Add function arraySplit and arrayReverseSplit which split an array by “cut off”
  conditions. They are useful in time sequence handling.
  #7294 (hcz)
• Add new functions that return the Array of all matched indices in multiMatch family of functions.
  #7299 (Danila
  Kutenin)
• Add a new database engine Lazy that is optimized for storing a large number of small -Log
  tables. #7171 (Nikita
  Vasilev)
• Add aggregate functions groupBitmapAnd, -Or, -Xor for bitmap columns. #7109 (Zhichang
  Yu)
• Add aggregate function combinators -OrNull and -OrDefault, which return null
  or default values when there is nothing to aggregate.
  #7331
  (hcz)
• Introduce CustomSeparated data format that supports custom escaping and
  delimiter rules. #7118
  (tavplubix)
• Support Redis as source of external dictionary. #4361 #6962 (comunodi, Anton
  Popov)

Bug Fix

• Fix wrong query result if it has WHERE IN (SELECT ...) section and optimize_read_in_order is
  used. #7371 (Anton
  Popov)
• Disabled MariaDB authentication plugin, which depends on files outside of project.
  #7140 (Yuriy
  Baranov)
• Fix exception Cannot convert column ... because it is constant but values of constants are different in source and result which could rarely happen when functions now(), today(),
  yesterday(), randConstant() are used.
  #7156 (Nikolai
  Kochetov)
• Fixed issue of using HTTP keep alive timeout instead of TCP keep alive timeout.
  #7351 (Vasily
  Nemkov)
• Fixed a segmentation fault in groupBitmapOr (issue #7109).
  #7289 (Zhichang
  Yu)
• For materialized views the commit for Kafka is called after all data were written.
  #7175 (Ivan)
• Fixed wrong duration_ms value in system.part_log table. It was ten times off.
  #7172 (Vladimir
  Chebotarev)
• A quick fix to resolve crash in LIVE VIEW table and re-enabling all LIVE VIEW tests.
  #7201
  (vzakaznikov)
• Serialize NULL values correctly in min/max indexes of MergeTree parts.
  #7234 (Alexander
  Kuzmenkov)
• Don’t put virtual columns to .sql metadata when table is created as CREATE TABLE AS.
  #7183 (Ivan)
• Fix segmentation fault in ATTACH PART query.
  #7185
  (alesapin)
• Fix wrong result for some queries given by the optimization of empty IN subqueries and empty
  INNER/RIGHT JOIN. #7284 (Nikolai
  Kochetov)
• Fixing AddressSanitizer error in the LIVE VIEW getHeader() method.
  #7271
  (vzakaznikov)

Improvement

• Add a message in case of queue_wait_max_ms wait takes place.
  #7390 (Azat
  Khuzhin)
• Made setting s3_min_upload_part_size table-level.
  #7059 (Vladimir
  Chebotarev)
• Check TTL in StorageFactory. #7304
  (sundyli)
• Squash left-hand blocks in partial merge join (optimization).
  #7122 (Artem
  Zuikov)
• Do not allow non-deterministic functions in mutations of Replicated table engines, because this
  can introduce inconsistencies between replicas.
  #7247 (Alexander
  Kazakov)
• Disable memory tracker while converting exception stack trace to string. It can prevent the loss
  of error messages of type Memory limit exceeded on server, which caused the Attempt to read after eof exception on client. #7264
  (Nikolai Kochetov)
• Miscellaneous format improvements. Resolves
  #6033,
  #2633,
  #6611,
  #6742
  #7215
  (tavplubix)
• ClickHouse ignores values on the right side of IN operator that are not convertible to the left
  side type. Make it work properly for compound types — Array and Tuple.
  #7283 (Alexander
  Kuzmenkov)
• Support missing inequalities for ASOF JOIN. It’s possible to join less-or-equal variant and strict
  greater and less variants for ASOF column in ON syntax.
  #7282 (Artem
  Zuikov)
• Optimize partial merge join. #7070
  (Artem Zuikov)
• Do not use more than 98K of memory in uniqCombined functions.
  #7236,
  #7270 (Azat
  Khuzhin)
• Flush parts of right-hand joining table on disk in PartialMergeJoin (if there is not enough
  memory). Load data back when needed. #7186
  (Artem Zuikov)

Performance Improvement

• Speed up joinGet with const arguments by avoiding data duplication.
  #7359 (Amos
  Bird)
• Return early if the subquery is empty.
  #7007 (小路)
• Optimize parsing of SQL expression in Values.
  #6781
  (tavplubix)

Build/Testing/Packaging Improvement

• Disable some contribs for cross-compilation to Mac OS.
  #7101 (Ivan)
• Add missing linking with PocoXML for clickhouse_common_io.
  #7200 (Azat
  Khuzhin)
• Accept multiple test filter arguments in clickhouse-test.
  #7226 (Alexander
  Kuzmenkov)
• Enable musl and jemalloc for ARM. #7300
  (Amos Bird)
• Added --client-option parameter to clickhouse-test to pass additional parameters to client.
  #7277 (Nikolai
  Kochetov)
• Preserve existing configs on rpm package upgrade.
  #7103
  (filimonov)
• Fix errors detected by PVS. #7153 (Artem
  Zuikov)
• Fix build for Darwin. #7149
  (Ivan)
• glibc 2.29 compatibility. #7142 (Amos
  Bird)
• Make sure dh_clean does not touch potential source files.
  #7205 (Amos
  Bird)
• Attempt to avoid conflict when updating from altinity rpm - it has config file packaged separately
  in clickhouse-server-common. #7073
  (filimonov)
• Optimize some header files for faster rebuilds.
  #7212,
  #7231 (Alexander
  Kuzmenkov)
• Add performance tests for Date and DateTime. #7332 (Vasily
  Nemkov)
• Fix some tests that contained non-deterministic mutations.
  #7132 (Alexander
  Kazakov)
• Add build with MemorySanitizer to CI. #7066
  (Alexander Kuzmenkov)
• Avoid use of uninitialized values in MetricsTransmitter.
  #7158 (Azat
  Khuzhin)
• Fix some issues in Fields found by MemorySanitizer.
  #7135,
  #7179 (Alexander
  Kuzmenkov), #7376
  (Amos Bird)
• Fix undefined behavior in murmurhash32. #7388 (Amos
  Bird)
• Fix undefined behavior in StoragesInfoStream. #7384
  (tavplubix)
• Fixed constant expressions folding for external database engines (MySQL, ODBC, JDBC). In previous
  versions it wasn’t working for multiple constant expressions and was not working at all for Date,
  DateTime and UUID. This fixes #7245
  #7252
  (alexey-milovidov)
• Fixing ThreadSanitizer data race error in the LIVE VIEW when accessing no_users_thread variable.
  #7353
  (vzakaznikov)
• Get rid of malloc symbols in libcommon
  #7134,
  #7065 (Amos
  Bird)
• Add global flag ENABLE_LIBRARIES for disabling all libraries.
  #7063
  (proller)

Code cleanup

• Generalize configuration repository to prepare for DDL for Dictionaries. #7155
  (alesapin)
• Parser for dictionaries DDL without any semantic.
  #7209
  (alesapin)
• Split ParserCreateQuery into different smaller parsers.
  #7253
  (alesapin)
• Small refactoring and renaming near external dictionaries.
  #7111
  (alesapin)
• Refactor some code to prepare for role-based access control. #7235 (Vitaly
  Baranov)
• Some improvements in DatabaseOrdinary code.
  #7086 (Nikita
  Vasilev)
• Do not use iterators in find() and emplace() methods of hash tables.
  #7026 (Alexander
  Kuzmenkov)
• Fix getMultipleValuesFromConfig in case when parameter root is not empty. #7374
  (Mikhail Korotov)
• Remove some copy-paste (TemporaryFile and TemporaryFileStream)
  #7166 (Artem
  Zuikov)
• Improved code readability a little bit (MergeTreeData::getActiveContainingPart).
  #7361 (Vladimir
  Chebotarev)
• Wait for all scheduled jobs, which are using local objects, if ThreadPool::schedule(...) throws
  an exception. Rename ThreadPool::schedule(...) to ThreadPool::scheduleOrThrowOnError(...) and
  fix comments to make obvious that it may throw.
  #7350
  (tavplubix)

ClickHouse release 19.15

ClickHouse release 19.15.4.10, 2019-10-31

Bug Fix

• Added handling of SQL_TINYINT and SQL_BIGINT, and fix handling of SQL_FLOAT data source types in ODBC Bridge.
  #7491 (Denis Glazachev)
• Allowed to have some parts on destination disk or volume in MOVE PARTITION.
  #7434 (Vladimir Chebotarev)
• Fixed NULL-values in nullable columns through ODBC-bridge.
  #7402 (Vasily Nemkov)
• Fixed INSERT into Distributed non local node with MATERIALIZED columns.
  #7377 (Azat Khuzhin)
• Fixed function getMultipleValuesFromConfig.
  #7374 (Mikhail Korotov)
• Fixed issue of using HTTP keep alive timeout instead of TCP keep alive timeout.
  #7351 (Vasily Nemkov)
• Wait for all jobs to finish on exception (fixes rare segfaults).
  #7350 (tavplubix)
• Don’t push to MVs when inserting into Kafka table.
  #7265 (Ivan)
• Disable memory tracker for exception stack.
  #7264 (Nikolai Kochetov)
• Fixed bad code in transforming query for external database.
  #7252 (alexey-milovidov)
• Avoid use of uninitialized values in MetricsTransmitter.
  #7158 (Azat Khuzhin)
• Added example config with macros for tests (alexey-milovidov)

ClickHouse release 19.15.3.6, 2019-10-09

Bug Fix

• Fixed bad_variant in hashed dictionary.
  (alesapin)
• Fixed up bug with segmentation fault in ATTACH PART query.
  (alesapin)
• Fixed time calculation in MergeTreeData.
  (Vladimir Chebotarev)
• Commit to Kafka explicitly after the writing is finalized.
  #7175 (Ivan)
• Serialize NULL values correctly in min/max indexes of MergeTree parts.
  #7234 (Alexander Kuzmenkov)

ClickHouse release 19.15.2.2, 2019-10-01

New Feature

• Tiered storage: support to use multiple storage volumes for tables with MergeTree engine. It’s possible to store fresh data on SSD and automatically move old data to HDD. (example). #4918 (Igr) #6489 (alesapin)
• Add table function input for reading incoming data in INSERT SELECT query. #5450 (palasonic1) #6832 (Anton Popov)
• Add a sparse_hashed dictionary layout, that is functionally equivalent to the hashed layout, but is more memory efficient. It uses about twice as less memory at the cost of slower value retrieval. #6894 (Azat Khuzhin)
• Implement ability to define list of users for access to dictionaries. Only current connected database using. #6907 (Guillaume Tassery)
• Add LIMIT option to SHOW query. #6944 (Philipp Malkovsky)
• Add bitmapSubsetLimit(bitmap, range_start, limit) function, that returns subset of the smallest limit values in set that is no smaller than range_start. #6957 (Zhichang Yu)
• Add bitmapMin and bitmapMax functions. #6970 (Zhichang Yu)
• Add function repeat related to issue-6648 #6999 (flynn)

Experimental Feature

• Implement (in memory) Merge Join variant that does not change current pipeline. Result is partially sorted by merge key. Set partial_merge_join = 1 to use this feature. The Merge Join is still in development. #6940 (Artem Zuikov)
• Add S3 engine and table function. It is still in development (no authentication support yet). #5596 (Vladimir Chebotarev)

Improvement

• Every message read from Kafka is inserted atomically. This resolves almost all known issues with Kafka engine. #6950 (Ivan)
• Improvements for failover of Distributed queries. Shorten recovery time, also it is now configurable and can be seen in system.clusters. #6399 (Vasily Nemkov)
• Support numeric values for Enums directly in IN section. #6766 #6941 (dimarub2000)
• Support (optional, disabled by default) redirects on URL storage. #6914 (maqroll)
• Add information message when client with an older version connects to a server. #6893 (Philipp Malkovsky)
• Remove maximum backoff sleep time limit for sending data in Distributed tables #6895 (Azat Khuzhin)
• Add ability to send profile events (counters) with cumulative values to graphite. It can be enabled under <events_cumulative> in server config.xml. #6969 (Azat Khuzhin)
• Add automatically cast type T to LowCardinality(T) while inserting data in column of type LowCardinality(T) in Native format via HTTP. #6891 (Nikolai Kochetov)
• Add ability to use function hex without using reinterpretAsString for Float32, Float64. #7024 (Mikhail Korotov)

Build/Testing/Packaging Improvement

• Add gdb-index to clickhouse binary with debug info. It will speed up startup time of gdb. #6947 (alesapin)
• Speed up deb packaging with patched dpkg-deb which uses pigz. #6960 (alesapin)
• Set enable_fuzzing = 1 to enable libfuzzer instrumentation of all the project code. #7042 (kyprizel)
• Add split build smoke test in CI. #7061 (alesapin)
• Add build with MemorySanitizer to CI. #7066 (Alexander Kuzmenkov)
• Replace libsparsehash with sparsehash-c11 #6965 (Azat Khuzhin)

Bug Fix

• Fixed performance degradation of index analysis on complex keys on large tables. This fixes #6924. #7075 (alexey-milovidov)
• Fix logical error causing segfaults when selecting from Kafka empty topic. #6909 (Ivan)
• Fix too early MySQL connection close in MySQLBlockInputStream.cpp. #6882 (Clément Rodriguez)
• Returned support for very old Linux kernels (fix #6841) #6853 (alexey-milovidov)
• Fix possible data loss in insert select query in case of empty block in input stream. #6834 #6862 #6911 (Nikolai Kochetov)
• Fix for function АrrayEnumerateUniqRanked with empty arrays in params #6928 (proller)
• Fix complex queries with array joins and global subqueries. #6934 (Ivan)
• Fix Unknown identifier error in ORDER BY and GROUP BY with multiple JOINs #7022 (Artem Zuikov)
• Fixed MSan warning while executing function with LowCardinality argument. #7062 (Nikolai Kochetov)

Backward Incompatible Change

• Changed serialization format of bitmap aggregate function states to improve performance. Serialized states of bitmap from previous versions cannot be read. #6908 (Zhichang Yu)

ClickHouse release 19.14

ClickHouse release 19.14.7.15, 2019-10-02

Bug Fix

• This release also contains all bug fixes from 19.11.12.69.
• Fixed compatibility for distributed queries between 19.14 and earlier versions. This fixes #7068. #7069 (alexey-milovidov)

ClickHouse release 19.14.6.12, 2019-09-19

Bug Fix

• Fix for function АrrayEnumerateUniqRanked with empty arrays in params. #6928 (proller)
• Fixed subquery name in queries with ARRAY JOIN and GLOBAL IN subquery with alias. Use subquery alias for external table name if it is specified. #6934 (Ivan)

Build/Testing/Packaging Improvement

• Fix flapping.html) test 00715_fetch_merged_or_mutated_part_zookeeper by rewriting it to a shell scripts because it needs to wait for mutations to apply. #6977 (Alexander Kazakov)
• Fixed UBSan and MemSan failure in function groupUniqArray with emtpy array argument. It was caused by placing of empty PaddedPODArray into hash table zero cell because constructor for zero cell value was not called. #6937 (Amos Bird)

ClickHouse release 19.14.3.3, 2019-09-10

New Feature

• WITH FILL modifier for ORDER BY. (continuation of #5069) #6610 (Anton Popov)
• WITH TIES modifier for LIMIT. (continuation of #5069) #6610 (Anton Popov)
• Parse unquoted NULL literal as NULL (if setting format_csv_unquoted_null_literal_as_null=1). Initialize null fields with default values if data type of this field is not nullable (if setting input_format_null_as_default=1). #5990 #6055 (tavplubix)
• Support for wildcards in paths of table functions file and hdfs. If the path contains wildcards, the table will be readonly. Example of usage: select * from hdfs('hdfs://hdfs1:9000/some_dir/another_dir/*/file{0..9}{0..9}') and select * from file('some_dir/{some_file,another_file,yet_another}.tsv', 'TSV', 'value UInt32'). #6092 (Olga Khvostikova)
• New system.metric_log table which stores values of system.events and system.metrics with specified time interval. #6363 #6467 (Nikita Mikhaylov) #6530 (alexey-milovidov)
• Allow to write ClickHouse text logs to system.text_log table. #6037 #6103 (Nikita Mikhaylov) #6164 (alexey-milovidov)
• Show private symbols in stack traces (this is done via parsing symbol tables of ELF files). Added information about file and line number in stack traces if debug info is present. Speedup symbol name lookup with indexing symbols present in program. Added new SQL functions for introspection: demangle and addressToLine. Renamed function symbolizeAddress to addressToSymbol for consistency. Function addressToSymbol will return mangled name for performance reasons and you have to apply demangle. Added setting allow_introspection_functions which is turned off by default. #6201 (alexey-milovidov)
• Table function values (the name is case-insensitive). It allows to read from VALUES list proposed in #5984. Example: SELECT * FROM VALUES('a UInt64, s String', (1, 'one'), (2, 'two'), (3, 'three')). #6217. #6209 (dimarub2000)
• Added an ability to alter storage settings. Syntax: ALTER TABLE <table> MODIFY SETTING <setting> = <value>. #6366 #6669 #6685 (alesapin)
• Support for removing of detached parts. Syntax: ALTER TABLE <table_name> DROP DETACHED PART '<part_id>'. #6158 (tavplubix)
• Table constraints. Allows to add constraint to table definition which will be checked at insert. #5273 (Gleb Novikov) #6652 (alexey-milovidov)
• Suppport for cascaded materialized views. #6324 (Amos Bird)
• Turn on query profiler by default to sample every query execution thread once a second. #6283 (alexey-milovidov)
• Input format ORC. #6454 #6703 (akonyaev90)
• Added two new functions: sigmoid and tanh (that are useful for machine learning applications). #6254 (alexey-milovidov)
• Function hasToken(haystack, token), hasTokenCaseInsensitive(haystack, token) to check if given token is in haystack. Token is a maximal length substring between two non alphanumeric ASCII characters (or boundaries of haystack). Token must be a constant string. Supported by tokenbf_v1 index specialization. #6596, #6662 (Vasily Nemkov)
• New function neighbor(value, offset[, default_value]). Allows to reach prev/next value within column in a block of data. #5925 (Alex Krash) 6685365ab8c5b74f9650492c88a012596eb1b0c6 341e2e4587a18065c2da1ca888c73389f48ce36c Alexey Milovidov
• Created a function currentUser(), returning login of authorized user. Added alias user() for compatibility with MySQL. #6470 (Alex Krash)
• New aggregate functions quantilesExactInclusive and quantilesExactExclusive which were proposed in #5885. #6477 (dimarub2000)
• Function bitmapRange(bitmap, range_begin, range_end) which returns new set with specified range (not include the range_end). #6314 (Zhichang Yu)
• Function geohashesInBox(longitude_min, latitude_min, longitude_max, latitude_max, precision) which creates array of precision-long strings of geohash-boxes covering provided area. #6127 (Vasily Nemkov)
• Implement support for INSERT query with Kafka tables. #6012 (Ivan)
• Added support for _partition and _timestamp virtual columns to Kafka engine. #6400 (Ivan)
• Possibility to remove sensitive data from query_log, server logs, process list with regexp-based rules. #5710 (filimonov)

Experimental Feature

• Input and output data format Template. It allows to specify custom format string for input and output. #4354 #6727 (tavplubix)
• Implementation of LIVE VIEW tables that were originally proposed in #2898, prepared in #3925, and then updated in #5541. See #5541 for detailed description. #5541 (vzakaznikov) #6425 (Nikolai Kochetov) #6656 (vzakaznikov) Note that LIVE VIEW feature may be removed in next versions.

Bug Fix

• This release also contains all bug fixes from 19.13 and 19.11.
• Fix segmentation fault when the table has skip indices and vertical merge happens. #6723 (alesapin)
• Fix per-column TTL with non-trivial column defaults. Previously in case of force TTL merge with OPTIMIZE ... FINAL query, expired values was replaced by type defaults instead of user-specified column defaults. #6796 (Anton Popov)
• Fix Kafka messages duplication problem on normal server restart. #6597 (Ivan)
• Fixed infinite loop when reading Kafka messages. Do not pause/resume consumer on subscription at all - otherwise it may get paused indefinitely in some scenarios. #6354 (Ivan)
• Fix Key expression contains comparison between inconvertible types exception in bitmapContains function. #6136 #6146 #6156 (dimarub2000)
• Fix segfault with enabled optimize_skip_unused_shards and missing sharding key. #6384 (Anton Popov)
• Fixed wrong code in mutations that may lead to memory corruption. Fixed segfault with read of address 0x14c0 that may happed due to concurrent DROP TABLE and SELECT from system.parts or system.parts_columns. Fixed race condition in preparation of mutation queries. Fixed deadlock caused by OPTIMIZE of Replicated tables and concurrent modification operations like ALTERs. #6514 (alexey-milovidov)
• Removed extra verbose logging in MySQL interface #6389 (alexey-milovidov)
• Return the ability to parse boolean settings from ‘true’ and ‘false’ in the configuration file. #6278 (alesapin)
• Fix crash in quantile and median function over Nullable(Decimal128). #6378 (Artem Zuikov)
• Fixed possible incomplete result returned by SELECT query with WHERE condition on primary key contained conversion to Float type. It was caused by incorrect checking of monotonicity in toFloat function. #6248 #6374 (dimarub2000)
• Check max_expanded_ast_elements setting for mutations. Clear mutations after TRUNCATE TABLE. #6205 (Winter Zhang)
• Fix JOIN results for key columns when used with join_use_nulls. Attach Nulls instead of columns defaults. #6249 (Artem Zuikov)
• Fix for skip indices with vertical merge and alter. Fix for Bad size of marks file exception. #6594 #6713 (alesapin)
• Fix rare crash in ALTER MODIFY COLUMN and vertical merge when one of merged/altered parts is empty (0 rows) #6746 #6780 (alesapin)
• Fixed bug in conversion of LowCardinality types in AggregateFunctionFactory. This fixes #6257. #6281 (Nikolai Kochetov)
• Fix wrong behavior and possible segfaults in topK and topKWeighted aggregated functions. #6404 (Anton Popov)
• Fixed unsafe code around getIdentifier function. #6401 #6409 (alexey-milovidov)
• Fixed bug in MySQL wire protocol (is used while connecting to ClickHouse form MySQL client). Caused by heap buffer overflow in PacketPayloadWriteBuffer. #6212 (Yuriy Baranov)
• Fixed memory leak in bitmapSubsetInRange function. #6819 (Zhichang Yu)
• Fix rare bug when mutation executed after granularity change. #6816 (alesapin)
• Allow protobuf message with all fields by default. #6132 (Vitaly Baranov)
• Resolve a bug with nullIf function when we send a NULL argument on the second argument. #6446 (Guillaume Tassery)
• Fix rare bug with wrong memory allocation/deallocation in complex key cache dictionaries with string fields which leads to infinite memory consumption (looks like memory leak). Bug reproduces when string size was a power of two starting from eight (8, 16, 32, etc). #6447 (alesapin)
• Fixed Gorilla encoding on small sequences which caused exception Cannot write after end of buffer. #6398 #6444 (Vasily Nemkov)
• Allow to use not nullable types in JOINs with join_use_nulls enabled. #6705 (Artem Zuikov)
• Disable Poco::AbstractConfiguration substitutions in query in clickhouse-client. #6706 (alexey-milovidov)
• Avoid deadlock in REPLACE PARTITION. #6677 (alexey-milovidov)
• Using arrayReduce for constant arguments may lead to segfault. #6242 #6326 (alexey-milovidov)
• Fix inconsistent parts which can appear if replica was restored after DROP PARTITION. #6522 #6523 (tavplubix)
• Fixed hang in JSONExtractRaw function. #6195 #6198 (alexey-milovidov)
• Fix bug with incorrect skip indices serialization and aggregation with adaptive granularity. #6594. #6748 (alesapin)
• Fix WITH ROLLUP and WITH CUBE modifiers of GROUP BY with two-level aggregation. #6225 (Anton Popov)
• Fix bug with writing secondary indices marks with adaptive granularity. #6126 (alesapin)
• Fix initialization order while server startup. Since StorageMergeTree::background_task_handle is initialized in startup() the MergeTreeBlockOutputStream::write() may try to use it before initialization. Just check if it is initialized. #6080 (Ivan)
• Clearing the data buffer from the previous read operation that was completed with an error. #6026 (Nikolay)
• Fix bug with enabling adaptive granularity when creating a new replica for Replicated*MergeTree table. #6394 #6452 (alesapin)
• Fixed possible crash during server startup in case of exception happened in libunwind during exception at access to uninitialized ThreadStatus structure. #6456 (Nikita Mikhaylov)
• Fix crash in yandexConsistentHash function. Found by fuzz test. #6304 #6305 (alexey-milovidov)
• Fixed the possibility of hanging queries when server is overloaded and global thread pool becomes near full. This have higher chance to happen on clusters with large number of shards (hundreds), because distributed queries allocate a thread per connection to each shard. For example, this issue may reproduce if a cluster of 330 shards is processing 30 concurrent distributed queries. This issue affects all versions starting from 19.2. #6301 (alexey-milovidov)
• Fixed logic of arrayEnumerateUniqRanked function. #6423 (alexey-milovidov)
• Fix segfault when decoding symbol table. #6603 (Amos Bird)
• Fixed irrelevant exception in cast of LowCardinality(Nullable) to not-Nullable column in case if it doesn’t contain Nulls (e.g. in query like SELECT CAST(CAST('Hello' AS LowCardinality(Nullable(String))) AS String). #6094 #6119 (Nikolai Kochetov)
• Removed extra quoting of description in system.settings table. #6696 #6699 (alexey-milovidov)
• Avoid possible deadlock in TRUNCATE of Replicated table. #6695 (alexey-milovidov)
• Fix reading in order of sorting key. #6189 (Anton Popov)
• Fix ALTER TABLE ... UPDATE query for tables with enable_mixed_granularity_parts=1. #6543 (alesapin)
• Fix bug opened by #4405 (since 19.4.0). Reproduces in queries to Distributed tables over MergeTree tables when we doesn’t query any columns (SELECT 1). #6236 (alesapin)
• Fixed overflow in integer division of signed type to unsigned type. The behaviour was exactly as in C or C++ language (integer promotion rules) that may be surprising. Please note that the overflow is still possible when dividing large signed number to large unsigned number or vice-versa (but that case is less usual). The issue existed in all server versions. #6214 #6233 (alexey-milovidov)
• Limit maximum sleep time for throttling when max_execution_speed or max_execution_speed_bytes is set. Fixed false errors like Estimated query execution time (inf seconds) is too long. #5547 #6232 (alexey-milovidov)
• Fixed issues about using MATERIALIZED columns and aliases in MaterializedView. #448 #3484 #3450 #2878 #2285 #3796 (Amos Bird) #6316 (alexey-milovidov)
• Fix FormatFactory behaviour for input streams which are not implemented as processor. #6495 (Nikolai Kochetov)
• Fixed typo. #6631 (Alex Ryndin)
• Typo in the error message ( is -> are ). #6839 (Denis Zhuravlev)
• Fixed error while parsing of columns list from string if type contained a comma (this issue was relevant for File, URL, HDFS storages) #6217. #6209 (dimarub2000)

Security Fix

• This release also contains all bug security fixes from 19.13 and 19.11.
• Fixed the possibility of a fabricated query to cause server crash due to stack overflow in SQL parser. Fixed the possibility of stack overflow in Merge and Distributed tables, materialized views and conditions for row-level security that involve subqueries. #6433 (alexey-milovidov)

Improvement

• Correct implementation of ternary logic for AND/OR. #6048 (Alexander Kazakov)
• Now values and rows with expired TTL will be removed after OPTIMIZE ... FINAL query from old parts without TTL infos or with outdated TTL infos, e.g. after ALTER ... MODIFY TTL query. Added queries SYSTEM STOP/START TTL MERGES to disallow/allow assign merges with TTL and filter expired values in all merges. #6274 (Anton Popov)
• Possibility to change the location of ClickHouse history file for client using CLICKHOUSE_HISTORY_FILE env. #6840 (filimonov)
• Remove dry_run flag from InterpreterSelectQuery. … #6375 (Nikolai Kochetov)
• Support ASOF JOIN with ON section. #6211 (Artem Zuikov)
• Better support of skip indexes for mutations and replication. Support for MATERIALIZE/CLEAR INDEX ... IN PARTITION query. UPDATE x = x recalculates all indices that use column x. #5053 (Nikita Vasilev)
• Allow to ATTACH live views (for example, at the server startup) regardless to allow_experimental_live_view setting. #6754 (alexey-milovidov)
• For stack traces gathered by query profiler, do not include stack frames generated by the query profiler itself. #6250 (alexey-milovidov)
• Now table functions values, file, url, hdfs have support for ALIAS columns. #6255 (alexey-milovidov)
• Throw an exception if config.d file doesn’t have the corresponding root element as the config file. #6123 (dimarub2000)
• Print extra info in exception message for no space left on device. #6182, #6252 #6352 (tavplubix)
• When determining shards of a Distributed table to be covered by a read query (for optimize_skip_unused_shards = 1) ClickHouse now checks conditions from both prewhere and where clauses of select statement. #6521 (Alexander Kazakov)
• Enabled SIMDJSON for machines without AVX2 but with SSE 4.2 and PCLMUL instruction set. #6285 #6320 (alexey-milovidov)
• ClickHouse can work on filesystems without O_DIRECT support (such as ZFS and BtrFS) without additional tuning. #4449 #6730 (alexey-milovidov)
• Support push down predicate for final subquery. #6120 (TCeason) #6162 (alexey-milovidov)
• Better JOIN ON keys extraction #6131 (Artem Zuikov)
• Upated SIMDJSON. #6285. #6306 (alexey-milovidov)
• Optimize selecting of smallest column for SELECT count() query. #6344 (Amos Bird)
• Added strict parameter in windowFunnel(). When the strict is set, the windowFunnel() applies conditions only for the unique values. #6548 (achimbab)
• Safer interface of mysqlxx::Pool. #6150 (avasiliev)
• Options line size when executing with --help option now corresponds with terminal size. #6590 (dimarub2000)
• Disable “read in order” optimization for aggregation without keys. #6599 (Anton Popov)
• HTTP status code for INCORRECT_DATA and TYPE_MISMATCH error codes was changed from default 500 Internal Server Error to 400 Bad Request. #6271 (Alexander Rodin)
• Move Join object from ExpressionAction into AnalyzedJoin. ExpressionAnalyzer and ExpressionAction do not know about Join class anymore. Its logic is hidden by AnalyzedJoin iface. #6801 (Artem Zuikov)
• Fixed possible deadlock of distributed queries when one of shards is localhost but the query is sent via network connection. #6759 (alexey-milovidov)
• Changed semantic of multiple tables RENAME to avoid possible deadlocks. #6757. #6756 (alexey-milovidov)
• Rewritten MySQL compatibility server to prevent loading full packet payload in memory. Decreased memory consumption for each connection to approximately 2 * DBMS_DEFAULT_BUFFER_SIZE (read/write buffers). #5811 (Yuriy Baranov)
• Move AST alias interpreting logic out of parser that doesn’t have to know anything about query semantics. #6108 (Artem Zuikov)
• Slightly more safe parsing of NamesAndTypesList. #6408. #6410 (alexey-milovidov)
• clickhouse-copier: Allow use where_condition from config with partition_key alias in query for checking partition existence (Earlier it was used only in reading data queries). #6577 (proller)
• Added optional message argument in throwIf. (#5772) #6329 (Vdimir)
• Server exception got while sending insertion data is now being processed in client as well. #5891 #6711 (dimarub2000)
• Added a metric DistributedFilesToInsert that shows the total number of files in filesystem that are selected to send to remote servers by Distributed tables. The number is summed across all shards. #6600 (alexey-milovidov)
• Move most of JOINs prepare logic from ExpressionAction/ExpressionAnalyzer to AnalyzedJoin. #6785 (Artem Zuikov)
• Fix TSan warning.html) ‘lock-order-inversion’. #6740 (Vasily Nemkov)
• Better information messages about lack of Linux capabilities. Logging fatal errors with “fatal” level, that will make it easier to find in system.text_log. #6441 (alexey-milovidov)
• When enable dumping temporary data to the disk to restrict memory usage during GROUP BY, ORDER BY, it didn’t check the free disk space. The fix add a new setting min_free_disk_space, when the free disk space it smaller then the threshold, the query will stop and throw ErrorCodes::NOT_ENOUGH_SPACE. #6678 (Weiqing Xu) #6691 (alexey-milovidov)
• Removed recursive rwlock by thread. It makes no sense, because threads are reused between queries. SELECT query may acquire a lock in one thread, hold a lock from another thread and exit from first thread. In the same time, first thread can be reused by DROP query. This will lead to false “Attempt to acquire exclusive lock recursively” messages. #6771 (alexey-milovidov)
• Split ExpressionAnalyzer.appendJoin(). Prepare a place in ExpressionAnalyzer for MergeJoin. #6524 (Artem Zuikov)
• Added mysql_native_password authentication plugin to MySQL compatibility server. #6194 (Yuriy Baranov)
• Less number of clock_gettime calls; fixed ABI compatibility between debug/release in Allocator (insignificant issue). #6197 (alexey-milovidov)
• Move collectUsedColumns from ExpressionAnalyzer to SyntaxAnalyzer. SyntaxAnalyzer makes required_source_columns itself now. #6416 (Artem Zuikov)
• Add setting joined_subquery_requires_alias to require aliases for subselects and table functions in FROM that more than one table is present (i.e. queries with JOINs). #6733 (Artem Zuikov)
• Extract GetAggregatesVisitor class from ExpressionAnalyzer. #6458 (Artem Zuikov)
• system.query_log: change data type of type column to Enum. #6265 (Nikita Mikhaylov)
• Static linking of sha256_password authentication plugin. #6512 (Yuriy Baranov)
• Avoid extra dependency for the setting compile to work. In previous versions, the user may get error like cannot open crti.o, unable to find library -lc etc. #6309 (alexey-milovidov)
• More validation of the input that may come from malicious replica. #6303 (alexey-milovidov)
• Now clickhouse-obfuscator file is available in clickhouse-client package. In previous versions it was available as clickhouse obfuscator (with whitespace). #5816 #6609 (dimarub2000)
• Fixed deadlock when we have at least two queries that read at least two tables in different order and another query that performs DDL operation on one of tables. Fixed another very rare deadlock. #6764 (alexey-milovidov)
• Added os_thread_ids column to system.processes and system.query_log for better debugging possibilities. #6763 (alexey-milovidov)
• A workaround for PHP mysqlnd extension bugs which occur when sha256_password is used as a default authentication plugin (described in #6031). #6113 (Yuriy Baranov)
• Remove unneeded place with changed nullability columns. #6693 (Artem Zuikov)
• Set default value of queue_max_wait_ms to zero, because current value (five seconds) makes no sense. There are rare circumstances when this settings has any use. Added settings replace_running_query_max_wait_ms, kafka_max_wait_ms and connection_pool_max_wait_ms for disambiguation. #6692 (alexey-milovidov)
• Extract SelectQueryExpressionAnalyzer from ExpressionAnalyzer. Keep the last one for non-select queries. #6499 (Artem Zuikov)
• Removed duplicating input and output formats. #6239 (Nikolai Kochetov)
• Allow user to override poll_interval and idle_connection_timeout settings on connection. #6230 (alexey-milovidov)
• MergeTree now has an additional option ttl_only_drop_parts (disabled by default) to avoid partial pruning of parts, so that they dropped completely when all the rows in a part are expired. #6191 (Sergi Vladykin)
• Type checks for set index functions. Throw exception if function got a wrong type. This fixes fuzz test with UBSan. #6511 (Nikita Vasilev)

Performance Improvement

• Optimize queries with ORDER BY expressions clause, where expressions have coinciding prefix with sorting key in MergeTree tables. This optimization is controlled by optimize_read_in_order setting. #6054 #6629 (Anton Popov)
• Allow to use multiple threads during parts loading and removal. #6372 #6074 #6438 (alexey-milovidov)
• Implemented batch variant of updating aggregate function states. It may lead to performance benefits. #6435 (alexey-milovidov)
• Using FastOps library for functions exp, log, sigmoid, tanh. FastOps is a fast vector math library from Michael Parakhin (Yandex CTO). Improved performance of exp and log functions more than 6 times. The functions exp and log from Float32 argument will return Float32 (in previous versions they always return Float64). Now exp(nan) may return inf. The result of exp and log functions may be not the nearest machine representable number to the true answer. #6254 (alexey-milovidov) Using Danila Kutenin variant to make fastops working #6317 (alexey-milovidov)
• Disable consecutive key optimization for UInt8/16. #6298 #6701 (akuzm)
• Improved performance of simdjson library by getting rid of dynamic allocation in ParsedJson::Iterator. #6479 (Vitaly Baranov)
• Pre-fault pages when allocating memory with mmap(). #6667 (akuzm)
• Fix performance bug in Decimal comparison. #6380 (Artem Zuikov)

Build/Testing/Packaging Improvement

• Remove Compiler (runtime template instantiation) because we’ve win over it’s performance. #6646 (alexey-milovidov)
• Added performance test to show degradation of performance in gcc-9 in more isolated way. #6302 (alexey-milovidov)
• Added table function numbers_mt, which is multithreaded version of numbers. Updated performance tests with hash functions. #6554 (Nikolai Kochetov)
• Comparison mode in clickhouse-benchmark #6220 #6343 (dimarub2000)
• Best effort for printing stack traces. Also added SIGPROF as a debugging signal to print stack trace of a running thread. #6529 (alexey-milovidov)
• Every function in its own file, part 10. #6321 (alexey-milovidov)
• Remove doubled const TABLE_IS_READ_ONLY. #6566 (filimonov)
• Formatting changes for StringHashMap PR #5417. #6700 (akuzm)
• Better subquery for join creation in ExpressionAnalyzer. #6824 (Artem Zuikov)
• Remove a redundant condition (found by PVS Studio). #6775 (akuzm)
• Separate the hash table interface for ReverseIndex. #6672 (akuzm)
• Refactoring of settings. #6689 (alesapin)
• Add comments for set index functions. #6319 (Nikita Vasilev)
• Increase OOM score in debug version on Linux. #6152 (akuzm)
• HDFS HA now work in debug build. #6650 (Weiqing Xu)
• Added a test to transform_query_for_external_database. #6388 (alexey-milovidov)
• Add test for multiple materialized views for Kafka table. #6509 (Ivan)
• Make a better build scheme. #6500 (Ivan)
• Fixed test_external_dictionaries integration in case it was executed under non root user. #6507 (Nikolai Kochetov)
• The bug reproduces when total size of written packets exceeds DBMS_DEFAULT_BUFFER_SIZE. #6204 (Yuriy Baranov)
• Added a test for RENAME table race condition #6752 (alexey-milovidov)
• Avoid data race on Settings in KILL QUERY. #6753 (alexey-milovidov)
• Add integration test for handling errors by a cache dictionary. #6755 (Vitaly Baranov)
• Disable parsing of ELF object files on Mac OS, because it makes no sense. #6578 (alexey-milovidov)
• Attempt to make changelog generator better. #6327 (alexey-milovidov)
• Adding -Wshadow switch to the GCC. #6325 (kreuzerkrieg)
• Removed obsolete code for mimalloc support. #6715 (alexey-milovidov)
• zlib-ng determines x86 capabilities and saves this info to global variables. This is done in defalteInit call, which may be made by different threads simultaneously. To avoid multithreaded writes, do it on library startup. #6141 (akuzm)
• Regression test for a bug which in join which was fixed in #5192. #6147 (Bakhtiyor Ruziev)
• Fixed MSan report. #6144 (alexey-milovidov)
• Fix flapping TTL test. #6782 (Anton Popov)
• Fixed false data race in MergeTreeDataPart::is_frozen field. #6583 (alexey-milovidov)
• Fixed timeouts in fuzz test. In previous version, it managed to find false hangup in query SELECT * FROM numbers_mt(gccMurmurHash('')). #6582 (alexey-milovidov)
• Added debug checks to static_cast of columns. #6581 (alexey-milovidov)
• Support for Oracle Linux in official RPM packages. #6356 #6585 (alexey-milovidov)
• Changed json perftests from once to loop type. #6536 (Nikolai Kochetov)
• odbc-bridge.cpp defines main() so it should not be included in clickhouse-lib. #6538 (Orivej Desh)
• Test for crash in FULL|RIGHT JOIN with nulls in right table’s keys. #6362 (Artem Zuikov)
• Added a test for the limit on expansion of aliases just in case. #6442 (alexey-milovidov)
• Switched from boost::filesystem to std::filesystem where appropriate. #6253 #6385 (alexey-milovidov)
• Added RPM packages to website. #6251 (alexey-milovidov)
• Add a test for fixed Unknown identifier exception in IN section. #6708 (Artem Zuikov)
• Simplify shared_ptr_helper because people facing difficulties understanding it. #6675 (alexey-milovidov)
• Added performance tests for fixed Gorilla and DoubleDelta codec. #6179 (Vasily Nemkov)
• Split the integration test test_dictionaries into 4 separate tests. #6776 (Vitaly Baranov)
• Fix PVS-Studio warning in PipelineExecutor. #6777 (Nikolai Kochetov)
• Allow to use library dictionary source with ASan. #6482 (alexey-milovidov)
• Added option to generate changelog from a list of PRs. #6350 (alexey-milovidov)
• Lock the TinyLog storage when reading. #6226 (akuzm)
• Check for broken symlinks in CI. #6634 (alexey-milovidov)
• Increase timeout for “stack overflow” test because it may take a long time in debug build. #6637 (alexey-milovidov)
• Added a check for double whitespaces. #6643 (alexey-milovidov)
• Fix new/delete memory tracking when build with sanitizers. Tracking is not clear. It only prevents memory limit exceptions in tests. #6450 (Artem Zuikov)
• Enable back the check of undefined symbols while linking. #6453 (Ivan)
• Avoid rebuilding hyperscan every day. #6307 (alexey-milovidov)
• Fixed UBSan report in ProtobufWriter. #6163 (alexey-milovidov)
• Don’t allow to use query profiler with sanitizers because it is not compatible. #6769 (alexey-milovidov)
• Add test for reloading a dictionary after fail by timer. #6114 (Vitaly Baranov)
• Fix inconsistency in PipelineExecutor::prepareProcessor argument type. #6494 (Nikolai Kochetov)
• Added a test for bad URIs. #6493 (alexey-milovidov)
• Added more checks to CAST function. This should get more information about segmentation fault in fuzzy test. #6346 (Nikolai Kochetov)
• Added gcc-9 support to docker/builder container that builds image locally. #6333 (Gleb Novikov)
• Test for primary key with LowCardinality(String). #5044 #6219 (dimarub2000)
• Fixed tests affected by slow stack traces printing. #6315 (alexey-milovidov)
• Add a test case for crash in groupUniqArray fixed in #6029. #4402 #6129 (akuzm)
• Fixed indices mutations tests. #6645 (Nikita Vasilev)
• In performance test, do not read query log for queries we didn’t run. #6427 (akuzm)
• Materialized view now could be created with any low cardinality types regardless to the setting about suspicious low cardinality types. #6428 (Olga Khvostikova)
• Updated tests for send_logs_level setting. #6207 (Nikolai Kochetov)
• Fix build under gcc-8.2. #6196 (Max Akhmedov)
• Fix build with internal libc++. #6724 (Ivan)
• Fix shared build with rdkafka library #6101 (Ivan)
• Fixes for Mac OS build (incomplete). #6390 (alexey-milovidov) #6429 (alex-zaitsev)
• Fix “splitted” build. #6618 (alexey-milovidov)
• Other build fixes: #6186 (Amos Bird) #6486 #6348 (vxider) #6744 (Ivan) #6016 #6421 #6491 (proller)

Backward Incompatible Change

• Removed rarely used table function catBoostPool and storage CatBoostPool. If you have used this table function, please write email to [[email protected]](https://clickhouse.tech/cdn-cgi/l/email-protection). Note that CatBoost integration remains and will be supported. #6279 (alexey-milovidov)
• Disable ANY RIGHT JOIN and ANY FULL JOIN by default. Set any_join_distinct_right_table_keys setting to enable them. #5126 #6351 (Artem Zuikov)

ClickHouse release 19.13

ClickHouse release 19.13.6.51, 2019-10-02

Bug Fix

• This release also contains all bug fixes from 19.11.12.69.

ClickHouse release 19.13.5.44, 2019-09-20

Bug Fix

• This release also contains all bug fixes from 19.14.6.12.
• Fixed possible inconsistent state of table while executing DROP query for replicated table while zookeeper is not accessible. #6045 #6413 (Nikita Mikhaylov)
• Fix for data race in StorageMerge #6717 (alexey-milovidov)
• Fix bug introduced in query profiler which leads to endless recv from socket. #6386 (alesapin)
• Fix excessive CPU usage while executing JSONExtractRaw function over a boolean value. #6208 (Vitaly Baranov)
• Fixes the regression while pushing to materialized view. #6415 (Ivan)
• Table function url had the vulnerability allowed the attacker to inject arbitrary HTTP headers in the request. This issue was found by Nikita Tikhomirov. #6466 (alexey-milovidov)
• Fix useless AST check in Set index. #6510 #6651 (Nikita Vasilev)
• Fixed parsing of AggregateFunction values embedded in query. #6575 #6773 (Zhichang Yu)
• Fixed wrong behaviour of trim functions family. #6647 (alexey-milovidov)

ClickHouse release 19.13.4.32, 2019-09-10

Bug Fix

• This release also contains all bug security fixes from 19.11.9.52 and 19.11.10.54.
• Fixed data race in system.parts table and ALTER query. #6245 #6513 (alexey-milovidov)
• Fixed mismatched header in streams happened in case of reading from empty distributed table with sample and prewhere. #6167 (Lixiang Qian) #6823 (Nikolai Kochetov)
• Fixed crash when using IN clause with a subquery with a tuple. #6125 #6550 (tavplubix)
• Fix case with same column names in GLOBAL JOIN ON section. #6181 (Artem Zuikov)
• Fix crash when casting types to Decimal that do not support it. Throw exception instead. #6297 (Artem Zuikov)
• Fixed crash in extractAll() function. #6644 (Artem Zuikov)
• Query transformation for MySQL, ODBC, JDBC table functions now works properly for SELECT WHERE queries with multiple AND expressions. #6381 #6676 (dimarub2000)
• Added previous declaration checks for MySQL 8 integration. #6569 (Rafael David Tinoco)

Security Fix

• Fix two vulnerabilities in codecs in decompression phase (malicious user can fabricate compressed data that will lead to buffer overflow in decompression). #6670 (Artem Zuikov)

ClickHouse release 19.13.3.26, 2019-08-22

Bug Fix

• Fix ALTER TABLE ... UPDATE query for tables with enable_mixed_granularity_parts=1. #6543 (alesapin)
• Fix NPE when using IN clause with a subquery with a tuple. #6125 #6550 (tavplubix)
• Fixed an issue that if a stale replica becomes alive, it may still have data parts that were removed by DROP PARTITION. #6522 #6523 (tavplubix)
• Fixed issue with parsing CSV #6426 #6559 (tavplubix)
• Fixed data race in system.parts table and ALTER query. This fixes #6245. #6513 (alexey-milovidov)
• Fixed wrong code in mutations that may lead to memory corruption. Fixed segfault with read of address 0x14c0 that may happed due to concurrent DROP TABLE and SELECT from system.parts or system.parts_columns. Fixed race condition in preparation of mutation queries. Fixed deadlock caused by OPTIMIZE of Replicated tables and concurrent modification operations like ALTERs. #6514 (alexey-milovidov)
• Fixed possible data loss after ALTER DELETE query on table with skipping index. #6224 #6282 (Nikita Vasilev)

Security Fix

• If the attacker has write access to ZooKeeper and is able to run custom server available from the network where ClickHouse run, it can create custom-built malicious server that will act as ClickHouse replica and register it in ZooKeeper. When another replica will fetch data part from malicious replica, it can force clickhouse-server to write to arbitrary path on filesystem. Found by Eldar Zaitov, information security team at Yandex. #6247 (alexey-milovidov)

ClickHouse release 19.13.2.19, 2019-08-14

New Feature

• Sampling profiler on query level. Example. #4247 (laplab) #6124 (alexey-milovidov) #6250 #6283 #6386
• Allow to specify a list of columns with COLUMNS('regexp') expression that works like a more sophisticated variant of * asterisk. #5951 (mfridental), (alexey-milovidov)
• CREATE TABLE AS table_function() is now possible #6057 (dimarub2000)
• Adam optimizer for stochastic gradient descent is used by default in stochasticLinearRegression() and stochasticLogisticRegression() aggregate functions, because it shows good quality without almost any tuning. #6000 (Quid37)
• Added functions for working with the сustom week number #5212 (Andy Yang)
• RENAME queries now work with all storages. #5953 (Ivan)
• Now client receive logs from server with any desired level by setting send_logs_level regardless to the log level specified in server settings. #5964 (Nikita Mikhaylov)

Backward Incompatible Change

• The setting input_format_defaults_for_omitted_fields is enabled by default. Inserts in Distributed tables need this setting to be the same on cluster (you need to set it before rolling update). It enables calculation of complex default expressions for omitted fields in JSONEachRow and CSV* formats. It should be the expected behavior but may lead to negligible performance difference. #6043 (Artem Zuikov), #5625 (akuzm)

Experimental features

• New query processing pipeline. Use experimental_use_processors=1 option to enable it. Use for your own trouble. #4914 (Nikolai Kochetov)

Bug Fix

• Kafka integration has been fixed in this version.
• Fixed DoubleDelta encoding of Int64 for large DoubleDelta values, improved DoubleDelta encoding for random data for Int32. #5998 (Vasily Nemkov)
• Fixed overestimation of max_rows_to_read if the setting merge_tree_uniform_read_distribution is set to 0. #6019 (alexey-milovidov)

Improvement

• Throws an exception if config.d file doesn’t have the corresponding root element as the config file #6123 (dimarub2000)

Performance Improvement

• Optimize count(). Now it uses the smallest column (if possible). #6028 (Amos Bird)

Build/Testing/Packaging Improvement

• Report memory usage in performance tests. #5899 (akuzm)
• Fix build with external libcxx #6010 (Ivan)
• Fix shared build with rdkafka library #6101 (Ivan)

ClickHouse release 19.11

ClickHouse release 19.11.13.74, 2019-11-01

Bug Fix

• Fixed rare crash in ALTER MODIFY COLUMN and vertical merge when one of merged/altered parts is empty (0 rows). #6780 (alesapin)
• Manual update of SIMDJSON. This fixes possible flooding of stderr files with bogus json diagnostic messages. #7548 (Alexander Kazakov)
• Fixed bug with mrk file extension for mutations (alesapin)

ClickHouse release 19.11.12.69, 2019-10-02

Bug Fix

• Fixed performance degradation of index analysis on complex keys on large tables. This fixes #6924. #7075 (alexey-milovidov)
• Avoid rare SIGSEGV while sending data in tables with Distributed engine (Failed to send batch: file with index XXXXX is absent). #7032 (Azat Khuzhin)
• Fix Unknown identifier with multiple joins. This fixes #5254. #7022 (Artem Zuikov)

ClickHouse release 19.11.11.57, 2019-09-13

• Fix logical error causing segfaults when selecting from Kafka empty topic. #6902 #6909 (Ivan)
• Fix for function АrrayEnumerateUniqRanked with empty arrays in params. #6928 (proller)

ClickHouse release 19.11.10.54, 2019-09-10

Bug Fix

• Do store offsets for Kafka messages manually to be able to commit them all at once for all partitions. Fixes potential duplication in “one consumer - many partitions” scenario. #6872 (Ivan)

ClickHouse release 19.11.9.52, 2019-09-6

• Improve error handling in cache dictionaries. #6737 (Vitaly Baranov)
• Fixed bug in function arrayEnumerateUniqRanked. #6779 (proller)
• Fix JSONExtract function while extracting a Tuple from JSON. #6718 (Vitaly Baranov)
• Fixed possible data loss after ALTER DELETE query on table with skipping index. #6224 #6282 (Nikita Vasilev)
• Fixed performance test. #6392 (alexey-milovidov)
• Parquet: Fix reading boolean columns. #6579 (alexey-milovidov)
• Fixed wrong behaviour of nullIf function for constant arguments. #6518 (Guillaume Tassery) #6580 (alexey-milovidov)
• Fix Kafka messages duplication problem on normal server restart. #6597 (Ivan)
• Fixed an issue when long ALTER UPDATE or ALTER DELETE may prevent regular merges to run. Prevent mutations from executing if there is no enough free threads available. #6502 #6617 (tavplubix)
• Fixed error with processing “timezone” in server configuration file. #6709 (alexey-milovidov)
• Fix kafka tests. #6805 (Ivan)

Security Fix

• If the attacker has write access to ZooKeeper and is able to run custom server available from the network where ClickHouse runs, it can create custom-built malicious server that will act as ClickHouse replica and register it in ZooKeeper. When another replica will fetch data part from malicious replica, it can force clickhouse-server to write to arbitrary path on filesystem. Found by Eldar Zaitov, information security team at Yandex. #6247 (alexey-milovidov)

ClickHouse release 19.11.8.46, 2019-08-22

Bug Fix

• Fix ALTER TABLE ... UPDATE query for tables with enable_mixed_granularity_parts=1. #6543 (alesapin)
• Fix NPE when using IN clause with a subquery with a tuple. #6125 #6550 (tavplubix)
• Fixed an issue that if a stale replica becomes alive, it may still have data parts that were removed by DROP PARTITION. #6522 #6523 (tavplubix)
• Fixed issue with parsing CSV #6426 #6559 (tavplubix)
• Fixed data race in system.parts table and ALTER query. This fixes #6245. #6513 (alexey-milovidov)
• Fixed wrong code in mutations that may lead to memory corruption. Fixed segfault with read of address 0x14c0 that may happed due to concurrent DROP TABLE and SELECT from system.parts or system.parts_columns. Fixed race condition in preparation of mutation queries. Fixed deadlock caused by OPTIMIZE of Replicated tables and concurrent modification operations like ALTERs. #6514 (alexey-milovidov)

ClickHouse release 19.11.7.40, 2019-08-14

Bug fix

• Kafka integration has been fixed in this version.
• Fix segfault when using arrayReduce for constant arguments. #6326 (alexey-milovidov)
• Fixed toFloat() monotonicity. #6374 (dimarub2000)
• Fix segfault with enabled optimize_skip_unused_shards and missing sharding key. #6384 (CurtizJ)
• Fixed logic of arrayEnumerateUniqRanked function. #6423 (alexey-milovidov)
• Removed extra verbose logging from MySQL handler. #6389 (alexey-milovidov)
• Fix wrong behavior and possible segfaults in topK and topKWeighted aggregated functions. #6404 (CurtizJ)
• Do not expose virtual columns in system.columns table. This is required for backward compatibility. #6406 (alexey-milovidov)
• Fix bug with memory allocation for string fields in complex key cache dictionary. #6447 (alesapin)
• Fix bug with enabling adaptive granularity when creating new replica for Replicated*MergeTree table. #6452 (alesapin)
• Fix infinite loop when reading Kafka messages. #6354 (abyss7)
• Fixed the possibility of a fabricated query to cause server crash due to stack overflow in SQL parser and possibility of stack overflow in Merge and Distributed tables #6433 (alexey-milovidov)
• Fixed Gorilla encoding error on small sequences. #6444 (Enmk)

Improvement

• Allow user to override poll_interval and idle_connection_timeout settings on connection. #6230 (alexey-milovidov)

ClickHouse release 19.11.5.28, 2019-08-05

Bug fix

• Fixed the possibility of hanging queries when server is overloaded. #6301 (alexey-milovidov)
• Fix FPE in yandexConsistentHash function. This fixes #6304. #6126 (alexey-milovidov)
• Fixed bug in conversion of LowCardinality types in AggregateFunctionFactory. This fixes #6257. #6281 (Nikolai Kochetov)
• Fix parsing of bool settings from true and false strings in configuration files. #6278 (alesapin)
• Fix rare bug with incompatible stream headers in queries to Distributed table over MergeTree table when part of WHERE moves to PREWHERE. #6236 (alesapin)
• Fixed overflow in integer division of signed type to unsigned type. This fixes #6214. #6233 (alexey-milovidov)

Backward Incompatible Change

• Kafka still broken.

ClickHouse release 19.11.4.24, 2019-08-01

Bug Fix

• Fix bug with writing secondary indices marks with adaptive granularity. #6126 (alesapin)
• Fix WITH ROLLUP and WITH CUBE modifiers of GROUP BY with two-level aggregation. #6225 (Anton Popov)
• Fixed hang in JSONExtractRaw function. Fixed #6195 #6198 (alexey-milovidov)
• Fix segfault in ExternalLoader::reloadOutdated(). #6082 (Vitaly Baranov)
• Fixed the case when server may close listening sockets but not shutdown and continue serving remaining queries. You may end up with two running clickhouse-server processes. Sometimes, the server may return an error bad_function_call for remaining queries. #6231 (alexey-milovidov)
• Fixed useless and incorrect condition on update field for initial loading of external dictionaries via ODBC, MySQL, ClickHouse and HTTP. This fixes #6069 #6083 (alexey-milovidov)
• Fixed irrelevant exception in cast of LowCardinality(Nullable) to not-Nullable column in case if it doesn’t contain Nulls (e.g. in query like SELECT CAST(CAST('Hello' AS LowCardinality(Nullable(String))) AS String). #6094 #6119 (Nikolai Kochetov)
• Fix non-deterministic result of “uniq” aggregate function in extreme rare cases. The bug was present in all ClickHouse versions. #6058 (alexey-milovidov)
• Segfault when we set a little bit too high CIDR on the function IPv6CIDRToRange. #6068 (Guillaume Tassery)
• Fixed small memory leak when server throw many exceptions from many different contexts. #6144 (alexey-milovidov)
• Fix the situation when consumer got paused before subscription and not resumed afterwards. #6075 (Ivan) Note that Kafka is broken in this version.
• Clearing the Kafka data buffer from the previous read operation that was completed with an error #6026 (Nikolay) Note that Kafka is broken in this version.
• Since StorageMergeTree::background_task_handle is initialized in startup() the MergeTreeBlockOutputStream::write() may try to use it before initialization. Just check if it is initialized. #6080 (Ivan)

Build/Testing/Packaging Improvement

• Added official rpm packages. #5740 (proller) (alesapin)
• Add an ability to build .rpm and .tgz packages with packager script. #5769 (alesapin)
• Fixes for “Arcadia” build system. #6223 (proller)

Backward Incompatible Change

• Kafka is broken in this version.

ClickHouse release 19.11.3.11, 2019-07-18

New Feature

• Added support for prepared statements. #5331 (Alexander) #5630 (alexey-milovidov)
• DoubleDelta and Gorilla column codecs #5600 (Vasily Nemkov)
• Added os_thread_priority setting that allows to control the “nice” value of query processing threads that is used by OS to adjust dynamic scheduling priority. It requires CAP_SYS_NICE capabilities to work. This implements #5858 #5909 (alexey-milovidov)
• Implement _topic, _offset, _key columns for Kafka engine #5382 (Ivan) Note that Kafka is broken in this version.
• Add aggregate function combinator -Resample #5590 (hcz)
• Aggregate functions groupArrayMovingSum(win_size)(x) and groupArrayMovingAvg(win_size)(x), which calculate moving sum/avg with or without window-size limitation. #5595 (inv2004)
• Add synonim arrayFlatten <-> flatten #5764 (hcz)
• Intergate H3 function geoToH3 from Uber. #4724 (Remen Ivan) #5805 (alexey-milovidov)

Bug Fix

• Implement DNS cache with asynchronous update. Separate thread resolves all hosts and updates DNS cache with period (setting dns_cache_update_period). It should help, when ip of hosts changes frequently. #5857 (Anton Popov)
• Fix segfault in Delta codec which affects columns with values less than 32 bits size. The bug led to random memory corruption. #5786 (alesapin)
• Fix segfault in TTL merge with non-physical columns in block. #5819 (Anton Popov)
• Fix rare bug in checking of part with LowCardinality column. Previously checkDataPart always fails for part with LowCardinality column. #5832 (alesapin)
• Avoid hanging connections when server thread pool is full. It is important for connections from remote table function or connections to a shard without replicas when there is long connection timeout. This fixes #5878 #5881 (alexey-milovidov)
• Support for constant arguments to evalMLModel function. This fixes #5817 #5820 (alexey-milovidov)
• Fixed the issue when ClickHouse determines default time zone as UCT instead of UTC. This fixes #5804. #5828 (alexey-milovidov)
• Fixed buffer underflow in visitParamExtractRaw. This fixes #5901 #5902 (alexey-milovidov)
• Now distributed DROP/ALTER/TRUNCATE/OPTIMIZE ON CLUSTER queries will be executed directly on leader replica. #5757 (alesapin)
• Fix coalesce for ColumnConst with ColumnNullable + related changes. #5755 (Artem Zuikov)
• Fix the ReadBufferFromKafkaConsumer so that it keeps reading new messages after commit() even if it was stalled before #5852 (Ivan)
• Fix FULL and RIGHT JOIN results when joining on Nullable keys in right table. #5859 (Artem Zuikov)
• Possible fix of infinite sleeping of low-priority queries. #5842 (alexey-milovidov)
• Fix race condition, which cause that some queries may not appear in query_log after SYSTEM FLUSH LOGS query. #5456 #5685 (Anton Popov)
• Fixed heap-use-after-free ASan warning in ClusterCopier caused by watch which try to use already removed copier object. #5871 (Nikolai Kochetov)
• Fixed wrong StringRef pointer returned by some implementations of IColumn::deserializeAndInsertFromArena. This bug affected only unit-tests. #5973 (Nikolai Kochetov)
• Prevent source and intermediate array join columns of masking same name columns. #5941 (Artem Zuikov)
• Fix insert and select query to MySQL engine with MySQL style identifier quoting. #5704 (Winter Zhang)
• Now CHECK TABLE query can work with MergeTree engine family. It returns check status and message if any for each part (or file in case of simplier engines). Also, fix bug in fetch of a broken part. #5865 (alesapin)
• Fix SPLIT_SHARED_LIBRARIES runtime #5793 (Danila Kutenin)
• Fixed time zone initialization when /etc/localtime is a relative symlink like ../usr/share/zoneinfo/Europe/Moscow #5922 (alexey-milovidov)
• clickhouse-copier: Fix use-after free on shutdown #5752 (proller)
• Updated simdjson. Fixed the issue that some invalid JSONs with zero bytes successfully parse. #5938 (alexey-milovidov)
• Fix shutdown of SystemLogs #5802 (Anton Popov)
• Fix hanging when condition in invalidate_query depends on a dictionary. #6011 (Vitaly Baranov)

Improvement

• Allow unresolvable addresses in cluster configuration. They will be considered unavailable and tried to resolve at every connection attempt. This is especially useful for Kubernetes. This fixes #5714 #5924 (alexey-milovidov)
• Close idle TCP connections (with one hour timeout by default). This is especially important for large clusters with multiple distributed tables on every server, because every server can possibly keep a connection pool to every other server, and after peak query concurrency, connections will stall. This fixes #5879 #5880 (alexey-milovidov)
• Better quality of topK function. Changed the SavingSpace set behavior to remove the last element if the new element have a bigger weight. #5833 #5850 (Guillaume Tassery)
• URL functions to work with domains now can work for incomplete URLs without scheme #5725 (alesapin)
• Checksums added to the system.parts_columns table. #5874 (Nikita Mikhaylov)
• Added Enum data type as a synonim for Enum8 or Enum16. #5886 (dimarub2000)
• Full bit transpose variant for T64 codec. Could lead to better compression with zstd. #5742 (Artem Zuikov)
• Condition on startsWith function now can uses primary key. This fixes #5310 and #5882 #5919 (dimarub2000)
• Allow to use clickhouse-copier with cross-replication cluster topology by permitting empty database name. #5745 (nvartolomei)
• Use UTC as default timezone on a system without tzdata (e.g. bare Docker container). Before this patch, error message Could not determine local time zone was printed and server or client refused to start. #5827 (alexey-milovidov)
• Returned back support for floating point argument in function quantileTiming for backward compatibility. #5911 (alexey-milovidov)
• Show which table is missing column in error messages. #5768 (Ivan)
• Disallow run query with same query_id by various users #5430 (proller)
• More robust code for sending metrics to Graphite. It will work even during long multiple RENAME TABLE operation. #5875 (alexey-milovidov)
• More informative error messages will be displayed when ThreadPool cannot schedule a task for execution. This fixes #5305 #5801 (alexey-milovidov)
• Inverting ngramSearch to be more intuitive #5807 (Danila Kutenin)
• Add user parsing in HDFS engine builder #5946 (akonyaev90)
• Update default value of max_ast_elements parameter #5933 (Artem Konovalov)
• Added a notion of obsolete settings. The obsolete setting allow_experimental_low_cardinality_type can be used with no effect. 0f15c01c6802f7ce1a1494c12c846be8c98944cd Alexey Milovidov

Performance Improvement

• Increase number of streams to SELECT from Merge table for more uniform distribution of threads. Added setting max_streams_multiplier_for_merge_tables. This fixes #5797 #5915 (alexey-milovidov)

Build/Testing/Packaging Improvement

• Add a backward compatibility test for client-server interaction with different versions of clickhouse. #5868 (alesapin)
• Test coverage information in every commit and pull request. #5896 (alesapin)
• Cooperate with address sanitizer to support our custom allocators (Arena and ArenaWithFreeLists) for better debugging of “use-after-free” errors. #5728 (akuzm)
• Switch to LLVM libunwind implementation for C++ exception handling and for stack traces printing #4828 (Nikita Lapkov)
• Add two more warnings from -Weverything #5923 (alexey-milovidov)
• Allow to build ClickHouse with Memory Sanitizer. #3949 (alexey-milovidov)
• Fixed ubsan report about bitTest function in fuzz test. #5943 (alexey-milovidov)
• Docker: added possibility to init a ClickHouse instance which requires authentication. #5727 (Korviakov Andrey)
• Update librdkafka to version 1.1.0 #5872 (Ivan)
• Add global timeout for integration tests and disable some of them in tests code. #5741 (alesapin)
• Fix some ThreadSanitizer failures. #5854 (akuzm)
• The --no-undefined option forces the linker to check all external names for existence while linking. It’s very useful to track real dependencies between libraries in the split build mode. #5855 (Ivan)
• Added performance test for #5797 #5914 (alexey-milovidov)
• Fixed compatibility with gcc-7. #5840 (alexey-milovidov)
• Added support for gcc-9. This fixes #5717 #5774 (alexey-milovidov)
• Fixed error when libunwind can be linked incorrectly. #5948 (alexey-milovidov)
• Fixed a few warnings found by PVS-Studio. #5921 (alexey-milovidov)
• Added initial support for clang-tidy static analyzer. #5806 (alexey-milovidov)
• Convert BSD/Linux endian macros( ‘be64toh’ and ‘htobe64’) to the Mac OS X equivalents #5785 (Fu Chen)
• Improved integration tests guide. #5796 (Vladimir Chebotarev)
• Fixing build at macosx + gcc9 #5822 (filimonov)
• Fix a hard-to-spot typo: aggreAGte -> aggregate. #5753 (akuzm)
• Fix freebsd build #5760 (proller)
• Add link to experimental YouTube channel to website #5845 (Ivan Blinkov)
• CMake: add option for coverage flags: WITH_COVERAGE #5776 (proller)
• Fix initial size of some inline PODArray’s. #5787 (akuzm)
• clickhouse-server.postinst: fix os detection for centos 6 #5788 (proller)
• Added Arch linux package generation. #5719 (Vladimir Chebotarev)
• Split Common/config.h by libs (dbms) #5715 (proller)
• Fixes for “Arcadia” build platform #5795 (proller)
• Fixes for unconventional build (gcc9, no submodules) #5792 (proller)
• Require explicit type in unalignedStore because it was proven to be bug-prone #5791 (akuzm)
• Fixes MacOS build #5830 (filimonov)
• Performance test concerning the new JIT feature with bigger dataset, as requested here #5263 #5887 (Guillaume Tassery)
• Run stateful tests in stress test 12693e568722f11e19859742f56428455501fd2a (alesapin)

Backward Incompatible Change

• Kafka is broken in this version.
• Enable adaptive_index_granularity = 10MB by default for new MergeTree tables. If you created new MergeTree tables on version 19.11+, downgrade to versions prior to 19.6 will be impossible. #5628 (alesapin)
• Removed obsolete undocumented embedded dictionaries that were used by Yandex.Metrica. The functions OSIn, SEIn, OSToRoot, SEToRoot, OSHierarchy, SEHierarchy are no longer available. If you are using these functions, write email to [email protected]. Note: at the last moment we decided to keep these functions for a while. #5780 (alexey-milovidov)

ClickHouse release 19.10

ClickHouse release 19.10.1.5, 2019-07-12

New Feature

• Add new column codec: T64. Made for (U)IntX/EnumX/Data(Time)/DecimalX columns. It should be good for columns with constant or small range values. Codec itself allows enlarge or shrink data type without re-compression. #5557 (Artem Zuikov)
• Add database engine MySQL that allow to view all the tables in remote MySQL server #5599 (Winter Zhang)
• bitmapContains implementation. It’s 2x faster than bitmapHasAny if the second bitmap contains one element. #5535 (Zhichang Yu)
• Support for crc32 function (with behaviour exactly as in MySQL or PHP). Do not use it if you need a hash function. #5661 (Remen Ivan)
• Implemented SYSTEM START/STOP DISTRIBUTED SENDS queries to control asynchronous inserts into Distributed tables. #4935 (Winter Zhang)

Bug Fix

• Ignore query execution limits and max parts size for merge limits while executing mutations. #5659 (Anton Popov)
• Fix bug which may lead to deduplication of normal blocks (extremely rare) and insertion of duplicate blocks (more often). #5549 (alesapin)
• Fix of function arrayEnumerateUniqRanked for arguments with empty arrays #5559 (proller)
• Don’t subscribe to Kafka topics without intent to poll any messages. #5698 (Ivan)
• Make setting join_use_nulls get no effect for types that cannot be inside Nullable #5700 (Olga Khvostikova)
• Fixed Incorrect size of index granularity errors #5720 (coraxster)
• Fix Float to Decimal convert overflow #5607 (coraxster)
• Flush buffer when WriteBufferFromHDFS‘s destructor is called. This fixes writing into HDFS. #5684 (Xindong Peng)

Improvement

• Treat empty cells in CSV as default values when the setting input_format_defaults_for_omitted_fields is enabled. #5625 (akuzm)
• Non-blocking loading of external dictionaries. #5567 (Vitaly Baranov)
• Network timeouts can be dynamically changed for already established connections according to the settings. #4558 (Konstantin Podshumok)
• Using “public_suffix_list” for functions firstSignificantSubdomain, cutToFirstSignificantSubdomain. It’s using a perfect hash table generated by gperf with a list generated from the file: https://publicsuffix.org/list/public_suffix_list.dat. (for example, now we recognize the domain ac.uk as non-significant). #5030 (Guillaume Tassery)
• Adopted IPv6 data type in system tables; unified client info columns in system.processes and system.query_log #5640 (alexey-milovidov)
• Using sessions for connections with MySQL compatibility protocol. #5476 #5646 (Yuriy Baranov)
• Support more ALTER queries ON CLUSTER. #5593 #5613 (sundyli)
• Support <logger> section in clickhouse-local config file. #5540 (proller)
• Allow run query with remote table function in clickhouse-local #5627 (proller)

Performance Improvement

• Add the possibility to write the final mark at the end of MergeTree columns. It allows to avoid useless reads for keys that are out of table data range. It is enabled only if adaptive index granularity is in use. #5624 (alesapin)
• Improved performance of MergeTree tables on very slow filesystems by reducing number of stat syscalls. #5648 (alexey-milovidov)
• Fixed performance degradation in reading from MergeTree tables that was introduced in version 19.6. Fixes #5631. #5633 (alexey-milovidov)

Build/Testing/Packaging Improvement

• Implemented TestKeeper as an implementation of ZooKeeper interface used for testing #5643 (alexey-milovidov) (levushkin aleksej)
• From now on .sql tests can be run isolated by server, in parallel, with random database. It allows to run them faster, add new tests with custom server configurations, and be sure that different tests doesn’t affect each other. #5554 (Ivan)
• Remove <name> and <metrics> from performance tests #5672 (Olga Khvostikova)
• Fixed “select_format” performance test for Pretty formats #5642 (alexey-milovidov)

ClickHouse release 19.9

ClickHouse release 19.9.3.31, 2019-07-05

Bug Fix

• Fix segfault in Delta codec which affects columns with values less than 32 bits size. The bug led to random memory corruption. #5786 (alesapin)
• Fix rare bug in checking of part with LowCardinality column. #5832 (alesapin)
• Fix segfault in TTL merge with non-physical columns in block. #5819 (Anton Popov)
• Fix potential infinite sleeping of low-priority queries. #5842 (alexey-milovidov)
• Fix how ClickHouse determines default time zone as UCT instead of UTC. #5828 (alexey-milovidov)
• Fix bug about executing distributed DROP/ALTER/TRUNCATE/OPTIMIZE ON CLUSTER queries on follower replica before leader replica. Now they will be executed directly on leader replica. #5757 (alesapin)
• Fix race condition, which cause that some queries may not appear in query_log instantly after SYSTEM FLUSH LOGS query. #5685 (Anton Popov)
• Added missing support for constant arguments to evalMLModel function. #5820 (alexey-milovidov)

ClickHouse release 19.9.2.4, 2019-06-24

New Feature

• Print information about frozen parts in system.parts table. #5471 (proller)
• Ask client password on clickhouse-client start on tty if not set in arguments #5092 (proller)
• Implement dictGet and dictGetOrDefault functions for Decimal types. #5394 (Artem Zuikov)

Improvement

• Debian init: Add service stop timeout #5522 (proller)
• Add setting forbidden by default to create table with suspicious types for LowCardinality #5448 (Olga Khvostikova)
• Regression functions return model weights when not used as State in function evalMLMethod. #5411 (Quid37)
• Rename and improve regression methods. #5492 (Quid37)
• Clearer interfaces of string searchers. #5586 (Danila Kutenin)

Bug Fix

• Fix potential data loss in Kafka #5445 (Ivan)
• Fix potential infinite loop in PrettySpace format when called with zero columns #5560 (Olga Khvostikova)
• Fixed UInt32 overflow bug in linear models. Allow eval ML model for non-const model argument. #5516 (Nikolai Kochetov)
• ALTER TABLE ... DROP INDEX IF EXISTS ... should not raise an exception if provided index does not exist #5524 (Gleb Novikov)
• Fix segfault with bitmapHasAny in scalar subquery #5528 (Zhichang Yu)
• Fixed error when replication connection pool doesn’t retry to resolve host, even when DNS cache was dropped. #5534 (alesapin)
• Fixed ALTER ... MODIFY TTL on ReplicatedMergeTree. #5539 (Anton Popov)
• Fix INSERT into Distributed table with MATERIALIZED column #5429 (Azat Khuzhin)
• Fix bad alloc when truncate Join storage #5437 (TCeason)
• In recent versions of package tzdata some of files are symlinks now. The current mechanism for detecting default timezone gets broken and gives wrong names for some timezones. Now at least we force the timezone name to the contents of TZ if provided. #5443 (Ivan)
• Fix some extremely rare cases with MultiVolnitsky searcher when the constant needles in sum are at least 16KB long. The algorithm missed or overwrote the previous results which can lead to the incorrect result of multiSearchAny. #5588 (Danila Kutenin)
• Fix the issue when settings for ExternalData requests couldn’t use ClickHouse settings. Also, for now, settings date_time_input_format and low_cardinality_allow_in_native_format cannot be used because of the ambiguity of names (in external data it can be interpreted as table format and in the query it can be a setting). #5455 (Danila Kutenin)
• Fix bug when parts were removed only from FS without dropping them from Zookeeper. #5520 (alesapin)
• Remove debug logging from MySQL protocol #5478 (alexey-milovidov)
• Skip ZNONODE during DDL query processing #5489 (Azat Khuzhin)
• Fix mix UNION ALL result column type. There were cases with inconsistent data and column types of resulting columns. #5503 (Artem Zuikov)
• Throw an exception on wrong integers in dictGetT functions instead of crash. #5446 (Artem Zuikov)
• Fix wrong element_count and load_factor for hashed dictionary in system.dictionaries table. #5440 (Azat Khuzhin)

Build/Testing/Packaging Improvement

• Fixed build without Brotli HTTP compression support (ENABLE_BROTLI=OFF cmake variable). #5521 (Anton Yuzhaninov)
• Include roaring.h as roaring/roaring.h #5523 (Orivej Desh)
• Fix gcc9 warnings in hyperscan (#line directive is evil!) #5546 (Danila Kutenin)
• Fix all warnings when compiling with gcc-9. Fix some contrib issues. Fix gcc9 ICE and submit it to bugzilla. #5498 (Danila Kutenin)
• Fixed linking with lld #5477 (alexey-milovidov)
• Remove unused specializations in dictionaries #5452 (Artem Zuikov)
• Improvement performance tests for formatting and parsing tables for different types of files #5497 (Olga Khvostikova)
• Fixes for parallel test run #5506 (proller)
• Docker: use configs from clickhouse-test #5531 (proller)
• Fix compile for FreeBSD #5447 (proller)
• Upgrade boost to 1.70 #5570 (proller)
• Fix build clickhouse as submodule #5574 (proller)
• Improve JSONExtract performance tests #5444 (Vitaly Baranov)

ClickHouse release 19.8

ClickHouse release 19.8.3.8, 2019-06-11

New Features

• Added functions to work with JSON #4686 (hcz) #5124. (Vitaly Baranov)
• Add a function basename, with a similar behaviour to a basename function, which exists in a lot of languages (os.path.basename in python, basename in PHP, etc…). Work with both an UNIX-like path or a Windows path. #5136 (Guillaume Tassery)
• Added LIMIT n, m BY or LIMIT m OFFSET n BY syntax to set offset of n for LIMIT BY clause. #5138 (Anton Popov)
• Added new data type SimpleAggregateFunction, which allows to have columns with light aggregation in an AggregatingMergeTree. This can only be used with simple functions like any, anyLast, sum, min, max. #4629 (Boris Granveaud)
• Added support for non-constant arguments in function ngramDistance #5198 (Danila Kutenin)
• Added functions skewPop, skewSamp, kurtPop and kurtSamp to compute for sequence skewness, sample skewness, kurtosis and sample kurtosis respectively. #5200 (hcz)
• Support rename operation for MaterializeView storage. #5209 (Guillaume Tassery)
• Added server which allows connecting to ClickHouse using MySQL client. #4715 (Yuriy Baranov)
• Add toDecimal*OrZero and toDecimal*OrNull functions. #5291 (Artem Zuikov)
• Support Decimal types in functions: quantile, quantiles, median, quantileExactWeighted, quantilesExactWeighted, medianExactWeighted. #5304 (Artem Zuikov)
• Added toValidUTF8 function, which replaces all invalid UTF-8 characters by replacement character � (U+FFFD). #5322 (Danila Kutenin)
• Added format function. Formatting constant pattern (simplified Python format pattern) with the strings listed in the arguments. #5330 (Danila Kutenin)
• Added system.detached_parts table containing information about detached parts of MergeTree tables. #5353 (akuzm)
• Added ngramSearch function to calculate the non-symmetric difference between needle and haystack. #5418#5422 (Danila Kutenin)
• Implementation of basic machine learning methods (stochastic linear regression and logistic regression) using aggregate functions interface. Has different strategies for updating model weights (simple gradient descent, momentum method, Nesterov method). Also supports mini-batches of custom size. #4943 (Quid37)
• Implementation of geohashEncode and geohashDecode functions. #5003 (Vasily Nemkov)
• Added aggregate function timeSeriesGroupSum, which can aggregate different time series that sample timestamp not alignment. It will use linear interpolation between two sample timestamp and then sum time-series together. Added aggregate function timeSeriesGroupRateSum, which calculates the rate of time-series and then sum rates together. #4542 (Yangkuan Liu)
• Added functions IPv4CIDRtoIPv4Range and IPv6CIDRtoIPv6Range to calculate the lower and higher bounds for an IP in the subnet using a CIDR. #5095 (Guillaume Tassery)
• Add a X-ClickHouse-Summary header when we send a query using HTTP with enabled setting send_progress_in_http_headers. Return the usual information of X-ClickHouse-Progress, with additional information like how many rows and bytes were inserted in the query. #5116 (Guillaume Tassery)

Improvements

• Added max_parts_in_total setting for MergeTree family of tables (default: 100 000) that prevents unsafe specification of partition key #5166. #5171 (alexey-milovidov)
• clickhouse-obfuscator: derive seed for individual columns by combining initial seed with column name, not column position. This is intended to transform datasets with multiple related tables, so that tables will remain JOINable after transformation. #5178 (alexey-milovidov)
• Added functions JSONExtractRaw, JSONExtractKeyAndValues. Renamed functions jsonExtract<type> to JSONExtract<type>. When something goes wrong these functions return the correspondent values, not NULL. Modified function JSONExtract, now it gets the return type from its last parameter and doesn’t inject nullables. Implemented fallback to RapidJSON in case AVX2 instructions are not available. Simdjson library updated to a new version. #5235 (Vitaly Baranov)
• Now if and multiIf functions don’t rely on the condition’s Nullable, but rely on the branches for sql compatibility. #5238 (Jian Wu)
• In predicate now generates Null result from Null input like the Equal function. #5152 (Jian Wu)
• Check the time limit every (flush_interval / poll_timeout) number of rows from Kafka. This allows to break the reading from Kafka consumer more frequently and to check the time limits for the top-level streams #5249 (Ivan)
• Link rdkafka with bundled SASL. It should allow to use SASL SCRAM authentication #5253 (Ivan)
• Batched version of RowRefList for ALL JOINS. #5267 (Artem Zuikov)
• clickhouse-server: more informative listen error messages. #5268 (proller)
• Support dictionaries in clickhouse-copier for functions in <sharding_key> #5270 (proller)
• Add new setting kafka_commit_every_batch to regulate Kafka committing policy.
  It allows to set commit mode: after every batch of messages is handled, or after the whole block is written to the storage. It’s a trade-off between losing some messages or reading them twice in some extreme situations. #5308 (Ivan)
• Make windowFunnel support other Unsigned Integer Types. #5320 (sundyli)
• Allow to shadow virtual column _table in Merge engine. #5325 (Ivan)
• Make sequenceMatch aggregate functions support other unsigned Integer types #5339 (sundyli)
• Better error messages if checksum mismatch is most likely caused by hardware failures. #5355 (alexey-milovidov)
• Check that underlying tables support sampling for StorageMerge #5366 (Ivan)
• Сlose MySQL connections after their usage in external dictionaries. It is related to issue #893. #5395 (Clément Rodriguez)
• Improvements of MySQL Wire Protocol. Changed name of format to MySQLWire. Using RAII for calling RSA_free. Disabling SSL if context cannot be created. #5419 (Yuriy Baranov)
• clickhouse-client: allow to run with unaccessable history file (read-only, no disk space, file is directory, …). #5431 (proller)
• Respect query settings in asynchronous INSERTs into Distributed tables. #4936 (TCeason)
• Renamed functions leastSqr to simpleLinearRegression, LinearRegression to linearRegression, LogisticRegression to logisticRegression. #5391 (Nikolai Kochetov)

Performance Improvements

• Parallelize processing of parts of non-replicated MergeTree tables in ALTER MODIFY query. #4639 (Ivan Kush)
• Optimizations in regular expressions extraction. #5193 #5191 (Danila Kutenin)
• Do not add right join key column to join result if it’s used only in join on section. #5260 (Artem Zuikov)
• Freeze the Kafka buffer after first empty response. It avoids multiple invokations of ReadBuffer::next() for empty result in some row-parsing streams. #5283 (Ivan)
• concat function optimization for multiple arguments. #5357 (Danila Kutenin)
• Query optimisation. Allow push down IN statement while rewriting commа/cross join into inner one. #5396 (Artem Zuikov)
• Upgrade our LZ4 implementation with reference one to have faster decompression. #5070 (Danila Kutenin)
• Implemented MSD radix sort (based on kxsort), and partial sorting. #5129 (Evgenii Pravda)

Bug Fixes

• Fix push require columns with join #5192 (Winter Zhang)
• Fixed bug, when ClickHouse is run by systemd, the command sudo service clickhouse-server forcerestart was not working as expected. #5204 (proller)
• Fix http error codes in DataPartsExchange (interserver http server on 9009 port always returned code 200, even on errors). #5216 (proller)
• Fix SimpleAggregateFunction for String longer than MAX_SMALL_STRING_SIZE #5311 (Azat Khuzhin)
• Fix error for Decimal to Nullable(Decimal) conversion in IN. Support other Decimal to Decimal conversions (including different scales). #5350 (Artem Zuikov)
• Fixed FPU clobbering in simdjson library that lead to wrong calculation of uniqHLL and uniqCombined aggregate function and math functions such as log. #5354 (alexey-milovidov)
• Fixed handling mixed const/nonconst cases in JSON functions. #5435 (Vitaly Baranov)
• Fix retention function. Now all conditions that satisfy in a row of data are added to the data state. #5119 (小路)
• Fix result type for quantileExact with Decimals. #5304 (Artem Zuikov)

Documentation

• Translate documentation for CollapsingMergeTree to chinese. #5168 (张风啸)
• Translate some documentation about table engines to chinese.
  #5134
  #5328
  (never lee)

Build/Testing/Packaging Improvements

• Fix some sanitizer reports that show probable use-after-free.#5139 #5143 #5393 (Ivan)
• Move performance tests out of separate directories for convenience. #5158 (alexey-milovidov)
• Fix incorrect performance tests. #5255 (alesapin)
• Added a tool to calculate checksums caused by bit flips to debug hardware issues. #5334 (alexey-milovidov)
• Make runner script more usable. #5340#5360 (filimonov)
• Add small instruction how to write performance tests. #5408 (alesapin)
• Add ability to make substitutions in create, fill and drop query in performance tests #5367 (Olga Khvostikova)

ClickHouse release 19.7

ClickHouse release 19.7.5.29, 2019-07-05

Bug Fix

• Fix performance regression in some queries with JOIN. #5192 (Winter Zhang)

ClickHouse release 19.7.5.27, 2019-06-09

New features

• Added bitmap related functions bitmapHasAny and bitmapHasAll analogous to hasAny and hasAll functions for arrays. #5279 (Sergi Vladykin)

Bug Fixes

• Fix segfault on minmax INDEX with Null value. #5246 (Nikita Vasilev)
• Mark all input columns in LIMIT BY as required output. It fixes ‘Not found column’ error in some distributed queries. #5407 (Constantin S. Pan)
• Fix “Column ‘0’ already exists” error in SELECT .. PREWHERE on column with DEFAULT #5397 (proller)
• Fix ALTER MODIFY TTL query on ReplicatedMergeTree. #5539 (Anton Popov)
• Don’t crash the server when Kafka consumers have failed to start. #5285 (Ivan)
• Fixed bitmap functions produce wrong result. #5359 (Andy Yang)
• Fix element_count for hashed dictionary (do not include duplicates) #5440 (Azat Khuzhin)
• Use contents of environment variable TZ as the name for timezone. It helps to correctly detect default timezone in some cases.#5443 (Ivan)
• Do not try to convert integers in dictGetT functions, because it doesn’t work correctly. Throw an exception instead. #5446 (Artem Zuikov)
• Fix settings in ExternalData HTTP request. #5455 (Danila
  Kutenin)
• Fix bug when parts were removed only from FS without dropping them from Zookeeper. #5520 (alesapin)
• Fix segmentation fault in bitmapHasAny function. #5528 (Zhichang Yu)
• Fixed error when replication connection pool doesn’t retry to resolve host, even when DNS cache was dropped. #5534 (alesapin)
• Fixed DROP INDEX IF EXISTS query. Now ALTER TABLE ... DROP INDEX IF EXISTS ... query doesn’t raise an exception if provided index does not exist. #5524 (Gleb Novikov)
• Fix union all supertype column. There were cases with inconsistent data and column types of resulting columns. #5503 (Artem Zuikov)
• Skip ZNONODE during DDL query processing. Before if another node removes the znode in task queue, the one that
  did not process it, but already get list of children, will terminate the DDLWorker thread. #5489 (Azat Khuzhin)
• Fix INSERT into Distributed() table with MATERIALIZED column. #5429 (Azat Khuzhin)

ClickHouse release 19.7.3.9, 2019-05-30

New Features

• Allow to limit the range of a setting that can be specified by user.
  These constraints can be set up in user settings profile.
  #4931 (Vitaly
  Baranov)
• Add a second version of the function groupUniqArray with an optional
  max_size parameter that limits the size of the resulting array. This
  behavior is similar to groupArray(max_size)(x) function.
  #5026 (Guillaume
  Tassery)
• For TSVWithNames/CSVWithNames input file formats, column order can now be
  determined from file header. This is controlled by
  input_format_with_names_use_header parameter.
  #5081
  (Alexander)

Bug Fixes

• Crash with uncompressed_cache + JOIN during merge (#5197)
  #5133 (Danila
  Kutenin)
• Segmentation fault on a clickhouse-client query to system tables. #5066
  #5127
  (Ivan)
• Data loss on heavy load via KafkaEngine (#4736)
  #5080
  (Ivan)
• Fixed very rare data race condition that could happen when executing a query with UNION ALL involving at least two SELECTs from system.columns, system.tables, system.parts, system.parts_tables or tables of Merge family and performing ALTER of columns of the related tables concurrently. #5189 (alexey-milovidov)

Performance Improvements

• Use radix sort for sorting by single numeric column in ORDER BY without
  LIMIT. #5106,
  #4439
  (Evgenii Pravda,
  alexey-milovidov)

Documentation

• Translate documentation for some table engines to Chinese.
  #5107,
  #5094,
  #5087
  (张风啸),
  #5068 (never
  lee)

Build/Testing/Packaging Improvements

• Print UTF-8 characters properly in clickhouse-test.
  #5084
  (alexey-milovidov)
• Add command line parameter for clickhouse-client to always load suggestion
  data. #5102
  (alexey-milovidov)
• Resolve some of PVS-Studio warnings.
  #5082
  (alexey-milovidov)
• Update LZ4 #5040 (Danila
  Kutenin)
• Add gperf to build requirements for upcoming pull request #5030.
  #5110
  (proller)

ClickHouse release 19.6

ClickHouse release 19.6.3.18, 2019-06-13

Bug Fixes

• Fixed IN condition pushdown for queries from table functions mysql and odbc and corresponding table engines. This fixes #3540 and #2384. #5313 (alexey-milovidov)
• Fix deadlock in Zookeeper. #5297 (github1youlc)
• Allow quoted decimals in CSV. #5284 (Artem Zuikov
• Disallow conversion from float Inf/NaN into Decimals (throw exception). #5282 (Artem Zuikov)
• Fix data race in rename query. #5247 (Winter Zhang)
• Temporarily disable LFAlloc. Usage of LFAlloc might lead to a lot of MAP_FAILED in allocating UncompressedCache and in a result to crashes of queries at high loaded servers. cfdba93(Danila Kutenin)

ClickHouse release 19.6.2.11, 2019-05-13

New Features

• TTL expressions for columns and tables. #4212 (Anton Popov)
• Added support for brotli compression for HTTP responses (Accept-Encoding: br) #4388 (Mikhail)
• Added new function isValidUTF8 for checking whether a set of bytes is correctly utf-8 encoded. #4934 (Danila Kutenin)
• Add new load balancing policy first_or_random which sends queries to the first specified host and if it’s inaccessible send queries to random hosts of shard. Useful for cross-replication topology setups. #5012 (nvartolomei)

Experimental Features

• Add setting index_granularity_bytes (adaptive index granularity) for MergeTree* tables family. #4826 (alesapin)

Improvements

• Added support for non-constant and negative size and length arguments for function substringUTF8. #4989 (alexey-milovidov)
• Disable push-down to right table in left join, left table in right join, and both tables in full join. This fixes wrong JOIN results in some cases. #4846 (Ivan)
• clickhouse-copier: auto upload task configuration from --task-file option #4876 (proller)
• Added typos handler for storage factory and table functions factory. #4891 (Danila Kutenin)
• Support asterisks and qualified asterisks for multiple joins without subqueries #4898 (Artem Zuikov)
• Make missing column error message more user friendly. #4915 (Artem Zuikov)

Performance Improvements

• Significant speedup of ASOF JOIN #4924 (Martijn Bakker)

Backward Incompatible Changes

• HTTP header Query-Id was renamed to X-ClickHouse-Query-Id for consistency. #4972 (Mikhail)

Bug Fixes

• Fixed potential null pointer dereference in clickhouse-copier. #4900 (proller)
• Fixed error on query with JOIN + ARRAY JOIN #4938 (Artem Zuikov)
• Fixed hanging on start of the server when a dictionary depends on another dictionary via a database with engine=Dictionary. #4962 (Vitaly Baranov)
• Partially fix distributed_product_mode = local. It’s possible to allow columns of local tables in where/having/order by/… via table aliases. Throw exception if table does not have alias. There’s not possible to access to the columns without table aliases yet. #4986 (Artem Zuikov)
• Fix potentially wrong result for SELECT DISTINCT with JOIN #5001 (Artem Zuikov)
• Fixed very rare data race condition that could happen when executing a query with UNION ALL involving at least two SELECTs from system.columns, system.tables, system.parts, system.parts_tables or tables of Merge family and performing ALTER of columns of the related tables concurrently. #5189 (alexey-milovidov)

Build/Testing/Packaging Improvements

• Fixed test failures when running clickhouse-server on different host #4713 (Vasily Nemkov)
• clickhouse-test: Disable color control sequences in non tty environment. #4937 (alesapin)
• clickhouse-test: Allow use any test database (remove test. qualification where it possible) #5008 (proller)
• Fix ubsan errors #5037 (Vitaly Baranov)
• Yandex LFAlloc was added to ClickHouse to allocate MarkCache and UncompressedCache data in different ways to catch segfaults more reliable #4995 (Danila Kutenin)
• Python util to help with backports and changelogs. #4949 (Ivan)

ClickHouse release 19.5

ClickHouse release 19.5.4.22, 2019-05-13

Bug fixes

• Fixed possible crash in bitmap* functions #5220 #5228 (Andy Yang)
• Fixed very rare data race condition that could happen when executing a query with UNION ALL involving at least two SELECTs from system.columns, system.tables, system.parts, system.parts_tables or tables of Merge family and performing ALTER of columns of the related tables concurrently. #5189 (alexey-milovidov)
• Fixed error Set for IN is not created yet in case of using single LowCardinality column in the left part of IN. This error happened if LowCardinality column was the part of primary key. #5031 #5154 (Nikolai Kochetov)
• Modification of retention function: If a row satisfies both the first and NTH condition, only the first satisfied condition is added to the data state. Now all conditions that satisfy in a row of data are added to the data state. #5119 (小路)

ClickHouse release 19.5.3.8, 2019-04-18

Bug fixes

• Fixed type of setting max_partitions_per_insert_block from boolean to UInt64. #5028 (Mohammad Hossein Sekhavat)

ClickHouse release 19.5.2.6, 2019-04-15

New Features

• Hyperscan multiple regular expression matching was added (functions multiMatchAny, multiMatchAnyIndex, multiFuzzyMatchAny, multiFuzzyMatchAnyIndex). #4780, #4841 (Danila Kutenin)
• multiSearchFirstPosition function was added. #4780 (Danila Kutenin)
• Implement the predefined expression filter per row for tables. #4792 (Ivan)
• A new type of data skipping indices based on bloom filters (can be used for equal, in and like functions). #4499 (Nikita Vasilev)
• Added ASOF JOIN which allows to run queries that join to the most recent value known. #4774 #4867 #4863 #4875 (Martijn Bakker, Artem Zuikov)
• Rewrite multiple COMMA JOIN to CROSS JOIN. Then rewrite them to INNER JOIN if possible. #4661 (Artem Zuikov)

Improvement

• topK and topKWeighted now supports custom loadFactor (fixes issue #4252). #4634 (Kirill Danshin)
• Allow to use parallel_replicas_count > 1 even for tables without sampling (the setting is simply ignored for them). In previous versions it was lead to exception. #4637 (Alexey Elymanov)
• Support for CREATE OR REPLACE VIEW. Allow to create a view or set a new definition in a single statement. #4654 (Boris Granveaud)
• Buffer table engine now supports PREWHERE. #4671 (Yangkuan Liu)
• Add ability to start replicated table without metadata in zookeeper in readonly mode. #4691 (alesapin)
• Fixed flicker of progress bar in clickhouse-client. The issue was most noticeable when using FORMAT Null with streaming queries. #4811 (alexey-milovidov)
• Allow to disable functions with hyperscan library on per user basis to limit potentially excessive and uncontrolled resource usage. #4816 (alexey-milovidov)
• Add version number logging in all errors. #4824 (proller)
• Added restriction to the multiMatch functions which requires string size to fit into unsigned int. Also added the number of arguments limit to the multiSearch functions. #4834 (Danila Kutenin)
• Improved usage of scratch space and error handling in Hyperscan. #4866 (Danila Kutenin)
• Fill system.graphite_detentions from a table config of *GraphiteMergeTree engine tables. #4584 (Mikhail f. Shiryaev)
• Rename trigramDistance function to ngramDistance and add more functions with CaseInsensitive and UTF. #4602 (Danila Kutenin)
• Improved data skipping indices calculation. #4640 (Nikita Vasilev)
• Keep ordinary, DEFAULT, MATERIALIZED and ALIAS columns in a single list (fixes issue #2867). #4707 (Alex Zatelepin)

Bug Fix

• Avoid std::terminate in case of memory allocation failure. Now std::bad_alloc exception is thrown as expected. #4665 (alexey-milovidov)
• Fixes capnproto reading from buffer. Sometimes files wasn’t loaded successfully by HTTP. #4674 (Vladislav)
• Fix error Unknown log entry type: 0 after OPTIMIZE TABLE FINAL query. #4683 (Amos Bird)
• Wrong arguments to hasAny or hasAll functions may lead to segfault. #4698 (alexey-milovidov)
• Deadlock may happen while executing DROP DATABASE dictionary query. #4701 (alexey-milovidov)
• Fix undefined behavior in median and quantile functions. #4702 (hcz)
• Fix compression level detection when network_compression_method in lowercase. Broken in v19.1. #4706 (proller)
• Fixed ignorance of <timezone>UTC</timezone> setting (fixes issue #4658). #4718 (proller)
• Fix histogram function behaviour with Distributed tables. #4741 (olegkv)
• Fixed tsan report destroy of a locked mutex. #4742 (alexey-milovidov)
• Fixed TSan report on shutdown due to race condition in system logs usage. Fixed potential use-after-free on shutdown when part_log is enabled. #4758 (alexey-milovidov)
• Fix recheck parts in ReplicatedMergeTreeAlterThread in case of error. #4772 (Nikolai Kochetov)
• Arithmetic operations on intermediate aggregate function states were not working for constant arguments (such as subquery results). #4776 (alexey-milovidov)
• Always backquote column names in metadata. Otherwise it’s impossible to create a table with column named index (server won’t restart due to malformed ATTACH query in metadata). #4782 (alexey-milovidov)
• Fix crash in ALTER ... MODIFY ORDER BY on Distributed table. #4790 (TCeason)
• Fix segfault in JOIN ON with enabled enable_optimize_predicate_expression. #4794 (Winter Zhang)
• Fix bug with adding an extraneous row after consuming a protobuf message from Kafka. #4808 (Vitaly Baranov)
• Fix crash of JOIN on not-nullable vs nullable column. Fix NULLs in right keys in ANY JOIN + join_use_nulls. #4815 (Artem Zuikov)
• Fix segmentation fault in clickhouse-copier. #4835 (proller)
• Fixed race condition in SELECT from system.tables if the table is renamed or altered concurrently. #4836 (alexey-milovidov)
• Fixed data race when fetching data part that is already obsolete. #4839 (alexey-milovidov)
• Fixed rare data race that can happen during RENAME table of MergeTree family. #4844 (alexey-milovidov)
• Fixed segmentation fault in function arrayIntersect. Segmentation fault could happen if function was called with mixed constant and ordinary arguments. #4847 (Lixiang Qian)
• Fixed reading from Array(LowCardinality) column in rare case when column contained a long sequence of empty arrays. #4850 (Nikolai Kochetov)
• Fix crash in FULL/RIGHT JOIN when we joining on nullable vs not nullable. #4855 (Artem Zuikov)
• Fix No message received exception while fetching parts between replicas. #4856 (alesapin)
• Fixed arrayIntersect function wrong result in case of several repeated values in single array. #4871 (Nikolai Kochetov)
• Fix a race condition during concurrent ALTER COLUMN queries that could lead to a server crash (fixes issue #3421). #4592 (Alex Zatelepin)
• Fix incorrect result in FULL/RIGHT JOIN with const column. #4723 (Artem Zuikov)
• Fix duplicates in GLOBAL JOIN with asterisk. #4705 (Artem Zuikov)
• Fix parameter deduction in ALTER MODIFY of column CODEC when column type is not specified. #4883 (alesapin)
• Functions cutQueryStringAndFragment() and queryStringAndFragment() now works correctly when URL contains a fragment and no query. #4894 (Vitaly Baranov)
• Fix rare bug when setting min_bytes_to_use_direct_io is greater than zero, which occures when thread have to seek backward in column file. #4897 (alesapin)
• Fix wrong argument types for aggregate functions with LowCardinality arguments (fixes issue #4919). #4922 (Nikolai Kochetov)
• Fix wrong name qualification in GLOBAL JOIN. #4969 (Artem Zuikov)
• Fix function toISOWeek result for year 1970. #4988 (alexey-milovidov)
• Fix DROP, TRUNCATE and OPTIMIZE queries duplication, when executed on ON CLUSTER for ReplicatedMergeTree* tables family. #4991 (alesapin)

Backward Incompatible Change

• Rename setting insert_sample_with_metadata to setting input_format_defaults_for_omitted_fields. #4771 (Artem Zuikov)
• Added setting max_partitions_per_insert_block (with value 100 by default). If inserted block contains larger number of partitions, an exception is thrown. Set it to 0 if you want to remove the limit (not recommended). #4845 (alexey-milovidov)
• Multi-search functions were renamed (multiPosition to multiSearchAllPositions, multiSearch to multiSearchAny, firstMatch to multiSearchFirstIndex). #4780 (Danila Kutenin)

Performance Improvement

• Optimize Volnitsky searcher by inlining, giving about 5-10% search improvement for queries with many needles or many similar bigrams. #4862 (Danila Kutenin)
• Fix performance issue when setting use_uncompressed_cache is greater than zero, which appeared when all read data contained in cache. #4913 (alesapin)

Build/Testing/Packaging Improvement

• Hardening debug build: more granular memory mappings and ASLR; add memory protection for mark cache and index. This allows to find more memory stomping bugs in case when ASan and MSan cannot do it. #4632 (alexey-milovidov)
• Add support for cmake variables ENABLE_PROTOBUF, ENABLE_PARQUET and ENABLE_BROTLI which allows to enable/disable the above features (same as we can do for librdkafka, mysql, etc). #4669 (Silviu Caragea)
• Add ability to print process list and stacktraces of all threads if some queries are hung after test run. #4675 (alesapin)
• Add retries on Connection loss error in clickhouse-test. #4682 (alesapin)
• Add freebsd build with vagrant and build with thread sanitizer to packager script. #4712 #4748 (alesapin)
• Now user asked for password for user 'default' during installation. #4725 (proller)
• Suppress warning in rdkafka library. #4740 (alexey-milovidov)
• Allow ability to build without ssl. #4750 (proller)
• Add a way to launch clickhouse-server image from a custom user. #4753 (Mikhail f. Shiryaev)
• Upgrade contrib boost to 1.69. #4793 (proller)
• Disable usage of mremap when compiled with Thread Sanitizer. Surprisingly enough, TSan does not intercept mremap (though it does intercept mmap, munmap) that leads to false positives. Fixed TSan report in stateful tests. #4859 (alexey-milovidov)
• Add test checking using format schema via HTTP interface. #4864 (Vitaly Baranov)

ClickHouse release 19.4

ClickHouse release 19.4.4.33, 2019-04-17

Bug Fixes

• Avoid std::terminate in case of memory allocation failure. Now std::bad_alloc exception is thrown as expected. #4665 (alexey-milovidov)
• Fixes capnproto reading from buffer. Sometimes files wasn’t loaded successfully by HTTP. #4674 (Vladislav)
• Fix error Unknown log entry type: 0 after OPTIMIZE TABLE FINAL query. #4683 (Amos Bird)
• Wrong arguments to hasAny or hasAll functions may lead to segfault. #4698 (alexey-milovidov)
• Deadlock may happen while executing DROP DATABASE dictionary query. #4701 (alexey-milovidov)
• Fix undefined behavior in median and quantile functions. #4702 (hcz)
• Fix compression level detection when network_compression_method in lowercase. Broken in v19.1. #4706 (proller)
• Fixed ignorance of <timezone>UTC</timezone> setting (fixes issue #4658). #4718 (proller)
• Fix histogram function behaviour with Distributed tables. #4741 (olegkv)
• Fixed tsan report destroy of a locked mutex. #4742 (alexey-milovidov)
• Fixed TSan report on shutdown due to race condition in system logs usage. Fixed potential use-after-free on shutdown when part_log is enabled. #4758 (alexey-milovidov)
• Fix recheck parts in ReplicatedMergeTreeAlterThread in case of error. #4772 (Nikolai Kochetov)
• Arithmetic operations on intermediate aggregate function states were not working for constant arguments (such as subquery results). #4776 (alexey-milovidov)
• Always backquote column names in metadata. Otherwise it’s impossible to create a table with column named index (server won’t restart due to malformed ATTACH query in metadata). #4782 (alexey-milovidov)
• Fix crash in ALTER ... MODIFY ORDER BY on Distributed table. #4790 (TCeason)
• Fix segfault in JOIN ON with enabled enable_optimize_predicate_expression. #4794 (Winter Zhang)
• Fix bug with adding an extraneous row after consuming a protobuf message from Kafka. #4808 (Vitaly Baranov)
• Fix segmentation fault in clickhouse-copier. #4835 (proller)
• Fixed race condition in SELECT from system.tables if the table is renamed or altered concurrently. #4836 (alexey-milovidov)
• Fixed data race when fetching data part that is already obsolete. #4839 (alexey-milovidov)
• Fixed rare data race that can happen during RENAME table of MergeTree family. #4844 (alexey-milovidov)
• Fixed segmentation fault in function arrayIntersect. Segmentation fault could happen if function was called with mixed constant and ordinary arguments. #4847 (Lixiang Qian)
• Fixed reading from Array(LowCardinality) column in rare case when column contained a long sequence of empty arrays. #4850 (Nikolai Kochetov)
• Fix No message received exception while fetching parts between replicas. #4856 (alesapin)
• Fixed arrayIntersect function wrong result in case of several repeated values in single array. #4871 (Nikolai Kochetov)
• Fix a race condition during concurrent ALTER COLUMN queries that could lead to a server crash (fixes issue #3421). #4592 (Alex Zatelepin)
• Fix parameter deduction in ALTER MODIFY of column CODEC when column type is not specified. #4883 (alesapin)
• Functions cutQueryStringAndFragment() and queryStringAndFragment() now works correctly when URL contains a fragment and no query. #4894 (Vitaly Baranov)
• Fix rare bug when setting min_bytes_to_use_direct_io is greater than zero, which occures when thread have to seek backward in column file. #4897 (alesapin)
• Fix wrong argument types for aggregate functions with LowCardinality arguments (fixes issue #4919). #4922 (Nikolai Kochetov)
• Fix function toISOWeek result for year 1970. #4988 (alexey-milovidov)
• Fix DROP, TRUNCATE and OPTIMIZE queries duplication, when executed on ON CLUSTER for ReplicatedMergeTree* tables family. #4991 (alesapin)

Improvements

• Keep ordinary, DEFAULT, MATERIALIZED and ALIAS columns in a single list (fixes issue #2867). #4707 (Alex Zatelepin)

ClickHouse release 19.4.3.11, 2019-04-02

Bug Fixes

• Fix crash in FULL/RIGHT JOIN when we joining on nullable vs not nullable. #4855 (Artem Zuikov)
• Fix segmentation fault in clickhouse-copier. #4835 (proller)

Build/Testing/Packaging Improvement

• Add a way to launch clickhouse-server image from a custom user. #4753 (Mikhail f. Shiryaev)

ClickHouse release 19.4.2.7, 2019-03-30

Bug Fixes

• Fixed reading from Array(LowCardinality) column in rare case when column contained a long sequence of empty arrays. #4850 (Nikolai Kochetov)

ClickHouse release 19.4.1.3, 2019-03-19

Bug Fixes

• Fixed remote queries which contain both LIMIT BY and LIMIT. Previously, if LIMIT BY and LIMIT were used for remote query, LIMIT could happen before LIMIT BY, which led to too filtered result. #4708 (Constantin S. Pan)

ClickHouse release 19.4.0.49, 2019-03-09

New Features

• Added full support for Protobuf format (input and output, nested data structures). #4174 #4493 (Vitaly Baranov)
• Added bitmap functions with Roaring Bitmaps. #4207 (Andy Yang) #4568 (Vitaly Baranov)
• Parquet format support. #4448 (proller)
• N-gram distance was added for fuzzy string comparison. It is similar to q-gram metrics in R language. #4466 (Danila Kutenin)
• Combine rules for graphite rollup from dedicated aggregation and retention patterns. #4426 (Mikhail f. Shiryaev)
• Added max_execution_speed and max_execution_speed_bytes to limit resource usage. Added min_execution_speed_bytes setting to complement the min_execution_speed. #4430 (Winter Zhang)
• Implemented function flatten. #4555 #4409 (alexey-milovidov, kzon)
• Added functions arrayEnumerateDenseRanked and arrayEnumerateUniqRanked (it’s like arrayEnumerateUniq but allows to fine tune array depth to look inside multidimensional arrays). #4475 (proller) #4601 (alexey-milovidov)
• Multiple JOINS with some restrictions: no asterisks, no complex aliases in ON/WHERE/GROUP BY/… #4462 (Artem Zuikov)

Bug Fixes

• This release also contains all bug fixes from 19.3 and 19.1.
• Fixed bug in data skipping indices: order of granules after INSERT was incorrect. #4407 (Nikita Vasilev)
• Fixed set index for Nullable and LowCardinality columns. Before it, set index with Nullable or LowCardinality column led to error Data type must be deserialized with multiple streams while selecting. #4594 (Nikolai Kochetov)
• Correctly set update_time on full executable dictionary update. #4551 (Tema Novikov)
• Fix broken progress bar in 19.3. #4627 (filimonov)
• Fixed inconsistent values of MemoryTracker when memory region was shrinked, in certain cases. #4619 (alexey-milovidov)
• Fixed undefined behaviour in ThreadPool. #4612 (alexey-milovidov)
• Fixed a very rare crash with the message mutex lock failed: Invalid argument that could happen when a MergeTree table was dropped concurrently with a SELECT. #4608 (Alex Zatelepin)
• ODBC driver compatibility with LowCardinality data type. #4381 (proller)
• FreeBSD: Fixup for AIOcontextPool: Found io_event with unknown id 0 error. #4438 (urgordeadbeef)
• system.part_log table was created regardless to configuration. #4483 (alexey-milovidov)
• Fix undefined behaviour in dictIsIn function for cache dictionaries. #4515 (alesapin)
• Fixed a deadlock when a SELECT query locks the same table multiple times (e.g. from different threads or when executing multiple subqueries) and there is a concurrent DDL query. #4535 (Alex Zatelepin)
• Disable compile_expressions by default until we get own llvm contrib and can test it with clang and asan. #4579 (alesapin)
• Prevent std::terminate when invalidate_query for clickhouse external dictionary source has returned wrong resultset (empty or more than one row or more than one column). Fixed issue when the invalidate_query was performed every five seconds regardless to the lifetime. #4583 (alexey-milovidov)
• Avoid deadlock when the invalidate_query for a dictionary with clickhouse source was involving system.dictionaries table or Dictionaries database (rare case). #4599 (alexey-milovidov)
• Fixes for CROSS JOIN with empty WHERE. #4598 (Artem Zuikov)
• Fixed segfault in function “replicate” when constant argument is passed. #4603 (alexey-milovidov)
• Fix lambda function with predicate optimizer. #4408 (Winter Zhang)
• Multiple JOINs multiple fixes. #4595 (Artem Zuikov)

Improvements

• Support aliases in JOIN ON section for right table columns. #4412 (Artem Zuikov)
• Result of multiple JOINs need correct result names to be used in subselects. Replace flat aliases with source names in result. #4474 (Artem Zuikov)
• Improve push-down logic for joined statements. #4387 (Ivan)

Performance Improvements

• Improved heuristics of “move to PREWHERE” optimization. #4405 (alexey-milovidov)
• Use proper lookup tables that uses HashTable’s API for 8-bit and 16-bit keys. #4536 (Amos Bird)
• Improved performance of string comparison. #4564 (alexey-milovidov)
• Cleanup distributed DDL queue in a separate thread so that it doesn’t slow down the main loop that processes distributed DDL tasks. #4502 (Alex Zatelepin)
• When min_bytes_to_use_direct_io is set to 1, not every file was opened with O_DIRECT mode because the data size to read was sometimes underestimated by the size of one compressed block. #4526 (alexey-milovidov)

Build/Testing/Packaging Improvement

• Added support for clang-9 #4604 (alexey-milovidov)
• Fix wrong __asm__ instructions (again) #4621 (Konstantin Podshumok)
• Add ability to specify settings for clickhouse-performance-test from command line. #4437 (alesapin)
• Add dictionaries tests to integration tests. #4477 (alesapin)
• Added queries from the benchmark on the website to automated performance tests. #4496 (alexey-milovidov)
• xxhash.h does not exist in external lz4 because it is an implementation detail and its symbols are namespaced with XXH_NAMESPACE macro. When lz4 is external, xxHash has to be external too, and the dependents have to link to it. #4495 (Orivej Desh)
• Fixed a case when quantileTiming aggregate function can be called with negative or floating point argument (this fixes fuzz test with undefined behaviour sanitizer). #4506 (alexey-milovidov)
• Spelling error correction. #4531 (sdk2)
• Fix compilation on Mac. #4371 (Vitaly Baranov)
• Build fixes for FreeBSD and various unusual build configurations. #4444 (proller)

ClickHouse release 19.3

ClickHouse release 19.3.9.1, 2019-04-02

Bug Fixes

• Fix crash in FULL/RIGHT JOIN when we joining on nullable vs not nullable. #4855 (Artem Zuikov)
• Fix segmentation fault in clickhouse-copier. #4835 (proller)
• Fixed reading from Array(LowCardinality) column in rare case when column contained a long sequence of empty arrays. #4850 (Nikolai Kochetov)

Build/Testing/Packaging Improvement

• Add a way to launch clickhouse-server image from a custom user #4753 (Mikhail f. Shiryaev)

ClickHouse release 19.3.7, 2019-03-12

Bug fixes

• Fixed error in #3920. This error manifests itself as random cache corruption (messages Unknown codec family code, Cannot seek through file) and segfaults. This bug first appeared in version 19.1 and is present in versions up to 19.1.10 and 19.3.6. #4623 (alexey-milovidov)

ClickHouse release 19.3.6, 2019-03-02

Bug fixes

• When there are more than 1000 threads in a thread pool, std::terminate may happen on thread exit. Azat Khuzhin #4485 #4505 (alexey-milovidov)
• Now it’s possible to create ReplicatedMergeTree* tables with comments on columns without defaults and tables with columns codecs without comments and defaults. Also fix comparison of codecs. #4523 (alesapin)
• Fixed crash on JOIN with array or tuple. #4552 (Artem Zuikov)
• Fixed crash in clickhouse-copier with the message ThreadStatus not created. #4540 (Artem Zuikov)
• Fixed hangup on server shutdown if distributed DDLs were used. #4472 (Alex Zatelepin)
• Incorrect column numbers were printed in error message about text format parsing for columns with number greater than 10. #4484 (alexey-milovidov)

Build/Testing/Packaging Improvements

• Fixed build with AVX enabled. #4527 (alexey-milovidov)
• Enable extended accounting and IO accounting based on good known version instead of kernel under which it is compiled. #4541 (nvartolomei)
• Allow to skip setting of core_dump.size_limit, warning instead of throw if limit set fail. #4473 (proller)
• Removed the inline tags of void readBinary(...) in Field.cpp. Also merged redundant namespace DB blocks. #4530 (hcz)

ClickHouse release 19.3.5, 2019-02-21

Bug fixes

• Fixed bug with large http insert queries processing. #4454 (alesapin)
• Fixed backward incompatibility with old versions due to wrong implementation of send_logs_level setting. #4445 (alexey-milovidov)
• Fixed backward incompatibility of table function remote introduced with column comments. #4446 (alexey-milovidov)

ClickHouse release 19.3.4, 2019-02-16

Improvements

• Table index size is not accounted for memory limits when doing ATTACH TABLE query. Avoided the possibility that a table cannot be attached after being detached. #4396 (alexey-milovidov)
• Slightly raised up the limit on max string and array size received from ZooKeeper. It allows to continue to work with increased size of CLIENT_JVMFLAGS=-Djute.maxbuffer=... on ZooKeeper. #4398 (alexey-milovidov)
• Allow to repair abandoned replica even if it already has huge number of nodes in its queue. #4399 (alexey-milovidov)
• Add one required argument to SET index (max stored rows number). #4386 (Nikita Vasilev)

Bug Fixes

• Fixed WITH ROLLUP result for group by single LowCardinality key. #4384 (Nikolai Kochetov)
• Fixed bug in the set index (dropping a granule if it contains more than max_rows rows). #4386 (Nikita Vasilev)
• A lot of FreeBSD build fixes. #4397 (proller)
• Fixed aliases substitution in queries with subquery containing same alias (issue #4110). #4351 (Artem Zuikov)

Build/Testing/Packaging Improvements

• Add ability to run clickhouse-server for stateless tests in docker image. #4347 (Vasily Nemkov)

ClickHouse release 19.3.3, 2019-02-13

New Features

• Added the KILL MUTATION statement that allows removing mutations that are for some reasons stuck. Added latest_failed_part, latest_fail_time, latest_fail_reason fields to the system.mutations table for easier troubleshooting. #4287 (Alex Zatelepin)
• Added aggregate function entropy which computes Shannon entropy. #4238 (Quid37)
• Added ability to send queries INSERT INTO tbl VALUES (.... to server without splitting on query and data parts. #4301 (alesapin)
• Generic implementation of arrayWithConstant function was added. #4322 (alexey-milovidov)
• Implemented NOT BETWEEN comparison operator. #4228 (Dmitry Naumov)
• Implement sumMapFiltered in order to be able to limit the number of keys for which values will be summed by sumMap. #4129 (Léo Ercolanelli)
• Added support of Nullable types in mysql table function. #4198 (Emmanuel Donin de Rosière)
• Support for arbitrary constant expressions in LIMIT clause. #4246 (k3box)
• Added topKWeighted aggregate function that takes additional argument with (unsigned integer) weight. #4245 (Andrew Golman)
• StorageJoin now supports join_any_take_last_row setting that allows overwriting existing values of the same key. #3973 (Amos Bird
• Added function toStartOfInterval. #4304 (Vitaly Baranov)
• Added RowBinaryWithNamesAndTypes format. #4200 (Oleg V. Kozlyuk)
• Added IPv4 and IPv6 data types. More effective implementations of IPv* functions. #3669 (Vasily Nemkov)
• Added function toStartOfTenMinutes(). #4298 (Vitaly Baranov)
• Added Protobuf output format. #4005 #4158 (Vitaly Baranov)
• Added brotli support for HTTP interface for data import (INSERTs). #4235 (Mikhail )
• Added hints while user make typo in function name or type in command line client. #4239 (Danila Kutenin)
• Added Query-Id to Server’s HTTP Response header. #4231 (Mikhail )

Experimental features

• Added minmax and set data skipping indices for MergeTree table engines family. #4143 (Nikita Vasilev)
• Added conversion of CROSS JOIN to INNER JOIN if possible. #4221 #4266 (Artem Zuikov)

Bug Fixes

• Fixed Not found column for duplicate columns in JOIN ON section. #4279 (Artem Zuikov)
• Make START REPLICATED SENDS command start replicated sends. #4229 (nvartolomei)
• Fixed aggregate functions execution with Array(LowCardinality) arguments. #4055 (KochetovNicolai)
• Fixed wrong behaviour when doing INSERT ... SELECT ... FROM file(...) query and file has CSVWithNames or TSVWIthNames format and the first data row is missing. #4297 (alexey-milovidov)
• Fixed crash on dictionary reload if dictionary not available. This bug was appeared in 19.1.6. #4188 (proller)
• Fixed ALL JOIN with duplicates in right table. #4184 (Artem Zuikov)
• Fixed segmentation fault with use_uncompressed_cache=1 and exception with wrong uncompressed size. This bug was appeared in 19.1.6. #4186 (alesapin)
• Fixed compile_expressions bug with comparison of big (more than int16) dates. #4341 (alesapin)
• Fixed infinite loop when selecting from table function numbers(0). #4280 (alexey-milovidov)
• Temporarily disable predicate optimization for ORDER BY. #3890 (Winter Zhang)
• Fixed Illegal instruction error when using base64 functions on old CPUs. This error has been reproduced only when ClickHouse was compiled with gcc-8. #4275 (alexey-milovidov)
• Fixed No message received error when interacting with PostgreSQL ODBC Driver through TLS connection. Also fixes segfault when using MySQL ODBC Driver. #4170 (alexey-milovidov)
• Fixed incorrect result when Date and DateTime arguments are used in branches of conditional operator (function if). Added generic case for function if. #4243 (alexey-milovidov)
• ClickHouse dictionaries now load within clickhouse process. #4166 (alexey-milovidov)
• Fixed deadlock when SELECT from a table with File engine was retried after No such file or directory error. #4161 (alexey-milovidov)
• Fixed race condition when selecting from system.tables may give table doesn't exist error. #4313 (alexey-milovidov)
• clickhouse-client can segfault on exit while loading data for command line suggestions if it was run in interactive mode. #4317 (alexey-milovidov)
• Fixed a bug when the execution of mutations containing IN operators was producing incorrect results. #4099 (Alex Zatelepin)
• Fixed error: if there is a database with Dictionary engine, all dictionaries forced to load at server startup, and if there is a dictionary with ClickHouse source from localhost, the dictionary cannot load. #4255 (alexey-milovidov)
• Fixed error when system logs are tried to create again at server shutdown. #4254 (alexey-milovidov)
• Correctly return the right type and properly handle locks in joinGet function. #4153 (Amos Bird)
• Added sumMapWithOverflow function. #4151 (Léo Ercolanelli)
• Fixed segfault with allow_experimental_multiple_joins_emulation. 52de2c (Artem Zuikov)
• Fixed bug with incorrect Date and DateTime comparison. #4237 (valexey)
• Fixed fuzz test under undefined behavior sanitizer: added parameter type check for quantile*Weighted family of functions. #4145 (alexey-milovidov)
• Fixed rare race condition when removing of old data parts can fail with File not found error. #4378 (alexey-milovidov)
• Fix install package with missing /etc/clickhouse-server/config.xml. #4343 (proller)

Build/Testing/Packaging Improvements

• Debian package: correct /etc/clickhouse-server/preprocessed link according to config. #4205 (proller)
• Various build fixes for FreeBSD. #4225 (proller)
• Added ability to create, fill and drop tables in perftest. #4220 (alesapin)
• Added a script to check for duplicate includes. #4326 (alexey-milovidov)
• Added ability to run queries by index in performance test. #4264 (alesapin)
• Package with debug symbols is suggested to be installed. #4274 (alexey-milovidov)
• Refactoring of performance-test. Better logging and signals handling. #4171 (alesapin)
• Added docs to anonymized Yandex.Metrika datasets. #4164 (alesapin)
• Аdded tool for converting an old month-partitioned part to the custom-partitioned format. #4195 (Alex Zatelepin)
• Added docs about two datasets in s3. #4144 (alesapin)
• Added script which creates changelog from pull requests description. #4169 #4173 (KochetovNicolai) (KochetovNicolai)
• Added puppet module for Clickhouse. #4182 (Maxim Fedotov)
• Added docs for a group of undocumented functions. #4168 (Winter Zhang)
• ARM build fixes. #4210#4306 #4291 (proller) (proller)
• Dictionary tests now able to run from ctest. #4189 (proller)
• Now /etc/ssl is used as default directory with SSL certificates. #4167 (alexey-milovidov)
• Added checking SSE and AVX instruction at start. #4234 (Igr)
• Init script will wait server until start. #4281 (proller)

Backward Incompatible Changes

• Removed allow_experimental_low_cardinality_type setting. LowCardinality data types are production ready. #4323 (alexey-milovidov)
• Reduce mark cache size and uncompressed cache size accordingly to available memory amount. #4240 (Lopatin Konstantin
• Added keyword INDEX in CREATE TABLE query. A column with name index must be quoted with backticks or double quotes: `index` . #4143 (Nikita Vasilev)
• sumMap now promote result type instead of overflow. The old sumMap behavior can be obtained by using sumMapWithOverflow function. #4151 (Léo Ercolanelli)

Performance Improvements

• std::sort replaced by pdqsort for queries without LIMIT. #4236 (Evgenii Pravda)
• Now server reuse threads from global thread pool. This affects performance in some corner cases. #4150 (alexey-milovidov)

Improvements

• Implemented AIO support for FreeBSD. #4305 (urgordeadbeef)
• SELECT * FROM a JOIN b USING a, b now return a and b columns only from the left table. #4141 (Artem Zuikov)
• Allow -C option of client to work as -c option. #4232 (syominsergey)
• Now option --password used without value requires password from stdin. #4230 (BSD_Conqueror)
• Added highlighting of unescaped metacharacters in string literals that contain LIKE expressions or regexps. #4327 (alexey-milovidov)
• Added cancelling of HTTP read only queries if client socket goes away. #4213 (nvartolomei)
• Now server reports progress to keep client connections alive. #4215 (Ivan)
• Slightly better message with reason for OPTIMIZE query with optimize_throw_if_noop setting enabled. #4294 (alexey-milovidov)
• Added support of --version option for clickhouse server. #4251 (Lopatin Konstantin)
• Added --help/-h option to clickhouse-server. #4233 (Yuriy Baranov)
• Added support for scalar subqueries with aggregate function state result. #4348 (Nikolai Kochetov)
• Improved server shutdown time and ALTERs waiting time. #4372 (alexey-milovidov)
• Added info about the replicated_can_become_leader setting to system.replicas and add logging if the replica won’t try to become leader. #4379 (Alex Zatelepin)

ClickHouse release 19.1

ClickHouse release 19.1.14, 2019-03-14

• Fixed error Column ... queried more than once that may happen if the setting asterisk_left_columns_only is set to 1 in case of using GLOBAL JOIN with SELECT * (rare case). The issue does not exist in 19.3 and newer. 6bac7d8d (Artem Zuikov)

ClickHouse release 19.1.13, 2019-03-12

This release contains exactly the same set of patches as 19.3.7.

ClickHouse release 19.1.10, 2019-03-03

This release contains exactly the same set of patches as 19.3.6.

ClickHouse release 19.1

ClickHouse release 19.1.9, 2019-02-21

Bug fixes

• Fixed backward incompatibility with old versions due to wrong implementation of send_logs_level setting. #4445 (alexey-milovidov)
• Fixed backward incompatibility of table function remote introduced with column comments. #4446 (alexey-milovidov)

ClickHouse release 19.1.8, 2019-02-16

Bug Fixes

• Fix install package with missing /etc/clickhouse-server/config.xml. #4343 (proller)

ClickHouse release 19.1

ClickHouse release 19.1.7, 2019-02-15

Bug Fixes

• Correctly return the right type and properly handle locks in joinGet function. #4153 (Amos Bird)
• Fixed error when system logs are tried to create again at server shutdown. #4254 (alexey-milovidov)
• Fixed error: if there is a database with Dictionary engine, all dictionaries forced to load at server startup, and if there is a dictionary with ClickHouse source from localhost, the dictionary cannot load. #4255 (alexey-milovidov)
• Fixed a bug when the execution of mutations containing IN operators was producing incorrect results. #4099 (Alex Zatelepin)
• clickhouse-client can segfault on exit while loading data for command line suggestions if it was run in interactive mode. #4317 (alexey-milovidov)
• Fixed race condition when selecting from system.tables may give table doesn't exist error. #4313 (alexey-milovidov)
• Fixed deadlock when SELECT from a table with File engine was retried after No such file or directory error. #4161 (alexey-milovidov)
• Fixed an issue: local ClickHouse dictionaries are loaded via TCP, but should load within process. #4166 (alexey-milovidov)
• Fixed No message received error when interacting with PostgreSQL ODBC Driver through TLS connection. Also fixes segfault when using MySQL ODBC Driver. #4170 (alexey-milovidov)
• Temporarily disable predicate optimization for ORDER BY. #3890 (Winter Zhang)
• Fixed infinite loop when selecting from table function numbers(0). #4280 (alexey-milovidov)
• Fixed compile_expressions bug with comparison of big (more than int16) dates. #4341 (alesapin)
• Fixed segmentation fault with uncompressed_cache=1 and exception with wrong uncompressed size. #4186 (alesapin)
• Fixed ALL JOIN with duplicates in right table. #4184 (Artem Zuikov)
• Fixed wrong behaviour when doing INSERT ... SELECT ... FROM file(...) query and file has CSVWithNames or TSVWIthNames format and the first data row is missing. #4297 (alexey-milovidov)
• Fixed aggregate functions execution with Array(LowCardinality) arguments. #4055 (KochetovNicolai)
• Debian package: correct /etc/clickhouse-server/preprocessed link according to config. #4205 (proller)
• Fixed fuzz test under undefined behavior sanitizer: added parameter type check for quantile*Weighted family of functions. #4145 (alexey-milovidov)
• Make START REPLICATED SENDS command start replicated sends. #4229 (nvartolomei)
• Fixed Not found column for duplicate columns in JOIN ON section. #4279 (Artem Zuikov)
• Now /etc/ssl is used as default directory with SSL certificates. #4167 (alexey-milovidov)
• Fixed crash on dictionary reload if dictionary not available. #4188 (proller)
• Fixed bug with incorrect Date and DateTime comparison. #4237 (valexey)
• Fixed incorrect result when Date and DateTime arguments are used in branches of conditional operator (function if). Added generic case for function if. #4243 (alexey-milovidov)