Elasticsearch version 7.0.0-alpha1

The changes listed below have been released for the first time in Elasticsearch 7.0.0-alpha1.

Breaking changes

Aggregations

  • Remove support for deprecated params._agg/_aggs for scripted metric aggregations #32979 (issues: #29328, #31597)
  • Percentile/Ranks should return null instead of NaN when empty #30460 (issue: #29066)
  • Render sum as zero if count is zero for stats aggregation #27193 (issue: #26893)

Analysis

Audit

  • Logfile auditing settings remove after deprecation #35205

Authentication

  • Security: remove wrapping in put user response #33512 (issue: #32332)

Authorization

  • Remove aliases resolution limitations when security is enabled #31952 (issue: #31516)

CRUD

  • Version conflict exception message enhancement #29432 (issue: #21278)
  • Using ObjectParser in UpdateRequest #29293 (issue: #28740)

Distributed

  • Remove undocumented action.master.force_local setting #29351
  • Remove tribe node support #28443
  • Forbid negative values for index.unassigned.node_left.delayed_timeout #26828

Features/Indices APIs

  • Indices Exists API should return 404 for empty wildcards #34499
  • Default to one shard #30539
  • Limit the number of nested documents #27405 (issue: #26962)

Features/Ingest

  • INGEST: Add Configuration Except. Data to Metdata #32322 (issue: #27728)

Features/Stats

  • Remove the suggest metric from stats APIs #29635 (issue: #29589)
  • Align cat thread pool info to thread pool config #29195 (issue: #29123)
  • Align thread pool info to thread pool configuration #29123 (issue: #29113)

Geo

  • Use geohash cell instead of just a corner in geo_bounding_box #30698 (issue: #25154)

Infra/Circuit Breakers

  • Introduce durability of circuit breaking exception #34460 (issue: #31986)
  • Circuit-break based on real memory usage #31767

Infra/Core

  • Core: Default node.name to the hostname #33677
  • Remove bulk fallback for write thread pool #29609
  • CCS: Drop http address from remote cluster info #29568 (issue: #29207)
  • Remove the index thread pool #29556
  • Main response should not have status 503 when okay #29045 (issue: #8902)
  • Automatically prepare indices for splitting #27451
  • Don’t refresh on _flush _force_merge and _upgrade #27000 (issue: #26972)

Infra/Packaging

  • Packaging: Remove windows bin files from the tar distribution #30596

Infra/REST API

  • REST: Remove GET support for clear cache indices #29525
  • REST : Clear Indices Cache API remove deprecated url params #29068

Infra/Scripting

  • Remove support for deprecated StoredScript contexts #31394 (issues: #27612, #28939)
  • Scripting: Remove getDate methods from ScriptDocValues #30690
  • Handle missing and multiple values in script #29611 (issue: #29286)
  • Drop ScriptDocValues#date and ScriptDocValues#dates in 7.0.0 [ISSUE] #23008

Infra/Settings

  • Remove config prompting for secrets and text #27216

Mapping

  • Match phrase queries against non-indexed fields should throw an exception #31060
  • Remove legacy mapping code. #29224
  • Reject updates to the _default_ mapping. #29165 (issues: #15613, #28248)
  • Remove the update_all_types option. #28288
  • Remove the _default_ mapping. #28248
  • Reject the index_options parameter for numeric fields #26668 (issue: #21475)

Network

  • Network: Remove http.enabled setting #29601 (issue: #12792)
  • Remove HTTP max content length leniency #29337

Percolator

  • remove deprecated percolator map_unmapped_fields_as_string setting #28060

Ranking

  • Add minimal sanity checks to custom/scripted similarities. #33564 (issue: #33309)
  • Scroll queries asking for rescore are considered invalid #32918 (issue: #31775)

Search

  • Remove deprecated url parameters _source_include and _source_exclude #35097 (issues: #22792, #33475)
  • Disallow negative query boost #34486 (issue: #33309)
  • Forbid negative weight in Function Score Query #33390 (issue: #31927)
  • In the field capabilities API, remove support for providing fields in the request body. #30185
  • Remove deprecated options for query_string #29203 (issue: #25551)
  • Fix Laplace scorer to multiply by alpha (and not add) #27125
  • Remove _primary and _replica shard preferences #26791 (issue: #26335)
  • Limit the number of expanded fields it query_string and simple_query_string #26541 (issue: #25105)
  • Make purely negative queries return scores of 0. #26015 (issue: #23449)

Snapshot/Restore

  • Include size of snapshot in snapshot metadata #30890 (issue: #18543)
  • Remove azure deprecated settings #26099 (issue: #23405)

Store

  • drop elasticsearch-translog for 7.0 #33373 (issues: #31389, #32281)
  • completely drop index.shard.check_on_startup: fix for 7.0 #33194

Suggesters

  • Fix threshold frequency computation in Suggesters #34312 (issue: #34282)
  • Make Geo Context Mapping Parsing More Strict #32821 (issues: #32202, #32412)
  • Make Geo Context Parsing More Strict #32412 (issue: #32202)
  • Remove the ability to index or query context suggestions without context #31007 (issue: #30712)

Breaking Java changes

Aggregations

  • Change GeoHashGrid.Bucket#getKey() to return String #31748 (issue: #30320)

Analysis

  • Remove deprecated AnalysisPlugin#requriesAnalysisSettings method #32037 (issue: #32025)

Features/Java High Level REST Client

  • API: Drop deprecated methods from Retry #33925
  • REST hl client: cluster health to default to cluster level #31268 (issue: #29331)
  • REST high-level Client: remove deprecated API methods #31200 (issue: #31069)

Features/Java Low Level REST Client

Geo

  • [Geo] Decouple geojson parse logic from ShapeBuilders #27212

Infra/Core

  • Core: Remove RequestBuilder from Action #30966

Infra/Transport API

Deprecations

Analysis

  • Replace parameter unicodeSetFilter with unicode_set_filter #29215 (issue: #22823)
  • Replace delimited_payload_filter by delimited_payload #26625 (issue: #21978)

Features/Indices APIs

  • Default copy settings to true and deprecate on the REST layer #30598

Infra/Transport API

  • Deprecate the transport client in favour of the high-level REST client #27085

Mapping

  • Deprecate type exists requests. #34663

Search

New features

Analysis

  • Relax TermVectors API to work with textual fields other than TextFieldType #31915 (issue: #31902)

CCR

  • Generalize search.remote settings to cluster.remote #33413

Distributed

Features/Ingest

  • Revert “Introduce a Hashing Processor (#31087)” #32178
  • Add ingest-attachment support for per document indexed_chars limit #28977 (issue: #28942)

Features/Java High Level REST Client

  • GraphClient for the high level REST client and associated tests #32366

Features/Monitoring

  • [Elasticsearch Monitoring] Collect only display_name (for now) #35265 (issue: #8445)

Infra/Core

  • Skip shard refreshes if shard is search idle #27500

Infra/Logging

  • Logging: Unify log rotation for index/search slow log #27298

Infra/Plugins

Infra/REST API

Machine Learning

  • [ML] Filter undefined job groups from update job calendar actions #30757

Mapping

Ranking

Recovery

  • Allow to trim all ops above a certain seq# with a term lower than X, … #31211 (issue: #10708)

SQL

  • SQL: Add basic support for ST_AsWKT geo function #34205
  • SQL: Add support for SYS GEOMETRY_COLUMNS #30496 (issue: #29872)

Search

  • Add “took” timing info to response for _msearch/template API #30961 (issue: #30957)
  • Expose the lucene Matches API to searches [ISSUE] #29631
  • Add allow_partial_search_results flag to search requests with default setting true #28440 (issue: #27435)
  • Enable adaptive replica selection by default #26522 (issue: #24915)

Suggesters

  • serialize suggestion responses as named writeables #30284 (issue: #26585)

Enhancements

Aggregations

  • Uses MergingDigest instead of AVLDigest in percentiles agg #28702 (issue: #19528)

Discovery-Plugins

  • Rename discovery.zen.minimum_master_nodes [ISSUE] #14058

Engine

  • Remove versionType from translog #31945
  • do retry if primary fails on AsyncAfterWriteAction #31857 (issues: #31716, #31755)
  • handle AsyncAfterWriteAction exception before listener is registered #31755 (issue: #31716)
  • Use IndexWriter#flushNextBuffer to free memory #27753
  • Remove pre 6.0.0 support from InternalEngine #27720

Features/Indices APIs

  • Add cluster-wide shard limit #32856 (issue: #20705)
  • Remove RestGetAllAliasesAction #31308 (issue: #31129)
  • Add rollover-creation-date setting to rolled over index #31144 (issue: #30887)
  • add is-write-index flag to aliases #30942
  • Make index and bulk APIs work without types. #29479

Features/Ingest

  • ingest: Add ignore_missing property to foreach filter (#22147) #31578 (issue: #22147)

Features/Java High Level REST Client

Features/Stats

  • Stats to record how often the ClusterState diff mechanism is used successfully #26973

Features/Watcher

  • Watcher: Validate email adresses when storing a watch #34042 (issue: #33980)

Infra/Circuit Breakers

  • Have circuit breaker succeed on unknown mem usage #33125 (issue: #31767)
  • Account for XContent overhead in in-flight breaker #31613
  • Script Stats: Add compilation limit counter to stats #26387

Infra/Core

  • Add RunOnce utility class that executes a Runnable exactly once #35484
  • Improved IndexNotFoundException’s default error message #34649 (issue: #34628)
  • Set a bounded default for http.max_warning_header_count [ISSUE] #33479

Infra/Packaging

  • Choose JVM options ergonomically #30684

Infra/REST API

Infra/Scripting

  • Tests: Add support for custom contexts to mock scripts #34100
  • Scripting: Reflect factory signatures in painless classloader #34088
  • Handle missing values in painless #32207 (issue: #29286)

Infra/Settings

  • Settings: Add keystore creation to add commands #26126

Infra/Transport API

  • Change BWC version for VerifyRepositoryResponse #30796 (issue: #30762)

Network

  • Add cors support to NioHttpServerTransport #30827 (issue: #28898)
  • Reintroduce mandatory http pipelining support #30820
  • Make http pipelining support mandatory #30695 (issues: #28898, #29500)
  • Add nio http server transport #29587 (issue: #28898)
  • Add class for serializing message to bytes #29384 (issue: #28898)
  • Selectors operate on channel contexts #28468 (issue: #27260)
  • Unify nio read / write channel contexts #28160 (issue: #27260)
  • Create nio-transport plugin for NioTransport #27949 (issue: #27260)
  • Add elasticsearch-nio jar for base nio classes #27801 (issue: #27802)

Ranking

  • Add k parameter to PrecisionAtK metric #27569

SQL

  • SQL: Introduce support for NULL values #34573 (issue: #32079)

Search

  • Make limit on number of expanded fields configurable #35284 (issues: #26541, #34778)
  • Search: Simply SingleFieldsVisitor #34052
  • Don’t count hits via the collector if the hit count can be computed from index stats. #33701
  • Limit the number of concurrent requests per node #31206 (issue: #31192)
  • Default max concurrent search req. numNodes * 5 #31171 (issues: #30783, #30994)
  • Change ScriptException status to 400 (bad request) #30861 (issue: #12315)
  • Change default value to true for transpositions parameter of fuzzy query #26901
  • Introducing “took” time (in ms) for _msearch #23767 (issue: #23131)

Snapshot/Restore

  • 31608 Add S3 Setting to Force Path Type Access #34721 (issue: #31608)

Store

ZenDiscovery

  • [Zen2] Introduce vote withdrawal #35446
  • Zen2: Add basic Zen1 transport-level BWC #35443
  • Zen2: Add diff-based publishing #35290
  • [Zen2] Introduce auto_shrink_voting_configuration setting #35217
  • Introduce transport API for cluster bootstrapping #34961
  • [Zen2] Reconfigure cluster as its membership changes #34592 (issue: #33924)
  • Zen2: Fail fast on disconnects #34503
  • [Zen2] Add storage-layer disruptions to CoordinatorTests #34347
  • [Zen2] Add low-level bootstrap implementation #34345
  • [Zen2] Gather votes from all nodes #34335
  • Zen2: Add Cluster State Applier #34257
  • [Zen2] Add safety phase to CoordinatorTests #34241
  • [Zen2] Integrate FollowerChecker with Coordinator #34075
  • Integrate LeaderChecker with Coordinator #34049
  • Zen2: Trigger join when active master detected #34008
  • Zen2: Update PeerFinder term on term bump #33992
  • [Zen2] Calculate optimal cluster configuration #33924
  • [Zen2] Introduce FollowersChecker #33917
  • Zen2: Integrate publication pipeline into Coordinator #33771
  • Zen2: Add DisruptableMockTransport #33713
  • [Zen2] Implement basic cluster formation #33668
  • [Zen2] Introduce LeaderChecker #33024
  • Zen2: Add leader-side join handling logic #33013
  • [Zen2] Add PeerFinder#onFoundPeersUpdated #32939
  • [Zen2] Introduce PreVoteCollector #32847
  • [Zen2] Introduce ElectionScheduler #32846
  • [Zen2] Introduce ElectionScheduler #32709
  • [Zen2] Add HandshakingTransportAddressConnector #32643 (issue: #32246)
  • [Zen2] Add UnicastConfiguredHostsResolver #32642 (issue: #32246)
  • Zen2: Cluster state publication pipeline #32584 (issue: #32006)
  • [Zen2] Introduce gossip-like discovery of master nodes #32246
  • Add core coordination algorithm for cluster state publishing #32171 (issue: #32006)
  • Add term and config to cluster state #32100 (issue: #32006)

Bug fixes

Aggregations

  • Fix InternalAutoDateHistogram reproducible failure #32723 (issue: #32215)

Analysis

  • Close #26771: beider_morse phonetic encoder failure when languageset unspecified #26848 (issue: #26771)

Authorization

Docs Infrastructure

  • Docs build fails due to missing nexus.png [ISSUE] #33101

Features/Indices APIs

  • Validate top-level keys for create index request (#23755) #23869 (issue: #23755)

Features/Ingest

  • INGEST: Fix Deprecation Warning in Script Proc. #32407

Features/Java High Level REST Client

  • HLRC: Drop extra level from user parser #34932

Features/Java Low Level REST Client

  • Remove I/O pool blocking sniffing call from onFailure callback, add some logic around host exclusion #27985 (issue: #27984)

Features/Watcher

  • Watcher: Ignore system locale/timezone in croneval CLI tool #33215

Geo

  • [build] Test GeoShapeQueryTests#testPointsOnly fails [ISSUE] #27454

Infra/Core

  • Ensure shard is refreshed once it’s inactive #27559 (issue: #27500)

Infra/Settings

  • Change format how settings represent lists / array #26723

Infra/Transport API

  • Remove version read/write logic in Verify Response #30879 (issue: #30807)
  • Enable muted Repository test #30875 (issue: #30807)
  • Bad regex in CORS settings should throw a nicer error #29108

License

  • Update versions for start_trial after backport #30218 (issue: #30135)

Mapping

  • Ensure that field aliases cannot be used in multi-fields. #32219

Network

  • Adjust SSLDriver behavior for JDK11 changes #32145 (issues: #32122, #32144)
  • Netty4SizeHeaderFrameDecoder error #31057
  • Fix memory leak in http pipelining #30815 (issue: #30801)
  • Fix issue with finishing handshake in ssl driver #30580

Search

  • Ensure realtime _get and _termvectors don’t run on the network thread #33814 (issue: #27500)
  • [bug] fuzziness custom auto #33462 (issue: #33454)
  • Fix inner hits retrieval when stored fields are disabled (none) #33018 (issue: #32941)
  • Set maxScore for empty TopDocs to Nan rather than 0 #32938
  • Handle leniency for cross_fields type in multi_match query #27045 (issue: #23210)
  • Raise IllegalArgumentException instead if query validation failed #26811 (issue: #26799)

Security

  • Handle 6.4.0+ BWC for Application Privileges #32929

ZenDiscovery

  • [Zen2] Remove duplicate discovered peers #35505

Upgrades

Geo

Infra/Core

Network

  • NETWORKING: Fix Netty Leaks by upgrading to 4.1.28 #32511 (issue: #32487)