Token count field type

A token count field type stores the number of analyzed tokens in a string.

Example

Create a mapping with a token count field:

  1. PUT testindex
  2. {
  3.   "mappings": {
  4.     "properties": {
  5.       "sentence": { 
  6.         "type": "text",
  7.         "fields": {
  8.           "num_words": { 
  9.             "type":     "token_count",
  10.             "analyzer": "english"
  11.           }
  12.         }
  13.       }
  14.     }
  15.   }
  16. }

copy

Index three documents with text fields:

  1. PUT testindex/_doc/1
  2. { "sentence": "To be, or not to be: that is the question." }

copy

  1. PUT testindex/_doc/2
  2. { "sentence": "All the world’s a stage, and all the men and women are merely players." }

copy

  1. PUT testindex/_doc/3
  2. { "sentence": "Now is the winter of our discontent." }

copy

Search for sentences with fewer than 10 words:

  1. GET testindex/_search
  2. {
  3.   "query": {
  4.     "range": {
  5.       "sentence.num_words": {
  6.         "lt": 10
  7.       }
  8.     }
  9.   }
  10. }

copy

The response contains one matching sentence:

  1. {
  2.   "took" : 8,
  3.   "timed_out" : false,
  4.   "_shards" : {
  5.     "total" : 1,
  6.     "successful" : 1,
  7.     "skipped" : 0,
  8.     "failed" : 0
  9.   },
  10.   "hits" : {
  11.     "total" : {
  12.       "value" : 1,
  13.       "relation" : "eq"
  14.     },
  15.     "max_score" : 1.0,
  16.     "hits" : [
  17.       {
  18.         "_index" : "testindex",
  19.         "_type" : "_doc",
  20.         "_id" : "3",
  21.         "_score" : 1.0,
  22.         "_source" : {
  23.           "sentence" : "Now is the winter of our discontent."
  24.         }
  25.       }
  26.     ]
  27.   }
  28. }

Parameters

The following table lists the parameters accepted by token count field types. The analyzer parameter is required; all other parameters are optional.

ParameterDescription
analyzerThe analyzer to be used for this field. Specify an analyzer without token filters for optimal performance. Required.
boostA floating-point value that specifies the weight of this field toward the relevance score. Values above 1.0 increase the field’s relevance. Values between 0.0 and 1.0 decrease the field’s relevance. Default is 1.0.
doc_valuesA Boolean value that specifies whether the field should be stored on disk so that it can be used for aggregations, sorting, or scripting. Default is false.
enable_position_incrementsA Boolean value that specifies whether position increments should be counted. To avoid removing stopwords, set this field to false. Default is true.
indexA Boolean value that specifies whether the field should be searchable. Default is true.
null_valueA value to be used in place of null. Must be of the same type as the field. If this parameter is not specified, the field is treated as missing when its value is null. Default is null.
storeA Boolean value that specifies whether the field value should be stored and can be retrieved separately from the _source field. Default is false.