Expressions in search

Manticore lets you use arbitrary arithmetic expressions both via SQL and HTTP, involving attribute values, internal attributes (document ID and relevance weight), arithmetic operations, a number of built-in functions, and user-defined functions. Here’s the complete reference list for quick access.

Arithmetic operators

+, -, *, /, %, DIV, MOD

The standard arithmetic operators. Arithmetic calculations involving those can be performed in three different modes:

1. using single-precision, 32-bit IEEE 754 floating point values (the default),
2. using signed 32-bit integers
3. using 64-bit signed integers

The expression parser will automatically switch to integer mode if there are no operations the result in a floating point value. Otherwise, it will use the default floating point mode. For instance, a+b will be computed using 32-bit integers if both arguments are 32-bit integers; or using 64-bit integers if both arguments are integers but one of them is 64-bit; or in floats otherwise. However, a/b or sqrt(a) will always be computed in floats, because these operations return a result of non-integer type. To avoid the first, you can either use IDIV(a,b) or a DIV b form. Also, a*b will not be automatically promoted to 64-bit when the arguments are 32-bit. To enforce 64-bit results, you can use BIGINT()), but note that if there are non-integer operations, BIGINT() will simply be ignored.

Comparison operators

<, > <=, >=, =, <>

Comparison operators return 1.0 when the condition is true and 0.0 otherwise. For instance, (a=b)+3 will evaluate to 4 when attribute a is equal to attribute b, and to 3 when a is not. Unlike MySQL, the equality comparisons (ie. = and <> operators) introduce a small equality threshold (1e-6 by default). If the difference between compared values is within the threshold, they will be considered equal. BETWEEN and IN operators in case of multi-value attribute return true if at least one value matches the condition(same as ANY())). IN doesn’t support JSON attributes. IS (NOT) NULL is supported only for JSON attributes.

Boolean operators

AND, OR, NOT

Boolean operators (AND, OR, NOT) behave as usual. They are left-associative and have the least priority compared to other operators. NOT has more priority than AND and OR but nevertheless less than any other operator. AND and OR have the same priority so brackets use is recommended to avoid confusion in complex expressions.

Bitwise operators

&, |

These operators perform bitwise AND and OR respectively. The operands must be of an integer types.

Functions:

• ABS())
• ALL())
• ANY())
• ATAN2())
• BIGINT())
• BITDOT())
• BM25F())
• CEIL())
• CONCAT())
• CONTAINS())
• COS())
• CRC32())
• DAY())
• DOUBLE())
• EXP())
• FIBONACCI())
• FLOOR())
• GEODIST())
• GEOPOLY2D())
• GREATEST())
• HOUR())
• IDIV())
• IF())
• IN())
• INDEXOF())
• INTEGER())
• INTERVAL())
• LAST_INSERT_ID())
• LEAST())
• LENGTH())
• LN())
• LOG10())
• LOG2())
• MAX())
• MIN())
• MINUTE())
• MIN_TOP_SORTVAL())
• MIN_TOP_WEIGHT())
• MONTH())
• NOW())
• PACKEDFACTORS())
• POLY2D())
• POW())
• RAND())
• REGEX())
• REMAP())
• SECOND())
• SIN())
• SINT())
• SQRT())
• SUBSTRING_INDEX())
• TO_STRING())
• UINT())
• YEAR())
• YEARMONTH())
• YEARMONTHDAY())
• WEIGHT())

Expressions in HTTP JSON

In HTTP JSON interface expressions are supported via script_fields and expressions

script_fields

{
    "index": "test",
    "query": { 
        "match_all": {} 
    }, "script_fields": {
        "add_all": { 
            "script": { 
                "inline": "( gid * 10 ) | crc32(title)" 
            } 
        },
        "title_len": { 
            "script": { 
                "inline": "crc32(title)" 
            } 
        }
    }
}

In this example two expressions are created: add_all and title_len. First expression calculates ( gid * 10 ) | crc32(title) and stores the result in the add_all attribute. Second expression calculates crc32(title) and stores the result in the title_len attribute.

Only inline expressions are supported for now. The value of inline property (the expression to compute) has the same syntax as SQL expressions.

The expression name can be used in filtering or sorting.

• script_fields

script_fields

{
    "index":"movies_rt",
    "script_fields":{
        "cond1":{
            "script":{
                "inline":"actor_2_facebook_likes =296 OR movie_facebook_likes =37000"
            }
        },
        "cond2":{
            "script":{
                "inline":"IF (IN (content_rating,'TV-PG','PG'),2, IF(IN(content_rating,'TV-14','PG-13'),1,0))"
            }
        }
    },
    "limit":10,
    "sort":[
        {
            "cond2":"desc"
        },
        {
            "actor_1_name":"asc"
        },
        {
            "actor_2_name":"desc"
        }
    ],
    "profile":true,
    "query":{
        "bool":{
            "must":[
                {
                    "match":{
                        "*":"star"
                    }
                },
                {
                    "equals":{
                        "cond1":1
                    }
                }
            ],
            "must_not":[
                {
                    "equals":{
                        "content_rating":"R"
                    }
                }
            ]
        }
    }
}

The expression values are by default included in the _source array of the result set. If the source is selective (see Source selection) the expressions name can be added to the _source parameter in the request.

expressions

expressions is an alternative to script_fields with a simpler syntax. Example request adds two expressions and stores the results into add_all and title_len attributes.

• expressions

expressions

{
  "index": "test",
  "query": { "match_all": {} },
  "expressions":
  {
      "add_all": "( gid * 10 ) | crc32(title)",
      "title_len": "crc32(title)"
  }
}

Search options

SQL SELECT clause and HTTP /search endpoint support a number of options that can be used to fine-tune search behaviour.

OPTION

General syntax

SQL:

SELECT ... [OPTION <optionname>=<value> [ , ... ]] [/*+ [NO_][ColumnarScan|DocidIndex|SecondaryIndex(<attribute>[,...])]] /*]

HTTP:

POST /search
{   
    "index" : "index_name",
    "options":   
    {
        "optionname": "value",
        "optionname2": <value2>
    }
}

• SQL
• JSON

SQL JSON

SELECT * FROM test WHERE MATCH('@title hello @body world')
OPTION ranker=bm25, max_matches=3000,
field_weights=(title=10, body=3), agent_query_timeout=10000

POST /search
{   
    "index" : "test",
    "query": {
      "match": {
        "title": "hello"
      },
      "match": {
        "body": "world"     
      }
    },
    "options":   
    {
        "ranker": "bm25",
        "max_matches": 3000,
        "field_weights": {
            "title": 10,
            "body": 3
        },
        "agent_query_timeout": 10000
    }
}

Response

+------+-------+-------+
| id   | title | body  |
+------+-------+-------+
|    1 | hello | world |
+------+-------+-------+
1 row in set (0.00 sec)

{
  "took": 0,
  "timed_out": false,
  "hits": {
    "total": 1,
    "total_relation": "eq",
    "hits": [
      {
        "_id": "1",
        "_score": 10500,
        "_source": {
          "title": "hello",
          "body": "world"
        }
      }
    ]
  }
}

Supported options are:

accurate_aggregation

Integer. Enables or disables guaranteed aggregate accuracy when running groupby queries in multiple threads. Default is 0.

When running a groupby query, it can be run in parallel on plain index on several pseudo shards (if pseudo_sharding is on). A similar approach works on RT indexes. Each shard/chunk executes the query, but the number of groups is limited by max_matches. If the result sets from different shards/chunks have different groups, the group counts and aggregates may be inaccurate. Note that Manticore tries to increase max_matches up to max_matches_increase_threshold based on the number of unique values of the groupby attribute (retrieved from secondary indexes). If it succeeds, there will be no loss in accuracy.

However, if the number of unique values of the groupby attribute is high, further increasing max_matches may not be a good strategy because it can lead to a loss in performance and higher memory usage. Setting accurate_aggregation to 1 forces groupby searches to run in a single thread, which fixes the accuracy issue. Note that running in a single thread is only enforced when max_matches cannot be set high enough; otherwise, searches with accurate_aggregation=1 will still run in multiple threads.

Overall, setting accurate_aggregation to 1 will guarantee group count and aggregate accuracy in RT indexes and plain indexes with pseudo_sharding\=1. The downside is that searches will run slower because they will be forced to run in a single thread.

agent_query_timeout

Integer. Max time in milliseconds to wait for remote queries to complete, see this section.

boolean_simplify

0 or 1 (0 by default). boolean_simplify=1 enables simplifying the query to speed it up.

comment

String, user comment that gets copied to a query log file.

cutoff

Integer. Max found matches threshold. The value is selected automatically if not specified.

• N = 0 disables the threshold
• N > 0: instructs Manticore to stop looking for results as soon as it finds N documents.
• not set: Manticore will decide automatically what the value should be.

In case Manticore cannot calculate the exact matching documents count you will see total_relation: gte in the query meta information, which means that the actual count is Greater Than or Equal to the total (total_found in SHOW META via SQL, hits.total in JSON via HTTP). If the total value is precise you’ll get total_relation: eq.

expand_keywords

0 or 1 (0 by default). Expands keywords with exact forms and/or stars when possible. Refer to expand_keywords for more details.

field_weights

Named integer list (per-field user weights for ranking).

Example:

SELECT ... OPTION field_weights=(title=10, body=3)

global_idf

Use global statistics (frequencies) from the global_idf file for IDF computations.

idf

Quoted, comma-separated list of IDF computation flags. Known flags are:

• normalized: BM25 variant, idf = log((N-n+1)/n), as per Robertson et al
• plain: plain variant, idf = log(N/n), as per Sparck-Jones
• tfidf_normalized: additionally divide IDF by query word count, so that TF*IDF fits into [0, 1] range
• tfidf_unnormalized: do not additionally divide IDF by query word count where N is the collection size and n is the number of matched documents

The historically default IDF (Inverse Document Frequency) in Manticore is equivalent to OPTION idf='normalized,tfidf_normalized', and those normalizations may cause several undesired effects.

First, idf=normalized causes keyword penalization. For instance, if you search for the | something and the occurs in more than 50% of the documents, then documents with both keywords the and something will get less weight than documents with just one keyword something. Using OPTION idf=plain avoids this. Plain IDF varies in [0, log(N)] range, and keywords are never penalized; while the normalized IDF varies in [-log(N), log(N)] range, and too frequent keywords are penalized.

Second, idf=tfidf_normalized causes IDF drift over queries. Historically, we additionally divided IDF by query keyword count, so that the entire sum(tf*idf) over all keywords would still fit into [0,1] range. However, that means that queries word1 and word1 | nonmatchingword2 would assign different weights to the exactly same result set, because the IDFs for both word1 and nonmatchingword2 would be divided by 2. OPTION idf='tfidf_unnormalized' fixes that. Note that BM25, BM25A, BM25F() ranking factors will be scale accordingly once you disable this normalization.

IDF flags can be mixed; plain and normalized are mutually exclusive; tfidf_unnormalized and tfidf_normalized are mutually exclusive; and unspecified flags in such a mutually exclusive group take their defaults. That means that OPTION idf=plain is equivalent to a complete OPTION idf='plain,tfidf_normalized' specification.

index_weights

Named integer list. Per-table user weights for ranking.

local_df

0 or 1,automatically sum DFs over all the local parts of a distributed table, so that the IDF is consistent (and precise) over a locally sharded table.

low_priority

0 or 1 (0 by default). low_priority=1 runs the query with low priority in terms of Linux CPU scheduling. Consider also option threads=1 instead, or use that together with low_priority=1, as it might be better in some use cases.

max_matches

Integer. Per-query max matches value.

Maximum amount of matches that the server keeps in RAM for each table and can return to the client. Default is 1000.

Introduced in order to control and limit RAM usage, max_matches setting defines how much matches will be kept in RAM while searching each table. Every match found will still be processed; but only best N of them will be kept in memory and return to the client in the end. Assume that the table contains 2,000,000 matches for the query. You rarely (if ever) need to retrieve all of them. Rather, you need to scan all of them, but only choose “best” at most, say, 500 by some criteria (ie. sorted by relevance, or price, or anything else), and display those 500 matches to the end user in pages of 20 to 100 matches. And tracking only the best 500 matches is much more RAM and CPU efficient than keeping all 2,000,000 matches, sorting them, and then discarding everything but the first 20 needed to display the search results page. max_matches controls N in that “best N” amount.

This parameter noticeably affects per-query RAM and CPU usage. Values of 1,000 to 10,000 are generally fine, but higher limits must be used with care. Recklessly raising max_matches to 1,000,000 means that searchd will have to allocate and initialize 1-million-entry matches buffer for every query. That will obviously increase per-query RAM usage, and in some cases can also noticeably impact performance.

See also max_matches_increase_threshold, which can affect the behavior of the max_matches option.

max_matches_increase_threshold

Integer. Sets the threshold that max_matches can be increased to. Default is 16384.

Manticore may increase max_matches to improve groupby and/or aggregation accuracy when pseudo_sharding is enabled and if it detects that the number of unique values of groupby attribute is less than this threshold. Loss of accuracy may occur when pseudo sharding executes the query in several threads or RT table performs parallel searches in disk chunks.

If the number of unique values of groupby attribute is less than the threshold, max_matches will be set to this number. Otherwise, default max_matches will be used.

If max_matches was set explicitly in query options, this threshold has no effect.

Note that if this threshold is set too high, the result will be increased memory consumption and general performance degradation.

You can also force guaranteed groupby/aggregate accuracy mode using accurate_aggregation option.

max_query_time

Sets maximum search query time, in milliseconds. Must be a non-negative integer. Default value is 0 which means “do not limit”. Local search queries will be stopped once that much time has elapsed. Note that if you’re performing a search which queries several local tables, this limit applies to each table separately. Note it may increase the query’s response time a little bit, the overhead is caused by constant tracking if it’s time to stop the query.

max_predicted_time

Integer. Max predicted search time, see predicted_time_costs.

morphology

none allows to replace all query terms with their exact forms if table was built with index_exact_words enabled. Useful to prevent stemming or lemmatizing query terms.

not_terms_only_allowed

0 or 1, allows standalone negation for the query. Default is 0. See also corresponding global setting.

• SQL

SQL

MySQL [(none)]> select * from tbl where match('-donald');
ERROR 1064 (42000): index t: query error: query is non-computable (single NOT operator)
MySQL [(none)]> select * from t where match('-donald') option not_terms_only_allowed=1;
+---------------------+-----------+
| id                  | field     |
+---------------------+-----------+
| 1658178727135150081 | smth else |
+---------------------+-----------+

ranker

Any of:

• proximity_bm25
• bm25
• none
• wordcount
• proximity
• matchany
• fieldmask
• sph04
• expr
• or export

Refer to Search results ranking for more details on each ranker.

rand_seed

Lets you specify a specific integer seed value for an ORDER BY RAND() query, for example: ... OPTION rand_seed=1234. By default, a new and different seed value is autogenerated for every query

retry_count

Integer. Distributed retries count.

retry_delay

Integer. Distributed retry delay, msec.

sort_method

• pq - priority queue, set by default
• kbuffer - gives faster sorting for already pre-sorted data, e.g. table data sorted by id The result set is in both cases the same; picking one option or the other may just improve (or worsen!) performance.

threads

Limits max number of threads to use for current query processing. Default - no limit (the query can occupy all threads as defined globally). For batch of queries the option must be attached to the very first query in the batch, and it is then applied when working queue is created and then is effective for the whole batch. This option has same meaning as option max_threads_per_query, but applied only to the current query or batch of queries.

token_filter

Quoted, colon-separated of library name:plugin name:optional string of settings. Query-time token filter gets created on search each time full-text invoked by every table involved and let you implement a custom tokenizer that makes tokens according to custom rules.

SELECT * FROM index WHERE MATCH ('yes@no') OPTION token_filter='mylib.so:blend:@'

Query optimizer hints

In rare cases, Manticore’s built-in query analyzer may be incorrect in understanding a query and determining whether a docid index, secondary indexes, or columnar scan should be used. To override the query optimizer’s decisions, you can use the following hints in your query:

• /*+ DocidIndex(id) */ to force the use of a docid index, /*+ NO_DocidIndex(id) */ to tell the optimizer to ignore it
• /*+ SecondaryIndex(<attr_name1>[, <attr_nameN>]) */ to force the use of a secondary index (if available), /*+ NO_SecondaryIndex(id) */ to tell the optimizer to ignore it
• /*+ ColumnarScan(<attr_name1>[, <attr_nameN>]) */ to force the use of a columnar scan (if the attribute is columnar), /*+ NO_ColumnarScan(id) */ to tell the optimizer to ignore it

For more information on how the query optimizer works, see the Cost based optimizer.

• SQL

SQL

SELECT * FROM students where age > 21 /*+ SecondaryIndex(age) */

Highlighting

Highlighting allows you to get highlighted text fragments (called snippets) from documents that contain matching keywords.

SQL’s HIGHLIGHT() function, "highlight" property in json queries via HTTP and highlight() function in the PHP client all use built-in document storage for retrieving original field contents (enabled by default).

• SQL
• JSON
• PHP
• Python
• Javascript
• Java

SQL JSON PHP Python Javascript Java

SELECT HIGHLIGHT() FROM books WHERE MATCH('try');

POST /search
{
  "index": "books",
  "query":  {  "match": { "*" : "try" }  },
  "highlight": {}
}

$results = $index->search('try')->highlight()->get();
foreach($results as $doc)
{
    echo 'Document: '.$doc->getId();
    foreach($doc->getData() as $field=>$value)
    {
        echo $field.': '.$value;
    }
    foreach($doc->getHighlight() as $field=>$snippets)
    {
        echo "Highlight for ".$field.":\n";
        foreach($snippets as $snippet)
        {
            echo "- ".$snippet."\n";
        }
    }
}

res = searchApi.search({"index":"books","query":{"match":{"*":"try"}},"highlight":{}})

res =  await searchApi.search({"index":"books","query":{"match":{"*":"try"}},"highlight":{}});

searchRequest = new SearchRequest();
searchRequest.setIndex("books");
query = new HashMap<String,Object>();
query.put("match",new HashMap<String,Object>(){{
    put("*","try|gets|down|said");
}});        
searchRequest.setQuery(query);
highlight = new HashMap<String,Object>(){{
}};
searchRequest.setHighlight(highlight);
searchResponse = searchApi.search(searchRequest);

Response

+----------------------------------------------------------+
| highlight()                                              |
+----------------------------------------------------------+
| Don`t <strong>try</strong> to compete in childishness, said Bliss. |
+----------------------------------------------------------+
1 row in set (0.00 sec)

{
  "took":1,
  "timed_out":false,
  "hits":
  {
    "total":1,
    "hits":
    [
      {
        "_id":"4",
        "_score":1704,
        "_source":
        {
          "title":"Book four",
          "content":"Don`t try to compete in childishness, said Bliss."
        },
        "highlight":
        {
          "title": ["Book four"],
          "content": ["Don`t <strong>try</strong> to compete in childishness, said Bliss."]
        }
      }
    ]
  }
}

Document: 14
title: Book four
content: Don`t try to compete in childishness, said Bliss.
Highlight for title:
- Book four
Highlight for content:
- Don`t <strong>try</strong> to compete in childishness, said Bliss.

{'aggregations': None,
 'hits': {'hits': [{u'_id': u'4',
                    u'_score': 1695,
                    u'_source': {u'content': u'Don`t try to compete in childishness, said Bliss.',
                                 u'title': u'Book four'},
                    u'highlight': {u'content': [u'Don`t <strong>try</strong> to compete in childishness, said Bliss.'],
                                   u'title': [u'Book four']}}],
          'max_score': None,
          'total': 1},
 'profile': None,
 'timed_out': False,
 'took': 0}

{"took":0,"timed_out":false,"hits":{"total":1,"hits":[{"_id":"4","_score":1695,"_source":{"title":"Book four","content":"Don`t try to compete in childishness, said Bliss."},"highlight":{"title":["Book four"],"content":["Don`t <strong>try</strong> to compete in childishness, said Bliss."]}}]}}

class SearchResponse {
    took: 0
    timedOut: false
    hits: class SearchResponseHits {
        total: 3
        maxScore: null
        hits: [{_id=3, _score=1597, _source={title=Book three, content=Trevize whispered, "It gets infantile pleasure out of display. I`d love to knock it down."}, highlight={title=[Book three], content=[, "It <strong>gets</strong> infantile pleasure ,  to knock it <strong>down</strong>."]}}, {_id=4, _score=1563, _source={title=Book four, content=Don`t try to compete in childishness, said Bliss.}, highlight={title=[Book four], content=[Don`t <strong>try</strong> to compete in childishness, <strong>said</strong> Bliss.]}}, {_id=5, _score=1514, _source={title=Books two, content=A door opened before them, revealing a small room. Bander said, "Come, half-humans, I want to show you how we live."}, highlight={title=[Books two], content=[ a small room. Bander <strong>said</strong>, "Come, half-humans, I]}}]
        aggregations: null
    }
    profile: null
}

When using SQL to highlight search results, you get different snippets from different fields concatenated as a single string. It is a limitation of mysql protocol. You can fine-tune concatenation separators with field_separator and snippet_separator options, see below.

When running json queries via HTTP or using the PHP client, there are no such limitations and the result set contains an array of fields which contains arrays of snippets (without the separators).

Note that snippet generation options such as limit, limit_words, limit_snippets are applied to each field separately (by default). You can change this behavior using the limits_per_field option, but it may lead to undesirable results. I.e. one of the fields has matching keywords, but no snippets from this field are included in the result set because they didn’t rank as high as the snippets from the other fields in the highlighting engine.

Highlighting algorithm currently favors better snippets (with closer phrase matches), and then snippets with keywords not yet included in the result. Generally, it will try to highlight the best match with the query, and it will also try to highlight all the query keywords, as made possible by the limits. If there are no matches in the current field, the beginning of the document trimmed down according to the limits will be return by default. You can also return an empty string instead by setting allow_empty option to 1.

Highlighting is performed on a so-called post limit stage, meaning that snippet generation is postponed not just until the entire final result set is ready, but even after the LIMIT clause is applied. For example, with a LIMIT 20,10 clause, HIGHLIGHT() function will be called at most 10 times.

Highlighting options

There are several additional optional highlighting options that can be used to fine-tune snippet generation. Most of them are common to SQL, HTTP and PHP client.

before_match

A string to insert before a keyword match. A %SNIPPET_ID% macro can be used in this string. The first match of the macro is replaced with an incrementing snippet number within a current snippet. Numbering starts at 1 by default but can be overridden with start_snippet_id option. %SNIPPET_ID% restarts at the start of every new document. Default is <strong>.

after_match

A string to insert after a keyword match. Default is </strong>.

limit

Maximum snippet size, in symbols (codepoints). Default is 256. Per-field by default, see limits_per_field.

limit_words

Limits the maximum number of words that can be included in the result. Note the limit applies to any words, and not just the matched keywords to highlight. For example, if we are highlighting Mary and a snippet Mary had a little lamb is selected, then it contributes 5 words to this limit, not just 1. Default is 0 (no limit). Per-field by default, see limits_per_field.

limit_snippets

Limits the maximum number of snippets that can be included in the result. Default is 0 (no limit). Per-field by default, see limits_per_field.

limits_per_field

Selects whether limit, limit_words and limit_snippets work as individual limits in every field of the document being highlighted or as global limits for the whole document. Setting this option to 0 means that all combined highlighting results for one document must be within the specified limits. The downside is that you may get several snippets highlighted in one field and none in another if the highlighting engine decides that they are more relevant. Default is 1 (use per-field limits).

around

How much words to pick around each matching keywords block. Default is 5.

use_boundaries

Whether to additionally break snippets by phrase boundary characters, as configured in table settings with phrase_boundary directive. Default is 0 (don’t use boundaries).

weight_order

Whether to sort the extracted snippets in order of relevance (decreasing weight), or in order of appearance in the document (increasing position). Default is 0 (don’t use weight order).

force_all_words

Ignores length limit until the result includes all the keywords. Default is 0 (don’t force all keywords).

start_snippet_id

Specifies the starting value of %SNIPPET_ID% macro (that gets detected and expanded in before_match, after_match strings). Default is 1.

html_strip_mode

HTML stripping mode setting. Defaults to index, which means that table settings will be used. The other values are none and strip, that forcibly skip or apply stripping irregardless of table settings; and retain, that retains HTML markup and protects it from highlighting. The retain mode can only be used when highlighting full documents and thus requires that no snippet size limits are set. String, allowed values are none, strip, index, and retain.

allow_empty

Allows empty string to be returned as highlighting result when no snippets could be generated in the current field (no keywords match, or no snippets fit the limit). By default, the beginning of original text would be returned instead of an empty string. Default is 0 (don’t allow empty result).

snippet_boundary

Ensures that snippets do not cross a sentence, paragraph, or zone boundary (when used with a table that has the respective indexing settings enabled). String, allowed values are sentence, paragraph, and zone.

emit_zones

Emits an HTML tag with an enclosing zone name before each snippet. Default is 0 (don’t emit zone names).

force_snippets

Whether to force snippet generation even if limits allow to highlight whole text. Default is 0 (don’t force snippet generation).

• SQL
• JSON
• PHP
• Python
• Javascript
• Java

SQL JSON PHP Python Javascript Java

SELECT HIGHLIGHT({limit=50}) FROM books WHERE MATCH('try|gets|down|said');

POST /search
{
  "index": "books",
  "query": {"query_string": "try|gets|down|said"},
  "highlight": { "limit":50 }
}

$results = $index->search('try|gets|down|said')->highlight([],['limit'=>50])->get();
foreach($results as $doc)
{
    echo 'Document: '.$doc->getId();
    foreach($doc->getData() as $field=>$value)
    {
        echo $field.': '.$value;
    }
    foreach($doc->getHighlight() as $field=>$snippets)
    {
        echo "Highlight for ".$field.":\n";
        foreach($snippets as $snippet)
        {
            echo  $snippet."\n";
        }
    }
}

res = searchApi.search({"index":"books","query":{"match":{"*":"try"}},"highlight":{"limit":50}})

res =  await searchApi.search({"index":"books","query":{"query_string":"try|gets|down|said"},"highlight":{"limit":50}});

searchRequest = new SearchRequest();
searchRequest.setIndex("books");
query = new HashMap<String,Object>();
query.put("match",new HashMap<String,Object>(){{
    put("*","try|gets|down|said");
}});        
searchRequest.setQuery(query);
highlight = new HashMap<String,Object>(){{
    put("limit",50);
}};
searchRequest.setHighlight(highlight);
searchResponse = searchApi.search(searchRequest);

Response

+---------------------------------------------------------------------------+
| highlight({limit=50})                                                     |
+---------------------------------------------------------------------------+
|  ... , "It <strong>gets</strong> infantile pleasure  ...  to knock it <strong>down</strong>." |
| Don`t <strong>try</strong> to compete in childishness, <strong>said</strong> Bliss.           |
|  ...  a small room. Bander <strong>said</strong>, "Come, half-humans, I ...         |
+---------------------------------------------------------------------------+
3 rows in set (0.00 sec)

{
  "took":2,
  "timed_out":false,
  "hits":
  {
    "total":3,
    "hits":
    [
      {
        "_id":"3",
        "_score":1602,
        "_source":
        {
          "title":"Book three",
          "content":"Trevize whispered, \"It gets infantile pleasure out of display. I`d love to knock it down.\""
        },
        "highlight":
        {
          "title":
          [
            "Book three"
          ],
          "content":
          [
            ", \"It <strong>gets</strong> infantile pleasure ",
            " to knock it <strong>down</strong>.\""
          ]
        }
      },
      {
        "_id":"4",
        "_score":1573,
        "_source":
        {
          "title":"Book four",
          "content":"Don`t try to compete in childishness, said Bliss."
        },
        "highlight":
        {
          "title":
          [
            "Book four"
          ],
          "content":
          [
            "Don`t <strong>try</strong> to compete in childishness, <strong>said</strong> Bliss."
          ]
        }
      },
      {
        "_id":"2",
        "_score":1521,
        "_source":
        {
          "title":"Book two",
          "content":"A door opened before them, revealing a small room. Bander said, \"Come, half-humans, I want to show you how we live.\""
        },
        "highlight":
        {
          "title":
          [
            "Book two"
          ],
          "content":
          [
            " a small room. Bander <strong>said</strong>, \"Come, half-humans, I"
          ]
        }
      }
    ]
  }
}

Document: 3
title: Book three
content: Trevize whispered, "It gets infantile pleasure out of display. I`d love to knock it down."
Highlight for title:
- Book four
Highlight for content:
, "It <strong>gets</strong> infantile pleasure
to knock it <strong>down</strong>."
Document: 4
title: Book four
content: Don`t try to compete in childishness, said Bliss.
Highlight for title:
- Book four
Highlight for content:
Don`t <strong>try</strong> to compete in childishness, <strong>said</strong> Bliss.
Document: 2
title: Book two
content: A door opened before them, revealing a small room. Bander said, "Come, half-humans, I want to show you how we live.
Highlight for title:
- Book two
Highlight for content:
 a small room. Bander <strong>said</strong>, \"Come, half-humans, I

{'aggregations': None,
 'hits': {'hits': [{u'_id': u'4',
                    u'_score': 1695,
                    u'_source': {u'content': u'Don`t try to compete in childishness, said Bliss.',
                                 u'title': u'Book four'},
                    u'highlight': {u'content': [u'Don`t <strong>try</strong> to compete in childishness, said Bliss.'],
                                   u'title': [u'Book four']}}],
          'max_score': None,
          'total': 1},
 'profile': None,
 'timed_out': False,
 'took': 0}

{"took":0,"timed_out":false,"hits":{"total":3,"hits":[{"_id":"3","_score":1597,"_source":{"title":"Book three","content":"Trevize whispered, \"It gets infantile pleasure out of display. I`d love to knock it down.\""},"highlight":{"title":["Book three"],"content":[", \"It <strong>gets</strong> infantile pleasure "," to knock it <strong>down</strong>.\""]}},{"_id":"4","_score":1563,"_source":{"title":"Book four","content":"Don`t try to compete in childishness, said Bliss."},"highlight":{"title":["Book four"],"content":["Don`t <strong>try</strong> to compete in childishness, <strong>said</strong> Bliss."]}},{"_id":"5","_score":1514,"_source":{"title":"Books two","content":"A door opened before them, revealing a small room. Bander said, \"Come, half-humans, I want to show you how we live.\""},"highlight":{"title":["Books two"],"content":[" a small room. Bander <strong>said</strong>, \"Come, half-humans, I"]}}]}}

class SearchResponse {
    took: 0
    timedOut: false
    hits: class SearchResponseHits {
        total: 3
        maxScore: null
        hits: [{_id=3, _score=1597, _source={title=Book three, content=Trevize whispered, "It gets infantile pleasure out of display. I`d love to knock it down."}, highlight={title=[Book three], content=[, "It <strong>gets</strong> infantile pleasure ,  to knock it <strong>down</strong>."]}}, {_id=4, _score=1563, _source={title=Book four, content=Don`t try to compete in childishness, said Bliss.}, highlight={title=[Book four], content=[Don`t <strong>try</strong> to compete in childishness, <strong>said</strong> Bliss.]}}, {_id=5, _score=1514, _source={title=Books two, content=A door opened before them, revealing a small room. Bander said, "Come, half-humans, I want to show you how we live."}, highlight={title=[Books two], content=[ a small room. Bander <strong>said</strong>, "Come, half-humans, I]}}]
        aggregations: null
    }
    profile: null
}

Highlighting via SQL

HIGHLIGHT() function can be used to highlight search results. Here’s the syntax:

HIGHLIGHT([options], [field_list], [query] )

By default, it works with no arguments.

• SQL

SQL

SELECT HIGHLIGHT() FROM books WHERE MATCH('before');

Response

+-----------------------------------------------------------+
| highlight()                                               |
+-----------------------------------------------------------+
| A door opened <strong>before</strong> them, revealing a small room. |
+-----------------------------------------------------------+
1 row in set (0.00 sec)

HIGHLIGHT() fetches all available full-text fields from document storage and highlights them against the given query. It supports field syntax in queries. Field text is separated by field_separator, which can be changed in the options.

• SQL

SQL

SELECT HIGHLIGHT() FROM books WHERE MATCH('@title one');

Response

+-----------------+
| highlight()     |
+-----------------+
| Book <strong>one</strong> |
+-----------------+
1 row in set (0.00 sec)

Optional first argument in HIGHLIGHT() is the list of options.

• SQL

SQL

SELECT HIGHLIGHT({before_match='[match]',after_match='[/match]'}) FROM books WHERE MATCH('@title one');

Response

+------------------------------------------------------------+
| highlight({before_match='[match]',after_match='[/match]'}) |
+------------------------------------------------------------+
| Book [match]one[/match]                                    |
+------------------------------------------------------------+
1 row in set (0.00 sec)

Optional second argument is a string containing a field or a comma-separated list of fields. If this argument is present, only the specified fields will be fetched from document storage and highlighted. An empty string as a second argument means “fetch all available fields”.

• SQL

SQL

SELECT HIGHLIGHT({},'title,content') FROM books WHERE MATCH('one|robots');

Response

+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| highlight({},'title,content')                                                                                                                                                         |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Book <strong>one</strong> | They followed Bander. The <strong>robots</strong> remained at a polite distance, but their presence was a constantly felt threat.                                             |
| Bander ushered all three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander gestured the other <strong>robots</strong> away and entered itself. The door closed behind it. |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)

Another way to use the second argument is to specify string attribute or field name without quotes. This way the supplied string will be highlighted against the provided query, however, field syntax will be ignored.

• SQL

SQL

SELECT HIGHLIGHT({}, title) FROM books WHERE MATCH('one');

Response

+---------------------+
| highlight({},title) |
+---------------------+
| Book <strong>one</strong>     |
| Book five           |
+---------------------+
2 rows in set (0.00 sec)

Optional third argument is the query. It is used to highlight search results against a query different than the one used for searching.

• SQL

SQL

SELECT HIGHLIGHT({},'title', 'five') FROM books WHERE MATCH('one');

Response

+-------------------------------+
| highlight({},'title', 'five') |
+-------------------------------+
| Book one                      |
| Book <strong>five</strong>              |
+-------------------------------+
2 rows in set (0.00 sec)

While HIGHLIGHT() is designed to work with stored full-text fields and string attributes, it can also be used to highlight arbitrary text. Note that if the query has any field search operators (@title hello @body world), the field part of them is ignored in this case.

• SQL

SQL

SELECT HIGHLIGHT({},TO_STRING('some text to highlight'), 'highlight') FROM books WHERE MATCH('@title one');

Response

+----------------------------------------------------------------+
| highlight({},TO_STRING('some text to highlight'), 'highlight') |
+----------------------------------------------------------------+
| some text to <strong>highlight</strong>                                  |
+----------------------------------------------------------------+
1 row in set (0.00 sec)

Several options make sense only when generating a single string as a result (not an array of snippets). This only applies to SQL’s HIGHLIGHT() function:

snippet_separator

A string to insert between snippets. Default is ....

field_separator

A string to insert between fields. Default is |.

Another way to highlight text is to use the CALL SNIPPETS statement. It mostly duplicates HIGHLIGHT() functionality, but it can’t use built-in document storage. It can, however, load source text from files.

Highlighting via HTTP

To highlight full-text search results in JSON queries via HTTP, field contents has to be stored in document storage (enabled by default). In the example full-text fields content and title are fetched from document storage and highlighted against the query specified in query clause.

Highlighted snippets are returned in the highlight property of the hits array.

• JSON
• PHP
• Python
• Javascript
• Java

JSON PHP Python Javascript Java

POST /search
{
  "index": "books",
  "query": { "match": { "*": "one|robots" } },
  "highlight":
  {
    "fields": ["content"]
  }
}

$index->setName('books');
$results = $index->search('one|robots')->highlight(['content'])->get();
foreach($results as $doc)
{
    echo 'Document: '.$doc->getId()."\n";
    foreach($doc->getData() as $field=>$value)
    {
        echo $field.' : '.$value."\n";
    }
    foreach($doc->getHighlight() as $field=>$snippets)
    {
        echo "Highlight for ".$field.":\n";
        foreach($snippets as $snippet)
        {
            echo "- ".$snippet."\n";
        }
    }
}

res = searchApi.search({"index":"books","query":{"match":{"*":"one|robots"}},"highlight":{"fields":["content"]}}))

res =  await searchApi.search({"index":"books","query":{"match":{"*":"one|robots"}},"highlight":{"fields":["content"]}});

searchRequest = new SearchRequest();
searchRequest.setIndex("books");
query = new HashMap<String,Object>();
query.put("match",new HashMap<String,Object>(){{
    put("*","one|robots");
}});        
searchRequest.setQuery(query);
highlight = new HashMap<String,Object>(){{
    put("fields",new String[] {"content"});
}};
searchRequest.setHighlight(highlight);
searchResponse = searchApi.search(searchRequest);

Response

{
  "took": 0,
  "timed_out": false,
  "hits": {
    "total": 1,
    "hits": [
      {
        "_id": "1",
        "_score": 2788,
        "_source": {
          "title": "Books one",
          "content": "They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. "
        },
        "highlight": {
          "content": [
            "They followed Bander. The <strong>robots</strong> remained at a polite distance, ",
            " three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander",
            " gestured the other <strong>robots</strong> away and entered itself. The"
          ]
        }
      }
    ]
  }
}

Document: 1
title : Books one
content : They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it.
Highlight for content:
- They followed Bander. The <strong>robots</strong> remained at a polite distance,
-  three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander
-  gestured the other <strong>robots</strong> away and entered itself. The

{'aggregations': None,
 'hits': {'hits': [{u'_id': u'1',
                    u'_score': 2788,
                    u'_source': {u'content': u'They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. ',
                                 u'title': u'Books one'},
                    u'highlight': {u'content': [u'They followed Bander. The <strong>robots</strong> remained at a polite distance, ',
                                                u' three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander',
                                                u' gestured the other <strong>robots</strong> away and entered itself. The']}}],
          'max_score': None,
          'total': 1},
 'profile': None,
 'timed_out': False,
 'took': 0}

{"took":0,"timed_out":false,"hits":{"total":1,"hits":[{"_id":"1","_score":2788,"_source":{"title":"Books one","content":"They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. "},"highlight":{"content":["They followed Bander. The <strong>robots</strong> remained at a polite distance, "," three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander"," gestured the other <strong>robots</strong> away and entered itself. The"]}}]}}

class SearchResponse {
    took: 0
    timedOut: false
    hits: class SearchResponseHits {
        total: 1
        maxScore: null
        hits: [{_id=1, _score=2788, _source={title=Books one, content=They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. }, highlight={title=[Books <strong>one</strong>], content=[They followed Bander. The <strong>robots</strong> remained at a polite distance, ,  three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander,  gestured the other <strong>robots</strong> away and entered itself. The]}}]
        aggregations: null
    }
    profile: null
}

To highlight all possible fields, pass an empty object as highlight propery.

• JSON
• PHP
• Python
• Javascript
• Java

JSON PHP Python Javascript Java

POST /search
{
  "index": "books",
  "query": { "match": { "*": "one|robots" } },
  "highlight": {}
}

$index->setName('books');
$results = $index->search('one|robots')->highlight()->get();
foreach($results as $doc)
{
    echo 'Document: '.$doc->getId()."\n";
    foreach($doc->getData() as $field=>$value)
    {
        echo $field.' : '.$value."\n";
    }
    foreach($doc->getHighlight() as $field=>$snippets)
    {
        echo "Highlight for ".$field.":\n";
        foreach($snippets as $snippet)
        {
            echo "- ".$snippet."\n";
        }
    }
}

res = searchApi.search({"index":"books","query":{"match":{"*":"one|robots"}},"highlight":{}})

res =  await searchApi.search({"index":"books","query":{"match":{"*":"one|robots"}},"highlight":{}});

searchRequest = new SearchRequest();
searchRequest.setIndex("books");
query = new HashMap<String,Object>();
query.put("match",new HashMap<String,Object>(){{
    put("*","one|robots");
}});        
searchRequest.setQuery(query);
highlight = new HashMap<String,Object>(){{
}};
searchRequest.setHighlight(highlight);
searchResponse = searchApi.search(searchRequest);

Response

{
  "took": 0,
  "timed_out": false,
  "hits": {
    "total": 1,
    "hits": [
      {
        "_id": "1",
        "_score": 2788,
        "_source": {
          "title": "Books one",
          "content": "They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. "
        },
        "highlight": {
          "title": [
            "Books <strong>one</strong>"
          ],
          "content": [
            "They followed Bander. The <strong>robots</strong> remained at a polite distance, ",
            " three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander",
            " gestured the other <strong>robots</strong> away and entered itself. The"
          ]
        }
      }
    ]
  }
}

Document: 1
title : Books one
content : They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it.
Highlight for title:
- Books <strong>one</strong>
Highlight for content:
- They followed Bander. The <strong>robots</strong> remained at a polite distance,
-  three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander
-  gestured the other <strong>robots</strong> away and entered itself. The

{'aggregations': None,
 'hits': {'hits': [{u'_id': u'1',
                    u'_score': 2788,
                    u'_source': {u'content': u'They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. ',
                                 u'title': u'Books one'},
                    u'highlight': {u'content': [u'They followed Bander. The <strong>robots</strong> remained at a polite distance, ',
                                                u' three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander',
                                                u' gestured the other <strong>robots</strong> away and entered itself. The'],
                                   u'title': [u'Books <strong>one</strong>']}}],
          'max_score': None,
          'total': 1},
 'profile': None,
 'timed_out': False,
 'took': 0}

{"took":0,"timed_out":false,"hits":{"total":1,"hits":[{"_id":"1","_score":2788,"_source":{"title":"Books one","content":"They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. "},"highlight":{"title":["Books <strong>one</strong>"],"content":["They followed Bander. The <strong>robots</strong> remained at a polite distance, "," three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander"," gestured the other <strong>robots</strong> away and entered itself. The"]}}]}}

class SearchResponse {
    took: 0
    timedOut: false
    hits: class SearchResponseHits {
        total: 1
        maxScore: null
        hits: [{_id=1, _score=2788, _source={title=Books one, content=They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. }, highlight={title=[Books <strong>one</strong>], content=[They followed Bander. The <strong>robots</strong> remained at a polite distance, ,  three into the room. <strong>One</strong> of the <strong>robots</strong> followed as well. Bander,  gestured the other <strong>robots</strong> away and entered itself. The]}}]
        aggregations: null
    }
    profile: null
}

In addition to common highlighting options, several synonyms are available for JSON queries via HTTP:

fields

fields object contains attribute names with options. It can also be an array of field names (without any options).

Note, by default the highlighting works the way it tries to highlight the results following the full-text query. I.e. in a general case when you don’t specify fields to highlight the highlight is based on your full-text query, but if you specify the fields to highlight it highlights only if the full-text query matches the selected fields.

encoder

encoder can be set to default or html. When set to html, retains html markup when highlighting. Works similar to html_strip_mode=retain option.

highlight_query

highlight_query makes it possible to highlight against a query other than our search query. Syntax is the same as in the main query.

• JSON
• PHP
• Python
• Javascript
• Java

JSON PHP Python Javascript Java

POST /search
{
  "index": "books",
  "query": { "match": { "content": "one|robots" } },
  "highlight":
  {
    "fields": [ "content"],
    "highlight_query": { "match": { "*":"polite distance" } }
   }
}

$index->setName('books');
$bool = new \Manticoresearch\Query\BoolQuery();
$bool->must(new \Manticoresearch\Query\Match(['query' => 'one|robots'], 'content'));
$results = $index->search($bool)->highlight(['content'],['highlight_query'=>['match'=>['*'=>'polite distance']]])->get();
foreach($results as $doc)
{
    echo 'Document: '.$doc->getId()."\n";
    foreach($doc->getData() as $field=>$value)
    {
        echo $field.' : '.$value."\n";
    }
    foreach($doc->getHighlight() as $field=>$snippets)
    {
        echo "Highlight for ".$field.":\n";
        foreach($snippets as $snippet)
        {
            echo "- ".$snippet."\n";
        }
    }
}

res = searchApi.search({"index":"books","query":{"match":{"content":"one|robots"}},"highlight":{"fields":["content"],"highlight_query":{"match":{"*":"polite distance"}}}})

res =  await searchApi.search({"index":"books","query":{"match":{"content":"one|robots"}},"highlight":{"fields":["content"],"highlight_query":{"match":{"*":"polite distance"}}}});

searchRequest = new SearchRequest();
searchRequest.setIndex("books");
query = new HashMap<String,Object>();
query.put("match",new HashMap<String,Object>(){{
    put("*","one|robots");
}});        
searchRequest.setQuery(query);
highlight = new HashMap<String,Object>(){{
put("fields",new String[] {"content","title"});
put("highlight_query",
    new HashMap<String,Object>(){{
        put("match", new HashMap<String,Object>(){{
            put("*","polite distance");
        }});
    }});
}};
searchRequest.setHighlight(highlight);
searchResponse = searchApi.search(searchRequest);

Response

{'aggregations': None,
 'hits': {'hits': [{u'_id': u'1',
                    u'_score': 1788,
                    u'_source': {u'content': u'They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. ',
                                 u'title': u'Books one'},
                    u'highlight': {u'content': [u'. The robots remained at a <strong>polite distance</strong>, but their presence was a']}}],
          'max_score': None,
          'total': 1},
 'profile': None,
 'timed_out': False,
 'took': 0}

{"took":0,"timed_out":false,"hits":{"total":1,"hits":[{"_id":"1","_score":1788,"_source":{"title":"Books one","content":"They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. "},"highlight":{"content":[". The robots remained at a <strong>polite distance</strong>, but their presence was a"]}}]}}

pre_tags and post_tags

pre_tags and post_tags set opening and closing tags for highlighted text snippets. They work similar to before_match and after_match options. Optional, defaults are <strong> and </strong>.

• JSON
• PHP
• Python
• Javascript
• Java

JSON PHP Python Javascript Java

POST /search
{
  "index": "books",
  "query": { "match": { "*": "one|robots" } },
  "highlight":
  {
    "fields": [ "content", "title" ],
    "pre_tags": "before_",
    "post_tags": "_after"
   }
}

$index->setName('books');
$bool = new \Manticoresearch\Query\BoolQuery();
$bool->must(new \Manticoresearch\Query\Match(['query' => 'one|robots'], '*'));
$results = $index->search($bool)->highlight(['content','title'],['pre_tags'=>'before_','post_tags'=>'_after'])->get();
foreach($results as $doc)
{
    echo 'Document: '.$doc->getId()."\n";
    foreach($doc->getData() as $field=>$value)
    {
        echo $field.' : '.$value."\n";
    }
    foreach($doc->getHighlight() as $field=>$snippets)
    {
        echo "Highlight for ".$field.":\n";
        foreach($snippets as $snippet)
        {
            echo "- ".$snippet."\n";
        }
    }
}

res = searchApi.search({"index":"books","query":{"match":{"*":"one|robots"}},"highlight":{"fields":["content","title"],"pre_tags":"before_","post_tags":"_after"}})

res =  await searchApi.search({"index":"books","query":{"match":{"*":"one|robots"}},"highlight":{"fields":["content","title"],"pre_tags":"before_","post_tags":"_after"}});

searchRequest = new SearchRequest();
searchRequest.setIndex("books");
query = new HashMap<String,Object>();
query.put("match",new HashMap<String,Object>(){{
    put("*","one|robots");
}});        
searchRequest.setQuery(query);
highlight = new HashMap<String,Object>(){{
    put("fields",new String[] {"content","title"});
    put("pre_tags","before_");
    put("post_tags","_after");
}};
searchRequest.setHighlight(highlight);
searchResponse = searchApi.search(searchRequest);

Response

Document: 1
title : Books one
content : They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it.
Highlight for content:
- They followed Bander. The before_robots_after remained at a polite distance,
-  three into the room. before_One_after of the before_robots_after followed as well. Bander
-  gestured the other before_robots_after away and entered itself. The
Highlight for title:
- Books before_one_after

{'aggregations': None,
 'hits': {'hits': [{u'_id': u'1',
                    u'_score': 2788,
                    u'_source': {u'content': u'They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. ',
                                 u'title': u'Books one'},
                    u'highlight': {u'content': [u'They followed Bander. The before_robots_after remained at a polite distance, ',
                                                u' three into the room. before_One_after of the before_robots_after followed as well. Bander',
                                                u' gestured the other before_robots_after away and entered itself. The'],
                                   u'title': [u'Books before_one_after']}}],
          'max_score': None,
          'total': 1},
 'profile': None,
 'timed_out': False,
 'took': 0}

{"took":0,"timed_out":false,"hits":{"total":1,"hits":[{"_id":"1","_score":2788,"_source":{"title":"Books one","content":"They followed Bander. The robots remained at a polite distance, but their presence was a constantly felt threat. Bander ushered all three into the room. One of the robots followed as well. Bander gestured the other robots away and entered itself. The door closed behind it. "},"highlight":{"content":["They followed Bander. The before_robots_after remained at a polite distance, "," three into the room. before_One_after of the before_robots_after followed as well. Bander"," gestured the other before_robots_after away and entered itself. The"],"title":["Books before_one_after"]}}]}}

no_match_size

no_match_size works similar to the allow_empty option. If set to 0, acts as allow_empty=1, i.e. allows empty string to be returned as highlighting result when a snippet could not be generated. Otherwise, the beginning of the field will be returned. Optional, default is 1.

• JSON
• PHP
• Python
• Javascript
• Java

JSON PHP Python Javascript Java

POST /search
{
  "index": "books",
  "query": { "match": { "*": "one|robots" } },
  "highlight":
  {
    "fields": [ "content", "title" ],
    "no_match_size": 0
  }
}

$index->setName('books');
$bool = new \Manticoresearch\Query\BoolQuery();
$bool->must(new \Manticoresearch\Query\Match(['query' => 'one|robots'], '*'));
$results = $index->search($bool)->highlight(['content','title'],['no_match_size'=>0])->get();
foreach($results as $doc)
{
    echo 'Document: '.$doc->getId()."\n";
    foreach($doc->getData() as $field=>$value)
    {
        echo $field.' : '.$value."\n";
    }
    foreach($doc->getHighlight() as $field=>$snippets)
    {
        echo "Highlight for ".$field.":\n";
        foreach($snippets as $snippet)
        {
            echo "- ".$snippet."\n";
        }
    }
}