Quarkus - Hibernate Search guide

You have a Hibernate ORM-based application? You want to provide a full-featured full-text search to your users? You’re at the right place.

With this guide, you’ll learn how to synchronize your entities to an Elasticsearch cluster in a heart beat with Hibernate Search. We will also explore how you can can query your Elasticsearch cluster using the Hibernate Search API.

This technology is considered preview.

In preview, backward compatibility and presence in the ecosystem is not guaranteed. Specific improvements might require to change configuration or APIs and plans to become stable are under way. Feedback is welcome on our mailing list or as issues in our GitHub issue tracker.

For a full list of possible extension statuses, check our FAQ entry.

This extension is based on a beta version of Hibernate Search. While APIs are quite stable and the code is of production quality and thoroughly tested, some features are still missing, performance might not be optimal and some APIs or configuration properties might change as the extension matures.

Prerequisites

To complete this guide, you need:

Architecture

The application described in this guide allows to manage a (simple) library: you manage authors and their books.

The entities are stored in a PostgreSQL database and indexed in an Elasticsearch cluster.

Solution

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can go right to the completed example.

Clone the Git repository: git clone [https://github.com/quarkusio/quarkus-quickstarts.git](https://github.com/quarkusio/quarkus-quickstarts.git), or download an archive.

The solution is located in the hibernate-search-elasticsearch-quickstart directory.

The provided solution contains a few additional elements such as tests and testing infrastructure.

Creating the Maven project

First, we need a new project. Create a new project with the following command:

  1. mvn io.quarkus:quarkus-maven-plugin:1.7.6.Final:create \
  2. -DprojectGroupId=org.acme \
  3. -DprojectArtifactId=hibernate-search-elasticsearch-quickstart \
  4. -DclassName="org.acme.hibernate.search.elasticsearch.LibraryResource" \
  5. -Dpath="/library" \
  6. -Dextensions="hibernate-orm-panache, hibernate-search-elasticsearch, resteasy-jsonb, jdbc-postgresql"
  7. cd hibernate-search-elasticsearch-quickstart

This command generates a Maven structure importing the following extensions:

  • Hibernate ORM with Panache,

  • the PostgreSQL JDBC driver,

  • Hibernate Search + Elasticsearch,

  • RESTEasy and JSON-B.

If you already have your Quarkus project configured, you can add the hibernate-search-elasticsearch extension to your project by running the following command in your project base directory:

  1. ./mvnw quarkus:add-extension -Dextensions="hibernate-search-elasticsearch"

This will add the following to your pom.xml:

  1. <dependency>
  2. <groupId>io.quarkus</groupId>
  3. <artifactId>quarkus-hibernate-search-elasticsearch</artifactId>
  4. </dependency>

For now, let’s delete the two generated tests LibraryResourceTest and NativeLibraryResourceIT present in src/test/java. If you are interested in how you can test this application, just refer to the solution in the quickstarts Git repository: it contains a lot of tests and the required testing infrastructure.

Creating the bare entities

First, let’s create our Hibernate ORM entities Book and Author in the model subpackage.

  1. package org.acme.hibernate.search.elasticsearch.model;
  2. import java.util.List;
  3. import java.util.Objects;
  4. import javax.persistence.CascadeType;
  5. import javax.persistence.Entity;
  6. import javax.persistence.FetchType;
  7. import javax.persistence.OneToMany;
  8. import io.quarkus.hibernate.orm.panache.PanacheEntity;
  9. @Entity
  10. public class Author extends PanacheEntity { (1)
  11. public String firstName;
  12. public String lastName;
  13. @OneToMany(mappedBy = "author", cascade = CascadeType.ALL, orphanRemoval = true, fetch = FetchType.EAGER) (2)
  14. public List<Book> books;
  15. @Override
  16. public boolean equals(Object o) {
  17. if (this == o) {
  18. return true;
  19. }
  20. if (!(o instanceof Author)) {
  21. return false;
  22. }
  23. Author other = (Author) o;
  24. return Objects.equals(id, other.id);
  25. }
  26. @Override
  27. public int hashCode() {
  28. return 31;
  29. }
  30. }
1We are using Hibernate ORM with Panache, it is not mandatory.
2We are loading these elements eagerly so that they are present in the JSON output. In a real world application, you should probably use a DTO approach.
  1. package org.acme.hibernate.search.elasticsearch.model;
  2. import java.util.Objects;
  3. import javax.json.bind.annotation.JsonbTransient;
  4. import javax.persistence.Entity;
  5. import javax.persistence.ManyToOne;
  6. import io.quarkus.hibernate.orm.panache.PanacheEntity;
  7. @Entity
  8. public class Book extends PanacheEntity {
  9. public String title;
  10. @ManyToOne
  11. @JsonbTransient (1)
  12. public Author author;
  13. @Override
  14. public boolean equals(Object o) {
  15. if (this == o) {
  16. return true;
  17. }
  18. if (!(o instanceof Book)) {
  19. return false;
  20. }
  21. Book other = (Book) o;
  22. return Objects.equals(id, other.id);
  23. }
  24. @Override
  25. public int hashCode() {
  26. return 31;
  27. }
  28. }
1We mark this property with @JsonbTransient to avoid infinite loops when serializing with JSON-B.

Initializing the REST service

While everything is not yet set up for our REST service, we can initialize it with the standard CRUD operations we will need.

Just copy this content in the LibraryResource file created by the Maven create-project command:

  1. package org.acme.hibernate.search.elasticsearch;
  2. import javax.inject.Inject;
  3. import javax.persistence.EntityManager;
  4. import javax.transaction.Transactional;
  5. import javax.ws.rs.Consumes;
  6. import javax.ws.rs.DELETE;
  7. import javax.ws.rs.POST;
  8. import javax.ws.rs.PUT;
  9. import javax.ws.rs.Path;
  10. import javax.ws.rs.Produces;
  11. import javax.ws.rs.core.MediaType;
  12. import org.acme.hibernate.search.elasticsearch.model.Author;
  13. import org.acme.hibernate.search.elasticsearch.model.Book;
  14. import org.jboss.resteasy.annotations.jaxrs.FormParam;
  15. import org.jboss.resteasy.annotations.jaxrs.PathParam;
  16. @Path("/library")
  17. @Produces(MediaType.APPLICATION_JSON)
  18. @Consumes(MediaType.APPLICATION_JSON)
  19. public class LibraryResource {
  20. @Inject
  21. EntityManager em;
  22. @PUT
  23. @Path("book")
  24. @Transactional
  25. @Consumes(MediaType.APPLICATION_FORM_URLENCODED)
  26. public void addBook(@FormParam String title, @FormParam Long authorId) {
  27. Author author = Author.findById(authorId);
  28. if (author == null) {
  29. return;
  30. }
  31. Book book = new Book();
  32. book.title = title;
  33. book.author = author;
  34. book.persist();
  35. author.books.add(book);
  36. author.persist();
  37. }
  38. @DELETE
  39. @Path("book/{id}")
  40. @Transactional
  41. public void deleteBook(@PathParam Long id) {
  42. Book book = Book.findById(id);
  43. if (book != null) {
  44. book.author.books.remove(book);
  45. book.delete();
  46. }
  47. }
  48. @PUT
  49. @Path("author")
  50. @Transactional
  51. @Consumes(MediaType.APPLICATION_FORM_URLENCODED)
  52. public void addAuthor(@FormParam String firstName, @FormParam String lastName) {
  53. Author author = new Author();
  54. author.firstName = firstName;
  55. author.lastName = lastName;
  56. author.persist();
  57. }
  58. @POST
  59. @Path("author/{id}")
  60. @Transactional
  61. @Consumes(MediaType.APPLICATION_FORM_URLENCODED)
  62. public void updateAuthor(@PathParam Long id, @FormParam String firstName, @FormParam String lastName) {
  63. Author author = Author.findById(id);
  64. if (author == null) {
  65. return;
  66. }
  67. author.firstName = firstName;
  68. author.lastName = lastName;
  69. author.persist();
  70. }
  71. @DELETE
  72. @Path("author/{id}")
  73. @Transactional
  74. public void deleteAuthor(@PathParam Long id) {
  75. Author author = Author.findById(id);
  76. if (author != null) {
  77. author.delete();
  78. }
  79. }
  80. }

Nothing out of the ordinary here: it is just good old Hibernate ORM with Panache operations in a standard JAX-RS service.

In fact, the interesting part is that we will need to add very few elements to make our full text search application working.

Using Hibernate Search annotations

Let’s go back to our entities.

Enabling full text search capabilities for them is as simple as adding a few annotations.

Let’s edit the Author entity again to include this content:

  1. package org.acme.hibernate.search.elasticsearch.model;
  2. import java.util.Objects;
  3. import javax.json.bind.annotation.JsonbTransient;
  4. import javax.persistence.Entity;
  5. import javax.persistence.ManyToOne;
  6. import org.hibernate.search.mapper.pojo.mapping.definition.annotation.FullTextField;
  7. import org.hibernate.search.mapper.pojo.mapping.definition.annotation.Indexed;
  8. import io.quarkus.hibernate.orm.panache.PanacheEntity;
  9. @Entity
  10. @Indexed (1)
  11. public class Book extends PanacheEntity {
  12. @FullTextField(analyzer = "english") (2)
  13. public String title;
  14. @ManyToOne
  15. @JsonbTransient
  16. public Author author;
  17. // Preexisting equals()/hashCode() methods
  18. }
1First, let’s use the @Indexed annotation to register our Book entity as part of the full text index.
2The @FullTextField annotation declares a field in the index specifically tailored for full text search. In particular, we have to define an analyzer to split and analyze the tokens (~ words) - more on this later.

Now that our books are indexed, we can do the same for the authors.

Open the Author class and include the content below.

Things are quite similar here: we use the @Indexed, @FullTextField and @KeywordField annotations.

There are a few differences/additions though. Let’s check them out.

  1. package org.acme.hibernate.search.elasticsearch.model;
  2. import java.util.List;
  3. import java.util.Objects;
  4. import javax.persistence.CascadeType;
  5. import javax.persistence.Entity;
  6. import javax.persistence.FetchType;
  7. import javax.persistence.OneToMany;
  8. import org.hibernate.search.engine.backend.types.Sortable;
  9. import org.hibernate.search.mapper.pojo.mapping.definition.annotation.FullTextField;
  10. import org.hibernate.search.mapper.pojo.mapping.definition.annotation.Indexed;
  11. import org.hibernate.search.mapper.pojo.mapping.definition.annotation.IndexedEmbedded;
  12. import org.hibernate.search.mapper.pojo.mapping.definition.annotation.KeywordField;
  13. import io.quarkus.hibernate.orm.panache.PanacheEntity;
  14. @Entity
  15. @Indexed
  16. public class Author extends PanacheEntity {
  17. @FullTextField(analyzer = "name") (1)
  18. @KeywordField(name = "firstName_sort", sortable = Sortable.YES, normalizer = "sort") (2)
  19. public String firstName;
  20. @FullTextField(analyzer = "name")
  21. @KeywordField(name = "lastName_sort", sortable = Sortable.YES, normalizer = "sort")
  22. public String lastName;
  23. @OneToMany(mappedBy = "author", cascade = CascadeType.ALL, orphanRemoval = true, fetch = FetchType.EAGER)
  24. @IndexedEmbedded (3)
  25. public List<Book> books;
  26. // Preexisting equals()/hashCode() methods
  27. }
1We use a @FullTextField similar to what we did for Book but you’ll notice that the analyzer is different - more on this later.
2As you can see, we can define several fields for the same property. Here, we define a @KeywordField with a specific name. The main difference is that a keyword field is not tokenized (the string is kept as one single token) but can be normalized (i.e. filtered) - more on this later. This field is marked as sortable as our intention is to use it for sorting our authors.
3The purpose of @IndexedEmbedded is to include the Book fields into the Author index. In this case, we just use the default configuration: all the fields of the associated Book entities are included in the index (i.e. the title field). The nice thing with @IndexedEmbedded is that it is able to automatically reindex an Author if one of its Books has been updated thanks to the bidirectional relation. @IndexedEmbedded also supports nested documents (using the storage = NESTED attribute) but we don’t need it here. You can also specify the fields you want to include in your parent index using the includePaths attribute if you don’t want them all.

Analyzers and normalizers

Introduction

Analysis is a big part of full text search: it defines how text will be processed when indexing or building search queries.

The role of analyzers is to split the text into tokens (~ words) and filter them (making it all lowercase and removing accents for instance).

Normalizers are a special type of analyzers that keeps the input as a single token. It is especially useful for sorting or indexing keywords.

There are a lot of bundled analyzers but you can also develop your own for your own specific purposes.

You can learn more about the Elasticsearch analysis framework in the Analysis section of the Elasticsearch documentation.

Defining the analyzers used

When we added the Hibernate Search annotations to our entities, we defined the analyzers and normalizers used. Typically:

  1. @FullTextField(analyzer = "english")
  1. @FullTextField(analyzer = "name")
  1. @KeywordField(name = "lastName_sort", sortable = Sortable.YES, normalizer = "sort")

We use:

  • an analyzer called name for person names,

  • an analyzer called english for book titles,

  • a normalizer called sort for our sort fields

but we haven’t set them up yet.

Let’s see how you can do it with Hibernate Search.

Setting up the analyzers

It is an easy task, we just need to create an implementation of ElasticsearchAnalysisConfigurer (and configure Quarkus to use it, more on that later).

To fulfill our requirements, let’s create the following implementation:

  1. package org.acme.hibernate.search.elasticsearch.config;
  2. import org.hibernate.search.backend.elasticsearch.analysis.ElasticsearchAnalysisConfigurationContext;
  3. import org.hibernate.search.backend.elasticsearch.analysis.ElasticsearchAnalysisConfigurer;
  4. public class AnalysisConfigurer implements ElasticsearchAnalysisConfigurer {
  5. @Override
  6. public void configure(ElasticsearchAnalysisConfigurationContext context) {
  7. context.analyzer("name").custom() (1)
  8. .tokenizer("standard")
  9. .tokenFilters("asciifolding", "lowercase");
  10. context.analyzer("english").custom() (2)
  11. .tokenizer("standard")
  12. .tokenFilters("asciifolding", "lowercase", "porter_stem");
  13. context.normalizer("sort").custom() (3)
  14. .tokenFilters("asciifolding", "lowercase");
  15. }
  16. }
1This is a simple analyzer separating the words on spaces, removing any non-ASCII characters by its ASCII counterpart (and thus removing accents) and putting everything in lowercase. It is used in our examples for the author’s names.
2We are a bit more aggressive with this one and we include some stemming: we will be able to search for mystery and get a result even if the indexed input contains mysteries. It is definitely too aggressive for person names but it is perfect for the book titles.
3Here is the normalizer used for sorting. Very similar to our first analyzer, except we don’t tokenize the words as we want one and only one token.

Adding full text capabilities to our REST service

In our existing LibraryResource, we just need to inject the following methods (and add a few imports):

  1. @Transactional (1)
  2. void onStart(@Observes StartupEvent ev) throws InterruptedException { (2)
  3. // only reindex if we imported some content
  4. if (Book.count() > 0) {
  5. Search.session(em)
  6. .massIndexer()
  7. .startAndWait();
  8. }
  9. }
  10. @GET
  11. @Path("author/search") (3)
  12. @Transactional
  13. public List<Author> searchAuthors(@QueryParam String pattern, (4)
  14. @QueryParam Optional<Integer> size) {
  15. return Search.session(em) (5)
  16. .search(Author.class) (6)
  17. .where(f ->
  18. pattern == null || pattern.trim().isEmpty() ?
  19. f.matchAll() : (7)
  20. f.simpleQueryString()
  21. .fields("firstName", "lastName", "books.title").matching(pattern) (8)
  22. )
  23. .sort(f -> f.field("lastName_sort").then().field("firstName_sort")) (9)
  24. .fetchHits(size.orElse(20)); (10)
  25. }
1Important point: we need a transactional context for these methods.
2As we will import data into the PostgreSQL database using an SQL script, we need to reindex the data at startup. For this, we use Hibernate Search’s mass indexer, which allows to index a lot of data efficiently (you can fine tune it for better performances). All the upcoming updates coming through Hibernate ORM operations will be synchronized automatically to the full text index. If you don’t import data manually in the database, you don’t need that: the mass indexer should then only be used when you change your indexing configuration (adding a new field, changing an analyzer’s configuration…​) and you want the new configuration to be applied to your existing entities.
3This is where the magic begins: just adding the annotations to our entities makes them available for full text search: we can now query the index using the Hibernate Search DSL.
4Use the org.jboss.resteasy.annotations.jaxrs.QueryParam annotation type to avoid repeating the parameter name.
5First, we get an Hibernate Search session from the injected entity manager.
6We indicate that we are searching for Authors.
7We create a predicate: if the pattern is empty, we use a matchAll() predicate.
8If we have a valid pattern, we create a simpleQueryString() predicate on the firstName, lastName and books.title fields matching our pattern.
9We define the sort order of our results. Here we sort by last name, then by first name. Note that we use the specific fields we created for sorting.
10Fetch the size top hits, 20 by default. Obviously, paging is also supported.

The Hibernate Search DSL supports a significant subset of the Elasticsearch predicates (match, range, nested, phrase, spatial…​). Feel free to explore the DSL using autocompletion.

When that’s not enough, you can always fall back to defining a predicate using JSON directly.

Configuring the application

As usual, we can configure everything in the Quarkus configuration file, application.properties.

Edit src/main/resources/application.properties and inject the following configuration:

  1. quarkus.ssl.native=false (1)
  2. quarkus.datasource.db-kind=postgresql (2)
  3. quarkus.datasource.username=quarkus_test
  4. quarkus.datasource.password=quarkus_test
  5. quarkus.datasource.jdbc.url=jdbc:postgresql:quarkus_test
  6. quarkus.hibernate-orm.database.generation=drop-and-create (3)
  7. quarkus.hibernate-orm.sql-load-script=import.sql (4)
  8. quarkus.hibernate-search.elasticsearch.version=7 (5)
  9. quarkus.hibernate-search.elasticsearch.analysis.configurer=org.acme.hibernate.search.elasticsearch.config.AnalysisConfigurer (6)
  10. quarkus.hibernate-search.schema-management.strategy=drop-and-create (7)
  11. quarkus.hibernate-search.elasticsearch.index-defaults.schema-management.required-status=yellow (8)
  12. quarkus.hibernate-search.automatic-indexing.synchronization.strategy=sync (9)
1We won’t use SSL so we disable it to have a more compact native executable.
2Let’s create a PostgreSQL datasource.
3We will drop and recreate the schema every time we start the application.
4We load some initial data.
5We need to tell Hibernate Search about the version of Elasticsearch we will use. It is important because there are significant differences between Elasticsearch mapping syntax depending on the version. Since the mapping is created at build time to reduce startup time, Hibernate Search cannot connect to the cluster to automatically detect the version.
6We point to the custom AnalysisConfigurer which defines the configuration of our analyzers and normalizers.
7Obviously, this is not for production: we drop and recreate the index every time we start the application.
8We consider the yellow status is sufficient to proceed after an index is created. This is for testing purposes with the Elasticsearch Docker container. It should not be used in production.
9This means that we wait for the entities to be searchable before considering a write complete. On a production setup, the write-sync default will provide better performance. Using sync is especially important when testing as you need the entities to be searchable immediately.
For more information about the Hibernate Search extension configuration please refer to the Configuration Reference.

Creating a frontend

Now let’s add a simple web page to interact with our LibraryResource. Quarkus automatically serves static resources located under the META-INF/resources directory. In the src/main/resources/META-INF/resources directory, overwrite the existing index.html file with the content from this index.html file.

Automatic import script

For the purpose of this demonstration, let’s import an initial dataset.

Let’s create a src/main/resources/import.sql file with the following content:

  1. INSERT INTO author(id, firstname, lastname) VALUES (nextval('hibernate_sequence'), 'John', 'Irving');
  2. INSERT INTO author(id, firstname, lastname) VALUES (nextval('hibernate_sequence'), 'Paul', 'Auster');
  3. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'The World According to Garp', 1);
  4. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'The Hotel New Hampshire', 1);
  5. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'The Cider House Rules', 1);
  6. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'A Prayer for Owen Meany', 1);
  7. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'Last Night in Twisted River', 1);
  8. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'In One Person', 1);
  9. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'Avenue of Mysteries', 1);
  10. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'The New York Trilogy', 2);
  11. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'Mr. Vertigo', 2);
  12. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'The Brooklyn Follies', 2);
  13. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'Invisible', 2);
  14. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), 'Sunset Park', 2);
  15. INSERT INTO book(id, title, author_id) VALUES (nextval('hibernate_sequence'), '4 3 2 1', 2);

Preparing the infrastructure

We need a PostgreSQL instance and an Elasticsearch cluster.

Let’s use Docker to start one of each:

  1. docker run -it --rm=true --name elasticsearch_quarkus_test -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch-oss:{elasticsearch-version}
  1. docker run --ulimit memlock=-1:-1 -it --rm=true --memory-swappiness=0 --name postgresql_quarkus_test -e POSTGRES_USER=quarkus_test -e POSTGRES_PASSWORD=quarkus_test -e POSTGRES_DB=quarkus_test -p 5432:5432 postgres:11.3

Time to play with your application

You can now interact with your REST service:

  • start Quarkus with ./mvnw compile quarkus:dev

  • open a browser to [http://localhost:8080/](http://localhost:8080/)

  • search for authors or book titles (we initialized some data for you)

  • create new authors and books and search for them too

As you can see, all your updates are automatically synchronized to the Elasticsearch cluster.

Building a native executable

You can build a native executable with the usual command ./mvnw package -Pnative.

As usual with native executable compilation, this operation consumes a lot of memory.

It might be safer to stop the two containers while you are building the native executable and start them again once you are done.

Running it is as simple as executing ./target/hibernate-search-elasticsearch-quickstart-1.0-SNAPSHOT-runner.

You can then point your browser to [http://localhost:8080/](http://localhost:8080/) and use your application.

The startup is a bit slower than usual: it is mostly due to us dropping and recreating the database schema and the Elasticsearch mapping every time at startup. We also inject some data and execute the mass indexer.

In a real life application, it is obviously something you won’t do at startup.

Further reading

If you are interested into learning more about Hibernate Search 6, the Hibernate team has some documentation in the works.

It is still work in progress but covers all main concepts and features, so it can guide you in your exploration.

FAQ

Why Hibernate Search 6 (and not a fully supported version)?

To optimize the Hibernate Search bootstrap for Quarkus, the Hibernate team had to reorganize things a bit (collect the metadata offline, start the Elasticsearch client later…​).

This couldn’t be done in the 5.x code base so we decided to go with the in-progress Hibernate Search 6.

Can I really use it?

While Hibernate Search 6 is still at Beta stage, the code is of production quality and can be relied on.

What we don’t guarantee is that there might be API changes along the way to the final release of Hibernate Search 6 and you might have to adapt your code.

If it is not a major issue for you, then sure you can use it.

Why Elasticsearch only?

Hibernate Search supports both a Lucene backend and an Elasticsearch backend.

In the context of Quarkus and to build microservices, we thought the latter would make more sense. Thus we focused our efforts on it.

We don’t have plans to support the Lucene backend in Quarkus for now.

Hibernate Search Configuration Reference

About the Duration format

The format for durations uses the standard java.time.Duration format. You can learn more about it in the Duration#parse() javadoc.

You can also provide duration values starting with a number. In this case, if the value consists only of a number, the converter treats the value as seconds. Otherwise, PT is implicitly prepended to the value to obtain a standard java.time.Duration format.