Outbox Quarkus Extension

Overview

This extension is inspired by the Outbox Event Router single message transformation (SMT). As discussed in the blog post Reliable Microservices Data Exchange with the Outbox Pattern, microservices often need to exchange information with one another and an excellent way to deal with that is using the Outbox pattern combined with Debezium’s Outbox Event Router SMT.

The following image shows the overall architecture of this pattern:

Outbox Pattern

The Outbox extension’s goal is to provide a Quarkus application with a reusable, highly configurable component that facilitates the use of the Outbox pattern paired with Debezium’s CDC connector pipeline to reliably and asynchronously share data with any consumer of said data.

Getting Started

In order to start using the Debezium Outbox Quarkus extension, the extension needs to be added as a part of the Quarkus application as follows:

  1. <dependency>
  2. <groupId>io.debezium.quarkus</groupId>
  3. <artfiactId>debezium-quarkus-outbox</artfiactId>
  4. <version>1.2.5.Final</version>
  5. </dependency>

The extension provides the application with the io.debezium.outbox.quarkus.ExportedEvent interface. It’s expected that an application class will implement this interface and that the event will be emitted using the javax.enterprise.event.Event class.

The ExportedEvent interface is parameterized to allow the application to designate the Java types used by several attributes emitted by the event. It’s important that for a given Quarkus application, all implementations of the ExportedEvent interface must use the same parameter types or a build failure will be thrown since all events will use the same underlying database table.

Example

The following illustrates an implementation of the ExportedEvent interface representing an order that has been created:

OrderCreatedEvent.java

  1. public class OrderCreatedEvent implements ExportedEvent<String, JsonNode> {
  2. private static final String TYPE = "Order";
  3. private static final String EVENT_TYPE = "OrderCreated";
  4. private final long orderId;
  5. private final JsonNode jsonNode;
  6. private final Instant timestamp;
  7. public OrderCreatedEvent(Instant createdAt, Order order) {
  8. this.orderId = order.getId();
  9. this.timestamp = createdAt;
  10. this.jsonNode = convertToJson(order);
  11. }
  12. @Override
  13. public String getAggregateId() {
  14. return String.valeuOf(orderId);
  15. }
  16. @Override
  17. public String getAggregateType() {
  18. return TYPE;
  19. }
  20. @Override
  21. public JsonNode getPayload() {
  22. return jsonNode;
  23. }
  24. @Override
  25. public String getType() {
  26. return EVENT_TYPE;
  27. }
  28. @Override
  29. public Instant getTimestamp() {
  30. return timestamp;
  31. }
  32. }

The following example illustrates an OrderService that emits the OrderCreatedEvent:

OrderService.java

  1. @ApplicationScoped
  2. public class OrderService {
  3. @Inject
  4. Event<ExportedEvent<?, ?>> event;
  5. @Transactional
  6. public Order addOrder(Order order) {
  7. order = orderRepository.save(order);
  8. event.fire(new OrderCreatedEvent(Instant.now(), order));
  9. return order;
  10. }
  11. }

When the application code fires the event by calling Event#fire(), the Outbox extension will be notified that the event occurred and persists the contents of the event into an outbox event table within the scope of the current transaction. The Debezium CDC connector in conjunction with the Outbox Event Router will be monitoring this table and will be responsible for relaying that data using CDC events.

To see a full end-to-end demo, the Outbox example illustrates two Quarkus microservice applications using the outbox pattern to share data between them when orders are placed or cancelled.

Configuration

The Outbox extension can be configured by setting options in the Quarkus application.properties file. The extension works out-of-the-box with a default configuration, but this configuration may not be ideal for every situation.

Build time configuration options

Configuration property

Type

Default

    quarkus.debezium-outbox.table-name

    The table name to be used when creating the outbox table.

string

OutboxEvent

    quarkus.debezium-outbox.id.name

    The column name for the event id column.
    e.g. uuid

string

id

    quarkus.debezium-outbox.id.column-definition

    The database-specific column definition for the event id column.
    e.g. uuid not null

string

UUID NOT NULL

    quarkus.debezium-outbox.aggregate-id.name

    The column name for the event key column.

string

aggregateid

    quarkus.debezium-outbox.aggregate-id.column-definition

    The database-specific column definition for the aggregate id.
    e.g. varchar(50) not null

string

VARCHAR(255) NOT NULL

    quarkus.debezium-outbox.aggregate-id.converter

    The JPA AttributeConverter for the event key column.
    e.g. com.company.TheAttributeConverter

string

    quarkus.debezium-outbox.aggregate-type.name

    The column name for the event aggregate type column.

string

aggregatetype

    quarkus.debezium-outbox.aggregate-type.column-definition

    The database-specific column definition for the aggregate type.
    e.g. varchar(15) not null

string

VARCHAR(255) NOT NULL

    quarkus.debezium-outbox.aggregate-type.converter

    The JPA AttributeConverter for the event aggregate type column.
    e.g. com.company.TheAttributeConverter

string

    quarkus.debezium-outbox.type.name

    The column name for the event type column.

string

type

    quarkus.debezium-outbox.type.column-definition

    The database-specific column definition for the event type.
    e.g. varchar(50) not null

string

VARCHAR(255) NOT NULL

    quarkus.debezium-outbox.type.converter

    The JPA AttributeConverter for the event type column.
    e.g. com.company.TheAttributeConverter

string

    quarkus.debezium-outbox.timestamp.name

    The column name for the event timestamp column.

string

timestamp

    quarkus.debezium-outbox.timestamp.column-definition

    The database-specific column definition for the event timestamp.
    e.g. timestamp not null

string

TIMESTAMP NOT NULL

    quarkus.debezium-outbox.timestamp.converter

    The JPA AttributeConverter for the event timestamp column.
    e.g. com.company.TheAttributeConverter

string

    quarkus.debezium-outbox.payload.name

    The column name for the event payload column.

string

payload

    quarkus.debezium-outbox.payload.column-definition

    The database-specific column definition for the event payload.
    e.g. text not null

string

VARCHAR(8000)

    quarkus.debezium-outbox.payload.converter

    The JPA AttributeConverter for the event payload column.
    e.g. com.company.TheAttributeConverter

string

The build time configuration defaults will work with the Outbox Event Router SMT out of the box. When not using the default values, be sure that the SMT configuration matches.

Runtime configuration options

Configuration property

Type

Default

    quarkus.debezium-outbox.remove-after-insert

    Whether the outbox entry is removed after having been inserted.

    The removal of the entry does not impact the Debezium connector from being able to emit CDC events. This is used as a way to keep the table’s underlying storage from growing over time.

boolean

true