Camunda 8 message correlation explained: names, keys and TTL

You publish a message, the call succeeds — and the process instance just keeps waiting. Message correlation is where many Camunda 8 projects get stuck, because a successful publish doesn’t mean the message reached anything. Here is how correlation really works, and how to debug it.

The basics: name and correlation key

A process waits for a message at a message catch event, a receive task or a message boundary event. Each one references a message with two parts:

• Message name — the type of message, for example payment-received. It must match exactly, including case.
• Correlation key — an expression evaluated for each instance, for example = orderId, that identifies which instance the message is for.

<bpmn:message id="Message_Payment" name="payment-received">
  <bpmn:extensionElements>
    <zeebe:subscription correlationKey="=orderId" />
  </bpmn:extensionElements>
</bpmn:message>

When an instance reaches the catch event, Zeebe evaluates the correlation key and opens a subscription: “instance 2251799813685249 is waiting for payment-received with key ORD-1042”. A published message with the same name and key completes that wait.

Publishing a message

client.newPublishMessageCommand()
    .messageName("payment-received")
    .correlationKey(orderId)              // e.g. "ORD-1042"
    .messageId("payment-" + paymentId)    // optional: rejects duplicates while buffered
    .timeToLive(Duration.ofMinutes(10))   // keep it if nobody is waiting yet
    .variables(Map.of("paymentStatus", "PAID"))
    .send()
    .join();

The correlation key must evaluate to a string or a number, and the value you publish must match it exactly — ORD-1042 and ord-1042 are different keys.

Timing and time-to-live

This is the most common source of “lost” messages. What happens depends on whether a subscription exists when the message is published:

• A matching subscription exists — the message correlates immediately.
• No subscription yet, TTL greater than zero — Zeebe buffers the message. If an instance reaches the matching catch event before the TTL expires, it correlates then.
• No subscription and TTL of zero — the message is discarded. It only correlates if an instance is already waiting.

So if your service publishes the payment confirmation a moment before the process reaches the “Wait for payment” event, a TTL of zero loses the message every time. Use a TTL that covers the realistic gap.

Correlation rules worth knowing

• Publishing is not correlating. The publish call succeeds whether or not any instance is waiting. Newer Camunda 8 releases also offer a “correlate message” API that reports whether the message matched a waiting instance — useful when you need to know immediately.
• Message IDs make publishing idempotent: while a message is buffered, a duplicate with the same ID is rejected — handy when the sender may retry.
• One message, one instance per process: a message can correlate with waiting instances of different processes, but only once per process.
• The key is evaluated on entry: if the variable behind the correlation key doesn’t exist when the instance reaches the event, an incident is raised instead of a subscription.
• Message variables are merged into the instance at the catching element; use output mappings to control exactly what lands in process scope.

Debugging a message that doesn’t correlate

1. Find the instance in Operate. Is it actually waiting at the message event — or is it stuck earlier, or at an incident?
2. Check the correlation key value in the instance’s variables. Compare it character by character with what was published: case, whitespace, prefixes, number versus string formatting.
3. Check the message name in the deployed model against the name in your code.
4. Check timing: was the message published before the instance reached the event, with a TTL of zero or a TTL that had expired?
5. Check the version: instances started on an older process version wait on the older model’s message definition.

Patterns that work well

• Use a business identifier as the correlation key — order ID, application number — never a technical key that the sender doesn’t know.
• Pair waiting events with a timer boundary event so an instance doesn’t wait forever if the message never arrives.
• For “start a process from an event” use a message start event; for “continue a running process” use a catch event or receive task.
• Log the message name, correlation key and message ID on every publish — you will need them.