Spring Kafka error handling: retries and dead-letter topics

A Kafka listener throws an exception. What happens next depends on how your Spring Boot service is configured — and the default is rarely what production needs. This guide covers retries, dead-letter topics and the “poison pill” records that can stall a consumer.

What happens by default

In current Spring for Apache Kafka versions, listener containers use the DefaultErrorHandler. When your listener throws, it seeks back and redelivers the record: by default it retries 9 times with no delay (10 attempts in total), then logs the failure and skips the record. The message is effectively lost unless you configure somewhere for it to go.

Some exceptions are treated as fatal and are not retried at all — for example deserialization and message conversion errors — because retrying them can never succeed.

Retries with backoff and a dead-letter topic

Declare your own error handler as a bean: Spring Boot applies it to the auto-configured listener container factory. This one retries with exponential backoff, then publishes the record to a dead-letter topic:

@Bean
public DefaultErrorHandler kafkaErrorHandler(KafkaTemplate<Object, Object> template) {
  var recoverer = new DeadLetterPublishingRecoverer(template);

  var backOff = new ExponentialBackOffWithMaxRetries(4); // 4 retries after the first attempt
  backOff.setInitialInterval(1_000);
  backOff.setMultiplier(2.0);
  backOff.setMaxInterval(10_000);

  var handler = new DefaultErrorHandler(recoverer, backOff);
  handler.addNotRetryableExceptions(InvalidOrderException.class); // retrying won't fix bad data
  return handler;
}

By default, DeadLetterPublishingRecoverer sends the record to a topic named after the original with a .DLT suffix (for example orders.DLT), to the same partition number — so create the DLT with at least as many partitions as the source topic. Exception details are added as record headers, which makes later investigation much easier.

These retries are blocking: while one record is retried, the rest of that partition waits. That preserves ordering, but keep the total retry time well below max.poll.interval.ms, or the consumer is kicked out of the group and a rebalance starts.

Non-blocking retries with @RetryableTopic

When a dependency might be down for minutes, blocking a partition is too costly. @RetryableTopic moves failed records to separate retry topics with delays, so the main topic keeps flowing:

@RetryableTopic(attempts = "4", backoff = @Backoff(delay = 1000, multiplier = 2.0))
@KafkaListener(topics = "orders")
public void onOrder(OrderEvent event) {
  orderService.process(event);
}

@DltHandler
public void onDeadLetter(OrderEvent event, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {
  log.error("Order moved to {} after retries: {}", topic, event);
}

The trade-off: records are no longer processed in order, because a retried record comes back later. Use this when ordering doesn’t matter — or when processing is keyed and idempotent enough to tolerate it.

Poison pills: records that can’t be deserialized

If a record can’t be deserialized, the failure happens before your listener is even called. Without a safety net, the consumer can get stuck failing on the same record. Wrap your real deserializer in ErrorHandlingDeserializer so the failure reaches the error handler, which can skip the record or send it to the dead-letter topic:

spring:
  kafka:
    consumer:
      value-deserializer: org.springframework.kafka.support.serializer.ErrorHandlingDeserializer
      properties:
        spring.deserializer.value.delegate.class: org.springframework.kafka.support.serializer.JsonDeserializer

Which approach to use

• Short, transient failures and ordering matters: blocking retries with backoff, then a DLT.
• Longer outages and ordering doesn’t matter: @RetryableTopic.
• Bad data: mark the exception as not retryable and send the record straight to the DLT.
• Always: ErrorHandlingDeserializer, idempotent processing, and alerts on DLT traffic.

Operating the dead-letter topic

• Alert when records arrive on a DLT — an unwatched DLT is just a slower way to lose data.
• Inspect headers on dead-lettered records to see the exception and original topic, partition and offset.
• Replay after fixing the cause, by re-publishing DLT records to the original topic — which is safe only if processing is idempotent.
• Set retention on DLTs long enough for someone to notice and act.