Skip to content
Advertisement

Disable auto topic creation from Spring Kafka Consumer

I don’t want to auto create topics from my Consumer application when the topics are not present.

I know it is a Kafka server level config to disable auto topic creation (auto.create.topics.enable = false), but I cannot do that change in my infrastructure.

So I am looking for a way to disable auto topic creation in my Consumer application (using Spring Kafka).

I tried setting

spring:
  kafka:
    consumer:
      properties:
        allow.auto.create.topics: false

but it is not working!

Seems like Kafka added this support: https://cwiki.apache.org/confluence/display/KAFKA/KIP-361%3A+Add+Consumer+Configuration+to+Disable+Auto+Topic+Creation

Can someone please help here?

Advertisement

Answer

The below configuration in application.yml works perfectly for Consumers based on Spring for Apache Kafka:

spring:
  kafka:
    consumer:
      properties:
        allow.auto.create.topics: false

Here is a reference project for a simple Spring Kafka Consumer.

However in my case, I was also using the non-blocking retries provided by Spring Kafka in the form of @RetryableTopic annotation.

In this case in order to turn off auto topic creation from the Consumer, along with the above mentioned property change, we also need to set a property named autoCreateTopics to "false" in the @RetryableTopic annotation like so:

  @RetryableTopic(
    attempts = "4",
    backoff = @Backoff(delay = 1000),
    fixedDelayTopicStrategy = FixedDelayStrategy.SINGLE_TOPIC,
    autoCreateTopics = "false"
  )

It’s default value is "true".

Here is a reference project for Spring Kafka Consumer with non-blocking retries.

Big thanks to tzolov for pointing me to the right direction.

User contributions licensed under: CC BY-SA
1 People found this is helpful
Advertisement