I don’t want to auto create topics from my Consumer application when the topics are not present.
I know it is a Kafka server level config to disable auto topic creation (auto.create.topics.enable = false
), but I cannot do that change in my infrastructure.
So I am looking for a way to disable auto topic creation in my Consumer application (using Spring Kafka).
I tried setting
spring: kafka: consumer: properties: allow.auto.create.topics: false
but it is not working!
Seems like Kafka added this support: https://cwiki.apache.org/confluence/display/KAFKA/KIP-361%3A+Add+Consumer+Configuration+to+Disable+Auto+Topic+Creation
Can someone please help here?
Advertisement
Answer
The below configuration in application.yml
works perfectly for Consumers based on Spring for Apache Kafka:
spring: kafka: consumer: properties: allow.auto.create.topics: false
Here is a reference project for a simple Spring Kafka Consumer.
However in my case, I was also using the non-blocking retries provided by Spring Kafka in the form of @RetryableTopic
annotation.
In this case in order to turn off auto topic creation from the Consumer, along with the above mentioned property change, we also need to set a property named autoCreateTopics
to "false"
in the @RetryableTopic
annotation like so:
@RetryableTopic( attempts = "4", backoff = @Backoff(delay = 1000), fixedDelayTopicStrategy = FixedDelayStrategy.SINGLE_TOPIC, autoCreateTopics = "false" )
It’s default value is "true"
.
Here is a reference project for Spring Kafka Consumer with non-blocking retries.
Big thanks to tzolov for pointing me to the right direction.