Skip to content

Tag: spring-kafka

kafka Offset commit failing org.apache.kafka.clients.consumer.CommitFailedException

I have written a kafka consumer using spring-kafka library (spring-boot-starter-parent 2.3.4.RELEASE). I have following consumer configuration in my code Following is my listener method : Here I am reading 1 message at a time, apply business logic and use ack.acknowledge() to commit offset, but what I have seen, sometime offset commit succeed but many time I get org.apache.kafka.clients.consumer.CommitFailedException on line

Use Kakfa connection to dynamically Subscribe/Unsubscribe the Kafka Topics using Spring Boot

I’m developing a SpringBoot application which exposes the APIs to sub/unsub the Kafka topics. All we need to do is to pass the topic-name in the API call and the application will subscribe to it and consume messages. Subscribe topic API : Unsubscribe topic API : Dependency I have created KafkaConsumerConfiguration in which stated some beans (as follows). and I

Spring Kafka multiple topic for one class dynamically

I recently wanted to add a new behavior in my project that uses spring-kafka. The idea is really simple : App1 create a new scenario name “SCENARIO_1” and publish this string in the topic “NEW_SCENARIO” App1 publish some message on topic “APP2-SCENARIO_1” and “APP3-SCENARIO_1” App2 (group-id=app2) listens on NEW_SCENARIO and creates a new consumer<Object,String> listening on a new topic “APP2-SCENARIO_1”

Spring Boot Kafka StreamsConfig or ConsumerConfig from application.yaml not applying

I have a very simple spring boot project with a KTable and I want to customize my configuration in application.yml, but the config seems to not be applied. This is my configuration file application.yml However, when starting the application the log outputs the following: ConsumerConfig: Below is the simple application class I’m using: The values from my application.yml seems to

Incoming kafka metrics are not detected in Appdynamcis – springkafka

We are using springboot and springkafka to consume and process the messages, and Appdynamics for capturing the Performance metrics. Appdynamics is capturing the out going topics and metrics, but not detecting the incoming topic and the metrics.The solutions we tried Custom configured the topics name in the backend Set enable-kafka-consumer to true Custom-interceptors.xml mentioned below In any of the cases