Having this Listener KafkaConsumerRebalanceListener class When I add into my application.properties file And adding this dependency into another class I’m having this exception when I run the service Any idea what’s wrong? Regards Answer You have to use same @Identifier(“responses”) ne…
Tag: apache-kafka
Spring Boot Kafka StreamsConfig or ConsumerConfig from application.yaml not applying
I have a very simple spring boot project with a KTable and I want to customize my configuration in application.yml, but the config seems to not be applied. This is my configuration file application.yml However, when starting the application the log outputs the following: ConsumerConfig: Below is the simple ap…
Kafka docker – NoSuchFileException: /opt/kafka.server.keystore.jks
I have installed kafka via docker. When i run the docker-compose up command I face following errors: Below is my docker-compose.yml file: Answer Change the variable values to use /certs
Kafka Streams: java.lang.IllegalArgumentException: Data should be null for a VoidDeserializer
I am working through my first sample of Kafka Streams: When I attempt to run it I get this error: This is a sample message from the “users” topic: Value: Header: Key: What should I do to avoid this problem? Answer If the key actually has data, you shouldn’t be using Serdes.Void() or KStream&…
Batch consumer camel kafka
I am unable to read in batch with the kafka camel consumer, despite following an example posted here. Are there changes I need to make to my producer, or is the problem most likely with my consumer configuration? The application in question utilizes the kafka camel component to ingest messages from a rest end…
Incoming kafka metrics are not detected in Appdynamcis – springkafka
We are using springboot and springkafka to consume and process the messages, and Appdynamics for capturing the Performance metrics. Appdynamics is capturing the out going topics and metrics, but not detecting the incoming topic and the metrics.The solutions we tried Custom configured the topics name in the ba…
The Kafka topic is here, a Java consumer program finds it, but lists none of its content, while a kafka-console-consumer is able to
It’s my first Kafka program. From a kafka_2.13-3.1.0 instance, I created a Kafka topic poids_garmin_brut and filled it with this csv: And at anytime now, before or after running the program I’ll show, its content can be displayed by a kafka-console-consumer command: Here is the Java program, based…
How to properly convert HashMap<String,List> to Json with array of json object
I have a stream of Kafka messages and wanted to build a HashMap<String,List<Object>> to be used as API response in Json format. expected response: actual response: Answer It is now working using these changes: Convert Kafka message string to JSONObject new JSONObject(consumerRecord.value()) Constr…
How to read headers in kafka partitioner
I’d like to extend Kafka DefaultPartitioner to make a custom one. However, I find no way to access the message headers as the partitioning should be based on a value present there. EDIT 1: The task is choosing a partition not based on the key but on another integer contained in the header. Answer You ca…
Does the Kafka Streams StreamBuilder always detect “duplicate” input topics?
This code creates two KStream instances separately, both are reading from the same topic: The topology looks like this: There is only one source definition which is then used two times: Now my question is: Is it always safe to assume that the StreamBuilder creates only one source (= only one consumer for the …