I am writing a little Kafka metrics exporter (Yes there are loads available like prometheus etc but I want a light weight custom one. Kindly excuse me on this). As part of this I would like to know as soon as first message is received (or topic has messages) in a Kafka topic. I am using Spring Boot and Kafka.
I don’t want to auto create topics from my Consumer application when the topics are not present. I know it is a Kafka server level config to disable auto topic creation (auto.create.topics.enable = false), but I cannot do that change in my infrastructure. So I am looking for a way to disable auto topic creation in my Consumer application (using
I set 2 different message keys for 2 partitions . This is my application.yml for producer application To test parallelism, I created a consumer application that reads messages from pf-topic. This is configuration from consumer application. . I created a function in consumer application to consume messages Now it is time for testing. To test parallelism, I run 2 instances
I’m using Log4j2 (v2.17.2) to send information directly to kafka, and am using XML to create the configuration (as many articles mention that XML can handle a lot more configuration options – properties do not). The issue that I have is that my console is filled to the brim with irrelevant INFO log-lines (as an example): I’ve tried all the
I recently wanted to add a new behavior in my project that uses spring-kafka. The idea is really simple : App1 create a new scenario name “SCENARIO_1” and publish this string in the topic “NEW_SCENARIO” App1 publish some message on topic “APP2-SCENARIO_1” and “APP3-SCENARIO_1” App2 (group-id=app2) listens on NEW_SCENARIO and creates a new consumer<Object,String> listening on a new topic “APP2-SCENARIO_1”
Having this Listener KafkaConsumerRebalanceListener class When I add into my application.properties file And adding this dependency into another class I’m having this exception when I run the service Any idea what’s wrong? Regards Answer You have to use same @Identifier(“responses”) near your injection point. It’s because @Identifier have @Qualifier over it.
I have a very simple spring boot project with a KTable and I want to customize my configuration in application.yml, but the config seems to not be applied. This is my configuration file application.yml However, when starting the application the log outputs the following: ConsumerConfig: Below is the simple application class I’m using: The values from my application.yml seems to
I have installed kafka via docker. When i run the docker-compose up command I face following errors: Below is my docker-compose.yml file: Answer Change the variable values to use /certs
I am working through my first sample of Kafka Streams: When I attempt to run it I get this error: This is a sample message from the “users” topic: Value: Header: Key: What should I do to avoid this problem? Answer If the key actually has data, you shouldn’t be using Serdes.Void() or KStream<Void,
I am unable to read in batch with the kafka camel consumer, despite following an example posted here. Are there changes I need to make to my producer, or is the problem most likely with my consumer configuration? The application in question utilizes the kafka camel component to ingest messages from a rest endpoint, validate them, and place them on