I’d like to extend Kafka DefaultPartitioner to make a custom one. However, I find no way to access the message headers as the partitioning should be based on a value present there. EDIT 1: The task is choosing a partition not based on the key but on another integer contained in the header. Answer You ca…
Tag: kafka-producer-api
kafka producer does not throw exception when broker is down
Created a cluster with two brokers using same zookeeper and trying to produce message to a topic whose details are as below. When the producer sets acks=”all” or -1,min.insync.replicas=”2″, it is supposed to receive acknowledgement from the brokers(leaders and replicas) but when one br…
I’m getting “Topic not present in metadata after 60000 ms” message on some computers
Here’s my program I run it on Windows and Linux. On some computers it runs fine, but on other computers, specifically a Linux machine which is not the kafka machine, it consistently gives me this error: This happens, of course, when trying to send a message in the run() function, specifically in the sen…
SpringBoot, Kafka : java.lang.NoSuchMethodError: org.apache.kafka.clients.producer.Producer.close(Ljava/time/Duration;)V
I’m using spring boot v2.2.4 and Apache Kafka in my project. Below is my pom.xml file: Below is the code which i have as part of kafka But when json message is sent to kafka queue, i’m getting below error however json message is getting reached to queue, but i want to understand why i’m gett…
Kafka message ordering in partition while producer retry
According to producer configs, there are: retries and max.in.flight.requests.per.connection. Suppose that retries > 0 and max.in.flight.requests.per.connection > 1. Can messages arrive out of order within ONE partition of topic (e.g. if first message has retries, but second message delivered to broker w…