Skip to content
Advertisement

How to read headers in kafka partitioner

I’d like to extend Kafka DefaultPartitioner to make a custom one. However, I find no way to access the message headers as the partitioning should be based on a value present there.

EDIT 1: The task is choosing a partition not based on the key but on another integer contained in the header.

Advertisement

Answer

You cannot access headers in custom partitioner class. But maybe you can create ProducerRecord manually on basis of your header value.

ProducerRecord has many overloaded constructor definitions. Some of them have partition argument. There you can specify partition number which says in which partition your ProducerRecord will go.

e.g ProducerRecord<String,String> rec = new ProducerRecord(topic, partitionNo, key, value);

Otherwise you will have to embed that specific header value inside key or value object and then access it inside partitioner class.

User contributions licensed under: CC BY-SA
7 People found this is helpful
Advertisement