Kafka consumer missing messages while consuming messages in loop

I am running my consumer code in loop due to memory constraints, committing my data and then loading into tables Following is the code which will run in loop // here is the main part of the component,…

Can an offset of an unassigned partition be committed by KafkaConsumer.commitSync/commitAsync

KafkaConsumer.commitSync(Map offsets) Can above method be used to commit offset of an unassigned TopicPartition ? I know ConsumerRebalanceListener….

Few kafka partitions are not getting assigned to any flink consumer

I have a kafka topic with 15 partitions [0-14] and I’m running flink with 5 parallelism. So ideally each parallel flink consumer should consume 3 partitions each. But even after multiple restarts, few …