Using Debezium 0.7 to read from MySQL but getting flush timeout and OutOfMemoryError errors in the initial snapshot phase. Looking at the logs below it seems like the connector is trying to write too many messages in one go: Wonder what the correct settings are http://debezium.io/docs/connectors/mysql/#connector-properties for sizeable databases (>50GB). I didn’t have this issue with smaller databases. Simply increasing
Tag: apache-kafka
Kafka message ordering in partition while producer retry
According to producer configs, there are: retries and max.in.flight.requests.per.connection. Suppose that retries > 0 and max.in.flight.requests.per.connection > 1. Can messages arrive out of order within ONE partition of topic (e.g. if first message has retries, but second message delivered to broker with the first attempt)? Or do out of order only happen across several partitions of topic, but within partition
Kafka Streams – Send on different topics depending on Streams Data
I have a kafka streams application waiting for records to be published on topic user_activity. It will receive json data and depending on the value of against a key I want to push that stream into different topics. This is my streams App code: In this code, I want to check operation type and then depending on that I want