Flink ElasticsearchSinkFunction not serializable in non-static method, but serializable in static method

I have a piece of code that only works inside static methods. If I put the code in a static method, then call it from a non-static method, it works. Never heard of anything like this and couldn’t find …

Few kafka partitions are not getting assigned to any flink consumer

I have a kafka topic with 15 partitions [0-14] and I’m running flink with 5 parallelism. So ideally each parallel flink consumer should consume 3 partitions each. But even after multiple restarts, few …

How do I time the checkpointing in Apache Flink streaming?

I am running the Fraud Detector example of Apache Flink with RocksDB as my state backend. I want to know how long does Apache Flink takes to checkpoint the state. My approach is to print time before …

Flink: does Flink support abstract operator which can process different data streams with common fields?

Suppose we have multiple data streams and they share some common features. For example, we have a stream of Teacher and a stream of Student, and they both have an age field. If I want to find out the …

Flink Avro Serialization shows “not serializable” error when working with GenericRecords

I’m really having a hard time making Flink to communicate properly with a running Kafka instance making use of an Avro schema from the Confluent Schema Registry (for both key and value). After a …

TimeCharacteristics & TimerService in Apache Flink

I’m currently working through this tutorial on Stream processing in Apache Flink and am a little confused on how the TimeCharacteristics of a StreamEnvironment effect the order of the data values in …

Apache Flink – custom java options are not recognized inside job

I’ve added the following line to flink-conf.yaml: env.java.opts: “-Ddy.props.path=/PATH/TO/PROPS/FILE” when starting jobmanager (jobmanager.sh start cluster) I see in logs that the jvm option is …