Skip to content
Advertisement

Tag: apache-flink

How to see restored value from savepoints?

I have tried to log info about states that recovered when I start a Flink app using savepoints. I see that the app was started using a savepoint correctly. But I can’t log state values that were recovered. The app calculates how many products were bought. Can I use CheckpointedFunction in such way? I see that context.isRestored() return true, but

Developing job for Flink

I am building a simple data pipeline for learning purposes. I have real time data coming from Kafka, I would like to do some transformations using Flink. Unfortunately, I’m not sure if I understand correctly deployment options. In the the Flink docs I have found section about Docker Compose and application mode. It says that I can deploy only one

What does latency metrics represent in Flink and are them really valid to evaluate the latency of an application?

I’ve a pipeline developed as shown below: I’m thinking hard about a way to value latency from input to output, but the relation between input and output isn’t 1 to 1 but there are a lot of transformations that make the latency evaluation conceptually very hard. I know about the latency metrics given by using the method: env.getConfig().setLatencyTrackingInterval(1000), but I

Flink: java.lang.NoSuchMethodError: AvroSchemaConverter

I am trying to connect to Kafka. When I run a simple JAR file, I get the following error: Yes, I am aware that NoSuchMethod can potentially mean that there is a conflict between versions; however, there is no Flink instance running on YARN cluster. I have also tried to play with pom.xml but no luck The versions are indicated

Advertisement