Skip to content
Advertisement

Kafka Connect: Convert message from bytes to Json

I am trying to use a Google PubSub source connector to fetch data from my google cloud to kafka. I do get the data, but the message comes as bytes. I refered here and as mentioned, I have used a JSON convertor to change it.

Here is my connector code part:

JavaScript

And this what I get in my kafka:

JavaScript

Even after providing the connector, details I get message as byte. Is there something further I should be doing to convert it to json format ?

Advertisement

Answer

The Cloud Pub/Sub Kafka connector does not inspect or convert the data in messages that it receives; it just passes the data field through as bytes, which is the type of the field in the PubsubMessage. There is currently no way to get the connector itself to read the contents of the message and convert it to JSON.

Advertisement