Skip to content

Tag: avro

AvroParquetOutputFormat – Unable to Write Arrays with Null Elements

I’m using v1.11.1 of the parquet-mr library as part of a Java application that takes Avro records and writes them into Parquet files using the AvroParquetOutputFormat. There are Avro records with array type fields that will have null elements, e.g. Here’s an example Avro schema: I’m trying to write the following record: I thought I could use the 3-level list

Deserialization of JSON and Avro without Schema

I have been trying to implement Avro deserialization without confluent schema registry. A quick research shows that I can embed the schema in header before sending the record to topic. But the schema again has to be serialized to bytes before embedding on the header which again makes this problematic. Is there anyway to achieve this? What are what are