I wrote an UDF like the below: And I get this error: InvalidRequest: Error from server: code=2200 [Invalid query] message=”Java source compilation failed: Line 1: The operator != is undefined for the argument type(s) long, null Line 1: The operator != is undefined for the argument type(s) double, null” While in manuals these comparisons seems ok (e.g. here). I am
Tag: user-defined-functions
Spark – Transforming Complex Data Types
Goal The goal I want to achieve is to read a CSV file (OK) encode it to Dataset<Person>, where Person object has a nested object Address[]. (Throws an exception) The Person CSV file In a file called person.csv, there is the following data describing some persons: The first line is the schema and address is a nested structure. Data classes