I am parsing a json file in SparkSQL in JAVA and I need to be able to access coordinates that are returned in what appears to be a WrappedArray of WrappedArrays. Here is the code:
df.registerTempTable("test_articles"); sql = "select gnip.profileLocations.geo.coordinates FROM test_articles"; DataFrame testData = sqlContext.sql(sql); Row [] rowArray = testData.collect(); for(Row r:rowArray) { if(r.get(0)!=null) System.out.println(r.get(0).toString()); }
OUTPUT: WrappedArray(WrappedArray(30.74806, 40.79944))
file.json
"gnip": { "profileLocations": [{ "objectType": "place", "geo": { "type": "point", "coordinates": [132.56111, 35.07693] } }] }
Advertisement
Answer
Spark SQL Row has getList
method, that returns a Java list instead of WrappedArray. So, in the above example, one could say r.getList(0)