I am parsing a json file in SparkSQL in JAVA and I need to be able to access coordinates that are returned in what appears to be a WrappedArray of WrappedArrays. Here is the code:
JavaScript
x
df.registerTempTable("test_articles");
sql = "select gnip.profileLocations.geo.coordinates FROM test_articles";
DataFrame testData = sqlContext.sql(sql);
Row [] rowArray = testData.collect();
for(Row r:rowArray)
{
if(r.get(0)!=null)
System.out.println(r.get(0).toString());
}
OUTPUT: WrappedArray(WrappedArray(30.74806, 40.79944))
file.json
JavaScript
"gnip": {
"profileLocations": [{
"objectType": "place",
"geo": {
"type": "point",
"coordinates": [132.56111,
35.07693]
}
}]
}
Advertisement
Answer
Spark SQL Row has getList
method, that returns a Java list instead of WrappedArray. So, in the above example, one could say r.getList(0)