Gson library is not working reliably as intended for parsing large JSON file

Tags: , ,



I have to parse a really huge JSON file (file size can get to several GBs), so I cannot just load the entire JSON String into the memory and parse it into an object, I have to somehow read the JSON String line by line and parse it. I am currently using JsonReader from Gson library, which was working great, but recently I discovered that it occasionally throws an error saying Unterminated string at line 1 column xxxxxxxxx path $.fieldname[random index].fieldname[random index].fieldname, but when I parsed it using a different library like Jackson, the parsing went flawlessly (this file is not that huge, only 50 MB so I can just load it into the memory and parse it into an object), so is this a BUG in Gson? And if it is, is there any other JAVA library I can use to do the same thing? I will be appreciated for any answer!

PS: I am using gson-2.8.2

EDIT: I have tested the same file again using Gson, the same error occurred but at the different line and different position, so is it confirmed this is BUG in Gson?

Answer

I tested the parsing with Jackson library as well and still got the same type of error, however, throughout many tests, it turns out Gson and Jackson libraries can have a problem (not always) handling JSON file that is NOT pretty printed (meaning that the JSON does not have proper indentation), because all the JSON files I tested are putting the entire JSON string on a single line (technically still a legit JSON), after I formatted it to have indentations, the parsing went through successfully (both Gson and Jackson), hope this can help anyone who encountered the same issue as I did



Source: stackoverflow