How to Parse Json containing Array of Arrays Java

I have a Json which contains array of Arrays, now I need to parse that Json and count the elements, and after reaching certain limit I need to put it into result Json. I was able to parse till one level and count the elements. How can I parse multiple levels and get the object in same format:

here is the sample code I tried for parsing one level with counting no.of elements:

private void handleJson(Object jsonObj, CountObject c, JSONObject jsonObj) {
        Map<String, Object> map= new HashMap<>();

        if (jsonObj instanceof JSONObject) {

            parseJson(inputJSON,c, map, jsonObj);

        } 

        }

    }


private void parseJson(Object inputObj, CountObject c, Map<String, Object> map, JSONObject jsonObj) {
        JSONObject nodeJson = (JSONObject) inputJSON;
        Iterator<String> keyIter = nodeJson.keySet().iterator();

        while (keyIter.hasNext()) {
            String key = keyIter.next();
            Object value = nodeJson.get(key);

            if (value instanceof JSONObject) {
                int offSet = c.getOffSet();
                if(c.getLimit() == c.getOffSet()) {
                    break;
                }
            keyIter.remove(); 
            map.put(key, value); 
            c.setOffSet(++offSet);;
            } else {
                handleJSONArray(value,k, map, key);
            }
        }
        for (Entry<String, Object> entry : map.entrySet()) {
            jsonObj.put(entry.getKey(), entry.getValue());
}
    }


private void handleJSONArray(Object inputJSON, CountObject c, Map<String, Object> map, String key) {
        JSONArray nodeJsonArr = (JSONArray) inputJSON;
        int offSet = c.getOffSet();
        List<Object> ll = new ArrayList<>();
        for (int i = 0; i < nodeJsonArr.length(); i++) {
            Object value = nodeJsonArr.get(i);
            if (value instanceof JSONArray) {
                handleJSONArray(value, c, map, key2);
            } else {
                
                if (k.getLimit() == k.getOffSet()) {
                    break;
                }
                ll.add(value);
                ++offSet;
            }
        }
        map.put(key2, ll);
        c.setOffSet(offSet);
    }

and here is my Json :

{
"emails": [
{
"emails": [
{
"email": {
"id": "ac9e95cf-3338-4094-b465-e0e1deca23c4",
"value": "[e-mail]"
}
}
]
},
{
"email": {
"id": "b61ffb48-ffc7-4ae6-81a2-78b632892fda",
"value": "[e-mail]"
}
}
],
"lastName": {
"id": "ffe19ece-819b-4680-8e0b-8566b34c973d",
"value": "FirstName"
},
"firstName": {
"id": "4ed234f4-f679-40f3-b76b-41d9fdef7390",
"value": "LastName"
}
}

And count Object is a Pojo which has offset and Limit variables , If I pass limit as 3 I should fetch only first 3 elements with same json format something like below :

{
"emails": [
{
"emails": [
{
"email": {
"id": "ac9e95cf-3338-4094-b465-e0e1deca23c4",
"value": "[e-mail]"
}
}
]
},
{
"email": {
"id": "b61ffb48-ffc7-4ae6-81a2-78b632892fda",
"value": "[e-mail]"
}
}
],
"lastName": {
"id": "ffe19ece-819b-4680-8e0b-8566b34c973d",
"value": "FirstName"
}

Here I gave one of the sample JSON file, and Json can contain any no.of inner Array of elements, logic should be able to parse any type of Json. Here I should do the pagination as well for Json elements, means if I pass offSet and limit and I should fetch the elements accordingly. In the above example CountObject contains limit and offSet based on that it should fetch the elements. TO give more explanation If I pass offSet as 10 and limit a 10 I should fetch the elements in from 10th element to 20th element and so on.

Answer

Here is an approach using Jackson (I used version 2.11.1).

An “item” here is defined as one of the id/value pairs in the source JSON – for example:

{
  "id": "b61ffb48-ffc7-4ae6-81a2-78b632892fda",
  "value": "[e-mail]"
}

I split the task into 2 parts:

1. Cut off the data when the required limit is reached, by deleting subsequent items.

2. Clean up any resulting empty objects or arrays.

Here is my input test data (based on the data provided in the question):

    private static final String JSON = "{n"
            + " "emails": [{n"
            + "         "emails": [{n"
            + "             "email": {n"
            + "                 "id": "ac9e95cf-3338-4094-b465-e0e1deca23c4",n"
            + "                 "value": "[e-mail]"n"
            + "             }n"
            + "         }]n"
            + "     },n"
            + "     {n"
            + "         "email": {n"
            + "             "id": "b61ffb48-ffc7-4ae6-81a2-78b632892fda",n"
            + "             "value": "[e-mail]"n"
            + "         }n"
            + "     }n"
            + " ],n"
            + " "lastName": {n"
            + "     "id": "ffe19ece-819b-4680-8e0b-8566b34c973d",n"
            + "     "value": "LastName"n"
            + " },n"
            + " "firstName": {n"
            + "     "id": "4ed234f4-f679-40f3-b76b-41d9fdef7390",n"
            + "     "value": "FirstName"n"
            + " }n"
            + "}";

The code:

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import java.util.Iterator;

public class JsonReducer {

    // get the first n "id/value" items:
    private final int limit = 2;
    // tracks how close we are to the cutoff limit:
    private int counter = 0;

    public void doParsing() throws JsonProcessingException {
        ObjectMapper mapper = new ObjectMapper();
        JsonNode json = mapper.readValue(JSON, JsonNode.class);
        
        // show the input JSON formatted, just for this demo:
        System.out.println(json.toPrettyString());

        // a copy of the original - we will use this when cleaning up:
        JsonNode prevJson = json.deepCopy();
        // remove unwanted items from the JSON
        json = reduce(json);

        // clean up empty nodes resulting from removals:
        while (!json.equals(prevJson)) {
            prevJson = json.deepCopy();
            json = stripEmpty(json);
        }

        System.out.println("---------------------------------");
        System.out.println(json.toPrettyString());
    }

    private JsonNode reduce(JsonNode json) {
        for (JsonNode node : json) {
            if (node.isObject()) {
                counter++;
                //System.out.println("obj " + counter + " - " + node.toString());
                if (counter > limit) {
                    ((ObjectNode) node).removeAll();
                } else {
                    reduce(node);
                }
            } else if (node.isArray()) {
                ArrayNode arrayNode = (ArrayNode) node;
                //System.out.println("array - " + arrayNode.toString());
                arrayNode.forEach((item) -> {
                    // assume each item is a JSON object - no arrays of arrays:
                    ObjectNode objectNode = (ObjectNode) item;
                    reduce(objectNode);
                });
            } //else if (node.isTextual()) {
            //System.out.println("text  - " + node.asText());
            //}
        }
        return json;
    }

    private JsonNode stripEmpty(JsonNode json) {
        Iterator<JsonNode> it = json.iterator();
        while (it.hasNext()) {
            JsonNode child = it.next();
            if (child.isContainerNode() && child.isEmpty()) {
                it.remove(); // remove empty arrays [], and objects {}
            } else {
                stripEmpty(child);
            }
        }
        return json;
    }

    private static final String JSON = ... // as shown above.

}

The reduce() method recursively iterates through the JSON, keeping track of the number of items collected – and then deletes any in excess of the required number.

However, this can leave empty [] arrays or {} objects in the JSON, so the stripEmpty() method handles that.

Because we are iterating sequentially through the JSON from top to bottom and from outer to inner, it’s possible that we may need more than one pass of the stripEmpty() method. There may be a more efficient approach, which only needs one pass, but this is approach is at least straightforward.

Examples of the results:

For limit = 2:

{
  "emails" : [ {
    "emails" : [ {
      "email" : {
        "id" : "ac9e95cf-3338-4094-b465-e0e1deca23c4",
        "value" : "[e-mail]"
      }
    } ]
  }, {
    "email" : {
      "id" : "b61ffb48-ffc7-4ae6-81a2-78b632892fda",
      "value" : "[e-mail]"
    }
  } ]
}

For limit = 1:

{
  "emails" : [ {
    "emails" : [ {
      "email" : {
        "id" : "ac9e95cf-3338-4094-b465-e0e1deca23c4",
        "value" : "[e-mail]"
      }
    } ]
  } ]
}

For limit = 0:

{ }

Additional Points:

Not Generic

The approach assumes there are never any arrays nested directly inside other arrays – so none of this: [ [ {...} ] ]. In other words, this is not a 100% generic parser, but does have some limitations in line with the sample data in the question.

Consider using POJOs

This solution does not define any POJO java objects into which the data is loaded – but it can often be easier to get what you want by doing that:

• load (deserialize) the data into one or more POJOs.
• remove unwanted data from the POJOs.
• serialize the remaining data back to JSON.

If the example were any more complicated than the one in the question, I think I would favor doing this instead of manipulating only JsonNode data.

Update

Given the changes to the question, I think the best approach I can suggest is to parse each “item” (see definition above) into a POJO which would simply contain 3 fields:

String attribute;
String id;
String value;

The code to do this is as follows:

    private void traverse(JsonNode json) {
        Iterator<Map.Entry<String, JsonNode>> it = json.fields();
        while (it.hasNext()) {
            Map.Entry<String, JsonNode> entry = it.next();
            String name = entry.getKey();
            JsonNode node = entry.getValue();

            if (node.isArray()) {
                ArrayNode arrayNode = (ArrayNode) node;
                arrayNode.forEach((item) -> {
                    // assume each item is a JSON object - no arrays of arrays:
                    ObjectNode objectNode = (ObjectNode) item;
                    traverse(objectNode);
                });
            } else {
                String id = node.get("id").asText();
                String value = node.get("value").asText();
                
                System.out.println("attr : " + name);
                System.out.println("id   : " + id);
                System.out.println("value: " + value);
                System.out.println("---");
            }
        }
    }

Instead of the println() statements, you would create a new instance of the POJO and add it to an ArrayList.

Now you have a standard list containing all your data – and you can access items 1 – 100, then 101 – 200… and so on, as needed for the user interface.

You would need to convert that raw POJO data back to whatever format the UI needs/expects, of course.

Using the example JSON from the question, the above approach prints this:

attr : email
id   : ac9e95cf-3338-4094-b465-e0e1deca23c4
value: [e-mail]
---
attr : email
id   : b61ffb48-ffc7-4ae6-81a2-78b632892fda
value: [e-mail]
---
attr : lastName
id   : ffe19ece-819b-4680-8e0b-8566b34c973d
value: LastName
---
attr : firstName
id   : 4ed234f4-f679-40f3-b76b-41d9fdef7390
value: FirstName