java.util.ConcurrentModificationException when combing results from Future’s into HashMap

I’m running into an issue where I intermittently get an java.util.ConcurrentModificationException error whilst merging the results of futures. I’ve tried making all my HashMaps concurrent and I tried using an iterator but the error still persists and I’m totally confused to how this error is occurring. I’m not looping through the same HashMap I’m inserting too either (as done in similar questions with this error).

For my code, to start off with I build a bunch of tasks that return’s HashMap<String, HashSet>, then using .invokeAll my tasks all return the above HashMap which I then try to merge together (I’m reading a large CSV to get all unique results for each column).

I start by defining all the keys.

HashMap<String, HashSet<String>> headersHashset = new HashMap<String, HashSet<String>>();
headers = Arrays.asList(br.readLine().split(delimiter));
for (String headerID : headers) {
    headersHashset.put(headerID, new HashSet<>());
}

I then make my tasks, which clone the keys, do the processing and return it’s result

tasks.add(() -> {
    HashMap<String, HashSet<String>> localHeadersHashset = (HashMap<String, HashSet<String>>) headersHashset.clone();
    
    for (String[] values : sampleSet.values()) {  // sampleSet is a SortedMap<Integer, String[]>
        int headerAsINT = 0;
        for (String value : values) {
            localHeadersHashset.get(headers.get(headerAsINT)).add(value);
            headerAsINT++;
        }
    }
        
    return localHeadersHashset;
});

I invokeAll the tasks, where the results are put into a List of Future’s

ExecutorService es = Executors.newCachedThreadPool();
List<Future<HashMap<String, HashSet<String>>>> futures = null;
try {
    futures = es.invokeAll(tasks);
    es.shutdown();
} catch (InterruptedException e) {
    e.printStackTrace();
}

I then, for very safe keeping, make a ConruentHashMap with a copy of all the keys pre-defined. I then loop through all the results from my tasks, put them into a ConcurrentHashMap as well, and addAll the results into the original copied ConcurrentHashMap with the pre-defined keys. This is when the error occurs (only sometimes though, which backs that it’s something to do with threads, but I thought all the threading stuff is done at this point?).

ConcurrentHashMap<String, HashSet<String>> headersHashsetConcurrent = new ConcurrentHashMap<>(headersHashset);

try {
    for (Future<HashMap<String, HashSet<String>>> f : futures) {
        ConcurrentHashMap<String, HashSet<String>> threadSafeItems = new ConcurrentHashMap<>(f.get());
        for (Map.Entry<String, HashSet<String>> items : threadSafeItems.entrySet()) {
            headersHashsetConcurrent.get(items.getKey()).addAll(items.getValue());  // ERROR SHOWS HERE 
        }
    }
} catch (Exception e) {
    e.printStackTrace();
}

Here is the error in full:

java.util.ConcurrentModificationException
    at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1584)
    at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1607)
    at java.base/java.util.AbstractCollection.addAll(AbstractCollection.java:335)
    at project/project.parseCSV.processFile(parseCSV.java:101)
    at project/project.parseCSV.call(parseCSV.java:126)
    at project/project.parseCSV.call(parseCSV.java:11)
    at javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Task.java:1425)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.lang.Thread.run(Thread.java:832)

I’m really not sure why this is causing an error, as addAll is only run on the main thread, and it’s inserting into a ConCurrent HashMap too anyway – any ideas on why this is happening would be hugely appreciated!

Answer

The problem is that headersHashset.clone() doesn’t clone HashSets in the values.

From the docs:

Returns a shallow copy of this HashMap instance: the keys and values themselves are not cloned.

It means that localHeadersHashset in your tasks, and your headersHashsetConcurrent, and threadSafeItems returned by futures — all of these use the same HashSet objects for the same keys.

Since tasks are executed in parallel threads, it is entirely possible that some task executes HashSet.add() at the same time when the main thread iterates over the elements of the same HashSet inside:

            headersHashsetConcurrent.get(items.getKey()).addAll(items.getValue());

This is what causes your ConcurrentModificationException.

Advertisement

Answer