I have a List of String arrays that I need to write into a CSV file efficiently. I’ve tried it two ways below.
Dataset is a file with at least 1 million records in size. ie. returnedList
has at least 1 million String[]
arrays.
My current code is terribly inefficient and takes far too long using the StringBuilder:
BufferedWriter br = new BufferedWriter(new FileWriter(filePath + "test.csv")); for(String[] listEntry : returnedList) { // convert each array to a string first for(String s : listEntry) { builder.append(s); } String str = builder.toString(); br.write(str); br.newLine(); } br.close(); assertThat(newFirstRow).isNotEmpty();
My second attempt was with using the OpenCsv API:
Writer writer = Files.newBufferedWriter(Paths.get(filePath + "test.csv")); StatefulBeanToCsv<String[]> beanToCsv = new StatefulBeanToCsvBuilder(writer) .withQuotechar(CSVWriter.NO_QUOTE_CHARACTER) .build(); beanToCsv.write(returnedList);
This method failed, where somehow, all the records that were inserted into the file was entirely empty.
Am I using the OpenCsv library wrongly?
Advertisement
Answer
Dont make use of StringBuilder. How BufferedWriter works is it stores data in main memory and flushes it on disk when asked for thus avoiding the clostly network trip. Can you try with below code.
for(String[] listEntry : returnedList) { for(String s : listEntry) { br.append(s); } br.flush(str); br.newLine(); } //If memory constrain allows Flush everything in one go here and close instead of doing flush() in for loop br.close();