writing to OutputStream having capacity restriction

Following the question I asked before: I am implementing an ByteArrayOutputStream having capacity restriction. My main limitation is an amount of available memory. So having such stream os:

1. When I write more than say 1MB to the output stream I need to “stop”. I prefer not throw exception but write the complete contents of os output stream to the specified other output stream argument. OutputStream out; os.writeTo(out); And after that continue the writings to os from its beginning

2. In order to prevent the situation described at 1. , I prefer to drain os, as freuqntely as possible. I mean copy the data from it to out in chuncks of 512KB Is it feasible? If yes any advices how can it be done? Or may be there is a built in class which answers my requirements

Edit: The amount of bytes written to out is also limited. I can write there up to 1GB. If I have more, I need to create other output stream in order to drain from os there. The proccess of writing to os. can be like that. 500MB was written there – I transfer it immidiately to out. After several seconds 700MB were written there – I need to drain only 500MB to out and other 200MB to other outputstream(out2), which I`ll need to create upon such situation

Answer

What you are describing is a BufferedOutputStream, which you can construct like that :

new BufferedOutputStream(out, 512000)

The first arg is the other outputstream you have and the second one is the size of the BufferedOutputStream internal buffer

EDIT:

ok, i did not fully understand your need at first. You will indeed need to extend OutputStream to achieve that. Here is a sample code :

Here is how to use the below code :

    public static void main(String[] args) throws IOException {
        AtomicLong idx = new AtomicLong(0);
        try (
            OutputStream out = new OutputStreamMultiVolume(10, () -> new FileOutputStream(getNextFilename(idx)));
            ) {

            out.write("01234567890123456789012345678901234567890123456789".getBytes("UTF-8"));
        }
    }

    private static File getNextFilename(AtomicLong idx) {
        return new File("sample.file." + idx.incrementAndGet() + ".txt");
    }

The first constructor arg of OutputStreamMultiVolume is the max size of a volume. If we reach this size, we will close the current outputStream, and call the OutputStreamSupplier to get the next one.

The example code here will write the String 01234567890123456789012345678901234567890123456789 (5 times 0123456789) to files named ‘sample.file.idx.txt’ where idx is increased each time we reach the outstream max size (so you’ll get 5 files).

and the class intself :

public class OutputStreamMultiVolume extends OutputStream {

    private final long maxBytePerVolume;
    private long bytesInCurrentVolume = 0;
    private OutputStream out;
    private OutputStreamSupplier outputStreamSupplier;

    static interface OutputStreamSupplier {
        OutputStream get() throws IOException;
    }

    public OutputStreamMultiVolume(long maxBytePerOutput, OutputStreamSupplier outputStreamSupplier) throws IOException {
        this.outputStreamSupplier = outputStreamSupplier;
        this.maxBytePerVolume = maxBytePerOutput;
        this.out = outputStreamSupplier.get();
    }

    @Override
    public synchronized void write(byte[] bytes) throws IOException {
        final int remainingBytesInVol = (int) (maxBytePerVolume - bytesInCurrentVolume);
        if (remainingBytesInVol >= bytes.length) {
            out.write(bytes);
            bytesInCurrentVolume += bytes.length;
            return;
        }

        out.write(bytes, 0, remainingBytesInVol);
        switchOutput();

        this.write(bytes, remainingBytesInVol, bytes.length - remainingBytesInVol);
    }

    @Override
    public synchronized void write(int b) throws IOException {
        if (bytesInCurrentVolume + 1 <= maxBytePerVolume) {
            out.write(b);
            bytesInCurrentVolume += 1;
            return;
        }

        switchOutput();
        out.write(b);
        bytesInCurrentVolume += 1;
    }

    @Override
    public synchronized void write(byte[] b, int off, int len) throws IOException {
        final int remainingBytesInVol = (int) (maxBytePerVolume - bytesInCurrentVolume);
        if (remainingBytesInVol >= len) {
            out.write(b, off, len);
            bytesInCurrentVolume += len;
            return;
        }

        out.write(b, off, remainingBytesInVol);
        switchOutput();
        this.write(b, off + remainingBytesInVol, len - remainingBytesInVol);
        bytesInCurrentVolume += len - remainingBytesInVol;
    }

    private void switchOutput() throws IOException {
        out.flush();
        out.close();

        out = outputStreamSupplier.get();
        bytesInCurrentVolume = 0;
    }

    @Override
    public synchronized void close() throws IOException {
        out.close();
    }

    @Override
    public synchronized void flush() throws IOException {
        out.flush();
    }
}