While loop doesn’t exit after file download

I’ve got the following code to download a file being transmitted over TCP:

        try (OutputStream out = new FileOutputStream(path); InputStream is = socket.getInputStream();) {
            byte[] bytes = new byte[1024];
            int count, xp = 0;
            while ((count = is.read(bytes)) > 0) {      // TODO after upload the service doesn't leave while loop
                out.write(bytes, 0, count);
            }
            System.out.println("hello");
         ...

upload code:

if (ready.equalsIgnoreCase(CdnResponse.READY.getContext())){
    int read = 0;
    byte[] bytes = new byte[1024];
    while ((read = inputStream.read(bytes)) != -1) {
        out.write(bytes, 0, read);
    }

}

The upload exits its loop fine.

Once all the bytes have been processed (they are always processed successfully, however the loop is never exited), the file is created, without a single issue, but, the loop doesn’t exit.

Answer

TCP/IP connections are designed to be long-lived streaming connections (built on top of the out-of-order, no-guarantee, packet-based IP protocol).

That means that is.read(bytes) does exactly what the spec says it will: It will wait until at least 1 byte is available, OR the ‘end of stream’ signal comes in. As long as neither occurs (no bytes arrive, but the stream isn’t closed), it will dutifully block. Forever if it has to.

The solution is to either [A] pre-send the size of the file, and then adjust the loop to just exit once you’ve received that amount of bytes, or [B] to close the stream.

To close the stream, close the socket. It kinda sounds like you don’t wanna do that (that you are multiplexing multiple things over the stream, i.e. that after transfering a file, you may then send other commands).

So, option A, that sounds better. However, option A has as a prerequisite that you know how many bytes are going to come out of inputStream. If it’s a file, that’s easy, just ask for its size. If it’s streamed data, that would require that, on the ‘upload code side’, you first stream the whole thing into a file and only then stream it over the network which is unwieldy and potentially inefficient.

If you DO know the size, it would look something like (and I’m going to use newer APIs here, you’re using some obsolete, 20 year old outdated stuff):

// upload side
void upload() {
  Path p = Paths.get("/path/to/file/you/want/to/send");
  long fileSize = Files.size(p);
  out.write(longToBytes(fileSize);
  try (InputStream in = Files.newInputStream(p)) {
    in.transferTo(out);
  }
}

public static byte[] longToBytes(long x) {
    ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
    buffer.putLong(x);
    return buffer.array();
}

This code has the following properties:

• First it sends 8 bytes, in big endian order, which is the size of the data to come.
• It uses the new java.nio.file API.
• It uses the new transferTo method in InputStream, which avoids the rigamarole of having to declare a byte array to serve as buffer, and a while loop.

Then on the download side:

void download() {
  long size = bytesToLong(in.readNBytes(8));
  Path p = Paths.get("/file/to/store/data/at");
  // Generally network transfer sizes are a bit higher than 1k;
  // up to 64k in fact. Best to declare a larger buffer!
  byte[] bb = new byte[65536];
  try (OutputStream out = Files.newOutputStream(p)) {
    while (size > 0) {
      int count = in.read(bb);
      if (count == -1) throw new IOException("Unexpected end of stream");
      out.write(bb, 0, count);
      size -= count;
    }
  }
}

public static long bytesToLong(byte[] bytes) {
    ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
    buffer.put(bytes);
    buffer.flip();//need flip 
    return buffer.getLong();
}

This code:

• Uses the new readNBytes method to read that size in, first.

If you do not know how large the incoming data is, you need to write a little protocol. For example:

• A size is sent, in the form of 2 bytes, in big endian order, unsigned. Then that many bytes follow, then another size is sent, ad infinitum.
• When the stream is completed, a size of 0 is sent (so, 2 bytes with value 0), which indicates the file is complete.

I’ll leave it as an exercise for you to implement the up- and download side if this is what you need.