I don't understand what I'm doing wrong. How to send both commands and files. Below are the server and client pipelines. client pipeline: server pipeline: The fact is that before I send the file, I send a command (FileInfo) that the file described in this FileInfo will be sent soon. Then the file itself is sent (byte Buf), Like FileInfo,

How to send both commands and files using Netty

I don’t understand what I’m doing wrong. How to send both commands and files. Below are the server and client pipelines.

client pipeline:

 pipeline.addLast(new ObjectDecoder(ClassResolvers.cacheDisabled(null)));    //in
            pipeline.addLast(new ObjectEncoder());                           //out
            pipeline.addLast(new ChunkedWriteHandler());                     //out
            pipeline.addLast(new FileSenderHandler());                       //in (write in responce)
            pipeline.addLast(new FileInfoSenderHandler());                   //out

server pipeline:

        pipeline.addLast(new ObjectEncoder());                                   //out
        pipeline.addLast(new ObjectDecoder(ClassResolvers.cacheDisabled(null))); //in
        pipeline.addLast(new FileReceiverHandler());                             //in
        pipeline.addLast(new FileInfoReceiverHandler());                         //in (write in responce)

The fact is that before I send the file, I send a command (FileInfo) that the file described in this FileInfo will be sent soon. Then the file itself is sent (byte Buf), Like FileInfo, ByteBuf also passes through to ObjectDecoder. At the same time , Java swears on the first chunk:

2021-10-09 01:26:02 WARN  DefaultChannelPipeline:1152 - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 1048576: 4292411364 - discarded
    at io.netty.handler.codec.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFrameDecoder.java:503)
    at io.netty.handler.codec.LengthFieldBasedFrameDecoder.failIfNecessary(LengthFieldBasedFrameDecoder.java:489)
    at io.netty.handler.codec.LengthFieldBasedFrameDecoder.exceededFrameLength(LengthFieldBasedFrameDecoder.java:376)
    at io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:419)
    at io.netty.handler.codec.serialization.ObjectDecoder.decode(ObjectDecoder.java:69)
    at io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:332)
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)

If you override decode() and add logs, then you can see that the file transfer does not end after that and each piece passes through the ObjectDecoder and is “decoded to null”. There is no such exception on the remaining chunks.

What have I tried:

In the ObjectDecoder constructor, set the number maxObjectSize = MAX.INTEGER, but it is still small.

In the decode() set the condition:

 @Override
 protected Object decode(ChannelHandlerContext ctx, ByteBuf in) throws Exception {
     System.out.println("decode()");
     Object o = super.decode(ctx, in);
     System.out.println(o);

     if(o == null){
         System.out.println(in);
         return in;
     }
     return o;
 }

In this case, as a result, 2048 bytes are missing in the file.

3)If I dynamically change the pipeline in the code (delete/add an ObjectDecoder depending on the state), then the files are accepted by the server. However, I am not sure about the correctness of this decision.

Maybe I’m doing something fundamentally wrong? I have read almost all the documentation, and I have been solving this problem for the second week. What to read? Please help me. Thanks!

Answer

It seems like you have a custom protocol that consists of metadata about the file and then actual file data. Since both metadata handler (ObjectEncoder/Decoder) and file handlers work on buffers, it seems there is no way to distinguish which message is metadata and which is data. If there is no indication as such on the wire and you know that the first message is always metadata then starting the pipeline with the metadata handler and then swapping it later with the file handlers seem like the best option for you.

Advertisement

Answer