My packets are too big, how do I solve this?

[crayonc...@gmail.com]

Hello,

I believe I discovered a 500 byte bottle neck between Adobe Air and my Netty server.
I tried splitting my packets up into smaller amounts, but sending them rapidly after each other
has the same effect as sending it all at once.

I can do a work around of waiting for .5 seconds between packets, but I don't think this is the proper way.

What is the proper way to read a long packet by a single user when many users are sending packets?
I can id my user, but sending packets rapidly means it is the same as me trying one giant packet.

,Jim

[Tim Boudreau]

There's not really enough info here to answer this well.  Can you post some code?

There are a lot of things that could be going on, but without knowing what your code is doing - particularly how you're splitting things up, what the network looks like in between the bunch of devices, whether it's TCP or UDP or SCTP, stuff like that - I don't think anybody could hazard a guess.

-Tim

[Zhu Ran]

Agree with Time boudreau, however, you can take a look at my earlie answer to 

the topic "Unable to get long String message complete" in this group for a general consideration.

[Tim Boudreau]

Agreed.  Some places to start, since you haven't replied with more detail:

 - If TCP, there shouldn't be anything that's "too big" since the network layer at the OS level application will take care of splitting data into packets.  If you're doing TCP, then something else is wrong.  Here's one that can happen that's not obvious to people new to asynchronous I/O:

    - If you are flushing some bytes to the socket, are you waiting until you get an event (in a ChannelFutureListener) before sending more, or are you just slamming them out there in a for-loop?  If the latter, it's entirely possible that you are sending the data, but it's being flushed to the stream out-of-order, so it looks like garbage, and something on the other end is discarding it silently.  Introducing a 0.5 second delay *would* look like it fixed the problem.

 - If it's UDP, maximum packet size depends on the network between you and the other end, and packets that are too large go into a black hole.  But if introducing a delay solves the problem, then this is not your problem.

 - If it's SCTP, messages larger than a threshold side get split up and must be reassembled on the other end.

At any rate, I think your problem is not packet *size* per se, since introducing a delay fixes the problem.  And if it's a TCP based protocol, you're likely misusing the word "packet" - when you flush a buffer, that might go out as one packet, or more, depending on the size of the data (it also might theoretically get split up into more or combined into fewer packets by any router between you and the endpoint).  A buffer is not a packet.

tcpflow is a handy unix utility if you want to just copy all outbound data from your computer, the way it looks on the wire, to files on disk and then examine them - that might help diagnose things.  Or wireshark if you prefer GUI tools.  Take a look at what's really going over the wire.  Your problem is almost surely not data size.

-Tim

[goodnews...@gmail.com]

Thank you guys for caring. This is my other gmail.

I think Tim is right. Though I reinvented the wheel and sent smaller packets, it only worked some of the time.

I'm using Adobe Air to send data as a client, and Netty Java as my server.

I send a string of information via Adobe Air, and Netty doesn't read the whole string.

The string is like 544 characters long, and what gets cut off is like 50-65 characters.

My adobe air looks like:
socket.writeObject(data)
socket.flush();

My Netty looks like:
Public void channelRead(ChannelHandlerContext ctz, Object msg){
ByteBuf in = (ByteBuf) msg;
Try{
String s1;
s1=in.toString(io.netty.util.CharsetUtil.US_ASCII);
in.release
System.out.println(s1); // Doesn't always print out all the characters sent via Adobe Air, seems like only errors out when
// hundreds of characters are sent, as I don't get errors on small packets. I get the first couple hundred and it cuts off
// the last 1-65 ish most of the time.

//Thank you for helping me. I should have an alpha video game out in two months or so if I can figure thisnone out

}
finally
{}

[Tim Boudreau]

What is the protocol, http?

If so, are you dealing with chunked encoding? If not, there is a class in codec.http for that that you can just put in the pipeline, and channelRead will be deferred until all the data is there. Otherwise, you are just reading the first chunk and thinking you're done (you can check the http headers you're getting to see if that's the case).

-Tim 

--
You received this message because you are subscribed to a topic in the Google Groups "Netty discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/netty/pekavYE2zbE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to netty+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netty/80886336-2182-4aa1-8c4b-9b9609fde0d2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

http://timboudreau.com

[goodnews...@gmail.com]

Tim,

I appreciate the quick response. I'm really into making this game and put hours in each day. Its almost finished. Help on this last brick wall is appreciated a bunch.

How can I tell which protocol I'm using.

I think it is http because I can't use html characters common like <>%'`&

I am using tcpip, I know that.

,Jim

[Tim Boudreau]

If you are using classes from the http codec, you are.  The fact that you don't know suggests you're not. What does the traffic on the wire look like?

-Tim 

--
You received this message because you are subscribed to a topic in the Google Groups "Netty discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/netty/pekavYE2zbE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to netty+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/netty/19428687-29e3-42c3-84c5-3c75b5bc2118%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

http://timboudreau.com

[goodnews...@gmail.com]

Here is my server:

EventLoopGroup bossGroup = new NioEventLoopGroup(1); // (1)
        //EventLoopGroup.shutdownGracefully();
        EventLoopGroup workerGroup = new NioEventLoopGroup(2);
        try {
            ServerBootstrap b = new ServerBootstrap(); // (2)
            b.group(bossGroup, workerGroup)
             .channel(NioServerSocketChannel.class) // (3)
             .childHandler(new ChannelInitializer<SocketChannel>() { // (4)
                 @Override
                
                 public void initChannel(SocketChannel ch) throws Exception {
                     dshstore.add(new DiscardServerHandler(t1));
     
                     ch.pipeline().addLast(dshstore.get(maxdshstore));
                     
                     maxdshstore++;
                     System.out.println("MAXDSHSTORE"+Integer.toString(maxdshstore));
                   shstore=ch;
                         
                 }
             })
             .option(ChannelOption.SO_BACKLOG, 128)          // (5)
             .childOption(ChannelOption.SO_KEEPALIVE, true); // (6)

            ChannelFuture f = b.bind(port).sync(); // (7)


           
           // f.channel().closeFuture().sync();
            wildloop();
        } finally {
            workerGroup.shutdownGracefully();
            bossGroup.shutdownGracefully();
           
     }

I would send a packet that looks like this:
00567BMHOZ.3d\J-eU|.0tXaA{.1v-f.6RD1H.2VPKV~]F1z.6Wv]Q.1sPU_o{U(_jP.0{\bCkoQRxa+JN.1c.0e96WD|~W.2J7[=k@x_Ds:CMuS.08P=NNF*(iURy1I;N|2jp.3]9G.5fS9Yx~G.3sE.0.41nbvAIA.4}Xoi=b6.6~.6CCVS2Lno[P_L*9Nsl|MzR.2xH;B...@H.2mdU~R8PB.5O[=YaU.3.3UM.1].3.0EKb|n8(p$sc[{$Gb$aK?.1b-ei$A6.5,@Q\Tu/I*88.51G,QEiRXthFbR@jC\GRw,YM5ng7m0Il.0HB+IiF.1USR(pKhi.2$qw=ECygrMPg8pX^JY/QF0[6.3.2\@@u.5j]No.35)gwhz(:?.2+|=\[1;K/V.1Zl^}Esci{2GRkae$y9;c;pHGBEs}(AHoxl.4CjYi[Dmnscf_.2p.2)RQF7H7.5829BLG1b9+//^BW+Vs$IX].0zY;V2*ad6qjnHdXn|aLwei)z/b|.1?$oB.6z,4p6-trhSYa/j/5r.6.39X1EK~ZOmI?701LBY,.5uHN_{9.0$/n2L)


I would receive a packet that looks like this:
00567BMHOZ.3d\J-eU|.0tXaA{.1v-f.6RD1H.2VPKV~]F1z.6Wv]Q.1sPU_o{U(_jP.0{\bCkoQRxa+JN.1c.0e96WD|~W.2J7[=k@x_Ds:CMuS.08P=NNF*(iURy1I;N|2jp.3]9G.5fS9Yx~G.3sE.0.41nbvAIA.4}Xoi=b6.6~.6CCVS2Lno[P_L*9Nsl|MzR.2xH;B...@H.2mdU~R8PB.5O[=YaU.3.3UM.1].3.0EKb|n8(p$sc[{$Gb$aK?.1b-ei$A6.5,@Q\Tu/I*88.51G,QEiRXthFbR@jC\GRw,YM5ng7m0Il.0HB+IiF.1USR(pKhi.2$qw=ECygrMPg8pX^JY/QF0[6.3.2\@@u.5j]No.35)gwhz(:?.2+|=\[1;K/V.1Zl^}Esci{2GRkae$y9;c;pHGBEs}(AHoxl.4CjYi[Dmnscf_.2p.2)RQF7H7.5829BLG1b9+//^BW+Vs$IX].0zY;V2*ad6qjnHdXn|aLwei)z


Notice how the received packet doesn't have:
/b|.1?$oB.6z,4p6-trhSYa/j/5r.6.39X1EK~ZOmI?701LBY,.5uHN_{9.0$/n2L)

[goodnews...@gmail.com]

Tim,

If you have more questions, let me know. I don't think this is one of those problems I can solve on my own. This seems like a brick wall to me. So your help is super appreciative. I'll post a link to alpha testing for you guys that helped once everything is working. Thanks again.

,Jim

[goodnews...@gmail.com]

I found a huge clue to this.

Not all of the packet gets read at once via

channelRead(ChannelHandlerContext ctx, Object msg)

{
ByteBuf in = (ByteBuf) msg;

try
{
String s1;
S1=in.toString(io.netty.util.CharsetUtil.US_ASCII);
}

I now can read a whole packet.
The problem is that I identify users by cookies in the packet.
So I'm not sure how to figure out how to construct a long string from two different packets(? Sections of message)

I might have to rely on IPaddress to id a user constructing a long string, but there might be errors with two players behind a router, right?

[goodnews...@gmail.com]

The solution I will be using is to concatonate the strings. I just didn't realize the data got sent in more than one chunk to channelRead.

Thanks everyone for the help. I should be good to go.

If anyone wants to play my game in alpha in a few months, I'll make a post.

[Zhu Ran]

On Tuesday, April 11, 2017 at 12:30:40 PM UTC+8, James Sager wrote:

Tim,

I appreciate the quick response.  I'm really into making this game and put hours in each day.  Its almost finished.  Help on this last brick wall is appreciated a bunch.

How can I tell which protocol I'm using.

Protocol, in Tim's context, is meaning that how do you know how many bytes do you need when you read from channels, I guess.

You should know your protocol first, and then you can know what to do, to choose a built-in decoder or you implement one.

 

I think you can look at ByteToMessageDecoder and some of its "children" in Netty.

If one child meets your need, use it. If none, you can extend ByteToMessageDecoder to do your own stuff.