Echo server example with the next generation API

["이희승 (Trustin Lee)"]

Hi,

I have been spending most of my time for revamping Netty API for version
4. The new API focuses on the following issues:

* Too many event objects are created and GC'd
* Netty does more memory copies where it should not.
(especially when JNI is used.)
* The terms like 'upstream' and 'downstream' are confusing.
* Confusing thread model

Currently, it's at pre-alpha stage and I did not finish retrofitting all
legacy components with the new API, but today I finally succeeded to run
the echo server example. :-)

http://j.mp/J0pf9E

The example does not make use of ServerBootstrap, so you can find how
Netty accepts a new connection in the new API.

You might find the changes are so dramatic that it looks like a
different framework, the basic idea is simple:

* No event object anymore - it's just a method invocation
* A user handler creates a buffer and accesses it directly
* Protocol decoders can do the same job with less copies
* With upcoming buffer pooling support, using direct buffers
becomes more feasible.
* You can create a bounded buffer to throttle traffic.
If the inbound buffer is full, Netty won't read until asked.
* Both inbound and outbound events are handled by event loop
* A user can add and remove a channel from event loop
whenever he/she wants.

Please feel free to give some feed back, share ideas, or ask any questions.

Cheers,
T
--
https://twitter.com/trustin
https://twitter.com/trustin_ko
https://twitter.com/netty_project

[Trustin Lee]

I must correct one thing - outbound events are still handled by a caller thread.  However, if your handler is so complex that it's likely to cause a dead lock, you can always call EventLoop.execute(Runnable) so that what you want to perform is always done in the event loop (i.e. I/O thread).  I'm gonna make some handlers like SslHandler take advantage of this feature to simplify them dramatically.

["이희승 (Trustin Lee)"]

Another important change to note is that all EventLoops must implement
ScheduledExecutorService. That is, you don't need to create a separate
timer such as HashedWheelTimer to schedule something. You can just use
the EventLoop your channel is registered to.

Currently, it's not a hashed wheel timer but traditional
DelayQueue-based one, but it should be probably OK for most cases.

[Andrei Pozolotin]

I can confirm, from my experience with netty, that this is true:

* Too many event objects are created and GC'd 
* Netty does more memory copies where it should not. 

* The terms like 'upstream' and 'downstream' are confusing. 
* Confusing thread model 

additionally:

* logic around selectors should be simplified

* thread pool / worker pool life cycle should be simplified

[Vibul Imtarnasan]

Hi Trustin,

What's the difference between the parentEventLoop and childEventLoop?  Is the equivalent to the old boss and worker threads?

Thanks

Vibul

[Netty project]

Correct.

--
Sent from a mobile device.

[Netty project]

Yes.  Worker and WorkerPool have been replaced with EventLoop to simplify them significantly.

--
Sent from a mobile device.

[komp...@googlemail.com]

Hi,

I've been working with Netty for half a year now. It is really a great framework. So I am very interested in the development of future versions.

I have studied the echo-sample and I am a bit confused right now. The concepts are clear, but the resulting code does not seem very nice to me. To return just the data the server received the EchoServerHandler now takes 6 lines of code. The old API did the same in 2 lines. The explicit call of a flush method feels out-dated to me. Also the method is triggered because a new message or inbound-buffer arrived, the new data is not given as a method parameter, but must be called from the context.

I hope I don't sound old-fashioned. I am really looking forward to the simplified workflows and processes. But just the echo-sample does not look like an improvement to me. May you give some explanations to that?

Cheers,
Martin

["이희승 (Trustin Lee)"]

Hi Martin,

I like the idea of providing the inbound buffer as a parameter. Let me
update the API.

Instead of writeBytes() + flush(), you can just call write(). The
example demonstrates you can buffer the output instead of flushing it
immediately. So, you have one more option. :-)

discardReadBytes() is a tricky part. Unlike Netty 3, which created a
new buffer for each read and write, Netty 4 keeps the buffers for reads
and writes and reuses them to reduce GC overhead. Because of this
change, the buffer you access is not always in a clean state, and thus
you sometimes have to call it to ensure the readerIndex of the buffer
does not grow up too large. Netty can call discardReadBytes() whenever
it can, but it often involves memory copy. The author of the handler
knows best when to call it, so Netty calls it only when it's sure it
does not involve memory copy.

In the echo example, Netty can be more smart and there's no need for a
user to call in.discardReadBytes() because it's always consumed
completely and then Netty can just clear the buffer away.

However, it is still necessary to call out.discardReadBytes() because
there's no guarantee that the outbound buffer will be consumed
completely by the remote peer. For example, a
fast-writing-slow-reading client will never allow the outbound buffer
become empty, so discardReadBytes() will always involve memory copy.
Netty does not want to waste memory bandwidth by calling
discardReadBytes() in this case, and the user should have control on
this.

So, the resulting code would look like the following, saving 2 lines:

void inboundBufferUpdated(ctx, in) {
ChannelBuffer out = ctx.nextOutboundByteBuffer();
out.discardReadBytes();
out.writeBytes(in);
ctx.flush();
}

If you don't care about doing an extra memory copy, you can do this:

void inboundBufferUpdated(ctx, in) {
ctx.write(in.readBytes(in.readableBytes()));
}

It is true that you can achieve the same without an extra memory copy
in Netty 3, but this is only the case in the echo example. In other
cases, you don't just reply a carbon copy. :-)

What do you think? Did you find something that could be simplified?
I'd love to listen!

Cheers,
T

> https://twitter.com/netty_project <https://twitter.com/netty_project>

[Martin]

Hi Trustin,

I have to admit I don't fully understand the need to call out.discardReadBytes. What I think I get is that you want to achieve the ultimate performance in I/O. But from my point of view as a simple user of the framework it seems like developing with netty will get more complicated with this. On the one hand you say you want to straighten concepts and make things easier to understand, but on the other hand this performance optimization code is the opposite of this. You may then have a nice API but the code I have to write is more complex and cares more about buffer-management then business logic. I see your last example for - when you don't care about copies -. That is really the code I want to write. But after the preamble of possible not clean buffer states and stuff I am unsettled if this is stable code.

And again I am afraid to sound lazy or something but that's my point of view. I want to have the performance but not the management-code in my methods. I know it's easy to criticize and I wish I could suggest some other solutions to that but I am not that into buffer management. As I wrote I am just a simple user of Netty.

Cheers,
Martin

[Vibul Imtarnasan]

Hi Martin,

I think in the new API, there are 2 types of writes are supported:

1. Buffered

 void inboundBufferUpdated(ctx, in) { 
    ChannelBuffer out = ctx.nextOutboundByteBuffer(); 
    out.discardReadBytes(); 
    out.writeBytes(in); 
    ctx.flush(); 
  }  

Note that "out" is the buffer which has to be flushed.

2. Not Buffered

  void inboundBufferUpdated(ctx, in) { 
    ctx.write(in.readBytes(in.readableBytes())); 
  }  

Is this correct Trustin?  If so, could the API be modified to 

  void inboundBufferUpdated(ctx, in) { 
    ctx.write(in); 
  }   

or  

  void inboundBufferUpdated(ctx, in) { 
    ctx.bufferredWrite(in); 
  }   

Vibul

["이희승 (Trustin Lee)"]

Hi Vibul,

Currently, the write operation is a shortcut of 1) adding the data to
the buffer, and 2) issuing a flush request. So, the following code
could work:

ctx.write(in);

.. only if write() moved in's readerIndex. Currently, it just does
not. I'm just unsure what the default behavior should be. Because the
content of 'in' is copied into the buffer, I feel no need for modifying
the state of 'in', but it really depends on how we define the contract
of the write() method. Probably, your idea is better because the
writeBytes() methods without index parameters updates the indexes of
the source buffer.

Therefore, I'm gonna add the following two methods:

- ctx.write(ChannelBuffer)
- ctx.write(ChannelBuffer, int index, int length);

the first one will move the readerIndex of the source buffer and the
second will not. Then, yeah, our example will become a little bit
shorter. However, this is not really the point.

In most applications, you do not work with ChannelBuffers in your
business logic handler. You work with codecs and message-oriented
handlers. With codec framework, all these details should not be
noticeable.

If you chose to work directly with ChannelBuffers in your biz logic,
then you chose to have as much control as possible, so I see no problem
making the resulting user code a little bit more verbose and take full
advantage of it.

T

On Fri 01 Jun 2012 06:18:21 PM PDT, Vibul Imtarnasan wrote:
>
> Hi Martin,
>
> I think in the new API, there are 2 types of writes are supported:
>

> *1. Buffered*

>
> void inboundBufferUpdated(ctx, in) {
> ChannelBuffer out = ctx.nextOutboundByteBuffer();
> out.discardReadBytes();
> out.writeBytes(in);
> ctx.flush();
> }
>
> Note that "out" is the buffer which has to be flushed.
>
>

> *2. Not Buffered*

[Vibul Imtarnasan]

Agreed Trustin.

I guess it is a matter of use case as to the method that you would use.

Just in the "buffered" use case, should:

    ChannelBuffer out = ctx.nextOutboundByteBuffer();
    out.discardReadBytes();

be combined to

    ChannelBuffer out = ctx.nextOutboundByteBufferForWriting();

or similar? In this way, there is clarity that the buffer will be used for writing.

Regards

Vibul

[fuyou]

 when I open the url,but response 404 code 

2012/6/14 <abrahamm...@gmail.com>

Hi,

Without knowing the full internal details of Netty(hence my q could be stupid) let me ask the following q's
1)  "A user can add and remove a channel from event loop whenever he/she wants" -> If we have multicore machines is there a strategy to create event loops themselves? Meaning on a 32 core machine would there be 32 event loops? If so, how do you assign new socket channels to event loops for e.g. round robin? Also are there options to set thread affinity etc.
2) Any plan to go down the disruptor way of IPC? They seem to have the fastest numbers published.

Regarding API, I have following suggestion. There should be a simple startServer(int port, ChannelInitializer factory) added.
It will be of no use to advanced users. However anybody beginning with Netty will be overwhelmed at the number of options(low level transport stuff like tcp no delay, backlog etc) needed to provide to just start a server.

Thanks,
Abraham Menacherry.

--
   =============================================

                                                                             fuyou001

          此致
敬礼！

[Netty project]

Try this:

https://github.com/netty/netty/blob/master/example/src/main/java/io/netty/example/echo/EchoServer.java

--
Sent from a mobile device.