[lwip-users] Delayed Ack causing problems? Where to call tcp_nagle_disable()?
FreeRTOS Info
lwIP V1.4.0.
I have moved the demo FreeRTOS example from the Windoze simulator onto
real hardware in order to check the timing behaviour (which can't really
be done in the simulator), and found a timing problem.
Using an umodified raw http server from the contrib distribution, I am
attempting to retrieve a large jpg file (approx 37K). The file is
retrieved in tact, but very slowly. I am using a point to point connection.
Taking a Wireshark log I see that data is only being sent by lwIP after
an ACK from Windows. I have attached the log for good measure.
192.168.0.200 is the lwIP end.
I have considered three things to fix this. First I have set
TCP_SND_QUEUELEN to (8 * TCP_SND_BUF/TCP_MSS), but that made no
difference. Second I considered the calls to tcp_output() after
tcp_write() to make sure data is actually going out immediately, but
don't think that is the issue as I'm using an unmodified HTTP server
from the contrib distribution.
Finally, I want to try disabling the nagle algorithm, and see this can
be done by calling tcp_nagle_disable(), but I am unsure where and how to
do this? I don't want to modify the core lwIP code, so assume it should
be in the http server code. It is using the raw API version, so it is
not obvious how or where to do to call the macro correctly.
I would be grateful for advise or an example. Thanks.
Regards,
Richard.
+ http://www.FreeRTOS.org
Designed for Microcontrollers.
More than 7000 downloads per month.
FreeRTOS Info
and everything is fine now.
/* Tell TCP that this is the structure we wish to be passed for our
callbacks. */
tcp_nagle_disable(pcb);//<<<<<<<<<<<<<<<<Added line
tcp_arg(pcb, hs);
/* Set up the various callback functions */
tcp_recv(pcb, http_recv);
tcp_err(pcb, http_err);
tcp_poll(pcb, http_poll, HTTPD_POLL_INTERVAL);
tcp_sent(pcb, http_sent);
Regards,
Richard.
+ http://www.FreeRTOS.org
Designed for Microcontrollers.
More than 7000 downloads per month.
> _______________________________________________
> lwip-users mailing list
> lwip-...@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users
_______________________________________________
lwip-users mailing list
lwip-...@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
Felipe de Andrade Neves L.
gold...@gmx.de
but if you want to do so before sending the first segment, any place of
these is equally good:
- at initialization time (as you did)
- in the accept callback (for passive connections)
- in the recv-callback before sending (for passive connections)
- in the connected-callback (for active connections)
Simon
gold...@gmx.de
> May I ask something, how came the Nagle's algorithm has caused the
> problem, as it is suppose to group small data until receive the next
> ack, but the JPG has lots of pending data to be sent and the algorithm
> shouldn't retain it.
Richard, can you tell us anything about this? The raw-httpd from contrib
shouldn't have problems sending a JPG with nagle enabled (unless limited
by available memory or config options)???
Simon
FreeRTOS Info
>> May I ask something, how came the Nagle's algorithm has caused the
>> problem, as it is suppose to group small data until receive the
>> next ack, but the JPG has lots of pending data to be sent and the
>> algorithm shouldn't retain it.
> That's a good, question.
>
> Richard, can you tell us anything about this? The raw-httpd from
> contrib shouldn't have problems sending a JPG with nagle enabled
> (unless limited by available memory or config options)???
Umm. I can describe the set up I have, if that helps. Actually, the
code is all available here if you want to look at that...
http://interactive.freertos.org/entries/20290712-freertos-win32-project-with-lwip-web-server
...but as far as I recall, the only modification I made was that in the
post you replied to before this.
I am using Windows XP, with all service packs. I was testing using
WinPCap developer pack (driver in the files above, that is not the same
as the contrib one) "through" a WiFi adaptor. I was also using a
FreeRTOS simulator (or should that be emulator?), which is not real real
time.
I have since also tried the code on a Microblaze, which is true real
time, but just used the same code so didn't try it without the nagle
algorithm disabled. Perhaps I should.
http://www.freertos.org/Free-RTOS-for-Xilinx-MicroBlaze-on-Spartan-6-FPGA.html
[note the link to the Microblaze is not yet complete - the network
driver works well at 100BaseT but is flaky (although working) at
10BaseT. It will not be released properly until the 10BaseT is fixed,
which will be whenever I get time to look at it again)]
Regards,
Richard.
+ http://www.FreeRTOS.org
Designed for Microcontrollers.
More than 7000 downloads per month.
Simon Goldschmidt
>> Richard, can you tell us anything about this? The raw-httpd from
>>
>> contrib shouldn't have problems sending a JPG with nagle enabled
>> (unless limited by available memory or config options)???
>
> Umm. I can describe the set up I have, if that helps. Actually, the
> code is all available here if you want to look at that...
>
> http://interactive.freertos.org/entries/20290712-freertos-win32-project-with-lwip-web-server
So that seems like a good place to start for me debugging this :-) I'll try that and report back...
gold...@gmx.de
> Umm. I can describe the set up I have, if that helps. Actually, the
> code is all available here if you want to look at that...
>
> http://interactive.freertos.org/entries/20290712-freertos-win32-project-with-lwip-web-server
>
> ...but as far as I recall, the only modification I made was that in the
> post you replied to before this.
Funny, I loaded that ZIP file, added the disable_nagle call and it still
loads slow, while it loads normally fast on my win32 test port.
However, even with the same lwipopts.h, the FreeRTOS sim is not as fast...
BTW: sys_mbox_new() should assum standard values if size==0!
FreeRTOS Info
On 08/09/2011 22:22, gold...@gmx.de wrote:
> FreeRTOS Info wrote:
>> Umm. I can describe the set up I have, if that helps. Actually, the
>> code is all available here if you want to look at that...
>>
>> http://interactive.freertos.org/entries/20290712-freertos-win32-project-with-lwip-web-server
>>
>>
>> ...but as far as I recall, the only modification I made was that in the
>> post you replied to before this.
> That took me a while...
>
> Funny, I loaded that ZIP file, added the disable_nagle call and it still
> loads slow, while it loads normally fast on my win32 test port.
> However, even with the same lwipopts.h, the FreeRTOS sim is not as fast...
The simulator is exactly that - it is running the FreeRTOS code, which
is normally real time, in a non real time windows environment. It is
simulated in the sense that the only way to get the algorithms to run as
expected (for the correct task to run at the correct time, in the
correct sequence) is to have simulated time move much more slowly that
actual time. If that makes sense. I believe your port is just running
the lwIP stack as fast as it can, whereas the FreeRTOS simulator is
designed to allow you to develop and debug FreeRTOS code in a
"convenient" (?) environment, before moving it to a "real" environment.
The lwIP stack being in a FreeRTOS task. There is a bit more here:
http://sourceforge.net/projects/freertos/forums/forum/382005/topic/4693065
>
> BTW: sys_mbox_new() should assum standard values if size==0!
Could you elaborate on that point please? Presumably there is something
in sysarch that needs changing.
Regards,
Richard.
+ http://www.FreeRTOS.org
Designed for Microcontrollers.
More than 7000 downloads per month.
>
Simon Goldschmidt
> The simulator is exactly that - it is running the FreeRTOS code, which
> is normally real time, in a non real time windows environment. It is
> simulated in the sense that the only way to get the algorithms to run as
> actual time.
That's OK with me, I only wanted to point out that the httpd does *not* need nagle to be disabled, it's fast with it enabled and doesn't gain anything from disabling it.
> > BTW: sys_mbox_new() should assum standard values if size==0!
>
> Could you elaborate on that point please? Presumably there is something
> in sysarch that needs changing.
As the defaults for the mbox size defines are 0, the sys_arch should just create a default-size mbox if 0 is passed in (to make sure it runs with the defaults from opt.h). I remember it being like that, but I can't find it documented anywhere... :-(
Anyway, since you have the port *and* the lwipopts.h file under your control, it's not too much of an issue, I guess.
Simon
--
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
FreeRTOS Info
On 09/09/2011 09:39, Simon Goldschmidt wrote:
> FreeRTOS Info <nos...@freertos.org> wrote:
>> The simulator is exactly that - it is running the FreeRTOS code, which
>> is normally real time, in a non real time windows environment. It is
>> simulated in the sense that the only way to get the algorithms to run as
>> expected [..] is to have simulated time move much more slowly that
>> actual time.
>
> That's OK with me, I only wanted to point out that the httpd does *not* need nagle to be disabled, it's fast with it enabled and doesn't gain anything from disabling it.
Yes - the windows simulator does not really tell me anything about the
timing of the TCP/IP applications, so I didn't recognise there was a
problem when using the simulator. I only recognised there was a problem
when I moved the code onto a proper platform, which happened to be a
Microblaze.
http://www.freertos.org/Free-RTOS-for-Xilinx-MicroBlaze-on-Spartan-6-FPGA.html
Note the microblaze example does not yet include the command console, so
is only using lwIP with its raw interface.
I can get the hardware out again and retry it with/without nagle, just
in case there was something else in the mix that was causing the
perceived problem. The source code running on the Microblaze (barring
the command console) is the same as that running in the simulator -
other than the MAC driver, naturally.
---
Regarding the default mbox sizes, there was a definite reason why I had
to change them. It was so definite I cannot remember off hand exactly
why the change was needed, but removing the new definitions would show
it up again right away.
Regards,
Richard.
+ http://www.FreeRTOS.org
Designed for Microcontrollers.
More than 7000 downloads per month.
_______________________________________________
Richard Barry
I have not got the Microblaze hardware out again, but I have tried the
same code on an LPC1769 now, and get the same result. I had to change
lwipopts.h slightly to reduce the RAM footprint, but the C code (barring
the MAC driver) and all other lwIP configuration is as per the FreeRTOS
simulator code you already looked at.
Attached are two logs, one using the raw http server with no
modification, and one with the insertion of the tcp_nagle_disable(pcb)
call, as per my previous post. If its not obvious, 192.168.0.200 is the
target that is running lwIP, and 192.168.0.100 is the host running WinXP.
The point of interest is the difference in the timing between each
frame, as you will see.
Regards,
Richard.
+ http://www.FreeRTOS.org
Designed for Microcontrollers.
More than 7000 downloads per month.
Simon Goldschmidt
> I had to change lwipopts.h slightly to reduce the RAM footprint
In fact, the lwipopts.h settings are the most interesting here. Would you mind sending your file (as attachment)?
There are some things I don't (yet) understand in the traces (e.g. why does the client announce an MSS of 1260 only? I would have expected 1460 here...)
The problem could be that you seem to be limited to 2048 byts of snd_buf and with nagle enabled, lwIP-tcp waits to fill a 2nd segment (of 1260 bytes), which it can't and thus doesn't send (maybe we should enhance the nagle-check here?).
For decent TCP performance, you should ensure that:
- TCP_WND is at least 4 * TCP_MSS (or the remote mss here)
- SND_BUF is as big as the window (4 * mss might also be enough)
- SND_QUEUELEN should not limit the transfer either (i.e. when passing the copy-flag to tcp_write, it should be at least 4 and you should call tcp_write with the biggest possible chunk size)
Not obeing the rules above will make the connection slow (i.e. you will prevent the sliding window algorithm to work smoothly and instead turn your TCP connections into a ping-pong style of DATA-ACK-DATA-ACK etc.).
I'll try to reproduce (and maybe fix) this with your lwipopts.h.
Simon
--
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
_______________________________________________
FreeRTOS Info
That would be excellent - and exactly the feedback I would look for. As
stated previously, my intention is for these to be "reference"
implementations, which is why I am using both the raw and sockets
interfaces, so there is an example of each.
Various lwipopts files are attached, renamed to indicate which project
they are used with. From your ping pong explanation, it sounds like the
cause lies within.
Regards,
Richard.
+ http://www.FreeRTOS.org
Designed for Microcontrollers.
More than 7000 downloads per month.
gold...@gmx.de
> That would be excellent - and exactly the feedback I would look for.
Alright, so it turns out the lwipopts.h *was* the reason for the poor
performance. Using the LPC17xx one, I had the same poor performance with
my win32 port. I got it working by changing the following line:
#define TCP_SND_BUF (TCP_MSS*2) // was: 2048
That ensures that TCP isn't limited too before sending 2 mss-sized
segments (which is the borderline for delayed ACKs on most systems).
BTW: there *are* sanity checks for these defines in init.c. However,
they are (yet) runtime checks that output a warning via
LWIP_PLATFORMDIAG in LWIP_DEBUG mode, so there's a high risk of not
seeing them. Given the time we spent on this, I'll change them to
compile-time errors that can be disabled. There are also other sanity
checks that fail with your configuration. While these can be ignored in
the very special case of the webserver, I'd strongly recommend to change
your lwipopts.h to obey all of them.
I'll update init.c in git these days, and I'll have to adapt the default
values in opt.h, too, since they fail the sanity checks, too :-( -->
bad example!
Simon
FreeRTOS Info
Output message are not always easy to view in embedded environments.
What I should do is run all lwipopts files through the Windows
environment, where viewing the warnings is easy.
Regards,
Richard.
+ http://www.FreeRTOS.org
Designed for Microcontrollers.
More than 7000 downloads per month.
Simon Goldschmidt
> Output message are not always easy to view in embedded environments.
I know, that's why I want to convert it to compile-time checks. The reason it wasn't so is that these are warnings rather than errors (it can work if you know what it does).
Simon
--
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
_______________________________________________