[lwip-users] LWIP Delay between Consecutive Packets
Gary Olson
|
Hi, I am trying to figure out if I can increase my TCP bandwidth. Using a PC timer I see that as I send TCP Packets that the delay is in the range of 500 us on average. There may be a couple at the start with much less delay between TCP Packets. But most of the packets after the first few are in the same range.
Does any one know what the LWIP Delayed Acknowledge timer is set to. I have been looking on the web in general for a solution to this. I program my PC in Visual C++ using Winsock so I am going to see if using TCP NO DELAY will have a positive change. Since this will cause the Nagle Algorithm to be disabled and packets are supposed to be sent immediately rather waiting for a delayed acknowledge to come first. Thank You, Gary |
|
Kieran Mansley
> I am trying to figure out if I can increase my TCP bandwidth. Using a
PC timer
> I see that as I send TCP Packets that the delay is in the range of 500
us on
> average.
If you have a delay in the order of hundreds of microseconds, the
delayed ACK timer is not something to worry about as delays from that
are measured in hundreds of milliseconds.
Kieran
_______________________________________________
lwip-users mailing list
lwip-...@nongnu.org
http://lists.nongnu.org/mailman/listinfo/lwip-users
Gary Olson
-----Original Message-----
From: lwip-users-bounces+golson=jnt...@nongnu.org
[mailto:lwip-users-bounces+golson=jnt...@nongnu.org] On Behalf Of
lwip-user...@nongnu.org
Sent: Tuesday, September 30, 2008 12:00 PM
To: lwip-...@nongnu.org
Subject: lwip-users Digest, Vol 61, Issue 33
Importance: Low
Send lwip-users mailing list submissions to
lwip-...@nongnu.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.nongnu.org/mailman/listinfo/lwip-users
or, via email, send a message with subject or body 'help' to
lwip-user...@nongnu.org
Hi Kieran,
Actually I am in trouble in my Xilinx Design since my bandwidth is not
good enough to handle the data rates required.
According to the PC Timing information. I get a bandwidth of about 7.0
Megabytes per second on a Gigabit Interface.
Or 56.5 Megabits per second.
Now each individual packet I send is 4040 bytes and can be sent in around 14
- 16 us by the PC. But the Time between each packet
Is approx 500 us. If I can figure out how to shorten this time I may
improve my bandwidth. I need to
double my bandwidth to about 14 Megabytes per second be successfull with
this design. My requirement is about 6 Picture frames
a second and at about 7 Megabytes I can probably get 3 Frames per second.
Gary
You can reach the person managing the list at
lwip-use...@nongnu.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of lwip-users digest..."
Today's Topics:
1. Re: LWIP Delay between Consecutive Packets (Kieran Mansley)
----------------------------------------------------------------------
Message: 1
Date: Tue, 30 Sep 2008 16:13:11 +0100
From: Kieran Mansley <kie...@recoil.org>
Subject: Re: [lwip-users] LWIP Delay between Consecutive Packets
To: Mailing list for lwIP users <lwip-...@nongnu.org>
Message-ID: <1222787591....@moonstone.uk.level5networks.com>
Content-Type: text/plain
On Tue, 2008-09-30 at 11:00 -0400, Gary Olson wrote:
> I am trying to figure out if I can increase my TCP bandwidth. Using a
PC timer
> I see that as I send TCP Packets that the delay is in the range of 500
us on
> average.
If you have a delay in the order of hundreds of microseconds, the
delayed ACK timer is not something to worry about as delays from that
are measured in hundreds of milliseconds.
Kieran
------------------------------
_______________________________________________
lwip-users mailing list
lwip-...@nongnu.org
http://lists.nongnu.org/mailman/listinfo/lwip-users
End of lwip-users Digest, Vol 61, Issue 33
******************************************
bill
Are you using RAWAPI? I have data only for that mode. Chances are you need to speed up your Ethernet driver and choose a faster checksum algorithm or write or find one in assembly language. Both make a huge difference.
I have clocked lwIP at 940MbS (megabits average – 960 peak) using a tcp_sent callback but doing nothing in the callback. When I use the data as our application needs it (copying it into sequential memory), I could get 500-600MbS using a DMA memory copy, but this would lock up the PowerPC intermittently. Using a high-speed memory copy (in assembly language) we can run our application with 300-400MbS – this includes an interrupt running at 36kHz consuming almost half of the processor bandwidth. We are sending 1 to 2MB blasts from the PC as many as 7 or 8 per second. One critical point is that we must call tcp_output after a tcp_write for the PC to respond quickly. Our application has 99% of the data coming from the PC, not going to it. Be warned that WireShark affects the bandwidth at higher speeds.
If you’re not using the RAWAPI, this won’t help you too much except to show you what we found lwIP to be capable of achieving.
Bill
Kieran Mansley
> Hi Kieran,
>
> Actually I am in trouble in my Xilinx Design since my bandwidth is not
> good enough to handle the data rates required.
> According to the PC Timing information. I get a bandwidth of about 7.0
> Megabytes per second on a Gigabit Interface.
> Or 56.5 Megabits per second.
>
> Now each individual packet I send is 4040 bytes and can be sent in around 14
> - 16 us by the PC. But the Time between each packet
> Is approx 500 us. If I can figure out how to shorten this time I may
> improve my bandwidth. I need to
> double my bandwidth to about 14 Megabytes per second be successfull with
> this design. My requirement is about 6 Picture frames
> a second and at about 7 Megabytes I can probably get 3 Frames per second.
Obviously the hardware you run on will affect the performance that is
achievable, but others have seen significantly higher bandwidth than
that. It might help to know which API you're using (sockets will be
much slower than the raw API) and which version of lwIP you're using.
Kieran
Gary Olson
Hi Kieran and Bill,
I thought I had setup checksum offloading but I found that the software settings had to be set up as well as the hardware. This did improve my bandwidth about 20 percent but still
To slow for my needs. Still have delays between my UDP packets around 400 us. My bandwidth for TCP transfers are around 70 – 80 Megabits per second.
I am using RAW API and a ML505 Xilinx Board with a Microblaze processor running at 100 Mhz and PLB 125 Mhz I can send 3 Picture Frames in .8 seconds one frame being 2 Million Bytes or 16 million bits.
3 * 16 Million bits = 54 million bits in .8 seconds = 67.5 million bits per second using TCP.
Here is my settings:
BEGIN LIBRARY
PARAMETER LIBRARY_NAME = lwip
PARAMETER LIBRARY_VER = 3.00.a
PARAMETER PROC_INSTANCE = microblaze_0
PARAMETER tcp_mss = 4560
PARAMETER ip_frag_max_mtu = 5500
PARAMETER ip_reass_bufsize = 9000
PARAMETER mem_size = 1000000
PARAMETER pbuf_pool_bufsize = 4600
PARAMETER tcp_wnd = 5560
PARAMETER tcp_snd_buf = 25000
PARAMETER tcp_tx_checksum_offload = true
PARAMETER tcp_rx_checksum_offload = true
PARAMETER pbuf_pool_size = 550
PARAMETER memp_n_pbuf = 32
END
I have been using LWIP I think Version 3, -llwip4 is my link against library command.
Additionally I played with tcp_wnd and tried raising it all the way to 60000 it might have helped some but brought it back down because
Some one said I should have the FIFO size for it in my hardware MAC. I did notice that when I brought the tcp_wnd that the GAP started to
Vary from shorter to longer in a serial type of sequence.
Thank You,
Gary Olson
Kieran Mansley
> Additionally I played with tcp_wnd and tried raising it all the way to
> 60000 it might have helped some but brought it back down because
>
> Some one said I should have the FIFO size for it in my hardware MAC.
> I did notice that when I brought the tcp_wnd that the GAP started to
The first thing I thought when I read this was that your window size was
too low. It will severely limit the amount of TCP data you can have in-
flight. Setting it to something like your tcp_snd_buffer would make
more sense.
Taranowski, Thomas (SWCOE)
This sounds to me like the driver isn’t sending out packets right away. I’m not specifically familiar with the Xilinx board, but Ethernet MACs are all pretty similar in design. I’d review the driver to ensure that the bit to mark the send buffer ready to send (Ready, Own, etc.) is set correctly when a packet is enqueued in the send queue, and that the hardware isn’t pending on something to be done.
From:
lwip-users-bounces+thomas.taranowski=honeyw...@nongnu.org
[mailto:lwip-users-bounces+thomas.taranowski=honeyw...@nongnu.org] On Behalf Of Gary Olson
Sent: Friday, October 03, 2008
7:59 AM
To: lwip-...@nongnu.org
Subject: [lwip-users] LWIP Delay
between Consecutive Packets
Hi Kieran and Bill,