WinHttp Internals / Effeciency
Sam Khavari
would I be able to build a more efficient web proxy using sockets and IO
completion ports?
Thanks,
Sam
Stephen Sulzer (Microsoft)
Internally, WinHTTP uses Winsock, I/O completion ports for asynchronous
requests, and a high-performance memory manager. So it is pretty efficient,
but it could certainly be beaten with some effort. You could try
implementing the first version, or a prototype, of your web proxy using
WinHTTP. After measuring your application's overall performance, you should
then have the information to decide whether to write your own HTTP client if
WinHTTP's performance proved unsatisfactory. (For best performance, I assume
you would use WinHTTP's Win32 C API and not it's COM component, and also use
the API in asynchronous mode.) WinHTTP includes an option to specify the
size of the worker thread pool that services its internal I/O completion
port; playing with this value should measurably impact performance.
Also keep in mind that WinHTTP is an HTTP client only; it does not implement
an HTTP server API. You would still need to implement the HTTP server part
of your web proxy, presumably using Winsock. Nor does WinHTTP implement a
persistent URL cache, which is often an important feature of a web proxy.
Good luck!
Regards,
Stephen Sulzer
Microsoft Corporation
This posting is provided "AS IS" with no warranties, and confers no rights.
"Sam Khavari" <skha...@cs.stanford.edu> wrote in message
news:Oa5nPabqBHA.2128@tkmsftngp07...
Sam Khavari
many pages I can download. So, in my world, the local machine has a billion
requests it must fetch as fast as possible.
> Internally, WinHTTP uses Winsock, I/O completion ports for asynchronous
> requests, and a high-performance memory manager. So it is pretty
efficient,
> but it could certainly be beaten with some effort.
What would be WinHTTP's bottleneck? Is the interface for this memory
manager public? I'm assuming its a custom Memory Allocator.... right?
>You could try
> implementing the first version, or a prototype, of your web proxy using
> WinHTTP. After measuring your application's overall performance, you
should
> then have the information to decide whether to write your own HTTP client
if
> WinHTTP's performance proved unsatisfactory.
Excellent idea. I assume there are no performance numbers of the throughput
one can achieve using WinHttp given all the random variables involved.
> (For best performance, I assume
> you would use WinHTTP's Win32 C API and not it's COM component, and also
use
> the API in asynchronous mode.)
Absolutely
> WinHTTP includes an option to specify the
> size of the worker thread pool that services its internal I/O completion
> port; playing with this value should measurably impact performance.
Is there any data roughly showing throughput vs number of threads for a
given machine with n cpu's?
> Also keep in mind that WinHTTP is an HTTP client only; it does not
implement
> an HTTP server API. You would still need to implement the HTTP server part
> of your web proxy, presumably using Winsock. Nor does WinHTTP implement a
> persistent URL cache, which is often an important feature of a web proxy.
Right now I'm focusing on sucking down pages as quick as possible, so this
isn't an issue.
> Good luck!
Thanks!
> This posting is provided "AS IS" with no warranties, and confers no
rights.
Damn lawyers.... :-)
Thanks for all your help!
Stephen Sulzer (Microsoft)
Given values for some key parameters, we can do some back-of-the-envelope
calculations to estimate the theoretical maximum throughput that could be
achieved, and whether WinHTTP would be up to the task.
You need the following numbers:
- The sustained download speed that your network connection can maintain.
Specifically, the network bandwidth between your machine and the target
server to which you'll be sending the requests. (T)
- The average size of the web page or resource you'll be requesting. (S)
Dividing T by S will give you the maximum requests per second throughput
that could possibly be attained.
Do you have such data yet for your setup? That would be interesting to know.
Example A. Suppose you have a very fast pipe between the two machines. Say
100 MB/s (about 1Gbps) sustained transfer rate. But the average file is
quite big: 1 megabyte. Then your HTTP client only needs to handle 100
requests/second on average to keep all that network bandwidth productive.
Example B. You have a more modest network connection, maybe a T3 line,
somewhere around 5MB/s (45Mbps) I think. And the average web page is 5K.
Then your HTTP client needs to handle 1,000 requests/second so as not to be
the bottleneck.
Example C. 100 MB/s sustained transfer rate; 10K average file size. Your
HTTP client will need to handle 10,000 r/s to keep up.
> What would be WinHTTP's bottleneck? Is the interface for this memory
> manager public? I'm assuming its a custom Memory Allocator.... right?
>
The memory manager is proprietary, sorry.
However, one parameter I can help fill in concerning WinHTTP's performance
(which you could measure independently) is: how many CPU cycles does WinHTTP
consume for a typical request. I don't have an exact figure, but for a
single, plain vanilla HTTP request, (no SSL, no authentication, no proxy
servers in between to deal with, etc.), WinHTTP requires somewhere around
500,000 CPU cycles. This number increases for larger requests, as more
ReadData calls consume more CPU cycles. But then, the larger the request
resource, the fewer that need to be processed per second due to network
bandwidth limits.
For a 1GHz CPU then, WinHTTP should be able to process about 2,000
requests/second--under ideal conditions, assuming network bandwidth and
latency are not bottlenecks, and you application doesn't hog the CPU for
other needs. Such a machine using WinHTTP should be expected to handle the
workloads in Examples A and B, but not a chance on example C.
Your application architecture should use a small set of threads to issue the
asynchronous requests. Probably no more than two threads per CPU. A single
thread should be able to keep "in flight" a dozen simultaneous requests.
(Let's call this the concurrency of the thread.) As one request completes,
the thread dispatches the next one. You might also try experimenting with
the WORKER_THREAD_COUNT WinHTTP option. By default, WinHTTP will create 1
worker thread per CPU. I would suggest setting this value to 2 * the number
of CPUs in the machine.
You do not want to create many worker threads as context switching will
degrade performance. Keep an eye on the CPU usage in Task Manager. If CPU
utilization is not over 90%, then try experimenting with adding more worker
threads, or bumping up the concurrency count for an application thread. If
such changes actually lower the CPU usage, then cut back on the number of
threads; your application is already suffering contention on some shared
resource.
WinHTTP's asynchronous programming model requires your application to
implement a callback function to receive status and completion
notifications, and to initiate the next stage of the request (i.e., the next
read request for more data). The callback function will often be called from
one of WinHTTP's worker threads. Be careful not to stall WinHTTP's async
engine by performing a lot of unnecessary or inefficient work during the
callback. Typically you will want to read the response data in 8K to 32K
chunks.
Well, I hope this gives you a framework to work in and to model your
scenario. Please report back with any results you obtain.
Regards,
Stephen Sulzer
Microsoft Corporation
This posting is provided "AS IS" with no warranties, and confers no rights.
"Sam Khavari" <skha...@cs.stanford.edu> wrote in message
news:OitLFLxqBHA.2304@tkmsftngp07...
Sam Khavari
know.
Yes and no. I'll have to parse some logs to get this data. Ill report back
when I have good numbers. I think the trend will be very small pages (~10K)
with a maybee 10Mbps throughput.
> However, one parameter I can help fill in concerning WinHTTP's
performance
> (which you could measure independently) is: how many CPU cycles does
WinHTTP
> consume for a typical request. I don't have an exact figure, but for a
> single, plain vanilla HTTP request, (no SSL, no authentication, no proxy
> servers in between to deal with, etc.), WinHTTP requires somewhere around
> 500,000 CPU cycles. This number increases for larger requests, as more
> ReadData calls consume more CPU cycles. But then, the larger the request
> resource, the fewer that need to be processed per second due to network
> bandwidth limits.
> Your application architecture should use a small set of threads to issue
the
> asynchronous requests. Probably no more than two threads per CPU. A single
> thread should be able to keep "in flight" a dozen simultaneous requests.
> (Let's call this the concurrency of the thread.) As one request completes,
> the thread dispatches the next one. You might also try experimenting with
> the WORKER_THREAD_COUNT WinHTTP option. By default, WinHTTP will create 1
> worker thread per CPU. I would suggest setting this value to 2 * the
number
> of CPUs in the machine.
Is there any way to throttle the number of threads or must that be static?
I wanted to experiment with an algorithm similiar to what is Richter's
"Programming Server-Side Applications for Windows 2000".
> Well, I hope this gives you a framework to work in and to model your
> scenario. Please report back with any results you obtain.
Absolutely, and I tremendously appreciate your help.
Thanks,
Sam Khavari