NSOperation + Networking: You're doing it wrong!

[Ken Collins]

Morning Ben,

I have still yet to finish my iPhone app using ASIHTTPRequest, but am looking forward to it. You are already in the 3rd party credits screen and I'll let you know when it launches. BTW, saw these series of tweets this morning. I could care less, but thought they might make an interesting discussion. Thoughts?

http://twitter.com/sophiestication/status/5761126862
http://twitter.com/sophiestication/status/5761282478
http://twitter.com/sophiestication/status/5761282784


- Thanks again,
Ken Collins

[Ben Copsey]

Hi Ken

I think she has a fair point. Using an NSOperation will certainly be more memory intensive, and a bit more CPU intensive.

ASIHTTPRequests do use a loop to prevent the operation finishing while the request is in progress, which will slow things down a little bit. Having said that, if there isn't any new data available yet, they'll probably spend most of their time in CFRunLoopRunInMode, which almost certainly means the thread will be sleeping when there's nothing to do.

The current implementation of the main loop could be made more efficient by running the runloop for timeoutSeconds, which as far as I understand should mean the thread will sleep the whole time if it gets no data, I hope to look at this fairly soon.

Unless you have loads of operations running at the same time, I don't believe the CPU hit will be that large.

IMO there are a several advantages to NSOperation for network stuff:

* Request queuing and managing the number of concurrent operations
* Request dependencies
* Ability to perform parsing of the data as it is received with a much smaller impact on ui responsiveness

On top of the NSOperation stuff, ASIHTTPRequests are much more flexible than NSURLRequests, and support a lot more features out of the box, including some things that aren't possible with NSURLRequest / NSURLConnection.

Great to hear your app is coming soon, I look forward to seeing it!

b

> --
>
> You received this message because you are subscribed to the Google Groups "ASIHTTPRequest" group.
> To post to this group, send email to asihttp...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/asihttprequest?hl=.
>
>

[Tim]

Hey Ben,

I've poked through the ASIHTTPRequest code some over the last few
months, and I've poked around the docs for NSOperation,
NSOperationQueue, and some of the CFNetwork stuff. I've done non-
blocking I/O stuff in Ruby and in Java, and I'm interested in your
advice on how to learn more about the Cocoa approach to non-blocking I/
O and how that relates to NSOperationQueues (and also how ASI stuff is
setup).

For example, I don't seem to get why the NSOperation docs say that you
can mark you operation concurrent if it calls a non-blocking syscall,
but that you really only do that if you call it without putting it in
a queue. In Ruby's EventMachine framework everything's on one thread,
and all I/O is evented, so you tend to go about making a bunch of
async requests one after the other, and then handling them each as
they complete. With using a blocking NSOperation, like ASI does and
what the docs suggest, it seems like even by using a non-blocking call
you don't let the thread keep busy by pushing more requests out but
instead force operation #2 to sit and wait for op #1 to fully complete
as if it were synchronous (to the queue's thread). So while this seems
to let the main UI thread keep moving, it feels different from what
I'm used to in other frameworks.

This is just one example of how I'm a novice at working with either
async operations in cocoa or with network system calls in Obj-C. How
can I wrap my head around the model right so I approach problems with
the right kind of idiomatic approach for Obj-C?

Thanks for your help.

Tim

[hartem]

Hi Ben,
First off, thanks for a great wrapper, it has been extremely useful
for me so far.
I need much of the flexibility that ASIHTTPRequest provides, and
NSURLRequest etc just won't cut it for some of the things I'm trying
to do.

I've only been using it a day or so, but I have noticed that it does
seem to be a lot more CPU intensive than the previous incarnation of
my mac app, which relied on NSURLDownload (I'm downloading large
files).

I'm going to attempt to look into why this would be the case over the
next few days, but will start with the timeoutSeconds for the runloop
as you have suggested.

If you have any other ideas why performance might be suffering, please
let me know.
Thanks again,
Martin.

[Ben Copsey]

Hi Tim

> I've poked through the ASIHTTPRequest code some over the last few
> months, and I've poked around the docs for NSOperation,
> NSOperationQueue, and some of the CFNetwork stuff. I've done non-
> blocking I/O stuff in Ruby and in Java, and I'm interested in your
> advice on how to learn more about the Cocoa approach to non-blocking I/
> O and how that relates to NSOperationQueues (and also how ASI stuff is
> setup).
>
> For example, I don't seem to get why the NSOperation docs say that you
> can mark you operation concurrent if it calls a non-blocking syscall,
> but that you really only do that if you call it without putting it in
> a queue.

The concurrency programming guide uses an example where a concurrent operation creates its own thread to do work. If you did this, there wouldn't be as much value in using queue (though queue management and dependencies would still be useful) , but I don't think you have to use concurrent operations this way.

NSOperationQueue creates the thread to allow your non-concurrent operation to run without blocking the main thread. ASIHTTPRequests are non-concurrent NSOperations, so they rely on a queue to make them run asynchronously.

It would be interesting to experiment with using a concurrent NSOperation for ASIHTTPRequest. Rather than creating its own thread, a request would continue to leave that up to the queue. It could just start the runloop, and use a timer that gets reset every time a network event happens for the timeout. That way, we could eliminate the main loop, though I'm guessing requests would still need something similar to the current approach to ensure synchronous requests still worked.

> In Ruby's EventMachine framework everything's on one thread,
> and all I/O is evented, so you tend to go about making a bunch of
> async requests one after the other, and then handling them each as
> they complete.

CFNetwork works in a similar way - you schedule a request on the run loop, and it generates events as things happen. You *can* schedule lots of requests on the same run loop / thread, but ASIHTTPRequest doesn't do that.

> With using a blocking NSOperation, like ASI does and
> what the docs suggest, it seems like even by using a non-blocking call
> you don't let the thread keep busy by pushing more requests out but
> instead force operation #2 to sit and wait for op #1 to fully complete
> as if it were synchronous (to the queue's thread).

Each request has its own thread. Requests don't know anything about each other, or how many (if any) requests are running at the same time. Only the queue can start new requests, and it will only do so when a slot becomes free (how many can run at once is defined by the queue's maxConcurrentOperationCount).

Obviously, network requests can't be load-balanced in quite the same way as CPU-bound operations. If a request hasn't received any data recently, it might be because the connection is slow, so the very worst thing to do would be to start off more requests.

> This is just one example of how I'm a novice at working with either
> async operations in cocoa or with network system calls in Obj-C. How
> can I wrap my head around the model right so I approach problems with
> the right kind of idiomatic approach for Obj-C?

I don't think it fundamentally works any differently to what you've been used to. If some things seem confusing, it's probably because NSOperationQueue and asynchronous networking have slightly different models for how things should work.

ASIHTTPRequest 's design is less about the conventions of Obj-C than it is about about creating the API that I think will make me (and hopefully other people) more productive. I think about the API I want to have for my programs, and build that. In some cases, this may mean that its mental model doesn't map directly onto the underlying APIs it uses... :)

As a side note, I really appreciate people taking the time to ask about stuff like this. Some of the design decisions in ASIHTTPRequest were made a couple of years ago, based around the needs I had for a project I was working on then. It has improved because so many people have asked good questions and have been prepared to question the implementation.

Best,

Ben

[Ben Copsey]

Hi Martin

> First off, thanks for a great wrapper, it has been extremely useful
> for me so far.
> I need much of the flexibility that ASIHTTPRequest provides, and
> NSURLRequest etc just won't cut it for some of the things I'm trying
> to do.

I'm glad you find it useful!

> I've only been using it a day or so, but I have noticed that it does
> seem to be a lot more CPU intensive than the previous incarnation of
> my mac app, which relied on NSURLDownload (I'm downloading large
> files).
>
> I'm going to attempt to look into why this would be the case over the
> next few days, but will start with the timeoutSeconds for the runloop
> as you have suggested.

I don't think this change will be quite as simple as changing the time interval parameter to CFRunLoopRunInMode.

Increasing the time the run loop is run for will improve performance in some cases, but it is called with the returnAfterSourceHandled flag set to true, so as soon as your request is able to send or receive data, it will return control to the main loop.

To make this change work properly, you'd also need to pass false as the last parameter, and record the fact that data had been sent or received to prevent a timeout when control returned to the main loop. Additionally, the progress reporting would need to be moved elsewhere, since the main loop is not going to be hit often enough.

> If you have any other ideas why performance might be suffering, please
> let me know.

Off the top of my head are two areas where performance might be improved are:

* Time spent in the main loop (as above)
* Frequency that progress delegates are notified of updates in progress (this is almost certainly happening too often)

If you can afford to wait, I'd leave this for now. I'll spend some time this week looking at improving performance, as there have been a few questions about this recently.

Best

Ben

[Ben Copsey]

I've just created a new experimental branch on GitHub that includes a 'concurrent' NSOperation version of ASIHTTPRequest: http://github.com/pokeb/asi-http-request/tree/concurrent

It can work in three ways:

* Synchronously
This works as before, but should offer improved performance because what was the old main loop now only runs every 1/4 of a second, rather than every 1/4 of a second OR when something happens with the request. Start a synchronous operation with [request startSynchronous] (not [request start]).

* Asynchronously, on a background thread (when using an NSOperationQueue/ASINetworkQueue)
Works as before, but should be faster, as there is no main loop at all, just a timer that checks in on the request every 1/4 of a second. What was [request startAsynchronous] is now [request startInBackgroundThread].

* Asynchronously, in the same thread
This is new. :) Calling [request start] will run the request asynchronously in the same thread. Not sure how well this will work (if at all) when called from background threads at present, but it seems to work nicely in the main thread.

These changes will hopefully result in better performance and lower cpu use, though I haven't done a lot of testing.

THIS IS ABSOLUTELY NOT READY FOR PRODUCTION USE. The API may change, some things are broken, lots of tests are failing.

Thanks

Ben

[Martin Harte]

Hi Ben,
Excellent stuff. Thanks for looking at this so quickly. I'm finding
that the new version is far less CPU intensive.

It is crashing occasionally on me though when handleBytesAvailable is
called (as in the attached image).
I assume this is something to do with the timer firing when the stream
is no longer available.
I'll try and investigate further now.

All the best,
Martin.

2009/11/17 Ben Copsey <b...@allseeing-i.com>:

[hartem]

The issue only seems to concern the asynchronous modes, and appears
randomly when downloading a large amount of data.

The error is as follows:
0x90b07020 <+0018> call 0x90b07025
<_ZN14HTTPReadFilter15canReadNoSignalEP13CFStreamErrorh+23>

which occurs when calling
CFIndex bytesRead = CFReadStreamRead(readStream, buffer, sizeof
(buffer));

I'm having trouble find the cause of it though, and there doesn't seem
to be much on the web regarding this.

Cheers,
Martin.

>  Screen shot 2009-11-17 at 20.05.12.png
> 192KViewDownload

[Ben Copsey]

Hi Martin

> The issue only seems to concern the asynchronous modes, and appears
> randomly when downloading a large amount of data.
>
> The error is as follows:
> 0x90b07020 <+0018> call 0x90b07025
> <_ZN14HTTPReadFilter15canReadNoSignalEP13CFStreamErrorh+23>
>
> which occurs when calling
> CFIndex bytesRead = CFReadStreamRead(readStream, buffer, sizeof
> (buffer));
>
> I'm having trouble find the cause of it though, and there doesn't seem
> to be much on the web regarding this.

If you're happy to use the main branch version for your development for the time being, I should have more time later this week to start fixing some of the problems with the new version.

ta

Ben

[Ben Copsey]

I've made some changes to the concurrent branch of ASIHTTPRequest, and it's a lot more stable, most tests pass now.

It has been necessary to change the API again:

[request startSynchronous] = synchronous request, runs in current thread
[request startAsynchronous] = asynchronous request, runs in current thread
[request startInBackgroundThread] = asynchronous request, in background thread

I have eliminated the global queue that startAsynchronous used to use, startInBackgroundThread now creates its own thread, avoiding a queue altogether.

[request start] will now do different things on different platforms. On Leopard and iPhone, it does the same as startInBackgroundThread. On Snow Leopard, it does the same as startAsynchronous (this is because GCD will manage the thread when the request runs from a queue).

Delegate authentication and ASIAuthenticationDialog should now work in synchronous and asynchronous requests on the main thread.

I am wondering if startInBackgroundThread is even necessary, this might be removed in future, it's a one line call to run it in a background thread yourself anyway.

If anyone has time to test this version of ASIHTTPRequest, you can grab a copy here: http://github.com/pokeb/asi-http-request/tree/concurrent

Thanks

Ben