node-webkit http.get errors

[Christopher Hall]

Cross posted to stackoverflow.

I'll explain briefly, I'm working on an RSS aggregator with node-webkit as the backend. I was running through my list of feeds, and I've found these four errors consistently occur when I'm making lots of requests, but not when I'm making fewer (roughly less than 50 requests). The errors that are plaguing my progress are HPE_INVALID_CONSTANT, ETIMEDOUT, EAGAIN, and HPE_INVALID_HEADER_TOKEN. The first one somehow isn't an issue now, now I appear to be getting just two EAGAIN and then many ETIMEDOUT. Invalid header token appeared once, but I thought it was worthwhile to record it.

For one I don't really understand what these errors are telling me, I believe the first one is that the head of the file (or the whole file?) isn't what node.js was expecting, and as such threw an error, however after a good hour and a bit of searching, and even less luck reproducing my issue, this is all I have. Any insight on both what these errors mean, what is causing them, and what I can change in my code to fix them, would be astoundingly appreciated. Thank you very much in advance!

[Michael Latta]

Just guessing, but is it 50 requests or 50 concurrent requests.  If the later I would first try throttling the rate you issue requests.  There may be something in the HTTP library that is getting messed up with too many concurrent requests.  If the feeds are al from different servers you may be hitting some underlying connection limit.

If reducing the number of concurrent requests, the next thing I would try is see if the errors occur if the requests are in a different order.  Try sorting the feed URLs both ascending and descending and see if that makes a difference.  If it is the same URLs that fail in each case see if blacklisting them from your requests makes it work, just to test if it is that URL that is an issue.

Michael

--
You received this message because you are subscribed to the Google Groups "node-webkit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-webkit...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Christopher Hall]

I'll see if I can whip up a system that keeps the amount of requests below 50, and only starts new ones as old ones are finished and see if that helps. Is there anything native in nodejs that could help with this? (I know how to make such a system in javascript, but just curious)