Questions about child process and their limitations.

[Peter Frazier]

Good morning|afternoon|evening,

I'm working on a custom news tracking nodejs api for web/android/chrome apps.  It starts with keywords, then builds a feed list from news.google.com and using readability-node and feed-reed npm modules to process the whole mess.  Once I got the kinks worked out of it i was ready to move the whole thing into its own thread to free up the server to take more incoming api requests.  A well keyword'ed search can reach up to 100 articles a go, and this is just using google news, i have future plans to add in many other sources and recursive searches into wikipedia as well as other available databases.

the issue i'm running into is the code seems to just stop at a Article.findOne() mongoose call.  any and all console.log()'s never fire.  Is there some limitation?  http://jsapp.us/s/515 is a link to the code..

is there limitations in queuing and saves databases?  Am I using processes wrong?

[Tom Boutell]

On Saturday, January 3, 2015 1:21:05 PM UTC-5, Peter Frazier wrote:

Good morning|afternoon|evening,

the issue i'm running into is the code seems to just stop at a Article.findOne() mongoose call.  any and all console.log()'s never fire.  Is there some limitation?  http://jsapp.us/s/515 is a link to the code..

is there limitations in queuing and saves databases?  Am I using processes wrong?

I don't see where you are spawning processes at all in this code. Without that part it is difficult to say what might be going wrong.

Does this code work fine if you do it without spawning processes?

What benefit, exactly, do you hope to get by spawning processes? Are the tasks CPU-intensive, so you're spinning your wheels in JavaScript and not answering new requests?

If every API request sooner or later winds up hitting this code, then there's no real benefit in responding to requests faster than you can actually process them.

Our approach in this situation is to run multiple independent copies of the application, no more than one per core usually, listening on separate ports, and use nginx as a round-robin load balancer, which is very easy to set up and a proven solution.