Server status/health, benchmark information

[Sang Do]

Could someone please post this information? 

I'm running VIDA.IO and I am experiencing slowness, login failure especially in the morning. 

We have a couple thousands of visits per day and we're expecting higher traffic but we're not sure meteor server is in a healthy state to do this. 

Thank you. 

Sang Do

[Michael Bishop]

Need lots more information. Performance tuning is a huge field with many items to consider.

What is your server setup? Are you running on a VPS or bare metal? How are you monitoring your performance? Do you have any baselines for performance comparison?

I had to reload your site a few times to get it all. It appears that I was getting timed out. Is the server getting over run by connection requests? What web server are you running? What's the configuration of the web server?

I see Font Awesome is being loaded 3 times. Two with same version, but different sizes and one with an older version. So you could save ~100K of loading by eliminating the extras.

Not knowing anything about your setup, I'd guess it's either the load/connection configuration of the web server or the hardware that is the bottleneck. For hardware it's usually either not enough RAM or the hard drives can't keep up with requests.

--
You received this message because you are subscribed to the Google Groups "meteor-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-talk...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Nick Martin]

Hi Sang,

Looks like you're hosting on the 'meteor deploy' servers. The issue is that your Meteor app process is using 100% CPU. The server is OK and has plenty of RAM and I/O bandwidth, but your app is hitting the limit of a single node process.

Your app was taking 1.7G of memory, which is quite a bit. It's possible you have a memory leak. I've restarted the app, in case the issue is a memory or CPU leak, but it quickly went back to 100% CPU and 1.5G RAM. (You can do the same thing by re-deploying your app.)

The first thing to try is probably to reduce the number of queries per client and the amount of data each client syncs. Be especially wary of queries that send lots of data to each client and change frequently. Wherever possible, include `{_id: xxx}` in queries and updates that should only affect one document. It also helps if each client does the exact same query, so the server can de-deduplicate the queries to mongodb. Another tip is to make it so that clients that are not logged in don't subscribe to publishes they don't need, and only do the subscription once they log in.

-- Nick

On Thu, Oct 10, 2013 at 9:55 AM, Sang Do <sdol...@gmail.com> wrote:

--

[Sang Do]

@Michael: 

Thanks for taking a closer look into the site network and performance.

I do see the fontawesome being loaded three times. This wasn't cleaned up when I applied the UX design to the website. I just got them removed. 

@Nick:

Yes, we're deploying it using 'meteor deploy' and we realize that we need to move it immediately before more traffic coming in. 

I did try to minimize the amount of queries to the minimal for both visitors and users. In this case, I am working on one very high specific request to minimize the queries.

For the memory leak, I'm not sure what's causing it. 

For pub/sub, should we have an API call to pause? Currently, I cannot prioritize requests to make it faster rather than relying on meteor engine to do this.

Thanks

[Phuoc Do]

Hi Nick,

Thanks for the tips. vida.io seems to be back online, it's loading ok again. We have seen similar behavior yesterday around morning time. My experience has been that the app works ok if we have less than live 20 visitors. This traffic is probably enough to fill up 100% CPU on AWS core. Once we pass 20, it's starting to slow down significantly. It could be we reach 1.7GB mem limit.

Is it typical for a meteor app to consume 1.5 GB memory at start? Our app is heavy on client side but not server side.

Phuoc

[Phuoc Do]

I got vida.io deployed to an Amazon instance now. My node process uses about 170-200 MB of memory on this server. It peaks the CPU from time to time.

We did find an inefficiency in pub/sub where we load more than 20 other documents along with the embed one. It'd be nice if there's a mechanism to help avoid unnecessary pub/sub. We've been building these endpoints by hand.

If anyone needs deployment to your own environment, I recommend distelli.com. The deployment tool works like a charm for our app. It saved me a lot of time digging through unmaintained meteor deploy scripts.

Phuoc

[Arunoda Susiripala]

I think I've a solution for the CPU hicking. It's because of the current polling logic or meteor.

You can avoid it with meteor's upcoming oplog branch.

For time being, you can use my SmartCollections package. It has full support for the oplog but, some of the meteor features are not exists.

--
Arunoda Susiripala

@arunoda

https://github.com/arunoda

http://www.linkedin.com/in/arunoda

[curiou...@gmail.com]

Does anyone know where I can find instructions for using distelli to deploy meteor application? 

I looked up the deployment guides section on their website but did not find a guide for either NodeJS or Meteor applications. 

Thank you.

[Valerio Santinelli]

Nick, 

is there a way to grab how much RAM and CPU the node process is consuming on the Meteor deploy servers? It'd be useful to do some performance testing and tuning before deploying to dedicated hosts. Thanks!

Valerio

[Nick Martin]

Memory is easy, there is a node function for that: http://nodejs.org/api/process.html#process_process_memoryusage

CPU is a little more complex, there is no built-in. But you can read /proc/{{process.pid}}/stat to get the info (http://stackoverflow.com/questions/7773826/how-to-find-out-the-cpu-usage-for-node-js-process)

Or there is a handy node module for getting this info: https://npmjs.org/package/usage. I haven't tried it, but it looks like it does exactly what you want.

-- Nick

[Arunoda Susiripala]

I wrote usage:

It reads proc information and shows you the cpu precentage used by your app. Since most of the cloud providers use shared cpu and limitations, you need to use usage with care. 

For an example: In nodejitsu(a nodejs pass) your process only can use ~`2% CPU for each app(drone). So keep that is mind.

[Valerio Santinelli]

Thanks to both you guys for the info. I'll check them out. 

Cheers,

Valerio