Recommendation for number of threads

[nanook]

I am currently in the process of setting up a Scribe server to log
messages from over 1000 apache virtual hosts.

I have Scribe setup with the following category:
category=web-*

And am logging to virtual hosts like this:
web-www.domain.com

Should I set new_thread_per_category to 'yes'? This will make Scribe
used over 1000 threads. Will that cause problems with context
switching?

Would it be preferable to set num_thrift_server_threads to a high
value instead? And if so, what would be a good value to use?

[Anthony Giardullo]

Just to clarify, you have a single scribe server that is processing messages for 1000 different category names and writing these messages to disk?

If you new new_thread_per_category to 'no', then all web-* messages will be processed by a single store (ie written to the same file). Otherwise, they will each be processed by their own store and written to 1000 files (creating 1000 threads).

On the hardware I've tested with, having 1000 threads wasn't too big of a problem (especially if these threads spend most their time waiting on io). While this isn't an ideal threading model, this has not been a problem for me so far as I usually run into io bottlenecks before I run into any issues with having too many threads. I don't currently run any individual scribe servers that are configured to write to more than 1000 files.

-Anthony