RAM usage and disk thrashing in mqsvc.exe for recoverable messages
Jim
the MSDN pages aren't helping much.
Our system in running Windows Server 2003 Standard (32-bit) with 4GB of RAM.
We are using a local private non-transactional queue is used and remote
reads are performed to receive messages.
Currently the machine quota is set to 1GB which equates to about 200K
messages (which would fill up in roughly 2 days downtime). We would like to
be able to support at least 1.5M messages and 15 days downtime, which would
require us to set the machine quota at 7.5GB. Is this possible with just 4GB
of RAM?
If the receiver app is disconnected and the queue fills up to 1GB, I see
that the "Private Bytes" for mqsvc.exe shows ~7MB and the "Virtual Bytes" and
"Working Set" numbers show 1GB+. Is MSMQ really using 25% of my RAM resources
of my 4GB physical RAM?
This server is not just performing message queuing, but also other mission
critical tasks and the MSMQ is required to not consume more than 10% of
system resources (RAM, disk, CPU). Does this mean, I need to set the quota
to 400MB?
I am concerned that the VM usage by MSMQ would cause disk thrashing and
result in other mission critical processes to degrade in performance.
Thanks.
John Breakwell
The quota you set on a system or queue is unrelated to the amount of
physical RAM.
MSMQ does not need to fit all memory mapped files in physical RAM.
That limitation only applied to Windows NT4/2000 (MSMQ 1.0/2.0).
So you can leave MSMQ to manage the memory mapped files as best it can.
"Private Bytes" - MSMQ uses 75 bytes per message so 1,500,000 messages is
112.5MB private bytes, which is probably about half what the system has
available at boot up.
I would recommend migrating to a 64-bit platform as 32-bit Windows may not
have enough Private Bytes for your needs.
Check Item 4 of this bog post:
http://blogs.msdn.com/johnbreakwell/archive/2006/09/18/insufficient-resources-run-away-run-away.aspx
This blog post should help too:
http://blogs.msdn.com/johnbreakwell/archive/2008/02/29/what-are-msmq-s-limits-if-i-had-a-farthing-for-every-time.aspx
"Working Set" - if you aren't using RAM for anything else, MSMQ will use it
for memory-mapped files. 1GB of messages taking up 1GB of RAM on a machine
with lots of free physical RAM would make sense.
"We would like to be able to support at least 1.5M messages and 15 days
downtime"
MSMQ should be handled as a transport rather than a storage system.
Although you can technically store the messages from 2 week's of downtime in
MSMQ, I would recommend writing the messages to an SQL database.
"MSMQ is required to not consume more than 10% of system resources"
MSMQ won't lock physical RAM - your working set should go down as physical
RAM comes under pressure.
I don't know how you're going to set MSMQ to use no more than 10% of system
resources with the volumes of messages you are discussing.
Technically it probably isn't implementable and may not be recommended if
you need good performance from MSMQ with these message volumes.
"I am concerned that the VM usage by MSMQ would cause disk thrashing and
result in other mission critical processes to degrade in performance."
MSMQ will page in/out memory-mapped files as required based on available
physical RAM.
Other factors will come into play too, such as how fragmented the memory
mapped files, how old the message is that is being read from a queue,
whether the queues are transactional, etc.
For more on disk performance:
http://blogs.msdn.com/johnbreakwell/archive/2008/02/13/msmq-disk-performance.aspx
Cheers
John Breakwell (MSFT)
"Jim" <J...@discussions.microsoft.com> wrote in message
news:D518545B-8534-41E6...@microsoft.com...
John Breakwell
The limitation of Windows NT4/2000 (MSMQ 1.0/2.0) was around having to fit
all messages in the process' 2GB virtual memory, NOT physical RAM.
Apologies for any confusion.
Cheers
John Breakwell (MSFT)
"John Breakwell" <john...@msft.com> wrote in message
news:un1zd2Vs...@TK2MSFTNGP05.phx.gbl...