[Rocks-Discuss] Creating queues for SGE jobs

[Geoff Scott]

All,

I have a small cluster (around 15 nodes) that is currently using SGE
heavily. I have many, many batch jobs coming up for two different
projects. I need to allocate one half of my cluster to each project.
I thought the best way to do this would be set up two different
queues.

I've searched all over and can't find a good write up of how to go
about this. The qconf command can get it done, but as to what I
should be entering for each variable in the queue file, I've no idea.
I also looked at Qmon, but this just seems to run qconf so I don't
really gain anything there except a little more documentation. I'm
very hesitant to monkey too much with the system while it's under such
a great load. I can't muck anything up.

At any rate, I realize this is fairly pedestrian, but any help would
be much appreciated. Two queues with each queue only containing half
of the available computing nodes. Nothing fancy.

Thanks in advance.

Geoff

[Dr. Hung-Sheng Tsao (LaoTsao)]

please check this blueprint
one simple way is use the fair share scheduler
http://wikis.sun.com/display/BluePrints/Scheduler+Policies+for+Job+Prioritization+in+the+N1+Grid+Engine+6+System

-------------- next part --------------
A non-text attachment was scrubbed...
Name: hung-sheng_tsao.vcf
Type: text/x-vcard
Size: 366 bytes
Desc: not available
Url : https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20081007/b30d82e1/hung-sheng_tsao.vcf

[Kevin Doman]

You can create two separate hostlists in addition to the @allhosts,
each new hostlist contain half of the compute nodes, then create two
additional queues. The all.q already default to the @allhosts group
and your other two queues each has one half of the cluster in it.

With SGE, you can do a lot of manipulation without affecting any
running jobs. Things like fair share would immediate affect queued
jobs, but not running jobs.

Try to learn qconf as it is very quick and you can work while the
system is under heavy load. Qmon requires GUI access and sometimes
slow under heavy load.

K.

[Geoff Scott]

Thanks for the suggestions...

I seem to be missing something obvious however. I have created two
host groups and two queues, each with half of the cluster resources.
If I load up one of the project queues, then execute jobs with out
specifying a queue (thereby using the @allhosts queue), I end up with
compute nodes that are overloaded. A node that can only execute one
job will be executing one job from the @allhosts queue and one job
from the project queue. I've poured through the documentation to try
and figure out how to keep this from happening, but haven't found
anything.

Any, and all, help would be greatly appreciated.

Geoff

[Alain Péan]

Hello Geoff,

I had the same problem and resolved it by limiting rhe number of jobs
per host to the number of cores. To do this, I used the graphic
interface to SGE, qmon. It is certaily feasible on a command line.

So, execute qmon, click on the 'Host Configuration Button', Then on the
'Exection Host' tab, select your node, and click 'Modify'.
On the 'Consumable/Fixed Attributes' tab, select 'slots', and set it to
the number of cores in your node.

If you have a lot a nodes, a script would be better, but perhaps another
one can give the corresponding command and options ?

Hope this help...

Alain

Geoff Scott a écrit :

--
==========================================================
Alain Péan - CETP/CNRS
Centre d'etude des Environnements Terrestre et Planetaires
Observatoire de Saint-Maur
4, av de Neptune, Bat. A
94100 Saint-Maur des Fossés
Tel : 01-45-11-42-39 - Fax : 01-48-89-44-33
==========================================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20081015/5790010f/attachment.html

[Nadya Williams]

Geoff,

having 2 host groups and 2 queues should be sufficient for your
specification.
Something like this sequence of steps will do it:
1. create 2 separate queues for your projects
qconf -aq project1
qconf -aq project2
2. create 2 separate host groups for your queues, put hlad of your
cluster nodes in each.
qconf -ahgrp @project1
qconf -ahgrp @project2

for a correct syntax to use see the output of the command
qconf -shgrp @allhosts

3. modify your queue configuration to use these hostlists
qconf -mq project1
qconf -mq project2

when the command opens a file in the editor,
(1) change the variable hostlist value from @allhosts to whatever is
appropriate choice for your queue.
(2) Also, change the slots variable value so that is shows only the
compute nodes
from that host group.

4. you can then add users that use different queues for specific
project jobs with.
Assuming that you have different set of users for each project :
qconf -au user1,user2,user3 project1
qconf -au user11,user12,user13 project2

by default, any local user can submit the job to the queue if the
user list is not specified.
Note in the above command that user names are separated only by
commas, no spaces.

These steps should give you 2 fully separate queues for your
projects. If all your jobs
fall for 2 projects then this is it. The above is simple and fast and
can be adjusted as needed.

If you need to have an original queue to run some other jobs a well
then you will need to play
with complex values for the queues configuration.

hope this helps.
Nadya

Nadya Williams University of Zurich
na...@oci.uzh.ch Winterthurerstrasse 190
Tel: +41 44 635 4296 CH-8057 Zurich
Fax: +41 44 635 6888 Switzerland