Qmon Download

[Brook Mithani]

Ihave a small cluster (around 15 nodes) that is currently using SGE

heavily. I have many, many batch jobs coming up for two different

projects. I need to allocate one half of my cluster to each project.

I thought the best way to do this would be set up two different

queues.I've searched all over and can't find a good write up of how to go

about this. The qconf command can get it done, but as to what I

should be entering for each variable in the queue file, I've no idea.

I also looked at Qmon, but this just seems to run qconf so I don't

really gain anything there except a little more documentation. I'm

very hesitant to monkey too much with the system while it's under such

a great load. I can't muck anything up.At any rate, I realize this is fairly pedestrian, but any help would

be much appreciated. Two queues with each queue only containing half

of the available computing nodes. Nothing fancy.Thanks in advance.Geoff

With SGE, you can do a lot of manipulation without affecting any

running jobs. Things like fair share would immediate affect queued

jobs, but not running jobs.Try to learn qconf as it is very quick and you can work while the

system is under heavy load. Qmon requires GUI access and sometimes

slow under heavy load.K.

I seem to be missing something obvious however. I have created two

host groups and two queues, each with half of the cluster resources.

If I load up one of the project queues, then execute jobs with out

specifying a queue (thereby using the @allhosts queue), I end up with

compute nodes that are overloaded. A node that can only execute one

job will be executing one job from the @allhosts queue and one job

from the project queue. I've poured through the documentation to try

and figure out how to keep this from happening, but haven't found

anything.Any, and all, help would be greatly appreciated.Geoff

I had the same problem and resolved it by limiting rhe number of jobs

per host to the number of cores. To do this, I used the graphic

interface to SGE, qmon. It is certaily feasible on a command line.So, execute qmon, click on the 'Host Configuration Button', Then on the

'Exection Host' tab, select your node, and click 'Modify'.

On the 'Consumable/Fixed Attributes' tab, select 'slots', and set it to

the number of cores in your node.If you have a lot a nodes, a script would be better, but perhaps another

one can give the corresponding command and options ?Hope this help...AlainGeoff Scott a crit :

having 2 host groups and 2 queues should be sufficient for your

specification.

Something like this sequence of steps will do it:

1. create 2 separate queues for your projects

qconf -aq project1

qconf -aq project2

2. create 2 separate host groups for your queues, put hlad of your

cluster nodes in each.

qconf -ahgrp @project1

qconf -ahgrp @project2

for a correct syntax to use see the output of the command

qconf -shgrp @allhosts3. modify your queue configuration to use these hostlists

qconf -mq project1

qconf -mq project2when the command opens a file in the editor,

(1) change the variable hostlist value from @allhosts to whatever is

appropriate choice for your queue.

(2) Also, change the slots variable value so that is shows only the

compute nodes

from that host group.4. you can then add users that use different queues for specific

project jobs with.

Assuming that you have different set of users for each project :

qconf -au user1,user2,user3 project1

qconf -au user11,user12,user13 project2by default, any local user can submit the job to the queue if the

user list is not specified.

Note in the above command that user names are separated only by

commas, no spaces.These steps should give you 2 fully separate queues for your

projects. If all your jobs

fall for 2 projects then this is it. The above is simple and fast and

can be adjusted as needed.If you need to have an original queue to run some other jobs a well

then you will need to play

with complex values for the queues configuration.hope this helps.

Nadya

Does anyone have knowledge on the inbound queuing functionality that relates to POSDM Tlog (e.g. transactions /posdw/qmon, /posdw/qdis)? From what I have heard this will be useful in reducing development costs and improving processing efficiency. However, before investigating on system I would like to have a clear idea of its requirements, functionality and limitations.

The Inbound queuing approach is used to support trickle feeding in POS DM. So when transaction coming in by trickle feed these are not created directly in POS DM but written into the inbound queue. Then you have to post the inbound queue records to generate the transactions in POS DM.

But it does not reduce development cost (maybe it can simplify the mapping, you do not have to spilt POS files containing more than one day/store combination) and it does not really improve processing efficiency.

Depending on the POS DM customizing settings (transaction: /N/POSDW/IMG) for the store sending the data: It is possible to post the incoming data immediately or to use the inbound queue of POS DM (transaction: /N/POSDW/QMON and /N/POSDW/QDIS) to post the data in a separate processing unit/step.

To identify if the imported message you are able to use the inbound queue of POS DM. The XML messages of the inbound service can get identified in the queue monitor with the fields "Key" that carries the global unique indentifier of the message instance and 'Object Type = XML_DOC'.

Both RFC interfaces (BAPI and EXT) and their related IDOCs, Services, ... will work with the Inbound Queue. From an interface perspective the only change for a user is that you can now send in multiple days and stores within one call.

The rules that applied to the "old interfaces" regarding potential errors are valid as before. So the system takes more or less whatever you send in and will raise errors when ISO code conversions are not possible or if you have entries that can not be clearly be assigne dto one transaction. Of course the locking issue will not occur but will be handeled in the second posting step. Encryption needs to be done before writing the data to the queue. (BADIs /POSDW/BAPIINPUT_PRE and PST are executed before posting to the queue)

No, there is a new monitoring transaction code /POSDW/QMON. Thiswill provide you with a view on the packages in the queue. If you are using the automated deletion of posted records (customizing) you will only sees packages in error or unprocessed. If you use the reorg report you will also find processed packages in there (reorg through /POSDW/REORG_TIBQ).

Queueing system is one of the most important component of the Coguepackage. Probably researchers around first-principles calculation arefamiliar with how to use queueing systems, however may not haveexperience how to build the queueing system. In fact, it is not verydifficult to install and configure a queueing system. In this section,installation and configuration of the Grid Engine software areexplained to take Ubuntu linux as an example of the operating system.

If the master node is located on the remote place, ssh -X is usedto login the master node. Then the graphics will be forwarded to yourcomputer. After opening qmon, if qmon reports someauthorization problem, then you should check the hostname written in/var/lib/gridengine/CELL_NAME/common/act_qmaster. If you modifythe hostname in it, you need to restart gridendinge-master by:

Have Anti-Virus or HIPS software installed? To avoid conflicts with Cloud Agent, ensure that you exclude the following files, directories, and processes from all security software installed on the system.

%ProgramFiles%\Qualys\QualysAgent - this is where the service and uninstall live. The service will create processes, so HIPS needs to make sure to unblock this action. This path is same for both x86 and x64-bit systems.

HKEY_LOCAL_MACHINE\SOFTWARE\Qualys - this is where breadcrumb information lives to merge agent and appliance scanner results. The agent needs c/r/w/d access here; setup needs to create the key; uninstall needs ability to delete the key.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\qmon - this is where the agent setup installs the driver into the system if Qualys File Integrity Monitoring (FIM) is activated or Self-protection is enabled or Qualys EDR is activated on the agent.

3a8082e126