Skip to first unread message
doyel banerjee
Jan 2, 2017, 8:11:40 AM1/2/17
to schedulix
Hi,
I am trying to run schedulix job from command line but getting some error. Please suggest.
>sdms-submit --host $SDMSHOST --port $SDMSPORT --verbose --job PARAMETER_TEST_PRG
Executing command:
GET SUBMITTAG
Feedback: Got Submittag
Executing command:
SUBMIT PARAMETER_TEST_PRG
WITH
SUBMITTAG = 'BIC#303102919'
ERROR:01111281413,SUBMIT PARAMETER_TEST_PRG
WITH
SUBMITTAG = 'BIC#303102919';
Syntax Error: expected a user command
I also passed the parameters with the job but still the same error.These parameters are already defined in the job.
>sdms-submit --host $SDMSHOST --port $SDMSPORT --verbose --job PARAMETER_TEST_PRG $PGDATABASE $PGHOST $PGPORT
Executing command:
GET SUBMITTAG
Feedback: Got Submittag
Executing command:
SUBMIT PARAMETER_TEST_PRG
WITH
SUBMITTAG = 'BIC#303099421'
ERROR:01111281413,SUBMIT PARAMETER_TEST_PRG
WITH
SUBMITTAG = 'BIC#303099421';
Syntax Error: expected a user command
Ronald Jeninga
Jan 2, 2017, 9:28:11 AM1/2/17
to schedulix
Hi Doyel,
first of all you need a full qualified job name. If you only specify the last part of the name, the server thinks that a job is trying to submit a job. (the name is interpreted as an alias).
Since you opened a user connection, job commands aren't allowed.
Second, if you specify parameters, you'll have to tell the server the names as well.
Hence, something like
>sdms-submit --host $SDMSHOST --port $SDMSPORT --verbose --job SYSTEM.SOME_FOLDER.PARAMETER_TEST_PRG DB $PGDATABASE DBHOST $PGHOST DBPORT $PGPORT
should do it, though it assumes that all used environment variables (SDMSHOST, SDMSPORT, PGDATABASE, PGHOST and PGPORT) are set.
If the parameters are already set, there will be no point in specifying them again (unless you want to set or override them).
HTH
Regards,
Ronald
PS. Happy New Year :-)
first of all you need a full qualified job name. If you only specify the last part of the name, the server thinks that a job is trying to submit a job. (the name is interpreted as an alias).
Since you opened a user connection, job commands aren't allowed.
Second, if you specify parameters, you'll have to tell the server the names as well.
Hence, something like
>sdms-submit --host $SDMSHOST --port $SDMSPORT --verbose --job SYSTEM.SOME_FOLDER.PARAMETER_TEST_PRG DB $PGDATABASE DBHOST $PGHOST DBPORT $PGPORT
should do it, though it assumes that all used environment variables (SDMSHOST, SDMSPORT, PGDATABASE, PGHOST and PGPORT) are set.
If the parameters are already set, there will be no point in specifying them again (unless you want to set or override them).
HTH
Regards,
Ronald
PS. Happy New Year :-)
doyel banerjee
Jan 2, 2017, 3:26:28 PM1/2/17
to schedulix
Thanks a lot for the explanation. It worked. I am able to call a child job using sdms-submit command inside a shell script. Here I am capturing some value say envir_id and want to pass the value of this parameter to the child job. Now I am stuck . I am using the below command in the run box .
./applications/common/scripts/call_child.ksh
The script is as below
more call_child.ksh
#!/bin/ksh
sdms-get_variable -h $SDMSHOST -p $SDMSPORT -j $JOBID -k $KEY -n ENVIR_ID
echo $ENVIR_ID
I am getting the error : ERROR:03209231453,Couldn't resolve Parameter ENVIR_ID
Please help.
Ronald Jeninga
Jan 3, 2017, 6:13:26 AM1/3/17
to schedulix
Hi Doyel,
in order to be able to retrieve the value of ENVIR_ID, the parameter must be known to the system.
It should be defined in the parent job/batch.
If it isn't declared and you submit the parent job, thereby specifying the parameter, it will be regarded a local parameter which is not inherited by the children.
AFAIK the sdms-get_variable utility writes the content of the parameter (if found of course) to stdout.
Your script should look like:
HTH
Ronald
in order to be able to retrieve the value of ENVIR_ID, the parameter must be known to the system.
It should be defined in the parent job/batch.
If it isn't declared and you submit the parent job, thereby specifying the parameter, it will be regarded a local parameter which is not inherited by the children.
AFAIK the sdms-get_variable utility writes the content of the parameter (if found of course) to stdout.
Your script should look like:
#!/bin/ksh
ENVIR_ID=`sdms-get_variable -h $SDMSHOST -p $SDMSPORT -j $JOBID -k $KEY -n ENVIR_ID`
echo $ENVIR_ID
HTH
Ronald
doyel banerjee
Jan 3, 2017, 7:18:46 AM1/3/17
to schedulix
Thanks Ronald for your help. As per your suggestion, I declared a parameter ENVIR_ID with default value NONE in my parent job and ran the parent job and made the changes in the child job as recommended by you. Also declared the parameter in my child job with default value NONE . The job is getting submitted now but ENVIR_ID is still blank. I might be doing some silly mistake but cant trace.
Ronald Jeninga
Jan 3, 2017, 7:41:23 AM1/3/17
to schedulix
Hi Doyel,
let's try a simple example first.
In this simple example the child job is only interested in retrieving the value from the parent job.
This means that the type of parameter in the child job should be IMPORT in this case.
(Side note: as long as the parameter is not set, IMPORT and PARAMETER as a type will show the same behaviour. The difference is that the value of a PARAMETER can be set, the value of an IMPORT can't).
We start with a simple example first. So you define the parent parameter to be a CONSTANT with some value.
This should work. If not, we'll have to find out why. And if it works we'll do a next step and try to construct a more dynamic example.
BTW. since you got a valid response (empty string means: Parameter has no value), we're nearly there already.
Regards,
Ronald
let's try a simple example first.
In this simple example the child job is only interested in retrieving the value from the parent job.
This means that the type of parameter in the child job should be IMPORT in this case.
(Side note: as long as the parameter is not set, IMPORT and PARAMETER as a type will show the same behaviour. The difference is that the value of a PARAMETER can be set, the value of an IMPORT can't).
We start with a simple example first. So you define the parent parameter to be a CONSTANT with some value.
This should work. If not, we'll have to find out why. And if it works we'll do a next step and try to construct a more dynamic example.
BTW. since you got a valid response (empty string means: Parameter has no value), we're nearly there already.
Regards,
Ronald
doyel banerjee
Jan 3, 2017, 9:08:16 AM1/3/17
to schedulix
I defined the parent parameter as CONSTANT with some value say 777 and child parameter as PARAMETER. It worked.
Then changed the parent parameter as IMPORT with no value and child as PARAMETER. It didnt work, ENVIR_ID is empty.
.
Also tried changing the parent to PARAMETER and child to IMPORT. No luck.
Ronald Jeninga
Jan 3, 2017, 9:35:22 AM1/3/17
to schedulix
Hi Doyel,
changing the parent parameter to type IMPORT doesn't make sense. You don't want to import it, you want to give it a fixed value, specify this value at submit time, or set it at run time.
If you change it from CONSTANT to PARAMETER _and_ specify a default value, it'll still work.
If you have it working, parent job with a PARAMETER and a default value, and a child job with an IMPORT, you can submit the parent and specify the value of the parameter at submit time.
If this works as well (it should), we can go a step further and try to set values at run time.
Regards,
Ronald
changing the parent parameter to type IMPORT doesn't make sense. You don't want to import it, you want to give it a fixed value, specify this value at submit time, or set it at run time.
If you change it from CONSTANT to PARAMETER _and_ specify a default value, it'll still work.
If you have it working, parent job with a PARAMETER and a default value, and a child job with an IMPORT, you can submit the parent and specify the value of the parameter at submit time.
If this works as well (it should), we can go a step further and try to set values at run time.
Regards,
Ronald
doyel banerjee
Jan 3, 2017, 10:13:06 AM1/3/17
to schedulix
"You don't want to import it, you want to give it a fixed value, specify this value at submit time, or set it at run time.
If you change it from CONSTANT to PARAMETER _and_ specify a default value, it'll still work." -- this is working
"If you have it working, parent job with a PARAMETER and a default value, and a child job with an IMPORT, you can submit the parent and specify the value of the parameter at submit time." -- this is also working
waiting for your next instructions :)
Ronald Jeninga
Jan 3, 2017, 11:50:18 AM1/3/17
to schedulix
Hi Doyel,
so we achieved a lot so far. You know now how to get parameter values from the parent.
The next step isn't as much intended to set a generic PATH variable (that's what you started with), but more to hand over information from one process to another.
In this case we will still use the same parent/child hierarchy.
The situation is that some parent job determines a value and wants to pass it to one or more children.
In the simplest case (which we have), there's only a single child, but it is easy to add more children.
The hierarchy looks like:
P
|
C
But we have a synchronisation problem now. Before the Child can retrieve the value of the parameter, it must be set first.
Hence, we must make sure that the parent runs first.
The easiest way to achieve that is to create a dependency. The Child should wait for the Parent.
Normally dependencies are created with an ALL_FINAL mode. This means that dependent jobs will have to wait until the required job _and_ all of its children have reached a final state.
If we would apply this to our situation, we would have a deadlock, since the child would first run after it has finished. This is clearly impossible.
The solution is to choose the JOB_FINAL mode for the dependency. This dependency will be fulfilled if the required job itself reaches a final state. Any children are ignored.
The hierarchy would then look like
P<--+
| |
C---+
Now you can set the parameter within the parent job (e.g. sdms-set_parameter), and the child job can retrieve its value afterwards.
The parameter in the parent job can be a RESULT (not known in advance, but will be set at runtime) or a PARAMETER (might be known in advance or at submit time and might be set at runtime).
The child job should declare its parameter as IMPORT (declared to be used, but it will be inherited from somewhere else).
Ronald
so we achieved a lot so far. You know now how to get parameter values from the parent.
The next step isn't as much intended to set a generic PATH variable (that's what you started with), but more to hand over information from one process to another.
In this case we will still use the same parent/child hierarchy.
The situation is that some parent job determines a value and wants to pass it to one or more children.
In the simplest case (which we have), there's only a single child, but it is easy to add more children.
The hierarchy looks like:
P
|
C
But we have a synchronisation problem now. Before the Child can retrieve the value of the parameter, it must be set first.
Hence, we must make sure that the parent runs first.
The easiest way to achieve that is to create a dependency. The Child should wait for the Parent.
Normally dependencies are created with an ALL_FINAL mode. This means that dependent jobs will have to wait until the required job _and_ all of its children have reached a final state.
If we would apply this to our situation, we would have a deadlock, since the child would first run after it has finished. This is clearly impossible.
The solution is to choose the JOB_FINAL mode for the dependency. This dependency will be fulfilled if the required job itself reaches a final state. Any children are ignored.
The hierarchy would then look like
P<--+
| |
C---+
Now you can set the parameter within the parent job (e.g. sdms-set_parameter), and the child job can retrieve its value afterwards.
The parameter in the parent job can be a RESULT (not known in advance, but will be set at runtime) or a PARAMETER (might be known in advance or at submit time and might be set at runtime).
The child job should declare its parameter as IMPORT (declared to be used, but it will be inherited from somewhere else).
Ronald
doyel banerjee
Jan 3, 2017, 2:59:25 PM1/3/17
to schedulix
Thanks Ronald for the detailed post. It seems I am missing out on something. The parent job has exitstate FAILURE (not sure why. It was able to do sdms-submit successfully) with state BROKEN_FINISHED and the child job is DEPENDENCY_WAIT ( this is as expected waiting for dependent object to finish first). I am sharing the snapshots.
1. Image 1 shows the hierarchy.
2. Image 2 shows different tabs of parent job.
3. Image 3 shows different tabs of child job.
4. Image 4 shows the log for submitted entity.
The log file reads as the jobid of child job and the error log file say "feedback: job submitted".
Ronald Jeninga
Jan 4, 2017, 5:56:07 AM1/4/17
to schedulix
Hi Doyel,
it is the BROKEN_FINISHED that has to be investigated. (The rest seems to work quite well, from my point of view).
If a job is executed by a jobserver, first of all a jobexecutor process is spawned. The jobexecutor process executes the job's run program.
The communication between jobserver and jobexecutor is done via the taskfile.
The reason for this construction is that a jobserver can be shut down an restarted without affecting running jobs. The parent process is always the jobexecutor.
On termination of the user's process, the jobexecutor will do a wait() to retrieve the exit code. If the jobexecutor dies, the exit code will be lost (swallowed by the init process ;).
Hence the health of the jobexecutor process is crucial.
If the jobexecutor terminates prematurely, the jobserver will detect this and set the state of the (still) running job to BROKEN_ACTIVE. If the user's process terminates, the jobserver will set the state to BROKEN_FINISHED.
Since your job shows a BROKEN_FINISHED state, the jobexecutor must have vanished somehow. Either the process died, or the jobserver failed to recognize the jobexecutor.
The latter can happen if you configured something else than BOOTTIME=NONE in the jobserver's configuration. The former can happen if some administrator kills the jobexecutor, the Linux OOM hits, or the program run into some bug and crashed.
Another way to confuse the system is to store the taskfiles (see the TASKFILEPREFIX in the jobserver's configuration) on an NFS mounted device. Since both the jobexecutor and the jobserver evaluate the contents of the taskfile regularly, it is extremely important that the file is in a consistent state. Unfortunately this can't be guaranteed with NFS. The bottom line is: if you love nonsensical problems, store the taskfiles on an NFS mounted device.
The best thing to do now is to cancel your job and retry it. You can cancel the job via the GUI (RIP button, yellow), or by sdmsh (ALTER JOB 97563 WITH CANCEL;).
But before you retry, please check the BOOTTIME and the TASKFILEPREFIX configuration, as explained above.
HTH
Regards,
Ronald
it is the BROKEN_FINISHED that has to be investigated. (The rest seems to work quite well, from my point of view).
If a job is executed by a jobserver, first of all a jobexecutor process is spawned. The jobexecutor process executes the job's run program.
The communication between jobserver and jobexecutor is done via the taskfile.
The reason for this construction is that a jobserver can be shut down an restarted without affecting running jobs. The parent process is always the jobexecutor.
On termination of the user's process, the jobexecutor will do a wait() to retrieve the exit code. If the jobexecutor dies, the exit code will be lost (swallowed by the init process ;).
Hence the health of the jobexecutor process is crucial.
If the jobexecutor terminates prematurely, the jobserver will detect this and set the state of the (still) running job to BROKEN_ACTIVE. If the user's process terminates, the jobserver will set the state to BROKEN_FINISHED.
Since your job shows a BROKEN_FINISHED state, the jobexecutor must have vanished somehow. Either the process died, or the jobserver failed to recognize the jobexecutor.
The latter can happen if you configured something else than BOOTTIME=NONE in the jobserver's configuration. The former can happen if some administrator kills the jobexecutor, the Linux OOM hits, or the program run into some bug and crashed.
Another way to confuse the system is to store the taskfiles (see the TASKFILEPREFIX in the jobserver's configuration) on an NFS mounted device. Since both the jobexecutor and the jobserver evaluate the contents of the taskfile regularly, it is extremely important that the file is in a consistent state. Unfortunately this can't be guaranteed with NFS. The bottom line is: if you love nonsensical problems, store the taskfiles on an NFS mounted device.
The best thing to do now is to cancel your job and retry it. You can cancel the job via the GUI (RIP button, yellow), or by sdmsh (ALTER JOB 97563 WITH CANCEL;).
But before you retry, please check the BOOTTIME and the TASKFILEPREFIX configuration, as explained above.
HTH
Regards,
Ronald
doyel banerjee
Jan 5, 2017, 1:58:08 AM1/5/17
to schedulix
Thanks Ronald for the detailed explanation. I have three issues now. Will come to each one in the order of occurrence.
I am running schedulix2.6 on linux redhat and using ext4 filesystem.
1. To fix the BROKEN_FINISHED, I added the taskfile path in my jobserver ( which was missing previously) and set BOOTTIME to blank ( which was set to BLANK and here I misunderstood ) and RIPed the job. Triggered my job and it was SUCCESS with state FINAL. But ENVIR_ID was still empty.
2. So this time I changed the parameter type of ENVIR_ID in parent job to RESULT and reran the job. It was BROKEN_FINISHED and child was in DEPENDENCY_WAIT. Was analyzing the cause for BROKEN_FINISHED and set BOOTTIME to NONE and submitted the job again . But no luck. The properties tab of jobserver shows some error. Attached is the screenshot. (The screenshot now shows disconnected but it was connected then)
3. This time I restarted my jobserver and tried the same . Now my job is in RUNNABLE state. I checked if my jobserver is up.
/opt/app/dataservices/schedulix/bin>ps -ef |grep jobserver|grep localhost
514 30988 1 0 00:18 ? 00:00:00 /opt/app/dataservices/schedulix/bin/scrolllog /opt/app/dataservices/log/localhost.out -e /usr/bin/java -client -XX:ParallelGCThreads=2 -cp /opt/app/dataservices/schedulix/lib/BICsuite.jar:/usr/share/java/postgresql-9.4-1200.jdbc41.jar:/usr/share/java/jna.jar:/usr/lib/java/swt-linux-gtk-x64.jar de.independit.scheduler.jobserver.JobServer /opt/app/dataservices/etc/localhost.conf jobserver
514 30989 30988 0 00:18 ? 00:00:03 /usr/bin/java -client -XX:ParallelGCThreads=2 -cp /opt/app/dataservices/schedulix/lib/BICsuite.jar:/usr/share/java/postgresql-9.4-1200.jdbc41.jar:/usr/share/java/jna.jar:/usr/lib/java/swt-linux-gtk-x64.jar de.independit.scheduler.jobserver.JobServer /opt/app/dataservices/etc/localhost.conf jobserver
Ronald Jeninga
Jan 5, 2017, 5:34:30 AM1/5/17
to schedulix
Hi Doyel,
the BOOTTIME must be set to NONE, not blank. And that's also what the jobserver is complaining about in its error message.
In fact it'll be best to set the BOOTTIME to NONE on GLOBAL level. Don't forget to set the inherit flag on all lower levels.
Does the parent job _set_ the parameter? You can check the job and look at the parameters.
If I find the time, I'll build a small working example for you. I don't (and can't) promise anything though.
Regards,
Ronald
the BOOTTIME must be set to NONE, not blank. And that's also what the jobserver is complaining about in its error message.
In fact it'll be best to set the BOOTTIME to NONE on GLOBAL level. Don't forget to set the inherit flag on all lower levels.
Does the parent job _set_ the parameter? You can check the job and look at the parameters.
If I find the time, I'll build a small working example for you. I don't (and can't) promise anything though.
Regards,
Ronald
Ronald Jeninga
Jan 5, 2017, 6:37:53 AM1/5/17
to schedulix
Hi Doyel,
I built some examples. If you download the attached file (it is in ASCII, human readable format) and pipe it through sdmsh, it'll load the examples into your system.
Please have a look at the file in advance. You might want to change the environment. I used "SERVER@LOCALHOST" from our examples, but I don't know if you installed them.
And please try to understand what's happening. You might want to read the file, or load it and look at the examples in the GUI (SYSTEM.DOYEL folder).
I created these examples on a v2.7. I edited the commands in an attempt to eliminate the newer keywords, but it is possible it won't install. In that case please give me the error message and I'll repair the file.
HTH
Regards,
Ronald
I built some examples. If you download the attached file (it is in ASCII, human readable format) and pipe it through sdmsh, it'll load the examples into your system.
Please have a look at the file in advance. You might want to change the environment. I used "SERVER@LOCALHOST" from our examples, but I don't know if you installed them.
And please try to understand what's happening. You might want to read the file, or load it and look at the examples in the GUI (SYSTEM.DOYEL folder).
I created these examples on a v2.7. I edited the commands in an attempt to eliminate the newer keywords, but it is possible it won't install. In that case please give me the error message and I'll repair the file.
HTH
Regards,
Ronald
doyel banerjee
Jan 5, 2017, 12:40:57 PM1/5/17
to schedulix
Thankyou so very much. I know I keep pestering you so much and unfortunately will be doing so for sometime more. Hopefully I will be in a better understanding of Schedulix by the time this is done. Good news is we are going full fledged with Schedulix as our scheduling server for production.
doyel banerjee
Jan 5, 2017, 2:08:05 PM1/5/17
to schedulix
I see you have used parameters differently in all three cases. Though the intention to capture the parameter in parent job remains the same. I just want to know the implications of using these parameters in this way.
Ronald Jeninga
Jan 6, 2017, 5:04:58 AM1/6/17
to schedulix
Hi Doyel,
well I created the examples the way I would implement it if it were my task.
In example1, the static parent to child, I used a parameter of type IMPORT to document that the job depends on this value. I used a PARAMETER in the parent job with a default value to show that the value is set in the parent job. If you change the run program to "true" it'll still work, but the child will get the default value. You can also change the type to RESULT and it'll still work.
The main differences between PARAMETER and RESULT types are:
- A PARAMETER value must be known at submit time (hence the default value), a RESULT is expected to be set at run time
- A PARAMETER will behave like an IMPORT if not set, a RESULT won't
- A PARAMETER can be set at submit time, a RESULT can't
If I were sure that the value would be set by the run program, I'd use a RESULT.
In example2, we have a different situation. The parameter value is set by a job next to the job that needs it. Walking up the hierarchy won't find the parameter.
That's why the "consumer" defines a REFERENCE. This forces the system to search for a job next to the referencing job. In fact the same search algorithm as for resolving dependencies is used.
The parameter in the"producer" job is of type RESULT as explained above.
In the third example, I don't bother setting some parameter in the parent, but I use the possibility to specify parameter values at submit time.
Specifying the parameter is mandatory (by requirement). Hence I use a PARAMETER _without_ a default value in the child. A submit without specifying the parameter value will fail.
Regards,
Ronald
well I created the examples the way I would implement it if it were my task.
In example1, the static parent to child, I used a parameter of type IMPORT to document that the job depends on this value. I used a PARAMETER in the parent job with a default value to show that the value is set in the parent job. If you change the run program to "true" it'll still work, but the child will get the default value. You can also change the type to RESULT and it'll still work.
The main differences between PARAMETER and RESULT types are:
- A PARAMETER value must be known at submit time (hence the default value), a RESULT is expected to be set at run time
- A PARAMETER will behave like an IMPORT if not set, a RESULT won't
- A PARAMETER can be set at submit time, a RESULT can't
If I were sure that the value would be set by the run program, I'd use a RESULT.
In example2, we have a different situation. The parameter value is set by a job next to the job that needs it. Walking up the hierarchy won't find the parameter.
That's why the "consumer" defines a REFERENCE. This forces the system to search for a job next to the referencing job. In fact the same search algorithm as for resolving dependencies is used.
The parameter in the"producer" job is of type RESULT as explained above.
In the third example, I don't bother setting some parameter in the parent, but I use the possibility to specify parameter values at submit time.
Specifying the parameter is mandatory (by requirement). Hence I use a PARAMETER _without_ a default value in the child. A submit without specifying the parameter value will fail.
Regards,
Ronald
Reply all
Reply to author
Forward
0 new messages