Retention policy

[dpo...@shaklee.com]

Hello again,

In the examples, they had a job that would clean up the log files. My current deployment does not have that job in it. However, I was trying to see how far back I can pull history for. There seems to be some limit. I need to know what that limit is, how to change it, and what are the consequences of changing it.

I am being requested to be able to pull up logging information for about 12-18 months. Right now I seem to only be able to pull up about 10days worth of logging. Where is the rest of it going? Do I need to archive it someplace somehow?

Just to let you all know. You have been a huge help and I really appreciate all the answers you have provided me.

[Ronald Jeninga]

Hi Dan,

schedulix doesn't do log file management. This means that nothing is deleted automatically.
The GUI only shows jobs that have been loaded into memory (and not have been cleaned out). The *History* settings in the server.conf define which jobs to load.
But your database still contains the jobs that have ever run (unless you cleaned up the database, either by server.conf or manually).

Assuming you didn't clean up the database and didn't delete log files, you'll be able to match the jobs of 12 months ago with their log files.
The columns LOGFILE and ERRLOGFILE of SCI_SUBMITTED_ENTITY are what you are looking for.
But note: if those columns don't specify a full qualified path, you'll have to prepend the workdir of the job, which is the jobserver's default workdir if not specified.
The column SCOPE_ID points to the jobserver that executed the job. This will enable you to find the host that run the job.

A common practice is to use $JOBID in the name of the log file. That is a good way to find the file, even if the jobserver's workdir has changed in the meanwhile.

I am happy our answers are valuable for you and help you doing your job.

Regards,

Ronald

[dpo...@shaklee.com]

I feel like I might be being greedy here. But I do have SOX compliance to worry about.

Would there by any chance be someone somewhere that has already built some reporting that could pull all the running master jobs for day X with pertinent information and then using the job information, pull another report of all the jobs that ran under that master with pertinent information?

[Ronald Jeninga]

Hi Dan,

I assume you mean the Sarbanes Oxley Act in this case?

Well, everything is stored in the database and won't be deleted unless you tell the system to do so.
Hence by querying the database you'll find the desired information.

The base tables store the data the way the system needs it, but we defined a bunch of views that makes it somewhat easier to interpret the data, most notably the date/time columns.

To retrieve the masters, you'll have to search for rows in the SCI_SUBMITTED_ENTITY where MASTER_ID = ID.
To retrieve the jobs for a single master, you simply test for the MASTER_ID to be something.

Joining the SCI_SUBMITTED_ENTITY view with the corresponding definition (SCHEDULING_ENTITY) has to take the version into account.
Basically it would look like

SELECT whatever_you_want_to_know
  FROM SCI_SUBMITTED_ENTITY sme,
       SCI_V_SCHEDULING_ENTITY se
 WHERE sme.SE_ID = se.ID
   AND sme.SE_VERSION BETWEEN se.VALID_FROM and se.VALID_TO
   AND some other conditions;

The only supported way of changing data within the database is to advice the server to do so.
Manual changes are a fast method to make your system unusable.
Since the system doesn't know any methods to delete submitted entities, you have the entire history of your jobs within the database.
Hence, I think the SOX requirements are implementable. (I don't know the exact requirements, but from what I read we must be pretty close to what's demanded).

As a side note: If you use BICsuite Professional, you'll also have an audit trail, which keeps track of all manual interventions and a few other events. (who did what, at what time ...)
And in case of a manual intervention (cancel, rerun, ignore dependency, ignore resource, ...) the user can provide a short comment to explain the reason of his action. (... and why)

The protection of the database itself is an organisational issue. Schedulix can't prevent that some administrator does a "drop database", or throws the computer out of the window.
And the same applies to read/write access to the database which should be allowed to the DBA only, but this can't be enforced by schedulix.
Since this is true for every system, that can't be part of the technical SOX requirements.

And I don't think you're greedy. I think you understand why you want to use a scheduling system.

Regards,

Ronald

[Ronald Jeninga]

a small addition:

I said

> Well, everything is stored in the database and won't be deleted unless you tell the system to do so.

This refers to the possibility of archiving, which also allows to delete "old" jobs.
This is not enabled by default though and you need write access to the server.conf file to enable it.
Hence only a privileged user can enable it. If he spoils it, he's misusing his privileges and in his next job he should be the assistant of the administrator that throws computers out of windows.

Regards,

Ronald

[fab...@heavybyte.com]

Hi,

I also need this information, but for statical information. 

How can i get the final state, actually translate the field EXIT code, base on its exit state mapping?

Fábio

[Ronald Jeninga]

Hi Fabio,

I'm sorry, I don't quite understand what you mean.

If a job terminates, it is reported by the jobserver to the scheduling server.

The exit code is mapped to an exit state by the exit state mapping.

Then this exit state is merged into it's parent's state using the parent's exit state profile.

If you have a look at the output of a "show job", you'll find all relevant information:

[SYSTEM@localhost:2507] SDMS> show job 1511206;

Job

                   ID : 1511206

              SE_NAME : SYSTEM.TESTS.EVENTS.BATCH_EXAMPLE.JOB4

             SE_OWNER : ADMIN

              SE_TYPE : JOB

       SE_RUN_PROGRAM : 0

     SE_RERUN_PROGRAM : <null>

      SE_KILL_PROGRAM : <null>

           SE_WORKDIR : <null>

           SE_LOGFILE : ${JOBID}.log

         SE_TRUNC_LOG : false

        SE_ERRLOGFILE : ${JOBID}.log

      SE_TRUNC_ERRLOG : false

  SE_EXPECTED_RUNTIME : 0

          SE_PRIORITY : 50

  SE_SUBMIT_SUSPENDED : false

SE_MASTER_SUBMITTABLE : false

   SE_DEPENDENCY_MODE : AND

          SE_ESP_NAME : STANDARD

          SE_ESM_NAME : <default>

          SE_ENV_NAME : SERVER@LOCALHOST

           SE_FP_NAME : <null>

            MASTER_ID : 1511199

            CHILD_TAG : <null>

           SE_VERSION : 48594361

                OWNER : ADMIN

...

                STATE : FINAL

          IS_DISABLED : false

         IS_CANCELLED : false

           JOB_ESD_ID : SUCCESS

         JOB_ESD_PREF : 2

         JOB_IS_FINAL : true

   JOB_IS_RESTARTABLE : false

         FINAL_ESD_ID : SUCCESS

            EXIT_CODE : 0

          COMMANDLINE : 0

...

Job shown

To make it easier to read, I deleted some of the output.

Anyway, you have the information exit_code, job_esd_id and final_esd_id.

The exit_code is, eh, the exit code.

The job_esd_id is the exit state of the job itself; the mapped exit code.

The final_esd_id is the merged exit code of the job and it's children.

All this information is also present in the database (of course). OK, the job_esd_id and final_esd_id are in fact ID's of exit state definitions.

I think the required join is so simple, I don't need to write it down.

HTH

Regards,

Ronald

[fab...@heavybyte.com]

Thanks Ronald,

this is exact what i need, the field to write the join  a get de description,

Thanks