WARNING: Parsing of '.cfg' files in pilot commands

[Luisa Arrabito]

Dear all,

I would like to report another issue that we observe in our pilots that was already present in py2 pilots.

All pilot commands raise the following warnings:

2021-10-12 14:04:16 UTC INFO [CheckCECapabilities] Executing command dirac-resource-get-parameters -S LCG.DESY-ZEUTHEN.de -N grid-htcondorce1.zeuthen.desy.de -Q share pilot.cfg -d
2021-10-12 14:04:16 UTC Framework WARN: Can't load /scratch/condor/dir_187744/DIRAC_YgzYzMpilot/diracos/etc/dirac.cfg file
WARNING: Parsing of '.cfg' files as command line arguments is changing!
          Set the environment variable 'export DIRAC_NO_CFG=1' to pass the file as a positional
          argument (this will become the default).

          To modify the local configuration use '--cfg <configfile>' instead.

For the moment the commands work fine, but I guess that it could be a problem in future. The problem is that this command line is built automatically. My understanding is that there is a check about the pilot version in order to add the --cfg option, as for instance here:

https://github.com/DIRACGrid/Pilot/blob/master/Pilot/pilotCommands.py#L483

The fact is that our case the pilot version is the one of the CTADIRAC extension, currently 2.0, so that the --cfg option is not added.

I've also seen that we could pass pilot.cfg as positional argument by setting DIRAC_NO_CFG=1, but I don't know if this will be maintained in future.

Could you please suggest a proper solution to get rid of these Warnings?

Thank you,

Luisa

[Federico Stagni]

Hi Luisa,

this warning 

WARNING: Parsing of '.cfg' files as command line arguments is changing!
          Set the environment variable 'export DIRAC_NO_CFG=1' to pass the file as a positional
          argument (this will become the default).

          To modify the local configuration use '--cfg <configfile>' instead.

is, as you said, not "dangerous". You can add the CTADIRAC version in your CTAPilot extension, if you have one, like it's done in https://gitlab.cern.ch/lhcb-dirac/LHCbPilot/-/blob/master/LHCbPilot/LHCbPilotCommands.py#L154 

Cheers,

Federico

--
You received this message because you are subscribed to the Google Groups "diracgrid-forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to diracgrid-for...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/diracgrid-forum/50629623-c292-4fb4-8d55-7cdd88e6a54an%40googlegroups.com.

[Luisa Arrabito]

Hi Federico,

Thank you for your answer.

However we don't have a CTAPilot extension, so I was just worried that in future the Warning becomes an Error...

Anyway, at some point we might be interested in using a CTAPilot extension.

I've already gave it some trial but without success.

Basically, I've added to the Operations/<Setup>/Pilot section:

pilotVORepo = pointing to the git with CTAPilot

Extensions = CTA

and tried to put some dummy commands in the CTAPilot repo, but it doesn't seem to work.

Am I missing something?

Thank you,

Luisa

[Luisa Arrabito]

Could you eventually give me the conf. of the Pilot section that you have in LHCb?

Thank you,

Luisa

[Federico Stagni]

Hi,

this warning is going to stay a warning for a long time, so you should not worry.

In LHCb we use:

    Pilot
    {
      CheckVersion = True
      Version = v10r2p13
      Version += v10r2p10
      Version += v10r2p9
      Version += 10.2.9
      Version += 10.2.13
      Project = LHCb
      Commands
      {
        Vac = NagiosProbes
        Vac += CheckWorkerNode
        Vac += LHCbInstallDIRAC
        Vac += LHCbConfigureBasics
        Vac += CheckCECapabilities
        Vac += LHCbAddCVMFSTags
        Vac += CheckWNCapabilities
        Vac += LHCbConfigureSite
        Vac += LHCbConfigureArchitecture
        Vac += LHCbConfigureCPURequirements
        Vac += LaunchAgent
        Vcycle = NagiosProbes
        Vcycle += CheckWorkerNode
        Vcycle += LHCbInstallDIRAC
        Vcycle += LHCbConfigureBasics
        Vcycle += CheckCECapabilities
        Vcycle += CheckWNCapabilities
        Vcycle += LHCbConfigureSite
        Vcycle += LHCbConfigureArchitecture
        Vcycle += LHCbConfigureCPURequirements
        Vcycle += MultiLaunchAgent
        Skygrid = CheckWorkerNode
        Skygrid += LHCbInstallDIRAC
        Skygrid += LHCbConfigureBasics
        Skygrid += LHCbConfigureSite
        Skygrid += LHCbConfigureArchitecture
        Skygrid += LHCbConfigureCPURequirements
        Skygrid += MultiLaunchAgent
        Online = CheckWorkerNode
        Online += LHCbInstallDIRAC
        Online += LHCbCleanPilotEnv
        Online += LHCbConfigureBasics
        Online += CheckCECapabilities
        Online += LHCbAddCVMFSTags
        Online += CheckWNCapabilities
        Online += LHCbConfigureSite
        Online += LHCbConfigureArchitecture
        Online += LHCbConfigureCPURequirements
        Online += LaunchAgent
        Defaults = CheckWorkerNode
        Defaults += LHCbInstallDIRAC
        Defaults += LHCbConfigureBasics
        Defaults += CheckCECapabilities
        Defaults += LHCbAddCVMFSTags
        Defaults += CheckWNCapabilities
        Defaults += LHCbConfigureSite
        Defaults += LHCbConfigureArchitecture
        Defaults += LHCbConfigureCPURequirements
        Defaults += LaunchAgent
      }
      pilotFileServer = lhcbdirac.s3.cern.ch/Pilot3
      pilotVORepo = https://gitlab.cern.ch/lhcb-dirac/LHCbPilot.git
      pilotVOScriptsPath = LHCbPilot
      workDir = /opt/dirac/work/Pilot3
    }

For explanation please check https://dirac.readthedocs.io/en/latest/AdministratorGuide/Systems/WorkloadManagement/Pilots/index.html

Cheers,

Federico

PS: I encourage you to try https://github.com/DIRACGrid/DIRAC/discussions instead of this forum, as it makes it easier to do cross-references.

To view this discussion on the web visit https://groups.google.com/d/msgid/diracgrid-forum/38e3784c-9760-46e3-b949-1581cad767cdn%40googlegroups.com.

[Luisa Arrabito]

Perfect. Thanks a lot!

And ok no problem, I will user the github discussions the next time.