testing failure at OSG

[Churamani Paudel]

Dear Experts: 

I wonder if OSG is running smoothly because when I tested my job it failed with following error that I got in the email (I have produced subsample in the ifarm for the quick analysis purpose). 

b"XROOTD is available for use if needed...


Detected bash shell
Radiator thickness set...
Polarization angle set...
Electron beam energy set...
Coherent peak set...
eBEAM energy set...
RCDB exe:
/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/rcdb/rcdb_0.06.00/rcnd
beam (on) current set...
Collimator size set...
WARNING: User requested GEN_MAX_ENERGY > eBEAM_ENERGY.  This is not possible. 
Setting GEN_MAX_ENERGY to eBEAM_ENERGY...
This job has been configured to run at:  -1 : scosg20.jlab.org
Job started:  Wed Jun 1 20:32:31 EDT 2022
Simulating the Experiment:  rcdb
ccdbsqlite path:  ./ sqlite:////tmp/TestProj_2544/ccdb.sqlite
rcdbsqlite path:  no_sqlite mysql://rc...@hallddb.jlab.org/rcdb
Producing file number: 50685
Containing:  0/file:/osgpool/halld/tbritton/REQUESTEDMC_CONFIGS/2544_jana.config events
Running location: ./
Output location: /osgpool/halld/tbritton/REQUESTEDMC_OUTPUT/TnP_eff_201808_MC
Environment file: 
/group/halld/www/halldweb/html/halld_versions/recon-2018_08-ver02_22.xml
Analysis Environment file: 
/group/halld/www/halldweb/html/halld_versions/analysis-2018_08-ver14.xml
Context: variation=500 calibtime=mc
Geometry URL: ccdb:///GEOMETRY/main_HDDS.xml
Reconstruction calibtime: none
Run Number: _20220601082615pm/
Electron beam current to use: unset uA
Electron beam energy to use: 0 GeV
Radiator Thickness to use: 0 m
Collimator Diameter: 0.0050 m
Photon Energy between rcdb and 0 GeV
Polarization Angle: -1.0 degrees
Coherent Peak position: 0
----------------------------------------------
Run generation step? notime  Will be cleaned? 1
Flux Hist to use:  0  :  ccdb
Polarization to use:  0  :  0.4
Using 1 with config:
/osgpool/halld/tbritton/REQUESTEDMC_CONFIGS/2544_eta958_pippimeta.in
Will run JLAB postprocessing after generator with configuration: No, event
definitions: Default and decay definitions: Default
----------------------------------------------
Run geant step? 1  Will be cleaned? 1
Using geantgenEtaRegge
Custom Gcontrol? None
Background to use: 4
Random trigger background to use: rcdb
BGRATE will be set to: rcdb GHz (if applicable)
Run mcsmear? 1  Will be cleaned? 1
----------------------------------------------
Run reconstruction? 1  Will be cleaned? 1
With additional plugins: I_dont_have_one
==============================================


=======SOFTWARE USED=======
MCwrapper version v0
MCwrapper location /scigroup/mcwrapper/gluex_MCwrapper/
LDPRELOAD:  /usr/lib64/libXrdPosixPreload.so
Streaming via xrootd? 1 Event Count: GlueX
BC /usr/bin/bc
python /usr/bin/python


/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/hdgeant4/hdgeant4-2.31.0^rec180827/bin/Linux-g++/hdgeant4
/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/halld_sim/halld_sim-4.35.0^rec180827/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin/mcsmear
/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/halld_recon/halld_recon-recon-2018_08-ver02.7/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin/hd_root


RECO PREFIX:   I_do
NO VALID GENERATOR GIVEN
only [genr8, bggen, genEtaRegge, gen_2pi_amp, gen_pi0, gen_omega_3pi, gen_2k,
bggen_jpsi, gen_ee, gen_ee_hb,  bggen_phi_ee, particle_gun, geantBEAM, genBH,
gen_omega_radiative, gen_amp, gen_compton, gen_npi, gen_compton_simple,
gen_primex_eta_he4, gen_whizard, gen_omegapi, mc_gen, gen_vec_ps] are supported
*********************************
Welcome to v2.6.2 of MCwrapper
Thomas Britton 05/25/22
*********************************
('WILL BE BUNDLED', False)
Locally simulating 500 2544_eta958_pippimeta Events
ending gluex_MC.py
"


Errors:


b"mkdir: cannot create directory './/_20220601082615pm/_50685': No such file or
directory
/scigroup/mcwrapper/gluex_MCwrapper//MakeMC.sh: line 214: cd:
.//_20220601082615pm/_50685: No such file or directory
cp: omitting directory './'
usage: rcnd.py [-h] [-c <srting>] [-v] [-l] [--create <name>]
               [--new-run <name>]
               [--type {bool,json,string,float,int,time,blob}]
               [-d DESCRIPTION] [--write <value>] [--replace] [--list-names]
               [run_number] [condition_name]
rcnd.py: error: argument run_number: invalid int value: '_20220601082615pm/'
usage: rcnd.py [-h] [-c <srting>] [-v] [-l] [--create <name>]
               [--new-run <name>]
               [--type {bool,json,string,float,int,time,blob}]
               [-d DESCRIPTION] [--write <value>] [--replace] [--list-names]
               [run_number] [condition_name]
rcnd.py: error: argument run_number: invalid int value: '_20220601082615pm/'
usage: rcnd.py [-h] [-c <srting>] [-v] [-l] [--create <name>]
               [--new-run <name>]
               [--type {bool,json,string,float,int,time,blob}]
               [-d DESCRIPTION] [--write <value>] [--replace] [--list-names]
               [run_number] [condition_name]
rcnd.py: error: argument run_number: invalid int value: '_20220601082615pm/'
usage: rcnd.py [-h] [-c <srting>] [-v] [-l] [--create <name>]
               [--new-run <name>]
               [--type {bool,json,string,float,int,time,blob}]
               [-d DESCRIPTION] [--write <value>] [--replace] [--list-names]
               [run_number] [condition_name]
rcnd.py: error: argument run_number: invalid int value: '_20220601082615pm/'
which: no 1 in
(/apps/cmake/cmake-3.5.1/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/gluex_root_analysis/gluex_root_analysis-1.20.0^rec180827/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/gluex_MCwrapper/gluex_MCwrapper-v2.5.1:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/hdgeant4/hdgeant4-2.31.0^rec180827/bin/Linux-g++:/home/tbritton/halld_my/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/halld_sim/halld_sim-4.35.0^rec180827/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/halld_recon/halld_recon-recon-2018_08-ver02.7/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/jana/jana_0.7.9p1^bs221/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/rcdb/rcdb_0.06.00:/group/halld/Soft
 ware/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/rcdb/rcdb_0.06.00/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/rcdb/rcdb_0.06.00/cpp/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/ccdb/ccdb_1.06.07/bin:/home/tbritton/geant4_workdir/bin/Linux-g++:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/geant4/geant4.10.02.p02/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/cernlib/2005/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/root/root-6.08.06^bs221/bin:/apps/perl/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
which: no JLAB in
(/apps/cmake/cmake-3.5.1/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/gluex_root_analysis/gluex_root_analysis-1.20.0^rec180827/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/gluex_MCwrapper/gluex_MCwrapper-v2.5.1:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/hdgeant4/hdgeant4-2.31.0^rec180827/bin/Linux-g++:/home/tbritton/halld_my/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/halld_sim/halld_sim-4.35.0^rec180827/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/halld_recon/halld_recon-recon-2018_08-ver02.7/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/jana/jana_0.7.9p1^bs221/Linux_CentOS7-x86_64-gcc4.8.5-cntr/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/rcdb/rcdb_0.06.00:/group/halld/S
 oftware/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/rcdb/rcdb_0.06.00/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/rcdb/rcdb_0.06.00/cpp/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/ccdb/ccdb_1.06.07/bin:/home/tbritton/geant4_workdir/bin/Linux-g++:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/geant4/geant4.10.02.p02/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/cernlib/2005/bin:/group/halld/Software/builds/Linux_CentOS7-x86_64-gcc4.8.5-cntr/root/root-6.08.06^bs221/bin:/apps/perl/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
cp: cannot stat 'None': No such file or directory
"

Any suggestions would be greatly appreciated, 

Kindest Regards

Churamani 

[Thomas Britton]

Something has gone wrong in configuration. Make sure any string contains no spaces or special characters (maybe a ‘ is used?) which screws up parsing

RECO PREFIX:   I_do
NO VALID GENERATOR GIVEN
only [genr8, bggen, genEtaRegge, gen_2pi_amp, gen_pi0, gen_omega_3pi, gen_2k,
bggen_jpsi, gen_ee, gen_ee_hb,  bggen_phi_ee, particle_gun, geantBEAM, genBH,
gen_omega_radiative, gen_amp, gen_compton, gen_npi, gen_compton_simple,
gen_primex_eta_he4, gen_whizard, gen_omegapi, mc_gen, gen_vec_ps] are supported

Is a clue

Thomas Britton

On Jun 1, 2022, at 9:01 PM, Churamani Paudel <cpau...@fiu.edu> wrote:



--
You received this message because you are subscribed to the Google Groups "GlueX Software Help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gluex-softwar...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gluex-software/CAMN%3DDsC9A6msJ-27m%3DAiPCJBgT_DNrQZa7qrV%3Dx7%2BPGyz_kpfg%40mail.gmail.com.

[Churamani Paudel]

Dr. Britten:

Yes I checked if it has happened and did not see, generator name is valid and provided from GUI while link to generator input file is also correct. genEtaRegge is well inside the list it mentions.  Towards the end  log, it says this as well: 

Errors:


b"mkdir: cannot create directory './/_20220601082230pm/_30274': No such file or
directory
/scigroup/mcwrapper/gluex_MCwrapper//MakeMC.sh: line 214: cd:
.//_20220601082230pm/_30274: No such file or directory
cp: omitting directory './'
usage: rcnd.py [-h] [-c <srting>] [-v] [-l] [--create <name>]
               [--new-run <name>]
               [--type {bool,json,string,float,int,time,blob}]
               [-d DESCRIPTION] [--write <value>] [--replace] [--list-names]
               [run_number] [condition_name]
rcnd.py: error: argument run_number: invalid int value: '_20220601082230pm/'
usage: rcnd.py [-h] [-c <srting>] [-v] [-l] [--create <name>]
               [--new-run <name>]
               [--type {bool,json,string,float,int,time,blob}]
               [-d DESCRIPTION] [--write <value>] [--replace] [--list-names]
               [run_number] [condition_name]
rcnd.py: error: argument run_number: invalid int value: '_20220601082230pm/'
usage: rcnd.py [-h] [-c <srting>] [-v] [-l] [--create <name>]
               [--new-run <name>]
               [--type {bool,json,string,float,int,time,blob}]
               [-d DESCRIPTION] [--write <value>] [--replace] [--list-names]
               [run_number] [condition_name]
rcnd.py: error: argument run_number: invalid int value: '_20220601082230pm/'
usage: rcnd.py [-h] [-c <srting>] [-v] [-l] [--create <name>]
               [--new-run <name>]
               [--type {bool,json,string,float,int,time,blob}]
               [-d DESCRIPTION] [--write <value>] [--replace] [--list-names]
               [run_number] [condition_name]
rcnd.py: error: argument run_number: invalid int value: '_20220601082230pm/'
which: no 1 in

so this seems to to error related to RCDB or something other? I think this could be an cause of crash. 

Churamani 

[Thomas Britton]

The parsing has gone awry. Whatever is in the generator variable is not valid, not because it is wrong in the drop down but because the parsing is wrong.  Some variable contains an invalid character (most likely).  For example:

Let’s say we have two variables x and y. They get parsed by the bash script (or csh) as “x y”

Now let’s say x= “my dir” and y = 5

“x y” becomes “my dir 5”…but this is 3 variables:

x=my

y=dir

z=5

So any assertion that y is an integer fails

directory './/_20220601082230pm/_30274

Looks garbled. I am suspect of the output directory field. I will take a look asasp

Thomas Britton

On Jun 1, 2022, at 10:42 PM, Churamani Paudel <cpau...@fiu.edu> wrote:



[Churamani Paudel]

Dr. Britton:

Thanks  a lot for your explanation and detection, it went through now. 

Churamani