Can't open the file "mainparams" - except that it can

[steve kimble]

Dear Group:

On a UNIX cluster, I'm trying to run structure, which works fine
during small test runs on the front end. However, when I submit it via
qsub, I get the can't open file "mainparams" message.

From my working directory:

-bash-3.2$ pwd
/scratch/scratch96/s/skimble/STRUCTURE

I have the necessary files:

-bash-3.2$ ls
extraparams infile mainparams structurejobsubmissionfile

and I've loaded structure:

-bash-3.2$ module list
Currently Loaded Modulefiles:
1) java/1.6.0_14 2) structure/2.3

and when I do a small test run:

-bash-3.2$ structure

it works fine and prints the outfile_f just fine.

But when I try to submit it to the cluster with qsub:

-bash-3.2$ qsub -l select=1:ncpus=1,walltime=60:00:00
structurejobsubmissionfile

in the output file that the cluster generates I get:

----------------------------------------------------
STRUCTURE by Pritchard, Stephens and Donnelly (2000)
and Falush, Stephens and Pritchard (2003)
Code by Pritchard, Falush and Hubisz
Version 2.3.2.1 (Oct 2009)
----------------------------------------------------

Can't open the file "mainparams".
Exiting the program due to error(s) listed above.

I would suspect the formatting of mainparams except that it works fine
when called from the front end. And I think the
structurejobsubmissionfile works because structure does appear to
load, as seen in the cluster's output file.

Any suggestions?

Thanks!
Steve

contents of mainparams file:
#define OUTFILE /usr/rmt_share/scratch96/s/skimble/STRUCTURE/outfile
#define INFILE /usr/rmt_share/scratch96/s/skimble/STRUCTURE/infile
#define NUMINDS 1558
#define NUMLOCI 11
#define LABEL 0
#define POPDATA 0
#define POPFLAG 0
#define LOCDATA 0
#define PHENOTYPE 0
#define MARKERNAMES 0
#define MAPDISTANCES 0
#define ONEROWPERIND 1
#define PHASEINFO 0
#define PHASED 0
#define RECESSIVEALLELES 0
#define EXTRACOLS 0
#define MISSING 000
#define PLOIDY 2
#define MAXPOPS 3
#define BURNIN 1000
#define NUMREPS 5000


#define NOADMIX 0
#define LINKAGE 0
#define USEPOPINFO 0

#define LOCPRIOR 0
#define INFERALPHA 1
#define ALPHA 1.0
#define POPALPHAS 0
#define UNIFPRIORALPHA 1
#define ALPHAMAX 10.0
#define ALPHAPROPSD 0.025

#define FREQSCORR 0
#define LAMBDA 1.0
#define COMPUTEPROB 1
#define PFROMPOPFLAGONLY 0
#define ANCESTDIST 1
#define NUMBOXES 1000
#define ANCESTPINT 0.9
#define STARTATPOPINFO 0
#define METROFREQ 10


#define UPDATEFREQ 1

[Vikram Chhatre]

Steve,

Could it be due to inappropriate file permissions? A different
system user may be executing the qsub than when you execute structure
directly. Just a thought.

V

> --
> You received this message because you are subscribed to the Google Groups "structure-software" group.
> To post to this group, send email to structure...@googlegroups.com.
> To unsubscribe from this group, send email to structure-softw...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/structure-software?hl=en.
>

[steve kimble]

Vikram:

Thanks for your post. I don't think so: the cluster is at a
University, with lots of users, and there are firewalls between them.
When, for example, I go to qdel a job if I accidentally try to kill
some other user's job it tells me I don't have permission. Plus I
write a place=excl argument into the qsub command so my jobs have the
node to themselves. Plus, this happens over and over.

Thanks,
Steve

[Vikram Chhatre]

Steve -

Sorry I wasn't very clear in the last post. What I meant to say was
that perhaps the qsub daemon uses a different user account (system
user, not some other individual on the cluster). But of course, now
that I think about it, if permissions was a problem, qsub wouldn't be
able to execute structure either.

Just out of curiosity, what are the permissions on the 'mainparams'
file? I remember seeing a similar problem posted to the list a few
weeks ago, but can't seem to recall what the solution was.

V

[steve kimble]

mainparams shows rw-r--r--

[Vikram Chhatre]

As you probably already know, that says: root: read write; user: read
only: group: read only.

Should be rwxrw-rw-.

Although this should still not stop it from being read by structure.
What user does the file belong to?

Please post the full output of:

$> ls -lha STRUCTURE/

Vikram

[steve kimble]

Vikram:

I finally got it work by adding cd $PBS_O_WORKDIR to the submission
file so that it read:

module load structure
cd $PBS_O_WORKDIR
structure

which had no problem with the mainparams file.

-bash-3.2$ ls -lha /scratch/scratch96/s/skimble/STRUCTURE/ gives:
total 4.0M
drwxr-xr-x 92 skimble fnr 14K Feb 15 13:22 .
drwx------ 15 skimble tgusers 6.0K Feb 14 21:44 ..
-rw-r--r-- 1 skimble fnr 2 Feb 8 18:33 extraparams
-rw-r--r-- 1 skimble fnr 134K Feb 3 14:48 infile
drwxr-xr-x 2 skimble fnr 2.0K Feb 15 13:04 k=12.1
drwxr-xr-x 2 skimble fnr 2.0K Feb 15 13:04 k=12.2
...
drwxr-xr-x 2 skimble fnr 2.0K Feb 15 13:11 k=25.5
drwxr-xr-x 2 skimble fnr 2.0K Feb 15 13:11 k=25.9
-rw-r--r-- 1 skimble fnr 992 Feb 15 13:10 mainparams
-rw------- 1 skimble fnr 639K Feb 15 13:39 outfile_f
-rw------- 1 skimble fnr 201K Feb 15 13:39 outfile_q
-rw-r--r-- 1 skimble fnr 77 Feb 14 14:33
structurejobsubmissionfile

Thanks!
Steve