Running moose/scripts/update_and_rebuild_petsc.sh on HPC
Tomas Mondragon
Fande Kong
--
You received this message because you are subscribed to the Google Groups "moose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/277eb13a-0590-4b1a-a089-09a7c35efd83%40googlegroups.com.
Tomas Mondragon
Fande Kong
I have attached the configure.log file from when I ran update_and_rebuild_petsc.sh on both machines. In the log from jim is from an attempt to rebuild after adding --with-mpiexec=mpiexec_mpt to the call to pets/configure in the update and rebuild script
--
You received this message because you are subscribed to the Google Groups "moose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/edd3a169-5012-4e46-a060-5088a199603c%40googlegroups.com.
Tomas Mondragon
Fande Kong
Yes, but now that I have tried this a couple of different ways with different --with-mpiexec options, I am beginning to suspect that I need to run this as a PBS job on compute nodes rather than a shell script on login nodes. Also, running it with --with-mpiexec="mpirun -n 1" and with --with-mpiexec="path/to/mpirun" don't get me results any different from using --with-mpiexec="mpirun"I have attached my modified update_and_rebuild_petsc.sh scripts from both machines plus their respective configure.log files.
--
You received this message because you are subscribed to the Google Groups "moose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/641b2c64-0e88-47e7-b33a-e2528287095a%40googlegroups.com.
Tomas Mondragon
Tomas Mondragon
wget https://bitbucket.org/petsc/pkg-fblaslapack/get/origin/barry/2019-08-22/fix-syntax-for-nag.tar.gz
wget https://bitbucket.org/petsc/pkg-fblaslapack/get/d28880efc974.zip
wget https://bitbucket.org/petsc/pkg-fblaslapack/get/d28880efc974.tar.gz
wget https://bitbucket.org/petsc/pkg-fblaslapack/get/fix-syntax-for-nag.tar.gz
wget https://bitbucket.org/petsc/pkg-fblaslapack/get/master.tar.gz
It may be because of the directory structure extracted from the fblaslapack tarball. Anyhow, the urls for the metis, parmetis, and scotch tarballs are the same ones that the configure script attempts to download from. Except for some reason .configure can't download them but wget can. I am attaching my customized update_and_rebuild_petsc script and the configure.log that resulted from running it.
Fande Kong
--
You received this message because you are subscribed to the Google Groups "moose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/0a8315b7-185e-44c9-b1d3-d3b8f52939d4%40googlegroups.com.
Tomas Mondragon
Tomas Mondragon
--
You received this message because you are subscribed to a topic in the Google Groups "moose-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/moose-users/2xZsBpG-DtY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/c230c461-db36-4257-a88f-a1479ac97fa5%40googlegroups.com.
Fande Kong
You received this message because you are subscribed to the Google Groups "moose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/CAP4KeS%2BSTRb1h99NS%2BtqrBC819uNAHQCPBpPT0b08WeBQ_VSJQ%40mail.gmail.com.
Tomas Mondragon
On Monday, January 13, 2020 at 11:02:07 AM UTC-6, Fande Kong wrote:
I see. We did not hit this issue because we often use "--download-xxx=1" instead of "--download-xxx=your/path/xxx.tar.gz".Anyway, I'm glad your issue was resolved.Thanks,Fande,
Tomas Mondragon
Fande Kong
--
You received this message because you are subscribed to the Google Groups "moose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/0b4c29ac-2261-404a-84f6-5e8e28e1c51f%40googlegroups.com.
Tomas Mondragon
./configure $(echo $PFX_STR) \
--download-hypre=downloaded_thirdParty_tarballs/hypre-93baaa8c9.tar.gz \
--with-debugging=no \
--with-shared-libraries=1 \
--download-fblaslapack=downloaded_thirdParty_tarballs/fblaslapack-v3.4.2-p2.tar.gz \
--download-metis=downloaded_thirdParty_tarballs/metis-v5.1.0-p7.tar.gz \
--download-ptscotch=downloaded_thirdParty_tarballs/scotch-v6.0.9.tar.gz \
--download-parmetis=downloaded_thirdParty_tarballs/parmetis-v4.0.3.tar.gz \
--download-superlu_dist=downloaded_thirdParty_tarballs/superLU-DIST-v6.2.0.tar.gz \
--download-mumps=downloaded_thirdParty_tarballs/mumps-v5.2.1-p2.tar.gz \
--download-scalapack=downloaded_thirdParty_tarballs/scalapack-v2.1.0-p1.tar.gz \
--download-slepc=downloaded_thirdParty_tarballs/slepc-bf89b9d.tar.gz \
--with-mpi=1 \
--with-cxx-dialect=C++11 \
--with-fortran-bindings=0 \
--with-sowing=0 \
--with-batch=1 \
--with-cudac=0 \
--with-lgrind=0 \
$*
On Monday, January 27, 2020 at 11:24:50 AM UTC-6, Fande Kong wrote:
You could try "--with-lgrind=0". Not sure why lgrind was on. By default, it is off.BTW, what did you have when you were using PETSc submodule with manual downloaded packages?Thanks,Fande,
On Fri, Jan 24, 2020 at 4:09 PM Tomas Mondragon <tom.alex...@gmail.com> wrote:
The "--with-cudac=0" flag did make it avoid the new code in getExecutable for a while, but it got tripped up while configure searched for lgrind. Is there any flag for avoiding lgrind or should I just wait on a fix to getExecutable?--
On Friday, January 24, 2020 at 4:26:32 PM UTC-6, Tomas Mondragon wrote:
On Monday, January 13, 2020 at 11:02:07 AM UTC-6, Fande Kong wrote:I see. We did not hit this issue because we often use "--download-xxx=1" instead of "--download-xxx=your/path/xxx.tar.gz".Anyway, I'm glad your issue was resolved.Thanks,Fande,Unfortunately, the issue is not resolved as I discovered that the nodes running the script have no access to the internet, so I am stuck with using "--download-xxx=your/path/xxx.tar.gz".Anyhow, I tried again using a newer petsc commit than the one that the moose git repo uses as a submodule, but ran in to trouble while petsc/config/BuildSystem/config/setCompilers.py was searching for a cuda compiler. The new bit of code in the getExecutable method in petsc/config/BuildSystem/config/base.py lines 294-318 will cause errors if a directory on the search path is inaccessible due to file permissions.I think running .configure with a "--with-cudac=0" flag might help me avoid that bit of code.I am attaching my new configure.log in case anyone sees anything else wrong with it.
You received this message because you are subscribed to the Google Groups "moose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to moose...@googlegroups.com.
Fande Kong
To unsubscribe from this group and stop receiving emails from it, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/095881e4-592d-427a-ad84-6cbe5fb8fe2e%40googlegroups.com.
Tomas Mondragon
def getExecutable(self, names, path = [], getFullPath = 0, useDefaultPath = 0, resultName = '', setMakeMacro = 1):
'''Search for an executable in the list names
- Each name in the list is tried for each entry in the path until a name is located, then it stops
- If found, the path is stored in the variable "name", or "resultName" if given
- By default, a make macro "resultName" will hold the path'''
found = 0
if isinstance(names,str) and names.startswith('/'):
path = os.path.dirname(names)
names = os.path.basename(names)
if isinstance(names, str):
names = [names]
if isinstance(path, str):
path = path.split(os.path.pathsep)
if not len(path):
useDefaultPath = 1
def getNames(name, resultName):
import re
prog = re.match(r'(.*?)(?<!\\)(\s.*)',name)
if prog:
name = prog.group(1)
options = prog.group(2)
else:
options = ''
if not resultName:
varName = name
else:
varName = resultName
return name, options, varName
varName = names[0]
varPath = ''
for d in path:
for name in names:
name, options, varName = getNames(name, resultName)
if self.checkExecutable(d, name):
found = 1
getFullPath = 1
varPath = d
break
if found: break
if useDefaultPath and not found:
for d in os.environ['PATH'].split(os.path.pathsep):
for name in names:
name, options, varName = getNames(name, resultName)
if self.checkExecutable(d, name):
found = 1
varPath = d
break
if found: break
if not found:
dirs = self.argDB['with-executables-search-path']
if not isinstance(dirs, list): dirs = [dirs]
for d in dirs:
for name in names:
name, options, varName = getNames(name, resultName)
if self.checkExecutable(d, name):
found = 1
getFullPath = 1
varPath = d
break
if found: break
if found:
if getFullPath:
setattr(self, varName, os.path.abspath(os.path.join(varPath, name))+options)
else:
setattr(self, varName, name+options)
if setMakeMacro:
self.addMakeMacro(varName.upper(), getattr(self, varName))
else:
self.logWrite(' Unable to find programs '+str(names)+' providing listing of each search directory to help debug\n')
self.logWrite(' Path provided in Python program\n')
for d in path:
if os.path.isdir(d):
try:
self.logWrite(' '+str(os.listdir(d))+'\n')
except OSError as e:
self.logWrite(' '+e.strerror+'\n')
else:
self.logWrite(' Warning '+d+' is not a directory\n')
if useDefaultPath:
if os.environ['PATH'].split(os.path.pathsep):
self.logWrite(' Path provided by default path\n')
for d in os.environ['PATH'].split(os.path.pathsep):
if os.path.isdir(d):
try:
self.logWrite(' '+str(os.listdir(d))+'\n')
except OSError as e:
self.logWrite(' '+e.strerror+'\n')
else:
self.logWrite(' Warning '+d+' is not a directory\n')
dirs = self.argDB['with-executables-search-path']
if not isinstance(dirs, list): dirs = [dirs]
if dirs:
self.logWrite(' Path provided by --with-executables-search-path\n')
for d in dirs:
if os.path.isdir(d):
try:
self.logWrite(' '+str(os.listdir(d))+'\n')
except OSError as e:
self.logWrite(' '+e.strerror+'\n')
else:
self.logWrite(' Warning '+d+' is not a directory\n')
return found
On Thursday, January 30, 2020 at 10:54:21 AM UTC-6, Tomas Mondragon wrote:
Configuration log is attached
Alexander Lindsay
--
You received this message because you are subscribed to the Google Groups "moose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/4d3bfd39-d68b-40d1-af4a-536f24860035%40googlegroups.com.
Fande Kong
Pushed one more change - move duplicate/similar code into a function.
Satish
On Thu, 30 Jan 2020, Satish Balay via petsc-users wrote:
> Ah - missed that part. I've updated the branch/MR.
>
> Thanks!
> Satish
>
> On Thu, 30 Jan 2020, Tomas Mondragon wrote:
>
> > Just to be extra safe, that fix should also be applied to the
> > 'with-executables-search-path' section as well, but your fix did help me
> > get past the checks for lgrind and c2html.
> >
> > On Thu, Jan 30, 2020, 3:47 PM Satish Balay <ba...@mcs.anl.gov> wrote:
> >
> > > I pushed a fix to branch balay/fix-check-files-in-path - please give it a
> > > try.
> > >
> > > https://gitlab.com/petsc/petsc/-/merge_requests/2490
> > >
> > > Satish
> > >
> > > On Thu, 30 Jan 2020, Satish Balay via petsc-users wrote:
> > >
> > > > The issue is:
> > > >
> > > > >>>
> > > > [Errno 13] Permission denied: '/pbs/SLB'
> > > > <<<
> > > >
> > > > Try removing this from PATH - and rerun configure.
> > > >
> > > > This part of configure code should be fixed.. [or protected with 'try']
> > > >
> > > > Satish
> > > >
> > > > On Thu, 30 Jan 2020, Fande Kong wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > It looks like a bug for me.
> > > > >
> > > > > PETSc was still trying to detect lgrind even we set "--with-lgrind=0".
> > > The
> > > > > configuration log is attached. Any way to disable lgrind detection.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Fande
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------- Forwarded message ---------
> > > > > From: Tomas Mondragon <tom.alex....@gmail.com>
> > > > > Date: Thu, Jan 30, 2020 at 9:54 AM
> > > > > Subject: Re: Running moose/scripts/update_and_rebuild_petsc.sh on HPC
> > > > > To: moose-users <moose...@googlegroups.com>
> > > > >
> > > > >
> > > > > Configuration log is attached
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
Tomas Mondragon
Tomas Mondragon
Thanks for the change to base.py. Pulling the commit, confirm was able to skip over lgrind and c2html. I did have a problem with Parmetis, but that was because I was using an old ParMetis commit accidentally. Fixed by downloading the right commit of ParMetis.My current problem is with Hypre. Apparently --download-hypre cannot be used with --with-batch=1 even if the download URL is on the local machine. The configuration.log that resulted is attached for anyone who may be interested.
--
You received this message because you are subscribed to a topic in the Google Groups "moose-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/moose-users/2xZsBpG-DtY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to moose-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/a34fa09e-a4f5-4225-8933-34eb36759260%40googlegroups.com.
Tomas Mondragon
You might find this option useful.
--with-packages-download-dir=<dir>
Skip network download of package tarballs and locate them in specified dir. If not found in dir, print package URL - so it can be obtained manually.
This generates a list of URLs to download so you don't need to look through the xxx.py files for that information. Conceivably a script could gather this information from the run of configure and get the tarballs for you.
Barry