Run test with Xyce6.8 Serial executable file

[Xiang Zhang]

Hello,

I recently tried to build the source code on my windows/Ubuntu system but unfortunately I did not make it through. So I decided to download the serial 6.8 executable files and want to give it try first. As the instruction says that "-library" is needed for executable version. Should I add this command within the long quote or start with a new quote?

#!/bin/bash

#$HOME/Xyce_Regression-6.8/TestScripts/run_xyce_regression \

rm -rf Xyce_Test

$HOME/EXECUTABLE_XYCE/Xyce_Regression-6.8/TestScripts/run_xyce_regression \

--output=`pwd`/Xyce_Test \

--xyce_test="$HOME/Xyce_Regression-6.8" \

--resultfile=`pwd`/serial_results \

--taglist="+serial+nightly?noverbose?klu-verbose?fft-library" \

`pwd`/src/Xyce 

Here is my current version.

Thanks,

Xiang Zhang

[Xiang Zhang]

BTW, I converted the rpm file to deb using command alien to fit my Ubuntu system and hope it works.

Without the "-library", when I ran ./RunTestSequence, it says that "ERROR: no valid xyce_verify.pl script found". But I do find the xyce_verify.pl in my folder as it is installed from the package.

The original RunTestSequence is attached.

Thanks,

Xiang Zhang

[xyce-users]

The Xyce project doesn't provide binaries that will work on Ubuntu (the RPM will only work on RHEL and CentOS), so we'll have to assume that you are trying this on Windows.  If you try to run the RPM on Ubuntu it will probably not work.  You will have to build Xyce from source on Ubuntu.

You should add the "-library" inside the quoted tags list, just as you have done here.  But there is more wrong with your invocation.  The final argument to run_xyce_regression has to be the executable file to run, which is only "`pwd`/src/Xyce" when you're building from source.  If you're using one of the binary installers, you have to change that so it points to the binary's actual location.  On Windows, that binary will be located in C:\Program Files\Xyce 6.8 OPENSOURCE\bin\Xyce.exe
We know that Xyce builds properly on Ubuntu (one of our developers tests it there with each release, when updating the building guide).  What problem did you encounter that made it fail for you?

You ask in a later email where you can find xyce_verify.pl --- this is part of the Xyce regression suite, and for run_xyce_regression to be able to find it, you must have "--xyce_test=" pointing at the place where that lives.  From what you posted, that should probably be $HOME/EXECUTABLE_XYCE/Xyce_Regression-6.8, but you are setting it to "$HOME/Xyce_Regression-6.8".

[Xiang Zhang]

Hi xyce-users,

Thanks for the quick reply, I really appreciate your help!

I am new to C++ so I am learning building Xyce along with C++. So sorry about my bad invocation.

Since I first failed to install the Xyce on my Win10 and Ubuntu, I tried to convert the rpm file to deb file using command 'alien'.

With the command you suggested, I just finished my regression test on both system (Win10 and Ubuntu with the converted deb version). I noticed two types of pass with "passed" and "passed[sh]". Do they both mean pass?

I also installed windows version as well. Enclosed are two output files from the two system. There are 42 failed tests on Ubuntu and 44 failed tests on Windows.

It took quite a while to run the whole process. Sorry about the wait.

I will try to reinstall the trilinos and Xyce again on Ubuntu/Win10 and show you the fault info.

Thanks for your advice and I really appreciate it!
Thanks,

Xiang

[xyce-users]

I believe the failures you are seeing may be related to python.  As far as I can tell, every test that uses a python script has failed.  For these to pass, you'd need python, numpy, and scipy all installed where the test suite could find them, and my guess is you don't have them installed.  If you install these and try again, you should find that those tests stop failing.  There may also be a few tests that are failing because you might have a more recent version of perl than we used when we released Xyce 6.8.  There were at least two tests whose perl scripts in the 6.8 regression suite used features that were removed from Perl 5.26.  We have fixed those in the upcoming Xyce 6.9 test suite.

I'm surprised that the RPM build of Xyce worked at all on Ubuntu --- I would not have expected the shared library dependencies to be pulled in properly.  It is interesting that you got as far as you did.  Could you post exactly what you did to convert the RPM to DEB?  Others might benefit from that information.

As far as "passed" vs. "passed[sh]", yes, both mean passed.  It is a relic of a very old test process we had in the group --- a long time ago, all tests in the test suite simply ran the binary under test and compared it directly to an archived gold standard.  Later, we added the capability of doing more complex tests, controlled by a script that did something special that was specific to that test (which could be anything from checking against an archived gold standard to checking that the code failed cleanly with an expected error message).  "passed" means its one of those old-style tests and the current run compared well to the archived gold standard, "passed[sh]" means it's a new style and the script said it passed (and one would have to read the script to see what "passing" really entailed).

[Xiang Zhang]

Thanks for the detailed explanation on "pass" and "pass[sh]".

The guide is as follows on how to convert the file from rpm to deb (Ubuntu compatible file) for the future users.

[Keywords: rpm, deb, Ubuntu, Red hat, convert]

Follow the instruction on the page:

https://stackoverflow.com/questions/28229024/how-to-install-rpm-on-ubuntu

https://help.ubuntu.com/community/RPM/AlienHowto

In case the link expired, here is the detailed steps:

1) Download the rpm file from the Xyce executable web.

2) unzip the .rpm file.

3) Install the conversion command as,

sudo apt-get install alien

4) Install the .rpm directly using,

sudo alien -i package_file.rpm

5) Alternatively, convert the file and then install,

sudo alien package_file.rpm

sudo dpkg -i package_file.deb

,where the package_file.rpm and package_file.deb are the filenames you just downloaded from the Xyce website.

[Xiang Zhang]

The problem that I encountered during reconfigure "make" for trilinos12.12 is that there are many warnings such as:

[  1%] Building CXX object packages/teuchos/core/src/CMakeFiles/teuchoscore.dir/Teuchos_RCPNode.cpp.o

In file included from /home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_Assert.hpp:46:0,

                 from /home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_any.hpp:51,

                 from /home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_RCPNode.hpp:53,

                 from /home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_RCPNode.cpp:42:

/home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_RCPNode.cpp: In destructor ‘Teuchos::ActiveRCPNodesSetup::~ActiveRCPNodesSetup()’:

/home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_TestForException.hpp:183:28: warning: throw will always call terminate() [-Wterminate]

     throw Exception(omsgstr); \

                            ^

/home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_TestForException.hpp:305:3: note: in expansion of macro ‘TEUCHOS_TEST_FOR_EXCEPTION’

   TEUCHOS_TEST_FOR_EXCEPTION(throw_exception_test, std::logic_error, "Error!")

   ^~~~~~~~~~~~~~~~~~~~~~~~~~

/home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_RCPNode.cpp:655:5: note: in expansion of macro ‘TEUCHOS_TEST_FOR_EXCEPT’

     TEUCHOS_TEST_FOR_EXCEPT(0==rcp_node_list());

     ^~~~~~~~~~~~~~~~~~~~~~~

/home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_TestForException.hpp:183:28: note: in C++11 destructors default to noexcept

     throw Exception(omsgstr); \

                            ^

/home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_TestForException.hpp:305:3: note: in expansion of macro ‘TEUCHOS_TEST_FOR_EXCEPTION’

   TEUCHOS_TEST_FOR_EXCEPTION(throw_exception_test, std::logic_error, "Error!")

   ^~~~~~~~~~~~~~~~~~~~~~~~~~

/home/firehawk/SGCC/trilinos-12.12.1-Source/packages/teuchos/core/src/Teuchos_RCPNode.cpp:655:5: note: in expansion of macro ‘TEUCHOS_TEST_FOR_EXCEPT’

     TEUCHOS_TEST_FOR_EXCEPT(0==rcp_node_list());

     ^~~~~~~~~~~~~~~~~~~~~~~

[  1%] Building CXX object packages/teuchos/core/src/CMakeFiles/teuchoscore.dir/Teuchos_Range1D.cpp.o

And there are several others as well (shown in the file trilinos_output_screen). But it still ran through make and make install process.
Do you know what the problem is?

Thanks,

Xiang

On Tuesday, May 29, 2018 at 2:33:35 PM UTC-7, xyce-users wrote:

[Xiang Zhang]

The ACTUAL error I encountered is during Xyce "make". The screen output and the reconfigure are attached.

Thanks,

Xiang

On Tuesday, May 29, 2018 at 2:33:35 PM UTC-7, xyce-users wrote:

[xyce-users]

The warnings are not important here, and are not relevant to the failures you're seeing.  I'll post about those shortly.

[xyce-users]

The errors you're seeing here are coming from files created by "flex" --- these files should be referencing "std::cerr", "std::cout" and "std::cin" and not "cerr", "cout" and "cin".  That your flex is creating an N_DEV_ReactionLexer.cxx that references the streams in this way is very strange, and not what we see .  It suggest that there may be some issue with the version of flex you're running.

What version of Ubuntu are you running?  What does "flex --version" report?  How did you install flex?

[xyce-users]

And for what it's worth, I just built Xyce 6.8 on Ubuntu 14.04 (a fairly old version) and the version of flex I've got installed (2.5.35, also a fairly old version, from 2008) created N_DEV_ReactionLexer.cxx that referenced the streams using the std:: namespace properly.  I'm very keen to know where this breakage is coming from.  This seems to point to your version of flex being quite old (it would be odd if a *newer* version of flex removed the namespace qualifiers).   As near as I can tell from looking at Flex's git history (at https://github.com/westes/flex), the lines of the flex skeleton scanner that match the ones that are showing up in your error message date back to before 2001 when they were replaced with lines that included the namespace prior to version 2.5.10 of flex (which was released in 2002).

The path forward to getting this to work will be to use a much newer version of flex than that, at least 2.5.35, but preferably a more recent one in the 2.6 series.

[Xiang Zhang]

Hi xyce-users,

Thanks for the reply.

My flex version was quite old , version 2.5.4 and my Ubuntu is 18.04. Once I updated the flex to 2.6.4. The system is able to run through the xyce "make" with no problem but a few warnings and abort (which is attached in the file). But when I ran makeinstall, there are some errors... The output screen is also attached.

Thanks for your effort!

Thanks,

Xiang

[xyce-users]

The warnings are meaningless noise from the compiler.  Recent versions of gcc (gcc 6 and 7) are much more willing than older versions were to warn about things that "might" be errors that are not in fact errors.  Further, it has become commonplace in Linux to declare certain system functions such that a warning is emitted when the return result of the function is ignored, even though legacy codes have been doing just that for decades with no problem.  It makes for an ugly mess while compiling, but the program still works exactly as intended. 

if you compile with "-Wno-unused-result" in your CXXFLAGS the warnings will go away.  They do not constitute real errors in Xyce, and it is unlikely that we'll have the staff available to pick through the code to satisfy the compiler's nit picking in these cases any time soon (though our developers do try to work on reducing the number of distracting warnings as much as we can).

Your make install error seems to be due to your trying to install into /usr/local without root permission.  Try "sudo make install" instead.  Or configure with "--prefix=" and a path that you do have permission to write into.  Both are documented in our building guide.  We highly recommend that you chose a unique prefix and not rely on configure's default install prefix (/usr/local).   That recommendation is also in the building guide.

It's surprising that Ubuntu 18.04 (which is very new) had Flex 2.5.4, which is unreasonably old.  The building guide does state that Xyce requires version 2.5.34 or later.  The ubuntu 14.04 system on which I built had 2.5.35.

[Xiang Zhang]

Hi xyce-users,

Thanks for your guide on the "make install". I actually read through and even highlighted the instruction on "--prefix=" before I installed the file...I just noticed that embarrassing moment. XD.

Enclosed are the two regression test results I received from the Serial build from Ubuntu and Win10 and the test result from converted RPM as well.

It seems that it still has minor case error with Win10. But again, I tried to install and update the python according to your guide yesterday but it appears that it is still not quite working.

Could you show me how you update python or is there a command that would link the python with Xyce?

BTW, I am going to install parallel build on two systems as I am writing for now. Again, thanks for your precious advice and help on the installation! I really appreciate it!

Thanks,

Xiang

[Xiang Zhang]

When I install parallel trilinos "make install", it says that 

checking for bison version 2.7... no

checking for bison version 2.6... no

checking for bison version 2.5... no

checking for bison version 2.4... no

checking for bison version 2.3... no

configure: error: "Requested reaction parser, but no bison of correct version number."

However, the bison version I used on Ubuntu is 

firehawk@firehawk-ThinkCentre-M710t:~/SGCC/Xyce/buildParallel$ bison --version

bison (GNU Bison) 3.0.4

Written by Robert Corbett and Richard Stallman.

But the this 3.0.4 version has no problem when I install serial build. Should I install a 2.7 on Ubuntu to continue on?

I do not encounter such a problem when i install on Win10 with Cygwin.

Thanks,

Xiang

[xyce-users]

You should not need to install an older version of bison.  Something else is wrong here.

Xyce's configure script is supposed to look for bison by starting at the latest version we support (3.0.x) and working backwards through every prior version that we are sure worked.  It does this by doing "bison --version" to see what version you have, then comparing that against the versions that are known to work (taking only the major and minor version numbers, skipping the third field).

It should have first printed "Checking for bison version 3.0" first, and should have found it.  That it didn't is puzzling.  That it worked fine for the serial version and not for parallel is bewildering.

The console output is not enough to diagnose what is wrong.   The answer (or at least a clue) will be in the "config.log" file that configure creates, and you should look in there to see what the actual error was when it tried to run bison.  If you post that file (as an attachment) we may be able to spot what's happening.

You may also just want to disable the reaction parser, which is a fairly esoteric bit of Xyce that you probably don't need, by adding "--disable-reaction_parser" to the configure line.  That will make Xyce skip its checks for bison and flex, and not bother compiling in that part of the code.  It will also make the handful of tests that rely on it fail unless you tell the regression suite to skip them (by adding "-rxn" to the tags list).

[Xiang Zhang]

Attached are my Xyce reconfigure and config.log.

Sorry what I just posted as "make install". It should be when I ./reconfigure.

Thanks,

Xiang

[Thomas Russo]

Look carefully at your reconfigure script.  You're trying to build Xyce 6.0, not Xyce 6.8, and you must still have an old copy of 6.0 lying around so that its configure can be run instead of failing immediately.  Xyce 6.0 had not yet been updated to accept Bison 3.0.

From your reconfigure script:

/home/firehawk/SGCC/Xyce/Xyce-6.0/configure

Make it run Xyce 6.8's configure instead, and you should not have this problem.

[Xiang Zhang]

you are right! My bad. It works now and I am proceeding on "make". Sorry about that typo. I really should have examined that more carefully!

BTW, any thoughts on regression test result issues?

Thanks,

Xiang

[Xiang Zhang]

I have got parallel build implemented on my Ubuntu System and currently running regression tests.

However, there are errors shown on my Win10 when I ran Xyce "make".

The last couple of lines of screen output is also attached.

Thanks,

Xiang

[xyce-users]

The errors indicate a linker problem and give the impression that the C compiler is being used to link C++ objects.  The configure script does not have any glaring errors.

Can you attach the results of compiling Xyce where you use 'make V=1' at the command line?  That should give more verbose compile and link lines that may help us determine the issue.

Thanks,
The Xyce Team

[Xiang Zhang]

Hi,

Attached is the output from the screen by typing in "make V=1 |& tee Xyce_build_output_screen.txt". However, since the file is too large to attach, I cut it into 5 pieces and created a Dropbox link you could use to view. The original txt is also included. Sorry about that.

https://www.dropbox.com/sh/tbvp1dcf3jjvzwd/AABsOKx_EJZO43Dt5JkjPwqZa?dl=0

Thanks,

Xiang

[xyce-users]

This is just a shot in the dark, but you are using "CXX=mpiCC" in your Windows reconfigure file for parallel.   This is probably the issue.

Windows has a case-insensitive file system, and "mpiCC" is actually the same thing as "mpicc" --- that is, the C compiler, instead of the expected C++ compiler.  Try using "CXX=mpicxx" or "CXX=mpic++" instead to refer to the C++ compiler wrapper.  Do a "make distclean" in your build directory, rerun your fixed reconfigure script, and see if that fixes things.

The Xyce team has never build Xyce on Windows in parallel --- in fact, Xyce 6.8 is the first version of Xyce that didn't require ParMETIS in parallel, and ParMETIS had serious compilation problems on Windows (or so we have been told).  You may be the first person ever to have tried it.

The Xyce 6.9 building guide has been modified to avoid mentioning "mpiCC" for the C++ compiler, and recommends mpicxx or mpic++.  Every OpenMPI installation will have mpicxx and mpic++ as synonyms for the C++ compiler wrapper, but only those on case-sensitive file systems will have mpiCC.

[Xiang Zhang]

Hi,

With the command you suggested, I have successfully built the Xyce with parallel version on my Win10! However, as it suggests on regression test, the test suite is not suitable for Win10 parallel. I noticed that it took roughly 200s average for each case to execute....(I started testing this morning and it is still half way to go (already ~9 hours!). So I am going to stick to the Ubuntu version!

One interesting thing I notice is that even on Ubuntu, it took 2 s for serialbuild to run, 13 s for 2 cores but 20 s for 4 cores with my Core i5 7400. I guess this case is not designed for parallel computing.

However, the cases involving python are still failing on Ubuntu even with the new python version, with Python 2.7.15rc1 when typing in "python --version" and Python 3.6.5 when typing in "python3 --version". But Python update did fix my Win10 Serial problem.

Attached are my output from file and screen shot. Do you know what I should do with this scenario?

Thanks,

Xiang