gmake error with upcxx-run

371 views
Skip to first unread message

Andrew

unread,
Apr 26, 2021, 10:15:05 AM4/26/21
to UPC++

Hello,

Thanks to some excellent help on the issues pages, I have managed to compile upcxx and metahipmer. However, I am still hitting a snag.

I have compiled metahipmer v2.1.0.2 using GCC 8.2.0 and cmake 3.14.0 (no GPU support).

mhm2_test works fine, and I can directly call the mhm2 executable. However, when I run the standard command line mhm2.py which runs mhm2 through the upcxx-run command I get the following error:

~/mhm2-v2.0.1.2/install/bin/mhm2.py -p pair1.fq.gz,pair2.fq.gz --post-asm-abd true INFO: may need to build the required runtime. Please be patient.
gmake[1]: Nothing to be done for `/rds/general/user/ajm3018/home/upcxx-2021.3.0/Makefile'.
gmake: *** gasnet.debug/-conduit: No such file or directory. Stop.
gmake[2]: *** [gasnet-single] Error 2
gmake[1]: *** [upcxx-run] Error 2
gmake: *** [upcxx-run] Error 2
upcxx-run: ERROR: build of the required runtime failed.

FATAL ERROR: Could not execute dir/upcxx-run: Command '['dir/upcxx-run', '-h']' returned non-zero exit status 1

If I call upcxx-run directly I get the same gmake errors.

I presume I've missed something simple!

All advice welcome!

Thanks,

Andrew

Rob Egan

unread,
Apr 26, 2021, 5:18:10 PM4/26/21
to UPC++
Hi Andrew,

An update that includes some of the behind-the-scenes direct messages that I've had with you and UPC++ developers (Dan).

It looks like your configuration is pointing to the UPC++ source tree, and not the UPC++ installation tree.  There does exist a upcxx-run in the source code that is NOT the same as the one which would be installed.  The source code version performs a build and the installation code would actually spawn and executes your program on your installed platform.

And for the sake of this list, this was my initial recommendation for you emailed separately:

I believe the installation of upcxx is incorrect on your platform, as the message you are getting implies that it is attempting to build upcxx, not actually run it.

I would suggest downloading the source again and following the build instructions.

Alternatively, MHM2 includes a simple build script which we use to install upcxx in a simple environment (i.e. 1 linux box) which performs the download, build and install all at once.

You can find it in the mhm2 source at: upcxx-utils/contrib/install_upcxx.sh

So I see in your cmake command that you include the "upc_source_dir".  That should really be the installation directory, not the source code... And then, if that installation's bin directory is in your PATH, then the CMAKE_PREFIX_PATH should not be necessary to find the cmake support that come packaged with UPC++

So when UPC++ is installed and in your PATH, you should be able to execute "upcxx --version" and "upcxx-run -h" without errors or odd building commands.

I hope this helps,
Rob

Andrew

unread,
Apr 27, 2021, 4:22:06 PM4/27/21
to UPC++
Hi Rob

Thanks so much. I have recompiled upcxx without error and confirmed the share directory exists as expected.

upcxx is on the path. I am following the upcxx advice to use a script to call g++ as this is not the native g++ (https://bitbucket.org/berkeleylab/upcxx/src/efd61603c0318056776b197172892ad948a4f012/docs/local-gcc.md)

Thus my cmake command is

cmake /xxx/mhm2-v2.0.1.2    -DCMAKE_BUILD_TYPE=Release    -DENABLE_CUDA=OFF    -DCMAKE_INSTALL_PREFIX=/xxx/mhm2-v2.0.1.2/install    -DCMAKE_C_COMPILER=/apps/gcc/8.2.0/bin/gcc    -DCMAKE_CXX_COMPILER=/.../g++.sh

I get a warning about using the "wrong" g++ (which I may be able to ignore, as it's really the same g++) but it fails due to not finding upcxx. I do not understand cmake's find_modules process enough to understand why it might be failing.

I apologise for probably wasting your time fixing issues which are probably arising because I don't have the expertise to compile complex software like this. Incidentally, is it just the .build folder I need to delete to have a clean rerun with cmake?

Command line output below!

Thanks,

Andrew

-- Building MHM2 with CMake 3.14.0
-- No CUDA environment detected
-- Building for CPU only
-- Found CpuArch: sandy-bridge  found components:  mmx avx sse sse2 ssse3
-- Compiling C with -mmmx option
-- Compiling CXX with -mmmx option
-- Compiling C with -mavx option
-- Compiling CXX with -mavx option
-- Compiling C with -msse option
-- Compiling CXX with -msse option
-- Compiling C with -msse2 option
-- Compiling CXX with -msse2 option
-- Compiling C with -mssse3 option
-- Compiling CXX with -mssse3 option
Building Release version
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_CXX_COMPILER=/apps/gcc/8.2.0/bin/g++
-- CMAKE_CXX_COMPILER=/xxx/mhm2-v2.0.1.2/g++.sh
-- ABS_UPCXX_CXX_PATH=/rds/general/applications/gcc/8.2.0/bin/g++
-- ABS_CMAKE_CXX_PATH=/xxx/mhm2-v2.0.1.2/g++.sh
-- compiler names: g++ vs g++.sh
CMake Warning at /xxx/upcxx-2021.3.0/install/share/cmake/UPCXX/UPCXXConfig.cmake:129 (message):
  Compiler compatibility check failed!

  UPCXX compiler provided by upcxx-meta CXX:

      /apps/gcc/8.2.0/bin/g++ ->
      /rds/general/applications/gcc/8.2.0/bin/g++

  is different from CMAKE_CXX_COMPILER:

      /xxx/mhm2-v2.0.1.2/g++.sh ->
      /xxx/mhm2-v2.0.1.2/g++.sh



  Please either pass cmake: -DCMAKE_CXX_COMPILER=/apps/gcc/8.2.0/bin/g++

  or re-install UPC++ with:
  CXX=/xxx/mhm2-v2.0.1.2/g++.sh

Call Stack (most recent call first):
  CMakeLists.txt:167 (find_package)


-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Could NOT find UPCXX (missing: UPCXX_COMPATIBLE_COMPILER) (found suitable version "2021.3.0", minimum required is "2020.3")
-- This UPC++ install requires the c++11 standard.
-- UPCXX_NETWORK=smp
-- UPCXX_THREADMODE=seq
-- UPCXX_CODEMODE=O3
CMake Warning at CMakeLists.txt:167 (find_package):
  Found package configuration file:

    /rds/general/user/ajm3018/home/upcxx-2021.3.0/install/share/cmake/UPCXX/UPCXXConfig.cmake

  but it set UPCXX_FOUND to FALSE so package "UPCXX" is considered to be NOT
  FOUND.


CMake Error at CMakeLists.txt:169 (message):
  UPCXX (version 2020.3) is required to build.

    Please verify the installation of upcxx
    and verify that upcxx is in your PATH: which upcxx && upcxx --version
    and has been installed in the correct location: find $(dirname $(which upcxx))/../share -name 'UPCXX*' -type f
      which should discover UPCXXConfig.cmake and UPCXXConfigVersion.cmake

Dan Bonachea

unread,
Apr 27, 2021, 5:02:44 PM4/27/21
to Andrew, UPC++
Hi Andrew -

I get a warning about using the "wrong" g++ (which I may be able to ignore, as it's really the same g++) but it fails due to not finding upcxx. I do not understand cmake's find_modules process enough to understand why it might be failing.

IIUC you cannot safely ignore this compiler compatibility warning, it appears to be the reason the build is dropping dead.

The message explains how to fix the problem:

  UPCXX compiler provided by upcxx-meta CXX:

      /apps/gcc/8.2.0/bin/g++ ->
      /rds/general/applications/gcc/8.2.0/bin/g++

  is different from CMAKE_CXX_COMPILER:

      /xxx/mhm2-v2.0.1.2/g++.sh ->
      /xxx/mhm2-v2.0.1.2/g++.sh

  Please either pass cmake: -DCMAKE_CXX_COMPILER=/apps/gcc/8.2.0/bin/g++
  or re-install UPC++ with:
  CXX=/xxx/mhm2-v2.0.1.2/g++.sh

IIUC, you are using the wrapper script to add a directory to the shared library path. I think your options are:

0. Rename your wrapper script from g++.sh to g++. The CMake logic sees the different base name, takes that as indication these are different compilers and stops trying to determine if these are in fact the same compiler.
-or-
1. Reconfigure/build/install UPC++ using your wrapper script as CXX=/xxx/mhm2-v2.0.1.2/g++.sh (as the message suggests)
-or-
2. Ditch the wrapper script and reconfigure/build/install UPC++ using configure --with-ldflags=-Wl,-rpath=/rds/general/applications/gcc/8.2.0/lib
-or-
3. Ditch the wrapper script and use LD_RUN_PATH/LD_LIBRARY_PATH to inject your shared library directory instead (but this assumes nothing else in your link is setting an rpath)

Unfortunately, I am not in a position to test these recommendations myself.  So, I’d appreciate knowing of both success and failure following any of these options.

Hope this helps..
-D

--
You received this message because you are subscribed to the Google Groups "UPC++" group.
To unsubscribe from this group and stop receiving emails from it, send an email to upcxx+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/upcxx/3bb1dec4-340b-4a0f-8fae-0e936f8f4c5cn%40googlegroups.com.

Andrew

unread,
Apr 28, 2021, 7:47:28 AM4/28/21
to UPC++
Dan and Rob,

Thanks so much for the hand-holding! You were right that the g++ warning seemed to be the issue (though the final console output made upcxx look like the culprit). As a true novice, I took the easy way out and renamed g++.sh and the compilation now works (I have guessed that I need to rm -rf CMakeFiles in the source dir each time). I also think I may have put the directory at the wrong bit of the cmake command (beginning rather than end) though I don't think that made a difference.

Interestingly, the .build folder from which I ran the cmake command remains empty (perhaps this is expected because I haven't run make yet).

However, > make -j all install

gives an error:

make: *** No rule to make target `all'.  Stop.


mkdir -p .build
cd .build
cmake -DCMAKE_INSTALL_PREFIX=path-to-install ..
make -j all install

Only my cmake command is:

cmake -DCMAKE_BUILD_TYPE=Release    -DENABLE_CUDA=OFF    -DCMAKE_INSTALL_PREFIX=/xxx/mhm2-v2.0.1.2/install    -DCMAKE_C_COMPILER=/apps/gcc/8.2.0/bin/gcc    -DCMAKE_CXX_COMPILER=/xxx/mhm2-v2.0.1.2/g++ /xxx/mhm2-v2.0.1.2

I, again, suspect a basic error on my part!

Thanks again for advice.

Best wishes,

Andrew

Andrew

unread,
Apr 28, 2021, 9:23:54 AM4/28/21
to UPC++
Just to note that the first upcxx test (test-hello_upcxx-smp) has worked on a two node job, but still warns about a possible need for compilation:

upcxx-run -n 2 ./test-hello_upcxx-smp
INFO: may need to build the required runtime.  Please be patient.
Hello from 0 of 2
Hello from 1 of 2

This bodes well for getting mhm2 to work!

Best wishes,

Andrew

Andrew

unread,
Apr 28, 2021, 9:54:00 AM4/28/21
to UPC++
And of note, this (below) is the output from make run-tests (smp, ibv and udp). I confess I have no understanding of our network topology or set up.

I had to set SSH_SERVERS environment variable first. Interestingly, it looks like test-hello_upcxx-smp and test-hello_upcxx-ibv passed and test-hello_upcxx-udp failed. When I call udp manually it passes:

upcxx-run -n 2 ./test-hello_upcxx-udp
INFO: may need to build the required runtime.  Please be patient.
Warning: Permanently added the ED25519 host key for IP address '2a0c:5bc0:89:ad4:ec4:7aff:fe3f:a3d5' to the list of known hosts.
Hello from 0 of 2
Hello from 1 of 2

The test-hello_upcxx-ibv gave a warning:

WARNING: Found 2 IB HCAs, but GASNet was configured without multi-rail support.  To utilize all your HCAs, you should reconfigure GASNet using '--with-ibv-max-hcas=2'.  You can silence this warning by setting the environment variable GASNET_IBV_PORTS as described in the file 'gasnet/ibv-conduit/README'.
WARNING: Found 2 IB HCAs, but GASNet was configured without multi-rail support.  To utilize all your HCAs, you should reconfigure GASNet using '--with-ibv-max-hcas=2'.  You can silence this warning by setting the environment variable GASNET_IBV_PORTS as described in the file 'gasnet/ibv-conduit/README'.
WARNING: ODP disabled on 2 of 2 processes which are missing support.
         To suppress this message set environment variable
         GASNET_ODP_VERBOSE=0 or reconfigure with --disable-ibv-odp
         (see ibv-conduit's README for more information).
         To see additional details set environment variable
         GASNET_ODP_VERBOSE=2 (or higher).
WARNING: Beginning a potentially slow probe of max pinnable memory...
WARNING: Probe of max pinnable memory completed in 26.3173s.
WARNING:   Probe of max pinnable memory has yielded '83 GB'.
WARNING:   If you have the same memory configuration on all nodes, then
WARNING:   to avoid this probe in the future either reconfigure using
WARNING:      --with-ibv-physmem-max='83 GB'
WARNING:   or run with environment variable
WARNING:      GASNET_PHYSMEM_MAX='83 GB'.
WARNING: For more information see "Slow PHYSMEM probe at start-up"
WARNING: in ibv-conduit's README.

I can look into that later!

Without further ado, here's the make tests-run output, in case it highlights obvious issues to you.

Running tests with RANKS=4
Running test-hello_upcxx-smp
make[6]: 'gasnet-smp-seq.pc' is up to date.
make[6]: 'libgasnet-smp-seq.a' is up to date.
Running test-hello_upcxx-udp
make[6]: 'gasnet-udp-seq.pc' is up to date.
cd ../other/amudp && make libamudp.a
make[8]: 'libgasnet_tools-seq.a' is up to date.
make[6]: 'libgasnet_tools-seq.a' is up to date.
FAILED (exitcode=1)
Running test-hello_upcxx-ibv
make[6]: 'gasnet-ibv-seq.pc' is up to date.
make[6]: 'libgasnet-ibv-seq.a' is up to date.
make[6]: Nothing to be done for 'all'.
Running test-atomics-smp
Test result: SUCCESS
Running test-atomics-udp
FAILED (exitcode=1)
Running test-atomics-ibv
Test result: SUCCESS
Running test-collectives-smp
Test result: SUCCESS
Running test-collectives-udp
FAILED (exitcode=1)
Running test-collectives-ibv
Test result: SUCCESS
Running test-dist_object-smp
Test result: SUCCESS
Running test-dist_object-udp
FAILED (exitcode=1)
Running test-dist_object-ibv
Test result: SUCCESS
Running test-global_ptr-smp
Test result: SUCCESS
Running test-global_ptr-udp
FAILED (exitcode=1)
Running test-global_ptr-ibv
Test result: SUCCESS
Running test-local_team-smp
Test result: SUCCESS
Running test-local_team-udp
FAILED (exitcode=1)
Running test-local_team-ibv
Test result: SUCCESS
Running test-barrier-smp
Test result: SUCCESS
Running test-barrier-udp
FAILED (exitcode=1)
Running test-barrier-ibv
Test result: SUCCESS
Running test-rpc_barrier-smp
Test result: SUCCESS
Running test-rpc_barrier-udp
FAILED (exitcode=1)
Running test-rpc_barrier-ibv
Test result: SUCCESS
Running test-rpc_ff_ring-smp
Test result: SUCCESS
Running test-rpc_ff_ring-udp
FAILED (exitcode=1)
Running test-rpc_ff_ring-ibv
Test result: SUCCESS
Running test-rput-smp
Test result: SUCCESS
Running test-rput-udp
FAILED (exitcode=1)
Running test-rput-ibv
Test result: SUCCESS
Running test-vis-smp
Test result: SUCCESS
Running test-vis-udp
FAILED (exitcode=1)
Running test-vis-ibv
Test result: SUCCESS
Running test-vis_stress-smp
Test result: SUCCESS
Running test-vis_stress-udp
FAILED (exitcode=1)
Running test-vis_stress-ibv
Test result: SUCCESS
Running test-uts_ranks-smp
Test result: SUCCESS
Running test-uts_ranks-udp
FAILED (exitcode=1)
Running test-uts_ranks-ibv
Test result: SUCCESS
Running test-rput_thread-smp
Test result: SUCCESS
Running test-rput_thread-udp
FAILED (exitcode=1)
Running test-rput_thread-ibv
Test result: SUCCESS
Running test-uts_hybrid-smp
Test result: SUCCESS
Running test-uts_hybrid-udp
FAILED (exitcode=1)
Running test-uts_hybrid-ibv
Test result: SUCCESS
Running test-view-smp
Test result: SUCCESS
Running test-view-udp
FAILED (exitcode=1)
Running test-view-ibv
Test result: SUCCESS

Dan Bonachea

unread,
Apr 28, 2021, 10:26:22 AM4/28/21
to Andrew, UPC++
Hi Andrew -

I'll leave the MHM2 build questions for Rob to answer, since he's the expert on that.

Answers to UPC++ related questions interspersed below..

On Wed, Apr 28, 2021 at 9:54 AM Andrew <andrew....@gmail.com> wrote:
And of note, this (below) is the output from make run-tests (smp, ibv and udp). I confess I have no understanding of our network topology or set up.

I had to set SSH_SERVERS environment variable first. Interestingly, it looks like test-hello_upcxx-smp and test-hello_upcxx-ibv passed and test-hello_upcxx-udp failed.

It's important to understand that the -smp, -ibv and -udp backends are all doing very different things.

In a nutshell:
smp runs all processes on a single node (your frontend)
udp runs a distributed job communicating over Ethernet (a portable, low-performance option)
ibv runs a distributed job communicating over high-performance InfiniBand hardware

Because they work differently, each has its own quirks for parallel job spawning. There's lots more details in the documentation.
 
When I call udp manually it passes:

upcxx-run -n 2 ./test-hello_upcxx-udp
INFO: may need to build the required runtime.  Please be patient.
Warning: Permanently added the ED25519 host key for IP address '2a0c:5bc0:89:ad4:ec4:7aff:fe3f:a3d5' to the list of known hosts.
Hello from 0 of 2
Hello from 1 of 2

This message: "INFO: may need to build the required runtime.  Please be patient."
indicates that your PATH is still using upcxx-run from the UPC++ build directory instead of the UPC++ install directory.
We highly recommend installing UPC++ and adding the install directory to your PATH.

 
The test-hello_upcxx-ibv gave a warning:

WARNING: Found 2 IB HCAs, but GASNet was configured without multi-rail support.  To utilize all your HCAs, you should reconfigure GASNet using '--with-ibv-max-hcas=2'.  You can silence this warning by setting the environment variable GASNET_IBV_PORTS as described in the file 'gasnet/ibv-conduit/README'.
WARNING: Found 2 IB HCAs, but GASNet was configured without multi-rail support.  To utilize all your HCAs, you should reconfigure GASNet using '--with-ibv-max-hcas=2'.  You can silence this warning by setting the environment variable GASNET_IBV_PORTS as described in the file 'gasnet/ibv-conduit/README'.
WARNING: ODP disabled on 2 of 2 processes which are missing support.
         To suppress this message set environment variable
         GASNET_ODP_VERBOSE=0 or reconfigure with --disable-ibv-odp
         (see ibv-conduit's README for more information).
         To see additional details set environment variable
         GASNET_ODP_VERBOSE=2 (or higher).
WARNING: Beginning a potentially slow probe of max pinnable memory...
WARNING: Probe of max pinnable memory completed in 26.3173s.
WARNING:   Probe of max pinnable memory has yielded '83 GB'.
WARNING:   If you have the same memory configuration on all nodes, then
WARNING:   to avoid this probe in the future either reconfigure using
WARNING:      --with-ibv-physmem-max='83 GB'
WARNING:   or run with environment variable
WARNING:      GASNET_PHYSMEM_MAX='83 GB'.
WARNING: For more information see "Slow PHYSMEM probe at start-up"
WARNING: in ibv-conduit's README.

I can look into that later!

Those warnings are all things that can/should be addressed, following the instructions provided in the messages.

It looks like you should probably re-do the UPC++ configure/build/install and use configure options:
configure --with-ibv-max-hcas=2 --with-ibv-physmem-max=83GB --disable-ibv-odp ...

Here I'm assuming you have actual InfiniBand network hardware (eg Mellanox or IBM branded) in your compute nodes, and NOT Intel Omni-Path (which emulates InfiniBand in software, so ibv-conduit is functional, but not recommended).

 Hope this helps..
-D

Andrew

unread,
Apr 28, 2021, 12:17:43 PM4/28/21
to UPC++
Thanks Dan,

I'm checking out about precise networking details. I imagine if we don't have InfiniBand and have to use UDP, that mhm2 might not be very performant?

The upcxx executable is within ~/upcxx-2021.3.0/install/bin/upcxx, and I compiled as per the instructions (~/upcxx-2021.3.0/install/bin/upcxx). 

The bin dir contains test-upcxx-install.sh, upcxx, upcxx-meta, upcxx-run, upcxx.sh

I followed the instructions from the test building command line, which directed me to the non-installed binaries:

I've confirmed that running from the install/bin dir drops the message about needing to compile.

So if I can get the make command for mhm2 working, and confirm we have InfiniBand, I may be good to go :-)

Thanks again,

Andrew

Steven Hofmeyr

unread,
Apr 28, 2021, 12:42:24 PM4/28/21
to Andrew, UPC++
Hi Andrew,

I'm a mhm2 developer. 

Yes, running with UDP will be bad for performance.

About building: the easiest way to build and install mhm2 is simply to run

./build.sh Release

from the repo root directory. 

If you do want to do it the hard way and do the individual steps explicitly, take a look inside the build script for the steps (cmake, make, etc)

Steve

Andrew

unread,
Apr 28, 2021, 12:45:28 PM4/28/21
to UPC++
Hi Steve,

Thanks for this. I had to do it the hard way because of the need to point to a proxy script for g++

I followed the instructions here (https://bitbucket.org/berkeleylab/mhm2/src/d18aa76b1efd4036b87d9a159b142a3ad45adb7d/docs/mhm_guide.md) and ran cmake (with modifications as described above) with apparent success. However, the call to make (from within the .build directory) gave an error and I do not understand the reason.

The details are in the message above (12:47) if you have a chance to review.

Best wishes,

Andrew

Steven Hofmeyr

unread,
Apr 28, 2021, 12:56:01 PM4/28/21
to Andrew, UPC++
Hi Andrew,

Comments below.

On Wed, Apr 28, 2021 at 4:47 AM Andrew <andrew....@gmail.com> wrote:
Dan and Rob,

Thanks so much for the hand-holding! You were right that the g++ warning seemed to be the issue (though the final console output made upcxx look like the culprit). As a true novice, I took the easy way out and renamed g++.sh and the compilation now works (I have guessed that I need to rm -rf CMakeFiles in the source dir each time). I also think I may have put the directory at the wrong bit of the cmake command (beginning rather than end) though I don't think that made a difference.

Interestingly, the .build folder from which I ran the cmake command remains empty (perhaps this is expected because I haven't run make yet).

However, > make -j all install

gives an error:

make: *** No rule to make target `all'.  Stop.


mkdir -p .build
cd .build
cmake -DCMAKE_INSTALL_PREFIX=path-to-install ..
make -j all install

Only my cmake command is:

cmake -DCMAKE_BUILD_TYPE=Release    -DENABLE_CUDA=OFF    -DCMAKE_INSTALL_PREFIX=/xxx/mhm2-v2.0.1.2/install    -DCMAKE_C_COMPILER=/apps/gcc/8.2.0/bin/gcc    -DCMAKE_CXX_COMPILER=/xxx/mhm2-v2.0.1.2/g++ /xxx/mhm2-v2.0.1.2

I think this is missing the path. You haven't specified where the main directory is. You can see above the '..' at the end, so you need to add that, or run:

cmake .. -DCMAKE_BUILD_TYPE=Release etc

or if your build directory is somewhere else:

cmake $repodir -Detc

Andrew

unread,
Apr 28, 2021, 1:01:07 PM4/28/21
to UPC++
Hi Steve,

I thought it was there at the end:

cmake -DCMAKE_BUILD_TYPE=Release    -DENABLE_CUDA=OFF    -DCMAKE_INSTALL_PREFIX=/xxx/mhm2-v2.0.1.2/install    -DCMAKE_C_COMPILER=/apps/gcc/8.2.0/bin/gcc    -DCMAKE_CXX_COMPILER=/xxx/mhm2-v2.0.1.2/g++ /xxx/mhm2-v2.0.1.2

Where that is the source directory (and .build is inside that). Of course I could replace with .., but that should make no difference I expect?

I had also previously run it with the directory first, and that also completed.

Best wishes,

Andrew

Steven Hofmeyr

unread,
Apr 28, 2021, 1:06:36 PM4/28/21
to Andrew, UPC++
Ah ok. So what exactly is the error you're getting? Could you post the output when you run that cmake command and the following make command?

To view this discussion on the web visit https://groups.google.com/d/msgid/upcxx/e8625f61-9ce4-414b-87d7-5bc748ccf0b8n%40googlegroups.com.

Andrew

unread,
Apr 28, 2021, 1:12:15 PM4/28/21
to UPC++
Sure,

.build > cmake -DCMAKE_BUILD_TYPE=Release    -DENABLE_CUDA=OFF    -DCMAKE_INSTALL_PREFIX=/xxx/mhm2-v2.0.1.2/install    -DCMAKE_C_COMPILER=/apps/gcc/8.2.0/bin/gcc    -DCMAKE_CXX_COMPILER=/xxx/home/mhm2-v2.0.1.2/g++ /xxx/home/mhm2-v2.0.1.2
-- The C compiler identification is GNU 8.2.0
-- The CXX compiler identification is GNU 8.2.0
-- Check for working C compiler: /apps/gcc/8.2.0/bin/gcc
-- Check for working C compiler: /apps/gcc/8.2.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /xxx/mhm2-v2.0.1.2/g++
-- Check for working CXX compiler: /xxx/mhm2-v2.0.1.2/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building MHM2 with CMake 3.14.0
-- No CUDA environment detected
-- Building for CPU only
-- Found CpuArch: sandy-bridge  found components:  mmx avx sse sse2 ssse3
-- Compiling C with -mmmx option
-- Compiling CXX with -mmmx option
-- Compiling C with -mavx option
-- Compiling CXX with -mavx option
-- Compiling C with -msse option
-- Compiling CXX with -msse option
-- Compiling C with -msse2 option
-- Compiling CXX with -msse2 option
-- Compiling C with -mssse3 option
-- Compiling CXX with -mssse3 option
Building Release version
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_CXX_COMPILER=/apps/gcc/8.2.0/bin/g++
-- CMAKE_CXX_COMPILER=/xxx/mhm2-v2.0.1.2/g++
-- ABS_UPCXX_CXX_PATH=/rds/general/applications/gcc/8.2.0/bin/g++
-- ABS_CMAKE_CXX_PATH=/xxx/mhm2-v2.0.1.2/g++
-- compiler names: g++ vs g++
-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta (found suitable version "2021.3.0", minimum required is "2020.3")
-- This UPC++ install requires the c++11 standard.
-- UPCXX_NETWORK=smp
-- UPCXX_THREADMODE=seq
-- UPCXX_CODEMODE=O3
-- UPCXX_INCLUDE_DIRS: /xxx/upcxx-2021.3.0/install/upcxx.opt.gasnet_seq.smp/include;/xxx/upcxx-2021.3.0/install/gasnet.opt/include;/xxx/upcxx-2021.3.0/install/gasnet.opt/include/smp-conduit
-- UPCXX_DEFINITIONS:  UPCXX_ASSERT_ENABLED=0;UPCXX_BACKEND=1;UPCXX_BACKEND_GASNET_SEQ=1;_GNU_SOURCE=1;GASNET_SEQ
-- UPCXX_OPTIONS:      -Wno-unused;-Wunused-result;-Wno-unused-parameter;-Wno-address
-- UPCXX_LINK_OPTIONS: SHELL:--param max-inline-insns-single=35000;SHELL:--param inline-unit-growth=10000;SHELL:--param large-function-growth=200000;-Wno-unused;-Wunused-result;-Wno-unused-parameter;-Wno-address
-- UPCXX_LIBRARIES:    -L/xxx/upcxx-2021.3.0/install/upcxx.opt.gasnet_seq.smp/lib -lupcxx -L/xxx/upcxx-2021.3.0/install/gasnet.opt/lib -lgasnet-smp-seq    -lrt -L/rds/general/applications/gcc/8.2.0/bin/../lib/gcc/x86_64-pc-linux-gnu/8.2.0 -lgcc -lm
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_CXX_COMPILER=/apps/gcc/8.2.0/bin/g++
-- CMAKE_CXX_COMPILER=/xxx/mhm2-v2.0.1.2/g++
-- ABS_UPCXX_CXX_PATH=/rds/general/applications/gcc/8.2.0/bin/g++
-- ABS_CMAKE_CXX_PATH=/xxx/mhm2-v2.0.1.2/g++
-- compiler names: g++ vs g++
-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta (found suitable version "2021.3.0", minimum required is "2020.3")
-- This UPC++ install requires the c++11 standard.
-- UPCXX_NETWORK=smp
-- UPCXX_THREADMODE=seq
-- UPCXX_CODEMODE=O3
-- Building for Release: CMAKE_CXX_FLAGS= -mmmx -mavx -msse -msse2 -mssse3
Adding ZLIB for -I/usr/include and -L/usr/lib64 -L/usr/lib)
-- Building MHM2 with a maximum kmer size of 160: 32;64;96;128;160
-- MHM2 base version 2.0.1.2 (2.0.1.2)
-- MHM2_BUILD_DATE=20210428_122840
-- Using upcxx-utils submodule in source tree
-- Did not find an installation of upcxx_utils, including submodule in src/upcxx_utils
-- Building for Release
-- UPCXX_UTILS base version 0.3.5 (0.3.5)
-- UPCXX_UTILS_BUILD_DATE=20210428_122840
-- Testing the environment..
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_CXX_COMPILER=/apps/gcc/8.2.0/bin/g++
-- CMAKE_CXX_COMPILER=/xxx/mhm2-v2.0.1.2/g++
-- ABS_UPCXX_CXX_PATH=/rds/general/applications/gcc/8.2.0/bin/g++
-- ABS_CMAKE_CXX_PATH=/xxx/mhm2-v2.0.1.2/g++
-- compiler names: g++ vs g++
-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta (found suitable version "2021.3.0", minimum required is "2020.3")
-- This UPC++ install requires the c++11 standard.
-- UPCXX_NETWORK=smp
-- UPCXX_THREADMODE=seq
-- UPCXX_CODEMODE=O3
-- Using UPCXX version - 2021.3.0 20210300
-- UPCXX_UTILS is using the UPCXX::upcxx library interface: UPCXX::upcxx;-L/xxx/home/upcxx-2021.3.0/install/upcxx.opt.gasnet_seq.smp/lib -lupcxx -L/xxx/home/upcxx-2021.3.0/install/gasnet.opt/lib -lgasnet-smp-seq    -lrt -L/rds/general/applications/gcc/8.2.0/bin/../lib/gcc/x86_64-pc-linux-gnu/8.2.0 -lgcc -lm;-L/xxx/upcxx-2021.3.0/install/upcxx.opt.gasnet_seq.smp/lib -lupcxx -L/xxx/upcxx-2021.3.0/install/gasnet.opt/lib -lgasnet-smp-seq    -lrt -L/rds/general/applications/gcc/8.2.0/bin/../lib/gcc/x86_64-pc-linux-gnu/8.2.0 -lgcc -lm
-- Linking with Threads
-- Building the Memory Allocators
-- Installing package version 0.3.5
-- UPCXX_UTILS-utils build type: Release
-- added upcxx_utils ()
-- MHM2 is using the UPCXX::upcxx library interface: UPCXX::upcxx;-L/xxx/upcxx-2021.3.0/install/upcxx.opt.gasnet_seq.smp/lib -lupcxx -L/xxx/upcxx-2021.3.0/install/gasnet.opt/lib -lgasnet-smp-seq    -lrt -L/rds/general/applications/gcc/8.2.0/bin/../lib/gcc/x86_64-pc-linux-gnu/8.2.0 -lgcc -lm
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta
-- UPCXX_CXX_COMPILER=/apps/gcc/8.2.0/bin/g++
-- CMAKE_CXX_COMPILER=/xxx/mhm2-v2.0.1.2/g++
-- ABS_UPCXX_CXX_PATH=/rds/general/applications/gcc/8.2.0/bin/g++
-- ABS_CMAKE_CXX_PATH=/xxx/mhm2-v2.0.1.2/g++
-- compiler names: g++ vs g++
-- UPCXX_VERSION_STRING: 20210300
-- UPCXX_VERSION: 2021.3.0
-- Found UPCXX: /xxx/upcxx-2021.3.0/install/bin/upcxx-meta (found version "2021.3.0")
-- This UPC++ install requires the c++11 standard.
-- UPCXX_NETWORK=smp
-- UPCXX_THREADMODE=seq
-- UPCXX_CODEMODE=O3
Building Release version
Adding ZLIB for -I/usr/include and -L/usr/lib64 -L/usr/lib)
Definitions:
  MAX_BUILD_KMER=160
  _XOPEN_SOURCE=700
  KCOUNT_QUAL_CUTOFF=20
  KCOUNT_FILTER_BAD_QUAL_IN_READ
  KCOUNT_BLOOM_FP=0.05
  KCOUNT_NO_BLOOM_ADJUSTMENT_FACTOR=0.2
  DYN_MIN_DEPTH=0.9
  USE_MINIMIZERS
  MINIMIZER_LEN=15
  CONFIG_USE_COLORS
  USE_BYTELL
  KLIGN_MAX_ALNS_PER_READ=20
  KLIGN_CTG_FETCH_BUF_SIZE=1000000
  KLIGN_SEED_SPACE=8
  KLIGN_UNALIGNED_THRES=1
  KLIGN_GPU_BLOCK_SIZE=20000
  CGRAPH_DEPTH_DIFF_THRES=0.1
  CGRAPH_MAX_DEPTH_DIFF=15
  CGRAPH_MIN_DEPTH_DIFF=5
  CGRAPH_MAX_SEARCH_LEVEL=5
  CGRAPH_MAX_QUEUE_SIZE=100
  CGRAPH_MAX_CACHE_SIZE=2000000
  CGRAPH_MAX_DEGREE=100
  CGRAPH_WALK_SUPPORT_THRES=3
  CGRAPH_GAP_CLOSING_OVERLAP_MISMATCH_THRES=1
  CGRAPH_SPAN_OVERLAP_MISMATCH_THRES=1
  CGRAPH_MAX_MISMATCHES_THRES=3
  CGRAPH_MIN_TNF_CLEN=1000
  CGRAPH_WALK_TNF_THRES=0.75
  ALN_MATCH_SCORE=1
  ALN_MISMATCH_COST=1
  ALN_GAP_OPENING_COST=1
  ALN_GAP_EXTENDING_COST=1
  ALN_AMBIGUITY_COST=1
  LASSM_MAX_KMER_LEN=121
  LASSM_MIN_KMER_LEN=21
  LASSM_SHIFT_SIZE=8
  LASSM_MIN_QUAL=10
  LASSM_MIN_HI_QUAL=20
  LASSM_MIN_VIABLE_DEPTH=0.2
  LASSM_MIN_EXPECTED_DEPTH=0.3
  LASSM_RATING_THRES=0
  LASSM_MAX_COUNT_MERS_READS=3000
MHM2_LINK_LIBRARIES: Threads::Threads;SSW_LIBRARY;/usr/lib64/libz.so;MHM2_VERSION_LIB
-- Configuring done
-- Generating done
-- Build files have been written to: /xxx/mhm2-v2.0.1.2

.build > make -j all install
make: *** No rule to make target `all'.  Stop.

Thanks,

Andrew

Steven Hofmeyr

unread,
Apr 28, 2021, 1:32:23 PM4/28/21
to Andrew, UPC++
It looks like the build files are getting written to the wrong location:

-- Build files have been written to: /xxx/mhm2-v2.0.1.2

That should be 

/xxx/mhm2-v2.0.1.2/.build

But there's an easier way to do this. First delete the .build directory, and then try the following two commands from the repo root:

export MHM2_CMAKE_EXTRAS="-DCMAKE_C_COMPILER=/apps/gcc/8.2.0/bin/gcc -DCMAKE_CXX_COMPILER=/xxx/home/mhm2-v2.0.1.2/g++"
./build.sh Release



Andrew

unread,
Apr 28, 2021, 2:36:08 PM4/28/21
to UPC++
Interestingly, even following this the build script console log ends with:

...
-- Configuring done
-- Generating done
-- Build files have been written to: /xxx/mhm2-v2.0.1.2
make: *** No rule to make target `all'.  Stop.

Is there any reason build files would consistently go to the wrong place?

Thanks,

Andrew

Steven Hofmeyr

unread,
Apr 28, 2021, 2:37:45 PM4/28/21
to Andrew, UPC++
Hi Andrew,

I'm moving this conversation out of the upc++ mailing list, since it seems to be more about MHM at this point.

Steve

To view this discussion on the web visit https://groups.google.com/d/msgid/upcxx/e8a66aef-ef09-46d6-b579-00b92ca036efn%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages