Test program?

[Powell, Amy Jo]

Hello,

I am attempting to build / install merbench, which requires UPC.

 

I had compile time failures of metamer.

 

The (merbench) errors point to a problem with the `upcc` compiler:

 

```

…

Testing UPC Compiler

-- Check for working UPC compiler: /ascldap/users/ajpowel/upc_exabiome/berkeley_upc-2022.5.0/install/bin/upcc

Loaded cached UPC Compiler Variables /ascldap/users/ajpowel/upc_exabiome/berkeley_upc-2022.5.0/install/bin/upcc

Building CMake UPC Information and Rules: /ascldap/users/ajpowel/upc_exabiome/berkeley_upc-2022.5.0/install/bin/upcc

Looking for Compiler/BUPC-UPC or Platform/Linux-BUPC-UPC or Platform/Linux-upcc or Platform/Linux

Loading Compiler/BUPC-UPC

-- Check for working UPC compiler: /ascldap/users/ajpowel/upc_exabiome/berkeley_upc-2022.5.0/install/bin/upcc -- broken

The UPC compiler "/ascldap/users/ajpowel/upc_exabiome/berkeley_upc-2022.5.0/install/bin/upcc" is not able to compile a simple test program.

It fails with the following output:

Change Dir: /ascldap/users/ajpowel/exabiome_merbench/merbench-code/build/CMakeFiles/CMakeTmp

 

Run Build Command(s):/usr/bin/gmake -f Makefile cmTC_a9dda/fast && /usr/bin/gmake  -f CMakeFiles/cmTC_a9dda.dir/build.make CMakeFiles/cmTC_a9dda.dir/build

gmake[1]: Entering directory `/home/ajpowel/exabiome_merbench/merbench-code/build/CMakeFiles/CMakeTmp'

Building UPC object CMakeFiles/cmTC_a9dda.dir/testUPCCompiler.o

/ascldap/users/ajpowel/upc_exabiome/berkeley_upc-2022.5.0/install/bin/upcc    -o CMakeFiles/cmTC_a9dda.dir/testUPCCompiler.o -c /ascldap/users/ajpowel/exabiome_merbench/merbench-code/build/CMakeFiles/CMakeTmp/testUPCCompiler.upc

upcc: Error: Can't connect to http://upc-translator.lbl.gov/upcc-2022.5.0.cgi: Connection timed out

…

 

```

 

So, I’m attempting to compile a “hello” test program, but it either hangs --

 

[ajpowel@kokkos-dev-2 bin]$ ./upcc -c hello_c.c

 

Or fails when the `--http-proxy` option is passed:

 

[ajpowel@kokkos-dev-2 bin]$ ./upcc --http-proxy=http://proxy.sandia.gov:80 -c hello_c.c

upcc: HTTP error: http://upc-translator.lbl.gov/upcc-2022.5.0.cgi 403 Forbidden

 

Can someone help me develop a solid build / install of UPC (to use with merbench)?

 

Many thanks for any support you can offer.

 

Best,

 

AJP

[Paul H. Hargrove]

Amy,

The preferred solution to your problem depends on which of multiple options you want for UPC code translation.
As I responded in your other thread, "cupc2c" is the one I believe MerBench prefers.

You should find additional information regarding cucp2c in the INSTALL.txt in the Berkeley UPC sources.
However, for your case I suggest (as noted in the other thread) that you first see what recommendations the ExaBiome team has regarding the preferred way to configure Berkeley UPC for their application.

The issue with a 403 error when using a http proxy might be a genuine problem independent of everything else.

We certainly don't test that configuration regularly.  So, I will try to reproduce that problem when I am able.

-Paul

--
You received this message because you are subscribed to the Google Groups "upc-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to upc-users+...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/upc-users/D3FAF0CA-603E-4E2F-820B-7633C408DE4A%40sandia.gov.

--

Paul H. Hargrove <PHHar...@lbl.gov>
Pronouns: he, him, his

Computer Languages & Systems Software (CLaSS) Group

Computer Science Department

Lawrence Berkeley National Laboratory

[Paul H. Hargrove]

Amy,

I have verified that the `--http-proxy` option is working on at least one system where it is needed:

$ upcc --translator http://upc-translator.lbl.gov/upcc-2022.5.0.cgi hello.upc && echo PASS || echo FAIL

upcc: Error: Can't connect to http://upc-translator.lbl.gov/upcc-2022.5.0.cgi: Network is unreachable

FAIL

$ upcc --translator http://upc-translator.lbl.gov/upcc-2022.5.0.cgi --http-proxy=http://proxy.ftm.alcf.anl.gov:3128 hello.upc && echo PASS || echo FAIL

PASS

So, I cannot account for your 403 error, unless it is the case that your local proxy is configured to limit which URLs you can access.

However, I do not know how to determine if that is the case or not.

-Paul

[Powell, Amy Jo]

Hi Paul et al.,

Many thanks for your rapid reply.

 

I am adding my collaborator, Gavin Conant, to this thread – we’re both struggling with UPC (and anticipate future struggles with Merbench and MetaHipMer, but we’ll set those aside for the moment).

 

Back to UPC; here’s what I see when I try your commands, I end up hitting this flavor of error:

 

```

upcc: HTTP error: http://upc-translator.lbl.gov/upcc-2022.5.0.cgi 403 Forbidden

```

 

Here is the basic command I’m trying to execute on an NVIDIA box:

 

```

./upcc \

--translator http://upc-translator.lbl.gov/upcc-2022.5.0.cgi \

--http-proxy http://proxy.sandia.gov:80 \

--network smp \

hello.upc

 

```

 

I can do a generic command to that URL:

 

```

wget  https://upc-translator.lbl.gov/upcc-2022.5.0.cgi

--2022-09-09 15:25:17--  https://upc-translator.lbl.gov/upcc-2022.5.0.cgi

Resolving proxy.sandia.gov (proxy.sandia.gov)... 10.192.2.17

Connecting to proxy.sandia.gov (proxy.sandia.gov)|10.192.2.17|:80... connected.

ERROR: cannot verify upc-translator.lbl.gov's certificate, issued by ‘/C=US/O=Let's Encrypt/CN=R3’:

  Issued certificate has expired.

To connect to upc-translator.lbl.gov insecurely, use `--no-check-certificate'.

[ajpowel@kokkos-dev-2 berkeley_upc-2022.5.0]$

[ajpowel@kokkos-dev-2 berkeley_upc-2022.5.0]$ wget  http://upc-translator.lbl.gov/upcc-2022.5.0.cgi

--2022-09-09 15:25:31--  http://upc-translator.lbl.gov/upcc-2022.5.0.cgi

Resolving proxy.sandia.gov (proxy.sandia.gov)... 10.192.2.17

Connecting to proxy.sandia.gov (proxy.sandia.gov)|10.192.2.17|:80... connected.

Proxy request sent, awaiting response... 200 OK

Length: 87 [text/plain]

Saving to: ‘upcc-2022.5.0.cgi’

 

100%[============================================================================================================>] 87          --.-K/s   in 0s     

 

2022-09-09 15:25:31 (6.49 MB/s) - ‘upcc-2022.5.0.cgi’ saved [87/87]

```

 

Do you have any suggestions about what I might try next?

 

Best,

 

AJP

 

 

 

 

 

 

[ajpowel@kokkos-dev-2 bin]$ ./upcc --translator ./upcc --translator http://upc-translator.lbl.gov/upcc-2022.5.0.cgi

upcc: Error: no input files

[ajpowel@kokkos-dev-2 bin]$ ./upcc --translator http://upc-translator.lbl.gov/upcc-2022.5.0.cgi hello.upc && echo PASS || echo FAIL

upcc: Error: Can't connect to http://upc-translator.lbl.gov/upcc-2022.5.0.cgi: Connection timed out

FAIL

 

 

[ajpowel@kokkos-dev-2 bin]$ ./upcc --translator http://upc-translator.lbl.gov/upcc-2022.5.0.cgi --http-proxy=http://proxy.sandia.gov hello.upc && echo PASS || echo FAIL

upcc: HTTP error: http://upc-translator.lbl.gov/upcc-2022.5.0.cgi 403 Forbidden

FAIL

 

 

[ajpowel@kokkos-dev-2 bin]$ env | grep -i proxy

http_proxy=http://proxy.sandia.gov:80

HTTPS_PROXY=nop...@proxy.sandia.gov:80>http://user:nop...@proxy.sandia.gov:80

https_proxy=http://proxy.sandia.gov:80

no_proxy=127.0.0.1,localhost,.sandia.gov

HTTP_PROXY=nop...@proxy.sandia.gov:80>http://user:nop...@proxy.sandia.gov:80

[Paul H. Hargrove]

Amy and Gavin,

The only thing I am entirely certain about in that output is the certificate expiration message.

Though not relevant to getting BUPC working, the server's certificate does not expire until Oct 23.

I suspect the expiration is of a certificate used to cross-sign LetsEncrypt's cert (see here)

The `upcc` compiler driver does not support `https:`,only `http:`

So, I don't think the certificate is relevant to the original 403 error, unless it is the case that your proxy is converting http to https.

I believe the best option is to avoid use of the network-based translation service entirely.

To do so, one should locally build either the Berkeley UPC source-to-source translator or the Clang UPC-to-C (CUPC2C) source-to-source translator.

The INSTALL.TXT in the Berkeley UPC sources briefly describe these two options and provide pointers to their respective download and installation instructions.

As I've mentioned earlier, I suspect that the ExaBiome codes prefer (perhaps even require) CUPC2C and it is generally the more stable option.

So, I recommend pursuing a local installation of CUPC2C first.

If I am not mistaken, the ExaBiome team has a script to fully automate the download and build of BUPC with CUPC2C.

If that is not the case, I can provide relatively simple instructions.

-Paul