Announcing OpenCoarrays 2.6.1

[Zaak Beekman]

-----BEGIN PGP SIGNED MESSAGE-----

Hash: SHA512

Greetings,

We are pleased to announce OpenCoarrays 2.6.1!

Open Coarrays 2.6.1 provides better support for:

* find_package(opencoarrays)

* relocatable installation

* reproducible builds

* transitive usage requirements via CMake target properties

The reproducibility and relocatability has only been thoroughly tested on macOS Mojave so far, but it should apply to Linux too. On macOS here are the recommended environment variable settings for creating reproducible builds:

SOURCE_DATE_EPOCH=<date of last commit/release in unix time> # for 2.6.1 this is 1553191566

ZERO_AR_DATE=1

TZ=UTC

LC_ALL=C

In addition, to experiment with the checksums of the build assets, a utility target `hash_installed` can be "built" after running `make install`. A subsequent `make install` when the sha256 checksums of the installed assets are present will install this manifest into <DATADIR>/opencoarrays/ The SHA256 checksummed installation manifest is of a format that should be suitable for verification with, e.g., `shasum -c sha256_install_manifest.txt`.

Reproducibility and relocatability have not been thoroughly tested on platforms other than macOS, so please report back with any issues you may encounter.

Furthermore, when consuming OpenCoarrays from other CMake based projects, the preferred method is now to use `find_package(OpenCoarrays)`. Then, you should link to either `OpenCoarrays::mpi_caf` if you want to use the default GFortran functionality, or link to `OpenCoarrays::opencoarrays_mod` if you want to `use opencoarrays` from your main program to access the OpenCoarrays extensions.

Some work remains to further simplify and abstract the build system, paying down technical debt. The top level CMakeLists.txt still has a few `add_definitions()` that should be changed to attach definitions to targets or meta-targets. A bunch of the introspection should be pushed into CMake modules. Introspection should be added to test for the compiler flags required for creating reproducible builds, and an option added to allow default builds. These improvements should continue to be implemented over the coming weeks and months.

Cheers,

Zaak

P.S. For your reference, when built on macOS mojave, using GCC/GFortran 8.3 and MPICH 3.3, these are the sha256 checksums of the build artifacts that I obtain:

440aad6cd173c4f39a27fcdd00674480a3203c78e774b227b8dcd6f20b409228 /usr/local/share/man/man1/caf.1

74eeb03b4c610d9f3f1f56719b10d18cad1e1bb53a17c271155d5fe0c563e02e /usr/local/share/man/man1/cafrun.1

2f050881cf9af775a2eb4887d434ba6021a65ad006afc44aa3a40f243c2e7bf6 /usr/local/include/OpenCoarrays-2.6.1_GNU-8.3.0/opencoarrays.mod

2f050881cf9af775a2eb4887d434ba6021a65ad006afc44aa3a40f243c2e7bf6 /usr/local/include/opencoarrays.mod

09200e6d71d5f72b7a6e64349852781aa160698dfb2d527221c63ffd852a0e2f /usr/local/lib/libopencoarrays_mod.a

27b513290111f1ad55c93187254e9fc2e5dc3b01929d606b24feefb55b9bd67f /usr/local/lib/libcaf_mpi.3.dylib

27b513290111f1ad55c93187254e9fc2e5dc3b01929d606b24feefb55b9bd67f /usr/local/lib/libcaf_mpi.dylib

e0ce7f5281640d4150306417340a6b362f7758341297049f61d8d52066129394 /usr/local/lib/libcaf_mpi.a

a7e8e1dd1beb590390d08f3b15a9467034fd45e3c514c9cebc3771f63e39cde0 /usr/local/bin/cafrun

fb2c73d2546c183e10c655601d47e42bb4807ed8b98da8b75e58bcbac6e4dd97 /usr/local/bin/caf

c476578320505faf59f4de828025bfedaae469b7dca607b35e77875568edbb9b /usr/local/include/libcaf.h

cc12011c52989ff450dac1e57249a03584d2e40fa7661ea2c0c0d2b51dbc754a /usr/local/include/libcaf-gfortran-descriptor.h

76ad4efaf697f740c186aea7b3eb4daa6efdf3e98f8c11cee5921453e1928cd3 /usr/local/include/libcaf-version-def.h

6ec62c2b8bbee3b82640e4d1b71979bc3717b7ca1448652436ef420ef54b8eb8 /usr/local/lib/cmake/opencoarrays/OpenCoarraysTargets.cmake

101989e7fb319ab66dd8468e47ce7d9294a1b78c97eb2de27218672f649926e9 /usr/local/lib/cmake/opencoarrays/OpenCoarraysTargets-release.cmake

69e48af077f11c91ce632abc1606f387c6c7a5e5ef5df15911b1abbf5c684064 /usr/local/lib/cmake/opencoarrays/OpenCoarraysConfig.cmake

daad7bedf60b541c67b9a462f37866c720f53be5c2474603c1e7a40939a4ea87 /usr/local/lib/cmake/opencoarrays/OpenCoarraysConfigVersion.cmake

The compilation was achieved by setting the variables listed above, and also:

```

brew unlink openmpi

brew link mpich

export FC="$(which gfortran-8)"

export CC="$(which gcc-8)"

mkdir build

cd build

cmake -Wdev ..

make -j install

make hash_installed

ctest # verify installation

make -j install # Install the manifest with hashes

```

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEqUSJ2TjLk2Zo+QkRqTznDYAhvQ8FAlyT614ACgkQqTznDYAh

vQ+ZCA/8DjGechYyzDJ/awBTWyPU7QYIffDcJ1HBo9elPWZupNeaupc1yYODM09M

XPcO6NZkJzFYXQhbxQ3ti4SimmhBLa6d7s3Gia5tOQ3ZYZmofYh6bBxuBPhwQi1N

UR5iL/q7fJgzP+4vrNxnKPcPUz5faU3Sq89bEww0cOPh3iLnjJkTa8zoHWYV5R1K

kd5h22qEXPztflZAQHYnuq2R9AiEnkIbTaeEYUw8dGCYveg5chgc5lGmm8so6L76

u0gJjZqUHzkCElt+VmFC04GbpXXPvazvz0EaWN5qqXlIu16s2eSHjIQVD8PjH8nD

9xnaHq31Z1lGNdONjipOJHXxF58DDcJZVgHlNKHTR2Hu/zqCdX+7WrvXN21MA9BS

obM9/mEKUw2SiqHE7kvGrvUe+bvK3rbIS1tFo0irj4BgERF7NlSAsshxZaNaywej

FRRj/Bp0cttTtVXOOpuc6iwOtWSVBK85zrLw5Z8H6ssIpq37fOrotVdgvqqfqksG

5eWewcVn1+i6pq1EqUiJHqJ6oQGeox+1Rlt7+G1HM6OOqydTJSQYNFQvoe66nVlZ

39WxZTYlFcaoF1zb9igAhpfUndEL40T5aehiSBmJCQ2lOk/7m1f/pWFkf5ta06FB

6tB6G57wp4TLT35hhbSHG0/qJb5rSL6aDDeNIagjPftKUcuCXIg=

=a0a3

-----END PGP SIGNATURE-----

[Vishnu V. Krishnan]

I just compiled it on Archlinux, and I see the following message as part of the build output:

==> WARNING: Package contains reference to $srcdir
usr/lib/libopencoarrays_mod.a

I think this breaks reproducibility, does it not?

[Zaak Beekman]

Hmmmm. I’m guessing that’s in the `caf` or `cafrun` script. Can you grep through the strings in the installation artifacts and figure out which file is doing that?

Also, you can have absolute and relative paths so long as others can transform their own paths to match yours.

That path in particular looks like a path in the install tree.

Final request: can you please remind me what district you’re on?

--
You received this message because you are subscribed to a topic in the Google Groups "OpenCoarrays" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/opencoarrays/9kaCMJwqA3o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to opencoarrays...@googlegroups.com.
Visit this group at https://groups.google.com/group/opencoarrays.
To view this discussion on the web visit https://groups.google.com/d/msgid/opencoarrays/6ef7857c-427c-43f4-8303-094180345c1a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Vishnu V. Krishnan]

I think we've discussed this before.

No, the reference is in the file `/usr/lib/libopencoarrays_mod.a`. The issue is that there is a mention of the location it was built. In this case, it is:




At line 590 of file /<build_directory>/opencoarrays/src/OpenCoarrays-2.6.1/src/extensions/opencoarrays.F90

[Zaak Beekman]

Vishnu,

Which compilers are you using? Have you fetched the latest 2.6.1
version? It should be scrubbing source file names from the build
directory (at least with gcc-8) due to `-fno-working-directory`. I'm
planning to add better introspection to test for that flag and others.
It is possible on linux that we also need to set ar flags, since macOS
uses ar from apple. Let me try to add a commit to a branch to see if
that will resolve the issue (and test on Fedora 29)

Thanks,
Zaak

> --
> You received this message because you are subscribed to a topic in the Google Groups "OpenCoarrays" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/opencoarrays/9kaCMJwqA3o/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to opencoarrays...@googlegroups.com.
> Visit this group at https://groups.google.com/group/opencoarrays.

> To view this discussion on the web visit https://groups.google.com/d/msgid/opencoarrays/fb8a5747-6760-4519-91b0-cfc787c1633e%40googlegroups.com.

[Vishnu V. Krishnan]

I'm using gcc version 8.2.1 20181127 (GCC)

Yes, I'm using the latest release. I compiled it using the AUR build-script:
https://aur.archlinux.org/packages/opencoarrays/

[Zaak Beekman]

Hi Vishnu,

I've isolated the issue and developed a work around. As far as I can
tell, this is due to a bug in GFortran: It is providing an error
message for the `repeat()` intrinsic function with a correlation to
the file and line number, even when flags are passed to try to
eliminate build paths. I haven't tested if `-P` might suppress this
output, but, at any rate, I believe this to be a bug.

Would it be possible for you to test the reproducibility of the build
using the `gfortran-non-determinism-repeat` branch available at
https://github.com/sourceryinstitute/OpenCoarrays/tree/gfortran-non-determinism-repeat
?

Thanks,
Zaak

> --
> You received this message because you are subscribed to a topic in the Google Groups "OpenCoarrays" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/opencoarrays/9kaCMJwqA3o/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to opencoarrays...@googlegroups.com.
> Visit this group at https://groups.google.com/group/opencoarrays.

> To view this discussion on the web visit https://groups.google.com/d/msgid/opencoarrays/1f934cdd-0176-431f-9882-a35d989203c3%40googlegroups.com.

[Vishnu V. Krishnan]

Hey Zaak!

I just built that branch, and can confirm that there is no longer a reference to the build location.

Thanks!