ABI Compatibility Checkers?

[Morgan, Ben]

Hi all,

Chris Burr’s presentation on Conda reminded me about the package compatibility problem (e.g. how to avoid rebuilding a package when it’s ABI/API hasn’t changed).

On the ABI side, is anyone, or colleagues, using some of the ABI checker tools as part of CI/Release Management (e.g. abi-compliance-checker)? Whilst this is closer to “development” than “packaging”,
it sounds like packaging is made easier if a project provides information with each release on ABI changes,
so perhaps useful to discuss.

Cheers,

Ben.

--
==================
Dr. Ben Morgan
Senior Research Fellow
Department of Physics
University of Warwick
Coventry CV4 7AL
==================

[Chris Burr]

Hi Ben,

Conda's docs about this are here, though they're missing a few of the newer features.

When finishing the ROOT recipe, I looked at this when deciding how to pin XRootD:

https://abi-laboratory.pro/index.php?view=timeline&l=xrootd

So far as I know the current conda system requires the version pinning is known at build time[1] so I have to guess if a hypothetical XRootD 4.8.6 will provide the same ABI as 4.8.5. Pinning on minor version looks reasonable but there has been two patch releases that broke the ABI. That said, ROOT probably doesn't use the parts of the ABI that changed so I could pin on minor version and hope for the best!

The verification step[2] after building is still being developed so I wouldn't be surprised if conda eventually automates this if it's possible to do so.

Cheers,

Chris

[1] The XRootD package specifies how it should be pinned, and the ROOT recipe can override this if appropriate. It is then included as metadata in the tarball that is distributed.

[2] Example from building XRootD, look at the log in the "./.circleci/run_docker_build.sh" step of https://circleci.com/gh/conda-forge/xrootd-feedstock/23. If you search for "INFO:conda_build.build:Packaging" you can some of the things that are already checked and the warnings will likely become errors in the future.

--
You received this message because you are subscribed to the Google Groups "HSF Build and Packaging Tools Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hsf-packaging-...@googlegroups.com.
To post to this group, send email to hsf-pack...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hsf-packaging-wg/FA97D261-2B31-4DD5-8D3D-605F9046D719%40warwick.ac.uk.
For more options, visit https://groups.google.com/d/optout.

[Patrick E Gartung]

Hi Ben,

As far as I know Spack enforces ABI compatibility because all packages are built with the same compiler and use the default compiler standard. It is up to the package maintainers to ensure the correct compiler std flags are used consistently among dependent packages, especially external packages from the system. There is a class to check ABI compatibility of libstdc++ https://github.com/spack/spack/blob/develop/lib/spack/spack/abi.py but I don't see any uses of it.

I think CMS has already encountered an ABI compatibility between Nvidia binaries compiled with the system compiler on SL6 and CMS binaries compiled with gcc 7.3 and -std=c++17.

Patrick

--
You received this message because you are subscribed to the Google Groups "HSF Build and Packaging Tools Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hsf-packaging-...@googlegroups.com.
To post to this group, send email to hsf-pack...@googlegroups.com.

To view this discussion on the web visit https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msgid_hsf-2Dpackaging-2Dwg_FA97D261-2D2B31-2D4DD5-2D8D3D-2D605F9046D719-2540warwick.ac.uk&d=DwIFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=aZ0rCqVqzdRNtV7duJqHzA&m=yoC6j3K4QAxvRDT8_weYcUrnbYHolv0scp6ql0CfDeQ&s=wKMelfEQPGm1aAROXgOfo-Fzdy_rst5YSCaKCJAjB0U&e=.
For more options, visit https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_optout&d=DwIFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=aZ0rCqVqzdRNtV7duJqHzA&m=yoC6j3K4QAxvRDT8_weYcUrnbYHolv0scp6ql0CfDeQ&s=t-OIT542Hp2kJrObpaqOcF4RGJdPvkELGs-chfaTcfs&e=.

[Patrick E Gartung]

On closer examination, Spack does attempt to check ABI compatibility of external and virtual packages by checking the default or system compiler libstdc++ during the concretization process:

https://github.com/spack/spack/blob/develop/lib/spack/spack/concretize.py#L131

[David Lange]

On Wed, Jan 30, 2019 at 5:15 PM Patrick E Gartung <gar...@fnal.gov> wrote:
>
> Hi Ben,
>
> As far as I know Spack enforces ABI compatibility because all packages are built with the same compiler and use the default compiler standard. It is up to the package maintainers to ensure the correct compiler std flags are used consistently among dependent packages, especially external packages from the system. There is a class to check ABI compatibility of libstdc++ https://github.com/spack/spack/blob/develop/lib/spack/spack/abi.py but I don't see any uses of it.
>
> I think CMS has already encountered an ABI compatibility between Nvidia binaries compiled with the system compiler on SL6 and CMS binaries compiled with gcc 7.3 and -std=c++17.

not Nvidia (afaik) but Oracle occi libraries (which were compiled
however Oracle compiles them rather than with the SL6 system
compiler). This ABI incompatibility wasn't necessarily fatal - though
it was in our case (at link time).

david

> To view this discussion on the web visit https://groups.google.com/d/msgid/hsf-packaging-wg/FBB85E22-11E6-4E9B-B8F9-2562EAC77711%40fnal.gov.
> For more options, visit https://groups.google.com/d/optout.