ABI incompatible change in ATF?

[Craig Rodrigues]

Hi,

I did the following:

(1)  took the FreeBSD-HEAD, checked it out, and built the world
(2)  I built kyua and atf from git and installed them
(3)  I ran the tests:

https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/6/

I saw some failures in lutok.  I ran this from the command-line:

/usr/test/local/lutok/c_gate_test local.lutok.c_gate_test

and got:

/usr/local/tests/lutok/c_gate_test: Undefined symbol "_ZN3atf5tests6run_tpEiPKPcPFvRNSt3__16vectorIPNS0_2tcENS4_9allocatorIS7_EEEEE"

The demangled name is:

"atf::tests::run_tp(int, char* const*, void (*)(std::__1::vector<atf::tests::tc*, std::__1::allocator<atf::tests::tc*> >&))"


So it looks like the ATF libraries in the FreeBSD base system are

ABI incompatible with the ATF libraries from a git checkout?

Any ideas?

--

Craig

     

[Julio Merino]

On Tue, Sep 30, 2014 at 10:00 AM, Craig Rodrigues <rod...@freebsd.org> wrote:
> So it looks like the ATF libraries in the FreeBSD base system are
>
> ABI incompatible with the ATF libraries from a git checkout?

Yes. ATF is 0.x intentionally to make no ABI guarantees for the time
being and there was a change in that function if I recall correctly --
but I'd need to look to be sure.

Also, and according to libtool best practices (which I don't
understand for this specific point, honestly), the ABI version of the
library should only be bumped in the source tree at the moment of
release, not earlier. This is why the libraries in Git still carry
the same major version as the ones in FreeBSD.

[Craig Rodrigues]

On Tue, Sep 30, 2014 at 1:46 AM, Julio Merino <jm...@meroh.net> wrote:

On Tue, Sep 30, 2014 at 10:00 AM, Craig Rodrigues <rod...@freebsd.org> wrote:
> So it looks like the ATF libraries in the FreeBSD base system are
>
> ABI incompatible with the ATF libraries from a git checkout?

Yes.  ATF is 0.x intentionally to make no ABI guarantees for the time
being and there was a change in that function if I recall correctly --
but I'd need to look to be sure.

OK, I hacked the Makefiles in my git checkout so that the libraries it had

were different versions than the ATF versions in the FreeBSD base system.

I re-ran the kyua test, but still got errors in the lutok tests

https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/lastCompletedBuild/

Is there a way to be able to run the FreeBSD ATF tests
with a kyua and atf checked out from git?

On a side not, I fixed the pkill and pgrep tests in FreeBSD.

You'll see in the latest test run that they are not failing any more.

--

Craig

[Julio Merino]

On Tue, Sep 30, 2014 at 10:56 PM, Craig Rodrigues <rod...@freebsd.org> wrote:

On Tue, Sep 30, 2014 at 1:46 AM, Julio Merino <jm...@meroh.net> wrote:

On Tue, Sep 30, 2014 at 10:00 AM, Craig Rodrigues <rod...@freebsd.org> wrote:
> So it looks like the ATF libraries in the FreeBSD base system are
>
> ABI incompatible with the ATF libraries from a git checkout?

Yes.  ATF is 0.x intentionally to make no ABI guarantees for the time
being and there was a change in that function if I recall correctly --
but I'd need to look to be sure.

OK, I hacked the Makefiles in my git checkout so that the libraries it had

were different versions than the ATF versions in the FreeBSD base system.

I re-ran the kyua test, but still got errors in the lutok tests

https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/lastCompletedBuild/

Is there a way to be able to run the FreeBSD ATF tests
with a kyua and atf checked out from git?

Kyua yes, mostly.

ATF no. It's part of the base system, so if you want to use a different ATF version, you have to properly replace the one in base. Same as with ports: if you want to use a different ATF, you'll have to properly update the port in place and rebuild all consumers as needed. Exactly the same as with any other library shipped by base or ports.

-- 

Julio Merino | BCD / PD SRE | jm...@google.com | +1 (347) 694-0576

[Craig Rodrigues]

On Wed, Oct 1, 2014 at 5:07 AM, 'Julio Merino' via kyua-discuss <kyua-d...@googlegroups.com> wrote:

Is there a way to be able to run the FreeBSD ATF tests
with a kyua and atf checked out from git?

Kyua yes, mostly.

ATF no. It's part of the base system, so if you want to use a different ATF version, you have to properly replace the one in base. Same as with ports: if you want to use a different ATF, you'll have to properly update the port in place and rebuild all consumers as needed. Exactly the same as with any other library shipped by base or ports.

If I "pkg install kyua", this pulls in the lutok package via a dependency.

If I do:
cd /usr/ports/devel/kyua
make -V RUN_DEPENDS

it shows atf.  If I am running the tests in the FreeBSD base system under kyua,
does this mean that if I install kyua from ports,
I need to make sure that atf is exactly the same version as in the
base system, and if it is not, I  need to forcefully uninstall the atf package?

--

Craig

[Julio Merino]

The atf in the base system is private (installed under lib/private/, no .pc files provided) to prevent anything outside of the base system from picking it up unexpectedly.

Ports explicitly depend on the ports-provided version of atf to prevent conflicts with the base system.

atf in base can be different from ports. This is expected to happen and therefore is a supported configuration, as long as you use the atf bundled within the base src tree and the atf package for ports.

Now... if you have any environment variables (say, CFLAGS or PKG_CONFIG_PATH) pointing at a custom copy of ATF... you'll have a bad time with ports. See https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182909 . Or you'll also have problems if you forcibly "make install" a git build under /usr/local/, which is doomed to break all sorts of stuff.

(By the way: this is all FreeBSD stuff and thus should have been discussed in freebsd-testing.)

[Craig Rodrigues]

On Wed, Oct 1, 2014 at 9:26 AM, 'Julio Merino' via kyua-discuss <kyua-d...@googlegroups.com> wrote:

On Wed, Oct 1, 2014 at 12:17 PM, Craig Rodrigues <rod...@freebsd.org> wrote:

On Wed, Oct 1, 2014 at 5:07 AM, 'Julio Merino' via kyua-discuss <kyua-d...@googlegroups.com> wrote:

Is there a way to be able to run the FreeBSD ATF tests
with a kyua and atf checked out from git?

Kyua yes, mostly.

ATF no. It's part of the base system, so if you want to use a different ATF version, you have to properly replace the one in base. Same as with ports: if you want to use a different ATF, you'll have to properly update the port in place and rebuild all consumers as needed. Exactly the same as with any other library shipped by base or ports.

If I "pkg install kyua", this pulls in the lutok package via a dependency.

If I do:
cd /usr/ports/devel/kyua
make -V RUN_DEPENDS

it shows atf.  If I am running the tests in the FreeBSD base system under kyua,
does this mean that if I install kyua from ports,
I need to make sure that atf is exactly the same version as in the
base system, and if it is not, I  need to forcefully uninstall the atf package?

The atf in the base system is private (installed under lib/private/, no .pc files provided) to prevent anything outside of the base system from picking it up unexpectedly.

Ports explicitly depend on the ports-provided version of atf to prevent conflicts with the base system.

atf in base can be different from ports. This is expected to happen and therefore is a supported configuration, as long as you use the atf bundled within the base src tree and the atf package for ports.

In my case, I have modified versions of the kyua port and the atf port to pull in newer versions.

If the kyua port pulls in a newer version of the atf port, but the FreeBSD base system has an older version
of atf, if I do "kyua test" in the /usr/test directory on FreeBSD, shouldn't that use the ATF libraries in the base system
and not from the atf port?

--

Craig

[Julio Merino]

On Wed, Oct 1, 2014 at 9:19 PM, Craig Rodrigues <rod...@freebsd.org> wrote:

In my case, I have modified versions of the kyua port and the atf port to pull in newer versions.

If the kyua port pulls in a newer version of the atf port, but the FreeBSD base system has an older version
of atf, if I do "kyua test" in the /usr/test directory on FreeBSD, shouldn't that use the ATF libraries in the base system
and not from the atf port?

Correct. Why do you think that's not happening?

[Craig Rodrigues]

On Wed, Oct 1, 2014 at 1:19 PM, 'Julio Merino' via kyua-discuss <kyua-d...@googlegroups.com> wrote:

On Wed, Oct 1, 2014 at 9:19 PM, Craig Rodrigues <rod...@freebsd.org> wrote:

In my case, I have modified versions of the kyua port and the atf port to pull in newer versions.

If the kyua port pulls in a newer version of the atf port, but the FreeBSD base system has an older version
of atf, if I do "kyua test" in the /usr/test directory on FreeBSD, shouldn't that use the ATF libraries in the base system
and not from the atf port?

Correct. Why do you think that's not happening?

Because I'm getting run-time errors from the lutok tests that seem to be
related to pulling in the pkg-installed version of the ATF libraries:  https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/7/

--

Craig

[Julio Merino]

On Wed, Oct 1, 2014 at 10:24 PM, Craig Rodrigues <rod...@freebsd.org> wrote:

Because I'm getting run-time errors from the lutok tests that seem to be
related to pulling in the pkg-installed version of the ATF libraries:  https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/7/

lutok is not in the base system, ergo it's using the atf version from ports.

[Craig Rodrigues]

On Wed, Oct 1, 2014 at 1:29 PM, 'Julio Merino' via kyua-discuss <kyua-d...@googlegroups.com> wrote:

On Wed, Oct 1, 2014 at 10:24 PM, Craig Rodrigues <rod...@freebsd.org> wrote:

Because I'm getting run-time errors from the lutok tests that seem to be
related to pulling in the pkg-installed version of the ATF libraries:  https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/7/

lutok is not in the base system, ergo it's using the atf version from ports.

OK, now I understand.  There is a symlink /usr/tests/local -> /usr/local/tests

If I do a top-level: cd /usr/tests; kyua test

it will execute all the tests (base system + tests installed by ports under /usr/local ).
I can understand the motivation to have all the tests in one place.

What I did not understand is that if any package includes ATF tests in C/C++,
then that package as a compile time dependency against the ATF package.
I need to recompile that package (such as lutok) against the ATF package that I have installed.

The other approach would be to recompile the lutok package, but configure it to not include tests.

Another approach would be to remove the /usr/tests/local symlink when

I do "cd /usr/test; kyua test", but I don't really want to do that.

--

Craig

[Julio Merino]

On Wed, Oct 1, 2014 at 10:48 PM, Craig Rodrigues <rod...@freebsd.org> wrote:

What I did not understand is that if any package includes ATF tests in C/C++,
then that package as a compile time dependency against the ATF package.
I need to recompile that package (such as lutok) against the ATF package that I have installed.

Correct. That's why, when there is an ABI change in a library package, all other packages that depend on the library must have their PORTREVISION bumped and thus rebuilt. If you are doing custom changes to the ports, you have to deal with this on your own in some manner.