meson build system for protobuf-c

[Robert Edmonds]

Hi,

I'm one of the maintainers for protobuf-c, a Protocol Buffers
implementation in C (https://github.com/protobuf-c/protobuf-c). I'm
considering replacing our existing autotools and cmake build systems
with a meson build system. I started porting the build system yesterday,
without having any previous experience with meson.

Here is what I have so far, which seems to be largely feature complete
compared to the autotools build system:

https://github.com/protobuf-c/protobuf-c/tree/edmonds/meson-build-system
(the development branch)

https://github.com/protobuf-c/protobuf-c/compare/next...edmonds/meson-build-system?expand=1
(the development branch compared against its merge target)

https://travis-ci.org/protobuf-c/protobuf-c/builds/423769279
(the most recent CI build log)

How does this look so far?

I also have a few questions:

1)

protobuf-c has a dependency on Google's protobuf, which in the CI
environment is built from source and installed to a non-system path, to
avoid needing root privileges (and also because this is how some users
install the protobuf dependency). During the build, a helper utility has
to be linked against the protobuf library and executed to generate data
for other build steps (the 'cxx-generate-packed-data' utility).

Previously, all I had to do was set the PKG_CONFIG_PATH environment
variable and autotools was able to pick up the protobuf dependency and
run the helper utility during the build.

With meson, I have to set LD_LIBRARY_PATH in addition to
PKG_CONFIG_PATH, or otherwise the build will fail when invoking the
built utility with an error like:

/tmp/protobuf-c/build/cxx-generate-packed-data: error while loading shared libraries: libprotobuf.so.17: cannot open shared object file: No such file or directory

Is there a way to get this to work without having to manually set
LD_LIBRARY_PATH in the build environment?

2)

At least on Linux, we need to install a few symlinks. The library's
public header file is installed so that it should be included as
'#include <protobuf-c/protobuf-c.h>', but for historical reasons it also
needs to be accessible as '#include <google/protobuf-c/protobuf-c.h>'
(eventually this compatibility symlink will be dropped).

The protobuf-c compiler (protoc-c) is also usable as a plugin for the
protobuf compiler (protoc), but in order to do so the plugin binary must
be named 'protoc-gen-c'. So the protobuf-c compiler changes behavior
based on whether it's invoked as 'protoc-c' or 'protoc-gen-c'.

It looks like meson does not have built-in support for symlinks, so I've
resorted to shelling out directly to 'mkdir -p' and 'ln -s'. Is there a
better way to do this? I would imagine this just doesn't work on
Windows, and the reason protobuf-c even has a cmake build system in the
first place is to support Windows users. (We can probably get away with
just not installing the symlinks on Windows.)

3)

protobuf-c is somewhat popular with developers that cross-compile to
embedded systems. Is there anything special I need to do to support
these kinds of users?

Thanks!

--
Robert Edmonds
edm...@debian.org

[Jussi Pakkanen]

On Mon, Sep 3, 2018 at 7:00 AM Robert Edmonds <edm...@debian.org> wrote:

> Here is what I have so far, which seems to be largely feature complete
> compared to the autotools build system:
>
> https://github.com/protobuf-c/protobuf-c/tree/edmonds/meson-build-system
> (the development branch)

Seems pretty good based on a quick lookup.

> /tmp/protobuf-c/build/cxx-generate-packed-data: error while loading shared libraries: libprotobuf.so.17: cannot open shared object file: No such file or directory
>
> Is there a way to get this to work without having to manually set
> LD_LIBRARY_PATH in the build environment?

Not at the moment. People are expected to set up their environment so
that executables can be run.

> It looks like meson does not have built-in support for symlinks, so I've
> resorted to shelling out directly to 'mkdir -p' and 'ln -s'. Is there a
> better way to do this? I would imagine this just doesn't work on
> Windows, and the reason protobuf-c even has a cmake build system in the
> first place is to support Windows users. (We can probably get away with
> just not installing the symlinks on Windows.)

The simple and ugly solution is that you compile all the actual
sources into a static library and then have two executable targets
that build the same main file to different executables (and link
against the static library). It's a bit ugly but it should not be
_that_ slow.

> 3)
>
> protobuf-c is somewhat popular with developers that cross-compile to
> embedded systems. Is there anything special I need to do to support
> these kinds of users?

You might want to make sure that you build the compiler as a native
target so it can be built and run as part of the cross compilation
transparently.

You might also consider building Protobuf as a Meson subproject. It is
available in http://wrapdb.mesonbuild.com/. There also seems to be a
submitted port of protobuf-c. Be sure to submit an update once this
lands in master.

This workflow will get easier once
https://github.com/mesonbuild/meson/pull/4055 lands.

[Rafael Avila de Espindola]

"Robert Edmonds" <edm...@debian.org> writes:

> 1)
>
> protobuf-c has a dependency on Google's protobuf, which in the CI
> environment is built from source and installed to a non-system path, to
> avoid needing root privileges (and also because this is how some users
> install the protobuf dependency). During the build, a helper utility has
> to be linked against the protobuf library and executed to generate data
> for other build steps (the 'cxx-generate-packed-data' utility).
>
> Previously, all I had to do was set the PKG_CONFIG_PATH environment
> variable and autotools was able to pick up the protobuf dependency and
> run the helper utility during the build.
>
> With meson, I have to set LD_LIBRARY_PATH in addition to
> PKG_CONFIG_PATH, or otherwise the build will fail when invoking the
> built utility with an error like:
>
> /tmp/protobuf-c/build/cxx-generate-packed-data: error while loading shared libraries: libprotobuf.so.17: cannot open shared object file: No such file or directory
>
> Is there a way to get this to work without having to manually set
> LD_LIBRARY_PATH in the build environment?

Do you know what autotools was doing? Was it passing -rpath to the
linker when creating the executables? If these are just test binaries it
is probably simpler than managing LD_LIBRARY_PATH.

Cheers,
Rafael

[Rafael Avila de Espindola]

>> Do you know what autotools was doing? Was it passing -rpath to the
>> linker when creating the executables? If these are just test binaries it
>> is probably simpler than managing LD_LIBRARY_PATH.
>

> Yes, it's just a helper utility used by the test suite that doesn't get
> installed. So, I could simply do something like this?
>
> gen_cxx_data = custom_target('cxx-generate-packed-data',
> output : ['test-full-cxx-output.inc'],
> command :
> executable('cxx-generate-packed-data', 't/generated-code2/cxx-generate-packed-data.cc',
> gen_protobuf.process('t/test-full.proto',
> preserve_path_from : meson.current_source_dir()),
> dependencies : protobuf,
> build_rpath : protobuf.get_pkgconfig_variable('libdir')),
> capture : true)

Sounds about right, but I never used build_rpath myself.

Cheers,
Rafael