Fwd: [COVID-19 Hackathon] Bazel build system pending NEW

[Yun Peng]

Hi all,

Since Bazel is almost in Debian, if you want to help packaging TensorFlow for Debian, please check this thread.

Thanks!

Best,

Yun

---------- Forwarded message ---------
From: Yun Peng <pcl...@google.com>
Date: Thu, Sep 3, 2020 at 10:21 AM
Subject: Re: [COVID-19 Hackathon] Bazel build system pending NEW
To: Anton Gladky <gl...@debian.org>
Cc: Mo Zhou <lu...@debian.org>, Michael R. Crusoe <cru...@debian.org>, Olek Wojnar <ol...@debian.org>, Debian Med Project List <debia...@lists.debian.org>, Debian Science List <debian-...@lists.debian.org>

Hi everyone,

Last time I checked was July 24. The result is the following, but looks like the situation has already improved.

> The remaining dependencies are:
> (In NEW queue)
> abseil-cpp

Need to try if the newly packaged abseil works for the TF build.

>
> (Packaged in Debian, but doesn't work due to version mismatch)
> re2 (too new)

TF uses an older version of re2, that's why the Debian version doesn't work. We might need to upgrade the re2 version in TF.

> boringssl (too old)
> 

And this time it's the Debian version that's too old.

> (Packaged in Debian, but not yet integrated with the TF_SYSTEM_LIBS approach)
> eigen

> highwayhash

Those two still need to be integrated with the TF_SYSTEM_LIBS approach.

>
> (Not available in Debian)
> fft2d

Maybe we can exclude this?

> farmhash

Needs to be verified if it works for the TF build.

> aws-sdk-cpp
> aws-c-event-stream
> aws-checksums
> aws-c-common

Needs to patch TF code to exclude those dependencies?

Since there are some changes that need to be done from the TensorFlow side, I'll ask if someone in the TF team can help.

Cheers,

Yun

On Tue, Sep 1, 2020 at 9:32 PM Anton Gladky <gl...@debian.org> wrote:

Hi,

> eigen: sometimes TF relies on new features that debian's libeigen3-dev
>        does not provide. In that case we have to use an embedded
>        tarball again.

The new 3.3.8 release seems to be scheduled on 7th of september [1]. 

If it has features that you need, please just wait a couple of days and it will be

uploaded into the archive.

[1] https://listengine.tuxfamily.org/lists.tuxfamily.org/eigen/2020/08/msg00001.html

Best regards

Anton

Am Di., 1. Sept. 2020 um 04:37 Uhr schrieb Mo Zhou <lu...@debian.org>:

Hi Michael,

I went through the list of missing dependencies.

 aws-*: I'm not interested in them. In the past I just (manually)
        filtered out all the code associated with aws functionalities.
        Most of them are possibly not packaged yet.

 abseil-cpp: this is a tricky one. Similar to facebook/folly. It does
             not guarantee a stable ABI. Once the system-provided abseil
             breaks the tensorflow, we have to embed one.

             Anyway it's present in our archive: libabsl-dev but I don't
             know whether tensorflow builds against it.

 re2: already present in archive. I guess there is no reason to embed it.

 boringssl: maybe a simple string replacement s/boring/open/g will just work?

 farmhash: in archive, and I'm the uploader. Ping me if it needs to be updated.

 eigen: sometimes TF relies on new features that debian's libeigen3-dev
        does not provide. In that case we have to use an embedded
        tarball again.

 highwayhash: in archive, and I'm the uploader.

 fft2d: TF uses merely 1 source file from this project for merely
        one OP/Kernel. Just keep it embedded.
        The last upstream update was ~10 yrs ago (IIRC).

On Mon, Aug 31, 2020 at 05:32:56PM +0200, Michael R. Crusoe wrote:
> Hey Mo, we have some unpackaged dependencies. Currently TensorFlow can be built
> using code copies, but obviously that isn't a great plan. Here are the current
> code copies: https://github.com/meteorcloudy/tensorflow/tree/r2.2-debian/debian
> /dist
>
> If you want to help out you can use the bazel-bootstrap and
> libcheck-framework-java binary packages from https://people.debian.org/~olek/
> packages/

Great! Thanks for the hint.

[Alexander Grund]

FWIW: As TF changes its required Bazel version with pretty much every release this might still be hard.

Also regarding BoringSSL: I successfully built TF with OpenSSL via TF_SYSTEM_LIBS. This is absolutely required when using the system libcurl as otherwise the symbols of OpenSSL and BoringSSL clash and everything breaks. Talking about ODR violations leading to invalid reads and writes here, so serious stuff.