Untarring using genrule can't find outs

[Garrett Garcia]

I'm having trouble with a genrule.  The tarball, mbedtls-2.2.1-apache.tgz, includes source code and headers that need to be compiled.  So I created the following genrule:

genrule(

    name = "unpack_mbedtls_tarball",

    srcs = [

        "mbedtls-2.2.1-apache.tgz",

    ],

    cmd = "tar xfz $(location mbedtls-2.2.1-apache.tgz)",

    outs = [

        "mbedtls-2.2.1/.gitignore",

    ],

)

There are other outs that I've removed for space.

I'm getting this error:

ERROR: /Users/slopirate/p4/depot/branches/bazel/all/cc/mbedtls/BUILD:5:1: declared output 'cc/mbedtls/mbedtls-2.2.1/.gitignore' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely).

I don't understand why it can't find the outputs.

[Austin Schuh]

Try setting cmd = "tar ...; ls -la" and seeing what you learn about what is being created.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/d7d53773-f729-432b-bb35-74d3f66fb244%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Garrett Garcia]

Here's the output:

drwxr-xr-x    7 slopirate  wheel    238 Feb 10 15:48 .

drwxr-xr-x   15 slopirate  wheel    510 Feb 10 15:48 ..

drwxr-xr-x  382 slopirate  wheel  12988 Feb 10 15:48 _bin

drwxr-xr-x    5 slopirate  wheel    170 Feb 10 15:48 bazel-out

lrwxr-xr-x    1 slopirate  wheel     52 Feb 10 15:48 cc -> /Users/slopirate/p4/depot/branches/bazel/all/cc

lrwxr-xr-x    1 slopirate  wheel     80 Feb 10 15:48 external -> /private/var/tmp/_bazel_slopirate/f351f0bff6fab7be16e685e7e1254d29/external

drwxr-xr-x   20 slopirate  wheel    680 Jan  4 20:38 mbedtls-2.2.1

I don't know enough to know if that's where the mbedtls-2.2.1 directory should be.  Should I add a "-C {something}" to the tar command?  I've read through the docs a few times and can't make sense of this.

[Brian Silverman]

That does sound like the problem. Try "-C $@D". The variables you can use there are called "Make" variables.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/cf0c4bf0-607d-4fd6-92df-8d34f9f15328%40googlegroups.com.

[Garrett Garcia]

Still no luck:

SON-00002219:all slopirate$ bazel build -s //cc/mbedtls:unpack_mbedtls_tarball

INFO: Found 1 target...

>>>>> # //cc/mbedtls:unpack_mbedtls_tarball [action 'Executing genrule //cc/mbedtls:unpack_mbedtls_tarball']

(cd /private/var/tmp/_bazel_slopirate/f351f0bff6fab7be16e685e7e1254d29/all && \

  exec env - \

    PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Users/slopirate/Development/adt-bundle-mac-x86_64-20131030/sdk/platform-tools:/Users/slopirate/Development/apache-ant-1.9.2/bin:/Users/slopirate/Development/adt-bundle-mac-x86_64-20131030/sdk/tools:/usr/local/apache-maven-3.3.9/bin:/Users/slopirate/bin \

    TMPDIR=/var/folders/gd/srx2mbtd04125jj31q9c0q_xxq73m_/T/ \

  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; tar xfz cc/mbedtls/mbedtls-2.2.1-apache.tgz -C bazel-out/local_darwin-fastbuild/genfiles/cc/mbedtls/mbedtls-2.2.1')

ERROR: /Users/slopirate/p4/depot/branches/bazel/all/cc/mbedtls/BUILD:5:1: declared output 'cc/mbedtls/mbedtls-2.2.1/.gitignore' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely).

[Brian Silverman]

Maybe you need --strip-components=1 too? Try building with --genrule_strategy=standalone and looking at where the output files actually vs where Bazel is telling you it expects them (which is bazel-genfiles/cc/mbedtls/mbedtls-2.2.1/.gitignore in this case).

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/e1d0c411-4709-459e-8ee3-69f8c915bb68%40googlegroups.com.

[Alex Humesky]

if the tar contains "mbedtls-2.2.1/.gitignore", and you do -C "$D", it might be creating:

bazel-out/local_darwin-fastbuild/genfiles/cc/mbedtls/mbedtls-2.2.1/mbedtls-2.2.1/.gitignore

whereas bazel expects

bazel-out/local_darwin-fastbuild/genfiles/cc/mbedtls/mbedtls-2.2.1/.gitignore

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/e1d0c411-4709-459e-8ee3-69f8c915bb68%40googlegroups.com.

[Garrett Garcia]

--strip-components 1 did it.  Thanks for the help!

As a follow-up question, is there any way to use glob() in the outs attribute or do I need to declare each file in the tarball explicitly?

[Garrett Garcia]

As an addendum:

--strip-components 1 was not the problem.  The problem was that having only a single entry in outs changes the behavior of $@D.  When I added more than one out file, it worked fine with cmd = "tar xfz $(location mbedtls-2.2.1-apache.tgz) -C $(@D)"

[Alex Humesky]

Each output must be declared individually in the outs (otherwise the genrule can't know if everything was created).

You could alternatively untar everything and keep them in your source tree, then you can glob them up as regular inputs to rules.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/19f5f479-9564-43b9-ac50-eae77852716a%40googlegroups.com.

[me...@spotify.com]

Alex, can you elaborate more on this?
Is there a placeholder I can use that is the BUILD file's folder?
Can the output of a genrule be empty list?

[Alex Humesky]

On Thu, May 5, 2016 at 2:48 PM menny via bazel-discuss <bazel-...@googlegroups.com> wrote:

Alex, can you elaborate more on this?

Do you have a specific question or an example?

 

Is there a placeholder I can use that is the BUILD file's folder?

There is the PACKAGE_NAME global that you can use inside build files to get the build file's package, which corresponds to folder it's in in the repo:

http://bazel.io/docs/skylark/lib/globals.html#PACKAGE_NAME

 

Can the output of a genrule be empty list?

The outs must be non-empty, bazel will give an error otherwise -- if a genrule doesn't generate anything, then there is no reason to run it :)

 

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/f55859ff-87fe-458f-94b0-9570853979ac%40googlegroups.com.