docker image with awscli pip package

[bajac...@gmail.com]

Hello,
I'm new to the Bazel build tool. I've made some decent progress on building docker images based on the rules_docker at https://github.com/bazelbuild/rules_docker
I'm having trouble creating a docker image that has the awscli python package installed. I've added this to my workspace:

git_repository(
name = "io_bazel_rules_python",
remote = "https://github.com/bazelbuild/rules_python.git",
commit = "979fca9ac17ffdb5f715854d6100f7d29dd17875",
)

load("@io_bazel_rules_python//python:pip.bzl", "pip_repositories", "pip_import")
pip_repositories()

pip_import(
name = "piptool_deps",
requirements = "@io_bazel_rules_python//python:requirements.txt",
)
load(
"@piptool_deps//:requirements.bzl",
_piptool_install = "pip_install",
)
_piptool_install()

pip_import(
name = "aws",
requirements = "//:requirements.txt",
)
load(
"@aws//:requirements.bzl",
_aws_install = "pip_install",
)
_aws_install()

My top level requirements.txt is:

awscli>=1.11.183

Im Building a docker images that looks like this in the BUILD file:

oad("@aws//:requirements.bzl", "requirement")

load(
"@io_bazel_rules_docker//container:container.bzl",
"container_image",
"container_push",
)
load("@distroless//package_manager:package_manager.bzl", "dpkg_list")
load("@package_bundle//file:packages.bzl", "packages")

container_image(
name = "build_image",
base = "@stretch//image",
debs = [
packages["zlib1g"],
packages["openjdk-8-jre-headless"],
packages["python2.7-minimal"],
packages["libpython2.7-minimal"],
packages["libpython2.7-stdlib"],
packages["git"],
],
files = [
requirement("awscli")
],
symlinks = {
"/usr/bin/java": "/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java",
"/usr/bin/python": "/usr/bin/python2.7",
},
cmd = ["cat"]
)

As you can see, I have no idea where/how to install the awscli package. When I run the resulting container of the above, I see that the awscli pacakge has been expanded in the top level of the container.

Any insight or help would be greatly appreciated. I am loving the Bazel build tool so far.

Thank you

[Matthew Moore]

Hi Dave,

   I believe your problem is here:

    files = [

        requirement("awscli")

    ],

The intent of this function is to be used with py_{libary,binary}, not to install that requirement in a container image (e.g. as pip would) directly.

A more typical practice would be to build a "PAR" file using github.com/google/subpar and include that bundle into your image, or to use the py_image rule.

That said, as the owner of both rules_python and rules_docker, your "mistake" inspires me to see if such a thing could be made possible! :)

In this case, I believe your interest is in the CLI (which is a script this likely installs onto your path) vs. just installing a library onto the system?

I'll go poke around a bit.  thanks for reaching out.

-M

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/b3764535-2d9d-41d8-a491-55f3d6a86fdf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matthew Moore

Container Development Uber-TL

Developer Infrastructure @ Google

[bajac...@gmail.com]

Thank you Matthew,

I will look into subpar. I made efforts into the py_image bazel rule and I believe that's the direction I'd like to go, It appears that using py_image would allow me to place the awscli dependency on it's own layer of the docker image? That would be great since it will rarely change and the image builds should be faster.

Thanks again.

[Matthew Moore]

Yeah, py_image is great! (no bias here) ;-)

Definitely reach out if you have any issues (or even just feedback), and I'll do my best to unblock you.

-M

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/cdf1e3f4-9b3c-4573-a6d6-105a30799bbd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Aistis Jokubauskas]

Hi, I'm looking for exactly the same functionality. I'm working on containerizing development environment and for that it would be great to have something like pip_install('aws', put_in_usr_local=true). Is this achievable with PAR?

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/b3764535-2d9d-41d8-a491-55f3d6a86fdf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.