Setting up a repo to work with Bazel

[Collin Winter]

Hi bazel-discuss,

I'm trying to convert one of my team's projects to use Bazel, and I'm a little confused by which of Bazel's dependencies I should check into source control.

http://bazel.io/docs/getting-started.html says, "The tools/ and third_party/ directories can be symbolically linked into the workspace, or they can be copied in whole, but must be named tools/ and third_party/ respectively." Based on that, I went ahead and checked in the contents of Bazel's tools and third_party directories to our repo. This seems to be fine for third_party (which is architecture-independent AFAICT), but tools/ contains binaries like tools/jdk/ijar which run on only a single platform.

What's the right way to do this? Should I be checking in any of Bazel's deps? Should I have a script in my repo that people run to symlink parts of their Bazel installation into their git clone? Something else?

Thanks,

Collin Winter

[Ulf Adams]

What platforms do you want to run on? Do you want to build from source (including tools)? Which rules do you want to use (i.e., C++, Java, ?) What's your timeline?

Unfortunately, most of what you're describing is still an open problem. We have some ideas, but haven't decided yet on what our proposed full solution is going to be. Current thinking is to support multiple models depending on what exactly you want, while supporting migration between the models.

- nothing checked in; instead, install tools to local machine and use --package_path (or some variant thereof) to make them show up in your build

- partially checked in; same as above, but some tools are checked into source control

- fully checked in; this needs a strategy for multi-platform - either build tools from source or, if that isn't possible, check in multiple binaries and use select()

-- Ulf

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To post to this group, send email to bazel-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAL%2B262mqVe96WxuxpDc-Q_q80%2BuOGEW44W%3DHDq-%2B%3D%3DT60OK%3D_g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Collin Winter]

On Mon, Apr 6, 2015 at 11:18 AM, Ulf Adams <ulf...@google.com> wrote:

What platforms do you want to run on?

At least Linux and OS X.

 

Do you want to build from source (including tools)?

I certainly want to build our project from source, but I don't particularly care about building the tools from source. I don't want to check in a full C++ or Java toolchain into our github repo, for instance.

 

Which rules do you want to use (i.e., C++, Java, ?)

C++, Java, and Go (though I realize Go is currently not supported).

 

What's your timeline?

We're currently using Kythe's Campfire system, and we'd like to get rid of it as soon as possible.

 

Unfortunately, most of what you're describing is still an open problem. We have some ideas, but haven't decided yet on what our proposed full solution is going to be. Current thinking is to support multiple models depending on what exactly you want, while supporting migration between the models.

- nothing checked in; instead, install tools to local machine and use --package_path (or some variant thereof) to make them show up in your build

- partially checked in; same as above, but some tools are checked into source control

- fully checked in; this needs a strategy for multi-platform - either build tools from source or, if that isn't possible, check in multiple binaries and use select()

I see, thanks for the clarification.

I'm going to explore getting the "nothing checked in" option to work for us.

[Thiago Farina]

On Mon, Apr 6, 2015 at 7:33 PM, 'Collin Winter' via bazel-discuss <bazel-...@googlegroups.com> wrote:

Do you want to build from source (including tools)?

I certainly want to build our project from source, but I don't particularly care about building the tools from source. I don't want to check in a full C++ or Java toolchain into our github repo, for instance.

 

Just my 2 cents, I think that is going to be the case for the vast majority of open source projects.

 

--

Thiago Farina

[Miguel Alcon]

Hi,

Now you have the option of using the tools/ folder from Bazel's installation dir, so that you do not have to check in base_workspace in the SCM:

http://bazel.io/blog/2015/04/06/Simplified-Workspace-Creation.html

Regards,

Mikel.

--

You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To post to this group, send email to bazel-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAFenBsYzRNhuhR22JPxP62QqD-%3DdMkhz4dbfTM%2BkOsNxLTF1Ow%40mail.gmail.com.

[Ming Zhao]

Thanks for update. But I have to say that's still sub-optimal in a
real deployment.
Having the installing process to update the .blazerc in the home is
going to yield inconsistent output across the users(in a corp
environment) and ends up being a support nightmare.

What's the recommendation if I'm willing to put a tools directory in a
source tree and just want to run compile.sh once and share the
generated bazle binary? Simply copying original tools directory to my
source tree doesn't work for java build because it needs
JavaBuilder_deploy.jar and SingleJar_deploy.jar to for java build, and
I don't like to put the jar in my source tree because it's generated
file and there should be some existing rule to generate these output
already, is there any easy way I can just use them?

> https://groups.google.com/d/msgid/bazel-discuss/CA%2BXQKfPE%2BReqxEN5qu7Uc-RAikyG1jb9vb2%2Bzf0%3DRLQBAjbekQ%40mail.gmail.com.

[Ulf Adams]

On Tue, Apr 7, 2015 at 5:49 AM, Ming Zhao <ming...@gmail.com> wrote:

Thanks for update. But I have to say that's still sub-optimal in a
real deployment.
Having the installing process to update the .blazerc in the home is
going to yield inconsistent output across the users(in a corp
environment) and ends up being a support nightmare.

What's the recommendation if I'm willing to put a tools directory in a
source tree and just want to run compile.sh once and share the
generated bazle binary? Simply copying original tools directory to my
source tree doesn't work for java build because it needs
JavaBuilder_deploy.jar and SingleJar_deploy.jar to for java build, and
I don't like to put the jar in my source tree because it's generated
file and there should be some existing rule to generate these output 

already, is there any easy way I can just use them?

Not easily. At Google, we actually do check in these deploy jars. You can't use the Java rules to generate them, because the Java rules depend on these files (such a setup would create a dependency cycle). You'd have to write genrules to invoke javac and jar (or similar; jar is non-deterministic by default, so we don't recommend that) to generate these files.

It might be possible to have yet another mechanism to allow you to put a version id into your source control such that everyone who's checked out the same version also gets the same tools, but without having to actually check the tools into source control.

[Volker Braun]

Why not use the location of the bazel binary? Relying on a ~/.bazelrc is annoying if you move the tree or have >=2 bazel installations around. Its a bit platform-specifc, e.g. /proc/self/exe on Linux, but has been done before.

On Tuesday, April 7, 2015 at 11:03:26 AM UTC+2, Ulf Adams wrote:

It might be possible to have yet another mechanism to allow you to put a version id into your source control such that everyone who's checked out the same version also gets the same tools, but without having to actually check the tools into source control.

I guess you mean git smudge/clean filters, e.g. git-fat

[Han-Wen Nienhuys]

On Tue, Apr 7, 2015 at 5:49 AM, Ming Zhao <ming...@gmail.com> wrote:

> Thanks for update. But I have to say that's still sub-optimal in a
> real deployment.
> Having the installing process to update the .blazerc in the home is
> going to yield inconsistent output across the users(in a corp
> environment) and ends up being a support nightmare.

There is also a global RC file, which -at google- we load from version
control. See

OptionProcessor::FindAlongsideBinaryBlazerc

and

OptionProcessor::FindDepotBlazerc

in src/main/cpp/option_processor.cc. Basically, you can put a
bazel.bazelrc next to the executable, and it will be loaded. Whatever
mechanism to deploy bazel should work for deploying rc changes.


--
Han-Wen Nienhuys
Google Munich
han...@google.com

[Ming Zhao]

Thanks! It's pretty useful.
It would be nice to include this setting to
http://bazel.io/docs/bazel-user-manual.html

[Han-Wen Nienhuys]

It's there, look for "master configuration file" on
http://bazel.io/docs/bazel-user-manual.html