Buck support merged; exit criteria

[Dave Borowitz]

Buck support was merged as fd6bb9f. We decided to go ahead with this while Shawn was sleeping for two reasons:

-It should make people at the hackathon and beyond more productive in the short term (especially those with slow laptops :).

-Shawn was adding incremental features that really belonged in new changes. It was becoming difficult to review incremental patch sets.

See the linked commit message for details about how to get started with Buck and what features are currently supported.

Now, here are the exit criteria we came up with for accepting buck and removing Maven support, within a tentative time horizon of 3 months.

Consider these exit criteria in a draft state and up for discussion. That said, if you have objections to Buck as a whole, please try to state them in terms of specific exit criteria: rather than "this is why I will always hate Buck" or "this is why I will always love Maven," please suggest "in order to accept a Buck-based build, Buck and/or Gerrit need to support X."

1. Windows support.

Facebook has an intern who will be working on this. Hopefully they succeed.

2. Bootstrap and stable version support.

From a fresh Gerrit clone on a machine without Buck (but with some reasonable subset of Buck's dependencies, e.g. Python 2.7), a new Gerrit developer should be able to set up and start building with Buck by running approximately one command. There should also be some idea of a "stable" version of Buck, even if we just tie our build to specific known-good SHAs. Binary distributions are another plus, which I believe the Buck team has planned.

3. Eclipse support.

Much of this is already there. The build needs to be at least as reliable as it is under Maven. (This is kind of a low bar, due to issues like Maven not handling generated Prolog source files or recompiling the GWT source.)

4. Fix all incidental issues.

Things come up that don't work. Martin just ran out of file descriptors, which sounds like an upstream bug. I was having problems with non-{dbg,opt} gwtui in Eclipse. Shawn had that bug that he fixed in his fork. We need to fix all of these so that everything works reliably for all core contributors all the time. Also, there should be a consensus that new bugs like this in upstream Buck are not constantly being introduced.

[Chad Horohoe]

On Thu, May 9, 2013 at 11:04 AM, Dave Borowitz <dbor...@google.com> wrote:

Buck support was merged as fd6bb9f. We decided to go ahead with this while Shawn was sleeping for two reasons:

-It should make people at the hackathon and beyond more productive in the short term (especially those with slow laptops :).

-Shawn was adding incremental features that really belonged in new changes. It was becoming difficult to review incremental patch sets.

See the linked commit message for details about how to get started with Buck and what features are currently supported.

Now, here are the exit criteria we came up with for accepting buck and removing Maven support, within a tentative time horizon of 3 months.

Consider these exit criteria in a draft state and up for discussion. That said, if you have objections to Buck as a whole, please try to state them in terms of specific exit criteria: rather than "this is why I will always hate Buck" or "this is why I will always love Maven," please suggest "in order to accept a Buck-based build, Buck and/or Gerrit need to support X."

This all sounds completely reasonable to me.

 

1. Windows support.

Facebook has an intern who will be working on this. Hopefully they succeed.

Good to know, best of luck to them.

 

2. Bootstrap and stable version support.

From a fresh Gerrit clone on a machine without Buck (but with some reasonable subset of Buck's dependencies, e.g. Python 2.7), a new Gerrit developer should be able to set up and start building with Buck by running approximately one command. There should also be some idea of a "stable" version of Buck, even if we just tie our build to specific known-good SHAs. Binary distributions are another plus, which I believe the Buck team has planned.

Stable versions & binary distributions sound great and would ease some of

my biggest worries re: Buck.

 

3. Eclipse support.

Much of this is already there. The build needs to be at least as reliable as it is under Maven. (This is kind of a low bar, due to issues like Maven not handling generated Prolog source files or recompiling the GWT source.)

*nod*

 

4. Fix all incidental issues.

Things come up that don't work. Martin just ran out of file descriptors, which sounds like an upstream bug. I was having problems with non-{dbg,opt} gwtui in Eclipse. Shawn had that bug that he fixed in his fork. We need to fix all of these so that everything works reliably for all core contributors all the time. Also, there should be a consensus that new bugs like this in upstream Buck are not constantly being introduced. 

 

All software has bugs. Minor annoyances may mean you file an issue,

major bugs mean you spend few days and then contribute upstream.

Heck, that's why I'm here :-)

Could we possibly add a #5: investigate impact on requiring the JDK7?

-Chad

[Shawn Pearce]

On Thu, May 9, 2013 at 8:20 AM, Chad Horohoe <chor...@wikimedia.org> wrote:
>
> Could we possibly add a #5: investigate impact on requiring the JDK7?

BUCK will not work on JDK 6, end of story. I talked about this with
the FB guys on Thursday. They really want to use the newer NIO APIs
that were added in Java 7. Work is already underway using some of
these APIs, and may make it into the open source version of BUCK
within the next couple of weeks.

Java 6 is already end-of-life. Oracle has declared there will be no
new versions of Java 6, even though Java 6 has a known security issue
that will not be patched. Its getting time to move on.

Gerrit itself will continue to target Java 6, but our it sounds like
if we use BUCK we will have to run the build tool under Java 7.

[Chad Horohoe]

I assumed as much. Again, it's not so much a problem as something I just wanted clarification on. Thanks for the clear answer.

-Chad

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Thomas Koch]

On Thursday, May 09, 2013 04:04:02 PM Dave Borowitz wrote:

> Now, here are the exit criteria we came up with for accepting buck and
> removing Maven support, within a tentative time horizon of 3 months.

While I don't have a voice on this, I'd propose a criteria:

* Buck should be packaged in the Debian (and Fedora) archive.

Having Buck in the Debian archive is of course a good thing on its own for
everybody working on a Debian based distribution. An additional benefit are the
sanity checks involved in getting buck accepted in the Debian archive:

- License checks
- All source code of all dependencies available and free? (Remember json.org?)
- Bootstrapable (build Buck without a pre-existing version of Buck)
- Easy hackability of Buck. (If a Debian Maintainer is able to build it, so
are you :-)

Regards,

Thomas Koch, http://www.koch.ro

[David Ostrovsky]

on current master

* Gerrit: 4deeefb2d3f26eb2f03b1cf45974a1e5d360dcce,
* buck (cloned from https://gerrit.googlesource.com/buck): 8159f3f522def00011a10493a501a3d1b55182dd)

i am trying to follow this description:
https://gerrit-review.googlesource.com/#/c/45565/2/Documentation/dev-buck.txt

buck build gerrit
BUILDING //:gerrit
BUILD SUCCESSFUL

buck build api
BUILD FAILED: No directory api when resolving target //api:api in context FULLY_QUALIFIED

However this works:

buck build //:api
BUILDING //:api
BUILD SUCCESSFUL

buck test --all -e slow
TESTS PASSED

buck test --all
TESTS FAILED: 81 Failures

[David Pursehouse]

On Monday, May 13, 2013 5:49:34 AM UTC+9, David Ostrovsky wrote:

buck build api

BUILD FAILED: No directory api when resolving target //api:api in context FULLY_QUALIFIED

This is because "api" does not exist as an alias for "://api" in the buck configuration.

I've uploaded a change that adds it.

https://gerrit-review.googlesource.com/#/c/45710/

 

[Blewitt, Alex [Tech]]

It must be possible to build against JARs that are derived from or follow the same path as a Maven-GAV compatible format, and that repository location must be able to be repointed from e.g. repo1.maven.org to an internal repository instead. For example, instead of having an absolute URL of the form ‘http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar’ in the constructed downloads, it should be possible to use ‘${repo}/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar’ where ${repo} is externally configurable to the build and defaults to http://repo1.maven.org. To allow reproducible builds of upstream source with the same commitid, the overrides file must be settable externally to the files that are checked in to Git; for example, an overrides file in a .gitignore reference, an external Java system property or environment variable.

 

Alex

--

[Dave Borowitz]

On Mon, May 13, 2013 at 3:16 AM, Blewitt, Alex [Tech] <Alex.B...@gs.com> wrote:

It must be possible to build against JARs that are derived from or follow the same path as a Maven-GAV compatible format, and that repository location must be able to be repointed from e.g. repo1.maven.org to an internal repository instead. For example, instead of having an absolute URL of the form ‘http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar’ in the constructed downloads, it should be possible to use ‘${repo}/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar’ where ${repo} is externally configurable to the build and defaults to http://repo1.maven.org. To allow reproducible builds of upstream source with the same commitid, the overrides file must be settable externally to the files that are checked in to Git; for example, an overrides file in a .gitignore reference, an external Java system property or environment variable.

I assume this is because of draconian proxy requirements? The SHA-1 of every external JAR we download is manually specified in the BUCK file, so it literally makes no difference from the build perspective where the files come from.

That said, should be technically easy enough.

[Shawn Pearce]

I have summarized the major themes of this thread in
https://gerrit-review.googlesource.com/45673 and will work to iterate
there. Importantly we will use Gerrit to vote on removal of items from
the list when we think they are completed.

[Shawn Pearce]

On Mon, May 13, 2013 at 11:43 AM, Shawn Pearce <s...@google.com> wrote:
> I have summarized the major themes of this thread in
> https://gerrit-review.googlesource.com/45673 and will work to iterate
> there.

Really I meant we will submit this change, and then further iterate by
proposing code reviews on this section of the file.

> Importantly we will use Gerrit to vote on removal of items from
> the list when we think they are completed.

https://gerrit-review.googlesource.com/45652 runs a vote to close the
Eclipse support item. Please +1 or -1 based on your experiences, and
leave commentary on the review as appropriate.

[Dave Borowitz]

On Sun, May 12, 2013 at 4:27 AM, Thomas Koch <tho...@koch.ro> wrote:

On Thursday, May 09, 2013 04:04:02 PM Dave Borowitz wrote:

> Now, here are the exit criteria we came up with for accepting buck and
> removing Maven support, within a tentative time horizon of 3 months.

While I don't have a voice on this, I'd propose a criteria:

* Buck should be packaged in the Debian (and Fedora) archive.

This is a good criterion and I would be surprised if it doesn't happen eventually, but if we can provide a script within Gerrit to bootstrap even without a Debian package, should it really be blocking?

 

Having Buck in the Debian archive is of course a good thing on its own for
everybody working on a Debian based distribution. An additional benefit are the
sanity checks involved in getting buck accepted in the Debian archive:

- License checks

- All source code of all dependencies available and free? (Remember json.org?)

I agree, in that it more or less goes without saying that we need to audit the license of any third-party dependency we pull into the project. Nothing special about Buck here. In the past we have done this ourselves, and while it's nice if Debian does legwork for us, I don't think we should block anything on Debian doing this work.

 

- Bootstrapable (build Buck without a pre-existing version of Buck)
- Easy hackability of Buck. (If a Debian Maintainer is able to build it, so
are you :-)

IMHO these are nice to have. I would prefer to keep the exit criteria that _Gerrit_ has an acceptable bootstrap procedure.

[Pursehouse, David (Sony Mobile)]

> It must be possible to build against JARs that are derived from or follow the same path as a Maven-GAV compatible format, and that repository location must be able to be repointed from e.g. repo1.maven.org to an internal repository instead.

Support for this was merged today [1]. I suggest we can remove it from the list of open exit criteria [2].

While I was documenting how to build plugins I realized the buck build does not have any support for generating a plugin skeleton like Maven does with the archetype framework. Do we need buck to do that? I've uploaded a change that adds it in the exit criteria [3]; anyone who needs it, please add +1 on it.


[1] https://gerrit-review.googlesource.com/#/c/45657/
[2] https://gerrit-review.googlesource.com/#/c/45684/
[3] https://gerrit-review.googlesource.com/#/c/45716/

[Thomas Broyer]

On Tuesday, May 14, 2013 11:44:45 AM UTC+2, David Pursehouse wrote:

> It must be possible to build against JARs that are derived from or follow the same path as a Maven-GAV compatible format, and that repository location must be able to be repointed from e.g. repo1.maven.org to an internal repository instead.

Support for this was merged today [1].  I suggest we can remove it from the list of open exit criteria [2].

While I was documenting how to build plugins I realized the buck build does not have any support for generating a plugin skeleton like Maven does with the archetype framework.  Do we need buck to do that?

How do you create such a skeleton in non-Java or non-Maven projects? I suspect you don't have a Make target?

Contrary to Maven which manages 3rd-party dependencies, builds, tests, releases, runs, etc. and scaffold, Buck is only a build tool: it builds and tests.

For a skeleton, IMO, there's no reason to ditch the Maven archetype (there's no reason plugins couldn't continue to be built with Maven and would have to switch to Buck), and for those who want to use Buck for their plugins, then I'd say maintain a skeleton in a Git repo that you simply clone to get started.

My 2¢ (as a user and lurker with recent interest in build systems)