PSA: Platform2 is being enabled!

[Liam McLoughlin]

Platform2 is going to be enabled by default today assuming all goes well. If you don't know what Platform2 is, see the doc about it at http://www.chromium.org/chromium-os/getting-started-with-platform2.

In short, makefiles/SCons files for some projects in src/platform are disappearing in favour of GYP files. If your project has a GYP file, you should be updating that once the transition happens and not the Makefile. The makefiles will be cleaned out for most projects shortly after the transition happens.

If your project is in src/platform and doesn't have a GYP file, you should write one. The syntax is very straightforward and you should be able to use an already ported project like cros-disks as a good reference.

Eventually everything in src/platform will use GYP files and src/platform will become a single repository. Anything where this isn't appropriate will be moved somewhere else outside src/platform.

If there's something you were doing with common-mk that you can't do with the new system, please let me know or file a bug and CC me on it.

[Olof Johansson]

2013/9/12 Liam McLoughlin <lmclo...@chromium.org>

Platform2 is going to be enabled by default today assuming all goes well. If you don't know what Platform2 is, see the doc about it at http://www.chromium.org/chromium-os/getting-started-with-platform2.

In short, makefiles/SCons files for some projects in src/platform are disappearing in favour of GYP files. If your project has a GYP file, you should be updating that once the transition happens and not the Makefile. The makefiles will be cleaned out for most projects shortly after the transition happens.

If your project is in src/platform and doesn't have a GYP file, you should write one. The syntax is very straightforward and you should be able to use an already ported project like cros-disks as a good reference.

Eventually everything in src/platform will use GYP files and src/platform will become a single repository. Anything where this isn't appropriate will be moved somewhere else outside src/platform.

What's considered "not appropriate"?

Switching over to one monolithic repo means that it makes it _much_ harder for some outside party to track just a few components of our platform, which is counterproductive for all parties. It seems like a step backwards. But maybe I'm missing what's actually considered here compared to what is already in that subtree.

-Olof

[Jorge Lucangeli Obes]

I agree with Olof. The docs on chromium.org are not clear on the benefit of combining things in one large repo. Parallel/incremental builds with GYP+Ninja seems orthogonal to the number of repos we have.

 

-Olof

--
--
Chromium OS Developers mailing list: chromiu...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-os-dev?hl=en
 

[Mike Frysinger]

On Thu, Sep 12, 2013 at 1:57 PM, Olof Johansson <ol...@chromium.org> wrote:

2013/9/12 Liam McLoughlin <lmclo...@chromium.org>

Platform2 is going to be enabled by default today assuming all goes well. If you don't know what Platform2 is, see the doc about it at http://www.chromium.org/chromium-os/getting-started-with-platform2.

In short, makefiles/SCons files for some projects in src/platform are disappearing in favour of GYP files. If your project has a GYP file, you should be updating that once the transition happens and not the Makefile. The makefiles will be cleaned out for most projects shortly after the transition happens.

If your project is in src/platform and doesn't have a GYP file, you should write one. The syntax is very straightforward and you should be able to use an already ported project like cros-disks as a good reference.

Eventually everything in src/platform will use GYP files and src/platform will become a single repository. Anything where this isn't appropriate will be moved somewhere else outside src/platform.

What's considered "not appropriate"?

things that better belong in third_party/ (like google-breakpad).  things that aren't really part of the CrOS "platform" (like some of the testsuites).  things that don't make sense being integrated into the single repo (like vboot).

Switching over to one monolithic repo means that it makes it _much_ harder for some outside party to track just a few components of our platform, which is counterproductive for all parties. It seems like a step backwards. But maybe I'm missing what's actually considered here compared to what is already in that subtree.

i'm not sure about it being a lot harder.  we don't intend on killing off the existing Makefile's right away for packages that we know are being used externally (like chaps).  we've been bad in the past in terms of making it easy for some of these projects to get pulled out and used outside of CrOS.  this is often due to things like dependency on libbase/libchromeos.  unifying these in one repo i think will actually make it easier to create distributable and standalone source packages.

-mike

[Mike Frysinger]

On Thu, Sep 12, 2013 at 2:04 PM, Jorge Lucangeli Obes <jor...@chromium.org> wrote:

On Thu, Sep 12, 2013 at 10:57 AM, Olof Johansson <ol...@chromium.org> wrote:

2013/9/12 Liam McLoughlin <lmclo...@chromium.org>

Platform2 is going to be enabled by default today assuming all goes well. If you don't know what Platform2 is, see the doc about it at http://www.chromium.org/chromium-os/getting-started-with-platform2.

In short, makefiles/SCons files for some projects in src/platform are disappearing in favour of GYP files. If your project has a GYP file, you should be updating that once the transition happens and not the Makefile. The makefiles will be cleaned out for most projects shortly after the transition happens.

If your project is in src/platform and doesn't have a GYP file, you should write one. The syntax is very straightforward and you should be able to use an already ported project like cros-disks as a good reference.

Eventually everything in src/platform will use GYP files and src/platform will become a single repository. Anything where this isn't appropriate will be moved somewhere else outside src/platform.

What's considered "not appropriate"?

Switching over to one monolithic repo means that it makes it _much_ harder for some outside party to track just a few components of our platform, which is counterproductive for all parties. It seems like a step backwards. But maybe I'm missing what's actually considered here compared to what is already in that subtree.

I agree with Olof. The docs on chromium.org are not clear on the benefit of combining things in one large repo. Parallel/incremental builds with GYP+Ninja seems orthogonal to the number of repos we have.

off the top of my head, the complaints we've heard from developers:

 * having to make multiple changes across repos when they want to add/change APIs/build settings/etc...

 * explosion of build systems (agreed that a single repo isn't strictly required to fix this, but it does help as it makes for a team-wide audit point -- it's obvious that there is now One True Solution)

 * explosion of utility libraries -- people have no idea what is already offered, or the common location (libbase/libchromeos) doesn't have what they need.  so they implement it once in their project.  then the next project does the same thing.

 * proliferation of a variety of third party libraries -- devs don't know what 3rd party libs they should be using, so they just reach for what they know or can find talking to others.

-mike

[Antoine Labour]

On Thu, Sep 12, 2013 at 10:57 AM, Olof Johansson <ol...@chromium.org> wrote:

I disagree. IMO this is perhaps the best thing that's happening to the Chrome OS repository since... it started.

A single repo means more correct dependencies and ability to bisect. Also a single build instead of recursion between ebuilds and makefiles. Faster/incremental etc.

As a kernel developer, you should appreciate that - imagine if each arch and top-level directory in the kernel was in a separate repo, with only loose dependencies between them, and a different makefile system in each of them. That's what Chrome OS is today, but this change definitely make us go into the right direction.

Antoine

[Chris Masone]

The explanation page doesn't explain several things I need to know in order to migrate over packages that I maintain.

1) What about generating dbus bindings for dbus-glib? Will I need to write my own generator for that? How do I get started doing that?

2) What's the expected edit/compile/test workflow using platform2? It asserts that I now get incremental builds, but doesn't tell me how to run them. And how do I run only the unit tests that I care about?

--

[Mike Frysinger]

On Thu, Sep 12, 2013 at 5:23 PM, Chris Masone <cma...@chromium.org> wrote:

The explanation page doesn't explain several things I need to know in order to migrate over packages that I maintain.

yeah, we need to flesh it out more

1) What about generating dbus bindings for dbus-glib? Will I need to write my own generator for that? How do I get started doing that?

if there isn't an existing .gypi in common-mk/, then you'll have to write a new one and store it in there

2) What's the expected edit/compile/test workflow using platform2? It asserts that I now get incremental builds, but doesn't tell me how to run them. And how do I run only the unit tests that I care about?

hmm, i'm not sure it is possible to run a subset of unittests other than by hand.  we probably need to fix that ;).

-mike

[Jorge Lucangeli Obes]

On Thu, Sep 12, 2013 at 2:17 PM, Antoine Labour <pi...@chromium.org> wrote:

On Thu, Sep 12, 2013 at 10:57 AM, Olof Johansson <ol...@chromium.org> wrote:

2013/9/12 Liam McLoughlin <lmclo...@chromium.org>

Platform2 is going to be enabled by default today assuming all goes well. If you don't know what Platform2 is, see the doc about it at http://www.chromium.org/chromium-os/getting-started-with-platform2.

In short, makefiles/SCons files for some projects in src/platform are disappearing in favour of GYP files. If your project has a GYP file, you should be updating that once the transition happens and not the Makefile. The makefiles will be cleaned out for most projects shortly after the transition happens.

If your project is in src/platform and doesn't have a GYP file, you should write one. The syntax is very straightforward and you should be able to use an already ported project like cros-disks as a good reference.

Eventually everything in src/platform will use GYP files and src/platform will become a single repository. Anything where this isn't appropriate will be moved somewhere else outside src/platform.

What's considered "not appropriate"?

Switching over to one monolithic repo means that it makes it _much_ harder for some outside party to track just a few components of our platform, which is counterproductive for all parties. It seems like a step backwards. But maybe I'm missing what's actually considered here compared to what is already in that subtree.

I disagree. IMO this is perhaps the best thing that's happening to the Chrome OS repository since... it started.

A single repo means more correct dependencies and ability to bisect. Also a single build instead of recursion between ebuilds and makefiles. Faster/incremental etc.

As a kernel developer, you should appreciate that - imagine if each arch and top-level directory in the kernel was in a separate repo, with only loose dependencies between them, and a different makefile system in each of them. That's what Chrome OS is today, but this change definitely make us go into the right direction.

These are very good reasons that should be in the chromium.org doc =)

[Chris Masone]

On Thu, Sep 12, 2013 at 2:43 PM, Mike Frysinger <vap...@chromium.org> wrote:

On Thu, Sep 12, 2013 at 5:23 PM, Chris Masone <cma...@chromium.org> wrote:

The explanation page doesn't explain several things I need to know in order to migrate over packages that I maintain.

yeah, we need to flesh it out more

1) What about generating dbus bindings for dbus-glib? Will I need to write my own generator for that? How do I get started doing that?

if there isn't an existing .gypi in common-mk/, then you'll have to write a new one and store it in there

And how do I start doing that?

 

2) What's the expected edit/compile/test workflow using platform2? It asserts that I now get incremental builds, but doesn't tell me how to run them. And how do I run only the unit tests that I care about?

hmm, i'm not sure it is possible to run a subset of unittests other than by hand.  we probably need to fix that ;).

Yeah. I'd recommend against anyone porting to platform2 until this is sorted. The negative impact on my productivity would be significant if I have to run all of everyone's unittests when I'm just trying to debug one of my own.

Also, I still don't know how to build the new code :-)

 

-mike

[Mike Frysinger]

On Thu, Sep 12, 2013 at 5:53 PM, Chris Masone <cma...@chromium.org> wrote:

On Thu, Sep 12, 2013 at 2:43 PM, Mike Frysinger <vap...@chromium.org> wrote:

On Thu, Sep 12, 2013 at 5:23 PM, Chris Masone <cma...@chromium.org> wrote:

The explanation page doesn't explain several things I need to know in order to migrate over packages that I maintain.

yeah, we need to flesh it out more

1) What about generating dbus bindings for dbus-glib? Will I need to write my own generator for that? How do I get started doing that?

if there isn't an existing .gypi in common-mk/, then you'll have to write a new one and store it in there

And how do I start doing that?

look at an existing .gypi.  there are generators already for dbusxx-xml2cpp and protoc.  i can't imagine dbus-glib binding generation would be far from those (or i'm just not creative enough).

2) What's the expected edit/compile/test workflow using platform2? It asserts that I now get incremental builds, but doesn't tell me how to run them. And how do I run only the unit tests that I care about?

hmm, i'm not sure it is possible to run a subset of unittests other than by hand.  we probably need to fix that ;).

Yeah. I'd recommend against anyone porting to platform2 until this is sorted. The negative impact on my productivity would be significant if I have to run all of everyone's unittests when I'm just trying to debug one of my own.

it hasn't quite been enabled yet (as liam hit a failure in one of his CLs when he tried landing things).  i would say this is more in extended dogfood phase right now than primetime.  adventurous dogfooders should approach so we can use their experience to expand the documentation.

Also, I still don't know how to build the new code :-)

you do `emerge-$BOARD platform2`.  it replaces the individual ebuilds with one.

-mike

[Chris Masone]

Part of the reason we wrote cros_workon_make was because running emerge wastes time packaging up the build results. What's the plan to avoid that, here? I understand this is still dogfood-ish, but we should preemptively address problems we realized years ago and already addressed in our existing tools.

 

-mike

[Mike Frysinger]

afaik, cros_workon_make should work with any cros-workon package (which platform2 is).  the gyp stuff is replacing the scons and common.mk and handwritten Makefile's.  we still have an ebuild which runs the gyp stages and such.

perhaps this pre-emptive strike by Liam could be considered a dry-run for actually enabling platform2.  we'll look at all the feedback, update the docs to answer any queries people have (if you guys still don't understand how something works, please ask specifics), and then try again at enabling it.

-mike

[Liam McLoughlin]

I've updated the docs and fixed the issues that stopped it being merged last time. Have I covered everyone's questions? I'd like to go ahead and try to enable it again soon if so.

[Liam McLoughlin]

By popular request, you can now run a subset of unit tests with Platform2: http://www.chromium.org/chromium-os/getting-started-with-platform2#TOC-Running-unit-tests