Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Demo for modularization Gaia repo
78 views
Skip to first unread message
Greg Weng
Dec 12, 2013, 2:43:21 AM12/12/13
to dev-gaia
Hello everyone,
About one month ago, Yuren and I have announced a idea about modularizing
the current Gaia repo, which would allow developers to develop built-in
apps in a way close to ordinary web apps.
(presentation):
https://docs.google.com/presentation/d/1gO6NLhYHRmscJF14MezfP9BDZIVRJ6Mw_o1ZqWhvbf4/edit?usp=sharing
Now the first runnable demo is available on GitHub, and it includes these
sub-projects:
1. Calendar app for demo: https://github.com/snowmantw/gaia-calendar
2. Homescreen for demo: https://github.com/snowmantw/gaia-homescreen
3. Essential parts of building profile:
https://github.com/snowmantw/gaia-essential
4. Template.js in Bower package: https://github.com/snowmantw/libgaia-html
5. Grunt task for building Gaia profile:
https://github.com/snowmantw/grunt-gaia-builder
There are two auxiliary tools for the Grunt plugin:
6. To build the profile: https://github.com/snowmantw/gaia-profile-builder
7. To build the config: https://github.com/snowmantw/gaia-config
Now I'm going to explain these tools and how to run the demo.
## Build a demo profile
Imaging you're a Calendar app developer, which should only care about the
app itself, and only do tests and publish for that. Now you want to build a
Gaia only contains Calendar and System, because you don't need any other
parts of Gaia, you'll need to:
1. Download the [1.] repo (gaia-calendar) to get the Calendar app
2. In the Calendar's directory, type:
npm install # Install other necessary tools and download bower
packages
3. Using the libraries downloaded by Bower and CRUD in your Calendar app
4. After CRUD, commit and push the changes back to GitHub
5. Prepare to build: in the Calendar directory, type:
grunt build-vanilla # Build a minimal build only contains Calendar and
System
6. You'll have a runnable profile in '/tmp/calendar-build-profile' after
the building process done.
*The build-vanilla can be substituted with build-full, which will build a
build with the additional homescreen app*
Short version:
git clone https://github.com/snowmantw/gaia-calendar.git && cd
gaia-calendar && npm install && grunt build-vanilla && <path to firefox>
-profile /tmp/calendar-build-profile
**Caveats**
1. Due to the mozilla-download can't work with xulrunner (yet), the
gaia-essential repo in fact include a specific version (MacOS) of XUL, so
the demo now is runnable only for MacOS. The patch won't birth quickly if
this project still remains only me as the contributor.
2. Download gaia-essential would let the program hang about 3-5 mins. It
depends on the network between your console and GitHub.
3. We propose to provide lots of pre-defined build settings to satisfy most
of our requests, but the settings mechanism isn't implemented yet.
4. You need to remove the demo build and profile directories every time to
start a new build. This is because the build path now is hard-coded in the
Gruntfile.js in the Calendar app.
5. Due to the vanilla build lacks of homescreen, you need to type the url '
http://gaia-calendar.gaiamobile.org:8080' to switch to the Calendar app
6. Cache is not implemented, so the demo would donwload repos form GitHub
every time.
## Major concepts in this demo
1. One package.json and bower.json per app to manage all app and libraries
dependencies of your app, and no more build script magics like parsing
index.html to find all related shared libraries.
2. It's possible to generate various builds with pre-defined or customized
settings.
3. If you find something is very strongly coupling with other apps, and
this building flow become the major obstacle of the development, you
probably found an immature part of our inter-app architecture, which should
allow loose the coupling among apps as much as possible.
## Future works
1. More discussions: we need more feedbacks and discussions to find out
whether this approach is good for Gaia, and how to polish this coarse demo
to become really usable.
2. More contributors... It's tired to handle these things alone. If build
system can become a formal, scheduled work, we may be able to handle these
issues better.
3. References: to find out how KDE, Gnome, Android and other similar
projects manage their sub-projects' dependencies and build system. We may
learn something helpful from their build system.
Feedbacks is welcome!
--
<http://about.me/snowmantw>Greg Weng
http://about.me/snowmantw
*Understand y f = f [ y f ] ; lose last remaining non-major friend*
* -- Anonymous *
About one month ago, Yuren and I have announced a idea about modularizing
the current Gaia repo, which would allow developers to develop built-in
apps in a way close to ordinary web apps.
(presentation):
https://docs.google.com/presentation/d/1gO6NLhYHRmscJF14MezfP9BDZIVRJ6Mw_o1ZqWhvbf4/edit?usp=sharing
Now the first runnable demo is available on GitHub, and it includes these
sub-projects:
1. Calendar app for demo: https://github.com/snowmantw/gaia-calendar
2. Homescreen for demo: https://github.com/snowmantw/gaia-homescreen
3. Essential parts of building profile:
https://github.com/snowmantw/gaia-essential
4. Template.js in Bower package: https://github.com/snowmantw/libgaia-html
5. Grunt task for building Gaia profile:
https://github.com/snowmantw/grunt-gaia-builder
There are two auxiliary tools for the Grunt plugin:
6. To build the profile: https://github.com/snowmantw/gaia-profile-builder
7. To build the config: https://github.com/snowmantw/gaia-config
Now I'm going to explain these tools and how to run the demo.
## Build a demo profile
Imaging you're a Calendar app developer, which should only care about the
app itself, and only do tests and publish for that. Now you want to build a
Gaia only contains Calendar and System, because you don't need any other
parts of Gaia, you'll need to:
1. Download the [1.] repo (gaia-calendar) to get the Calendar app
2. In the Calendar's directory, type:
npm install # Install other necessary tools and download bower
packages
3. Using the libraries downloaded by Bower and CRUD in your Calendar app
4. After CRUD, commit and push the changes back to GitHub
5. Prepare to build: in the Calendar directory, type:
grunt build-vanilla # Build a minimal build only contains Calendar and
System
6. You'll have a runnable profile in '/tmp/calendar-build-profile' after
the building process done.
*The build-vanilla can be substituted with build-full, which will build a
build with the additional homescreen app*
Short version:
git clone https://github.com/snowmantw/gaia-calendar.git && cd
gaia-calendar && npm install && grunt build-vanilla && <path to firefox>
-profile /tmp/calendar-build-profile
**Caveats**
1. Due to the mozilla-download can't work with xulrunner (yet), the
gaia-essential repo in fact include a specific version (MacOS) of XUL, so
the demo now is runnable only for MacOS. The patch won't birth quickly if
this project still remains only me as the contributor.
2. Download gaia-essential would let the program hang about 3-5 mins. It
depends on the network between your console and GitHub.
3. We propose to provide lots of pre-defined build settings to satisfy most
of our requests, but the settings mechanism isn't implemented yet.
4. You need to remove the demo build and profile directories every time to
start a new build. This is because the build path now is hard-coded in the
Gruntfile.js in the Calendar app.
5. Due to the vanilla build lacks of homescreen, you need to type the url '
http://gaia-calendar.gaiamobile.org:8080' to switch to the Calendar app
6. Cache is not implemented, so the demo would donwload repos form GitHub
every time.
## Major concepts in this demo
1. One package.json and bower.json per app to manage all app and libraries
dependencies of your app, and no more build script magics like parsing
index.html to find all related shared libraries.
2. It's possible to generate various builds with pre-defined or customized
settings.
3. If you find something is very strongly coupling with other apps, and
this building flow become the major obstacle of the development, you
probably found an immature part of our inter-app architecture, which should
allow loose the coupling among apps as much as possible.
## Future works
1. More discussions: we need more feedbacks and discussions to find out
whether this approach is good for Gaia, and how to polish this coarse demo
to become really usable.
2. More contributors... It's tired to handle these things alone. If build
system can become a formal, scheduled work, we may be able to handle these
issues better.
3. References: to find out how KDE, Gnome, Android and other similar
projects manage their sub-projects' dependencies and build system. We may
learn something helpful from their build system.
Feedbacks is welcome!
--
<http://about.me/snowmantw>Greg Weng
http://about.me/snowmantw
*Understand y f = f [ y f ] ; lose last remaining non-major friend*
* -- Anonymous *
Yuren Ju
Dec 12, 2013, 3:18:37 AM12/12/13
to Greg Weng, dev-gaia
Hi,
GNOME use jhbuild to automate downloading, building and running gnome apps.
https://wiki.gnome.org/GnomeLove/JhbuildIntroduction
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
GNOME use jhbuild to automate downloading, building and running gnome apps.
https://wiki.gnome.org/GnomeLove/JhbuildIntroduction
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
Dietrich Ayala
Dec 12, 2013, 2:03:04 PM12/12/13
to Greg Weng, dev-gaia
Greg and Yuren thanks VERY much for doing this research and
demonstration. Changes like these have the potential to radically
improve developer productivity and sanity, which in turn will improve
the quality of Firefox OS :D
demonstration. Changes like these have the potential to radically
improve developer productivity and sanity, which in turn will improve
the quality of Firefox OS :D
Rick Waldron
Dec 12, 2013, 5:27:09 PM12/12/13
to Greg Weng, dev-gaia
On Thu, Dec 12, 2013 at 2:43 AM, Greg Weng <snow...@gmail.com> wrote:
> Hello everyone,
>
> About one month ago, Yuren and I have announced a idea about modularizing
> the current Gaia repo, which would allow developers to develop built-in
> apps in a way close to ordinary web apps.
>
> (presentation):
>
> https://docs.google.com/presentation/d/1gO6NLhYHRmscJF14MezfP9BDZIVRJ6Mw_o1ZqWhvbf4/edit?usp=sharing
>
> Now the first runnable demo is available on GitHub, and it includes these
> sub-projects:
>
> 1. Calendar app for demo: https://github.com/snowmantw/gaia-calendar
> 2. Homescreen for demo: https://github.com/snowmantw/gaia-homescreen
> 3. Essential parts of building profile:
> https://github.com/snowmantw/gaia-essential
> 4. Template.js in Bower package: https://github.com/snowmantw/libgaia-html
This is missing all of the tests for Template.
> Hello everyone,
>
> About one month ago, Yuren and I have announced a idea about modularizing
> the current Gaia repo, which would allow developers to develop built-in
> apps in a way close to ordinary web apps.
>
> (presentation):
>
> https://docs.google.com/presentation/d/1gO6NLhYHRmscJF14MezfP9BDZIVRJ6Mw_o1ZqWhvbf4/edit?usp=sharing
>
> Now the first runnable demo is available on GitHub, and it includes these
> sub-projects:
>
> 1. Calendar app for demo: https://github.com/snowmantw/gaia-calendar
> 2. Homescreen for demo: https://github.com/snowmantw/gaia-homescreen
> 3. Essential parts of building profile:
> https://github.com/snowmantw/gaia-essential
> 4. Template.js in Bower package: https://github.com/snowmantw/libgaia-html
Rick
Rick Waldron
Dec 12, 2013, 5:30:07 PM12/12/13
to Greg Weng, dev-gaia
Sorry, hit "send" too soon.
This is pretty awesome and very closely resembles the rough plan that
Bocoup originally had for our contribution to Gaia. I wish you the best of
luck in adoption of the patterns you've presented here.
Rick
This is pretty awesome and very closely resembles the rough plan that
Bocoup originally had for our contribution to Gaia. I wish you the best of
luck in adoption of the patterns you've presented here.
Rick
Greg Weng
Dec 13, 2013, 2:58:41 AM12/13/13
to Rick Waldron, dev-gaia
We may start from some harmless parts like adopt require.js + bower in Gaia
repo, to solve the dependencies of libraries for each app first. Then, a
standalone building tool would be implemented, which should be able to
build a Gaia build like this demo shows, but it would be more generic and
mature.
To split Gaia apps would be the last step, and we're finding and trying
ways to keep the existing information, like git logs, would still be usable
after the splitting. We also would schedule a meeting to clarify all
necessary and detailed steps and obstacles.
2013/12/13 Rick Waldron <waldro...@gmail.com>
repo, to solve the dependencies of libraries for each app first. Then, a
standalone building tool would be implemented, which should be able to
build a Gaia build like this demo shows, but it would be more generic and
mature.
To split Gaia apps would be the last step, and we're finding and trying
ways to keep the existing information, like git logs, would still be usable
after the splitting. We also would schedule a meeting to clarify all
necessary and detailed steps and obstacles.
2013/12/13 Rick Waldron <waldro...@gmail.com>
Jan Jongboom
Dec 13, 2013, 5:31:41 AM12/13/13
to
Greg Weng
Dec 13, 2013, 5:54:39 AM12/13/13
to Jan Jongboom, dev-gaia
Yes, this would be difficult to balance the need of testing and project
managements, and the need of developing.
One things on my mind is how these things going on Linux distributions,
Android apps, Gnome, KDE and other similar projects?
Either they have good ways to solve these issues that we haven't adopted
and should adopt, or they all sucks on these things.
Another thing is that if we seem built-in apps as individual apps as they
should be, we may be able to consider that the easy parts you mentioned are
actually short-cuts,
and should be replaced with better patterns or tools. Maybe use a single
repo without partially checkout ability as a OS application layers and even
including apps is not a good idea at all.
2013/12/13 Jan Jongboom <janjo...@gmail.com>
managements, and the need of developing.
One things on my mind is how these things going on Linux distributions,
Android apps, Gnome, KDE and other similar projects?
Either they have good ways to solve these issues that we haven't adopted
and should adopt, or they all sucks on these things.
Another thing is that if we seem built-in apps as individual apps as they
should be, we may be able to consider that the easy parts you mentioned are
actually short-cuts,
and should be replaced with better patterns or tools. Maybe use a single
repo without partially checkout ability as a OS application layers and even
including apps is not a good idea at all.
2013/12/13 Jan Jongboom <janjo...@gmail.com>
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
Wilson Page
Dec 13, 2013, 8:38:30 AM12/13/13
to Greg Weng, dev-gaia, Jan Jongboom
Can't wait to have a setup like this. Full steam ahead!
----- Original Message -----
From: "Greg Weng" <snow...@gmail.com>
To: "Jan Jongboom" <janjo...@gmail.com>
Cc: "dev-gaia" <dev-...@lists.mozilla.org>
Sent: Friday, December 13, 2013 10:54:39 AM
Subject: Re: Demo for modularization Gaia repo
Yes, this would be difficult to balance the need of testing and project
managements, and the need of developing.
One things on my mind is how these things going on Linux distributions,
Android apps, Gnome, KDE and other similar projects?
Either they have good ways to solve these issues that we haven't adopted
and should adopt, or they all sucks on these things.
Another thing is that if we seem built-in apps as individual apps as they
should be, we may be able to consider that the easy parts you mentioned are
actually short-cuts,
and should be replaced with better patterns or tools. Maybe use a single
repo without partially checkout ability as a OS application layers and even
including apps is not a good idea at all.
2013/12/13 Jan Jongboom <janjo...@gmail.com>
----- Original Message -----
From: "Greg Weng" <snow...@gmail.com>
To: "Jan Jongboom" <janjo...@gmail.com>
Cc: "dev-gaia" <dev-...@lists.mozilla.org>
Sent: Friday, December 13, 2013 10:54:39 AM
Subject: Re: Demo for modularization Gaia repo
Yes, this would be difficult to balance the need of testing and project
managements, and the need of developing.
One things on my mind is how these things going on Linux distributions,
Android apps, Gnome, KDE and other similar projects?
Either they have good ways to solve these issues that we haven't adopted
and should adopt, or they all sucks on these things.
Another thing is that if we seem built-in apps as individual apps as they
should be, we may be able to consider that the easy parts you mentioned are
actually short-cuts,
and should be replaced with better patterns or tools. Maybe use a single
repo without partially checkout ability as a OS application layers and even
including apps is not a good idea at all.
2013/12/13 Jan Jongboom <janjo...@gmail.com>
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
--
<http://about.me/snowmantw>Greg Weng
http://about.me/snowmantw
*Understand y f = f [ y f ] ; lose last remaining non-major friend*
* -- Anonymous*
<http://about.me/snowmantw>Greg Weng
http://about.me/snowmantw
*Understand y f = f [ y f ] ; lose last remaining non-major friend*
* -- Anonymous*
Julien Wajsberg
Dec 13, 2013, 10:47:32 AM12/13/13
to Greg Weng, Jan Jongboom, dev-gaia
Le 13/12/2013 11:54, Greg Weng a écrit :
> Yes, this would be difficult to balance the need of testing and project
> managements, and the need of developing.
>
> One things on my mind is how these things going on Linux distributions,
Linux distributions have their own repository containing the files they
> Yes, this would be difficult to balance the need of testing and project
> managements, and the need of developing.
>
> One things on my mind is how these things going on Linux distributions,
embed (eg: debian's apt mirrors).
Axel Hecht
Dec 13, 2013, 11:11:30 AM12/13/13
to mozilla-...@lists.mozilla.org
Did you think about how that maps out for l10n yet?
Axel
Axel
Andreas Gal
Dec 13, 2013, 11:17:38 AM12/13/13
to Axel Hecht, mozilla-...@lists.mozilla.org
I would love for l10n to live much closer to the source (in each repo). With proper tooling and automation we should be able to not impact the way our l10n community submits and updates translations.
Andreas
On Dec 13, 2013, at 8:11 AM, Axel Hecht <l1...@mozilla.com> wrote:
> Did you think about how that maps out for l10n yet?
>
> Axel
>
> On 12/12/13 12:43 AM, Greg Weng wrote:
Greg Weng
Dec 13, 2013, 11:27:30 AM12/13/13
to dev-gaia
Forgot to forward the list.
----- 轉寄的郵件 -----
寄件者: "Greg Weng" <gw...@mozilla.com>
收件者: "Axel Hecht" <l1...@mozilla.com>
寄件備份: 2013 12 月 14 星期六 上午 12:26:50
主旨: Re: Demo for modularization Gaia repo
We currently can only focus on the most basic flow, because we're still discovering its pros and cons,
and how to make it smoother to try more cases.
However, I believe there're no reasons that individual apps can't own its l10n solution.
The worst case we may have a package like `gaia-locales-zh-TW`, just like in Linux distros,
and it would be possible to handle the build in some proper stage.
But I must say that I'm not expert at l10n, so if there're some concerns about why other projects' usual solutions are improper for us,
please correct me.
----- 原始郵件 -----
寄件者: "Axel Hecht" <l1...@mozilla.com>
收件者: mozilla-...@lists.mozilla.org
寄件備份: 2013 12 月 14 星期六 上午 12:11:30
主旨: Re: Demo for modularization Gaia repo
Did you think about how that maps out for l10n yet?
Axel
On 12/12/13 12:43 AM, Greg Weng wrote:
----- 轉寄的郵件 -----
寄件者: "Greg Weng" <gw...@mozilla.com>
收件者: "Axel Hecht" <l1...@mozilla.com>
寄件備份: 2013 12 月 14 星期六 上午 12:26:50
主旨: Re: Demo for modularization Gaia repo
We currently can only focus on the most basic flow, because we're still discovering its pros and cons,
and how to make it smoother to try more cases.
However, I believe there're no reasons that individual apps can't own its l10n solution.
The worst case we may have a package like `gaia-locales-zh-TW`, just like in Linux distros,
and it would be possible to handle the build in some proper stage.
But I must say that I'm not expert at l10n, so if there're some concerns about why other projects' usual solutions are improper for us,
please correct me.
----- 原始郵件 -----
寄件者: "Axel Hecht" <l1...@mozilla.com>
收件者: mozilla-...@lists.mozilla.org
寄件備份: 2013 12 月 14 星期六 上午 12:11:30
主旨: Re: Demo for modularization Gaia repo
Did you think about how that maps out for l10n yet?
Axel
On 12/12/13 12:43 AM, Greg Weng wrote:
Yuren Ju
Dec 13, 2013, 6:46:36 PM12/13/13
to Greg Weng, dev-gaia
we can list all things we do in build script, and think how we use this new
build tools to do those stuffs and make life easier
- generate profile
- js/css optimization
- l10n optimization (for performance)
- customization (https://wiki.mozilla.org/B2G/MarketCustomizations)
- build for different targets (DEBUG=1, SIMULATOR=1, PRODUCTION=1,
DESKTOP=1)
- generate default mozSettings
- generate preference
- generate manifest
- branding
- jslint/jshint
- localization
- unit test
- integration test
- install sample files
imagine we already had this tool — how do we use this tool to do these
things? it can help to figure out details, what the scope is and think how
do we design this tool.
build tools to do those stuffs and make life easier
- generate profile
- js/css optimization
- l10n optimization (for performance)
- customization (https://wiki.mozilla.org/B2G/MarketCustomizations)
- build for different targets (DEBUG=1, SIMULATOR=1, PRODUCTION=1,
DESKTOP=1)
- generate default mozSettings
- generate preference
- generate manifest
- branding
- jslint/jshint
- localization
- unit test
- integration test
- install sample files
imagine we already had this tool — how do we use this tool to do these
things? it can help to figure out details, what the scope is and think how
do we design this tool.
Axel Hecht
Dec 13, 2013, 7:05:33 PM12/13/13
to mozilla-...@lists.mozilla.org
On 12/13/13 9:17 AM, Andreas Gal wrote:
> I would love for l10n to live much closer to the source (in each repo). With proper tooling and automation we should be able to not impact the way our l10n community submits and updates translations.
>
> Andreas
To put down the expectations on l10n as I see them these days:
> I would love for l10n to live much closer to the source (in each repo). With proper tooling and automation we should be able to not impact the way our l10n community submits and updates translations.
>
> Andreas
The l10n community expects us to expose the strings in a common file
format with a single target. Right now that's a cumbersome and manual
process, though mostly tooled.
https://blog.mozilla.org/axel/2013/09/19/a-tale-of-convert-and-convert/
has details. That one target is then absorbed by the various different
toolchains people use. Sadly, no size fits two there.
Project drivers expect us to deliver a status on how l10n is doing for,
say, 1.2. I guess our answers today are already on the verbose side. As
we're growing locale coverage, this is getting more complicated as it is.
The less strict we get on the gaia side as to where the strings are and
which are used, the more complicated the system gets, and we may loose
insight at the end that we really want.
The good news is, all the stuff I'm doing is in public repos, and the
l10n automation and dashboard can be set up locally without any
dependency on upstream repos. So if someone or a group wants to come in
and help to create what's needed, that's totally doable. The code is
pretty much python throughout, http://pike.github.io/a10n/test-setup/
and http://pike.github.io/master-ball/test-master/ give you an idea what
it takes to run things locally.
Axel
Greg Weng
Dec 13, 2013, 10:52:27 PM12/13/13
to Yuren Ju, Greg Weng, dev-gaia
In fact I'm developing a tool named Aether (which born Gaia in some version
of Greek mythology)(Chaos is not a good name at all LOL).
I should be able run features written in cucumber next week.
2013/12/14 Yuren Ju <y...@mozilla.com>
of Greek mythology)(Chaos is not a good name at all LOL).
I should be able run features written in cucumber next week.
2013/12/14 Yuren Ju <y...@mozilla.com>
> > Did you think about how that maps out for l10n yet?
> >
> > Axel
> >
> > On 12/12/13 12:43 AM, Greg Weng wrote:
> >
> > Axel
> >
> > On 12/12/13 12:43 AM, Greg Weng wrote:
> > _______________________________________________
> > dev-gaia mailing list
> > dev-...@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-gaia
> > _______________________________________________
> > dev-gaia mailing list
> > dev-...@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-gaia
> >
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
> > dev-gaia mailing list
> > dev-...@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-gaia
> > _______________________________________________
> > dev-gaia mailing list
> > dev-...@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-gaia
> >
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
Greg Weng
Jan 2, 2014, 9:32:20 AM1/2/14
to Gareth Aye, dev-gaia
(Reply public because these may be other's questions, too.)
Bower manages "client" JavaScript libraries. That is, if you have a library
used in App like our Template.js, you can pack it as a bower package and
publish it as a real library. And if this library require others to make it
work, bower can also download and manage them. It is not NPM's
responsibilities because NPM only handle Node.js' libraries, and we use no
Node.js code in published Gaia.
Grunt has it's easy to use and customized plugin system, which means we can
easily modularize our building modules. Things like grunt-jshint, grunt-git
and grunt-jsdoc can be plugged into our building process with no inventing
wheels by ourselves. Moreover, these plugins can be managed by NPM, too.
I'm not automake's antagonist, but I think to have a dependency manager and
formal way to extend our building system is important. And we can uniform
our language to JS to reduce overhead of maintain multiple languages in our
codebase.
2014/1/2 Gareth Aye <garet...@gmail.com>
> Is there a meta bug to track gaia repo modularization? There are lots of
> small fixes we'll need like making sure the integration and unit test
> runners do the right thing in each repo.
>
> Also why did we move from make => grunt and npm => bower? I don't know
> terribly much about grunt and bower, but make and npm worked for me.
>
>
> On Thu, Jan 2, 2014 at 8:57 AM, Gareth Aye <garet...@gmail.com> wrote:
>
>> I want this. When can I have it?
>> Gareth
>>
>>
>
>
> --
> Best,
> Gareth
Bower manages "client" JavaScript libraries. That is, if you have a library
used in App like our Template.js, you can pack it as a bower package and
publish it as a real library. And if this library require others to make it
work, bower can also download and manage them. It is not NPM's
responsibilities because NPM only handle Node.js' libraries, and we use no
Node.js code in published Gaia.
Grunt has it's easy to use and customized plugin system, which means we can
easily modularize our building modules. Things like grunt-jshint, grunt-git
and grunt-jsdoc can be plugged into our building process with no inventing
wheels by ourselves. Moreover, these plugins can be managed by NPM, too.
I'm not automake's antagonist, but I think to have a dependency manager and
formal way to extend our building system is important. And we can uniform
our language to JS to reduce overhead of maintain multiple languages in our
codebase.
2014/1/2 Gareth Aye <garet...@gmail.com>
> Is there a meta bug to track gaia repo modularization? There are lots of
> small fixes we'll need like making sure the integration and unit test
> runners do the right thing in each repo.
>
> Also why did we move from make => grunt and npm => bower? I don't know
> terribly much about grunt and bower, but make and npm worked for me.
>
>
> On Thu, Jan 2, 2014 at 8:57 AM, Gareth Aye <garet...@gmail.com> wrote:
>
>> I want this. When can I have it?
>>> > > _______________________________________________
>>> > > dev-gaia mailing list
>>> > > dev-...@lists.mozilla.org
>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>> > > _______________________________________________
>>> > > dev-gaia mailing list
>>> > > dev-...@lists.mozilla.org
>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>> > >
>>> > _______________________________________________
>>> > dev-gaia mailing list
>>> > dev-...@lists.mozilla.org
>>> > https://lists.mozilla.org/listinfo/dev-gaia
>>> >
>>>
>>>
>>>
>>> > > dev-gaia mailing list
>>> > > dev-...@lists.mozilla.org
>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>> > > _______________________________________________
>>> > > dev-gaia mailing list
>>> > > dev-...@lists.mozilla.org
>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>> > >
>>> > _______________________________________________
>>> > dev-gaia mailing list
>>> > dev-...@lists.mozilla.org
>>> > https://lists.mozilla.org/listinfo/dev-gaia
>>> >
>>>
>>>
>>>
>>> --
>>> <http://about.me/snowmantw>Greg Weng
>>>
>>> http://about.me/snowmantw
>>>
>>> *Understand y f = f [ y f ] ; lose last remaining non-major friend*
>>>
>>> * -- Anonymous*
>>> <http://about.me/snowmantw>Greg Weng
>>>
>>> http://about.me/snowmantw
>>>
>>> *Understand y f = f [ y f ] ; lose last remaining non-major friend*
>>>
>>> * -- Anonymous*
>>> _______________________________________________
>>> dev-gaia mailing list
>>> dev-...@lists.mozilla.org
>>> https://lists.mozilla.org/listinfo/dev-gaia
>>>
>>
>>
>>
>> --
>> Best,
>>> dev-gaia mailing list
>>> dev-...@lists.mozilla.org
>>> https://lists.mozilla.org/listinfo/dev-gaia
>>>
>>
>>
>>
>> --
>> Gareth
>>
>>
>
>
> --
> Best,
> Gareth
Greg Weng
Jan 2, 2014, 9:34:54 AM1/2/14
to Gareth Aye, dev-gaia
Well I must clarify that "we use no Node.js code in published Gaia" means
that our Gaia runs on Gecko/SpiderMonkey, and there is no Node.js code in
runtime, so NPM can't and shouldn't manage libraries used in App in the
runtime.
2014/1/2 Greg Weng <snow...@gmail.com>
that our Gaia runs on Gecko/SpiderMonkey, and there is no Node.js code in
runtime, so NPM can't and shouldn't manage libraries used in App in the
runtime.
2014/1/2 Greg Weng <snow...@gmail.com>
>>>> > > _______________________________________________
>>>> > > dev-gaia mailing list
>>>> > > dev-...@lists.mozilla.org
>>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>>> > > _______________________________________________
>>>> > > dev-gaia mailing list
>>>> > > dev-...@lists.mozilla.org
>>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>>> > >
>>>> > _______________________________________________
>>>> > dev-gaia mailing list
>>>> > dev-...@lists.mozilla.org
>>>> > https://lists.mozilla.org/listinfo/dev-gaia
>>>> >
>>>>
>>>>
>>>>
>>>> > > dev-gaia mailing list
>>>> > > dev-...@lists.mozilla.org
>>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>>> > > _______________________________________________
>>>> > > dev-gaia mailing list
>>>> > > dev-...@lists.mozilla.org
>>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>>> > >
>>>> > _______________________________________________
>>>> > dev-gaia mailing list
>>>> > dev-...@lists.mozilla.org
>>>> > https://lists.mozilla.org/listinfo/dev-gaia
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> <http://about.me/snowmantw>Greg Weng
>>>>
>>>> http://about.me/snowmantw
>>>>
>>>> *Understand y f = f [ y f ] ; lose last remaining non-major friend*
>>>>
>>>> <http://about.me/snowmantw>Greg Weng
>>>>
>>>> http://about.me/snowmantw
>>>>
>>>> *Understand y f = f [ y f ] ; lose last remaining non-major friend*
>>>>
>>>> * -- Anonymous*
>>>> _______________________________________________
>>>> dev-gaia mailing list
>>>> dev-...@lists.mozilla.org
>>>> https://lists.mozilla.org/listinfo/dev-gaia
>>>>
>>>
>>>
>>>
>>> --
>>> Best,
>>> Gareth
>>>
>>>
>>
>>
>> --
>> Best,
>> Gareth
>>
>>
>
>
>>>> _______________________________________________
>>>> dev-gaia mailing list
>>>> dev-...@lists.mozilla.org
>>>> https://lists.mozilla.org/listinfo/dev-gaia
>>>>
>>>
>>>
>>>
>>> --
>>> Best,
>>> Gareth
>>>
>>>
>>
>>
>> --
>> Best,
>> Gareth
>>
>>
>
>
Greg Weng
Jan 2, 2014, 2:41:18 PM1/2/14
to Gareth Aye, dev-gaia
(Replied public because references)
The difference between bower and npm:
http://stackoverflow.com/questions/18641899/difference-between-bower-and-npm
http://tech.pro/tutorial/1190/package-managers-an-introductory-guide-for-the-uninitiated-front-end-developer#what_is_bower
http://stackoverflow.com/questions/15092345/javascript-dependency-management-npm-vs-bower-vs-volo
About the bug for building tool: well, it's still in an very experimental
stage so I treat it as a standalone and personal project before. Maybe
you're right that we can start to treat it as a formal feature in our
building system, therefore a bug for it would be suitable.
I can leave its GitHub URL here:
https://github.com/snowmantw/aether
But there're so many concepts and plans are still missing, and I'm also
trying to discover which is a better way to develop such tool. The main
difference between it and Gaia (or other Mozilla products?) is that I'm
trying using BDD to develop this tool (with Cucumber in Node.js), so you
may see there is a "features" directory in the repo.
2014/1/2 Gareth Aye <garet...@gmail.com>
> Thanks Greg! How about a bug to track this work? I would love to
> contribute too but don't know where to start necessarily.
>
> Also, I don't think I understand fully what bower does that npm doesn't.
> Is there some reading you could recommend?
>
>
> On Thu, Jan 2, 2014 at 9:48 AM, Greg Weng <snow...@gmail.com> wrote:
>
>> I'm developing it daily after I solved some critical or blocked bugs, but
>> it's still in a unusable stage.
>> I may publish a partial workable version next week.
>>
>>
>> 2014/1/2 Gareth Aye <garet...@gmail.com>
> Best,
> Gareth
The difference between bower and npm:
http://stackoverflow.com/questions/18641899/difference-between-bower-and-npm
http://tech.pro/tutorial/1190/package-managers-an-introductory-guide-for-the-uninitiated-front-end-developer#what_is_bower
http://stackoverflow.com/questions/15092345/javascript-dependency-management-npm-vs-bower-vs-volo
About the bug for building tool: well, it's still in an very experimental
stage so I treat it as a standalone and personal project before. Maybe
you're right that we can start to treat it as a formal feature in our
building system, therefore a bug for it would be suitable.
I can leave its GitHub URL here:
https://github.com/snowmantw/aether
But there're so many concepts and plans are still missing, and I'm also
trying to discover which is a better way to develop such tool. The main
difference between it and Gaia (or other Mozilla products?) is that I'm
trying using BDD to develop this tool (with Cucumber in Node.js), so you
may see there is a "features" directory in the repo.
2014/1/2 Gareth Aye <garet...@gmail.com>
> Thanks Greg! How about a bug to track this work? I would love to
> contribute too but don't know where to start necessarily.
>
> Also, I don't think I understand fully what bower does that npm doesn't.
> Is there some reading you could recommend?
>
>
> On Thu, Jan 2, 2014 at 9:48 AM, Greg Weng <snow...@gmail.com> wrote:
>
>> I'm developing it daily after I solved some critical or blocked bugs, but
>> it's still in a unusable stage.
>> I may publish a partial workable version next week.
>>
>>
>> 2014/1/2 Gareth Aye <garet...@gmail.com>
>>>> > > _______________________________________________
>>>> > > dev-gaia mailing list
>>>> > > dev-...@lists.mozilla.org
>>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>>> > > _______________________________________________
>>>> > > dev-gaia mailing list
>>>> > > dev-...@lists.mozilla.org
>>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>>> > >
>>>> > _______________________________________________
>>>> > dev-gaia mailing list
>>>> > dev-...@lists.mozilla.org
>>>> > https://lists.mozilla.org/listinfo/dev-gaia
>>>> >
>>>>
>>>>
>>>>
>>>> > > dev-gaia mailing list
>>>> > > dev-...@lists.mozilla.org
>>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>>> > > _______________________________________________
>>>> > > dev-gaia mailing list
>>>> > > dev-...@lists.mozilla.org
>>>> > > https://lists.mozilla.org/listinfo/dev-gaia
>>>> > >
>>>> > _______________________________________________
>>>> > dev-gaia mailing list
>>>> > dev-...@lists.mozilla.org
>>>> > https://lists.mozilla.org/listinfo/dev-gaia
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> <http://about.me/snowmantw>Greg Weng
>>>>
>>>> http://about.me/snowmantw
>>>>
>>>> *Understand y f = f [ y f ] ; lose last remaining non-major friend*
>>>>
>>>>
>>>> <http://about.me/snowmantw>Greg Weng
>>>>
>>>> http://about.me/snowmantw
>>>>
>>>> *Understand y f = f [ y f ] ; lose last remaining non-major friend*
>>>>
>>>> * -- Anonymous*
>>>> _______________________________________________
>>>> dev-gaia mailing list
>>>> dev-...@lists.mozilla.org
>>>> https://lists.mozilla.org/listinfo/dev-gaia
>>>>
>>>
>>>
>>>
>>> --
>>> Best,
>>> Gareth
>>>
>>>
>>
>>
>>>> _______________________________________________
>>>> dev-gaia mailing list
>>>> dev-...@lists.mozilla.org
>>>> https://lists.mozilla.org/listinfo/dev-gaia
>>>>
>>>
>>>
>>>
>>> --
>>> Best,
>>> Gareth
>>>
>>>
>>
>>
>> --
>> <http://about.me/snowmantw>Greg Weng
>>
>> http://about.me/snowmantw
>>
>> *Understand y f = f [ y f ] ; lose last remaining non-major friend*
>>
>> * -- Anonymous*
>>
>>
>
>
> --
>> <http://about.me/snowmantw>Greg Weng
>>
>> http://about.me/snowmantw
>>
>> *Understand y f = f [ y f ] ; lose last remaining non-major friend*
>>
>> * -- Anonymous*
>>
>>
>
>
> Best,
> Gareth
Alexandre poirot
Jan 2, 2014, 6:00:14 PM1/2/14
to Greg Weng, dev-gaia
In order to be concrete and start landing patches, I'd suggest you to start
a draft plan of
possible iterations we may execute that would start heading toward better
modularization
and tell us how much effort it would take to solve specific issues and what
it would change for developers.
Smaller focused iterations would help synchronizing different tools
(travis, tbpl, each app custom build script),
and teams (localization, releng and partners).
I'm not convinced we will tackle all gaia itchy points by executing such
big move.
Ideally, we would focus on the different topics we want to improve one by
one:
* tests workflow,
* repo size,
* shared folder story,
* localization workflow.
* ...
Speaking about repo size, I don't know how much keyboard dictionaries are
compressed by git,
but they are taking 42% of uncompressed gaia size...
We also have 28% for media-samples and test_media.
So there is easier/faster ways to strim down gaia repo!
If we move these very specific resources to external repo,
we would strip down from 600MB to 136MB (uncompressed gaia).
What could download a versioned tarball of these data files using magic
github urls:
https://github.com/mozilla-b2g/gaia-dictionaries/tarball/67d5c214cee036bdaa937c7a8644f776424f7970
Also that would be really great to ensure that it will still be possible to
build gaia from Firefox,
via the gaia-build addon Yuren wrote.
a draft plan of
possible iterations we may execute that would start heading toward better
modularization
and tell us how much effort it would take to solve specific issues and what
it would change for developers.
Smaller focused iterations would help synchronizing different tools
(travis, tbpl, each app custom build script),
and teams (localization, releng and partners).
I'm not convinced we will tackle all gaia itchy points by executing such
big move.
Ideally, we would focus on the different topics we want to improve one by
one:
* tests workflow,
* repo size,
* shared folder story,
* localization workflow.
* ...
Speaking about repo size, I don't know how much keyboard dictionaries are
compressed by git,
but they are taking 42% of uncompressed gaia size...
We also have 28% for media-samples and test_media.
So there is easier/faster ways to strim down gaia repo!
If we move these very specific resources to external repo,
we would strip down from 600MB to 136MB (uncompressed gaia).
What could download a versioned tarball of these data files using magic
github urls:
https://github.com/mozilla-b2g/gaia-dictionaries/tarball/67d5c214cee036bdaa937c7a8644f776424f7970
Also that would be really great to ensure that it will still be possible to
build gaia from Firefox,
via the gaia-build addon Yuren wrote.
Greg Weng
Jan 2, 2014, 6:44:55 PM1/2/14
to Alexandre poirot, dev-gaia
Yes, we don't want to make a such long jump to the destination. It's
apparently impossible, and would cause a disaster. All demos and projects
you can see are experimental, and their usage is to show how many effort we
may need to put and where we may get stuck, of course they would also show
what we can gain and what we would lose in such different way.
We would also discuss this and submit some drafts in this new year. This
thing had been discussed to be scheduled several weeks ago, but you know it
would be hard to progress during the vacation.
But I think the original ideas and issues is still valid: giant meta
project, especially when it includes so many sub units/projects, should or
should not include these theoretical independent units as a part of its
repo? Yes, it's convenient enough to do a complete test, and people can
always easily to solve some unavoidable cross-units problem. But sometimes
this model also shows some inconveniences, especially when one modification
may only concern some single and isolated unit (app, in Gaia), it shouldn't
be intervened by other units which should be isolated, too. I think these
issues had been discussed in previous threads, and the size is not the only
one we concerns.
2014/1/3 Alexandre poirot <poiro...@gmail.com>
apparently impossible, and would cause a disaster. All demos and projects
you can see are experimental, and their usage is to show how many effort we
may need to put and where we may get stuck, of course they would also show
what we can gain and what we would lose in such different way.
We would also discuss this and submit some drafts in this new year. This
thing had been discussed to be scheduled several weeks ago, but you know it
would be hard to progress during the vacation.
But I think the original ideas and issues is still valid: giant meta
project, especially when it includes so many sub units/projects, should or
should not include these theoretical independent units as a part of its
repo? Yes, it's convenient enough to do a complete test, and people can
always easily to solve some unavoidable cross-units problem. But sometimes
this model also shows some inconveniences, especially when one modification
may only concern some single and isolated unit (app, in Gaia), it shouldn't
be intervened by other units which should be isolated, too. I think these
issues had been discussed in previous threads, and the size is not the only
one we concerns.
2014/1/3 Alexandre poirot <poiro...@gmail.com>
Jan Jongboom
Jan 8, 2014, 9:30:51 AM1/8/14
to
So I gave this some more thought yesterday and I think modularizing apps into separate repo's is a terrible idea. We already have a ton of pain with the fact that gaia and gecko don't live in the same repo and break tests for that matter and I think this'll only get worse. When working at Cloud9 we also had this, part of our code was open source so it was in a different repo, and then we had to constantly commit to two branches, sync up, make sure everything was in order. There are always refactorings (activities or how keyboard/system work together) that will require touching more repo's; and that gets very painful.
Also the article I saw yesterday from FB (not that that would mean we should do it):
"Our code base has grown organically and its internal dependencies are very complex. We could have spent a lot of time making it more modular in a way that would be friendly to a source control tool, but there are a number of benefits to using a single repository. Even at our current scale, we often make large changes throughout our code base, and having a single repository is useful for continuous modernization. Splitting it up would make large, atomic refactorings more difficult. "
https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/
Every time Ryan pokes me about some test that fails because there's a breaking change in gecko and the gaia commit is not compatible bla bla I die a little inside and I don't want that to happen more.
Also the article I saw yesterday from FB (not that that would mean we should do it):
"Our code base has grown organically and its internal dependencies are very complex. We could have spent a lot of time making it more modular in a way that would be friendly to a source control tool, but there are a number of benefits to using a single repository. Even at our current scale, we often make large changes throughout our code base, and having a single repository is useful for continuous modernization. Splitting it up would make large, atomic refactorings more difficult. "
https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/
Every time Ryan pokes me about some test that fails because there's a breaking change in gecko and the gaia commit is not compatible bla bla I die a little inside and I don't want that to happen more.
Greg Weng
Jan 8, 2014, 11:53:08 AM1/8/14
to Jan Jongboom, dev-gaia
Today we just discussed build script roadmap and I mentioned that our Gaia
repo is more like a semi-source repo, which is more like Linux distros'
repo, rather than the applications' source code repo. This means we treat
it as a ready-to-build-and-deploy unit, rather than a meta-project that
include small and independent units.
So maybe this is the problem: we're doing source-level developing in a
distro repo. However, I'm not sure how this thing be solved between distro
and applications, like Gnome's widgets in Debian distro.
And as I read discussions here, I've found that we may only be able to
choose one extreme solution instead of a compromised one: to fully satisfy
the request of testing and debugging at the built build, it's would be
convenient to make a giant repo with all code, including
Gonk/Gecko/Gaia/NPM modules/shared libraries/resources, just like what
previous opinions mentioned. If you leave any single part of these
project's code not in this colossal repo, you can always find some glitches
to complain that to test and find commits is hard, and the fault is because
we haven't merge code into such repo.
On the other hand, I can't find any wrong about to put applications in an
OS like project in different repo, and develop them individually.
Especially when you're a developer of, say a simple application like a
desktop memo widget; you may not be happy if you must wait from kernel to
libc, X, libgtk and libgnome to compile itself and pass all tests (not to
mention some of them may fail at some irrelevant stages frequently) *every
time* you just committed few CSS changes to change your widget's
appearance. It also odd to tell an Mplayer (Video in our case) developer
that if he want to contribute to the project, he must clone a repo includes
everything from kernel to desktop environment (if we really merged all
FirefoxOS projects to eliminate all of the testing glitches as I
mentioned). I don't think that (not to do this) is only for to be friend to
a source control tool.
Also, I think while we're trying to kill the pain of testing, finding
commits and some other similar things, we also need to care about the need
of development, and the problems in the current and the colossal repo
model. If we can find a win-win way here, it would be a huge victory.
2014/1/8 Jan Jongboom <janjo...@gmail.com>
repo is more like a semi-source repo, which is more like Linux distros'
repo, rather than the applications' source code repo. This means we treat
it as a ready-to-build-and-deploy unit, rather than a meta-project that
include small and independent units.
So maybe this is the problem: we're doing source-level developing in a
distro repo. However, I'm not sure how this thing be solved between distro
and applications, like Gnome's widgets in Debian distro.
And as I read discussions here, I've found that we may only be able to
choose one extreme solution instead of a compromised one: to fully satisfy
the request of testing and debugging at the built build, it's would be
convenient to make a giant repo with all code, including
Gonk/Gecko/Gaia/NPM modules/shared libraries/resources, just like what
previous opinions mentioned. If you leave any single part of these
project's code not in this colossal repo, you can always find some glitches
to complain that to test and find commits is hard, and the fault is because
we haven't merge code into such repo.
On the other hand, I can't find any wrong about to put applications in an
OS like project in different repo, and develop them individually.
Especially when you're a developer of, say a simple application like a
desktop memo widget; you may not be happy if you must wait from kernel to
libc, X, libgtk and libgnome to compile itself and pass all tests (not to
mention some of them may fail at some irrelevant stages frequently) *every
time* you just committed few CSS changes to change your widget's
appearance. It also odd to tell an Mplayer (Video in our case) developer
that if he want to contribute to the project, he must clone a repo includes
everything from kernel to desktop environment (if we really merged all
FirefoxOS projects to eliminate all of the testing glitches as I
mentioned). I don't think that (not to do this) is only for to be friend to
a source control tool.
Also, I think while we're trying to kill the pain of testing, finding
commits and some other similar things, we also need to care about the need
of development, and the problems in the current and the colossal repo
model. If we can find a win-win way here, it would be a huge victory.
2014/1/8 Jan Jongboom <janjo...@gmail.com>
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
Kevin Grandon
Jan 8, 2014, 12:12:18 PM1/8/14
to Greg Weng, dev-gaia, Jan Jongboom
Just chiming in here with my thoughts. There are pros and cons of leaving as-is, or splitting up the repo, and I could go either way with this. Some things would be harder one way, and some the other way.
In any case, I think we're biting off too much right now. It's fun to think about, but I think a better approach would be to iteratively move towards some solution. I would propose something like:
For 1.4 - Take a single file from shared/ and move it to an external repo. Something like l10n, or lazy loader.
For 1.5 - Expand the idea to an entire folder, probably the shared/ folder, or all of building blocks.
For 1.6+ - Start thinking about moving apps into their own repos.
After each step we should evaluate the workflow at that time, solve any painpoints, or re-consider the idea. Like others have said, we need to understand what impact this has on testing, localization, and future refactoring.
Best,
Kevin
----- Original Message -----
From: "Greg Weng" <snow...@gmail.com>
To: "Jan Jongboom" <janjo...@gmail.com>
Cc: "dev-gaia" <dev-...@lists.mozilla.org>
In any case, I think we're biting off too much right now. It's fun to think about, but I think a better approach would be to iteratively move towards some solution. I would propose something like:
For 1.4 - Take a single file from shared/ and move it to an external repo. Something like l10n, or lazy loader.
For 1.5 - Expand the idea to an entire folder, probably the shared/ folder, or all of building blocks.
For 1.6+ - Start thinking about moving apps into their own repos.
After each step we should evaluate the workflow at that time, solve any painpoints, or re-consider the idea. Like others have said, we need to understand what impact this has on testing, localization, and future refactoring.
Best,
Kevin
----- Original Message -----
From: "Greg Weng" <snow...@gmail.com>
To: "Jan Jongboom" <janjo...@gmail.com>
Cc: "dev-gaia" <dev-...@lists.mozilla.org>
> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>
--
<http://about.me/snowmantw>Greg Weng
http://about.me/snowmantw
*Understand y f = f [ y f ] ; lose last remaining non-major friend*
* -- Anonymous*
<http://about.me/snowmantw>Greg Weng
http://about.me/snowmantw
*Understand y f = f [ y f ] ; lose last remaining non-major friend*
* -- Anonymous*
Fabrice Desré
Jan 8, 2014, 12:35:29 PM1/8/14
to Kevin Grandon, Greg Weng, dev-gaia, Jan Jongboom
but... this is clearly taking one side here Kevin? :P
How do people expect to bisect once we have a gazillions repos? I don't
think that gecko+gaia would be "huge" repository by today's standards.
Big, yes, but you're not cloning on your phone ;)
The platform is still changing too much to decouple the core apps.
Keep in mind that certified apps are pretty much part of the platform -
they are different beasts than mplayer on linux/unix, which is a 20
years old environment.
Fabrice
b2g team
Mozilla Corporation
How do people expect to bisect once we have a gazillions repos? I don't
think that gecko+gaia would be "huge" repository by today's standards.
Big, yes, but you're not cloning on your phone ;)
The platform is still changing too much to decouple the core apps.
Keep in mind that certified apps are pretty much part of the platform -
they are different beasts than mplayer on linux/unix, which is a 20
years old environment.
Fabrice
>> _______________________________________________
>> dev-gaia mailing list
>> dev-...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-gaia
>>
>
>
>
--
Fabrice Desr�
>> dev-gaia mailing list
>> dev-...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-gaia
>>
>
>
>
--
b2g team
Mozilla Corporation
Kevin Grandon
Jan 8, 2014, 1:35:42 PM1/8/14
to Fabrice Desré, Greg Weng, dev-gaia, Jan Jongboom
Sorry, I suppose what I meant was that I saw that as the only path forward if we were going to go down that path.
My opinion is that stuff in shared/ should be split out into their own repos, but apps should stay in the primary gaia repo for the time being. I don't think we can make an informed decision about apps until this happens.
b2g team
Mozilla Corporation
My opinion is that stuff in shared/ should be split out into their own repos, but apps should stay in the primary gaia repo for the time being. I don't think we can make an informed decision about apps until this happens.
>> _______________________________________________
>> dev-gaia mailing list
>> dev-...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-gaia
>>
>
>
>
--
Fabrice Desré
>> dev-gaia mailing list
>> dev-...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-gaia
>>
>
>
>
--
b2g team
Mozilla Corporation
Greg Weng
Jan 8, 2014, 8:53:35 PM1/8/14
to Fabrice Desré, dev-gaia, Jan Jongboom, Kevin Grandon
Well, I said "huge" not for its size, but the architecture. We will need to
make a single repo include one OS.
Beside that, I've thought what improvements can be done in the current
model (single repo for this OS' all builtin applications)
1. Can we just test those parts relevant to my patch? I feel really bad
every time my patch got stuck because irrelevant timeout and others'
failures, especially when the patch just changed some text or styles (hey,
it didn't even change any code!). Sometime it event got stuck from 3 days
to 1 week, and you have to poke Travis daily and manually to see if it can
pass the tests of "the other parts". Not to mention that your code may be
conflicted with others during this waiting time.
2. Is it possible to arrange the full-scanning test runs individually?
Maybe it can run every hour or half hour after pulling the last changes,
but it would not block patches until it got broken (really broken, not just
timeout!). We can still back out code according to this test, while patches
can be test more quickly.
3. Even we want to keep a single repo for all built-in apps, our shared
libraries, which just like other open source libraries, still need to be
split as encapsulated unit and can be published on GitHub. In this way we
can benefit other web developers (they may include these libraries in their
website or web apps), and eliminate the black-magic in build script by
introducing Bower as our dependency and requiring tool. I believe it would
bring us clear enough way to manage these libraries.
4. Resources like dictionaries may be in individual repos. And maybe test
apps and experimental apps, especially when you don't need *all* these apps.
5. Is it possible to partially checkout our repo, just like SVN provides?
This concerns maybe some problems we discussed here are relevant to the
tool itself, not our way to use it.
Also, we can start from splitting one existing or new app as an individual
repo (maybe the LockScreen app in the 'near' future?), and try to develop
and manage it to see what would happen if we use the splitting model.
2014/1/9 Fabrice Desré <fab...@mozilla.com>
make a single repo include one OS.
Beside that, I've thought what improvements can be done in the current
model (single repo for this OS' all builtin applications)
1. Can we just test those parts relevant to my patch? I feel really bad
every time my patch got stuck because irrelevant timeout and others'
failures, especially when the patch just changed some text or styles (hey,
it didn't even change any code!). Sometime it event got stuck from 3 days
to 1 week, and you have to poke Travis daily and manually to see if it can
pass the tests of "the other parts". Not to mention that your code may be
conflicted with others during this waiting time.
2. Is it possible to arrange the full-scanning test runs individually?
Maybe it can run every hour or half hour after pulling the last changes,
but it would not block patches until it got broken (really broken, not just
timeout!). We can still back out code according to this test, while patches
can be test more quickly.
3. Even we want to keep a single repo for all built-in apps, our shared
libraries, which just like other open source libraries, still need to be
split as encapsulated unit and can be published on GitHub. In this way we
can benefit other web developers (they may include these libraries in their
website or web apps), and eliminate the black-magic in build script by
introducing Bower as our dependency and requiring tool. I believe it would
bring us clear enough way to manage these libraries.
4. Resources like dictionaries may be in individual repos. And maybe test
apps and experimental apps, especially when you don't need *all* these apps.
5. Is it possible to partially checkout our repo, just like SVN provides?
This concerns maybe some problems we discussed here are relevant to the
tool itself, not our way to use it.
Also, we can start from splitting one existing or new app as an individual
repo (maybe the LockScreen app in the 'near' future?), and try to develop
and manage it to see what would happen if we use the splitting model.
2014/1/9 Fabrice Desré <fab...@mozilla.com>
> >> _______________________________________________
> >> dev-gaia mailing list
> >> dev-...@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-gaia
> >>
> >
> >
> >
>
>
> --
> Fabrice Desré
> b2g team
> Mozilla Corporation
>
> >> dev-gaia mailing list
> >> dev-...@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-gaia
> >>
> >
> >
> >
>
>
> --
> Fabrice Desré
> b2g team
> Mozilla Corporation
>
Axel Hecht
Jan 9, 2014, 8:44:15 AM1/9/14
to mozilla-...@lists.mozilla.org
One the engineering complexity side, shared/ is one of those things that
I expect show all the problems of modularization and packages.
If a change to shared breaks an app, who's fixing that? How do we get
all apps to actually use the same version of shared by the time we
integrate a product? If we don't, could we actually localize different
versions of shared at the same time? (nobody does anything like that today)
There's much evidence in package managers out there that the problem is
hard. Linux builds and xulrunner and nsprpub plus localizations for
example is bubbling up every now and then. 20 different versions some
nifty node module inside node_modules because it's used 20 times. Kinda
works, but we also want to run on 128mb devices.
Which reminds of a distraction, has anyone ever tried to package shared/
inside a gecko addon in the gaia profile to expose them as res:// URLs?
That might buy us a bunch of package size, and possibly some caching wins.
Axel
I expect show all the problems of modularization and packages.
If a change to shared breaks an app, who's fixing that? How do we get
all apps to actually use the same version of shared by the time we
integrate a product? If we don't, could we actually localize different
versions of shared at the same time? (nobody does anything like that today)
There's much evidence in package managers out there that the problem is
hard. Linux builds and xulrunner and nsprpub plus localizations for
example is bubbling up every now and then. 20 different versions some
nifty node module inside node_modules because it's used 20 times. Kinda
works, but we also want to run on 128mb devices.
Which reminds of a distraction, has anyone ever tried to package shared/
inside a gecko addon in the gaia profile to expose them as res:// URLs?
That might buy us a bunch of package size, and possibly some caching wins.
Axel
Gareth Aye
Feb 13, 2014, 12:04:36 PM2/13/14
to Fabrice Desré, Greg Weng, dev-gaia, Jan Jongboom, Kevin Grandon
Fabrice,
We can use git submodules so that development that happens in app-only
repositories can be upstreamed into gaia proper. The submodule updates can
be bisected like any other commits. I would actually be willing to do this
for calendar this week if no one has any strong objections...
On Wed, Jan 8, 2014 at 12:35 PM, Fabrice Desré <fab...@mozilla.com> wrote:
> but... this is clearly taking one side here Kevin? :P
>
> How do people expect to bisect once we have a gazillions repos? I don't
> think that gecko+gaia would be "huge" repository by today's standards.
> Big, yes, but you're not cloning on your phone ;)
>
> The platform is still changing too much to decouple the core apps.
> Keep in mind that certified apps are pretty much part of the platform -
> they are different beasts than mplayer on linux/unix, which is a 20
> years old environment.
>
> Fabrice
>
We can use git submodules so that development that happens in app-only
repositories can be upstreamed into gaia proper. The submodule updates can
be bisected like any other commits. I would actually be willing to do this
for calendar this week if no one has any strong objections...
On Wed, Jan 8, 2014 at 12:35 PM, Fabrice Desré <fab...@mozilla.com> wrote:
> but... this is clearly taking one side here Kevin? :P
>
> How do people expect to bisect once we have a gazillions repos? I don't
> think that gecko+gaia would be "huge" repository by today's standards.
> Big, yes, but you're not cloning on your phone ;)
>
> The platform is still changing too much to decouple the core apps.
> Keep in mind that certified apps are pretty much part of the platform -
> they are different beasts than mplayer on linux/unix, which is a 20
> years old environment.
>
> Fabrice
>
> >> _______________________________________________
> >> dev-gaia mailing list
> >> dev-...@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-gaia
> >>
> >
> >
> >
>
>
> >> dev-gaia mailing list
> >> dev-...@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-gaia
> >>
> >
> >
> >
>
>
> --
> Fabrice Desré
> b2g team
> Mozilla Corporation
> Fabrice Desré
> b2g team
> Mozilla Corporation
Gareth Aye
Feb 13, 2014, 12:10:19 PM2/13/14
to Fabrice Desré, Greg Weng, dev-gaia, Jan Jongboom, Kevin Grandon
s/upstream/downstream/ sorry!
On Thu, Feb 13, 2014 at 12:04 PM, Gareth Aye
<gar...@alumni.middlebury.edu>wrote:
On Thu, Feb 13, 2014 at 12:04 PM, Gareth Aye
<gar...@alumni.middlebury.edu>wrote:
Kevin Grandon
Feb 13, 2014, 12:22:56 PM2/13/14
to gar...@alumni.middlebury.edu, Greg Weng, Fabrice Desré, dev-gaia, Jan Jongboom
I actually did the same thing before with git submodules, but rel eng does not support them yet - so no dice. We would need to put this on their roadmap if we really wanted to go with git submodules.
They are clunky, but I think they could get the job done. Note: I still would not yet recommend placing apps in submodules. I think starting with shared/ librares would make the most sense initially.
They are clunky, but I think they could get the job done. Note: I still would not yet recommend placing apps in submodules. I think starting with shared/ librares would make the most sense initially.
Gareth Aye
Feb 13, 2014, 1:52:01 PM2/13/14
to Kevin Grandon, Greg Weng, Fabrice Desré, dev-gaia, Jan Jongboom
I went ahead and filed
https://bugzilla.mozilla.org/show_bug.cgi?id=972489for moving apps and
libs out of gaia proper via git submodules and
https://bugzilla.mozilla.org/show_bug.cgi?id=972481 for making python hg
build magic support git submodules.
https://bugzilla.mozilla.org/show_bug.cgi?id=972489for moving apps and
libs out of gaia proper via git submodules and
https://bugzilla.mozilla.org/show_bug.cgi?id=972481 for making python hg
build magic support git submodules.
Miller Medeiros
Feb 13, 2014, 2:11:45 PM2/13/14
to dev-...@lists.mozilla.org
**TL;DR;** git submodules have issues (I do not recommend them for this
case) and by splitting into individual modules we will need to write
"smarter" build/integration scripts. BUT I'm all in favor of
smaller/individual modules.
I'm still new to the whole process, but let's do some "brain dump"..
I would avoid git submodules, they "never" point to latest commits and
will be a PITA to do "git submodule update && git commit" all the time..
what if, instead of submodules, we only cared about the
"application.zip" + "manifest.webapp" files for the releases? -
Something like "https://gaia-apps.mozilla.com/v1.3/calendar.zip"
One thing that would be a drawback (if not planned properly), is how
changes to the "system" can propagate to other apps. If the system "app"
doesn't execute the tests for all the apps, it might introduce errors
that will be hard to catch... - imagine if the affected app doesn't have
any commits after change that introduced the error. So we would need to
be smart about how the tests are executed (run tests for all apps at
each system change).
For each individual app repository, I'm thinking of a shell script (npm
module) that downloads the latest "gaia" and executes ONLY the required
tests into that context. - If app is on branch v1.3, it should download
gaia@v1.3 as well.
There are many advantages about individual repos: long lived branches;
less "noise"; forces us to split and version "shared" libs; faster CI
cycle (no wasted resources running unrelated tests)...
A drawback for individual repos is that we wouldn't show up on the most
merged pull request on "octoverse" :P
Cheers,
Miller Medeiros
PS: gaia repo is HUGE, we would save a lot of time on CI by downloading
and executing less stuff. even with "--depth=1" it still downloads
250MB!!! on travis it downloads 580MB since it does "--depth=50".
PPS: we would need a simple way to reuse some of the make targets (like
the one that installs app/gaia into the phone), maybe more npm packages!
case) and by splitting into individual modules we will need to write
"smarter" build/integration scripts. BUT I'm all in favor of
smaller/individual modules.
I'm still new to the whole process, but let's do some "brain dump"..
I would avoid git submodules, they "never" point to latest commits and
will be a PITA to do "git submodule update && git commit" all the time..
what if, instead of submodules, we only cared about the
"application.zip" + "manifest.webapp" files for the releases? -
Something like "https://gaia-apps.mozilla.com/v1.3/calendar.zip"
One thing that would be a drawback (if not planned properly), is how
changes to the "system" can propagate to other apps. If the system "app"
doesn't execute the tests for all the apps, it might introduce errors
that will be hard to catch... - imagine if the affected app doesn't have
any commits after change that introduced the error. So we would need to
be smart about how the tests are executed (run tests for all apps at
each system change).
For each individual app repository, I'm thinking of a shell script (npm
module) that downloads the latest "gaia" and executes ONLY the required
tests into that context. - If app is on branch v1.3, it should download
gaia@v1.3 as well.
There are many advantages about individual repos: long lived branches;
less "noise"; forces us to split and version "shared" libs; faster CI
cycle (no wasted resources running unrelated tests)...
A drawback for individual repos is that we wouldn't show up on the most
merged pull request on "octoverse" :P
Cheers,
Miller Medeiros
PS: gaia repo is HUGE, we would save a lot of time on CI by downloading
and executing less stuff. even with "--depth=1" it still downloads
250MB!!! on travis it downloads 580MB since it does "--depth=50".
PPS: we would need a simple way to reuse some of the make targets (like
the one that installs app/gaia into the phone), maybe more npm packages!
Fabrice Desré
Feb 13, 2014, 5:56:30 PM2/13/14
to gar...@alumni.middlebury.edu, Greg Weng, dev-gaia, Jan Jongboom, Kevin Grandon
On 02/13/2014 09:04 AM, Gareth Aye wrote:
> Fabrice,
>
> We can use git submodules so that development that happens in app-only
> repositories can be upstreamed into gaia proper. The submodule updates
> can be bisected like any other commits. I would actually be willing to
> do this for calendar this week if no one has any strong objections...
We used git submodules initially in the main B2G repository and it was a
> Fabrice,
>
> We can use git submodules so that development that happens in app-only
> repositories can be upstreamed into gaia proper. The submodule updates
> can be bisected like any other commits. I would actually be willing to
> do this for calendar this week if no one has any strong objections...
PITA. using repo has been a clear win on this front. Any reason we don't
want to also use it for gaia?
Fabrice
Kevin Grandon
Feb 13, 2014, 6:02:34 PM2/13/14
to Fabrice Desré, Greg Weng, dev-gaia, Jan Jongboom, gar...@alumni.middlebury.edu
I posted the following in one of the bugs:
So I did some investigation of using repo with shared libraries in the past, and one thing that did not work was that we would want to be able to include different versions of the library within the same project. Some people said that this was possible, but I did not have luck when I went to implement it. If anyone can prove that it does work I would say repo would work just fine. E.g., we would want to be able to pull in different versions as follows:
gaia/foo/ExternalLib@v1.1/
gaia/bar/ExternalLib@v2.0/
If that does work then I'd say just go with repo because it's supported, and we can actually make progress on this front.
Best,
Kevin
----- Original Message -----
From: "Fabrice Desré" <fab...@mozilla.com>
To: gar...@alumni.middlebury.edu
Cc: "Kevin Grandon" <kgra...@mozilla.com>, "Greg Weng" <snow...@gmail.com>, "dev-gaia" <dev-...@lists.mozilla.org>, "Jan Jongboom" <janjo...@gmail.com>
Sent: Thursday, February 13, 2014 2:56:30 PM
Subject: Re: Demo for modularization Gaia repo
So I did some investigation of using repo with shared libraries in the past, and one thing that did not work was that we would want to be able to include different versions of the library within the same project. Some people said that this was possible, but I did not have luck when I went to implement it. If anyone can prove that it does work I would say repo would work just fine. E.g., we would want to be able to pull in different versions as follows:
gaia/foo/ExternalLib@v1.1/
gaia/bar/ExternalLib@v2.0/
If that does work then I'd say just go with repo because it's supported, and we can actually make progress on this front.
Best,
Kevin
----- Original Message -----
From: "Fabrice Desré" <fab...@mozilla.com>
To: gar...@alumni.middlebury.edu
Cc: "Kevin Grandon" <kgra...@mozilla.com>, "Greg Weng" <snow...@gmail.com>, "dev-gaia" <dev-...@lists.mozilla.org>, "Jan Jongboom" <janjo...@gmail.com>
Sent: Thursday, February 13, 2014 2:56:30 PM
Subject: Re: Demo for modularization Gaia repo
Fred Lin
Feb 14, 2014, 1:26:04 AM2/14/14
to Kevin Grandon, Greg Weng, Fabrice Desré, dev-gaia, Jan Jongboom, gar...@alumni.middlebury.edu
Replied in bug 883711.
I found its possible to fetch different branch with repo, see
https://github.com/gasolin/demo883711
I've a project called fxosbmi that have several branches. with the https://github.com/gasolin/demo883711/blob/master/default.xml file I can sync different branches to different folder.
the instructions are:
$ mkdir demo && cd demo
$ repo init -u http://github.com/gasolin/demo883711
$ repo sync
I already use repo to manage my gaia repository with https://github.com/gasolin/gaia-repo
So the issue is really goes to how we organize them properly and benefit our dev process, instead of introduce more issues.
regards
--
Fred
----- 原始郵件 -----
I found its possible to fetch different branch with repo, see
https://github.com/gasolin/demo883711
I've a project called fxosbmi that have several branches. with the https://github.com/gasolin/demo883711/blob/master/default.xml file I can sync different branches to different folder.
the instructions are:
$ mkdir demo && cd demo
$ repo init -u http://github.com/gasolin/demo883711
$ repo sync
I already use repo to manage my gaia repository with https://github.com/gasolin/gaia-repo
So the issue is really goes to how we organize them properly and benefit our dev process, instead of introduce more issues.
regards
--
Fred
----- 原始郵件 -----
Kevin Grandon
Feb 14, 2014, 2:14:32 AM2/14/14
to Fred Lin, Greg Weng, Fabrice Desré, dev-gaia, Jan Jongboom, gar...@alumni.middlebury.edu
Awesome! If no one has any serious objections I would strongly recommend that we start using repo for some shared libraries initially. We've been spinning our wheels on this for far too long here, so it would be nice to finally start working on it. Thanks for the investigation into this Fred.
Yuren Ju
Feb 14, 2014, 4:27:29 AM2/14/14
to Kevin Grandon, gar...@alumni.middlebury.edu, Fabrice Desré, Fred Lin, Greg Weng, dev-gaia, Jan Jongboom
I think we should use bower for extracting shared library and use repo for
split each app to different repositories since bower is used on a lot of
web app libraries.
split each app to different repositories since bower is used on a lot of
web app libraries.
Nicolas B. Pierron
Feb 14, 2014, 4:57:51 AM2/14/14
to mozilla-...@lists.mozilla.org
used to identify the source of issues. It always take the last of each
repository, which might not be consistent.
"git submodules" is awful in the sense that one need to update the top-level
repository each time they want to share their changes with other teams. It
is (was?) also awful for merging changes.
Currently, the only work-around to "git-repo" mess, is in
gecko/b2g/config/<device>/sources.xml . Which implies that if you want to
bisect gecko, then you can find out which version of Gaia which has been
tested with EACH device. (where is the unagi? oh, we are no longer testing
on it …)
--
Nicolas B. Pierron
Axel Hecht
Feb 14, 2014, 8:49:24 AM2/14/14
to mozilla-...@lists.mozilla.org
I haven't seen anyone getting the l10n side of things going?
Like everything else in life, repos are venn diagrams. What works for
one thing doesn't need to work for another thing.
I won't have cycles for this project, I need to work on actually
shipping software.
Axel
Like everything else in life, repos are venn diagrams. What works for
one thing doesn't need to work for another thing.
I won't have cycles for this project, I need to work on actually
shipping software.
Axel
Fabrice Desré
Feb 14, 2014, 9:08:48 AM2/14/14
to Axel Hecht, mozilla-...@lists.mozilla.org
On 02/14/2014 05:49 AM, Axel Hecht wrote:
> I haven't seen anyone getting the l10n side of things going?
>
> Like everything else in life, repos are venn diagrams. What works for
> one thing doesn't need to work for another thing.
>
> I won't have cycles for this project, I need to work on actually
> shipping software.
I'm not sure that being passive aggressive will help you or anyone here.
> I haven't seen anyone getting the l10n side of things going?
>
> Like everything else in life, repos are venn diagrams. What works for
> one thing doesn't need to work for another thing.
>
> I won't have cycles for this project, I need to work on actually
> shipping software.
Gaia had l10n support since day one (maybe two, ok) and I don't see that
being left in the ditch.
Fabrice
Gareth Aye
Feb 21, 2014, 5:48:50 PM2/21/14
to Kevin Grandon, Greg Weng, Fabrice Desré, dev-gaia, Jan Jongboom, Fred Lin
I want to revive this conversation because there are obvious testing
benefits to doing this and we need more testing benefits. Does anyone
object to prototyping managing shared libraries with repo in the hope of
moving toward decoupling core apps from gaia eventually? I don't completely
understand the l10n concern, but I also don't know what the l10n workflow
looks like?
On Fri, Feb 14, 2014 at 2:14 AM, Kevin Grandon <kgra...@mozilla.com> wrote:
> Awesome! If no one has any serious objections I would strongly recommend
> that we start using repo for some shared libraries initially. We've been
> spinning our wheels on this for far too long here, so it would be nice to
> finally start working on it. Thanks for the investigation into this Fred.
>
benefits to doing this and we need more testing benefits. Does anyone
object to prototyping managing shared libraries with repo in the hope of
moving toward decoupling core apps from gaia eventually? I don't completely
understand the l10n concern, but I also don't know what the l10n workflow
looks like?
On Fri, Feb 14, 2014 at 2:14 AM, Kevin Grandon <kgra...@mozilla.com> wrote:
> Awesome! If no one has any serious objections I would strongly recommend
> that we start using repo for some shared libraries initially. We've been
> spinning our wheels on this for far too long here, so it would be nice to
> finally start working on it. Thanks for the investigation into this Fred.
>
> Best,
> Kevin
>
> ----- Original Message -----
> Kevin
>
> ----- Original Message -----
> Cc: "Fabrice Desré" <fab...@mozilla.com>, "Greg Weng" <
> snow...@gmail.com>, "dev-gaia" <dev-...@lists.mozilla.org>, "Jan
> Jongboom" <janjo...@gmail.com>, gar...@alumni.middlebury.edu
> Sent: Thursday, February 13, 2014 10:26:04 PM
> Subject: Re: Demo for modularization Gaia repo
>
> snow...@gmail.com>, "dev-gaia" <dev-...@lists.mozilla.org>, "Jan
> Jongboom" <janjo...@gmail.com>, gar...@alumni.middlebury.edu
> Sent: Thursday, February 13, 2014 10:26:04 PM
> Subject: Re: Demo for modularization Gaia repo
>
> Best,
> Kevin
>
> ----- Original Message -----
> Kevin
>
> ----- Original Message -----
> Cc: "Kevin Grandon" <kgra...@mozilla.com>, "Greg Weng" <
> snow...@gmail.com>, "dev-gaia" <dev-...@lists.mozilla.org>, "Jan
> Jongboom" <janjo...@gmail.com>
> Sent: Thursday, February 13, 2014 2:56:30 PM
> Subject: Re: Demo for modularization Gaia repo
>
> snow...@gmail.com>, "dev-gaia" <dev-...@lists.mozilla.org>, "Jan
> Jongboom" <janjo...@gmail.com>
> Sent: Thursday, February 13, 2014 2:56:30 PM
> Subject: Re: Demo for modularization Gaia repo
>
> On 02/13/2014 09:04 AM, Gareth Aye wrote:
> > Fabrice,
> >
> > We can use git submodules so that development that happens in app-only
> > repositories can be upstreamed into gaia proper. The submodule updates
> > can be bisected like any other commits. I would actually be willing to
> > do this for calendar this week if no one has any strong objections...
>
> >
> > We can use git submodules so that development that happens in app-only
> > repositories can be upstreamed into gaia proper. The submodule updates
> > can be bisected like any other commits. I would actually be willing to
> > do this for calendar this week if no one has any strong objections...
>
> We used git submodules initially in the main B2G repository and it was a
> PITA. using repo has been a clear win on this front. Any reason we don't
> want to also use it for gaia?
>
> PITA. using repo has been a clear win on this front. Any reason we don't
> want to also use it for gaia?
>
> Fabrice
> --
> Fabrice Desré
> b2g team
> Mozilla Corporation
> --
> Fabrice Desré
> b2g team
> Mozilla Corporation
Kevin Grandon
Feb 21, 2014, 10:35:51 PM2/21/14
to gar...@alumni.middlebury.edu, Greg Weng, Fabrice Desré, dev-gaia, Jan Jongboom, Fred Lin
Can you elaborate on the testing benefits? I can think of benefits for shared code/, but when it comes to apps it seems that testing just gets much more complex. How do we then test things like the system app and activities? If we split an app out, do we then need to pull in the the system app, and other apps as we'll to have an integration test check activities? Or do we rely on tons of mock handlers and only do some testing for this at the system level?
I think we can certainly come up with clever solutions to splitting out apps. I do think it would be a lot of work, with a lot of unknown implications and risk. We don't clearly know what the l10n workflow looks like, and yet we still push for this without getting clarification. There are probably many other concerns along these lines.
I definitely support us investigating this and starting small with something like building blocks. There is zero existing localization/integration testing concerns there, so it should be an easy one.
Best,
Kevin
I think we can certainly come up with clever solutions to splitting out apps. I do think it would be a lot of work, with a lot of unknown implications and risk. We don't clearly know what the l10n workflow looks like, and yet we still push for this without getting clarification. There are probably many other concerns along these lines.
I definitely support us investigating this and starting small with something like building blocks. There is zero existing localization/integration testing concerns there, so it should be an easy one.
Best,
Kevin
Gareth Aye
Feb 22, 2014, 9:51:20 AM2/22/14
to Kevin Grandon, Greg Weng, Fabrice Desré, dev-gaia, Jan Jongboom, Fred Lin
> Can you elaborate on the testing benefits? I can think of benefits for
> shared code/, but when it comes to apps it seems that testing just gets
> much more complex.
These are all very good questions. As for the testing benefits, I think
> shared code/, but when it comes to apps it seems that testing just gets
> much more complex.
there are a few. Off the top of my head...
1. When one app regresses, most others can stay open. For instance, in our
most recent episode, we could have closed the email and system app trees
while leaving the other apps open.
2. Like :snowmantw already mentioned in his slides, when we decouple apps
from the core, we speed up builds and tests since (for example) changes to
the dialer app won't (by default) build and test clock.
How do we then test things like the system app and activities? If we split
> an app out, do we then need to pull in the the system app, and other apps
> as we'll to have an integration test check activities? Or do we rely on
> tons of mock handlers and only do some testing for this at the system level?
apps they expect to exchange activities calls with. The system app is
tightly coupled with the keyboard and the comms apps, so I would expect (at
the very least) for it to pull those in for tests.
I think we can certainly come up with clever solutions to splitting out
> apps. I do think it would be a lot of work, with a lot of unknown
> implications and risk.
and architecting ways that we can wire things together to minimize the
risks.
> We don't clearly know what the l10n workflow looks like, and yet we still
> push for this without getting clarification. There are probably many other
> concerns along these lines.
>
https://bugzilla.mozilla.org/show_bug.cgi?id=975713 and needinfo'd Axel to
clarify the l10n concerns. I would love for any other groups that are
downstream from changes like this (like releng) to weigh in.
> I definitely support us investigating this and starting small with
> something like building blocks. There is zero existing
> localization/integration testing concerns there, so it should be an easy
> one.
https://bugzilla.mozilla.org/show_bug.cgi?id=975714
Axel Hecht
Feb 22, 2014, 2:51:03 PM2/22/14
to mozilla-...@lists.mozilla.org
I would point to sicking's post about closing trees. Closing trees is
unknown bustage, everything else is a simple back-out.
The idea that unknown bustage would affect only one app doesn't seem likely.
This reminds me of our grand modular ideas back in the 90s, when XPCOM
was all the rage. And now we spent the past 10 years ripping all that
modularity back out, as it only costs in a system that is only expected
to work in one way, and no other.
As far as l10n goes, we'll not localize 26 different things. 25 apps and
shared. There's just no reason to assume that several dozens of teams
working on several dozen apps has any chance of success.
Fx OS will need to provide a single localization target.
Axel
unknown bustage, everything else is a simple back-out.
The idea that unknown bustage would affect only one app doesn't seem likely.
This reminds me of our grand modular ideas back in the 90s, when XPCOM
was all the rage. And now we spent the past 10 years ripping all that
modularity back out, as it only costs in a system that is only expected
to work in one way, and no other.
As far as l10n goes, we'll not localize 26 different things. 25 apps and
shared. There's just no reason to assume that several dozens of teams
working on several dozen apps has any chance of success.
Fx OS will need to provide a single localization target.
Axel
Gareth Aye
Feb 22, 2014, 8:26:13 PM2/22/14
to Axel Hecht, mozilla-...@lists.mozilla.org
> I would point to sicking's post about closing trees. Closing trees is
> unknown bustage, everything else is a simple back-out.
>
Yes I agree. Who are you arguing with?
> unknown bustage, everything else is a simple back-out.
>
> The idea that unknown bustage would affect only one app doesn't seem
> likely.
>
likelihood? Your claim is unfalsifiable, so I can't argue with you. In the
future, I encourage you to make your claims in a way that facilitates
healthy discussion.
> This reminds me of our grand modular ideas back in the 90s, when XPCOM was
> all the rage. And now we spent the past 10 years ripping all that
> modularity back out, as it only costs in a system that is only expected to
> work in one way, and no other.
>
better because components can be exchanged, reused, and modified without
affecting entire systems. Modular designs (as opposed to monolithic ones)
are actually better suited for environments where project requirements are
subject to change for this reason.
> As far as l10n goes, we'll not localize 26 different things. 25 apps and
> shared.
decoupling core apps from the gaia repository creates more work for the
l10n team. It doesn't matter if the calendar app lives in mozilla-b2g/gaia
or mozilla-b2g/gaia-calendar. The same strings must still be translated to
our target languages.
> There's just no reason to assume that several dozens of teams working on
> several dozen apps has any chance of success.
>
as Greg's) are twofold. If we move core apps out of gaia, gaia developers
will be more productive and gaia will be more maintainable. My supporting
points are:
1. app developers will not be slowed down by tests which are irrelevant to
their changesets
2. the build system will be more maintainable if all of the apps aren't
generated by a single Makefile
3. tree-closing regressions that break only a proper subset of all core
apps will not stall apps which have not been regressed
> Fx OS will need to provide a single localization target.
>
from decoupling core apps from mozilla-b2g/gaia, but if you want to be
taken seriously, you must make falsifiable claims for which you provide
evidence.
![Francesco Lodolo [:flod]'s profile photo](http://lh3.googleusercontent.com/a-/ALV-UjXs0woEjV8IBRZaZsGBgw87TNzFgo_G2aPycSH6tCfbedeoye4=s40-c)
Francesco Lodolo [:flod]
Feb 23, 2014, 2:21:39 AM2/23/14
to dev-...@lists.mozilla.org
Il 23/02/14 02:26, Gareth Aye ha scritto:
string-only Hg repository per branch, they have no idea what's behind,
or how it's created. I've seen the process once, and that's far from
simple. [1]
How do you plan localizers, and tools to work on strings scattered
across 30 something Git repositories? Most developers will work on one
or two apps, localization teams need to work on all of them.
Francesco
[1] From this very same thread
https://groups.google.com/d/msg/mozilla.dev.gaia/nq7FgDbS4QM/Kerw4pu0LlMJ
> Aren't you already localizing 26 different things?
No. Localizers (and external tools) are exposed to one single plain
string-only Hg repository per branch, they have no idea what's behind,
or how it's created. I've seen the process once, and that's far from
simple. [1]
How do you plan localizers, and tools to work on strings scattered
across 30 something Git repositories? Most developers will work on one
or two apps, localization teams need to work on all of them.
Francesco
[1] From this very same thread
https://groups.google.com/d/msg/mozilla.dev.gaia/nq7FgDbS4QM/Kerw4pu0LlMJ
Gareth Aye
Feb 23, 2014, 8:15:02 AM2/23/14
to Francesco Lodolo [:flod], dev-...@lists.mozilla.org
On Sun, Feb 23, 2014 at 2:21 AM, Francesco Lodolo [:flod]
<fl...@lodolo.net>wrote:
> Il 23/02/14 02:26, Gareth Aye ha scritto:
>
> Aren't you already localizing 26 different things?
>>
> No. Localizers (and external tools) are exposed to one single plain
> string-only Hg repository per branch, they have no idea what's behind, or
> how it's created. I've seen the process once, and that's far from simple.
> [1]
>
Okay I understand and thank you for clarifying. That process should not be
affected by the changes proposed here.
> How do you plan localizers, and tools to work on strings scattered across
> 30 something Git repositories? Most developers will work on one or two
> apps, localization teams need to work on all of them.
>
We can maintain the existing localizers' workflows with the existing
tooling. This change won't be visible to the localizers.
<fl...@lodolo.net>wrote:
> Il 23/02/14 02:26, Gareth Aye ha scritto:
>
> Aren't you already localizing 26 different things?
>>
> No. Localizers (and external tools) are exposed to one single plain
> string-only Hg repository per branch, they have no idea what's behind, or
> how it's created. I've seen the process once, and that's far from simple.
> [1]
>
affected by the changes proposed here.
> How do you plan localizers, and tools to work on strings scattered across
> 30 something Git repositories? Most developers will work on one or two
> apps, localization teams need to work on all of them.
>
tooling. This change won't be visible to the localizers.
Fred Lin
Feb 23, 2014, 9:37:16 PM2/23/14
to gar...@alumni.middlebury.edu, Yuren Ju, Fabrice Desré, Greg Weng, dev-gaia, Jan Jongboom, Kevin Grandon
I support use repo in some place but not for every libraries inclusion by each app.
As I know @yuren is starting to refactor the build script, the modularization task would be more clear after the refactor.
== maybe bower? ==
For the case if we tend to manage libraries by app instead of by shared/ (which is still in plan stage). We could do it more `web` way, like bower (npm for client side libraries).
The good thing is we've the experimental bower support for building-blocks https://github.com/buildingfirefoxos/Building-Blocks
It make include building-blocks for developer as easy as npm install:
$ bower install building-blocks
As :snowmantw's experimental, we could do that for most shared/ js libraries (ex: bower install gaia-templatejs) as well.
== repo support issue ==
For repo usage we've to check if repo can support inherit repo first (does top level repo can handle sub-folder's repo?), or with this approach each B2G manifest list would need to expose many gaia-related gits.
If we include libraries by repo but it does not support inheritance, we have to ask every vendor to update their gaia related git in their B2G manifest, which is error prone.
We could get benefits from repo, but may not for every libs in each app.
regards
--
Fred
----- 原始郵件 -----
寄件人: "Gareth Aye" <gar...@alumni.middlebury.edu>
收件人: "Kevin Grandon" <kgra...@mozilla.com>
副本: "Fred Lin" <fl...@mozilla.com>, "Fabrice Desré" <fab...@mozilla.com>, "Greg Weng" <snow...@gmail.com>, "dev-gaia" <dev-...@lists.mozilla.org>, "Jan Jongboom" <janjo...@gmail.com>
寄件箱: 2014年2 月22日, 星期六 上午 6:48:50
標題: Re: Demo for modularization Gaia repo
As I know @yuren is starting to refactor the build script, the modularization task would be more clear after the refactor.
== maybe bower? ==
For the case if we tend to manage libraries by app instead of by shared/ (which is still in plan stage). We could do it more `web` way, like bower (npm for client side libraries).
The good thing is we've the experimental bower support for building-blocks https://github.com/buildingfirefoxos/Building-Blocks
It make include building-blocks for developer as easy as npm install:
$ bower install building-blocks
As :snowmantw's experimental, we could do that for most shared/ js libraries (ex: bower install gaia-templatejs) as well.
== repo support issue ==
For repo usage we've to check if repo can support inherit repo first (does top level repo can handle sub-folder's repo?), or with this approach each B2G manifest list would need to expose many gaia-related gits.
If we include libraries by repo but it does not support inheritance, we have to ask every vendor to update their gaia related git in their B2G manifest, which is error prone.
We could get benefits from repo, but may not for every libs in each app.
regards
--
Fred
----- 原始郵件 -----
收件人: "Kevin Grandon" <kgra...@mozilla.com>
副本: "Fred Lin" <fl...@mozilla.com>, "Fabrice Desré" <fab...@mozilla.com>, "Greg Weng" <snow...@gmail.com>, "dev-gaia" <dev-...@lists.mozilla.org>, "Jan Jongboom" <janjo...@gmail.com>
寄件箱: 2014年2 月22日, 星期六 上午 6:48:50
標題: Re: Demo for modularization Gaia repo
Axel Hecht
Feb 24, 2014, 7:54:46 AM2/24/14
to mozilla-...@lists.mozilla.org
github.com/mozilla-b2g/gaia branches get converted to hg repositories at
https://hg.mozilla.org/integration/, a repository per supported branch.
I take a fork of hg convert to port the DAGs per hg repository into an
en-US only repository per branch. [*]
Those get translated in various locale-dependent tools, and eventually
commited back to https://hg.mozilla.org/gaia-l10n/
Releng automation then takes those repositories (and the
releases/l10n/mozilla-* ones) and converts those back into git
repositories, on http://git.mozilla.org/ (not github). Example l10n
repos would be
http://git.mozilla.org/?p=releases/l10n/de/gaia.git;a=summary and
http://git.mozilla.org/?p=releases/l10n/de/gecko.git;a=summary
[*] A fork of hg convert, as I need to rewrite apps/foo/webapp.manifest
into apps/foo/manifest.properties by rewriting the content of the file.
Also, the l10n repo generation is fully scripted, which ensures that all
changesets are exposed, with proper version control. It's not automated
as the strings in gaia are too often not ready for product, and thus
ready for localization. So every so often, I need to hold back on
exposing the strings to localization until they're ready for prime time.
I don't know of any tool in this chain that would support merging of
unrelated repository histories into one. Which is what we would need,
unless we redo the workflows. From my experience with forking `hg
convert`, I wouldn't have a starting point for such a tool, and I expect
the patch algebra and graph theory to be "advanced", at least.
That is what I expect you to find or create. If this relates to the
needs that releng has or not, you'd probably need to discuss with Hal.
Axel
0 new messages