Pushing back the release?

[Jono]

Hey everybody,
Software is hard.
I feel bad to even suggest this after how hard all the Ubiquity
contributors have been working towards this release over the past few
weeks, but I am thinking about pushing the 0.5 release back a couple
more weeks, until I have returned from my trip. I just got done
talking to Atul and he strongly suggested doing so, for the following
reasons:

1. The server-side components (bug reporter, command search engine,
etc) are not anywhere near ready. I've been working on them all day
and discovered that the job is much, much bigger than I thought. I
thought it was going to be a one-day thing but it looks more like we
will have to seriously rethink the server infrastructure if we want to
offer bug reporting and command search services to our users.

2. Support: After we release 0.5 onto addons.mozilla.com, since it is
such a big change to how ubiquity works, there is bound to be a huge
influx of users with support requests, maybe more serious bugs
discovered, etc. This would be happening while I was gone and not
able to help with any of the support work. Leaving the users hanging
with new problems due to 0.5 might be a worse thing to do to them than
making them wait another couple of weeks to get 0.5.

3. Features: If we push it back, we will have time to try to put in
some of the features we cut out: Suggestion memory, provider-plugin
arguments, and most of all, asynchronous noun-first suggestion (in a
way that doesn't count as a DDOS attack on Yelp).

So, if we push it back, then we'll be able to do a more polished
release, with the server-side components working, and at a time when
I'll be available to help with the support and bug fixing and the
inevitable patches that will be needed.

This isn't just my decision, though. What does everybody else think
of pushing the release back?
--Jono

[Jono]

P.S.
I think we should keep releasing 0.5pre versions periodically during
the next three weeks. Aza will take care of organizing that effort.
--Jono

[Brandon Pung]

I agree with the decision to push back the release until Jono's return from his trip, for the same reasons Jono has described. I was listening in on the conversation with Atul and it seems like the logical choice. Jono being gone for the whole support phase following the release would be pretty bad, and coupled with the server-side problems I think we need to push it back. On a side note, with this extra time I'm excited about the possibility of getting noun first suggestions into 0.5 in a way that's more reflective of what we were advertising with the 0.5pre blog posts and so forth, though modified to avoid DDOS attacks of course =)

-Brandon

--
Brandon James Pung
Massachusetts Institute of Technology Class of 2010
Department of Electrical Engineering and Computer Science
828.777.8640 | bp...@mit.edu

[Jono]

How about we release Tuesday? Mitcho points out the following reasons
we should release after all:

1. if we push it back we will be letting down all the people who have
already blogged/written/read about the 0.5 release
2. the XPI itself is ready
3. The server components are not ready, but 0.5 without bug reporter /
herd is at least not any worse than 0.1.8 without bug reporter/herd
4. The rest of the community can handle support without me ;-)
5. Getting people onto 0.5 will reduce the load on the ubiquity.mozilla.com.

We should replace the herd and bug reporter pages with notifications
that tell users those services are down for repair, and then focus on
getting them working after the 0.5 release is out.

Atul and Aza have agreed to manage the release while I'm gone.
--Jono

[Jono]

P.S. remember that if upgrading to 0.5 breaks a user's favorite
third-party command, they can always go back to parser 1 using the
Settings page.
(Anybody doing support should keep this in mind as it will be a common
complaint.)
--Jono

[Blair McBride]

I think Personas has a dual release schedule - one for the client, one
for the server-side stuff. It seems to work pretty well, and I don't see
why that method couldn't be applied to Ubiquity. To me, client
(Ubiquity) and server (Herd) are two separate products.

I don't think the release should be pushed back for new features. For
serious bugs yes, but there's nothing major blocking that I know of.
There's no such thing as a perfect release - there will always be
something that could have been fixed, could have been better.

So I'm for releasing now. Or Tuesday if really necessary. But not any later.

- Blair

["mitcho (Michael 芳貴 Erlewine)"]

As Jono already mentioned a number of the reasons I gave to strongly
push for a release today, it suffices to say that I agree whole-
heartedly with Blair. Amen.

I'm fine with a Tuesday release, but we should make it a firm
commitment and we should make it crystal clear on trac and/or the list
(ideally both) *EXACTLY* what the blockers are that we are up against
(if any).

m

--
mitcho (Michael 芳貴 Erlewine)
mit...@mitcho.com
http://mitcho.com/
linguist, coder, teacher

[Jono]

Hey everybody,
I apologize for my panicky email on Thursday afternoon; I was very
frustrated at my inability to get the server-side components working,
and I got pretty emotional (as those who have to share an office with
me can attest). Anyway, after hearing from everyone about it, I think
we are good to launch early Tuesday morning.

Today I pushed a 0.5pre4 (up on the site now). It includes changes to
the web content and the Report Bug command, so that users who are
looking to report bugs are directed to Get Satisfaction, and users
looking for more commands are directed to the Commands In The Wild
wiki page. I think both of these will be more helpful than directing
them to non-functional Herd and Bug Reporter services.

I think 0.5pre4 is ready to be renamed 0.5.0 and released, but it
should be stress-tested by everyone involved to make sure it can
upgrade cleanly, that it has decent performance and generates
reasonable suggestions, and that the tutorial works.

The Herd and the report-bug service will likely require a complete
rewrite of the backend infrastructure before they are functional.
They both depend on a single couchdb install which currently stands at
6 GB, and which cannot run for more than a few minutes without
crashing. I was unable to get it to stay up for any length of time,
even after spending most of yesterday on it. So I think we'll need to
start some new server infrastructure from scratch in order to support
Herd and the bug reporter the way we want to. That should be one of
the main priorities after we release 0.5.0.

As I said, I think we're good to release Tuesday morning. I agree
with Christian about pushing to get translations done for that.
Please, no new features or functional changes: just translations,
testing, and bug fixes.

Aza has agreed to take responsibility while I'm gone, to write the
release blog post for 0.5 and to manage the release of any dot-patches
that are afterwards found to be neccessary (0.5.1 etc).

I am leaving on my trip tomorrow morning, so I can do no more on this
release. The rest of you will have to decide the exact timing of the
0.5.0 release, and manage its execution.

Good luck!
--Jono

["mitcho (Michael 芳貴 Erlewine)"]

Jono, this sounds great. The server-side is definitely a priority
worth seriously working on in the near future. Looking forward to
seeing you on Monday! :D

I will email ubiquity-i18n about the localization time schedule. I
personally think the more public outreach to the broader localization
community (through dev-l10n), especially given the limited time until
next Tuesday, is better suited to right after the 0.5 release.

mitcho

[Blair McBride]

Just gotta remind ourselves that we're not designing the Therac-25 here
;) 0.5 is going to be (is?) the most awesome Ubiquity release to date.


- Blair

PS, 6Gb is kinda crazy. I mean, wtf?

[Zach]

What about folding the Ubiquity commands into the pre-existing add-ons
site? Can we bring SUMO on board to get the documentation up to
snuff? That would get us back on track much more quickly- and it's
what is going to happen sometime down the line anyway.

Wasn't there a plan to get someone working on the HERD full time for
the summer- an intern or something...

-Zak-the-guy-who-promises-to-do-more-work-instead-of-complaining-:-)



On Jul 3, 8:38 pm, "mitcho (Michael 芳貴 Erlewine)" <mit...@mitcho.com>

[Fernando Takai]

I think Tuesday is a good day ;D

Btw, i emailed the couchdb user-list, but i really need the logs from
the server (so they can say what's going on).
Also, i prototyped another bug-report app [1], using MongoDB [2] - if
we can't make couchdb work, i think this is a good replacement.

[1] http://bitbucket.org/fernandotakai/ubiquity_bugreport/
[2] http://www.mongodb.org

--
Fernando Takai
http://twitter.com/fernando.takai

[Brandon Pung]

Hi everybody,

Aza, Atul, and I just had a lengthy conversation about the release plans for tomorrow, and we have decided to somewhat tweak the way we are doing this release. Ubiquity 0.5 will still be released tomorrow, but it will be released as a separate entity from 0.1.8, and will be called Ubiquity.Next or something along those lines. It will be on addons.mozilla.org under this separate name, so both 0.1.8 and 0.5 will be available there. The two versions of Ubiquity will be kept separate until 0.5 is in a state that we are ready to push on to all of our ~400,000 users. The requirements for this merge will be something like:

1) Have the top x percent of 3rd party commands converted to Parser 2 format

2) Fix the network call problem in a way that allows us to re-enable async network calls for noun first suggestions (through server-side caching or other means)

3) Resolve bugs by design that will surely be identified as we get a bunch of new users on Ubiquity.Next

The reason we do not want to force everyone to upgrade to 0.5 at this point is because the benefits of 0.5 in it's current state don't necessarily outweigh the negative changes for many of our users. With the async suggestions basically disabled for noun first suggestions, the biggest positive changes made between 0.1.8 and 0.5 are the more sophisticated parser, resulting in the removal of hyphens, and internationalization. While this stuff is very cool, and moving in the right direction, 0.5 breaks all 3rd party commands and also breaks some user habits (like the way delimiters force you to type "weather near boston", whereas in 0.1.8 you can just type "weather boston"). Breaking user habits is ok if the positive gains outweigh the necessary changes in behavior. We think that 0.1.8 and 0.5 should be kept separate until it is clear that 0.5 is a valuable upgrade to nearly all of our users, which will be true when the requirements listed above are fulfilled. While this is a labs project, we also have hundreds of thousands of users who depend on Ubiquity to make their computing lives easier, so we need to be cautious about pushing major changes on everybody.

Aza will be creating a screencast further describing the reason for maintaining the two separate extensions soon. How does this sound to everyone? Any questions or concerns?

-Brandon

[Heather]

I really like this idea and think it will make the end user experience
much smoother for english language users who are mostly interested in
Ubiquity as a productivity enhancer. I also have a few concerns.

1. I think it is important that we figure out a way to differentiate
0.1.8 from Ubiquity.Next for purposes of support/command feeds/
documentation. I think the documentation needs some major cleanup -
some of which I am willing to do tonight if we can come up with a good
way to explain the differences. Anyone have any suggestions about the
best way to do this? This means maintaining two sets of user
documentation and two sets of command developer documentation.

2. In terms of command developers, is there any easy way we can make
Ubiquity handle command feeds for both versions? We have some
developers that have done great work getting their commands ready for
Parser2, and we're making their lives harder temporarily by making
them maintain two versions of their commands. It would be nice if
Ubiquity could handle subscribing to one feed for 0.1 and one feed for
0.5 without having to make links to both.

3. Another limitation of this approach is it weakens "ubiquity the
platform" a little in terms of being easy to use and understand, since
the Parser1 documentation is not so good right now (the auto generated
docs, including the command API, are generated from the source tip),
and as a community we'll need to expect some people developing for
Parser1 and some for Parser2. I think most of this concern is
remedied by urging people interested in command development to use 0.5
and develop for Parser2.

4. It's a little late to do this this time around, but I think in
general the core development team should think about maintaining a
release trunk and a beta branch (haha, i'm using developer words,
probably incorrectly). Right now maintaining 0.1.8 is actually fairly
difficult if I understand correctly. I think 0.1.8 is fairly stable,
but it has a few bugs that would be worth fixing if we're going to
keep it out there. I think it's worth having a conversation about how
we're going to handle ubiquity the product vs. ubiquity the experiment
in the future, but I think this idea is a step in the right direction.

5. I think this gives us a great opportunity to think about how to
best engineer the herd to smooth over the upgrade issues of when 0.5
is pushed to be the "used by everybody" version. If the herd is
parser1/parser2 aware, then we can use it to measure how many 3rd
party commands have been ported to parser2 and when people make the
switch, they can use the herd to actually find commands, which would
be pretty sweet! Fern has done some awesome work on the Herd, but I
think a complete rewrite will require some developer resources that
Ubiquity just doesn't have right now.

I think that's about it.

Heather

> Department of Electrical Engineering and Computer ...
>
> read more »