Post-mortem: Duo

[Michael Johnston]

Duo didn't make it into the top 100 (let alone the top 50), which has
been very hard for me (I hate losing!) but since people here expressed
a lot of enthusiam for our project, I thought you might enjoy reading
about its development in more detail. You can watch a short video of
Duo here if you haven't already:
http://www.youtube.com/watch?v=fwSBLdGCjnY

Although we lost the contest, Duo is alive and kicking. For now we
are shifting our focus to other platforms (for practical reasons, not
out of spite). If anyone is interested in investing in Duo or
contributing to it, or if you know someone who might be, please
contact me here:
michael AT codality DOT com

So this is the story of Duo and its development. If you're more
interested in technical details, you might want to skip to the next
post where I talk about "what went right" and "what went wrong".

Awhile back I quit my job as a game developer to work on my own
project. I used my savings to support myself while I developed a new
game development tool. Over the course of several months I made good
progress. In November of 2007 I heard about Android and Google's
development competition. I thought: hey, here's a good short-term
opportunity. But my project at the time wasn't a good fit for Android
and I didn't have any good ideas for new mobile applications, so I
didn't look into Android any further.

Then, one evening in late 2007, I had an enlightening conversation
with an industrial designer friend of mine. She had recently
interviewed for a position with Microsoft's mobile division and she
told me about their grueling interview process. They were hiring for
people to help develop innovative new mobile applications. She told
me that one of their interview exercises was to design a solution for
telling stories using your mobile phone. This was a deliberately open-
ended exercise. The solution could be anything you imagined. I love
exercises like this, so it planted a seed in my mind.

That seed germinated in early January 2008. One night I thought to
myself: what would my solution to the storytelling exercise be? I
thought it would be cool to use your phone to record information as
you traveled around, and then to share that information with your
friends. This could include text, tags, photos, audio, video, and
ratings. I thought that if there was a way to combine all of these
forms of information using a cohesive and elegant interface, you'd
have a very powerful tool for recording and sharing a huge variety of
location-based information, including not just your personal stories,
but also things like restaurant ratings. I called these potential
blobs of information "footprints". The idea would be to create
software that lets you leave footprints in your wake for your friends
and perhaps the public to experience.

In hindsight I realize that this wasn't a particularly innovative
idea, but at the time I was fairly naive about the state of mobile
application development, so it seemed new and exciting to me. I kept
thinking about it and, after sharing the idea with some trusted
friends, we started coming up with a bunch of other (fairly obvious)
ideas: automatic friend finding, connecting footprints together to
create paths for things like guided tours, creating "future
footprints" for coordinating events, various corresponding business
models, etc.

In early January I decided to download Android and start tinkering to
see what we might be able to develop in the time remaining before the
contest deadline. The results were promising. I hadn't coded in Java
in years (my background is C/C++), but I was able to get up and
running with Android very quickly. I found the documentation to be
very good for an early release of a new SDK. Where documentation was
lacking, decompliation could be used to look under the hood, revealing
a solid design with code that was easy to follow (except the
obfuscated mapping classes...argh!)

Satisfied with Android as a development platform, I knew I wouldn't be
able to do this alone, so the next task was to find a team. A very
good designer friend of mine was also excited by the idea and wanted
to contribute part-time. His involvement would prove invaluable,
particularly during the final stage of UI iteration. He also had a
good friend who was a server development wiz. We approached him with
our idea and he wanted to contribute part-time to develop all the
backend tech, which was great.

Within a few days I developed a very simple prototype comprised of a
map and a list of fake footprints. Our server wiz created a simple
backend and we connected the two successfully. This was extremely
encouraging. So much progress in such a short time!

The three of us were also actively posting new ideas in a private
forum. As a result of our early prototyping and brainstorming, I
experienced one of those "aha" moments where you end up pacing around
the room excitedly, thinking non-stop about all the implications of an
idea. The idea in question emerged in the following way. Our
prototype was built with GPS in mind. You'd create footprints at your
GPS location. This worked well for creating footprints, but what
about browsing them? What about finding footprints in places you
visited previously, or in places you hoped to visit in the future?

What if we introduced a lightweight, game-like interface that allowed
you to easily travel "out-of-body" to explore your city and the entire
world? I wrote a lengthy post about this and the other guys were
excited by it too. They extended it with lots of ideas of their own.
We knew it would be challenging to combine a fun interface together
with locational information, but we felt we were onto something, and
there was huge potential. The business models with this approach
would be much more varied and potentially lucrative. Thus Duo was
born. Many more ideas kept flowing after that.

Meanwhile, I had been in the process of applying for a government
grant for my game development tool. After a great deal of inner
turmoil, I decided to change this grant application to be for Duo
instead. While awaiting the outcome of the grant application, I
continued developing the client prototype, spending a lot of time
establishing a solid framework for networking and storage, and testing
different UI ideas. I also converted everything to the new SDK
release.

At the end of February our grant application was successful, which
meant the government would cover half the cost of hiring two part-time
Canadian contractors to help us (I paid for the other half). A friend
of mine from university was interested in helping us with programming,
and I advertised for an artist to help us too. As of Feb 27th we
entered proper development based on a loose but ambitious schedule.
Our team was comprised of:
- Me, full-time client programmer
- Fred, part-time server programmer
- Eric C, part-time designer
- Eric T, part-time client programmer
- Jasper, artist

Everything progressed very smoothly throughout March. The framework I
had built in the previous month allowed me to fairly quickly connect
to the backend as Fred found time in his busy schedule to implement
it. I developed the tech we'd need for an avatar and buildings while
Jasper worked with Eric C to start cranking out art. Eric T made
solid progress each week with the interior tile renderer.

By April 1st, we had an app that could do all kinds of stuff, but the
interface was still very rough around the edges. Other major problems
included: you couldn't properly login yet, you couldn't delete what
you created, dynamic updates weren't really working so it was
impossible to see friends creating footprints, etc. The amount of
work we did in the final two weeks to bring it all together was
remarkable and I'm really proud of our team.

We spent all our time developing, and barely any time on documentation
or "marketing" because we believed that this competition would be
about results. We thought the best way for judges to understand Duo
would be to try it, and we thought the judging process would involve
several people using it for at least several minutes each.

Unfortunately, according to our logs, only a few judges tried Duo, and
their experiences were brief. Maybe the client crashed and they
couldn't be bothered to login again. Maybe they didn't know you can
leave your house and go outside. Maybe they thought the underlying
premise wasn't promising enough. Maybe they didn't read our
document. Maybe they did read our document but they didn't like it.
We'll probably never know, but in retrospect I wish I had spent time
creating a demonstration video to submit as part of our document
(rather than creating it after-the-fact). Perhaps that would've
helped us score better.

That's the story of Duo in a nutshell. Next I'll go into more
technical details about what went right and what went wrong during
development.

[Michael Johnston]

WHAT WENT RIGHT
We built on top of some existing technologies to try and get a leg up
on the competition. Our tile renderer uses the well known Mappy
library, our chat system is built on top of IRC, and our backend is
built on Amazon's scalable EC2 service. This strategy was very
successful. We wouldn't have been able to implement what we did in
such a short period of time if we hadn't done this.

Early on we made the decision to use a web service model based on JSON
and a RESTful interface. Our server wiz got this up and running
quickly and we were able to extend it with ease as our needs grew.
Accessing the server's JSON data from within Android was also quite
easy. All the fetched data gets cached in a MySQL ContentProvider,
which was a bit cumbersome to setup but is easy to use thereafter.
Parsing JSON objects in Android is a snap.

I think we must've gone through about a dozen UI iterations. That's
quite a lot for a 3 month project, but I think it ultimately resulted
in an interface that is consistent and elegant. We went through two
phases of heavy UI iteration, one at the beginning, and one at the
end. Experience gained during the first phase allowed me to be very
productive during the second phase when our designer was cranking out
new concepts.

The development environment and Eclipse in particular were a dream for
the most part. This was a welcome change from Visual Studio.

On the couple of occasions where I couldn't solve a particular Android
problem by sheer perseverence or by looking for solutions online, I
posted on the development forums and was promptly assisted by a Google
employee.


WHAT WENT WRONG
From the start, performance was a huge concern for us. There was an
immediate performance limit imposed by our use of Google Maps, which
eats a good chunk of CPU and requires regular garbage collection. To
improve this situation, I employed strategies to circumvent the
default map rendering logic in order to more intelligently prefetch
tiles as you fly around and more efficiently render our wide variety
of overlays. This had the added benefit of reducing the somewhat ugly-
looking tile loading that you usually see when people use Google Maps.

However, this only got us so far. Early on I investigated rendering
the map to a 3D plane using an orthographic projection and moving that
around instead, but due to lack of support for hardware acceleration
in the emulator, this didn't gain us much. Then I flip-flopped back
and forth a couple times between using the event-based MapView and
using a thread-based SurfaceView. In the final weeks of development I
made a final switch to using a SurfaceView with a proper time-
sensitive game loop. I employed all the strategies mentioned in
Google's performance tips page, but I probably spent too much time
tinkering with algorithms, time that could've otherwise been spent
improving the stability of the heavily multithreaded code. In the
version we delivered to Google, emulated performance on a modern
computer was quite good, with some chugging when you fly into new
areas, but emulated performance on low-end computers was tolerable at
best.

There were times when an interesting new problem arose, and I probably
spent too much time on it. For example, we came up with an idea to
build and render generic buildings using voxel-like brick images
rendered at different angles to give us the flexibility of 3D with the
convenience of 2D. I spent a couple days getting this tech working
before I came to my senses and realized how much extra work on top of
that would be required to integrate and polish it. I suppose this is
probably a problem that most self-managing programmers face.

We spent a lot of time implementing features we thought actual users
would expect, like editing and deleting footprints, and some options
to help manage performance and battery life, and live notifications
for when friends create footprints. None of these were actually seen
by the judges. So rather than focusing on these secondary (though
important) features, we could've spent that time creating a rock solid
3-minute introductory walkthrough experience within the world that
would have had a better chance of improving our score. Instead, we
spent a little time at the end to create a more open-ended experience
in the hopes that judges would be curious and interested enough to
explore.

I already mentioned multithreading but it's worth mentioning again.
Java and Android in particular make it pretty easy to use threads, but
I think this is pretty dangerous. We ended up having a UI thread,
networking threads, a rendering thread and some miscellaneous worker
threads for stuff like animation and loading, not to mention whatever
threads get launched as a result of the Android features we were
using. The language and the SDK help you set all this up and even
manage it to a certain extent, but the usual pitfalls are still
there. I didn't employ enough rigourous testing of my multithreaded
code, and as a result, we shipped with some known indeterminate
crashes that sometimes happen when moving the map in strange ways
(exiting buildings, flying up and down, teleporting, etc). We
discovered very late that these crashes occur more frequently on lower-
end computers.

GPS became a problem for us. Although I was able to setup a mock
location provider, it wasn't always stable, and there was confusion on
the forums about whether or not judges would be able to load mock
providers. We couldn't use the default mock provider because it was
in California and most of our content was based in New York City. So
we didn't use the proper GPS provider and displayed a hard-coded GPS
location instead.

We were unable to include music and sound effects due to instability
with Android's MediaPlayer and extreme audio distortion when running
the emulator on Windows Vista. Since my main development box is Vista
and judges might be running Vista too, we didn't include any music or
sound effects in our submission.

As our UIs increased in complexity (despite our best efforts to keep
them simple!) we encountered a stack size limitation that effectively
capped the possible depth of our view hierarchies. This was
frustrating because the error message was obscure and I spent a great
deal of time trying to work around it (I assumed it was my fault). A
Google employee was kind enough to explain the situation to us, and
apparently view hierarchies will be greatly improved in the next
version of the SDK.

I experienced similar frustration with the animation system, which
works brilliantly in simple situations but seems to break when you try
to get fancy. I ended up abandoning some ambitious ideas as a result.

File management for our art assets started to become a chore because
you can't use subdirectories in the "drawable" directory. So we ended
up having almost 400 very carefully named images.

I was able to successfully record a 3gp video using the emulator's
camera, but playback of the resulting file is unsupported in the
current version of the SDK. As a result, we disabled video recording
for our submission (although you can still take pictures with the
camera).

Lastly, the lack of tools for visually designing UIs meant that by the
end of the project I was speaking and dreaming in XML. I explored
some of the community-made tools but found them to be insufficient for
the complexity of our view hierarchies. I spent a huge amount of time
creating and hand-tweaking multiple complex XML files (not to mention
styles and strings), time that could've been spent elsewhere if there
had been a more robust tool for creating and managing all that UI
data. Ideally the tool would be simple and powerful enough that our
designer could use it without a programmer's involvement. Having said
that, we were still able to accomplish a lot with the UIs...it just
took more time than I felt it should have.

Thanks for reading and I hope other people post their post-mortems,
too. See you on the other platforms!
Michael

[John P.]

"As a result of our early prototyping and brainstorming, I
experienced one of those "aha" moments where you end up pacing around
the room excitedly, thinking non-stop about all the implications of an
idea."

haha, this made me laugth. Good to know that I'm not the only one who
paces around in circles when contemplating things.

[nick]

Wow Michael, thanks for the detailed report. I wish you luck with
future development.

I have say though that I'm really surprised Duo didn't win. I think
the general consensus was that this was one of the most complete and
polished applications submitted to the challenge (that we knew of).
I'm still trying to get over the smarts of losing (submitted Fluid
Nexus), and I'm really interested to see who won. I definitely don't
want to disparage the winners at all, and offer them the fullest
congratulations--I'm just interested to see what the judges (i.e., the
companies in the alliance) thought was important and, by extension,
what they thought wasn't important.

nick

[mathiastck]

Thanks for posting this, I hope others do post similarly useful post-
mortems.

[freeanderson]

I'm too surprised Duo didn't win.
Actually I'm interested to know what a criterion of judging for
learning the reason I've failed.
And like Duo, I hope more to succeed in the business world.

andy

[Biosopher]

Hi Michael,

Ditto on my many thanks for your postmortem analysis. It takes spirit,
guts, & true belief in your project to both get this far and to then
open up to everyone for comments.

After reviewing some of the winners & losers, I'm also surprised at
some who have won & lost. I'll send you my thoughts on Duo post-
Monday when we can see all of the winners as we'll each have better
perspective then.

I'd love to get my hands on a working .apk for Duo so keep me down for
a beta release.

Cheers,
Anthony

[Dave]

On May 9, 1:42 pm, Michael Johnston <kineticp...@gmail.com> wrote:
> Duo didn't make it into the top 100 (let alone the top 50).

How do you know whether an app made it into the top 100? I didn't see
any information on that. Could have missed it, I suppose. Please let
me know.

[Lateef Jackson]

Hands down the best demo I have seen. This would be the second (mine,
weruat.com being the first) application I am planning on putting on my
phone (when they come out).

[fubin]

It is hard to believe such a good application failed...
You are still the winner in my heart!

[Michael Johnston]

Thanks to everyone for your support, and I'm happy some of you enjoyed
reading the postmortem. After we released the video last week, the
response we received was so positive that we were very hopeful. We
only posted the video in this forum and it trickled out to all sorts
of interesting places. It's very disappointing that we lost.

Some people have asked about any differences there might be between
our submission and the video I made. The video is authentic. The
transitions from login to indoors to outdoors, etc., are all real.
But I accelerated time in places (like for typing) and made some cuts
to improve the flow (like when you open a web browser). Our
submission also didn't have music or audio (due to Vista problems) and
didn't contain any of the administrator tools for creating your own
buildings.

[[nirajswami]]

Great stuff Michael!

NCAA coach Bobby Knight said "Many have the will to win, few have the
will to do the work to win"... Duo definitely reflects the latter...
and I am sure it'll be a success in the long run... good luck!

I look forward to using it in the future and spreading the good word
it deserves.

Niraj

> ...
>
> read more >>

[Hong]

OMG!!! Duo didn't win????

it's unbelievable!!!  Duo is the best I've seen so far...

so unless the top 100, top 50 are far exceeding what Duo does... which is quite frankly, impossible...

we'll see on monday who are the winners... so sad...

[jman]

I remembered when I first saw Duo, it really impressed me in several
ways. In a competition like this, judges are from all over the
world with very different cultural background. Some will certainly
tend to be stricter than others. If Duo happened to be put in the
one of the unlucky batches, then it would get a tough start because of
judges or apps in the batch. Let's assume I were a judge asked to
score a batch of 15 apps and only one should make it to top 100. If
2 best apps happen to be in this same batch, I would potentially
eliminate the 2nd best (which might be much better than other batches'
best) by separating them by a noticeable scoring margin.

Anyway, you work your best but some time in life, you need lots of
luck to be successful. There is no doubt about your talent and great
work. Just keep up your good work and hope you get the right
opportunity at the right time.

L.C.Y.

[BQ]

Surprise to me too.
Maybe a crash at the start of your app, that's the thing I can only
imagine of at this time.
Like I said in response to your debut announcement, I am still
interested in joining your team for future adventure.

> ...
>
> read more >>

[Greg Fodor]

Just wanted to add that Duo looked great. I hope at least you guys
move forward with it, I am pretty shocked you didn't come out on top.

[Ninad]

Its a big surprise to me that Duo didnt win!
Now I would really love to know the judging criterias!

For me, Duo is a winner!

[Shooter]

Really awesome demo. The hardwork shows. I think either the app
crashed or something during the demo or it was too complicated for the
judge to try out all the features on their own.
Keep up the hardwork and do market this thing.

Cheers,
Shooter.

[Alex Pisarev]

Quite surprised that Duo didn't win too...
May be there was a problem with Duo's presentation to judges?

[luckydroid]

I was sure Duo would be one of the top apps!
Keep up the good work. Duo looks awesome.
-mac

[jacek]

Duo is a winner for us, and the judging process a loser.
I too, had just a single server hit and I am 95% sure the (single)
judge
didn't bother to explore the menus leading to most of the
functionality.
Brutally, the strategy should have been to go for a slick UI
experience,
nice icons, something simple, 100% of the time working.
That, btw, describes a great commercial app :-)
Strange challenge... too black-and-white too early in the process

[whitemice]

You were in my top 50
Which means that the winners must be f**king hot, corporately biased
or someone somewhere has made a mistake.

Did someone say we find out on Monday?

[luckydroid]

omg- yes. the aha and the pacing! I've never paced so much in my life.
I was waking up going, Shit. I have another idea. And then, like you
said, all the elaborate and wonderful implications of the idea made me
get out of bed, write furiously in my notebook and pace in circles,
and across my apartment and back, and in another circle, and one idea
would lead to another so I'd write until I was satisfied that it had
been documented, so I could finally "force quit" and get a few hours
of sleep before it would start again. Luckily this occurs, and luckily
not daily because I would be a certified, sleepless, lunatic if it
did.

Thanks for the documentation, Michael. That will help a lot of us. I
am very surprised, as I posted before, that you were not in the 50.
Best of luck to you. Great app.

-mac

[Michael Johnston]

I thought I'd post a short update given what we've recently learned
about the judging process. Apparently, if there was some kind of
critical error while judging Duo, this would've been caught by the
judging process (either by eliminating one judge as an outlier, or by
assigning a new judge if there were two judges who experienced
problems).

We know from our server logs that two judges created Duo accounts
(along with a certain Google employee who isn't a judge). This means
that at least one, and probably two judges scored us based only on our
documentation. This doesn't surprise me because lots of other people
reported this as well.

Even if the two judges who created accounts scored us perfectly, the
other judges who weren't impressed enough by our documentation to
actually launch the app would've given us average scores at best,
which would be enough to eliminate us after the first cut.

So, to reiterate one of my post-mortem notes, we should've spent more
time on our documentation (which was good, but not great), and we
should've created our video before submitting, and provided a link to
it at the top of the document.

Thanks again to everyone for your encouragement. Our team really
appreciates it.