Developer Engagement Leads Offsite Day 1

[Stormy Peters]

Hi Mozillians interested in Developer Engagement!


Yesterday we discussed:

-

2014 goals and in particular Q1 goals
-

Process for proposing contributions to developer engagement projects
-

Working remotely


2014 Goals


Back in August a group of us met in Toronto to discuss the Developer
Program, what it meant and how we’d measure its success. We agreed
that the goal
of the Mozilla Developer Program is to enable, inspire and collaborate to
make the Web the primary platform used to create connected experiences
across devices.

We also decided on 5 metrics for 2014. Since then Patrick Finch figured out
some ways we could measure those goals and we’ve iterated on the goal for
2014 taking into account organic growth and our impact.


-

24.5 % of mobile web developers using open web technologies. (17%, July
2013.)
-

30% of web developers are using Firefox tools. (15%, July 2013.)
-

24.5% of apps developed in HTML5 across all platforms. (17%, July 2013.)
-

13,000 apps in Firefox Marketplace. (1,448, July 2013.)
-

1,000 developers/month contributing to Mozilla Developer
Program.(473/month, July 2013.)


We’ll update these numbers for the end of 2013 and make sure we’re on track.

Q1 Goals

Keeping in mind the 2014 goals, we brainstormed as a group what we could
measure and target for Q1.

For each of the 2014 goals we came up with a quarterly goal for Q1:


-

18% of mobile web developers using open web technologies.
-

xx% of web developers are using Firefox tools.
-

Need to sync up with the Firefox tools team
-

18% of apps developed in HTML5 across all platforms.
-

4,200 apps in Firefox Marketplace.
-

Need to sync up with the Marketplace team
-

539 developers/month contributing to Mozilla Developer Program.
-

Need to add Stack Overflow numbers to this


We also brainstormed other metrics that might show if we are making
progress or not. Here’s some of the ones we came up with. We expect that
every group will take the bigger goals and figure out how their team can
impact them. For example, the documentation team might measure how many
visitors to the MDN Apps page end up submitting an app on Marketplace to
help understand how their work impacts the apps goal.


-

21,000 newsletter subscribers (current number: 19,899)
-

x # of Firefox OS tagged questions or answers on StackOverflow
-

# of MDN active contributors
-

# of code base committers
-

# of apps from PhoneGap
-

# of apps submitted from MDN referrals to Marketplace
-

# of page hits for HTML5 pages on MDN
-

# new simulator add-on downloads
-

what does the developer tools team measure?
-

# of page hits on tools on MDN (doubled with the redesign)
-

# of MDN referrals to simulator
-

measure the funnel from MDN to submit an app


We also talked about all the work we are currently doing and how it maps
into the top level goals. There’s lots of great stuff already happening
from newsletters to workshops to engagements with MADE to localization to
badges to ... Each team/individual can evaluate the impact they are having
and what changes if any they need to make.

Process for proposing contributions

We discussed how to turn ideas into actions in a way that encourages
autonomy for every person and team that contributes to the developer
ecosystem. Tomorrow we will draft a process for any contributor to propose
new projects in a way that scales across our traditional team structures to
create more spontaneous, effective, and efficient collaboration on projects
that directly impact the higher-level goals. Some of the key aspects we’re
considering:


-

You champion your own idea. The process shows how you can make it happen.
-

We will establish a channel for reviewing proposals as a group.
-

Every idea will pass thru a minimal set of questions that will help
convey impact and requirements. Potential questions:
-

What is the Project?
-

Who is the Champion?
-

Who else do you need to work on it?
-

Which goal does the project impact?
-

How can we measure the project impact against the goal?
-

Do you need official Mozilla “sponsorship” (like partnering with
another org)


Working Remotely


We have a very geodistributed team that is functioning extremely well. We
regularly get other people who approach us for best practices. Here are
some of our current best practices:

-

IRC - we are all on irc whenever we are working. This is hugely
beneficial to our team. An even bigger bonus would be if we could get all
of Mozilla to adopt this.


-

Asynchronous. While we have our fair share of meetings, we work hard to
make sure that communication can happen asynchronously to accommodate as
many schedules and working styles as possible. We do a lot of our
communication in mailing lists, wiki pages and bugs. Decisions made in
meetings are shared out through email, wiki or bugs as well.
-

Bugs. We track all of our work in bugzilla. We use needinfo to flag
individuals. We document decisions made in meetings with a comment in the
bug. We encourage people to file bugs for all issues and make sure we
capture key information there.
-

Meetings. We use meetings wisely. We have as few of them as possible. We
have many IRC meetings - instead of vidyo - to enable more people to
participate and to facilitate information sharing. We document decisions
that are made. We have much of the discussion in email before hand - this
enables people to participate even if they can’t make the meeting time.
-

Encourage the “do-ers” to make decisions and work across teams, as
opposed to working through managers.


Things we could do better:

-

Instead of saying no to proposals, we should encourage others to develop
their idea and help them figure out how to implement it.
-

Agenda for all meetings. We should have a clear agenda for all meetings
ahead of time.
-

Bugs. We communicate well in bugs but we don’t always have bugs for all
work.
-

Devengage Cafe. Create a Vidyo room just for hanging out together.


Today we’ll discuss Developer Program value add - what we can best offer
developers to help them create open web content - and the process for
proposing contributions.

Best,

Stormy, Luke, Ali, Mark, Robert & Sakina

[Jean-Yves Perrier]

Hi!

Nice info, thank you very much. A few comments about these.

On 19/12/2013 14:42, Stormy Peters wrote:
> We also decided on 5 metrics for 2014. Since then Patrick Finch figured out

> some ways we could measure those goals and we�ve iterated on the goal for

> 2014 taking into account organic growth and our impact.

I suppose the 5 metrics listed are the one decided? Because until now
these have never been communicated to the whole team.

>
>
> Working Remotely
>
>
> We have a very geodistributed team that is functioning extremely well. We
> regularly get other people who approach us for best practices. Here are
> some of our current best practices:
>
> -
>
> IRC - we are all on irc whenever we are working. This is hugely
> beneficial to our team. An even bigger bonus would be if we could get all
> of Mozilla to adopt this.

Don't forget that IRC is synchronous. Though it is an very useful mean
to have a question answered, important discussion must be logged. There
is nothing worse as hearing that something was discussed during the
European night the other day via IRC. A IRC log can be added to the
relevant bug, or e-mail.

Decisions (other than trivial, technical) shouldn't be taken outside
regular announced meetings, even on IRC.

>
> -
>
> Asynchronous. While we have our fair share of meetings, we work hard to
> make sure that communication can happen asynchronously to accommodate as
> many schedules and working styles as possible. We do a lot of our
> communication in mailing lists, wiki pages and bugs. Decisions made in
> meetings are shared out through email, wiki or bugs as well.

Each time something is done synchronously, we exclude 99% of the
volunteers and 30% of the employees.

> -
>
> Bugs. We track all of our work in bugzilla. We use needinfo to flag
> individuals. We document decisions made in meetings with a comment in the
> bug. We encourage people to file bugs for all issues and make sure we
> capture key information there.

Does this rule out github?

> -
>
> Meetings. We use meetings wisely. We have as few of them as possible. We
> have many IRC meetings - instead of vidyo - to enable more people to
> participate and to facilitate information sharing. We document decisions
> that are made. We have much of the discussion in email before hand - this

> enables people to participate even if they can�t make the meeting time.
Meetings must be announced in advance (not at 5pm for a meeting the same
day). We are all grown-up, exceptions can happen for good reasons of course.
Also the agenda for the meeting must be published at that point: it
allows us to know if we need to attend the meeting or if the minutes
will be enough. It also allows us to _prepare_ for the meeting.

Also by default, all meetings should be announced on a mailing list.
Closed meetings should be the exception, not the default. Agenda + time
of the meeting is what drive my decision to attend it.

IRC is also easier for non-native speaker and for by-listener. Or for
shy contributors.

I know that a good meeting hygiene is difficult (public announcement,
agenda, meeting leading, published minutes). It implies mastering the
time (almost no impromptu meeting) and the communication. If this is
successful, 90% of the communication problems will be solved.
> -
>
> Encourage the �do-ers� to make decisions and work across teams, as

> opposed to working through managers.
>
>
> Things we could do better:
>
>
> -
>

> Agenda for all meetings. We should have a clear agenda for all meetings
> ahead of time.

This is critical.

> Devengage Cafe. Create a Vidyo room just for hanging out together.

I don't think this is a good idea, hanging in #devrel is much better. I
will not sit in front of a camera for an improbable chat. I love IRC
chat with the rest of team. I keep root in the French community or with
#london coworkers when out of the office that way.


Thank you for the feedback on your discussion :-) Much appreciated.

Have a nice day!

--
Jean-Yves Perrier
Technical Writer / Mozilla Developer Network

[Jeremie Patonnier]

Hi!

Thanks for sharing.
Here are a few comments and questions :)

tl;dr: I'm highly skeptical about the goals as they are currently defined.


2013/12/19 Stormy Peters <spe...@mozilla.com>

> 2014 Goals
>
> Back in August a group of us met in Toronto to discuss the Developer
> Program, what it meant and how we’d measure its success. We agreed
> that the goal
> of the Mozilla Developer Program is to enable, inspire and collaborate to
> make the Web the primary platform used to create connected experiences
> across devices.
>

I should miss something but what exactly are the goals? I mean,
practically? "enable, inspire and collaborate" are not goals, they are
inspirational purpose.

I would be interesting on how each dev engage sub team are turning those
top level ideas into concrete goals? As a volunteer it would be easier for
me to have such concrete goals to help Mozilla reach them.

>From my experience a good practical goal is define as such:

- It's written short
- It gives a point to reach
- It provide a way to measure its success

Say another way: it must be realistic, actionable and measurable. Otherwise
it's just speech in the wind.

> We also decided on 5 metrics for 2014. Since then Patrick Finch figured out

> some ways we could measure those goals and we’ve iterated on the goal for

> 2014 taking into account organic growth and our impact.
>

I would be curious to know how you are measuring those goals. To be
accurate the measurement need to be very accurate in order to avoid false
impression.

> 24.5 % of mobile web developers using open web technologies. (17%, July
> 2013.)
>

How did you get such numbers? By itself it can mean anything. The way you
get such metrics is a very strong indicator of what it exactly means.

> 30% of web developers are using Firefox tools. (15%, July 2013.)
>

I'm skeptical about the interested of such metrics (and again, would be
interested in knowing how such value is gathered). It's a quantitative
information that mean nothing about the adoption of the web as the
platform. There is nothing wrong by using other developer tools, and
FIrefox (even without other browsers' tools) has a long tradition of third
party tools.

> 24.5% of apps developed in HTML5 across all platforms. (17%, July 2013.)
>

Again, I'm not sure about what it means. Do you mean that a quarter of HTML
Web Apps are running on all browser or does it mean 75% of the Web Apps in
the wild are made for one browser only?

And what does "platforms" mean? Browser? OS? Device? All of those three?
And in that case I'm impressed if you truly are able to check "all"
platforms.

So a metric that is so much debatable is not necessarily a good one. Maybe
it needs to be polished.

> 13,000 apps in Firefox Marketplace. (1,448, July 2013.)
>

Nice metric but it mean nothing about having the web being the platform. It
just mean that people are pushing their web app to the Mozilla Marketplace.
I agree that it's a strong metric for Mozilla but beware, it can be tree
hiding the forest.

> 1,000 developers/month contributing to Mozilla Developer
> Program.(473/month, July 2013.)
>

What means "contributing" in that area? Pushing an app to the marketplace?
Editing MDN? Something else?

So as you can see, once pushed in the wild, those metrics needs some
serious explanation about how they are gathered and what they mean exactly.
Otherwise it's a troll prone topic.

Q1 Goals
>
> Keeping in mind the 2014 goals, we brainstormed as a group what we could
> measure and target for Q1.
>
> For each of the 2014 goals we came up with a quarterly goal for Q1:
>
> - 18% of mobile web developers using open web technologies.
> - xx% of web developers are using Firefox tools.

> - 18% of apps developed in HTML5 across all platforms.
> - 4,200 apps in Firefox Marketplace.

> - 539 developers/month contributing to Mozilla Developer Program.
>

I'm curious about how you get to those values? As I'm especially involved
with the MDN team and I cannot figure how it can be actionable for that
team.

> - Need to add Stack Overflow numbers to this
>

This is something that really puzzled me a lot. Yes Stack Overflow is an
awesome place for Web dev. But it's only for English speaking people. How
to engage more people that do not speak English? Mozilla has an incredible
community beyond the English speakers. There is also so many amazing web
developers who do not speak English at all. How do we plan to reach all
those people? English should not be the alpha and omega of our work. We
need some serious plan to move forward English. That way we will be able to
reach our goals faster and more efficiently in the long run.

> We also brainstormed other metrics that might show if we are making
> progress or not.

See my first point, but I strongly think we should first think in terms of
action and only think about metrics to measure the success of those action.
Absolute metrics are hard to get, difficult to interpret and subject to
outside side effects.

> Here’s some of the ones we came up with. We expect that
> every group will take the bigger goals and figure out how their team can
> impact them. For example, the documentation team might measure how many
> visitors to the MDN Apps page end up submitting an app on Marketplace to
> help understand how their work impacts the apps goal.
>

Okay, let's take that example. First let's assume it's possible to get such
information (this would be in itself an interesting topic but it's not the
place). This typically a bad goals in the long run. If the MDN team is
expected to improve that transformation chain, they will focus on having
the MDN Apps page promoting the Firefox Marketplace on many point from the
obvious to the more subtle. Such a hard quantitative goal will force them
to "forget" about the quality of the documentation in favor of some kind of
marketing message. In the end we will lose developers that will start to
think about MDN as the place to get information about Mozilla's Web Apps
instead of having them remembering of MDN as the best place to know
everything about the Web Platform. In the long run the will turn their back
on us... but the short terms goals would have been reached.

A better goal would be to measure how many web apps are updated to be
available cross platform after a visitor has read the MDN Apps page (but I
seriously doubt it's possible to measure that in the wild but for the app
on the marketplace, it could be possible).

> - 21,000 newsletter subscribers (current number: 19,899)
>

I'm so sorry to be that skeptical... but measuring newsletter subscribers!
in 2014? Seriously? With so many web dev using social network, I'm not sure
it's a relevant metric. I think we should better focus on a social networks
strategy.

> - x # of Firefox OS tagged questions or answers on StackOverflow
> - # of MDN active contributors
>

We need to define what "active" means.

> - # of code base committers
>

Which code base?

> - # of apps from PhoneGap
> - # of apps submitted from MDN referrals to Marketplace
> - # of page hits for HTML5 pages on MDN
>

We should globally improved the visibility of MDN and focus a lot more on
localization for that.

> - # new simulator add-on downloads

> - # of page hits on tools on MDN (doubled with the redesign)
> - # of MDN referrals to simulator
>

What's the point of this?

> - measure the funnel from MDN to submit an app
>

If you wish but it a very short term goal that mean nothing in the regard
of having the web becoming the preferred platform for developing Apps.

> Working Remotely
>
> We have a very geodistributed team that is functioning extremely well. We
> regularly get other people who approach us for best practices. Here are
> some of our current best practices:
>
> - IRC - we are all on irc whenever we are working. This is hugely
> beneficial to our team. An even bigger bonus would be if we could get
> all
> of Mozilla to adopt this.

> - Asynchronous. While we have our fair share of meetings, we work hard
> to
> make sure that communication can happen asynchronously to accommodate as
> many schedules and working styles as possible. We do a lot of our
> communication in mailing lists, wiki pages and bugs. Decisions made in
> meetings are shared out through email, wiki or bugs as well.
>

It's true but it seriously need to be improved. Mailing list are underused
across the dev engage teams.

> - Bugs. We track all of our work in bugzilla. We use needinfo to flag
> individuals. We document decisions made in meetings with a comment in
> the
> bug. We encourage people to file bugs for all issues and make sure we
> capture key information there.

> - Meetings. We use meetings wisely. We have as few of them as possible.
> We
> have many IRC meetings - instead of vidyo - to enable more people to
> participate and to facilitate information sharing. We document decisions
> that are made. We have much of the discussion in email before hand -
> this

> enables people to participate even if they can’t make the meeting time.
>

We should avoid "closed doors" meeting. There were way to much of them in
2013. Volunteers could have the false impression they are not welcome as
they have to dig deep to have the necessary information about schedule, and
decision process.


> - Encourage the “do-ers” to make decisions and work across teams, as

> opposed to working through managers.
>
>
> Things we could do better:
>

> - Instead of saying no to proposals, we should encourage others to
> develop
> their idea and help them figure out how to implement it.
>

It would also be good to explain avery "no". Sometimes, "no" is a valid
answer but it must be explain, espacially when the decision making process
is fuzzy (and sometimes it's more than fuzzy as even the payed staff does
not know where the answer come from).

> - Agenda for all meetings. We should have a clear agenda for all
> meetings
> ahead of time.
>

And a reasonable schedule that allow people to prepare the meeting.

> - Bugs. We communicate well in bugs but we don’t always have bugs for
> all
> work.

> - Devengage Cafe. Create a Vidyo room just for hanging out together.
>

Interesting but IRC already provide this, what's the advantage of having a
video conference room here?

> Today we’ll discuss Developer Program value add - what we can best offer
> developers to help them create open web content - and the process for
> proposing contributions.
>

It's good to remember that "open web content" is something very different
than "Firefox OS content" or "Having Apps in the marketplace". Mozilla's
tools and projects are just a means to promote the web platform. They are
not a goals by themselves, if they fail it's not a problem as long as the
standard web succeed.

Best,
--
Jeremie
.............................
Web : http://jeremie.patonnier.net
Twitter : @JeremiePat <http://twitter.com/JeremiePat>

[Robert Nyman]

Hi,

Thanks for the e-mail, glad you appreciated the sharing of info!

> Decisions (other than trivial, technical) shouldn't be taken outside regular announced meetings, even on IRC.

I agree. We talked about that any decisions being taken, way to move forward etc - whether it came up on IRC, in a meeting, e-mail etc - it should always be documented in an existing bug belonging to that feature. I.e. a short list of decisions that have been made/discussed.

>> Devengage Cafe. Create a Vidyo room just for hanging out together.

> I don't think this is a good idea, hanging in #devrel is much better. I will not sit in front of a camera for an improbable chat. I love IRC chat with the rest of team. I keep root in the French community or with #london coworkers when out of the office that way.

I think this is a really interesting area, so let me elaborate on it. First, let me emphasize that this is completely optional and adding a way for us to be more social. A lot of people work remotely and never (very seldom) see their colleagues in person, and this is a way to offer that. To get that office feel, of just chatting with people, seeing each other, potentially having a quick discussion on an interesting topic (instead of 15 e-mails going back and forth).

The tone in writing - be it IRC, e-mail or similar - can also be very different than talking to other people. And I would venture a guess that you work and communicate in a written form much better with people you have actually met/seen in person - that's definitely my own experience at least. And if such an open Vidyo room can help with us getting closer to each other and feel more connected, and at times be more efficient in solving some things, I definitely think it' worth a try for those who are interested in testing that.


Best regards,
Robert

On 19 Dec 2013, at 15:19, Jean-Yves Perrier <jper...@mozilla.com> wrote:

> Hi!
>
> Nice info, thank you very much. A few comments about these.
>
> On 19/12/2013 14:42, Stormy Peters wrote:

>> We also decided on 5 metrics for 2014. Since then Patrick Finch figured out
>> some ways we could measure those goals and we’ve iterated on the goal for
>> 2014 taking into account organic growth and our impact.

> I suppose the 5 metrics listed are the one decided? Because until now these have never been communicated to the whole team.
>>
>>

>> Working Remotely
>>
>>
>> We have a very geodistributed team that is functioning extremely well. We
>> regularly get other people who approach us for best practices. Here are
>> some of our current best practices:
>>
>> -
>>
>> IRC - we are all on irc whenever we are working. This is hugely
>> beneficial to our team. An even bigger bonus would be if we could get all
>> of Mozilla to adopt this.

> Don't forget that IRC is synchronous. Though it is an very useful mean to have a question answered, important discussion must be logged. There is nothing worse as hearing that something was discussed during the European night the other day via IRC. A IRC log can be added to the relevant bug, or e-mail.
>
> Decisions (other than trivial, technical) shouldn't be taken outside regular announced meetings, even on IRC.
>>

>> -
>>
>> Asynchronous. While we have our fair share of meetings, we work hard to
>> make sure that communication can happen asynchronously to accommodate as
>> many schedules and working styles as possible. We do a lot of our
>> communication in mailing lists, wiki pages and bugs. Decisions made in
>> meetings are shared out through email, wiki or bugs as well.

> Each time something is done synchronously, we exclude 99% of the volunteers and 30% of the employees.

>> -
>>
>> Bugs. We track all of our work in bugzilla. We use needinfo to flag
>> individuals. We document decisions made in meetings with a comment in the
>> bug. We encourage people to file bugs for all issues and make sure we
>> capture key information there.

> Does this rule out github?

>> -
>>
>> Meetings. We use meetings wisely. We have as few of them as possible. We
>> have many IRC meetings - instead of vidyo - to enable more people to
>> participate and to facilitate information sharing. We document decisions
>> that are made. We have much of the discussion in email before hand - this
>> enables people to participate even if they can’t make the meeting time.

> Meetings must be announced in advance (not at 5pm for a meeting the same day). We are all grown-up, exceptions can happen for good reasons of course.
> Also the agenda for the meeting must be published at that point: it allows us to know if we need to attend the meeting or if the minutes will be enough. It also allows us to _prepare_ for the meeting.
>
> Also by default, all meetings should be announced on a mailing list. Closed meetings should be the exception, not the default. Agenda + time of the meeting is what drive my decision to attend it.
>
> IRC is also easier for non-native speaker and for by-listener. Or for shy contributors.
>
> I know that a good meeting hygiene is difficult (public announcement, agenda, meeting leading, published minutes). It implies mastering the time (almost no impromptu meeting) and the communication. If this is successful, 90% of the communication problems will be solved.

>> -
>>
>> Encourage the “do-ers” to make decisions and work across teams, as
>> opposed to working through managers.
>>
>>
>> Things we could do better:
>>
>>
>> -
>>

>> Agenda for all meetings. We should have a clear agenda for all meetings
>> ahead of time.

> This is critical.

>> Devengage Cafe. Create a Vidyo room just for hanging out together.

> I don't think this is a good idea, hanging in #devrel is much better. I will not sit in front of a camera for an improbable chat. I love IRC chat with the rest of team. I keep root in the French community or with #london coworkers when out of the office that way.
>
>
> Thank you for the feedback on your discussion :-) Much appreciated.
>
> Have a nice day!
>
> --
> Jean-Yves Perrier
> Technical Writer / Mozilla Developer Network
>

> _______________________________________________
> engagement-developers mailing list
> engagement...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/engagement-developers

[Eric Shepherd]

I don't like the suggestion here that the docs team should be responsible for tracking metrics as well as acting to improve them. We need to be focused on building content, not on gathering data to meet someone else's goals.

Eric Shepherd
Sent from my iPad

[Stormy Peters]

All teams are responsible for making sure what you are doing is making a
difference in the way we want, i.e. metrics. Someone on your team (you,
your manager, someone) should be making sure you are developing content
that aligns with your Mozilla goals.

Stormy

[Luke Crouch]

I won't speak for others, but I'm +1 to managers taking responsibility
for measuring group & project impacts on goals, and helping everyone
measure individual impact too.

We have to do some kind of visible work, right? :)

-L

--
Q: Why is this email five sentences or less?
A: http://five.sentenc.es