MDN: Moving KumaScript Macros to GitHub

[rjoh...@mozilla.com]

If you're a past or present contributor to MDN’s macros (sometimes called templates), we want to share some important news with you! We're moving the management of MDN's macros to GitHub. Starting today at 11am PST, Wednesday November 23, 2016, we're asking that you stop creating and updating macros using the editor on the MDN website, and then starting Monday November 28, 2016, instead use the kumascript repository on GitHub (https://github.com/mozilla/kumascript) to create and edit KumaScript macros. This new workflow will open up contributions to our macros to more people while making them safer and more reliable, since we’ll have a proper workflow for updating them. Take a look at the documentation in the repo’s README for more information, and please don’t hesitate to ask questions in #mdn.

We hope you find this as exciting as we do!

[William Bamberg]

This is really exciting, Ryan.

I had some questions.

* where under https://github.com/mozilla/kumascript will they live?
* what is the workflow? Especially, how can we test our changes? Will we
need to set up local instances of Kuma?

Will

> _______________________________________________
> dev-mdc mailing list
> dev...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-mdc
> MDN contributor guide: http://bit.ly/ContributorGuide
>

[Sebastian Zartner]

These are really exciting news!

I'm wondering the same as Will, though. Also, will the macros be
copied over to GitHub as they are currently stored on the MDN website?
There should have been some communication about this in advance.

Sebastian

[John Whitlock]

On Wed, Nov 23, 2016 at 3:40 PM, Sebastian Zartner <
sebastia...@gmail.com> wrote:

> These are really exciting news!
>
> I'm wondering the same as Will, though. Also, will the macros be
> copied over to GitHub as they are currently stored on the MDN website?
> There should have been some communication about this in advance.
>
>

This is the advanced communication! And this, from November 2nd:

https://groups.google.com/forum/#!topic/mozilla.dev.mdn/K_uCWXLbqEg

And this, from November 2015:

https://groups.google.com/d/msg/mozilla.dev.mdn/PEuH7WW2Xtk/3y7Tv9_rEgAJ

The macros will be in the "macros" folder of the kumascipt project, and
will be the versions as of today. If changes are missed, you can get them
includes with a PR.

You can see the new layout in the branch:

https://github.com/escattone/kumascript/tree/1314058-import-macros

Specifically, the macros:

https://github.com/escattone/kumascript/tree/1314058-import-macros/macros

John

[William Bamberg]

Thanks John!

For the workflow question, a relevant bit from the linked email[1] is:

> Moving the macros to GitHub... will require that macro authors
> test their changes in a local development environment.
> The Docker development environment makes this move possible.

[1] https://groups.google.com/forum/#!topic/mozilla.dev.mdn/K_uCWXLbqEg



On Wed, Nov 23, 2016 at 2:13 PM, John Whitlock <jwhi...@mozilla.com>
wrote:

[Sebastian Zartner]

> On Wed, Nov 23, 2016 at 2:13 PM, John Whitlock <jwhi...@mozilla.com>
> wrote:
>>
>> On Wed, Nov 23, 2016 at 3:40 PM, Sebastian Zartner
>> <sebastia...@gmail.com> wrote:
>>>
>>> These are really exciting news!
>>>
>>> I'm wondering the same as Will, though. Also, will the macros be
>>> copied over to GitHub as they are currently stored on the MDN website?
>>> There should have been some communication about this in advance.
>>>
>>
>> This is the advanced communication! And this, from November 2nd:
>>
>> https://groups.google.com/forum/#!topic/mozilla.dev.mdn/K_uCWXLbqEg
>>
>> And this, from November 2015:
>>
>> https://groups.google.com/d/msg/mozilla.dev.mdn/PEuH7WW2Xtk/3y7Tv9_rEgAJ
>>
>> The macros will be in the "macros" folder of the kumascipt project, and
>> will be the versions as of today. If changes are missed, you can get them
>> includes with a PR.
>>
>> You can see the new layout in the branch:
>>
>> https://github.com/escattone/kumascript/tree/1314058-import-macros
>>
>> Specifically, the macros:
>>
>> https://github.com/escattone/kumascript/tree/1314058-import-macros/macros
>>
>> John

I see. Sorry that I missed/forgot about that communication, John!

On 23 November 2016 at 23:23, William Bamberg <wbam...@mozilla.com> wrote:
> Thanks John!
>
> For the workflow question, a relevant bit from the linked email[1] is:
>
>> Moving the macros to GitHub... will require that macro authors
>> test their changes in a local development environment.
>> The Docker development environment makes this move possible.
>
> [1] https://groups.google.com/forum/#!topic/mozilla.dev.mdn/K_uCWXLbqEg

Unfortunately I wasn't able yet to properly set up the Docker
environment on my Windows 10. Though I also just quickly tried it
shortly after your announcement and didn't take the time yet to
investigate the problems further.

Sebastian

[rjoh...@mozilla.com]

As of around 11:30am PST, this went live! From now on, all changes to KumaScript macros must be made via the kumascript repository on GitHub (https://github.com/mozilla/kumascript). Please see the README on the main page of the repo for more information.

[Jeremie Patonnier]

That's great, well done Ryan :D

Just two quick questions about the life cycle of deployment:

- What is the delay between a pull request being merged and its
deployment in production environment?
- Is there a way to do some testing of macros on staging before moving
to production?

2016-11-28 11:55 GMT-08:00 <rjoh...@mozilla.com>:

> As of around 11:30am PST, this went live! From now on, all changes to

> KumaScript macros must be made via the kumascript repository on GitHub (

> https://github.com/mozilla/kumascript). Please see the README on the main
> page of the repo for more information.
>

> _______________________________________________
> dev-mdc mailing list
> dev...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-mdc
> MDN contributor guide: http://bit.ly/ContributorGuide
>

--
Jeremie
.............................
Web : http://jeremie.patonnier.net
Twitter : @JeremiePat <http://twitter.com/JeremiePat>

[John Whitlock]

On Tue, Nov 29, 2016 at 5:04 PM, Jeremie Patonnier <
jeremie....@gmail.com> wrote:

>
> Just two quick questions about the life cycle of deployment:
>
> - What is the delay between a pull request being merged and its
> deployment in production environment?
> - Is there a way to do some testing of macros on staging before moving
> to production?
>
>

Short answers:
- It takes between an hour and a week to deploy merged code, based on the
severity of the change
- Yes, macros can be tested on staging, but should not be developed there

We'll deploy KumaScript changes at the same time as Kuma changes, so that's
probably the best model for deployments. We'll see how that changes with
KumaScript macros in the mix.

The Kuma process is something like:
1. A developer creates a change in their development environment in a git
branch, including tests and documentation.
2. The developer creates a pull request (PR) on the branch, adds some
helpful notes about the change, and recruits a developer to code review it.
3. A second developer reviews the code, which include reading and
understanding the change, trying it out in their local development
environment, thinking about tests, documentation, deployment, etc.
4. The developers go through one or more rounds of review, adjusting the PR
until it is approved.
5. The PR is merged to the master branch.
6. The master branch is deployed to staging and manually tested
7. The same code is deployed to production

We deploy when there's a compelling reason. If there's a security fix
needed in production, we'll deploy at midnight on a Saturday. If it isn't
an emergency, then we deploy during work hours, when we're well rested and
have at least an hour available. If a bug has been in production for
months or years, it can wait until the morning to get fixed. And if
something unexpected happens, it's good to have time to deal with it, which
may include coding, merging, and deploying a second time in a day.

We don't deploy every code change, but instead try to batch a few, since it
requires at least one developer focusing on deployment and monitoring
instead of doing other tasks. For small stuff, a deploy a week is about
right. During busy times, it may be a deploy every day.

The deploy process itself is manually started but automated. A developer
actively monitors the production environment for an hour after deployment,
and takes care of other tasks like closing bugs, moving Taiga cards, and
sending emails related to the deployment.

Part of the process is testing on staging, which can potentially find
issues with merged code. A few things can happen:

1. The code works as planned. We deploy to production.
2. A minor bug is found, but the change is an overall positive. The
production deployment proceeds, and the developer begins a new branch to
fix the new bug.
3. The bug is bad, but a quick fix is identified and the deploy isn't
critical. Staging is frozen while a fix is coded, reviewed, merged, and
tested, then is deployed to production.
4. The bug is bad, but there are other changes that must be deployed. A
"reverse" PR is created, reviewed, and merged, which does the opposite of
the previous merged commits, and the new master is deployed to production,
while the developer tries again in a new branch.

We don't "roll back" the master branch, only add commits and move forward.
It is too late to go back in time and fix things once they are merged. This
is why only code reviewed changes should be merged to master, and why we
don't let un-deployed changes pile up in the master branch.

If staging isn't needed for production deploys, a branch can be temporarily
deployed there, for testing or demonstration. This happens once or twice a
quarter. This could be done with macros, after reasonable testing in a
development environment.

John