Kuma Report, December 2016, and year-in-review

[johnww...@gmail.com]

Here's what happened in December in Kuma, the engine of MDN:

- Not much
- Looking back at 2016

Here's the plan for January:

- Upgrade node.js
- Plan the next phase of moving to AWS

Done in December
===

Not Much
---
We didn't ship any big features in December. Between work weeks, holidays, and vacations, there wasn't a lot of time to write and push code.

The MDN staff attended the Mozilla All-Hands in Hawaii, which you could jealously follow with the #MozAloha tag [1] on Twitter. We have a distributed team, so it is good to see each other face-to-face, including long-time volunteers SphinxKnight and Sebastianz, and to have discussions that are hard to have on IRC or video chat. It was also a great time to talk with Mozillians on other teams, and to work out new ways to collaborate in the coming six months.

[1] https://twitter.com/hashtag/mozaloha

Our contributors were busy:

- mverity fixed bug 1306087, allowing MDN users to use their profiles on es.stackoverflow.com and other non-English StackOverflow sites on their MDN profile.
- arai-a fixed bug 1321963, which caused preview mode to break when the AddonSidebar is used. aria-a continues to track down issues with preview and macros, tracked in bug 973678.
- jpmedley made improvements to the SpecName macro, and PuercoPop added a Spanish translation. Their contributions also helped us figure out the new KumaScript macro editing workflow, from reviewing pull requests to deploying them to production.

We did get some collaborative work done. We worked with Matt Pressman (mpressman) of the Data team to schedule a maintenance window for schema updates against the production database. Ryan Watson (w0ts0n) from WebOps tuned MDN's HTTP headers, making MDN more secure for all our visitors. April King of the Security team helped ensure we did the right things, and helped us sketch a plan for a Content Security Policy in 2017.

Staff members also worked on the functional test suite, the sample database, planned front-end fixes and upgrades, and and made the KumaScript code work on Windows file systems.

OK, maybe a lot did ship in December :)

Looking Back at 2016
---
The winter break was a good time to look back on the progress made in a year, and see how the plans matched the execution. My memory of the MDN plan in January 2016 was:

- Upgrade to Django 1.8 and switch to hashed requirements files
- Move MDN hosting from the SCL3 datacenter to AWS
- Switch the database from MySQL to PostgreSQL
- Import MDN Browser Compatibility data into a new data service
- Build a modern contribution interface for compatibility data

We accomplished 1 out of 5. Sometimes the universe has different plans.

This was going to be a short report, but then I started digging into emails and commit history, and now it is long. Here's a few of the highlights of 2016:

- January
* Shipped Django 1.8 (jezdez, WebOps)
* Migrated from submodules to pip 8 and hashed requirements (jezdez)
* Added "./manage.py ihavepower" (willkg)
* Switched from nose to py.test (willkg)
* Monitored JS errors with Google Analytics (stephaniehobson)
* Decommissioned Demo Studio (robhudson)
- February
* Massive spam attack shuts down new account creation
* Refactored attachments (jezdez, robhudson)
* Refactored product details (willkg, robhudson)
* Merged first Docker config (robhudson)
* Fixed compile_stylesheets (matonanthony's first PR)
* Fixed random languages in the admin (willkg)
* Demonstrated a successful transition from MySQL to PostgreSQL (robhudson)
- March
* Started the move of the MDN dev team from Firefox org to Marketing org
* Shelved Browser Compat project
* Dropped SVG fonts (stephaniehobson)
* Shipped expanded contributor bar (stephaniehobson)
* Improved code sample performance (jezdez, jwhitlock)
* Added filtering to the default search (robhudson)
- April
* Jannis Leidel (jezdez), Rob Hudson (robhudson), and Will Kahn-Greene (willkg) leave for other Firefox products, leaving two developers on MDN
* Erin Mullaney (emullaney) and Dmitriy Chukhin (dchukin) (Caktus contractors) start
* Fixed and improved drafts code fixed (stephaniehobson)
* Improved Akismet integration (jwhitlock)
* Added Page Creation permissions (jwhitlock)
* Updated Homepage links (BychekRU's first PRs)
* Started work on the new Docker configuration (jgmize)
- May
* Shipped Zone fixes for Firefox OS -> B2G OS (chrismills, jwhitlock)
* Added a "Known Authors" group and filter (dchukin)
* Improved Spam submission UI (emullaney)
* Fixed Profile page and removed spammy fields (stephaniehobson)
* Added AJAX method for marking spam/ham (safwanrahman's first PR)
- June
* Account creation re-opens
* Jeff Bradberry (jbradberry), Caktus contractor, starts
* Approved MDN Tech Plan 2016 at London All-Hands
* Shipped phase 1 of the One Click Ban Hammer (emullaney, dchukin, stephaniehobson)
- July
* Announced Persona plan (MDN team)
* Disabled account creation w/ Persona (jwhitlock, stephaniehobson)
* Added tracking of spam viewers with Google Analytics (jbradberry, stephaniehobson, jwhitlock)
- August
* Caktus Contractors leave at end of contract
* Added an "All Topics" filter on search (emullaney)
* Discovered massive schema issues in production (jbradberry, jwhitlock)
* Shipped task completion survey (stephaniehobson)
* Shipped spam report dashboard (jbradberry, jwhitlock)
* Shipped CKEditor 4.5.10 and fixed the build process (safwanrahman)
* Cleaned up homepage links (stephaniehobson)
- September
* Ryan Johnson (rjohnson, escattone) starts
* Merged new Docker configuration (jgmize, jwhitlock)
* Started functional tests w/ py.test (stephaniehobson, mbrandt)
* Cleaned up template issue discovered during testing (stephaniehobson)
* Refreshed development documentation (jwhitlock, safwanrahman, jgmize)
* Added Jenkins pipeline (jgmize)
- October
* Added the newsletter signup (stephaniehobson)
* Prepared for Persona shutdown (jwhitlock, stephaniehobson)
- November
* Disabled Persona for login (jwhitlock, stephaniehobson)
* Moved Kumascript Macros from the MDN database to the GitHub repository (escattone)
* Shipped the beta sample database (jwhitlock)
- December
* Fixed most database schema issues in production (jwhitlock, mpressman, etc.)
* Shipped the first KumaScript macro updates using the new processes

Planned for January
===

Upgrade node.js
---
KumaScript runs in node.js, and we also use it in our static asset pipelines. We're thrilled that the Node.js foundation has announced a Long-Term Support policy, and we're taking the leap from v0.10 to v6.

https://github.com/nodejs/LTS

We expect most of the pain of this upgrade to be on the server side, with little effect on MDN viewers or on page rendering. After the upgrade, we'll be on a supported version, and the KumaScript project will look more like a current node.js project.

Plan the next phase of moving to AWS
---
We've done most of the safe tasks for preparing for the AWS move. The next steps involve moving production data and traffic to AWS, and could negatively impact the experience of MDN users. We're going to take time to plan the next steps in more detail, ensure we have the resources we need, and get buy-in from other groups at Mozilla.

We're currently estimating that the move will happen in Q2 2017. This includes completing work from 2016, like shipping the sample database and running the functional tests in our deployment pipeline. 2017 will be exciting and scary.