MDN Developer Services Survey Q1'15 results & updates

[Luke Crouch]

dev-webdev, cloudservices-all, mdn-drivers,

(If you'd like to stay in the loop on MDN Developer Services, please sign
up for a new list where we'll move most of the discussions:
https://lists.mozilla.org/listinfo/mdn-services)

We ran our 2nd iteration survey of MDN visitors about their use of
developer services.

tl;dr ...

We see our audience saying they need the same "Top 3" services as Q4:
Security Scanning, Cross-Browser Testing, and Accessibility Testing.
Load-Testing also debuted as the #4 service needed.

What we're doing in these areas ...

* Yvan Boily, David Walsh, & team are working on a security offering.

* Stephanie Hobson & John Whitlock are working on a data store for browser
compatibility data [2]

* We are looking into doiuse as a precursor or prototype of a compat
linting tool we could leverage [3])

* Tarek Ziade, David Walsh, & team are working on a Load-Testing tool [4]

* David Walsh is starting a Firefox Add-on with the security offering to
deliver any or all of these services into our audience's existing tooling
and workflow. [5]

* Justin Crawford sent a follow-up survey to those developers who said "No,
but I need to" [use a service to test accessibility]. [6]



Details ...

Changes in the methodology this time:

1. We uplifted some demographics questions (occupation, organization, etc.)
from our detailed follow-up surveys to the general survey.

2. We uplifted some workflow questions (i.e., tool usage) from our detailed
follow-up surveys to the general survey.

3. We expanded and updated tool & service response options to learn better
*which* tools and services our audience uses

4. We added questions for Backend-as-a-Service, File-hosting-as-a-service,
and Load-testing-as-a-service to learn if/what those opportunities might be.

We got 1,401 responses over 1 week. [1]

Some of the noteables:

* The most popular tools used in workflows are:
1. Code editor with plugins (78%)
2. Browser plugins (69%)
3. Code linters (46%)
4. Code pre-compilers (42%)

* "Other" is the most popular code editor, followed by Sublime Text, Vim,
Eclipse, Visual Studio, then Emacs

* Stack Overflow is the king of reference sites - it's more popular with
MDN visitors than MDN itself, because logic? ;)

* The most popular services currently used:
1. Code Hosting (60%)
2. Analytics (54%)
3. Project Management (47%)
4. Site/App Hosting (40%)
5. Performance Testing (37%)
6. File Hosting (33%)
7. Cross-Browser Testing (22%)
8. Monitoring (22%)
9. Continuous Integration (17%)
10. Optimization (12%)
11. Test Coverage Reporting (8%)
12. Security Scanning (9%)
13. Backend-as-a-Service (8%)
14. Accessibility Testing (7%)
15. Load-Testing (6%)

* GitHub is the king of Code Hosting - 78% vs. 35% for #2 Bitbucket

* Google is the titan of its categories - 93% of analytics (Google
Analytics), 74% of optimization (Google Analytics Content Experiements),
62% (Google Analytics) and 77% (Google PageSpeed) of performance testing

* Security Scanning has no clear king in the market. HP WebInspect is the
single most popular, but it's followed closely by "Other", and there's
significant responses using multiple vendors

* Cross-Browser Testing is dominated by BrowserStack (60%) and then by W3C
validators (46%).

* Similarly, WAVE/WebAIM dominates (65%) Accessibility Testing service
market.

* The most popular services developers say they *need* to use:
1. Security Scanning (26%)
2. Cross-Browser Testing (20%)
3. Accessibility Testing (20%)
4. Load-Testing (17%)
5. Performance Testing (16%)
6. Test Coverage Reporting (16%)
7. Optimization (15%)
8. Continuous Integration (13%)
9. Monitoring (12%)
10. Project Management (9%)
11. Analytics (9%)
12. Site/App Hosting (8%)
13. Code Hosting (7%)
14. File Hosting (7%)
15. Backend-as-a-Service (7%)

(If you've read this far, you should really join
https://lists.mozilla.org/listinfo/mdn-services :)

-L

[1]
http://qsurvey.mozilla.com/Reportsview?key=28049-4041795-f8a227baddefc3ca7a0eefbd3d5975e1&mode=html
[2] https://github.com/mozilla/web-platform-compat
[3] https://github.com/anandthakker/doiuse
[4] https://github.com/loads/loads-web
[5] https://github.com/mozilla/mdn-devtool
[6]
http://qsurvey.mozilla.com/Reportsview?key=28049-4087696-03f8eeb9946bf5a943789943f1981f55&mode=html