[frontend-infra] Where to put all the common code?

[Kyle Lahnakoski]

For those working on frontend code,

My dream is to have dashboards be easy to create: To build something
like the android main page https://health.graphics/android in just one
day.  My experience with dashboards is they have short lifespan.  A good
number are project dashboards which die at the end of a project. Status
dashboards show data that does not change much from day-to-day, so are
used once and not again until months later (at which point we forget
they exist).  Only the rare cases where a dashboard supports a business
process does it last long enough to justify development cost. In all
cases, making dashboards easy to create (and easy to throw away) is a
benefit to Mozilla.

To achieve this goal of dashboard-a-day we require a library of tools,
and the corresponding expertise to use those tools. My hope is the
frontend-infra team will be sharing their techniques via this mailing
list. This leaves the question of where to put the tools, or pointers to
the tools.

WHAT IS A TOOL?  A tool can be a debugging library, a linting standard,
components we find useful, code to pull data from particular sources,
common styles, dynamic layout components, etc.

I am new the frontend work (my experience with 2012 javascript does not
count).  I have been working on the Health Dashboard [1], and there some
tools that I believe are useful to other dashboarding projects. For now
I have put that code into a `vendor` subdirectory [2] in anticipation
for some more official location/process to replace it.

PLEASE NOTE: Do not let my poor choice of word `vendor` distract you
from the contents; which are more like `mozbase` [3] than a repository
of 3rd party source code.

MAIN QUESTION: How should we be sharing this suite of tools? What does
it look like when we launch a new dashboard project?


The rest is my proposal,

I imagine there would be a github repo called "make-new-dashboard" which
has a predefined `package.json` with all our favorite third party
modules.  It will have a `index.jsx` file that can handle one route to
one page that shows one chart.  It would include the `vendor` directory
for all the tools that are big enough to be useful, but too small to
justify packaging as a module.  It would look a lot like Firefox Health,
but stripped to the bones.  A mono-repo of all our tools, if you will.

The dashboard life-cycle would be: Forking the repo, changing the name
(and other configs), writing the dashboard, deploying to some (static)
site, and leaving it to end-of-life. 

PAN-PROJECT SYNCHRONIZATION:

We do have some long lived dashboards (which I hope includes Treeherder
and Perfherder, eventually). They raise the issue of code
synchronization: Given that all dashboards are forks of (or "derivatives
of") `make-new-dashboard` how do improvements in one get propagated to
another (if at all)?

I suggest this be left to the derivative dashboard maintainer to decide:

* Share as module -  The `make-new-dashboard` could be basis for a
formal module that is just-another-import for the derivative dashboard
to use.  Tool development proceeds in "make-new-dashboard", while
dashboard development proceeds in the derivative. The maintainers of the
derivative can deal with module/version synchronization.   

* Share as code - I use a VCS to synchronize code between projects. As
long as their is a common subdirectory (the `vendor` directory) between
`make-new-dashboard` and the derivative dashboards, they can be
synchronized.  Code changes in a a derivative dashboard can be packaged
as a PR in `make-new_dashboard` and vise-versa.  The details of this
process need not be known to the maintainers of other derivative
dashboards; code changes appear as PRs against `make-new-dashboard`,
just like in the above case for modules.

Feedback? Suggestions?

Thank you



[0] https://github.com/mozilla-frontend-infra

[1] https://github.com/mozilla-frontend-infra/firefox-health-dashboard/

[2]
https://github.com/mozilla-frontend-infra/firefox-health-dashboard/tree/master/src/vendor

[3] https://hg.mozilla.org/mozilla-central/file/tip/testing/mozbase

[Hassan Ali]

For low-level UI interactions, we use material-ui[1]. For high level
components, we have @mozilla-frontend-infra/components[2]. If you want to
create a reusable chart component, please feel free to create a PR so that
it can be shared with other people. We also have a styleguide site that
hosts all of the components[3]. Regarding linting, we have
@mozilla-frontend-infra/lint[4] which are mozilla addition rules extending
the Airbnb styleguide. For bootstrapping new dashboards, we recommend using
Neutrino[5].

[1] https://material-ui.com/
[2] https://github.com/mozilla-frontend-infra/components
[3] https://mozilla-frontend-infra.github.io/components/
[4] https://github.com/mozilla-frontend-infra/lint/
[5] https://master.neutrinojs.org/

Hassan


On Fri, Mar 15, 2019 at 12:38 PM Kyle Lahnakoski <klahn...@mozilla.com>
wrote:

> _______________________________________________
> tools mailing list
> to...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/tools
>

[Armen Zambrano Gasparnian]

More content inline.

regards,
Armen

On Mar 15, 2019, 12:38 PM -0400, Kyle Lahnakoski , wrote:
> For those working on frontend code,
>
> My dream is to have dashboards be easy to create: To build something
> like the android main page https://health.graphics/android in just one
> day.  My experience with dashboards is they have short lifespan.  A good
> number are project dashboards which die at the end of a project. Status
> dashboards show data that does not change much from day-to-day, so are
> used once and not again until months later (at which point we forget
> they exist).  Only the rare cases where a dashboard supports a business
> process does it last long enough to justify development cost. In all
> cases, making dashboards easy to create (and easy to throw away) is a
> benefit to Mozilla.
>
> To achieve this goal of dashboard-a-day we require a library of tools,
> and the corresponding expertise to use those tools. My hope is the
> frontend-infra team will be sharing their techniques via this mailing
> list. This leaves the question of where to put the tools, or pointers to
> the tools.

After the various interactions we've had I believe a guide tackling the following topics would help:
* How to create a frontend app (e.g. neutrino create project)
* How to handle routing (e.g. react routing)
* Recommended query parsing techniques
* How to deploy the app (e.g. GitHub pages or Netlify)
* How to do development logging versus production logging (avoid console)
* Default timezone for all tools
* Recommended time/data manipulation library
* Recommended linting tool for React apps vs libraries

I think if we put a Google Doc gathering this information we could work on it quite fast and eventually put something a bit more formal.

>
> WHAT IS A TOOL?  A tool can be a debugging library, a linting standard,
> components we find useful, code to pull data from particular sources,
> common styles, dynamic layout components, etc.
>
> I am new the frontend work (my experience with 2012 javascript does not
> count).  I have been working on the Health Dashboard [1], and there some
> tools that I believe are useful to other dashboarding projects. For now
> I have put that code into a `vendor` subdirectory [2] in anticipation
> for some more official location/process to replace it.
>
> PLEASE NOTE: Do not let my poor choice of word `vendor` distract you
> from the contents; which are more like `mozbase` [3] than a repository
> of 3rd party source code.
>
> MAIN QUESTION: How should we be sharing this suite of tools? What does
> it look like when we launch a new dashboard project?
>
>
> The rest is my proposal,
>
> I imagine there would be a github repo called "make-new-dashboard" which
> has a predefined `package.json` with all our favorite third party
> modules.  It will have a `index.jsx` file that can handle one route to
> one page that shows one chart.  It would include the `vendor` directory
> for all the tools that are big enough to be useful, but too small to
> justify packaging as a module.  It would look a lot like Firefox Health,
> but stripped to the bones.  A mono-repo of all our tools, if you will.
>
> The dashboard life-cycle would be: Forking the repo, changing the name
> (and other configs), writing the dashboard, deploying to some (static)
> site, and leaving it to end-of-life.

When using a fork the repo approach there are some operational aspects of the project that cannot be forked.

Some of these need to be set up manually for every project:
* Deployments - e.g. Heroku or Netlify
* Per-branch deployments - e.g. review apps for Heroku and Netlify
* CI - e.g. TravisCI
* Code coverage - e.g. coveralls
* Alerting - each of the tools mentioned have their own alerting customizations
* Front end error reporting - e.g. Sentry

Some of these could be skipped for certain projects, however, things like Sentry have always helped me noticed issues before users notice. There are trade-offs to not setting them up.
Documenting these would reduce the cost of creating new projects.

Another alternative to the forked approach is by using Glitch.
Glitch allows you to see the code and the final deployment at the same place.
No need for development environment, users can fork and edit their own version of the canonical dashboard. This means they are more likely to maintain their own dashboard.

An approach like this would require very good documenation. We should maintain the canonical project. This would still require discussing how to share common code.

My counter proposal is to have a single repo where different dashboards can be built within in. You just have different routes to get to it. For the most part it reduces having to create a method to share common code. Nevertheless, we could still put widgets/code that could be used by others as packages within this repo. Even if we choose my proposal, I believe we should put a guide to help people create their own independent projects.

>  PAN-PROJECT SYNCHRONIZATION:
>
> We do have some long lived dashboards (which I hope includes Treeherder
> and Perfherder, eventually). They raise the issue of code
> synchronization: Given that all dashboards are forks of (or "derivatives
> of") `make-new-dashboard` how do improvements in one get propagated to
> another (if at all)?
>
> I suggest this be left to the derivative dashboard maintainer to decide:
>
> * Share as module -  The `make-new-dashboard` could be basis for a
> formal module that is just-another-import for the derivative dashboard
> to use.  Tool development proceeds in "make-new-dashboard", while
> dashboard development proceeds in the derivative. The maintainers of the
> derivative can deal with module/version synchronization.

Right now, this is the case for most projects.
The maintainer of a project needs to decide when to update the dependencies of the projects.

At large, dependencies have been using semantic versioning (accurate following of it is another topic) to indicate major/breaking releases versus minor/security releases.
This generally means that taking minor/security releases should not be costly to update to.
Updating to major releases might require re-testing or even following a guide (e.g. upgrade from Neutrino 8 to Neutrino 9).
Sharing as a module is the de-facto method these days.

>  * Share as code - I use a VCS to synchronize code between projects. As
> long as their is a common subdirectory (the `vendor` directory) between
> `make-new-dashboard` and the derivative dashboards, they can be
> synchronized.  Code changes in a a derivative dashboard can be packaged
> as a PR in `make-new_dashboard` and vise-versa.  The details of this
> process need not be known to the maintainers of other derivative
> dashboards; code changes appear as PRs against `make-new-dashboard`,
> just like in the above case for modules.

This could be done, however, it adds a custom solution that contributors would have to learn and we would need to maintain.
Sticking to the standard methods (Npm ecosystem) is what I propose we stick to for the lower barrier of entry.