A precursor to my talk on Wed about React/Flux

[James Long]

Hey all,

I'm talking on Wednesday (10:50AM) about some of the stuff I've been doing
recently on the debugger. Some of the new stuff I'm trying involves Flux, a
style of writing UIs typically used with React. I've discovered that it's
very helpful to steal ideas from it even without using React, and it's
actually a required first step in refactoring if we ever want to use React.

React/Flux is actually pretty simple, but because it's quite different from
previous "best practices" it can hard to understand quickly. To avoid a big
dump of info in a short talk, I wanted to write down everything I'm going
to talk about. If you read this first you'll probably get more out of my
talk.

Meant to send this out earlier, I know the week has already started and
you're already busy, sorry!

# Goals

Generally my goals for the debugger frontend are the following:

* No global variables, use modules
* Reduce side effects, especially DOM mutations
* Reduce race conditions, make it very explicit how actions are executed
* Easier testing, both unit testing and also while developing

# About churn and tech fatigue

Abstractions and constraints are good, but the wrong ones are poisonous and
crippling. You should be wary of adding dependencies, and be careful which
abstractions and constraints you are building into your architecture.

But it's just as dangerous to be too conservative and reject new things
because of unknowns. Be conservative with dependencies, but be aggressive
about adopting good ideas. Separate trends from fads. The tech worlds looks
like it's all fads

[James Long]

OMG freaking gmail interpreted my keystrokes as "send this email" even
though I was in the middle of it. Well, this is the beginning :) will
finish it soon and send the rest.

[James Long]

Full email below, sorry about sending the unfinished one. Read this only if
you want to be familiar with the stuff I'm going to be talking about.

---

I'm talking on Wednesday (10:50AM) about some of the stuff I've been doing
recently on the debugger. Some of the new stuff I'm trying involves Flux, a
style of writing UIs typically used with React. I've discovered that it's
very helpful to steal ideas from it even without using React, and it's
actually a required first step in refactoring if we ever want to use React.

React/Flux is actually pretty simple, but because it's quite different from
previous "best practices" it can hard to understand quickly. To avoid a big
dump of info in a short talk, I wanted to write down everything I'm going
to talk about. If you read this first you'll probably get more out of my
talk.

Ok, so this is pretty long, but it's hard to transcribe an entire talk and
keep it short. Read on only if you are interested. Meant to send this out

earlier, I know the week has already started and you're already busy, sorry!

# Goals

Generally my goals for the debugger frontend are the following:

* No global variables, use modules
* Reduce side effects, especially DOM mutations
* Reduce race conditions, make it very explicit how actions are executed
* Easier testing, both unit testing and also while developing

# About churn, tech fatigue, and when to adopt ideas

In my talk I'm going to discuss how to decide when to adopt ideas or not.
While we don't want to always be in churn, we also don't want to reject
legitimately good ideas that will help us a lot.

React is a case where I feel like it will legitimately help us. It's not a
fad, it's a significant shift in UI development. Ember is even rewriting a
lot of their internals and rethinking components because of React.

A way to think about React is it's a black box that lets you render data
into UIs. It's that simple. Instead of poking the DOM in various places,
React allows you to avoid side effects and treat it functionally: give it
some state, and rerender the entire UI based on it, and it will make sure
the *real* UI reflects it (minimizing actual updates by way of diffing the
new structure with the last). It's as if you have a `render` function that
takes state and returns a UI: `render(state) -> UI`.

Shared mutation makes things incredibly hard to understand, and the DOM is
basically a huge shared mutable structure.

# From React to Flux

I'm not actually going to go in-depth into React. We're not close to
being able to use it. Converting a widget to use React isn't really that
helpful, we only get anything out of it if we use React as part of the core
UI architecture.

If you know absolutely nothing about React, you may want to peruse this
material first:

- http://facebook.github.io/react/docs/thinking-in-react.html
- https://www.youtube.com/watch?v=DgVS-zXgMTk
-
http://jlongster.com/Removing-User-Interface-Complexity,-or-Why-React-is-Awesome

I'll quick a little more introduction to it in my talk.

Turns out right now we're actually more interested in Flux. What's flux?
Well first let's take a quick look at React.

Let's say we have a simple React component called `App`, and it contains
some state. With React you do this by implementing the `getInitialState`
method. And you set state by calling `setState` within the component (like
in response to a user event), which rerenders `App`. (In my talk I will
show code for this). Effectively the UI is updated like this:

+-----+
| | <-render
v |
App ---+

(I formatted that as fixed width, if you've turned formatting off that's
going to look weird)

In reality, your UI is made up of multiple components which form a tree
hierarchy. But still, if `App` contains state and passes it down to these
components as properties, it also passes callbacks which subcomponents call
in response to various events. These callbacks change the state within
`App`. So the flow of updating the UI still looks like this:

+---------------------+
| |
v |
App +-> Items |
| |
+-> Toolbar --+
|
+-> Button

This is a simplified view, but it helps show the flow. Any component can
actually have local state (like subcomponents), so sometimes only
sub-sections of the UI tree are rerendered when state changes. But
typically the flow is state flows down as props, and callbacks are fired to
notify components upward to change state, which triggers a rerender.

You may think always rerendering is slow. It works because there are two
phases of rendering: first render out all the components into virtual dom,
which is just simple JS objects that components return in their `render`
method that represent the desired structure. Secondly React will diff this
structure with what's already present in the DOM, and only make any changes
necessary.

Making virtual DOM is actually super fast. Way, way faster than touching
the DOM. In cases where it is a problem, it's super easy to implement a
method in a component to control when exactly it should be "updated",
meaning when it should create the vdom structure and diff.

Given this capability, we can now ask more interesting questions: where is
state stored? how do we update it? React only cares about applying changes
to the DOM and event handling from the DOM. It's up to us to figure out the
right state and component structure.

That's where Flux comes in: it complements React by prescribing a way to
manage data outside of the component tree. Most of your data should NOT be
component local state, since most of the time any component should be able
to read from the "app state" and render anything it wants to.

So Flux actually looks like this:

+----------------------------+
| |
v |
App +-> Items |
| |
+-> Toolbar -----> State
|
+-> Button

Now we have explicit state which the components can change and read
from.

There's actually a lot more details about flux. The state is split up into
multiple "stores", so you have something like the breakpoint store, and the
sources store. The components fire "actions" that any of the stores can
listen to, and stores emit change notifications so components can rerender.

But there's a critical element here: state is NOT stored in the DOM. For
example, currently the source listing in the debugger is treated as the
list of sources. If you need to lookup a source based on URL, you look it
up from the literal DOM list. Getting the currently selected source is
querying the DOM list.

There are multiple problems with this. First it makes testing harder.
Anything that needs access to the source list needs it actually exist in
the DOM, which makes it impossible to unit test any UI elements. Second, it
creates weird cyclical dependencies with events: if you select a source
from the list, it will update the editor. But if the editor changes the
currently selected source, the list will turn around and tell the editor to
update (even though it's the one that originated the event, and it doesn't
want to update). It needs to awkwardly pass flags around to specify where
events originated so it breaks the cyclical dependency.

Probably the worst is that it binds the state down to how the UI looks, so
any refactoring of the UI requires *everything* that touches it to change.
The state should be completely separate from the DOM structure.

# Stores and Actions

There are several more specific flux concepts. Each of them can be
interpreted loosely, and implemented how we please. The important part is
the general flow.

* Stores: stores are simply just methods separated into modules to deal
with different pieces of state. We could have a breakpoint store and a
sources store, for example. When state changes within a store, the store
emits a change notification (so components can "observe" stores).

* Actions: Actions are simple JS objects tagged with an "action type" like
"ADD_BREAKPOINT", and the various properties as arguments. Any component
can dispatch any action.

* Dispatcher: The dispatcher is a central object which components use to
actually fire actions. All stores are registered to the dispatcher. When an
action is disptached, it's broadcasted to all stores. It's very much like a
pubsub system.

An example dispatch within a component would be:

dispatcher.dispatch({
type: "ADD_BREAKPOINT",
location: { actor: "actor1", line: 5 }
});

There are several critical pieces here:

* Data flow always goes forward. Components *never* write to stores
directly (but they can read from them).
* *All* actions are broadcasted to *all* stores. A store can listen for any
action.
* Actions are simple JS objects with a string type name
* A dispatch can never occur in the middle of another dispatch. Stores
never fire dispatches.

What this means is you get a very clear structure of changes in the system,
and all the UI will update automatically when something is changed. Only
one thing can happen at a time: one action is processed, it's rendered out.
This constraint is good: you can log all the actions going through the
system and get a very clear idea what's going on.

Another small bit: it's helpful to have things called "action creators",
and all they do is wrap the dispatch into a method call so it's nicer. So
you could just do this instead of manually firing an "add breakpoint"
action:

actions.addBreakpoint({ actor: "actor1", line: 5 });

What's more powerful about action creators is they can fire multiple
actions, so you can wrap up more complex flows into a single method call.
More on this later (or at least in my talk). This concentrates tricky
interactions (mostly async) into a single place.

# Migrating

This looks cool and all, but is it realistic to migrate? I discovered that
there's a great migration path, which involves several passes which first
makes our code more Flux-like, and then eventually we could use React:

- split up the views into separate files
- treat each view as a component and controller as store. refactor each
view to not touch the controllers, instead emit actions. refactor each
store to not touch the views, instead emit change events.
- views new simple receive data and emit actions, should be trivial to
rewrite as a React component
- wrap existing components into React ones as needed to avoid work

We can do steps 1 and 2 right now without even using React, and it has
greatly simplified the debugger. Now instead of manually hooking up
dependency chains for specific UI elements, they are updated automatically
when a specific piece of state changes, and that piece of state can only be
changed by a store, which only happens because of an action coming through
the system.

It's not as nice without React. We need to emit the *specific* piece of
data which has changed, instead if broadcasting a generic "update" event,
but it's still way better.

# Adding a breakpoint

Ok, let's take a look at what adding a breakpoint was like before. Let's
assume you click the gutter in the editor, the control flow would look like
this:

DebuggerView.editor.addBreakpoint ->
Breakpoints.prototype._onEditorBreakpointAdd ("breakpointAdded" event
handler) ->
Breakpoints.prototype.addBreakpoint ->
Breakpoints.prototype._showBreakpoint

Now `_showBreakpoint` calls `DebuggerView.editor.addBreakpoint` if the
`noEditorUpdate` option wasn't passed, and also calls
`DebuggerView.Sources.addBreakpoint` is the `noPaneUpdate` option wasn't
passed. In this case `noEditorUpdate` would be true because the event
originated from the editor, and the breakpoint has already been added.
Yikes.

Additionally, breakpoint state is stored in three places: the editor, the
source listing DOM, and the Breakpoints controller. It's hard to keep these
in sync.

There are several other interactions with breakpoints that I will show in
my talk. You enable a breakpoint in the source list, and it needs to appear
in the editor. You add a breakpoint from the key command. All of these
right now pass around various flags to ensure only the things that should
update are updated, and it's a very weird and complex flow.

With the new architecture, it looks like this:

actions.addBreakpoint (fires ADD_BREAKPOINT action) ->
ADD_BREAKPOINT handler in Breakpoints store ->
'breakpoint-added' change event fired

Both the editor and the sourcel listing listen for the "breakpoint-added"
change event and make sure that the breakpoint exists.

It's hard to show this in text, and this is where I'll start showing a lot
more demos in my talk.

# Testing

All of this paves the way for some pretty incredible testing capabilities.
We can unit test the stores without a DOM at all by simply firing actions
and making sure the right change notifications occur. We can test
components by manually firing specific change notifications and making sure
the right things happen.

I think a lot of our mochitests eventually could be written as smaller
xpcshell unit tests. Mochitests woud still be good for integration tests of
course.

# Demos

I'm going to show several demos:

* A debugger with the backend server mocked out, and the ability to fire
actions from the console and watch changes happen in the UI real-time

* Using a single atom app state instead of splitting it up in between
stores, and snapshotting the UI by serializing the app state with
`JSON.stringify` after every change notification, and replaying the entire
UI history for debugging.

* Similar to above, but instead of logging the app state, simply log every
single action coming through the system. This lets you replay the entire
history and gets you back to a *working state* of where you were
(breakpoints are set, etc). Imaging a coworker sending you a serialized
action history that you can replay to watch a bug unfold.

* The SourcesView component wrapped up into a React component, and the new
ability to rerender it live to change its state. The allows you to play
with a specific component live, and change the attributes its rendered with
to see what happens. This will show how we can still use all of our
existing components as React ones.