So, what's been going on with Q?

[Wout Mertens]

It's mighty quiet here - how about a quick recap of the state of everything? Q, Q-IO, Joey, etc? Roadmaps for 2015? Impact of ES6 promises on Q?

[Kris Kowal]

Thanks, Wout. Here's my State of the Vision for 2015. I find it particularly

fitting that this comes as a *reaction* to a request, and not in anticipation

of a need.

A few years ago, I dedicated a sizable chunk of my evenings and weekends to the

development and maintenance of the Q continuum. For this work I have been

rewarded to see promises flung far and wide. However, this time is no longer

available and I have returned to persuing different projects, both personal and

professional. My contributions continue at a trickle. Please consider the

following a vision and feel free to take up your own arms to any of these ends.

If governance of the projects I have begun becomes an issue, I am open to

discussion and would be glad to extend others a greater deal of autonomy and

responsibility within the Q continuum. I also encourage the creation of new

projects and coordination here.

# The Year of the Three R's

In summary, I have stable and experimental version trains for all of the

following projects. The stable trains will not see much change going forward. I

have some thoughts about experiments to conduct on the experimental chains and

their future is not set in stone. For the foreseeable future, the experimental

tracks will have to be installed with an explicit major version, e.g., q@2,

even after they become stable, emphasizing that they are all opt-in.

This has been accomplished with the following annotation in package.json across

all experimental packages in the continuum:

  "publishConfig": {

    "tag": "future"

  },

I see considerable opportunity to expand the Q ecosystem (as well as related

work on collections and reactive programming in general) as browsers implement

promises, streams, the HTTP fetch API, service workers, and ES6. We have an

opportunity to provide uniform interfaces that work in Node.js/IO.js, service

workers, workers, and window contexts for The Three R's: routing, rendering,

and reactivity.

This year I would like us to create a web site that provides a hub for

documentation, tutorials, and other articles for the Q v2 stack. The logo will

probably be a stylized cursive upper-case Q, playing on the ambiguity of that

glyph with a cursive 2. The Q stack is a set of coherent packages that version

in lock-step.

I may elect to entertain a bottom-up effort to migrate the entire continuum to

ES6 with 6to5 in the release automation.

I continue to endorse use of Browserify and Webpack but use Montage Studio's Mr

(Montage Require). The experimental branch of Mr includes a tool called Mrs

that is akin to Browserify and Webpack. More on that later.

Also, I have not discussed governance of the Montage Studio projects Jasminum,

Joey, and Q-Git. I am of course at liberty to fork them on my own Github

account, but would be outstepping my bounds to publish new versions. I am

hoping to take responsibility for the experimental versions of these projects

and to avoid interfering with production code that depends on the stable

version with the guarantee that the new versions will be opt-in indefinitely.

However, emphatically, I have not discussed this with my former colleagues at

Montage Studio who own those projects and have a stake in their ongoing

stability.

# ASAP

https://github.com/kriskowal/asap

ASAP is a micro-task event queue interface that works across browsers, workers,

and Node.js. It is made for compatibility and speed. This is an olive branch to

other continua, including Ember and Angular.

ASAP v2 establishes a new minimum quality bar for the entire experimental

package ecosystem, introducing a framework for testing automation accross

Node.js and browser window and worker contexts using SauceLabs, S3, and

TravisCI. I would like to see this generalized and applied to all of the other

packages in the Q continuum.

There is more work to be done in ASAP for supporting AMD and <script> loading.

This will involve release automation. Published versions will go to S3, just

like test versions and release candidates.

# Q

https://github.com/kriskowal/q

Q v2 remains highly experimental. The interface is not final. Some methods have

been trimmed or consolidated.

Goals:

- Provide a leaner core.

- Approach parity with ES6 promises.

- Use modules; introduce dependencies.

- Introduce release automation to mitigate the cost of dependency management

  for consumers who do not want to absorb that cost. (I posit that Q v1 does

  not have dependencies and works in source form on Node.js and web browsers

  with choice of package management solution has been instrumental in its

  broad adoption.)

  The new version of Q is CommonJS in source and will eventually have a release

  process like ASAP that will build AMD and <script> tag versions and publish

  them to S3.

- Introduce test automation that engages a broad spectrum of browsers in worker

  and window contexts. This remains to be done.

- Speed. Bluebird has shown us how to make promises faster for V8. Howver, I

  intend for Q to take a more balanced approach. We will not double the size of

  Q to make it faster.

  It remains an open question to what extent we should compromise principles

  that have been baked into Q version 1. Particularly, Q version 1 makes

  extensive use of closures to make their interface tamper-resistant and

  future-flexible. For example, there is no property of Q or promises that a

  user could accidentally or deliberately come to depend upon, breaking their

  application in a patch or minor release.

  At present, the version 2 of Q takes more advantage of traditional prototype

  oriented internals in order to play better with optimizers, but uses a

  WeakMap to keep the private interface private. As a consequence, regardless

  of whether we use a WeakMap shim or a genuine WeakMap, Q version 2 runs at

  about the same speed as version 1.

  We have the option of using a simple property of a promise, e.g.,

  `promise._keepOffMyLawn` to refer to the private handler. By ditching WeakMap

  outright, we can get another order of magnitude of operations per second.

  Promises will become more spoofable and subclassable.

  https://github.com/kriskowal/q/pull/602

- Estimated Time to Fulfillment. v2 will introduce an interface (in flux) for

  observing the estimated time to fulfillment for a promise. This will require

  a companion library for signals and behaviors.

- There remains an open question to what degree Q version 2 should embrace the

  native Promise when it is available.

  There is no question that Q should work regardless of whether a native

  Promise is available.

  Embracing the native promise will give us better debugging experiences in

  Chrome and Firefox at time of writing.

  It is entirely possible that Q's own Promise could be implemented with a

  focus on operations per second if it gave up its message passing internals.

  Message passing internals could be an aspect implemented exclusively for the

  use of promises for remote objects or far references, provided that Q

  provides the hooks to enable the recognition of such objects as promises.

  As such, a native promise would be a more amenable core for near references.

  Q would of course need to subclass the native Promise. I do not consider

  duck-punching, monkey-patching, or extension of the prototype by any other

  name an option.

  Of course, regardless, Q's promise will be a strict superset of native

  Promise and will consume native promise as a thenable.

- Q needs to be factored into a make-q module that instantiates the q module

  with the asap scheduler. makeQ may have other options, for example, deep

  freezing promises.

# Q-IO

https://github.com/kriskowal/q-io

https://github.com/kriskowal/q-io/issues

Q-IO version 2 is about new promise streams, or asynchronous iterators. I have

made much ado about this approach in A General Theory of Reactivity. The

interface is not final.

https://github.com/kriskowal/gtor

Most of the Q-IO internals need to be reevaluated in terms of these streams.

Particularly, listTree is an expensive and slow operation that should be made

much faster and have an alternate interface that uses a stream and responds to

back-pressure.

Streams of known length now support a read() method that returns a promise,

which in turn advertises the estimated time to fulfillment as a signal!

I have already refactored the file system, making it much easier to subclass

and avoiding the use of closures.

My approach to path joining needs re-evaluation. There are edge cases I am not

comfortable with, which Stuart Knightley has noted.

It would make sense for a lot of Q-IO operations to return Tasks instead of

Promises, so they could be cancelled. For example, reading a stream should be

cancellable. This merits further discussion. The Task interface is a bit more

prickly than a promise.

# Q-Connection

https://github.com/kriskowal/q-connection

Version 2 is in a decent state but as always needs review. It's been rewritten

to be much more rigorous about pass-by-copy vs pass-by-reference, and the

default has been switched to pass-by-reference.

How to mark an object for pass-by-copy remains an open question, especially if

we forego the use of WeakMap in Q.

Remotes are also an area where it might make sense to use Task as the primary

interface since it would enable explicit reference counting on far references.

# Q-Git

https://github.com/montagejs/q-git

Q Git is a new project that provides the Q-IO file system interface for a

JS-Git repository. JS-Git repositories are great since they can be backed by

the local .git folder or just about anything else.

Unfortunately, Q-Git stopped working at some point because JS-Git is not

stable. This requires further attention.

Q-Git enables a plethora of opportunities for better tooling. Committing build

products directly to a branch, especially documentation to the gh-pages branch,

is an amazing workflow.

# Joey

https://github.com/montagejs/joey

I am content to say Joey is in its final state. There is a great deal of room

for further experimentation with HTTP routing based on what we learned here. I

would like to see a solution that is factored into parts suitable for use both

on the back-end and the front-end, or even the service worker middle-end.

The browser history API and service workers will play well together. A server

that can do a first render to a route and use the same router on the client

side when offline is the golden egg this year.

There is also room to explore the possibility of adapter chains that compose

well per path. I am convinced that a smarter system is needed for composing

adapter chains, routing, and content negotiation.

This is a topic you will likely hear more from me about over the coming year,

but perhaps not in the context of the Q continuum.

# Jasminum

https://github.com/montagejs/jasminum

I encourage folks to start looking at Jasminum as an alternative to Jasmine for

testing in the Q continuum. Jasminum is 1 KLOC on top of the work in Q and

Collections.

I will probably factor the polymorphic operators for `equals` and `compare` out

of Collections to lighten the Jasminum dependencies.

I have been considering building out a name-space in npm for polymorphic

operators. A polymorphic operator is a function that treats its first argument

as an object. The operator defines a behavior for objects that do not implement

the eponymous method and otherwise forwards the remaining arguments to the

method.  For example, `equals(['a'], ['b'])` would be true despite Array not

implementing `equals`, and the Point2 from the `ndim` package would support

`equals(p1, p2)` by implementing `p1.equals(p2)`.

# Mr

Mr version 2 introduces Mrs, the Browserify-alike build script.

Mr also now supports user-defined module translators, compilers, and

optimizers. These can be configured per-package in package.json. This needs to

become a standard.

Mr needs to add support for generation of source maps along side.

We have a Mr-Coffee package. I would like to see a Mr-Essix that supports ES6

using 6to5.

# General Theory of Reactivity

I expect to spend some time amending my GtoR article and taking it on the road

this year. I feel this is one area where I can have a lot of positive impact

with my limited time. On the side, I also continue to fiddle with the formula

for "web components", and the Q continuum is as always my foundation for that

work.

With your support I hope to see great things this year!

[christ...@christophdorn.com]

On January 18, 2015 06:18:22 pm PST, "Kris Kowal"

<kris....@cixar.com> wrote:

> I continue to endorse use of Browserify and Webpack but use Montage
> Studio's Mr
> (Montage Require). The experimental branch of Mr includes a tool called Mrs
> that is akin to Browserify and Webpack. More on that later.

> # Q-IO
> https://github.com/kriskowal/q-io


FYI, I am building out a view into q-io stack using the pinf bundler
here: https://github.com/pinf/q-io-for-pinf

Nothing pushed yet as its not working yet but its coming along.

The end result will be like this:

* Sample project: https://github.com/fireconsole/harviewer
* Deployed to github pages: http://fireconsole.github.io/harviewer/
* Pull request neede to patch original source to get it working with
the PINF bundler: https://github.com/janodvarko/harviewer/pull/11

Some more PINF-bundler samples:
https://github.com/pinf-to/pinf-to-github-pages

The point is that you can build JS bundles from the source following
static and dynamic links that can be loaded into any JS context
conflict-free.

My goal is to generate these bundles for q-io for the browser and
nodejs to provide a mechanism to add full q-io functionality to any
distributed js app all coming from optimized files that will load
conflict free guaranteed.

See how the harviewer can be embedded using jsfiddle for an example of
how generated bundles can be used: http://jsfiddle.net/cadorn/z9py15jw/

IIRC I am currently stuck on some dependency resolution issues on the
v2 branch. Has been a while and need to take a new look at it.

It is my goal to build out this distribution model in parallel to
whatever Kris/Q-community chooses to validate all abstraction decisions
made in the Q stack and provide further momentum to the codebase and
theory.

Christoph

[Petka Antonov]

maanantai 19. tammikuuta 2015 4.18.20 UTC+2 Kris Kowal kirjoitti:

- Speed. Bluebird has shown us how to make promises faster for V8. Howver, I

  intend for Q to take a more balanced approach. We will not double the size of

  Q to make it faster.

Actually I learned the hard way that the optimizations are not that of a big contributor to the file size, otherwise I would have already

published a bluebird-lite (RIP the time I spent on that) version to be used in browsers.