Hypothesis System Architecture
Nick Stenning
Hi all,
These are our current thoughts on the direction we're taking the Hypothesis
software over the next little while. This has been discussed internally for a
few months now, and we're very close to making the first few major changes we
outline below. I thought it might be interesting to some people on this list.
Hypothesis System Architecture
What follows is a concrete proposal for the future architecture of the
Hypothesis software system. In it, I'm going to introduce some names for things
that might previously have had several names, crappy names, or none at all.
Broadly, there are four main parts of the Hypothesis software system, two of
which are primarily intended for use by third-party developers (the first two
in the following list):
1. The Hypothesis client library (hypothesis.js)
This is the bulk of our JavaScript codebase. It is intended to be a library
(including style/image/template assets) which can be used on its own to annotate
web pages, retrieve annotations from remote web services (which may eventually
include other services than our own) and send annotations to those web services
for storage.
As a library, it will have external APIs that serve multiple platforms,
including browser/web page consumers (for people who wish to embed the
Hypothesis annotator into their web pages with a <script> tag), as well as
providing support hooks for browser extension runtimes.
This library is semantically versioned and released independently, and is
deployed as built assets to a CDN (as well as, very probably, npm).
2. Annotation storage/retrieval library (memex)
This is what is currently referred to as h.api, and will supersede
annotator-store. I'm calling it memex, in homage to the device of the same
name for storing and retrieving annotations made on human knowledge, proposed by
Vannevar Bush in his 1945 essay As We May Think.
It is a Python package containing a Pyramid module and associated support
libraries, which provide, respectively: Pyramid views implementing an HTTP API
for management of annotation and document metadata, and programmatic interfaces
for the same. It also provides appropriate hooks and extension points to assist
developers integrating the module into a more complicated application (such as
our own -- see 3.).
It can be used by those who wish to integrate a Hypothesis-like annotation API
into their Pyramid web application. As discussed in a previous
thread,
it does not concern itself with authentication or an accounts system.
The library is semantically versioned and released independently, and is
deployed to the Python Package Index.
3. The Hypothesis Web service (https://hypothes.is)
This is the bulk of the current h Python codebase, and serves the public web
application at https://hypothes.is. It is not intended for broad reuse, and is
coded in the open
rather than strictly open source. For the purposes of this list, this category
also includes auxiliary services such as
bouncer and
via.
Versioning is whatever makes most sense to support release and monitoring of a
continuously delivered web application, and it has an upstream dependency on
memex (number 2 in this list).
4. Hypothesis browser extension(s)
Currently Chrome (and very experimental Firefox), in future we will support
browser extensions for multiple platforms. These are primarily integration and
glue code between the Hypothesis client library (number 1 in this list), which
will usually be bundled in the extension, and the extension runtime.
Each extension will be an attempt to provide an excellent user experience on
each platform, rather than "lowest-common-denominator" support across the
various extension platforms.
Versioning is whatever makes most sense to support release and monitoring of the
extension given the constraints of the vendor platform. The extensions will have
upstream dependencies on the deployed versions of the Hypothesis client library.
It is likely that the build process for an extension will fetch client library
assets from the CDN.
Next steps
Over the coming weeks and months you'll be seeing some evolutionary
reorganisation of our software along the lines laid out above.
It is far too early to be talking about creating new projects and git
repositories for most of the components described above. We need first to
extract the various components into their own part of the h repository, and
define and document the interfaces between them. That process will help us to
establish if the divisions of responsibility are the right ones, while allowing
us to avoid the friction associated with having more moving parts in the
Hypothesis software system.
I believe that what's outlined is broadly compatible with many steps already
taken (including the cleaner separation of h.api and h responsibilities, and the
recent frontend build improvements).
-N
--
Nick Stenning
Developer, Hypothesis