Please advice on strategy moving forward...

4 views

Skip to first unread message

tav

unread,

Dec 18, 2010, 1:03:40 PM12/18/10

to Espians, Espians, openk...@googlegroups.com

Hey all,

I'd really appreciate your advice with regards the next steps of
the Espian strategy.

In short: I'm thinking of focusing on creating a fairly simple
Twitter-like collaboration system over the next two months. I
describe it below and have tried to make it easy to understand.

The idea is that with this system done, we could use it as the
basis to both kick off some crowd-funding as well as get the
general Espian movement going in full swing...

The app won't be decentralised, nor will it scale beyond about
10,000 users, but it will include many of the ideas that we've
talked about — Trust Maps, Pecus, Amp Creds, Confluence, etc.

It should be a good, usable, proof of many of the Espian ideas.
It will live on https://www.espians.com and be fully open source
of course.

The alternative to creating this app will be to keep on with the
never-ending development of the decentralised Ampify. Whilst
doing that appeals to the perfectionist in me, I fear it might
end up being "too little, too late".

What do you all think? Will the following be good enough? Should
I just focus on creating the decentralised system and not get
distracted?

I'd really appreciate your thoughts. Thank you!!

--------------------

At the heart of it, the site is very much like Twitter. You type
your message, press enter, and boom! It appears in the stream of
everyone who is following you.

Well, it's a lot better than Twitter because your messages
actually get sent in real-time. No need to click or refresh the
page, new messages simply appear like with chat/IM.

There's also a separate "To:" field like with email. Here you
can state that the message is to a:

* specific user, e.g. @olasofia
* public space, e.g. #espians
* controlled space, e.g. +family

This simple addition, combined with the ability to trust people
for specific spaces (Trust Maps), means that you can limit
spaces to only contain messages from those that you trust.

So if I only trust @evangineer for the #bigsociety space, I'll
see messages from him when I look at #bigsociety, but won't see
what he's sending to spaces I don't care about, e.g. #android.

In each space, you can also switch to seeing messages from
everyone and browse everyone else's Trust Maps to discover
others that you might want to add to your own Trust Maps.

It is worth noting here that we take a fairly unique approach to
privacy control. Instead of marking your messages as public or
private, privacy is determined by where you send messages to.

Everyone can send messages to and see messages sent to public
spaces, e.g. #bigsociety. The style of public space names was
chosen to be similar to Twitter #hashtags and IRC #channels.

However, we also introduce a new type of space, a "controlled"
space. These spaces can be created by anyone and access to them
is determined by whether you have an appropriate "access key".

These access keys specify whether you can:

* send a message to the space
* read messages sent to the space
* delete messages from the space
* set admin options for the space
* send messages to a space as another specific user

And any constraints on its validity, e.g.

* it can only be used N number of times
* it has to be used before a specific date/time

These access keys are both revocable and transferable. That is,
once someone gives you one, it can be revoked whenever they want
— but you can also give it to someone else if you want.

Access keys enable a range of different setups. For example, you
might give paying subscribers access to read messages from a
space, but limit the ability to send messages to just yourself.

In fact, the individual @spaces, e.g. @jeffarch, are just very
special controlled spaces. By default, the respective user has
an access key with all the privileges.

And whenever they connect with anyone (e.g. adding them to a
Trust Map), they give the other user an access key to send
messages to their @space.

The @space also has an admin option set to allow those without
access keys to also send messages to it if they pay the amount
(in Amp Creds) set by the user.

So, if I'd set the option to 0.2 Amp Creds, then whenever
someone whom I'm not connected with sends a message to @tav,
they'll have to pay 0.2 Amp Creds.

The idea behind this is to:

* provide an effective anti-spam protection
* enable people's attention to be really valued

For now, the @space names are global. That is, whenever anyone
refers to @jeffarch, they all mean the same @jeffarch. However,
all other controlled spaces are not global.

Instead, they are unique to each individual user and represented
with a leading plus sign, e.g. +family. People can call these
spaces whatever they want — including the ones that others give
them access.

And in addition to the respective @space, a number of default
controlled spaces are created for a user:

* +drafts
* +settings

The +drafts space is used to auto-save your messages as you're
writing them in case your browser crashes. And +settings is used
to store your various custom settings.

You can view multiple spaces at the same time thanks to a very
useful user interface innovation that thruflo has termed
"dolumns" — dynamic columns.

This is a bit like the columns on TweetDeck which allow you to
see multiple spaces at the same time, but with the ability to
re-arrange and minimise them with ease.

The number of spaces you can see at any one time is dependent on
your screen size which is auto-detected — but works well on
everything from iPhones to 27-inch iMacs.

Now that spaces are hopefully a bit clearer, let's switch focus
back to messages. Your messages don't need to be just plain
text. For starters the following are auto-linked for you:

* usernames, e.g. @thruflo
* spaces, e.g. #london
* links, e.g. http://inamidst.com/

All shortened links, e.g. http://tinyurl.com/24ot9no are
automatically expanded so that you know where they lead to and
warnings shown for sites on the Google Safe Browsing blacklist.

All links to music, photos, videos and info on the following
sites are specially marked so that you can view them without
having to leave the site:

* Flickr
* GitHub
* Skitch
* SoundCloud
* TwitPic
* Vimeo
* YouTube

All linked mp3 and video files (including SoundCloud recordings
and YouTube videos) can be queued up to be played in the builtin
media player.

Similarly, PDF and Powerpoint links can be viewed inline using
the Google Docs viewer without having to separately download and
open them.

You can also directly attach rich data to your message:

* Files up to 2 gigabytes in size. If these are image files then
a thumbnail is automatically created for them.

* Geolocation — you can specify the exact location (latitude,
longitude) by dropping a marker on a Google Map.

* Date/time which can be described in a number of natural
languages, e.g. "next Friday" and is fully aware of timezones
with daylight saving adjustments, etc.

* Number value, e.g. 1.61803398874989

* Currency value, e.g. ¥500

* Transboolean value, i.e. yes/no/maybe

* List and choice values.

* Rich text up to 100 kilobytes in size.

* Links — which automatically show the title, description and
let you choose a preview image just like on Facebook.

Besides attaching these values to a message, you can also use
those values to define an "aspect" of something. These aspects
are denoted with a leading slash, e.g. /price.

And you can make up whatever /aspect you like, e.g.

/need
/offer
/intention
/like

This simple addition enables people to define structured data
really easily, e.g.

/intention Take a break somewhere warm.

/price £14.00

/need Some advice on sound setup for recording voice-overs.

In fact, much of the functionality of the site is implemented by
using /aspects behind the scenes. For example, when you trust
someone for a specific space, you're actually doing something
like::

/trust <#literature> @sbp

The <about> can be left out normally. By default, it's assumed
that you are defining an /aspect about the space you're sending
the message to, i.e. if I send a message to #literature, it can
just be:

/trust @sbp

You can define aspects about #spaces, @users, http/https links
and even specific messages! You can refer to a specific message
by using the special amp link style, e.g.

~tav/2491

This is constructed by using the tilde sign followed by the
username of the sender and the unique numeric id associated with
each message.

But since numbers are not necessarily easy for most of us to
remember, you can also create "named links" which can point to
a specific message, e.g.

~tav/some-article -> ~tav/2491

Not only does this make it easier to remember, but since named
links can be updated, you can use it to point to updated content
(i.e. another message) in the future.

This is complimentary to the fact that messages are versioned
and you can redirect them to new versions. You can quickly
correct typos in your previous message with a new one saying::

s/complimentary/complementary/

Instead of sending a message with that text, it will instead
send a message with the fixed-up content of your previous one
and set a redirect on your previous message to the new one.

Now, as you might have guessed, you can also refer to named
links in your messages and define /aspects about them, e.g.

/intention <~tav/books> read the Abyssinian Moon by @sbp

/location <~lcl/party> 42 Brick Lane, London

For bonus points, you can connect your Facebook and Twitter
accounts and have the option of cross-posting individual
messages to any of those accounts.

Note however that we allow messages to be up to 1,000 bytes long
which is compatible with Facebook but not with the 140 character
limit on Twitter.

Connecting your external social network accounts also enables
you to see those on the site who you're already connected to on
Twitter and Facebook to further help with discovering people.

And when browsing, in addition to the usual message view, you
can also render the messages using custom views:

* calendar view
* map view
* network view
* trend view

The calendar view shows items in an easy to use calendar setting
and also generates output in iCal format so that it can be used
with Google Calendar, etc.

There's also a builtin app which provides an easy way for groups
of people to decide on the best time for everyone to attend an
event, e.g. a meeting.

This app is aware of timezone differences — so confusions over
the right time for international meetings can hopefully be a
thing of the past. It also provides a helpful countdown too,
e.g.

6 hours and 23 minutes left before the meeting.

The map view plots the items on a Google Map and includes a cool
"auto-zoom" functionality, which zooms up/down from your map's
centre to find the level with the adequate number of results.

We also support voice search on the map view which lets you do
searches using voice in some browsers — which can be useful on
your mobile when you're on the move.

The network view shows a pretty network timeline similar to
GitHub's network graph:

* https://github.com/tav/redis/network

The intention behind this view is to enable Confluence by
helping people to see how their intentions are evolving and
aligning around conditions over time.

The trend view shows useful statistics about what's trending
right now. Unlike Google trends, this is personal to you and
those that you trust and is also used to prioritise the various
autocompletion fields in the user interface.

You can affect what's trending through your interactions on the
site and you can help the works of others to trend by "amping"
their messages.

This is a bit like retweeting on Twitter, but works a little
better. By amping a message, you combine three actions into one:

* re-send a message into different spaces
* attach your own comment to the message (optional)
* allocate some Pecus to those who created and amped the message

By default, one of your Pecus is allocated to the creator of the
message and 0.5 Pecus are equally split amongst everyone who
amped the message before it reached you.

You can configure this in settings and you can do Pecu payouts
using PayPal to everyone who you've given Pecus to. Note: you
can only payout to users who've connected their PayPal accounts.

You can view the top and trending links in a space for the last
day, week and month. However, to act as a dampener against the
"Top 40" effect, any amping that results in following a link
from this view is not incorporated in the top list.

You can filter spaces using a search mechanism. In fact, the
entire site is built on top of a powerful search system. So when
you look at say #espians, you're actually doing a search for:

* to:#espians

All search results get continuously updated in real-time and a
number of search operators are supported:

* simple keywords, e.g. bradley manning wikileaks
* phrase searches, e.g. "inform 7"
* to: operator, e.g. to:#espians
* from: operator, e.g. from:@tav
* aspect search, e.g. /intention
* aspect comparison search, e.g. /rating ≥ 3

The different operators can be used together and the aspect
comparison search is triggered if any of the following appear
after an /aspect::

is ==
>= ≥ >
<= ≤ <

And the special ``near`` and ``within`` can be used to do
geo-queries, e.g.

café /location near ~tav/brushfield-riva

Or perhaps more specifically::

café /location near:200m ~tav/brushfield-riva

You can also specify ``near me`` which will use the Geolocation
to find out where you are and use that to centre the results
around you.

The search mechanism is clever enough to handle case differences
in various languages. So when you search for "Straße", the
results will also include messages with "strasse".

And on a related note, there is extensive support for Unicode.
So you can write messages in whatever language you want and even
have usernames in foreign scripts, e.g. @建国, @வாகீசன், etc.

You can also specify which languages you know and then messages
in other languages will be automatically translated into your
primary language for you!

And, finally, a key feature of the site is that it's extremely
"programmable". This is enabled by the "rich text" that you can
attach to your messages.

By default, any rich text is presumed to be a simple paste that
you want to share with others. But if you start the text with a
#! shebang line then it behaves quite differently.

The simplest one which does nothing to your content is::

#!raw

You can also include code in various programming languages and
have them syntax highlighted, e.g.

#!code ruby

But the one that people are most likely to use is::

#!text

This means you can write articles using the a subset of the
relatively popular Markdown syntax which also allows you to
embed HTML directly into the text:

* http://daringfireball.net/projects/markdown/syntax

You can use most HTML tags and even a fair amount of CSS styles
to present your content as you want to. You can create links
like in messages using [[double square bracket]] syntax, e.g.

[[~tav/some-article]]

[[http://www.youtube.com/watch?v=WL5ud5_K-GY]]

[[#espians]]

And even define custom link text, e.g.

[[~tav/some-article | info about Ampify]]

Mathematicians will be pleased to know that they'll be able to
embed equations using TeX/LaTeX formats and it'll be rendered
correctly, e.g.

<div class="math">
\[P(E) = {n \choose k} p^k (1-p)^{n-k} \]
</div>

There's also super cool support for {{transclusion}} to include
content from other messages:

* http://en.wikipedia.org/wiki/Transclusion

For example, to create a summary of an event, you could simply
include the various messages with explanatory text, e.g.

#!text

The event started with John giving an introduction:

Nadine chipped in with a very insightful comment:

The messages inside the {{curly brackets}} will be rendered as
they normally get rendered. You can render just the ``content``
or any attached ``value`` with, e.g.

You can also include specific sections from other texts, e.g.

Or even specific lines, e.g.

{{~tav/some-article#L12-20}

Note that in these cases it'll be implicitly using the attached
text ``value``. In fact, behind the scenes, the above is being
rewritten into a service call::

And the plain::

Is actually a shortcut for the service call::

There are a fair number of builtin services, e.g.

You can mix them together by enclosing them in parentheses or
putting them into a piped stream, e.g.

You can also use a markdown text as a template for another text,
e.g. let's define a ~tav/blog-template::

#!text
----
title: Default Title
value: Blog Post Content
----

<h1>{{$title}}</h1>
<div>
{{$value}}
</div>

This makes use of ``$variables``, which can be passed in via
service calls explicitly, e.g.

If no variables are passed in, then it will try to use any
default metadata found inside the triple-dashed section, i.e.

---
title: Default Title
value: Blog Post Content
---

The template can also be used by another text explicitly, e.g.

#!tav/blog-template title: $this.title value: $this.value
---
title: 7 Days of Templates
---

On the first day ...

Or even simpler by using the utility ``template`` service which
will pass the variables in automatically::

#!template ~tav/blog-template
---
title: 7 Days of Templates
---

On the first day ...

You can also define your own services using the beautiful
CoffeeScript syntax:

* http://jashkenas.github.com/coffee-script/

For example, to define a service that counts the number of words
in a message::

#!coffee

main = (message) ->
message.content.length

You can also call other services using the builtin ``call``
function, e.g.

call("~thruflo/count ~sbp/abyssinian-moon")

There are a number of other builtins, e.g.

open("http://inamidst.com/stuff/sappho/").read()

memcache.set("rate", 12.12)

send(message)

getUser()

Services using the more dangerous builtins, e.g. ``send`` which
sends a message on the user's behalf, would need to first get an
appropriate access key from the user.

Services can also be directly called from the message input box
by prefixing the command with a full stop (period), e.g.

.convert ¥23 to: GBP

There's also the builtin ``help`` service which displays some
help info for the given service::

.help convert

The help service also has a minimal shortcut::

? convert

The output of a service can be formatted differently when it's
called via an {{include}} and when it's called from the message
input box.

This is determined by the values for the ``/include-formatter``
and ``/command-formatter`` aspects respectively. You can point
these aspects at other services or templates.

Some services may not have any display to format though, e.g.
the ``.pause`` service which simply pauses what's currently
playing on the builtin media player.

People can also register services to run when specific patterns
are matched, e.g. you might want to have an SMS sent to you when
someone mentions "Milking". This is called placing a sensor.

There's also a comprehensive API using the JSON format which
other apps can use to integrate with the site. I've even come up
with an OAuth 3.0 spec using hashcash to make this very easy:

* http://tav.espians.com/oauth-3.0-the-sane-and-simple-way-to-do-it.html

Now, all of this functionality is likely not going to be used by
the majority of users, and the site will be more than useful for
them without having to use these features.

But creating services and templates should hopefully be easy
enough for the class of users who are not yet programmers but
are comfortable making Excel spreadsheets and installing
WordPress plugins.

For security, all interactions with the site happens over HTTPS,
i.e. it's encrypted. This turned out to be quite a challenge as
most APIs, e.g. Flickr, don't support HTTPS.

But, on the flip side, everyone can now use the platform without
having to worry about attacks like Firesheep:

* http://codebutler.com/firesheep

All image/form references except for those to trusted sites like
Google Charts are also rewritten and proxied as part of the
broader sanitation of user input against XSS attacks.

The whole site needs JavaScript to run, but the features are
progressively enhanced and thus will work well on everything
from the ancient IE 6 to modern iPhone/Android browsers.

And, to finish with, all user accounts have a usage quota in
terms of the number of controlled spaces and sensors they can
create and the total size (in gigabytes) of their messages and
attached data.

Let me know what you think. Thanks!

--
love, tav

plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

tav

unread,

Dec 21, 2010, 9:47:13 AM12/21/10

to openk...@googlegroups.com, Espians, Espians

In relation to the question of using StatusNet/OStatus, the short
answer is no — the data models between what I'm proposing and what's
offered by them are fundamentally different structures.

In effect I'm proposing a hybrid between twitter, wikis and irc with
rich programmability — whereas their work is more of an extension of
(micro) blogging.

> I'm curious whether you've looked at Status.Net (best known as the software
> for Identi.ca) as the platform for this. It's probably much friendlier,
> being open source, and already has some very superior features. They've
> also pioneered the open protocol, OStatus, for interoperability.

I've been following StatusNet since back in the day when it was called
Laconica. And likewise with OStatus from back when it was
OpenMicroBlogging. So I think I've got a fairly good understanding of
the pros and cons of it all.

> groups do a simpler version of what you describe: "the ability to trust
> people for specific spaces (Trust Maps)" - in this case it's just the
> ability to follow specific spaces. Mostly that works very well, and it's
> just a matter of using e.g. !green for the green group instead of #green.
> archives are permanent - you can access all past posts.
> "In context" shows a threaded conversation

The notion of groups (bangtags) on StatusNet is a great example of the
distinction between the two models.

StatusNet follows the traditional notion of groups — the group as a
fixed membership set. So anyone can join a group like !London and then
send messages to all other members of that group.

So if spammers (or just people you don't care to hear from) join a
group that you're part of and start sending messages then you have to
either put up with the noise or leave the group.

The needless distinction between #hashtags (tags) and !bangtags
(groups) on StatusNet also results in people being confused and using
a random mixture of the two, e.g.

* http://identi.ca/tag/london
* http://identi.ca/group/london

In contrast, the system I've proposed unifies the notion of tags and
groups under the simpler concept of a #space. And everyone can define
the membership of a space according to their own subjective
perspective.

Personally, I find this to be far more optimal for collaboration as
you can limit your information flows contextually.

And, as you can probably imagine, the underlying data models are quite
different for even just this difference. And it gets even more so when
you consider that, unlike StatusNet, we'll be supporting real-time
flows of these subjective data streams for arbitrary search queries.

And, while some of the ideas, e.g. /aspects-based structured data,
could be built on top of OStatus by introducing yet another format and
Schema on top of Atom, I think it's too complex as it is.

For instance, it'd take at least 10,000 lines of code/libraries to
write a simple, comprehensive client for StatusNet. There are so many
different elements for you to handle, e.g.

* Atom
* Activity Streams
* PubSubHubbub
* Salmon
* Webfinger
* LRDD
* OStatus
* OpenID
* OAuth 1.0
* etc.

And considering the fact that OStatus doesn't even define how basic
features like private messages/streams should be done, putting in so
much effort to support it seems a tad questionable.

In contrast, it'd take less than 300 lines to write a comprehensive
client for what I'm proposing using just the standard libraries of
languages like Ruby and Python.

This is still not as elegant as the Facebook API which takes less than
100 lines of code, but it's still a lot simpler and more feature-rich
than what is offered by StatusNet and OStatus.

And I haven't even mentioned the extensibility/programmability of what
I'm proposing — nor how instead of bolting on security as an
afterthought like StatusNet, we build on top of solid
object-capability security principles:

* https://www.cypherpunks.to/erights/talks/thesis/markm-thesis.pdf

In short, the differences between StatusNet and what I'm proposing is
a bit like the difference between FTP and HTTP. Both of them seem
quite similar in that they involve shuffling data between machines,
but the end result is dramatically different.

And I believe that what I'm proposing is far far better for
collaboration than anything built on StatusNet could ever deliver.