Protocols and APIs, not just implementations please

[knarf]

I realise that Diaspora has only just hatched so it is a bit early to
fix protocols and APIs but I hope that in a few months there will be
some documentation on the latter.

As to the 'why' and 'why not just use the original implementation' I
can say a few words: router, embedded, low memory, relatively slow
cpu.

To get something like Diaspora running on a WRT54GL (with the SD mod
for storage) it'd have to be written in a high performance compiled
language - there is no space for hulking interpreters and 'frameworks'
and such.

So please... to give this project a chance beyond the reach of those
who think in terms of 'cool'... don't tie it down to one specific
language implementation.

[Alex Wright]

Couldn't agree more, really really hope we see a spec soon.

Embedded is a really interesting place to go with this. But even without
requirements like that, alternative implementations would be great to
see. Built into other webapps etc, or just in languages / frameworks
people are more comfortable with, or already have investments in. Like a
J2EE or .Net shop for example. Deploying a *nix server with Rails isn't
likely something they want to do either.

[Ben Francis]

On Sep 16, 4:00 pm, knarf <googlecod...@unternet.org> wrote:
> I realise that Diaspora has only just hatched so it is a bit early to
> fix protocols and APIs but I hope that in a few months there will be
> some documentation on the latter.

> ... don't tie it down to one specific
> language implementation.

Perhaps what you're looking for is OStatus? http://ostatus.org

Or are you talking about more than that? Diaspora is a potential
future implementation of OStatus, but maybe there are more protocols &
APIs required too?

Ben

[John Favorite]

An embedded version would be awesome! Think of an oem box at brick and mortar stores. Plug it in to your home network and go. 

[Jarin Udom]

Definitely +1 on this.

[wenjie han]

quite a fair bit of people are F@#$booking on the move with mobile phones

it seems we are expecting each node/user to
>go through the tenuous installation
>>leave their boxes running all day
>>>just to serve their details
>>>> connect with others who want to/are able to do the same
not very compelling

there is much sense in Bug#1 ubuntu honestly

people (even rubyists) put up with the nonsense of Gates and *uckerburg
because they take our hardware/socialnet hostage

cud be wrong
bummer

---

From: John Favorite <john.f...@gmail.com>
To: diaspo...@googlegroups.com
Sent: Thu, September 16, 2010 11:12:54 PM
Subject: Re: Protocols and APIs, not just implementations please

[Alex Wright]

I'm fairly sure that is not the intended usage model. Email users don't have to run their own email server at home, or on their handset, to be able to use email.

[John Favorite]

or have a webhosting service that is free for the user paid by ads or bake sales.

[Marc Byrd]

What's the use case behind the desire for embedded, appliance, etc.?  private, disconnected social networks like Ning?  

Or are we thinking peer-to-peer (like tor, seti at home, bittorrent, etc.), a network of networks, layers?  

My impression is that the goal is to have a single big social network that respects people's privacy - in that case we're probably talking about a server farm, web app stack, service oriented architecture, fault tolerant, HA, etc.  In that case RoR is a perfectly good choice.  MongoDB on the other hand... (zing!  let the #nosql debate begin! ;)

I note that twitter's recent announcement re web app mentioned that they will use their own API's - this is huge for developers, as it means they are on equal footing with twitter, and that if twitter needs something in order to accomplish a new feature, developers should also get it.  In open source world on one hand that's not as important because any one can see, contribute, etc. - However it's a scope issue - having great API's would help contributors focus.  

m

[knarf]

$#%#^ firefox just crashed and ate my whole long reply but here it
goes again...

The use case behind the desire for embedded Diaspora is the following:

John H. Doe opens up the box, lifts out the router and has a good long
look at the user manual. It does not mean much to him but this damn
box is supposed to give him the internet so it bloody well better
work.

He puts the plug in the hole which seems to fit it, plugs the cables
in where they seem to fit and starts his internet. In his internet he
now sees the welcome page for the router asking him if he wants to
share pictures with his friends and such. 'Sure', Joe says and clicks
the button. The thing now asks him his name, password (uhhhh
'password1') and, at the end of the process, gives him a url:

http://johndoe223.diaspora.org/

Using that url he 'magically' is transported to a web page giving him
all the trimming of a social network. He does not know (or care) that
that page is actually generated on his own router. He just puts some
pictures of him and his buddy drinking beer/fishing on the page and
sends the link to those buddies. 'Hey Bubba, look at mah internet'!

Any technology distinguishable from magic is too complex. Let's make
this simple, and let's make it work. Joe and Bubba don't want their
stuff mined by BigEvilZuckerbergCorp and neither do I.

[Atrus]

> Any technology distinguishable from magic is too complex.

Off-Topic, but I love this.

[Eric Alan Solo]

Hypothetically speaking... what if John H. Doe's Diaspora box gets hit
by lightning?
Is the info of a seed stored anywhere else than on it's server?

I'm just curious, because if that data is not anywhere else then I'd
recommend he use a public seed instead, maybe one set up by a company
he trusts (who can you trust really? but I do trust google with my
emails so its what happens nowdays)

[knarf]

On Sep 16, 10:40 pm, Eric Alan Solo <sambarino1...@gmail.com> wrote:
> Hypothetically speaking... what if John H. Doe's Diaspora box gets hit
> by lightning?
> Is the info of a seed stored anywhere else than on it's server?

Well, it could be replicated, optionally in encrypted form if Joe only
wants his own server to be able to serve his content. As long as Joe's
server is around to seed the content the - potentially encrypted -
replicated bits stay alive on the net, comparable to the way Freenet
works.If his server gets hit by lightning he can rebuild it using
those replicated bits. If he waits to long with the rebuilt he might
find out that his content has expired. As to whether replication
should cover only textual content or both text as well as media files
is something for another discussion. Maybe the whole replication idea
is over the top? Maybe Joe should be told to make backups, just like
he should do for his email if he wants to keep that around after
lightning takes out his internet?

As to trusting a company with his data, well... that is up to Joe.
Having one form of Diaspora does not exclude having others... If Joe'd
rather hand his data to ${company} that is his business, not mine. I
don't trust Google with my email so I run my own server(s). Joe
probably does not even think in terms of trust, he just wants to be
able to send and receive mail.

As long as the protocols allow for it it should be possible to have my
router-based instance talk with Joe's and Megabuxx Inc's and ... etc.
I have backups. Megabuxx might, or might not - remember the Sidekick.
You trust Google? They have lost mail as well...

[Eric Alan Solo]

Maybe the company selling these boxes could sell an automated back-up
service too so that if u need to replace the box they can give you one
with your latest backup already on it... hmm...

And yeah, I do trust google... they make life way too easy for me :)

[Matěj Cepl]

Dne 17.9.2010 00:46, Eric Alan Solo napsal(a):

> Maybe the company selling these boxes could sell an automated back-up
> service too so that if u need to replace the box they can give you one
> with your latest backup already on it... hmm...

Of course, if the backup is encrypted BEFORE sending data to the backup
server (in the style of Firefox Sync or I believe Carbonite ...
listening to TWiT podcasts too much :)).

Matěj

[Ori Pekelman]

Interesting that no body stayed on topic.

This is not "just an ostatus" implementation. This is a whole stack
and I have yet to see it be described. You can not do security last,
and you can not do privacy last. Without a detailed description of the
underlining protocols (or more precisely how the existing protocols
are stacked together and used) we can not know what are the system
properties of Diaspora.

Is it really decentralized? Does it really protect privacy? What
information does it leak by default? What does it divulge by essence?
For example does traversing the graph forcefully means no anonymity
for the "browser". Are there any GUIDs (URLs, unique public keys)
publicly exposed, are the same identifiers exposed to different
parties? Are they necessary?
These are not stuff you resolve through bugfixes. These are protocol
design issues, and I have not succeeded yet in finding where are those
described.

You can not hope a secure protocol will organically form itself. So,
where are the diagrams?

[shadowfirebird]

I think Ori has made a damn good point. I'm nervous about the lack
of documentation too.

Mostly I'm nervous because I'm not a security expert; because I don't
know what I'm talking about.

But once built, I'm going to trust secrets to this thing
(presumably).

[Alex Andrews]

People - a bit of perspective - this isn't going to be in Beta before
March or so next year...

Alex

[shadowfirebird]

Well, that's true. But it's been opened to our contributions now.
Can we really help, if we don't know the underlying principals of the
software we are supposed to be collaborating on?

On Sep 17, 5:14 pm, Alex Andrews <awgandr...@gmail.com> wrote:
> People - a bit of perspective - this isn't going to be in Beta before
> March or so next year...
>
> Alex
>

[Alex Andrews]

I'll agree that its under documented but so is 90% of software in the
wild no? Hackers just don't like to do documentation...

Alex

[John Favorite]

I have heard rumors some do like to.. yet to meet one though :)

[Ori Pekelman]

Oh please, please stop with the pre-alpha argument. And listen to the
counter one: THIS is not about software development, this is about
protocols. You don't discover protocols, the way this distributed
thing works through atomic commits. I have no problems with this
specific piece of pre alpha software having a billion XSS exploits.
This can be easily corrected by the open source process. But the
underlying ideas, the way privacy is thought and implemented, are not
subject to incremental enhancement. This is either well thought
before, or never, ever happens.

People Like Eran Lahav-Hammer, Blaine Cook, Henry Story have put a lot
of work in thinking about this. More work is needed, but naively
thinking, that a patch here and a patch there we will get at a
"privacy aware" system... well. Won't happen.

I might not run a Diaspora server in Rails (Though I love Rails3 and
Mongo). I might want to run my thing on en Erlangish/nodejsque
implementation. So this is not the specific code that bothers me. But
what are the ideas behind. I have not seen the authors of Diaspora
speak clearly to that. And of what I could understand this seems like
really half-baked. The intentions are good. I love this project. I
want to kill Facebook like anyone else. But they need to go back to
the drawing board, and talk a bit about how this works as a system.
When, where under what conditions is information leaked. Can we safely
transfer public keys. Can we discover other people. My greatest fear
now is that the community will patch the gaping security holes. That
through the amazing public relations work these guys did, it will
succeed. And we will still not get our lives back. This is very, very
important. And must be done right. I am willing to wait until november
even, if it means I get: Anonymous traversal and no Information
leakage in the discovery phases. Revocability. The ability to lie.
If these are not in the building blocks these will never be in.

And lying, contrary to common belief is a feature, not a bug.

On Sep 17, 6:24 pm, Alex Andrews <awgandr...@gmail.com> wrote:
> I'll agree that its under documented but so is 90% of software in the
> wild no? Hackers just don't like to do documentation...
>
> Alex
>

[Tristan Sloughter]

I think I can safely say +1 to everything Ori says on an thread :)

[Singpolyma]

On Sep 16, 11:10 am, Ben Francis <b...@tola.me.uk> wrote:
> On Sep 16, 4:00 pm, knarf <googlecod...@unternet.org> wrote:
>
> > I realise that Diaspora has only just hatched so it is a bit early to
> > fix protocols and APIs but I hope that in a few months there will be
> > some documentation on the latter.
>

> Perhaps what you're looking for is OStatus?http://ostatus.org

Diaspora will almost definitely need some stuff not in OStatus (like
OpenPGP for crypto stuff, probably, and SMTP for private messaging)
but OStatus is definitely the right protocol to use for 80% of what
they have now.

[Singpolyma]

On Sep 16, 11:53 am, Alex Wright <alexkwri...@gmail.com> wrote:
>   I'm fairly sure that is not the intended usage model. Email users
> don't have to run their own email server at home, or on their handset,
> to be able to use email.

Right, don't have to, but can. Can is very key.

Should, maybe, even. But that comes later.

[Russell Whitaker]

Off-topic commentary follows:

It's a misquote variant on someone else's .sig line from around 2001,
seen on linux-misc:

"Any technology distinguishable from magic is insufficiently complex," which in
turn is taken from Clarke's (3rd) Law, about which this guy has choice words to
say with which I largely agree:

http://unintentional-irony.blogspot.com/2007/07/any-sufficiently-advanced-technology-is.html

I'm posting this only because it now becomes a matter of record, searchable, and
a ripstop in the fabric of attribution. No one does their homework nowadays...

--
Russell Whitaker
http://twitter.com/OrthoNormalRuss

[codethief]

On 17 Sep., 22:19, Tristan Sloughter <tristan.slough...@gmail.com>
wrote:

> I think I can safely say +1 to everything Ori says on an thread :)

Absolutely.
And count my +1 for knarf as well, please.

The ironic thing is:
I started development of an application for distributed social
networking, too, but designing the protocols all on my own without
receiving constant feedback / having a second opinion was no fun.
Then, when I heard about Diaspora I pretty much halted the project
because I thought:

"These guys are cs students, they are a group of four, they certainly
know better than I do how to approach such a project." (I'm
programming experienced but don't have a degree + know not much about
CS theory)

And now I don't see any documentation. I can't deny that I am more
than just a bit disappointed. Anyway, I won't give up my hopes just
yet due to the fact that they have the money to actually force it and
make Diaspora a true alternative.

[Jkob]

Oh thanks, you are totally right. Many people around here, who don't
understand your points.

The overall concept is missing at all. And the question is, what to do
if there is no one. And this is likely the case. Normally you have the
concept _before_ you start coding. To open the source code to the
community but to keep the concept secret would be nonsense.

The good thing now is that a lot of interested and capable people are
gathered here, but someone will need to make a start.

Regards,
Jakob

[scwh]

fully agree on the need for this to be more protocol rather than ui/
implementation focused. ideally the rails + mongo setup functions as
an example implementation.

> Anonymous traversal and no Information leakage in the discovery phases. Revocability.

i naively understand what you mean, but any links you have that
further explain these concepts would be helpful

> The ability to lie.
> If these are not in the building blocks these will never be in.
>
> And lying, contrary to common belief is a feature, not a bug.

do you mean in terms of identity or content or both? can you elaborate
on why that's desirable?

[wenjie han]

yea the whole Rails3+Mongo must be like a 'reference implementation' - java parlance
otherwise this is much like a project to develop an open source client for running a proprietary protocol

btw what harm is it if there were alternative implementations on the Diaspora API (php, nodejs etc...)
or different user interfaces - thick clients...appliances... mobile clients....
i can only see this as a greater reach for the eventual D. network.

based on vague notion of privacy and F@#$book hate?
we're not asking for some long essay-thesis
salient points or even a deck of slides can help

---

From: scwh <stephen...@gmail.com>
To: diaspora-dev <diaspo...@googlegroups.com>
Sent: Sat, September 18, 2010 9:12:24 AM

Subject: Re: Protocols and APIs, not just implementations please

[codethief]

Don't get me wrong but I feel that the lack of a protocol is actually
heavily slowing down the process. I mean there're hundreds of
developers around here who are ready to get going on this
distributable network thing yet there's still no thorough concept to
hold on. How much faster could we be if there was a plan? And no
offense intended to the Diaspora guys, but with almost absolute
certainty many people here are much more competent and experienced in
terms of (secure) protocol development and project planning.

On a side note: I don't think that the HTT protocol should be
(mis-)used for everything. I think a separate protocol dedicated to
the demands of such a project would fit much better (and also be
simpler in terms of implementation). Also, already existing protocols
like XMPP (Jabber) should be considered as basis.
I for myself am very ready to draft an own protocol with you guys.
Anyone who'd like to jump into the boat? (This is not meant as
competition. It's just that I'd like to get started after having
waited and neglected my original plans for months. I know that
splitting up should be avoided but it's the only perspective I'm
seeing at the moment.)

[codethief]

I should clarify my previous message:
As this is not intended as a call to create a competing project I
would very much like to build something upon Diaspora('s protocol). I
just don't think it (the protocol) should be completely up to Daniel,
Maxwell, Raphael and Ilya. As other intelligent people noted elsewhere
protocols don't grow organically.

[Sainath]

I pretty much agree with everything said, most of all I find lack of
participation from the original developers in these discussions
concerning.

--
-Sainath

[shadowfirebird]

I've been trying to give them the benefit of the doubt. But I agree:
the lack of a published design is scary; the lack of team presense
here; the lack of any sort of distributed network in the code(?) - all
raise my 'uh-oh' factor.

And, is it just me? The contract they want you to fill in before
they'll pull your changes is scary? I mean, I've never really done
serious FOSS work before. Maybe it's standard...

On Sep 21, 4:16 am, Sainath <sainathdre...@gmail.com> wrote:
> I pretty much agree with everything said, most of all I find lack of
> participation from the original developers in these discussions
> concerning.
>

[Jakob Keres]

On 09/21/10 08:31, shadowfirebird wrote:
> I've been trying to give them the benefit of the doubt. But I agree:
> the lack of a published design is scary; the lack of team presense
> here; the lack of any sort of distributed network in the code(?) - all
> raise my 'uh-oh' factor.

I also wonder about how many commits since the "development" has been
"opened" to the community concern the most unimportant things of
software in pre-alpha state: the layout (UI) and translations

> And, is it just me? The contract they want you to fill in before
> they'll pull your changes is scary? I mean, I've never really done
> serious FOSS work before. Maybe it's standard...

You are not the first one wondering about this contract.

[shadowfirebird]

Don't want to derail this interesting topic, moving my response to a
new thread...

[distributed]

+1 here.

Frankly, I am quite disappointed by the open source release of
Diaspora. I was looking for documentation first and then, as I didn't
find any, manifestation of communication concepts in the Diaspora
code, but I found none. I wasn't expecting total and full
documentation, but more something along the lines of "if you want to
be friends with someone on another seed, then your seed does a POST
request to the other seed, port 1234, with parameters ..... encoded as
JSON and the reply is the following fields .... encoded as JSON" and
some high level system overview like docs laying out how and when
seeds are expected to communicate. Or even what is the vision of how
and about what seeds communication. But not even something like this
is present. Honestly, I don't know Ruby nor Rails, but functions
filled with "# DO SOMETHING" do not look like the implementation of
something meaningful.

As Jakob Keres noted, most commits I see on Github are addressing
issues which are, at least for me and at the moment, totally
unimportant like UI changes and translations. Also The team is
absolutely not present on the list. They want to build the distributed
Facebook killer, but there seems to be no distributed system. I
understand that the team might be overwhelmed by the attention they
receive after their September 15 release of the code. But I think that
they should get present _now_ and comment on how they want to proceed.
If they don't have any communication concepts, then they should tell
so. If they exist, they should publish their ideas. I find the lack of
team involvement baffling. If this continues like this, I will
absolutely lose interest in Diaspora and I guess so will many others.

[Ori Pekelman]

+1 hear hear.

In order to be constructive I would propose as a first step to try to
help them by reverse engineering their stack and documenting it over
at http://github.com/diaspora/diaspora/wiki/Concept-idea (which
currently is shamefully naive, simplistic, and .. empty )

I have links to basically all the relevant protocols and
implementations over at http://leavediaspora.com

[distributed]

I will dig in and try to find out something. I will post my findings,
if any, to the concepts page.

On Sep 21, 12:34 pm, Ori Pekelman <o...@af83.com> wrote:
> +1 hear hear.
>
> In order to be constructive I would propose as a first step to try to
> help them by reverse engineering their stack and documenting it over
> athttp://github.com/diaspora/diaspora/wiki/Concept-idea(which
> currently is shamefully naive, simplistic, and .. empty )
>
> I have links to basically all the relevant protocols and

> implementations over athttp://leavediaspora.com

[Raphael Sofaer]

Hello,

There isn't an unchanging protocol being used in Diaspora right now.  We're still figuring out what we need to have, so publishing the current state of the software as v0 of a protocol seems like a recipe for incompatibility problems.  I've put up a summary at http://github.com/diaspora/diaspora/wiki/Message-passing-in-Diaspora

Suggestions are welcome.

[shadowfirebird]

Many thanks. Now I feel like maybe we can be of some help to you!

On Sep 21, 6:40 pm, Raphael Sofaer <raph...@joindiaspora.com> wrote:
> Hello,
>
> There isn't an unchanging protocol being used in Diaspora right now.  We're
> still figuring out what we need to have, so publishing the current state of
> the software as v0 of a protocol seems like a recipe for incompatibility

> problems.  I've put up a summary athttp://github.com/diaspora/diaspora/wiki/Message-passing-in-Diaspora
>
> Suggestions are welcome.

[Simon B.]

Thanks for the wikipage. On June 16th I made up an example to show why
we want to have encryption on message passing: http://pix.zash.se/i/md.png
(link stays 1 month, free to use for any purpose)
While the example says Diaspora* there is no guarantee that it is
anything like how the alpha release works. But perhaps someone in-the-
know can comment on my diagram?

[Ori Pekelman]

This is great! Will read, analyse, and hope be able to bring 2 cents
to the table

On Sep 21, 7:40 pm, Raphael Sofaer <raph...@joindiaspora.com> wrote:
> Hello,
>
> There isn't an unchanging protocol being used in Diaspora right now.  We're
> still figuring out what we need to have, so publishing the current state of
> the software as v0 of a protocol seems like a recipe for incompatibility

> problems.  I've put up a summary athttp://github.com/diaspora/diaspora/wiki/Message-passing-in-Diaspora
>
> Suggestions are welcome.

[shadowfirebird]

In my carzy, half-arsed "design" this can't happen, because Alice can
get the photos anyway -- probably from Bob's server, and he can't
fiddle with them easily because he'll change the signatures. Also,
all photos are probably signed, so the only way Bob could really do
this is to run a man-in-the-middle attack and intercept the keys.
He's need to change the code to do that in Diaspora.