Doozer - a better "Apache Zookeeper" for Useraddress.net ?
47 views
Skip to first unread message
Thad Guidry
Jul 29, 2012, 1:48:42 AM7/29/12
to unho...@googlegroups.com
The ideas and code are aligned quite nicely on this project. Don't you think ?
Interesting, none the less, as a potential Useraddress.net backend ?
-Thad
http://www.freebase.com/view/en/thad_guidry
Michiel de Jong
Jul 29, 2012, 4:53:13 AM7/29/12
to unho...@googlegroups.com
cool, yeah if we want to run UserAddress in production we would need
things like that. Right now it's just one nodejs process that keeps
all data in memory and dumps it to a JSON document every time data is
added, so very unscalable. :)
For me, the main questions for UserAddress right now are:
- do we agree we need this, and would people (i.e. Meute but also
StatusNet, Diaspora, Friendica, BuddyCloud) use it?
- who can maintain it (both code maintenance and server maintenance,
probably 1 full-time person)
- how does this person get paid
- the indexing data we will collect over time will become a valuable
social knowledge graph. Can we publish that data set in the public
domain as Open Data, or would that lead to problems? I don't care
about influencing Facebook's stock price as advertisers moves from
their platform to fedsocweb, but making information about people more
easily accessible can lead to problems like spam. Also we would have
to warn users that any URL they put in may end up being indexed and
listed, because they may expect their input to stay within the context
of the search session. If we decide not to publish it, then what do we
do with it to make sure this valuable data is owned by The People?
Once we answer those questions, the person who would run this service
may choose to use doozerd for part of the architecture, yes! :)
PS if anyone is reading this and would like to be responsible for
running and maintaining UserAddress.net in production, let me know! :)
things like that. Right now it's just one nodejs process that keeps
all data in memory and dumps it to a JSON document every time data is
added, so very unscalable. :)
For me, the main questions for UserAddress right now are:
- do we agree we need this, and would people (i.e. Meute but also
StatusNet, Diaspora, Friendica, BuddyCloud) use it?
- who can maintain it (both code maintenance and server maintenance,
probably 1 full-time person)
- how does this person get paid
- the indexing data we will collect over time will become a valuable
social knowledge graph. Can we publish that data set in the public
domain as Open Data, or would that lead to problems? I don't care
about influencing Facebook's stock price as advertisers moves from
their platform to fedsocweb, but making information about people more
easily accessible can lead to problems like spam. Also we would have
to warn users that any URL they put in may end up being indexed and
listed, because they may expect their input to stay within the context
of the search session. If we decide not to publish it, then what do we
do with it to make sure this valuable data is owned by The People?
Once we answer those questions, the person who would run this service
may choose to use doozerd for part of the architecture, yes! :)
PS if anyone is reading this and would like to be responsible for
running and maintaining UserAddress.net in production, let me know! :)
elf Pavlik
Jul 29, 2012, 5:21:46 AM7/29/12
to unhosted, Melvin Carvalho
Excerpts from Michiel de Jong's message of 2012-07-29 08:53:13 +0000:
http://openprofile.com/
? :)
> cool, yeah if we want to run UserAddress in production we would need
> things like that. Right now it's just one nodejs process that keeps
> all data in memory and dumps it to a JSON document every time data is
> added, so very unscalable. :)
>
> For me, the main questions for UserAddress right now are:
>
> - do we agree we need this, and would people (i.e. Meute but also
> StatusNet, Diaspora, Friendica, BuddyCloud) use it?
> - who can maintain it (both code maintenance and server maintenance,
> probably 1 full-time person)
> - how does this person get paid
> - the indexing data we will collect over time will become a valuable
> social knowledge graph. Can we publish that data set in the public
> domain as Open Data, or would that lead to problems? I don't care
> about influencing Facebook's stock price as advertisers moves from
> their platform to fedsocweb, but making information about people more
> easily accessible can lead to problems like spam. Also we would have
> to warn users that any URL they put in may end up being indexed and
> listed, because they may expect their input to stay within the context
> of the search session. If we decide not to publish it, then what do we
> do with it to make sure this valuable data is owned by The People?
>
> Once we answer those questions, the person who would run this service
> may choose to use doozerd for part of the architecture, yes! :)
>
> PS if anyone is reading this and would like to be responsible for
> running and maintaining UserAddress.net in production, let me know! :)
>
this one seams related to some extent:
> things like that. Right now it's just one nodejs process that keeps
> all data in memory and dumps it to a JSON document every time data is
> added, so very unscalable. :)
>
> For me, the main questions for UserAddress right now are:
>
> - do we agree we need this, and would people (i.e. Meute but also
> StatusNet, Diaspora, Friendica, BuddyCloud) use it?
> - who can maintain it (both code maintenance and server maintenance,
> probably 1 full-time person)
> - how does this person get paid
> - the indexing data we will collect over time will become a valuable
> social knowledge graph. Can we publish that data set in the public
> domain as Open Data, or would that lead to problems? I don't care
> about influencing Facebook's stock price as advertisers moves from
> their platform to fedsocweb, but making information about people more
> easily accessible can lead to problems like spam. Also we would have
> to warn users that any URL they put in may end up being indexed and
> listed, because they may expect their input to stay within the context
> of the search session. If we decide not to publish it, then what do we
> do with it to make sure this valuable data is owned by The People?
>
> Once we answer those questions, the person who would run this service
> may choose to use doozerd for part of the architecture, yes! :)
>
> PS if anyone is reading this and would like to be responsible for
> running and maintaining UserAddress.net in production, let me know! :)
>
http://openprofile.com/
? :)
Michiel de Jong
Jul 29, 2012, 5:46:07 AM7/29/12
to unho...@googlegroups.com, Melvin Carvalho
On Sun, Jul 29, 2012 at 11:21 AM, elf Pavlik
<perpetua...@wwelves.org> wrote:
> this one seams related to some extent:
> http://openprofile.com/
> ? :)
yes and also to http://openfollow.net
<perpetua...@wwelves.org> wrote:
> this one seams related to some extent:
> http://openprofile.com/
> ? :)
what makes UserAddress different from both of those though is that it
creates a text index on full names and returns as-you-type results
based on that. Afaics neither openprofile nor openfollow cache search
results, let alone index them.
Another difference between UserAddress/OpenFollow on the one hand and
OpenProfile on the other is that the first two are meant to be used as
an API that you can integrate into other app, although i guess
OpenProfile could easily offer an API for third-party integration (and
maybe they do this already but i just didn't find it). Also if i put
my gmail address into OpenProfile it doesn't find anything (except
people tweeting about gmail), so it seems you're not supposed to use
it that way, although i couldn't find documentation about that.
Melvin Carvalho
Jul 29, 2012, 6:14:57 AM7/29/12
to unho...@googlegroups.com
one of mine ... but i never really got time to maintin it ... lol
idea is to be a meta search engine
i can hook it up to useraddress.net at some stage if you allow search by URI
some comments
which uri does this map to?
as above
mailto: ?
- https://api.twitter.com/1/users/show.json?screen_name=michielbdejong
- https://graph.facebook.com/dejong.michiel
documents
which URI?
its kind of hard to understand, i dont know if im looking at documents, data, simple strings or URIs
i sometimes think it would have been easier to understand if things were called foo.doc rather than foo.html
you might want to grab some of the google data if it's still there
? :)
Michiel de Jong
Jul 29, 2012, 6:28:32 AM7/29/12
to unho...@googlegroups.com
On Sun, Jul 29, 2012 at 12:14 PM, Melvin Carvalho
<melvinc...@gmail.com> wrote:
> michiel...@identi.ca
>
> which uri does this map to?
see the 'The user address query format' section on
<melvinc...@gmail.com> wrote:
> michiel...@identi.ca
>
> which uri does this map to?
http://useraddress.net/ - these are all mapped to URLs where we can
retrieve documents about them, namely
https://{host}/.well-known/host-meta?resource=acct:{user}@{host}
> mailto: ?
no, see that same section - URIs in general are not necessarily
supported, only URLs that return a (redirect to) a http- or https-
retrievable document whose content the master parser understands.
> its kind of hard to understand, i dont know if im looking at documents,
> data, simple strings or URIs
document describing the user. If you give UserAddress a URI that is
not a URL, it will have no way to discover information.
> you might want to grab some of the google data if it's still there
https://developers.google.com/social-graph/
Melvin Carvalho
Jul 29, 2012, 8:17:34 AM7/29/12
to unho...@googlegroups.com
On 29 July 2012 12:28, Michiel de Jong <mic...@unhosted.org> wrote:
On Sun, Jul 29, 2012 at 12:14 PM, Melvin Carvalhosee the 'The user address query format' section on
<melvinc...@gmail.com> wrote:
> michiel...@identi.ca
>
> which uri does this map to?
http://useraddress.net/ - these are all mapped to URLs where we can
retrieve documents about them, namely
https://{host}/.well-known/host-meta?resource=acct:{user}@{host}
> mailto: ?
no, see that same section - URIs in general are not necessarily
supported, only URLs that return a (redirect to) a http- or https-
retrievable document whose content the master parser understands.
A user address can take one of two formats: user@host, or the URL of a
> its kind of hard to understand, i dont know if im looking at documents,
> data, simple strings or URIs
document describing the user. If you give UserAddress a URI that is
not a URL, it will have no way to discover information.
Thanks for clarifying. Exclusion of URIs seems to be a strange design decision, imho.
which google data where? you mean Social Graph API?
> you might want to grab some of the google data if it's still there
https://developers.google.com/social-graph/
Pity seems they've shut it off permanently now. Perhaps someone in google might have a dump of the data to share with FSW.
Melvin Carvalho
Jul 29, 2012, 8:23:37 AM7/29/12
to unho...@googlegroups.com
On 29 July 2012 12:28, Michiel de Jong <mic...@unhosted.org> wrote:
On Sun, Jul 29, 2012 at 12:14 PM, Melvin Carvalhosee the 'The user address query format' section on
<melvinc...@gmail.com> wrote:
> michiel...@identi.ca
>
> which uri does this map to?
http://useraddress.net/ - these are all mapped to URLs where we can
retrieve documents about them, namely
https://{host}/.well-known/host-meta?resource=acct:{user}@{host}
> mailto: ?
no, see that same section - URIs in general are not necessarily
supported, only URLs that return a (redirect to) a http- or https-
retrievable document whose content the master parser understands.
A user address can take one of two formats: user@host, or the URL of a
> its kind of hard to understand, i dont know if im looking at documents,
> data, simple strings or URIs
document describing the user. If you give UserAddress a URI that is
not a URL, it will have no way to discover information.
i think what you call a user address is really just a foaf : nick
I could type any of
@michiel
michiel@host
Michiel ( foaf : name )
and a good search engine should be able to do a lookup
you may want to look at uriburner.com too which is a very powerful seach engine with many years of dev time behind it
Reply all
Reply to author
Forward
0 new messages