Open repository distribution system

[Peli]

I feel honored to post the first message in this forum :-)

Based on the discussion here:
http://groups.google.com/group/android-challenge/browse_frm/thread/32a797801e7f8a2f#
I really like the idea of a shared open repository system. The
benefits are obvious: There is no need to upload ones application to
several repositories, but all could be handled transparently, and the
user can benefit from finding the right software from a vast selection
through a single client of choice.

I suppose it is still a far way to get there, but the most important
is to get the interested people together as early as possible.

From the thread above there are:
* JVending: http://code.google.com/p/jvending/
* Droidstor: http://sadko.mobi/droidstor
* HelloAndroid.com: http://www.helloandroid.com/

How would this be related to other systems, like
* bolt: http://code.google.com/p/android-bolt/

Would this be a competing system, or could it work together as well?

Also, how are issues like payment handled if the respository is
distributed over several competing sites?

Peli

[Shane Isbell]

This is actually pretty doable. http://openrdf.org has everything we
need. We can generate and RDF from the provisioning.xml file, use Elmo
to bind to it and then load up the RDF workbench. We can then query
across multiple nodes using SPARQL. This does mean that the client
will need to support RDF.

Shane

On Apr 10, 3:20 pm, Peli <peli0...@googlemail.com> wrote:
> I feel honored to post the first message in this forum :-)
>

> Based on the discussion here:http://groups.google.com/group/android-challenge/browse_frm/thread/32...

[Zach Hobbs]

I'm on board for this also. I had also mentioned to Shane that it's
also my goal to be able to include discussions, ratings and
screenshots for the application listings as well. Maybe these
additional features are outside the scope of JVendor, and may also add
a lot of complications because of different user accounts with
different securities with different repositories, etc...but maybe we
can come up with a solution (via plugin architecture or separate spec
to piggyback on JVendor, etc)

But either way if we get off on the right foot collaborating instead
of competing it'll save everyone a lot of time and in the long run
will benefit the Android community.

I'm pretty busy with my ADC projects so I haven't had a chance to
really take a look at JVendor, Bolt, etc., but after Monday I'll have
a little more free time.

Also, Peli you asked about how to handle payment options. I think
this would be difficult, especially with multiple repositories
offering the same commercial application (who gets the cut?). So let
me know your guys thoughts on this aspect.

--

Zach Hobbs
HelloAndroid.com
Android OS news, tutorials, downloads

On Apr 10, 6:20 pm, Peli <peli0...@googlemail.com> wrote:
> I feel honored to post the first message in this forum :-)
>

> Based on the discussion here:http://groups.google.com/group/android-challenge/browse_frm/thread/32...

[Shane Isbell]

If there's three people willing to colloborate to add rating and comments to JV and the SAMce client, I'm on board as well. No point in duplicating it. We will need to sit down and define what the catalog properties should be in the provisioning.xml and how to push (or pull) updates to JV. I'm also pretty busy with the contest, so lets sync up after the 14th and figure it out.

 

Shane

[Peli]

Exactly. I'm interested. It could even be a common entry to challenge
II :-)
I'll also have more time after the 14th and a couple of days of
break :-)

Peli

[Carl H.]

I am in as well... we can also use androforge.net

[Shane Isbell]

Man, you need to talk to George, he's been throwing out ideas on how to hook up external services for SVN, mailing lists,etc to slideme. androforge looks perfect for this. George? Going off topic, I know.

 

Shane 

[george_c]

Hi Carl,

We would definitely need to discuss this further as its along our project plans.

I would need also some more time after this late hectic period to get
everything in order so we can discuss options for how repositories
could be hosted, content shared amongst us, %'s etc.

There is indeed an nice 'open model' I have in mind that could work
well. Just need to iron out all the details so we can take this
discussion further.

Carl, if you need server allocation space I can arrange possibly with
our DC. btw: we're managing it all with 'virtual iron'.

All the best, and lets get this moving.. before the big boys come into play ;)

Regards
George

[Peli]

Hi,

We host our project OpenIntents currently on Google code, and I have
to say I got to like it.
Specifically, we use:
* the wiki pages: http://code.google.com/p/openintents/w/list
* issue tracker in grid mode for good overview:
http://code.google.com/p/openintents/issues/list?can=2&mode=grid
* Online browsing of code: http://code.google.com/p/openintents/source/browse
* Online viewing of code changes: http://code.google.com/p/openintents/source/list
* Download page with counter: http://code.google.com/p/openintents/downloads/list
* Google groups for developer discussions: http://groups.google.com/group/openintents

It is ironic that right now when i write this reply, the site is "read-
only for maintenance", but it was online 99.9% during the round I :-)
(maybe they waited with maintenance work for the deadline of the ADC
even?)

Peli

[Shane Isbell]

Before we get off topic too much, let me throw something out:

We form a challenge II entry. Peli (others from openintents?), Zach,
Carl, me (and the other guys from slideme). The end game would be
something where a client (either Android, desktop client, web
interface) can do a distributed search for Android catalog info,
across multiple site. Presumably, we would have slideme site,
HelloAndroid site, Androforge project releases all hooked together. If
other projects want, they can plug right into the distributed repo.

Some specifics:
1) Define the RDF interface to lay over the catalog
2) Build RDF interface (using openrdf) on top of JV. JV will be
reference implementation. Other systems can be built, as long as they
follow the standard RDF.
3) Build android client that somehow handles the RDF distributed
queries. Can either handle SPARQL directly or proxies the query
through a server that knows SPARQL.

(1) and (2) are pretty easy, (3) will be the tough nut to crack.

If we are in general agreement on this, then lets form a project on
Androforge for the client and make the announcements. Thoughts?

Shane

[Shane Isbell]

Carl, I just registered an account on androforge. What's the version control system using? I can't find that info anywhere.

 

Shane

On Sat, Apr 12, 2008 at 11:09 AM, Carl H. <char...@gmail.com> wrote:

[Peli]

> 3) Build android client that somehow handles the RDF distributed
> queries. Can either handle SPARQL directly or proxies the query
> through a server that knows SPARQL.
>
> (1) and (2) are pretty easy, (3) will be the tough nut to crack.

What is the part of (3) that you think will be tougher than the other
2 parts? Talking about a "reference system", one should try to
separate the concepts of the logical handling of the queries from how
the client appears to the end user (i.e. the graphical user
interface).

The core of the client could or should even be implemented to be as
independent as possible - i.e. run also on Java platforms other than
Android. Different developers may then put different "backends" or
"graphical user interfaces" with whatever fancy additional
functionality on top of that, or incorporate the core search and
installation in their existing larger client applications.

I just wonder - should all the logic of the distributed query sit in
the client, or rather in the server? I can't imagine my small phone
connecting to several servers for a single query, but this is exactly
what I expect the servers to handle for me: Servers should cover their
own catalog as well as be able to forward requests to related servers
and send back a useful list of possible applications to the client,
based on my query conditions.

So, I rather see the bulk of the problem on the server-side rather
than on the client side.

Since I'm not an expert on this, i may be completely wrong, so if you
have some web site or paper where the basic mechanism or RDF
distributed queries is explained, that would be very useful for a
better overview how the whole system should work.

Peli

[Peli]

> Some specifics:
> 1) Define the RDF interface to lay over the catalog
> 2) Build RDF interface (using openrdf) on top of JV. JV will be
> reference implementation. Other systems can be built, as long as they
> follow the standard RDF.
> 3) Build android client that somehow handles the RDF distributed
> queries. Can either handle SPARQL directly or proxies the query
> through a server that knows SPARQL.

Now that I looked at http://www.openrdf.org/ and http://www.w3.org/TR/rdf-sparql-query/
I think I have a better impression of what you want to do. It seems
rather unlikely that the Android client will handle SPARQL queries,
because that would mean that all data that one wants to search through
are on the client - which could eventually be something huge
(combining all repositories world-wide!)

So, I really see the client as a thin client:
A) The user interacts with a GUI Interface and enters some search
parameters.
B) The Android client forms a SPARQL query out of this and sends it to
a server.
C) The server interprets the SPARQL query, redirects the query to
other servers, collects the answers, and sends back a solution
sequence.
D) The client GUI displays the solution sequence in a list with fancy
images, star rating, bla bla.

So, the Android client does not have to understand SPARQL, maybe it is
not even necessary to include openRDF / Sesame libraries (unless only
the relevant parts can be extracted).

Now, in this scenario, the server has a lot to do: It has to keep a
list of its own content, as well as know other repositories to either
query them or host a copy of the RDF files of the other sites.

Do I get the basic picture right? I also think for a quick spreading
of the open repository system, it would be very beneficial to keep the
basic client as thin as possible. Then also cheap / weak phones could
access it.

Peli

[Zach Hobbs]

On Thursday 17 April 2008 11:02:46 Peli wrote:
> C) The server interprets the SPARQL query, redirects the query to
> other servers, collects the answers, and sends back a solution
> sequence.

Where do we keep the lists of servers? Will this be part of the AndroForge
project? Also, does our initiative have a name yet?

Also, we might want to give the client options about what sources to use (in
cause they want to include a source not normally included, or only want to
pull only from one source, etc).

Thanks,
Zach

[Shane Isbell]

On Thu, Apr 17, 2008 at 9:15 AM, Zach Hobbs <ho...@helloandroid.com> wrote:

On Thursday 17 April 2008 11:02:46 Peli wrote:
> C) The server interprets the SPARQL query, redirects the query to
> other servers, collects the answers, and sends back a solution
> sequence.

Where do we keep the lists of servers?  

I'm in agreement with Peli that the full search should be done on a proxy. If we take this approach, then the client could designate the query, list of servers and send that to the proxy/relay, which would do the actual distributed search.  

 

Will this be part of the AndroForge
project?  

 

Still under discussion. I know Peli had some questions and not getting a response from Carl is not a good sign. I imagine the RDF layer over JV would be part of jvending project, but clients and proxies, would be a completely separate project. This separate project would only be dependent on the defined RDF interface, so other backend catalog implementations besides JV could be plugged in.

 

Also, does our initiative have a name yet?

No, not yet.

Also, we might want to give the client options about what sources to use (in
cause they want to include a source not normally included, or only want to
pull only from one source, etc).

This is doable.

 

Shane

Thanks,
Zach

[Peli]

Ok, let's break this down to small managable pieces :-) We at
OpenIntents believe in the KISS principle ( http://en.wikipedia.org/wiki/KISS_principle
). So let's attack the big problem in small steps:

1) Since we agree of having all the multi-server part be handled by
proxy-servers, we can first leave this option out. Simplest setup:
* Client GUI (really basic)
* Client forms SPARQL query and sends it to a single server.
* Server receives SPARQL, searches through its RDF content. ( no
forwarding to other servers yet), and sends back solution sequence.
* Client interprets solution sequence and displays it in a list.
* Client accesses JVending server directly (?) to download new
software.

This will already have benefits to the end user. Through the SPARQL/
RDF abstraction layer, any provisioning system could be hooked into
the back end, and any client into the front end.

QUESTION 1: At some point the client has to access JVending directly
to download the software. Are there any possibilities of
incompatibility here? Or will it be standard http or https requests to
download the application, possibly with some security restrictions
(for payment options).

2) Once this works, it is just a matter of changing the SPARQL server
to obtain multi-server search. No need to change the client. It would
still receive a single solution sequence, but the addresses where the
applications are located point to different servers.

QUESTION 2: Would we really need a separate server process to handle
basic SPARQL requests, or could this be simply added on top of
JVending? In one case, the important thing is only the RDF and SPARQL
definitions, in the other one has a working implementation to plug it
on top of other projects.

So, the first step in any case is to define the RDF and SPARQL
interfaces. Once this is done, I could imagine that OpenIntents could
build a basic client while JVending builds the basic server. We should
strive to have a working sample as soon as possible - independent
improvements on each side are then possible independently.

What do you think?

Peli

> > Zach- Hide quoted text -
>
> - Show quoted text -

[Peli]

Until recently, I was not fully aware of the SlideME client - which
does a good job.

The open repository system really only makes sense if there are at
least two different vending systems that we want to combine. Right
now, I see only JVending in this discussion. I wonder whether the
better strategy would not be to rather improve the existing JVending
system to support multiple servers transparently, rather than build
another layer on top of that.

I start a wish list in a separate thread. :-)

Peli

On Apr 18, 9:50 am, Peli <peli0...@googlemail.com> wrote:
> Ok, let's break this down to small managable pieces :-) We at

> OpenIntents believe in the KISS principle (http://en.wikipedia.org/wiki/KISS_principle

[Shane Isbell]

I probably need to clarify what I am thinking.

First, we build an RDF interface that can handle SPARQL queries on top of JV. Anybody can use an instance of JV for their own catalog. They could also build their own catalog and can plugin as long as they support SPARQL. So this leaves us with a number of catalog servers (some JV, some not) that can be queried in a distributed fashion.

If this were a PC, we could just build a fat client using existing RDF client libraries. The user would define the list of catalogs and do their queries. But to support Android, we need to setup a relay/proxy that can take a query request, translate it to SPARQL and then do the distributed query on behalf of the client. Certainly, a JV server could also function as a proxy, but the abstraction still needs to be there.

Shane