Backend API?

[Norman Walsh]

Hello,

I've just started looking at Perkeep. I'm interested in implementing a different back end for storage. Can that be done by implementing a REST API?

Pointers to docs, etc. most humbly solicited. I haven't actually tried reverse engineering it from sources yet :-)

--norm

[Mathieu Lonjaret]

Hi,

The minimum is to implement a blob server:
https://perkeep.org/doc/terms#blobserver
but what you probably want is to implement a full storage interface:
https://perkeep.org/pkg/blobserver#Storage
you can look at any of the other existing implementations in
pkg/blobserver (https://perkeep.org/pkg/blobserver#pkg-subdirectories)
for examples.

regards,
Mathieu

> --
> You received this message because you are subscribed to the Google Groups
> "Camlistore" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to camlistore+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Norman Walsh]

Mathieu Lonjaret <mathieu....@gmail.com> writes:
> The minimum is to implement a blob server:
> https://perkeep.org/doc/terms#blobserver
> but what you probably want is to implement a full storage interface:
> https://perkeep.org/pkg/blobserver#Storage
> you can look at any of the other existing implementations in
> pkg/blobserver (https://perkeep.org/pkg/blobserver#pkg-subdirectories)
> for examples.

Ok. It might be fun to learn a bit of Go and try to implement the
whole thing as a new backend, but have you considered providing a
generalized interface that communicates with the actual backend
database through a defined REST interface?

Be seeing you,
norm

--
Norman Walsh <n...@nwalsh.com> | Knowledge comes, but wisdom
http://nwalsh.com/ | lingers.--Alfred Lord Tennyson

[Mathieu Lonjaret]

I forget if we have a better place where it's described, but I believe
something like https://perkeep.org/doc/protocol/ might be what you're
asking for?
Then your server written in another lang can be integrated to the
Perkeep server with https://perkeep.org/pkg/blobserver/remote

[Brad Fitzpatrick]

On Thu, Dec 21, 2017 at 7:57 AM, Norman Walsh <n...@nwalsh.com> wrote:

Mathieu Lonjaret <mathieu....@gmail.com> writes:
> The minimum is to implement a blob server:
> https://perkeep.org/doc/terms#blobserver
> but what you probably want is to implement a full storage interface:
> https://perkeep.org/pkg/blobserver#Storage
> you can look at any of the other existing implementations in
> pkg/blobserver (https://perkeep.org/pkg/blobserver#pkg-subdirectories)
> for examples.

Ok. It might be fun to learn a bit of Go and try to implement the
whole thing as a new backend, but have you considered providing a
generalized interface that communicates with the actual backend
database through a defined REST interface?

We actually have one already. You don't need to write in Go.

If you just want to write a blob server in your favorite language, you need to implement the blob server protocol:

https://camlistore.googlesource.com/camlistore/+/master/doc/protocol/

 

And then you can use it from the main perkeep server (still called camlistored for a bit) by using the "remote" blob storage target. The "remote" blob server implements the blobserver.Storage interface that Mathieu had linked by being a client to a remote server implementation.

 

[Norman Walsh]

Mathieu Lonjaret <mathieu....@gmail.com> writes:
> I forget if we have a better place where it's described, but I believe
> something like https://perkeep.org/doc/protocol/ might be what you're
> asking for?
> Then your server written in another lang can be integrated to the
> Perkeep server with https://perkeep.org/pkg/blobserver/remote

Perfect. Are there unit tests for testing the /protocol/ backend?

Be seeing you,
norm

--

Norman Walsh <n...@nwalsh.com> | There has never been a perfect
http://nwalsh.com/ | government, because men have passions;
| and if they did not have passions,
| there would be no need for
| government.--Voltaire

[Mathieu Lonjaret]

On 22 December 2017 at 10:07, Norman Walsh <n...@nwalsh.com> wrote:
> Mathieu Lonjaret <mathieu....@gmail.com> writes:
>> I forget if we have a better place where it's described, but I believe
>> something like https://perkeep.org/doc/protocol/ might be what you're
>> asking for?
>> Then your server written in another lang can be integrated to the
>> Perkeep server with https://perkeep.org/pkg/blobserver/remote
>
> Perfect. Are there unit tests for testing the /protocol/ backend?

I don't think so. Then again, if you write tests similar enough to the
various Go implementations in pkg/blobserver/ (e.g. localdisk) for
your server, then you should be good.

> Be seeing you,
> norm
>
> --
> Norman Walsh <n...@nwalsh.com> | There has never been a perfect
> http://nwalsh.com/ | government, because men have passions;
> | and if they did not have passions,
> | there would be no need for
> | government.--Voltaire
>