Is Perkeep still alive?

[Lucian Daniliuc]

Hello,

I've been studying Perkeep for a few weeks now, and even though the technology and use cases looks solid, I see there's no active community around it (or I don't know where to loo for it). 

Giving the rise of AI LLMs I am thinking of building a project on Perkeep and not sure that it is still being developed or maintained.

So, if you stopped using Perkeep (or just gave up), please reply with your reason and what are you using instead?

Cheers,

Lucian.

[Aaron Peterson]

I installed a few weeks ago, and did a git pull,  a few hundred files were updated...  and I am able to get my Google files imported with ssl

[Gina White]

I have a pull request that has been unanswered for 3 months now.  I gave up.

--

You received this message because you are subscribed to the Google Groups "Perkeep" group.

To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/e6b06baa-9e44-4df7-8f4b-99841d41b86fn%40googlegroups.com.

[Michael Farr]

Hi Lucian,

I got an initial system working, but despite its many great features, I couldn't get it to stretch around what I needed. The maintenance doesn't appear to be quite dead yet, but it seems to have slowed down considerably in the past couple of years.

The first issue I had was the lack of a system to control the lifecycle of data that I didn't want to keep forever. An example would be to support data that the legal owner only gives me temporary and conditional rights to hold. The second issue was that it couldn't manage data stored on other systems; in general, it just wants to manage files in its own blob storage system and lacks features to link or push external data (it has some good tools for importing data from some sources).

I like this idea of wrapping AI into a data management system also. Feel free to contact me directly if you want to discuss some ideas.

Mike

--

[Lucian D.]

Hey Michael,

In the meantime I realized that PerKeep might not be the best option. 

Please see https://betterprogramming.pub/the-personal-vault-project-7cdd692b08b0 about my vision about the AI-enhanced personal vault that I am working on.

I have described the system how I see it.

I am curious what you have on your mind.

Some more thoughts on the subject (mine + other peoples) are on the Discussion section: https://github.com/dlucian/pvp/discussions

Cheers,

Lucian.

You received this message because you are subscribed to a topic in the Google Groups "Perkeep" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/perkeep/eCoO2zm9Plk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to perkeep+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/CACXS6nQr%2BrxkiZzYBRzgW5bBRNrtm13t%2BU7iCaqKZ9nZtj-cyw%40mail.gmail.com.

[Michael Farr]

Hi Lucian,

Thats so great, I'll contribute something there soon. I had also started a discussion page, but I think I like the idea of running it in github much more. My concept is to separate the metadata from the archive, but the 3 project goals seem the same.

Cheers,

Mike

To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/CAOXX1pJu9BsAYqM5znP-xr8eMWn%2B7c574hdN1_1p-7Nk1VkxKw%40mail.gmail.com.

[Aaron Peterson]

can you incorporate those features into perkeep?  fragmentation is kinda rough.

Using the blobs as a backend is great.

To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/CAOXX1pJu9BsAYqM5znP-xr8eMWn%2B7c574hdN1_1p-7Nk1VkxKw%40mail.gmail.com.

[Quinn]

Has anyone continued this work? I am very interested in having a general purpose blob storage that I can use for personal projects as a backend/db. Ideally, something I can use as library in go code rather than a server. There really isn't anything that I can find online that fits this, I've spent a lot of time googling stuff like "blob store" or "Content addressable storage" and nothing really comes up. 

[Quinn]

Here's a random selection of two:

https://github.com/bobg/bs
https://github.com/dennwc/cas

Both are seemingly abandoned, and they both cite perkeep as inspiration.

[Markus Peröbner]

I'm still using perkeep for storing lots of my personal data.

On the other side I've started to implement an alternative blob store with a seperate index called everarch.

You can have a look if you are interested:

https://evr.ma300k.de/#/title

https://git.ma300k.de/everarch/

To view this discussion visit https://groups.google.com/d/msgid/perkeep/9f819edd-5f12-4223-9d99-ebe2cbdcbad3n%40googlegroups.com.

[Quinn]

Cool! I'm reading thru your README, and it sounds a lot like perkeep and/or restic:

- store your tiny and GB sized files
- provide a customized index over your data
- make backups complete and fast
- access your files via a virtual file system
- store your data in a format which you still can read in 20 years in
_future programming language_
- store your data in a way that allows you to easily adopt new data
structures without the need to migrate the existing data
- detect corrupt or externally modified data and restore it with a
partial backup

Why are you building this instead of just using perkeep (or restic)? Also, if you are dissatisfied with the claim structure, etc. you could just use perkeep as a dumb blob store and build your own index using enumeration. 

Really, want i want is this:

 * blob store
 * some sort of extensible mechanic for indexing
 * code that isn't in a dir called "internal" (i want to use it as a lib not a server)
 * written in go

I really like perkeep's method of indexing any blob that starts with 0x7b ({) as an easy way to build meaningful types on top of generic blobs of content-addressed data. I would like this indexing function to be more extensible so I can use it to store my own types.

Also, I get why you chose xml for the claims (xslt and namespaces), but I think I would prefer to use json anyway 😅

Is anyone interested in trying to pick up maintenance of perkeep as a group? 

Quinn

[Markus Peröbner]

There are many reasons why I build everarch but my personal most important ones are:

1) Perkeep has a quite big memory footprint. I think it was probably built for a HDD-era were random hard disk access was super slow. I assume this because the whole index is always in memory. I have a >1TB everarch repository which consumes less than 100MB of RAM.

2) You probably know that 10 years later you might do things differently than today. The properties you write with perkeep into your JSON are pretty much fixed. Changing their meaning can hardly be done without rewriting attributes in probably lots of permanodes. I hope that with the everarch setup I will only need to introduce a new XML namespace in 10 years and then adjust my XSLT transformation to tell how to map the old data into my new way of thinking. We will see if this will work out :D

Still I think perkeep is a pretty neat tool. Being able to declare handlers in the server's config in a layered structure is pretty awesome! Of course I would never have written everarch without the many great ideas from perkeep.

To view this discussion visit https://groups.google.com/d/msgid/perkeep/6afda2b4-92ec-420e-96fd-322b86f85b33n%40googlegroups.com.

[Jim DeLaHunt]

Am So., 24. Nov. 2024 um 19:55 Uhr schrieb Quinn <q.sha...@gmail.com>:

… I get why you chose xml for the claims (xslt and namespaces), but I think I would prefer to use json anyway 😅…

On 2024-11-24 22:59, Markus Peröbner wrote:

…There are many reasons why I build everarch but my personal most important ones are: …

2) You probably know that 10 years later you might do things differently than today. The properties you write with perkeep into your JSON are pretty much fixed. Changing their meaning can hardly be done without rewriting attributes in probably lots of permanodes. I hope that with the everarch setup I will only need to introduce a new XML namespace in 10 years and then adjust my XSLT transformation to tell how to map the old data into my new way of thinking. We will see if this will work out :D

This comment by Markus reminds me of an interesting design issue for archives which are supposed to preserve data for decades and centuries into the future. You have to assume that the software which reads the archive's metadata a century in the future will not be the same software which wrote the metadata today. The original source code will probably have been lost, the programming language and libraries will have moved on. You have to assume that the future reader will need to reverse-engineer an implementation of your archive's format. 

I like a comment I read once that JSON objects tend to represent a program's in-memory data structure. Future software which aspires to reverse-engineer JSON objects need to figure out the generating program's data structures. XML documents tend to be more self-contained. They tend to describe the semantics of the information being stored, rather than the data structures of the generating program. The XML language is designed with an assumption that the reading software is different than the writing software, and that the document may persist through multiple generations of reading software.

Yes, JSON is currently a popular format, and XML is out of fashion. However, I believe that archive metadata expressed as a document with XML markup will probably communicate the semantics of their content more effectively to future software writers than will a JSON object. Whatever the description language, I think the archive designer would do well to think of the needs of the reverse-engineer a century in the future.

Best regards,
     —Jim DeLaHunt

-- 
.   --Jim DeLaHunt, jd...@jdlh.com     http://blog.jdlh.com/ (http://jdlh.com/)
      multilingual websites consultant, Vancouver, B.C., Canada