Per-contributor focuses for next major release, hack sessions

[Eric Drechsel]

Many open source projects have a tradition to kick off a new dev cycle with a thread where contributors list tasks/issues they are planning to work on in the coming months. 

It seems like PerKeep has been building some momentum over the Winter, and I'm really curious to hear what people have on their personal roadmaps, both core commiters and new/sometime contribs.

I'll kick it off :D While I haven't been very involved in the past year, I am interested to make more contributions. Here are the things I'm keen to work on:

• Update this old change list exposing permanode IDs as a FUSE xattr [1].
• Have build infrastructure produce Synology packages, or figure out best practices for deploying to Synology using Docker etc [2].
• Write a client side example app/guide using the GopherJS data access layer but without using GopherJS for the app code.
• If this isn't currently possible I would like to work on making sure the GopherJS blob exports a nice data access API to JS.
• Write additional content importers:
• Git repository fetcher
• Website crawler
• Work on maintainence/new features in the Web UI

 [1] https://camlistore-review.googlesource.com/c/camlistore/+/2869

 [2] https://github.com/camlistore/camlistore/issues/986

Of course I'd be happy if anyone else wants to work on any of these features too :D

Also, are there any PerKeep meetups/hack sessions planned? LinuxFestNW CFP is open till the end of the month... :)

--

best, Eric
eric.pdxhub.org

[clive boulton]

PerKeep perfect... 

Linuxfest Northwest 2018: "Message In A Bottle"

LinuxFest is celebrating 19 years of service to the Free and Open Source community and we want you to be a part of it! Please join us by being a presenter at one of the largest, technically diverse, educational, and fun open source conferences here in the Pacific Northwest.

We are encouraging (but not requiring) that presenters touch on our theme "Message in a Bottle": Security, Containerization, and Personal Data. If you have any questions please don't hesitate to contact us at pre...@linuxfestnorthwest.org.

https://linuxfestnorthwest.org/conferences/lfnw18

Terse from phone

--
You received this message because you are subscribed to the Google Groups "Camlistore" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camlistore+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Adam Shannon]

I'd like to help out. To get familiar with the code I've been fixing low hanging doc related fruit.

I've seen some things as label:"Help Wanted", but I'm open to trying anything.

https://github.com/perkeep/perkeep/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22

[Brad Fitzpatrick]

Good idea!

First off, let's agree on the case of the new name: Perkeep, without a capital 'K'. :-)

My priorities in the short term:

-- finish the project rename

-- be more aggressive at unblocking people and reviewing code, or fixing it up when it's stalled but close

-- work on the website, especially more docs

-- fix the Google Photos importer (https://github.com/perkeep/perkeep/issues/987)

-- focus on importers in general, and make Perkeep more user-friendly by making importers work out of the box (https://github.com/perkeep/perkeep/issues/1005)

-- migrate away from SHA-1 (https://github.com/perkeep/perkeep/issues/537)

-- triage the bug tracker more aggresively

-- help new contributors ramped up

[Brad Fitzpatrick]

Also, just doing lots of cleanups, fixing ancient & embarrassing things in the code. :)

[Mathieu Lonjaret]

What i will actually end up doing, and what I would love to land are
of course only weakly correlated, but here are my most personal wishes
anyway:

-keep on making the sharing story better
-add a Google Drive importer (after Google Photos is done)
-fix all the flaky tests, and have a more reliable CI (builds and
tests) setup overall

And of course I would love for LFNW to happen again.

>>> email to camlistore+...@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>

> --
> You received this message because you are subscribed to the Google Groups
> "Camlistore" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to camlistore+...@googlegroups.com.

[Will Norris]

+1 to another meetup at LFNW... I'd be up for that.

Things I'm hoping to accomplish:

 - update site markup and css, both for some minor general cleanup but also to make it render a bit better on mobile (I've been doing a lot of re-reading of the docs and code on my phone with a baby in arm in the wee hours of the night, so would love to make this a slightly better experience).  Also, some minor doc cleanup that I've run across along the way.

 - work on a chrome os filesystem provider (#1017).  I've been using a chromebook as my primary machine for close to a year now, so would love to be able to access files from the native Files app.  This may end up being more ambitious than I have time to commit to it right now, but I'd like to see how far I can get.

>>> email to camlistore+unsubscribe@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Camlistore" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to camlistore+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Camlistore" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camlistore+unsubscribe@googlegroups.com.

[Mathieu Lonjaret]

On 8 January 2018 at 05:11, Will Norris <wi...@willnorris.com> wrote:
> +1 to another meetup at LFNW... I'd be up for that.
>
> Things I'm hoping to accomplish:
> - update site markup and css, both for some minor general cleanup but also
> to make it render a bit better on mobile (I've been doing a lot of
> re-reading of the docs and code on my phone with a baby in arm in the wee
> hours of the night, so would love to make this a slightly better
> experience). Also, some minor doc cleanup that I've run across along the
> way.

thanks. speaking of mobile, if while doing that for the website, you
realize similar changes could be done to the web UI, feel free to
bring my attention to it. I think it already kinda works on mobile,
but any little bit to make it better counts.

>> >>> email to camlistore+...@googlegroups.com.

>> >>> For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "Camlistore" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to camlistore+...@googlegroups.com.

>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Camlistore" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to camlistore+...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Camlistore" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to camlistore+...@googlegroups.com.

[Paul Lindner]

Okay, here goes short term:

- modernize our dependencies so it's easier to work well with other related projects and infrastructure.

- learn the block/kv backends, possibly implement for ethereum swarm and IPFS.

- Promote, promote, promote; organize, organize, organize

And of course my Long term focus:

- Make Perkeep usable by typical users

- Fulfill the 100 year promise, via economic and organizational changes.

- Develop protocols and standards that enable archiving by default for major services and obsolete importers.

[Adam Shannon]

I'm going to look at building a web based rss reader ontop of perkeep. Maybe something that reminds me of Google reader. 

>>> email to camlistore+unsubscribe@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Camlistore" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to camlistore+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Camlistore" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camlistore+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Camlistore" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camlistore+unsubscribe@googlegroups.com.

[Stephen Searles]

I saw some activity on my ancient sharding merge request. I've been dusting that off and am actually trying to tackle the problem I've been stalled on all this time, which I'll describe here.

The naive sharding approach in the current MR assumes the same shards will always exist. This sounds too naive to be practical. When you get that brand new hard drive, possibly because another one bit the dust, you want a permanent data repository to be able to handle replicating and migrating data. So at the very least, I'm working on a generational configuration that allows a single blob to map onto multiple shards, and to allow the shards available to change over time. The default selector will still map blobs to shards deterministically based on their hash and size.

That said, my true goal is to sort-of automatically manage these storage generations. Basically, I want to be able to throw disks into the configuration and have data automatically shuffled to take advantage of the new disk. I also want to configure a redundancy threshold and be alerted if my data grows too large or if usable disk space grows too small to provide the level of redundancy requested. Additional gravy: to monitor the performance and S.M.A.R.T. data from the drives to make inferences about disk health and warn ahead of time when redundancy may fall. Even further gravy: use that performance data to specifically shuffle data onto disks that are fast at reading and have open space on disks that are fast at writing.

Anyway, I'm not sure what of this (if any) I'll actually finish by the next release cycle, but I realize I'm quiet here and wanted to chime in with what I've been tinkering with.

Thanks for the thread! Good idea!

[Brad Fitzpatrick]

Rather than explicitly track generations, I'd just do consistent hashing, where each disk has a weight proportional to its size (by default) or explicit configuration.

On read, try its ideal location. If you miss, search N shards in parallel looking for it. Optionally enqueue a migration task. On any layout change, have an optional background worker doing migrations to ideal locations.

[Eric Drechsel]

Thanks everyone who responded, great to hear peoples' plans!

Having thought a bit more about my priorities for this cycle, i think my amended priority list would be:

 - implement a browser-side test suite for the ui, integrate with current testing.

 - update the android app. implement android Storage Access Framework providers so you can browse/upload photos/videos/files to/from your store from camlistore. Probably reuse react stuff for the browse UI

 - Start of IndieWeb integration so i can publish to the Fediverse straight from Perkeep. This has been a longtime dogfooding goal for me, and the main use case that initially drew me to the project.

--

best, Eric
eric.pdxhub.org