Ordered photo albums/slideshows

[Dan Cutting]

Hi all,

I’m thinking of using Perkeep to store and manage my large photo library (hundreds of GB).

I’ve been reading through all the Perkeep docs/specs/blog posts/mailing lists/etc. but still have many questions.

I understand each photo could be a file blob referencing a permanode.

But what would be a good way to represent collections of photos such as albums or slideshow that would be arranged in a specific order?

I’m not sure the best underlying blob representation for this use case. I see there are concepts like sets which appear to be unordered. Or directories which are key value but also appear to be unordered (or at least, not easily "reorderable").

So perhaps a better approach would be to store the order of photos in a custom "slideshow" data blob and just rewrite that whenever I want to reorder it.

Has anybody tried something like this?

I have lots of other questions about photos, but I'll just start with this. :)

Thanks for a great project!

Dan

[Mathieu Lonjaret]

Hi,

quick answer, so I'll have to double check, but if I wanted to go the
simplest route for that, I'd use the camliPath attribute. If we sort
(say lexicographically) - but I don't remember if we currently do -
the camliPaths then reordering within a group of camliPaths, simply
means renaming the ones you want to reorder.

I'll try to go for a better/longer answer later.

hth,
Mathieu

> --
> You received this message because you are subscribed to the Google Groups "Perkeep" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/80d6b316-f19f-44f4-95a2-f00633a68d23%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Dan Cutting]

Thanks for your reply Mathieu.

I think I see what you mean. Something like this:

item_1: sha224-abc

item_2: sha224-def

item_3: sha224-ghi

Would this mean potentially renaming all camliPath keys to move an item to the front of the list?

item_2: sha224-abc

item_3: sha224-def

item_1: sha224-ghi

Thanks,

Dan

On Thursday, May 9, 2019 at 1:27:35 PM UTC+1, mathieu.lonjaret wrote:

Hi,

quick answer, so I'll have to double check, but if I wanted to go the
simplest route for that, I'd use the camliPath attribute. If we sort
(say lexicographically) - but I don't remember if we currently do -
the camliPaths then reordering within a group of camliPaths, simply
means renaming the ones you want to reorder.

I'll try to go for a better/longer answer later.

hth,
Mathieu

On Thu, 9 May 2019 at 00:43, Dan Cutting <dcut...@gmail.com> wrote:
>
> Hi all,
>
> I’m thinking of using Perkeep to store and manage my large photo library (hundreds of GB).
>
> I’ve been reading through all the Perkeep docs/specs/blog posts/mailing lists/etc. but still have many questions.
>
> I understand each photo could be a file blob referencing a permanode.
>
> But what would be a good way to represent collections of photos such as albums or slideshow that would be arranged in a specific order?
>
> I’m not sure the best underlying blob representation for this use case. I see there are concepts like sets which appear to be unordered. Or directories which are key value but also appear to be unordered (or at least, not easily "reorderable").
>
> So perhaps a better approach would be to store the order of photos in a custom "slideshow" data blob and just rewrite that whenever I want to reorder it.
>
> Has anybody tried something like this?
>
> I have lots of other questions about photos, but I'll just start with this. :)
>
> Thanks for a great project!
>
> Dan
>
> --
> You received this message because you are subscribed to the Google Groups "Perkeep" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to per...@googlegroups.com.

[Mathieu Lonjaret]

On Thu, 9 May 2019 at 16:09, Dan Cutting <dcut...@gmail.com> wrote:
>
> Thanks for your reply Mathieu.
>
> I think I see what you mean. Something like this:
>
> item_1: sha224-abc
> item_2: sha224-def
> item_3: sha224-ghi

More like:

camliPath:/collections/photo1 = sha224-blabla

but yeah, you got the idea.

> Would this mean potentially renaming all camliPath keys to move an item to the front of the list?

Not necessarily. If the top of the list is e.g. camliPath:/photo1 and
you want to move another photo before that one, you'd just have to
name it to something that comes before in alphabetical order (again,
assuming we are sorting them this way), so to e.g. camliPath:/photo0.
And you wouldn't have to rename any of the others. But yeah, maybe
some situations would involve quite some shuffling around, I'm not
sure.
Btw, I've just remembered two things:
1) in the scanning cabinet we use camliPath:page_number to represent
the set of the scanned pages of a document. It's a bit like what you
want, except we indeed do not mutate the order.
2) In this thread, we were talking about another potential attribute,
i.e. camliPathOrder, but we never ended up actually using it. Maybe we
should revive it at some point.
https://groups.google.com/d/msg/camlistore/xApHFjJKn3M/dyYpJDDAjvgJ

> To unsubscribe from this group and stop receiving emails from it, send an email to perkeep+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/0a345d45-0e5b-47b2-8e22-975004636b50%40googlegroups.com.

[Dan Cutting]

On Thursday, May 9, 2019 at 9:29:47 PM UTC+1, mathieu.lonjaret wrote:

...

Btw, I've just remembered two things:
1) in the scanning cabinet we use camliPath:page_number to represent
the set of the scanned pages of a document. It's a bit like what you
want, except we indeed do not mutate the order.

Right; this would work well for archival documents like document scans but less well for mutable collections like albums.

 

2) In this thread, we were talking about another potential attribute,
i.e. camliPathOrder, but we never ended up actually using it. Maybe we
should revive it at some point.
https://groups.google.com/d/msg/camlistore/xApHFjJKn3M/dyYpJDDAjvgJ

This thread is really interesting.

The camliPathOrder attribute would let you reorder by spacing out values and then using midpoints when things change. Floating points would be better for this since you wouldn't have to worry about picking the right spacing to begin with. Either way, this approach does limit the number of reorders, especially in pathological cases. Perhaps it could periodically be "repacked" somehow.

The other approach mentioned seems to be building a singly/doubly linked list structure with "previous" and "next" attributes, which sounds plausible.

But it makes me wonder if this could be done more simply by appending insert/move/delete operations to the permanode.

E.g.:

item_0: sha224-abc

item_1: sha224-def

item_2: sha224-ghi

move:2:0

delete:1

insert:0:sha224-jkl

The indexer would just run through these operations to reconstruct the final order:

sha224-jkl

sha224-ghi

sha224-def

Again, maybe periodic repacking checkpoints could limit the work the indexer needs to do after many operations are appended.

Clearly I don't know exactly what this should look like, but would this kind of approach make any sense?

...

[Ian Denhardt]

Quoting Dan Cutting (2019-05-09 19:07:24)

> Right; this would work well for archival documents like document scans
> but less well for mutable collections like albums.

Two thoughts on this:

1. For something like a photo album, it seems unlikely to me that a
single album would get so big that having to shift everything down
would actually be a problem.
2. If you do want to improve the asymptotics, you could basically
construct a trie using the names; something like the following:

On initial import, use keys like:

photo-0
photo-1
photo-2
...

On subsequent imports, if you have something that's supposed to go
between photo-0 and photo 1, you could rename photo-1 to photo-11,
and name the new image photo-10.

You could fiddle with the details of the scheme a bit, but it gives you
logarithmic insert, instead of linear.

Still feels like a bit of a hack though, so maybe still worth
investigating other solutions.

-Ian

[Dan Cutting]

On Friday, May 10, 2019 at 8:39:08 PM UTC+1, ian wrote:

Two thoughts on this:

1. For something like a photo album, it seems unlikely to me that a
   single album would get so big that having to shift everything down
   would actually be a problem.

This is almost certainly true. :) But it also seems like a good opportunity to check whether there's a simple-ish general solution.

 

2. If you do want to improve the asymptotics, you could basically
   construct a trie using the names; something like the following:

On initial import, use keys like:

    photo-0
    photo-1
    photo-2
    ...

On subsequent imports, if you have something that's supposed to go
between photo-0 and photo 1, you could rename photo-1 to photo-11,
and name the new image photo-10.

You could fiddle with the details of the scheme a bit, but it gives you
logarithmic insert, instead of linear.

Yeah, I like this idea. To put something between photo-0 and photo-1, you might be able to just give it the name photo-01 without renaming others, which would be constant insert? (I think the only time you couldn't do it this way would be to insert something at the front of the list.)

 

Still feels like a bit of a hack though, so maybe still worth
investigating other solutions.

 

This problem feels similar to the list labelling problem, so perhaps there are ideas to explore there.

https://en.wikipedia.org/wiki/Order-maintenance_problem#List-labeling