publisher setup

82 views
Skip to first unread message

Attila Tajti

unread,
Jan 17, 2017, 12:28:58 PM1/17/17
to Camlistore
I tried publisher the first time today, and it worked fine after adding just

    "publish": {
        "/pics/": {
            "camliRoot": "mypics",
            "cacheRoot": "/home/ata/var/camlistore/blobs/cache",
            "goTemplate": "gallery.html"
        }
    }

to the server-config, and I could access «camlihost:port»/pics/ just fine. However after updating camlistore it is broken. The CL/issue


seems to be the culprit, I can access «camlihost:port»/pics/ with d864e3580281c26ff3507d1df60f8d2542ee1c2c but not with master (abc6561c60c863589bb146784e70432806c14d4d).

Since I have no experience with publisher, my question if one should publish image permanodes or sets with children? I've checked the CL and the issue, and the issue mentions how one should be able to access "subdirectories" under /pics/. Should the URL ("foo" and "bar" in the example in the issue) be set through camliPath, camliMember/title?

Btw, the thumbnails in the publisher show up with an incorrect aspect ratio (apparently the image is streched into a square shape) but standalone images appear fine. Is there anything I can do about it?

Mathieu Lonjaret

unread,
Jan 17, 2017, 2:14:59 PM1/17/17
to camli...@googlegroups.com
There's something else broken with the publisher that I have to fix before I can answer your questions for sure. More soon.


--
You received this message because you are subscribed to the Google Groups "Camlistore" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camlistore+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Mathieu Lonjaret

unread,
Jan 17, 2017, 8:01:52 PM1/17/17
to camli...@googlegroups.com
Thanks for the question, I wouldn't have noticed https://camlistore-review.googlesource.com/#/c/9146/ otherwise! Well at least not until later, as it was a conjunction of two different CLs that introduced the breakage.

On 17 January 2017 at 18:28, Attila Tajti <attila...@gmail.com> wrote:
I tried publisher the first time today, and it worked fine after adding just

    "publish": {
        "/pics/": {
            "camliRoot": "mypics",
            "cacheRoot": "/home/ata/var/camlistore/blobs/cache",
            "goTemplate": "gallery.html"
        }
    }

to the server-config, and I could access «camlihost:port»/pics/ just fine. However after updating camlistore it is broken. The CL/issue


seems to be the culprit, I can access «camlihost:port»/pics/ with d864e3580281c26ff3507d1df60f8d2542ee1c2c but not with master (abc6561c60c863589bb146784e70432806c14d4d).

Yes, that is the point of the CL. It is a debatable decision, but I found that I wouldn't usually let the world know of all my published stuff at the same time. I usually want to share a particular item, or set of items, with an audience, not unveil everything that I have. It is of course not a security measure by any means, as everything is still published and discoverable with enough effort, but it's more of an attempt at basic privacy. 
 
Since I have no experience with publisher, my question if one should publish image permanodes or sets with children? I've checked the CL and the issue, and the issue mentions how one should be able to access "subdirectories" under /pics/. Should the URL ("foo" and "bar" in the example in the issue) be set through camliPath, camliMember/title?

The camliRoot itself only supports camilPath, so foo and bar are camliPath attributes set on the camliRoot, such as:

camilPath:foo = sha1-foo
camliPath:bar = sha1-bar

sha1-foo (and sha1-bar) should be either a permanode with some camliContent, or a permanode with some camliMembers. But I notice that a permanode with camliContent (directly at the root) seems to be broken. I'll look into it.
You can then of course have anything deeper in the path (the camliMembers of sha1-foo can have camliMembers themselves, or have a camliContent file, or directory, etc.), but only sets (camliMembers) are browsable. directories (static-sets) aren't yet browsable, because we need to finish https://camlistore-review.googlesource.com/8326 first.
 
Btw, the thumbnails in the publisher show up with an incorrect aspect ratio (apparently the image is streched into a square shape) but standalone images appear fine. Is there anything I can do about it?

Yeah, I did the very minimum amount of math for the thumbs.
You can:
1) as usual file an issue :-) (but that probably won't be in my priorities, sorry) 
2) fix the code at app/publisher/js/members.go. Aaron had done it pretty well for the web UI (I'm still seeing some bugs if I mix images with very different sizes though), so that might be the way to go for inspiration.
 
hth,
Mathieu

Attila Tajti

unread,
Jan 18, 2017, 2:19:14 AM1/18/17
to Camlistore
Thank you!

I've added the hints I needed/used to the publisher readme, see


Btw, I believe publishRequest.serveHTTP for the root path (added in CL 8866) should perhaps respond with 403 instead of 404. The status code is 404 is technically correct according to the RFC, but 403 would have saved some time for me because I would have thought of "directory listing prohibited". But maybe that's only me.

Mathieu Lonjaret

unread,
Jan 18, 2017, 12:34:24 PM1/18/17
to camli...@googlegroups.com
On 18 January 2017 at 08:19, Attila Tajti <attila...@gmail.com> wrote:
Thank you!

I've added the hints I needed/used to the publisher readme, see


thanks!
 
Btw, I believe publishRequest.serveHTTP for the root path (added in CL 8866) should perhaps respond with 403 instead of 404. The status code is 404 is technically correct according to the RFC, but 403 would have saved some time for me because I would have thought of "directory listing prohibited". But maybe that's only me.


SGTM. Do you want to send a CL for it?
 


On Tuesday, January 17, 2017 at 6:28:58 PM UTC+1, Attila Tajti wrote:
I tried publisher the first time today, and it worked fine after adding just

    "publish": {
        "/pics/": {
            "camliRoot": "mypics",
            "cacheRoot": "/home/ata/var/camlistore/blobs/cache",
            "goTemplate": "gallery.html"
        }
    }

to the server-config, and I could access «camlihost:port»/pics/ just fine. However after updating camlistore it is broken. The CL/issue


seems to be the culprit, I can access «camlihost:port»/pics/ with d864e3580281c26ff3507d1df60f8d2542ee1c2c but not with master (abc6561c60c863589bb146784e70432806c14d4d).

Since I have no experience with publisher, my question if one should publish image permanodes or sets with children? I've checked the CL and the issue, and the issue mentions how one should be able to access "subdirectories" under /pics/. Should the URL ("foo" and "bar" in the example in the issue) be set through camliPath, camliMember/title?

Btw, the thumbnails in the publisher show up with an incorrect aspect ratio (apparently the image is streched into a square shape) but standalone images appear fine. Is there anything I can do about it?

Attila Tajti

unread,
Jan 18, 2017, 1:16:05 PM1/18/17
to Camlistore


On Wednesday, January 18, 2017 at 6:34:24 PM UTC+1, mpl wrote:
 
Btw, I believe publishRequest.serveHTTP for the root path (added in CL 8866) should perhaps respond with 403 instead of 404. The status code is 404 is technically correct according to the RFC, but 403 would have saved some time for me because I would have thought of "directory listing prohibited". But maybe that's only me.


SGTM. Do you want to send a CL for it?


Mathieu Lonjaret

unread,
Jan 19, 2017, 10:54:00 AM1/19/17
to camli...@googlegroups.com

 
Since I have no experience with publisher, my question if one should publish image permanodes or sets with children? I've checked the CL and the issue, and the issue mentions how one should be able to access "subdirectories" under /pics/. Should the URL ("foo" and "bar" in the example in the issue) be set through camliPath, camliMember/title?

The camliRoot itself only supports camilPath, so foo and bar are camliPath attributes set on the camliRoot, such as:

camilPath:foo = sha1-foo
camliPath:bar = sha1-bar

sha1-foo (and sha1-bar) should be either a permanode with some camliContent, or a permanode with some camliMembers. But I notice that a permanode with camliContent (directly at the root) seems to be broken. I'll look into it.

Attila Tajti

unread,
Jan 19, 2017, 11:32:27 AM1/19/17
to camli...@googlegroups.com
This doesn't seem to work for me, but I'm not sure it is related to your publisher change, so I'm responding here.

Ubuntu 16.04.1 LTS

Camlistore: 2017-01-18-eba4897
Go: linux/amd64 go1.7.4, cgo=true
djpeg: djpeg available

When I add a permanode of a jpeg image to my publisher root as camlipath:face then publisher shows only the title of the image (IMG_20150410_141410.jpg, which is correct), the image is missing. Apart from this attr, I've the followin attrs:

title = "My pictures"
camliRoot = mypics
camliMember = sha-xxx (multiple members from my first attempt setting up publisher)
camliPath:szk = sha-yyy (permanode set, works in publisher)
camliPath:face = sha1-300831992389538979400db5cc8e587bc6316ff8 (permanode with camlicontent, only title is shown)

I get the following in javascript console:

publisher.js:97 Your current jQuery version is: 2.2.3
publisher.js:21 error creating file container: full blobref for prefix sha1-pics not found

What is strange I can't access the permanode on the UI from http://linda:3179/ui/sha1-300831992389538979400db5cc8e587bc6316ff8 either, the XHR http://linda:3179/my-search/camli/search/query responds with

{
  "error": "can only sort by ctime when all results are permanodes",
  "errorType": "Internal Server Error"
}

But that blob seems to be a permanode:

ata@linda:~/src/camlistore.org$ camget sha1-300831992389538979400db5cc8e587bc6316ff8
{"camliVersion": 1,
  "camliSigner": "sha1-5c788ddba4fa84a820119282054100e5ae51c928",
  "camliType": "permanode",
  "key": "sha1-a2b5a6c48b1ae1ac2166154222e3b6abf9cb9e27"
,"camliSig":"wsBcBAABCAAQBQIAAAAACRA9fFgva8QEPQAAyA8IAKwzhD7hdfhIhCm2Z4igiHWOfpPXcLSM7eotuWAL9m8+GEEq8yOTZotu8MNheIvQQ407eZ9BcFoac8gNHG3mVQcvAQWhn1k1DmNDvRhU7XFB3n03Mo93VaA3hxBr84Ubvv6zu6KHH2m6PrhAxSvX1UPlJuK84q4XJV1FQ6HDkiR6+33tgQxaFCecUufe3dTP2oJEHVQf80noq/MqIcWK238B6D8ooNyT2lbVYZckmZ94eCQilXnRWNVD3szeR9QxeC+hoWOlQjcshmfDHUPwK8PYTO3yoys7cOsloufs0CtxGdYCsnwl/ETSs8eR1nwRAbZ+ItMAmhFGZZb+/U84gbY==o0Bt"}

The camliContent of sha1-300831992389538979400db5cc8e587bc6316ff8 is sha1-1abe01663003b2b7beabdf34e7f2babd796eaa16:

ata@linda:~/src/camlistore.org$ camget sha1-1abe01663003b2b7beabdf34e7f2babd796eaa16
{"camliVersion": 1,
  "camliType": "file",
  "fileName": "IMG_20150410_141410.jpg",
  "parts": [
    {
      "blobRef": "sha1-f69e3908f5d520640e5b777801e4bd96c2f2e7bd",
      "size": 262144
    },
    {
      "blobRef": "sha1-d537820ce88a0d3dd6af83dd7af1424643117f53",
      "size": 23186
    }
  ],
  "unixMtime": "2015-04-22T08:31:38Z",
  "unixPermission": "0666"

However http://linda:3179/ui/sha1-300831992389538979400db5cc8e587bc6316ff8?q=%20 (note query parameter) seems to work. This means also pressing enter in a permanode view breaks the view, because it removes the query parameter. This happens with all permanodes I've checked.


Attila
--
You received this message because you are subscribed to a topic in the Google Groups "Camlistore" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/camlistore/vIM2dAFsyTo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to camlistore+...@googlegroups.com.

Mathieu Lonjaret

unread,
Jan 19, 2017, 12:01:02 PM1/19/17
to camli...@googlegroups.com
On 19 January 2017 at 17:32, Attila Tajti <attila...@gmail.com> wrote:
This doesn't seem to work for me, but I'm not sure it is related to your publisher change, so I'm responding here.

Ubuntu 16.04.1 LTS

Camlistore: 2017-01-18-eba4897
Go: linux/amd64 go1.7.4, cgo=true
djpeg: djpeg available

When I add a permanode of a jpeg image to my publisher root as camlipath:face then publisher shows only the title of the image (IMG_20150410_141410.jpg, which is correct), the image is missing. Apart from this attr, I've the followin attrs:

title = "My pictures"
camliRoot = mypics
camliMember = sha-xxx (multiple members from my first attempt setting up publisher)
camliPath:szk = sha-yyy (permanode set, works in publisher)
camliPath:face = sha1-300831992389538979400db5cc8e587bc6316ff8 (permanode with camlicontent, only title is shown)

I get the following in javascript console:

publisher.js:97 Your current jQuery version is: 2.2.3
publisher.js:21 error creating file container: full blobref for prefix sha1-pics not found


the fact that you're getting this message means the check

  if !strings.Contains(basePath, "/-") {

didn't work, and face didn't get detected as a "top node"

Can you add a

println(basePath)

right after

isTopNode := false

and report what it says please?

To unsubscribe from this group and all its topics, send an email to camlistore+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Camlistore" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camlistore+unsubscribe@googlegroups.com.

Attila Tajti

unread,
Jan 19, 2017, 12:21:57 PM1/19/17
to camli...@googlegroups.com
Sorry, it was my mistake. I haven't worked with static files served with their modTimes in a while...

 A refresh in Chrome fixed it. The camliContent is shown in publisher with the CL, and also the UI works fine now.

Attila
To unsubscribe from this group and all its topics, send an email to camlistore+...@googlegroups.com.

Attila Tajti

unread,
Jan 20, 2017, 2:47:05 AM1/20/17
to Camlistore


On Wednesday, January 18, 2017 at 2:01:52 AM UTC+1, mpl wrote:

On 17 January 2017 at 18:28, Attila Tajti <attila...@gmail.com> wrote:
 
Btw, the thumbnails in the publisher show up with an incorrect aspect ratio (apparently the image is streched into a square shape) but standalone images appear fine. Is there anything I can do about it?

Yeah, I did the very minimum amount of math for the thumbs.
You can:
1) as usual file an issue :-) (but that probably won't be in my priorities, sorry) 
2) fix the code at app/publisher/js/members.go. Aaron had done it pretty well for the web UI (I'm still seeing some bugs if I mix images with very different sizes though), so that might be the way to go for inspiration.
 

For square thumbnails to work thumbnail images are needed with proper size (i.e. cropped to 200x200px), perhaps with new query parameters cw and ch (crop width/height) on the server.

The cropping could alternatively be done in the browser, but then an image should be proportionally scaled so that (width >= 200px && height >= 200px && (height == 200px || width == 200px)). Then the server would need store bits of the image that are never shown, therefore it would be better to have it cropped on the server side

The handler would scale the image until either (scaledwidth == cw && scaledheight >= ch) or (scaledheight == ch || scaledwidth >= cw) is true, crop the top/bottom or left/right sides of it so that the thumb will have the exact dimensions cw × ch.

WDYT?
 

Mathieu Lonjaret

unread,
Jan 20, 2017, 7:05:56 PM1/20/17
to camli...@googlegroups.com
wait, why do we need anything new on the server-side, since the web UI is already capable of doing this job pretty well? Why can't the publisher code be improved to do the same thing the web UI does? What am I missing?


--

Attila Tajti

unread,
Jan 24, 2017, 2:29:24 AM1/24/17
to Camlistore
I was wondering if there is a good use case where square thumbnails are needed in the first place. They are easy to work with and I liked iPhoto and Photofloat that used them, but I agree something like the web UI would be far superior.
To unsubscribe from this group and stop receiving emails from it, send an email to camlistore+...@googlegroups.com.

Mathieu Lonjaret

unread,
Jan 24, 2017, 8:34:26 AM1/24/17
to camli...@googlegroups.com
ah, I get it I think. You were trying to see if we could fix things while still keeping only square thumbnails?


To unsubscribe from this group and stop receiving emails from it, send an email to camlistore+unsubscribe@googlegroups.com.

Attila Tajti

unread,
Jan 25, 2017, 1:17:23 AM1/25/17
to Camlistore
Exactly. The square thumbnails with rounded edges in publisher seemed like an artistic/design choice, so I tried to keep them. :)

Mathieu Lonjaret

unread,
Jan 25, 2017, 8:22:43 AM1/25/17
to camli...@googlegroups.com
haha, no. more of "path of least resistance" choice. Any kind of improvement is welcome. Notably, I noticed some scrolling behaviour bug (it scrolls back up by itself) when browsing lots of images on mobile. I hadn't noticed until recently  because I think it does not happen on "desktop".
Anyway, I'm not supposed to dedicate too much time to the publisher these days, but if you want to send some CLs for it I sure will review them. As usual, it's probably better if it can be done in small, incremental CLs but I understand it might be difficult if you want to take a completely different approach/design.

To unsubscribe from this group and stop receiving emails from it, send an email to camlistore+unsubscribe@googlegroups.com.

Attila Tajti

unread,
Apr 25, 2017, 5:54:13 AM4/25/17
to Camlistore
PTAL at https://camlistore-review.googlesource.com/9947 for a simple fix to have thumbnails with the correct image ratio.
Reply all
Reply to author
Forward
0 new messages