PUBLISHER: could not find permanode for root "pics" of publish handler: file does not exist

[Tamás Gulácsi]

This is what I got in camlistored log for

gthomas@tequila:~$ cat .config/camlistore/server-config.json
{
    "listen": "0.0.0.0:3179",
    "baseURL": "https://camli.gthomas.eu",
    "shareHandler": true,
    "https": false,
    "httpsCert": "/home/gthomas/.config/camlistore/camli.gthomas.eu.crt",
    "httpsKey": "/home/gthomas/.config/camlistore/camli.gthomas.eu.key",
    "auth": "userpass:gthomas:majdhafagy:+localhost",
    "identity": "974EA38B",
    "identitySecretRing": "/home/gthomas/.config/camlistore/identity-secring.gpg", 
    "levelDB": "/home/gthomas/var/camlistore/camli-index.leveldb",
    "blobPath": "/home/gthomas/var/camlistore/packs", 
    "packBlobs": true,
        "runIndex": true,
        "copyIndexToMemory": true,
    "mysql": "", 
    "mongo": "",
    "postgres": "", 
    "sqlite": "",
    "s3": "", 
    "replicateTo": [],
        "publish": {
          "/pics/": { 
             "camliRoot": "pics",
                 "cacheRoot": "/home/gthomas/var/camlistore/blobs/cache", 
                 "goTemplate": "gallery.html"
       }
     }
}

gthomas@tequila:~$ camget sha1-de682600ddf64620b322971c94e7911bebe4865b
{"camliVersion": 1,
  "camliSigner": "sha1-01c5e458c48552abac802d4f8b52b093efbb2caa",
  "camliType": "permanode",
  "key": "pics"
,"camliSig":"wsBcBAABCAAQBQJXE4QgCRAdaMP0l06jiwAAG0gIAIZlrpoWvCFnjptlprCE2QnlTma+R63G/2PLKlY5oZPC7p/yBQp+6ESBN5le3ohhDvp1TMWcuq1bFDushTYqdqparu6ZFGgf0NKKXaO47PYbPdDDUJcuZQ3dsCWguXcbT0Vaik297sQirakGUZ+TRisveqiWdswAx3OYFq1YpxQG542uF70RK6lHPO3si4mf5l30A6KNNc28W0/lHfSHteKNZxn1sFXw2nbRPF6JdXXq8YXXt28mOetcqXj9XhBfZbP0zrMEZ2NPOljIzaUSGCmAb2Xu/oTix5w7j/Zg7C+dBJ9JElXGfmYoTZcMpjw4UqS6CGhdjHVcgB6nX5elKJE==4waQ"}

I get same error if I change the "camliRoot" to "sha1-de682600ddf64620b322971c94e7911bebe4865b", so now I don't know what to do.


What kind of permanode does publish need for its root?
Why is that "mypics" (the key) in the example config?
Why doesn't this work?

Thanks,
Tamás Gulácsi

[Mathieu Lonjaret]

The permanode that you want to use as your root for the publisher just
needs to have its "camliRoot" attribute set to the value that you
define in your server-config, there's nothing more to it.
So based on the server-config you showed, if you want to use
sha1-de682600ddf64620b322971c94e7911bebe4865b as the root, just make
sure its "camliRoot" attribute is set to "pics".

Then every time you want to publish a permanode/set, you add it as a
camliPath attribute to the above root permanode, e.g.
"camliPath:seaside-vacation-2016" : "sha1-ref_of_set_of_pictures"

> --
> You received this message because you are subscribed to the Google Groups
> "Camlistore" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to camlistore+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Tamás Gulácsi]

Thanks!

So this two commands was missing:

camput attr sha1-de682600ddf64620b322971c94e7911bebe4865b camliRoot pics
camput attr -add sha1-de682600ddf64620b322971c94e7911bebe4865b camliPath:20160507-Bori_mazsi sha1-7591a5e16ce83625b6e7a7e65a099142af328d9c

Is this documented, just I was blind?


Now another obstacle:
 PUBLISHER: 2016/05/09 21:34:47 Listen: Failed to listen on camli.gthomas.eu:42902: listen tcp 176.63.179.118:42902: bind: cannot assign requested address

That IP is my server's outer ip, but I'm behind a router, so can't bind on it. It should bind to 127.0.0.1 only.
It is derived (in pkg/server/app/app.go) from the base URL of the handlerLoader.

If I change it to "http://localhost:3179" (to the address where my camlistored is listening on), then the publisher can start, bat fails with
PUBLISHER: 2016/05/09 21:59:19 could not get app config at http://localhost/pics/config.json: client: got status code 404 from URL http://localhost/pics/config.json; body No such host at :80


Maybe the culprit is that I have camlistored listen on http://localhost:3179 and have caddy fiddle with SSL (Let's Encrypt) as a proxy listening on https://camli.gthomas.eu:443.

Do you have some idea how to straighten this up?

[Tamás Gulácsi]

2016. május 9., hétfő 22:03:00 UTC+2 időpontban Tamás Gulácsi a következőt írta:

2016. május 9., hétfő 16:04:30 UTC+2 időpontban mpl a következőt írta:

The permanode that you want to use as your root for the publisher just
needs to have its "camliRoot" attribute set to the value that you
define in your server-config, there's nothing more to it.
So based on the server-config you showed, if you want to use
sha1-de682600ddf64620b322971c94e7911bebe4865b as the root, just make
sure its "camliRoot" attribute is set to "pics".

Then every time you want to publish a permanode/set, you add it as a
camliPath attribute to the above root permanode, e.g.
"camliPath:seaside-vacation-2016" : "sha1-ref_of_set_of_pictures"

Thanks!

So this two commands was missing:

camput attr sha1-de682600ddf64620b322971c94e7911bebe4865b camliRoot pics
camput attr -add sha1-de682600ddf64620b322971c94e7911bebe4865b camliPath:20160507-Bori_mazsi sha1-7591a5e16ce83625b6e7a7e65a099142af328d9c

Is this documented, just I was blind?


Now another obstacle:
 PUBLISHER: 2016/05/09 21:34:47 Listen: Failed to listen on camli.gthomas.eu:42902: listen tcp 176.63.179.118:42902: bind: cannot assign requested address

That IP is my server's outer ip, but I'm behind a router, so can't bind on it. It should bind to 127.0.0.1 only.
It is derived (in pkg/server/app/app.go) from the base URL of the handlerLoader.

If I change it to "http://localhost:3179" (to the address where my camlistored is listening on), then the publisher can start, bat fails with
PUBLISHER: 2016/05/09 21:59:19 could not get app config at http://localhost/pics/config.json: client: got status code 404 from URL http://localhost/pics/config.json; body No such host at :80


Maybe the culprit is that I have camlistored listen on http://localhost:3179 and have caddy fiddle with SSL (Let's Encrypt) as a proxy listening on https://camli.gthomas.eu:443.

Do you have some idea how to straighten this up?

To answer my question: adding camli.gthomas.eu to /etc/hosts as 127.0.0.1 fixes this problem, publisher starts flawlessly.

But something is wrong with the proxy:

 PUBLISHER: 2016/05/10 19:52:28 Starting publisher version 2016-05-10-044c7b3; Go go1.6.2 (linux/amd64)
2016/05/10 19:52:28 Starting regular periodic import for picasa importer account, sha1-d9452176e1f13387052dc5951ad80a3776901a93
2016/05/10 19:52:28 Starting picasa importer account, sha1-d9452176e1f13387052dc5951ad80a3776901a93: userid 110475319045955272364 (Tamás Gulácsi)
2016/05/10 19:52:28 auth: local connection uid 33 doesn't match server uid 1000
2016/05/10 19:52:28 auth: local connection uid 33 doesn't match server uid 1000
PUBLISHER: 2016/05/10 19:52:28 Starting to listen on http://127.0.0.1:40792
2016/05/10 19:52:28 Available on https://camli.gthomas.eu/ui/
2016/05/10 19:52:33 picasa importer account, sha1-d9452176e1f13387052dc5951ad80a3776901a93 finished.
2016/05/10 19:52:33 picasa importer account, sha1-d9452176e1f13387052dc5951ad80a3776901a93 ran recently enough. Sleeping for 29m59.999823374s.
2016/05/10 19:52:51 auth: local connection uid 33 doesn't match server uid 1000
2016/05/10 19:53:02 auth: local connection uid 33 doesn't match server uid 1000
2016/05/10 19:53:16 auth: local connection uid 33 doesn't match server uid 1000
2016/05/10 19:55:00 http: proxy error: tls: oversized record received with length 20527


As far as I know, caddy ends TLS, so camlistored shouldn't want to proxy HTTPS.

[Tamás Gulácsi]

But something is wrong with the proxy:

 PUBLISHER: 2016/05/10 19:52:28 Starting publisher version 2016-05-10-044c7b3; Go go1.6.2 (linux/amd64)
2016/05/10 19:52:28 Starting regular periodic import for picasa importer account, sha1-d9452176e1f13387052dc5951ad80a3776901a93
2016/05/10 19:52:28 Starting picasa importer account, sha1-d9452176e1f13387052dc5951ad80a3776901a93: userid 110475319045955272364 (Tamás Gulácsi)
2016/05/10 19:52:28 auth: local connection uid 33 doesn't match server uid 1000
2016/05/10 19:52:28 auth: local connection uid 33 doesn't match server uid 1000
PUBLISHER: 2016/05/10 19:52:28 Starting to listen on http://127.0.0.1:40792
2016/05/10 19:52:28 Available on https://camli.gthomas.eu/ui/
2016/05/10 19:52:33 picasa importer account, sha1-d9452176e1f13387052dc5951ad80a3776901a93 finished.
2016/05/10 19:52:33 picasa importer account, sha1-d9452176e1f13387052dc5951ad80a3776901a93 ran recently enough. Sleeping for 29m59.999823374s.
2016/05/10 19:52:51 auth: local connection uid 33 doesn't match server uid 1000
2016/05/10 19:53:02 auth: local connection uid 33 doesn't match server uid 1000
2016/05/10 19:53:16 auth: local connection uid 33 doesn't match server uid 1000
2016/05/10 19:55:00 http: proxy error: tls: oversized record received with length 20527


As far as I know, caddy ends TLS, so camlistored shouldn't want to proxy HTTPS.

Forcing the proxyURL to be http helps:

diff --git a/pkg/server/app/app.go b/pkg/server/app/app.go
index b025247..020e2a6 100644
--- a/pkg/server/app/app.go
+++ b/pkg/server/app/app.go
@@ -166,6 +166,9 @@ func NewHandler(conf jsonconfig.Obj, apiHost, appHandlerPrefix string) (*Handler
        if err != nil {
                return nil, fmt.Errorf("could not parse backendURL %q: %v", backendURL, err)
        }
+       if proxyURL.Scheme == "https" {
+               proxyURL.Scheme = "http"
+       }
        return &Handler{
                name:       name,
                envVars:    envVars,
 
But I don't know how to make this a more acceptable, less barbaric change...

[Mathieu Lonjaret]

Sorry, I forgot about that thread.
Shouldn't you rather change listen to:

"listen": "camli.gthomas.eu:3179",
?
Does that conflict with your caddy proying?

[Mathieu Lonjaret]

fwiw, as part of the scanning cabinet CL
(https://camlistore-review.googlesource.com/5416) I have also reworked
a lot of the app handler business, so maybe the changes in there would
also fix things for you?

I should probably pull these changes out of that CL so we don't have
to wait for the scanning cabinet to land them. I think I'll do it
soon.

[Mathieu Lonjaret]

From https://camlistore.org/doc/server-config

"One can create any permanode with camput or the UI, and set its
camliRoot attribute to the value set in the config, to use it as the
root permanode for publishing."

> camput attr -add sha1-de682600ddf64620b322971c94e7911bebe4865b
> camliPath:20160507-Bori_mazsi sha1-7591a5e16ce83625b6e7a7e65a099142af328d9c

But indeed, in https://camlistore.org/doc/schema/attributes.md
camliRoot could use some doc.

> Is this documented, just I was blind?

So you're only half-blind ;P

> Now another obstacle:
> PUBLISHER: 2016/05/09 21:34:47 Listen: Failed to listen on
> camli.gthomas.eu:42902: listen tcp 176.63.179.118:42902: bind: cannot assign
> requested address
>
> That IP is my server's outer ip, but I'm behind a router, so can't bind on
> it. It should bind to 127.0.0.1 only.
> It is derived (in pkg/server/app/app.go) from the base URL of the
> handlerLoader.
>
> If I change it to "http://localhost:3179" (to the address where my
> camlistored is listening on), then the publisher can start, bat fails with
> PUBLISHER: 2016/05/09 21:59:19 could not get app config at
> http://localhost/pics/config.json: client: got status code 404 from URL
> http://localhost/pics/config.json; body No such host at :80
>
>
> Maybe the culprit is that I have camlistored listen on http://localhost:3179
> and have caddy fiddle with SSL (Let's Encrypt) as a proxy listening on
> https://camli.gthomas.eu:443.
>
> Do you have some idea how to straighten this up?
>

[Adrian Tritschler]

I'm not sure if you intended it, but you've posted the URL, login and password of your camlistore.  You may wish to change the password.

  Adrian

[Gulácsi Tamás]

Thanks!

Changed.

--
You received this message because you are subscribed to a topic in the Google Groups "Camlistore" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/camlistore/bQlWEjy0i7o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to camlistore+...@googlegroups.com.

[Mathieu Lonjaret]

Hey Tamás,

Could you please let me know if
https://camlistore-review.googlesource.com/6646 makes it easier for
you to run the publisher with your setup?
Note that in the high-level config for the publisher, baseURL is now
backendURL, and that you can now specify "listen" too.

thanks,
Mathieu

> You received this message because you are subscribed to the Google Groups
> "Camlistore" group.
> To unsubscribe from this group and stop receiving emails from it, send an

[Mathieu Lonjaret]

Alright, so let's discuss your setup so I can try to reproduce it please.

Is Caddy the software facing the outside world? And it is listening on
https://camli.gthomas.eu:443 ?
And do I understand correctly that the end goal for you is to have the
publisher displayed when someone hits https://camli.gthomas.eu/pics/ ?
How do you make Caddy proxy the relevant requests to your Camlistore instance?

[Gulácsi Tamás]

Yes.

Caddy is listening on 0.0.0.0:443, forwarding anything for https://camli.gthomas.eu:443/ to http://localhost:3179.

So Camlistored is listening on http://127.0.0.1:3179.

I can make Caddy proxy https://camli.gthomas.eu/pics/ to somewhere else, but now it goes to camlistored, without modification.

[Mathieu Lonjaret]

I've been making changes so that the app should work by itself (i.e.
when getting requests directly, not proxied through camlistored), but
let's stay with the usual case for now, i.e. camlistored gets the
requests and its app handler proxies them to the publisher. So yes,
let's keep your Caddy proxying as it is.

Next question: if Caddy is listening on TLS, why aren't you doing the
same thing for camlistored? Why do you want unencrypted traffic
between Caddy and Camlistore?

[Gulácsi Tamás]

It's been Caddy, as at that time only she offered no fuss automatic Let's Encrypt certs.

I see now camlistored supports it, too.

I've transformed my config to:

Caddy listens on https://camli.gthomas.eu, forwards the connection to https://127.0.0.1:3179, where camlistored listens.

But I get
PUBLISHER: 2016/05/27 20:51:26 Starting publisher version 2016-05-23-8d4f18e; Go go1.6.2 (linux/amd64)
PUBLISHER: 2016/05/27 20:51:26 Starting to listen on https://127.0.0.1:33699
2016/05/27 20:51:26 Available on https://camli.gthomas.eu/ui/
2016/05/27 20:51:26 http: TLS handshake error from 127.0.0.1:50550: EOF
2016/05/27 20:51:30 http: proxy error: x509: certificate signed by unknown authority
2016/05/27 20:51:30 http: TLS handshake error from 127.0.0.1:50556: remote error: bad certificate

even with 6646/2.

What kind of config works?

[Mathieu Lonjaret]

It looks like your camlistored is not using your let's encrypt cert,
but an auto-gen one instead, isn't it? If you don't specify the cert
to use with httpsCert and httpsKey in the server config, it will use
the ones found at .config/camlistore/tls.[crt|key]. So I think you
need to fix that first.

Then, to answer your question about the config, I've just tried with a
config like that:

{
"auth": "userpass:foo:bar",
"listen": ":3179",
"baseURL": "https://camli.gthomas.eu:3179",
"https": true,
"identity": "BF117793",
"identitySecretRing": "/home/mpl/.config/camlistore/identity-secring.gpg",
"blobPath": "/home/mpl/var/camlistore/blobs",
"packRelated": true,
"levelDB": "/home/mpl/var/camlistore/index.leveldb",
"publish": {
"/pics/": {
"camliRoot": "picsRoot",
"goTemplate": "gallery.html"
}
},
"dbNames": null
}

and it looks like everything is working for me. Except I'm not hitting
a Caddy front-end first of course. I can try to setup an equivalent
proxy if you think the Caddy part is still making a difference.

[Gulácsi Tamás]

Hmm.. I've never thought about it before, but symlinking the .caddy/sites/camli.gthomas.eu/*.{key,crt} to their tls.{key,crt} pairs just works!

TL;DR; now it works, if I allow everything to use HTTPS with the correct letsencrypt certificates.

But publisher says

PUBLISHER: 2016/05/30 20:56:16 Starting publisher version 2016-05-29-7b9b9d5; Go go1.6.2 (linux/amd64)
2016/05/30 20:56:16 Starting regular periodic import for picasa importer account, sha1-d9452176e1f13387052dc5951ad80a3776901a93
PUBLISHER: 2016/05/30 20:56:16 Starting to listen on https://127.0.0.1:41599
2016/05/30 20:56:16 Available on https://camli.gthomas.eu/ui/
2016/05/30 20:56:16 http: TLS handshake error from 127.0.0.1:46730: EOF

Either with tip or with cl-6646. Maybe Camlistore proxies for the publisher app, but that app serves an unknown cert?


gthomas@tequila:~$ curl -k -v https://localhost:41613
* Rebuilt URL to: https://localhost:41613/
*   Trying ::1...
* connect to ::1 port 41613 failed: Connection refused
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 41613 (#0)
* found 175 certificates in /etc/ssl/certs/ca-certificates.crt
* found 800 certificates in /etc/ssl/certs
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
*        server certificate verification SKIPPED
*        server certificate status verification SKIPPED
*        common name: camli.gthomas.eu (does not match 'localhost')
*        server certificate expiration date FAILED
*        server certificate activation date OK
*        certificate public key: RSA
*        certificate version: #3
*        subject: CN=camli.gthomas.eu
*        start date: Sat, 05 Dec 2015 19:32:00 GMT
*        expire date: Fri, 04 Mar 2016 19:32:00 GMT
*        issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X1
*        compression: NULL
* ALPN, server accepted to use h2
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* TCP_NODELAY set
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55b937028700)
> GET / HTTP/1.1
> Host: localhost:41613
> User-Agent: curl/7.47.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2.0 200
< content-type:text/html; charset=utf-8
< date:Mon, 30 May 2016 19:12:14 GMT
<

This is where I learnt that I need to use the proper letsencrypt certs - as caddy is running as www-data, its under that's home, /var/www.

After fixing the permissions, everything works - both with cl-6646, and tip!

Thanks Mathieu!

[Mathieu Lonjaret]

Cool, I'm glad it's working for you.

What do you think about the CL though? Does it make things easier/clearer?

It might be possible to make things work with Caddy listening on https
and camlistored behind doing only http, but I feel that's asking for
trouble. And I don't think that's a priority, so until someone
convinces me their setup requires them to do that, I'm probably not
gonna think about it.