VSG OpenStreetMap test

[Bruce Clay]

Walking through various parts of VSG tests I encountered an error with the OpenStreetMap

file vsgExamples\data\models\openstreetmap.vsgt

When I run the viewer on the osm vsgt file I  get the following error

Error: libcurl read error, file=https://a.tile.openstreetmap.org/0/0/0.png error = SSL peer certificate or SSH remote key was not OK

if I run curl https://a.tile.openstreetmap.org/0/0/0.png -o0.png from the command line I get a valid image.

Has anyone tried this file and have any clues to what is happening?

Bruce

[Robert Osfield]

Hi Bruce,

I have just tested on my linux system and it's still working fine. 

I haven't seen reports of libcurl errors when attempting to use openstreetmap.vsgt, but perhaps others have seen this error and simply not reported it.

As a test I've copied the openstreetmap.vsgt and changed the imageLayer URL from https to http, this works just like the unmodified version for me.  Could you try the attached osm.vsgt?

Could you also try the readymap.vsgt?  I.e.

  vsgviewer models/readymap.vsgt

Looking at its imageLayer URL it's http.

I don't know the specific cause or what the solution is but I do wonder if the Linux and Windows defaults used by libcurl aren't the same, if this is the case then we probably need to explicitly set the values in the vsgXchange::curl plugin.

Cheers,

Robert.

[Bruce Clay]

Robert:

  OSM was the only sample that had a problem.  If I change https in the openstreetmap.vsgt to http it works as well.

BTW:I can rotate and zoom vsgViewer but cannot pan.  Is that normal?

Bruce

  

--
You received this message because you are subscribed to a topic in the Google Groups "vsg-users : VulkanSceneGraph Developer Discussion Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vsg-users/X4gwsApENHg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vsg-users+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CAFN7Y%2BWk8cSbwHOKnJ0sR6gf8As3jAbK1dWHqMWX5xOr4He_xA%40mail.gmail.com.

[Robert Osfield]

On Wed, 2 Nov 2022 at 15:05, Bruce Clay <bcla...@gmail.com> wrote:

  OSM was the only sample that had a problem.  If I change https in the openstreetmap.vsgt to http it works as well.

OK, that looks like some https variation in curl between Windows and Linux.  I would like to get this resolved before 1.0 so would appreciate community help on figuring out what might be going on.

 

BTW:I can rotate and zoom vsgViewer but cannot pan.  Is that normal?

It should behave similar to the OSG's trackball manipulator except when associated with an EllipsoidModel like a whole earth model.  Rotation default to using the left mouse button, pan the middle mouse button and zoom the right mouse button. What happens when you do:

   vsgviewer models/lz.vsgt

vs

    vsgviewer models/readymap.vsgt

When associated with an EllipsoidModel the pan will "look" a rotation when a long way from the surface, but if you zoom in you'll find that the pan is happening relative to the surface and will feel more naturally like a pan.  The associated with the EllipsoidModel is do by the vsgviewer itself - it checks for a ElliposoidModel attached to the root node of the loaded scene graph then use that for computing the near/far and constraining the trackball to the EllipsoidModel.

 

This is quite different to the OSG's camera manipulators that require a totally different camera manipulator for standard vs whole earth models.

Does this explanation make sense of the behavior you are seeing?  If not then perhaps the middle mouse button isn't being mapped correctly on your system.

Cheers,

Robert.

["François Cami"]

On Wed, Nov 2, 2022 at 4:23 PM Robert Osfield <robert....@gmail.com> wrote:
> On Wed, 2 Nov 2022 at 15:05, Bruce Clay <bcla...@gmail.com> wrote:
>>
>> OSM was the only sample that had a problem. If I change https in the openstreetmap.vsgt to http it works as well.
>
>
> OK, that looks like some https variation in curl between Windows and Linux. I would like to get this resolved before 1.0 so would appreciate community help on figuring out what might be going on.

It looks like CURLSSLOPT_NATIVE_CA is needed:
https://curl.se/libcurl/c/CURLOPT_SSL_OPTIONS.html
"Tell libcurl to use the operating system's native CA store for
certificate verification. Works only on Windows when built to use
OpenSSL."

See also:
https://www.apachelounge.com/viewtopic.php?t=8627
(search for "CURLSSLOPT_NATIVE_CA" or "set->ssl.native_ca_store = TRUE;")

Bruce, is your curl built using WinSSL?
E.g. what's the output of "curl.exe -V"?
It should be something like this:
curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL

>> BTW:I can rotate and zoom vsgViewer but cannot pan. Is that normal?
>
>
> It should behave similar to the OSG's trackball manipulator except when associated with an EllipsoidModel like a whole earth model. Rotation default to using the left mouse button, pan the middle mouse button and zoom the right mouse button. What happens when you do:
>
> vsgviewer models/lz.vsgt
>
> vs
>
> vsgviewer models/readymap.vsgt
>
> When associated with an EllipsoidModel the pan will "look" a rotation when a long way from the surface, but if you zoom in you'll find that the pan is happening relative to the surface and will feel more naturally like a pan. The associated with the EllipsoidModel is do by the vsgviewer itself - it checks for a ElliposoidModel attached to the root node of the loaded scene graph then use that for computing the near/far and constraining the trackball to the EllipsoidModel.
>
> This is quite different to the OSG's camera manipulators that require a totally different camera manipulator for standard vs whole earth models.
>
> Does this explanation make sense of the behavior you are seeing? If not then perhaps the middle mouse button isn't being mapped correctly on your system.
>
> Cheers,
> Robert.
>

> --
> You received this message because you are subscribed to the Google Groups "vsg-users : VulkanSceneGraph Developer Discussion Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vsg-users+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CAFN7Y%2BWbhj6zYUXqLQUmZsNESFjE07%2B0z0Eq_Sa9JfLmVUfD%2Bw%40mail.gmail.com.

[Bruce Clay]

Francois:

The output of curl is

curl 7.83.1 (Windows) libcurl/7.83.1 Schannel
Release-Date: 2022-05-13
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS HSTS IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI UnixSockets

Bruce

To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CADa9dq0-bzp1zOGmr08qnHqnYgdtPg8DFuRfT--S8PgH_ooVKw%40mail.gmail.com.

[Robert Osfield]

When I wrote vsgXchange::curl I wrote it using OSG's curl plugin and libcurl docs as a guide, developing/testing it under Linux.

The implementation isn't as complete as the OSG's and doesn't have the OSG's password support, so consider it as a first pass to implement basic functionality that could help test the rest of the VSG such as database paging.

My expectation is that we'd need to return to the vsgXchange::curl implementation and expand support when the VSG user base required it.   It might be that we'll need to revisit it now or shortly after VSG-1.0 goes out.

["François Cami"]

On Linux, curl is linked against libcurl so there should not be any discrepancy.
However, it looks like in Windows, you can have a curl using Schannel
(Windows' implementation of SSL, using its own Certficate Store) and a
libcurl compiled against OpenSSL (expecting a CA bundle).

This is explained very well there:
https://www.mail-archive.com/curl-l...@lists.haxx.se/msg00521.html

Bruce, is that the case in your system? Where did you get libcurl from?

> To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CAHemx53xf4GTEyRW9FEkUEf8Ax%3DKk2wLns_zGtvxEQ7EmEFwyA%40mail.gmail.com.

[Bruce Clay]

I downloaded the curl-7.86.0.zip version from https://curl.se/download.html.  I rand cmake and do not recall making any changes but I was pulling and building several packages so I ma not totally sure.  I can send the CMakeCache file if it will help

You received this message because you are subscribed to a topic in the Google Groups "vsg-users : VulkanSceneGraph Developer Discussion Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vsg-users/X4gwsApENHg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vsg-users+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CADa9dq2jw%2BcsqqjRn88Le3dRpgBfG43MxyeqykAJTOtEKM1GrQ%40mail.gmail.com.

["François Cami"]

Can you determine yourself from the CMakeCache whether curl was
configured to build against OpenSSL or Schannel? If not, I'll have a
look.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CAHemx50CuKU1ii_4NMkEiDfG-wj_QvS5iUhxyYuNfoQ%2Bcj9Mnw%40mail.gmail.com.

[Bruce Clay]

It appears that it was not using OpenSSL.  the SSL settings are shown below

//Disable automatic loading of OpenSSL configuration
CURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG:BOOL=OFF

//Use OpenSSL code. Experimental
CURL_USE_OPENSSL:BOOL=OFF

//Enable SSL support
CURL_ENABLE_SSL:BOOL=ON

//Enable BearSSL for SSL/TLS
CURL_USE_BEARSSL:BOOL=OFF

//Enable mbedTLS for SSL/TLS
CURL_USE_MBEDTLS:BOOL=OFF

//Enable NSS for SSL/TLS
CURL_USE_NSS:BOOL=OFF

//enable wolfSSL for SSL/TLS
CURL_USE_WOLFSSL:BOOL=OFF

To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CADa9dq12QsuxgVZpgjcHOLnSMWvzNqvQRNe%3DKMbqHHFi7JcW%3DQ%40mail.gmail.com.

[Robert Osfield]

Another data point in figuring out what is going wrong might be to look at the differences between the OSG's and the simpler VSG curl plugin. 

The OSG doesn't have CURLSSLOPT_NATIVE_CA but it does set CURLOPT_SSL_VERIFYPEER to 0 or 1, defaulting to 1 (enabled), the relevant line in the OSG's curl plugin is:

https://github.com/openscenegraph/OpenSceneGraph/blob/master/src/osgPlugins/curl/ReaderWriterCURL.cpp#L257 Bruce, you could try adding a line similar to the OSG's curl plugin and the CURLSSLOPT_NATIVE_CA suggested by Francois.

["François Cami"]

Maybe with this on top:
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);

I'm starting to think that even if this is fixable in Bruce's case,
this is going to be hard to test, given libcurl can be compiled in
multiple ways.

> --
> You received this message because you are subscribed to the Google Groups "vsg-users : VulkanSceneGraph Developer Discussion Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vsg-users+...@googlegroups.com.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CAFN7Y%2BVsKj-QgnpQD4yYdMeUVbyA13x%2Beo%3Du%3DC7nvkKv7LXOnA%40mail.gmail.com.

[Bruce Clay]

Francois & Robert:

Sorry for the delay. I have been rather swamped.  Hopefully I updated everything correctly.  I believe the error message changed with that update but still does not load with https

Current error message is

libcurl error responseCode = 1, Unsupported protocol
*   Trying 146.75.81.91:443...
* Connected to a.tile.openstreetmap.org (146.75.81.91) port 443 (#0)
* ALPN, offering http/1.1
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0

Error: libcurl read error, file=https://a.tile.openstreetmap.org/0/0/0.png error = SSL peer certificate or SSH remote key was not OK

Bruce

To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CADa9dq2VODYD532KnoGHc5maCV27seinr8Ahx2TtN1Uvj5d3hw%40mail.gmail.com.

["François Cami"]

Hey,

On Fri, Nov 4, 2022 at 6:31 PM Bruce Clay <bcla...@gmail.com> wrote:
>
> Francois & Robert:
>
> Sorry for the delay. I have been rather swamped. Hopefully I updated everything correctly. I believe the error message changed with that update but still does not load with https
>
> Current error message is
>
> libcurl error responseCode = 1, Unsupported protocol
> * Trying 146.75.81.91:443...
> * Connected to a.tile.openstreetmap.org (146.75.81.91) port 443 (#0)
> * ALPN, offering http/1.1
> * SSL certificate problem: unable to get local issuer certificate

Now that's a slightly more deterministic error message.
That's essentially saying it cannot find the CA bundle (OpenSSL) or
your CA bundle is too old.
The issuer's certificate is very recent (2022 Q3) so any slightly
obsolete bundle will not have it:
"issuer=C = BE, O = GlobalSign nv-sa, CN = GlobalSign Atlas R3 DV TLS
CA 2022 Q3"

I tried on an old Linux system ( ca-certificates-2022.2.54-1.2.fc35.noarch ):

$ openssl s_client -CApath /etc/ssl/certs/ -connect a.tile.openstreetmap.org:443

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3390 bytes and written 398 bytes
Verification error: unable to get local issuer certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 20 (unable to get local issuer certificate)
---

then on an updated Linux system ( ca-certificates-2022.2.54-5.fc37.noarch ):

$ openssl s_client -CApath /etc/ssl/certs/ -connect a.tile.openstreetmap.org:443

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3390 bytes and written 408 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

Now I don't know why the curl project does not provide Windows
binaries compiled against Schannel, but on the other hand, they do
provide a libcurl built against OpenSSL with a CA bundle.
I therefore tried openssl using curl's CA bundle (note -CAfile vs -CApath):

curl-7.86.0_1-win64-mingw]$ openssl s_client -CAfile
bin/curl-ca-bundle.crt -connect a.tile.openstreetmap.org:443

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3390 bytes and written 398 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

Success!

So here is what I suggest:

Step #1:
Download "curl for 64-bit" (Windows) there:
https://curl.se/windows/
Essentially this:
https://curl.se/windows/dl-7.86.0_1/curl-7.86.0_1-win64-mingw.zip
This seems to be an OpenSSL build, complete with a curl-ca-bundle.crt (in bin).

Please rebuild vsgXchange using these downloaded (lib)curl binaries.


Step #2
Add the following ( https://curl.se/libcurl/c/CURLINFO_CAINFO.html ):

curl_easy_getinfo(curl, CURLINFO_CAINFO, &cainfo);
if(cainfo)
printf("default ca info path: %s\n", cainfo);
}

If there is a default path: copy the CA bundle (from curl's bin) to
the default CA path determined above.
Otherwise add ( https://curl.se/libcurl/c/CURLOPT_CAINFO.html ):

curl_easy_setopt(curl, CURLOPT_CAINFO, "C:\path\to\your\bundle");

Step #3
Retry :)

Bruce, thanks for your patience. I'm totally working blind here, I
don't have Windows.

I think that using the curl project provided binaries and their
enclosed CA bundle is the way to go.
IMHO this is not vsgXchange's problem. The system should be setup to
work properly, but it's really hard on Windows...

François

> To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CAHemx53er5cUjyWEQdQfcOmoDG5f3hytZ3Cky972xndc0NaJ6A%40mail.gmail.com.

[Bruce Clay]

Francois:

  I rebuilt vsgXchange with the curl from the link you mentioned then I added the following to the top of vsgconv.

    char* cainfo = NULL;
    CURL* curl = curl_easy_init();

    curl_easy_getinfo(curl, CURLINFO_CAINFO, &cainfo);
    if (cainfo)

        printf("default ca info path: %s\n", cainfo);

    else
        curl_easy_setopt(curl, CURLOPT_CAINFO, "G:/SupportLibraries3/curl-7.86.0_1-win64-mingw/bin/curl-ca-bundle.crt");

I waited to copy the new dll into the path to make sure it was looking for the right file then I ran vsgconv from the debugger using 

G:\SupportLibraries3\VulkanSceneGraph\vsgExamples\data\models\openstreetmap.vsgt osm.png as the arguments.

cainfo was returned empty do the bundle with the ming package was used

It still did not work.  the error message was

libcurl error responseCode = 60, SSL peer certificate or SSH remote key was not OK

Error: libcurl read error, file=https://a.tile.openstreetmap.org/0/0/0.png error = SSL peer certificate or SSH remote key was not OK

Thanks for your help with this problem.

Bruce

To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CADa9dq0CcPLN1vfwjp0vm3H8mSQxT1%2BRQxOR42806GCtsE79yw%40mail.gmail.com.

["François Cami"]

On Sat, Nov 5, 2022 at 1:30 PM Bruce Clay <bcla...@gmail.com> wrote:
>
> Francois:

>
> I rebuilt vsgXchange with the curl from the link you mentioned then I added the following to the top of vsgconv.
>
> char* cainfo = NULL;
> CURL* curl = curl_easy_init();
>
> curl_easy_getinfo(curl, CURLINFO_CAINFO, &cainfo);
> if (cainfo)
> printf("default ca info path: %s\n", cainfo);
> else
> curl_easy_setopt(curl, CURLOPT_CAINFO, "G:/SupportLibraries3/curl-7.86.0_1-win64-mingw/bin/curl-ca-bundle.crt");
>
> I waited to copy the new dll into the path to make sure it was looking for the right file then I ran vsgconv from the debugger using
> G:\SupportLibraries3\VulkanSceneGraph\vsgExamples\data\models\openstreetmap.vsgt osm.png as the arguments.
>
> cainfo was returned empty do the bundle with the ming package was used
>
> It still did not work. the error message was
>
> libcurl error responseCode = 60, SSL peer certificate or SSH remote key was not OK
> Error: libcurl read error, file=https://a.tile.openstreetmap.org/0/0/0.png error = SSL peer certificate or SSH remote key was not OK
>
> Thanks for your help with this problem.

You're welcome, but it's obvious I have no idea what's going on on
your system ;)

I built a little demo, adapting curl upstream's url2file.c for more debugging:
https://github.com/fcami/gists/tree/main/curl_ssl_demo

The tool can either use the default CA bundle, or a specified one (see README).
I've included two bundles, one of them fails on
https://a.tile.openstreetmap.org/0/0/0.png .

Let us know what happens.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CAHemx510Fj33w9qsG%3DmrgM--Kp2VYTOYh%3DzA8J8My9dcfHM0RQ%40mail.gmail.com.

[Robert Osfield]

Unfortunately I don't have any insights to add for getting to the bottom of the https issues Bruce is seeing, and as I have a bunch of VSG-1.0 review and prep work will have to concentrate on this for now.

For the release I think changing openstreetmap.vsgt to use http instead of https is the quickest workaround.  I don't consider it a solution  as vsgXchange::curl looks like it'll need further work, or at minimum for us to come up with a set of recommendations of how to install curl on Windows to avoid gotchas.

Cheers,

Robert.