Stardog Connection Pool using HTTPS w/ Apache causes weird stardog error

[Kevin Cho]

Hello,

I have an urgent question ask (we do have official license).  When we do Stardog Connection Pool using HTTP/Snarl then everything seems ok but not so when it involves SSL using Apache Proxy Server.  The strange errors we see is mostly related to data being missing from SPARQL queries.  However, if we use non-ssl then we never get those errors.  Anyways, I'm beginning to suspect that pooling using proxy server is not working as intended.  For example

Our Web App -> HTTPS to Apache -> HTTP to Stardog

From our web app, we pool the connection using HTTPS.  So, I do know that our connection pool is good from Web App -> Apache.  However, I'm not so sure about connection between Apache -> Stardog.  Not quite sure if HTTP stays alive or gets timed out or something else.  Can you guys help to troubleshoot on why it's not working? I wish I can give more details but I'm very puzzled at this point.

Also, do you guys know of any customers who has successfully used Connection Pooling w/ HTTPS? I'd appreciate a fast response if possible.  Thank you for your support.

Kevin

[Mike Grove]

On Mon, Feb 4, 2013 at 11:12 AM, Kevin Cho <kch...@gmail.com> wrote:

Hello,

I have an urgent question ask (we do have official license).  When we do Stardog Connection Pool using HTTP/Snarl then everything seems ok but not so when it involves SSL using Apache Proxy Server.  The strange errors we see is mostly related to data being missing from SPARQL queries. 

Can you clarify how data is missing from a query?

 

However, if we use non-ssl then we never get those errors.  Anyways, I'm beginning to suspect that pooling using proxy server is not working as intended.  For example

Our Web App -> HTTPS to Apache -> HTTP to Stardog

So the bit in the middle, that's some custom code you wrote?  What precisely is it doing, and how is your webapp using it?

It's hard to say what could be going on without some details.  

Also, you might want to reference the section in the documention [1] that talks about secure deployments of Stardog, in particular, using SSL with Stardog.

Cheers,

Mike

[1] http://stardog.com/docs/security/#deployment

 

From our web app, we pool the connection using HTTPS.  So, I do know that our connection pool is good from Web App -> Apache.  However, I'm not so sure about connection between Apache -> Stardog.  Not quite sure if HTTP stays alive or gets timed out or something else.  Can you guys help to troubleshoot on why it's not working? I wish I can give more details but I'm very puzzled at this point.

Also, do you guys know of any customers who has successfully used Connection Pooling w/ HTTPS? I'd appreciate a fast response if possible.  Thank you for your support.

Kevin

--
-- --
You received this message because you are subscribed to the C&P "Stardog" group.
To post to this group, send email to sta...@clarkparsia.com
To unsubscribe from this group, send email to
stardog+u...@clarkparsia.com
For more options, visit this group at
http://groups.google.com/a/clarkparsia.com/group/stardog?hl=en
 
 

[Kevin Cho]

Thanks for the quick reply.  I do not have any custom code in the middle.  I simply followed the SSL documentation for the stardog

<VirtualHost stardog-dev.dpm.com:443>
        SSLEngine on
        SSLCertificateFile /etc/pki/tls/certs/stardog-dev.crt
        SSLCertificateKeyFile /etc/pki/tls/private/stardog-dev.key
        <Directory /var/www/vhosts/stardog-dev/httpsdocs>
         AllowOverride All
      SSLRequireSSL
        </Directory>
    ProxyPass        /        http://127.0.0.1:5822/
        DocumentRoot /var/www/vhosts/stardog-dev/httpsdocs
        ServerName stardog-dev.dpm.com
</VirtualHost>

[Mike Grove]

On Mon, Feb 4, 2013 at 11:51 AM, Kevin Cho <kch...@gmail.com> wrote:

Thanks for the quick reply.  I do not have any custom code in the middle.  I simply followed the SSL documentation for the stardog

<VirtualHost stardog-dev.dpm.com:443>
        SSLEngine on
        SSLCertificateFile /etc/pki/tls/certs/stardog-dev.crt
        SSLCertificateKeyFile /etc/pki/tls/private/stardog-dev.key
        <Directory /var/www/vhosts/stardog-dev/httpsdocs>
         AllowOverride All
      SSLRequireSSL
        </Directory>
    ProxyPass        /        http://127.0.0.1:5822/
        DocumentRoot /var/www/vhosts/stardog-dev/httpsdocs
        ServerName stardog-dev.dpm.com
</VirtualHost>

Kevin,

I am not sure this configuration is correct.  First, specifying a path on disk that you're trying to secure will not work.  I guess the httpsdocs directly is your stardog home?

I think the configuration needs to be as shown in the documentation:

<Directory />
        SSLRequireSSL
</Directory>
ProxyPass        /        http://127.0.0.1:5822/




Was there a reason you tried to specify a document root?

Further, hitting the stardog-dev.dpm.com server via a browser or telnet yields SSL errors.  Via telnet, I actually get a plaintext response.  So I think the configuration of the SSL proxy is not correct, which might be the explanation for whatever issues you are experiencing.

Cheers,

Mike

[Kevin Cho]

I tried to use your configuration but issue still remains.  I do belive the HTTPS does work because about 90% of the requests goes through fine.  I've changed the configuration to

<VirtualHost stardog-dev.dpm.com:443>
        SSLEngine on
        SSLCertificateFile /etc/pki/tls/certs/stardog-dev.crt
        SSLCertificateKeyFile /etc/pki/tls/private/stardog-dev.key

        <Directory />
      SSLRequireSSL
        </Directory>
    ProxyPass        /        http://127.0.0.1:5822/

        ServerName stardog-dev.dpm.com
</VirtualHost>

Just curious, do you have any customer who uses connection pool w/ https?

[Mike Grove]

On Mon, Feb 4, 2013 at 12:55 PM, Kevin Cho <kch...@gmail.com> wrote:

I tried to use your configuration but issue still remains.  I do belive the HTTPS does work because about 90% of the requests goes through fine.  I've changed the configuration to

I don't know what else to suggest then; you still have not given a complete description of what the issue precisely is, so I do not know what else to suggest.  From what you've described, and what I can tell trying to access your server via telnet or my browser, SSL is misconfigured.

 

<VirtualHost stardog-dev.dpm.com:443>
        SSLEngine on
        SSLCertificateFile /etc/pki/tls/certs/stardog-dev.crt
        SSLCertificateKeyFile /etc/pki/tls/private/stardog-dev.key

        <Directory />
      SSLRequireSSL
        </Directory>
    ProxyPass        /        http://127.0.0.1:5822/

        ServerName stardog-dev.dpm.com
</VirtualHost>

Just curious, do you have any customer who uses connection pool w/ https?

Yes, we do.

[Kevin Cho]

That's exactly what I'm trying to pin point because the error happens in random requets.  If I rerun the same requests after the init error then goes through fine.  However, if I use either HTTP/Snarl w/o proxy server then I don't see any problem... I do understand I can't give more details because I just don't know what's going on in the Apache.  With that said, do you have any suggestions I should try to pinpoint the error?

[Kendall Clark]

What is the thing that happens randomly? That's the thing we're still
not clear about.

Also, Wireshark (or the like) is yr friend here.

Cheers,
Kendall

[Mike Grove]

On Mon, Feb 4, 2013 at 1:28 PM, Kendall Clark <ken...@clarkparsia.com> wrote:

What is the thing that happens randomly? That's the thing we're still
not clear about.

Also, Wireshark (or the like) is yr friend here.

You might also tail your apache log to see if there is anything useful that Apache is complaining about.

Cheers,

Mike

[Kevin Cho]

The part that acts randomly is that some sparql are returning empty results under HTTPS and in our java code would fail due to that.  Yup, we have used Wireshark to verify the SSL is working.  I thought that maybe the request received by Apache is not going in the same order to Stardog.  However, when I compare Apache request log vs Wireshark..they matched the sequences as well. 

We would really like try deploying stardog to a web server if possible.  Is this something we can try w/ latest version?

[Mike Grove]

On Mon, Feb 4, 2013 at 2:38 PM, Kevin Cho <kch...@gmail.com> wrote:

The part that acts randomly is that some sparql are returning empty results under HTTPS and in our java code would fail due to that.  Yup, we have used Wireshark to verify the SSL is working.  I thought that maybe the request received by Apache is not going in the same order to Stardog.  However, when I compare Apache request log vs Wireshark..they matched the sequences as well. 

Stardog isn't going to randomly return empty results, and since when you're not using Apache to handle HTTPS traffic and forward onto the server, Stardog works, it seems unlikely that this is indicative of a new bug in Stardog.  I still suggest you double check your setup because based on your description, it seems like the issue is with the forwarding configuration.  Again, I get SSL configuration errors when simply trying to connect to your server over HTTPS via a browser or telnet.  You might want to also verify that there is no caching taking place with Apache.

 

We would really like try deploying stardog to a web server if possible.  Is this something we can try w/ latest version?

We don't provide any war artifact that you can drop into a servlet container.  You'd have to write one that would wrap Stardog running in embedded mode.

[Kevin Cho]

Thanks Mike.

Out of pure luck, we found that the same error happens w/ non SSL using HTTP.  When using Snarl, the query always returns a result but when using HTTP it comes empty randomly.  I'd say about 1 out of 20~50 try.  We're trying to determine to see if it's our code doing that or the stardog.  I don't think it was the apache issue as I thought it was..it just happned that when I turn on HTTP..the error suddenly disappeared while it randomly appeared on HTTPS.

Kevin