SonarQube 6.0 fails to detect that the server is using SSL

[bra...@smartercode.net]

I've been trying to diagnose why my sonar-runner is failing, and I've discovered the following. It looks like in 6.0, the logic to determine whether the server is running over HTTPS for creating links and redirects now exclusively looks for the X-Forwarded-Proto header (and ignores the serverBaseURL variable) as described in this issue: https://jira.sonarsource.com/browse/SONAR-7856

However this does not appear to be working correctly. I have SonarQube behind nginx, and am setting the correct X-Forwarded-Proto header. However SonarQube is still incorrectly generating a redirect link with the http scheme, as shown in the traffic capture below:

> curl -v https://my.sonarhost.com:8443/batch_bootstrap/index

GET /batch_bootstrap/index HTTP/1.0

X-Real-IP: 111.222.111.222

X-Forwarded-For: 111.222.111.222

Host: my.sonarhost.com:8443

X-Forwarded-Proto: https

Connection: close

User-Agent: curl/7.43.0

Accept: */*

HTTP/1.1 302 Found

Server: Apache-Coyote/1.1

Location: http://my.sonarhost.com:8443/batch/index

Content-Length: 0

Date: Mon, 15 Aug 2016 21:34:53 GMT

Connection: close

[Julien Lancelot]

Hi Brandon,

This ticket SONAR-7856 is only about authentication, when creating cookies.

You still need to correctly set "sonar.core.serverBaseURL" in order to correctly create links and do redirection.

Regards,

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/f682a833-dad2-4d62-9022-455871735e33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Julien LANCELOT | SonarSource

http://sonarsource.com

[bra...@smartercode.net]

Hi Julien,

I can confirm I do have sonar.core.serverBaseURL set to https://my.sonarhost.com:8443 and am still seeing the behavior described above. Is there something else I might be missing? This was working previously, before upgrading from SonarQube 5.6 to 6.0.

[Eric Hartmann]

Hi Brandon,

SonarQube is sending a relative redirect (/batch/index) to the browser, however according to your curl example the redirect becomes an absolute redirect (http://my.sonarhost.com....).

So something is broken with the reverse proxy, since the Server header in the response is set to "Apache-Coyote/1.1", I suppose that you are not using libtcnative for the connection between Nginx and SonarQube.

So can you post your nginx configuration ? And it may worth trying a classic http connection between Nginx and SonarQube,

Cheers,

[bra...@smartercode.net]

Hi Eric,

To clarify, the request-response I included in my first message was captured using tcpdump between nginx and SonarQube while processing my cURL request - nginx sends the GET request in blue to SonarQube, and SonarQube replies with the 302 back to nginx. That's why you're seeing 'Apache-Coyote' in the Server header. However it also shows that SonarQube is sending the location header as an absolute URL before it even reaches nginx. I've included my nginx configuration below in case it's of some help, but given the info above, it doesn't seem likely that it's the culprit here.

server {
    server_name my.sonarhost.com;
    root /var/www/sonarqube;

    listen 8443 ssl http2;

    ssl_certificate /etc/ssl/certs/my_ssl.crt;
    ssl_certificate_key /etc/ssl/private/my_ssl.key;
    add_header Strict-Transport-Security max-age=31536000;

    client_max_body_size 150M;

    location / {
            proxy_pass http://127.0.0.1:9000;
            proxy_set_header        X-Real-IP $remote_addr;
            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header        Host $http_host;
            proxy_set_header        X-Forwarded-Proto https;
    }
}

[Eric Hartmann]

Hi Brandon,

I've reproduced your issue but it's due to the Host header set in the configuration. You can remove it and the redirects will be set correctly.

Cheers,

You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/9an4MWxfMwY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/b354bd6c-4cd8-46f5-ab09-385823171063%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Eric HARTMANN | SonarSource

http://sonarsource.com

[pa...@cyoda.com]

Worked for me. Thanks!

On Friday, August 19, 2016 at 11:42:09 AM UTC+2, Eric Hartmann wrote:

Hi Brandon,

[hartma...@gmail.com]

Great, thanks for the feedback.

[pa...@cyoda.com]

Sorry, I was too quick. This solved the redirect, but within sonar, all links reverted to localhost.

This is my nginx config before:
        location /sonar {
            proxy_pass http://localhost:9000/sonar;
            proxy_set_header Host $host;

            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header    Upgrade $http_upgrade;
            proxy_set_header    Connection $connection_upgrade;
        }

I changed $scheme to https, which seems to have solved the problem:

        location /sonar {
            proxy_pass http://localhost:9000/sonar;
            proxy_set_header Host $host;

            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header    Upgrade $http_upgrade;
            proxy_set_header    Connection $connection_upgrade;

[pa...@cyoda.com]

Nope. Still not working. Sorry for the spam.

[Brandon Siegel]

I can confirm, I'm seeing the same issue as Paul. Removing the `proxy_set_header Host` line did allow the sonar-runner to work successfully, however in the Web UI when navigating from the dashboard to a project, it looks like a redirect occurs which bounces my browser from `https://my.sonarhost.com:8443/dashboard/index/<id>` to `https://127.0.0.1:9000/overview?id=<id>`

[Eric Hartmann]

Hi guys,

Here is a sample configuration that is working correctly with Nginx (1.11.4 with SonarQube 6.1) with SSL and a web context:

  server { 

    listen       443 ssl;

    server_name  localhost;

    add_header Strict-Transport-Security max-age=31536000;

    client_max_body_size 150M;

    ssl_certificate /etc/nginx/localhost.crt;

    ssl_certificate_key /etc/nginx/localhost.key;

    location /sonar {

      proxy_pass        http://sonarqube:9000/sonar;

      proxy_set_header  X-Forwarded-Proto https;

    }

  }

@Paul, you must remove the header Host as said previously.

According to your configuration you are using nginx plus, I'm not able to test it th Upgrade and Connection header, but you should remove them. 

You should also remove http2 since SonarQube is not supporting http2 for the time being, this is useless.

@Brandon, I'm unable to reproduce your issue. Please post your nginx configuration and the versions (nginx and SonarQube) you are using.

Please try using my snippet, don't forget to update according to your setup :

• ssl_certificate, 
• ssl_certificate_key,
  
• the location,
• the proxy_pass directive

Cheers,

[Brandon Siegel]

Thanks for the example Eric, I was adding X-Real-IP and X-Forwarded-For headers when forwarding to Sonarqube, and for whatever reason that was causing the behavior I was seeing. Removing those and just using the two lines as in your example solved the problem.

[hartma...@gmail.com]

Thanks for the feedback, Brandon. I'll investigate this behaviour.

Cheers

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/9an4MWxfMwY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/CAAM9xd3d1xFP3Mh%3DOf5U1jHvrRXtWZ%3D81bhEVFfhRve%2BjbhLuA%40mail.gmail.com.

[pa...@cyoda.com]

Hi folks,

after some fiddling my suspicion fell on my /etc/hosts file. I had a line in there to point the machine name to the public IP address rather than 127.0.0.1.
After changing the /etc/hosts file, my original configuration works.

So /etc/hosts has now (assuming uname -n returns node.domain.com)

127.0.0.1 node node.domain.com localhost

Since I have several servers sitting behind the proxy, I've got this for my nginx.conf (replace whatever is in blue with the appropriate values)

user www-data;
worker_processes auto;
pid /run/nginx.pid;

events {
    worker_connections 768;
}

http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_session_cache shared:SSL:20m;
    ssl_session_timeout 180m;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;
    ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:!ADH:!AECDH:!MD5;

    ssl_dhparam /etc/nginx/cert/dhparam.pem;
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/nginx/cert/incertme.pem;
    resolver 8.8.8.8 8.8.4.4;

    ##
    # Logging Settings
    ##

    log_format main '$remote_addr - $remote_user [$time_local] "$request" $status $bytes_sent "$http_referer" "$http_user_agent" "$gzip_ratio"';
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    proxy_send_timeout 1200;
    proxy_read_timeout 300;
    proxy_buffering    off;
    proxy_connect_timeout  240;

    map $http_upgrade $connection_upgrade {
            default upgrade;
            ''      close;
    }

    # Reroute everything to https
    server {
            listen 80 default_server;
            listen [::]:80 default_server;
            server_name _;
            return 301 https://$host$request_uri;
    }

    server {
        listen 443;
        listen [::]:443;
        server_name  node.domain.com;
   
        # allow large uploads of files - refer to nginx documentation
        client_max_body_size 0;
   
        ssl on;
        ssl_certificate      /etc/nginx/cert/incertme.crt;
        ssl_certificate_key  /etc/nginx/cert/incertme.key;
   
        add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
        ##
        # Virtual Host Configs
        ##
   
        include /etc/nginx/conf.d/*.conf;
    }
}

And in /etc/nginx/conf.d/sonar.conf

        location /sonar {
            proxy_pass http://localhost:9000/sonar;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
        }

I'm running

ubuntu 16.04
apt nginx package version 1.10.0-0ubuntu0.16.04.2
apt sonar package version 6.0

Might be useful for others...

Best
Paul

[pa...@cyoda.com]

Sorry,
this works for browswer access, not for the TeamCity SonarQube Runner plugin.

If I use the configuration suggested by Eric, I get an error in the browser:

ssl_error_rx_record_too_long
(and I need to open up port 9000 on my firewall)

With my settings, the browser based access works fine, but the plugin fails with 
Status returned by url : 'https://blablabla/sonar/batch_bootstrap/index' is invalid : 302

Apologize for the confusion. I let this rest too long and am reinventing my wheel.

[pa...@cyoda.com]

Considering that my config works for browser-based access, and my issue was with the teamcity sonarqube plugin getting a redirect from sonar/batch_bootstrap/index to sonar/batch/index with a protocol change from https to http, which made the sonarqube runner unhappy, I did a brute-force thing and added to my sonar.conf:

location /sonar/batch_bootstrap/index {
     return 302 https://node.machine.com/sonar/batch/index;
}

I'm now able to push the builds to sonar from teamcity, and have no http loopholes in the proxy.

Phew, at least it looks to be working now for both: plugin and browser.

[arlek...@gmail.com]

In the sonarqube documentation in https://docs.sonarqube.org/display/SONAR/Securing+the+Server+Behind+a+Proxy

For the Apache reverse proxy it is also used ProxyPassReverse, but an equivalent parameter is not used in the nginx example.

Accordingly to Apache documentation https://httpd.apache.org/docs/current/mod/mod_proxy.html#proxypassreverse

This directive lets Apache httpd adjust the URL in the Location, Content-Location and URI headers on HTTP redirect responses. This is essential when Apache httpd is used as a reverse proxy (or gateway) to avoid bypassing the reverse proxy because of HTTP redirects on the backend servers which stay behind the reverse proxy.

In the nginx documentation in https://www.nginx.com/resources/wiki/start/topics/examples/likeapache/  they propose this solution:

server {
    listen myhost:80;
    server_name  myhost;
    location / {
        root /path/to/myapp/public;
        proxy_set_header X-Forwarded-Host $host:$server_port;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://myapp:8080;
    }
}

I worked for me. I am working with sonarqube and nginx docker containers. This is my configuration:

    upstream sonarqube {

        server sonarqube:9000;

    }

    server {

        listen   *:443;

        server_name  Mysonar.com;

        client_max_body_size 1G;

        ssl on;

        ssl_certificate      /etc/letsencrypt/live/Mysonar.com/fullchain.pem;

        ssl_certificate_key  /etc/letsencrypt/live/Mysonar.com/privkey.pem;

        location / {

            proxy_pass http://sonarqube;

            proxy_set_header X-Real-IP $remote_addr;

            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            proxy_set_header X-Forwarded-Proto "https";

            proxy_set_header X-Forwarded-Host $host:$server_port;

            proxy_set_header X-Forwarded-Server $host;