installation problems (nginx related?)

[Isaac Johnson]

Hi, I've tried installing vFense several times between Ubuntu 12.04/14.04 and CentOS (using repos and git). The CentOS installer doesn't work because the install script can't create the user, not a big issue for me because I mainly administer Ubuntu anyhow; however, I can not get vFense up and running. The first issue I had on Ubuntu (both 12.04 and 14.04) was an error coming up related to dateutil not being found when I ran the install script, so I ran:

sudo apt-get install python-dateutil

After package install I can run the script and it completes, even completes all the CVEs and whatnot without throwing errors, eventually giving me a couple lines:

admin:thepasswordichose

agent:arandompassword

Next I try to run:

sudo service nginx restart

but I get a FAIL every time. I can run:

sudo service rethinkdb restart

sudo python /opt/TopPatch/tp/src/daemon/vFensed start

and both work fine and I see vFense get a bunch of updates to 3rd party things but nginx is still really unhappy. My understanding is that the installer script should create:

/etc/nginx/sites-available/vFense.conf

but I don't see a vFense.conf file being created for nginx. Likewise I can't get windows or linux agents to connect to the server using IP or domain as they all seem to time out.

Anybody have any ideas on what could be going wrong or possibly point out if I'm missing something obvious? Thanks!

[Humberto Chacon]

Hey there Isaac, did the "sudo python /opt/TopPatch/tp/src/scripts/initialize_vFense.py ....." command run without errors?

Also, can you show us what version of nginx you're on by running "nginx -v"?

[Allen Sanabria]

if you run sudo nginx -t, does the output look like this...

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conf test is successful

If so, than your nginx config is good to go. If not you may have the wrong version of nginx installed. Which means you either need to uninstall nginx or if you do not care about uploading packages to vFense, then you can comment out the upload section in vFense.conf

Line 47 to lines 64 should be commented out if you do not want to upload custom packages to vFense.

On Thursday, May 8, 2014 8:44:42 PM UTC-4, Isaac Johnson wrote:

[Isaac Johnson]

Hi, thanks for the responses I'm completely new to nginx and didn't have the slightest clue where to start! Anyhow, I didn't have any installation errors pop up after I installed python-dateutil, the install script seems to run fine with no errors when I ran it. Also if I move /etc/nginx/sites-enabled/vFense.conf out of the nginx directory then nginx starts/stops/restarts without issue so it appears I have a configuration issue in that file. Here is the output I get from those nginx commands:

isaac@vfense:~$ nginx -v

nginx version: nginx/1.4.6 (Ubuntu)

isaac@vfense:~$ sudo nginx -t

[sudo] password for isaac:

nginx: [emerg] unknown directive "upload_store" in /etc/nginx/sites-enabled/vFense.conf:48

nginx: configuration file /etc/nginx/nginx.conf test failed

Here is the output of cat /etc/nginx/sites-enabled/vFense.conf:

isaac@vfense:~$ cat /etc/nginx/sites-enabled/vFense.conf

upstream rvlistener {

    server 127.0.0.1:9020;

    server 127.0.0.1:9021;

    server 127.0.0.1:9022;

    server 127.0.0.1:9023;

    server 127.0.0.1:9024;

    server 127.0.0.1:9025;

    server 127.0.0.1:9026;

    server 127.0.0.1:9027;

    server 127.0.0.1:9028;

    server 127.0.0.1:9029;

}

upstream rvweb {

    server 127.0.0.1:9060;

}

server {

    listen         80;

    server_name    vfense.local localhost;

    rewrite        ^ https://$server_name$request_uri? permanent;

}

server {

    listen                      443;

    server_name                 _;

    ssl                         on;

    ssl_certificate             /opt/TopPatch/tp/data/ssl/server.crt;

    ssl_certificate_key         /opt/TopPatch/tp/data/ssl/server.key;

    ssl_session_timeout         5m;

    ssl_protocols               SSLv3 TLSv1;

    ssl_ciphers                 ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;

    ssl_prefer_server_ciphers   on;

    client_max_body_size            1G;

    client_body_buffer_size     100m;

    location /nginx_status {

        stub_status on;

        access_log   off;

        allow 192.168.0.0/16;

        allow 127.0.0.1;

        deny all;

    }

    location /upload/package {

        upload_store /opt/TopPatch/var/packages/tmp/;

        upload_store_access user:rw group:rw all:rw;

        upload_set_form_field $upload_field_name.name "$upload_file_name";

        upload_set_form_field $upload_field_name.content_type "$upload_content_type";

        upload_set_form_field $upload_field_name.path "$upload_tmp_path";

        upload_aggregate_form_field "$upload_field_name.md5" "$upload_file_md5";

        upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";

        upload_pass @after_upload;

        upload_pass_form_field "^id$";

        upload_pass_form_field ".*";

        upload_cleanup 400 404 499 500-505;

    }

    location @after_upload {

        proxy_pass              https://rvweb;

    }

    location ^~ /api/ {

        proxy_pass              https://rvweb;

        proxy_set_header        Host $host;

        proxy_set_header        X-Real-IP $remote_addr;

        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_intercept_errors  off;

        proxy_redirect          http:// https://;

    }

    location ~ /ra/websockify/(.*)/([0-9]+) {                                                                                               

        proxy_pass              http://$1:$2/websockify;

        proxy_read_timeout      2592000;

        proxy_http_version      1.1;

        proxy_set_header        Upgrade $http_upgrade;

        proxy_set_header        Connection "upgrade";

    }

    location ~ /ra/(.*)/([0-9]+)/(.*$) {

        proxy_pass              http://$1:$2/$3;

        proxy_set_header        Host $host;

        proxy_set_header        X-Real-IP $remote_addr;

        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_intercept_errors  off;

        proxy_redirect          http:// https://;

        #echo                   "im in the location";

    }

    location  ^~ /ws/ {

        proxy_pass              https://rvweb;

        proxy_read_timeout      604800; # 7 days

        proxy_http_version      1.1;

        proxy_set_header        Upgrade $http_upgrade;

        proxy_set_header        Connection "upgrade";

        proxy_set_header        X-Real-IP $remote_addr;

        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

        #proxy_send_timeout      300;

    }

    location ^~ /rvl/ {

        proxy_pass              https://rvlistener;

        proxy_set_header        Host $host;

        proxy_set_header        X-Real-IP $remote_addr;

        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_intercept_errors  off;

        proxy_redirect          http:// https://;

    }

    location ~* \.(?:ico|css|js|gif|jpe?g|png)$ {

        root                    /opt/TopPatch/tp/wwwstatic;

        expires                 max;

        add_header              Pragma public;

        add_header              Cache-Control "public, must-revalidate, proxy-revalidate";

    }

    location ~ /var/packages {

        root                    /opt/TopPatch/var/packages;

        expires                 max;

        add_header              Pragma public;

        add_header              Cache-Control "public, must-revalidate, proxy-revalidate";

    }

    location  ^~ /# {

        proxy_pass              https://rvweb;

        proxy_set_header        Host $host;

        proxy_set_header        X-Real-IP $remote_addr;

        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_redirect          http:// https://;

    }

    location  / {

        proxy_pass              https://rvweb;

        proxy_set_header        Host $host;

        proxy_set_header        X-Real-IP $remote_addr;

        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_redirect          http:// https://;

    }

[Dinesh Kumar]

Issac, it seems its upload issue. Can you please go to the conf file "/etc/nginx/sites-enabled/vFense.conf" and comment from line number 47 to 64, like below:
==============

    #location /upload/package {
    #   upload_store /opt/TopPatch/var/packages/tmp/;
    #    upload_store_access user:rw group:rw all:rw;
    #   upload_set_form_field $upload_field_name.name "$upload_file_name";
    #    upload_set_form_field $upload_field_name.content_type "$upload_content_type";
    #    upload_set_form_field $upload_field_name.path "$upload_tmp_path";
    #    upload_aggregate_form_field "$upload_field_name.md5" "$upload_file_md5";
    #    upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";
    #    upload_pass @after_upload;
    #    upload_pass_form_field "^id$";
    #    upload_pass_form_field ".*";
    #    upload_cleanup 400 404 499 500-505;
    #}
    # 
    #location @after_upload {
    #    proxy_pass              https://rvweb;
    #}
   #
======================

Once commented, please run the command nginx -t and lets us know if the test failed.

Thanks

[Isaac Johnson]

No failure, syntax runs fine. I can now browse to https://vfense.local and receive a certificate, but get a "502 Ba Gateway" after that (which is further than I could get before) so we're on the right track. At any rate, it appears that the nginx-extras package does not include nginx-upload-module, I upgraded to nginx 1.6 via the launchpad PPA and it doesn't show up under nginx -V (it didn't under 1.4 either).

I take it I'll have to compile nginx from source with the nginx-upload-module?

[Dinesh Kumar]

Issac, bit confused here. Are you still facing the 502-Bad Gateway error while accessing the UI or you already fixed it. If still facing, as test went successful please restart the nginx once. Also redis-server and rethinkdb. Execute  following commands as root or sudo user:

python /opt/TopPatch/tp/src/daemon/vFensed stop
service nginx restart
service redis-server restart
service rethinkdb restart
python /opt/TopPatch/tp/src/daemon/vFensed start

and let us know if the issue persists.

Thanks

[Isaac Johnson]

Ah, I had only restarted nginx; after restarting all the services I can log into the dashboard now and I have managed to join linux and windows agents. You guys are awesome!

So I did a bit more research and it looks like the package nginx-extras for Debian had the nginx-upload-module (which powers the upload_store instruction) removed due to incompatibility with nginx 1.4 and I suspect the same has been done to Ubuntu. While debian apparently added it back in after compatibility was resolved, it doesn't appear that the package maintainers of nginx-extras on Ubuntu ever did. (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=729003)

That's mostly why I ask if I'll need to compile nginx from source with the necessary modules at some point. I'm not entirely sure what upload_store does for vFense, but I suspect it's serving some kind of function or it wouldn't have been included? ;-D

Thanks again, you guys rock!

[Humberto Chacon]

The vFense server currently relies on nginx 1.1.19 (the default on Ubuntu official repo) to be able to upload custom applications. If you'd prefer to use nginx 1.4+ you would have to comment out the lines mentioned by Allen and Dinesh. This would mean that you can not use the custom uploader, however.

[Sergio Jurado]

Hello, I have the same problem after installing vFense. But I restart nginx and the others service and 502 still happens. I have reviewed nginx configurations several times and I have not found any solution.

Any help will be usefull.

[Allen Sanabria]

If the Nginx settings are correct, than the next thing to check, is to see if rethinkdb is running.
ps -fe |grep rethinkdb

If rethinkdb is running, than try this...

bpython
from vFense.db.client import *
conn = db_connect()
list(r.table('agents').run(conn))

If this returns data, than rethinkdb is responding correctly.

Next step is to tail the logs..

tail -f /var/log/rvapi_file.log
Are there any errors??

[Sergio Jurado]

Hello Allen, thanks for your answer. I just done that you say.

- Rethink is running.

- The bpython output is the following:



I don't know exactly if it's returning data....

- The output of rvapi_file.log is:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/vFense/core/agent/_db.py", line 358, in fetch_agent_info
    .run(conn)
  File "/usr/local/lib/python2.7/dist-packages/rethinkdb/ast.py", line 108, in run
    return c._start(self, **global_opt_args)
  File "/usr/local/lib/python2.7/dist-packages/rethinkdb/net.py", line 199, in _start
    return self._send_query(query, term, global_opt_args)
  File "/usr/local/lib/python2.7/dist-packages/rethinkdb/net.py", line 308, in _send_query
    self._check_error_response(response, term)
  File "/usr/local/lib/python2.7/dist-packages/rethinkdb/net.py", line 276, in _check_error_response
    raise RqlRuntimeError(message, term, frames)
RqlRuntimeError: Cannot perform merge on a non-object non-sequence `null`. in:
r.table('agents').get(u'f5d18715-ce38-4685-a7af-a42ccf2136d6').merge(lambda var_3: {'tags': r.table('tag_per_agent').get_all(r.row['agent_id'], index='age$
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                            $

Any ideas?

Thanks in advance.

[mandar kanade]

Hello Allen,

I've git-cloned bumblebee release "https://github.com/vFense/vFense/releases". I get rethink web console on port 8080, I can see nginx when i open localhost in IE, but i do not get the https://localhost portal open. I tried the way guided in this post & i get the below mentioned error. Requesting your advise on this please.

>>> from vFense.db.client import *

Traceback (most recent call last):

  File "<input>", line 1, in <module>

  File "/usr/local/lib/python2.7/dist-packages/vFense/db/client.py", line 6, in

 <module>

    import rethinkdb as r

ImportError: No module named rethinkdb

>>> conn = db_connect()

Traceback (most recent call last):

  File "<input>", line 1, in <module>

NameError: name 'db_connect' is not defined

>>> list(r.table('agents').run(conn))

Traceback (most recent call last):

  File "<input>", line 1, in <module>

NameError: name 'r' is not defined

>>> 

Thanks & Regards

Mandar.K

[Allen Sanabria]

Please make sure to follow the instructions here https://github.com/vFense/vFense/wiki/Installing-vFense-Server

Since it looks like you do not have the rethinkdb module installed, which means you could not have initialized the server. Let me know, how it goes after you followed the instructions.

--
You received this message because you are subscribed to a topic in the Google Groups "vFense" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vfense/WCwgJ46Pe9U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vfense+un...@googlegroups.com.
To post to this group, send email to vfe...@googlegroups.com.
Visit this group at http://groups.google.com/group/vfense.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfense/3d8363c9-f887-48f5-b5a4-f4e03f1fe6d4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.