VirusTotal Integration in SGUIL

[BBCan177]

****************************************************************
** SGUIL Integration with Virus Total and BOTHunter (SRI int.)** ****************************************************************

I have tested the following improved functionality of SGUIL WHOIS Resolution.

In SGUILs "IP Resolution" Dialog Box, the only data provided are "Whois" results.

I wrote the following scripts to have it lookup additional details to help gather better intelligence on an alerted IP address. The Script will output results based on IP (SRC or DST) and display the results in the "DNS Resolution Window in SGUIL"

So if the IP is a Private Address, it will report-

1) nslookup

2) Known IP Info - This will lookup a local text file that you can edit to display any information you want about an IP. I use this to show the User name of the IP as a typical DNS Lookup will only resolve the Hostname. ("sguil-alias")

If the IP is an external address, it will report-

1) "Selected" Lines from VirusTotal "Reputation Report"

a) VT IP Scan Date
b) # of "Positives" alerts
c) Other Comment/Error alerts

2) Known IP Info - This will lookup a local text file that you can edit to display any information you want about an IP. I use this to show the Username of the IP as a DNS lookup will only resolve the Hostname. ("sguil-alias")

3) "BOTHUNTER" lookup which provides a "Threat Level" and also contains GEOIP info. (SRI Int.)

4) WHOis Result

5) Full VirusTotal "Reputation Report"

6) Full VirusTotal "IP Information Report"

Please see the attachments-

"Sample Output" - Is a sample of what the report should look like.

"sguil-External Lookup" - png file of SGUIL DNS resolution Dialog Box

"sguil-Local Lookup" - png file of SGUIL DNS resolution Dialog Box

"sguil-whois" - Main BASH script that outputs the various reports and calls
the python script.

"vt.py" - Python Script that uses VirusTotal API 2.0 to compare an IP to
the VT database. It originally came from this website "
http://salawank.com/blog2/ " with a lot of mods.

"sguil-alias" is the local text file for "Known IP Information"

You will need to register with VirusTotal to receive a free "API KEY" before using the script. The API Key needs to be inserted into the sguil-whois file.

https://www.virustotal.com/en/#signup

"TO TEST WITHOUT modifying SGUIL"

The sguil-whois can be run as a test from the terminal window. Open the file and Un-comment "only one" of the listed IP's. From the command line run (save files in /etc/sguil)

./sguil-whois (you may need to run chmod as shown below)

TO modify SGUIL's WHOis lookup and add this functionality, do the following-

I installed the files in "/etc/sguil"

create the files sguil-whois, sguil-alias and vt.py
chmod +x vt.py
chmod +x sguil-whois

Edit the "sguil-whois" file. and remove the test IP's, leaving the variable

IP=$*

Finally, you need to edit "/etc/sguil/sguil.conf"

and change the following

#set WHOIS_PATH SimpleWhois
set WHOIS_PATH /etc/sguil/sguil-whois

You will need to sudo nsm_server_ps-restart, Exit SGUIL and Re-open.

In "DNS RESOLUTION" box, select SRC or DST and the reports will appear.

----------------------------------------

Any additional lookup functionality can be implemented.
If you find other solutions, please share.

Hope you try it out. Feedback is welcome.... and wanted!
My first python script so pls be gentle....

-----------------------------------------

[BBCan177]

*UPDATE*

I added a timeout (--max-time 10) to the BOTHunter lookup. (in the sguil-whois file)

curl --max-time 10 -s "http://kb.bothunter.net/ipInfo/nowait.php?IP=$IP" | tail -n +5 | head -n -5 | sed -r 's/<\/{0,1}b>//g'

I never noticed it timing out before. Not sure if its a Network issue or BOTHunter limiting requests?

Does anyone know more info about SRI's BOTHunter lookup.

As Squert/ELSA is using GEOIP, can a GEOIP request be made from the commandline utilizing the Maxmind package?

[Paul Halliday]

Squert uses its own geo database created using data from the RIRs. You
could query it like this:

~$ mysql -N -B -u <user> -p -D sguildb -e "SELECT registry, cc,
c_long, type, date, status FROM ip2c WHERE $ip >=start_ip AND $ip <=
end_ip LIMIT 1;"

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Paul Halliday
http://www.pintumbler.org/

[BBCan177]

On Wednesday, February 19, 2014 12:42:27 PM UTC-5, Paul Halliday wrote:
> Squert uses its own geo database created using data from the RIRs. You
> could query it like this:
> ~$ mysql -N -B -u <user> -p -D sguildb -e "SELECT registry, cc,
> c_long, type, date, status FROM ip2c WHERE $ip >=start_ip AND $ip <=
> end_ip LIMIT 1;"

Thanks Paul I tried this but am getting this error

mysql -N -B -u <user> -p -D sguildb -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE 8.8.8.8 >=start_ip AND 8.8.8.8 <=end_ip LIMIT 1;"

Enter password:
ERROR 1045 (28000): Access denied for user 'xxxxxxx'@'localhost' (using password: YES)

I used the same password as Squert. Also tried root? tried a couple different IPs also?

Unfortunately, I it asks for a password and I'm not sure if adding the password in plain text to my BASH script would be good?

[Paul Halliday]

Perhaps a constrained read-only user:

mysql> GRANT SELECT ON sguildb.ip2c TO 'iplookup'@'localhost'
IDENTIFIED BY 'apassword';

All this user would be able to do is read the ip2c table.

[BBCan177]

On Wednesday, February 19, 2014 1:55:09 PM UTC-5, BBCan177 wrote:

> On Wednesday, February 19, 2014 1:19:48 PM UTC-5, Paul Halliday wrote:
> > Perhaps a constrained read-only user:
>

> mysql> GRANT SELECT ON sguildb.ip2c TO 'user'@localhost IDENTIFIED BY 'password';
> ERROR 1146 (42S02): Table 'sguildb.ip2c' doesn't exist
>
> Had to use this
>
> mysql> GRANT SELECT ON securityonion_db.ip2c TO user@localhost IDENTIFIED BY 'password';
> Query OK, 0 rows affected (0.00 sec)
>
>
> mysql -N -B -u user -p -D sguildb.ip2c -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE 8.8.8.8 >=start_ip AND 8.8.8.8 <=end_ip LIMIT 1;"
>
> Enter password:
>
> ERROR 1044 (42000): Access denied for user 'user'@'localhost' to database 'sguildb.ip2c'
>
>
> mysql -N -B -u user -p -D securityonion_db.ip2c -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE 8.8.8.8 >=start_ip AND 8.8.8.8 <=end_ip LIMIT 1;"
>
> ERROR 1045 (28000): Access denied for user 'user'@'localhost' (using password: YES)
>
>
> I tried to add the password in the cmd line after the "-p" without success?
>
>
> mysql> show databases;
> +--------------------+
> | Database |
> +--------------------+
> | information_schema |
> | elsa_web |
> | mysql |
> | performance_schema |
> | securityonion_db |
> | snorby |
> | syslog |
> | syslog_data |
> | test |
> +--------------------+
> 9 rows in set (0.00 sec)

[BBCan177]

I have tried in the past to use

"geoiplookup" from the cmd line. But it is not installed as a package in Security Onion base installation. So I didn't want to necessarily add more packages at first.

But that is also an option.

[Paul Halliday]

Are you trying the connection from localhost? You might need to use
127.0.0.1 or the systems hostname. Also, when you specify it on the
command line it must immediately follow the p. So -ppassword.

On Wed, Feb 19, 2014 at 2:55 PM, BBCan177 <bbca...@gmail.com> wrote:

> On Wednesday, February 19, 2014 1:19:48 PM UTC-5, Paul Halliday wrote:
>> Perhaps a constrained read-only user:
>

> mysql> GRANT SELECT ON sguildb.ip2c TO 'user'@localhost IDENTIFIED BY 'password';
> ERROR 1146 (42S02): Table 'sguildb.ip2c' doesn't exist
>
> Had to use this
>
> mysql> GRANT SELECT ON securityonion_db.ip2c TO user@localhost IDENTIFIED BY 'password';
> Query OK, 0 rows affected (0.00 sec)
>
>

> mysql -N -B -u user -p -D sguildb.ip2c -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE 8.8.8.8 >=start_ip AND 8.8.8.8 <=end_ip LIMIT 1;"
>
> Enter password:
>

> ERROR 1044 (42000): Access denied for user 'user'@'localhost' to database 'sguildb.ip2c'
>
>

> mysql -N -B -u user -p -D securityonion_db.ip2c -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE 24.138.103.244 >=start_ip AND 24.138.103.244 <=end_ip LIMIT 1;"
>
> ERROR 1045 (28000): Access denied for user 'user'@'localhost' (using password: YES)

>
>
> I tried to add the password in the cmd line after the "-p" without success?
>
>
> mysql> show databases;
> +--------------------+
> | Database |
> +--------------------+
> | information_schema |
> | elsa_web |
> | mysql |
> | performance_schema |
> | securityonion_db |
> | snorby |
> | syslog |
> | syslog_data |
> | test |
> +--------------------+
> 9 rows in set (0.00 sec)
>
>
>

[Doug Burks]

Have you tried simply "mysql -uroot"?

https://code.google.com/p/security-onion/wiki/Passwords#MySQL

On Wed, Feb 19, 2014 at 1:55 PM, BBCan177 <bbca...@gmail.com> wrote:

> On Wednesday, February 19, 2014 1:19:48 PM UTC-5, Paul Halliday wrote:
>> Perhaps a constrained read-only user:
>

> mysql> GRANT SELECT ON sguildb.ip2c TO 'user'@localhost IDENTIFIED BY 'password';
> ERROR 1146 (42S02): Table 'sguildb.ip2c' doesn't exist
>
> Had to use this
>
> mysql> GRANT SELECT ON securityonion_db.ip2c TO user@localhost IDENTIFIED BY 'password';
> Query OK, 0 rows affected (0.00 sec)
>
>

> mysql -N -B -u user -p -D sguildb.ip2c -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE 8.8.8.8 >=start_ip AND 8.8.8.8 <=end_ip LIMIT 1;"
>
> Enter password:
>

> ERROR 1044 (42000): Access denied for user 'user'@'localhost' to database 'sguildb.ip2c'
>
>
> mysql -N -B -u user -p -D securityonion_db.ip2c -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE 24.138.103.244 >=start_ip AND 24.138.103.244 <=end_ip LIMIT 1;"
>
> ERROR 1045 (28000): Access denied for user 'user'@'localhost' (using password: YES)
>
>
> I tried to add the password in the cmd line after the "-p" without success?
>
>
> mysql> show databases;
> +--------------------+
> | Database |
> +--------------------+
> | information_schema |
> | elsa_web |
> | mysql |
> | performance_schema |
> | securityonion_db |
> | snorby |
> | syslog |
> | syslog_data |
> | test |
> +--------------------+
> 9 rows in set (0.00 sec)
>
>
>

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/groups/opt_out.



--

Doug Burks

[BBCan177]

On Wednesday, February 19, 2014 5:18:30 PM UTC-5, Doug Burks wrote:
> Have you tried simply "mysql -uroot"?

Hi Doug,

I don't think I'm querying the correct database to get the geoip data.

None of the recommendations above seem to work with either users.

[Keith Butler]

Remove the ".ip2c" from "-D securityonion_db.ip2c"
Just: "-D securityonion_db"

[Keith Butler]

nice work on the integration

[BBCan177]

On Wednesday, February 19, 2014 5:51:51 PM UTC-5, Keith Butler wrote:
> nice work on the integration

Thanks Keith.... Have you tested it out?

mysql -N -B -uroot -D securityonion_db -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE 8.8.8.8 >=start_ip AND 8.8.8.8 <=end_ip LIMIT 1;"

ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.8.8 >=start_ip AND 8.8.8.8 <=end_ip LIMIT 1' at line 1

Pauls original line had $ip. Did I enter the correct syntax for the IP addr?
I tried it in my script also with Variables without success.

I also tried to change the "From ip2c" to "From securityonion_db"

[Keith Butler]

> Thanks Keith.... Have you tested it out?

Not yet. But I will this weekend!

> mysql -N -B -uroot -D securityonion_db -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE 8.8.8.8 >=start_ip AND 8.8.8.8 <=end_ip LIMIT 1;"
>
> ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.8.8 >=start_ip AND 8.8.8.8 <=end_ip LIMIT 1' at line 1

IP addresses in the table are stored as integers not strings, so you've got to reference them using the inet_aton function:

mysql> grant select on securityonion_db.ip2c to user@localhost identified by 'password';

Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;

Query OK, 0 rows affected (0.00 sec)

$ mysql -N -B -u user -p -D securityonion_db -e "Select * from ip2c limit 2"
Enter password:
RFC1918 LO RFC1918 ipv4 167772160 184549375 1996-02-01 00:00:00 allocated
RFC1918 LO RFC1918 ipv4 2886729728 2886795263 1996-02-01 00:00:00 allocated

mysql> select * from ip2c where start_ip <= inet_aton('41.0.0.1') and end_ip >= inet_aton('41.0.0.1');
+----------+------+--------------+------+-----------+-----------+---------------------+-----------+
| registry | cc | c_long | type | start_ip | end_ip | date | status |
+----------+------+--------------+------+-----------+-----------+---------------------+-----------+
| afrinic | ZA | SOUTH AFRICA | ipv4 | 687865856 | 689963007 | 2007-11-26 00:00:00 | allocated |
+----------+------+--------------+------+-----------+-----------+---------------------+-----------+

[Keith Butler]

Perhaps using BETWEEN instead of the >=/<= syntax might be cleaner and possibly more optimized? I'm not sure. I didn't think this would work, but it seems to.

$ mysql -N -B -u user -p -D securityonion_db -e "SELECT registry, cc, c_long, type, date, status FROM ip2c WHERE INET_ATON('8.8.8.8') BETWEEN start_ip and end_ip"
Enter password:
arin US UNITED STATES ipv4 1992-12-01 00:00:00 allocated

[BBCan177]

Hi Keith,

I think that the following two queries provide more info and a lot easier.
The nice thing about the script is that it can be customized to each persons needs quite easily.

If you have any suggestions on other Intel. lookups, please let me know.

curl --max-time 10 -s "http://kb.bothunter.net/ipInfo/nowait.php?IP=$IP" | tail -n +5 | head -n -5 | sed -r 's/<\/{0,1}b>//g'

========== > BOTHUNTER INFO < ==========

Threat Level = High
Threat Category = Malware Propagator, Malware Controller
Threat Description = Malware drive-by exploit site, Malware scan and infect source
Hostname = No hostname found
Service Provider = INTERNET FOR EVERYTHING LTD
Domain Name = -
ASN Number = 39779
ASN Name = MESHDIGITAL Mesh Digital Ltd
Network Speed = DSL
Country CC = UK
Country = UNITED KINGDOM
Region = -
City = -
Longitude = -4.47300004959106
Latitude = 54.1500015258789
Zipcode = -
TimeZone = +00:00

========== > GEOip RESULT < ==========

geoiplookup 72.32.173.45

GeoIP City Edition, Rev 1: US, TX, Texas, San Antonio, 78218, 29.488899, -98.398697, 641, 210

(This would need "apt-get install geoiplookup" as its not in SO"

[Keith Butler]

> If you have any suggestions on other Intel. lookups, please let me know.

Will do.

[Brian Kellogg]

I've edited the /etc/sguil/sguil.conf file and the /etc/nsm/securityonion/sguil.conf file and I still cannot get squil to use this script. I have restarted squil via nsm_server_ps-restart as well, several times. What am I missing? The script works great from the command line.

I've also changed the owner and group on the three script files to sguil:sguil.

thanks,
Brian

[BBCan177]

Hi Brian,

I think the rights need to be set as root. You only need to edit the /etc/sguil/sguil.conf to get this integration to work. No changes are needed to the /etc/nsm/securityonion/sguild.conf file.

ls -al

drwxr-xr-x 2 root root 4096 Feb 28 04:07 .
drwxr-xr-x 159 root root 12288 Feb 22 20:32 ..
-rw-r--r-- 1 root root 2572 Feb 16 05:58 sguil-alias
-rw-r--r-- 1 root root 4831 Feb 16 06:01 sguil.conf
-rwxrwxr-x 1 root root 3922 Feb 21 04:06 sguil-whois
-rwxr--r-- 1 root root 5281 Feb 18 22:23 vt.py

It should also work if you copy the files to your SO user home directory and run it from there.

[Brian Kellogg]

I tried editing the /etc/nsm/securityonion/sguild.conf as a last resort.

I edited the /etc/sguil/sguil.conf file as stated above on the SO server and all I get in SGuil is "error: connection to whois.arin.net timed out". This error is because I'm still waiting for the FW admins to open the whois port for the SO server.

From a bash shell I can run sguil-whois and get back information from VirusTotal and Botnet Hunter.

My SO server ls -al /etc/sguil

drwxr-xr-x 2 root root 4096 Mar 3 21:51 .
drwxr-xr-x 151 root root 12288 Mar 3 17:17 ..
-rw-r--r-- 1 root root 157 Mar 3 16:19 sguil-alias
-rw-r--r-- 1 root root 4802 Mar 3 21:45 sguil.conf
-rwxrwxr-x 1 root root 3451 Mar 3 16:58 sguil-whois
-rwxr--r-- 1 root root 5283 Mar 3 16:17 vt.py

From squil.conf in /etc/sguil

# Path to a whois script.
# awhois.sh is an example. Get it at ftp://ftp.weird.com/pub/local/awhois.sh
# NEW: sguil.tk has a built in whois proc called SimpleWhois although
# you can continue to use tools like awhois.sh.
set WHOIS_PATH /etc/sguil/sguil-whois

I'm at a loss unless its a typo I keep missing.

[Brian Kellogg]

Argh, nevermind.

I've been using X forwarding so much that it never occurred to me that I was using the local install of the sguil client on my local machine and not the remote sguil client on the SO server. My apologies and thanks.

[BBCan177]

On Monday, March 3, 2014 5:15:18 PM UTC-5, Brian Kellogg wrote:
> Argh, nevermind.
>
> I've been using X forwarding so much that it never occurred to me that I was using the local install of the sguil client on my local machine and not the remote sguil client on the SO server. My apologies and thanks.

When it doubt get a bigger hammer.... Your Welcome..

[Rex Underwood]

This is an outstanding feature. I have been doing a lot of right-clicking and going to IPvoid.com to test each event's IP address; what a time-saver this will be. Accordingly, I have worked for hours to implement it, but without success. I have crawled the code and read this page a dozen times, but I can't make it work in sguil or from the command line (just individual commands, not the whole script). Here's what I did:

1. I downloaded the files and saved them to /etc/sguil

2. I renamed them without extensions, and in lower case letters (to match the references in the code).

3. I changed the permissions according to this page's post. (In my frustration, I gave everyone read, write, and execute privileges, but it didn't help).

4. I changed APIKEY= to apikey= in sguil-whois (in order to match the reference in vt.py). I also checked for white space that might have interfered with the script.

5. I tried running the test case: ./sguil-whois (without editing sguil.conf) as myself and as root. When I run under my account, I get:

joel@leigh:/etc/sguil$ ./sguil-whois
bash: ./sguil-whois: /bin/bash^M: bad interpreter: No such file or directory

and when I run it as root, it does nothing:

joel@leigh:/etc/sguil$ sudo ./sguil-whois
[sudo] password for joel:
joel@leigh:/etc/sguil$

6. I isolated each command in the script, entered values for the variables, and ran it in the terminal successfully:

joel@leigh:/etc/sguil$ grep 8.8.8.8 /etc/sguil/sguil-alias
8.8.8.8 is Google DNS

joel@leigh:python /etc/sguil/vt.py "8.8.8.8" "I pasted my APIKEY here"
a long list of URLs was returned

joel@leigh:/etc/sguil$ curl -s http://kb.bothunter.net/ipInfo/nowait.php?IP=8.8.8.8 | tail -n +5 | head -n -5 | sed -r 's/<\/{0,1}b>//g'
Threat Level = Unverified
Threat Category =
Threat Description =
Hostname = google-public-dns-a.google.com
Service Provider = GOOGLE INCORPORATED
Domain Name = LEVEL3.NET
ASN Number = 15169
ASN Name = GOOGLE - Google Inc.
Network Speed = DSL
Country CC = US
Country = UNITED STATES
Region = CALIFORNIA
City = MOUNTAIN VIEW
Longitude = -122.075996398926
Latitude = 37.3955993652344
Zipcode = 94043
TimeZone = -08:00

joel@leigh:/etc/sguil$ /usr/bin/whois 8.8.8.8 |grep -v "#" | tail -n +5 | head -n -4
and Google's whois information was faithfully returned

7. I had similarly frustrating results when I made the edits and added my VT API key to sguil-whois, uncommented the IP=$*, exited the sguil gui, stopped and started the nsm service, and tried to run it within the DNS lookup pane in sguil. As soon as I clicked the source or destination IP radio button, whether it was a private or public IP address, I got an immediate error:

couldn't execute "/etc/sguil/sguil-whois": no such file or directory
couldn't execute "/etc/sguil/sguil-whois": no such file or directory
while executing
"open "| $WHOIS_PATH $ip" r"
(procedure "GetWhoisData" line 36)
invoked from within
"GetWhoisData"
invoked from within
".eventPane.pane1.childsite.detailPane.pane0.childsite.detailTabs.canvas.notebook.cs.page1.cs.ipDataFrame.whoisDataFrame.whoisDataButtonsFrame.whoisSrc..."
("uplevel" body line 1)
invoked from within
"uplevel #0 [list $w $cmd]"
(procedure "tk::CheckRadioInvoke" line 3)
invoked from within
"tk::CheckRadioInvoke .eventPane.pane1.childsite.detailPane.pane0.childsite.detailTabs.canvas.notebook.cs.page1.cs.ipDataFrame.whoisDataFrame.whoisData..."
(command bound to event)

Now, the file IS there, and the permissions are set correctly (I even made them ugo+rwx), and I made the proper edits to sguil-whois (apikey, IP=$*) and sguil.conf ($WHOIS-PATH - I tried both the full file path and the short one, to no avail), so I'm out of things I can think of to check. Any help would be greatly appreciated!

BTW- I used Leafpad to do the edits, and I wonder if that introduced an unwanted factor in the code of vt.py or sguil-whois. Unlike gedit, Leafpad doesn't make a hidden backup copy of the file.

[BBCan177]

> 4. I changed APIKEY= to apikey= in sguil-whois (in order to match the reference in vt.py). I also checked for white space that might have interfered with the script.

This needs to stay in Upper case as this line below pushes the APIKEY to the vt.py script and the vt.py receives the APIKEY as $1.

python /etc/sguil/vt.py "$IP" "$APIKEY"

(If you want to keep it consistent with the vt.py being lower case, then make sure the line above "$APIKEY" is also changed to lower case.)

> 5. I tried running the test case: ./sguil-whois (without editing sguil.conf) as myself and as root. When I run under my account, I get:
>
> joel@leigh:/etc/sguil$ ./sguil-whois
> bash: ./sguil-whois: /bin/bash^M: bad interpreter: No such file or directory

It looks like it can't find the BASH interpreter.

Make sure the sguil-whois file has the first line as

#!/bin/bash

The bash file should be in " ls /bin/bash "

> BTW- I used Leafpad to do the edits, and I wonder if that introduced an unwanted factor in the code of vt.py or sguil-whois. Unlike gedit, Leafpad doesn't make a hidden backup copy of the file.

I would suggest using "vi" to edit these files.

Hopefully this will lead you in the right direction.

[BBCan177]

> This needs to stay in Upper case as this line below pushes the APIKEY to the vt.py script and the vt.py receives the APIKEY as $1.

Should have said $2 as $1 is the IP address to lookup.

[David Vasil]

On Friday, March 28, 2014 9:39:35 AM UTC-5, BBCan177 wrote:
> > 5. I tried running the test case: ./sguil-whois (without editing sguil.conf) as myself and as root. When I run under my account, I get:
> >
> > joel@leigh:/etc/sguil$ ./sguil-whois
> > bash: ./sguil-whois: /bin/bash^M: bad interpreter: No such file or directory

The ^M is indicative of a file that is using DOS CRLF line terminators. Try using dos2unix on the file.

vasil@so:~/Downloads$ ./testfile
bash: ./testfile: /bin/bash^M: bad interpreter: No such file or directory
vasil@so:~/Downloads$ file testfile
testfile: Bourne-Again shell script, ASCII text executable, with CRLF line terminators
vasil@so:~/Downloads$ dos2unix testfile
dos2unix: converting file testfile to Unix format ...
vasil@so:~/Downloads$ ./testfile
hello world

-dave

[Rex Underwood]

Thank you BBCan177 and David Vasil; you guys are studs, and your kung-fubuntu is powerful. Both of you provided a part of the solution. Here's what I did that worked:

1. I downloaded fresh copies of each file, deleting the .txt extentions. But I left them in upper case, except for the python scripy, which I renamed from VY.PY to vt.py (in order to minimize edits to the code).

2. I edited SGUIL-WHOIS and sguil.conf as directed above. I forgot and used Leafpad, which introduced the DOS CRLF line terminators David mentioned, so I just ran dos2unix on everything in the /etc/sguil directory.

3. I set the permissions as suggested by BBCan177 in the directions above.

4. Restarted the nsm service, and voila!

I really appreciate your help. In a matter of a couple of hours I was able to retire over 20K events, going back to mid-summer. I sorted the event table in sguil by IP address, and once I verified a site as legitimate (Netflix, or something) I was able to batch-retire whole sections of the table at once with the f8 key. Sguil is lean and mean and frisky again. The old way was to right-click on each event and copy the IP address, past it into IPvoid.com, and return to sguil to retire the event (or dig deeper). It was a round-trip of 9 mouse clicks (right and left clicks). This is SO much faster, BBCan177. Awesome work! And thank you for the PM; I appreciate you help and responsiveness.

[Grant]

everything is working but virustotal lookup. i keep getting "invalid VT API KEY" i know its the right key as i am also using it on the same machine for virus total uploader. any ideas?

[Grant]

sorry but i solved my own issue. i was putting the VT Key in the parentheses instead of erasing all text between quotes and simply putting the key in between the quotes. rookie...

[Grant]

is this solution still working? i am now getting some python errors in the output (see below) and the VT info is not displaying. the whois and bothunter info is still working

python errors...

"Traceback (most recent call last):
File "/etc/sguil/vt.py", line 126, in <module>
main()
File "/etc/sguil/vt.py", line 29, in main
response = urllib.urlopen('%s?%s' % (url, urllib.urlencode(parameters))).read()
File "/usr/lib/python2.7/urllib.py", line 86, in urlopen
return opener.open(url)
File "/usr/lib/python2.7/urllib.py", line 207, in open
return getattr(self, name)(url)
File "/usr/lib/python2.7/urllib.py", line 436, in open_https
h.endheaders(data)
File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 776, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 1161, in connect
self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
File "/usr/lib/python2.7/ssl.py", line 381, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 143, in __init__
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 305, in do_handshake
self._sslobj.do_handshake()
IOError: [Errno socket error] [Errno 8] _ssl.c:504: EOF occurred in violation of protocol
"