ARIN databse ftp download started at 6AM UTC & is chewing up bandwidth 16 GB dl and counting

[jsm]

Hi everybody

I run Zabbix for Network monitoring and I noticed that today at 6 AM UTC today something on the network started downloading at around 2 Mbps and is till going. By my maths it's probably downloaded 16 GB already.

I have SO running and using ELSA and CapMe it appears the culprit is ... the SO server.

It is downloading ARIN ASN registration data from ftp://ftp.arin.net/pub/stats/

Which of the SO components is doing this and is anybody else finding this issue?

What extra data would you like me to provide? I have attached the sostat-redacted output and some screenshots.

Thank you all

jsm

[BBCan177]

I polled my sancp data for that address and had this event today see attachment.

Very small bytes compared to yours.

[jsm]

Cheers BBCan177

If it helps for t'shooting the netstat and ps output points to a tcl script "ip2c.tcl"

[Doug Burks]

Hi jsm,

ip2c.tcl is used by Squert to update the IP-to-country mappings. As
you noticed, it runs at 06:00 UTC on Sunday:

# /etc/cron.d/squert-ip2c
#
# crontab entry to update SQueRT ip2c mappings

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

*/5 * * * * root [ -d /var/lib/mysql/securityonion_db/ ] &&
/usr/bin/php -e /var/www/squert/.inc/ip2c.php 1 > /dev/null 2>&1
00 6 * * 0 root [ -d /var/lib/mysql/securityonion_db/ ] && (cd
/var/www/squert/.scripts/ && ./ip2c.tcl > /dev/null 2>&1)

It should download the latest mappings and then apply them to the
database. If yours is consuming that much data, my guess would be
that it's stuck in a loop and continually downloading the mappings.
Please kill any running ip2c.tcl, then run it manually and include all
output in your reply:

sudo -i
cd /var/www/squert/.scripts/ && ./ip2c.tcl

On Mon, Feb 3, 2014 at 4:56 AM, jsm <jsand...@gmail.com> wrote:
> Cheers BBCan177
>
> If it helps for t'shooting the netstat and ps output points to a tcl script "ip2c.tcl"
>

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Doug Burks

[jsm]

Thank you Doug. I killed the ps & ran the command as requested:

soadmin@soserver:~$ sudo -i
root@soserver:~# cd /var/www/squert/.scripts/ && ./ip2c.tcl
Fetching AFRINIC Checksum..
Bookmark not found, Fetching AFRINIC Data.
Verifying transfer.. Looks good, processing..
Processed 2673 IPv4 records and skipped 3990.

Fetching APNIC Checksum..
Bookmark not found, Fetching APNIC Data.
Verifying transfer.. Looks good, processing..
Processed 23647 IPv4 records and skipped 28874.

Fetching ARIN Checksum..
Bookmark found, looking for changes.. Fetching new data from ARIN

Verifying transfer.. Looks good, processing..
Processed 54048 IPv4 records and skipped 54054.

Fetching LACNIC Checksum..
Bookmark found, looking for changes.. Fetching new data from LACNIC

Verifying transfer.. Looks good, processing..
Processed 8678 IPv4 records and skipped 14382.

Fetching RIPE Checksum..
Bookmark found, looking for changes.. Fetching new data from RIPE

Verifying transfer.. Looks good, processing..
Processed 51051 IPv4 records and skipped 69168.

Updating database..
root@soserver:/var/www/squert/.scripts#

It looks to me this ran successfully and Zabbix confirms it only downloaded what it needed to and then stopped.

I'm not sure why it got stuck in a loop and am happy it's working now. Thank you all

[sfear...@gmail.com]

This also happened to me. I didn't notice it until I looked at my own bandwidth graphs almost 15 hours after it ended. Looks like the same FTP attempt after looking through pcaps. It lasted about 24 hours and ended on its own staying at ~1.5 Mbps the entire time.

This SO install had been powered off since 12/30/13. I booted it back up on 2/12, upgraded via soup and then ran setup again if that helps. Looks like the download started not long after I ran setup.