Pcap file location and finding URL requests in logs

[Bryan Jones]

I have setup an Intel NUC with 16 GB Ram, 120 GB SSD and 1 TB second drive with a USB nic for the management interface and the internal nic on the mirrored port. I am running the server/sensor in full packet capture mode. I have been watching the disk grow and only see snort log files taking space. Are these the pcap files?

btw, SO is amazing, thanks for all the work! My current goal is to find out what full http/https urls are being requested on my home network. For our young children, we use opendns family shield, but some youtube content is not appropriate for young children. With three ipads on Saturday afternoon throughout the house, it is hard to monitor without technology. I know google has "youtube for schools" but our house is not a school.

To grep for full http/https urls, will I need to install a squid proxy server with ssl_bump or can SO provide the info I am looking for? I prefer not to have to setup a MITM for https. Note I need the url info to be on the command line so I can email myself a daily summary.

thanks

[Doug Burks]

Hi Bryan,

Replies inline.

On Thu, May 7, 2015 at 11:10 AM, Bryan Jones <brya...@gmail.com> wrote:
> I have setup an Intel NUC with 16 GB Ram, 120 GB SSD and 1 TB second drive with a USB nic for the management interface and the internal nic on the mirrored port. I am running the server/sensor in full packet capture mode. I have been watching the disk grow and only see snort log files taking space. Are these the pcap files?

Full packet capture is stored in
/nsm/sensor_data/HOSTNAME-INTERFACE/dailylogs/YYYY-MM-DD/snort.log.TIMESTAMP

> btw, SO is amazing, thanks for all the work! My current goal is to find out what full http/https urls are being requested on my home network. For our young children, we use opendns family shield, but some youtube content is not appropriate for young children. With three ipads on Saturday afternoon throughout the house, it is hard to monitor without technology. I know google has "youtube for schools" but our house is not a school.
>
> To grep for full http/https urls, will I need to install a squid proxy server with ssl_bump or can SO provide the info I am looking for? I prefer not to have to setup a MITM for https. Note I need the url info to be on the command line so I can email myself a daily summary.

When you ran Setup, did you enable Bro? If so, Bro logs all HTTP URLs
to /nsm/bro/logs/current/http.log. If you enabled ELSA, you can use
it to report on and slice and dice your HTTP logs. Bro also records
SSL certificate hostnames in ssl.log and they are available via ELSA
as well.


--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

[Doug Burks]

Replies inline.

On Mon, May 11, 2015 at 7:54 AM, Bryan Jones <brya...@gmail.com> wrote:
> Redacted so file attached.
>
> I am getting a better understanding of how some of security onion works. but still failing to reach my goal of seeing all unencrypted https / ssl get requests (ie full https://www.youtube.com/watch?v=gqmmpoO1JrY url) As I said in the previous post I am using full capture mode. After grepping all over and going through the biggest database tables I see things like the payload column and x509.log but I can figure out how to put the pieces together.
>
> If there is an ELSA query that can show the full https url let me know.

Security Onion can show you the HTTPS hostname (since it's sent in the
certificate in cleartext), but it does not do any SSL decryption by
default.

> Note I did a apt-get update, I assume this is safe to do, the OS said over 100 packages needed updating.

Please use soup instead:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Upgrade

> A few things are not working, maybe caused by using remote desktop from windows 8.1 laptop? In ELSA, capMe says sending request, but nothing ever happens. In sgiel, some of the right click options work, but wireshark and networkminer won't start no matter what right click/click trick I try to use.

Have you checked /var/log/nsm/securityonion/sguild.log for additional clues?

> I also know there are some command line tools that can take a pcap and key file and unencrypt it but I am not sure how to pick the right key file for that to work.
>
> Last question for now, is there online training that I can purchase and watch on demand? Specific dates are hard to fit into my work schedule.

Sorry, we don't have any on-demand training right now.

> thanks

> On Friday, May 8, 2015 at 4:47:14 PM UTC-4, Doug Burks wrote:
>> Hi Bryan,
>>
>> Replies inline.
>>

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.