Sniffing Decrypted TLS Traffic With Security Onion
Tanesha Prately
I have Security Onion installed and doing full packet capture on my home network. I also have a firewall (pfSense) which does my routing. I have Security Onion sitting just behind my firewall and mirrored from a switch. It works great except I want to decrypt my SSL traffic. I would very much prefer to have the decryption take place on the fly instead of manually decrypting with wireshark that way Snort sees everything as well. I installed squid on the firewall to try to proxy my traffic but it is not working right. It forwards everything on but I can't get it to decrypt (run live capture on the firewall and it is still encrypted). If I can get this working, will that be enough (using the firewall's private key to decrypt)? If so, how would I go about setting this up in Security Onion? Thank you for your help.
The SslBump feature in Squid allows the proxy to inspect the decrypted web traffic, but the tools on the Security Onion node (Snort, Zeek etc.) will still only see the encrypted traffic because they don't support passive TLS decryption.
Wouldn't it be awesome to have a NIDS like Snort, Suricata or Zeek inspect HTTP requests leaving your network inside TLS encrypted HTTPS traffic? Yeah, we think so too! We have therefore created this guide on how to configure Security Onion to sniff decrypted TLS traffic with help of PolarProxy.
PolarProxy is a forward TLS proxy that decrypts incoming TLS traffic from clients, re-encrypts it and forwards it to the server. One of the key features in PolarProxy is the ability to export the proxied traffic in decrypted form using the PCAP format (a.k.a. libpcap/tcpdump format). This makes it possible to read the decrypted traffic with external tools, without having to perform the decryption again. It also enables packet analysis using tools that don't have built-in TLS decryption support.
Run the Security Onion setup utility by double-clicking the "Setup" desktop shortcut or executing "sudo sosetup" from a terminal. Follow the setup steps in the Production Deployment documentation and select "decrypted" as your sniffing interface.
Wait for the Elastic stack to initialize, so that the intercepted network traffic becomes available through the Kibana GUI. You can check the status of the elastic initialization with "sudo so-elastic-status".
As I said the IPS modes mentioned are able to receive traffic and return them aftewards, either from interface to interface directly with AF_PACKET or within netfilter via NFQUEUE where you can use routing etc.
Many commercial firewalls can be deployed out of band (as a sniffer) for IDS/IPS. They would need a decrypted feed to be really useful. SSL/TLS decryption needs to be done by an in-band device, because you literally man-in-the-middle the traffic.
With how much traffic is encrypted now, and the generally very high false positive rate, a part of me thinks it might not be worth the effort of tuning anymore. While I do love to play around with it, as it's intriguing from a netsec standpoint, I really just am not sure even in an enterprise environment, if it's worth the time. To me, things like PFBlocker with a very strict setup might be a better option overall for security.
Fortigates, as of FortiOS 6.0 offer you the ability to export a packet flow that has been SSL inspected by a firewall, which is what I am going to do in this post. Unfortunately I dont have hardware to plug into a SPAN port, and I want to inspect my traffic in a virtual machine so im going to use GRE enapsulation to send a decrypt SPAN to aSecurity Onion installation. If you arent aware of security onion, you should get familiar with it, it has a lot of REALLY cool features that are super useful for analyzing traffic flows.
I did a live demo at the CS3Sthlm conference last year, titled "TLS Interception and Decryption", where I showed how TLS interception can be used to decrypt and analyze malicious HTTPS network traffic. During the demo I used DNS-over-HTTPS (DoH) and posted messages to Pastebin and Twitter, pretending to be a malware or malicious actor. The HTTPS network traffic was decrypted and analyzed live as part of my demo. The CS3Sthlm organizers have posted a video recording of the live demo on YouTube.
The TLS decryption was performed by connecting a laptop to a custom WiFi access point, which was a Raspberry Pi configured as in our "Raspberry Pi WiFi Access Point with TLS Inspection" blog post. I additionally enabled the PCAP-over-IP feature in PolarProxy by starting it with the "--pcapoverip 57012" option. This allowed me to connect with Wireshark andNetworkMiner to TCP port 57012 on the TLS proxy and stream the decrypted traffic in order to perform live network traffic analysis.
Wireshark was one of the first network traffic analysis tools to implement HTTP/2 support, much thanks to Alexis La Goutte. However, Wireshark's excellent "File > Export Objects" doesn't yet support extraction of files from HTTP/2 traffic. There are other ways to extract HTTP/2 file transfers with Wireshark, but they require a few additional steps in order to carve out the file to disk.
Wouldn't it be awesome to have a NIDS likeSnort,Suricata orZeek inspect HTTP requests leaving your network inside TLS encrypted HTTPS traffic? Yeah, we think so too! We have therefore created this guide on how to configureSecurity Onion to sniff decrypted TLS traffic with help of PolarProxy.
The decrypted traffic can now be accessed via PolarProxy's PCAP-over-IP service on TCP 57012. We can leverage tcpreplay and netcat to replay these packets to our dummy network interface in order to have them picked up by Security Onion.
PolarProxy is primarily a TLS forward proxy, but it can also be used as a TLS termination proxy or reverse TLS proxy to intercept and decrypt incoming TLS traffic, such as HTTPS or IMAPS, before it is forwarded to a server. The proxied traffic can be accessed in decrypted form as a PCAP formatted data stream, which allowsreal-time analysis of the decrypted traffic by an IDS as well as post incident forensics with Wireshark.
There are setups for which it is preferable to also encrypt the internal sessions between PolarProxy and the final server. One such setup is when the server is hosting a web service with support for the HTTP/2 protocol, which in practice always uses TLS. Luckily PolarProxy is designed to decrypt and re-encrypt proxied traffic while also forwarding important TLS parameters, such asALPN andSNI, between the internal and external TLS sessions.
I have previously blogged abouthow to extract John-the-Ripper hashes from Kerberos network traffic with NetworkMiner. We have now added support for presenting LANMAN and NTLM credentials as JtR hashes as well.
Jan Hesse sent us a feature request on Twitter earlier this year, where asked about support for FritzBox captures. We are happy to announce that NetworkMiner now supports the modified pcap format you get whensniffing network traffic with a FritzBox gateway.
Our transparent TLS proxyPolarProxy is gaining lots of popularity due to how effective it is at generating decrypted PCAP files in combination with how easy it is to deploy. In this blog post we will show how to run PolarProxy in Docker.
The Docker file we used in this blog post defines two volumes. The first volume is mounted on "/var/log/PolarProxy" in the container, which is where the decrypted network traffic will be stored as hourly rotated PCAP files. The second volume is the polarproxy home directory, under which PolarProxy will store its private root CA certificate.
Create a container called "polarproxy", which has the "pcap" and "polarproxy" directories mounted as volumes. The service on TCP 10080 will serve the proxy's public root cert over HTTP. The localhost:57012 service is a Pcap-over-IP server, from which the decrypted network traffic can be streamed in real-time.
It probably makes more sense to forward the decrypted traffic to an IDS or other type of network security monitoring tool though. See our blog post "Sniffing Decrypted TLS Traffic with Security Onion" for instructions on how to use netcat and tcpreplay to send the decrypted traffic to a monitor interface.
The new "--pcapoveripconnect" option can be used to let PolarProxy connect to aPCAP-over-IP listener and send it a live PCAP stream of decrypted traffic over TCP. This option complements PolarProxy's "--pcapoverip" option, which sets up a PCAP-over-IP listener that serves clients with the same PCAP stream. Thanks to Andy Wick for suggesting adding a PCAP-over-IP connector to PolarProxy!
e2b47a7662