Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Suricata Ip Reputation List Download
140 views
Skip to first unread message
Patricia Strawbridge
Jan 3, 2024, 6:13:02 AM1/3/24
to
I figured out how to fix the duplicate error and also SELKS team told me how to fix the error that I was getting from Scirius. This time I was able to add the custom categories, signatures/rules and reputation file via Scirius without errors.
suricata ip reputation list download
DOWNLOAD https://t.co/eZ38nhihCh
The purpose of the IP reputation component is the ranking of IP Addresses within the Suricata Engine. It will collect, store, update and distribute reputation intelligence on IP Addresses. The hub and spoke architecture will allows the central database (The Hub) to collect, store and compile updated IP reputation details that are then distributed to user-side sensor databases (Spokes) for inclusion in user security systems. The reputation data update frequency and security action taken, is defined in the user security configuration.
The intent of IP Reputation is to allow sharing of intelligence regarding a vast number of IP addresses. This can be positive or negative intelligence classified into a number of categories. The technical implementation requires three major efforts; engine integration, the hub that redistributes reputation, and the communication protocol between hubs and sensors. The hub will have a number of responsibilities. This will be a separate module running on a separate system as any sensor. Most often it would run on a central database that all sensors already have communication with. It will be able to subscribe to one or more external feeds. The local admin should be able to define the feeds to be subscribed to, provide authentication credentials if required, and give a weight to that feed. The weight can be an overall number or a by category weight. This will allow the admin to minimize the influence a feed has on their overall reputation if they distrust a particular category or feed, or trust another implicitly. Feeds can be configured to accept feedback or not and will report so on connect. The admin can override and choose not to give any feedback, but the sensor should report these to the Hub upstream on connect. The hub will take all of these feeds and aggregate them into an average single score for each IP or IP Block, and then redistribute this data to all local sensors as configured. It should receive connections from sensors. The sensor will have to provide authentication and will provide feedback. The hub should redistribute that feedback from sensors to all other sensors as well as up to any feeds that accept feedback. The hub should also have an API to allow outside statistical analysis to be done to the database and fed back in to the stream. For instance a local site may choose to change the reputation on all Russian IP blocks, etc.
I'd like to start with an acknowledgement: It's almost a certainty that the attackers responsible for Sunburst have burned these C2s and moved on, and thusly adding these IPs to an IP Reputation list is kind of pointless. But I just want to demonstrate how to set up IPREP within Security Onion and I figured the Sunburst addition is topical.
Knowing which file to edit and which file to leave for salt to handle is not always easy. For example, changing configuration for suricata.yaml will be instead be done in /opt/so/saltstack/local/pillar/minions/onion_eval.sls.
Though we've saved these files in our onion machine, none of them are actually in the suricata docker container. In order to get them there we need to bind the files we created earlier to the suricata container.
The IP is an IPv4 address in the quad-dotted notation or an IPv6 address. Both IP types support networks in CIDR notation. The category is the number as defined in the categories file. The reputation score is the confidence that this IP is in the specified category, represented by a number between 1 and 127 (0 means no data).
Navigate to line 14, the address-groups list the definitions for HOME_NETand EXTERNAL_NET.
If need be, uncomment the fourth occurrence ofHOME_NET, this should be on line 18.
Add IPs here that you want aspart of your HOME_NET, or home network.
The next step is to read over the "Step 2"section of the file, and decide which outputs you want enabled. Forreference, we have the following enabled on our suricata.yaml:
Some of the rules include in the Emerging Threats, and other open sourcerulesets are just too strict for standard operations. For this reason, andbased on other recommendations found on their forums, we have compiled alist of rules that we think should be disabled. Open the filedisable.conf with your favorite text editor. If the fileis blank, there is no need to worry. The following command opens thefile in nano:
Move the script into the /etc/suricata/lua-output directory that you just made. This is the default directory where Suricata will look for Lua output scripts to execute. Change the permissions of the script to be an executable using:
Create the file ufw_unblock.pyEither through a text editor or the terminal, enter the following code into the file and save it.import osfrom datetime import datetimeoutput = []with open('/var/log/suricata/blocked_ips.log') as file: for line in file: lineArray = line.split() dateArray = lineArray[1].split("-") ip = lineArray[0] reportDate = dateArray[0] reportDate = datetime.strptime(reportDate, "%m/%d/%Y") today = datetime.today() # Get the difference in days difference = abs((today - reportDate).days) # If it has been a week or more, unblock the ip from ufw if difference >= 7: os.system("sudo ufw delete deny from " + str(ip)) # Otherwise, add the line to the output list to write to the file else: output.append(line)# Remove the lines that have been unblocked by writing over with the ones that were notfile = open('blocked_ips.log', 'w')file.writelines(output)file.close()Modify the abuseipdb_report.lua script located in /etc/suricata/lua-output/
In these examples the DNS query string is checked against three differentreputation lists. A MD5 list, a SHA256 list, and a raw string (buffer) list.The rules will only match if the data is in the list and the reputationvalue is higher than 200.
Starting out slowly, before jumping into anything like external reputation sources, I'm honestly not sure how I even get into Suricata alarm data in the SIEM? I can see alarms in the Suricata Dashboards and the "Filebeat Suricata" counter on the SIEM Overview page is incrementing, but when I go into the Network view it appears that I can only drag objects from the "Top Talkers" and "Top DNS Domains" tables into Timeline.
Switching over to the SIEM, I can follow what you are doing on the Host view, but I'm not sure that helps me - mainly because Suricata events are Network events. Following your example, I can see the host in my environment, and I can drag it into the Timeline, but that gives me all of the Suricata alarms from that "probe" (to use our parlance) - what I need to dig into is the device/IP that generated the event. As an aside, I do not seem to be able to use Suricata filters (e.g. NOT suricata.eve.event_type:"stats") on the Host view, but they do work in the Network view.
It might help if I explain our workflow: A Suricata alarm is generated and we first evaluate the source.ip and destination.ip. We check Packetbeat data for other DNS/HTTP/HTTPS/etc activity from the source.ip to see if anything else suspicious happened. We check the destination.ip for traffic with other internal IPs, compare it against IP reputation lists, threat intel feeds, etc.
What I really need is a Suricata dashboard instead of say "Top Talkers" as the focal point of the Network view. In that view I would be able to see source.ip, destination.ip, source.port, destination.port, and a bunch of the suricata.eve.* fields - very similar to what is in the Suricata Event and Alert dashboards, but with the ability to drag the contents of those fields into the timeline.
The Host and Network views are generally aligned with their respective event types, but there is no strict delineation. For example you can view your Suricata events in the events table at the bottom of the hosts view by filtering the host view on event.module:"suricata" Then, if you choose to, you can drag the source.ip or destination.ip values from the events table into the timeline.
Likewise you can start by viewing all Suricata events in the Timeline by adding the same filter to the KQL Bar in the timeline: event.module:"suricata". Then you can continue your investigation by dragging items from the Suricata events themselves into the query builder as filters to expand or narrow your search. To include non-Suricata events in your results, you'll need to change the AND/OR Search selector (the the left of the KQL bar in Timeline) to OR search.
Right know I have DMZs (each exposed VLAN got a different one), GIANT IP Blocklists (edrop, firehol level1, etc) that update daily on the WAN, millions of DNS blocked for the LAN (I was totally crazy with this setup, I really need to lower the blocks).
Translated to "human languaje"? My exposed machines now are isolated from my network. They totally block the "famous" attackers (ip blocklists) and cant navigate to the dangerous places (dns blocklists).
About the IPS
1.- Yes, I could just enable IPS on the firewall but...first of all, I am already using big and very very active blocklists, updated daily. I THINK the most dangerous traffic is just blocked by default, no need to check it.
2.- I have found posts 6 YEARS old saying something similar to "IPS is not worth it, they just work for non ecncrypted traffic". 6 years old this was a way of thinking. There is any stat about how much attacking traffic is non encrypted on 2022? If it were a 50% normal 50% encrypted IPS could still worth it, but I FEAR (total personal opinion) the number will be something similar to 10%-90% now
3.- In the case I finally use the IPS...we are speaking of attack patterns that can appear by surprise, we need good lists to be ready and have updated block rules. The free rules dont update very much (and remember I got IP Blocklists that update every few hours). So dont know were to find good lists.
4.- This is related to the prior thought. I know there is ET Telemetry, but dont find much reviews around. But I have found some posts (nor much! 2 or 3) that made a scary statements: ET feed from your alerts list so you get their rules in exchange, some people claim if that list is always empty you wont receive new rules (and using large lP Bocklists hopefully it will be always empty). What is your opinion about this? Its true? There only 2 or 3 posts, dont get me wrong, but there arent much messages about ET Telemetry and thi sis totally important.
35fe9a5643
suricata ip reputation list download
DOWNLOAD https://t.co/eZ38nhihCh
The purpose of the IP reputation component is the ranking of IP Addresses within the Suricata Engine. It will collect, store, update and distribute reputation intelligence on IP Addresses. The hub and spoke architecture will allows the central database (The Hub) to collect, store and compile updated IP reputation details that are then distributed to user-side sensor databases (Spokes) for inclusion in user security systems. The reputation data update frequency and security action taken, is defined in the user security configuration.
The intent of IP Reputation is to allow sharing of intelligence regarding a vast number of IP addresses. This can be positive or negative intelligence classified into a number of categories. The technical implementation requires three major efforts; engine integration, the hub that redistributes reputation, and the communication protocol between hubs and sensors. The hub will have a number of responsibilities. This will be a separate module running on a separate system as any sensor. Most often it would run on a central database that all sensors already have communication with. It will be able to subscribe to one or more external feeds. The local admin should be able to define the feeds to be subscribed to, provide authentication credentials if required, and give a weight to that feed. The weight can be an overall number or a by category weight. This will allow the admin to minimize the influence a feed has on their overall reputation if they distrust a particular category or feed, or trust another implicitly. Feeds can be configured to accept feedback or not and will report so on connect. The admin can override and choose not to give any feedback, but the sensor should report these to the Hub upstream on connect. The hub will take all of these feeds and aggregate them into an average single score for each IP or IP Block, and then redistribute this data to all local sensors as configured. It should receive connections from sensors. The sensor will have to provide authentication and will provide feedback. The hub should redistribute that feedback from sensors to all other sensors as well as up to any feeds that accept feedback. The hub should also have an API to allow outside statistical analysis to be done to the database and fed back in to the stream. For instance a local site may choose to change the reputation on all Russian IP blocks, etc.
I'd like to start with an acknowledgement: It's almost a certainty that the attackers responsible for Sunburst have burned these C2s and moved on, and thusly adding these IPs to an IP Reputation list is kind of pointless. But I just want to demonstrate how to set up IPREP within Security Onion and I figured the Sunburst addition is topical.
Knowing which file to edit and which file to leave for salt to handle is not always easy. For example, changing configuration for suricata.yaml will be instead be done in /opt/so/saltstack/local/pillar/minions/onion_eval.sls.
Though we've saved these files in our onion machine, none of them are actually in the suricata docker container. In order to get them there we need to bind the files we created earlier to the suricata container.
The IP is an IPv4 address in the quad-dotted notation or an IPv6 address. Both IP types support networks in CIDR notation. The category is the number as defined in the categories file. The reputation score is the confidence that this IP is in the specified category, represented by a number between 1 and 127 (0 means no data).
Navigate to line 14, the address-groups list the definitions for HOME_NETand EXTERNAL_NET.
If need be, uncomment the fourth occurrence ofHOME_NET, this should be on line 18.
Add IPs here that you want aspart of your HOME_NET, or home network.
The next step is to read over the "Step 2"section of the file, and decide which outputs you want enabled. Forreference, we have the following enabled on our suricata.yaml:
Some of the rules include in the Emerging Threats, and other open sourcerulesets are just too strict for standard operations. For this reason, andbased on other recommendations found on their forums, we have compiled alist of rules that we think should be disabled. Open the filedisable.conf with your favorite text editor. If the fileis blank, there is no need to worry. The following command opens thefile in nano:
Move the script into the /etc/suricata/lua-output directory that you just made. This is the default directory where Suricata will look for Lua output scripts to execute. Change the permissions of the script to be an executable using:
Create the file ufw_unblock.pyEither through a text editor or the terminal, enter the following code into the file and save it.import osfrom datetime import datetimeoutput = []with open('/var/log/suricata/blocked_ips.log') as file: for line in file: lineArray = line.split() dateArray = lineArray[1].split("-") ip = lineArray[0] reportDate = dateArray[0] reportDate = datetime.strptime(reportDate, "%m/%d/%Y") today = datetime.today() # Get the difference in days difference = abs((today - reportDate).days) # If it has been a week or more, unblock the ip from ufw if difference >= 7: os.system("sudo ufw delete deny from " + str(ip)) # Otherwise, add the line to the output list to write to the file else: output.append(line)# Remove the lines that have been unblocked by writing over with the ones that were notfile = open('blocked_ips.log', 'w')file.writelines(output)file.close()Modify the abuseipdb_report.lua script located in /etc/suricata/lua-output/
In these examples the DNS query string is checked against three differentreputation lists. A MD5 list, a SHA256 list, and a raw string (buffer) list.The rules will only match if the data is in the list and the reputationvalue is higher than 200.
Starting out slowly, before jumping into anything like external reputation sources, I'm honestly not sure how I even get into Suricata alarm data in the SIEM? I can see alarms in the Suricata Dashboards and the "Filebeat Suricata" counter on the SIEM Overview page is incrementing, but when I go into the Network view it appears that I can only drag objects from the "Top Talkers" and "Top DNS Domains" tables into Timeline.
Switching over to the SIEM, I can follow what you are doing on the Host view, but I'm not sure that helps me - mainly because Suricata events are Network events. Following your example, I can see the host in my environment, and I can drag it into the Timeline, but that gives me all of the Suricata alarms from that "probe" (to use our parlance) - what I need to dig into is the device/IP that generated the event. As an aside, I do not seem to be able to use Suricata filters (e.g. NOT suricata.eve.event_type:"stats") on the Host view, but they do work in the Network view.
It might help if I explain our workflow: A Suricata alarm is generated and we first evaluate the source.ip and destination.ip. We check Packetbeat data for other DNS/HTTP/HTTPS/etc activity from the source.ip to see if anything else suspicious happened. We check the destination.ip for traffic with other internal IPs, compare it against IP reputation lists, threat intel feeds, etc.
What I really need is a Suricata dashboard instead of say "Top Talkers" as the focal point of the Network view. In that view I would be able to see source.ip, destination.ip, source.port, destination.port, and a bunch of the suricata.eve.* fields - very similar to what is in the Suricata Event and Alert dashboards, but with the ability to drag the contents of those fields into the timeline.
The Host and Network views are generally aligned with their respective event types, but there is no strict delineation. For example you can view your Suricata events in the events table at the bottom of the hosts view by filtering the host view on event.module:"suricata" Then, if you choose to, you can drag the source.ip or destination.ip values from the events table into the timeline.
Likewise you can start by viewing all Suricata events in the Timeline by adding the same filter to the KQL Bar in the timeline: event.module:"suricata". Then you can continue your investigation by dragging items from the Suricata events themselves into the query builder as filters to expand or narrow your search. To include non-Suricata events in your results, you'll need to change the AND/OR Search selector (the the left of the KQL bar in Timeline) to OR search.
Right know I have DMZs (each exposed VLAN got a different one), GIANT IP Blocklists (edrop, firehol level1, etc) that update daily on the WAN, millions of DNS blocked for the LAN (I was totally crazy with this setup, I really need to lower the blocks).
Translated to "human languaje"? My exposed machines now are isolated from my network. They totally block the "famous" attackers (ip blocklists) and cant navigate to the dangerous places (dns blocklists).
About the IPS
1.- Yes, I could just enable IPS on the firewall but...first of all, I am already using big and very very active blocklists, updated daily. I THINK the most dangerous traffic is just blocked by default, no need to check it.
2.- I have found posts 6 YEARS old saying something similar to "IPS is not worth it, they just work for non ecncrypted traffic". 6 years old this was a way of thinking. There is any stat about how much attacking traffic is non encrypted on 2022? If it were a 50% normal 50% encrypted IPS could still worth it, but I FEAR (total personal opinion) the number will be something similar to 10%-90% now
3.- In the case I finally use the IPS...we are speaking of attack patterns that can appear by surprise, we need good lists to be ready and have updated block rules. The free rules dont update very much (and remember I got IP Blocklists that update every few hours). So dont know were to find good lists.
4.- This is related to the prior thought. I know there is ET Telemetry, but dont find much reviews around. But I have found some posts (nor much! 2 or 3) that made a scary statements: ET feed from your alerts list so you get their rules in exchange, some people claim if that list is always empty you wont receive new rules (and using large lP Bocklists hopefully it will be always empty). What is your opinion about this? Its true? There only 2 or 3 posts, dont get me wrong, but there arent much messages about ET Telemetry and thi sis totally important.
35fe9a5643
0 new messages