CDB working example

[Louis Bernardo]

Hi All,

Does anyone have a working CDB example for me (lists, ossec.conf, and a rule)? It seems no matter what I do it simply won't read my list and insert the text.

Thanks,

L

[Pedro Sánchez]

Hi Louis,

Here you can read a blog post publish a while ago, it is using CDB lists and create one of them from the scratch: https://blog.wazuh.com/cdb-lists/ (Using OSINT to create CDB lists and block malicious IPs)

You can check as well this thread: https://groups.google.com/forum/#!topic/wazuh/JnV-IHanV7E.

Let me know if it is what you are looking for.

I hope it helps,

Pedro 'snaow' Sanchez.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/67f6e6a1-37a6-4942-984e-de7c439a5f65%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Louis Bernardo]

Thanks Pedro,

Will have a look.

On Monday, 3 September 2018 10:06:20 UTC+2, Pedro Sanchez wrote:

Hi Louis,

Here you can read a blog post publish a while ago, it is using CDB lists and create one of them from the scratch: https://blog.wazuh.com/cdb-lists/ (Using OSINT to create CDB lists and block malicious IPs)

You can check as well this thread: https://groups.google.com/forum/#!topic/wazuh/JnV-IHanV7E.

Let me know if it is what you are looking for.

I hope it helps,

Pedro 'snaow' Sanchez.

On Mon, Sep 3, 2018 at 9:51 AM Louis Bernardo <lo...@louisbernardo.com> wrote:

Hi All,

Does anyone have a working CDB example for me (lists, ossec.conf, and a rule)? It seems no matter what I do it simply won't read my list and insert the text.

Thanks,

L

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

[Louis Bernardo]

Ok, checked the blog, it matches what I have done. Will give a quick breakdown of what I am working on.

Objective: Forcepoint Proxy Decoder (done) and Rules(in progress) for full ingestion

Subobjective: Perform CDB lookups for fixed log fields based on Forcepoint logins. The first is to get the Category to be displayed in text instead of a numeral. In the test example the category is 76 by number, the description is "76:Information Technology - Search Engines and Portals". 

Decoder:

<decoder name="forcepoint-proxy-traffic">

    <parent>forcepoint-proxy</parent>

    <prematch offset="after_parent">product=Security</prematch>

    <regex offset="after_prematch">\.*action=(\S+) severity=(\d+) category=(\d+) user=(\.*) loginID=(\.*) src_host=(\S+) src_port=(\d+) dst_host=(\.*) dst_ip=(\S+) dst_port=(\d+) bytes_out=(\d+) bytes_in=(\d+) http_response=(\d+) http_method=(\S+) http_content_type=(\.*) http_user_agent=(\.*) http_proxy_status_code=(\d+) reason=(\w+) disposition=(\d+) policy=(\.*) role=(\d+) duration=(\d+) url=(\.*)</regex>

    <order>action,severity,category,user,loginid,srcip,srcport,dsthost,dstip,dstport,bytes_out,bytes_in,http_response,http_method,http_content_type,http_user_agent,http_proxy_status_code,reason,disposition,policy,role,duration,url</order>

</decoder>

Rules:

<group name="forcepoint,syslog,">

    <rule id="120000" level="0">

        <decoded_as>forcepoint-proxy</decoded_as>

        <description>Forcepoint proxy messages grouped.</description>

    </rule>

    <rule id="120001" level="0">

        <if_sid>120000</if_sid>

        <list field="category" lookup="match_key">/etc/lists/forcepoint/forcepoint-category</list>

        <description>Forcepoint: $(category)</description>

        <group>proxy_drop,Forcepoint,</group>

    </rule>

</group>

Ossec.conf:

 <list>etc/lists/forcepoint/forcepoint-category</list>

Test line for list:

76:Information Technology - Search Engines and Portals

Log example:

<159>Aug 28 10:21:02 10.10.10.10 vendor=Forcepoint product=Security product_version=8.4.0 action=blocked severity=7 category=76 user=LDAP://thisisfake.com OU\=Desktop Users,OU\=Users,OU\=IT Service Desk,DC\=com/Akuna Matata loginID=AM99 src_host=10.10.10.20 src_port=53728 dst_host=clients1.google.com dst_ip=216.58.223.46 dst_port=443 bytes_out=0 bytes_in=0 http_response=0 http_method=CONNECT http_content_type=- http_user_agent=Mozilla/5.0_(Windows_NT_6.1)_AppleWebKit/537.36_(KHTML,_like_Gecko)_Chrome/61.0.3163.79_Safari/537.36 http_proxy_status_code=403 reason=- disposition=1031 policy=**WWW_Casual_Time_Policy**Authenticated_User_access role=8 duration=5 url=https://clients1.google.com

Decoding functions correctly:

**Phase 2: Completed decoding.

       decoder: 'forcepoint-proxy'

       action: 'blocked'

       severity: '7'

       category: '76'

       dstuser: 'LDAP://thisisfake.com OU\=Desktop Users,OU\=Users,OU\=IT Service Desk,DC\=com/Akuna Matata'

       loginid: 'AM99'

       srcip: '10.10.10.20'

       srcport: '53728'

       dsthost: 'clients1.google.com'

       dstip: '216.58.223.46'

       dstport: '443'

       bytes_out: '0'

       bytes_in: '0'

       http_response: '0'

       http_method: 'CONNECT'

       http_content_type: '-'

       http_user_agent: 'Mozilla/5.0_(Windows_NT_6.1)_AppleWebKit/537.36_(KHTML,_like_Gecko)_Chrome/61.0.3163.79_Safari/537.36'

       http_proxy_status_code: '403'

       reason: '-'

       disposition: '1031'

       policy: '**WWW_Casual_Time_Policy**Authenticated_User_access'

       role: '8'

       duration: '5'

       url: 'https://clients1.google.com'

**Phase 3: Completed filtering (rules).

       Rule id: '120000'

       Level: '0'

       Description: 'Forcepoint proxy messages grouped.'

So it all works great up to the point where it has to perform the lookup, I also moved the lookup line to the 120000 rule to no avail. 

Any ideas welcome.

Thanks,

L

[Louis Bernardo]

It seems I have been trying to do something the CDB isn't built for. Any suggestions on how to enrich the log events with a category description from a text source? There are hundreds, don't really want to create hundreds of rules. 

On Monday, 3 September 2018 09:51:28 UTC+2, Louis Bernardo wrote:

[migue...@wazuh.com]

Hi Louis,

thanks for all the files and the information.

I'm not sure about what you are trying to achieve, but there might be several reasons why it isn't working.

There is a mistake in this line of the rule 120001

<list field="category" lookup="match_key">/etc/lists/forcepoint/forcepoint-category</list>

It should be

<list field="category" lookup="match_key">etc/lists/forcepoint/forcepoint-category</list>

without the initial "/" . Because the manager uses a relative path to the list file starting from /var/ossec/.

Also, in order to work, the CDB list need to be compiled using the binary

/var/ossec/bin/ossec-makelists

Make sure you execute the command after any change in the list file to update the .cdb file.

Hope it solves the problem.

Best regards.

[Louis Bernardo]

Hi Miquel,

I was trying to resolve the disposition and category from the CDB (I was under the mistaken impression that that was possible). There are hundreds of those and was hoping to avoid creating a rule for each one (cpu cycles, time constraints and some other reasons). I have attached the files that I used for the list (I did compile the lists). 

Is there a way to perform a lookup of a value and replace the numeric value with a text value without creating a rule for each one?

Thanks,

L

[migue...@wazuh.com]

What you purpose is an interesting use case to use the CDB lists as a translation table for rule description, but is not possible to do yet as far as I know.

If you are interested in that capability, you can post an issue in our Github repository in order to be developed in a future. Or do it by yourself and make a pull request.

If you are using an ELK Stack, I think it's possible to make what you said using the Logstash translate filter, but probably won't be the best way to avoid increasing cpu cycles.

Best Regards,

Miguel R.

[Louis Bernardo]

Thanks Miquel,

I will work on the filters for now, but will also log a request to have the CDB function expanded to potentially include this. Considering that the function is already there to iterate over the data in the lists it would be nice if one can just add an amendment to return the value found in as text based on the key index. My python skills are not advanced enough to gauge the complexity of the request. This may be useful for more than just the one set of rules, I have some other ideas I am playing with that could use the same method for data enrichment. 

Thanks for all the help.

[Louis Bernardo]

Morning Miguel, 

I have opened an issue for this. 

https://github.com/wazuh/wazuh/issues/1218

I have also started working on the filters, once I have the rules complete with the filters I will post the implementation method here so that they can be checked for integration into the platform. 

Thanks for all the help.

On Monday, 3 September 2018 18:28:31 UTC+2, migue...@wazuh.com wrote:

[migue...@wazuh.com]

Morning Louis,

Thank you for your contribution, this kind of ideas help us to improve our product, it is a pleasure to have your feedback.

Our team now will have a look at your idea and consider to develop it for future releases.

Thanks again and if you need something else, don't hesitate to contact us!

Best regards,

Miguel R.