Some issues configuring remote Syslog on OSSEC

[Jérémie Belmudes]

Hi,

First thing, I’m new with OSSEC and didn’t find any clue in archives.

 

I would like an OSSEC server to receive logs from a Juniper firewall, in order to analyse them. Note there is nothing else than OSSEC on the server.

 

Here is the issue:

• Firewall side: everything is fine, logs are received on OSSEC server on default port 514(checked via tcpdump).
• OSSEC side:
•  remote syslog was enabled during setup
•  I change ossec.conf file :

<remote>
  <connection>syslog</connection>
  <allowed-ips>firewall IP</allowed-ips>
</remote>

• OSSEC was restarted
•  I get error “2011/07/18 15:06:56 ossec-remoted(1402): ERROR: Authentication key file '/etc/client.keys' not found. 2011/07/18 15:06:56 ossec-remoted(1750): ERROR: No remote connection configured. Exiting.”

Here is what I tried, to make it works, but still fails :

• I added firewall IP via manage agents ; restart ; configuring logs on firewall to be sent on port 1514 ; getting error for each log received from the firewall “2011/07/18 14:47:44 ossec-remoted(1403): ERROR: Incorrectly formated message from 'IP adress'.”
• Tried to receive logs on another port, assigning in ossec.conf file <port>port_number</port> in <remote>…</remote> ; OSSEC restarted ; no error, but can’t find firewall logs in any log file (/var/log/* & /var/ossec/logs/*)

Now, I’m wondering what I’m missing.

Can OSSEC receive logs from device with syslog ? I read many articles answering yes, but can’t make it works.

If you have any idea…

Thanks,

J

[dan (ddp)]

Hi Jérémie,

On Mon, Jul 18, 2011 at 10:22 AM, Jérémie Belmudes
<jeremie....@gmail.com> wrote:
> Hi,
>
> First thing, I’m new with OSSEC and didn’t find any clue in archives.
>
>
>
> I would like an OSSEC server to receive logs from a Juniper firewall, in
> order to analyse them. Note there is nothing else than OSSEC on the server.
>

If you are not planning on adding some real clients, you should
probably use a local installation.

>
>
> Here is the issue:
>
> Firewall side: everything is fine, logs are received on OSSEC server on
> default port 514(checked via tcpdump).
> OSSEC side:
>
>  remote syslog was enabled during setup
>  I change ossec.conf file :
>
> <remote>
>   <connection>syslog</connection>
>   <allowed-ips>firewall IP</allowed-ips>
> </remote>
>
> OSSEC was restarted
>  I get error “2011/07/18 15:06:56 ossec-remoted(1402): ERROR: Authentication
> key file '/etc/client.keys' not found. 2011/07/18 15:06:56

Either setup a local install or add an agent (even a fake agent should be ok).

> ossec-remoted(1750): ERROR: No remote connection configured. Exiting.”
>

You can check to see if an ossec-remoted is still alive listening on
514/UDP with "netstat -pan | grep 514" (I'm assuming your manager is
Linux).

> Here is what I tried, to make it works, but still fails :
>
> I added firewall IP via manage agents ; restart ; configuring logs on
> firewall to be sent on port 1514 ; getting error for each log received from
> the firewall “2011/07/18 14:47:44 ossec-remoted(1403): ERROR: Incorrectly
> formated message from 'IP adress'.”

1514 is for OSSEC's secure messaging, not syslog. The error is correct.

> Tried to receive logs on another port, assigning in ossec.conf file
> <port>port_number</port> in <remote>…</remote> ; OSSEC restarted ; no error,
> but can’t find firewall logs in any log file (/var/log/* &
> /var/ossec/logs/*)
>

You won't find the firewall logs in any of these files, unless an
alert is triggered.
OSSEC does not log to /var/log at all, so nothing should go from it to
any file there.
/var/ossec/logs/alerts/alerts.log will contain the alerts.
By default OSSEC does not log all log messages, but you can force it
to do so by adding <logall>yes</logall> to the <global> section.
After restarting OSSEC you can find all logs received by OSSEC in
/var/ossec/logs/archives/archive.log. I think this is really helpful
to figure out what is happening on an OSSEC system, and I generally
turn it on. If you have a few copies of various logs you can even use
ossec-logtest to see how they are being decoded.

> Now, I’m wondering what I’m missing.
>
> Can OSSEC receive logs from device with syslog ? I read many articles
> answering yes, but can’t make it works.
>

Yes, it can. I do it, as do many others daily. :)

[Jérémie Belmudes]

Hi,

Thanks for your hints Dan, I make OSSEC starts without any error, but still have somes issues.

Here is where I am :

1. Starting with a clean setup of OSSEC 2.6 on a linux server. Choose "server mode" to enable syslog in OSSEC.
2. Adding a fake agent to avoid errors with manage_agents tools.
3. Adding <allowed-ips> for syslog in ossec.conf file
4. Finally starting ossec.

I can see that OSSEC listen on the port UDP514 (via netstat -navup), and also, that traffic still arrives on the server (via tcpdump).

As you said when logall option is set up, I retrieve all my logs in logs/archives/archives.log.

I tried as you recommanded to test my logs, but they don't pass the test (ossec-logtest -f). 

I found out the problem comes from the Date prefix of the log. 

The prefix generated by OSSEC isn't recognized, but it works when I replace the prefix by the one presented in decoder.xml as exemple.

• generated prefix = 2011 Jul 20 16:57:21 nameOf Server->@IP_firewall
• prefix from decoder.xml = Jan  1 10:02:11 xx

First thing, I don't know where that prefix regex can be updated, surely not in decoder.xml.

At first I thought the remote syslog would be automatically analyzed. If it's not the case I would like to retrieve them in a specific file. I think that archives.log file shouldn't be analyzed (<localfile> in ossec.conf).

Thanks,

J 

2011/7/18 dan (ddp) <ddp...@gmail.com>

--
Cordialement,

Jérémie BELMUDES
jeremie....@gmail.com

[dan (ddp)]

On Wed, Jul 20, 2011 at 11:56 AM, Jérémie Belmudes
<jeremie....@gmail.com> wrote:
> Hi,

> Thanks for your hints Dan, I make OSSEC starts without any error, but still
> have somes issues.
> Here is where I am :
>
> Starting with a clean setup of OSSEC 2.6 on a linux server. Choose "server
> mode" to enable syslog in OSSEC.
> Adding a fake agent to avoid errors with manage_agents tools.
> Adding <allowed-ips> for syslog in ossec.conf file
> Finally starting ossec.
>
> I can see that OSSEC listen on the port UDP514 (via netstat -navup), and
> also, that traffic still arrives on the server (via tcpdump).
> As you said when logall option is set up, I retrieve all my logs in
> logs/archives/archives.log.
> I tried as you recommanded to test my logs, but they don't pass the test
> (ossec-logtest -f).
> I found out the problem comes from the Date prefix of the log.
> The prefix generated by OSSEC isn't recognized, but it works when I replace
> the prefix by the one presented in decoder.xml as exemple.
>
> generated prefix = 2011 Jul 20 16:57:21 nameOf Server->@IP_firewall

This is just a header that's added to the entries. You should remove
it before pushing it through ossec-logtest. ossec-analysisd does not
see this header when trying to decode the messages.

> prefix from decoder.xml = Jan  1 10:02:11 xx
>
> First thing, I don't know where that prefix regex can be updated, surely not
> in decoder.xml.

It may not have to be. Send a few example logs so I can see what's going on.

> At first I thought the remote syslog would be automatically analyzed. If
> it's not the case I would like to retrieve them in a specific file. I think
> that archives.log file shouldn't be analyzed (<localfile> in ossec.conf).

The log messages coming in via syslog are being looked at. They may
not be triggering anything, or they may not be decoded properly. It's
tough to tell without a log sample.

[Jérémie Belmudes]

Here is what I receive with tcpdump:

E....z..?..{........Rl....&.<133>FIREWALL: NetScreen device_id=0100000000000001  [Root]system-notification-00257(traffic): start_time="2011-07-21 10:16:20" duration=0 policy_id=1 service=http proto=6 src zone=ALPHA dst zone=BETA action=Permit sent=0 rcvd=0 src=172.16.0.1 dst=10.0.0.1 src_port=65341 dst_port=80 src-xlated ip=172.16.0.1 port=65341 dst-xlated ip=10.0.0.1 port=80 session_id=46083 reason=Creation.

Now what is written in logs/archives/archives.log file:

2011 Jul 21 10:16:23 OSSEC->192.168.0.1 FIREWALL: NetScreen device_id=0100000000000001  [Root]system-notification-00257(traffic): start_time="2011-07-21 10:16:20" duration=0 policy_id=1 service=http proto=6 src zone=ALPHA dst zone=BETA action=Permit sent=0 rcvd=0 src=172.16.0.1 dst=10.0.0.1 src_port=65341 dst_port=80 src-xlated ip=172.16.0.1 port=65341 dst-xlated ip=10.0.0.1 port=80 session_id=46083 reason=Creation

None of these ones passes bin/ossec-logtest. 

I tried to just remove the green header following decoder regex (see below), but it fails.

It finally works when I replace the green header by the one as the following example given in decoder.xml

Jan  1 10:02:11 xx ns5gt: NetScreen device_id=ns5gt  [No Name]system-notification-00257(traffic): start_time="2006-01-01 10:09:38" duration=0 policy_id=310101 service=tcp/port:1526 proto=6 src zone=Null dst zone=self action=Deny sent=0 rcvd=38 src=10.1.2.3 dst=10.1.1.1 src_port=51350 dst_port=1426

decoder.xml says netscreen logs begin with "Netscreen device_id" :

<decoder name="netscreenfw">

  <program_name />

  <prematch>^NetScreen device_id</prematch>

</decoder>

I tried to modify decoder to accept any characters before "Netscreen device_id", not working. 

Now I don't understand why bin/ossec-logtest requires a date as a prefix whereas decoder.xml says the contrary.

Assuming that header is needed, I'm wondering where it can be configured.

Comparing logs/archives/archives.log and the working example from decoder.xml, the problem seems coming from the date format.

The fact is logs received from the firewall are in standard format. Also date format can't be changed.

Have an idea about it ?

 

2011/7/20 dan (ddp) <ddp...@gmail.com>

[dan (ddp)]

You should see if there is a way to enable the timestamp at the
beginning on your netscreen. OSSEC doesn't seem to like it when that
bit's missing.

On Thu, Jul 21, 2011 at 5:10 AM, Jérémie Belmudes

[Michael Barrett]

How do I make a feature request?

I would like OSSEC to show me the agent info before I confirm delete


____________________________________________
Michael Barrett | Information Security Analyst - Lead | Mortgage Guaranty Insurance Corporation
270 E. Kilbourn Ave. | Milwaukee, WI  53202 USA | ( 1.414.347.6271 | 7 1.888.601.4440 | * michael...@MGIC.com

This message is intended for use only by the person(s) addressed above and may contain privileged and confidential information. Disclosure or use of this message by any other person is strictly prohibited. If this message is received in error, please notify the sender immediately and delete this message.

[Michael Barrett]

Is there a way to configure the ossec agent to forward its local logs to a syslog server?