Merge EventChannel fix into 2.8?

[DefensiveDepth]

Is it possible to merge the EventChannel bug fix into 2.8 so that stable binaries with this issue fixed could be released?

Thanks,

-Josh

[Kevin Branch]

+1

I sure would appreciate if this bug fix could be back ported to 2.8.  I've been eager to use OSSEC to monitor Sysmon events for a long time but have held back due to this issue.  

Kevin

[dan (ddp)]

I'm afraid it will fall to the same issues 2.9 is having right now, but I will give it a shot.

--

---
You received this message because you are subscribed to the Google Groups "ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brent Morris]

Would it be easier to host a compiled version of the fixed client?  I think that might solve some of the challenges here...

[DefensiveDepth]

@Brent, the 2.9 beta that has it fixed?

-Josh 

[Brent Morris]

(I'm assuming it is fixed in 2.9) - sure!  Compile and post the 2.9 client binaries on ossec.net with checksums, etc.

Or would this create other issues?

[dan (ddp)]

On Mon, Sep 21, 2015 at 6:09 PM, Brent Morris <brent....@gmail.com> wrote:
> (I'm assuming it is fixed in 2.9) - sure! Compile and post the 2.9 client
> binaries on ossec.net with checksums, etc.
>
> Or would this create other issues?
>

The issue is finding the time to do a complete release. Find that time
for the powers tha tbe, and it'll get done.

If anyone happens to remember which commit fixed the issues, let me
know. It's not jumping out from the commit log and I ignore a lot of
the Windows stuff.

>
>
> On Monday, September 21, 2015 at 2:19:58 PM UTC-7, DefensiveDepth wrote:
>>>>>
>>>>> @Brent, the 2.9 beta that has it fixed?
>>
>>
>> -Josh
>

[dan (ddp)]

Never mind, I think I found it.
If anyone wants to test this out before I look into what else needs to
be done for a release, I'd really appreciate it:
https://github.com/ddpbsd/ossec-hids/tree/283

I guess I should see if my fix for hybrid mode was in 2.8.2 or just pre-2.9...

[DefensiveDepth]

I will attempt to build the binary tomorrow morning and do some testing...

-Josh 

[DefensiveDepth]

Got most of the way through the build, then hit a wall, see errors here:

http://screencast.com/t/jHFO69Ml 

I will take another stab at it tonight/tomorrow--If anybody has any comments on the current errors, let me know.

Thanks

-Josh

[dan (ddp)]

On Thu, Sep 24, 2015 at 6:31 AM, DefensiveDepth <joshb...@gmail.com> wrote:
> Got most of the way through the build, then hit a wall, see errors here:
>
> http://screencast.com/t/jHFO69Ml
>

I didn't even make any changes there. Try adding "#include <unistd.h>"
to src/syscheckd/seechanges.c.

> I will take another stab at it tonight/tomorrow--If anybody has any comments
> on the current errors, let me know.
>
> Thanks
>
> -Josh
>
>

[DefensiveDepth]

@Dan, added and tried the build again - errored out with the same exact message.

 

-Josh 

[DefensiveDepth]

I should make it clear that I am using this as a guide:

http://ossec-docs.readthedocs.org/en/latest/manual/installation/compile-ossec-mingw.html

[SoulAuctioneer]

That is my doing. When fixing CVE-2015-3222 I inadvertantly broke the Windows builds with my backport to 2.8.2. I fixed in the master branch so 2.9 wouldn't have the problem but never felt the need to backport the fix but since we are doing another 2.8.x release it seems like we should. You need some form of this to get things working again:

https://github.com/awiddersheim/ossec-hids/commit/d65dc132b5da831ec3c3c8b20b9c19862616cfac

Sorry about that.

[SoulAuctioneer]

Was talking to Dan today. Will try to put together some merge requests to his branch and 2.8.3 that will hopefully fix these things. Hopefully will find some time in the next few days to make that happen.

[DefensiveDepth]

Sounds great, thanks!

Let me know how I can help.....

-Josh

[dan (ddp)]

I've updated my branch with Andrew's changes. Please give it another
shot when you get a chance.
https://github.com/ddpbsd/ossec-hids/tree/283

[DefensiveDepth]

Thanks, but unfortunately, new errors:

http://screencast.com/t/bB8BGgoYSj

-Josh

[dan (ddp)]

Thanks. I'll recheck my pull from Andrew, and try to remember to boot my linux machine tonight.

[DefensiveDepth]

Looks like @Brent was able to successfully build it on Centos... Not sure if my env is borked or not....

Either way, I will test the binary in the next day and let you know how what is looks like... 

Thanks

-Josh

[dan (ddp)]

Great, thanks!

[SoulAuctioneer]

The compile errors you posted look like they might be because Dan's branch is missing some things.

[DefensiveDepth]

Ok, so Brent & I confirmed that good build we had was not the correct branch.

So we are back to square one: the errors here are still holding up the build. http://screencast.com/t/bB8BGgoYSj

Thanks,

-Josh

[SoulAuctioneer]

Might just need to add this line into error_messages.h in Dan's branch:

https://github.com/awiddersheim/ossec-hids/blob/master/src/error_messages/error_messages.h#L44

[dan (ddp)]

There's definitely more than that. Adding that line I still get:
/tmp/ccw4cOwc.o:read_win_event_channel.c:(.text+0xcdb): undefined
reference to `mkstemp_ex'
/tmp/ccw4cOwc.o:read_win_event_channel.c:(.text+0xe19): undefined
reference to `rename_ex'
/usr/bin/i686-w64-mingw32-ld: /tmp/ccw4cOwc.o: bad reloc address 0xd84
in section `.rdata'
collect2: error: ld returned 1 exit status

Unfortunately, google doesn't help with mkstemp_ex or rename_ex.

[dan (ddp)]

On Wed, Sep 30, 2015 at 10:31 PM, dan (ddp) <ddp...@gmail.com> wrote:
> On Wed, Sep 30, 2015 at 8:22 PM, SoulAuctioneer
> <awidde...@hotmail.com> wrote:
>> Might just need to add this line into error_messages.h in Dan's branch:
>>
>> https://github.com/awiddersheim/ossec-hids/blob/master/src/error_messages/error_messages.h#L44
>>
>
> There's definitely more than that. Adding that line I still get:
> /tmp/ccw4cOwc.o:read_win_event_channel.c:(.text+0xcdb): undefined
> reference to `mkstemp_ex'
> /tmp/ccw4cOwc.o:read_win_event_channel.c:(.text+0xe19): undefined
> reference to `rename_ex'
> /usr/bin/i686-w64-mingw32-ld: /tmp/ccw4cOwc.o: bad reloc address 0xd84
> in section `.rdata'
> collect2: error: ld returned 1 exit status
>
> Unfortunately, google doesn't help with mkstemp_ex or rename_ex.
>

Derp, found those. I probably shouldn't have settled for decaf.

[DefensiveDepth]

When in doubt, caffeinate!

Is the mkstemp error possibly related to the version of mingw32 we are running?

[dan (ddp)]

I've updated the branch again. I managed to compile a binary, but
can't test it at the moment.
I'm running a *nix build or two in the mean time to make sure I didn't
mess anything up there.

[dan (ddp)]

(Hint: I did, but I'll deal with that fallout later :-P)

[DefensiveDepth]

Built great. (Thanks!)

Installed and running on 2008 R2 right now. Appears to be working correctly.  Getting a massive number of the following errors in the client log:

=====================

2015/10/01 16:24:59 ossec-agent: ERROR: Could not mkstemp_ex() temporary bookmark (tmp/Microsoft-Windows-Sysmon_Operational-a03592) for (Microsoft-Windows-Sysmon/Operational)

2015/10/01 16:24:59 ossec-agent: ERROR: Could not create temporary file (tmp/Microsoft-Windows-Sysmon_Operational-a03592) which returned (3)

2015/10/01 16:24:59 ossec-agent: ERROR: Could not mkstemp_ex() temporary bookmark (tmp/Microsoft-Windows-Sysmon_Operational-a03592) for (Microsoft-Windows-Sysmon/Operational)

2015/10/01 16:24:59 ossec-agent: ERROR: Could not create temporary file (tmp/Microsoft-Windows-Sysmon_Operational-a03592) which returned (3)

==================

Will check in the morning to make sure everything is still working right

-Josh

[dan (ddp)]

On Thu, Oct 1, 2015 at 4:34 PM, DefensiveDepth <joshb...@gmail.com> wrote:
> Built great. (Thanks!)
>
> Installed and running on 2008 R2 right now. Appears to be working correctly.
> Getting a massive number of the following errors in the client log:
>
> =====================
>
> 2015/10/01 16:24:59 ossec-agent: ERROR: Could not mkstemp_ex() temporary
> bookmark (tmp/Microsoft-Windows-Sysmon_Operational-a03592) for
> (Microsoft-Windows-Sysmon/Operational)
>
> 2015/10/01 16:24:59 ossec-agent: ERROR: Could not create temporary file
> (tmp/Microsoft-Windows-Sysmon_Operational-a03592) which returned (3)
>
> 2015/10/01 16:24:59 ossec-agent: ERROR: Could not mkstemp_ex() temporary
> bookmark (tmp/Microsoft-Windows-Sysmon_Operational-a03592) for
> (Microsoft-Windows-Sysmon/Operational)
>
> 2015/10/01 16:24:59 ossec-agent: ERROR: Could not create temporary file
> (tmp/Microsoft-Windows-Sysmon_Operational-a03592) which returned (3)
> ==================
>

I'll try to look at these this weekend to try and figure out if
they're a big deal or not.

[dan (ddp)]

I've also made a couple of smaller changes to the branch. It still
compiles for win32 and now compiles for *nix as well.
I still need to make sure the hybrid fix is in, and do some more
testing. After that it's document the changes and submit them. I still
have to figure out the whole git tagging thing, to make sure I don't
clobber anything important.

[DefensiveDepth]

Looks like the client is still stable this morning.  

Do you want me to re-build and test the new changes you made, or wait?

-Josh

[dan (ddp)]

I don't think it would hurt to do it.

[dan (ddp)]

All right, here's my plan if it still seems to be working on the Windows hosts:
1. I need to test on linux.
a. Test upgrades from 2.8.2
b. Test server installs
c. Test agent installs
d. Test hybrid
2. I need to write up some release notes
3. Tag and pull request
4. Coordinate an actual release with the powers that be.
a. PGP signing
b. Website updates
c. Announcements

Anyone see anything I've forgotten?

Here's a zip of the source for anyone who wants to do any testing
(Solaris/OS X testers would be great!):
https://github.com/ddpbsd/ossec-hids/archive/283.zip

[DefensiveDepth]

I will get the current changes tested on Windows this weekend....

[SoulAuctioneer]

Those bookmark failures shouldn't be happening so if you continue to see those I think we will probably need to dig in a bit. Especially if the OSSEC version I gave you (Josh) a few months ago isn't doing the same thing.

[DefensiveDepth]

Just rebuilt (including the changes made at 10:00AM EDT today), and still getting the bookmark failures... @SoulAuctioneer, I thought we saw this issue previously, and you fixed it?

Thanks

-Josh

[SoulAuctioneer]

Yeah, there was this:

https://github.com/awiddersheim/ossec-hids/commit/262630f63674c8e0e5928bf8a002d0a31114e2d6

Not sure that is the problem. Could be a number of things potentially. Is there a tmp directory in the OSSEC directory? Maybe something stupid with permissions? Might be worth using some of the pstools (ProcMon, ProcExp) to see where OSSEC is trying to make those files and see what it might be dying on. Those bookmarks are used to keep track of where OSSEC was last reading from the eventlog so that when you stop/start the service it can pick up where it left off.

[DefensiveDepth]

I uninstalled, deleted the entire ossec install folder, rebooted & then reinstalled.

The bookmark error only occurs after I drop the eventchannel line into the config.

Here is what the install folder looks like:  http://screencast.com/t/5I8UxnusQ44V

Here is a view of the error from procmon:  http://screencast.com/t/D4fGNnfWwhY

I manually created the tmp folder, and that took care of one of the procmon errors (Path Not Found), now I just get a Name Not Found, when it can't find the file in tmp.

Thoughts?

-Josh

[SoulAuctioneer]

Are there errors in the OSSEC log after you create the tmp directory in the OSSEC directory and restart everything?

Looks like the installer needs the following:

https://github.com/ossec/ossec-hids/blame/master/src/win32/ossec-installer.nsi#L146

https://github.com/ossec/ossec-hids/blame/master/src/win32/ossec-installer.nsi#L422

https://github.com/ossec/ossec-hids/blame/master/src/win32/ossec-installer.nsi#L438

Some Procmon errors like "Name Not Found" can probably be expected when things first start up since OSSEC will try to ascertain if a bookmark file exists but that shouldn't result in an error in the OSSEC logs.

[DefensiveDepth]

Creating the tmp dir and restarting services appeared to have fixed it. 

To be sure, I did a clean re-install and created the tmp dir prior to the eventchannel config--After startup, there are currently no bookmark errors.

I also confirmed once again that the eventchannel logs are being parsed correctly.

-Josh

[Vilius Benetis]

Just wanted to congratulate the involved people with success and thank for the long effort!

Vilius

[Swati]

Hi Josh,

Will your fix going to rectify the issue that I reported here : https://groups.google.com/forum/#!searchin/ossec-list/bookmarks/ossec-list/2NPMEfA6NLk/bSecI-CHAAAJ

If so, is it possible to get hold of the binary.

Kind Regards

Swati

[DefensiveDepth]

Swati,

I replied on the linked thread, so as to keep this one focused on the topic at hand.

Thanks

-Josh

[dan (ddp)]

On Fri, Oct 9, 2015 at 8:16 PM, SoulAuctioneer <awidde...@hotmail.com> wrote:
> Are there errors in the OSSEC log after you create the tmp directory in the
> OSSEC directory and restart everything?
>
> Looks like the installer needs the following:
>
> https://github.com/ossec/ossec-hids/blame/master/src/win32/ossec-installer.nsi#L146
> https://github.com/ossec/ossec-hids/blame/master/src/win32/ossec-installer.nsi#L422
> https://github.com/ossec/ossec-hids/blame/master/src/win32/ossec-installer.nsi#L438
>

These have been added to my branch.

> Some Procmon errors like "Name Not Found" can probably be expected when
> things first start up since OSSEC will try to ascertain if a bookmark file
> exists but that shouldn't result in an error in the OSSEC logs.
>

[DefensiveDepth]

Looks great!

New build creates tmp dir, no bookmark errors.

EventChannel logs still being successfully processed.

-Josh

[dan (ddp)]

On Tue, Oct 13, 2015 at 6:46 AM, DefensiveDepth <joshb...@gmail.com> wrote:
> Looks great!
>
> New build creates tmp dir, no bookmark errors.
>
> EventChannel logs still being successfully processed.
>

Awesome. I haven't installed on a win7+ system, does an administrators
group need to be created for it to run properly (saw this in the list
somewhere)?
If so, anyone know how to add that the installer?

[DefensiveDepth]

I believe this is the relevant thread.

I have always installed the client with a user that has local admin privileges, so I have never run into this issue.... 

Anybody else have any input?

[SoulAuctioneer]

If I had to guess, that thread and some of the others you might remember seeing are about the installer setting permissions to the 'Administrators' group. The problem is when Windows is set to use another language that group isn't named the same. The proper way to do this is with well known SID's which some stuff has been updated to use and other stuff not so much. Was working on fixing that completely in 2.9 or 3.0.

[DefensiveDepth]

Is there anything else that would be an issue with continuing to move forward on this?

[DefensiveDepth]

I should clarify - move forward with the release, as is?

-Josh

[dan (ddp)]

I think I was seeing some instability in analysisd on OpenBSD, but
I've been unable to trigger it in the past day. I've seen no crashes
on my linux system.
I want to give it the weekend before declaring this done, but I can
still move ahead with other parts.
I have a basic release notes written up in the ossec-docs repo
(https://github.com/ddpbsd/ossec-docs/blob/283/docs/whatsnew/release-notes/ossec-hids-2.8.3-release-note.txt).
I've also tried to get the attention of Vic Hargrave and Jeremy Rossi
behind the scenes, but haven't heard back. I'll try emailing them this
time.
If anyone can think of something I'm missing, let me know!

[DefensiveDepth]

This all looks good to me, but I have never been involved in a release in the past, so what do I know?  :)

[dan (ddp)]

There is some headway being made on a release. Too many things going on at once, as always.

[DefensiveDepth]

Thanks for the update Dan.

[DefensiveDepth]

And the continued blood & sweat!

[DefensiveDepth]

Looks like 2.8.3 was released Nov 5, somehow I missed that....

Thanks Dan, Andrew & everybody else who got this released!

-Josh