Wincollect 7.3.1
Totaly Pavlina
I have windows data coming into a Splunk instance from a specific domain "DC XYZ". The data from DC XYZ is sent from wincollect. So the format is different. Is there an app or has any solved this before. Installing Splunk Windows TA is not an option.
Ok, not too ugly. Field names largely correspond with what it would normally look like. I'd just write REPORT based extractions for the key=value and key: value pieces. Maybe a few fields that need to be renamed and then just copy all relevant logic from the windows TA (props,transforms,eventtypes,tags,lookups). Make sure to give the copied content unique stanza names (transforms, eventtypes, lookups) to prevent conflicts.
Typically it boils down to writing the field extractions for the custom format in a way that results in similar fields as what the Splunk TA uses and then copy pasting as much as possible from the windows TA to prevent having to reinvent all that logic for CIM mapping and eventtyping etc. How easy that is depends on how different the custom format is from the 'normal' format.
I've been trying to install Wincollect + Wincollect Configuration Console on Windows 2016, 2019 and Windows 10 systems but have not been able to run Wincollect Configuration Console. A message appears MMC could not create Snap-In. I took a look at this: -wincollect-%E2%80%9Cmmc-could-not-create-snap
IBM TechXchange Community offers a constant stream of freshly updated content including featured blogs and forums for discussion and collaboration; access to the latest white papers, webcasts, presentations, and research uniquely for members, by members.
Okay, have been talking with development and here are items to consider based on your questions.
1- Is managed WinCollect deployment supported on a cloud?
Let's break these up by cloud types:
A. QRadar installed in Azure
In Azure hosted QRadar, the WinCollect icon would still be available and you can use managed as long as you have direct line of sight to the QRadar appliance and port 8413 isn't blocked by some resource group/security profile in Azure then yes they can run in managed.
I would guess that due to NAT or other cloud networks/port issues that you would be best off with going standalone. However, I'm not sure if these XP/Win7 boxes are in the cloud as well or external. You might need to clarify the location of the remote polling targets, are they also in Azure? My thoughts here is that if these are collecting a set type of event data that you would go standalone and allow these to forward in. This answer comes down to more infrastructure questions (network issues, locations, visibility) than actual collection. I believe that most large rollouts use standalone and some form of set collection that can come in via Syslog or TLS Syslong from the WinCollect agent or they post an Event Collector or Data Gateway in specific areas were infrastructure and networks cause issues with collection to act as a local collection point.
A lot of users find that going Microsoft Windows Event Forwarding (WEF) is an easy solution as it can be managed via Group Policy. It is a good idea to consider. I think WEF is supported on XP SP2, but not sure what point of sale OS levels we are talking about here without more specific info. WEF and local WinCollect agents are the easiest large implementation you can have as you can change XML configs and push changes easily without impacting WinCollect collection.
Optionally, if line of sight and ports are not an issue, you could go managed. However, with that size of deployment we find that customers typically prefer using standalone and SCCM for management. Things are easier now that IBM has released the Log Source Management app, but I think it comes down to available tools and how your team perfers to work. You can go WEF, managed, or standalone and the tools you have might drive this factor. We often tell users to mix/match collection in an 80/20 rollout if they can. No one ever seems to have a large network where one solution would fix. So, cover most of your network using the easiest collection, then cover the gaps with selected agent rollouts in standalone or managed, pending network (NAT, ports, line of sight) issues.
B. QRadar on Cloud (IBM Softlayer)
As documented here, we do not support managed WinCollect with QRadar on Cloud (QRoC) and users are required to use Standalone and send events from the WinCollect agents to a Data Gateway appliance. I don't know if this applies also to Azure hosted QRadar or not, but take a look at this: _wincollect_capabilities_cloud.html. As for QRoC (IBMs Softlayer hosted installs), there is no Managed WinCollect and the icon is removed from the product.
2- We plan to do WinCollect standalone deployment vis BigFix but trying to figure if there is a more recommended approach or easier way?
Not sure if you mean standalone or managed, as your first question was about managed. That being said, our dev team has some experience with BigFix installs, but this is typically done via IBM Professional Services as far as I'm aware. As mentioned above, you might consider WEF too in the mix of options, if your OSes can support it. If you have a large deployment and you are considering standalone, you are going to want some form of management to replace AgentConfig.xml with a new template, which can be done by SCCM, BigFix, and probably even via Powershell, but some of these tools not might be available or restricted to you.
3- Will a managed deployment be better if we do remote polling? so we install 60 WinCollect agents to collect (500 workstations per agent) events from 30,000 workstations. What's the downside to this approach?
The downside with any large deployment is managing updates. Typically users with large deployments go standalone and have some sort of scriptable update method to push changes either via SCCM, BigFix, or they have another tool they use. The good news is that the new Log Source Management app, if you are not already using it will make bulk edits easier in QRadar as the existing UI was an issue in the standard QRadar UI. If you don't have the Log Source Management app, go install it. Typically, ports and the number of agents a QRadar appliance can manage (500/appliance) is the limiting factor.
4- Or a standalone deployment is better wherein we install one agent per workstation
Again, this comes down to management. One agent per workstation allows you to do a set install on each box and walk away if you do not need to constantly adjust what you poll and just need core logs (System, application, security). If you need to adjust the collection often, you might consider standalone and see how you can push AgentConfig.xml updates or try the log source management app. It is important to decide how you want to collect (unless going WEF) because you want the installer to handle the log source auto creation for you, which is one less step to worry about.
Other
Resource wise, there really isn't much difference if you are talking less than 1 EPS on typical boxes. The more events we need to collect, parse, and convert to Syslog, the more CPU/memory WinCollect will need to get the job done quickly. This is why our performance metrics are wide, as some users are installing agents 1-to-1 where they are collecting only 1 EPS versus a remote polling agent that is collecting 2,500 EPS and trying to turn all of the data from the payloads in to name=value pairs for the event going to QRadar. The root limitation is how much EPS needs to be processed by QRadar as we can collect 2,500 EPS via remote polling and 5,000 via local (1-to-1) collection. There are fuzzy performance numbers for WEF, but Microsoft claims you can do from 0-50k+ via WEF.
As mentioned in chat, this is a good discussion in general. As for Windows XP and Windows 7. As you know we do not support Windows XP, or any operating system that Microsoft deems end of life (EoL), but I can say that development still tests against a few XP boxes and it should collect, but we will not take cases/phone support on legacy operating systems. You are likely running a security risk as I'm sure you are aware with these versions, but collection should occur. If you need to get help either in the support forums (ibm.biz/qradarforums) or here in reddit unofficially, since any cases would be closed. Collection should work for both operating systems unless you have some kind of non-standard setup or are on older XP/Win7 patch versions and are hitting some strange Windows issue.
Look in to WEF, but I'll see if I can append other info here from your core post too. Take a look at what I wrote and let me know if you have follow-up questions. I will cross-post this to both areas were you opened questions.
- Jonathan