Kiwi Syslog Download
Meghan Beas
Typically, when I am successful on a longer SolarWinds project I like to document that success on Thwack. My thought is that if I couldn't find the solution when I started then someone else might come to Thwack with the same question I had. By posting my solution, I hope I'm paying it forward. This is a pay it forward post.
We have a VM that runs the Kiwi Syslog Server and also the Kiwi Syslog Web Access application. I think the Web Access instructions may have asked for a large local drive but our Systems team gave us a max of 5GB.
I have about 30 network devices sending data to my Kiwi server as part of a pilot project. I will add over 300 more eventually but during the pilot I could already see that my hard drive was maxed out. The first thing I learned is that Kiwi Syslog Web Access keeps its data in a separate database from Kiwi Syslog Server. And the Web Access application puts a 4GB cap on the amount of data it will store. Between the files that run Kiwi and the Web Access log files, I filled the hard drive to capacity in 2-3 weeks! The solution there was to increase the size of my local D: drive from 5GB to 10GB. That gave me some breathing room. I now had enough space to store data from Kiwi Syslog.
I considered writing scripts in DOS or Python to move the daily syslog files from the Kiwi server over to the network share but I eventually realized that there is a feature in the Kiwi Syslog Service Manager (File -> Setup) that will archive the data for me. Below the Rules section is a grouping called Schedules that had nothing within it. I right clicked on Schedules and told it Add New Schedule. I now had a GUI I could configure that would back up files based on date or size. It will even append a date to the name of the *.zip file that is one day older than that of the datafile. Let me share why this is important ...
Let's say that today is January 1. Kiwi writes all of its data to a file and at the end of the day it closes that file and starts a new file for the next day. The problem is that the file is written at midnight ... on January 2. Thus, if I look at the file creation date for my log file, I will find data for Jan 1 in the file created on Jan 2. The archive process has the option to append a date (today's or yesterday's) to the filename making it easier to identify when the logs were created.
I ran into a problem using this renaming feature, however, which I haven't been able to resolve. I have Kiwi append the date to my filename each day. If I didn't then the Jan 1 data would be in the same file as the Jan 2 data. That's good. But when the archive job runs, it appends the date (today's or yesterday's) to the log file name and that becomes the zip file name. So if the data collected on Jan 1 is written into file Syslog-2020-01-02.txt, the archive name becomes Syslog-2020-01-02_2020-01-01.zip. I haven't found a way to strip today's date out of the log file name before applying yesterday's date to the archive file name. Even if I didn't add a date to the log file name so that the archive file had the correct name ... and archived daily ... the system would have a conflict when trying to create a new file for Jan 2 with the same name as the Jan 1 file name that hadn't yet been archived and erased. I haven't solved this challenge yet though admittedly, it is very a very minor issue.
The bigger challenge that I faced was that I could get my archive job to run when I kicked it off manually but it would not run via the scheduler. It turns out that the issue was that the Kiwi Syslog Server application was being launched by the Local System account, and that account did not have authorization to write to my network share. To solve the problem, I went into the Windows settings for Services and told it to run that application using a network service account (an Active Directory account created for just this sort of thing) that has admin access to my Kiwi server and also has write access to the network share. Now, when I schedule a job, it runs the way I expect: moving syslog files that are at least 7 days old each into their own zip file and deleting the original datafile. It also now writes a report to the network share telling me which files were moved.
It is a little bit of work but you can also log directly to a SQL database with Kiwi. We had a set of servers we wanted to keep the messages from long term so we set up a SQL database and created an action to write those messages there. It was much easier to search in.
I am setting up a kiwi syslog server. Running into a problem with the filtering not working the way I would expect. I have used Kiwi but that was several years ago. I have setup a display for a specific switch and have tried several different filter possibilities but still getting syslog messages on the display that dont belong to the switch I am trying to watch.
I have tried a ip address - simple filter with the ip address of the switch "10.1.1.2". On the cisco switch, I have used the command logging source-interface vlan 254 which should send out the syslog messages using the ip address in the simple filter I setup. I have also tried the hostname option with the hostname of the switch "Switch1" but same problem.
Along with the Message Text filter, the IP address and hostname filters are only available in the licensed version. They will not work in the free version. I your case you'll need to configure your Cisco switch to send syslog messages using a specific facility, e.g. Local0.
You should then be able to create a priority based filter within Kiwi Syslog to filter these Local0 messages and have them sent to your chosen display. Information on priority filters can be found in the Kiwi Syslog Help file at the following link, www.kiwisyslog.com/.../filters_priority.htm
Thanks for all the replies. I would have like to have know what didnt work in the free version before getting started. I wasted several hours trying to evaluate something that was disabled. Some features told you that they wouldnt work. These didnt. I have looked for information on the page that I downloaded the syslog server from at SW and nothing was listed.
If a feature is disabled, I would suggest that some warning or popup be shown so that you dont spend hours of troubleshooting like I did trying to figure out why a feature wasnt working. Either that or provide a list with the "free" product that lists what features are disabled.
While all these links tell about installing a forwarder, we can directly use the feature in our kiwi syslog to forward logs to our splunk on any of the TCP port, which we can later configure in our splunk as well.
I wouldn't recommend that solution. You'd have to create multiple ports if you want to classify the data differently. With the forwarder that's easy, just create multiple monitor stanzas. The forwarder handles failures much better as well. A bare TCP listener won't properly handle loadbalancing across multiple Splunk servers nor will it gracefully handle connection failures.
I have same problem (wireshark detected UDP log go inside the computer but not displaying to console) when tried to use kiwi 9.4.1 free and evaluation. Sometime, it will pop up with some errors from licensing. How the tools to remove/reset this? Plesae help.
The diagnostics will show you some basic stats for the server itself, top talkers, dns stats, static host entries, and various message stats. If you scroll down, towards the bottom half of the report, you should find some stats relating to message buffers. I would check those first, and see how if you have any overflow messages, and what percentage free is available. I have had numerous different issues cause the service to stop. While I have not performed the same actions you have, the last time I ran into this issue, I was simply adding a new rule. Another time, they narrowed it down to a database issue, as I had several rules dumping data into different tables in the same database.
I had a similar problem except I couldn't get kiwi syslog server running on w2k12R2 server that was also a DC... we're going to move it to a windows 7 host instead as a solution. I couldn't get the service to start and stay running at all.
It seems that some .net updates or something screwed up the compatibility , coworker found the main issue with the setup of the Kiwi Syslog Service Manager and updates our support ticket with this information. Now we are a week into this ticket and we didn't get a response anymore.
To solve the issue with the manager, I can confirm that copying the DLL to the program folder itself solves the messages and errors for about 98% (as in everything works, but sometimes there is a new error that pops-up because of error handling (according to .NET explanation)).
I am new to Splunk. I have installed Splunk ES 6.2.3 as an Indexer on a Windows 2008 R2 server. As an initial test, I installed the application and Forwarder App on another Windows 2008 server (which happens to be a Domain Controller). This seems to work fine as I am able to run searches and reports on the events from the remote server. So far so good ...
We have previously deployed Kiwi Syslog Server ver. 9.4.2. This is already collecting events and alerts from all of our network devices and servers. Ideally, I would like to send the data from Kiwi Syslog into Splunk (rather than have every single device forward log information to the Splunk Indexer directly). Now, I installed the Splunk ES on the same Windows 2008 server that is running Kiwi Syslog. Now, if Kiwi Syslog had been on a different server, I guess I would simply set up a Forwarder there. But how do I get the syslog information into Splunk if it resides on the same server? This may seem like a strange question, but remember I'm a newbie
Best practice: Create a directory to contain the syslog data. Have the Kiwi syslog write to that directory, rotating files regularly. In Splunk, set up a monitor input that tracks the directory. (FYI, it is a local input.) Choose "continuously monitor" rather than "index once."
c80f0f1006