Snare and rsyslog
Information from the developers:
Snare uses tabs to separate fields from the windows event log. If TAB characters are detected within a windows 'field', they will be converted to space characters. If newlines (CR/LF) were included within the original event, they will be converted to single spaces
When Snare is configured to send data using the standard log format, the event will be sent using the following format:
HostName<TAB>MSWinEventLog<TAB>Criticality<TAB>EventLogSource<TAB>SnareCounter<TAB>SubmitTime<TAB>EventID<TAB>SourceName<TAB>UserName<TAB>SIDType<TAB>EventLogType<TAB>ComputerName<TAB>CategoryString<TAB>DataString<TAB>ExpandedString<TAB>MD5 checksum (optional)
Where:
Hostname is the name that the server has been configured with, or alternatively, the value set as a host name override, in the Snare Agent configuration.
Criticality is the criticality assigned to the matching 'objective' within the snare configuration.
EventLogSource is Security, Application, System, Directory Service, DNS Server, File Replication Service, or (for supported enterprise agents), any custom defined log definition.
SnareCounter is a sequential event counter, designed to assist the process of determining delivery percentages when using non-guaranteed transmission protocols.
SubmitTime is the date time stamp of the event record.
EventID is the Windows Event ID.
SourceName is the Windows Event Log from which the event record was derived.
UserName This is the Window's user name.
SIDType This is the type of SID used. In the above example, it is a 'user' SID, but it may also be a 'computer' or other type of SID.
EventLogType This can be anyone of 'Success Audit', 'Failure Audit', 'Error', 'Information', or 'Warning'.
ComputerName This is the Windows computer name.
CategoryString This is the category of audit event, as detailed by the Windows event logging system.
DataString This contains the data strings.
ExpandedString This contains the expanded data strings.
MD5 Checksum (optional) An md5 checksum of the event can optionally be included with each event sent over the network by the Snare for Windows agent. Note that the application that evaluates each record will need to strip the final delimiter, plus the checksum, prior to evaluating the event.
When Snare sends a log in 'syslog format', the format is very similar:
<SYSLOGNUM>CurrentDate<SPACE>HostName<SPACE>MSWinEventLog<TAB>Criticality<TAB>Criticality<TAB>EventLogSource<TAB>SnareCounter<TAB>SubmitTime<TAB>EventID<TAB>SourceName<TAB>UserName<TAB>SIDType<TAB>EventLogType<TAB>ComputerName<TAB>CategoryString<TAB>DataString<TAB>ExpandedString<TAB>MD5 checksum (optional)
Where:
<SYSLOGNUM> is the appropriate numeric syslog facility/priority combination for the objective, as defined in the snare configuratin. (eg: <99>)
More information available from the InterSect Alliance web site: www dot intersectalliance dot com/
We hope you enjoy the combination of Snare and Rsyslog! - Leigh
note that rsyslog will then escape the tab characters, changing them to #011 producing a line that looks like:
mail.abc.com#011MSWinEventLog#0111#011Security#0114169#011Fri
One way to deal with this is to just replace the tabs with spaces
To get rid of #011 we add the following lines to /etc/rsyslog.conf
$EscapeControlCharactersOnReceive off
then everywhere that you would use %msg% in a format, instead use
%msg:::space-cc%
The second line replaces all control characters (in this case we want to get rid of ASCI-11) with spaces. The first line is required to allow the space-cc to work. The resulting messages look like:
mail.abc.com.pk MSWinEventLog 1 Security 4169 Fri
This will replace all other control characters with a space as well, though this is probably a good thing.
A second way is to create custom formats for dealing with this, but preserving the field seperators
if you just replace the tabs with a spaces you loose the ability to do things like look for field 10 because field 8 may contain spaces.
In addition there are two formats of logs that snare can output to syslog
the biggest difference between the two syslog formats is the space vs tab between the host and MSWinEventLog.
in some cases it seems like there also sometimes a "1#011" before the field with Security in it in this example and sometimes not. I have not figured out the pattern here yet (it may be the format difference, my parsing logic just looks for a '1' in the first field, and if it finds it, drops that field)
what I do is to setup the following rules to reformat the syslog formatted messages to look like
Jan 1 01:01:01 mail.abc.com MSWinEventLog 1#011Security#0114169#011Fri
I can then filter on the program name MSWinEventLog and later parsing tools can examine specific fields of the log message
$template fixsnareFormat,"%timereported% %HOSTNAME% MSWinEventLog %syslogtag:18:$%%msg:::drop-last-lf%\n" $template fixsnareForwardFormat,"<%pri%>%timereported% %HOSTNAME% MSWinEventLog %syslogtag:18:$:%%msg:::drop-last-lf%\n"
$template fixsnareFormat2,"%timereported% %fromhost-ip% broken-MSWinEventLog %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" $template fixsnareForwardFormat2,"<%pri%>%timereported% %fromhost-ip% broken-MSWinEventLog %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"
:hostname, contains ,"MSWinEventLog" /var/log/messages;fixsnareFormat2 & @192.168.1.8;fixsnareForwardFormat2 & ~
:syslogtag, startswith, "MSWinEventLog#011" /var/log/messages;fixsnareFormat & @192.168.1.8;fixsnareForwardFormat & ~
*.* /var/log/messages;TraditionalFormat *.* @192.168.1.8
note that the *2 formats do not really fix things up, but they at least give you a sane hostname and program field (failure to do this can get very interesting, especially if you do something like dynafiles per host where you can end up with thousands of different files)
