LongTermLogRotatation

From rsyslog wiki

Long Term Log Rotation

If you believe that you will be having a large amount of traffic and will want to keep it for some time and really want to be able to find things. One of the better methods would be to have a directory structure of:

 (host|distinct object)/Year/Month/Day/files per hour.

Here is how you can do this with rsyslog: All you have to do is make a directory /var/log/logdir and rsyslog will create everything below it. Note, only the mails are being placed in an hour by hour basis but you can do this with any of

$template DailyPerHostLogs,"/var/log/logdir/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%_messages.log"
*.info;mail.none;authpriv.none;cron.none                -?DailyPerHostLogs

# The authpriv file has restricted access.
# We will use the secure log for two purpose one for history 
$template DailySecureLog,"/var/log/logdir/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%_secure.log"
authpriv.*                                              -?DailySecureLog
authpriv.*                                              /var/log/secure

$template HourlyMailLog,"/var/log/logdir/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%_mail.%$HOUR%"
# Log all the mail messages in one place.
mail.*                                                  -?HourlyMailLog
#mail.*                                                  -/var/log/maillog

I do not have seriously great tools to work with this format but they should not be that bad. This is easier to setup, manage, understand than a database.

(databases are not that difficult but if new people walk into your environment having a file /etc/syslog.conf say read rsyslog.conf and it having the above will be pretty quick)