Here comes the first story
From rsyslog wiki
[edit] Pre-amble
The reason I started to look at rsyslog as an alternative to the stock syslog, was that I needed support staff to look at maillog entries when diagnosing and trouble-shooting email problems for clients. The best way to do this, without giving staff access to any shell account on the various mail hosts, was to get those mail hosts to log to a central MySQL server and allow access to that server from a web browser.
These instructions detail step-by-step how I performed this using (originally) rsyslog 1.17.5 running on Scientific Linux 4.5 (a Red Hat Enterprise Linux 4 Update 5 derivative) and using phpLogCon as the web interface to the MySQL database. I'm now using the latest rsyslog versions and will continue to update as new releases become made available.
These instructions should hold true not just for Scientific Linux, but for any RHEL-based derivative including CentOS, Tao, Whitebox Linux, etc as they all basically use the Red Hat SRPMs. These instructions also hold true for any rsyslog version up from 1.17.5 to 1.19.x.
[edit] Purpose of this document
To outline the installation instructions for rsyslog on Scientific Linux 4.5
[edit] Resources
The rsyslog homepage is here:
Original spec was obtained from:
http://people.redhat.com/pvrabec/rpms/rsyslog.spec
however pvrabec doesn't maintain this spec so I used it as a basis and updated accordingly on each rsyslog upgrade.
[edit] Spec files
All spec files I maintain can be found here:
ftp://ftp.npgx.com.au/mmansour/specs
Currently I have the following spec files available:
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.17.5.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.18.0.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.18.1.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.18.2.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.19.0.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.19.1.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.19.2.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.19.3.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.19.7.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.19.9.spec
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog_v1.19.10.spec
For the purposes of saving me time in the future, I've submitted my spec file to the excellent rpmforge repository and hope someday they can take over and maintain it.
Note 1: although all these spec files are configured with MySQL enabled (the "--enable-mysql" setting in the spec file), they can still be used even if you're not using MySQL logging.
Note 2: if you're using rsyslog v1.19.0 and above, read the bottom of the wiki page for special instructions on the 1.19.x logging to MySQL.
Note 3: after reviewing my instructions from scratch, I discovered that I also needed the files:
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog.init
ftp://ftp.npgx.com.au/mmansour/specs/rsyslog.sysconfig
in the /usr/src/redhat/SOURCES directory to properly build from the spec files provided.
[edit] Build RPM(s)
Once one of the spec files above have been downloaded and copied into:
/usr/src/redhat/SPECS
and the accompanying rsyslog tarball (that the spec file references) has been downloaded and copied into:
/usr/src/redhat/SOURCES
then it's simply a matter of doing:
# rpmbuild -bb /usr/src/redhat/SPECS/<specfilename>
to build the rsyslog binary RPM's.
[edit] Install the newly built rsyslog RPM
# rpm -Uvh /usr/src/redhat/RPMS/i386/rsyslog-1.17.5-1.i386.rpm
Preparing... ########################################### [100%]
1:rsyslog ########################################### [100%]
warning: /etc/logrotate.d/syslog saved as /etc/logrotate.d/syslog.rpmsave
Note: The installation of this RPM will remove your old syslog setup, as we want it to do, and replace it with rsyslog.
[edit] Start rsyslog
By default rsyslog will start on boot:
# chkconfig --list rsyslog
rsyslog 0:off 1:off 2:on 3:on 4:on 5:on 6:off
Let's start rsyslog:
# service rsyslog start
Starting system logger (rsyslog): [ OK ]
Starting kernel logger (rklogd): [ OK ]
and in the /var/log/messages file we see:
Aug 7 20:09:24 servername1 rsyslogd: [origin software="rsyslogd" swVersion="1.17.5" x-pid="25699"]
[x-configInfo udpReception="No" udpPort="514" tcpReception="No" tcpPort="0"] restart
Aug 7 20:09:24 servername1 rsyslog: rsyslogd startup succeeded
Aug 7 20:09:24 servername1 kernel: rklogd 1.17.5, log source = /proc/kmsg started.
Aug 7 20:09:24 servername1 rsyslog: rklogd startup succeeded
Aug 7 20:05:34 servername1 syslog: syslogd shutdown succeeded#012#000
[edit] Configure MySQL logging
Setup instructions are here:
http://www.rsyslog.com/module-Static_Docs-view-f-rsyslog_mysql.html.phtml
From that URL we see:
The schema definition is contained in the file "createDB.sql". It comes with the rsyslog package. Review it to check that the database name is acceptable for you.
So we need the tarball to extract this file from it (as I don't include this file in the RPM, I will modify the spec some day to include it):
# tar zxvf rsyslog-1.18.0.tar.gz
# cd rsyslog-1.18.0
# ll *sql
-rw-rw-r-- 1 root root 1046 Jun 25 19:42 createDB.sql
Note: in rsyslog v1.19.10 this file can be found here ./plugins/ommysql/createDB.sql after tarball extraction.
We can use this file directly or, if you'd like to specify the database name (which I do):
# vi createDB.sql
modify:
CREATE DATABASE Syslog;
USE Syslog;
to change the "Syslog" name to whatever database name you like.
[edit] Execute the createDB.sql file
Execute the createDB.sql on the MySQL database, which will produce the Syslog database and populate it with the relevant tables.
[edit] Assign MySQL account name and password to the Syslog database
For simplicity, I perform my MySQL work using phpMyAdmin, of course you can use whatever you choose including mysql command prompt. For security purposes, we’ll create a new MySQL username and password which will be used and displayed in the /etc/rsyslog.conf file.
Using phpMyAdmin, it is sufficient to grant this user INSERT privileges to the systemevents table only.
We also create the syslogwriter MySQL user and assign INSERT privileges to the systemevents table only.
The following syslogwriter MySQL accounts were created:
syslogwriter@servername1.example.com.au
syslogwriter@servername2.example.com.au
syslogwriter@servername3.example.com.au
syslogwriter@servernameN.example.com.au
up to N servers that will be logging into this MySQL database.
Note: the above MySQL accounts are needed for each server logging into a "central" MySQL server. If the MySQL server is not going to act as a central repository for syslog messages, then only syslogwriter@servername1.example.com.au is needed.
[edit] Tell rsyslog to log to the database
As explained in the URL above:
Next, we need to tell rsyslogd to write data to the database. As we use the default schema, we do NOT need to define a template for this. We can use the hardcoded one (rsyslogd handles the proper template linking). So all we need to do is add a simple selector line to /etc/rsyslog.conf
*.* >database-server,database-name,database-userid,database-password
So we make this change:
# vi /etc/rsyslog.conf
add the following to the end of the file:
*.* >servername1.example.com.au,Syslog,syslogwriter,LongEncryPtedPaSsWord
Note: it’s important to use TAB spaces between entries above and to modify the values for your setup.
[edit] Secure the /etc/rsyslog.conf file
As discussed in the URL above:
As a side note, it is strongly advisable to make the rsyslog.conf file readable by root only - if you make it world-readable, everybody could obtain the password (and eventually other vital information from it).
So:
# ll /etc/rsyslog.conf
-rw-r--r-- 1 root root 929 Aug 7 21:57 /etc/rsyslog.conf
# chmod 640 /etc/rsyslog.conf
[edit] Restart rsyslog
# service rsyslog restart
Shutting down kernel logger (rklogd): [ OK ]
Shutting down system logger (rsyslog): [ OK ]
Starting system logger (rsyslog): [ OK ]
Starting kernel logger (rklogd): [ OK ]
and:
Aug 7 22:00:13 servername1 kernel: Kernel logging (proc) stopped.
Aug 7 22:00:13 servername1 kernel: Kernel log daemon terminating.
Aug 7 22:00:14 servername1 rsyslog: rklogd shutdown succeeded
Aug 7 22:00:14 servername1 rsyslogd: [origin software="rsyslogd" swVersion="1.17.5" x-pid="25699"]
exiting on signal 15.
Aug 7 22:00:14 servername1 rsyslogd: [origin software="rsyslogd" swVersion="1.17.5" x-pid="6774"]
[x-configInfo udpReception="No" udpPort="514" tcpReception="No" tcpPort="0"] restart
Aug 7 22:00:14 servername1 rsyslog: rsyslogd startup succeeded
Aug 7 22:00:14 servername1 kernel: rklogd 1.17.5, log source = /proc/kmsg started.
Aug 7 22:00:14 servername1 rsyslog: rklogd startup succeeded
Aug 7 22:00:14 servername1 rsyslog: rsyslogd shutdown succeeded#012#000
At this point, we’re now logging into the MySQL database. To view the logs we need to use phpLogCon.
[edit] Install phpLogCon http://www.phplogcon.org/
phpLogCon is used to browse the MySQL database from any Web browser. It also supports creating user accounts for people to view those logged rsyslog entries. This is obviously extremely handy and the real power behind the viewing of the rsyslog entries on a central MySQL server.
The file can be downloaded from here:
http://www.phplogcon.org/Downloads-req-getit-lid-6.phtml
[edit] Extract the tarball
# tar zxvf phplogcon-1.2.3.tar.gz
[edit] Create web path or virtual server
I normally create Apache virtual servers for any of these types of clustered setups, but you can just as easily copy the extracted phplogcon directory to:
/var/www/html/phplogcon
if you choose.
[edit] Move phplogcon to web directory
# mv phplogcon-v1.2.3/* /var/www/html/phplogcon
[edit] Run install.php for phplogcon
Prior to doing this, I had to assign ALL PRIVILEGES access to the Syslog database for the syslogwriter MySQL user otherwise the install.php script would not have the correct privileges to write to the tables. I didn't bother trying to work out exactly the privileges needed.
As explained on the URL:
http://new.phplogcon.org/index.php?module=Static_Docs&func=view&f=/setup.htm
Correct permissions so we can have the config.php created by the install script:
# chown apache.apache /var/www/html/phplogcon/config.php
# chmod 777 /var/www/html/phplogcon/config.php
Note: we'll change to a more secure setup after the install.php is complete.
Go to http://servername1.example.com.au/install/install.php
Click Submit on the first "Welcome to the install of phpLogCon etc" page.
Complete details for Connection Type, Host/IP, Port, etc.
Click "Install phpLogCon" which outputs:
Checking users input... Done!
Creating required tables... Done!
Inserting values into tables... Done!
Creating your config file (config.php)... Done!
All processes have been done clearly!
Congratulations! You've successfully installed phpLogCon!
A file named 'config.php' is stored in the root directory of phpLogCon. In this file there are the whole information you
have entered before! You can edit it to your needs if you want to.
Move to 'index.php' in root directory to start working with phpLogCon!
Don't forget to delete whole 'install/' directory!
These files could be user for a DoS on your phpLogCon!
After deleting the directory, you can go to index.
[edit] Secure the phpLogCon web environment
# rm -fr /var/www/html/phplogcon/install/
# chmod 644 /var/www/html/phplogcon/config.php
[edit] Login to phpLogCon
Click the index link, enter the username and password you created on the install screen.
Using phpMyAdmin, we can now look at the SystemEvents table in the Syslog database to show entries are actually being logged by counting the number of entries in the table.
We should also be able to now see these rsyslog entries in phpLogCon in the "Show Events" tab. If we don't, we need to turn off the UTC time setting as explained below.
[edit] Modify UTC time setting
As explained here:
http://www.phplogcon.org/Topic3.phtml
We need to modify the config.php to turn off the UTC time setting.
# vi /var/www/html/phplogcon/config.php
modify:
// Use UTC time, indicates if the records are stored in utc time
define('_UTCtime', 1);
to:
// Use UTC time, indicates if the records are stored in utc time
define('_UTCtime', 0);
Now going back to the phplogon webpage, select to view entries for today, we should see our syslog entries.
[edit] Cleaning up and Migrating old configs
After installing the rsyslog RPM, I also had to perform some cleanups and migrations, for example:
# rpm -Uvh /usr/src/redhat/RPMS/i386/rsyslog-1.18.0-1.i386.rpm
Preparing... ########################################### [100%]
1:rsyslog ########################################### [100%]
warning: /etc/logrotate.d/syslog saved as /etc/logrotate.d/syslog.rpmsave
As we're now using /etc/logrotate.d/rsyslog instead of /etc/logrotate.d/syslog, I migrated my settings in /etc/logrotate.d/syslog to /etc/logrotate.d/rsyslog, then:
# rm -f syslog.rpmsave
otherwise logrotate will produce a nightly error about that rpmsave file existing in that directory and being ignored.
Note: because the /etc/init.d/syslog stopstart script no longer exists, and I still have software which looks for specifically that stopstart script, I also created a link from /etc/init.d/syslog to /etc/init.d/rsyslog using:
# cd /etc/init.d
# ln -s rsyslog syslog
[edit] Logging just maillog entries
My purpose for using rsyslog was not to log anything of any level from any server. Doing that would cause the central MySQL database to grow so large and be cluttered with unnecessary information, causing support staff to waste time looking through items which have no relevance to them.
For my requirements, I'm only interested in the maillog entries, so I changed the entry in /etc/rsyslog.conf from:
*.* >servername1.example.com.au,Syslog,syslogwriter,LongEncryPtedPaSsWord
to:
mail.*,*.emerg >servername1.example.com.au,Syslog,syslogwriter,LongEncryPtedPaSsWord
(which logs all maillog and emerg) and restarted rsyslog with:
# service rsyslog restart
[edit] Maintaining the MySQL database
Thankfully, the rsyslog developers are very responsive and appreciative of contributions from people like me. Because of such developers, it's projects like this I love working on and contributing to.
With 6 mail servers running and logging to a central MySQL server, after only 5 days I had well over 119,000 lines logged, which dramatically slows down the web access to the database. I don't have a requirement to keep maillog entries for longer than a week, so allowing the database too keep more than 7 days of entries is unnecessary.
In the rsyslog tarball within the "contrib" directory, exists a MySQL maintenance script called delete_mysql. This script doesn't in fact "delete" MySQL, but prunes the MySQL tables for you, so you should modify this script to add your database name, username, etc and use it via cron to prune and optimise your database periodically.
[edit] Changes in rsyslog v1.19.0 and above
From the rsyslog v1.19.0 release, MySQL is now a plugin so the rsyslog_v1.19.0 spec files (and above) now produce two RPM's, namely:
/usr/src/redhat/RPMS/i386/rsyslog-1.19.0-1.i386.rpm
/usr/src/redhat/RPMS/i386/rsyslog-mysql-1.19.0-1.i386.rpm
which you need to install consecutively. Obviously, the rsyslog-mysql RPM provides just the libraries for rsyslog with mysql enabled.
Once you've installed both RPM's, you need to modify the /etc/rsyslog.conf file to add the:
$ModLoad MySQL
option at the top of the file or above the entry which sends rsyslog information to the MySQL server.
[edit] About the author
I've been using Linux at administrator and hosting (web and dedicated hosting) levels since 1994, from the early days of Yggdrasil, to Slackware, Debian, Red Hat and now to Scientific Linux[1], my preferred distribution.
I've worked for various companies over the years primarily as a Senior UNIX Administrator supporting medium to large UNIX environments, single server or clustered with Linux, Tru64 UNIX, Solaris and HP-UX.
Over the years I've also contributed my time and efforts into many Open Source projects including linuxha.net[2], Squid[3], Webmin[4], Virtualmin GPL[5], Virtualmin Pro[6], Webminstats[7] (and various other Webmin modules), smartmontools[8], clusterssh[9], BlockHosts[10], Brim[11], Kimai[12], DCC[13], OpenWebmail[14], mrepo[15], whichcd[16], rkhunter[17], csync2[18], drbdlinks[19], various perl modules, Scientific Linux[20], etc ... and now rsyslog[21] and phpLogCon[22].
I'm using rsyslog for various servers in a hosting environment which provides Web hosting, Webmail, Domain names and Mail scanning (Virus, Spam) solutions for various clients.
Michael Mansour.
