LogAnalyzer Delete Records

From rsyslog wiki
Jump to: navigation, search

Over time, the volume stored in LogAnalyzer data sources can grow considerably. To permit this, options exist to purge old records from data sources. This can either be done interactively or via the command line. The later also works excellent as a cron-script, which can trim your database on a regular schedule.

Please note that record purging is supported for database log sources, only. Entries from text files can not be purged (but that is almost everywhere handled by logrotate and similar means).

Currently, records can only be purged, but are not archived while doing so. This may be added in later releases. Be sure you really want to delete that data, because you can not undo the operation.

Purging via the Web Application

In LogAnalyzer, click on the "Admin Center", there, select sources. That will list all data sources being defined. To the right of each source description there is a small trashcan icon. Click it. Then, a dialog will open that enables to purge the records. Select from the various options.

Please note that, depending on the settings, a purge may be a very lengthy operation. As such, php may time out and the operation not properly be completed. As such, it is recommended to use interactive record purging only for small-volume deletions and the command line tool for all others.

Purging via the Command Line

The command line tool enables record purging in a non-interactive way. It is great for use with cron or the Windows task scheduler. The tool itself as well as a sample Windows batch file and bash script is provided in ./src/cron in the distribution tarball.

The tool takes at least 3 parameters:

  • what to do (the operation)
  • the data source (ID) to operate on
  • filter specifying which records shall be purged

The only currently supported operation is "cleandata", which means all data matching the criteria is deleted.

The second parameter is the numerical data source ID. You can look up this ID in LogAnalyzer's admin center, it is provided by each datasource detail.

The "purge filter" is either the word "all", "olderthan" or "date". All purges everything and needs no further parameter. Use with care. olderthan is followed by time specified in seconds. For example, to delete messages that are older than 1 day, use "olderthan 86400" (there are 86,460 seconds in each day). Similarly, "olderthan 5184000" deletes everything that is older than 60 days. "cleardata" is followed by a month (1-12), day and year (4-digit) and deletes everything that is older than this date.

System Requirements

The command line tool is written in php and as such needs the php command line interpreter. This may eventually not be installed. A typical package name is "php5-cli".

The maintenance script must be present in /var/www directory structure, as it is in the tarball. This is required because of the include file structure.


All samples assume that the data to be deleted is contained in logstream 2.

Delete all data

  php maintenance.php cleardata 2 all

Delete all data older then 1 hour

  php maintenance.php cleardata 2 olderthan 3600

Some typical values are

  • 60 - one minute
  • 3,600 - one hour
  • 86400 - one day
  • 2592000 - 30 days, roughly one month

Delete all data before 2008-11-18

  php maintenance.php cleardata 2 date 11 18 2008