Chroot support

From rsyslog wiki

rsyslog shall be able to actively support running in a root jail. That is, it should jail itself (via config statement). This feature, if implemented, will become part of the main rsyslog distribution.

Note that rsyslog v4 already contains some support for setuid, but this needs to be enhanced. Most importantly, the current implementation begins to listen to network ports before privileges are dropped. This was a compromise, because we felt it was better to provide some functionality rather than no until we have time to do it 100% perfect.

1 Specification
- 1.1 Requirements
- 1.2 Problems
2 Implementation
3 References

[edit] Specification

[edit] Requirements

support for privilege separation, setuid, and chroot (both for local and remote use cases)
traditional HUP behavior must be given up
Must not read from network port(s) (and local devices?) until privilege has been dropped and chrooted.
sufficient documentation, especially on security considerations

[edit] Problems

To read the kernel log, root privileges are required (at least on many platforms). So imklog seems to be totally incompatible with dropping privileges. A solution would be to enable plug-ins to be run out of process. While this already is on the todo-list, it has low priority and requires potentially large effort.

A more realistic (effort-wise) solution is to run two instances of rsyslogd on the same system, one as root and loading imklog, then forwarding messages (via TCP syslog) to the other instance, which runs with dropped privileges.

[edit] Implementation

We differentiate between two use-cases, local-syslog and remote-server. The local-syslog case processes the kernel logs but does nothing (except discarding) than forwarding messages to the remote server instance. It will always need to run with root privileges.

In contrast, remote-server will run inside a root jail and with dropped privileges. It can do anything that does not require root privileges after setup time.

Both the local-syslog and the remote-server configurations need to be run on a local machine in order to gain access to both local and remote logs. If no remote receptions is required, it may be sufficient to use a non-chroot local-syslog with root permissions on the system in question.

Configuration statements will exist to request rsyslog to run in a root jail and also to drop privileges (those already present in v4, but with improved processing). I will focus on the remote-server case from now on.

[edit] Code Changes

An important change that must be done to all inputs, the main processing queue and potentially to all action queues is that they must wait for a signal before they begin process messages. The same applies to the outputs, most importantly file writers etc. (reason is that path names must be relative to root and thus file names are different inside and outside of the root jail).

Thus, the startup sequence needs to be changed:

load/instantiate objects (inputs, outputs, whatever)
wait for config file to be read completely
drop privileges, switch to root jail
tell all objects to start processing messages (order inputs to outputs)

New interfaces must be defined to cover this. That breaks plugin backward compatibility, but this is probably acceptable towards the goal of a secure system. The output module interface will require equal modification as the input module interface (don't underestimate that).

We need to check in detail which file descriptors are opened before the chroot call and for what reason. We must identify what do to with those file descriptors. Current thinking (but must be verified) is that we can simply close fd stdout, stdin and stderr, the config fds can be closed after the config file is done (already how it happens) and the rest should probably only be opened as part of the inputs (TLS certificate files must be carefully checked!). So there should not be many changes resulting from this requirement. However, we do not know what the runtime libraries may keep open (like dlopen and the various dynamically loaded libraries, e.g. for MySQL). We have no control over them, so this probably is mostly something we need to document, but not address in code.

As today in v4, when privilege drop or root jail is configured, HUP behavior must be changed to non-restart type of HUP.

It needs to be discussed how rsyslog needs to handle the situation if the config instructs it to chroot, but does not contain a privilege drop instruction. Terminating rsyslog may be an option, but not one we can take by default (because the syslogd is too important to be disabled by a simple config error, that is against the "run under all circumstances" philosophy of rsyslog). So the issue what to do exists in any case. I currently think that the correct handling is to

emit a warning log message (regular message)
do the chroot, but do not drop privileges (as we do not have credentials)

That would leave us wide open to attacks, but may be a bit more secure than not chrooting at all - plus it probably is an operational requirement as otherwise given path names are incorrect.

For the same reason, it must be specified what to do if the the chroot call cannot be executed. This is a very bad case, as it invalidates many of the actions. One solution would be to abort the run (bad), load the emergency config (probably good, but far from perfect) or make each module handle the situation gracefully (probably much effort). I'd approach this from an effort-based side, that is would do the better solutions only if the effort required is low. In extreme cases, we can abort here (I think this failure case is quite uncommon). However, that would lead the door open to an attack, where an attacker could remove the jail root directory and with this prevent rsyslog from starting (requires an already compromised machine).

A solution my prossibly be the introduction of a new fatal_abort() handler, that halts rsyslog, but emits a wall message every 5 minutes (or so) telling every user that the syslogd is not operational. This may at least help getting some attention to the situation (which is better than silently dying).

[edit] User Contributions

Some user contributions are required for testing this functionality. Most importantly, we need to see how a real person needs to set up rsyslog on a real machine so that we can fix all the subtleties that we have probably overlooked (our own testing would not bring these up). So it is absolutely vital to have some external collaborators for this effort.

[edit] Testing

Testing the jail functionality is problematic, as breaking out of it is not desired, so any tests that actually permit us to break out of it would be vulnerabilities, which we obviously would fix if we know about them. So far, I can envision only a single test, with a deliberately misconfigured rsyslogd. In that, we could build an (unescaped) filename from message content and then emit a message that leads to a file name outside of the root. This can be used as a base verification that the jail works.

[edit] Documentation

Some documentation is useful for educating end users. Defining the issues in depth would fill a book, so we need to strip it down to the essentials:

provide overview of security measures
provide overview of security threads addressed
provide overview of remaining threads (as far as can be seen - the ones you do not envision are the worst)