rsyslogd - reliable and extended syslogd

rsyslogd [-4][-6][-A][-d][-f config file]
[-i pid file][-l hostlist][-n][-N level]
[-q][-Q][-s domainlist][-u userlevel][-v][-w][-x]

Support for message local and remote logging. Started on RaspberryPi via /etc/init.d/rsyslog with -c5

K04rsyslog
RSYSLOGD_BIN=/usr/sbin/rsyslogd
RSYSLOGD_OPTIONS="-c5"
RSYSLOGD_PIDFILE=/var/run/rsyslogd.pid

This document severly adapted by Dennis German, assumes an understanding of various syslog utilities.

See the html documentation, man pages only cover basics rsyslog.com/doc

Extensions include free definition of output formats via templates, precise timestamps and writing to databases (tools like phpLogCon can be used to view the log data.)

"Drop in replacment" use a standard syslog.conf, act like the original syslogd and interacts with the standard libraries. Used on raspberryPi.

-c version compatibility mode. Must be first .
-c0 compatible to sysklogd Default
-c3: use the rsyslog v3 native interface,
issues warning messages if the -c3 command line option is not given. order to aid you in this process, rsyslog logs every compatibility-mode config file directive it has generated. So you can simply copy them from your logfile and paste them to the config.
-N 1 CoNfig check. Do NOT run in regular mode. (only level 1 ) sample
-A UDP messages are sent to all targets. may improve reliability, but causes message duplication.
When sending UDP messages, there are potentially multiple paths to the target destination. default, only send to the first target it can successfully send to.
-4 listen to IPv4 addresses only. If neither -4 nor -6 is given, listens to all configured addresses
-6 listen to IPv6 addresses only.
-d Turns on debug mode. Not fork(2) to background, writes on stdout.
‑f config_fileDefault /etc/rsyslog.conf
-i pid_file must be used if multiple instances are run
-l hostlist hostnames to be logged with simple hostname and not the FQDN. Seperated multiple hosts with :
-n No backgrounding. needed if started and controlled by init(8).
-q add hostname if DNS fails during ACL processing when hostnames are resolved to IP addresses. If DNS initally fails the hostname is added as wildcard text, which results in proper, but somewhat slower operation once DNS is up again.
-Q do not resolve hostnames to IPs during ACL processing.
-s domainlist domainnames to be stripped, seperated by :
. Example -s north.de is specified. Host logging resolves to satu.infodrom.north.de no domain would be cut, specify two domains wi: -s north.de:infodrom.north.de.
-u userlevel 1 prevents parsing hostnames and tags inside messages.
2 prevents changing to the root directory. Almost never a good idea
3 both.
-v
rsyslogd 5.8.11, compiled with:
    FEATURE_REGEXP:             Yes
    FEATURE_LARGEFILE:          Yes
    GSSAPI Kerberos 5 support:      Yes
    FEATURE_DEBUG (debug build, slow code): No
    32bit Atomic operations supported:  Yes
    64bit Atomic operations supported:  No
    Runtime Instrumentation (slow code):    No
See http://www.rsyslog.com
-w warnings suppressed when messages are received from machines in no AllowedSender list.
-x Disable DNS for remote messages.

SIGNALS

To send a signal to rsyslogd use: sudo service rsyslog restart sudo kill -HUP $(cat /var/run/rsyslogd.pid) ????
HUP STOP then START:
close open files, TCP and other connections are torn down, queues are not running in disk assisted mode or not set to persist data on shutdown, data is lost.
Start, Read changed configuration files.
Extremely expensive operation and should only be done when actually necessary.
TERM,
INT
Well, it TERMinates!
USR1 toggle debugging if started with -d
CHLD Wait for childs if some were born, because of wall messages.

SECURITY THREATS

Can be used in a denial of service attack. A rogue program(mer) could flood with syslog messages resulting in logs consuming all the space on a filesystem.
Activating logging over the inet domain sockets exposes a system to risks outside of programs in addition to individuals on the local machine.
    Best practices:
  1. Implement kernel firewalling to limit which hosts or networks have access to the 514/UDP socket.
  2. Log to non-root filesystem which, if filled, will not impair the machine.
  3. Configure an ext2 filesystem to limit a percentage to usage by root only and run as a non-root process. (prevents remote logging )
  4. Disabling inet domain sockets

Message replay and spoofing

If remote logging is enabled, messages can be spoofed and replayed. Messages are transmitted in clear-text, the information obtained from the packet is viewable.
An attacker might replay recorded messages or spoof a sender's IP address, which could lead to a wrong perception of system activity. Use GSS-API authentication and encryption to prevent this.

Files

/etc/rsyslog.conf Configuration See filter at rsyslogd.com
/dev/log Unix domain socket to from where local syslog messages are read.
/var/run/rsyslogd.pid contains the process id of rsyslogd.
prefix/lib/rsyslog Default directory for rsyslogd modules. The prefix is specified during compilation (e.g. /usr/local).

ENVIRONMENT

RSYSLOG_DEBUG Controls runtime debug support. contains an option string of:
LogFuncFlow output the logical flow of functions (entering and exiting them)
FileTrace files to trace LogFuncFlow. Defaults to all files.
May be specified multiple times, one file each (e.g.
export RSYSLOG_DEBUG="LogFuncFlow FileTrace=vm.c FileTrace=expr.c"
PrintFuncDB content of the debug function database whenever debug information is output (e.g. abort case)!
PrintAllDebugInfoOnExit (currently not implemented!)
PrintMutexAction as it happens. Useful for finding deadlocks and such.
NoLogTimeStamp Do not prefix log lines with a timestamp (default is to do that).
NoStdOut Do not emit debug messages to stdout. If RSYSLOG_DEBUGLOG is not set, this means no messages will be displayed at all.
Help Display a very short list of commands
RSYSLOG_DEBUGLOG writes debug messages to the specified log file in addition to stdout.
RSYSLOG_MODDIR directory in which loadable modules reside.

See also

./doc subdirectory From: rsyslog.com

Regarding Memory usage:
On raspberrypi top reports VIRT:27,968 ; RES:1536; SHR: 1096

Debug output

9958.925905116:4007d000: rsyslogd 5.8.11 startup, compatibility mode 0, module path '', cwd:/var/log
9958.930543938:4007d000: caller requested object 'net', not found (iRet -3003)
9958.932890848:4007d000: Requested to load module 'lmnet'
9958.935678741:4007d000: loading module '/usr/lib/rsyslog/lmnet.so'
9958.938752623:4007d000: module of type 2 being loaded.
9958.941091533:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.942347484:4007d000: source file conf.c requested reference for module 'lmnet', reference count now 1
9958.943439443:4007d000: rsyslog runtime initialized, version 5.8.11, current users 1
9958.945011382:4007d000: source file syslogd.c requested reference for module 'lmnet', reference count now 2

9958.950204184:4007d000: GenerateLocalHostName uses 'raspberrypi'

9958.952694088:4007d000: omfile: using transactional output interface.

9958.955500980:4007d000: module of type 1 being loaded.
9958.957956886:4007d000: module of type 1 being loaded.
9958.959589824:4007d000: entry point 'beginTransaction' not present in module
9958.960286798:4007d000: entry point 'endTransaction' not present in module
9958.960987771:4007d000: source file omfwd.c requested reference for module 'lmnet', reference count now 3

9958.962230723:4007d000: module of type 1 being loaded.
9958.962463714:4007d000: entry point 'doHUP' not present in module
9958.963362679:4007d000: entry point 'beginTransaction' not present in module
9958.964361641:4007d000: entry point 'endTransaction' not present in module

9958.964630631:4007d000: module of type 1 being loaded.
9958.964846622:4007d000: entry point 'doHUP' not present in module
9958.965789586:4007d000: entry point 'beginTransaction' not present in module
9958.965995578:4007d000: entry point 'endTransaction' not present in module

9958.966842546:4007d000: module of type 1 being loaded.
9958.967063537:4007d000: entry point 'doHUP' not present in module
9958.967417524:4007d000: entry point 'beginTransaction' not present in module
9958.968164495:4007d000: entry point 'endTransaction' not present in module

9958.968405486:4007d000: module of type 1 being loaded.
9958.969365449:4007d000: entry point 'doHUP' not present in module
9958.969570441:4007d000: entry point 'beginTransaction' not present in module
9958.969766433:4007d000: entry point 'endTransaction' not present in module
9958.970248415:4007d000: rfc5424 parser init called
9958.971006386:4007d000: GetParserName addr 0x19ff4
9958.971207378:4007d000: module of type 3 being loaded.
9958.971886352:4007d000: Parser 'rsyslog.rfc5424' added to list of available parsers.
9958.972389333:4007d000: rfc3164 parser init called
9958.973297298:4007d000: module of type 3 being loaded.
9958.973529289:4007d000: Parser 'rsyslog.rfc3164' added to list of available parsers.
9958.973937273:4007d000: Parser 'rsyslog.rfc5424' added to default parser set.
9958.974674245:4007d000: Parser 'rsyslog.rfc3164' added to default parser set.
9958.975514213:4007d000: rsyslog standard file format strgen init called, compiled with version 5.8.11
9958.975736204:4007d000: module of type 4 being loaded.
9958.976600171:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.976844162:4007d000: Strgen 'RSYSLOG_FileFormat' added to list of available strgens.
9958.977224147:4007d000: traditional file format strgen init called, compiled with version 5.8.11
9958.977968118:4007d000: module of type 4 being loaded.
9958.978165111:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.979068076:4007d000: Strgen 'RSYSLOG_TraditionalFileFormat' added to list of available strgens.
9958.979309067:4007d000: rsyslog standard (network) forward format strgen init called, compiled with version 5.8.11
9958.979511059:4007d000: module of type 4 being loaded.
9958.980409025:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.980638016:4007d000: Strgen 'RSYSLOG_ForwardFormat' added to list of available strgens.
9958.981503983:4007d000: rsyslog traditional (network) forward format strgen init called, compiled with version 5.8.11
9958.981718974:4007d000: module of type 4 being loaded.
9958.982101960:4007d000: entry point 'isCompatibleWithFeature' not present in module
9958.982850931:4007d000: Strgen 'RSYSLOG_TraditionalForwardFormat' added to list of available strgens.
9958.984849854:4007d000: Called LogError, msg: WARNING: rsyslogd is running in compatibility mode. Automatically generated config directives may interfer with your rsyslog.conf settings. We suggest upgrading your config and adding -c5 as the first rsyslogd option.
9958.987708744:4007d000: Checking pidfile.
9958.993729513:4007d000: Writing pidfile /var/run/rsyslogd.pid.
Can't open or create /var/run/rsyslogd.pid.
Can't write pid.
^C


#  /etc/rsyslog.conf    Configuration file for rsyslog.
# 1/27/13 DGG
#### MODULES :
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support
#$ModLoad immark  # provides --MARK-- message capability

# provides UDP syslog reception    DGG enabled
$ModLoad imudp
$UDPServerRun 514

# provides TCP syslog reception   DGG enabled
$ModLoad imtcp
$InputTCPServerRun 514

#### RULES :
# First some standard log files.  Log by facility.
auth,authpriv.*         /var/log/auth.log
*.*;auth,authpriv.none         -/var/log/syslog
#cron.*             /var/log/cron.log
daemon.*               -/var/log/daemon.log
kern.*                 -/var/log/kern.log
mail.*;news.*;lpr.*        -/var/log/unused.log
user.*                 -/var/log/user.log
dhclient.info           -/var/dhcpclient.log

# Some "catch-all" log files.
*.=debug;auth,authpriv.none     -/var/log/07_debug.log
*.=crit             -/var/log/02_crit.log
*.err               -/var/log/03_err.log
*.warn              -/var/log/04_warn.log
*.notice            -/var/log/05_notice.log
*.info              -/var/log/06_info.log
*.info              -/var/log/06_info.log2
*.emerg                         -/var/log/01_crit.log
cron,daemon.none;       -/var/log/messages

# Emergencies are sent to everybody logged in.
*.emerg             :omusrmsg:*

# place spool and state files
$WorkDirectory /var/spool/rsyslog

# Include all config files in /etc/rsyslog.d/ DGG: There aren't any
$IncludeConfig /etc/rsyslog.d/*.conf

#### GLOBAL DIRECTIVES :  Set the default permissions for all log files.
$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022

# The named pipe /dev/xconsole is for the `xconsole' utility.
# To use it, invoke `xconsole' with the `-file' option:
#
#    $ xconsole -file /dev/xconsole [...]
#
# NOTE: adjust the list below, or you'll go crazy if you have a reasonably   busy site..
daemon.*;*.=debug;*.=info;*.=notice;*.=warn |/dev/xconsole