linux syslogd

syslogd

Apple System Log server (ASL)
ASLManager log file manager

syslogd [-d] [-D] [-m mark_interval]
            [-c log_cutoff] [-l lib_path] [-db_max size]
            [-utmp_ttl time] [-fs_ttl time] [-mps_limit quota]
            [-dup_delay time] [-module_name {0|1}]

The syslogd server receives and processes log messages. Several modules receive input messages through various channels, including UNIX domain sockets associated with the syslog, asl, and kernel printf APIs, and optionally on a UDP socket(514) from network clients.

Comprised of asl APIs, a new syslogd server, syslog commandline utility, and a data store file manager: aslmanager.

Log messages are retained in a 'data store', subject to automatic archival, and input filtering, to simplify the task of locating log messages and to facilitate browsing and searching.

syslogd is started by launchd using /System/Library/LaunchDaemons/com.apple.syslogd.plist

-d debugging output to STDERR.

Following example after
sudo killall -hup syslogd
caused syslogd to read /etc/asl.config.

Displayed by tail -f /var/log/syslogderr
  (if /System/Library/LaunchDaemons/com.apple.syslogd.plist
   included: <key>StandardErrorPath</key><string>/var/log/syslogderr</string> and
                        ProgramArguments included <string>-d<string>

asl_in: accepting connection
asl_in: error connecting socket fd 4: Resource temporarily unavailable
register_session: 14367 PID 21882
action = 6 options = /var/log/DiagnosticMessages exclude_asldb
action = 4 options = 0 80
action = 4 options = 0 80
action = 3 options = (null)
action = ACTION_STORE options = NULL
action = ACTION_STORE options = NULL
action = ACTION_STORE options = NULL
action = ACTION_STORE options = NULL
action = ACTION_STORE options = NULL
action = ACTION_STORE options = NULL
action = ACTION_STORE options = NULL

-D Start as a daemon. syslogd forks and the child process becomes a daemon.
-m mm minutes between --mark-- messages used to indicate that syslogd was alive.
Default: 20 minutes.
If not specified: Disabled i.e. mm is 0.
-c log_cutoff Cutoff filter for log priorities for messages to be retained in the log message data store, between 0 and 7, corresponding to log priorities:
00)LOG_EMERG or ASL_LEVEL_EMERG

07)LOG_DEBUG or ASL_LEVEL_DEBU
as defined in the syslog(3) and asl(3) header files.
Received messages with a priority or level value greater than the cutoff will not be saved in the data store.
Default: allow all.

value may be adjusted while running using the syslog command-line utility. See the syslog(1) manual.

-l plug_path alternate path for loading plug-in modules. Default: /usr/lib/asl.
-db_max bbb limit, in bytes, for files in the data store. Default 25MB Files are closed upon reaching the maximum size, and a new file is opened for subsequent messages.
-utmp_ttl sss time-to-live in seconds for messages used by utmp, wtmp, and lastlog .
Default 31,622,400 seconds (approximately 1 year).
If archival is enabled (see aslmanager ), these messages will be copied to an archive after the regular time-to-live interval, but will persist in the data store until their own expiry time.
-fs_ttl sss time-to-live in seconds for filesystem error messages generated by the kernel.
Default: is 31,622,400 seconds (approximately 1 year).
If archival is enabled, these messages will be copied to an archive after the regular time-to- live interval but will persist in the data store until their own expiry time.
-mps_limit mm messages per second, per-process, excess are ignored.
An error message is logged on behalf of the limited process, stating that its message quota has been exceeded, and that remaining messages for the current second will be discarded.
Default: 500 messages per second per process. of 0 turns off the quota mechanism.
-dup_delay sss seconds to delay for coalescing duplicate message in log files.
If a process logs multiple messages with the same text, syslogd will wait for sss to coalesce duplicates with a message similar to:
May 7 12:34:56: --- last message repeated 17 times ---
Default: 30, 0 disables coalescing.
-asl_in 0|1 receives log messages on the UNIX domain socket associated with the
asl API, disabled using -asl_in 0, Initially enabled.
-bsd_in 0|1 receives log messages on the UNIX domain socket associated with the
syslog API. disabled using -bsd_in 0, normally enabled.
-klog_in 0|1 receives log messages on the UNIX domain socket associated with the
kernel logging API, disabled using -klog_in 0, Initially enabled.
-udp_in 0|1 receives log messages on the UDP socket associated with the
Internet syslog message protocol, normally enabled, but inactive.
UDP sockets are managed by launched, and configured in the syslogd configuration file /System/Library/LaunchDaemons/com.apple.syslogd.plist.reformated ed
Dict {
    Label = com.apple.syslogd
    ProgramArguments = Array { /usr/sbin/syslogd }
    JetsamProperties = Dict { JetsamPriority = -49 JetsamMemoryLimit = 300 }
    EnvironmentVariables = Dict { ASL_DISABLE = 1 }
    POSIXSpawnType = Adaptive
    MachServices = Dict { com.apple.system.logger = Dict { ResetAtClose = true } }
    EnableTransactions = true
    Sockets = Dict { BSDSystemLogger = Dict { SockPathName = /var/run/syslog SockType = dgram SockPathMode = 438 } }
    HopefullyExitsLast = true
    OnDemand = false
}
By default , launchd does not open any sockets for the syslog UDP service, so no sockets are provided to udp_in module. If no sockets are provided, the module remains inactive.
disabled using -udp_in 0.
-bsd_out This module exists for backward compatibility
Use of the syslog(1) and asl(3) search APIs over the use of the log files that are specified in /etc/syslog.conf .
Future versions of Mac OS will move functions that are currently handled by bsd_out to asl_action .

Acts on messages according to the rules in /etc/syslog.conf.

-asl_action 0|1 acts on message according to the rules specified in /etc/asl.conf. See asl.conf

syslogd reinitializes in response to a HUP signal.

MESSAGE EXPIRY AND ARCHIVAL

syslogd invokes aslmanager after it starts up, at midnight and when a data store file reaches the -db_max size limit.
aslmanager utility, which manages files in the ASL data store.
Files are removed or optionally copied to an archival directory after a (default) 2 day time-to-live.

DATA STORE SECURITY

The ASL message store are in /var/log/asl.
The files are given read access corresponding to the read UID and GID specified in the messages as per "ReadUID" and/or "ReadGID" message keys.

Clients may use any value for the facility.
Attemptys by non-UID 0 processes to specify facility prefix of com.apple.system will be logged with facility user

FILES

/etc/syslog.conf bsd_out module configuration file
/etc/asl.conf asl_action module configuration file
/var/run/syslog.pid process ID file
/var/run/log name of the UNIX domain datagram log socket
/dev/klog kernel log device
/var/log/asl data store directory, (notice odd naming:: file from Dec 25 2009 called bb2012.12.31.…)
 /var/log/asl > ls -ltr
   2116 Dec 25  2009 BB.2010.12.31.U0.G80.asl
  43965 Dec 30  2009 BB.2010.12.31.asl
  13622 Jan 15 16:06 BB.2011.01.31.U0.G80.asl
 108000 Jan 30 08:54 BB.2011.01.31.asl
  39003 Feb 27 20:26 BB.2011.02.28.asl
 101250 Mar 30 23:57 BB.2011.03.31.asl
  12217 Apr 25 20:06 BB.2011.04.30.U0.G80.asl
 111523 Apr 29 21:28 BB.2011.04.30.asl
 117686 May 30 20:45 BB.2011.05.31.asl
  85135 Jun 29 19:51 BB.2011.06.30.asl
    794 Jul  4 18:39 2010.07.04.U501.asl
 103964 Jul  4 21:33 2010.07.04.asl
   5273 Jul  5 10:18 2010.07.04.U0.G80.asl
    506 Jul  5 21:23 2010.07.05.U92.asl
  26502 Jul  5 21:51 2010.07.05.U501.asl
 227054 Jul  5 23:15 2010.07.05.asl
  29155 Jul  6 09:23 2010.07.05.U0.G80.asl
    506 Jul  6 09:29 2010.07.06.U92.asl
 206955 Jul  6 23:08 2010.07.06.asl
   4777 Jul  6 23:08 2010.07.06.U501.asl
  29640 Jul  6 23:08 2010.07.06.U0.G80.asl
   2661 Jul  7 21:33 2010.07.07.U501.asl
  92111 Jul  8 08:32 2010.07.07.asl
  24902 Jul  8 08:32 2010.07.07.U0.G80.asl
  18846 Jul  8 20:26 2010.07.08.U501.asl
 214894 Jul  8 22:48 2010.07.08.asl
  18044 Jul  9 09:42 2010.07.08.U0.G80.asl
  72816 Jul  9 22:34 2010.07.09.asl
  33822 Jul  9 22:44 2010.07.09.U501.asl
      8 Jul 10 09:40 SweepStore
  26739 Jul 10 09:40 2010.07.09.U0.G80.asl
  15352 Jul 10 17:58 2010.07.10.U0.G80.asl
  31542 Jul 10 18:29 BB.2011.07.31.asl
   8328 Jul 10 20:21 2010.07.10.U501.asl
     12 Jul 10 20:47 StoreData
  42737 Jul 10 20:47 2010.07.10.asl 
/var/log/asl.archive default archive directory


/etc/asl.conf syslogd reads the asl.conf file at startup and when a HUP signal is received .
aslmanager daemon reads the file when it starts.
See the ASLMANAGER parameter settings section below for details on those Settings which override command-line options and query action rules that trigger actions when messages match

Parameter Settings

begin with = and are of the form:
= parameter_name value ...
debug 0|1 [ffff] option file name debug messages are written to that file.
cutoff n ASL data store cutoff level, 0 to 7 default: 7 allowing any message that matches a "store" action (see QUERY-ACTION RULES below) to be saved.
A lower value will prevent messages with log priority levels numerically greater that the specified cutoff from being saved
mark_time ss time interval for the mark facility. default 0 seconds, which indicates that mark messages are not generated.
dup_delay ss maximum time that the bsd_out module will allow before writing a "last message repeated times" message in a log file specified in /etc/syslog.conf. default: 30 seconds.
utmp_ttl ssss time-to-live by utmp, wtmp, and lastlog subsystems. default: 31622400 seconds (approximately 1 year).
fs_ttl sss time-to-live for filesystem error messages generated by the kernel. default: 31622400 seconds (approximately 1 year).
mps_limit nnn per-process message per second quota. default: 500. 0 disables the quota mechanism.
max_file_size bbbb for individual files in the ASL data store. default: 25,600,000 bytes.

QUERY-ACTION RULES

begin with ? or Q and have the form:
? query action ...

For example:
   ? [= Sender sname] [<= Level error] notify com.apple.label

Query

Queries comprise one or more message matching components, each of which has the form:

      OP KEY VAL

OP is a comparison operator: T true (always matches),
= equal, ! not equal,
> greater than, >= greater than or equal , < less than, <= less than or equal

can be preceded by modifiers:
    C casefold, N numeric comparison, S substring, A prefix, Z suffix

KEY and VAL are from the message .
For example
    [= Sender local1 ] matches any message with key="Sender" and val="local1 ".

The query
   [CA= Color gr]

matches any message with key=Color and val beginning with the letters GR, Gr, gr, or gR (C meaning casefold, A meaning prefix).

The example query :
   [= Sender mail] [N< Level 3]

matches any message from "mail" with a level numerically less than 3 (i.e. more important than notice)
The text values may be used equivalently for the Level key, so the example above may also be written as:
   [= Sender mail] [< Level Error]

text values for levels may be emergency, alert, critical, error, warning, notice, info, or debug

T useful to test for the presence of a particular key.

   [T Flavor] Will match any message that has a "Flavor" key, regardless of its value.

Actions

notify key post a notification with notify_post().
access UID# GID# Sets read access controls for messages that match the associated query pattern. restricting read access to matching messages .
store [fff] save matching messages in:
the ASL data store,
or
in a separate log message file fff.
A separate data store file may be accessed using the syslog command line utility.
A new file will be created if one does not exist with the UID, GID, and mode "uid=UUU", "gid=GGG", and "mode=0MMM".

stayopen will improve performance if a high volume of messages is expected, .

exclude_asldb will cause syslogd to save matching messages in the specificed file, but exclude them from the main ASL data store.

If no matching rules for the ASL data store, then syslogd will save all messages, subject to filtering in accordance with the log cutoff level.

store_directory ddd matching messages to be stored in a log message data store file in an existing directory. files are named yyyy.mm.dd.asl
"exclude_asldb", "uid=UUU", "gid=GGG", and "mode=0MMM"
broadcast constant write the text of matching messages or msg to all terminal windows.
ignore message is ignored.

5/19/13
# configuration file for syslogd and aslmanager  see also /etc/syslog.conf 
#  alert,crit,error,warn,notice,info,debug 
? [T com.apple.message.domain] store_dir /var/log/DiagnosticMessages
? [A= Facility com.apple.performance] store_dir /var/log/performance
? [A= Facility com.apple.eventmonitor] store_dir /var/log/eventmonitor

# authpriv messages are root/admin readable
? [= Facility authpriv] access 0 80 
? [= Facility remoteauth] [<= Level critical] access 0 80

? [= Level emergency] broadcast

# save kernel [PID 0] and launchd [PID 1] messages
? [<= PID 1] store

? [= Facility internal] ignore

# save everything from emergency to notice (i.e. not info, debug)
? [<= Level notice] store

# Flat file configurations formerly in syslog.conf 
? [= Facility install] file /var/log/install.log format=bsd
? [= Facility install] ignore

? [= Sender kernel] file /var/log/system.log mode=0600 gid=80 format=bsd

# emergency - notice  (i.e. not info, debug)
? [<= Level notice] file /var/log/system.log

? [= Facility auth] [<= Level info] file /var/log/system.log 
? [= Facility authpriv] file /var/log/system.log 
? [= Facility mail] file /var/log/mail.log mode=0644 format=bsd
? [= Facility com.apple.alf.logging] file /var/log/appfirewall.log

previously
# redirect com.apple.message.domain to /var/log/DiagnosticMessages
? [T com.apple.message.domain] store_dir /var/log/DiagnosticMessages exclude_asldb

# authpriv messages are root/admin readable
? [= Facility authpriv] access 0 80

# remoteauth critical, alert, and emergency messages are root/admin readable
? [= Facility remoteauth] [<= Level critical] access 0 80

# broadcast emergency messages
? [= Level emergency] broadcast

# save kernel [PID 0] and launchd [PID 1] messages
? [<= PID 1] store



# save everything from emergency to notice
? [<= Level notice] store

# save lpr info level and above
? [<= Level info] [= Facility lpr] store

# save all mail, ftp, local0, and local1 messages
? [= Facility mail] store
? [= Facility ftp] store
? [= Facility local0] store
? [= Facility local1] store 
> grep  -h " selected me" *g|sort|uniq |sed "s/ASL Module \"com.apple.//; s/\" claims selected messages.//"
AccountPolicyHelper, Accounts, CoreDuetAdmissionControl, MessageTracer, authd, authkit.osx.asl, awdd, callhistory.asl.conf, cdscheduler, cloudd, clouddocs, commerce.asl, contacts.ContactsAutocomplete, corecdp.osx.asl, corespotlight, eventmonitor, family.asl, ical, icloud.FindMyDevice, icloud.fmfd, install, iokit.power, mail, mobileme.fmf1, mobileme.fmf1.internal, networking.symptoms, performance, secinitd, securityd, xpc.activity
> grep  -h " shar" /var/log/*.log
    com.apple.authd" sharing output destination "/var/log/asl" with ASL Module "com.apple.asl".
    com.apple.authd" sharing output destination "/var/log/system.log" with ASL Module "com.apple.asl".
    com.apple.authkit.osx.asl" sharing output destination "/var/log/Accounts" with ASL Module "com.apple.Accounts".
    com.apple.corecdp.osx.asl" sharing output destination "/var/log/Accounts" with ASL Module "com.apple.Accounts".
    com.apple.mobileme.fmf1.internal" sharing output destination "/var/log/FindMyFriendsApp" with ASL Module "com.apple.mobileme.fmf1".
    com.apple.mobileme.fmf1.internal" sharing output destination "/var/log/FindMyFriendsApp/FindMyFriendsApp.asl" with ASL Module "com.apple.mobileme.fmf1".


Apple Syslog (file) Manager

aslmanager [-s store_dir] [-a [archive_dir]] [-ttl days] [-size max_size]

Scans through the files in the ASL data store directory, archiveing or deleting files that are older than time-to-live
similar to newsyslog

Files that contain messages with explicit expire times are removed or archived monthly after all thier contents expire.

-a files are copied to /var/log/asl.archive . An alternate directory
-ttl zero allows files to remain in the store with no time limit.

-size maximum size for the entire data store Default: 150MB, zero: unlimited and should not be used as a runaway process could fill all disk space.
Files are removed from oldest to newest, then in lexigraphic sort order by file name. May be copied to an archive directory.


Options may be specified in the aslmanager launch plist file and /etc/asl.conf(not usually ed) which override those on the command line.

store_ttl ddd time-to-live in days for messages in the syslog data store, zero:no time limit. Default 7 days.
max_store_size bbb maximum size for for the ASL data store. Default 150MB
archive e[dir Enables (1) or disables(0) archiving.
Option archive directory, files removed from the ASL data store. default /var/log/asl.archive.
store_path dir The data store path used by aslmanager. Default /var/log/asl.
Ignored by syslogd if in asl.conf.
archive_mode mmmm default 0400, so archive files will only be readable by root.
aslmanager_debug eEnables (1) or disables(0) internal debugging . little interest to most users. Debug messages are sent to syslogd.
As of 4/6/13 Mac OS X 10.8.3

/System/Library/LaunchDaemons/com.apple.ServerPerfLog.aslmanager.plist
reformatted by ed

 Disabled:false
    Label com.apple.ServerPerfLog.aslmanager
    ProgramArguments /usr/sbin/aslmanager -s /var/log/performance      
    StartCalendarInterval: Hour 3; Minute 25; Weekday 6 

hexdump -d /var/log/performance/StoreData
0000000   00000   00000   00000   59398     (eh?)

reformatted by ed
/System/Library/LaunchDaemons > plutil  -p  com.apple.syslogd.plist
  Label => com.apple.syslogd
  JetsamProperties => { JetsamPriority => -49 JetsamMemoryLimit => 300 }
  EnvironmentVariables => { ASL_DISABLE => 1 }
  MachServices => { com.apple.system.logger => { ResetAtClose => 1 } }
  EnableTransactions => 1
  ProgramArguments => [ 0 => /usr/sbin/syslogd ]
  Sockets => { AppleSystemLogger => { SockPathName => /var/run/asl_input SockPathMode => 438 }
               BSDSystemLogger   => { SockPathName => /var/run/syslog SockType => dgram SockPathMode => 438 }
                 }
  HopefullyExitsLast => 1
  OnDemand => 0


See: linux syslogd
syslog, logger(1), syslog(3), syslog.conf,asl.conf(5) (5), asl(3)

rsyslogd


Forward messages

sudo cp /etc/syslog.conf /etc/syslog.`date +%y%m%d%H%M`.conf #backup current version 
sudo vi /etc/syslog.conf
    *.*    @IP.ip.ip.ip[:port]
Use sudo tcpdump -i en1 host 192.168.1.12 and udp port 514 to verify log entries are being forwarded.