Internet Engineering Task Force syslog
Internet Draft: Informational Chris Lonvick
draft-ietf-syslog-syslog-03.txt Cisco Systems
January 3, 2001 Expires: July, 2001
syslog Protocol
draft-ietf-syslog-syslog-03.txt
STATUS OF THIS MEMO
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC 2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This work is a product of the IETF syslog Working Group. More
information about this effort may be found at
http://www.ietf.org/html.charters/syslog-charter.html
Comments about this draft should be directed to the syslog working
group at the mailing list of syslog-sec@employees.org.
When written in uppercase, The key words "MUST", "MUST NOT",
"REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT",
"RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
interpreted as described in RFC 2119.
Copyright Notice
Copyright (C) The Internet Society (2000). All Rights Reserved.
Abstract
This draft describes the observed behavior of the syslog protocol.
This protocol has been used for the transmission of event
notification messages across networks for many years.
Expires July 2001 [Page 1]
Draft syslog January 2001
1 Introduction
Since the beginning, life has relied upon the transmission of
messages. For the self-aware organic unit, these messages can relay
many different things. The messages may signal danger, the presence
of food or the other necessities of life, and many other things. In
many cases, these messages are informative to other units and require
no acknowledgement. As people created processes and machines, this
same principle was applied to societal communications. As an
example, severe weather warnings may be delivered through any number
of channels - a siren blowing, warnings delivered over television and
radio stations, and even through the use of flags on ships. The
expectation is that people hearing or seeing these warnings would
realize their significance and take appropriate action. In most
cases, no responding acknowledgement of receipt of the warning is
required or even desired.
Along these same lines, operating systems, processes and applications
were written to send messages of their own status, or messages to
indicate that certain events had occurred. These event messages
generally had local significance to the machine operators. As the
operating systems, processes and applications grew ever more complex,
systems were devised to categorize and log these diverse messages and
allow the operations staff to more quickly differentiate the
notifications of problems from simple status messages. The syslog
process was one such system that has been widely accepted in many
operating systems. Flexibility was designed into this process so
the operations staff have the ability to configure the destination of
messages sent from the processes running on the device. In one
dimension, the events that were received by the syslog process could
be logged to different files and also displayed on the console of the
device. In another dimension, the syslog process could be configured
to forward the messages across a network to the syslog process on
another machine. The syslog process had to be built network-aware
for some modicum of scalability since it was known that the operators
of multiple systems would not have the time to access each system to
review the messages logged there. The syslog process running on the
remote devices could therefore be configured to either add the
message to a file, or to subsequently forward it to another machine.
In its most simplistic terms, the syslog protocol is an event
notification protocol that allows a machine to send event
notification messages across IP networks to event message collectors
-also known as syslog servers. Since each process, application and
operating system was written somewhat independently, there is little
uniformity to the content of syslog messages. For this reason, no
assumption is made upon the contents of the messages. The protocol
is simply designed to transport these event messages. In all cases,
there is one device that originates the message. The syslog process
on that machine may send the message to a collector. No
acknowledgement of the receipt is made.
Expires July 2001 [Page 2]
Draft syslog January 2001
1.1 Events and Generated Messages
The writers of the operating systems, processes and applications have
had total control over the circumstances that would generate any
message. In some cases, messages are generated to give status.
These can be either of a certain period of time, or at some other
interval such as the invocation or exit of a program. In other
cases, the messages may be generated due to a set of conditions being
met. In that case, either a status message or a message containing
an alarm of some type may be generated.
The contents of a message have also been at the discretion of its
creator. It has been considered to be good form to write the
messages so that they are informative to the person who may be
reading them. It has also been considered good practice to include a
timestamp and some indication of the sending device and the process
that originated it in the messages. However, none of those are
required.
It should be assumed that any process on any device might generate an
event message. This may include processes on machines that do not
have any local storage - e.g. printers, routers, hubs, switches, and
diskless workstations. In that case, it may be imperative that event
messages are transported to a collector so that they may be recorded
and hopefully viewed by an operator.
1.2 Operations of the Message Receivers
It is beyond the scope of this Internet Draft to specify how event
messages should be processed. It was considered that the writers of
the operating systems, processes and applications would quantify
their messages into one of several broad categories. This was so
that the operations staff could be presented with the more important
and time sensitive messages quickly, while also having the ability to
place status or informative messages in a file for later perusal.
2 Transport Layer Protocol
syslog uses the user datagram protocol (UDP) [1] as its underlying
transport layer mechanism. The UDP port that has been assigned to
syslog is 514. It is RECOMMENDED that the source port also be 514
to indicate that the message is from the syslog process of the
sender, but there have been cases seen where valid syslog messages
have come from a sender with a source port other than 514. If the
sender uses a source port other than 514 then it is RECOMMENDED and
has been considered good form that subsequent messages are from a
single consistent port.
Expires July 2001 [Page 3]
Draft syslog January 2001
3 Definitions and Architecture
Any device may generate and send a syslog message. For the purpose
of this document, we'll make the following definitions:
- A machine that can generate a message will be called a "device".
- A machine that can receive the message and relay it to another
machine will be called a "relay".
- A machine that recieves the message and does not relay it to any
other machines will be called a "collector". This has been
commonly known as a "syslog server".
- Any device or relay will be known as the "sender" when it sends a
message.
- Any relay or collector will be known as the "receiver" when it
receives the message.
The archiecture of the devices may be summarized as follows:
- Devices send messages to relays or collectors with no knowledge of
whether it is a collector or relay.
- Devices and relays may be configured to send the same message to
multiple receivers.
Expires July 2001 [Page 4]
Draft syslog January 2001
The following architectures shown in Diagram 1 are valid while the
first one has been known to be the most prevalent.
+------+ +---------+
|Device|---->----|Collector|
+------+ +---------+
+------+ +-----+ +---------+
|Device|---->----|Relay|---->----|Collector|
+------+ +-----+ +---------+
+------+ +-----+ +-----+ +---------+
|Device|-->--|Relay|-->--..-->--|Relay|-->--|Collector|
+------+ +-----+ +-----+ +---------+
+------+ +-----+ +---------+
|Device|---->----|Relay|---->----|Collector|
| |-\ +-----+ +---------+
+------+ \
\ +-----+ +---------+
\-->--|Relay|---->----|Collector|
+-----+ +---------+
+------+ +---------+
|Device|---->----|Collector|
| |-\ +---------+
+------+ \
\ +-----+ +---------+
\-->--|Relay|---->----|Collector|
+-----+ +---------+
+------+ +-----+ +---------+
|Device|---->----|Relay|---->-------|Collector|
| |-\ +-----+ /--| |
+------+ \ / +---------+
\ +-----+ /
\-->--|Relay|-->--/
+-----+
Diagram 1. Some Possible syslog Architectures
Pemutations of the above examples are also acceptable.
As a very general rule, there are usually many devices sending
messages to relatively fewer collectors. This fan-in process allows
an administrator to aggregate messages into relatively few
repositories.
Expires July 2001 [Page 5]
Draft syslog January 2001
4 Packet Format and Contents
The syslog packet has two parts. The first part is the PRI and the
second part is the MSG. The PRI has three, four, five, or six
characters. The MSG will fill the remainder of the syslog packet.
There is no ending delimiter but the total length of the packet MUST
be 1024 bytes or less. There is no minimum length of the MSG
although sending a syslog packet with no contents is worthless and
SHOULD NOT be done. The MSG part of the packet has additional fields
that are described in Section 4.2 below.
4.1 PRI Part of a syslog Packet
The PRI part starts with a leading "<" ('less-than' character),
followed by a number, which is followed by a ">" ('greater-than'
character). This is OPTIONALLY followed by a single space character.
The code set used in this part MUST be seven-bit ASCII in an
eight-bit field as described in RFC 2234 [2]. These are the ASCII
codes as defined in "USA Standard Code for Information Interchange"
[3]. In this, the "<" character is defined as the Augmented
Backus-Naur Form (ABNF) %d60, and the ">" character has ABNF value
%d62. The number is known as the Priority code and represents both
the Facility and Severity as described below. The Priority code
consists of one, two, or three decimal integers (ABNF DIGITS) using
values of %d48 (for "0") through %d57 (for "9"). The OPTIONAL space
character at the end of this field is %d32.
The Facilities and Severities of the messages are numerically coded
with decimal values. The operating system and some of the daemons
and processes have been assigned a Facility parameter. Processes and
applications that have not been assigned a Facility, or that have not
been configured to use one of the "local use" Facilities SHOULD use
the "user" Facility which has the numerical Facility code of decimal
8. All Facilities are shown in the following table along with their
numerical code values.
Expires July 2001 [Page 6]
Draft syslog January 2001
Numerical Facility
Code
0 kernel messages
8 user-level messages
16 mail system
24 system daemons
32 security/authorization messages (note 1)
40 messages generated internally by syslogd
48 line printer subsystem
56 network news subsystem
64 UUCP subsystem
72 clock daemon (note 2)
80 security/authorization messages (note 1)
88 FTP daemon
96 NTP subsystem
104 log audit (note 1)
112 log alert (note 1)
120 clock daemon (note 2)
128 local use 0 (local0)
136 local use 1 (local1)
144 local use 2 (local2)
152 local use 3 (local3)
160 local use 4 (local4)
168 local use 5 (local5)
176 local use 6 (local6)
184 local use 7 (local7)
Table 1. syslog Message Facilities
Note 1 - Various operating systems have been found to utilize
Facilities 32, 80, 112 and 120 for security/authorization,
audit and alert messages which seem to be similar.
Note 2 - Various operating systems have been found to utilize both
Facilities 72 and 120 for clock (cron/at) messages.
Expires July 2001 [Page 7]
Draft syslog January 2001
Each message Priority also has a decimal Severity level indicator.
These are described in the following table along with their numerical
values.
Numerical Severity
Code
0 Emergency: system is unusable
1 Alert: action must be taken immediately
2 Critical: critical conditions
3 Error: error conditions
4 Warning: warning conditions
5 Notice: normal but significant condition
6 Informational: informational messages
7 Debug: debug-level messages
Table 2. syslog Message Severities
The Priority code is calculated by summing the numerical values of
the codes of the Facility and Severity. For example, A kernel
message with a Severity of Emergency would have a Priority code of
decimal 0, while a "local use 4" message with a Severity of Notice
would have a Priority code of decimal 165.
4.2 MSG Part of a syslog Packet
The MSG part of the syslog packet MUST contain visible (printing)
characters. The code set traditionally and most often used has also
been seven-bit ASCII in an eight-bit field like that used in the PRI
part. In this code set, the only allowable characters are the ABNF
VCHAR values (%d21-126) and spaces (SP value %d20). However, no
indication of the code set used within the MSG is required, nor is it
expected. Other code sets MAY be used as long as the characters used
in the MSG are exclusively visible characters and spaces similar to
those described above. The selection of a code set used in the MSG
SHOULD be made with thoughts of the intended receiver. A message
containing characters in a code set that cannot be viewed by a
recipient will yield no information of value to an operator or
administrator looking at it.
Any device receiving a syslog packet may change the format of the
MSG before retransmitting it but only in ways specified in section
4.2.2. While a relay MUST adhere to the rules described in that
section, it is beyond the scope of this memo to say how recieved
packets are handled by a collector. It has generally been observed,
however, that collectors will behave as a relay in the manner in
which they will modify the MSG before displaying or recording them.
Expires July 2001 [Page 8]
Draft syslog January 2001
4.2.1 Original syslog Packets
A device SHOULD compose a syslog packet with the PRI, a space
character and then the MSG. There are no set requirements on the
contents of the MSG as it is originally sent from a device. It is
RECOMMENDED that the MSG have the following fields: TIMESTAMP,
HOSTNAME, TAG and then the CONTENT. The contents of these fields are
described here and their formats are detailed in Section 4.2.3.
If the originally formed message has a TIMESTAMP, then it is the
local time of the device.
If the originally formed message has a HOSTNAME field, then it will
contain the hostname as it knows itself. If it does not have a
hostname, then it will contain its own IP address.
If the originally formed message has a TAG value, then that will be
the name of the program or process that generated the message.
The CONTENT contains the details of the message. This has
traditionally been a freeform message that gives some detailed
information of the event. Even if nothing else is sent, the packet
MUST contain something in the CONTENT field.
4.2.2 Relayed syslog Packets
When a relay receives a packet, it will check for a valid PRI. If
the first character is not a less-than sign, the relay MUST assume
that the packet does not contain a valid PRI. If the 3rd, 4th, or
5th character is not a greater-than sign, the relay again MUST assume
that the PRI is not valid. If the relay has been configured to
forward packets with a Priority value of 14 (User Facility=8 and
Informational Severity=6) then the relay MUST insert a PRI with a
Priority value of 14 as well as a MSG as described below. The
contents of the received packet will be treated as the CONTENT of the
MSG and appended. This new packet will be sent to the next relay or
collector.
If the relay finds a valid PRI then it will check its internal
configuration. Relays MUST be configured to forward syslog packets
on the basis of their Priority value. If the relay finds that it
is configured to forward the received packet, then it MUST check for
a valid TIMESTAMP as the first field in the MSG. If it finds a valid
TIMESTAMP, then it MUST relay the entire packet unchanged. However,
if it does not find a valid TIMESTAMP, then it MUST add a TIMESTAMP.
It SHOULD additionally add a HOSTNAME. These fields are described
here and detailed in Section 4.2.3. The remainder of the packet MUST
be treated as the CONTENT field of the MSG and appended.
The TIMESTAMP will be the current local time of the relay.
Expires July 2001 [Page 9]
Draft syslog January 2001
The HOSTNAME will be the name of the device as it is known by the
relay. If the name cannot be determined, the IP address of the
device will be used. The Domain Name MUST NOT be included in the
HOSTNAME field.
If the relay adds a TIMESTAMP and HOSTNAME at the front of the
MSG then it MUST check that the total length of the packet is still
1024 octets or less. If the packet has been expanded beyond 1024
octets, then the relay MUST truncate the packet to be 1024 octets.
This may cause the loss of vital information from the end of original
packets. It is for this reason that it is RECOMMENDED that the MSG
part of syslog packets contain the fields documented in Section
4.2.1.
4.2.3 Formats of the Fields in the MSG
The TIMESTAMP field is in the format of "MMM DD hh:mm:ss" (without
the quote marks) where:
MMM is the English language month of the year with the first
character in uppercase and the other two characters in lowercase.
The following are the only acceptable values:
Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec
DD is the day of the month. If the day of the month is less than
10, then it MUST be represented as a space and then the number.
hh:mm:ss is the local time. This is represented in a 24-hour
format in the hour field.
The TIMESTAMP field is followed by a space character before the
next field.
The HOSTNAME field MUST contain the hostname of the device as
specified in STD-13 [4]. This field is also followed with a space
character before the next field.
The TAG is a set of ABNF alpha-numeric string that MUST NOT exceed 32
characters. The TAG field will be terminated by any non-alpha-
numeric character. While any non-alpha-numeric character will
terminate this field, it is usually terminated by either the space
character or the left square bracket character ("["). This is
explained more in Section 5.3.
The CONTEXT field may fill the remainder of the MESSAGE.
Expires July 2001 [Page 10]
Draft syslog January 2001
5 Conventions
Although Section 4 of this document specifies all requirements for
the syslog protocol format and contents, certain conventions have
come about over time for the inclusion of additional information
within the syslog message. It must be plainly stated that these
datum are not mandated but may be included for completeness and to
give the recipient some additional clues of their origin and nature.
5.1 Dates and Times
It has been found that some network administrators like to archive
their syslog messages over long periods of time. For the convenience
of these people and for automated message parsers, a more explicit
time stamp has been seen to have been added to some messages. Some
devices will send an original syslog message with a 2 character or 4
character year field immediately after the space following the
TIMESTAMP. Relays will only check the validity of the TIMESTAMP
field and will not check the validity of the remaining fields. This
is not consistent with the original intent of the order and format of
the fields. If implementors wish to contain a more specific date and
time stamp within the message, it should be within the CONTEXT field.
Implementors may wish to utilize the ISO 8601 [5] date and time
formats if they wish to include more explicit date and time
information.
5.2 Domain Name and Address
To readily identify the device that originated the message, it may be
a good practice to include its fully qualified domain name (FQDN) and
its IP address within the CONTEXT field. Traditionally, however,
only the hostname has been included in the HOSTNAME field.
5.3 Originating Process Information
It has also considered to be good practice to include some
information about the process on the device that generated the
message - if that concept exists. This is usually the process name
and process id for robust operating systems. Some of the information
is displayed in the TAG field. Quite often, additional information
is included at the beginning of the CONTEXT field. The format of
"TAG[pid]:" - without the quote marks - is common. The left square
bracket is used to terminate the TAG field in this case. If the
process id is immaterial, it may be left off. In that case, the TAG
MAY be followed by a colon. This would be displayed as "TAG:"
without the quotes.
Expires July 2001 [Page 11]
Draft syslog January 2001
5.4 Examples
As examples, these are valid messages as they may be observed on the
wire between two devices. In the following examples, each message
starts with the less-than character but has been indented, with line
breaks inserted for readability.
Example 1
<37> Oct 11 16:00:15 mymachine su: 'su root' failed for lonvick
on /dev/pts/8
This example shows an authentication error in an attempt to acquire
additional privileges. It also shows the command attempted and the
user attempting it. This was recorded as an original message from
the device called mymachine. A relay receiving this would not make
any changes before sending it along as it contains a properly
formatted TIMESTAME field. The TAG value in this example is "su" and
it has been terminated by the colon which is the first character of
the CONTEXT field.
Example 2
<14>Use the BFG!
While this is a valid message, it has extraordinarily little useful
information. Since it is a user-generated message, it is consistent
that it is not associated with a process. However, it does not
contain a timestamp or any indication of the source of the message.
If this message is stored on paper or disk, subsequent review of the
message will not yield anything of value.
This example is obviously an original message from a device. A relay
MUST make changes to the message as described in Section 4.2.2 before
forwarding it. The resulting message is shown below.
<14> Dec 7 05:16:32 quakesrvr Use the BFG!
In this relayed message, a TIMESTAMP has been added along with a
HOSTNAME. Subsequent relays will not make any further changes to
this message.
Example 3
<160> Aug 24 03:24:00 am CST 1987 mymachine myproc[10]: %% It's
time to make the do-nuts. %% Ingredients: Mix=OK, Jelly=OK #
Devices: Mixer=OK, Jelly_Injector=OK, Frier=OK # Transport:
Conveyer1=OK, Conveyer2=OK # %%
Expires July 2001 [Page 12]
Draft syslog January 2001
This message does have a proper TIMESTAMP field in the message. A
relay will not modify this message before sending it, however, the
HOSTNAME and TAG fields are not consistent with the definitions in
Section 4.2.1. The HOSTNAME field would be construed to be "am" and
the TAG value would be "CST".
It should be noted that the information contained in the CONTEXT of
this example is not telemetry data, nor is it supervisory control or
data acquisition information. Due to the security concerns listed in
Section 6 of this document, information of that nature should
probably not be conveyed across this protocol.
Example 4
<0> 1990 Oct 22 08:22:59 TZ-6 scapegoat.dmz.example.org 10.1.2.3
sched[0]: That's All Folks!
This example has sufficient date and time information as well as a
fully qualified domain name (FQDN) [4] and IP address. It
appropriately lists the process name as well as the process id that
generated the message. The information given after those datum is
limited. Due to the indicated severity of the event, the process may
not have been able to gather anything more informative. It may have
been fortunate to have generated and sent this message at all.
This example is obviously an original message from a device. While
it does have all of the information required, it MUST be modified by
a relay since the first field is not a valid TIMESTAMP as described
in Section 4.2.3. A relay will add a TIMESTAMP and HOSTNAME as
follows.
<0> Oct 22 08:23:00 scapegoat 1990 Oct 22 08:22:59 TZ-6
scapegoat.dmz.example.org 10.1.2.3 sched[0]: That's All Folks!
6 Security Considerations
An odor may be considered to be a message that does not require any
acknowledgement. People tend to avoid bad odors but are drawn to
odors that they associate with good food. The acknowledgement of the
receipt of the odor or scent is not required and indeed it may be the
height of discretion to totally ignore some odors. On the other
hand, it is usually considered good civility to acknowledge the
prowess of the cook merely from the ambiance wafting from the
kitchen. Similarly, various species have been found to utilize odors
to attract mates. One species of moths use this scent to find each
other. However, it has been found that bolas spiders can mimic the
odor of the female moths of this species. This scent will then
attract male moths which will follow it with the expectation of
finding a mate. Instead, when they arrive at the source of the
Expires July 2001 [Page 13]
Draft syslog January 2001
scent, they will be eaten. [6] This is a case of a false message
being sent out with inimical intent.
In its local use, the syslog process places event notification
messages into files on that system. This relies upon the integrity
of the system for the protection of the messages. The subsequent
configuration of the syslog process to use the syslog protocol to
transport the messages to another collector was an extension of the
delivery of event notification messages and exhibits the same trust
of the network. As such there are some concerns about the
applicability of this protocol in situations that require robust
delivery. Along the lines of the analogy, computer event messages
may be sent accidentally, erroneously and even maliciously. At the
time of this writing, however, there have not been any reports of any
machine consuming any other machine.
6.1 Packet Parameters
As was described above, the message length MUST NOT exceed 1024
bytes. Attacks have seen where syslog messages are sent to a
receiver that have message lengths greater than 1024 bytes. In some
older versions of syslog, the receipt of syslog packets that had a
message greater than 1024 bytes caused problems. syslog message
receivers must not malfunction upon the receipt of packets where the
message length is greater than 1024 bytes. If a message receiver
does receive a message whose length is greater than 1024 bytes, it
may log all of, or the source address and some of the contents of the
message, or it may discard the message altogether. Devices MUST NOT
retransmit messages whose received length exceeds 1024 bytes.
Similarly, the receiver must rigidly enforce the correctness of the
message body. syslog collectors must not malfunction if received
messages do not have the less-than and greater-than characters around
a valid Priority value. The receiver may locally log some or all of
a received invalid message or they may discard it totally. Also,
received messages must contain printable text in the message as was
described in section 3. Devices must not malfunction if they receive
a message containing characters other than the characters described
above.
6.2 Message authenticity
The syslog delivery mechanism does not strongly associate the message
with the message sender. Any device can generate any syslog message
and send it to any other machine through the syslog protocol. The
receiver of that packet will not be able to ascertain that the
message was indeed sent from the reported sender, or if the packet
was sent from another device.
Expires July 2001 [Page 14]
Draft syslog January 2001
One possible consequence of this is that a misconfigured machine may
send syslog messages to a collector representing itself as another
machine. The administrative staff may become confused that the
status of the supposed sender of the messages may not be accurately
reflected in the received messages. The administrators may not be
able to readily discern that there are two or more machines
representing themselves as the same machine.
Another consequence happens when the event messages are forwarded.
Unless the identification of the device is contained within the body
of the event message, the source of the message may be lost since it
may only be self-identified by its IP address contained in the IP
header. It should be noted that some cases of embedding the identity
of a device may only have local significance and that may only be
ephemeral. The inclusion of a fully qualified domain name in the
message may give the administrators the best chance of identifying
the source of each message if it can always be associated with an IP
address. However, if the device had obtained an IP address from a
DHCP pool, then that association would not always hold true.
Malicious exploits of this vulnerability have also been noted. An
attacker may transmit syslog messages (either from the machine that
the messages purport from which to be sent or from any other machine)
to a collector. The attacks that have been noted run along these
lines:
- An attacker may perform a Denial of Service attack by filling the
disk of the collector with false messages, or otherwise
overwhelming the collector by sending more messages than it can
receive or process.
- An attacker may hide the true nature of an attack amidst many
other messages. As an example, an attacker may start generating
messages indicating a problem on some machine. This may get the
attention of the system administrators who will spend their time
investigating the alleged problem. During this time, the attacker
may be able to compromise a different machine, or a different
process on the same machine.
- An attacker may generate false syslog messages to give untrue
indications of status or of events. As an example, an attacker
may stop a critical process on a machine, which may generate a
notification of exit. The attacker may subsequently generate a
false notification that the process had been restarted from
another machine already under the control of the attacker. The
system administrators may accept that misinformation and not
verify that the process had indeed been restarted.
Expires July 2001 [Page 15]
Draft syslog January 2001
6.3 Sequenced delivery
As a general rule, the forensics of a network anomaly rely upon
reconstructing the sequence of events. In a perfect world, the
messages would be received on the syslog collector in their order of
generation from the other devices and anyone looking at these records
would have an accurate picture of the sequence of events.
Unfortunately, the syslog process and protocol do not ensure ordered
delivery. This section details some of the problems that may be
encountered from this.
6.3.1 Single Source to a Destination
The syslog records are usually presented (placed in a file, displayed
on the console, etc.) in the order in which they are received. This
is not always in accordance with the sequence in which they were
generated. As they are transported across an IP network, some out of
order receipt should be expected. This may lead to some confusion as
messages may be received that would indicate that a process has
stopped before it was started. This may be somewhat rectified if the
originating process had timestamped or numbered each of the messages
before transmission. To be as effective as possible, both the source
of the message and the syslog collector should both timestamp the
messages. In this, both the sending device as well as the collector
should utilize the same authoritative time source. It should be
remembered, however, that not all devices are capable of receiving
time updates, and not all processes timestamp their messages.
6.3.2 Multiple Sources to a Destination
In syslog, there is no concept of unified event numbering. Single
devices are free to include a sequence number within the event
message but that can hardly be coordinated between multiple devices.
In such cases, multiple devices may report that they are each
sending message number one. Again, this may be rectified somewhat if
the sending devices utilize a timestamp from an authoritative source
in their messages. As has been noted, however, even messages from a
single device to a single collector may be received out of order.
This situation is compounded when there are several devices
configured to send their syslog messages to a single collector.
Messages from one device may be delayed so that messages from another
device are received by the collector even though the messages from
the first were generated before the messages from the second. If
there is no timestamp or sequence number, then the messages may be
presented in the order in which they were received which may give an
inaccurate view of the sequence of actual events.
Expires July 2001 [Page 16]
Draft syslog January 2001
6.3.3 Multiple Sources to Multiple Destinations
The plethora of configuration options available to the network
administrators may further skew the perception of the order of
events. It is possible to configure a group of devices to send the
status messages -or other informative messages- to one collector,
while sending messages of relatively higher importance to another
collector. Additionally, the messages may be sent to different
files on the same collector. If the messages do not contain
timestamps from the source, it may be difficult to order the messages
if they are kept in different places. An administrator may not be
able to determine if a record in one file occurred before or after a
record in a different file. This may be somewhat alleviated by
placing marking messages with a timestamp into all destination files.
If these have coordinated timestamps, then there will be some
indication of the time of receipt of the individual messages.
6.3.4 Replaying
Also, without any sequence indication or timestamp, messages may be
recorded and replayed at a later time. An attacker may record a set
of messages that indicate normal activity of a machine. At a later
time, that attacker may remove that machine from the network and
replay the syslog messages to the collector. The administrators
would find nothing unusual in the received messages and their
receipt would falsely indicate normal activity of the machine.
6.4 Reliable Delivery
As there is no mechanism within either the syslog process or the
protocol to ensure delivery, and since the underlying transport is
UDP, some messages may be lost. They may either be dropped through
network congestion, or they may be maliciously intercepted and
discarded. The consequences of the drop of one or more syslog
messages cannot be determined. If the messages are simple status
updates, then their non-receipt may either not be noticed, or it may
cause an annoyance for the system operators. On the other hand, if
the messages are more critical, then the administrators may not
become aware of a developing and potentially serious problem.
Messages may also be intercepted and discarded by an attacker as a
way to hide unauthorized activities.
6.5 Message Integrity
Besides being discarded, syslog messages may be damaged in transit,
or an attacker may maliciously modify them. In the case of a packet
containing a syslog message being damaged, there are various
mechanisms built into the link layer as well as into the IP [7] and
Expires July 2001 [Page 17]
Draft syslog January 2001
UDP protocols which may detect the damage. A damaged IP packet may
be discarded by an intermediary router [8]. Damage to a UDP packet
may be detected by the receiving UDP module which may silently
discard it. In any case, the original contents of the message will
not be delivered to the collector. Additionally, if an attacker is
positioned between the sender and collector of syslog messages, then
he may be able to intercept and modify those messages in transit to
hide unauthorized activities.
6.6 Message Observation
While there are no strict guidelines pertaining to the event message
format, most syslog messages are generated in human readable form
with the assumption that capable administrators should be able to
read them and understand their meaning. Neither the syslog protocol
nor the syslog application has any mechanism to provide
confidentiality of the messages in transit. In most cases passing
the clear-text messages is a benefit to the operations staff if they
are sniffing the packets off of the wire. The operations staff may
be able to read the messages and associate them with other events
seen from other packets crossing the wire to track down and correct
problems. Unfortunately, an attacker may also be able to observe the
human-readable contents of syslog messages. The attacker may then
use the knowledge gained from those messages to compromise a machine
or do other damage.
6.7 Message Prioritization and Differentiation
While the processes that create the messages may signify the
importance of the events through the use of the message Priority
value, there is no distinct association between this value and the
importance of delivery of the packet. As an example of this consider
an application that generates two event messages. The first is a
normal status message but the second could be an important message
denoting a problem with the process. This second message would have
an appropriately higher Severity value associated with the importance
of that event. If the operators had configured that both of these
messages be transported to a syslog collector then they would, in
turn, be given to UDP for transmission. Under normal conditions, no
distinction would be made between them and they would be transmitted
in their order.
Again, under normal circumstances, the receiver would accept syslog
messages as they are received. If many devices are transmitting
normal status messages, but one is transmitting an important event
message, there is no inherent mechanism within the syslog protocol to
prioritize the important message over the other messages.
Expires July 2001 [Page 18]
Draft syslog January 2001
On a case-by-case basis, device operators may find some way to
associate the different levels with the quality of service
identifiers. As an example, the operators may elect to define some
linkage between syslog messages that have a specific Priority with a
specific value to be used in the IPv4 Precedence field [7], the IPv6
Traffic Class octet [9], or the Differentiated Services field [10].
In the above example, the operators may have the ability to associate
the status message with normal delivery while associating the message
indicating a problem with a high reliability, low latency queue as it
goes through the network. This would have the affect of prioritizing
the essential messages before the normal status messages. Even with
this hop-by-hop prioritization, this queuing mechanism could still
lead to head of line blocking on the transmitting device as well as
buffer starvation on the receiving device if there are many
near-simultaneous messages being sent or received. In this same
line, some implementations of the syslog application may have
mechanisms for the prioritization of the more important messages
within the transmission queue. This behavior is not unique to syslog
but is endemic to all operations that transmit messages serially.
There are security concerns for this behavior. Head of line blocking
of the transmission of important event messages may relegate the
conveyance of important messages behind less important messages. If
the queue is cleared appropriately, this may only add seconds to the
transmission of the important message. On the other hand, if the
queue is not cleared, then important messages may not be transmitted.
Also at the receiving side, if the syslog receiver is suffering from
buffer starvation due to large numbers of messages being received
near-simultaneously, important messages may be dropped
indiscriminately along with other messages. While these are problems
with the devices and their capacities, the protocol security concern
is that there is no prioritization of the relatively more important
messages over the less important messages.
6.8 Misconfiguration
Since there is no control information distributed about any messages
or configurations, it is wholly the responsibility of the network
administrator to ensure that the messages are actually going to the
intended recipient. Cases have been noted where devices were
inadvertently configured to send syslog messages to the wrong
receiver. In many cases, the inadvertent receiver may not be
configured to receive syslog messages and it will probably discard
them. In certain other cases, the receipt of syslog messages has
been known to cause problems for the unintended recipient. [11] If
messages are not going to the intended recipient, then they cannot be
reviewed or processed.
Expires July 2001 [Page 19]
Draft syslog January 2001
6.9 Forwarding Loop
As it is noted in Figure 1, machines may be configured to relay
syslog messages to subsequent relays before reaching a collector. In
one particular case, an administrator found that he had mistakenly
configured two relays to forward messages with certain priority
values to each other. When either of these machines either received
or generated that type of message, it would forward it to the other
relay. That relay would, in turn, forward it back. This cycle did
cause degradation to the intervening network as well as to the
processing availability on the two devices. Network administrators
must take care to not cause such a death spiral.
7 Conclusion and Other Efforts
The syslog protocol may be effectively used to transport event
notification messages across a network. It is highly recommended
that the network operators who choose to use this understand the
characteristics of the protocol and its security implications.
There have been attempts in the past to standardize the format of the
syslog message. The most notable attempt culminated in a BOF at the
Fortieth Internet Engineering Task Force meeting in 1997. This was
the Universal Logging Protocol (ulp) BOF and the minutes of their
meeting is on-line at the IETF Proceedings web site. [12] Many good
thoughts came from that effort and interested implementers may want
to find some of the notes or papers produced from that effort.
Expires July 2001 [Page 20]
Draft syslog January 2001
8 Acknowledgements
The following people provided content feedback during the writing of
this draft:
Jon Knight <J.P.Knight@lboro.ac.uk>
Magosanyi Arpad <mag@bunuel.tii.matav.hu>
Balazs Scheidler <bazsi@balabit.hu>
Jon Callas <jon@counterpane.com>
Eliot Lear <lear@cisco.com>
Petter Reinholdtsen <pere@hungry.com>
Darren Reed <darrenr@reed.wattle.id.au>
Alfonso De Gregorio <dira@speedcom.it>
Eric Allman <eric@sendmail.com>
Eric Allman is the original inventor and author of the syslog
daemon and protocol. The author of this draft and the community at
large would like to express their appreciation for this work and for
the usefulness that it has provided over the years.
A large amount of additional information about this de-facto standard
operating system feature may usually be found in the syslog.conf file
as well as in the man pages for syslog.conf, syslog, syslogd, and
logger, of many Unix and Unix-like devices.
9 Bibliography
[1] Postel, J., "User Datagram Protocol", STD 6, RFC 768,
USC/Information Sciences Institute, August 1980.
[2] Crocker, D., and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", RFC 2234, November, 1997
[3] USA Standard Code for Information Interchange, USASI X3.4-1968.
[4] Mockapetris, P.V., "Domain names - concepts and facilities",
RFC 1034, STD 13, Nov 1987.
[5] Data elements and interchange formats - Information exchange -
Representation of dates and times, International Organization
for Standardization, Reference number ISO 8601 : 1988 (E), 1988
[6] Stowe, M., et al, "Chemical Mimicry: Bolas Spiders Emit
Components of Moth Prey Species Sex Pheromones", Science,
1987
[7] Postel, J., "Internet Protocol", STD 5, RFC 791, USC/Information
Sciences Institute, September 1981.
[8] Baker, F., "Requirements for IP Version 4 Routers", RFC 1812,
June 1995.
Expires July 2001 [Page 21]
Draft syslog January 2001
[9] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6)
Specification", RFC 2460, December 1998.
[10] Nichols, K., S. Blake, F. Baker, D. Black, "Definition of the
Differentiated Services Field (DS Field) in the IPv4 and
IPv6 Headers", RFC 2474, December 1998
[11] Cisco Systems Product Security Incident Response Team (PSIRT),
"Field Notice: Cisco IOS(r) Syslog Crash", January 11, 1999
http://www.cisco.com/warp/public/707/advisory.html
[12] Walker, D., IETF Secretariat, "Proceedings of the Fortieth
Internet Engineering Task Force, Washington, DC, USA, December
8-12, 1997
http://www.ietf.org/proceedings/97dec/index.html
Expires July 2001 [Page 22]
Draft syslog January 2001
A Author's Address
Chris Lonvick
Cisco Systems
12515 Research Blvd.
Austin, TX 78759
USA
+1.512.378.1182
clonvick@cisco.com
B Full Copyright Statement
Copyright (C) The Internet Society (2000). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it or
assist in its implementation may be prepared, copied, published and
distributed, in whole or in part, without restriction of any kind,
provided that the above copyright notice and this paragraph are included
on all such copies and derivative works. However, this document itself
may not be modified in any way, such as by removing the copyright notice
or references to the Internet Society or other Internet organizations,
except as needed for the purpose of developing Internet standards in
which case the procedures for copyrights defined in the Internet
Standards process must be followed, or as required to translate it into
languages other than English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an "AS
IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK
FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT
INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR
FITNESS FOR A PARTICULAR PURPOSE.
Expires July 2001 [Page 23]
