INTERNET DRAFT J. Abela Expires: November 17, 1999 T. Debeaupuis <draft-abela-ulm-04.txt> Herv Schauer Consultants 17 May 1999 Universal Format for Logger Messages 1. Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. This document is an Internet-Draft. Internet-Drafts are working doc¡ uments of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute work¡ ing documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference mate¡ rial or to cite them other than as ``work in progress.'' The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. To learn the current status of any Internet-Draft, please check the ``1id-abstracts.txt'' listing contained in the Internet- Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED', 'MAY', and 'OPTIONAL' in this document are to be interpreted as described in RFC 2119 [RFC2119]. 2. Copyright Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this doc¡ ument itself may not be modified in any way, such as by removing the Abela & Debeaupuis [Page 1]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of develop¡ ing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MER¡ CHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 3. Abstract This document presents a format to describe system events for logging purpose. Some of the features presented here are already in use with the common syslog facility, but most of them are lost in the crowd of syslog format freedom. 4. Table of Contents 1. Status of this Memo . . . . . . . . . . . . . . . . . . . . . . . 1 2. Copyright . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 3. Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 4. Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . 2 5. Changes since last version . . . . . . . . . . . . . . . . . . . 2 6. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 3 7. Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 8. Mandatory Fields . . . . . . . . . . . . . . . . . . . . . . . . 4 9. Optional Fields . . . . . . . . . . . . . . . . . . . . . . . . . 7 10. Other more fields . . . . . . . . . . . . . . . . . . . . . . . 10 11. Security Considerations . . . . . . . . . . . . . . . . . . . . 11 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 13. Thanks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 14. Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 11 5. Changes since last version - Correction of the expire date, Abela & Debeaupuis [Page 2]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 - Field names now represent a hierachical tree, - Typo in field order in section "Mandatory fields". 6. Introduction At the beginning, logs were scanned by the administrator after an incident to detect the failure cause. With the increasing concern on computer security, and wide area network manangement, the need for automated log examination, extraction and reporting has grown. The Universal Logger Message (ULM) format presented here is a set of guidelines to formalize the semantics of such messages. In a ULM, each piece of data is marked with a tag to specify its meaning. In the message "gandalf is unavailable", even a human administrator could difficultly determine whether "gandalf" is a host, a user nickname, a network, or an application. But the "file¡ name=gandalf status=unavailable" notation desambiguates the meaning of each piece of data, and allows an automatic processing of the mes¡ sage. 7. Syntax This specification uses the Augmented Backus-Naur Form - ABNF [RFC2234] - notation specified in another document. Just note that "a*b xxx" means "xxx repeated between a and b times", the default values being 0 for a, and infinite for b. logfile = *(logline LF) logline = field *(SP field) field = fieldname "=" fieldvalue fieldname = fieldtag *("." fieldtag) fieldtag = ALPHA *(ALPHAEXT) (Case is not significant for the field names) fieldvalue = *(ALPHAEXT) / string string = QUOTE *(ANY) QUOTE (inside a string, quotes and backslashes must be escaped with a backslash) Abela & Debeaupuis [Page 3]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 ANY = <any ISO-8859-1 character excluding control characters> ALPHAEXT = ALPHA / DIGIT / "." / "-" / "" QUOTE = <the double quote character (ASCII decimal code 34)> ALPHA = <any one of the 52 alphabetic characters (A through Z in upper case and a through z in lower case)> DIGIT = <any one of the 10 numeric characters (0 through 9)> LF = <the line-feed character (ASCII decimal code 10)> SP = <the space character (ASCII decimal code 32)> To use the field names as a hierarchical tree, field name elements can be concatenated with dots in between. Due to the verbosity of ULM, field names should be chosen as short as possible: 4 characters are a maximum for a field name element. Here is a correct syslog message: Jan 20 00:05:03 myhost: 10.0.2.1 tuttle from space.foo-bar.com (10.1.3.5) 3456 Its meaning is however context and reader dependant, and could not be parsed successfully without a precise knowledge of all the type of messages wich could happen. An ULM for this could be: DATE=19970120000503 HOST=itesec PROG=foo-gw LVL=debug PS=3456 DST.IP=10.3.2.1 DST.USR=tuttle SRC.IP=10.3.3.5 SRC.FQDN=space.foo- bar.com 8. Mandatory Fields The following fields should be present in any ULM: DATE, HOST, PROG, LVL in this order. For readability purpose, this requirement is not enforced through the examples of this document. LVL Format: "Emergency" / "Alert" / "Error" / "Warning" / "Auth" / "Security" / "Usage" / "System" / "Important" / "Debug" Abela & Debeaupuis [Page 4]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 The level (LVL) field specifies the importance and category of the ULM. The meaning for the different values are: Emergency A panic condition. It should be broadcast to all users. Alert A condition that should be corrected immediately. Error A system error. This level and the previous ones are reserved for system conditions. Warning A program error. A program has detected an incorrect behaviour of his own. To clarify the differences between these last levels: Absence of a system configuration file is an Error, failed assert is a Warning, and erroneous data given by a user is never anything more than a Debug (except when asked for authentication pasphrase, see below). Auth An authentication happened. Potential senders for such an ULM could be su and login. The STAT field may give more information; if not specified, an authentication failure is assumed, as in: PROG=su LVL=Auth PS=2894 SRC.USR=tuttle DST.USR=root PROG=su LVL=Auth PS=2895 SRC.USR=tuttle DST.USR=root STAT=Success Security A standard protection was raised against what could be an intrusion. A connection denial based on the remote network address falls into this category. The STAT field may give more information; if not specified, a failure is assumed, as in: PROG=tcpwrapper LVL=Security SRC.NAME=evil.foo-bar.com Usage Abela & Debeaupuis [Page 5]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 Normal, standard, authorized day-to-day usage information. If an application has to report delayed information about what is used, it should be reported as debugging information first (for crashproof logging), then summaried into an only usage message. DATE=19970120000503 PROG=www-client LVL=Debug STAT=Start DATE=19970120000551 PROG=www-client LVL=Debug STAT=End DATE=19970120000551 PROG=www-client LVL=Usage DUR=48 System Tangible usage, but not traceable to any user. Automated pro¡ cessus or system batch jobs fall into this category. PROG=sshd LVL=System PS=175 MSG="RSA key generation" DUR=37 Important Information which could become critical, but is not yet. A configuration change may be an important information. Debug The Debug level ULMs are not interesting at all in the normal course of life, they add no information when everything is in order. HOST Format: string The HOST field contains the name of the host which issues the ULM. PROG Format: string The PROG field contains the name of the software component which issues the ULM. If a software component is a member of a software suite, it should be expressed in a hierarchical structure ("suite.component.subcomponent") as in "inn.innfeed". DATE Abela & Debeaupuis [Page 6]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 Format: <YYYY> <MM> <DD> <hh> <mm> <ss> [ '.' <decimals> ] [ ( '+' / '-' ) <hh> <mm> ] The DATE field contains the instantaneous date of the event. If the event lasts a sufficient amount of time, different ULM sould be issued, each marked with its own date. The given date must either be in GMT, or include a timezone information, from a GMT point of view (+0300 means GMT+0300). We recommend to use GMT time and not timezones to avoid discontinu¡ ous logging information during changes between daylight and stan¡ dard times. 9. Optional Fields The following fields could be added in any ULM. Any application read¡ ing log files should understand them: LANG, DUR, PS, ID, SRC.IP, SRC.FQDN, SRC.NAME, SRC.PORT, SRC.USR, SRC.MAIL, DST.*, REL.*, VOL, VOL.SENT, VOL.RCVD, CNT, CNT.SENT, CNT.RCVD, PROG.FILE, PROG.LINE, STAT, TTY, DOC, PROT, CMD, MSG. LANG Format: <ISO-3366 2-letters code> Language used for the text fields. Default is english (EN). DUR Format: <seconds> [ '.' <decimals> ] The DURATION indicates the duration (in seconds) of the event which end is given by the DATE field. This field is mandatory if the ULM announces the end of an event started earlier. PS Format: integer In a multi-tasking environment, this field specifies the process id which issued the ULM. On some systems, this id may not be unique, but it should however be unique on the specified HOST, over the specified DUR, if appropriate. Thus, the ULM announcing the end of a session should specify the duration of the session, and guarantee that all the ULM issued between the beginning of the session and this ULM with the same HOST value and the same PS values concern that session. Abela & Debeaupuis [Page 7]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 ID Format: string The ID field is a system reference to the concerned document. It could be a mail or Usenet news message-id, or an incremented counter. It should not be mistaken with the DOC field, which a user-level name. SRC.IP Format: ipv4 / ipv6 ipv4 = 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT ipv6 = 16HEX The SRC.IP field contains the IP number of the source host. Other SRC.* fields could describe network source addresses in other realms (IPX, X25, ...). The SRC.* fields all contain information about the connected, connecting, or trying to connect host. SRC.FQDN Format: string Fully Qualified Domain Name for the source host. SRC.NAME Format: string Generic name qualifying the source: a host name if fqdn is not available, or a full user name. SRC.PORT Format: 2-byte unsigned integer Port number for TCP, UDP or another protocol. SRC.USR Format: string Abela & Debeaupuis [Page 8]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 User name or user id. SRC.MAIL Format: string E-mail address. DST.IP DST.FQDN DST.NAME DST.PORT DST.USR DST.MAIL All the DST.* fields have the same meaning as the SRC.* fields, except that they qualify the destination. REL.IP REL.FQDN REL.NAME REL.PORT REL.USR REL.MAIL All the REL.* fields have the same meaning as the SRC.* fields, except that they qualify a proxy, or relayer, or some sort of man-in-the-middle. VOL VOL.SENT VOL.RCVD CNT CNT.SENT CNT.RCVD Volume (number of bytes) and count (of articles, files, events) sent, and received, from the source point of view. CNT=1 is always implicit. These fields allow ULM to be merged together, in which case, the VOL and CNT fields may be added together, if the other ones match. PROG.FILE Name of the program source file from which the ULM was generated. This field is useful for warning messages, such as "out of mem¡ ory", "can't fork", or "assert failed". PROG.LINE Line number of the PROG.FILE source file from which the ULM was generated. STAT The STAT field describes the state or status of the designed pro¡ cess. Possible values for this field may include "Failure", "Success", "Start", "End". TTY Format: string Abela & Debeaupuis [Page 9]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 The tty field describes the user's physical connection to the host. DOC Format: string The document (DOC) field is the name of an accessed document, like the path of an ftp file, the name of a newsgroup, or the non-host part of an URL. PROT Format: string The protocol (PROT) field specifies the protocol used. PROG=sendmail VOL=2238 PROT=ESMTP PROG=sshd PROT=RSA LVL=Auth STAT=Success CMD Format: string The command (CMD) field is an issued command, as in: PROG=cron LVL=Usage SRC.USR=news CMD="/local/news/bin/news.daily expireover delayrm" DUR=927 PROG=ftpd LVL=Usage PS=10359 CMD=RETR DOC=x11.tar.gz MSG Format: string The message (MSG) field is the only field which should contain arbitrary data. Any information that doesn't fit any of the other standard fields may be stored here. Using the message field for information fitting in standard field is forbidden. 10. Other more fields It is expected that the set of field names will grow significantly with time. In order to ensure that the set of such names is developed in an orderly, well-specified, and public manner, ULM defines a Abela & Debeaupuis [Page 10]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 registration process which uses the Internet Assigned Numbers Author¡ ity (IANA) as a central registry for such values. Here is a template for the registration of new field names with IANA. To: IANA@isi.edu Subject: Registration of new ULM field name ULM field name: Encoding specifications: Interpretation: (full explanation for the expected use of this field) Submitted by: (person and email address to contact for further information) 11. Security Considerations ULM includes no security functions. However, sites should worry about the vulnerabilites of their logging architecture, especially when networks are used to transport ULM, as these messages may be critical for the security. 12. References Here is a list of references to document referred in this document. [RFC2234] Augmented BNF for Syntax Specifications: ABNF. November 1997. D. Crocker et al. RFC 2234. <URL:ftp://ftp.nordu.net/rfc/rfc2234.txt> [RFC2119] Key works for use in RFCs to Indicate Requirement Levels, S. Bradner, RFC 2119, March 1997. 13. Thanks Authors would like to thank Herve Schauer <Herve.Schauer@hsc.fr>, Shane Kerr <kerr@arin.net> and Dale Wilson <dale@dra.com> for their useful comments. 14. Author's Address Jerome Abela Herve Schauer Consultants 142, rue de Rivoli 75001 Paris Abela & Debeaupuis [Page 11]
INTERNET-DRAFT Universal Logger Messages 17 May 1999 France Phone: +33 141 409 700 Fax: +33 141 409 709 EMail: Jerome.Abela@hsc.fr Tristan Debeaupuis Herve Schauer Consultants 142, rue de Rivoli 75001 Paris France Phone: +33 141 409 700 Fax: +33 141 409 709 EMail: Tristan.Debeaupuis@hsc.fr Comments to this document should be sent to : Tristan.Debeaupuis@hsc.fr We have started an archived mailing list. The archive is at http://www.hsc.fr/gulp or retrievable via email by sending "index gulp" in the body of email to sympa@hsc.fr. To subscribe, please send "sub gulp" to the same address. Abela & Debeaupuis