Format for the Session Initiation Protocol (SIP) Common Log Format (CLF)
draft-ietf-sipclf-format-11

[Ballot comment]
Thanks for addressing my earlier comments. However, the changes you made to the IP Address definitions in section 4.2 got things a bit mashed up. For example, the way you have it written now sounds like the port number is only in decimal notation if used with IPv4 addresses, which I know was not your intention. The other thing I noticed was that the reference to RFC 5952 is to section 6 (i.e., just the port number discussion). You don't say specifically what the numeric representation (hex or decimal) of the address itself is. I presume you are simply taking the rules from sections 4 and 5, which use hex for everything except in the case of special addresses where the last 32 bits are dotted decimal.

I suggest the following replacements (which also get rid of the superfluous MUSTs):

  Destination IP address:port The IP address of the downstream server
        and the port number, separated by a single ':'. IPv4 addresses
        are represented in "dotted decimal" notation as per [RFC1166].
        IPv6 addresses are represented using the hexadecimal notation
        detailed in Section 4 of [RFC5952] (or the special-case mixed
        hexadecimal and decimal notation detailed in Section 5 of
        [RFC5952]) and enclosed in square brackets ('[' and ']').

  Source IP address:port The IP address of the upstream client and the
        port number  over which the SIP message was received, separated
        by a single ':'. IPv4 addresses are represented in "dotted
        decimal" notation as per [RFC1166]. IPv6 addresses are
        represented using the hexadecimal notation detailed in Section 4
        of [RFC5952] (or the special-case mixed hexadecimal and decimal
        notation detailed in Section 5 of [RFC5952]) and enclosed in
        square brackets ('[' and ']').

[Ballot discuss]
I have no objection to seeing this document go forward, and would just like to discuss the registration policies for the SIP CLF Version and SIP CLF Transport Flag registries that are created in Section 9.  Why did the working group select Standards Action and IETF Review, respectively?  What are the reasons for requiring Standards Track for the first, and for requiring both to go through the IETF process (not allowing registrations from other standards-related organizations, for example).

I ask because this makes the registration process very heavyweight, which often discourages registrations and results in use of codepoints without registration.  Even in the IETF, we often don't want to bother cranking out an RFC "just for that", and we sometimes regret having made too-strict choices at the start.  I'd like to understand the thinking here.

Note: I want to have a discussion, and it might well be that these are the right policies for these registries.  Please do not respond to this by changing anything until we've discussed it.  Thanks.

[Ballot discuss]
- following the information versus data model DISCUSS for draft-ietf-sipclf-problem-statement, same remark here.
Just one example of text to be changed.
OLD:

  This file format adheres to the SIP CLF
  data model and provides an effective encoding scheme for all
  mandatory and optional fields that appear in a SIP CLF record.

NEW:

  This file format adheres to the SIP CLF
  information model and provides an effective encoding scheme for all
  mandatory and optional fields that appear in a SIP CLF record.

[Ballot discuss]
I will move to "Yes" once these are cleared up. I was going to leave them in the Comments, but I think they are important enough technical omissions that they need to be fixed before proceeding:


Section 4.2: Are the timestamps encoded as decimal or hexadecimal? Either way, you need to say which.

I take it IPv4 addresses are in dotted *decimal* notation. You need to say that. (You could even refer to RFC 1166 if you were feeling nostalgic.)

Is the port number also decimal encoded? If so, you need to say that.

Section 4.3, Paragraph 5 and Section 4.4, last paragraph of Vendor 00000000 / Tag 00: Neither of these sections say how to encode or replace a CRLF that does appear in a field. They need to explain.

[Ballot comment]
Section 3, last paragraph: Please don't refer to RFC 5735. It is about to become obsolete. Referring only to 5737 will suffice.

Section 4, first two paragraphs: The info in the first paragraph is redundant ("mandatory" and "MUST") and the MUST isn't needed since it's a reference, not a new mandate of this document, and the second paragraph is not clear whether it's referring to SIP CLF records in general or this document. (Obviously it is this document, but it took me a moment to figure that out.) May I suggest instead:

  The Common Log Format for the Session Initiation Protocol
  [I-D.ietf-sipclf-problem-statement] defines a data model to which
  this logging format format adheres, and section 8.1 of that
  document defines all the mandatory data model elements.

  This document defines the format of SIP CLF records as follows:

Section 4, last paragraph: You should probably explicitly say that hexadecimal encoded values are always unsigned. Someone is bound to treat the value as signed, and bad things will happen. (Then again, records longer than 32K will probably cause bad things to happen anyway. But still....)

Section 4.1: I find "MUST be >= 82" silly. How about "are therefore >= 82" instead?

Section 4.2: I find "MUST be separated by a '.'" silly. How about "are separated by a '.'" instead?

Section 4.3: The paragraph break between paragraphs 3 and 4 is incorrect and could actually cause confusion. It took me a bit to realize that the "there is no ambiguity" bit also applied to "-". Combine those two paragraphs.

[Ballot discuss]


A hopefully easy one:

There is a new security consideration here - if I wanted
to obfuscate a field in a CLF record and there is any b64
encoded field, how do I tell if the non-obfuscated value
is present after b64 decoding? I'm ok if you just note
that, e.g. by saying:

  "[I-D.ietf-sipclf-problem-statement] notes that values
  might need to be obfuscated for privacy reasons when
  CLF files are exchanged between domains. If a base64
  encoded string contains the non-obfuscated value then
  that would also need to be obfuscated before base64
  encoding."

IANA has reviewed draft-ietf-sipclf-format-09 and has the following comments:

IANA has questions about the IANA actions requested in this document.

The document says "IANA has created a registry of Version values entitled "SIP CLF Version Values" and "IANA has created a registry of SIP CLF Transport Flag values entitled "SIP CLF Transport Flag Values".  However, IANA can find no location where those registries have been established.

IANA Question -->  Is it possible that the authors intended to have IANA establish these as two new registries upon approval of this document?

If that is the case, then upon approval of this document IANA will complete two actions.

First, a new registry will be created called the "SIP CLF Version Values" registry.  New version numbers are registered via standards action as defined in RFC 5226.  The values for version values will be 'A' through 'Z' (Ox41 through Ox5A) inclusive.  There is a single initial registration in this new registry:

+---------+----------------------+------------------
| Version |      FORMAT        | Reference      |
+---------+----------------------+-----------------+
|  0x41  | Defined in RFC-to-be |  [ RFC-to-be ]  |
+---------+----------------------+-----------------+

Second, another new registry will be created called the "SIP CLF "Transport Flag" registry.  Transport Flags are assigned via the IETF Review procedure as defined in RFC 5226.  There are initial registrations in this new registry as follows:

+-------+--------------------+-----------------+
| Value | Transport Protocol | Reference      |
+-------+--------------------+-----------------+
|  U  |        UDP        |  [ RFC-to-be ]  |
|  T  |        TCP        |  [ RFC-to-be ]  |
|  S  |        SCTP        |  [ RFC-to-be ]  |
+-------+--------------------+-----------------+

IANA understands that, upon approval of this document, theses are the only actions required to be completed by IANA.

Note:  The actions requested in this document will not be completed
until the document has been approved for publication as an RFC.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Subject: Last Call:  (Format for the Session Initiation Protocol (SIP) Common Log Format (CLF)) to Proposed Standard


The IESG has received a request from the SIP Common Log Format WG
(sipclf) to consider the following document:
- 'Format for the Session Initiation Protocol (SIP) Common Log Format
  (CLF)'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2012-12-17. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  The SIPCLF Workgroup has defined a common log format framework for
  Session Initiation Protocol (SIP) servers.  This common log format
  mimics the successful event logging format found in well-known web
  servers like Apache and web proxies like Squid.  This document
  proposes an indexed text encoding format for the SIP Common Log
  Format (CLF) that retains the key advantages of a text-based format,
  while significantly increasing processing performance over a purely
  text-based implementation.  This file format adheres to the SIP CLF
  data model and provides an effective encoding scheme for all
  mandatory and optional fields that appear in a SIP CLF record.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-sipclf-format/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-sipclf-format/ballot/


No IPR declarations have been submitted directly on this I-D.

draft-ietf-sipclf-format-09

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Proposed Standard.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:

The Session Initiation Protocol does not have a common log format, and as a result, each server supports a distinct log format that makes it unnecessarily complex to produce tools to do trend analysis and security detection. This document describes a text based format based on the data model defined in the companion document sipclf-problem-statement. This format uses an indexed text approach to facilitate rapid searching of the log contents while retaining the advantages of a generic text search and filtering tools.

Working Group Summary:

The discussion of indexed-ascii specifically did not result in any contentious issues in the WG discussion. More generally the choice of indexed ascii was more difficult for the WG. That process is described in the write-up for sipclf-problem-statement.

Document Quality:

There were sample implementations of the indexed ascii format written by Peter Musgrave and placed on the sipclf Wiki page.

Additional information from Vijay Gurbani:

At least three implementations are known, although none deployed as far
as I can tell.

One was being implemented by Anders Nygren, as posted on the list [1].
ALU has an early implementation, but it is conditionally compiled into
the code. Illinois Institute of Technology did an implementation based
on Asterix codebase, but I do not think the students gave it back to
Asterix.

[1] http://www.ietf.org/mail-archive/web/sip-clf/current/msg00435.html


Personnel:

Peter Musgrave is the shepherd.

Robert Sparks is the responsible AD.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

This document has been read fully by the shepherd. Nits were checked.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Yes.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

None.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

The WG understands this document. As described in the write-up for sip-clf-problem-statement there were two competing strategies. This document describes the strategy that was adopted. Once discussion focused on the specifics of HOW to log SIP in a text format the work group understood the objective and worked towards it.



(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

Yes. There is one error (a normative reference to the problem statement). This reference will be correct once the AD changes the status of the problem statement to proposed standard.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews.

N/A.

(13) Have all references within this document been identified as either normative or informative?

Yes.

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

There are reference to a parallel document (sipclf-problem-statement). These need to move in parallel to avoid issues with downward normative references.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226).

This document defines two new registries. The new registries are:

IANA has created a registry of Version values entitled "SIP CLF
Version Values".
IANA has created a registry of SIP CLF Transport Flag values entitled "SIP CLF Transport
Flag Values".
It is the opinion of the shepherd that these are reasonable names.



(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

The two new registries are "SIPCLF Version values" and "SIP CLF Transport Flags".

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc.

N/A

(1.a) Who is the Document Shepherd for this document? Has the
Document Shepherd personally reviewed this version of the
document and, in particular, does he or she believe this
version is ready for forwarding to the IESG for publication?
Peter Musgrave is the document shepherd. He has read this version and believes it is ready for publication.

(1.b) Has the document had adequate review both from key WG members
and from key non-WG members? Does the Document Shepherd have
any concerns about the depth or breadth of the reviews that
have been performed?
The document has had adequate review from key members of the WG as well as members active in SIP and IPFIX WGs.

(1.c) Does the Document Shepherd have concerns that the document
needs more review from a particular or broader perspective,
e.g., security, operational complexity, someone familiar with
AAA, internationalization or XML?
No.

(1.d) Does the Document Shepherd have any specific concerns or
issues with this document that the Responsible Area Director
and/or the IESG should be aware of? For example, perhaps he
or she is uncomfortable with certain parts of the document, or
has concerns whether there really is a need for it. In any
event, if the WG has discussed those issues and has indicated
that it still wishes to advance the document, detail those
concerns here. Has an IPR disclosure related to this document
been filed? If so, please include a reference to the
disclosure and summarize the WG discussion and conclusion on
this issue.
There are no specific concerns.
There are no IPR disclosures for this document.

(1.e) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with
others being silent, or does the WG as a whole understand and
agree with it?
The WG as a whole has accepted this document as a reasonable format. Historically the WG considered
two approaches - the text based format described here and a binary format motivated by the desire to
stream to an IPFIX logging entity. This stalled progress in the WG for some time. Eventually the WG
was persuaded that for a file logging format in the SIPCLF charter the text format met the needs and that
the datamodel developed could be used as the basis for specifying a binary log streaming format within the
IPFIX community if there was sufficient interest in doing so.

Since this decision there has not been any contrary statements made during the last several review cycles of this document.

(1.f) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in
separate email messages to the Responsible Area Director. (It
should be in a separate email because this questionnaire is
entered into the ID Tracker.)
No.

(1.g) Has the Document Shepherd personally verified that the
document satisfies all ID nits? (See the Internet-Drafts Checklist
and http://tools.ietf.org/tools/idnits/). Boilerplate checks are
not enough; this check needs to be thorough. Has the document
met all formal review criteria it needs to, such as the MIB
Doctor, media type and URI type reviews?
There is one NIT error (normative reference to the problem statement) that can be resolved when there is an
RFC number for the problem statement.

One reference is reported as unused - even though the reference is on page 21 and appears properly formatted.

(1.h) Has the document split its references into normative and
informative? Are there normative references to documents that
are not ready for advancement or are otherwise in an unclear
state? If such normative references exist, what is the
strategy for their completion? Are there normative references
that are downward references, as described in [RFC3967]? If
so, list these downward references to support the Area
Director in the Last Call procedure for them [RFC3967].
See comment about NIT error above.

(1.i) Has the Document Shepherd verified that the document IANA
consideration section exists and is consistent with the body
of the document? If the document specifies protocol
extensions, are reservations requested in appropriate IANA
registries? Are the IANA registries clearly identified? If
the document creates a new registry, does it define the
proposed initial contents of the registry and an allocation
procedure for future registrations? Does it suggest a
reasonable name for the new registry? See [RFC5226]. If the
document describes an Expert Review process has Shepherd
conferred with the Responsible Area Director so that the IESG
can appoint the needed Expert during the IESG Evaluation?
Yes.

(1.j) Has the Document Shepherd verified that sections of the
document that are written in a formal language, such as XML
code, BNF rules, MIB definitions, etc., validate correctly in
an automated checker?
Not applicable.

(1.k) The IESG approval announcement includes a Document
Announcement Write-Up. Please provide such a Document
Announcement Write-Up? Recent examples can be found in the
"Action" announcements for approved documents. The approval
announcement contains the following sections:

Technical Summary
The document describes a standard indexed ASCII format for the logging of SIP messages
sent and received by a SIP entity. The format adheres to the SIPCLF data model defined by
[insert RFC reference]. The format is fully extensible as it provides a mechanism for
logging any optional element, including vendor specific extensions to allow it to be
tailored to the needs of specific implementations.

Working Group Summary
The WG debated the relative merits of ASCII versus binary encoding of logging
information. This was primarily a debate from people who wanted to use IPFIX
style logging servers versus those who wanted to use text based log tools and
scripts.

Document Quality
There were two implementations of the sipclf logging format. These implementors
helped find areas in earlier drafts where details were incomplete or ambiguous.