Skip to main content

Compacted-DNS (C-DNS): A Format for DNS Packet Capture
draft-ietf-dnsop-dns-capture-format-10

Revision differences

Document history

Date Rev. By Action
2019-08-19
10 Gunter Van de Velde Closed request for Telechat review by OPSDIR with state 'Overtaken by Events'
2019-08-19
10 Gunter Van de Velde Assignment of request for Telechat review by OPSDIR to Joel Jaeggli was marked no-response
2019-08-19
10 (System) RFC Editor state changed to AUTH48-DONE from AUTH48
2019-06-10
10 (System) RFC Editor state changed to AUTH48 from RFC-EDITOR
2019-05-28
10 (System) RFC Editor state changed to RFC-EDITOR from EDIT
2019-03-25
10 (System) RFC Editor state changed to EDIT from MISSREF
2019-01-10
10 Tero Kivinen Closed request for Last Call review by SECDIR with state 'No Response'
2019-01-08
10 (System) IANA Action state changed to RFC-Ed-Ack from Waiting on RFC Editor
2019-01-08
10 (System) IANA Action state changed to Waiting on RFC Editor from In Progress
2019-01-08
10 (System) IANA Action state changed to In Progress from Waiting on Authors
2019-01-07
10 (System) IANA Action state changed to Waiting on Authors from In Progress
2019-01-07
10 (System) IANA Action state changed to In Progress from Waiting on Authors
2019-01-04
10 (System) RFC Editor state changed to MISSREF
2019-01-04
10 (System) IESG state changed to RFC Ed Queue from Approved-announcement sent
2019-01-04
10 (System) Announcement was received by RFC Editor
2019-01-04
10 (System) IANA Action state changed to Waiting on Authors from In Progress
2019-01-04
10 (System) IANA Action state changed to In Progress
2019-01-04
10 Cindy Morgan IESG state changed to Approved-announcement sent from IESG Evaluation::AD Followup
2019-01-04
10 Cindy Morgan IESG has approved the document
2019-01-04
10 Cindy Morgan Closed "Approve" ballot
2019-01-04
10 Cindy Morgan Ballot approval text was generated
2019-01-03
10 Sabrina Tanamal IANA Review state changed to IANA OK - Actions Needed from Version Changed - Review Needed
2018-12-21
10 Sabrina Tanamal
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has completed its review of draft-ietf-dnsop-dns-capture-format-10. If any part of this review is inaccurate, please let …
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has completed its review of draft-ietf-dnsop-dns-capture-format-10. If any part of this review is inaccurate, please let us know.

The IANA Functions Operator has a question about one of the actions requested in the IANA Considerations section of this document.

The IANA Functions Operator understands that, upon approval of this document, there are four actions which we must complete.

First, a new registry page is to be created at:

https://www.iana.org/protocols

The new registry page will be titled "C-DNS DNS Capture Format"

Second, on the new registry page created in the first action above, a new registry will be created called C-DNS Transports. The registry will be managed via Expert Review as defined in RFC 8126. There are initial registrations in the new registry as follows:

+------------+------------+--------------+
| Identifier | Name | Reference |
+------------+------------+--------------+
| 0 | UDP | [[this RFC]] |
| 1 | TCP | [[this RFC]] |
| 2 | TLS | [[this RFC]] |
| 3 | DTLS | [[this RFC]] |
| 4 | DoH | [[this RFC]] |
| 5-15 | Unassigned | |
+------------+------------+--------------+

Third, also on the new registry page created in the first action above, a new registry will be created called C-DNS Storage Flags. The registry will be managed via Expert Review as defined in RFC 8126. There are initial registrations in the new registry as follows:

+------+------------------+-----------------------------+-------------+
| Bit | Name | Description | Reference |
+------+------------------+-----------------------------+-------------+
| 0 | anonymised-data | The data has been |[ RFC-to-be ]|
| | | anonymised. | |
| 1 | sampled-data | The data is sampled data. |[ RFC-to-be ]|
| 2 | normalized-names | Names in the data have been |[ RFC-to-be ]|
| | | normalized. | |
| 3-63 | Unassigned | | |
+------+------------------+-----------------------------+-------------+

Fourth, also on the new registry page created in the first action above, a new registry will be created called C-DNS Address Event Types. The registry will be managed via Expert Review as defined in RFC 8126. There are initial registrations in the new registry as follows:

+------------+----------------------+-------------------+-------------+
| Identifier | Event Type | ae-code contents | Reference |
+------------+----------------------+-------------------+-------------+
| 0 | TCP reset | None |[ RFC-to-be ]|
| 1 | ICMP time exceeded | ICMP code |[ RFC-to-be ]|
| 2 | ICMP destination | ICMP code |[ RFC-to-be ]|
| | unreachable | [icmpcodes] | |
| 3 | ICMPv6 time exceeded | ICMPv6 code |[ RFC-to-be ]|
| | | [icmp6codes] | |
| 4 | ICMPv6 destination | ICMPv6 code |[ RFC-to-be ]|
| | unreachable | [icmp6codes] | |
| 5 | ICMPv6 packet too | ICMPv6 code |[ RFC-to-be ]|
| | big | [icmp6codes] | |
| >5 | Unassigned | | |
+------------+----------------------+-------------------+-------------+

IANA Question --> What is the largest value that an Identifier in the C-DNS Address Event Types registry can take?

The IANA Functions Operator understands that these are the only actions required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is meant only to confirm the list of actions that will be performed.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist
2018-12-19
10 Magnus Westerlund Closed request for Last Call review by TSVART with state 'Overtaken by Events'
2018-12-12
10 Sara Dickinson New version available: draft-ietf-dnsop-dns-capture-format-10.txt
2018-12-12
10 (System) New version approved
2018-12-12
10 (System) Request for posting confirmation emailed to previous authors: Sara Dickinson , John Dickinson , Jim Hague , John Bond , Terry Manderson
2018-12-12
10 Sara Dickinson Uploaded new revision
2018-12-02
09 Benjamin Kaduk
[Ballot comment]
Thank you for addressing (or being in the process of addressing) my Discuss points!
Adding IANA registries for the bitmaps will address that …
[Ballot comment]
Thank you for addressing (or being in the process of addressing) my Discuss points!
Adding IANA registries for the bitmaps will address that concern, so I am proactively
clearing my position on the basis that those registry creations are in the works.

[Original COMMENT section preserved below]

Section 2

Please consider using the RFC 8174 version of the BCP 14 boilerplate.

Section 3

  Because of these considerations, a major factor in the design of the
  format is minimal storage size of the capture files.

maybe "storage and transmission"?

Section 6

In Figure 2, the Query name is marked as "(q)" (only present if there is a
query), but the running text in Section 4 (bullet 1) says that the Question
section from the response can be used as an identifying QNAME if there is a
response with no corresponding query.  Am I misexpanding QNAME here, or is
there a disagreement between these two parts of the text?  In particular, I
do not see a part of Figure 2 that would correspond to a Question section
in the response, given the various "(q)"/"(r)" markings.

Section 6.2.2

  Messages with OPCODES known to the recording application but not
  listed in the Storage Parameters are discarded (regardless of whether
  they are malformed or not).

(Do we need to say anything that the "discarded" is only w.r.t. the capture
process, and not meant to imply that DNS queries would not get a normal
response?)

Section 6.2.4

Please consider using IPv6 examples, per
https://www.iab.org/2016/11/07/iab-statement-on-ipv6/ .

Section 7.2

  o  The column T gives the CBOR data type of the item.

      *  U - Unsigned integer

      *  I - Signed integer

This is venturing a bit far from my normal area of expertise, but my
understanding is that CBOR native major types are only provided for
unsigned integer and negative integer, with "signed integer" being an
abstraction at a slightly higher layer that needs to be managed in the
application.  Do we need to add any clarifying text here or will the
meaning be clear to the reader?

Section 7.4

Should probably forward-reference section 8 for the format version numbers'
semantics.

Section 7.4.1.1

We should we reference the IANA registries by name for any of these fields
(e.g., opcodes, rr-types, etc.).  (Also in Section 7.5.3.1, etc.)

Are the storage flags going to be allocated in sequence by updating
standards-track documents, or some other mechanism?  (Is a registry
necessary?)

For the various address prefix fields, do we need to specify that the full
addresses are stored when the corresponding prefix field is absent?

Section 7.4.1.1.1

Am I parsing the "query-response-hints" text correctly to say that a bit is
set in the bitmap if the corresponding field is recorded (if present) by
the collecting implementation?  The causality of "if the field is omitted
the bit is unset" goes in a direction that is not what I expected.
(Similarly for the other fields in this table.)

Section 7.4.2

Do we need a reference for "promiscuous mode"?

Just to check: in "server-addresses", I just infer the IP version from the
length of the byte string?

Do we need to say more about where the vlan-ids identifiers are taken from?

Is the "generator-id" string intended to only be human readable?  Only
within a specific (administrative) context?

Section 7.5.1

Does "earliest-time" include leap seconds?

Section 7.5.3

The "ip-address" description seems to imply that very short ipv6 prefix
lengths could cause confusion as to the address type being indicated (e.g.,
setting to 32 when no ipv4 prefix length is set, or setting to the same
value as the ipv4 prefix length).  Do we need to restrict the ipv6 prefix
lengths to being 33 or larger?

Are the "name-rdata" contents in wire format or presentation format?

Section 7.5.3.2

What's the allocation policy/procedure for the remaining
qr-transport-flags transport values?  For additional bits in any/all of the
flags fields listed here?

Something of a side note, what's the mnemonic for the "sig" in
"qr-sig-flags"?  That is, what is it a signature of or over (it doesn't
seem like it's a cryptographic signature, which may be what is confusing
me)?

For "query-rcode"/"response-rcode", should there be a reference for "OPT",
and/or for any of the EDNS stuff in here?  (The Terminology section only
mentions using the naming from RFC 1035, that I can see.)

The "mm-transport-flags" here bear a striking resemblance to the
"qr-transport-flags" from Section 7.5.3.2; should there be a shared
registry for their contents?  (I guess the TransportFlags CDDL to some
extent serves this function.)

Section 7.7

How is the value of the "ae-code" determined?

Appendix A

We could perhaps apply some constraints on (e.g.) the address-prefex length
fields to be .le the relevant lengths.

Appendix C.6

                                          Using a strong compression,
  block sizes over 10,000 query/response pairs would seem to offer
  limited improvements.

nit: Using a strong compression scheme
2018-12-02
09 Benjamin Kaduk [Ballot Position Update] Position for Benjamin Kaduk has been changed to No Objection from Discuss
2018-11-30
09 Alexey Melnikov [Ballot comment]
Thank you for addressing my DISCUSS and comments.
2018-11-30
09 Alexey Melnikov [Ballot Position Update] Position for Alexey Melnikov has been changed to No Objection from Discuss
2018-11-30
09 (System) Sub state has been changed to AD Followup from Revised ID Needed
2018-11-30
09 (System) IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed
2018-11-30
09 Sara Dickinson New version available: draft-ietf-dnsop-dns-capture-format-09.txt
2018-11-30
09 (System) New version approved
2018-11-30
09 (System) Request for posting confirmation emailed to previous authors: Sara Dickinson , John Dickinson , Jim Hague , John Bond , Terry Manderson
2018-11-30
09 Sara Dickinson Uploaded new revision
2018-11-21
08 Cindy Morgan IESG state changed to IESG Evaluation::Revised I-D Needed from IESG Evaluation
2018-11-21
08 Suresh Krishnan
[Ballot comment]
* Section 7.4.1.1.

Looks like you can limit the {client,server}-address-prefix-{ipv4,ipv6} fields to one byte to restrict the range. e.g.

client-address-prefix-ipv6 => uint .size …
[Ballot comment]
* Section 7.4.1.1.

Looks like you can limit the {client,server}-address-prefix-{ipv4,ipv6} fields to one byte to restrict the range. e.g.

client-address-prefix-ipv6 => uint .size 1

Similar restrictions can be used for port (2) and TTL/hop limit (1) fields.
2018-11-21
08 Suresh Krishnan [Ballot Position Update] New position, No Objection, has been recorded for Suresh Krishnan
2018-11-21
08 Ignas Bagdonas [Ballot Position Update] New position, No Objection, has been recorded for Ignas Bagdonas
2018-11-21
08 Martin Vigoureux
[Ballot comment]
few nits:

s/types are are omitted/types are omitted/
s/type is are omitted/type is omitted/
s/common collection and and storage parameters/common collection and storage …
[Ballot comment]
few nits:

s/types are are omitted/types are omitted/
s/type is are omitted/type is omitted/
s/common collection and and storage parameters/common collection and storage parameters/
2018-11-21
08 Martin Vigoureux [Ballot Position Update] New position, No Objection, has been recorded for Martin Vigoureux
2018-11-20
08 Terry Manderson [Ballot comment]
I'm a co-author and financial sponsor of this work - recusing.
2018-11-20
08 Terry Manderson [Ballot Position Update] New position, Recuse, has been recorded for Terry Manderson
2018-11-20
08 Ben Campbell
[Ballot comment]
I support Benjamin's DISCUSS point about privacy considerations.

§2: Is there a reason not to use the boilerplate from RFC 8174? There …
[Ballot comment]
I support Benjamin's DISCUSS point about privacy considerations.

§2: Is there a reason not to use the boilerplate from RFC 8174? There are a number of lower case instances of normative keywords. These are ambiguous under the RFC 2119 boilerplate.
2018-11-20
08 Ben Campbell [Ballot Position Update] New position, No Objection, has been recorded for Ben Campbell
2018-11-20
08 Spencer Dawkins [Ballot Position Update] New position, No Objection, has been recorded for Spencer Dawkins
2018-11-20
08 Adam Roach
[Ballot comment]


I support Benjamin's DISCUSS regarding a treatment of the privacy issues related
to this capture format.

---------------------------------------------------------------------------

id-nits reports:

  ** There are …
[Ballot comment]


I support Benjamin's DISCUSS regarding a treatment of the privacy issues related
to this capture format.

---------------------------------------------------------------------------

id-nits reports:

  ** There are 11 instances of too long lines in the document, the longest
    one being 9 characters in excess of 72.

  -- The document has examples using IPv4 documentation addresses according
    to RFC6890, but does not use any IPv6 documentation addresses.  Maybe
    there should be IPv6 examples, too?

(See https://www.iab.org/2016/11/07/iab-statement-on-ipv6/ for more information
about the second issue)

---------------------------------------------------------------------------

§5:

>  o  CBOR is an IETF standard and familiar to IETF participants.  It is

While CBOR is standards-track, it's nowhere near standard yet. Suggest:
"...is an IETF specification..." (See BCP 9)

---------------------------------------------------------------------------

§9.1:

>  DNS style name compression is used on the individual names within the

Nit: "DNS-style"

---------------------------------------------------------------------------

Appendix A:

>  file-type-id  : tstr .regexp "C-DNS",

I'm far from a CDDL expert, but I just read through that specification, and it
seems to me that this is a bit overwrought. I think you can accomplish the same
with the much simpler production:

    file-type-id  : "C-DNS",

Similarly:

>  major-format-version => uint .eq 1,
>  minor-format-version => uint .eq 0,

would seem to mean the same as the simpler:

>  major-format-version => 1,
>  minor-format-version => 0,

---------------------------------------------------------------------------

Appendix B:

>  The next name added is bar.com.  This is matched against example.com.

bar.com is allocated to a private individual who has already had to contend with
a lot of unwanted traffic (see https://www.bar.com/ for details). We should
consider not making things worse for them. Please use an RFC 2606 address
instead.
2018-11-20
08 Adam Roach [Ballot Position Update] New position, No Objection, has been recorded for Adam Roach
2018-11-20
08 Eric Rescorla
[Ballot comment]
Rich version of this review at:
https://mozphab-ietf.devsvcdev.mozaws.net/D4670


I support Ben's DISCUSS.

Am I understanding the design correctly in that you can't have
literals …
[Ballot comment]
Rich version of this review at:
https://mozphab-ietf.devsvcdev.mozaws.net/D4670


I support Ben's DISCUSS.

Am I understanding the design correctly in that you can't have
literals in the individual per-query values, but instead references to
the block tables, so you can't stream inside a block?

IMPORTANT
S 7.5.1.
>      | earliest-time    | O | A | A timestamp (2 unsigned integers,      |
>      |                  |  |  | "Timestamp") for the earliest record  |
>      |                  |  |  | in the "Block" item. The first integer |
>      |                  |  |  | is the number of seconds since the    |
>      |                  |  |  | Posix epoch ("time_t"). The second    |
>      |                  |  |  | integer is the number of ticks since  |

I don't know what a "tick" is, so you need some definitionf or this.

COMMENTS
S 7.2.

>      o  The column O marks whether items in a map are optional.

>        *  O - Optional.  The item may be omitted.

>        *  M - Mandatory.  The item must be present.

FWIW, I think you might be happier with a mandatory "X" and optional
nothing, but that's up to you.


S 7.4.1.1.1.

>      +------------------+---+---+----------------------------------------+
>      | Field            | O | T | Description                            |
>      +------------------+---+---+----------------------------------------+
>      | query-response  | M | U | Hints indicating which "QueryResponse" |
>      | -hints          |  |  | fields are omitted, see section        |

Nit: generally, you would say "indicating which fields are included"
if 1 means included.


S 7.5.3.
>      +---------------------+---+---+-------------------------------------+

>  7.5.3.  "BlockTables"

>      Arrays containing data referenced by individual "QueryResponse" or
>      "MalformedMessage" items in this "Block".  Each element is an array

This is not a sentence.


S 7.5.3.
>      | qrr              | O | A | Array of type "Question". Each entry  |
>      |                  |  |  | is the contents of a single question, |
>      |                  |  |  | where a question is the second or    |
>      |                  |  |  | subsequent question in a query. See  |
>      |                  |  |  | Section 7.5.3.3.                      |
>      |                  |  |  |                                      |

So if I understand this correctly:

QRR is a map of integers to question
QLIST is a map of integers to question lists, with each question list
consisting of a set of indexes int o QRR?



S 7.5.3.2.

>      +--------------------+---+---+--------------------------------------+
>      | Field              | O | T | Description                          |
>      +--------------------+---+---+--------------------------------------+
>      | server-address    | O | U | The index in the item in the "ip-    |
>      | -index            |  |  | address" array of the server IP      |

So am I understanding correctly that you can't put the server address
literally in here. It has to be in the block tables?


S 7.5.3.2.
>      +--------------------+---+---+--------------------------------------+
>      | server-address    | O | U | The index in the item in the "ip-    |
>      | -index            |  |  | address" array of the server IP      |
>      |                    |  |  | address. See Section 7.5.3.          |
>      |                    |  |  |                                      |
>      | server-port        | O | U | The server port.                    |

Isn't the server port generally constant? It seems like you might save
some bits by having this attached to the server and then indixed
abvoe.



S 7.5.3.2.
>      |                    |  |  | used to service the query.          |
>      |                    |  |  | Bit 0. IP version. 0 if IPv4, 1 if  |
>      |                    |  |  | IPv6                                |
>      |                    |  |  | Bit 1-4. Transport. 4 bit unsigned  |
>      |                    |  |  | value where 0 = UDP, 1 = TCP, 2 =    |
>      |                    |  |  | TLS, 3 = DTLS. Values 4-15 are      |

You might want to specify DoH


S 7.5.3.5.
>      |                    |  |  | Bit 0. IP version. 0 if IPv4, 1 if  |
>      |                    |  |  | IPv6                                |
>      |                    |  |  | Bit 1-4. Transport. 4 bit unsigned  |
>      |                    |  |  | value where 0 = UDP, 1 = TCP, 2 =    |
>      |                    |  |  | TLS, 3 = DTLS. Values 4-15 are      |
>      |                    |  |  | reserved for future use.            |

Again, probably want to specify DoH.


S 17.3.
>      [18] https://github.com/dns-stats/draft-dns-capture-
>          format/blob/master/file-size-versus-block-size.svg

>  Appendix A.  CDDL

>      This appendix gives a CDDL [I-D.ietf-cbor-cddl] specification for

Is this a normative appendix?
2018-11-20
08 Eric Rescorla [Ballot Position Update] New position, No Objection, has been recorded for Eric Rescorla
2018-11-20
08 Alissa Cooper
[Ballot comment]
I support Benjamin's first DISCUSS point. In addition to documenting the privacy considerations, I think it's important for this document to be crystal …
[Ballot comment]
I support Benjamin's first DISCUSS point. In addition to documenting the privacy considerations, I think it's important for this document to be crystal clear about who is meant to be doing the data collection -- namely, the server operator. There are some statements in the document that otherwise could be construed to be encouraging third-party passive monitoring of DNS traffic without explaining why, which seems like a problem:

Section 1:

"There has long been a need to collect DNS queries and responses on
  authoritative and recursive name servers for monitoring and analysis."

Section 3:

"In an ideal world, it would be optimal to collect full packet
  captures of all packets going in or out of a name server."
2018-11-20
08 Alissa Cooper [Ballot Position Update] New position, No Objection, has been recorded for Alissa Cooper
2018-11-19
08 Alvaro Retana [Ballot comment]
I support Benjamin's DISCUSS.
2018-11-19
08 Alvaro Retana [Ballot Position Update] New position, No Objection, has been recorded for Alvaro Retana
2018-11-19
08 Deborah Brungard [Ballot Position Update] New position, No Objection, has been recorded for Deborah Brungard
2018-11-19
08 Alexey Melnikov
[Ballot discuss]
Thank you for this document, it is a useful contribution to RFC series. I enjoyed reading it.

I have a small list of …
[Ballot discuss]
Thank you for this document, it is a useful contribution to RFC series. I enjoyed reading it.

I have a small list of issues that is hopefully easy to fix:

1)

In 7.4.2:

  | filter          | O | T | "tcpdump" [pcap] style filter for      |
  |                  |  |  | input.                                |

This makes the [pcap] reference Normative. If you don't want to do that, please fully specify syntax in this document.

2) I believe [I-D.ietf-cbor-cddl] reference needs to be Normative due to use of CDDL in Appendix A. If you don't think CDDL is normative, you need to state that in Appendix A.
2018-11-19
08 Alexey Melnikov
[Ballot comment]
Was CDDL in Appendix A validated with a tool? This information is missing from the shepherding write-up.

6.2.3.  Storage flags

  The Storage …
[Ballot comment]
Was CDDL in Appendix A validated with a tool? This information is missing from the shepherding write-up.

6.2.3.  Storage flags

  The Storage Parameters also contains optional fields holding details
  of the sampling method used and the anonymisation method used.  It is
  RECOMMENDED these fields contain URIs pointing to resources
  describing the methods used.

Please add a Normative Reference for URI spec here (RFC 3986).



7.5.3.2.  "QueryResponseSignature"

  | qr-transport-flags | O | U | Bit flags describing the transport  |
  |                    |  |  | used to service the query.          |
  |                    |  |  | Bit 0. IP version. 0 if IPv4, 1 if  |
  |                    |  |  | IPv6                                |
  |                    |  |  | Bit 1-4. Transport. 4 bit unsigned  |
  |                    |  |  | value where 0 = UDP, 1 = TCP, 2 =    |
  |                    |  |  | TLS, 3 = DTLS. Values 4-15 are      |
  |                    |  |  | reserved for future use.            |
  |                    |  |  | Bit 5. 1 if trailing bytes in query  |
  |                    |  |  | packet. See Section 11.2.            |

Would something like DoH appear as a separate transport choice?

How can new values be added to the list if there are no IANA Considerations?


7.5.3.5.  "MalformedMessageData"

  |                    |  |  |                                      |
  | mm-transport-flags | O | U | Bit flags describing the transport  |
  |                    |  |  | used to service the query. Bit 0 is  |
  |                    |  |  | the least significant bit.          |
  |                    |  |  | Bit 0. IP version. 0 if IPv4, 1 if  |
  |                    |  |  | IPv6                                |
  |                    |  |  | Bit 1-4. Transport. 4 bit unsigned  |
  |                    |  |  | value where 0 = UDP, 1 = TCP, 2 =    |
  |                    |  |  | TLS, 3 = DTLS. Values 4-15 are      |
  |                    |  |  | reserved for future use.            |

If the above bits supposed to be the same as for qr-transport-flags,
maybe it is worth defining them once and referencing in all relevant places?


7.6.  "QueryResponse"
  | query-size          | O | U | DNS query message size (see        |
  |                      |  |  | below).                            |
  |                      |  |  |                                    |
  | response-size        | O | U | DNS query message size (see        |
  |                      |  |  | below).                            |

I think "DNS response message size" for response-size.


It is odd to have RFC 2119 language in B.2, which is itself informative.
2018-11-19
08 Alexey Melnikov [Ballot Position Update] New position, Discuss, has been recorded for Alexey Melnikov
2018-11-19
08 Gunter Van de Velde Request for Telechat review by OPSDIR is assigned to Joel Jaeggli
2018-11-19
08 Gunter Van de Velde Request for Telechat review by OPSDIR is assigned to Joel Jaeggli
2018-11-18
08 Benjamin Kaduk
[Ballot discuss]
It is pretty shocking to not see any discussion of the privacy
considerations of storing data including client addresses (and ports)
alongside DNS …
[Ballot discuss]
It is pretty shocking to not see any discussion of the privacy
considerations of storing data including client addresses (and ports)
alongside DNS transactions, given how central DNS resolution is to user
behavior on the web.  (Note that there are mentions of potentially
anonymized data in Sections 6.2 and 6.2.3 which would presumably
forward-reference the privacy considerations.)  Data normalization would
probably also be mentioned in this section, since (e.g.) the case used for
a query/response could be used in fingerprinting an implementation.

I'm also concerned about the policy/procedure for allocating/extending the
various bitfields and similar potential extension points in the data
structures.  Section 8 covers the major/minor versioning semantics with
respect to new map keys and new maps, but not addition of new bits within
existing (uint) bitmaps.  Given the usage of the CDDL .bits constraint,
it's not really clear that an IANA registry is the right tool to use, but I
think some indication of the expected way to allocate new bits is in order,
whether it's "a future standards-track document that updates this document"
or otherwise.  (I've noted many, but not all, instances of such bitmaps in
my COMMENT section.)

There are also a couple of fields whose semantics don't seem to be
sufficiently well specified for a proposed-standard document, such as
vlan-ids, generator-id, name-rdata, and ae-code.  (I understand that some
of them are probably only going to have locally relevant semantics, but we
should be explicit about when that's the case.)

If I'm reading things correctly that the IP address type is inferred from
the bytestring length, then I think we need to enforce a restriction on the
address prefix length(s) to allow for that inference to be unambiguous
(noting that we only have the *byte* length of the address fields at our
disposal for disabmgituation, and not the more precise bit-length).
2018-11-18
08 Benjamin Kaduk
[Ballot comment]
Section 2

Please consider using the RFC 8174 version of the BCP 14 boilerplate.

Section 3

  Because of these considerations, a major …
[Ballot comment]
Section 2

Please consider using the RFC 8174 version of the BCP 14 boilerplate.

Section 3

  Because of these considerations, a major factor in the design of the
  format is minimal storage size of the capture files.

maybe "storage and transmission"?

Section 6

In Figure 2, the Query name is marked as "(q)" (only present if there is a
query), but the running text in Section 4 (bullet 1) says that the Question
section from the response can be used as an identifying QNAME if there is a
response with no corresponding query.  Am I misexpanding QNAME here, or is
there a disagreement between these two parts of the text?  In particular, I
do not see a part of Figure 2 that would correspond to a Question section
in the response, given the various "(q)"/"(r)" markings.

Section 6.2.2

  Messages with OPCODES known to the recording application but not
  listed in the Storage Parameters are discarded (regardless of whether
  they are malformed or not).

(Do we need to say anything that the "discarded" is only w.r.t. the capture
process, and not meant to imply that DNS queries would not get a normal
response?)

Section 6.2.4

Please consider using IPv6 examples, per
https://www.iab.org/2016/11/07/iab-statement-on-ipv6/ .

Section 7.2

  o  The column T gives the CBOR data type of the item.

      *  U - Unsigned integer

      *  I - Signed integer

This is venturing a bit far from my normal area of expertise, but my
understanding is that CBOR native major types are only provided for
unsigned integer and negative integer, with "signed integer" being an
abstraction at a slightly higher layer that needs to be managed in the
application.  Do we need to add any clarifying text here or will the
meaning be clear to the reader?

Section 7.4

Should probably forward-reference section 8 for the format version numbers'
semantics.

Section 7.4.1.1

We should we reference the IANA registries by name for any of these fields
(e.g., opcodes, rr-types, etc.).  (Also in Section 7.5.3.1, etc.)

Are the storage flags going to be allocated in sequence by updating
standards-track documents, or some other mechanism?  (Is a registry
necessary?)

For the various address prefix fields, do we need to specify that the full
addresses are stored when the corresponding prefix field is absent?

Section 7.4.1.1.1

Am I parsing the "query-response-hints" text correctly to say that a bit is
set in the bitmap if the corresponding field is recorded (if present) by
the collecting implementation?  The causality of "if the field is omitted
the bit is unset" goes in a direction that is not what I expected.
(Similarly for the other fields in this table.)

Section 7.4.2

Do we need a reference for "promiscuous mode"?

Just to check: in "server-addresses", I just infer the IP version from the
length of the byte string?

Do we need to say more about where the vlan-ids identifiers are taken from?

Is the "generator-id" string intended to only be human readable?  Only
within a specific (administrative) context?

Section 7.5.1

Does "earliest-time" include leap seconds?

Section 7.5.3

The "ip-address" description seems to imply that very short ipv6 prefix
lengths could cause confusion as to the address type being indicated (e.g.,
setting to 32 when no ipv4 prefix length is set, or setting to the same
value as the ipv4 prefix length).  Do we need to restrict the ipv6 prefix
lengths to being 33 or larger?

Are the "name-rdata" contents in wire format or presentation format?

Section 7.5.3.2

What's the allocation policy/procedure for the remaining
qr-transport-flags transport values?  For additional bits in any/all of the
flags fields listed here?

Something of a side note, what's the mnemonic for the "sig" in
"qr-sig-flags"?  That is, what is it a signature of or over (it doesn't
seem like it's a cryptographic signature, which may be what is confusing
me)?

For "query-rcode"/"response-rcode", should there be a reference for "OPT",
and/or for any of the EDNS stuff in here?  (The Terminology section only
mentions using the naming from RFC 1035, that I can see.)

The "mm-transport-flags" here bear a striking resemblance to the
"qr-transport-flags" from Section 7.5.3.2; should there be a shared
registry for their contents?  (I guess the TransportFlags CDDL to some
extent serves this function.)

Section 7.7

How is the value of the "ae-code" determined?

Appendix A

We could perhaps apply some constraints on (e.g.) the address-prefex length
fields to be .le the relevant lengths.

Appendix C.6

                                          Using a strong compression,
  block sizes over 10,000 query/response pairs would seem to offer
  limited improvements.

nit: Using a strong compression scheme
2018-11-18
08 Benjamin Kaduk [Ballot Position Update] New position, Discuss, has been recorded for Benjamin Kaduk
2018-11-17
08 Warren Kumari IESG state changed to IESG Evaluation from Waiting for AD Go-Ahead
2018-11-15
08 Jean Mahoney Request for Last Call review by GENART Completed: Ready with Nits. Reviewer: Meral Shirazipour.
2018-11-13
08 (System) IANA Review state changed to IANA OK - No Actions Needed from IANA - Review Needed
2018-11-13
08 Amanda Baber
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-dnsop-dns-capture-format-08, which is currently in Last Call, and has the following comments:

We …
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-dnsop-dns-capture-format-08, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Amanda Baber
Lead IANA Services Specialist
2018-11-13
08 (System) IESG state changed to Waiting for AD Go-Ahead from In Last Call
2018-11-12
08 Magnus Westerlund Request for Last Call review by TSVART is assigned to Brian Trammell
2018-11-12
08 Magnus Westerlund Request for Last Call review by TSVART is assigned to Brian Trammell
2018-11-02
08 Magnus Westerlund Request for Last Call review by TSVART is assigned to Michael Tüxen
2018-11-02
08 Magnus Westerlund Request for Last Call review by TSVART is assigned to Michael Tüxen
2018-10-31
08 Tero Kivinen Request for Last Call review by SECDIR is assigned to Samuel Weiler
2018-10-31
08 Tero Kivinen Request for Last Call review by SECDIR is assigned to Samuel Weiler
2018-10-30
08 Jean Mahoney Request for Last Call review by GENART is assigned to Meral Shirazipour
2018-10-30
08 Jean Mahoney Request for Last Call review by GENART is assigned to Meral Shirazipour
2018-10-30
08 Cindy Morgan IANA Review state changed to IANA - Review Needed
2018-10-30
08 Cindy Morgan
The following Last Call announcement was sent out (ends 2018-11-13):

From: The IESG
To: IETF-Announce
CC: tjw.ietf@gmail.com, dnsop-chairs@ietf.org, draft-ietf-dnsop-dns-capture-format@ietf.org, dnsop@ietf.org, Tim …
The following Last Call announcement was sent out (ends 2018-11-13):

From: The IESG
To: IETF-Announce
CC: tjw.ietf@gmail.com, dnsop-chairs@ietf.org, draft-ietf-dnsop-dns-capture-format@ietf.org, dnsop@ietf.org, Tim Wicinski , warren@kumari.net
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (C-DNS: A DNS Packet Capture Format) to Proposed Standard


The IESG has received a request from the Domain Name System Operations WG
(dnsop) to consider the following document: - 'C-DNS: A DNS Packet Capture
Format'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2018-11-13. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract


  This document describes a data representation for collections of DNS
  messages.  The format is designed for efficient storage and
  transmission of large packet captures of DNS traffic; it attempts to
  minimize the size of such packet capture files but retain the full
  DNS message contents along with the most useful transport metadata.
  It is intended to assist with the development of DNS traffic
  monitoring applications.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-dnsop-dns-capture-format/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-dnsop-dns-capture-format/ballot/

The following IPR Declarations may be related to this I-D:

  https://datatracker.ietf.org/ipr/2909/





2018-10-30
08 Cindy Morgan IESG state changed to In Last Call from Last Call Requested
2018-10-30
08 Warren Kumari Last call was requested
2018-10-30
08 Warren Kumari Last call announcement was generated
2018-10-30
08 Warren Kumari IESG state changed to Last Call Requested from AD Evaluation
2018-10-29
08 Amy Vezza Placed on agenda for telechat - 2018-11-21
2018-10-29
08 Warren Kumari Ballot has been issued
2018-10-29
08 Warren Kumari Ballot approval text was generated
2018-10-29
08 Warren Kumari [Ballot Position Update] New position, Yes, has been recorded for Warren Kumari
2018-10-29
08 Warren Kumari Created "Approve" ballot
2018-10-29
08 Warren Kumari Ballot writeup was changed
2018-10-29
08 Warren Kumari Ballot writeup was changed
2018-10-15
08 Warren Kumari IESG state changed to AD Evaluation from Publication Requested
2018-10-02
08 Tim Wicinski

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  …

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

    The Document is labeled as "Standards Track", and this is the proper
    RFC type.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

    This document describes a data representation for collections of DNS
    messages.  The format is designed for efficient storage and
    transmission of large packet captures of DNS traffic; it attempts to
    minimize the size of such packet capture files but retain the full
    DNS message contents along with the most useful transport metadata.

Working Group Summary

    There was no controvesy with the working group. However, during an
    IETF Hackathon, several issues were during the proof of concept.
    The document was corrected to address these issues, and is a
    stronger document because of this.

Document Quality

    There is an existing implementation, as well as converters from
    this format to other packet formats.

Personnel

Document Shepherd: Tim Wicinski
Area Director: Warren Kumari


(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

    The Document Shepherd did both a technical review, as well as an
    editorial review.  Document is ready for publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

    No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

    No

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of?

    There are no concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

    Authors have confirmed IPR disclosures.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

    An IPR disclosure has been filed, by an engineer who feels that
    their former employer *may* have IPR implications.  They can not
    speak for their former employee, and the owner of the
    patent never came forward.

    The working group initially felt the document should not
    move forward until the IPR was resolved.  But after much
    discussion, and no solid claim placed, the working group decided
    to move this forward.
    https://datatracker.ietf.org/ipr/2909/

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

    Document has solid WG Consensus as well as wide consensus.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent?

    No

(11) Identify any ID nits the Document Shepherd has found in this
document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

    N/A

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

    None 

(13) Have all references within this document been identified as
either normative or informative?

    Yes

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state?

    No

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in
the Last Call procedure.

    No

(16) Will publication of this document change the status of any
existing RFCs?

    No

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document.

    There are no IANA Considerations.


(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

    N/A
   
(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

2018-10-02
08 Tim Wicinski Responsible AD changed to Warren Kumari
2018-10-02
08 Tim Wicinski IETF WG state changed to Submitted to IESG for Publication from WG Consensus: Waiting for Write-Up
2018-10-02
08 Tim Wicinski IESG state changed to Publication Requested
2018-10-02
08 Tim Wicinski IESG process started in state Publication Requested
2018-10-02
08 Tim Wicinski Tag Revised I-D Needed - Issue raised by WG cleared.
2018-10-02
08 Tim Wicinski Changed document writeup
2018-08-10
08 Sara Dickinson New version available: draft-ietf-dnsop-dns-capture-format-08.txt
2018-08-10
08 (System) New version approved
2018-08-10
08 (System) Request for posting confirmation emailed to previous authors: Sara Dickinson , John Dickinson , Jim Hague , John Bond , Terry Manderson
2018-08-10
08 Sara Dickinson Uploaded new revision
2018-07-20
07 Tim Wicinski Tag Revised I-D Needed - Issue raised by WG set.
2018-07-20
07 Tim Wicinski IETF WG state changed to WG Consensus: Waiting for Write-Up from In WG Last Call
2018-07-06
07 Tim Wicinski Notification list changed to Tim Wicinski <tjw.ietf@gmail.com>
2018-07-06
07 Tim Wicinski Document shepherd changed to Tim Wicinski
2018-07-06
07 Tim Wicinski IETF WG state changed to In WG Last Call from WG Document
2018-05-08
07 Sara Dickinson New version available: draft-ietf-dnsop-dns-capture-format-07.txt
2018-05-08
07 (System) New version approved
2018-05-08
07 (System) Request for posting confirmation emailed to previous authors: Sara Dickinson , John Dickinson , Jim Hague , John Bond , Terry Manderson
2018-05-08
07 Sara Dickinson Uploaded new revision
2018-03-05
06 Sara Dickinson New version available: draft-ietf-dnsop-dns-capture-format-06.txt
2018-03-05
06 (System) New version approved
2018-03-05
06 (System) Request for posting confirmation emailed to previous authors: Sara Dickinson , John Dickinson , Jim Hague , John Bond , Terry Manderson
2018-03-05
06 Sara Dickinson Uploaded new revision
2018-02-22
05 Sara Dickinson New version available: draft-ietf-dnsop-dns-capture-format-05.txt
2018-02-22
05 (System) New version approved
2018-02-22
05 (System) Request for posting confirmation emailed to previous authors: Sara Dickinson , John Dickinson , Jim Hague , John Bond , Terry Manderson
2018-02-22
05 Sara Dickinson Uploaded new revision
2018-01-03
04 Sara Dickinson New version available: draft-ietf-dnsop-dns-capture-format-04.txt
2018-01-03
04 (System) New version approved
2018-01-03
04 (System) Request for posting confirmation emailed to previous authors: Sara Dickinson , John Dickinson , Jim Hague , John Bond , Terry Manderson
2018-01-03
04 Sara Dickinson Uploaded new revision
2017-07-03
03 John Dickinson New version available: draft-ietf-dnsop-dns-capture-format-03.txt
2017-07-03
03 (System) New version approved
2017-07-03
03 (System) Request for posting confirmation emailed to previous authors: Sara Dickinson , John Dickinson , Jim Hague , John Bond , Terry Manderson
2017-07-03
03 John Dickinson Uploaded new revision
2017-04-18
02 Jim Hague New version available: draft-ietf-dnsop-dns-capture-format-02.txt
2017-04-18
02 (System) New version approved
2017-04-18
02 (System) Request for posting confirmation emailed to previous authors: Sara Dickinson , John Dickinson , Jim Hague , John Bond , Terry Manderson
2017-04-18
02 Jim Hague Uploaded new revision
2017-02-21
01 Sara Dickinson New version available: draft-ietf-dnsop-dns-capture-format-01.txt
2017-02-21
01 (System) New version approved
2017-02-21
01 (System) Request for posting confirmation emailed to previous authors: "John Dickinson" , "John Bond" , "Sara Dickinson" , "Jim Hague" , "Terry Manderson"
2017-02-21
01 Sara Dickinson Uploaded new revision
2016-12-08
00 Tim Wicinski Changed consensus to Yes from Unknown
2016-12-08
00 Tim Wicinski Intended Status changed to Proposed Standard from None
2016-12-06
00 Suzanne Woolf This document now replaces draft-dickinson-dnsop-dns-capture-format instead of None
2016-12-06
00 John Dickinson New version available: draft-ietf-dnsop-dns-capture-format-00.txt
2016-12-06
00 (System) WG -00 approved
2016-12-06
00 John Dickinson Set submitter to "John Dickinson ", replaces to draft-dickinson-dnsop-dns-capture-format and sent approval email to group chairs: dnsop-chairs@ietf.org
2016-12-06
00 John Dickinson Uploaded new revision