Main logging schema for qlog
draft-marx-qlog-main-schema-02

Document Type Active Internet-Draft (individual)
Author Robin Marx 
Last updated 2020-11-02
Stream (None)
Intended RFC status (None)
Formats plain text html xml pdf htmlized (tools) htmlized bibtex
Stream Stream state (No stream defined)
Consensus Boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date
Responsible AD (None)
Send notices to (None)
QUIC                                                             R. Marx
Internet-Draft                                        Hasselt University
Intended status: Standards Track                         2 November 2020
Expires: 6 May 2021

                      Main logging schema for qlog
                     draft-marx-qlog-main-schema-02

Abstract

   This document describes a high-level schema for a standardized
   logging format called qlog.  This format allows easy sharing of data
   and the creation of reusable visualization and debugging tools.  The
   high-level schema in this document is intended to be protocol-
   agnostic.  Separate documents specify how the format should be used
   for specific protocol data.  The schema is also format-agnostic, and
   can be represented in for example JSON, csv or protobuf.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 6 May 2021.

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

Marx                       Expires 6 May 2021                   [Page 1]
Internet-Draft        Main logging schema for qlog         November 2020

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Simplified BSD License text
   as described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Notational Conventions  . . . . . . . . . . . . . . . . .   3
   2.  Design goals  . . . . . . . . . . . . . . . . . . . . . . . .   5
   3.  The high level qlog schema  . . . . . . . . . . . . . . . . .   6
     3.1.  summary . . . . . . . . . . . . . . . . . . . . . . . . .   7
     3.2.  traces  . . . . . . . . . . . . . . . . . . . . . . . . .   7
     3.3.  Individual Trace containers . . . . . . . . . . . . . . .   8
       3.3.1.  configuration . . . . . . . . . . . . . . . . . . . .   9
       3.3.2.  vantage_point . . . . . . . . . . . . . . . . . . . .  11
     3.4.  Field name semantics  . . . . . . . . . . . . . . . . . .  13
       3.4.1.  timestamps  . . . . . . . . . . . . . . . . . . . . .  14
       3.4.2.  category and event  . . . . . . . . . . . . . . . . .  16
       3.4.3.  data  . . . . . . . . . . . . . . . . . . . . . . . .  17
       3.4.4.  protocol_type . . . . . . . . . . . . . . . . . . . .  18
       3.4.5.  custom fields . . . . . . . . . . . . . . . . . . . .  18
       3.4.6.  triggers  . . . . . . . . . . . . . . . . . . . . . .  18
       3.4.7.  group_id  . . . . . . . . . . . . . . . . . . . . . .  19
       3.4.8.  common_fields . . . . . . . . . . . . . . . . . . . .  20
   4.  Serializing qlog  . . . . . . . . . . . . . . . . . . . . . .  22
     4.1.  qlog to JSON mapping  . . . . . . . . . . . . . . . . . .  23
       4.1.1.  numbers . . . . . . . . . . . . . . . . . . . . . . .  23
       4.1.2.  bytes . . . . . . . . . . . . . . . . . . . . . . . .  24
       4.1.3.  Summarizing table . . . . . . . . . . . . . . . . . .  25
       4.1.4.  Other JSON specifics  . . . . . . . . . . . . . . . .  26
     4.2.  qlog to NDJSON mapping  . . . . . . . . . . . . . . . . .  27
       4.2.1.  Supporting NDJSON in tooling  . . . . . . . . . . . .  28
     4.3.  Other optimizated formatting options  . . . . . . . . . .  28
       4.3.1.  Data structure optimizations  . . . . . . . . . . . .  29
       4.3.2.  Compression . . . . . . . . . . . . . . . . . . . . .  30
       4.3.3.  Binary formats  . . . . . . . . . . . . . . . . . . .  31
       4.3.4.  Overview and summary  . . . . . . . . . . . . . . . .  32
     4.4.  Conversion between formats  . . . . . . . . . . . . . . .  33
   5.  Methods of access and generation  . . . . . . . . . . . . . .  34
     5.1.  Set file output destination via an environment
           variable  . . . . . . . . . . . . . . . . . . . . . . . .  34
     5.2.  Access logs via a well-known endpoint . . . . . . . . . .  35
Show full document text