CBOR Working Group                                         M. Richardson
Internet-Draft                                  Sandelman Software Works
Intended status: Best Current Practice                     21 April 2021
Expires: 23 October 2021


            On storing CBOR encoded items on stable storage
                     draft-ietf-cbor-file-magic-01

Abstract

   This document proposes an on-disk format for CBOR objects that is
   friendly to common on-disk recognition systems like the Unix file(1)
   command.

   This document is being discussed at: https://github.com/cbor-wg/cbor-
   magic-number

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 23 October 2021.

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Simplified BSD License text
   as described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Simplified BSD License.



Richardson               Expires 23 October 2021                [Page 1]


Internet-Draft               cbor-file-magic                  April 2021


Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Requirements for a Magic Number . . . . . . . . . . . . . . .   3
   3.  Protocol  . . . . . . . . . . . . . . . . . . . . . . . . . .   4
     3.1.  The CBOR Protocol Specific Tag  . . . . . . . . . . . . .   4
     3.2.  CBOR Tag Wrapped  . . . . . . . . . . . . . . . . . . . .   4
     3.3.  CBOR Tag Sequence . . . . . . . . . . . . . . . . . . . .   4
   4.  Security Considerations . . . . . . . . . . . . . . . . . . .   5
   5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   5
     5.1.  CBOR Sequence Tag . . . . . . . . . . . . . . . . . . . .   5
   6.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   6
   7.  Changelog . . . . . . . . . . . . . . . . . . . . . . . . . .   6
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   6
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .   6
     8.2.  Informative References  . . . . . . . . . . . . . . . . .   6
   Appendix A.  Example from Openswan  . . . . . . . . . . . . . . .   7
   Contributors  . . . . . . . . . . . . . . . . . . . . . . . . . .   8
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .   8

1.  Introduction

   Since very early in computing, operating systems have sought ways to
   mark which files could be processed by which programs.

   For instance, the Unix file(1) command, which has existed since 1973
   [file], has been able to identify many file formats for decades.
   Many systems (Linux, MacOS, Windows) will select the correct
   application based upon the file contents, if the system can not
   determine it by other means: for instance, the classic MacOS
   maintained a resource fork that includes media type ("MIME type")
   information and therefore ideally never needs to know what anything
   about the file.  Other systems do this by file extensions.

   While having a media type associated with the file is a better
   solution in general, when files become disconnected from their type
   information, such as when attempting to do forensics on a damaged
   system, then being able to identify a file type can become very
   important.

   It is noted that in the media type registration, that a magic number
   is asked for, if available, as is a file extension.









Richardson               Expires 23 October 2021                [Page 2]


Internet-Draft               cbor-file-magic                  April 2021


   A challenge for the file(1) program is often that it can be confused
   by the encoding vs the content.  For instance, an Android "apk" used
   to transfer and store an application may be identified as a ZIP file.
   Both OpenOffice or MSOffice files are ZIP files of XML files.
   (Unless OpenOffice files are flat (fodp) files, in which case they
   may appear to be generic XML files.)

   As CBOR becomes a more and more common encoding for a wide variety of
   artifacts, identifying them as just "CBOR" is probably not
   sufficient.  This document provides a way to encode a magic number
   into the beginning of a CBOR format file.  Two options are presented:
   typically a CBOR Protocol author will specify one.

   A CBOR Protocol is a specification which uses CBOR as its encoding.
   Examples of CBOR Protocols currently under development include CoSWID
   [I-D.ietf-sacm-coswid], and EAT [I-D.ietf-rats-eat].  COSE itself
   [RFC8152] is considered infrastructure, however the encoding of
   public keys in CBOR as described in
   [I-D.mattsson-cose-cbor-cert-compress] would be an identified CBOR
   Protocol.

   A major inspiration for this document is observing the mess in ASN.1
   based systems where most files are PEM encoded, identified by the
   extension "pem", confusing public keys, private keys, certificate
   requests and SIME content.

   These proposals are invasive to how CBOR protocols are written to
   disk, but in both cases, the proposed envelope does not require that
   the tag be transfered on the wire.

   In addition to the on-disk identification aspects, there are some
   protocols which may benefit from having such a magic number on the
   wire if they presently using a different (legacy) encoding scheme.
   The presence of the identifiable magic sequence signals that CBOR is
   being used or a legacy scheme.

2.  Requirements for a Magic Number

   A magic number is ideally a unique fingerprint, present in the first
   4 or 8 bytes of the file, which does not change when the contents
   change, and does not depend upon the length of the file.

   Less ideal solutions have a pattern that needs to be matched, but in
   which some bytes need to be ignored.  While the Unix file(1) command
   can be told to ignore bytes, this can lead to ambiguities.






Richardson               Expires 23 October 2021                [Page 3]


Internet-Draft               cbor-file-magic                  April 2021


3.  Protocol

   There are two variations of this practice.  Both use CBOR Tags in a
   way that results in a deterministic first 8 to 12 bytes.

3.1.  The CBOR Protocol Specific Tag

   CBOR Protocol designers should obtain a tag for each major type of
   object that they might store on disk.  As there are more than 4
   million available 4-byte tags, there should be little issue in
   allocating a few to each available CBOR Protocol.

   The policy is First Come First Served, so all that is required is an
   email to IANA, having filled in the small template provided in
   section 9.2 of [RFC8949].

   This tag should be allocated by the author of the CBOR Protocol, and
   to be in the four-byte range, it should be at least 0x01000000
   (decimal 16777216) in value.

   The use of a sequence of four US-ASCII codes which are mnemonic to
   the protocol is encouraged, but not required.

3.2.  CBOR Tag Wrapped

   This proposal starts with the Self-described CBOR tag, 55799, as
   described in [RFC8949] section 3.4.6.

   A second CBOR Tag is then allocated to describe the specific Protocol
   involved, as described above.

   This proposal wraps the CBOR value as tags usually do.  Applications
   that need to send the CBOR value across a constrained link may wish
   to remove the two tags if the use is implicitly understood.  This is
   a decision of the CBOR Protocol specification.

3.3.  CBOR Tag Sequence

   This proposal makes use of CBOR Sequences as described in [RFC8742].

   This proposal consists of two tags and a constant string for a total
   of 12 bytes.

   1.  The file shall start with the Self-described CBOR Sequence tag,
       55800.






Richardson               Expires 23 October 2021                [Page 4]


Internet-Draft               cbor-file-magic                  April 2021


   2.  The file shall continue with a CBOR tag, from the First Come
       First Served space, which uniquely identifies the CBOR Protocol.
       The use of a four-byte tag is encouraged.

   3.  The three byte CBOR byte string containing 0x42_4F_52.  When
       encoded it shows up as "CBOR"

   The first part identifies the file as being CBOR, and does so with
   all the desirable properties explained in [RFC8949] section 3.4.6.
   Specifically, it does not seem to conflict with any known file types,
   and it is not valid Unicode in any Unicode encoding.

   The second part identifies which CBOR Protocol is used, as described
   above.

   The third part is a constant value 0x43_42_4f_52, "CBOR".  That is,
   it the three byte sequence 0x42_4f_52 ("BOR").  This is the data item
   that is tagged.

   The actual CBOR Protocol value then follows as the next data item(s)
   in the CBOR sequence, without a need for any further specific tag.
   The use of a CBOR Sequence allows the application to trivially remove
   the first item with the two tags.

   This means that should a file be reviewed by a human (directly in an
   editor, or in a hexdump display), it will include the string "CBOR"
   prominently.  This value is also included simply because the two tags
   need to tag something.

4.  Security Considerations

   This document provides a way to identify CBOR Protocol objects.
   Clearly identifying CBOR contents on disk may have a variety of
   impacts.

   The most obvious is that it may allow malware to identify interesting
   objects on disk, and then corrupt them.

5.  IANA Considerations

   There are no IANA actions.  This section documents the allocation
   that was done.

5.1.  CBOR Sequence Tag

   IANA has allocated tag 55800 as the CBOR Sequence tag.  This tag is
   from the First Come/First Served area.




Richardson               Expires 23 October 2021                [Page 5]


Internet-Draft               cbor-file-magic                  April 2021


   The value has been picked to have properties similiar to the 55799
   tag.

   The hexadecimal representation is: 0xd9_\d9_f8.

   This is not valid UTF-8: the first 0xd9 puts the value into the
   three-byte value of UTF-8, but the 0xd9 as the second value is not a
   valid second byte for UTF-8.

   This is not valid UTF-16: the byte sequence 0xd9d9 (in either endian
   order), puts this value into the UTF-16 high-half zone, which would
   signal that this a 32-bit Unicode value.  However, the following
   16-bit big-endian value 0xf8.. is not a valid second sequence
   according to [RFC2781].  On a little-endian system, it would be
   necessary to examine the fourth byte to determine if it is valid.
   That next byte is determined by the subsequent encoding, and
   [RFC8949] section 3.4.6 has already determined that no valid CBOR
   encodings result in a valid UTF-16.

   Data Item: byte string
   Semantics: indicates that the file contains CBOR Sequences

6.  Acknowledgements

   The CBOR WG brainstormed this protocol on January 20, 2021.

7.  Changelog

8.  References

8.1.  Normative References

   [BCP14]    Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC8742]  Bormann, C., "Concise Binary Object Representation (CBOR)
              Sequences", RFC 8742, DOI 10.17487/RFC8742, February 2020,
              <https://www.rfc-editor.org/info/rfc8742>.

   [RFC8949]  Bormann, C. and P. Hoffman, "Concise Binary Object
              Representation (CBOR)", STD 94, RFC 8949,
              DOI 10.17487/RFC8949, December 2020,
              <https://www.rfc-editor.org/info/rfc8949>.

8.2.  Informative References





Richardson               Expires 23 October 2021                [Page 6]


Internet-Draft               cbor-file-magic                  April 2021


   [file]     Wikipedia, "file (command)", 20 January 2021,
              <https://en.wikipedia.org/wiki/File_%28command%29>.

   [I-D.ietf-rats-eat]
              Mandyam, G., Lundblade, L., Ballesteros, M., and J.
              O'Donoghue, "The Entity Attestation Token (EAT)", Work in
              Progress, Internet-Draft, draft-ietf-rats-eat-09, 7 March
              2021, <https://www.ietf.org/archive/id/draft-ietf-rats-
              eat-09.txt>.

   [I-D.ietf-sacm-coswid]
              Birkholz, H., Fitzgerald-McKay, J., Schmidt, C., and D.
              Waltermire, "Concise Software Identification Tags", Work
              in Progress, Internet-Draft, draft-ietf-sacm-coswid-17, 22
              February 2021, <https://www.ietf.org/archive/id/draft-
              ietf-sacm-coswid-17.txt>.

   [I-D.mattsson-cose-cbor-cert-compress]
              Raza, S., Höglund, J., Selander, G., Mattsson, J. P., and
              M. Furuhed, "CBOR Encoded X.509 Certificates (C509
              Certificates)", Work in Progress, Internet-Draft, draft-
              mattsson-cose-cbor-cert-compress-08, 22 February 2021,
              <https://www.ietf.org/archive/id/draft-mattsson-cose-cbor-
              cert-compress-08.txt>.

   [ilbm]     Wikipedia, "Interleaved BitMap", 20 January 2021,
              <https://en.wikipedia.org/wiki/ILBM>.

   [RFC2781]  Hoffman, P. and F. Yergeau, "UTF-16, an encoding of ISO
              10646", RFC 2781, DOI 10.17487/RFC2781, February 2000,
              <https://www.rfc-editor.org/info/rfc2781>.

   [RFC8152]  Schaad, J., "CBOR Object Signing and Encryption (COSE)",
              RFC 8152, DOI 10.17487/RFC8152, July 2017,
              <https://www.rfc-editor.org/info/rfc8152>.

Appendix A.  Example from Openswan

   The Openswan IPsec project has a daemon ("pluto"), and two control
   programs ("addconn", and "whack").  They communicate via a Unix-
   domain socket, over which a C-structure containing pointers to
   strings is serialized using a bespoke mechanism.  This is normally
   not a problem as the structure is compiled by the same compiler; but
   when there are upgrades it is possible for the daemon and the control
   programs to get out of sync by the bespoke serialization.  As a
   result, there are extra compensations to deal with shutting the
   daemon down.  During testing it is sometimes the case that upgrades
   are backed out.



Richardson               Expires 23 October 2021                [Page 7]


Internet-Draft               cbor-file-magic                  April 2021


   In addition, when doing unit testing, the easiest way to load policy
   is to use the normal policy reading process, but that is not normally
   loaded in the daemon.  Instead the IPC that is normally sent across
   the wire is compiled/serialized and placed in a file.  The above
   magic number is included in the file, and also on the IPC in order to
   distinguish the "shutdown" command CBOR operation.

   In order to reduce the problems due to serialization, the
   serialization is being changed to CBOR.  Additionally, this change
   allows the IPC to be described by CDDL, and for any language that
   encode to CBOR can be used.

   IANA has allocated the tag 1330664270, or 0x4f_50_ 53_ 4e for this
   purpose.  As a result, each file and each IPC is prefixed with:

   In diagnostic notation: ~~~~ 55800(1330664270(h'424F52')) ~~~~

   Or in hex: ~~~~ 00000000 d9 d9 f9 da 4f 50 53 4e 43 42 4f
   52 |....OPSNCBOR| ~~~~

Contributors

   Carsten Bormann

   Email: cabo@tzi.org


   Josef 'Jeff' Sipek

   Email: jeffpc@josefsipek.net


Author's Address

   Michael Richardson
   Sandelman Software Works

   Email: mcr+ietf@sandelman.ca













Richardson               Expires 23 October 2021                [Page 8]