Common PGP/MIME Message Mangling
draft-dkg-openpgp-pgpmime-message-mangling-00

Document Type Active Internet-Draft (individual)
Last updated 2019-05-22
Stream (None)
Intended RFC status (None)
Formats plain text xml pdf html bibtex
Stream Stream state (No stream defined)
Consensus Boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date
Responsible AD (None)
Send notices to (None)
openpgp                                                       D. Gillmor
Internet-Draft                                                      ACLU
Intended status: Informational                              May 22, 2019
Expires: November 23, 2019

                    Common PGP/MIME Message Mangling
             draft-dkg-openpgp-pgpmime-message-mangling-00

Abstract

   An e-mail message with OpenPGP encryption and/or signature has
   standard form.  However, some mail transport agents damage those
   forms in transit.  A user-friendly mail user agent that encounters
   such a mangled message may wish to try to process it anyway, despite
   the message's non-standard structure.

   This document aims to be a sort of bestiary of common message
   mangling patterns, along with guidance for implementers for how to
   cope with messages structured in these common ways.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on November 23, 2019.

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect

Gillmor                 Expires November 23, 2019               [Page 1]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
     1.2.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Problem Statement . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Cryptographic Properties  . . . . . . . . . . . . . . . . . .   4
   4.  Encrypted Messages  . . . . . . . . . . . . . . . . . . . . .   4
     4.1.  "Mixed Up" Encryption . . . . . . . . . . . . . . . . . .   4
       4.1.1.  Detection of "Mixed Up" Encryption  . . . . . . . . .   5
       4.1.2.  Repair of "Mixed Up" Encryption . . . . . . . . . . .   5
   5.  Clearsigned Messages  . . . . . . . . . . . . . . . . . . . .   5
     5.1.  Extra MIME footer . . . . . . . . . . . . . . . . . . . .   6
     5.2.  Content-Re-Encoding . . . . . . . . . . . . . . . . . . .   6
   6.  Performance Considerations  . . . . . . . . . . . . . . . . .   6
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .   6
     7.1.  Only Repair for Cryptographic Improvement . . . . . . . .   6
     7.2.  Avoiding Another E-Fail . . . . . . . . . . . . . . . . .   6
     7.3.  Do Not Attempt Repair Inside End-to-End Encryption  . . .   7
     7.4.  Interactions with DKIM Verification . . . . . . . . . . .   7
     7.5.  "Helpful" MTA mangling  . . . . . . . . . . . . . . . . .   7
   8.  Privacy Considerations  . . . . . . . . . . . . . . . . . . .   7
   9.  User Considerations . . . . . . . . . . . . . . . . . . . . .   8
     9.1.  Indications of Mangling Can Be Confusing  . . . . . . . .   8
     9.2.  Repair Indicators . . . . . . . . . . . . . . . . . . . .   8
     9.3.  Undoing Repair  . . . . . . . . . . . . . . . . . . . . .   8
     9.4.  Attempting Decryption During Repair . . . . . . . . . . .   8
   10. IANA Considerations . . . . . . . . . . . . . . . . . . . . .   8
   11. Document Considerations . . . . . . . . . . . . . . . . . . .   8
   12. Example Messages  . . . . . . . . . . . . . . . . . . . . . .   9
     12.1.  Example Input Encrypted Message  . . . . . . . . . . . .   9
     12.2.  Input Message Mangled by "Mixed Up"  . . . . . . . . . .   9
   13. Example Implemenations  . . . . . . . . . . . . . . . . . . .  10
     13.1.  Example: Detecting "Mixed Up" Encryption . . . . . . . .  10
     13.2.  Example: Repairing "Mixed Up" Encryption . . . . . . . .  11
   14. References  . . . . . . . . . . . . . . . . . . . . . . . . .  11
     14.1.  Normative References . . . . . . . . . . . . . . . . . .  11
     14.2.  Informative References . . . . . . . . . . . . . . . . .  12
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  12

Gillmor                 Expires November 23, 2019               [Page 2]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

1.  Introduction

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

1.2.  Terminology

   o  "Mail User Agent" (or "MUA") refers to a program designed to send
      and receive e-mail.  Common examples include Thunderbird, Outlook,
      and Mail.app.  Webmail implementations, like Roundcube or the
      Gmail interface can also be considered a MUA.

   o  "Mail Transport Agent" (or "MTA") refers to mail-handling software
      within the network.  Common examples include Postfix and Exchange.

   o  "PGP/MIME" is the message structure for OpenPGP protections
      described in [RFC3156].

   o  "encrypted message" is used to mean any message where the
      outermost PGP/MIME cryptographic feature of the message itself is
      encryption.  Some encrypted messages might be signed, but the
      inner signature cannot typically be mangled by an "MTA" that can't
      decrypt.

   o  "clearsigned message" is used here to refer specifically to
      messages that are not encrypted, but are signed with a PGP/MIME
      signature.

   o  "signed message" is used to refer to a message that has a valid
      OpenPGP cryptographic signature applied by the message sender,
      whether the message is encrypted or not.

2.  Problem Statement

   Some MTAs modify message headers, bodies, and structure in transit.
   Some of those modifications ("manglings") can transform a message in
   such a way that the end-to-end cryptographic properties of the
   message are lost.

   Clearsigned messages can become unverifiable, and encrypted messages
   can become impossible to decrypt.  MUAs that receive these mangled
   messages are unable to fulfil their users goals because of damage
   done to the message by the mangling MTA.

Gillmor                 Expires November 23, 2019               [Page 3]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

   In some cases, those manglings can take a regular enough form that a
   clever MUA can "repair" them, restoring the cryptographic properties
   of the message for the user.

   This document aims to collect common manglings, and document how an
   MUA can detect them and (if possible) repair them safely for the
   user.

3.  Cryptographic Properties

   TODO: describe message-wide cryptographic envelope vs. cryptographic
   payload and their relevance to this draft.

   TODO: we're assuming a single layer of cryptographic protection.
   These steps get more complicated if there are multiple nested layers
   (encrypted and then signed?  signed and then encrypted? triple-
   wrapped?).  Should this document address that situation?

4.  Encrypted Messages

   This section aims to collect examples of repairable mangling known to
   be performed by some currently-active MTA on encrypted messages.
   Each section offers a brief description of the mangling, a way to
   detect it, and a proposed method of repair.

4.1.  "Mixed Up" Encryption

   One common mangling takes an incoming multipart/encrypted e-mail and
   transforms it into a multipart/mixed e-mail, shuffling body parts at
   the same time.

   An encrypted message typically has this clean structure C (see
   example in Section 12.1):

   C   multipart/encrypted
   C.1  application/pgp-encrypted
   C.2  application/octet-stream

   But after processing, it has mangled structure M (see example in
   Section 12.2):

   M   multipart/mixed
   M.1  text/plain (0 bytes)
   M.2  application/pgp-encrypted
   M.3  application/octet-stream

   Some versions of Microsoft Exchange are known to do this.

Gillmor                 Expires November 23, 2019               [Page 4]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

4.1.1.  Detection of "Mixed Up" Encryption

   Check that all of the following conditions hold:

   o  The message itself (M) is Content-Type: multipart/mixed.

   o  The message has exactly three MIME subparts, all direct children
      of the message itself.

   o  The first part (M.1) is Content-Type: text/plain, and has 0 bytes.

   o  The second part (M.2) is Content-Type: application/pgp-encrypted,
      and contains (after removing any Content-Transfer-Encoding)
      exactly the string "Version: 1"

   o  The third part (M.3) is Content-Type: application/octet-stream.

   o  After removing any Content-Transfer-Encoding, the decoded third
      part (M.3) is an ASCII-armored OpenPGP block of type "PGP MESSAGE"
      (see see section 6.2 of [RFC4880])

   Please see Section 13.1 for an example implementation.

4.1.2.  Repair of "Mixed Up" Encryption

   If the detection is successful, repair can be attempted with the
   following steps:

   o  Convert the message's Content-Type: value to multipart/encrypted,
      while leaving any other parameters untouched.

   o  Add (or reset) the parameter protocol=application/pgp-encrypted to
      the Content-Type: header.

   o  Drop the first sub-part (M.1).

   Please see Section 13.2 for an example implementation.

5.  Clearsigned Messages

   This section aims to collect examples of repairable mangling known to
   be performed by some currently-active MTA on clearsigned messages.
   Each section offers a brief description of the mangling, a way to
   detect it, and a proposed method of repair.

Gillmor                 Expires November 23, 2019               [Page 5]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

5.1.  Extra MIME footer

   TODO: describe the mailman case

5.2.  Content-Re-Encoding

   TODO: despite the content of multipart/signed parts being ostensibly
   invariant (see page 4 of [RFC1847]), some MTAs mangle them.  Describe
   some reversible manglings, like gratuitous application of Content-
   Transfer-Encoding.

6.  Performance Considerations

   Trying multiple repair techniques can be expensive.  A resource-
   constrained MUA may want to limit itself to detection of possible
   mangling, rather than trying every possible repair it knows of and
   indicating

7.  Security Considerations

   There are several ways that attempting repair on a mangled message
   can go dangerously wrong.

7.1.  Only Repair for Cryptographic Improvement

   If a message repair produces a seemingly-better-structured message,
   but the repaired message still cannot be decrypted or verified, then
   the MUA should abandon the repair and instead render the message in
   its original form.  Additional message modification that does not
   provide a cryptographic advantage over the received form of the
   message offers no benefit to the user.

   TODO: is a decryptable, unsigned message "better" than a verifiably
   signed message?  What is "an improvement" isn't necessarily clear
   here.

7.2.  Avoiding Another E-Fail

   Promiscuous message repair may inject new vulnerabilities, or modify
   a message beyond recognition.  An MUA attempting message repair
   should use caution to avoid re-combining potentially malicious
   material with material that has better cryptographic guarantees.

   In particular, a responsible MUA should take care not to claim
   cryptographic protections over material that does not have them.

Gillmor                 Expires November 23, 2019               [Page 6]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

7.3.  Do Not Attempt Repair Inside End-to-End Encryption

   If a message or part of a message arrived end-to-end encrypted, none
   of the recommendations in this draft should be read as encouraging
   their application to any cleartext material within that end-to-end
   encryption.  These techniques are specifically for recovering from
   damage done by the MTA, which by design does not have access to the
   cleartext of an end-to-end encrypted message.

7.4.  Interactions with DKIM Verification

   A mangling MTA may also add DKIM headers after mangling.  A
   subsequent MTA that handles the message may verify the message by
   checking DKIM headers, storing the results of that check in an
   Authentication-Results header (see [RFC7601]).  If the MUA performs
   message repair, the repaired message might not pass DKIM check.  An
   MUA that relies on Authentication-Results headers for any purpose on
   such a repaired message should consider re-validating the DKIM header
   itself directly after message repair (though this introduces new
   problems for stored/old messages, as older DKIM signing keys may no
   longer be published in the DNS).

   Note that a repaired message may have cryptographic properties that
   obviates the need for DKIM verification; an MUA that requires
   messages to have a valid Authentication-Results: header might waive
   that requirement for a signed message (whether any repair routine was
   invoked or not).

7.5.  "Helpful" MTA mangling

   An MTA may intend to help the user by modifying the message.  For
   example, in a clearsigned message, the MTA may modify a potentially-
   malicious URL in the text, or it might change an attachment name or
   MIME-type in an attachment.  If an MUA attempts message repair in a
   way that reverses these modifications, it may negate whatever
   security advantages the MTA hoped to offer.

   It is not clear that this is a risk for repair of an encrypted
   message, though, since the MTA by design cannot perform such a
   modification on the cleartext of an encrypted message.

8.  Privacy Considerations

   TODO: privacy considerations?

Gillmor                 Expires November 23, 2019               [Page 7]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

9.  User Considerations

   Cryptographic properties of messages are only useful if the user
   understands them.

9.1.  Indications of Mangling Can Be Confusing

   o  TODO: about how indicators of mangled messages can cause user-
      experience confusion (e.g. parts of https://efail.de/ and
      https://github.com/RUB-NDS/Johnny-You-Are-Fired )

9.2.  Repair Indicators

   o  TODO: how should a repairing MUA indicate that a repair has
      happened (or is possible) to the user?

9.3.  Undoing Repair

   o  TODO: Should a repairing MUA allow the user to reverse the repair
      and see the broken message?  If so, what else should happen if the
      user takes this action (e.g., how is a subsequent "Reply" or
      "Forward" action affected?)

9.4.  Attempting Decryption During Repair

   Attempting decryption may require access to secret key material,
   which itself may cause interaction with the user.  If multiple
   repairs are attempted, each attempt to decrypt may require multiple
   decryptions.  In some scenarios, multiple use of the secret key may
   be annoying to the user.  It might be preferable to try to decrypt
   the apparent underlying block of text exactly once, regardless of
   message structure or repair, and if successful, either stash the
   session key, or store the cleartext in memory, while manipulating
   exterior structure.

10.  IANA Considerations

   This document requires no actions from IANA.

11.  Document Considerations

   [ RFC Editor: please remove this section before publication ]

   This document is currently edited as markdown.  Minor editorial
   changes can be suggested via merge requests at
   https://gitlab.com/dkg/draft-openpgp-pgpmime-message-mangling or by
   e-mail to the author.  Please direct all significant commentary to
   the public IETF OpenPGP mailing list: openpgp@ietf.org

Gillmor                 Expires November 23, 2019               [Page 8]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

12.  Example Messages

12.1.  Example Input Encrypted Message

   From: Michael Elkins <elkins@aero.org>
   To: Michael Elkins <elkins@aero.org>
   Mime-Version: 1.0
   Content-Type: multipart/encrypted; boundary=foo;
      protocol="application/pgp-encrypted"

   --foo
   Content-Type: application/pgp-encrypted

   Version: 1

   --foo
   Content-Type: application/octet-stream

   -----BEGIN PGP MESSAGE-----

   hIwDY32hYGCE8MkBA/wOu7d45aUxF4Q0RKJprD3v5Z9K1YcRJ2fve87lMlDlx4Oj
   eW4GDdBfLbJE7VUpp13N19GL8e/AqbyyjHH4aS0YoTk10QQ9nnRvjY8nZL3MPXSZ
   g9VGQxFeGqzykzmykU6A26MSMexR4ApeeON6xzZWfo+0yOqAq6lb46wsvldZ96YA
   AABH78hyX7YX4uT1tNCWEIIBoqqvCeIMpp7UQ2IzBrXg6GtukS8NxbukLeamqVW3
   1yt21DYOjuLzcMNe/JNsD9vDVCvOOG3OCi8=
   =zzaA
   -----END PGP MESSAGE-----

   --foo--

12.2.  Input Message Mangled by "Mixed Up"

Gillmor                 Expires November 23, 2019               [Page 9]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

   From: Michael Elkins <elkins@aero.org>
   To: Michael Elkins <elkins@aero.org>
   Mime-Version: 1.0
   Content-Type: multipart/mixed; boundary=foo

   --foo
   Content-Type: text/plain; charset="us-ascii"

   --foo
   Content-Type: application/pgp-encrypted

   Version: 1

   --foo
   Content-Type: application/octet-stream

   -----BEGIN PGP MESSAGE-----

   hIwDY32hYGCE8MkBA/wOu7d45aUxF4Q0RKJprD3v5Z9K1YcRJ2fve87lMlDlx4Oj
   eW4GDdBfLbJE7VUpp13N19GL8e/AqbyyjHH4aS0YoTk10QQ9nnRvjY8nZL3MPXSZ
   g9VGQxFeGqzykzmykU6A26MSMexR4ApeeON6xzZWfo+0yOqAq6lb46wsvldZ96YA
   AABH78hyX7YX4uT1tNCWEIIBoqqvCeIMpp7UQ2IzBrXg6GtukS8NxbukLeamqVW3
   1yt21DYOjuLzcMNe/JNsD9vDVCvOOG3OCi8=
   =zzaA
   -----END PGP MESSAGE-----

   --foo--

13.  Example Implemenations

   These examples use python3.  As Code Examples, they are licensed
   under the Simplified BSD License as noted above.

13.1.  Example: Detecting "Mixed Up" Encryption

Gillmor                 Expires November 23, 2019              [Page 10]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

   import email.message

   def is_mixed_up(msg: email.message.Message) -> bool:
     if msg.get_content_type() == 'multipart/mixed':
       pts = msg.get_payload()
       if len(pts) == 3 and \
          pts[0].get_content_type() == 'text/plain' and \
          pts[0].get_payload(decode=True) == b'' and \
          pts[1].get_content_type() == 'application/pgp-encrypted' and \
          pts[1].get_payload(decode=True).strip() == b'Version: 1' and \
          pts[2].get_content_type() == 'application/octet-stream':
         encpart = pts[2].get_payload(decode=True)
         # FIXME: this is not a true check for RFC4880 ASCII armor:
         lines = encpart.replace(b'\r\n', b'\n').split(b'\n')
         if lines[0] == b'-----BEGIN PGP MESSAGE-----' and \
            lines[-1] == b'-----END PGP MESSAGE-----':
           return True
     return False

13.2.  Example: Repairing "Mixed Up" Encryption

   import copy
   import email.message
   import typing

   def un_mix_up(msg: email.message.Message) -> \
       typing.Optional[email.message.Message]:
     if is_mixed_up(msg):
       ret = copy.deepcopy(msg)
       ret.set_type('multipart/encrypted')
       ret.set_param('protocol', 'application/pgp-encrypted')
       ret.set_payload(msg.get_payload()[1:])
       return ret
     return None

14.  References

14.1.  Normative References

   [RFC1847]  Galvin, J., Murphy, S., Crocker, S., and N. Freed,
              "Security Multiparts for MIME: Multipart/Signed and
              Multipart/Encrypted", RFC 1847, DOI 10.17487/RFC1847,
              October 1995, <https://www.rfc-editor.org/info/rfc1847>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

Gillmor                 Expires November 23, 2019              [Page 11]
Internet-Draft      Common PGP/MIME Message Mangling            May 2019

   [RFC3156]  Elkins, M., Del Torto, D., Levien, R., and T. Roessler,
              "MIME Security with OpenPGP", RFC 3156,
              DOI 10.17487/RFC3156, August 2001,
              <https://www.rfc-editor.org/info/rfc3156>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

14.2.  Informative References

   [RFC4880]  Callas, J., Donnerhacke, L., Finney, H., Shaw, D., and R.
              Thayer, "OpenPGP Message Format", RFC 4880,
              DOI 10.17487/RFC4880, November 2007,
              <https://www.rfc-editor.org/info/rfc4880>.

   [RFC7601]  Kucherawy, M., "Message Header Field for Indicating
              Message Authentication Status", RFC 7601,
              DOI 10.17487/RFC7601, August 2015,
              <https://www.rfc-editor.org/info/rfc7601>.

Author's Address

   Daniel Kahn Gillmor
   American Civil Liberties Union
   125 Broad St.
   New York, NY  10004
   USA

   Email: dkg@fifthhorseman.net

Gillmor                 Expires November 23, 2019              [Page 12]