IPsec and IKE anti-replay sequence number subspaces for multi-path tunnels and multi-core processing
draft-ponchon-ipsecme-anti-replay-subspaces-00

ipsecme                                                       P. Ponchon
Internet-Draft                                                 M. Shaikh
Intended status: Standards Track                              P. Pfister
Expires: 27 April 2023                                       G. Solignac
                                                            Cisco Meraki
                                                         24 October 2022

   IPsec and IKE anti-replay sequence number subspaces for multi-path
                   tunnels and multi-core processing
             draft-ponchon-ipsecme-anti-replay-subspaces-00

Abstract

   This document discusses the challenges of running IPsec with anti-
   replay in environments where packets may be re-ordered (e.g., when
   sent over multiple IP paths, traffic-engineered paths and/or using
   different QoS classes) as well as when processed on multiple cores.
   Different approaches to solving this problem are discussed, and a new
   solution based on splitting the anti-replay sequence number space
   into multiple different sequencing subspaces is proposed.  Since this
   solution requires support on both parties, an IKE extension is
   proposed in order to negotiate the use of the Anti-Replay sequence
   number subspaces.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 27 April 2023.

Copyright Notice

   Copyright (c) 2022 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

Ponchon, et al.           Expires 27 April 2023                 [Page 1]
Internet-Draft         IPsec anti-replay subspaces          October 2022

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Problem Statement . . . . . . . . . . . . . . . . . . . . . .   3
     2.1.  Solution Space  . . . . . . . . . . . . . . . . . . . . .   4
   3.  Conventions and Definitions . . . . . . . . . . . . . . . . .   4
   4.  Multiple sequence number subspaces  . . . . . . . . . . . . .   4
     4.1.  Sequence number subspace encoding in IPSec  . . . . . . .   5
     4.2.  Sender Behavior . . . . . . . . . . . . . . . . . . . . .   5
     4.3.  Receiver Behavior . . . . . . . . . . . . . . . . . . . .   6
     4.4.  Extended Sequence Numbers (ESN) considerations  . . . . .   6
     4.5.  Negotiating sequence-number subspaces using IKE . . . . .   7
     4.6.  Solution Analysis . . . . . . . . . . . . . . . . . . . .   7
   5.  Security Considerations . . . . . . . . . . . . . . . . . . .   8
   6.  Implementation Considerations . . . . . . . . . . . . . . . .   8
     6.1.  Initialization Vector (IV) Considerations . . . . . . . .   9
   7.  Operational Considerations  . . . . . . . . . . . . . . . . .   9
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  10
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  10
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .  10
     9.2.  Informative References  . . . . . . . . . . . . . . . . .  10
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  10

1.  Introduction

   The IPsec and IKE protocol suite is very commonly used in secure
   overlay networks, often interconnecting thousands or tens of
   thousands of sites.  Each of these sites often have two or more ways
   to connect to the internet (e.g., multiple fiber/cable, cellular or
   MPLS uplinks), with the promise that leveraging the different paths
   between nodes could bring greater throughput, availability and
   quality of service.

Ponchon, et al.           Expires 27 April 2023                 [Page 2]
Internet-Draft         IPsec anti-replay subspaces          October 2022

   Such scale and multi-paths requirements conflict with how anti-replay
   currently works.  This document first describes the problems related
   to running IPsec with anti-replay in multi-core and multi-paths
   environments, as well as their existing solutions.  A new solution
   where the IPsec sequence number sequencing space is divided into
   multiple subspaces is then specified.  Finally, implementation,
   security and operational considerations are discussed.

2.  Problem Statement

   A common approach to leverage multiple paths between IPsec peers is
   to allocate one Child SA per path, but given the quadratic number of
   paths that may exist between peers ('number of uplinks of peer 1' x
   'number of uplinks of peer 2'), this shows scalability issues in both
   IKE and IPsec implementations:

   *  Increased number of IKE negotiations and re-key operations.

   *  Increased IKE memory usage.

   *  Data-plane performance degradation due to the use of a larger
      number of keys.

   *  Data-plane reduced number of connected peers, due to a hard limit
      to the number of supported Child SAs.

   On the other hand, using a single Child SA per peer on multiple paths
   comes with challenges related to the anti-replay mechanism.  Packets
   traveling on different paths may arrive out-of-order.  A packet could
   travel on a slower network path, compared to another faster path, and
   arrive too late for the anti-replay window to be able to check
   whether the packet is a replay or not, causing the packet to be
   dropped.

   The same problem may also be observed when multiple QoS policies are
   used: Packets may be re-ordered, and a lower priority queue could
   have its packets arrive too late compared to others for the anti-
   replay mechanism to function properly.

   Finally, anti-replay implementations also suffer from performance
   issues when multiple threads are involved in sending or receiving
   encrypted packets for the same Child SA.  This is discussed in
   [I-D.pwouters-ipsecme-multi-sa-performance], which mainly focuses on
   high-throughput IPsec tunnels, but the problem also arises with small
   tunnels since multiple inner flows processed by multiple threads
   often need to be transmitted on the same tunnel (causing multiple
   threads to need to access shared resources).

Ponchon, et al.           Expires 27 April 2023                 [Page 3]
Internet-Draft         IPsec anti-replay subspaces          October 2022

2.1.  Solution Space

   The solution detailed in [I-D.pwouters-ipsecme-multi-sa-performance]
   proposes to use one Child SA per core, and could be extended to
   provide a solution involving one SA per path as well.  But allocating
   one SA per path, as well as per core, would further multiply the
   number of Child SAs.  For example, with 6 paths between two peers
   (one peer has 2 uplinks, the other has 3), and 4 cores to process the
   IPsec packets, the number of Child SAs would be multiplied by 24.
   This will divide by 24 the number of peers that an IPsec concentrator
   supports.

   Alternatively, a single Child SA per peer could be used with a very
   large anti-replay window (e.g., 128k bits), in order to mitigate the
   risk of packets being dropped when packets are sent on multiple
   paths.  But this solution has some serious drawbacks:

   *  The window size is guessed based on the expected throughput and
      delay difference between the best and the worst path, but the best
      value depends on many factors, and hence cannot be guessed.

   *  The window size can get very large, even more so as the throughput
      increases.

   *  Large window sizes cause performance degradation and scalability
      limitations by increasing the amount of memory accessed by the
      data-plane.

3.  Conventions and Definitions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in
   BCP14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

4.  Multiple sequence number subspaces

   The problems of using a single Child SA on multiple paths and
   processing the packets associated with a single Child SA on multiple
   cores both suffer from limitations due to the anti-replay mechanism.

Ponchon, et al.           Expires 27 April 2023                 [Page 4]
Internet-Draft         IPsec anti-replay subspaces          October 2022

   As a result, this section describes a solution which modifies the
   anti-replay mechanism by allowing the 32 bits or 64 bits (with ESN)
   anti-replay sequence number space into multiple subspaces.  Each
   path, or core, or combination of both, can then use their own unique
   anti-replay sequence number subspace.  The changes needed to the ESP
   header and IPsec protocol are described in Section 4.1, Section 4.2
   and Section 4.3.

   Since this specification requires both IPsec peers to implement this
   specification, an IKE extension is presented in Section 4.5, allowing
   peers to coordinate on the use, or not, of this specification.

4.1.  Sequence number subspace encoding in IPSec

   The 32-bit sequence number field of the ESP header is split into two
   sub-fields:

   *  The higher order 8 bits contain the new sequence number subspace
      ID.

   *  The lower order 24 bits continue to be used as the explicit part
      of the sequence number.

     0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |               Security Parameters Index (SPI)                 |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |  Subspace ID  |                Sequence Number                |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                    Rest of the ESP payload

4.2.  Sender Behavior

   The sender MAY set the sequence number subspace ID to any value.  For
   example, the sender would use different values per path or per
   processing core.

   The sender MUST maintain one sequence number counter per sequence
   number subspace that it makes use of.  But the sender MAY use only
   some (and as few as a single one) of the available 256 subspaces.

   When transmitting a packet, the sender MUST use the sequence number
   counter associated with the sequence number subspace in use for that
   packet.  The lower-order 24 bits of the sequence number counter are
   placed in the sequence number field, as specified in [RFC4303].

Ponchon, et al.           Expires 27 April 2023                 [Page 5]
Internet-Draft         IPsec anti-replay subspaces          October 2022

4.3.  Receiver Behavior

   The receiver MUST maintain one anti-replay window and counter for
   each sequence number subspace being used.

   When receiving a packet, the receiver MUST use the anti-replay window
   and counter associated with the sequence number subspace identified
   with the subspace ID field.

   Note: Since the sender may decide to only use a subset of the
   available subspace values, receivers SHOULD NOT allocate 256 anti-
   replay windows per peer by default.  Two mitigation mechanisms may be
   used to reduce the number of anti-replay windows:

   *  The receiver SHOULD limit the number of allocated anti-replay
      windows to the number of subspaces negotiated during the IKE Child
      SA creation exchange, as specified in Section 4.5.

   *  The receiver MAY reactively allocate an anti-replay window when
      receiving the first packet for a given subspace, since the sender
      may decide to not use all of the available values.  When doing so,
      the receiver SHOULD first check the authenticity of the packet
      before allocating the new anti-replay window.

4.4.  Extended Sequence Numbers (ESN) considerations

   When using Extended Sequence Numbers (ESNs), the sender and receiver
   MUST consider the 8 bits at indices 24 to 31 of the sequence number
   64 bits long counter to be implicit.

   Note that the reduction of the sequence number space to 24 bits
   specified in this document has some implications over the speed at
   which the explicit part of the sequence number will loop.  At a
   10Gbps rate, with 1500B ethernet frames, the 24 bits sequence number
   space will loop every 20 seconds.

   As a result:

   *  The peers SHOULD use ESN, unless the tunnel is known to run at low
      speed rates.

   *  Any outage lasting longer than 20 seconds might cause a resync (as
      defined in appendix A3 of [RFC4303]).

Ponchon, et al.           Expires 27 April 2023                 [Page 6]
Internet-Draft         IPsec anti-replay subspaces          October 2022

   Note: A peer may unilaterally enforce the use of ESN by specifying
   the IKE ESN transform with the value "1" (implying that the peer
   suggests to use ESN) in its CHILD_SA proposal, while ommitting to
   include the IKE ESN transform with the value "0" (implying that
   disabling ESN is not accepted).

   Authors' note: Is it too much of a problem ? Should we consider
   adding a new 32 bits field in the ESP header to contain the subspace
   ID, as well as future ESP extensions ?

4.5.  Negotiating sequence-number subspaces using IKE

   A mechanism will be needed to:

   *  Make sure both parties agree to use sequence-number subspaces when
      creating a Child SA.

   *  Let the sender express a preferred number of subspaces it wants to
      use, as well as the maximum number of subspaces it is capable of
      using.

   *  Let the receiver peer decide on a number of subspaces to be used
      by the sender, between zero (meaning that subspaces should not be
      used), and the maximum number of subspaces that the sender
      supports.

   This document will specify such negotiation.

   The authors would like to request the ipsecme working group for input
   on how to best implement the negotiation of this new functionality in
   IKEv2 [RFC5996].

4.6.  Solution Analysis

   As described in Section 2, anti-replay comes with implementation and
   scalability challenges when running in environments where IPsec peers
   may leverage multiple paths to send packets or multiple cores to
   process them.

   Since the anti-replay mechanism seems to be the source cause of these
   observed challenges, this document provides a solution which relies
   on a small and optional change at the anti-replay level.

   By using sequence number subspaces, IPsec peers may:

   *  use different subspaces for different cores, which allows
      distributing a Child SA between cores to increase performance

Ponchon, et al.           Expires 27 April 2023                 [Page 7]
Internet-Draft         IPsec anti-replay subspaces          October 2022

   *  use different subspaces for different QoS classes or different
      paths, which avoids unwanted drops due to potential reordering of
      packets, either at the egress or during its flight.

   *  combine the above per-path and per-core approaches without
      multiplying the number of required Child SAs.

5.  Security Considerations

   The sequence number is used by the anti-replay mechanism to ensure a
   packet could not be accepted twice by the receiver.  This prevents an
   attacker from trying to replay one or multiple packets from an IPsec
   tunnel.

   In this proposal, a single Child SA is associated with multiple anti-
   replay windows and counters.  When trying to replay a packet, the
   sequence number subspace ID must remain the same since the Subspace
   ID field is authenticated.  As a result the receiver will use the
   same anti-replay state when processing the replayed packet as the one
   used when the first packet was first received.  This ensures that a
   replayed packet will be detected and dropped by the receiver.

6.  Implementation Considerations

   When a single sequence number space is used within a given Child SA,
   encryption and decryption operations must always happen on the same
   core (locking anti-replay structures or using contended atomic
   operations has a dramatic performance hit).

   *  On reception, this requires packets which are received (and load-
      balanced to cores) to be often handoff to a different thread for
      processing.

   *  On transmisson, multiple flows, processed by different cores, need
      to be transmitted using the same Child SA.  This requires the
      packets to be handoff to the thread in charge of the given Child
      SA.

   To avoid the performance degradation caused by packet handoffs, each
   thread may use its own sequence number subspace:

   *  On transmission, the core will always select the subspace it is
      assigned when generating the ESP header.

   *  On reception, the subspace ID could be used to load-balance the
      packets to their proper thread.

   Similarly, when multiple paths are used:

Ponchon, et al.           Expires 27 April 2023                 [Page 8]
Internet-Draft         IPsec anti-replay subspaces          October 2022

   *  On transmission, a different sequence number subspace is used for
      each packet path.  Ensuring that out-of-order packets are not
      dropped by the anti-replay mechanism.

   *  On reception, the 5-tuple based packet steering would provide a
      decent level of load-balancing between threads, since different IP
      paths would use different 5-tuples.

   If a combination of both multi-path and multi-core load-balancing is
   needed, the subspace field could be used partly to encode a path ID,
   partly to encode a core ID.  But this is purely implementation
   specific and does not require coordination between the peers.

6.1.  Initialization Vector (IV) Considerations

   Depending on the cryptographic mode of operations, the Initialization
   Vector (IV) comes with specific requirements.

   Some modes (e.g., CBC) make use of random IV values.  When
   implementing this specification, each thread independently generates
   its independent stream of random values, ensuring the IV randomness
   property.  Care must be taken as to limit the global number of
   transmitted packets using the same Child SA in order to avoid
   birthday paradox attacks.  A lockless counter, or batched token
   bucket mechanism, may be used to efficiently implement this process
   without performance degradation.

   Other cryptographic modes (e.g., GCM) do not have randomness
   requirements over the IV, but the IV values must only be used once.
   RFC4106 Section 3.1 states that "The most natural way to implement
   this is with a counter, but anything that guarantees uniqueness can
   be used, such as a linear feedback shift register (LFSR).  Note that
   the encrypter can use any IV generation method that meets the
   uniqueness requirement, without coordinating with the decrypter." .
   One simple way to implement this specification is to divide the IV
   into a subspace field, which reuses the ESP sequence number subspace
   value, and a variable IV part, which is simply incremented for each
   encrypted packet.

   Author's note: Are there other cryptographic modes with different
   requirements over the IV ?

7.  Operational Considerations

   TBD

Ponchon, et al.           Expires 27 April 2023                 [Page 9]
Internet-Draft         IPsec anti-replay subspaces          October 2022

8.  IANA Considerations

   TBD

9.  References

9.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC4303]  Kent, S., "IP Encapsulating Security Payload (ESP)",
              RFC 4303, DOI 10.17487/RFC4303, December 2005,
              <https://www.rfc-editor.org/info/rfc4303>.

   [RFC5996]  Kaufman, C., Hoffman, P., Nir, Y., and P. Eronen,
              "Internet Key Exchange Protocol Version 2 (IKEv2)",
              RFC 5996, DOI 10.17487/RFC5996, September 2010,
              <https://www.rfc-editor.org/info/rfc5996>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

9.2.  Informative References

   [I-D.pwouters-ipsecme-multi-sa-performance]
              Antony, A., Brunner, T., Klassert, S., and P. Wouters,
              "IKEv2 support for per-queue Child SAs", Work in Progress,
              Internet-Draft, draft-pwouters-ipsecme-multi-sa-
              performance-04, 31 August 2022,
              <https://www.ietf.org/archive/id/draft-pwouters-ipsecme-
              multi-sa-performance-04.txt>.

Authors' Addresses

   Paul Ponchon
   Cisco Meraki
   Email: pponchon@cisco.com

   Mohsin Shaikh
   Cisco Meraki
   Email: mohsisha@cisco.com

Ponchon, et al.           Expires 27 April 2023                [Page 10]
Internet-Draft         IPsec anti-replay subspaces          October 2022

   Pierre Pfister
   Cisco Meraki
   Email: ppfister@cisco.com

   Guillaume Solignac
   Cisco Meraki
   Email: gsoligna@cisco.com

Ponchon, et al.           Expires 27 April 2023                [Page 11]