Skip to main content

Framework of Fast Fault Detection for IP-baesd Networks
draft-wang-ffd-framework-00

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft whose latest revision state is "Expired".
Authors Haibo Wang , Fengwei Qin , Lily Zhao , Shuanglong Chen
Last updated 2022-10-24
RFC stream (None)
Formats
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-wang-ffd-framework-00
Network Working Group                                            H. Wang
Internet-Draft                                                    Huawei
Intended status: Informational                                    F. Qin
Expires: 27 April 2023                                      China Mobile
                                                                 L. Zhao
                                                                 S. Chen
                                                                  Huawei
                                                         24 October 2022

        Framework of Fast Fault Detection for IP-baesd Networks
                      draft-wang-ffd-framework-00

Abstract

   The IP-based distributed system and software application layer often
   use heartbeat to maintain the network topology status.  However, the
   heartbeat setting is long, which prolongs the system fault detection
   time.  IP-based storage network is the typical usage of that
   scenario.  When an IP-based storage network fault occurs, NVMe
   connections need to be switched over.  Currently, no effective method
   is available for quick detection, switchover is performed only based
   on keepalive timeout, resulting in low performance.

   This document defines the basic framework of how network assisted
   host devices can quickly detect application connection failures
   caused by network faults.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

Wang, et al.              Expires 27 April 2023                 [Page 1]
Internet-Draft              Abbreviated-Title               October 2022

   This Internet-Draft will expire on 27 April 2023.

Copyright Notice

   Copyright (c) 2022 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Reference Models  . . . . . . . . . . . . . . . . . . . . . .   3
   4.  Functional Components . . . . . . . . . . . . . . . . . . . .   4
     4.1.  Server Endpoint (Storage Device)  . . . . . . . . . . . .   5
     4.2.  Client Endpoint (Host)  . . . . . . . . . . . . . . . . .   5
     4.3.  Network Device  . . . . . . . . . . . . . . . . . . . . .   6
   5.  Procedures  . . . . . . . . . . . . . . . . . . . . . . . . .   6
     5.1.  Network Deployment  . . . . . . . . . . . . . . . . . . .   7
     5.2.  Hosts and Storage devices . . . . . . . . . . . . . . . .   7
     5.3.  Status Infomation Sync And Notification . . . . . . . . .   7
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .   8
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   9
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .   9
     8.2.  Informative References  . . . . . . . . . . . . . . . . .   9
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   9

1.  Introduction

   IP-based distributed systems are widely used, and the network is
   opaque to application-side systems.  When an IP network connected to
   a distributed system encounters a fault that affects IP connectivity,
   the application system cannot quickly detect the fault.  To enable
   the application system to quickly detect the fault, the application
   system needs to accelerate keepalive or deploy a detection mechanism,
   which brings extra overheads to the application system.

Wang, et al.              Expires 27 April 2023                 [Page 2]
Internet-Draft              Abbreviated-Title               October 2022

   The [I-D.guo-ffd-requirement] describes the requirements for these
   applications.  The most typical application scenario is the IP-based
   NVMe scenario.

   IP-based NVMe is an implementation of NVMe over Fabrics that best
   fits NVMe semantics.  It is the development trend of high-speed
   storage networks in the future.  IP-based NVMe for high-speed storage
   has high requirements on IP networks.  In an IP-based NVMe network,
   when a failure that affects an IP connection occurs, for example, an
   access link failure or a switch network failure that cannot perform
   route convergence, the NVMe connection cannot immediately detect the
   fault.  In the current implementation mechanism, this failure can
   only be detected based on keepalive timeout.  Generally, this failure
   lasts more than 10s.  To speed up detection, hosts and storage
   devices can use fast keepalive or BFD for fast detection.  However,
   the solution introduces additional load on hosts and storage devices,
   making it difficult to use in large-scale IP-based NVMe.

2.  Terminology

   NoF : NVMe of Fabrics

   FC : Fiber Channel

   NVMe : Non-Volatile Memory Express

   SAN: Storage Area Network

3.  Reference Models

   This document describes the framework based of IP-based NVMe as a
   typical application.

   An IP-based NVMe mainly includes three types of roles: an initiator
   (referred to as a host), a switch, and a target (referred to as a
   storage device).  Initiators and targets are also referred to as
   client endpoint and server endpoint.  Hosts and storage devices use
   the IP-based NVMe protocol to transmit data over the network to
   provide high-performance storage services.

Wang, et al.              Expires 27 April 2023                 [Page 3]
Internet-Draft              Abbreviated-Title               October 2022

                          +--+      +--+      +--+      +--+
              Host        |H1|      |H2|      |H3|      |H4|
           (Initiator)    +/-+      +-,+      +.-+      +/-+
                           |         | '.   ,-`|         |
                           |         |   `',   |         |
                           |         | ,-`  '. |         |
                         +-\--+    +--`-+    +`'--+    +-\--+
                         | SW1|    | SW2|    | SW3|    | SW4|
                         +--,-+    +---,,    +,.--+    +-.--+
                             `.          `'.,`         .`
                               `.   _,-'`    ``'.,   .`
               IP              +--'`+            +`-`-+
             Network           | SW5|            | SW6|
                               +--,,+            +,.,-+
                               .`   `'.,     ,.-``   ',
                             .`         _,-'`          `.
                         +--`-+    +--'`+    `'---+    +-`'-+
                         | SW7|    | SW8|    | SW9|    |SW10|
                         +-.,-+    +-..-+    +-.,-+    +-_.-+
                           | '.   ,-` |        | `.,   .' |
                           |   `',    |        |    '.`   |
                           | ,-`  '.  |        | ,-`  `', |
             Storage      +-`+      `'\+      +-`+      +`'+
             (Target)     |S1|      |S2|      |S3|      |S4|
                          +--+      +--+      +--+      +--+
                          Figure 1 : NVMe over IP-based Network

   This is a dual-plane NVMe over IP-based Network which applies to a
   large-scale storage device access network.  Storage devices on the
   dual-homed access network provide NVMe services using two different
   IP addresses.

   When an access link (for example, the S1-SW7 link) or a network-side
   link (for example, the SW7-SW5 link) fails, H1 cannot access the IP
   address of S1 connected to SW1.  H1 cannot quickly detect the
   failure.  After the keepalive timeout, H1 can detect the failure and
   then switch the NVMe connection to the IP address that S1 accesses
   through SW8.

4.  Functional Components

   The NVMe IP-based SANs consists of storage devices, hosts and
   switches.  The storage device provides services.  The host initiates
   an NVMe connection to the storage device.  That is, the host is the
   Client Endpoint, and the storage device is the Server Endpoint.

Wang, et al.              Expires 27 April 2023                 [Page 4]
Internet-Draft              Abbreviated-Title               October 2022

4.1.  Server Endpoint (Storage Device)

   As a service provider, the server endpoint does not need to detect
   the status of the client.  To enable the network to know the
   information about the server, the server needs to advertise its
   information to the access switch.

   To reduce the complexity of server endpoint, it is suggested to
   extend the LLDP protocol to support registration.

                     +-----------+           +--------+
                     | Server EP |           | Switch |
                     | (Storage) |           |        |
                     +----/------+           +----/---+
                          |                       |
                          |    Register Msg       |
                          |---------------------->|
                          |                       |
                          \                       \
                       Figure 2 : Server Endpoint

4.2.  Client Endpoint (Host)

   The client needs to quickly obtain the IP reachability status of the
   service endpoint.  In this case, the client needs to send a
   subscription request to the access switch.  In addition, to
   facilitate the network to know the location of the client endpoint,
   the client endpoint needs to register its information to the access
   switch.  When the switch network senses a failure required by the
   client endpoint, the access switch notifies the corresponding client
   endpoint of the fault state.

   Also, to reduce the complexity of client endpoints, it is recommended
   that the LLDP protocol be extended to support subscriptions.  For
   notification messages initiated by the switch to client endpoints, it
   is recommended that the L2 extension protocol be used to control the
   notification scope.

Wang, et al.              Expires 27 April 2023                 [Page 5]
Internet-Draft              Abbreviated-Title               October 2022

    +-----------+           +--------+
    | Client EP |           | Switch |
    |  (Host)   |           |        |
    +----/------+           +----/---+
         |                       |
         |    Register Msg       |
         |---------------------->|
         |                       |
         |    Subscribe Msg      |
         |---------------------->|
         |                       |
         |   Notification Msg    |
         |<----------------------|
         |                       |
         \                       \
       Figure 3 : Client Endpoint

4.3.  Network Device

   Network devices, such as access switches, can quickly detect failures
   on local access links.  The client endpoint that needs to obtain the
   failure may not be connected to that switch.  Therefore, the switch
   that detects the failure needs to synchronize the information to
   other switches so that the other switches can notify the required
   endpoint as required.

   On a large-scale network, reflector can be used to reduce the number
   of connections for information synchronization between switches.

   To ensure that synchronization messages can be reliably synchronized
   to other switches, a reliable transmission protocol, such as TCP or
   Quic, must be used.

    +--------+    +-----------+   +--------+
    | Switch |    | Reflector |   | Switch |
    +----/---+    +-----/-----+   +---/----+
         |              |             |
         |   Sync Msg   |             |
         |------------->|   Sync Msg  |
         |              |------------>|
         \              \             \
       Figure 4 : Network Device

5.  Procedures

   Here use the IP-based NVMe interaction example to see the complete
   deployment process of this framework.

Wang, et al.              Expires 27 April 2023                 [Page 6]
Internet-Draft              Abbreviated-Title               October 2022

5.1.  Network Deployment

   The IP-based NVMe uses the standard IP technology.  Network
   deployments typically use the current IP technologies.  For example,
   OSPF is usually deployed as an underlay protocol.

5.2.  Hosts and Storage devices

   Hosts and storage devices are connected to the IP network.  As shown
   by Figure 1, they may access the network in single-homing or dual-
   homing mode.  The administrator assigns access IP addresses to the
   hosts and storage devices.  In most scenarios, these routes can be
   advertised through the underlay protocol.

   To enable IP network devices to know the information about these
   access nodes, hosts and storage devices need to register their own
   network information, such as IP addresses and roles, with the access
   switches after accessing the network.  In addition, the host needs to
   initiate a subscription request to the access switch to notify the
   access switch of the information about the storage device it cares
   about.

5.3.  Status Infomation Sync And Notification

   Hosts and storage devices are connected to different switches.  To
   enable these switches to obtain the registration and subscription
   information of these hosts and storage devices, synchronizing the
   information between the switches is needed.

Wang, et al.              Expires 27 April 2023                 [Page 7]
Internet-Draft              Abbreviated-Title               October 2022

 +------+        +--------+   +-----------+   +--------+     +---------+
 | Host |        | Switch |   | Reflector |   | Switch |     | Storage |
 +--/---+        +----/---+   +-----/-----+   +---/----+     +----/----+
    |  Register Msg   |             |             |               |
    |---------------->|             |             |               |
    |  Subscribe Msg  |             |             |               |
    |---------------->|  Sync Msg   |             |               |
    |                 |------------>|   Sync Msg  |               |
    |                 |             |------------>|  Register Msg |
    |                 |             |   Sync Msg  |<--------------|
    |                 |   Sync Msg  |<------------|               |
    |                 |<------------|             |--/            |
    |                 |             |             |  |Fault       |
    |                 |             |             |  |Detection   |
    |                 |             |   Sync Msg  |<--            |
    |                 |   Sync Msg  |<------------|               |
    | Notification Msg|<------------|             |               |
    |<----------------|             |             |               |
    \                 \             \             \               \
            Figure 7 : Information Advertisement

   After detecting a local failure, the switch calculates the IP address
   affected by the failure.  If another access endpoint on the switch
   wants to obtain the IP address of the failure, the switch notifies
   that access endpoint of the fault.  In addition, the switch needs to
   synchronize the failure IP address to other switches on the network.
   After receiving the failure IP address information, other switches
   notify the access endpoints who need the information.

   When a link between network devices or a network device is failure,
   routes are converged on the network.  If services cannot be restored
   even after route convergence, such as SW7-SW5 shown in Figure 1, is
   faulty.  As a result, H1 cannot access the IP address used by S1 to
   access SW7.  In this case, after detecting the failure, the network
   device calculates the IP addresses affected by the failure.  Then,
   the network device notifies the required access endpoint of the
   failure information.  As shown in Figure 1, SW1 calculates that the
   IP address used by S1 to connect to SW7 is unreachable.  Therefore,
   SW1 notifies H1 of the failure so that H1 can quickly switch to
   another storage device.

6.  Security Considerations

   NA

Wang, et al.              Expires 27 April 2023                 [Page 8]
Internet-Draft              Abbreviated-Title               October 2022

7.  IANA Considerations

   NA

8.  References

8.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

8.2.  Informative References

   [I-D.guo-ffd-requirement]
              Guo, L., Feng, Y., Zhao, J., Qin, F., Zhao, L., and H.
              Wang, "Requirement of Fast Fault Detection for IP-based
              Network", Work in Progress, Internet-Draft, draft-guo-ffd-
              requirement-00, 24 October 2022,
              <https://www.ietf.org/archive/id/draft-guo-ffd-
              requirement-00.txt>.

Authors' Addresses

   Haibo Wang
   Huawei
   No. 156 Beiqing Road
   Beijing
   100095
   P.R. China
   Email: rainsword.wang@huawei.com

   Fengwei Qin
   China Mobile
   Beijing
   China
   Email: qinfengwei@chinamobile.com

   Lily Zhao
   Huawei
   No. 3 Shangdi Information Road
   Beijing
   100085
   P.R. China
   Email: Lily.zhao@huawei.com

Wang, et al.              Expires 27 April 2023                 [Page 9]
Internet-Draft              Abbreviated-Title               October 2022

   Shuanglong Chen
   Huawei
   No. 156 Beiqing Road
   Beijing
   100095
   P.R. China
   Email: chenshuanglong@huawei.com

Wang, et al.              Expires 27 April 2023                [Page 10]