[Search] [pdf|bibtex] [Tracker] [Email] [Nits]

Versions: 00 01 02                                                      
Transport Working Group                                         P. Heist
Internet-Draft                                          18 February 2021
Intended status: Informational
Expires: 22 August 2021


     Explicit Congestion Notification (ECN) Deployment Observations
            draft-heist-tsvwg-ecn-deployment-observations-00

Abstract

   This note presents data gathered at an Internet Service Provider's
   gateway on the observed deployment and usage of ECN.  Relevant IP
   counter and flow tracking data was collected and analyzed for TCP and
   other protocols.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 22 August 2021.

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Simplified BSD License text
   as described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Simplified BSD License.





Heist                    Expires 22 August 2021                 [Page 1]


Internet-Draft                   ecndep                    February 2021


Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Collection Details  . . . . . . . . . . . . . . . . . . . . .   3
   3.  Observations  . . . . . . . . . . . . . . . . . . . . . . . .   3
     3.1.  ECN Endpoint Activity . . . . . . . . . . . . . . . . . .   3
       3.1.1.  Client Initiation . . . . . . . . . . . . . . . . . .   3
       3.1.2.  Server Acceptance . . . . . . . . . . . . . . . . . .   4
     3.2.  RFC3168 AQM Activity  . . . . . . . . . . . . . . . . . .   4
     3.3.  ECN Codepoints on Non-TCP Protocols . . . . . . . . . . .   4
       3.3.1.  Tunneled Traffic  . . . . . . . . . . . . . . . . . .   5
       3.3.2.  Use of the ECN Field for Historical Reasons . . . . .   6
       3.3.3.  Use of the ECN Field Inadvertently  . . . . . . . . .   6
       3.3.4.  Use of the ECN Field Maliciously  . . . . . . . . . .   7
   4.  Study Limitations and Recommendations for Future Work . . . .   7
     4.1.  ECN Acceptance Rate . . . . . . . . . . . . . . . . . . .   7
     4.2.  Tunnels . . . . . . . . . . . . . . . . . . . . . . . . .   7
     4.3.  Non-TCP Protocols . . . . . . . . . . . . . . . . . . . .   7
     4.4.  Other Protocols . . . . . . . . . . . . . . . . . . . . .   8
     4.5.  NS Flag . . . . . . . . . . . . . . . . . . . . . . . . .   8
   5.  Abbreviated Output from ecn-stats . . . . . . . . . . . . . .   8
     5.1.  All IP  . . . . . . . . . . . . . . . . . . . . . . . . .   8
     5.2.  TCP initiated from LAN to WAN . . . . . . . . . . . . . .   9
     5.3.  Non-TCP conntrack-supported protocols initiated from LAN to
           WAN . . . . . . . . . . . . . . . . . . . . . . . . . . .  12
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  26
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  26
   8.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  26
   9.  Informative References  . . . . . . . . . . . . . . . . . . .  26
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  27

1.  Introduction

   To help guide the evolution of ECN, there is a need for more data on
   current deployment status, and observed usage of the ECN related
   bits, including:

   *  the initiation and acceptance of ECN capable TCP flows

   *  marking via CE, and feedback for TCP via the ECE and CWR flags

   *  codepoints set on packets for protocols other than TCP

   For several weeks, we gathered data on all traffic through an
   Internet Service Provider's gateway.  Though some of the results are
   informative, we caution that a larger, more widely reviewed and
   geographically distributed survey would be needed to be
   authoritative.



Heist                    Expires 22 August 2021                 [Page 2]


Internet-Draft                   ecndep                    February 2021


2.  Collection Details

   From December 28, 2020 to January 20, 2021, data was gathered on all
   traffic into and out of the Internet gateway at FreeNet Liberec, a
   cooperative WISP in an urban area of the Czech Republic.  A total of
   122.5 TB of incoming data and 12 TB of outgoing data was seen.

   Around 660 members belong to the ISP, and 861 member IP addresses on
   the LAN were considered active during data collection.  Most member
   IPs are used by a household of users, while others are for individual
   devices and public locations.

   [IPTABLES-ECN] was used to collect and analyze the data.  This
   consists of a script to gather the data using iptables and ipsets in
   Linux, and an analysis program that produces textual output.  An
   abbreviated version of the output is included in Section 5.  See the
   referred to source repository for more details and full output.

3.  Observations

   Our key observations are summarized as follows, and further expanded
   upon in the following sections:

   *  1.44% of TCP flows initiated ECN, across 45% of member IPs.

   *  The acceptance rate for ECN flows was likely >50%.

   *  24% of member IPs that negotiated TCP ECN flows saw apparent AQM
      marking via CE or ECE, with more congestion observed on the
      downstream.

   *  42% of the member IPs that saw CE or ECE were from subnets that
      have known AQM instances in the ISP's backhaul, and the remainder
      appear to be from unknown AQMs.

   *  Nonzero ECN codepoints were observed on 0.053% of non-TCP packets,
      with possible attribution to tunneled ECN and/or misuse of the ECN
      field.

3.1.  ECN Endpoint Activity

3.1.1.  Client Initiation

   Of 319.5 million TCP SYNs from LAN to WAN, 1.44% indicated ECN
   capability.  Of 861 active member IP addresses, 390 (45.3%) attempted
   initiation for at least one ECN flow.  A large proportion of the ECN
   flows are thought to come from Apple devices.




Heist                    Expires 22 August 2021                 [Page 3]


Internet-Draft                   ecndep                    February 2021


3.1.2.  Server Acceptance

   While 4.6 million ECN TCP SYNs were seen from LAN to WAN, 3.3 million
   ECN SYN-ACKs were seen in return.  While it's not possible to get an
   exact ECN acceptance rate from this, it appears to be reasonably
   high, likely due to default acceptance on prevailing server operating
   systems like Linux, FreeBSD and recent versions of Windows Server.

3.2.  RFC3168 AQM Activity

   There appears to be evidence of [RFC3168] marking AQMs.  Of 861
   active member IP addresses:

   *  382 member IPs, or 44%, successfully negotiated any TCP ECN flows

   *  90 member IPs, or 24% of those that negotiated ECN, saw any CE or
      ECE marks on negotiated TCP ECN flows

   Two backhaul links have fq_codel [RFC8290] deployed, serving the
   10.45.64.0/24 and 10.45.235.0/24 subnets.  This accounts for 38 of
   the 90 member IP addresses that saw CE or ECE, with the source of the
   remaining CE and ECE marks unknown.  These are presumed to be from
   other [RFC3168] marking AQM instances.

   Note that depending on the position of the marking AQM relative to
   the gateway, CE marks may not be seen on some packets, while TCP ECE
   flags are seen in the opposite direction.  For a number of member IP
   addresses, we saw 0 CE marks downstream, but ECE flags set upstream,
   suggesting an AQM downstream from the gateway marking downstream
   traffic.

3.3.  ECN Codepoints on Non-TCP Protocols

   Referring to the packet counts in the _All IP_ / _Both Directions_
   table in the stats output in Section 5, where M indicates megapackets
   and G, gigapackets:















Heist                    Expires 22 August 2021                 [Page 4]


Internet-Draft                   ecndep                    February 2021


        +========+==========+===============+=========+==========+
        |        | TCP      | Conntrack (X) | Other   | Total    |
        +========+==========+===============+=========+==========+
        | All    | 76.60 G  | ->            | 43.52 G | 120.14 G |
        +--------+----------+---------------+---------+----------+
        | CE     | 10031    | 3.38 M        | 813951  | 4.20 M   |
        +--------+----------+---------------+---------+----------+
        | ECT(0) | 523.91 M | 9.66 M        | 2.55 M  | 536.12 M |
        +--------+----------+---------------+---------+----------+
        | ECT(1) | 63       | 6.68 M        | 182928  | 6.86 M   |
        +--------+----------+---------------+---------+----------+

                                 Table 1

   (X) UDP, ICMP, DCCP, SCTP, GRE (Conntrack All packets included in
   Other)

   We note the following:

   *  TCP accounted for 97.7% of the 536 million ECT(0) marks

   *  0.68% of all TCP packets were marked with a nonzero ECN codepoint

   *  0.053% of all non-TCP packets were marked with a nonzero ECN
      codepoint

   *  Non-TCP accounted for 99.8% of the 4.2 million CE marks

   *  Non-TCP accounted for virtually all of the ECT(1) marks

   Possible explanations for ECN marks on non-TCP packets are explored
   further in this section.

3.3.1.  Tunneled Traffic

   There are several different encapsulation methods used when handling
   the ECN field through tunnels, as per [RFC3168] and [RFC6040]:

   1.  copy the ECN field from the inner to the outer packet

   2.  reset the ECN field on the outer packet to ECT(0)

   3.  set Not-ECT on the outer packet

   When method 3 is used at both ends of a tunnel, we would not expect
   to see ECN codepoint usage in either direction.





Heist                    Expires 22 August 2021                 [Page 5]


Internet-Draft                   ecndep                    February 2021


   When methods 1 or 2 are used at both ends of a tunnel, we would
   expect to see ECT(0) on both incoming and outgoing packets.  We would
   also expect a bias towards incoming packets, since more data is
   generally downloaded than uploaded, and pure ACKs do not have ECT(0)
   marks.

   When method 3 is used at only one end of the tunnel, we would expect
   to see ECT(0) on packets in only one direction.

   We note the following:

   *  Bi-directional ECT(0) marks were observed for two member IP / port
      pairs, on UDP port 443 and 60001.

   *  Uni-directional ECT(0) marks were observed for:

      -  UDP port 4500 (IPSec NAT traversal [RFC3948]) with 23 member IP
         addresses downstream, and 1 member IP address upstream.

      -  UDP port 51820 [WIREGUARD] with 2 member IP addresses
         downstream.

      -  Numerous UDP ports in other ranges, mostly on the downstream.

   While it's possible that some of the data observed was from tunneled
   ECN traffic, this can't be established definitively.

3.3.2.  Use of the ECN Field for Historical Reasons

   Some applications may still use historical definitions of the former
   TOS byte.  Although RFC791 reserved the ECN field for future use, the
   now obsolete [RFC1349] defined the TOS field as four bits within the
   Type of Service octet, one of which overlaps with the ECN field.
   This may account for some of the observed usage of ECT(0), since the
   value for "minimize monetary cost" was 0001, shifted to the left one
   bit, coinciding with ECT(0).

3.3.3.  Use of the ECN Field Inadvertently

   Users of operating system's socket APIs wishing to set a DiffServ
   codepoint may be confused as to whether or not they need to shift the
   desired value left two bits before passing it in.  Additionally, OS
   header files have been seen with out-of-date definitions for obsolete
   values in the former Type of Service octet, and obsolete definitions
   from [RFC2481].






Heist                    Expires 22 August 2021                 [Page 6]


Internet-Draft                   ecndep                    February 2021


   Another possible source of confusion is the TOS field values listed
   in the now obsolete [RFC1349], without having been shifted.  A casual
   reader could see the value 0001 for "minimize monetary cost" and
   think that they should use this value in the TOS byte, conflicting
   with ECT(1), not realizing that:

   *  [RFC1349] is obsolete

   *  even if it weren't obsolete, the TOS values must be shifted to the
      left *by one bit*

   To reduce incorrect usages of the DS field, OS header files should be
   sanitized, obsolete RFCs more prominently marked as such, and API
   documentation brought up to date.

3.3.4.  Use of the ECN Field Maliciously

   It's possible that some software is using the ECN field to gain an
   advantage in Internet queues or for some other nefarious purpose.
   Further analysis would be needed to determine if this is the case.

4.  Study Limitations and Recommendations for Future Work

4.1.  ECN Acceptance Rate

   While we captured the ratio of ECN SYNs to ECN SYN-ACKs, we do not
   have an exact count of flows that were accepted or rejected.  It may
   be possible to do this more accurately with additional iptables rules
   in [IPTABLES-ECN].

4.2.  Tunnels

   Tunnel protocols are challenging because of the different
   encapsulation methods and protocols used.  An analysis at the flow
   level, rather than by IP address and destination port pairs, might be
   more useful in identifying the usage of ECN over tunnels.

4.3.  Non-TCP Protocols

   More research is needed into the reasons for ECN codepoints being set
   on non-TCP traffic.  Given the relatively low volume of this traffic,
   it might be practical to take packet captures of it for further
   analysis.

   Additionally, we are currently not able to differentiate between the
   total number of packets for conntrack-supported and Other protocols.
   This could be improved with some changes to [IPTABLES-ECN].




Heist                    Expires 22 August 2021                 [Page 7]


Internet-Draft                   ecndep                    February 2021


4.4.  Other Protocols

   While this study looked at signals by IP address for TCP and IP/port
   for conntrack-supported protocols, it does not break down signals for
   Other protocols by IP address.  Among those protocols is IPSec ESP
   packets, using IP protocol 50.  The [IPTABLES-ECN] script could be
   modified to create more ipsets of type hash:ip, parallel to what was
   done for IP traffic as a whole, to further analyze these protocols
   for tunnel activity.

4.5.  NS Flag

   Since [RFC8311] declared that the NS (Nonce Sum) flag is again
   Reserved, after its now historical use by [RFC3540], we could collect
   any observed usages of this flag, to confirm that it's available for
   use in practice.

5.  Abbreviated Output from ecn-stats

   This abbreviated output only includes LAN to WAN flows, and a small
   subset of the non-TCP conntrack protocols by member IP address.  For
   full output, see the [IPTABLES-ECN] repository.

   *Note* the IP addresses shown here have been anonymized within the
   10.0.0.0/8 address space, in a way that retains the subnet structure.

5.1.  All IP
























Heist                    Expires 22 August 2021                 [Page 8]


Internet-Draft                   ecndep                    February 2021


   Packets, CE, ECT(0) and ECT(1) are packet counts, and use
   units of M, G or T for mega, giga, or terapackets.

   Total (both directions):

                  TCP         Conntrack [*]   Other      Total
                  ---         -------------   -----      -----
       Bytes      101.22 TB   ->              33.22 TB   134.46 TB
       Packets    76.60 G     ->              43.52 G    120.14 G
       |-CE       10031       3.38 M          813951     4.20 M
       |-ECT(0)   523.91 M    9.66 M          2.55 M     536.12 M
       |-ECT(1)   63          6.68 M          182928     6.86 M

   WAN to LAN:

                  TCP        Conntrack [*]   Other      Total
                  ---        -------------   -----      -----
       Bytes      95.79 TB   ->              26.65 TB   122.45 TB
       Packets    41.43 G    ->              30.29 G    71.72 G
       |-CE       9298       3.38 M          721002     4.11 M
       |-ECT(0)   480.35 M   9.62 M          1.93 M     491.91 M
       |-ECT(1)   62         6.68 M          65111      6.74 M

   LAN to WAN:

                  TCP       Conntrack [*]   Other     Total
                  ---       -------------   -----     -----
       Bytes      5.43 TB   ->              6.57 TB   12.00 TB
       Packets    35.17 G   ->              13.23 G   48.41 G
       |-CE       733       60              92949     93742
       |-ECT(0)   43.56 M   40366           614623    44.21 M
       |-ECT(1)   1         28              117817    117846

       [*] Conntrack protocols: UDP, ICMP, DCCP, SCTP, GRE
           Conntrack total Bytes and Packets included in Other

5.2.  TCP initiated from LAN to WAN

   IP address counts with TCP and ECN activity:

       Active (sent >= 10 SYNs):  861 (of 1195)
       Initiated any ECN flows:   390 (45.3%)
       Negotiated any ECN flows:  382 (44.4%)
       Saw CE or ECE on ECN flow: 90 (23.6% of ECN, 10.5% of all)
       Saw ECT(1) on ECN flow:    5

   SYN packet count totals for active IPs:




Heist                    Expires 22 August 2021                 [Page 9]


Internet-Draft                   ecndep                    February 2021


       All SYNs:     319560652
       ECN SYNs:     4601118 (1.44% of all)
       ECN SYN/ACKs: 3273815 (71.15% of ECN SYNs)

   ECN packet count totals for active IPs:

       Direction CE   ECE    ECT(0)    ECT(1)
       --------- --   ---    ------    ------
       From LAN  733  502985 42903861  1
       From WAN  9298 19367  479756419 62

   ECN congestion signals by active IP:

       IP            CE from WAN ECE from LAN CE from LAN ECE from WAN
       --            ----------- ------------ ----------- ------------
       10.45.9.88    0           0            0           431
       10.45.64.3    36          13348        0           45
       10.45.64.4    0           2192         0           0
       10.45.64.7    28          4610         0           35
       10.45.64.11   0           335          0           0
       10.45.64.12   0           14955        3           0
       10.45.64.13   0           223          0           0
       10.45.64.14   13          20863        0           23
       10.45.64.15   0           9            0           0
       10.45.64.16   0           1396         0           0
       10.45.64.17   0           464          0           0
       10.45.64.31   0           46740        12          0
       10.45.64.39   0           11019        0           0
       10.45.64.45   0           363          0           0
       10.45.64.47   0           15731        321         6041
       10.45.64.59   0           44           0           0
       10.45.64.85   0           57           0           0
       10.45.64.93   0           16530        0           0
       10.45.64.103  0           10649        0           0
       10.45.64.105  0           2046         0           0
       10.45.64.112  0           1135         1           1
       10.45.64.116  0           1042         0           0
       10.45.64.118  163         710          0           170
       10.45.64.123  0           3118         0           0
       10.45.64.125  0           52960        49          0
       10.45.64.126  0           12579        122         0
       10.45.65.7    0           176          0           0
       10.45.65.16   0           4483         0           0
       10.45.65.110  0           1530         0           0
       10.45.65.112  0           2313         0           0
       10.45.65.124  5           6            0           9
       10.45.86.39   1           13           0           0
       10.45.86.41   72          3228         0           0



Heist                    Expires 22 August 2021                [Page 10]


Internet-Draft                   ecndep                    February 2021


       10.45.87.32   0           64           0           0
       10.45.87.45   1           0            0           0
       10.45.87.50   3           3            0           0
       10.45.87.127  17          22           0           39
       10.45.101.96  155         156          0           151
       10.45.104.24  55          63           0           77
       10.45.107.73  400         416          0           430
       10.45.108.24  0           0            0           36
       10.45.113.6   168         191          0           174
       10.45.113.106 34          37           0           40
       10.45.114.98  1619        1792         0           1739
       10.45.138.66  43          56           0           47
       10.45.140.73  510         551          0           520
       10.45.140.74  39          46           0           38
       10.45.141.85  39          50           0           85
       10.45.145.2   10          15           0           25
       10.45.145.73  1           0            0           0
       10.45.153.10  6           11           0           0
       10.45.154.82  22          25           0           44
       10.45.155.68  1           1            0           0
       10.45.155.71  144         143          1           152
       10.45.158.197 493         53           0           0
       10.45.158.198 13          13           0           25
       10.45.176.114 32          46           0           62
       10.45.176.119 38          47           0           68
       10.45.177.68  22          24           0           27
       10.45.182.75  6           7            0           13
       10.45.183.117 131         145          6           152
       10.45.183.204 8           10           0           0
       10.45.212.82  18          23           0           48
       10.45.229.81  268         2104         1           0
       10.45.230.25  3132        18481        0           0
       10.45.230.204 1           1            0           0
       10.45.231.31  16          9            0           30
       10.45.234.197 188         225          0           153
       10.45.235.6   0           217          0           0
       10.45.235.24  0           388          0           0
       10.45.235.59  16          897          0           30
       10.45.235.89  56          31899        176         5630
       10.45.235.90  727         4278         0           709
       10.45.235.92  151         169965       41          1784
       10.45.235.94  0           1394         0           0
       10.45.235.196 0           157          0           0
       10.45.235.199 0           56           0           0
       10.45.235.200 0           220          0           0
       10.45.235.203 0           234          0           0
       10.45.235.206 0           3484         0           0
       10.45.235.208 0           378          0           0



Heist                    Expires 22 August 2021                [Page 11]


Internet-Draft                   ecndep                    February 2021


       10.45.238.75  196         262          0           229
       10.45.241.101 0           740          0           0
       10.45.242.72  5           5            0           11
       10.45.242.146 21          25           0           44
       10.45.243.69  2           3            0           0
       10.45.249.6   0           2461         0           0
       10.45.249.34  0           2260         0           0
       10.45.251.37  39          171          0           0
       10.45.251.114 134         13794        0           0

5.3.  Non-TCP conntrack-supported protocols initiated from LAN to WAN

   Protocols included:

       UDP, ICMP, DCCP, SCTP, GRE

   Active IPs:

       Active IPs with ECN signals:              420
       Active IP/dstport pairs with ECN signals: 24972

   ECN packet count totals for active IPs:

       Direction CE      ECT(0)  ECT(1)
       --------- --      ------  ------
       From LAN  59      26692   28
       From WAN  2838929 9562002 6632561

   ECN codepoint packet counts by client IP, with selected ports:
       (ports with '*' had >100 ECT(0) marks)

                              ECT(0) CE   ECT(1) ECT(0)  CE      ECT(1)
                              from   from from   from    from    from
       IP/Port                LAN    LAN  LAN    WAN     WAN     WAN
       -------                ---    ---  ---    ---     ---     ---
       10.45.10.0             201    0    0      0       0       0
       10.45.10.4             14     0    0      0       0       0
       10.45.10.5             20     0    0      0       0       0
       10.45.10.6             9      0    0      0       0       0
       10.45.10.7             8      0    0      0       0       0
       10.45.10.8             39     0    0      0       0       0
       10.45.10.11            8      0    0      0       0       0
       10.45.10.12            2      0    0      0       0       0
       10.45.10.42            6      0    0      0       0       0
       10.45.10.61            2      0    0      0       0       0
       10.45.10.70            44     0    0      0       0       0
       10.45.10.71            5      0    0      0       0       0
       10.45.10.73            7      0    0      0       0       0



Heist                    Expires 22 August 2021                [Page 12]


Internet-Draft                   ecndep                    February 2021


       10.45.10.77            13     0    0      0       0       0
       10.45.10.81            10     0    0      0       0       0
       10.45.10.82            8      0    0      0       0       0
       10.45.10.83            3      0    0      0       0       0
       10.45.10.95            59     0    0      0       0       0
       10.45.10.96            39     0    0      0       0       0
       10.45.10.129           0      0    0      0       403     1
       10.45.10.196           80     0    0      0       0       0
       10.45.10.197           63     0    0      0       0       0
       10.45.10.201           3      0    0      0       0       0
       10.45.10.204           25     0    0      0       0       0
       10.45.10.227           40     0    0      0       0       0
       10.45.10.228           7      0    0      0       0       0
       10.45.10.244           14     0    0      0       0       0
       10.45.10.245           7      0    0      0       0       0
       10.45.64.3             100    0    0      0       0       0
       10.45.64.4             31     0    0      0       0       0
       10.45.64.6             2      0    0      0       0       0
       10.45.64.7             8      0    0      12      126     20
       10.45.64.10            29     0    0      0       0       0
       10.45.64.11            67     0    0      0       0       0
       10.45.64.12            6      0    0      0       0       0
       10.45.64.13            35     0    0      0       0       0
       10.45.64.14            121    0    0      0       0       0
       10.45.64.15            52     0    0      0       0       0
       10.45.64.16            18     0    0      0       0       0
       10.45.64.19            0      0    0      16      0       0
         udp:4500 (ipsec-na.. 0      0    0      11      0       0
       10.45.64.31            27     0    0      34129   2468    58304
         udp:37658            0      0    0      0       0       4346
       * udp:38129            0      0    0      24957   2468    15281
         udp:38884            0      0    0      0       0       10409
       * udp:40871            0      0    0      288     0       2269
       * udp:41621            0      0    0      3057    0       14609
       * udp:41744            0      0    0      171     0       61
         udp:43588            0      0    0      0       0       6746
         udp:45444            0      0    0      0       0       1292
       * udp:45465            0      0    0      866     0       0
         udp:45483            0      0    0      0       0       1838
       * udp:45522            0      0    0      4764    0       708
       10.45.64.39            75     0    0      0       0       0
       10.45.64.45            50     0    0      0       0       0
       10.45.64.47            11     0    0      0       0       0
       10.45.64.51            2      0    0      0       0       0
       10.45.64.59            593    0    0      56      1624    10
         udp:3478 (stun)      0      0    0      56      1624    10
       10.45.64.85            4      0    0      0       0       0
       10.45.64.86            9      0    0      7       434404  3



Heist                    Expires 22 August 2021                [Page 13]


Internet-Draft                   ecndep                    February 2021


         udp:4400 (ds-srv)    0      0    0      0       29065   0
         udp:14757            0      0    0      0       97175   0
         udp:24173            0      0    0      0       35437   0
         udp:29493            0      0    0      0       120959  0
         udp:44495            0      0    0      0       41547   0
         udp:53678            0      0    0      0       109978  0
       10.45.64.89            4      0    0      7       50      0
       10.45.64.93            75     0    0      598     2971    341
       * udp:3478 (stun)      0      0    0      598     2971    341
       10.45.64.98            0      0    0      0       0       32780
         udp:6008             0      0    0      0       0       9234
         udp:7008 (afs3-upd.. 0      0    0      0       0       23546
       10.45.64.99            0      0    0      132     2094    73
         udp:3478 (stun)      0      0    0      0       3       0
       10.45.64.103           47     0    0      0       0       0
       10.45.64.104           1      0    0      70      293     31
       10.45.64.105           7      0    0      213     33440   0
       * udp:443 (https)      0      0    0      213     33440   0
       10.45.64.107           2      0    0      0       0       0
       10.45.64.108           1      0    0      0       0       0
       10.45.64.111           0      0    0      1       1       0
       10.45.64.112           48     0    0      0       421     0
       10.45.64.116           64     0    8      4       143     8
       10.45.64.118           77     0    0      0       0       0
       10.45.64.121           0      0    0      0       2107    0
         udp:38603            0      0    0      0       2100    0
       10.45.64.123           13     0    0      0       0       0
       10.45.64.124           0      0    0      6       0       0
         udp:443 (https)      0      0    0      6       0       0
       10.45.64.125           22     0    0      0       0       0
       10.45.64.126           37     0    0      1       10      0
       10.45.65.0             42     0    0      0       0       0
       10.45.65.1             45     0    0      0       0       0
       10.45.65.5             17     0    0      0       0       0
       10.45.65.7             30     0    0      0       0       0
       10.45.65.11            6      0    0      0       0       0
       10.45.65.16            505    0    0      1686    40141   36888
       * udp:3478 (stun)      0      0    0      1595    22049   4
         udp:26808            0      0    0      0       0       36805
         udp:62348            0      0    0      0       15738   0
       10.45.65.17            0      0    0      0       4       0
       10.45.65.66            94     0    0      0       17      0
         udp:3478 (stun)      0      0    0      0       17      0
       10.45.65.94            25     0    0      319     0       1
         udp:3478 (stun)      0      0    0      0       0       1
       10.45.65.95            8      0    0      0       0       0
       10.45.65.104           41     0    0      0       0       0
       10.45.65.107           5      0    0      12      77      2



Heist                    Expires 22 August 2021                [Page 14]


Internet-Draft                   ecndep                    February 2021


       10.45.65.110           38     0    0      0       0       0
       10.45.65.112           75     0    0      39      1168    18
       10.45.65.122           0      0    0      2       5       0
       10.45.65.123           1      0    0      0       0       0
       10.45.65.124           11     0    0      0       0       0
       10.45.65.127           5      0    0      0       0       0
       10.45.75.90            1      0    0      0       0       0
       10.45.80.28            0      0    0      2       8       1
       10.45.80.79            2      0    0      4       7       0
       10.45.80.85            10     0    0      0       0       0
       10.45.80.99            11     0    0      0       0       0
       10.45.83.76            3      0    0      0       0       0
       10.45.83.80            0      0    0      28      51      11
       10.45.85.127           68     0    0      301     174     30747
       * udp:599 (acp)        0      0    0      222     174     45
         udp:6008             0      0    0      0       0       30702
       * udp:60001            65     0    0      49      0       0
       10.45.86.16            2      0    0      13      0       0
         udp:4500 (ipsec-na.. 0      0    0      8       0       0
         udp:51820 (wiregua.. 0      0    0      5       0       0
       10.45.86.36            4      0    0      0       0       0
       10.45.86.39            50     0    0      205     37619   107
         udp:29492            0      0    0      0       2512    0
         udp:64733            0      0    0      0       30711   0
       10.45.86.40            0      0    0      2       0       0
         udp:443 (https)      0      0    0      2       0       0
       10.45.86.43            532    0    0      0       11      0
       10.45.86.68            325    0    0      760     3528    614
         udp:80 (http)        0      0    0      0       2       0
       10.45.87.32            14     0    0      12      0       0
       10.45.87.44            0      0    0      709     4963    623
         udp:80 (http)        0      0    0      0       1       0
         udp:6881             0      0    0      3       1313    43
       10.45.87.45            185    0    0      0       0       0
       10.45.87.48            82     0    0      0       0       0
       10.45.87.50            68     0    0      3       0       9
         udp:4500 (ipsec-na.. 0      0    0      3       0       9
       10.45.87.103           2      0    0      0       0       0
       10.45.87.112           0      0    0      0       1       0
       10.45.87.113           33     0    0      0       0       0
       10.45.87.127           44     0    0      0       0       0
       10.45.92.74            2      0    0      31      0       1
       10.45.93.69            0      0    0      15      122     6
       10.45.93.75            4      0    0      361     2945    278
       10.45.93.79            8      0    0      0       0       0
       10.45.98.71            0      0    0      2       8       0
       10.45.98.72            40     0    0      0       1       0
         udp:3478 (stun)      0      0    0      0       1       0



Heist                    Expires 22 August 2021                [Page 15]


Internet-Draft                   ecndep                    February 2021


       10.45.101.96           140    0    0      0       0       0
       10.45.101.100          12     0    0      0       0       0
       10.45.101.101          0      0    0      2       10      7
       10.45.101.103          0      0    0      21      21899   15
         udp:58479            0      0    0      0       21372   0
       10.45.101.104          33     0    0      0       0       10
       10.45.104.24           324    0    0      0       0       0
       10.45.104.104          60     0    0      16      72      2
       10.45.107.73           58     0    0      32      0       1
         udp:4500 (ipsec-na.. 0      0    0      32      0       1
       10.45.107.79           70     0    0      34      0       0
         udp:443 (https)      0      0    0      34      0       0
       10.45.107.81           3      0    0      0       4421    0
         udp:61094            0      0    0      0       4421    0
       10.45.108.3            1      0    0      0       0       0
       10.45.108.4            1      0    0      33      5079    90
         udp:33027            0      0    0      0       2978    0
       10.45.108.13           14     0    0      0       0       0
       10.45.108.24           117    0    0      799     5543    1059
       * udp:40211            0      0    0      107     0       0
       10.45.108.25           799    0    0      1       2       1
       10.45.108.66           0      0    1      0       0       0
       10.45.108.69           2      0    0      0       0       0
       10.45.108.71           0      0    0      28      12830   0
         udp:34665            0      0    0      0       12462   0
       10.45.108.75           38     0    0      0       0       6395176
         udp:6008             0      0    0      0       0       1755476
         udp:7008 (afs3-upd.. 0      0    0      0       0       1827173
         udp:8008 (http-alt)  0      0    0      0       0       740987
         udp:9008             0      0    0      0       0       809024
         udp:10008 (octopus)  0      0    0      0       0       380001
         udp:11008            0      0    0      0       0       578400
         udp:12008 (accurac.. 0      0    0      0       0       231619
         udp:13008            0      0    0      0       0       72496
       10.45.108.76           2      0    0      0       0       0
       10.45.108.77           31     0    0      0       0       0
       10.45.108.80           10     0    0      337     1566    173
       10.45.108.95           283    0    0      1       5       0
       10.45.108.126          12     0    0      0       0       0
       10.45.112.74           371    0    0      9       95      4
       10.45.112.102          29     0    0      0       0       0
       10.45.112.139          5      0    0      0       0       0
       10.45.112.154          4      0    0      0       0       0
       10.45.112.165          24     0    0      0       0       0
       10.45.112.172          0      0    0      6333    0       0
       * udp:443 (https)      0      0    0      6333    0       0
       10.45.112.216          2      0    0      0       0       0
       10.45.113.6            136    0    0      147184  0       0



Heist                    Expires 22 August 2021                [Page 16]


Internet-Draft                   ecndep                    February 2021


       * udp:4500 (ipsec-na.. 0      0    0      147184  0       0
       10.45.113.7            52     0    0      453     0       10
       * udp:443 (https)      0      0    0      309     0       0
       * udp:4500 (ipsec-na.. 0      0    0      144     0       10
       10.45.113.9            60     0    0      0       0       0
       10.45.113.11           187    0    0      0       0       0
       10.45.113.27           1      0    0      0       0       0
       10.45.113.30           4      0    0      0       0       0
       10.45.113.33           2      0    0      0       0       0
       10.45.113.34           58     0    0      0       0       0
       10.45.113.35           6      0    0      0       0       0
       10.45.113.36           2      0    0      0       0       0
       10.45.113.66           0      0    0      1       11      0
       10.45.113.90           163    0    0      0       0       0
       10.45.113.94           0      0    0      17      62      2
       10.45.113.97           19     0    0      0       0       0
       10.45.113.99           15     0    0      11      76      12
       10.45.113.104          0      0    0      818     0       0
       * udp:4500 (ipsec-na.. 0      0    0      818     0       0
       10.45.113.106          10     0    0      0       0       0
       10.45.113.119          313    0    0      0       178     0
         udp:3478 (stun)      0      0    0      0       178     0
       10.45.113.122          0      0    0      36      0       0
         udp:4500 (ipsec-na.. 0      0    0      36      0       0
       10.45.113.124          201    0    0      0       0       0
       10.45.114.8            0      0    0      0       3       0
       10.45.114.10           3      0    0      0       0       0
       10.45.114.42           3      0    0      286     12      67
       * udp:51820 (wiregua.. 0      0    0      286     0       66
       10.45.114.98           10     0    0      0       0       0
       10.45.120.25           53     0    0      0       0       0
       10.45.120.34           12     0    0      0       0       0
       10.45.120.78           715    0    0      0       0       0
       10.45.122.51           66     0    0      686     28190   122
         udp:45622            0      0    0      0       5782    0
         udp:59437            0      0    0      0       17791   0
       10.45.124.31           105    0    0      1720    5946    16897
         udp:3478 (stun)      0      0    0      0       6       0
       * udp:50451            0      0    0      1720    0       15875
         udp:50919            0      0    0      0       2428    0
         udp:50996            0      0    0      0       0       1016
         udp:57403            0      0    0      0       1944    0
       10.45.124.43           12     0    0      0       0       0
       10.45.124.73           0      0    0      37      0       0
         udp:4500 (ipsec-na.. 0      0    0      37      0       0
       10.45.124.74           1      0    0      0       0       0
       10.45.124.89           2      0    0      0       0       0
       10.45.124.107          0      0    0      142     626895  83



Heist                    Expires 22 August 2021                [Page 17]


Internet-Draft                   ecndep                    February 2021


         udp:24616            0      0    0      0       501142  0
         udp:51123            0      0    0      0       124060  0
       10.45.124.111          0      0    0      0       1538    166
         udp:4748             0      0    0      0       1491    166
       10.45.124.117          248    0    0      0       0       0
       10.45.125.97           2      0    0      0       0       0
       10.45.125.99           1      0    0      130     6235    29
         udp:8609 (canon-cp.. 0      0    0      0       3002    0
       10.45.125.104          3      0    0      0       0       0
       10.45.125.105          7      0    0      0       0       0
       10.45.136.82           1      0    0      0       0       0
       10.45.136.198          8      0    0      0       0       0
       10.45.136.199          0      0    0      68      3210    7
         udp:22312            0      0    0      0       2452    0
       10.45.136.200          0      0    0      0       44      1
       10.45.137.4            1882   0    0      4603    0       0
       * udp:443 (https)      1882   0    0      4603    0       0
       10.45.137.21           118    0    0      0       0       0
       10.45.137.27           63     0    0      4       0       0
       10.45.137.29           0      0    1      0       0       0
       10.45.137.46           6      0    0      9       154     0
         udp:443 (https)      0      0    0      9       0       0
       10.45.137.53           7      0    0      0       0       0
       10.45.137.55           37     0    0      0       0       1
       10.45.137.62           14     0    0      5       29      1
         udp:443 (https)      0      0    0      2       0       0
       10.45.137.119          4      0    0      16      203825  12
         udp:16772            0      0    0      0       55846   0
         udp:25135            0      0    0      0       24694   0
         udp:25476            0      0    0      0       66965   0
         udp:51123            0      0    0      0       54265   0
         udp:55430            0      0    0      0       1138    0
       10.45.137.123          1      0    0      2       4190    1
         udp:29363            0      0    0      0       3283    0
       10.45.138.52           0      0    0      3093    18938   0
       * udp:42420            0      0    0      3087    18871   0
       10.45.138.66           249    0    0      0       0       0
       10.45.138.88           0      0    0      43      107     10
       10.45.138.95           20     0    0      0       0       0
       10.45.140.0            84     0    0      0       0       0
       10.45.140.5            2      0    0      0       0       0
       10.45.140.28           1      0    0      0       0       0
       10.45.140.74           12     0    0      0       0       0
       10.45.140.81           26     0    0      0       0       0
       10.45.140.100          0      0    0      143     465     37
       10.45.140.103          16     0    0      0       0       0
       10.45.140.104          4      0    0      0       0       0
       10.45.140.109          2      0    0      0       0       0



Heist                    Expires 22 August 2021                [Page 18]


Internet-Draft                   ecndep                    February 2021


       10.45.140.118          27     0    0      0       0       0
       10.45.140.121          17     0    0      0       7032    0
         udp:49710            0      0    0      0       1160    0
         udp:53984            0      0    0      0       2694    0
         udp:58704            0      0    0      0       1597    0
       10.45.140.122          0      0    0      0       3       0
       10.45.140.123          0      0    0      0       0       4
       10.45.140.127          15     0    0      0       0       0
       10.45.140.133          0      1    0      0       0       0
       10.45.140.169          59     0    0      0       0       0
       10.45.140.171          14     0    0      0       0       0
       10.45.141.2            12     0    0      91      0       0
         udp:443 (https)      0      0    0      91      0       0
       10.45.141.6            24     0    0      0       0       0
       10.45.141.14           2      0    0      0       0       0
       10.45.141.17           17     0    0      2       37      1
       10.45.141.19           2      0    0      0       0       0
       10.45.141.82           21     0    0      579     0       0
       * udp:443 (https)      0      0    0      579     0       0
       10.45.141.83           14     0    0      0       0       0
       10.45.141.84           90     0    0      0       0       0
       10.45.141.85           518    0    0      0       0       0
       10.45.141.86           6      0    0      0       0       0
       10.45.141.87           2      0    0      0       0       0
       10.45.141.103          57     0    0      0       0       0
       10.45.141.106          1079   0    0      7       190     3947
         udp:3478 (stun)      0      0    0      0       24      12
       * udp:5001 (commplex.. 1072   0    0      0       0       0
         udp:40208            0      0    0      0       0       3932
       10.45.141.125          2      0    0      0       0       0
       10.45.144.20           1      0    0      2       6       2
       10.45.144.43           3      0    0      0       0       0
       10.45.144.55           2      0    0      0       0       0
       10.45.144.68           363    0    0      0       0       0
       10.45.144.73           14     0    0      0       0       0
       10.45.144.75           51     0    0      0       0       3
       10.45.144.77           24     0    0      51      289     35
       10.45.144.105          1      0    0      413     0       11
       * udp:4500 (ipsec-na.. 0      0    0      413     0       11
       10.45.144.139          0      0    0      1496    0       0
       * udp:443 (https)      0      0    0      1496    0       0
       10.45.144.197          102    0    0      0       0       0
       10.45.145.2            15     0    0      0       0       0
       10.45.145.26           44     0    0      0       0       0
       10.45.145.39           11     0    0      2503039 0       0
         udp:443 (https)      0      0    0      4       0       0
       * udp:4500 (ipsec-na.. 0      0    0      2503035 0       0
       10.45.145.56           3      0    0      0       0       0



Heist                    Expires 22 August 2021                [Page 19]


Internet-Draft                   ecndep                    February 2021


       10.45.145.72           32     0    0      0       0       0
       10.45.145.75           0      0    0      3024    0       0
       * udp:443 (https)      0      0    0      3024    0       0
       10.45.145.81           292    0    0      8691    107114  8245
         udp:80 (http)        0      0    0      0       2       0
       * udp:6881             0      0    0      355     8092    672
         udp:19517            0      0    0      0       1097    0
         udp:22784            0      0    0      0       3441    0
       * udp:25223            0      0    0      110     0       0
       * udp:37526            0      0    0      139     0       0
       * udp:40631            0      0    0      191     0       0
         udp:40990            0      0    0      0       33415   0
         udp:51820 (wiregua.. 0      0    0      0       3       0
       10.45.145.96           7      0    0      0       0       0
       10.45.145.98           3      0    0      0       0       0
       10.45.145.107          0      0    0      0       9       0
       10.45.145.109          9      35   0      0       0       0
       10.45.145.115          11     0    0      0       0       0
       10.45.146.66           26     0    0      52      88      7
       10.45.146.195          2      0    0      0       0       0
       10.45.146.200          49     0    0      1471    0       0
       * udp:4500 (ipsec-na.. 0      0    0      1471    0       0
       10.45.146.201          9      0    0      0       0       0
       10.45.153.10           33     0    0      0       0       0
       10.45.153.194          0      0    0      2       86      2
       10.45.154.6            9      0    0      0       0       0
       10.45.154.81           4      0    0      0       0       0
       10.45.154.82           140    0    0      0       0       0
       10.45.154.100          14     0    0      0       0       0
       10.45.154.105          17     0    0      0       0       0
       10.45.154.112          5      0    0      0       0       0
       10.45.154.113          3      0    0      1       88      2
       10.45.154.115          224    0    0      0       0       0
       10.45.155.12           11     0    0      0       0       0
       10.45.155.67           1      0    0      0       0       0
       10.45.155.68           237    0    0      0       0       0
       10.45.155.69           1      0    0      0       0       0
       10.45.155.71           246    0    0      0       0       0
       10.45.155.73           72     0    0      0       0       0
       10.45.155.74           0      0    0      0       1       0
         udp:3478 (stun)      0      0    0      0       1       0
       10.45.155.75           0      0    0      0       4       0
       10.45.155.76           0      0    0      0       1       0
       10.45.155.217          15     0    0      0       0       0
       10.45.155.229          48     0    0      4       42      6
       10.45.156.94           0      0    0      25      152     8
       10.45.156.105          19     0    0      0       5362    0
         udp:58796            0      0    0      0       5362    0



Heist                    Expires 22 August 2021                [Page 20]


Internet-Draft                   ecndep                    February 2021


       10.45.156.127          22     0    0      0       0       0
       10.45.158.115          402    0    0      0       0       0
       10.45.158.124          4      0    0      0       0       0
       10.45.158.127          3      0    0      0       0       0
       10.45.158.195          25     0    0      0       1630    3
         udp:6881             0      0    0      0       1610    0
       10.45.158.197          82     0    0      0       0       0
       10.45.158.198          204    0    0      0       0       0
       10.45.158.204          118    0    0      0       0       0
       10.45.158.206          0      0    0      9       32      2
       10.45.176.114          68     0    0      0       0       0
       10.45.176.116          1      0    0      188     1702    191
       10.45.176.117          35     0    0      0       0       0
       10.45.176.119          218    0    0      9320    1028270 11302
         udp:6881             0      0    0      0       91498   83
       * udp:6900             0      0    0      322     0       0
         udp:8999 (bctp)      0      0    0      0       405853  3
       * udp:10556            0      0    0      741     0       0
         udp:11778            0      0    0      0       311705  0
       * udp:12111            0      0    0      274     0       0
         udp:21606            0      0    0      0       5678    0
         udp:23578            0      0    0      0       4281    0
         udp:24488            0      0    0      0       2140    0
         udp:35849            0      0    0      0       2632    0
       * udp:37758            0      0    0      212     721     0
         udp:40954            0      0    0      0       27113   0
       * udp:42012            0      0    0      380     26      101
         udp:48235            0      0    0      0       3182    0
       * udp:50321            0      0    0      2066    14226   5982
       * udp:50838            0      0    0      389     0       0
         udp:50884            0      0    0      0       0       2743
         udp:51413            0      0    0      39      1712    0
         udp:54457            0      0    0      0       3504    0
         udp:56769            0      0    0      0       23761   0
         udp:59025            0      0    0      0       3034    0
       * udp:60050            0      0    0      3000    3961    1478
         udp:60062            0      0    0      0       13672   0
         udp:64329            0      0    0      0       75590   0
       10.45.176.120          73     21   18     0       0       0
       10.45.176.206          34     0    0      37      689     3
         udp:3478 (stun)      0      0    0      37      685     3
       10.45.176.207          5      0    0      8       143     0
       10.45.176.209          11     0    0      12      88      1
       10.45.176.210          1      0    0      10      32      4
       10.45.176.214          18     0    0      25      8900    0
         udp:6672 (vision-s.. 0      0    0      23      8900    0
       10.45.176.224          114    0    0      1       0       0
       10.45.176.225          1      0    0      120     786     137



Heist                    Expires 22 August 2021                [Page 21]


Internet-Draft                   ecndep                    February 2021


       10.45.176.226          4      0    0      0       0       0
       10.45.176.237          0      0    0      4       0       0
         udp:443 (https)      0      0    0      4       0       0
       10.45.177.66           0      0    0      9       213     8349
         udp:6672 (vision-s.. 0      0    0      0       0       8334
       10.45.177.68           124    0    0      12      64      8
       10.45.177.75           66     0    0      0       2       0
       10.45.177.197          0      0    0      0       2       1
       10.45.182.75           25     0    0      44      71      17
       10.45.182.85           0      0    0      41      2612    5024
         udp:45864            0      0    0      0       0       4985
       10.45.182.136          8      0    0      0       0       0
       10.45.183.117          15     0    0      0       0       0
       10.45.183.199          8      0    0      45      1579    0
         udp:3478 (stun)      0      0    0      45      1578    0
       10.45.183.204          731    0    0      0       9478    0
       * udp:4500 (ipsec-na.. 237    0    0      0       0       0
         udp:22885            0      0    0      0       9404    0
       10.45.183.205          3      0    0      0       0       1
         udp:4500 (ipsec-na.. 0      0    0      0       0       1
       10.45.183.209          280    0    0      3       1       0
       10.45.183.219          61     0    0      0       0       0
       10.45.203.6            2      0    0      0       0       0
       10.45.212.17           0      0    0      10472   25127   16430
       * udp:62503            0      0    0      10452   23528   16423
       10.45.212.27           1      0    0      0       0       0
       10.45.212.29           30     0    0      0       0       0
       10.45.212.51           2      0    0      0       0       0
       10.45.212.82           28     0    0      0       1       1
       10.45.212.84           2      0    0      0       0       0
       10.45.212.199          1      0    0      0       0       0
       10.45.212.202          4      0    0      0       0       0
       10.45.212.205          299    0    0      0       0       0
       10.45.212.207          85     0    0      0       0       0
       10.45.229.75           0      0    0      3       0       0
         udp:443 (https)      0      0    0      3       0       0
       10.45.229.78           113    0    0      6694314 0       0
       * udp:4500 (ipsec-na.. 0      0    0      6694314 0       0
       10.45.229.79           27     0    0      0       0       0
       10.45.229.81           3      0    0      0       0       0
       10.45.229.101          69     0    0      0       0       0
       10.45.229.104          0      0    0      128     525     128
       10.45.229.119          20     0    0      0       0       0
       10.45.230.20           1      0    0      0       0       0
       10.45.230.25           32     0    0      10      0       72
         udp:4500 (ipsec-na.. 0      0    0      10      0       72
       10.45.230.89           4      0    0      495     3537    296
       10.45.230.99           2      0    0      7       0       5



Heist                    Expires 22 August 2021                [Page 22]


Internet-Draft                   ecndep                    February 2021


         udp:4500 (ipsec-na.. 0      0    0      7       0       5
       10.45.230.204          110    0    0      9       57      18
       10.45.230.207          1      0    0      18      33      1
       10.45.230.212          2      0    0      0       0       0
       10.45.230.223          3      0    0      0       0       0
       10.45.230.224          0      0    0      27927   93      13
       * udp:50323            0      0    0      322     0       0
       * udp:50361            0      0    0      128     0       0
       * udp:52065            0      0    0      409     0       0
       * udp:55236            0      0    0      257     0       0
       * udp:57072            0      0    0      142     0       0
       * udp:58494            0      0    0      170     0       0
       * udp:59465            0      0    0      160     0       0
       * udp:59659            0      0    0      445     0       0
       * udp:60874            0      0    0      129     0       0
       * udp:60898            0      0    0      102     0       0
       * udp:61122            0      0    0      302     0       0
       * udp:61312            0      0    0      137     0       0
       * udp:61669            0      0    0      124     0       0
       * udp:62889            0      0    0      24738   0       0
       * udp:63354            0      0    0      122     0       0
       * udp:63474            0      0    0      107     0       0
       10.45.230.226          3      0    0      0       0       0
       10.45.230.228          0      1    0      0       45      0
       10.45.230.229          0      0    0      682     21      3
       * udp:4500 (ipsec-na.. 0      0    0      682     0       0
       10.45.231.16           24     0    0      433     0       0
       * udp:4500 (ipsec-na.. 0      0    0      433     0       0
       10.45.231.21           0      0    0      40      256     81
       10.45.231.31           32     0    0      0       0       0
       10.45.231.53           0      0    0      2       46      0
       10.45.231.61           13     0    0      4151    0       1
       * udp:4500 (ipsec-na.. 0      0    0      4151    0       1
       10.45.231.80           6      0    0      0       0       0
       10.45.231.99           40     0    0      0       0       0
       10.45.231.102          11     0    0      0       0       0
       10.45.231.114          47     0    0      0       0       0
       10.45.233.16           55     0    0      0       0       0
       10.45.233.39           1      0    0      1       13      2
       10.45.233.41           4      0    0      0       0       3
       10.45.233.42           115    0    0      0       0       0
       10.45.233.47           1      0    0      0       0       0
       10.45.233.55           3      0    0      0       0       0
       10.45.234.197          2      0    0      320     0       11
       * udp:4500 (ipsec-na.. 0      0    0      320     0       11
       10.45.235.6            6      0    0      107     454     62
       10.45.235.11           0      0    0      250     0       0
       * udp:443 (https)      0      0    0      249     0       0



Heist                    Expires 22 August 2021                [Page 23]


Internet-Draft                   ecndep                    February 2021


       10.45.235.13           4      0    0      0       0       0
       10.45.235.16           0      0    0      24      56      3
       10.45.235.19           3      0    0      0       0       0
       10.45.235.24           33     0    0      0       2       0
       10.45.235.25           17     0    0      2310    28152   68
       * udp:443 (https)      0      0    0      2214    0       0
         udp:6881             0      0    0      0       13339   0
         udp:31708            0      0    0      0       4595    0
         udp:51413            0      0    0      0       5367    0
         udp:52372            0      0    0      0       3975    0
       10.45.235.49           0      0    0      672     3165    14
       * udp:443 (https)      0      0    0      672     79      0
         udp:59418            0      0    0      0       3078    0
       10.45.235.52           0      0    0      23      0       0
         udp:4500 (ipsec-na.. 0      0    0      23      0       0
       10.45.235.59           58     0    0      0       0       0
       10.45.235.66           4      0    0      0       0       0
       10.45.235.89           582    0    0      165     2580    23
       * udp:3478 (stun)      0      0    0      165     2580    23
       10.45.235.90           332    0    0      0       0       0
       10.45.235.92           1007   0    0      0       0       0
       10.45.235.93           13     0    0      229     3272    306
       10.45.235.94           10     0    0      0       0       0
       10.45.238.75           1744   0    0      0       0       0
       10.45.238.104          7      0    0      0       2576    0
         udp:443 (https)      0      0    0      0       2576    0
       10.45.239.66           0      0    0      40      0       5
         udp:4500 (ipsec-na.. 0      0    0      40      0       5
       10.45.239.219          18     0    0      1       0       0
         udp:443 (https)      0      0    0      1       0       0
       10.45.240.86           5      0    0      0       0       0
       10.45.241.57           0      0    0      216     66079   437
         udp:4500 (ipsec-na.. 0      0    0      0       0       21
         udp:33522            0      0    0      0       37844   0
         udp:37859            0      0    0      0       27536   0
       10.45.241.94           44     0    0      0       0       0
       10.45.241.98           4      0    0      0       0       0
       10.45.241.101          120    0    0      68946   10      2
       * udp:4500 (ipsec-na.. 0      0    0      68942   0       0
       10.45.241.121          2      0    0      0       0       0
       10.45.242.72           4      0    0      0       0       0
       10.45.242.81           14     1    0      0       0       0
       10.45.242.144          5      0    0      0       0       0
       10.45.242.146          30     0    0      0       0       0
       10.45.242.161          139    0    0      143     134     2297
       * udp:4500 (ipsec-na.. 0      0    0      115     0       4
         udp:27032            0      0    0      0       78      2293
       10.45.243.13           0      0    0      13877   63      1



Heist                    Expires 22 August 2021                [Page 24]


Internet-Draft                   ecndep                    February 2021


       * udp:20911            0      0    0      13853   0       0
       10.45.243.41           14     0    0      12      0       0
         udp:443 (https)      0      0    0      12      0       0
       10.45.243.69           66     0    0      0       0       0
       10.45.243.71           2      0    0      0       28      0
         udp:80 (http)        0      0    0      0       28      0
       10.45.243.109          7      0    0      0       2008    0
         udp:41697            0      0    0      0       2002    0
       10.45.248.33           10     0    0      2       8       0
         udp:3478 (stun)      0      0    0      2       8       0
       10.45.248.94           11     0    0      0       0       0
       10.45.248.118          2      0    0      0       0       0
       10.45.249.6            1502   0    0      0       0       0
       10.45.249.34           154    0    0      25      0       0
         udp:443 (https)      0      0    0      25      0       0
       10.45.249.99           0      0    0      68      558     88
       10.45.249.104          7      0    0      0       0       0
         udp:4500 (ipsec-na.. 6      0    0      0       0       0
       10.45.250.89           5      0    0      0       0       0
       10.45.251.37           19     0    0      0       0       0
       10.45.251.110          0      0    0      9       72      1
       10.45.251.119          23     0    0      0       0       0
       10.45.253.59           1      0    0      0       0       0
         udp:4500 (ipsec-na.. 1      0    0      0       0       0
       10.45.253.61           53     0    0      0       0       0
       10.45.253.84           16     0    0      121     0       0
       * udp:443 (https)      0      0    0      121     0       0
       10.45.253.93           4      0    0      0       0       0
       10.45.253.100          142    0    0      0       0       0
       10.45.253.121          2      0    0      0       0       0
       10.45.254.94           12     0    0      0       0       0
       10.45.255.90           0      0    0      1       125     0
       10.45.255.97           36     0    0      0       0       0

   ECN codepoint packet counts for selected ports:

                              ECT(0) CE   ECT(1) ECT(0)  CE      ECT(1)
                              from   from from   from    from    from
       Port                   LAN    LAN  LAN    WAN     WAN     WAN
       ----                   ---    ---  ---    ---     ---     ---
       icmp:port-unreachable  404    0    0      6632    40795   3539
       icmp:network-unreach.. 0      0    0      321     4       0
       icmp:ttl-zero-during.. 0      0    0      65      2       66
       icmp:host-unreachable  22990  0    0      1171    2575    43
       ipencap:0              1      0    0      0       0       0
       udp:53 (domain)        0      0    0      0       403     1
       udp:80 (http)          0      0    0      0       33      0
       udp:443 (https)        1882   0    0      20006   36095   0



Heist                    Expires 22 August 2021                [Page 25]


Internet-Draft                   ecndep                    February 2021


       udp:599 (acp)          0      0    0      238     261     59
       udp:1024-3457 [81]     34     59   28     100     618     9
       udp:3478 (stun)        0      0    0      2498    31725   394
       udp:3553-4492 [19]     0      0    0      1       29449   0
       udp:4500 (ipsec-nat-t) 244    0    0      9422229 0       151
       udp:4548-51819 [8177]  1072   0    0      62692   2291117 6604184
       udp:51820 (wireguard)  0      0    0      291     3       66
       udp:51821-65535 [9371] 65     0    0      45758   405849  24049

6.  IANA Considerations

   This document has no IANA actions.

7.  Security Considerations

   There are no known security considerations introduced by this note.

8.  Acknowledgements

   Thanks go to:

   *  Adam Pribyl, for gathering data at the FreeNet gateway

   *  Jonathan Morton and Rodney Grimes, for helping to analyze the
      results

   *  FreeNet Liberec, for allowing access for data collection

9.  Informative References

   [IPTABLES-ECN]
              Heist, P.G., "iptables-ecn GitHub Repository",
              <https://github.com/heistp/iptables-ecn/>.

   [RFC1349]  Almquist, P., "Type of Service in the Internet Protocol
              Suite", RFC 1349, DOI 10.17487/RFC1349, July 1992,
              <https://www.rfc-editor.org/info/rfc1349>.

   [RFC2481]  Ramakrishnan, K. and S. Floyd, "A Proposal to add Explicit
              Congestion Notification (ECN) to IP", RFC 2481,
              DOI 10.17487/RFC2481, January 1999,
              <https://www.rfc-editor.org/info/rfc2481>.

   [RFC3168]  Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
              of Explicit Congestion Notification (ECN) to IP",
              RFC 3168, DOI 10.17487/RFC3168, September 2001,
              <https://www.rfc-editor.org/info/rfc3168>.




Heist                    Expires 22 August 2021                [Page 26]


Internet-Draft                   ecndep                    February 2021


   [RFC3540]  Spring, N., Wetherall, D., and D. Ely, "Robust Explicit
              Congestion Notification (ECN) Signaling with Nonces",
              RFC 3540, DOI 10.17487/RFC3540, June 2003,
              <https://www.rfc-editor.org/info/rfc3540>.

   [RFC3948]  Huttunen, A., Swander, B., Volpe, V., DiBurro, L., and M.
              Stenberg, "UDP Encapsulation of IPsec ESP Packets",
              RFC 3948, DOI 10.17487/RFC3948, January 2005,
              <https://www.rfc-editor.org/info/rfc3948>.

   [RFC6040]  Briscoe, B., "Tunnelling of Explicit Congestion
              Notification", RFC 6040, DOI 10.17487/RFC6040, November
              2010, <https://www.rfc-editor.org/info/rfc6040>.

   [RFC8290]  Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys,
              J., and E. Dumazet, "The Flow Queue CoDel Packet Scheduler
              and Active Queue Management Algorithm", RFC 8290,
              DOI 10.17487/RFC8290, January 2018,
              <https://www.rfc-editor.org/info/rfc8290>.

   [RFC8311]  Black, D., "Relaxing Restrictions on Explicit Congestion
              Notification (ECN) Experimentation", RFC 8311,
              DOI 10.17487/RFC8311, January 2018,
              <https://www.rfc-editor.org/info/rfc8311>.

   [WIREGUARD]
              "WireGuard web site", <https://www.wireguard.com>.

Author's Address

   Peter G. Heist
   463 11 Liberec 30
   Czech Republic

   Email: pete@heistp.net
















Heist                    Expires 22 August 2021                [Page 27]