MPLS Working Group                                                 T. Li
Internet-Draft                                          Juniper Networks
Intended status: Informational                               12 May 2022
Expires: 13 November 2022


                             Entropy Values
                        draft-li-mpls-entropy-01

Abstract

   Equal Cost Multi-Path (ECMP) forwarding is an essential function in
   distributing traffic across parallel paths.  Packets within a flow
   must be kept on a single path to avoid reordering, while different
   flows must be distributed across paths to achieve parallelism.

   Previously, MPLS has addressed this through the use of an entropy
   label, providing up to 20 bits of entropy that can be added to the
   label stack to distinguish different flows.  [RFC6790] With the
   interest in MPLS Network Actions, there are proposals to embedding
   entropy into alternate structures, so it is an appropriate time to
   consider how many bits should be used for entropy in the future.
   [I-D.bocci-mpls-miad-adi-requirements][I-D.andersson-mpls-mna-fwk]

   In this document, we examine the question of how to provide adequate
   entropy through a simple stochastic simulation.  This is not intended
   to be a comprehensive and extensive treatise, but rather a simple
   investigation to build intuition into the issues.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 13 November 2022.






Li                      Expires 13 November 2022                [Page 1]


Internet-Draft               Entropy Values                     May 2022


Copyright Notice

   Copyright (c) 2022 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Model . . . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.2.  Methodology . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Randomness  . . . . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Hashing . . . . . . . . . . . . . . . . . . . . . . . . . . .   4
   4.  Future Hardware Considerations  . . . . . . . . . . . . . . .   5
   5.  Results . . . . . . . . . . . . . . . . . . . . . . . . . . .   5
   6.  Conclusion  . . . . . . . . . . . . . . . . . . . . . . . . .   6
   7.  Validation  . . . . . . . . . . . . . . . . . . . . . . . . .   8
   8.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   9.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
   10. References  . . . . . . . . . . . . . . . . . . . . . . . . .   9
     10.1.  Normative References . . . . . . . . . . . . . . . . . .   9
     10.2.  Informative References . . . . . . . . . . . . . . . . .   9
   Appendix A.  Source Code  . . . . . . . . . . . . . . . . . . . .  10
   Appendix B.  Full results . . . . . . . . . . . . . . . . . . . .  17
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  41

1.  Introduction

   Equal Cost Multi-Path (ECMP) forwarding is an essential function in
   distributing traffic across parallel paths.  Packets within a flow
   must be kept on a single path to avoid reordering, while different
   flows must be distributed across paths to achieve parallelism.

   Previously, MPLS has addressed this through the use of an entropy
   label, providing up to 20 bits of entropy that can be added to the
   label stack to distinguish different flows.  [RFC6790] With the
   interest in MPLS Network Actions, there are proposals to embedding
   entropy into alternate structures, so it is an appropriate time to
   consider how many bits should be used for entropy in the future.
   [I-D.bocci-mpls-miad-adi-requirements][I-D.andersson-mpls-mna-fwk]



Li                      Expires 13 November 2022                [Page 2]


Internet-Draft               Entropy Values                     May 2022


   In this document, we examine the question of how to provide adequate
   entropy through a simple stochastic simulation.  This is not intended
   to be a comprehensive and extensive treatise, but rather a simple
   investigation to build intuition into the issues.

1.1.  Model

   In a typical ECMP situation, a Label Edge Router (LER) [RFC4221]
   would examine incoming traffic and use values in the incoming packet
   to compute an Entropy Label (EL).  This would be placed in the label
   stack.  Subsequent routers on the path could use this EL as input to
   a hashing function and the resulting hash value would then be used to
   select one of the possible output paths.  If the entropy label and
   the hashing function are implemented correctly, packets would be
   uniformly distributed across the possible paths.  If packet sizes are
   uniformly distributed, this would also imply that bandwidth would be
   uniformly distributed across paths.

   Moving forward, we expect that entropy will continue to be carried in
   the label stack, but may not occupy a Label field inside of a Label
   Stack Entry (LSE) [RFC3032].  We generalize this to an Entropy Value
   (EV) and seek to understand the necessary size of this field.

1.2.  Methodology

   To study this, we employ a pseudo-random number generator to create
   Entropy Values and then we apply a hashing function to them.
   Ideally, the results of the hash will be uniformly distributed.
   Error metrics are used to characterize how far results are from this
   ideal.

2.  Randomness

   Real hardware does not use a random number generator to create
   entropy.  Doing so would require mapping each flow to an entropy
   value and this would be painful to implement due to scale.  Rather,
   implementations that we are familiar with use a hashing function on
   the packet header to create an entropy value.  Ideally, the results
   of this would be uniformly distributed.  However, we cannot assume
   that all traffic will be quite so cooperative.

   Conversely, we also cannot assume that the LER will generate no
   entropy.  If an LER does nothing except place a constant in the
   entropy value, no downstream hashing function will be able to extract
   any entropy.






Li                      Expires 13 November 2022                [Page 3]


Internet-Draft               Entropy Values                     May 2022


   To model this, we intentionally choose an imperfect random number
   generator, specifically, we choose one that has a normal distribution
   (i.e., Gaussian or bell curve).  This should produce many different
   entropy values and if the hashing function is doing its job, the
   resulting entropy should result in a uniform distribution of hash
   values.

3.  Hashing

   Similarly, there is no standardized way of implementing the hash
   function that downstream routers will use to convert an entropy value
   into a hash value.  In cases where the entropy value has more bits
   than the hash value can support, it is up to the hashing function to
   fold the entropy from all of the entropy value bits into the hash
   value.

   Thus, we consider a number of hashing functions:

   *  Addition: High order bits are added to the low order bits to
      produce the hash value.

   *  Masking: High order bits are simply discarded.

   *  Mixing and Masking: Bits are shifted, added, and then masked.

   *  Xor: High order bits are XOR'ed into the low order bits.

   *  CRC-CCITT: Compute a CRC over the entropy value and XOR the
      resulting bits together.

   *  MD4: Compute the MD4 digest over the entropy value and XOR the
      resulting bits together.

   *  MD4: Compute the MD5 digest over the entropy value and XOR the
      resulting bits together.

   *  SHA1: Compute the SHA1 digest over the entropy value and XOR the
      resulting bits together.

   *  SHA256: Compute the SHA256 digest over the entropy value and XOR
      the resulting bits together.

   Some of these hash functions work better than others.  Additional
   hash functions could easily be added.







Li                      Expires 13 November 2022                [Page 4]


Internet-Draft               Entropy Values                     May 2022


   To simplify our simulation, we have primarily focused on hashes that
   have a range that is a power of two.  However, in a practical
   application, that is not appropriate and the range must be an
   arbitrary integer, so that we can support n-way ECMP.  This is
   discussed further below.

4.  Future Hardware Considerations

   When ECMP was first implemented, 4-way ECMP was the norm.  In the
   early days of sparse networks, that was frequently adequate.  As the
   number of interfaces on a router has grown, ECMP has followed suit,
   and 128-way and 256-way ECMP is now in production.

   While hardware has provided an impetetus to grow the breadth of ECMP,
   it is also suggesting that there is an upper bound to what needs to
   be supported.  It is fairly clear that with current networking
   trends, we will never need more paths from a single router than it
   has physical interfaces.  While logically, more would be possible,
   the growth in bandwidth has driven the need for ECMP and that
   bandwidth is most easily provided through the use of non-fractional
   interfaces.

   Concurrently, we note a trend away from building multi-chassis
   routers with an arbitrary number of physical interfaces, so this
   implies a bound of around 1000 physical interface per router for the
   foreseable future.

   Hardware is also most conveniently impelemented on power of two
   boundaries.  Thus, we investigate ECMP from 4-way to 4096-way, in
   powers of two.

5.  Results

   For hash values from 8 to 12 bits, we consider entropy values with 0
   to 5 additional bits (i.e., 8 to 17 bits).  We generate enough random
   samples so that each hash bucket should ideally have 1000 entries.

   The error for each has bucket is then difference between the number
   of hash results and 1000.  We then compute the following error
   metrics for each combination of algorithm, hash value bits, and
   additional bits:

   *  Root Mean Square Error (RMSE): The Root Mean Square Error is the
      square root of the sum of the squares of the errors for each
      bucket, divided by the number of buckets.

   *  Mean Average Error (MAE): The Mean Average Error is the arithmetic
      mean of the absolute value of the error for each bucket.



Li                      Expires 13 November 2022                [Page 5]


Internet-Draft               Entropy Values                     May 2022


   *  Mean Average Percentage Error (MAPE): The Mean Average Percentage
      Error is the arithmetic mean of the percentage of error for each
      bucket.

   We then summarize the results based on the number of additional bits
   and on algorithm, computing the mean of each of the above metrics.

6.  Conclusion

   The arguments in Section 4 suggest that 4096-way ECMP will be
   sufficient for some time to come.  This suggests that a base of 12
   bits would be a good start.

   We arbitrarily select a MAPE of 5.0 as a threshold of acceptability.
   This represents a distribution that is within 5% of our goal of a
   uniform distribution.

   A solution to the issue would be a number of bits entropy value and a
   set of hashing algorithms that would provide acceptable performance
   over the full range of hash values, from 2 to 12 bits.

   We observe that for some entropy values, we get unacceptable
   solutions across all algorithms:

   For 2-way ECMP with 0 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add | 687.0 | 687.0 | 68.7 |
   |      mask | 378.0 | 378.0 | 37.8 |
   |       xor | 687.0 | 687.0 | 68.7 |
   | crc-ccitt | 687.0 | 687.0 | 68.7 |
   |       md4 | 687.0 | 687.0 | 68.7 |
   |       md5 | 376.0 | 376.0 | 37.6 |
   |      sha1 | 378.0 | 378.0 | 37.8 |
   |    sha256 | 378.0 | 378.0 | 37.8 |
   +-----------+-------+-------+------+

   Clearly, a 2 bit entropy value is not acceptable.

   We also observe that some algorithms perform better than others:










Li                      Expires 13 November 2022                [Page 6]


Internet-Draft               Entropy Values                     May 2022


   For 16-way ECMP with 1 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  46.0 |  35.0 |  3.5 |
   |      mask |  30.1 |  27.2 |  2.7 |
   |       xor |  48.3 |  43.9 |  4.4 |
   | crc-ccitt |  24.4 |  19.9 |  2.0 |
   |       md4 | 234.5 | 197.4 | 19.7 |
   |       md5 | 262.5 | 196.1 | 19.6 |
   |      sha1 | 183.4 | 150.0 | 15.0 |
   |    sha256 | 281.1 | 251.2 | 25.1 |
   +-----------+-------+-------+------+

   Understanding why some algorithms perform poorly is not understood at
   this time and a subject for future research.

   Once we reach about 10 bits of entropy value, most algorithms work
   well, most of the time:

   For 1024-way ECMP with 0 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 32.2 | 25.8 |  2.6 |
   |      mask | 32.6 | 26.2 |  2.6 |
   |       xor | 33.5 | 26.6 |  2.7 |
   | crc-ccitt | 31.2 | 24.9 |  2.5 |
   |       md4 | 55.4 | 44.4 |  4.4 |
   |       md5 | 55.9 | 44.7 |  4.5 |
   |      sha1 | 57.2 | 45.4 |  4.5 |
   |    sha256 | 57.0 | 46.1 |  4.6 |
   +-----------+------+------+------+

   For 256-way ECMP with 2 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 30.3 | 23.9 |  2.4 |
   |      mask | 32.6 | 26.2 |  2.6 |
   |       xor | 28.9 | 23.1 |  2.3 |
   | crc-ccitt | 35.2 | 28.0 |  2.8 |
   |       md4 | 59.0 | 46.5 |  4.6 |
   |       md5 | 54.7 | 44.0 |  4.4 |
   |      sha1 | 56.8 | 44.3 |  4.4 |
   |    sha256 | 54.5 | 44.1 |  4.4 |
   +-----------+------+------+------+




Li                      Expires 13 November 2022                [Page 7]


Internet-Draft               Entropy Values                     May 2022


   If we look at summaries by additional bits and by algorithm we see:

   Error per added bit
   +------------+-------+-------+-------+
   | Added bits | MRMSE |  MMAE | MMAPE |
   +------------+-------+-------+-------+
   |          0 | 171.0 | 150.6 |  15.1 |
   |          1 | 115.8 | 100.2 |  10.0 |
   |          2 |  74.9 |  61.6 |   6.2 |
   |          3 |  58.5 |  49.0 |   4.9 |
   |          4 |  51.1 |  43.2 |   4.3 |
   |          5 |  47.1 |  39.8 |   4.0 |
   +------------+-------+-------+-------+

   Error per algorithm
   +-----------+-------+-------+-------+
   | Algorithm | MRMSE |  MMAE | MMAPE |
   +-----------+-------+-------+-------+
   |       add |  52.7 |  45.8 |   4.6 |
   |      mask |  34.1 |  28.6 |   2.9 |
   |       xor |  54.0 |  48.0 |   4.8 |
   | crc-ccitt |  48.7 |  43.2 |   4.3 |
   |       md4 | 137.7 | 119.8 |  12.0 |
   |       md5 | 131.3 | 110.7 |  11.1 |
   |      sha1 | 110.5 |  93.0 |   9.3 |
   |    sha256 | 122.2 | 103.3 |  10.3 |
   +-----------+-------+-------+-------+

   This leads us to propose that use at least 4 additional bits for our
   entropy value, resulting in total of 16 (12 + 4).  Please note that
   this is a minimum.  Most proposals are proposing higher numbers of
   bits, thus exceeding this minimum.

7.  Validation

   To further validate that 16 bits of entropy is sufficient, we also
   simulated hashing with a non-power of two result to support n-way
   ECMP.  To map the hash to a range of n possible results, we performed
   a 'mod n' operation after each of the above hashes.  We then selected
   10 random n values that are not powers of two and repeated the above
   experiment.

   In this step, we generated 16 bits of entropy, hashed that and then
   folded it to down to 16 bits.  We then applied the 'mod n' operation
   to get down to a range of n results.  In these circumstances, the
   'add', 'mask', and 'xor' hashes are all identical and result in no
   changes, as they simply are the identity function on 16 bits.  The
   'add' and 'xor' functions were removed for efficiency.



Li                      Expires 13 November 2022                [Page 8]


Internet-Draft               Entropy Values                     May 2022


   The results demonstrate that the 'mask' (identity) function performs
   adequately in this circumstance, again supporting the claim that 16
   bits is sufficient entropy.

   For 3023-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  31.8 |  25.6 |  2.6 |
   | crc-ccitt | 238.2 | 191.1 | 19.1 |
   |       md4 | 320.7 | 254.9 | 25.5 |
   |       md5 | 319.7 | 253.8 | 25.4 |
   |      sha1 | 329.1 | 262.8 | 26.3 |
   |    sha256 | 328.0 | 261.6 | 26.2 |
   +-----------+-------+-------+------+

8.  Security Considerations

   No security issues are discussed in this document.

9.  IANA Considerations

   This document makes no requests of IANA.

10.  References

10.1.  Normative References

   [RFC6790]  Kompella, K., Drake, J., Amante, S., Henderickx, W., and
              L. Yong, "The Use of Entropy Labels in MPLS Forwarding",
              RFC 6790, DOI 10.17487/RFC6790, November 2012,
              <https://www.rfc-editor.org/info/rfc6790>.

   [RFC4221]  Nadeau, T., Srinivasan, C., and A. Farrel, "Multiprotocol
              Label Switching (MPLS) Management Overview", RFC 4221,
              DOI 10.17487/RFC4221, November 2005,
              <https://www.rfc-editor.org/info/rfc4221>.

   [RFC3032]  Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y.,
              Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack
              Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001,
              <https://www.rfc-editor.org/info/rfc3032>.

10.2.  Informative References

   [I-D.bocci-mpls-miad-adi-requirements]
              Bocci, M. and S. Bryant, "Requirements for MPLS Network
              Action Indicators and MPLS Ancillary Data", Work in



Li                      Expires 13 November 2022                [Page 9]


Internet-Draft               Entropy Values                     May 2022


              Progress, Internet-Draft, draft-bocci-mpls-miad-adi-
              requirements-04, 11 April 2022,
              <https://www.ietf.org/archive/id/draft-bocci-mpls-miad-
              adi-requirements-04.txt>.

   [I-D.andersson-mpls-mna-fwk]
              Andersson, L., Bryant, S., Bocci, M., and T. Li, "MPLS
              Network Actions Framework", Work in Progress, Internet-
              Draft, draft-andersson-mpls-mna-fwk-01, 27 April 2022,
              <https://www.ietf.org/archive/id/draft-andersson-mpls-mna-
              fwk-01.txt>.

Appendix A.  Source Code

   #!/usr/bin/python3
   #
   # Copyright 2022, Juniper Networks, Inc.
   # Tony Li (tli)
   #
   # Statistical study on the number of entropy bits necessary.
   #
   # Equal Cost Multi-Path (ECMP) forwarding directs traffic down
   # multiple paths. An entropy label is included in the MPLS label
   # stack or in the MPLS Network Action sub-stack to help
   # implementations keep individual flows on a consistent path but
   # spread different flows across different paths.
   #
   # The entropy label is supposed to be used by an implementation
   # as input to a hash function, producing a hash value that is an
   # index into the set of paths. If the hash values are uniformly
   # distributed, then the paths will be equally used. If the hash
   # function is poor or there is not enough entropy in the entropy
   # label, then the hash values will not be uniformly distribute.
   #
   # To gain some intuition about how many bits of entropy are
   # necessary, we simulate the hashing of a large number of random
   # values and then measure the uniformity of the results.
   #

   import crcmod
   from numpy import random, round
   import matplotlib.pyplot as plt
   import math
   from tabulate import tabulate
   import binascii
   import hashlib

   # To get an interesting volume of data, we need about this many



Li                      Expires 13 November 2022               [Page 10]


Internet-Draft               Entropy Values                     May 2022


   # samples per bucket.
   DEPTH = 1000

   # ECMP is typically up to N way, where N is 2 ** P.  We study a
   # range of P values to get an understanding of the number of
   # necessary bits as a function of P.

   P_RANGE = range(1, 13)
   #P_RANGE = range(2, 6)

   # Consider this many additional bits of entropy, above and beyond
   # P.
   MORE_BITS_RANGE = range(0, 6)

   # Format for floating point numbers
   FLT = '%.1f'

   class HashFunc(object):
      def __init__ (self, p, name):
         self.bits = p
         self.name = name
         # Mask of bits inside of the target range
         self.lowmask = (2 ** p - 1)
         # Mask of bits outside of the target range
         self.highmask = ~self.lowmask

      def hashfunc(self, v):
         raise NotImplementedError()

   class AddHash(HashFunc):
      def __init__(self, p):
         super().__init__(p, 'add')

      def hashfunc(self, v):
         old = v
         if self.highmask & v:
            v = (v & self.lowmask) + ((v & self.highmask) >>
                                      self.bits)
         return v & self.lowmask

   class MaskHash(HashFunc):
      def __init__(self, p):
         super().__init__(p, 'mask')

      def hashfunc(self, v):
         return v & self.lowmask

   class XorHash(HashFunc):



Li                      Expires 13 November 2022               [Page 11]


Internet-Draft               Entropy Values                     May 2022


      def __init__(self, p):
         super().__init__(p, 'xor')

      def hashfunc(self, v):
         while self.highmask & v:
            v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                      self.bits)
         return v

   class CrcHash(HashFunc):
      # CRC-CCITT
      def __init__(self, p):
         super().__init__(p, 'crc-ccitt')

      def hashfunc(self, v):
         v = binascii.crc_hqx(v.to_bytes(4, 'little'), 0)
         while self.highmask & v:
            v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                      self.bits)
         return v

   class Md4Hash(HashFunc):
      def __init__(self, p):
         super().__init__(p, 'md4')
         self.func = hashlib.new('md4')

      def hashfunc(self, v):
         m = self.func.copy()
         m.update(v.to_bytes(4, 'little'))
         d = m.digest()
         v = int.from_bytes(d, 'little')
         while self.highmask & v:
            v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                      self.bits)
         return v

   class Md5Hash(HashFunc):
      def __init__(self, p):
         super().__init__(p, 'md5')
         self.func = hashlib.md5()

      def hashfunc(self, v):
         m = self.func.copy()
         m.update(v.to_bytes(4, 'little'))
         d = m.digest()
         v = int.from_bytes(d, 'little')
         while self.highmask & v:
            v = (v & self.lowmask) ^ ((v & self.highmask) >>



Li                      Expires 13 November 2022               [Page 12]


Internet-Draft               Entropy Values                     May 2022


                                      self.bits)
         return v

   class Sha1Hash(HashFunc):
      def __init__(self, p):
         super().__init__(p, 'sha1')
         self.func = hashlib.sha1()

      def hashfunc(self, v):
         m = self.func.copy()
         m.update(v.to_bytes(4, 'little'))
         d = m.digest()
         v = int.from_bytes(d, 'little')
         while self.highmask & v:
            v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                      self.bits)
         return v

   class Sha256Hash(HashFunc):
      def __init__(self, p):
         super().__init__(p, 'sha256')
         self.func = hashlib.sha256()

      def hashfunc(self, v):
         m = self.func.copy()
         m.update(v.to_bytes(4, 'little'))
         d = m.digest()
         v = int.from_bytes(d, 'little')
         while self.highmask & v:
            v = (v & self.lowmask) ^ ((v & self.highmask) >>
                                      self.bits)
         return v

   def rmse( error ):
      # Compute the root mean square error of this run.
      errsq = [ err ** 2 for err in error ]
      rmse = math.sqrt( sum( errsq ) / len( error ) )
      return rmse

   def mae( error ):
      # Compute the mean average error of this run.
      mae = sum( [ abs( err ) for err in error ] ) / len( error )
      return mae

   def mape( error ):
      # Compute the mean average percentage error of this run.
      mape = 0.0
      for ( ind, err ) in enumerate( error ):



Li                      Expires 13 November 2022               [Page 13]


Internet-Draft               Entropy Values                     May 2022


         if err != 0.0:
            mape += abs( err / DEPTH)
      mape = mape / len( error )
      return mape * 100.0

   def mean(vec):
      return sum(vec)/len(vec)

   def samples(buckets, bits):
      # Uniform distribution
      #samples = random.randint(0, (2 ** bits), buckets * DEPTH)
      # Normal distribution
      # This emulates a 'poor' random number generator
      floats = random.normal(2 ** (bits - 1),
                             scale=2 ** (bits - 3),
                             size=(buckets * DEPTH))
      rounded = round(floats)
      samples = [ int(x) for x in rounded ]
      samples = [ x if x > 0 else -x for x in samples ]
      samples = [ x if x < 2 ** bits else (2 ** bits - 1) for x in
                  samples ]
      # Debugging
      # print (samples)
      #
      #count, bins, ignored = plt.hist(samples, bins = 2 ** bits)
      #plt.title('Random samples using %d bits of entropy' %
      #          bits)
      #plt.show()
      #
      return samples

   headers = ['Hash', 'RMSE', 'MAE', 'MAPE']
   summary_headers = ['Added bits', 'MRMSE', 'MMAE', 'MMAPE']
   algo_summary_headers = ['Algorithm', 'MRMSE', 'MMAE', 'MMAPE']

   def power_of_two():

      def simulate(p, bits):
         total = p + bits
         entropy = samples(2 ** p, total)
         hashes = [ AddHash(p), MaskHash(p), XorHash(p), CrcHash(p),
                    Md4Hash(p), Md5Hash(p), Sha1Hash(p),
                    Sha256Hash(p) ]
         results = []
         for h in hashes:
            hashvals = [ h.hashfunc(x) for x in entropy ]
            # print(hashvals)
            buckets = [ 0 ] * 2 ** p



Li                      Expires 13 November 2022               [Page 14]


Internet-Draft               Entropy Values                     May 2022


            for x in hashvals:
               buckets[x] += 1
            # print(h.name, buckets, sum(buckets))
            error = [ DEPTH - x for x in buckets ]
            # print(error)
            # print(rmse(error), mae(error), mape(error))
            results.append( [ h.name, FLT % rmse(error), FLT %
                              mae(error), FLT % mape(error) ] )
         return results

      summary = []
      per_algo = {}
      for more in MORE_BITS_RANGE:
         per_bit = []
         for p in P_RANGE:
            print( 'For %d-way ECMP with %d more bits of entropy:' %
                   (2 ** p, more))
            results = simulate(p, p + more)
            print(tabulate(results, headers=headers, tablefmt='pretty',
                           stralign='right'))
            rmsevec = [ float(x[1]) for x in results ]
            maevec = [ float(x[2]) for x in results ]
            mapevec = [ float(x[3]) for x in results ]
            per_bit.append([FLT % mean(rmsevec), FLT % mean(maevec),
                            FLT % mean(mapevec)])
            print()

            for result in results:
               if not per_algo.get(result[0]):
                  per_algo[result[0]] = [ [], [], [] ]
               for i in range(1,4):
                  per_algo[result[0]][i-1].append(float(result[i]))

         rmsevec = [ float(x[0]) for x in per_bit ]
         maevec = [ float(x[1]) for x in per_bit ]
         mapevec = [ float(x[2]) for x in per_bit ]
         summary.append([more, FLT % mean(rmsevec), FLT % mean(maevec),
                         FLT % mean(mapevec)])

      print('Error per added bit')
      print(tabulate(summary, headers=summary_headers,
                     tablefmt='pretty', stralign='right'))

      algo_summary = []
      for algo in per_algo:
         algo_summary.append([algo,
                              FLT % mean(per_algo[algo][0]),
                              FLT % mean(per_algo[algo][1]),



Li                      Expires 13 November 2022               [Page 15]


Internet-Draft               Entropy Values                     May 2022


                              FLT % mean(per_algo[algo][2])])
      print('\nError per algorithm')
      print(tabulate(algo_summary, headers=algo_summary_headers,
                     tablefmt='pretty', stralign='right'))

   def multibit():
      # Analyze hashes for non-power-of-two ranges.
      ENTROPY_BITS = 16
      SAMPLES = 10

      # For this case, Add, Xor, and Mask are equivalent: a NOP
      hashes = [ #AddHash(ENTROPY_BITS), XorHash(ENTROPY_BITS),
         MaskHash(ENTROPY_BITS), CrcHash(ENTROPY_BITS),
         Md4Hash(ENTROPY_BITS), Md5Hash(ENTROPY_BITS),
         Sha1Hash(ENTROPY_BITS), Sha256Hash(ENTROPY_BITS) ]

      def waygen():
         start = 2 ** min(P_RANGE)
         stop = 2 ** max(P_RANGE)
         for x in range(start, stop):
            # Don't bother with powers of two again.
            if x & (x - 1) != 0:
               yield x

      def simulate(nways):
         entropy = samples(nways, ENTROPY_BITS)
         results = []
         for h in hashes:
            hashvals = [ h.hashfunc(x) for x in entropy ]
            # print(hashvals)
            buckets = [ 0 ] * nways
            for x in hashvals:
               buckets[x % nways] += 1
            #print(h.name, buckets, sum(buckets))
            error = [ DEPTH - x for x in buckets ]
            # print(error)
            results.append( [ h.name, FLT % rmse(error), FLT %
                              mae(error), FLT % mape(error) ] )
         return results

      per_way = []
      nways = [ x for x in waygen() ]
      for nway in random.choice(nways, size=SAMPLES, replace=False):
         results = simulate(nway)
         print('\nFor %d-way ECMP, %d bits of entropy:' %
               (nway, ENTROPY_BITS))
         print(tabulate(results, headers=headers, tablefmt='pretty',
                        stralign='right'))



Li                      Expires 13 November 2022               [Page 16]


Internet-Draft               Entropy Values                     May 2022


         # print('%d-way' % nway)

         rmsevec = [ float(x[1]) for x in results ]
         maevec = [ float(x[2]) for x in results ]
         mapevec = [ float(x[3]) for x in results ]
         per_way.append([nway, FLT % mean(rmsevec),
                         FLT % mean(maevec), FLT % mean(mapevec)])

      summary_headers = ['n-ways', 'MRMSE', 'MMAE', 'MMAPE']
      print('\nFor n-way ECMP, with %d bits of entropy, '
            'across hash functions' % ENTROPY_BITS)
      print(tabulate(per_way, headers=summary_headers,
                     tablefmt='pretty',
                     stralign='right'))

   power_of_two()
   multibit()

Appendix B.  Full results

   For 2-way ECMP with 0 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add | 687.0 | 687.0 | 68.7 |
   |      mask | 378.0 | 378.0 | 37.8 |
   |       xor | 687.0 | 687.0 | 68.7 |
   | crc-ccitt | 687.0 | 687.0 | 68.7 |
   |       md4 | 687.0 | 687.0 | 68.7 |
   |       md5 | 376.0 | 376.0 | 37.6 |
   |      sha1 | 378.0 | 378.0 | 37.8 |
   |    sha256 | 378.0 | 378.0 | 37.8 |
   +-----------+-------+-------+------+

   For 4-way ECMP with 0 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add | 376.0 | 315.5 | 31.6 |
   |      mask |  39.0 |  36.5 |  3.7 |
   |       xor | 470.9 | 444.0 | 44.4 |
   | crc-ccitt |  56.5 |  48.5 |  4.9 |
   |       md4 | 990.4 | 936.0 | 93.6 |
   |       md5 | 532.6 | 504.0 | 50.4 |
   |      sha1 | 593.1 | 500.0 | 50.0 |
   |    sha256 | 358.4 | 260.5 | 26.1 |
   +-----------+-------+-------+------+




Li                      Expires 13 November 2022               [Page 17]


Internet-Draft               Entropy Values                     May 2022


   For 8-way ECMP with 0 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add | 167.9 | 150.8 | 15.1 |
   |      mask |  35.5 |  28.8 |  2.9 |
   |       xor | 227.2 | 213.2 | 21.3 |
   | crc-ccitt | 443.4 | 442.5 | 44.2 |
   |       md4 | 355.3 | 277.8 | 27.8 |
   |       md5 | 292.2 | 252.2 | 25.2 |
   |      sha1 | 321.3 | 284.8 | 28.5 |
   |    sha256 | 439.9 | 377.8 | 37.8 |
   +-----------+-------+-------+------+

   For 16-way ECMP with 0 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  75.4 |  60.1 |  6.0 |
   |      mask |  40.2 |  35.8 |  3.6 |
   |       xor | 120.2 | 112.0 | 11.2 |
   | crc-ccitt |  33.6 |  28.6 |  2.9 |
   |       md4 | 423.0 | 316.2 | 31.6 |
   |       md5 | 374.6 | 272.2 | 27.2 |
   |      sha1 | 345.7 | 282.9 | 28.3 |
   |    sha256 | 407.9 | 285.5 | 28.5 |
   +-----------+-------+-------+------+

   For 32-way ECMP with 0 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  50.9 |  44.6 |  4.5 |
   |      mask |  32.1 |  26.8 |  2.7 |
   |       xor |  58.5 |  48.7 |  4.9 |
   | crc-ccitt |  32.7 |  29.1 |  2.9 |
   |       md4 | 251.6 | 191.8 | 19.2 |
   |       md5 | 245.9 | 202.4 | 20.2 |
   |      sha1 | 274.4 | 228.1 | 22.8 |
   |    sha256 | 257.5 | 211.2 | 21.1 |
   +-----------+-------+-------+------+

   For 64-way ECMP with 0 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  33.5 |  26.2 |  2.6 |
   |      mask |  25.1 |  18.7 |  1.9 |



Li                      Expires 13 November 2022               [Page 18]


Internet-Draft               Entropy Values                     May 2022


   |       xor |  33.1 |  27.1 |  2.7 |
   | crc-ccitt |  32.9 |  26.4 |  2.6 |
   |       md4 | 204.7 | 168.9 | 16.9 |
   |       md5 | 173.4 | 140.5 | 14.0 |
   |      sha1 | 227.9 | 182.1 | 18.2 |
   |    sha256 | 165.2 | 131.5 | 13.2 |
   +-----------+-------+-------+------+

   For 128-way ECMP with 0 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  27.8 |  22.4 |  2.2 |
   |      mask |  30.9 |  24.8 |  2.5 |
   |       xor |  34.2 |  26.9 |  2.7 |
   | crc-ccitt |  33.2 |  27.0 |  2.7 |
   |       md4 | 131.9 | 105.5 | 10.5 |
   |       md5 | 123.1 | 100.9 | 10.1 |
   |      sha1 | 129.5 | 106.8 | 10.7 |
   |    sha256 | 133.4 | 105.0 | 10.5 |
   +-----------+-------+-------+------+

   For 256-way ECMP with 0 more bits of entropy:
   +-----------+-------+------+------+
   |      Hash |  RMSE |  MAE | MAPE |
   +-----------+-------+------+------+
   |       add |  29.6 | 23.8 |  2.4 |
   |      mask |  31.4 | 25.1 |  2.5 |
   |       xor |  32.5 | 26.1 |  2.6 |
   | crc-ccitt |  31.2 | 24.9 |  2.5 |
   |       md4 |  94.6 | 76.1 |  7.6 |
   |       md5 | 100.6 | 81.3 |  8.1 |
   |      sha1 | 102.3 | 81.1 |  8.1 |
   |    sha256 |  98.6 | 79.6 |  8.0 |
   +-----------+-------+------+------+

   For 512-way ECMP with 0 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.4 | 24.9 |  2.5 |
   |      mask | 32.2 | 25.8 |  2.6 |
   |       xor | 34.1 | 27.0 |  2.7 |
   | crc-ccitt | 30.7 | 25.0 |  2.5 |
   |       md4 | 72.6 | 58.2 |  5.8 |
   |       md5 | 71.7 | 56.6 |  5.7 |
   |      sha1 | 77.6 | 62.1 |  6.2 |
   |    sha256 | 72.2 | 57.6 |  5.8 |



Li                      Expires 13 November 2022               [Page 19]


Internet-Draft               Entropy Values                     May 2022


   +-----------+------+------+------+

   For 1024-way ECMP with 0 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 32.2 | 25.8 |  2.6 |
   |      mask | 32.6 | 26.2 |  2.6 |
   |       xor | 33.5 | 26.6 |  2.7 |
   | crc-ccitt | 31.2 | 24.9 |  2.5 |
   |       md4 | 55.4 | 44.4 |  4.4 |
   |       md5 | 55.9 | 44.7 |  4.5 |
   |      sha1 | 57.2 | 45.4 |  4.5 |
   |    sha256 | 57.0 | 46.1 |  4.6 |
   +-----------+------+------+------+

   For 2048-way ECMP with 0 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.5 | 25.1 |  2.5 |
   |      mask | 31.3 | 25.0 |  2.5 |
   |       xor | 31.9 | 25.6 |  2.6 |
   | crc-ccitt | 31.9 | 25.2 |  2.5 |
   |       md4 | 46.1 | 36.8 |  3.7 |
   |       md5 | 45.8 | 36.4 |  3.6 |
   |      sha1 | 46.9 | 38.0 |  3.8 |
   |    sha256 | 47.0 | 37.6 |  3.8 |
   +-----------+------+------+------+

   For 4096-way ECMP with 0 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.4 | 24.9 |  2.5 |
   |      mask | 31.5 | 24.9 |  2.5 |
   |       xor | 31.2 | 24.9 |  2.5 |
   | crc-ccitt | 31.7 | 25.2 |  2.5 |
   |       md4 | 39.0 | 31.2 |  3.1 |
   |       md5 | 39.8 | 31.9 |  3.2 |
   |      sha1 | 39.1 | 31.1 |  3.1 |
   |    sha256 | 39.2 | 31.2 |  3.1 |
   +-----------+------+------+------+

   For 2-way ECMP with 1 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+



Li                      Expires 13 November 2022               [Page 20]


Internet-Draft               Entropy Values                     May 2022


   |       add | 269.0 | 269.0 | 26.9 |
   |      mask |   3.0 |   3.0 |  0.3 |
   |       xor |  94.0 |  94.0 |  9.4 |
   | crc-ccitt |  94.0 |  94.0 |  9.4 |
   |       md4 | 761.0 | 761.0 | 76.1 |
   |       md5 | 381.0 | 381.0 | 38.1 |
   |      sha1 | 104.0 | 104.0 | 10.4 |
   |    sha256 | 771.0 | 771.0 | 77.1 |
   +-----------+-------+-------+------+

   For 4-way ECMP with 1 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add | 153.0 | 131.5 | 13.2 |
   |      mask |  34.5 |  27.0 |  2.7 |
   |       xor | 161.6 | 161.0 | 16.1 |
   | crc-ccitt |  36.9 |  31.0 |  3.1 |
   |       md4 | 332.6 | 307.0 | 30.7 |
   |       md5 | 562.9 | 443.5 | 44.4 |
   |      sha1 | 219.2 | 184.0 | 18.4 |
   |    sha256 | 358.9 | 325.0 | 32.5 |
   +-----------+-------+-------+------+

   For 8-way ECMP with 1 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  74.6 |  67.2 |  6.7 |
   |      mask |  17.3 |  15.8 |  1.6 |
   |       xor |  76.0 |  75.5 |  7.5 |
   | crc-ccitt | 167.8 | 166.5 | 16.6 |
   |       md4 | 458.2 | 408.5 | 40.9 |
   |       md5 | 340.7 | 247.8 | 24.8 |
   |      sha1 | 487.6 | 382.2 | 38.2 |
   |    sha256 | 355.5 | 282.5 | 28.2 |
   +-----------+-------+-------+------+

   For 16-way ECMP with 1 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  46.0 |  35.0 |  3.5 |
   |      mask |  30.1 |  27.2 |  2.7 |
   |       xor |  48.3 |  43.9 |  4.4 |
   | crc-ccitt |  24.4 |  19.9 |  2.0 |
   |       md4 | 234.5 | 197.4 | 19.7 |
   |       md5 | 262.5 | 196.1 | 19.6 |



Li                      Expires 13 November 2022               [Page 21]


Internet-Draft               Entropy Values                     May 2022


   |      sha1 | 183.4 | 150.0 | 15.0 |
   |    sha256 | 281.1 | 251.2 | 25.1 |
   +-----------+-------+-------+------+

   For 32-way ECMP with 1 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  35.9 |  28.9 |  2.9 |
   |      mask |  31.2 |  26.1 |  2.6 |
   |       xor |  32.6 |  26.8 |  2.7 |
   | crc-ccitt |  40.0 |  32.5 |  3.3 |
   |       md4 | 190.5 | 164.8 | 16.5 |
   |       md5 | 194.2 | 152.2 | 15.2 |
   |      sha1 | 188.4 | 155.7 | 15.6 |
   |    sha256 | 145.6 | 121.2 | 12.1 |
   +-----------+-------+-------+------+

   For 64-way ECMP with 1 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  34.4 |  27.7 |  2.8 |
   |      mask |  31.3 |  26.4 |  2.6 |
   |       xor |  38.2 |  31.6 |  3.2 |
   | crc-ccitt |  32.3 |  24.3 |  2.4 |
   |       md4 | 127.3 |  96.7 |  9.7 |
   |       md5 | 123.8 | 101.7 | 10.2 |
   |      sha1 | 150.6 | 122.3 | 12.2 |
   |    sha256 | 145.4 | 108.1 | 10.8 |
   +-----------+-------+-------+------+

   For 128-way ECMP with 1 more bits of entropy:
   +-----------+-------+------+------+
   |      Hash |  RMSE |  MAE | MAPE |
   +-----------+-------+------+------+
   |       add |  32.2 | 25.5 |  2.5 |
   |      mask |  33.4 | 25.7 |  2.6 |
   |       xor |  34.5 | 27.6 |  2.8 |
   | crc-ccitt |  27.7 | 20.7 |  2.1 |
   |       md4 |  94.6 | 77.5 |  7.7 |
   |       md5 | 105.4 | 83.5 |  8.4 |
   |      sha1 |  96.2 | 77.4 |  7.7 |
   |    sha256 | 100.2 | 75.9 |  7.6 |
   +-----------+-------+------+------+

   For 256-way ECMP with 1 more bits of entropy:
   +-----------+------+------+------+



Li                      Expires 13 November 2022               [Page 22]


Internet-Draft               Entropy Values                     May 2022


   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.0 | 24.9 |  2.5 |
   |      mask | 30.8 | 25.1 |  2.5 |
   |       xor | 28.7 | 22.5 |  2.3 |
   | crc-ccitt | 33.8 | 27.6 |  2.8 |
   |       md4 | 72.6 | 59.9 |  6.0 |
   |       md5 | 69.4 | 55.5 |  5.5 |
   |      sha1 | 72.8 | 58.6 |  5.9 |
   |    sha256 | 72.7 | 56.7 |  5.7 |
   +-----------+------+------+------+

   For 512-way ECMP with 1 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.7 | 25.4 |  2.5 |
   |      mask | 29.7 | 23.5 |  2.3 |
   |       xor | 31.9 | 25.4 |  2.5 |
   | crc-ccitt | 30.9 | 24.7 |  2.5 |
   |       md4 | 53.8 | 43.2 |  4.3 |
   |       md5 | 57.3 | 46.7 |  4.7 |
   |      sha1 | 59.7 | 47.2 |  4.7 |
   |    sha256 | 56.2 | 45.4 |  4.5 |
   +-----------+------+------+------+

   For 1024-way ECMP with 1 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.6 | 25.1 |  2.5 |
   |      mask | 31.0 | 25.0 |  2.5 |
   |       xor | 31.5 | 25.1 |  2.5 |
   | crc-ccitt | 30.2 | 24.1 |  2.4 |
   |       md4 | 45.0 | 35.4 |  3.5 |
   |       md5 | 47.6 | 38.0 |  3.8 |
   |      sha1 | 45.7 | 36.7 |  3.7 |
   |    sha256 | 46.5 | 36.8 |  3.7 |
   +-----------+------+------+------+

   For 2048-way ECMP with 1 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 32.7 | 26.0 |  2.6 |
   |      mask | 31.9 | 25.5 |  2.6 |
   |       xor | 31.8 | 25.4 |  2.5 |
   | crc-ccitt | 32.4 | 25.8 |  2.6 |



Li                      Expires 13 November 2022               [Page 23]


Internet-Draft               Entropy Values                     May 2022


   |       md4 | 39.5 | 31.8 |  3.2 |
   |       md5 | 38.6 | 30.8 |  3.1 |
   |      sha1 | 39.3 | 31.1 |  3.1 |
   |    sha256 | 39.7 | 31.9 |  3.2 |
   +-----------+------+------+------+

   For 4096-way ECMP with 1 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.7 | 25.3 |  2.5 |
   |      mask | 31.5 | 25.1 |  2.5 |
   |       xor | 31.8 | 25.5 |  2.6 |
   | crc-ccitt | 32.3 | 25.9 |  2.6 |
   |       md4 | 35.9 | 28.5 |  2.9 |
   |       md5 | 35.7 | 28.5 |  2.9 |
   |      sha1 | 35.2 | 28.1 |  2.8 |
   |    sha256 | 36.3 | 28.9 |  2.9 |
   +-----------+------+------+------+

   For 2-way ECMP with 2 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |   2.0 |   2.0 |  0.2 |
   |      mask |  15.0 |  15.0 |  1.5 |
   |       xor |  44.0 |  44.0 |  4.4 |
   | crc-ccitt |  44.0 |  44.0 |  4.4 |
   |       md4 | 197.0 | 197.0 | 19.7 |
   |       md5 | 123.0 | 123.0 | 12.3 |
   |      sha1 | 238.0 | 238.0 | 23.8 |
   |    sha256 |  20.0 |  20.0 |  2.0 |
   +-----------+-------+-------+------+

   For 4-way ECMP with 2 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  14.1 |  12.5 |  1.2 |
   |      mask |   8.2 |   8.0 |  0.8 |
   |       xor |   9.5 |   7.5 |  0.8 |
   | crc-ccitt |  66.0 |  57.5 |  5.8 |
   |       md4 | 119.6 |  94.5 |  9.4 |
   |       md5 | 553.7 | 472.0 | 47.2 |
   |      sha1 | 163.8 | 134.5 | 13.5 |
   |    sha256 | 366.7 | 298.5 | 29.8 |
   +-----------+-------+-------+------+




Li                      Expires 13 November 2022               [Page 24]


Internet-Draft               Entropy Values                     May 2022


   For 8-way ECMP with 2 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  30.2 |  22.5 |  2.2 |
   |      mask |  24.0 |  19.8 |  2.0 |
   |       xor |  43.9 |  36.5 |  3.7 |
   | crc-ccitt |  25.8 |  22.2 |  2.2 |
   |       md4 | 245.6 | 163.5 | 16.3 |
   |       md5 | 233.6 | 187.2 | 18.7 |
   |      sha1 | 337.1 | 295.8 | 29.6 |
   |    sha256 | 247.0 | 194.2 | 19.4 |
   +-----------+-------+-------+------+

   For 16-way ECMP with 2 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  23.0 |  19.0 |  1.9 |
   |      mask |  33.5 |  27.9 |  2.8 |
   |       xor |  26.4 |  21.5 |  2.2 |
   | crc-ccitt |  26.1 |  22.4 |  2.2 |
   |       md4 | 212.4 | 151.2 | 15.1 |
   |       md5 | 227.4 | 174.5 | 17.5 |
   |      sha1 | 180.4 | 146.9 | 14.7 |
   |    sha256 | 193.3 | 155.8 | 15.6 |
   +-----------+-------+-------+------+

   For 32-way ECMP with 2 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  35.4 |  28.9 |  2.9 |
   |      mask |  26.6 |  22.8 |  2.3 |
   |       xor |  30.8 |  23.7 |  2.4 |
   | crc-ccitt |  30.4 |  23.6 |  2.4 |
   |       md4 | 131.6 | 107.1 | 10.7 |
   |       md5 | 136.4 | 104.9 | 10.5 |
   |      sha1 | 121.8 | 101.3 | 10.1 |
   |    sha256 | 137.0 | 112.8 | 11.3 |
   +-----------+-------+-------+------+

   For 64-way ECMP with 2 more bits of entropy:
   +-----------+-------+------+------+
   |      Hash |  RMSE |  MAE | MAPE |
   +-----------+-------+------+------+
   |       add |  30.6 | 24.7 |  2.5 |
   |      mask |  28.6 | 21.5 |  2.1 |



Li                      Expires 13 November 2022               [Page 25]


Internet-Draft               Entropy Values                     May 2022


   |       xor |  35.1 | 28.8 |  2.9 |
   | crc-ccitt |  37.4 | 30.3 |  3.0 |
   |       md4 |  95.3 | 80.0 |  8.0 |
   |       md5 | 110.5 | 89.8 |  9.0 |
   |      sha1 | 106.2 | 85.4 |  8.5 |
   |    sha256 | 104.7 | 84.9 |  8.5 |
   +-----------+-------+------+------+

   For 128-way ECMP with 2 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 30.5 | 24.5 |  2.4 |
   |      mask | 33.6 | 27.0 |  2.7 |
   |       xor | 31.1 | 25.5 |  2.6 |
   | crc-ccitt | 31.7 | 25.4 |  2.5 |
   |       md4 | 69.0 | 55.2 |  5.5 |
   |       md5 | 72.6 | 55.3 |  5.5 |
   |      sha1 | 76.9 | 60.5 |  6.1 |
   |    sha256 | 75.9 | 59.8 |  6.0 |
   +-----------+------+------+------+

   For 256-way ECMP with 2 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 30.3 | 23.9 |  2.4 |
   |      mask | 32.6 | 26.2 |  2.6 |
   |       xor | 28.9 | 23.1 |  2.3 |
   | crc-ccitt | 35.2 | 28.0 |  2.8 |
   |       md4 | 59.0 | 46.5 |  4.6 |
   |       md5 | 54.7 | 44.0 |  4.4 |
   |      sha1 | 56.8 | 44.3 |  4.4 |
   |    sha256 | 54.5 | 44.1 |  4.4 |
   +-----------+------+------+------+

   For 512-way ECMP with 2 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 32.2 | 25.4 |  2.5 |
   |      mask | 32.4 | 25.8 |  2.6 |
   |       xor | 31.8 | 25.1 |  2.5 |
   | crc-ccitt | 33.2 | 26.6 |  2.7 |
   |       md4 | 47.0 | 36.5 |  3.7 |
   |       md5 | 45.4 | 36.5 |  3.6 |
   |      sha1 | 44.1 | 35.6 |  3.6 |
   |    sha256 | 45.7 | 36.5 |  3.6 |



Li                      Expires 13 November 2022               [Page 26]


Internet-Draft               Entropy Values                     May 2022


   +-----------+------+------+------+

   For 1024-way ECMP with 2 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.2 | 25.2 |  2.5 |
   |      mask | 30.8 | 24.5 |  2.5 |
   |       xor | 31.6 | 24.8 |  2.5 |
   | crc-ccitt | 31.9 | 25.5 |  2.5 |
   |       md4 | 39.5 | 31.4 |  3.1 |
   |       md5 | 39.4 | 31.8 |  3.2 |
   |      sha1 | 39.3 | 31.4 |  3.1 |
   |    sha256 | 38.6 | 31.0 |  3.1 |
   +-----------+------+------+------+

   For 2048-way ECMP with 2 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.5 | 25.1 |  2.5 |
   |      mask | 32.4 | 25.6 |  2.6 |
   |       xor | 32.7 | 26.0 |  2.6 |
   | crc-ccitt | 32.1 | 25.5 |  2.6 |
   |       md4 | 36.4 | 29.1 |  2.9 |
   |       md5 | 35.8 | 28.6 |  2.9 |
   |      sha1 | 34.9 | 27.6 |  2.8 |
   |    sha256 | 35.3 | 27.9 |  2.8 |
   +-----------+------+------+------+

   For 4096-way ECMP with 2 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.9 | 25.4 |  2.5 |
   |      mask | 31.2 | 24.9 |  2.5 |
   |       xor | 32.1 | 25.8 |  2.6 |
   | crc-ccitt | 31.4 | 25.0 |  2.5 |
   |       md4 | 33.3 | 26.7 |  2.7 |
   |       md5 | 33.8 | 27.0 |  2.7 |
   |      sha1 | 34.0 | 27.1 |  2.7 |
   |    sha256 | 34.3 | 27.2 |  2.7 |
   +-----------+------+------+------+

   For 2-way ECMP with 3 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+



Li                      Expires 13 November 2022               [Page 27]


Internet-Draft               Entropy Values                     May 2022


   |       add |  10.0 |  10.0 |  1.0 |
   |      mask |   0.0 |   0.0 |  0.0 |
   |       xor |   1.0 |   1.0 |  0.1 |
   | crc-ccitt |   1.0 |   1.0 |  0.1 |
   |       md4 |  28.0 |  28.0 |  2.8 |
   |       md5 | 372.0 | 372.0 | 37.2 |
   |      sha1 |  36.0 |  36.0 |  3.6 |
   |    sha256 |  99.0 |  99.0 |  9.9 |
   +-----------+-------+-------+------+

   For 4-way ECMP with 3 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  16.4 |  13.5 |  1.3 |
   |      mask |  26.5 |  21.5 |  2.1 |
   |       xor |  25.8 |  22.5 |  2.2 |
   | crc-ccitt |  24.4 |  21.0 |  2.1 |
   |       md4 | 411.4 | 408.5 | 40.8 |
   |       md5 | 191.2 | 163.0 | 16.3 |
   |      sha1 |  88.6 |  72.0 |  7.2 |
   |    sha256 | 177.6 | 152.5 | 15.2 |
   +-----------+-------+-------+------+

   For 8-way ECMP with 3 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  29.4 |  27.0 |  2.7 |
   |      mask |  34.4 |  30.0 |  3.0 |
   |       xor |  38.2 |  25.5 |  2.5 |
   | crc-ccitt |  35.2 |  32.0 |  3.2 |
   |       md4 | 133.3 | 115.2 | 11.5 |
   |       md5 | 257.9 | 179.2 | 17.9 |
   |      sha1 | 139.7 | 109.5 | 10.9 |
   |    sha256 | 120.1 | 108.8 | 10.9 |
   +-----------+-------+-------+------+

   For 16-way ECMP with 3 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  29.5 |  22.5 |  2.3 |
   |      mask |  21.8 |  17.5 |  1.8 |
   |       xor |  29.8 |  26.1 |  2.6 |
   | crc-ccitt |  23.4 |  18.9 |  1.9 |
   |       md4 | 116.8 |  83.8 |  8.4 |
   |       md5 | 156.1 | 130.5 | 13.1 |



Li                      Expires 13 November 2022               [Page 28]


Internet-Draft               Entropy Values                     May 2022


   |      sha1 | 144.3 | 116.9 | 11.7 |
   |    sha256 | 138.1 | 113.6 | 11.4 |
   +-----------+-------+-------+------+

   For 32-way ECMP with 3 more bits of entropy:
   +-----------+-------+------+------+
   |      Hash |  RMSE |  MAE | MAPE |
   +-----------+-------+------+------+
   |       add |  27.9 | 22.9 |  2.3 |
   |      mask |  33.4 | 23.9 |  2.4 |
   |       xor |  27.4 | 22.1 |  2.2 |
   | crc-ccitt |  27.6 | 21.4 |  2.1 |
   |       md4 |  90.3 | 70.6 |  7.1 |
   |       md5 |  75.9 | 60.9 |  6.1 |
   |      sha1 |  96.7 | 76.5 |  7.7 |
   |    sha256 | 116.8 | 96.2 |  9.6 |
   +-----------+-------+------+------+

   For 64-way ECMP with 3 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 32.0 | 25.9 |  2.6 |
   |      mask | 27.7 | 22.1 |  2.2 |
   |       xor | 32.2 | 26.0 |  2.6 |
   | crc-ccitt | 32.1 | 25.1 |  2.5 |
   |       md4 | 64.5 | 51.1 |  5.1 |
   |       md5 | 76.9 | 60.4 |  6.0 |
   |      sha1 | 77.1 | 62.1 |  6.2 |
   |    sha256 | 78.3 | 62.7 |  6.3 |
   +-----------+------+------+------+

   For 128-way ECMP with 3 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 29.4 | 24.2 |  2.4 |
   |      mask | 32.1 | 25.5 |  2.5 |
   |       xor | 30.7 | 24.7 |  2.5 |
   | crc-ccitt | 33.8 | 26.7 |  2.7 |
   |       md4 | 52.7 | 42.8 |  4.3 |
   |       md5 | 55.5 | 45.0 |  4.5 |
   |      sha1 | 60.3 | 48.5 |  4.8 |
   |    sha256 | 57.0 | 46.2 |  4.6 |
   +-----------+------+------+------+

   For 256-way ECMP with 3 more bits of entropy:
   +-----------+------+------+------+



Li                      Expires 13 November 2022               [Page 29]


Internet-Draft               Entropy Values                     May 2022


   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.2 | 25.5 |  2.5 |
   |      mask | 30.3 | 24.3 |  2.4 |
   |       xor | 32.6 | 26.4 |  2.6 |
   | crc-ccitt | 31.4 | 25.7 |  2.6 |
   |       md4 | 50.2 | 39.8 |  4.0 |
   |       md5 | 44.2 | 34.9 |  3.5 |
   |      sha1 | 46.7 | 37.0 |  3.7 |
   |    sha256 | 46.9 | 36.5 |  3.6 |
   +-----------+------+------+------+

   For 512-way ECMP with 3 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.8 | 25.4 |  2.5 |
   |      mask | 31.7 | 25.5 |  2.5 |
   |       xor | 31.5 | 25.2 |  2.5 |
   | crc-ccitt | 32.1 | 25.5 |  2.6 |
   |       md4 | 39.8 | 32.2 |  3.2 |
   |       md5 | 38.7 | 30.8 |  3.1 |
   |      sha1 | 38.6 | 31.4 |  3.1 |
   |    sha256 | 39.9 | 31.2 |  3.1 |
   +-----------+------+------+------+

   For 1024-way ECMP with 3 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 33.1 | 26.5 |  2.6 |
   |      mask | 32.7 | 26.3 |  2.6 |
   |       xor | 31.5 | 25.1 |  2.5 |
   | crc-ccitt | 31.9 | 25.5 |  2.5 |
   |       md4 | 35.5 | 28.1 |  2.8 |
   |       md5 | 36.9 | 29.2 |  2.9 |
   |      sha1 | 35.7 | 28.6 |  2.9 |
   |    sha256 | 35.8 | 28.5 |  2.8 |
   +-----------+------+------+------+

   For 2048-way ECMP with 3 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.3 | 24.9 |  2.5 |
   |      mask | 31.5 | 24.9 |  2.5 |
   |       xor | 32.3 | 25.7 |  2.6 |
   | crc-ccitt | 30.9 | 24.4 |  2.4 |



Li                      Expires 13 November 2022               [Page 30]


Internet-Draft               Entropy Values                     May 2022


   |       md4 | 33.7 | 26.6 |  2.7 |
   |       md5 | 33.8 | 27.3 |  2.7 |
   |      sha1 | 32.6 | 26.4 |  2.6 |
   |    sha256 | 33.5 | 26.6 |  2.7 |
   +-----------+------+------+------+

   For 4096-way ECMP with 3 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 32.2 | 25.5 |  2.5 |
   |      mask | 31.3 | 25.1 |  2.5 |
   |       xor | 31.8 | 25.2 |  2.5 |
   | crc-ccitt | 31.7 | 25.1 |  2.5 |
   |       md4 | 33.3 | 26.4 |  2.6 |
   |       md5 | 32.8 | 26.2 |  2.6 |
   |      sha1 | 32.9 | 26.3 |  2.6 |
   |    sha256 | 32.8 | 26.2 |  2.6 |
   +-----------+------+------+------+

   For 2-way ECMP with 4 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  24.0 |  24.0 |  2.4 |
   |      mask |   5.0 |   5.0 |  0.5 |
   |       xor |  26.0 |  26.0 |  2.6 |
   | crc-ccitt |  26.0 |  26.0 |  2.6 |
   |       md4 |  46.0 |  46.0 |  4.6 |
   |       md5 | 250.0 | 250.0 | 25.0 |
   |      sha1 |  75.0 |  75.0 |  7.5 |
   |    sha256 | 262.0 | 262.0 | 26.2 |
   +-----------+-------+-------+------+

   For 4-way ECMP with 4 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  39.3 |  32.0 |  3.2 |
   |      mask |  17.7 |  13.5 |  1.4 |
   |       xor |  28.4 |  22.5 |  2.2 |
   | crc-ccitt |  29.5 |  22.0 |  2.2 |
   |       md4 | 180.0 | 151.5 | 15.2 |
   |       md5 | 148.5 | 130.0 | 13.0 |
   |      sha1 | 116.0 | 100.0 | 10.0 |
   |    sha256 | 128.4 | 113.5 | 11.3 |
   +-----------+-------+-------+------+




Li                      Expires 13 November 2022               [Page 31]


Internet-Draft               Entropy Values                     May 2022


   For 8-way ECMP with 4 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  32.7 |  27.0 |  2.7 |
   |      mask |  32.7 |  25.2 |  2.5 |
   |       xor |  37.7 |  26.2 |  2.6 |
   | crc-ccitt |  30.3 |  23.2 |  2.3 |
   |       md4 |  79.2 |  68.5 |  6.9 |
   |       md5 | 148.2 | 133.0 | 13.3 |
   |      sha1 | 112.1 | 101.8 | 10.2 |
   |    sha256 | 120.7 | 108.8 | 10.9 |
   +-----------+-------+-------+------+

   For 16-way ECMP with 4 more bits of entropy:
   +-----------+-------+------+------+
   |      Hash |  RMSE |  MAE | MAPE |
   +-----------+-------+------+------+
   |       add |  26.7 | 21.0 |  2.1 |
   |      mask |  42.9 | 34.9 |  3.5 |
   |       xor |  31.5 | 24.0 |  2.4 |
   | crc-ccitt |  33.9 | 29.1 |  2.9 |
   |       md4 | 120.1 | 94.6 |  9.5 |
   |       md5 | 106.8 | 86.4 |  8.6 |
   |      sha1 |  90.1 | 71.1 |  7.1 |
   |    sha256 |  65.7 | 58.9 |  5.9 |
   +-----------+-------+------+------+

   For 32-way ECMP with 4 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 33.2 | 25.9 |  2.6 |
   |      mask | 30.3 | 24.7 |  2.5 |
   |       xor | 38.8 | 32.2 |  3.2 |
   | crc-ccitt | 28.0 | 21.9 |  2.2 |
   |       md4 | 63.9 | 48.1 |  4.8 |
   |       md5 | 79.4 | 61.7 |  6.2 |
   |      sha1 | 63.1 | 49.9 |  5.0 |
   |    sha256 | 83.0 | 69.1 |  6.9 |
   +-----------+------+------+------+

   For 64-way ECMP with 4 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 28.7 | 22.3 |  2.2 |
   |      mask | 30.2 | 25.5 |  2.6 |



Li                      Expires 13 November 2022               [Page 32]


Internet-Draft               Entropy Values                     May 2022


   |       xor | 33.2 | 26.8 |  2.7 |
   | crc-ccitt | 30.2 | 23.3 |  2.3 |
   |       md4 | 52.5 | 42.5 |  4.2 |
   |       md5 | 62.2 | 51.2 |  5.1 |
   |      sha1 | 55.1 | 45.0 |  4.5 |
   |    sha256 | 52.0 | 42.3 |  4.2 |
   +-----------+------+------+------+

   For 128-way ECMP with 4 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.6 | 25.0 |  2.5 |
   |      mask | 29.0 | 21.8 |  2.2 |
   |       xor | 30.1 | 23.0 |  2.3 |
   | crc-ccitt | 29.7 | 24.4 |  2.4 |
   |       md4 | 46.5 | 36.9 |  3.7 |
   |       md5 | 46.1 | 36.6 |  3.7 |
   |      sha1 | 42.9 | 33.8 |  3.4 |
   |    sha256 | 50.0 | 40.4 |  4.0 |
   +-----------+------+------+------+

   For 256-way ECMP with 4 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.3 | 24.9 |  2.5 |
   |      mask | 30.5 | 23.8 |  2.4 |
   |       xor | 31.0 | 25.4 |  2.5 |
   | crc-ccitt | 32.7 | 26.1 |  2.6 |
   |       md4 | 40.8 | 32.7 |  3.3 |
   |       md5 | 41.5 | 33.0 |  3.3 |
   |      sha1 | 36.4 | 28.5 |  2.8 |
   |    sha256 | 38.4 | 30.1 |  3.0 |
   +-----------+------+------+------+

   For 512-way ECMP with 4 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.4 | 25.3 |  2.5 |
   |      mask | 32.5 | 26.0 |  2.6 |
   |       xor | 31.5 | 24.7 |  2.5 |
   | crc-ccitt | 31.6 | 25.1 |  2.5 |
   |       md4 | 34.3 | 27.3 |  2.7 |
   |       md5 | 36.1 | 29.2 |  2.9 |
   |      sha1 | 35.9 | 28.6 |  2.9 |
   |    sha256 | 37.1 | 29.5 |  2.9 |



Li                      Expires 13 November 2022               [Page 33]


Internet-Draft               Entropy Values                     May 2022


   +-----------+------+------+------+

   For 1024-way ECMP with 4 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.3 | 24.8 |  2.5 |
   |      mask | 31.7 | 25.2 |  2.5 |
   |       xor | 31.4 | 25.1 |  2.5 |
   | crc-ccitt | 31.5 | 25.3 |  2.5 |
   |       md4 | 33.1 | 26.3 |  2.6 |
   |       md5 | 34.1 | 27.0 |  2.7 |
   |      sha1 | 33.7 | 27.1 |  2.7 |
   |    sha256 | 32.8 | 26.1 |  2.6 |
   +-----------+------+------+------+

   For 2048-way ECMP with 4 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.9 | 25.4 |  2.5 |
   |      mask | 31.8 | 25.5 |  2.5 |
   |       xor | 32.2 | 25.7 |  2.6 |
   | crc-ccitt | 31.4 | 24.8 |  2.5 |
   |       md4 | 32.7 | 26.1 |  2.6 |
   |       md5 | 32.9 | 26.1 |  2.6 |
   |      sha1 | 33.0 | 26.3 |  2.6 |
   |    sha256 | 31.9 | 25.7 |  2.6 |
   +-----------+------+------+------+

   For 4096-way ECMP with 4 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.7 | 25.4 |  2.5 |
   |      mask | 32.5 | 25.8 |  2.6 |
   |       xor | 31.7 | 25.2 |  2.5 |
   | crc-ccitt | 32.0 | 25.7 |  2.6 |
   |       md4 | 32.6 | 26.0 |  2.6 |
   |       md5 | 31.6 | 25.1 |  2.5 |
   |      sha1 | 31.7 | 25.1 |  2.5 |
   |    sha256 | 32.3 | 25.9 |  2.6 |
   +-----------+------+------+------+

   For 2-way ECMP with 5 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+



Li                      Expires 13 November 2022               [Page 34]


Internet-Draft               Entropy Values                     May 2022


   |       add |  29.0 |  29.0 |  2.9 |
   |      mask |  11.0 |  11.0 |  1.1 |
   |       xor |   7.0 |   7.0 |  0.7 |
   | crc-ccitt |   7.0 |   7.0 |  0.7 |
   |       md4 | 358.0 | 358.0 | 35.8 |
   |       md5 | 174.0 | 174.0 | 17.4 |
   |      sha1 | 108.0 | 108.0 | 10.8 |
   |    sha256 |  10.0 |  10.0 |  1.0 |
   +-----------+-------+-------+------+

   For 4-way ECMP with 5 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  41.3 |  35.0 |  3.5 |
   |      mask |  42.7 |  33.5 |  3.4 |
   |       xor |  17.5 |  15.5 |  1.6 |
   | crc-ccitt |  14.2 |  12.5 |  1.2 |
   |       md4 | 135.3 | 125.0 | 12.5 |
   |       md5 | 128.0 | 109.0 | 10.9 |
   |      sha1 | 114.3 | 106.5 | 10.7 |
   |    sha256 | 175.5 | 141.5 | 14.2 |
   +-----------+-------+-------+------+

   For 8-way ECMP with 5 more bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |       add |  23.5 |  20.0 |  2.0 |
   |      mask |  29.0 |  20.2 |  2.0 |
   |       xor |  27.1 |  24.5 |  2.4 |
   | crc-ccitt |  27.6 |  20.2 |  2.0 |
   |       md4 | 105.4 |  91.5 |  9.2 |
   |       md5 |  53.3 |  43.5 |  4.3 |
   |      sha1 |  74.8 |  63.8 |  6.4 |
   |    sha256 | 124.7 | 102.2 | 10.2 |
   +-----------+-------+-------+------+

   For 16-way ECMP with 5 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 39.1 | 33.2 |  3.3 |
   |      mask | 38.5 | 29.5 |  2.9 |
   |       xor | 41.3 | 33.6 |  3.4 |
   | crc-ccitt | 27.0 | 21.2 |  2.1 |
   |       md4 | 71.6 | 59.1 |  5.9 |
   |       md5 | 86.5 | 73.8 |  7.4 |



Li                      Expires 13 November 2022               [Page 35]


Internet-Draft               Entropy Values                     May 2022


   |      sha1 | 79.3 | 63.4 |  6.3 |
   |    sha256 | 59.3 | 50.9 |  5.1 |
   +-----------+------+------+------+

   For 32-way ECMP with 5 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.2 | 25.1 |  2.5 |
   |      mask | 33.7 | 28.6 |  2.9 |
   |       xor | 32.6 | 27.6 |  2.8 |
   | crc-ccitt | 28.7 | 22.0 |  2.2 |
   |       md4 | 51.4 | 42.0 |  4.2 |
   |       md5 | 54.2 | 44.8 |  4.5 |
   |      sha1 | 68.5 | 49.2 |  4.9 |
   |    sha256 | 56.7 | 47.8 |  4.8 |
   +-----------+------+------+------+

   For 64-way ECMP with 5 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 33.0 | 26.6 |  2.7 |
   |      mask | 29.4 | 22.9 |  2.3 |
   |       xor | 33.9 | 27.1 |  2.7 |
   | crc-ccitt | 29.6 | 22.0 |  2.2 |
   |       md4 | 46.6 | 35.8 |  3.6 |
   |       md5 | 48.5 | 39.8 |  4.0 |
   |      sha1 | 43.4 | 33.8 |  3.4 |
   |    sha256 | 44.4 | 36.2 |  3.6 |
   +-----------+------+------+------+

   For 128-way ECMP with 5 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 32.3 | 25.7 |  2.6 |
   |      mask | 28.8 | 22.9 |  2.3 |
   |       xor | 30.0 | 24.5 |  2.5 |
   | crc-ccitt | 31.3 | 23.9 |  2.4 |
   |       md4 | 41.7 | 34.7 |  3.5 |
   |       md5 | 38.5 | 31.1 |  3.1 |
   |      sha1 | 36.4 | 28.8 |  2.9 |
   |    sha256 | 35.0 | 28.0 |  2.8 |
   +-----------+------+------+------+

   For 256-way ECMP with 5 more bits of entropy:
   +-----------+------+------+------+



Li                      Expires 13 November 2022               [Page 36]


Internet-Draft               Entropy Values                     May 2022


   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 30.8 | 24.4 |  2.4 |
   |      mask | 31.0 | 24.6 |  2.5 |
   |       xor | 31.3 | 24.7 |  2.5 |
   | crc-ccitt | 31.1 | 24.3 |  2.4 |
   |       md4 | 37.0 | 29.6 |  3.0 |
   |       md5 | 31.5 | 26.0 |  2.6 |
   |      sha1 | 35.5 | 27.7 |  2.8 |
   |    sha256 | 37.0 | 29.6 |  3.0 |
   +-----------+------+------+------+

   For 512-way ECMP with 5 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 32.0 | 25.2 |  2.5 |
   |      mask | 30.9 | 24.7 |  2.5 |
   |       xor | 30.3 | 24.3 |  2.4 |
   | crc-ccitt | 31.6 | 25.0 |  2.5 |
   |       md4 | 36.0 | 28.6 |  2.9 |
   |       md5 | 33.9 | 26.5 |  2.6 |
   |      sha1 | 32.7 | 25.9 |  2.6 |
   |    sha256 | 33.7 | 26.5 |  2.6 |
   +-----------+------+------+------+

   For 1024-way ECMP with 5 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 32.5 | 25.7 |  2.6 |
   |      mask | 32.1 | 25.7 |  2.6 |
   |       xor | 32.4 | 26.1 |  2.6 |
   | crc-ccitt | 31.2 | 25.5 |  2.5 |
   |       md4 | 33.4 | 26.8 |  2.7 |
   |       md5 | 32.2 | 25.9 |  2.6 |
   |      sha1 | 33.4 | 26.3 |  2.6 |
   |    sha256 | 33.3 | 26.5 |  2.7 |
   +-----------+------+------+------+

   For 2048-way ECMP with 5 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 30.9 | 24.8 |  2.5 |
   |      mask | 32.1 | 25.9 |  2.6 |
   |       xor | 31.9 | 25.5 |  2.5 |
   | crc-ccitt | 31.3 | 24.9 |  2.5 |



Li                      Expires 13 November 2022               [Page 37]


Internet-Draft               Entropy Values                     May 2022


   |       md4 | 32.3 | 25.8 |  2.6 |
   |       md5 | 32.4 | 25.9 |  2.6 |
   |      sha1 | 32.6 | 26.1 |  2.6 |
   |    sha256 | 32.6 | 25.9 |  2.6 |
   +-----------+------+------+------+

   For 4096-way ECMP with 5 more bits of entropy:
   +-----------+------+------+------+
   |      Hash | RMSE |  MAE | MAPE |
   +-----------+------+------+------+
   |       add | 31.8 | 25.2 |  2.5 |
   |      mask | 31.5 | 24.9 |  2.5 |
   |       xor | 31.6 | 25.1 |  2.5 |
   | crc-ccitt | 31.6 | 25.3 |  2.5 |
   |       md4 | 31.9 | 25.5 |  2.6 |
   |       md5 | 31.8 | 25.2 |  2.5 |
   |      sha1 | 32.2 | 25.7 |  2.6 |
   |    sha256 | 32.2 | 26.0 |  2.6 |
   +-----------+------+------+------+

   Error per added bit
   +------------+-------+-------+-------+
   | Added bits | MRMSE |  MMAE | MMAPE |
   +------------+-------+-------+-------+
   |          0 | 171.0 | 150.6 |  15.1 |
   |          1 | 115.8 | 100.2 |  10.0 |
   |          2 |  74.9 |  61.6 |   6.2 |
   |          3 |  58.5 |  49.0 |   4.9 |
   |          4 |  51.1 |  43.2 |   4.3 |
   |          5 |  47.1 |  39.8 |   4.0 |
   +------------+-------+-------+-------+

   Error per algorithm
   +-----------+-------+-------+-------+
   | Algorithm | MRMSE |  MMAE | MMAPE |
   +-----------+-------+-------+-------+
   |       add |  52.7 |  45.8 |   4.6 |
   |      mask |  34.1 |  28.6 |   2.9 |
   |       xor |  54.0 |  48.0 |   4.8 |
   | crc-ccitt |  48.7 |  43.2 |   4.3 |
   |       md4 | 137.7 | 119.8 |  12.0 |
   |       md5 | 131.3 | 110.7 |  11.1 |
   |      sha1 | 110.5 |  93.0 |   9.3 |
   |    sha256 | 122.2 | 103.3 |  10.3 |
   +-----------+-------+-------+-------+

   For 3597-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+



Li                      Expires 13 November 2022               [Page 38]


Internet-Draft               Entropy Values                     May 2022


   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  31.7 |  25.2 |  2.5 |
   | crc-ccitt | 264.8 | 210.5 | 21.0 |
   |       md4 | 356.1 | 283.5 | 28.3 |
   |       md5 | 357.9 | 286.4 | 28.6 |
   |      sha1 | 358.9 | 284.9 | 28.5 |
   |    sha256 | 357.1 | 283.1 | 28.3 |
   +-----------+-------+-------+------+

   For 911-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  31.5 |  25.4 |  2.5 |
   | crc-ccitt | 163.2 | 133.3 | 13.3 |
   |       md4 | 181.9 | 144.9 | 14.5 |
   |       md5 | 190.0 | 150.5 | 15.0 |
   |      sha1 | 181.6 | 147.2 | 14.7 |
   |    sha256 | 180.3 | 143.8 | 14.4 |
   +-----------+-------+-------+------+

   For 3023-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  31.8 |  25.6 |  2.6 |
   | crc-ccitt | 238.2 | 191.1 | 19.1 |
   |       md4 | 320.7 | 254.9 | 25.5 |
   |       md5 | 319.7 | 253.8 | 25.4 |
   |      sha1 | 329.1 | 262.8 | 26.3 |
   |    sha256 | 328.0 | 261.6 | 26.2 |
   +-----------+-------+-------+------+

   For 2288-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  31.5 |  25.2 |  2.5 |
   | crc-ccitt | 190.8 | 156.0 | 15.6 |
   |       md4 | 282.4 | 225.8 | 22.6 |
   |       md5 | 279.8 | 220.4 | 22.0 |
   |      sha1 | 280.8 | 225.2 | 22.5 |
   |    sha256 | 281.9 | 223.5 | 22.3 |
   +-----------+-------+-------+------+

   For 2761-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+



Li                      Expires 13 November 2022               [Page 39]


Internet-Draft               Entropy Values                     May 2022


   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  32.1 |  25.4 |  2.5 |
   | crc-ccitt | 225.4 | 179.9 | 18.0 |
   |       md4 | 318.4 | 253.0 | 25.3 |
   |       md5 | 309.4 | 247.8 | 24.8 |
   |      sha1 | 304.6 | 244.0 | 24.4 |
   |    sha256 | 309.2 | 247.2 | 24.7 |
   +-----------+-------+-------+------+

   For 1845-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  31.4 |  25.0 |  2.5 |
   | crc-ccitt | 161.2 | 129.6 | 13.0 |
   |       md4 | 260.3 | 207.3 | 20.7 |
   |       md5 | 257.3 | 203.1 | 20.3 |
   |      sha1 | 259.9 | 206.1 | 20.6 |
   |    sha256 | 245.7 | 196.3 | 19.6 |
   +-----------+-------+-------+------+

   For 3508-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  31.4 |  25.2 |  2.5 |
   | crc-ccitt | 209.0 | 167.7 | 16.8 |
   |       md4 | 352.0 | 280.6 | 28.1 |
   |       md5 | 346.1 | 277.6 | 27.8 |
   |      sha1 | 357.4 | 284.2 | 28.4 |
   |    sha256 | 344.9 | 274.3 | 27.4 |
   +-----------+-------+-------+------+

   For 1963-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  31.4 |  25.3 |  2.5 |
   | crc-ccitt | 163.5 | 131.9 | 13.2 |
   |       md4 | 257.7 | 205.5 | 20.6 |
   |       md5 | 257.2 | 204.8 | 20.5 |
   |      sha1 | 264.4 | 213.0 | 21.3 |
   |    sha256 | 266.7 | 214.1 | 21.4 |
   +-----------+-------+-------+------+

   For 1168-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+



Li                      Expires 13 November 2022               [Page 40]


Internet-Draft               Entropy Values                     May 2022


   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  32.2 |  25.4 |  2.5 |
   | crc-ccitt | 107.5 |  86.1 |  8.6 |
   |       md4 | 198.6 | 158.4 | 15.8 |
   |       md5 | 203.3 | 161.8 | 16.2 |
   |      sha1 | 201.1 | 162.3 | 16.2 |
   |    sha256 | 201.4 | 159.0 | 15.9 |
   +-----------+-------+-------+------+

   For 2274-way ECMP, 16 bits of entropy:
   +-----------+-------+-------+------+
   |      Hash |  RMSE |   MAE | MAPE |
   +-----------+-------+-------+------+
   |      mask |  31.7 |  25.0 |  2.5 |
   | crc-ccitt | 208.9 | 166.0 | 16.6 |
   |       md4 | 277.2 | 221.8 | 22.2 |
   |       md5 | 279.8 | 224.9 | 22.5 |
   |      sha1 | 285.5 | 225.7 | 22.6 |
   |    sha256 | 288.4 | 229.6 | 23.0 |
   +-----------+-------+-------+------+

   For n-way ECMP, with 16 bits of entropy, across hash functions
   +--------+-------+-------+-------+
   | n-ways | MRMSE |  MMAE | MMAPE |
   +--------+-------+-------+-------+
   |   3597 | 287.8 | 228.9 |  22.9 |
   |    911 | 154.8 | 124.2 |  12.4 |
   |   3023 | 261.2 | 208.3 |  20.8 |
   |   2288 | 224.5 | 179.3 |  17.9 |
   |   2761 | 249.9 | 199.5 |  19.9 |
   |   1845 | 202.6 | 161.2 |  16.1 |
   |   3508 | 273.5 | 218.3 |  21.8 |
   |   1963 | 206.8 | 165.8 |  16.6 |
   |   1168 | 157.3 | 125.5 |  12.5 |
   |   2274 | 228.6 | 182.2 |  18.2 |
   +--------+-------+-------+-------+

Author's Address

   Tony Li
   Juniper Networks
   Email: tony.li@tony.li








Li                      Expires 13 November 2022               [Page 41]