IP Flow Information Accounting and Export Benchmarking Methodology
draft-ietf-bmwg-ipflow-meth-10
Revision differences
Document history
Date | Rev. | By | Action |
---|---|---|---|
2012-04-24
|
10 | (System) | IANA Action state changed to No IC |
2012-04-24
|
10 | Amy Vezza | State changed to RFC Ed Queue from Approved-announcement sent |
2012-04-23
|
10 | Amy Vezza | State changed to Approved-announcement sent from Approved-announcement to be sent |
2012-04-23
|
10 | Amy Vezza | IESG has approved the document |
2012-04-23
|
10 | Amy Vezza | Closed "Approve" ballot |
2012-04-23
|
10 | Amy Vezza | Ballot approval text was generated |
2012-04-23
|
10 | Amy Vezza | Ballot writeup was changed |
2012-04-23
|
10 | Ron Bonica | State changed to Approved-announcement to be sent from IESG Evaluation::AD Followup |
2012-04-23
|
10 | Ron Bonica | [Ballot Position Update] Position for Ronald Bonica has been changed to Yes from Discuss |
2012-04-23
|
10 | (System) | Sub state has been changed to AD Followup from Revised ID Needed |
2012-04-23
|
10 | Jan Novak | New version available: draft-ietf-bmwg-ipflow-meth-10.txt |
2012-04-12
|
09 | Cindy Morgan | State changed to IESG Evaluation::Revised ID Needed from Waiting for AD Go-Ahead |
2012-04-12
|
09 | Benoît Claise | [Ballot comment] Discussing with Ron, since I was an author for 2 versions of the individual draft, I will recuse myself. ========================================================================== Introduction … [Ballot comment] Discussing with Ron, since I was an author for 2 versions of the individual draft, I will recuse myself. ========================================================================== Introduction Monitoring of IP flows (Flow monitoring) is defined in the Architecture for IP Flow Information Export [RFC5470] and related IPFIX documents. Which documents? Do we expect the BMWG community to know about the relevant IPFIX documents. Please refer to "them" - Abstract mentions This document provides a methodology and framework for quantifying the performance impact of monitoring of IP flows on a network device and export of this information to a collector. It identifies the rate at which the IP flows are created, expired, and successfully exported as a new performance metric in combination with traditional throughput. The metric is only applicable to the devices compliant with the Architecture for IP Flow Information Export [RFC5470]. However, the introduction mentions: This document provides a methodology for measuring Flow monitoring performance so that network operators have a framework for measurements of impact on the network and network equipment. So if this document covers both "impact on the network and network equipment", it should be clearly mentioned in the abstract. Maybe this is what you mean by "export of this information to a collector", but it can be understood in different ways: impact on the Exporter, and/or Collector, and/or network. - A more complete understanding of the stress points of a particular device can be attained using this internal information and the tester MAY choose to gather this information during the measurement iterations. replace "device" by "DUT" - 2.1 Existing Terminology -> I would refer to RFC5101 instead of RFC5470 when possible. Because RFC5101 will be updated by RFC5101bis. And one important change in RFC5101bis will be the new "Flow" definition, which will be removed "IP". RFC5470 will most likely not have a bis version. - 2.2.5 Flow Export Rate Definition: The number of Cache entries that expire from the Cache (as defined by the Flow Expiration term) and are exported to the Collector within a measurement time interval. There SHOULD NOT be any export filtering, so that all the expired cache entries are exported. If there is export filtering and it can't be disabled, this needs to be noted. If you use to use RFC2119 terms in the definition, be consistent: replace "this needs to be noted. " by this MUST be noted". - 3.2 Device Applicability The Flow monitoring performance metric is applicable to network devices that implement [RFC5470] architecture. Replace the end with something similar to: "that implement RFC5101 and RFC5102, according to [RFC5470] architecture." After reading the entire draft, I see the sentence I was looking in section 4.3 The DUT MUST support the Flow monitoring architecture as specified by [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful results comparison due to the standardized export protocol. You should have something similar in section 3.2 - NetFlow is mentioned. At least refer to RFC3954. - "The Cache entries are expired from the Cache depending on the Cache configuration (ie, the Active and Inactive Timeouts, number of Cache entries and the Cache Size)" The cache entries is not a configuration parameter. - The DUT's export interface (connecting the Collector) MUST NOT be used for forwarding the test traffic but only for the Flow Export data containing the Flow Records. In all measurements, the export interface MUST have enough bandwidth to transmit Flow Export data without congestion. In other words, the export interface MUST NOT be a bottleneck during the measurement. I guess that the "the collector (interface) MUST NOT be a bottleneck during the measurement". Exactly like you wrote for the traffic receiver " The traffic receiver MUST have sufficient resources to measure all test traffic transferred successfully by the DUT." After reading section 4.4, I see this sentence: The Collector MUST be capable of capturing the export packets sent from the DUT at the full rate without losing any of them. This is a source of confusion for me in this draft. It seems that the information is fragmented throughout the draft... Therefore, the reading flow is sometimes not easy... - what is the difference between "Any such feature configuration MUST be part of the measurement report." and "All configurations MUST be fully documented." Should the latter say "All configurations MUST be fully documented in the measurement report"? - The DUT configuration and any existing Cache MUST be erased before application of any new configuration for the currently executed measurement. replace Cache by Cache entries? - Section 4.3.2 The Cache Size available to the DUT MUST be known and taken into account when designing the measurement as specified in section 5. What about Metering Process features that increase the cache size. Is this allowed? Should this be documented in the measurement report? Or should the cache size be set up to its maximum before starting the measurement? - Section 4.3.2 The configuration of the Metering Process MUST be recorded. -> MUST be included in the measurement report? - Section 4.3.3 The templates used by the tested implementations SHOULD be analysed and reported as part of the measurement report. Ideally only tests with same templates layout should be compared. "template layout" = Template Record in RFC5101. Please add to the terminology and use the term. - Section 4.3.3 Only benchmarks with the same transport layer protocol should be compared. should -> SHOULD - Section 4.3.4. For all the examples such as the following, please include the IPFIX Information Element. Flow Keys: Source IP address Destination IP address MPLS label (for MPLS traffic type only) Transport layer source port Transport layer destination port IP protocol number (IPv6 next header) IP type of service (IPv6 traffic class) Rational: - what does the MPLS label mean? In IPFIX IANA, we report mplsTopLabelStackSection, which is a combination of "The Label, Exp, and S fields". Note: clarify this as well in the section 4.3.6 - packet counters, byte counters: we have multiple IEs for those. Which one should the DUT chose? - IP addresses: IPv4 and/or IPv6? - 4.6 Frame Formats Flow monitoring itself is not dependent in any way on the media used on the input and output ports. Any media can be used as supported by the DUT and the test equipment. What about the export interface? - section 4.8 The used packet size SHOULD be part of the measurement report Why not a MUST? - section 5.1 I read multiple times the following sentences, and still could not understand the link between the first two. The number of unique Flow Keys sets that the traffic generator (sender) provides should be multiple times larger than the Cache Size. This ensures that the existing Cache entries are never updated before Flow Expiration and Flow Export. The Cache Size MUST be known in order to define the measurement circumstances properly. - Section 7 Flow Monitoring Accuracy Don't you have to say a few words about the accuracy of DUT that also does forwarding, i.e. the traffic analysis at the "traffic receiver" in figure 1 must also check that no packets were lost in the case of ingress monitoring. Note: there is also the case of egress monitoring. Interestingly, bidirectional is mentioned in the "Appendix A: Recommended Report Format", but not a single time in the draft. - Appendix A: would be great to mention which entries are SHOULD and MUST Note: not sure why it's an appendix, as there is normative text referring to each entries - Appendix A could be very helpful to poll the IPFIX-MIB (http://tools.ietf.org/html/draft-ietf-ipfix-rfc5815bis-03, currently in RFC-editor queue) and IPFIX-CONF (http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-10, currently in RFC-editor queue) as MAY in the measurement report. Note: [IPFIX-CONF] is almost ready, simply waiting for PSAMP-MIB. I believe it makes sense to wait for this RFC. - Appendix B. Not sure whether the use of "SHOULD" in the appendix is appropriate. To be checked. - Appendix B.6 Tests With Bidirectional Traffic Not sure why this is an appendix. The recommended report mentions Traffic Direction unidirectional, bidirectional Direction ingress, egress, both ... and the bidirectionality is only mentioned in the appendix? - Section 5.2 Traffic Generation The traffic generator needs to increment the Flow Keys values with each sent packet. This way each packet represents one Cache entry in the DUT Cache. Here is a comment I made to you years ago. You will find surprises if you increment the IP addresses by one in your generator, as opposed to random traffic, as the hash function are not optimized for incremental IP addresses. You should say a few words about this. |
2012-04-12
|
09 | Benoît Claise | [Ballot Position Update] Position for Benoit Claise has been changed to Recuse from Discuss |
2012-04-12
|
09 | Ron Bonica | [Ballot comment] The following are Benoit's DISCUSS comments: - Introduction The most significant performance parameter is the rate at which IP … [Ballot comment] The following are Benoit's DISCUSS comments: - Introduction The most significant performance parameter is the rate at which IP flows are created and expired in the network device's memory and exported to a collector One or multiple different rates? I guess different ones (but reading the document further will tell). So: The most significant performance parameters are the rates at which IP flows are created, expired in the network device's memory, and exported to a collector However, looking at the terminology section, it seems that you have only one benchmark metric: "Flow Monitoring Throughput". BMWG is about black box testing, but it doesn't mean that we don't have 3 different rates. The section 3.1 about "the Flow Monitoring Throughput" proves I'm right. Please improve the text. - See email "No active/inactive timeout definitions in any IPFIX RFCs? Idle versus inactive terminology? (part of draft-ietf-bmwg-ipflow-meth-09 review)" sent to the IPFIX WG. - Section 4.3.1 The (*) in Figure 2 designates the Observation Points in the default configuration. Other DUT Observation Points might be configured depending on the specific measurement needs as follows: a. ingress port/ports only b. egress port/ports only c. both ingress and egress If I refer to figure 2, there is no return traffic to the "traffic sender". Therefore, how could it be b. or c.? Am I dreaming or you had in the past a similar figure that explains that the return traffic could come back to the "traffic sender"? Figure 2 should be updated, or a new figure added, because, for egress, the traffic analysis must happen also on the "traffic sender" - Section 4.3.3 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as the protocol to use to format the Flow Export data You want a MUST here, as IP Flow = IPFIX at the IETF. Same remark for this sentence in 4.3 The DUT MUST support the Flow monitoring architecture as specified by [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful results comparison due to the standardized export protocol. Same remark for this sentence in 4.4 However if the Collector is also used to decode the Flow Export data then it SHOULD support IPFIX [RFC5101] for meaningful results However, looking at figure 1, you mention NetFlow, others. So you want to add that additional export mechanism MAY use the same benchmarking mechanism, i.e. NetFlow v9 [RFC3954] Comment (2012-04-10) - Introduction Monitoring of IP flows (Flow monitoring) is defined in the Architecture for IP Flow Information Export [RFC5470] and related IPFIX documents. Which documents? Do we expect the BMWG community to know about the relevant IPFIX documents. Please refer to "them" - Abstract mentions This document provides a methodology and framework for quantifying the performance impact of monitoring of IP flows on a network device and export of this information to a collector. It identifies the rate at which the IP flows are created, expired, and successfully exported as a new performance metric in combination with traditional throughput. The metric is only applicable to the devices compliant with the Architecture for IP Flow Information Export [RFC5470]. However, the introduction mentions: This document provides a methodology for measuring Flow monitoring performance so that network operators have a framework for measurements of impact on the network and network equipment. So if this document covers both "impact on the network and network equipment", it should be clearly mentioned in the abstract. Maybe this is what you mean by "export of this information to a collector", but it can be understood in different ways: impact on the Exporter, and/or Collector, and/or network. - A more complete understanding of the stress points of a particular device can be attained using this internal information and the tester MAY choose to gather this information during the measurement iterations. replace "device" by "DUT" - 2.1 Existing Terminology -> I would refer to RFC5101 instead of RFC5470 when possible. Because RFC5101 will be updated by RFC5101bis. And one important change in RFC5101bis will be the new "Flow" definition, which will be removed "IP". RFC5470 will most likely not have a bis version. - 2.2.5 Flow Export Rate Definition: The number of Cache entries that expire from the Cache (as defined by the Flow Expiration term) and are exported to the Collector within a measurement time interval. There SHOULD NOT be any export filtering, so that all the expired cache entries are exported. If there is export filtering and it can't be disabled, this needs to be noted. If you use to use RFC2119 terms in the definition, be consistent: replace "this needs to be noted. " by this MUST be noted". - 3.2 Device Applicability The Flow monitoring performance metric is applicable to network devices that implement [RFC5470] architecture. Replace the end with something similar to: "that implement RFC5101 and RFC5102, according to [RFC5470] architecture." After reading the entire draft, I see the sentence I was looking in section 4.3 The DUT MUST support the Flow monitoring architecture as specified by [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful results comparison due to the standardized export protocol. You should have something similar in section 3.2 - NetFlow is mentioned. At least refer to RFC3954. - "The Cache entries are expired from the Cache depending on the Cache configuration (ie, the Active and Inactive Timeouts, number of Cache entries and the Cache Size)" The cache entries is not a configuration parameter. - The DUT's export interface (connecting the Collector) MUST NOT be used for forwarding the test traffic but only for the Flow Export data containing the Flow Records. In all measurements, the export interface MUST have enough bandwidth to transmit Flow Export data without congestion. In other words, the export interface MUST NOT be a bottleneck during the measurement. I guess that the "the collector (interface) MUST NOT be a bottleneck during the measurement". Exactly like you wrote for the traffic receiver " The traffic receiver MUST have sufficient resources to measure all test traffic transferred successfully by the DUT." After reading section 4.4, I see this sentence: The Collector MUST be capable of capturing the export packets sent from the DUT at the full rate without losing any of them. This is a source of confusion for me in this draft. It seems that the information is fragmented throughout the draft... Therefore, the reading flow is sometimes not easy... - what is the difference between "Any such feature configuration MUST be part of the measurement report." and "All configurations MUST be fully documented." Should the latter say "All configurations MUST be fully documented in the measurement report"? - The DUT configuration and any existing Cache MUST be erased before application of any new configuration for the currently executed measurement. replace Cache by Cache entries? - Section 4.3.2 The Cache Size available to the DUT MUST be known and taken into account when designing the measurement as specified in section 5. What about Metering Process features that increase the cache size. Is this allowed? Should this be documented in the measurement report? Or should the cache size be set up to its maximum before starting the measurement? - Section 4.3.2 The configuration of the Metering Process MUST be recorded. -> MUST be included in the measurement report? - Section 4.3.3 The templates used by the tested implementations SHOULD be analysed and reported as part of the measurement report. Ideally only tests with same templates layout should be compared. "template layout" = Template Record in RFC5101. Please add to the terminology and use the term. - Section 4.3.3 Only benchmarks with the same transport layer protocol should be compared. should -> SHOULD - Section 4.3.4. For all the examples such as the following, please include the IPFIX Information Element. Flow Keys: Source IP address Destination IP address MPLS label (for MPLS traffic type only) Transport layer source port Transport layer destination port IP protocol number (IPv6 next header) IP type of service (IPv6 traffic class) Rational: - what does the MPLS label mean? In IPFIX IANA, we report mplsTopLabelStackSection, which is a combination of "The Label, Exp, and S fields". Note: clarify this as well in the section 4.3.6 - packet counters, byte counters: we have multiple IEs for those. Which one should the DUT chose? - IP addresses: IPv4 and/or IPv6? - 4.6 Frame Formats Flow monitoring itself is not dependent in any way on the media used on the input and output ports. Any media can be used as supported by the DUT and the test equipment. What about the export interface? - section 4.8 The used packet size SHOULD be part of the measurement report Why not a MUST? - section 5.1 I read multiple times the following sentences, and still could not understand the link between the first two. The number of unique Flow Keys sets that the traffic generator (sender) provides should be multiple times larger than the Cache Size. This ensures that the existing Cache entries are never updated before Flow Expiration and Flow Export. The Cache Size MUST be known in order to define the measurement circumstances properly. - Section 7 Flow Monitoring Accuracy Don't you have to say a few words about the accuracy of DUT that also does forwarding, i.e. the traffic analysis at the "traffic receiver" in figure 1 must also check that no packets were lost in the case of ingress monitoring. Note: there is also the case of egress monitoring. Interestingly, bidirectional is mentioned in the "Appendix A: Recommended Report Format", but not a single time in the draft. - Appendix A: would be great to mention which entries are SHOULD and MUST Note: not sure why it's an appendix, as there is normative text referring to each entries - Appendix A could be very helpful to poll the IPFIX-MIB (http://tools.ietf.org/html/draft-ietf-ipfix-rfc5815bis-03, currently in RFC- editor queue) and IPFIX-CONF (http://tools.ietf.org/html/draft-ietf-ipfix- configuration-model-10, currently in RFC-editor queue) as MAY in the measurement report. Note: [IPFIX-CONF] is almost ready, simply waiting for PSAMP-MIB. I believe it makes sense to wait for this RFC. - Appendix B. Not sure whether the use of "SHOULD" in the appendix is appropriate. To be checked. - Appendix B.6 Tests With Bidirectional Traffic Not sure why this is an appendix. The recommended report mentions Traffic Direction unidirectional, bidirectional Direction ingress, egress, both ... and the bidirectionality is only mentioned in the appendix? - Section 5.2 Traffic Generation The traffic generator needs to increment the Flow Keys values with each sent packet. This way each packet represents one Cache entry in the DUT Cache. Here is a comment I made to you years ago. You will find surprises if you increment the IP addresses by one in your generator, as opposed to random traffic, as the hash function are not optimized for incremental IP addresses. You should say a few words about this. |
2012-04-12
|
09 | Ron Bonica | Ballot comment text updated for Ronald Bonica |
2012-04-12
|
09 | Ron Bonica | [Ballot discuss] I am taking the very strange action of posting a DISCUSS on my own document. Benoit has some significant issues that need to … [Ballot discuss] I am taking the very strange action of posting a DISCUSS on my own document. Benoit has some significant issues that need to be worked out with the author and the WG. However, he must recuse because he was once a co-author of this draft. So, I will carry his DISCUSS and work towards resolving it. |
2012-04-12
|
09 | Ron Bonica | [Ballot Position Update] Position for Ronald Bonica has been changed to Discuss from Yes |
2012-04-12
|
09 | Gonzalo Camarillo | [Ballot Position Update] New position, No Objection, has been recorded for Gonzalo Camarillo |
2012-04-12
|
09 | Stewart Bryant | [Ballot comment] On the basis of a quick read and complete confidence that Benoit will work with the authors to make the draft perfect. |
2012-04-12
|
09 | Stewart Bryant | [Ballot Position Update] New position, No Objection, has been recorded for Stewart Bryant |
2012-04-12
|
09 | Pete Resnick | [Ballot comment] As usual, I find most of the uses of 2119 language in a document of this sort to be bizarre: I don't understand … [Ballot comment] As usual, I find most of the uses of 2119 language in a document of this sort to be bizarre: I don't understand how a statement like "MUST be part of the measurement report" constitutes something "required for interoperation or to limit behavior which has the potential for causing harm", and is not simply trying "to impose a particular method on implementors where the method is not required for interoperability." I'm hard pressed to find any occurrence of 2119 language in this document that is used as 2119 intended. |
2012-04-12
|
09 | Pete Resnick | [Ballot Position Update] New position, No Objection, has been recorded for Pete Resnick |
2012-04-11
|
09 | Wesley Eddy | [Ballot Position Update] New position, No Objection, has been recorded for Wesley Eddy |
2012-04-11
|
09 | Sean Turner | [Ballot Position Update] New position, No Objection, has been recorded for Sean Turner |
2012-04-11
|
09 | Robert Sparks | [Ballot Position Update] New position, No Objection, has been recorded for Robert Sparks |
2012-04-11
|
09 | Stephen Farrell | [Ballot Position Update] New position, No Objection, has been recorded for Stephen Farrell |
2012-04-11
|
09 | Brian Haberman | [Ballot Position Update] New position, No Objection, has been recorded for Brian Haberman |
2012-04-10
|
09 | Ralph Droms | [Ballot comment] A statement like this in section 1 begs for a little more explanation: The most significant performance parameter is the rate … [Ballot comment] A statement like this in section 1 begs for a little more explanation: The most significant performance parameter is the rate at which IP flows are created and expired in the network device's memory and exported to a collector. Is there a reference or some other justification for this statement? Could this statement simply be elided without losing the importance of the document? And here are a couple of minor editorial or clarification suggestions: In section 3.4.2: Mainly the Flow Export Rate caused by the test traffic during an [RFC2544] measurement MUST be known and reported. s/Mainly/Most importantly/ ?? In section 4.9.1 and 4.9.2, is it intended that the destination IP address recycles to the address for stream 1 after stream 10000? |
2012-04-10
|
09 | Ralph Droms | [Ballot Position Update] New position, No Objection, has been recorded for Ralph Droms |
2012-04-10
|
09 | Benoît Claise | [Ballot discuss] - Introduction The most significant performance parameter is the rate at which IP flows are created and expired in … [Ballot discuss] - Introduction The most significant performance parameter is the rate at which IP flows are created and expired in the network device's memory and exported to a collector One or multiple different rates? I guess different ones (but reading the document further will tell). So: The most significant performance parameters are the rates at which IP flows are created, expired in the network device's memory, and exported to a collector However, looking at the terminology section, it seems that you have only one benchmark metric: "Flow Monitoring Throughput". BMWG is about black box testing, but it doesn't mean that we don't have 3 different rates. The section 3.1 about "the Flow Monitoring Throughput" proves I'm right. Please improve the text. - See email "No active/inactive timeout definitions in any IPFIX RFCs? Idle versus inactive terminology? (part of draft-ietf-bmwg-ipflow-meth-09 review)" sent to the IPFIX WG. - Section 4.3.1 The (*) in Figure 2 designates the Observation Points in the default configuration. Other DUT Observation Points might be configured depending on the specific measurement needs as follows: a. ingress port/ports only b. egress port/ports only c. both ingress and egress If I refer to figure 2, there is no return traffic to the "traffic sender". Therefore, how could it be b. or c.? Am I dreaming or you had in the past a similar figure that explains that the return traffic could come back to the "traffic sender"? Figure 2 should be updated, or a new figure added, because, for egress, the traffic analysis must happen also on the "traffic sender" - Section 4.3.3 The Exporting Process SHOULD be configured with IPFIX [RFC5101] as the protocol to use to format the Flow Export data You want a MUST here, as IP Flow = IPFIX at the IETF. Same remark for this sentence in 4.3 The DUT MUST support the Flow monitoring architecture as specified by [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful results comparison due to the standardized export protocol. Same remark for this sentence in 4.4 However if the Collector is also used to decode the Flow Export data then it SHOULD support IPFIX [RFC5101] for meaningful results However, looking at figure 1, you mention NetFlow, others. So you want to add that additional export mechanism MAY use the same benchmarking mechanism, i.e. NetFlow v9 [RFC3954] |
2012-04-10
|
09 | Benoît Claise | [Ballot comment] - Introduction Monitoring of IP flows (Flow monitoring) is defined in the Architecture for IP Flow Information Export [ … [Ballot comment] - Introduction Monitoring of IP flows (Flow monitoring) is defined in the Architecture for IP Flow Information Export [RFC5470] and related IPFIX documents. Which documents? Do we expect the BMWG community to know about the relevant IPFIX documents. Please refer to "them" - Abstract mentions This document provides a methodology and framework for quantifying the performance impact of monitoring of IP flows on a network device and export of this information to a collector. It identifies the rate at which the IP flows are created, expired, and successfully exported as a new performance metric in combination with traditional throughput. The metric is only applicable to the devices compliant with the Architecture for IP Flow Information Export [RFC5470]. However, the introduction mentions: This document provides a methodology for measuring Flow monitoring performance so that network operators have a framework for measurements of impact on the network and network equipment. So if this document covers both "impact on the network and network equipment", it should be clearly mentioned in the abstract. Maybe this is what you mean by "export of this information to a collector", but it can be understood in different ways: impact on the Exporter, and/or Collector, and/or network. - A more complete understanding of the stress points of a particular device can be attained using this internal information and the tester MAY choose to gather this information during the measurement iterations. replace "device" by "DUT" - 2.1 Existing Terminology -> I would refer to RFC5101 instead of RFC5470 when possible. Because RFC5101 will be updated by RFC5101bis. And one important change in RFC5101bis will be the new "Flow" definition, which will be removed "IP". RFC5470 will most likely not have a bis version. - 2.2.5 Flow Export Rate Definition: The number of Cache entries that expire from the Cache (as defined by the Flow Expiration term) and are exported to the Collector within a measurement time interval. There SHOULD NOT be any export filtering, so that all the expired cache entries are exported. If there is export filtering and it can't be disabled, this needs to be noted. If you use to use RFC2119 terms in the definition, be consistent: replace "this needs to be noted. " by this MUST be noted". - 3.2 Device Applicability The Flow monitoring performance metric is applicable to network devices that implement [RFC5470] architecture. Replace the end with something similar to: "that implement RFC5101 and RFC5102, according to [RFC5470] architecture." After reading the entire draft, I see the sentence I was looking in section 4.3 The DUT MUST support the Flow monitoring architecture as specified by [RFC5470]. The DUT SHOULD support IPFIX [RFC5101] to allow meaningful results comparison due to the standardized export protocol. You should have something similar in section 3.2 - NetFlow is mentioned. At least refer to RFC3954. - "The Cache entries are expired from the Cache depending on the Cache configuration (ie, the Active and Inactive Timeouts, number of Cache entries and the Cache Size)" The cache entries is not a configuration parameter. - The DUT's export interface (connecting the Collector) MUST NOT be used for forwarding the test traffic but only for the Flow Export data containing the Flow Records. In all measurements, the export interface MUST have enough bandwidth to transmit Flow Export data without congestion. In other words, the export interface MUST NOT be a bottleneck during the measurement. I guess that the "the collector (interface) MUST NOT be a bottleneck during the measurement". Exactly like you wrote for the traffic receiver " The traffic receiver MUST have sufficient resources to measure all test traffic transferred successfully by the DUT." After reading section 4.4, I see this sentence: The Collector MUST be capable of capturing the export packets sent from the DUT at the full rate without losing any of them. This is a source of confusion for me in this draft. It seems that the information is fragmented throughout the draft... Therefore, the reading flow is sometimes not easy... - what is the difference between "Any such feature configuration MUST be part of the measurement report." and "All configurations MUST be fully documented." Should the latter say "All configurations MUST be fully documented in the measurement report"? - The DUT configuration and any existing Cache MUST be erased before application of any new configuration for the currently executed measurement. replace Cache by Cache entries? - Section 4.3.2 The Cache Size available to the DUT MUST be known and taken into account when designing the measurement as specified in section 5. What about Metering Process features that increase the cache size. Is this allowed? Should this be documented in the measurement report? Or should the cache size be set up to its maximum before starting the measurement? - Section 4.3.2 The configuration of the Metering Process MUST be recorded. -> MUST be included in the measurement report? - Section 4.3.3 The templates used by the tested implementations SHOULD be analysed and reported as part of the measurement report. Ideally only tests with same templates layout should be compared. "template layout" = Template Record in RFC5101. Please add to the terminology and use the term. - Section 4.3.3 Only benchmarks with the same transport layer protocol should be compared. should -> SHOULD - Section 4.3.4. For all the examples such as the following, please include the IPFIX Information Element. Flow Keys: Source IP address Destination IP address MPLS label (for MPLS traffic type only) Transport layer source port Transport layer destination port IP protocol number (IPv6 next header) IP type of service (IPv6 traffic class) Rational: - what does the MPLS label mean? In IPFIX IANA, we report mplsTopLabelStackSection, which is a combination of "The Label, Exp, and S fields". Note: clarify this as well in the section 4.3.6 - packet counters, byte counters: we have multiple IEs for those. Which one should the DUT chose? - IP addresses: IPv4 and/or IPv6? - 4.6 Frame Formats Flow monitoring itself is not dependent in any way on the media used on the input and output ports. Any media can be used as supported by the DUT and the test equipment. What about the export interface? - section 4.8 The used packet size SHOULD be part of the measurement report Why not a MUST? - section 5.1 I read multiple times the following sentences, and still could not understand the link between the first two. The number of unique Flow Keys sets that the traffic generator (sender) provides should be multiple times larger than the Cache Size. This ensures that the existing Cache entries are never updated before Flow Expiration and Flow Export. The Cache Size MUST be known in order to define the measurement circumstances properly. - Section 7 Flow Monitoring Accuracy Don't you have to say a few words about the accuracy of DUT that also does forwarding, i.e. the traffic analysis at the "traffic receiver" in figure 1 must also check that no packets were lost in the case of ingress monitoring. Note: there is also the case of egress monitoring. Interestingly, bidirectional is mentioned in the "Appendix A: Recommended Report Format", but not a single time in the draft. - Appendix A: would be great to mention which entries are SHOULD and MUST Note: not sure why it's an appendix, as there is normative text referring to each entries - Appendix A could be very helpful to poll the IPFIX-MIB (http://tools.ietf.org/html/draft-ietf-ipfix-rfc5815bis-03, currently in RFC-editor queue) and IPFIX-CONF (http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-10, currently in RFC-editor queue) as MAY in the measurement report. Note: [IPFIX-CONF] is almost ready, simply waiting for PSAMP-MIB. I believe it makes sense to wait for this RFC. - Appendix B. Not sure whether the use of "SHOULD" in the appendix is appropriate. To be checked. - Appendix B.6 Tests With Bidirectional Traffic Not sure why this is an appendix. The recommended report mentions Traffic Direction unidirectional, bidirectional Direction ingress, egress, both ... and the bidirectionality is only mentioned in the appendix? - Section 5.2 Traffic Generation The traffic generator needs to increment the Flow Keys values with each sent packet. This way each packet represents one Cache entry in the DUT Cache. Here is a comment I made to you years ago. You will find surprises if you increment the IP addresses by one in your generator, as opposed to random traffic, as the hash function are not optimized for incremental IP addresses. You should say a few words about this. |
2012-04-10
|
09 | Benoît Claise | [Ballot Position Update] New position, Discuss, has been recorded for Benoit Claise |
2012-04-09
|
09 | Russ Housley | [Ballot Position Update] New position, No Objection, has been recorded for Russ Housley |
2012-04-06
|
09 | Adrian Farrel | [Ballot Position Update] New position, No Objection, has been recorded for Adrian Farrel |
2012-04-03
|
09 | Amanda Baber | We understand that this document doesn't require any IANA actions. |
2012-04-03
|
09 | Amanda Baber | We understand that this document doesn't require any IANA actions. |
2012-03-26
|
09 | (System) | State changed to Waiting for AD Go-Ahead from In Last Call |
2012-03-16
|
09 | Samuel Weiler | Request for Last Call review by SECDIR is assigned to Ondřej Surý |
2012-03-16
|
09 | Samuel Weiler | Request for Last Call review by SECDIR is assigned to Ondřej Surý |
2012-03-15
|
09 | Jean Mahoney | Request for Last Call review by GENART is assigned to Joel Halpern |
2012-03-15
|
09 | Jean Mahoney | Request for Last Call review by GENART is assigned to Joel Halpern |
2012-03-12
|
09 | Amy Vezza | Last call sent |
2012-03-12
|
09 | Amy Vezza | State changed to In Last Call from Last Call Requested The following Last Call Announcement was sent out: From: The IESG To: IETF-Announce CC: Reply-To: … State changed to In Last Call from Last Call Requested The following Last Call Announcement was sent out: From: The IESG To: IETF-Announce CC: Reply-To: ietf@ietf.org Subject: Last Call: (IP Flow Information Accounting and Export Benchmarking Methodology) to Informational RFC The IESG has received a request from the Benchmarking Methodology WG (bmwg) to consider the following document: - 'IP Flow Information Accounting and Export Benchmarking Methodology' as an Informational RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2012-03-26. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract This document provides a methodology and framework for quantifying the performance impact of monitoring of IP flows on a network device and export of this information to a collector. It identifies the rate at which the IP flows are created, expired, and successfully exported as a new performance metric in combination with traditional throughput. The metric is only applicable to the devices compliant with the Architecture for IP Flow Information Export [RFC5470]. The file can be obtained via http://datatracker.ietf.org/doc/draft-ietf-bmwg-ipflow-meth/ IESG discussion can be tracked via http://datatracker.ietf.org/doc/draft-ietf-bmwg-ipflow-meth/ballot/ No IPR declarations have been submitted directly on this I-D. |
2012-03-12
|
09 | Ron Bonica | Placed on agenda for telechat - 2012-04-12 |
2012-03-12
|
09 | Ron Bonica | Last call was requested |
2012-03-12
|
09 | Ron Bonica | Last call announcement was generated |
2012-03-12
|
09 | Ron Bonica | State changed to Last Call Requested from AD Evaluation::AD Followup |
2012-03-12
|
09 | Ron Bonica | Ballot has been issued |
2012-03-12
|
09 | Ron Bonica | Ballot approval text was generated |
2012-03-12
|
09 | Ron Bonica | [Ballot Position Update] New position, Yes, has been recorded for Ronald Bonica |
2012-03-12
|
09 | Ron Bonica | Ballot writeup was changed |
2012-03-12
|
09 | Ron Bonica | Created "Approve" ballot |
2012-03-12
|
09 | Jan Novak | New version available: draft-ietf-bmwg-ipflow-meth-09.txt |
2012-03-09
|
08 | (System) | Sub state has been changed to AD Followup from Revised ID Needed |
2012-03-09
|
08 | Jan Novak | New version available: draft-ietf-bmwg-ipflow-meth-08.txt |
2012-03-05
|
07 | Ron Bonica | State changed to AD Evaluation::Revised ID Needed from Publication Requested |
2012-02-27
|
07 | Ron Bonica | Ballot writeup was generated |
2012-02-09
|
07 | Al Morton | Changed protocol writeup |
2012-02-09
|
07 | Al Morton | Submitted Jan 31 |
2012-02-09
|
07 | Al Morton | IETF state changed to Submitted to IESG for Publication from WG Document |
2012-02-09
|
07 | Al Morton | Publication Requested |
2012-02-09
|
07 | Al Morton | Annotation tag Other - see Comment Log set. Annotation tag Revised I-D Needed - Issue raised by WGLC cleared. |
2012-01-30
|
07 | Cindy Morgan | This is a Publication Request from bmwg: Document Title: IP Flow Information Accounting and Export Benchmarking Methodology Filename: draft-ietf-bmwg-ipflow-meth-07.txt Intended Status: Informational … This is a Publication Request from bmwg: Document Title: IP Flow Information Accounting and Export Benchmarking Methodology Filename: draft-ietf-bmwg-ipflow-meth-07.txt Intended Status: Informational (1.a) Who is the Document Shepherd for this document? Has the Document Shepherd personally reviewed this version of the document and, in particular, does he or she believe this version is ready for forwarding to the IESG for publication? Al Morton has reviewed the memo, it is ready (+/- a few minor edits, see the end of this write-up). This will be a difficult read for those not familiar with at least one of the areas covered here (IPFIX and Benchmarking), but the intended audience of testers should be adequately served. (1.b) Has the document had adequate review both from key WG members and from key non-WG members? Does the Document Shepherd have any concerns about the depth or breadth of the reviews that have been performed? The Current draft reflects extensive feedback, beginning with its first review at a session in Dublin. Throughout the process, non-wg people have been involved in reviews and WG last calls, especially the IPFIX WG (obviously). (1.c) Does the Document Shepherd have concerns that the document needs more review from a particular or broader perspective, e.g., security, operational complexity, someone familiar with AAA, internationalization or XML? No. (1.d) Does the Document Shepherd have any specific concerns or issues with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here. Has an IPR disclosure related to this document been filed? If so, please include a reference to the disclosure and summarize the WG discussion and conclusion on this issue. No concerns and No IPR disclosures. (1.e) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it? Quite a few bmwg participants and ipfix participants have given this a look and now concur with the results. It took several WGLC before the version reached consensus (with a few minor editorial changes). Examples of Test Implementation and Results were presented during development, which is compelling evidence of practicality. There were WGLCs yielding long lists of comments/issues to deal with, and this was finally accomplished. (1.f) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is entered into the ID Tracker.) No Appeals Threatened. (1.g) Has the Document Shepherd personally verified that the document satisfies all ID nits? (See the Internet-Drafts Checklist and http://tools.ietf.org/tools/idnits/). Boilerplate checks are not enough; this check needs to be thorough. Has the document met all formal review criteria it needs to, such as the MIB Doctor, media type and URI type reviews? All nits appear to be satisfied, with a few minor editorial changes needed. (1.h) Has the document split its references into normative and informative? Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the strategy for their completion? Are there normative references that are downward references, as described in [RFC3967]? If so, list these downward references to support the Area Director in the Last Call procedure for them [RFC3967]. The Refs are split and the Normative Refs are stable. No DownRefs. (1.i) Has the Document Shepherd verified that the document IANA consideration section exists and is consistent with the body of the document? If the document specifies protocol extensions, are reservations requested in appropriate IANA registries? Are the IANA registries clearly identified? If the document creates a new registry, does it define the proposed initial contents of the registry and an allocation procedure for future registrations? Does it suggest a reasonable name for the new registry? See [RFC5226]. If the document describes an Expert Review process has Shepherd conferred with the Responsible Area Director so that the IESG can appoint the needed Expert during the IESG Evaluation? IANA section exists, making no IANA requests, as is usual in bmwg. (1.j) Has the Document Shepherd verified that sections of the document that are written in a formal language, such as XML code, BNF rules, MIB definitions, etc., validate correctly in an automated checker? NA (1.k) The IESG approval announcement includes a Document Announcement Write-Up. Technical Summary For inter-networking devices that perform routing or switching as their primary function, the likely reduction in traffic-handling capacity when traffic monitoring is active continues to be a relevant question many years after it was first asked ("What happens when you turn-on Netflow?"). This document provides a methodology and framework for quantifying the performance impact of monitoring of IP flows on a network device and export of this information to a collector. It identifies the rate at which the IP flows are created, expired, and successfully exported as a new performance metric in combination with traditional throughput. The metric is only applicable to the devices compliant with the Architecture for IP Flow Information Export [RFC5470]. The methods are applicable to both inter-networking devices that forward traffic and other devices that simply monitor traffic with non-intrusive access to transmission facilities. The Forwarding Plane and Monitoring Plane represent two separate functional blocks, each with its own performance capability. The Forwarding Plane handles user data packets and is fully characterised by the metrics defined by [RFC2544]. The Monitoring Plane handles Flows which reflect the analysed traffic. The metric for Monitoring Plane performance is Flow Export Rate, and the benchmark is the Flow Monitoring Throughput. Working Group Summary Quite a few bmwg participants and ipfix participants have given this a look and now concur with the results. Examples of Test Implementation and Results were presented during development, which is compelling evidence of practicality. There were WGLCs yielding long lists of comments/issues to deal with, and this was finally accomplished. It took several WGLCs before this version reached consensus (with a few minor editorial changes). Document Quality All would agree that Paul Aitken provided very careful and complete reviews throughout the development process; he left no stone unturned. -=-=-=-=-=-=-=-=-=-=-=- Minor Editorial Points: Sec3.3 s/each with it's own performance/each with its own performance/ Sec3.4.2, last para s/The details how/The details of how/ Sec4 s/4. Measurement Set Up/4. Measurement Set-up/ Section 6.4 and 6.5 >>> since these are options of 6.3, it makes more sense if they are numbered 6.3.1 and 6.3.2: s/6.4/6.3.1/ s/6.5/6.3.5/ everywhere on pages 24 and 25 Sec 7 s/c. all the possible Flow Record fields values/c. all the possible Flow Record field values/ Sec 8 s/Packet per flow/Packets per flow/ and s/Be required to process/be required to process/ |
2012-01-30
|
07 | Cindy Morgan | Draft added in state Publication Requested |
2012-01-30
|
07 | Cindy Morgan | [Note]: 'Al Morton (acmorton@att.com) is the document shepherd.' added |
2012-01-30
|
07 | (System) | New version available: draft-ietf-bmwg-ipflow-meth-07.txt |
2012-01-22
|
06 | (System) | New version available: draft-ietf-bmwg-ipflow-meth-06.txt |
2011-12-06
|
05 | (System) | New version available: draft-ietf-bmwg-ipflow-meth-05.txt |
2011-12-05
|
07 | Al Morton | IETF state changed to WG Document from In WG Last Call |
2011-12-05
|
07 | Al Morton | 5 Issues need to be closed |
2011-12-05
|
07 | Al Morton | Annotation tag Revised I-D Needed - Issue raised by WGLC set. |
2011-10-02
|
04 | (System) | New version available: draft-ietf-bmwg-ipflow-meth-04.txt |
2011-07-11
|
03 | (System) | New version available: draft-ietf-bmwg-ipflow-meth-03.txt |
2011-06-21
|
02 | (System) | New version available: draft-ietf-bmwg-ipflow-meth-02.txt |
2011-05-30
|
07 | Al Morton | LC ends June 20, 2011 |
2011-05-30
|
07 | Al Morton | IETF state changed to In WG Last Call from WG Document |
2011-04-15
|
01 | (System) | New version available: draft-ietf-bmwg-ipflow-meth-01.txt |
2010-12-13
|
00 | (System) | New version available: draft-ietf-bmwg-ipflow-meth-00.txt |