Scenarios and Simulation Results of PCE in a Native IP Network
draft-ietf-teas-native-ip-scenarios-12

Note: This ballot was opened for revision 08 and is now closed.

Deborah Brungard Yes

Benjamin Kaduk (was Discuss) No Objection

Comment (2019-10-25 for -11)
Thanks for the updates in the -11!
This version does a much better job at convincing the reader that CCDR
can be an effective solution to the stated problems in real-world networks.

(Adam Roach) No Objection

Magnus Westerlund (was Discuss) No Objection

Alissa Cooper Abstain

Comment (2019-10-02 for -09)
I agree with Éric and Alvaro. It's not clear what is gained by publishing this as an RFC rather than a research paper, especially given the lack of support in the WG.

Roman Danyliw Abstain

Comment (2019-10-02 for -09)
==[ Summary
I am balloting as an ABSTAIN because the primary contribution of this draft appears to be the simulation results.  However, these results appear to be already published in greater detail in http://ieeexplore.ieee.org/document/8657733.

I have no commentary on the utility of the CDDR Scenarios found in Section 3.

==[ Details

I have reservations about the approach taken in Section 4.  As a stand-alone write-up, it insufficiently describes the simulations in question.  Furthermore, the content in this section copies verbatim or re-phrases the source material of these simulations from an already published academic paper (without citation).  See http://ieeexplore.ieee.org/document/8657733.  In particular:

-- The entirely of Section 4.1 (text and diagrams) is a cut-and-paste from Section V.B of the academic paper.  

-- The simulation results of Section 4.4 in this draft align with the analysis in Section V.C of the academic paper.  

-- The simulation results of Section 4.5 in this draft align with analysis in Section V.D of the academic paper.

Without the benefit of the academic paper cited above, I would have had the following feedback on Section 4:

-- Section 4.  What use case is being simulated isn’t clear.  The text is Section 3.1 states that “Section 4 of this document describes the simulation results of this use case”.  However, the topology for the simulation described in Section 4.2 looks different than the one in Figure 1 (of Section 3.1).  

-- Section 4.  A few questions about the simulation design:
Only the results were presented.  How were the simulations constructed and results evaluated?  

Section 4.0 states that this section “illustrate[s the] CCDR algorithm”.  Where is that algorithm defined?  Is that Section 7 of draft-ietf-teas-pce-native-ip or http://ieeexplore.ieee.org/document/8657733?

What specific algorithms was used to create the contrasting charts in Figures 7 and 8?  

Section 4.2 said that “[t]he number of links connecting one edge node to the set of core nodes is randomly between 2 to 30, and the total number of links is more than 20000.”  Section 4.3 said that “[i]n the CCDR simulation, the dimension of the traffic matrix is 500*500.  About 20% links are overloaded when the Open Shortest Path First (OSPF) protocol is used in the network.”, What is the basis of these choices?  Are they representative of a given use case?

What is the relationship between the Sections 4.2 and 4.3, and the simulation results in Sections 4.4 and 4.5

Additional feedback includes:

-- Section 5.  The purpose of this section isn’t clear.

-- Missing references
Section 1.  A reference to CCDR is needed.  Also, given that CCDR is the basis of the simulation, it needs to be normative.

Section 5.  A reference is needed for the IEEE document.

Barry Leiba (was No Objection) Abstain

Comment (2019-10-03 for -09)
In light of Roman's comments about the academic paper, I agree and join in the "Abstain" group.  It would be better just to have the other documents refer to the academic paper, rather than to republish a portion of that paper in the RFC series.

Alvaro Retana Abstain

Comment (2019-09-27 for -08)
The integration of distributed protocols and centralized control in a Hybrid network is not a new topic.  I feel conflicted about the publication of this document as an RFC because it doesn't contribute much to the existing literature...except maybe for the explicit suggestion of using "PCE in a native IP network".  Specifically, the interaction of the distributed and centralized components can result in added operational complexity and new security vulnerabilities.  Neither of these issues are mentioned.  

I appreciate the work that the authors have put into this document, but because of it being incomplete and only having "tepid support" [1] from the WG, I have decided to ABSTAIN and not stand in the way of publication.

[1] https://datatracker.ietf.org/doc/draft-ietf-teas-native-ip-scenarios/shepherdwriteup/

Martin Vigoureux Abstain

Comment (2019-10-03 for -09)
No email
send info
Thank you Roman for finding the paper. It seems to me that there is much more details in the paper than in the draft. As such I don't really see the value of publishing a subset as a draft and then an RFC.

Éric Vyncke Abstain

Comment (2019-10-01 for -09)
I share Alvaro's and TSV reviewer feeling about this is not a usual RFC but rather a scientific paper (even if the results are not really a surprise). So, I am abstaining.

Some comments though:
- why using OSPF and not IS-IS for the comparison ? Not that it would change a lot IMHO
- when using generated topologies, little is written on how it is generated (as it could introduce some bias changing the results of section 4.4)
- using the new format for XML2RFC could have included SVG for graphics

Interesting read anyway

-éric