Problem Statement: Overlays for Network Virtualization
RFC 7364

[Ballot comment]
The document is weirdly though non-specifically ipv4-centric. I don't think there are any particular fixes to be applied. I would observe however that address reuse while common in parallel rfc1918 addressing planes would not I imagine be very common in ipv6 in the umbering plans of ipv6 enabled DCs. that the longest possible route is not a /32, and that a signficant scaling consideration with L3 --> L2 mappings is the duplication between the arp cache and the NDP cache.

[Ballot comment]
Not much OPS feedback in this draft. I'm dying to see the "Operational Requirements submitted for IESG review" chartered item.

Editorial:
"Tenant Systems" should not be capitalized. Alternatively, you can define the term.
Please expand ARMD
Explain/Expand: C-VID, B-VID, I-VID

And here is Melinda's feedback from OPS-DIR:
I was asked to perform an OPS-DIR review of
draft-ietf-nvo3-overlay-problem-statement.

The document specifically targets multitenancy in large data
center networks, describing problems arising from that
scenario and how they may be addressed by overlay networks.
That this document made it through working group last call
at all should be seen as a major political accomplishment,
given the level of rancor in the working group, and much
respect is due to the chairs and the document authors for
getting this done.

The underlying assumption is that these virtual networks
will provide traffic isolation.

Minor issues:

Section 3.1: "Cloud computing" - the document would benefit
from eliminating that terminology and just describing the
scenario ("Some service providers offer elastic services
... ").  "Cloud" is imprecise and evocative of marketing
jargon.  We can talk about the need for dynamic provisioning
more carefully, I think.

Section 3.2, second sentence: "A VM can be migrated from
one server to another, [ ... ]."  I'm afraid it's servers
all the way down - may be clearer to say that VMs may be
migrated between hypervisors.

An operational consideration for this section (3.2) is that
there may be state associated with specific data flows to a
VM that is not on the VM - that's resident on some sort of
middlebox (firewall, application proxy, accelerator, cache,
etc.).  I tend to think that network state will, in
practice, be topologically close to the VM, but care must be
taken.

Doesn't really matter but it appears that the section header
for section 3.6 is marked up incorrectly (font and bolding).

3.7 is probably one of the clearest descriptions I've seen
of this issue - well done.

10: I'm not sure the security considerations are quite right, or
at least not the discussion of data plane security issues.
What are the characteristics of an overlay network that
differ from a physical network or VPN, and how do they
impact design decisions for the overlay?

Also, may be worth saying something about data leakage from
interception of control plane traffic (what inferences can
be made from changes in topology, etc.?).

[Ballot comment]

The nodes of a virtual network, once running, can look
after securing their own traffic. That might lead one to
say that nvo3 traffic isolation doesn't need to consider
confidentiality. However, if the nodes in a virtual network
are VMs and if VMs can be moved, then any secrets required
for the virtual network to secure its traffic will be
exposed to the underlay during the move.

I'm not clear if this wg will try address that issue or
not. Section 10 does say that some environments might be
concerned about confidentiality but is vague about whether
or not the wg will work on the topic.

Such a confidentiality service isn't a panacea of course,
the underlay components providing the confidentiality
service could leak the relevant keys, but it could still be
useful nonetheless. (BTW, I've no idea if it'd make sense
to have such a service that's separated from whatever
technology is used to move the VM or not.)

So I was wondering: will the wg actually define such a
confidentiality service or not? The response is that yes,
this'll be considered for the requirements documents
which is fine.

Note that I'm not trying to insist on a "yes" answer, even
though I think that'd be good. Even a "maybe, and that'll
be answered in the requirements specs before we re-charter"
would be ok. But regardless of the answer, I think it'd be
good to at least note this issue in the security
considerations section.

- 4.1, bullets: I found the use of ingress/egress
non-intuitive here. You mean ingress to the underlay and
egress from the underlay, right? It'd be good to explicitly
say that, though I figured it out eventually (or not, if
I'm wrong above:-)

[Ballot comment]
Minor nit:
  While an overlay-based approach may address some of the
  "pain points" that were raised in ARMD (e.g., better support for
  multi-tenancy).  Analysis will be needed to understand the scaling
  tradeoffs of an overlay based approach compared with existing
  approaches.

I think you want a comma between these two chunks; otherwise it doesn't really parse.

In 5.7, trill-fine-labeling is in the RFC editor queue, so I think that should be described as completed work, rather than something TRILL is investigating.

In general this draft is very clearly written, and does a good job of analyzing the problem space.  Thanks for doing such a good job on it!

[Ballot comment]
Thank you for writing this document. It is well written and easy to read, and documents the space well.

I had one question when reading Section 4. I was wondering why MTU was not mentioned, MTU issues being one of the impacts of overlay designs.

[Ballot comment]
Thanks for this document which I believe is a major step towards scoping
and documenting the real problems in this space.  I have a number of
fairly editorial concerns that I hope you can work through with your AD
and document shepherd.

---

In reading this document, I found it difficult to distinguish the
requirements that arrise from the provision of multiple virtual networks
on a common infrastrucutre (traffic isolation, address space isolation,
virtual network creation and configuration) from those that are specific
to the NVO3 scope (massive scaling, multi-tenancy on individual physical
servers, no constraints on physical location of hosted services).

---

[I-D.ietf-nvo3-framework] is used as a normative reference because it
defines terminology used in this document.

---

I would move the definiton of "in-band virtual network" from section 2
to section 5.3 (the only place the term is used) to avoid complicating
the definitions with concepts that appear to only be applied to L2
networks.

---

Why is the example of an Overlay Virtual Network in section picked from
the layer 2 space when this work is supposed to consider only layer 3
overlays?

OTOH, since this term is not used anywhere in the document, I suggest
deleting it.

---

I believe section 3.1 could be rewritten without the need to say "cloud"
or "elastic services".  This would be helpful because those marketting
phrases do not add to the meaning.

I think the final sentence of the paragraph captures the issues, but
could be pulled out into a little more explanation of what happens and
what problems it causes.

---

Section 5.3 uses the terms C-VLAN, S-VLAN, and B-VLAN, but only C-VLAN
has been defined.

--

Section 10 seems to me to be missing the impact that one virtual network
might be able to have on another (for example by stressing network
resources to cause undesirable VM mobility, or by consuming shared
resources to make b/w or CPU unavailable).

This is a type of self-consuming DoS.

[Ballot discuss]

The nodes of a virtual network, once running, can look
after securing their own traffic. That might lead one to
say that nvo3 traffic isolation doesn't need to consider
confidentiality. However, if the nodes in a virtual network
are VMs and if VMs can be moved, then any secrets required
for the virtual network to secure its traffic will be
exposed to the underlay during the move.

I'm not clear if this wg will try address that issue or
not. Section 10 does say that some environments might be
concerned about confidentiality but is vague about whether
or not the wg will work on the topic.

Such a confidentiality service isn't a panacea of course,
the underlay components providing the confidentiality
service could leak the relevant keys, but it could still be
useful nonetheless. (BTW, I've no idea if it'd make sense
to have such a service that's separated from whatever
technology is used to move the VM or not.)

So the discuss point is: will the wg actually define such a
confidentiality service or not?

Note that I'm not trying to insist on a "yes" answer, even
though I think that'd be good. Even a "maybe, and that'll
be answered in the requirements specs before we re-charter"
would be ok. But regardless of the answer, I think it'd be
good to at least note this issue in the security
considerations section.

[Ballot comment]

- 4.1, bullets: I found the use of ingress/egress
non-intuitive here. You mean ingress to the underlay and
egress from the underlay, right? It'd be good to explicitly
say that, though I figured it out eventually (or not, if
I'm wrong above:-)

[Ballot comment]
I did have one comment. Please consider it along with any other comments you receive.

This draft uses the word "cloud", which has been an extremely imprecise term in the IETF.

The second occurrence in Section 3.1 provides some guidance on which of the (many) aspects of clouds you're talking about:

  Cloud computing involves on-demand provisioning of resources for
  multi-tenant environments.

Could this explanation be moved to Section 1, with the first occurrence of "cloud"?

IESG/Authors/WG Chairs:

IANA has reviewed draft-ietf-nvo3-overlay-problem-statement-03, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any IANA actions.

If this assessment is not accurate, please respond as soon as possible.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Problem Statement: Overlays for Network Virtualization) to Informational RFC


The IESG has received a request from the Network Virtualization Overlays
WG (nvo3) to consider the following document:
- 'Problem Statement: Overlays for Network Virtualization'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2013-06-19. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  This document describes issues associated with providing multi-
  tenancy in large data center networks and how these issues may be
  addressed using an overlay-based network virtualization approach.  A
  key multi-tenancy requirement is traffic isolation, so that one
  tenant's traffic is not visible to any other tenant.  Another
  requirement is address space isolation, so that different tenants can
  use the same address space within different virtual networks.
  Traffic and address space isolation is achieved by assigning one or
  more virtual networks to each tenant, where traffic within a virtual
  network can only cross into another virtual network in a controlled
  fashion (e.g., via a configured router and/or a security gateway).
  Additional functionality is required to provision virtual networks,
  associating a virtual machine's network interface(s) with the
  appropriate virtual network, and maintaining that association as the
  virtual machine is activated, migrated and/or deactivated.  Use of an
  overlay-based approach enables scalable deployment on large network
  infrastructures.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-nvo3-overlay-problem-statement/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-nvo3-overlay-problem-statement/ballot/


No IPR declarations have been submitted directly on this I-D.

draft-ietf-nvo3-overlay-problem-statement-03.txt

Document Shepherd Write-Up

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

  Informational.

  This is appropriate as the draft describes the issues associated
  with providing multi-tenancy in large data centers, as well as
  work areas for the attention of the NVO3 working group. It does not
  specify new protocol elements, but rather provides the background
  requirements and architecture work in this area.

  The intended status is properly indicated.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

This document describes issues associated with providing multi-
  tenancy in large data center networks and how these issues may be
  addressed using an overlay-based network virtualization approach.


Working Group Summary

  The NVO3 working group is chartered to consider approaches to multi-tenancy
  that reside at the
    network layer rather than using traditional isolation mechanisms
    that rely on the underlying layer 2 technology (e.g., VLANs).
    The NVO3 WG will determine which types of connectivity services
    are needed by typical DC deployments (for example, IP and/or
    Ethernet). The basis for this work is a problem statement that identifies the
  key issues that must be considered and that are challenging to current data
  centre deployments, and must be considered as a part of a standardised approach.


  There are no IPR declarations on the draft.

Document Quality


  I have no concerns about the quality of the document. I believe it represents
  WG consensus, and it has been widely reviewed and discussed on the list since
  formation of the NVO3 working group.

  The document does not specify any MIB changes or additions which would need review.


Personnel

  The document shepherd is Matthew Bocci (matthew.bocci@alcatel-lucent.com).
  The responsible Area Director is Stewart Bryant (stbryant@cisco.com).

(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

  The document shepherd reviewed the document. I had no significant technical
  or editorial comments. There is a change log at the end of the document that
  should be removed prior to publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

  No concerns. The document has received adequate review. The document has
  been developed within the WG and reviewed over a
  period of a number of IETFs, as well as being a major focus of the BoF
  that led to the creation of the NVO3 working group.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

  No further review required.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

  No specific concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

  Each author listed in the Authors Addresses section has personally indicated that
  they are not aware of any IPR that has not already been declared in accordance
  with BCP 78 and 79.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

  None

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it?

    I am comfortable that the document represents WG consensus and has
    been reviewed by a reasonable number of active WG participants. It has been
    discussed over a long period , both in face to face IETF meetings
    and on the list. It received a number of comments in WG last call that
    were addressed by the authors.

    The document is the result of the combination of text from an original problem
  statement draft, that was used as a basis for the formation of the NVO3 working
  group, and the input of others in who felt that the original
  draft did not consider L3VPN solutions and issues sufficiently. A design team
  with a new editor was formed to resolve these comments and co-edit the
  combined draft. I am comfortable that the resulting draft has WG consensus.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

  None indicated.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

      ID-Nits passes.


(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

  There are no relevant formal review criteria.

(13) Have all references within this document been identified as
either normative or informative?

  Yes. All references are explicitly identified as informative.

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

  No

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in the
Last Call procedure.

  No. All references are informative.

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

  This document does not change the status of any existing RFCs.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

  There are no IANA actions.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

  There are no IANA actions.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

  There are no sections containing formal language that needs reviewing.

draft-ietf-nvo3-overlay-problem-statement-03.txt

Document Shepherd Write-Up

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

  Informational.
 
  This is appropriate as the draft describes the issues associated
  with providing multi-tenancy in large data centers, as well as
  work areas for the attention of the NVO3 working group. It does not
  specify new protocol elements, but rather provides the background
  requirements and architecture work in this area.
 
  The intended status is properly indicated.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

This document describes issues associated with providing multi-
  tenancy in large data center networks and how these issues may be
  addressed using an overlay-based network virtualization approach.


Working Group Summary

  The NVO3 working group is chartered to consider approaches to multi-tenancy
  that reside at the
    network layer rather than using traditional isolation mechanisms
    that rely on the underlying layer 2 technology (e.g., VLANs).
    The NVO3 WG will determine which types of connectivity services
    are needed by typical DC deployments (for example, IP and/or
    Ethernet). The basis for this work is a problem statement that identifies the
  key issues that must be considered and that are challenging to current data
  centre deployments, and must be considered as a part of a standardised approach.
     

  There are no IPR declarations on the draft.

Document Quality

   
  I have no concerns about the quality of the document. I believe it represents
  WG consensus, and it has been widely reviewed and discussed on the list since
  formation of the NVO3 working group.

  The document does not specify any MIB changes or additions which would need review.

   
Personnel

  The document shepherd is Matthew Bocci (matthew.bocci@alcatel-lucent.com).
  The responsible Area Director is Stewart Bryant (stbryant@cisco.com).

(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

  The document shepherd reviewed the document. I had no significant technical
  or editorial comments. There is a change log at the end of the document that
  should be removed prior to publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

  No concerns. The document has received adequate review. The document has
  been developed within the WG and reviewed over a
  period of a number of IETFs, as well as being a major focus of the BoF
  that led to the creation of the NVO3 working group.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

  No further review required.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

  No specific concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

  Each author listed in the Authors Addresses section has personally indicated that
  they are not aware of any IPR that has not already been declared in accordance
  with BCP 78 and 79.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

  None
 
(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it?

    I am comfortable that the document represents WG consensus and has
    been reviewed by a reasonable number of active WG participants. It has been
    discussed over a long period , both in face to face IETF meetings
    and on the list. It received a number of comments in WG last call that
    were addressed by the authors.

    The document is the result of the combination of text from an original problem
  statement draft, that was used as a basis for the formation of the NVO3 working
  group, and the input of others in who felt that the original
  draft did not consider L3VPN solutions and issues sufficiently. A design team
  with a new editor was formed to resolve these comments and co-edit the
  combined draft. I am comfortable that the resulting draft has WG consensus.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

  None indicated.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

      ID-Nits passes.


(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

  There are no relevant formal review criteria.

(13) Have all references within this document been identified as
either normative or informative?

  Yes. All references are explicitly identified as informative.

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

  No

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in the
Last Call procedure.

  No. All references are informative.

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

  This document does not change the status of any existing RFCs.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

  There are no IANA actions.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

  There are no IANA actions.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

  There are no sections containing formal language that needs reviewing.