Methodology for Benchmarking Session Initiation Protocol (SIP) Devices: Basic Session Setup and Registration
draft-ietf-bmwg-sip-bench-meth-12
Revision differences
Document history
Date | Rev. | By | Action |
---|---|---|---|
2015-04-06
|
12 | (System) | RFC Editor state changed to AUTH48-DONE from AUTH48 |
2015-03-23
|
12 | (System) | RFC Editor state changed to AUTH48 from RFC-EDITOR |
2015-03-12
|
12 | (System) | RFC Editor state changed to RFC-EDITOR from EDIT |
2015-02-05
|
12 | Amy Vezza | IESG state changed to RFC Ed Queue from Approved-announcement sent |
2015-02-05
|
12 | (System) | RFC Editor state changed to EDIT |
2015-02-05
|
12 | (System) | Announcement was received by RFC Editor |
2015-02-04
|
12 | (System) | IANA Action state changed to No IC from In Progress |
2015-02-04
|
12 | (System) | IANA Action state changed to In Progress |
2015-02-04
|
12 | Amy Vezza | IESG state changed to Approved-announcement sent from Approved-announcement to be sent |
2015-02-04
|
12 | Amy Vezza | IESG has approved the document |
2015-02-04
|
12 | Amy Vezza | Closed "Approve" ballot |
2015-02-03
|
12 | Joel Jaeggli | Ballot writeup was changed |
2015-02-03
|
12 | Joel Jaeggli | IESG state changed to Approved-announcement to be sent from Approved-announcement to be sent::Point Raised - writeup needed |
2015-02-03
|
12 | Joel Jaeggli | Ballot approval text was generated |
2014-11-28
|
12 | Jean Mahoney | Closed request for Last Call review by GENART with state 'No Response' |
2014-11-12
|
12 | Vijay Gurbani | IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed |
2014-11-12
|
12 | Vijay Gurbani | New version available: draft-ietf-bmwg-sip-bench-meth-12.txt |
2014-11-11
|
11 | Gunter Van de Velde | Closed request for Last Call review by OPSDIR with state 'No Response' |
2014-10-30
|
11 | Cindy Morgan | IESG state changed to Approved-announcement to be sent::Point Raised - writeup needed from IESG Evaluation |
2014-10-30
|
11 | Jari Arkko | [Ballot Position Update] New position, No Objection, has been recorded for Jari Arkko |
2014-10-29
|
11 | Richard Barnes | [Ballot comment] It seems like it might be worth noting for the media-on-DUT cases that the degree of SIP performance degradation due to media might … [Ballot comment] It seems like it might be worth noting for the media-on-DUT cases that the degree of SIP performance degradation due to media might differ depending on factors like what the DUT does with the media (e.g., relaying vs. transcoding), the type of media (audio vs. video vs. data), and the codec used for the media. There may also be cases where there is no performance impact, if the DUT has dedicated media-path hardware. |
2014-10-29
|
11 | Richard Barnes | Ballot comment text updated for Richard Barnes |
2014-10-29
|
11 | Richard Barnes | [Ballot Position Update] New position, No Objection, has been recorded for Richard Barnes |
2014-10-29
|
11 | Alissa Cooper | [Ballot comment] = Section 4.1 = "It is best ; off to start with the defaults (w = 0.10 … [Ballot comment] = Section 4.1 = "It is best ; off to start with the defaults (w = 0.10 and ; r >= 10)" I thought the default was r = 100? |
2014-10-29
|
11 | Alissa Cooper | [Ballot Position Update] New position, No Objection, has been recorded for Alissa Cooper |
2014-10-29
|
11 | Martin Stiemerling | [Ballot Position Update] New position, No Objection, has been recorded for Martin Stiemerling |
2014-10-28
|
11 | Barry Leiba | [Ballot Position Update] New position, No Objection, has been recorded for Barry Leiba |
2014-10-28
|
11 | Adrian Farrel | [Ballot Position Update] New position, No Objection, has been recorded for Adrian Farrel |
2014-10-28
|
11 | Kathleen Moriarty | [Ballot comment] Since this is a set of benchmarks for use in a lab environment, I agree with the security considerations section statement that there … [Ballot comment] Since this is a set of benchmarks for use in a lab environment, I agree with the security considerations section statement that there are no considerations when used in the way described. Thanks. |
2014-10-28
|
11 | Kathleen Moriarty | [Ballot Position Update] New position, No Objection, has been recorded for Kathleen Moriarty |
2014-10-27
|
11 | Benoît Claise | [Ballot Position Update] New position, No Objection, has been recorded for Benoit Claise |
2014-10-27
|
11 | Alia Atlas | [Ballot Position Update] New position, No Objection, has been recorded for Alia Atlas |
2014-10-20
|
11 | Brian Haberman | [Ballot comment] I trust the shepherding AD and his review of this document. |
2014-10-20
|
11 | Brian Haberman | [Ballot Position Update] New position, No Objection, has been recorded for Brian Haberman |
2014-10-19
|
11 | Joel Jaeggli | IESG state changed to IESG Evaluation from Waiting for Writeup |
2014-10-19
|
11 | Joel Jaeggli | Ballot has been issued |
2014-10-19
|
11 | Joel Jaeggli | [Ballot Position Update] New position, Yes, has been recorded for Joel Jaeggli |
2014-10-19
|
11 | Joel Jaeggli | Created "Approve" ballot |
2014-10-19
|
11 | Joel Jaeggli | Ballot writeup was changed |
2014-10-19
|
11 | Joel Jaeggli | Placed on agenda for telechat - 2014-10-30 |
2014-10-19
|
11 | Joel Jaeggli | Changed consensus to Yes from Unknown |
2014-10-09
|
11 | (System) | IESG state changed to Waiting for Writeup from In Last Call |
2014-10-02
|
11 | Gunter Van de Velde | Request for Last Call review by OPSDIR is assigned to Victor Fajardo |
2014-10-02
|
11 | Gunter Van de Velde | Request for Last Call review by OPSDIR is assigned to Victor Fajardo |
2014-10-02
|
11 | Gunter Van de Velde | Closed request for Last Call review by OPSDIR with state 'Withdrawn' |
2014-09-29
|
11 | Gunter Van de Velde | Request for Last Call review by OPSDIR is assigned to Bernard Aboba |
2014-09-29
|
11 | Gunter Van de Velde | Request for Last Call review by OPSDIR is assigned to Bernard Aboba |
2014-09-26
|
11 | (System) | IANA Review state changed to IANA OK - No Actions Needed from IANA - Review Needed |
2014-09-26
|
11 | Amanda Baber | IESG/Authors/WG Chairs: IANA has reviewed draft-ietf-bmwg-sip-bench-meth-11, which is currently in Last Call, and has the following comments: We understand that this document doesn't require … IESG/Authors/WG Chairs: IANA has reviewed draft-ietf-bmwg-sip-bench-meth-11, which is currently in Last Call, and has the following comments: We understand that this document doesn't require any IANA actions. While it is helpful for the IANA Considerations section of the document to remain in place upon publication, if the authors prefer to remove it, IANA doesn't object. If this assessment is not accurate, please respond as soon as possible. |
2014-09-25
|
11 | Jean Mahoney | Request for Last Call review by GENART is assigned to Tom Taylor |
2014-09-25
|
11 | Jean Mahoney | Request for Last Call review by GENART is assigned to Tom Taylor |
2014-09-25
|
11 | Amy Vezza | IANA Review state changed to IANA - Review Needed |
2014-09-25
|
11 | Amy Vezza | The following Last Call announcement was sent out: From: The IESG To: IETF-Announce CC: Reply-To: ietf@ietf.org Sender: Subject: Last Call: (Methodology for Benchmarking Session Initiation … The following Last Call announcement was sent out: From: The IESG To: IETF-Announce CC: Reply-To: ietf@ietf.org Sender: Subject: Last Call: (Methodology for Benchmarking Session Initiation Protocol (SIP) Devices: Basic session setup and registration) to Informational RFC The IESG has received a request from the Benchmarking Methodology WG (bmwg) to consider the following document: - 'Methodology for Benchmarking Session Initiation Protocol (SIP) Devices: Basic session setup and registration' as Informational RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2014-10-09. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract This document provides a methodology for benchmarking the Session Initiation Protocol (SIP) performance of devices. Terminology related to benchmarking SIP devices is described in the companion terminology document. Using these two documents, benchmarks can be obtained and compared for different types of devices such as SIP Proxy Servers, Registrars and Session Border Controllers. The term "performance" in this context means the capacity of the device-under- test (DUT) to process SIP messages. Media streams are used only to study how they impact the signaling behavior. The intent of the two documents is to provide a normalized set of tests that will enable an objective comparison of the capacity of SIP devices. Test setup parameters and a methodology are necessary because SIP allows a wide range of configuration and operational conditions that can influence performance benchmark measurements. The file can be obtained via http://datatracker.ietf.org/doc/draft-ietf-bmwg-sip-bench-meth/ IESG discussion can be tracked via http://datatracker.ietf.org/doc/draft-ietf-bmwg-sip-bench-meth/ballot/ No IPR declarations have been submitted directly on this I-D. |
2014-09-25
|
11 | Amy Vezza | IESG state changed to In Last Call from Last Call Requested |
2014-09-25
|
11 | Joel Jaeggli | Last call was requested |
2014-09-25
|
11 | Joel Jaeggli | IESG state changed to Last Call Requested from AD Evaluation |
2014-09-25
|
11 | Joel Jaeggli | Last call announcement was generated |
2014-09-24
|
11 | Joel Jaeggli | IESG state changed to AD Evaluation from Dead |
2014-07-19
|
11 | Al Morton | Tag Doc Shepherd Follow-up Underway cleared. |
2014-07-19
|
11 | Al Morton | IETF WG state changed to Submitted to IESG for Publication from WG Consensus: Waiting for Write-Up |
2014-07-19
|
11 | Al Morton | This is a publication request for: draft-ietf-bmwg-sip-bench-meth -11 2014-07-02 Active draft-ietf-bmwg-sip-bench-term -11 2014-07-02 Active Al Morton is the Document Shepherd, and prepared this form. (1) … This is a publication request for: draft-ietf-bmwg-sip-bench-meth -11 2014-07-02 Active draft-ietf-bmwg-sip-bench-term -11 2014-07-02 Active Al Morton is the Document Shepherd, and prepared this form. (1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header? Informational, as indicated on the title page. All BMWG RFCs are traditionally Informational, in part because they do not define protocols and the traditional conditions for Stds track advancement did not apply. However, they are specifications and the RFC 2119 terms are applicable to identify the level of requirements. (2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections: Technical Summary: All networking devices have a limited capacity to serve their purpose. In some cases these limits can be ascertained by counting physical features (e.g., interface card slots), but in other cases standardized tests are required to be sure that all vendors count their protocol-handling capacity in the same way, to avoid specmanship. This draft addresses one such case, where the SIP session-serving capacity of a device can only be discovered and rigorously compared with other devices through isolated laboratory testing. This document describes the methodology for benchmarking Session -or- This document describes the terminology for benchmarking Session Initiation Protocol (SIP) performance as described in SIP benchmarking terminology document. The methodology and terminology are to be used for benchmarking signaling plane performance with varying signaling and media load. Both scale and establishment rate are measured by signaling plane performance. The SIP Devices to be benchmarked may be a single device under test or a system under test. Benchmarks can be obtained and compared for different types of devices such as SIP Proxy Server, Session Border Controller, and server paired with a media relay or Firewall/NAT device. Working Group Summary: There were periods of intense and constructive feedback on this draft, but also several pauses in progress during development. The most lively discussions were prompted by presentation of actual test results using the draft methods, which require significant time investment but are well- worth the result. These drafts serve a useful purpose for the industry. Document Quality: There are existing implementations of the method, as noted above. Dale Worley conducted an early review, following BMWG's request of the RAI area. Dales's comments were addressed in version 05. Henning Schulzrinne commented on the original work proposal. Personnel: Who is the Document Shepherd? Who is the Responsible Area Director? Al Morton is Shepherd, Joel Jaeggli is Responsible AD. (3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG. The shepherd has reviewed the drafts many times, and his comments are in the BMWG archive. (4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed? No. (5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place. No. Cross-area review has been obtained, however it impossible to get the attention of everyone who considers themselves a SIP expert. (6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here. No concerns, this is still a valuable memo, as mentioned above. (7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why? There are not outstanding IPR disclosures, according to the authors. (8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures. No. (9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it? Although the comments and review intensity was highly variable, it now appears that the WG is satisfied. The first WGLC was completed on 5 April 2010 with comments. The second WGLC was completed on 18 May 2012 with comments. The third WGLC was completed on 10 Dec 2012 with comments, and the 1st Pub Request. A IETF Last Call followed, and completed on 30 Jan 2013 with comments. A fourth WGLC was completed 11 June 2014 with comments from expert review. The current versions (11) address Dale Worley's RAI area early review and Robert Spark's reviews. The fifth WGLC completed quietly on July 14th, 2014. (10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.) No. (11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough. Nits are warnings requiring no action for these drafts. (12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews. N/A (13) Have all references within this document been identified as either normative or informative? Yes. (14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion? The -term and -meth drafts are proceeding toward publication as a pair. (15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure. No. (16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary. No. (17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226). No requests of IANA. (18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries. N/A (19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc. N/A |
2014-07-19
|
11 | Al Morton | Tag Doc Shepherd Follow-up Underway set. |
2014-07-19
|
11 | Al Morton | IETF WG state changed to WG Consensus: Waiting for Write-Up from In WG Last Call |
2014-07-02
|
11 | Vijay Gurbani | New version available: draft-ietf-bmwg-sip-bench-meth-11.txt |
2014-05-29
|
10 | Al Morton | Tag Revised I-D Needed - Issue raised by WG cleared. |
2014-05-29
|
10 | Al Morton | IETF WG state changed to In WG Last Call from WG Document |
2014-05-28
|
10 | Vijay Gurbani | New version available: draft-ietf-bmwg-sip-bench-meth-10.txt |
2014-04-25
|
09 | Al Morton | Document shepherd changed to Al C. Morton |
2014-04-25
|
09 | Al Morton | Tag Revised I-D Needed - Issue raised by WG set. Tag Doc Shepherd Follow-up Underway cleared. |
2014-04-25
|
09 | Al Morton | IETF WG state changed to WG Document from Submitted to IESG for Publication |
2014-02-14
|
09 | Vijay Gurbani | New version available: draft-ietf-bmwg-sip-bench-meth-09.txt |
2013-08-02
|
08 | Tero Kivinen | Request for Last Call review by SECDIR Completed: Has Nits. Reviewer: Donald Eastlake. |
2013-07-31
|
08 | (System) | Document has expired |
2013-07-31
|
08 | (System) | State changed to Dead from AD is watching |
2013-07-30
|
08 | Joel Jaeggli | State changed to AD is watching from Waiting for AD Go-Ahead::Revised I-D Needed |
2013-06-13
|
08 | Tero Kivinen | Closed request for Last Call review by SECDIR with state 'No Response' |
2013-06-13
|
08 | Tero Kivinen | Request for Last Call review by SECDIR is assigned to Donald Eastlake |
2013-06-13
|
08 | Tero Kivinen | Request for Last Call review by SECDIR is assigned to Donald Eastlake |
2013-05-14
|
08 | Al Morton | Annotation tag Doc Shepherd Follow-up Underway set. |
2013-03-13
|
08 | Cindy Morgan | Shepherding AD changed to Joel Jaeggli |
2013-02-04
|
08 | Miguel García | Request for Last Call review by GENART Completed: Almost Ready. Reviewer: Miguel Garcia. |
2013-01-30
|
08 | Ron Bonica | State changed to Waiting for AD Go-Ahead::Revised ID Needed from Waiting for AD Go-Ahead |
2013-01-30
|
08 | (System) | State changed to Waiting for AD Go-Ahead from In Last Call |
2013-01-24
|
08 | Pearl Liang | IANA has reviewed draft-ietf-bmwg-sip-bench-meth-08, which is currently in Last Call, and has the following comments: IANA understands that, upon approval of this document, there … IANA has reviewed draft-ietf-bmwg-sip-bench-meth-08, which is currently in Last Call, and has the following comments: IANA understands that, upon approval of this document, there are no IANA Actions that need completion. |
2013-01-17
|
08 | Jean Mahoney | Request for Last Call review by GENART is assigned to Miguel Garcia |
2013-01-17
|
08 | Jean Mahoney | Request for Last Call review by GENART is assigned to Miguel Garcia |
2013-01-17
|
08 | Tero Kivinen | Request for Last Call review by SECDIR is assigned to Lt. Mundy |
2013-01-17
|
08 | Tero Kivinen | Request for Last Call review by SECDIR is assigned to Lt. Mundy |
2013-01-16
|
08 | Amy Vezza | The following Last Call announcement was sent out: From: The IESG To: IETF-Announce CC: Reply-To: ietf@ietf.org Subject: Last Call: (Methodology for Benchmarking SIP Networking Devices) … The following Last Call announcement was sent out: From: The IESG To: IETF-Announce CC: Reply-To: ietf@ietf.org Subject: Last Call: (Methodology for Benchmarking SIP Networking Devices) to Informational RFC The IESG has received a request from the Benchmarking Methodology WG (bmwg) to consider the following document: - 'Methodology for Benchmarking SIP Networking Devices' as Informational RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2013-01-30. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract This document describes the methodology for benchmarking Session Initiation Protocol (SIP) performance as described in SIP benchmarking terminology document. The methodology and terminology are to be used for benchmarking signaling plane performance with varying signaling and media load. Both scale and establishment rate are measured by signaling plane performance. The SIP Devices to be benchmarked may be a single device under test (DUT) or a system under test (SUT). Benchmarks can be obtained and compared for different types of devices such as SIP Proxy Server, SBC, and server paired with a media relay or Firewall/NAT device. The file can be obtained via http://datatracker.ietf.org/doc/draft-ietf-bmwg-sip-bench-meth/ IESG discussion can be tracked via http://datatracker.ietf.org/doc/draft-ietf-bmwg-sip-bench-meth/ballot/ No IPR declarations have been submitted directly on this I-D. |
2013-01-16
|
08 | Amy Vezza | State changed to In Last Call from Last Call Requested |
2013-01-16
|
08 | Ron Bonica | Last call was requested |
2013-01-16
|
08 | Ron Bonica | Last call announcement was generated |
2013-01-16
|
08 | Ron Bonica | Ballot approval text was generated |
2013-01-16
|
08 | Ron Bonica | Ballot writeup was generated |
2013-01-16
|
08 | Ron Bonica | State changed to Last Call Requested from AD Evaluation |
2013-01-16
|
08 | Al Morton | IETF state changed to Submitted to IESG for Publication from In WG Last Call |
2013-01-16
|
08 | Al Morton | Annotation tags Awaiting Expert Review/Resolution of Issues Raised, Other - see Comment Log cleared. |
2013-01-16
|
08 | Ron Bonica | State changed to AD Evaluation from Publication Requested |
2013-01-08
|
08 | Vijay Gurbani | New version available: draft-ietf-bmwg-sip-bench-meth-08.txt |
2013-01-07
|
07 | Cindy Morgan | (1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? … (1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header? Informational, as indicated on the title page. All BMWG RFCs are traditionally Informational, in part because they do not define protocols and the traditional conditions for Stds track advancement did not apply. However, they are specifications and the RFC 2119 terms are applicable to identify the level of requirements. (2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections: Technical Summary: All networking devices have a limited capacity to serve their purpose. In some cases these limits can be ascertained by counting physical features (e.g., interface card slots), but in other cases standard ized tests are required to be sure that all vendors count their protocol-handling capacity in the same way, to avoid specmanship. This draft addresses one such case, where the SIP session-serving capacity of a device can only be discovered and rigorously compared with other devices through isolated laboratory testing. This document describes the methodology for benchmarking Session -or- This document describes the terminology for benchmarking Session Initiation Protocol (SIP) performance as described in SIP benchmarking terminology document. The methodology and terminology are to be used for benchmarking signaling plane performance with varying signaling and media load. Both scale and establishment rate are measured by signaling plane performance. The SIP Devices to be benchmarked may be a single device under test or a system under test. Benchmarks can be obtained and compared for different types of devices such as SIP Proxy Server, Session Border Controller, and server paired with a media relay or Firewall/NAT device. Working Group Summary: There were periods of intense and constructive feedback on this draft, but also several pauses in progress during development. The most lively discussions were prompted by presentation of actual test results using the draft methods, which require significant time investment but are well- worth the result. These drafts serve a useful purpose for the industry. Document Quality: There are existing implementations of the method, as noted above. Dale Worley conducted an early review, following BMWG's request of the RAI area. Dales's comments were addressed in version 05. Henning Schulzrinne commented on the original work proposal. Personnel: Who is the Document Shepherd? Who is the Responsible Area Director? Al Morton is Shepherd, Ron Bonica is Responsible AD. (3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG. The shepherd has reviewed the drafts many times, and his comments are in the BMWG archive. The last review resulted in post-WGLC revisions. (4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed? No. (5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place. No. Cross-area review has been obtained, however it impossible to get the attention of everyone who considers themselves a SIP expert. (6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here. No concerns, this is still a valuable memo, as mentioned above. (7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why? There are no outstanding IPR disclosures, according to the authors. (8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures. No. (9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it? Although the comments and review intensity was highly variable, it now appears that the WG is satisfied. The 3rd WGLC went quietly. The first WGLC was completed on 5 April 2010 with comments. The second WGLC was completed on 18 May 2012 with comments. (10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.) No. (11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough. Nits (version 2.12.13) are warnings requiring no action for this draft. (12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews. N/A (13) Have all references within this document been identified as either normative or informative? Yes. (14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion? The -term and -meth drafts are proceeding to publication as a pair. (15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure. No. (16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary. No. (17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226). No requests of IANA. (18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries. N/A (19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc. N/A This is a publication request for: draft-ietf-bmwg-sip-bench-meth -07 2012-01-06 Active draft-ietf-bmwg-sip-bench-term -07 2012-01-06 Active using the shepherding form dated 24 February 2012, now available from http://www.ietf.org/iesg/template/doc-writeup.html Al Morton is the Document Shepherd, and prepared this form. (1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header? Informational, as indicated on the title page. All BMWG RFCs are traditionally Informational, in part because they do not define protocols and the traditional conditions for Stds track advancement did not apply. However, they are specifications and the RFC 2119 terms are applicable to identify the level of requirements. (2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections: Technical Summary: All networking devices have a limited capacity to serve their purpose. In some cases these limits can be ascertained by counting physical features (e.g., interface card slots), but in other cases standard ized tests are required to be sure that all vendors count their protocol-handling capacity in the same way, to avoid specmanship. This draft addresses one such case, where the SIP session-serving capacity of a device can only be discovered and rigorously compared with other devices through isolated laboratory testing. This document describes the methodology for benchmarking Session -or- This document describes the terminology for benchmarking Session Initiation Protocol (SIP) performance as described in SIP benchmarking terminology document. The methodology and terminology are to be used for benchmarking signaling plane performance with varying signaling and media load. Both scale and establishment rate are measured by signaling plane performance. The SIP Devices to be benchmarked may be a single device under test or a system under test. Benchmarks can be obtained and compared for different types of devices such as SIP Proxy Server, Session Border Controller, and server paired with a media relay or Firewall/NAT device. Working Group Summary: There were periods of intense and constructive feedback on this draft, but also several pauses in progress during development. The most lively discussions were prompted by presentation of actual test results using the draft methods, which require significant time investment but are well- worth the result. These drafts serve a useful purpose for the industry. Document Quality: There are existing implementations of the method, as noted above. Dale Worley conducted an early review, following BMWG's request of the RAI area. Dales's comments were addressed in version 05. Henning Schulzrinne commented on the original work proposal. Personnel: Who is the Document Shepherd? Who is the Responsible Area Director? Al Morton is Shepherd, Ron Bonica is Responsible AD. (3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG. The shepherd has reviewed the drafts many times, and his comments are in the BMWG archive. The last review resulted in post-WGLC revisions. (4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed? No. (5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place. No. Cross-area review has been obtained, however it impossible to get the attention of everyone who considers themselves a SIP expert. (6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here. No concerns, this is still a valuable memo, as mentioned above. (7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why? There are no outstanding IPR disclosures, according to the authors. (8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures. No. (9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it? Although the comments and review intensity was highly variable, it now appears that the WG is satisfied. The 3rd WGLC went quietly. The first WGLC was completed on 5 April 2010 with comments. The second WGLC was completed on 18 May 2012 with comments. (10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.) No. (11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough. Nits (version 2.12.13) are warnings requiring no action for this draft. (12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews. N/A (13) Have all references within this document been identified as either normative or informative? Yes. (14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion? The -term and -meth drafts are proceeding to publication as a pair. (15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure. No. (16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary. No. (17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226). No requests of IANA. (18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries. N/A (19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc. N/A |
2013-01-07
|
07 | Cindy Morgan | Note added 'Al Morton (acmorton@att.com) is the document shepherd.' |
2013-01-07
|
07 | Cindy Morgan | Intended Status changed to Informational |
2013-01-07
|
07 | Cindy Morgan | IESG process started in state Publication Requested |
2013-01-07
|
07 | (System) | Earlier history may be found in the Comment Log for draft-poretsky-bmwg-sip-bench-meth |
2013-01-06
|
07 | Al Morton | New version available: draft-ietf-bmwg-sip-bench-meth-07.txt |
2012-11-08
|
06 | Vijay Gurbani | New version available: draft-ietf-bmwg-sip-bench-meth-06.txt |
2012-10-22
|
05 | Carol Davids | New version available: draft-ietf-bmwg-sip-bench-meth-05.txt |
2012-05-03
|
04 | Al Morton | IETF state changed to In WG Last Call from WG Document |
2012-05-03
|
04 | Al Morton | Annotation tag Awaiting Expert Review/Resolution of Issues Raised set. |
2012-03-12
|
04 | Al Morton | ends May 18 |
2012-03-12
|
04 | Vijay Gurbani | New version available: draft-ietf-bmwg-sip-bench-meth-04.txt |
2011-09-15
|
03 | (System) | Document has expired |
2011-05-30
|
03 | Al Morton | Revised I-D Needed - Issues raised at IETF-80 |
2011-05-30
|
03 | Al Morton | IETF state changed to WG Document from Adopted by a WG |
2011-05-30
|
03 | Al Morton | IETF state changed to Adopted by a WG from WG Document |
2011-05-30
|
03 | Al Morton | Revised I-D Needed - Issues raised at IETF-80 |
2011-05-30
|
03 | Al Morton | Annotation tag Other - see Comment Log set. |
2011-03-14
|
03 | (System) | New version available: draft-ietf-bmwg-sip-bench-meth-03.txt |
2010-07-12
|
02 | (System) | New version available: draft-ietf-bmwg-sip-bench-meth-02.txt |
2010-02-08
|
01 | (System) | New version available: draft-ietf-bmwg-sip-bench-meth-01.txt |
2009-03-04
|
00 | (System) | New version available: draft-ietf-bmwg-sip-bench-meth-00.txt |