IP Performance Metrics WG (ippm) Tuesday, November 7, 2006 - 09:00--11:30 ======================================== The meeting was chaired by Henk Uijterwaal and Matt Zekauskas. Al Morton took notes, which were edited into these minutes by the chairs. AGENDA: 1. Administrivia 2. Status of milestones and drafts 3. Reporting metrics a) draft-ietf-ippm-reporting-01.txt b) Different points of view 3.5 On the IPPM Registry 4. TWAMP 5. Composition drafts 6. Traceroutes draft 7. Delay Variation AS draft 8. Metric for media application level performance 9. AOB 1. Administrivia 2. Status of milestones and drafts (Chairs) Henk opened the meeting with a review of draft status and the current sent of milestones. OWAMP has finally become RFC 4656. The reordering draft is in the RFC Editor's queue. We need readers on the capacity draft; we think there is one more revision needed before last call. The chairs intend to update the milestones immediately after the meeting, and need input. Al Morton noted the jitter applicability statement milestone was too aggressive from the start; assuming the draft he presents today is accepted as a working group document, he thought closure might occur around January 2008. 3. Reporting Metrics a) draft-ietf-ippm-reporting-01.txt --Stanislav Shalunov Stanislav Shalunov gave an update on the Reporting Metrics draft. The intent is to provide a simple set of items to report to humans for tests done on-demand. There was some discussion of the "duplication" metric, since the WG does not have it's own definition yet. Al Morton though that the WG needed a definition, but this draft should not be the first place to do it. Al also noted that there was a draft definition in the latest revision of ITU-T Y.1540, and he had a copy that he could share; IPPM should make sure any new definition is consistent with Y.1540. Emile Stefan asked about the sample source slide. Why not provide default values for the sample source? You seem to provide 3. Stanislav replied that they provide two, and you would use one or the other depending on the use case. Ping does RTT, owping does one-way measurements. owping has to print something at the end, and that was the spur for this work. Emile thought this wasn't consistent with the rest of the document -- that we should provide one way to provide clear results. The result is that we would prefer one-way active measurements, but one could use round-trip measurements if clock synchronization was a problem. 3b) draft-morton-ippm-reporting-metrics-01.txt --Al Morton Al Morton then presented his "different points of view" document -- that you need to consider how the measurements will be used, and thus what should be done if you want to lift the "on-the-fly" processing restriction needed in Stanislav's document to support nearly real-time reporting. A main point of contention is the use of delay distributions conditioned on arrival, making it possible to report the mean. In Al's view this conditional distribution maintains orthogonality between loss and delay, but in return requires that you must report both loss and delay. He also suggests that measurement reports could include both the median and the mean. Al would like the group to take his draft and use it as a basis for long-term reporting. People need to read this; Al will take more discussion to the list. 3.5 Reporting Metrics -- Emile Stefan Emile gave a quick report on reporting metrics. He thinks we need to merge with the ipfix information model, and will need working group input on it. Juergen Quittek had a short comment. He said that we could rely on the ipfix/psamp model, but we should have a list of information elements as an RFC, and then hand off maintaining the list to IANA. Then it's easy to add missing elements. 4. TWAMP --Kaynam Hedayat There are two new vendors that have implemented the draft as it stands. There is definitely interest to get the draft finalized. Al Morton reviewed and commented as requested by the WG chairs. Matt Z. will send email about organization and the security section -- the desire is to align with the version of OWAMP that has finally passed through IESG review, and refer to it, so that a protracted review for TWAMP will not be necessary. Please read the next version when it comes out, because it will likely be last called soon after. 5. Composition drafts --Al Morton Al reviewed the framework for composition draft, and the spatial composition draft. He has found a potential co-author for the temporal aggregation draft. Al asked who in the audience had read the drafts -- approximately ten folks raised their hands. Al thought the drafts were ready for a lot more readers; he would like to finish up the framework draft this year or at least stabilize it until the other draft are fleshed out). All the issues raised so far on the Spatial Composition draft have been resolved (they are documented in section 10). The next steps are to add new metrics, primarily for composition of delays and delay variation. There is only one reference to a delay variation function; one could take histograms and perform an approximate convolution. Al encouraged additional contributions in this area. There is also some redundancy left in the spatial draft that Al is going to excise. Al wanted more readers, or perhaps a pseudo-last call on the framework. It's not finished, but it's stable pending the new additions elsewhere. Two folks in the audience volunteered to read and put reviews on the mailing list. Al then put up a slide on the multimetrics draft. He showed how they had come up with a way to preserve the original IPPM terminology, but have introduced two new terms: vector and matrix. We measure singletons in a sample, and then produce statistics, as with the other IPPM metrics. A vector collects the singletons across space for a single path, or across multiple parties (multi-point). A matrix is formed by a set of vectors, essentially the sample at each point of interest. More readership would be useful here as well. There are more multicast and multiparty metrics that are in the works; it's not ready for last call but needs more review and input. 6. Traceroutes draft --Juergen Quittek Juergen reported that they believe this draft is almost complete. There have been very few changes since the last report, mostly editorial. One substantive change has been the resolution of an MPLS label stack issue; although they intended to report the entire stack, they found it was not possible in general, so they are reporting just the top label. Since the last meeting, the chairs talked with the chairs of the GGF (now OGF, Open Grid Forum) Network Measurement Working Group (NMWG), and Martin Swany (present today) has been identified to act as a liaison between the two bodies. Martin reported that the current state is that there are drafts in preparation for generic XMPL reporting framework. The current IPPM draft is essentially conformant and can be thought of as an instance of the larger draft. The remaining problem is naming; here we try to align with the DISMAN traceroute MIB; there the names are shorter and in the context of a particular name space. A couple of solutions were discussed. The first is to define this as a "mib profile compliant" version of the NMWG XML framework, where there is a defined mapping, even if the names are not the same. Another is to propose that the NMWG adopt an IANA name space. A third is to have this draft adopt an NMWG name space (or define the namespace according to NMWG guidelines). The prefix is the main difference, so the mapping is "trivial". Venkant asked about why it was difficult to extract the whole label stack in MPLS? It depends on the MPLS control information, and sometimes you don't have that information. You can always pop the top information. Matt Mathis was concerned that since the NMWG framework is still a draft, would it change out from underneath us. Martin said a semantic specification within the document might help, but understands the issue. Lars asked if the GGF has a formal consensus process, and if there is a rough timeline for the process there; we would like to move forward. Lars proposed making changes to element and namespace names, and then freezing the version. Martin noted that the important thing was to align the prefixes; if individual elements changed names there can always be a direct translation. Martin is to get together with the current authors, work out a solution, and present it at an upcoming NMWG meeting to make sure we are all in agreement. 7. Delay Variation AS draft --Al Morton Al said that he had come up with the current draft as a starting point from the discussion from the last two IETFs. It is an analysis of the situation with delay variation. Al invites comments, and is looking for more use cases. With some work, he thinks this draft can satisfy the outstanding working group milestone. Emile asked if Al thought the number of uses cases was finite. If you want to put something like this in the metric registry, you need to be able to enumerate them. 8. Metric for media application level performance --Kaynam Hedayat and Nagarjuna Venna Nagarjuna Venna then reported on Application of Loss Metrics to Multimedia Streams. This work involves using existing IPPM metrics as a base, along with some basic stream type information to provide metrics that might be useful to operators -- giving them the ability to raise alarms when users might be unhappy. One group member noted that they were part of a research project trying to go from path related metrics to media quality. They found it really depended on the codec used. There may be some derived metric that gives more value than simple ones, but it's not clear you can generically map to media quality. Lars Eggert asked if there was data on correlation from loss to quality. He agrees it correlates, but it would be nice to see how strongly it correlated. Kaynam Hedayat noted that some of the metrics might be implementation dependent, not just codec dependent. Eric Voit noted that with voice strings, you can come up with indications based on the derived information here, but the quality of experience is not well articulated. They felt that it would be worthwhile to have a mapping from metrics to quality of experience for a couple of different configurations. Yet another person felt that for things like noticeable loss events, noticeable loss rates, you could make inferences based on codecs and types of frame, and that would be useful. The presenters were directed to write a draft so that the group could evaluate if it wanted to take on the work, and if the work was in charter. If the work depends on a deep understanding of the particular application, then it would place the work outside the IPPM charter; if one can truly create inferences based on IPPM metrics, then it could perhaps be an applicability statement. 9. AOB Finally, Henk closed by noting there was another application-related personal draft presented to SIPPING that the group might want to take a look at if they are interested (and it appears that the group will be asked to do this formally at some point in the near future). This is currently draft-malas-performance-metrics-05.txt.