Early Review of draft-ietf-cats-metric-definition-10
review-ietf-cats-metric-definition-10-opsdir-early-lim-2026-07-02-00

Hi,

I have been selected as the Operational Directorate (opsdir) reviewer for this
Internet-Draft.

The Operational Directorate reviews all operational and management-related
Internet-Drafts to ensure alignment with operational best practices and that
adequate operational considerations are covered.

A complete set of _"Guidelines for Considering Operations and Management in
IETF Specifications"_ can be found at
https://datatracker.ietf.org/doc/draft-ietf-opsawg-rfc5706bis/.

While these comments are primarily for the Operations and Management Area
Directors (Ops ADs), the authors should consider them alongside other feedback
received.

- Document: draft-ietf-cats-metric-definition-10
- Reviewer: Fung Lim
- Review Date: 3rd July 2026
- Intended Status: Standards Track

---

## Summary

- Has Issues: I have concerns about this document that I think should be
resolved before publication.

This document defines a three-level hierarchical abstraction for
Computing-Aware Traffic Steering (CATS) metrics, standardizing Level 1 category
metrics (computing, communication, service, composed) and a Level 2 global
normalized score, along with a metric field framework. The design is
well-motivated: it deliberately avoids standardizing fine-grained Level 0
metrics while providing interoperable encodings and normative score semantics
at higher levels. The metric framework described in this document is sound and
several operational concerns are implicitly addressed.

However, as a Standards Track document that defines inputs to traffic-steering
decisions, it requires a dedicated Operational Considerations section (per
Section 3.1 of draft-ietf-opsawg-rfc5706bis). Several operational topics that
are partially scattered across design principles, registry entries, and
security text would benefit from consolidation into operator-facing guidance.
In particular, cross-vendor score comparability, metric collection failure
handling, control-plane churn from metric updates, and management interface
expectations should be addressed.

1. Missing Operational Considerations section (§3.1 of rfc5706bis)

Per draft-ietf-opsawg-rfc5706bis, a technical specification advancing as an
IETF RFC MUST include an Operational Considerations section immediately before
Security Considerations, or explicit boilerplate with rationale if none apply.
This document jumps from Section 6 (CATS Metric Registry Entries) directly to
Section 7 (Security Considerations).

For a Standards Track document whose metrics directly influence instance
selection and traffic steering, an Operational Considerations section is
essential rather than optional. The document already contains operationally
relevant material — score semantics (Section 4.3), level selection trade-offs
(Section 5), role assignments (C-SMA, C-NMA, C-PS in Section 6), calibration
guidance, and security requirements — but it is not consolidated where
operators will look for it.

2. Cross-vendor score comparability needs explicit operational guidance (§4.1,
§4.5 of rfc5706bis)

Section 4.3 warns that normalized scores "may not be directly comparable across
different implementations" because aggregation and normalization functions are
implementation-specific. Section 4.2.1 and 4.2.2 state these functions "are not
standardized" and are "controlled by operator policies."

This is a fundamental operational constraint that affects multi-vendor and
multi-provider CATS deployments. An operator comparing a score of 7 from
Provider A against a score of 7 from Provider B has no assurance of equivalent
capability. The normative 0–3 / 4–7 / 8–10 steering bands (Section 4.3) help
within a single administrative domain but do not solve cross-domain comparison.

The Operational Considerations section should address:

a. When scores are safe to use for steering (intra-domain only vs. inter-domain)
b. Whether operators should prefer Level 1 category metrics over Level 2 global
scores for multi-vendor environments c. How calibration (referenced in Section
6 registry entries) should be applied operationally — who runs benchmarks, how
often, and what deviation thresholds trigger re-calibration d. Whether future
CATS signaling documents will carry normalization policy identifiers to aid
comparison

3. No fault management or troubleshooting guidance (§5.4 of rfc5706bis)

CATS metrics drive steering decisions. The document does not discuss how
operators detect or respond to metric-related faults:

Stale or missing metrics from C-SMA or C-NMA (Section 7 addresses freshness at
the security level via SEC-4, but not operational fallback behavior) Collection
agent failures or telemetry pipeline outages Score anomalies (sudden drops,
stuck values, contradictory Level 1 vs. Level 2 scores) Integrity violation
handling (SEC-1 says metrics SHOULD trigger alerts, but no alert schema or
operational procedure) Per §5.4 of rfc5706bis, essential faults, alarms, and
events should be identified. At minimum, the document should discuss:

a. Fallback steering behavior when metrics are unavailable or fail freshness
checks b. Whether last-known-good values are used or instances are
de-prioritized c. Recommended health indicators for C-SMA, C-NMA, and C-PS
components

4. No guidance on verifying correct operation (§4.7 of rfc5706bis)

The registry entries include calibration methods (e.g., "score deviation ...
less than 0.1 over repeated test rounds") but the document does not describe
how operators verify end-to-end CATS metric operation in production:

Active vs. passive validation approaches
Synthetic workload injection for score validation
Correlation between normalized scores and observed application QoE
Simple health checks (e.g., confirming C-SMA is publishing expected metric
types at expected intervals) Section 6 registry entries reference benchmark
calibration but treat it as an implementation activity without operator-facing
procedures.

5. Metric update frequency and control-plane impact not addressed (§4.5 of
rfc5706bis)

Section 3.1 discusses the trade-off between metric update frequency and
"signaling overhead, reducing scalability of the metric distribution protocol."
Section 6 suggests continuous sampling every 10 seconds with a default
Measurement_Window of 10 seconds.

However, the document does not quantify or guide:

Recommended maximum metric advertisement rate per service instance
Impact of high churn on CATS control-plane protocols (to be defined in
companion documents) When coarser sampling or Level 2-only advertisement is
preferable to reduce overhead Interaction between Statistics field choices
(max, min, mean, cur) and update frequency This is particularly important
because CATS may scale to hundreds or thousands of service instances (Section
5).

Nits

6. No management interoperability discussion (§5.2 of rfc5706bis)

The document references Prometheus, NETCONF, and IPFIX for Level 0 metric
collection but does not discuss:

Whether a YANG data model for CATS metric configuration, policy, and exposed
state is needed or planned How normalized scores and runtime parameters
(Measurement_Window, aggregation weights) would be configured via standard
management interfaces Distinction between configuration data (normalization
bounds, weights), operational state (current scores), and statistics
(historical trends) For a Standards Track metric definition consumed by network
elements, some discussion of management model expectations — even if deferred
to a companion document — would help operators plan deployments.

7. Migration and coexistence with existing metrics (§4.3 of rfc5706bis)

The Introduction references existing TE metrics (RFC 7471) and network
performance registries (RFC 8911, RFC 8912). The document does not discuss:

How CATS Level 1 communication metrics relate to or subsume existing IGP TE
metrics Whether operators can incrementally adopt CATS metrics alongside
traditional TE without dual-metric steering conflicts Mapping or equivalence
between ALTO performance cost metrics (RFC 9439) and CATS normalized scores A
brief migration narrative would help operators who already run TE metric
extensions.

8. Role assignment ambiguity in registry entries

Section 6.4.3.6 assigns metric calculation to "Service contact instance" rather
than C-SMA (used consistently elsewhere). Section 6.1.3.6 contains a formatting
error where ### Output appears inline after the C-PS role description,
disrupting the registry structure.

Section 6.4.3.6:

Service contact instace: Collects Level 0 service raw metrics...

This should likely reference C-SMA for consistency with the framework
terminology.

9. Editorial and reference nits

"Section Section 4.4" (Section 4.1, line 605) — duplicate word
"To-be-assgined" → "To-be-assigned" (Sections 6.1.4.2, 6.2.5.2, 6.3.5.2)
"Service contact instace" → "Service contact instance" (Section 6.4.3.6)
Normative self-reference [I-D.ietf-cats-metric-definition] points to draft-08
(Section 9.1) — should reference the current draft version Appendix A examples
use inconsistent Source capitalization ("Directly measured" vs. lowercase
'directly measured' in Section 4.1) IANA placeholder "XXX" identifiers
throughout Section 6 — expected for draft stage, but Operational Considerations
should note operators need stable identifiers before deployment

10. Accounting and performance management partially addressed

Section 6 registry entries define singleton output values and calibration
targets, which is a good start. However:

No counter rollover behavior for Measurement_Window accumulators
No guidance on historical metric retention for capacity planning
No discussion of metric export for billing or SLA reporting (may be N/A for
some deployments, but worth a sentence)

Recommendations

1. Add an Operational Considerations section immediately before Security
Considerations. Consolidate scattered operational material and address: metric
level selection, runtime parameter defaults, cross-vendor score limitations,
metric collection failures and fallback steering, update rate guidance, and
coexistence with existing TE metrics.

2. Document fault management and fallback behavior. Specify what happens when
C-SMA/C-NMA stops publishing, metrics fail freshness checks (SEC-4), or scores
are contradictory. Define minimum health indicators and recommended alert
conditions.

3. Add verification guidance. Describe how operators validate that normalized
scores reflect actual instance capability — referencing the calibration methods
already in Section 6 registry entries and extending them into deployable
procedures (synthetic workloads, periodic re-calibration, QoE correlation).

4. Clarify cross-domain and multi-vendor deployment constraints. Expand Section
4.3's comparability warning into explicit operational guidance: when scores are
trustworthy, when Level 1 categories are preferable to Level 2 global scores,
and how calibration maintains consistency within an administrative domain.

5. Discuss management model expectations. Even if a YANG module is deferred,
state whether configuration of normalization bounds, aggregation weights, and
Measurement_Window is expected via NETCONF/RESTCONF or operator-local policy,
and how config vs. operational state vs. statistics should be distinguished.

6. Address metric advertisement churn. Provide guidance on sampling intervals,
maximum update rates, and when to prefer Level 2 over Level 1 to limit
control-plane overhead at scale.

7. Fix editorial issues in Section 6 registry entries (inline ### Output, role
naming inconsistency, typos) and update the normative self-reference to the
current draft version.

8. Add a brief migration narrative explaining how CATS metrics complement or
replace existing IGP TE metrics and ALTO cost metrics during incremental
adoption.

I hope these comments are useful and constructive. The hierarchical metric
framework is well-conceived and addresses a genuine interoperability need for
CATS; adding consolidated operational guidance will significantly improve its
deployability in multi-vendor environments.

Lim Fung