Network Working Group S. Dikshit
Internet-Draft Aruba Networks, HPE
Intended status: Informational 30 August 2026
Expires: 1 March 2027
Comparable Sequence Numbers Across Publishers in YANG
Datastore Telemetry
draft-dikshit-netconf-yang-push-causal-ordering-00
Abstract
YANG Datastore Telemetry, YANG-Push version 2
[I-D.ietf-netconf-yang-push-2], assigns each publisher a
monotonically increasing sequence number so a receiver can detect
loss and reordering. The current design leaves two questions
unanswered: what happens when the counter wraps, and how a
receiver aggregating records from more than one publisher is to
compare sequence numbers that were never defined to be comparable
across publishers in the first place. Both questions have already
been studied, and largely settled, in the distributed systems
literature under the heading of logical and hybrid logical clocks,
and in deployed streaming systems under the heading of offset and
epoch numbering. This document evaluates three existing causal-
ordering primitives against YANG-Push's specific constraints and
proposes adopting a Hybrid Logical Clock so that the wraparound and
cross-publisher comparison problems are removed by construction
rather than patched with a wider counter.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current
Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as "work in
progress."
This Internet-Draft will expire on 1 March 2027.
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this
document. Please review these documents carefully, as they
describe your rights and restrictions with respect to this
document.
Table of Contents
1. Introduction
2. Problem Statement
3. Three Existing Causal-Ordering Primitives
3.1. Per-Publisher Monotonic Counter (Current Design)
3.2. Vector Clocks
3.3. Hybrid Logical Clocks
3.4. Comparison
4. Proposed Mechanism: HLC-Tagged Notifications
5. Applicability to Wraparound and Cross-Publisher Comparison
6. Relationship to Existing Work
7. Security Considerations
8. IANA Considerations
9. References
9.1. Normative References
9.2. Informative References
Author's Address
1. Introduction
A receiver of YANG-Push version 2 [I-D.ietf-netconf-yang-push-2]
notifications, particularly in the distributed subscription
architecture defined by
[I-D.ietf-netconf-distributed-notif], must answer two questions
for every record it receives: did this record arrive out of order
relative to others from the same publisher, and, when correlating
records from more than one publisher, which of two records
happened first. The current design answers the first question
with a per-publisher monotonically increasing sequence number and
leaves the second question, and the counter's own wraparound
behavior, unspecified.
This is not a novel problem invented by YANG-Push. It is the
problem of assigning a total or partial order to events generated
by more than one independent process without a shared clock, and
it has an existing body of work with known, provably correct
solutions and known trade-offs between them. This document applies
that body of work to YANG-Push's specific setting rather than
re-deriving a weaker answer from scratch.
2. Problem Statement
[I-D.ietf-netconf-yang-push-2] Section 10.1 defines a sequence
number that increases monotonically per publisher-id in
distributed subscription mode. Two properties are left unstated:
1. Wraparound: the field's width bounds how long it can run
before repeating. Nothing in the current text tells a
receiver how to distinguish a genuinely new record whose
sequence number has wrapped from a replayed or duplicated old
record with the same numeric value.
2. Cross-publisher comparability: the sequence number is scoped
per publisher-id. Nothing states, or could state without an
additional mechanism, whether a sequence number 105 from
publisher A happened before, after, or concurrently with
sequence number 40 from publisher B. A receiver fusing
multiple publishers' streams, which is the entire purpose of
the distributed architecture in
[I-D.ietf-netconf-distributed-notif], has no way to establish
this relationship from the sequence number alone.
3. Three Existing Causal-Ordering Primitives
3.1. Per-Publisher Monotonic Counter (Current Design)
The mechanism currently in the notification envelope traces to
[I-D.tgraf-netconf-notif-sequencing] (Graf, Quilbeuf, Huang Feng),
which added a sysName and a per-publisher sequenceNumber to the
NETCONF notification header specifically to let a downstream
system "recognize loss, delay and reordering between the
publisher and the downstream system." That mechanism, since folded
into the notification envelope YANG-Push v2 now uses (visible as
the "hostname" and "sequence-number" fields in
[I-D.ietf-netconf-yang-push-2] Section 7.1's own example), was
designed to solve loss and reordering detection for a single
publisher's own stream, and it solves that problem well. It was
not designed to give a receiver a way to compare sequence numbers
across two different publishers, and does not, on its own text,
claim to. This document's problem, cross-publisher causal
ordering and wraparound at scale, is a different, harder problem
that the same field has not yet separately named.
State cost: O(1) per publisher. Causality precision: none across
publishers; total order only within a single publisher's own
stream. Wraparound handling: none defined. This is the weakest of
the three options along every axis except state cost, which is
why it is a reasonable starting point, and the right choice for
the loss/reorder problem [I-D.tgraf-netconf-notif-sequencing]
solves, but not a sufficient end state for the cross-publisher,
distributed architecture [I-D.ietf-netconf-distributed-notif]
adds on top of it.
3.2. Vector Clocks
Vector clocks (Fidge, 1988; Mattern, 1989) assign each process a
vector with one component per known process; a component is
incremented on a local event and the whole vector is carried on
every message, with the receiver taking the componentwise maximum.
Two vectors are comparable exactly when one is componentwise less
than or equal to the other; otherwise the events are, correctly,
reported as concurrent. This gives exact causality, at a state and
wire cost of O(n) in the number of publishers, which grows exactly
when a distributed telemetry architecture also grows: as more
publishers are added to a subscription.
3.3. Hybrid Logical Clocks
A Hybrid Logical Clock (HLC) [Kulkarni2014] pairs a physical
timestamp with a logical counter: on a local event, the physical
component takes the maximum of the local clock and the largest
physical component seen so far, and the logical counter increments
only when two events would otherwise tie on the physical
component. HLC values from different publishers are totally
ordered by the pair (physical, logical), require O(1) state per
publisher (not O(n)), remain within a bounded skew "epsilon" of
true physical time (making them usable directly as an approximate
wall-clock timestamp as well as a causal order), and have been in
production use for this exact purpose in distributed databases
(CockroachDB, MongoDB) since the cited paper.
3.4. Comparison
+====================+===========+===============+=============+
| Primitive | State | Cross-pub | Wraparound |
| | cost | causality | |
+====================+===========+===============+=============+
| Monotonic counter | O(1) | none | undefined |
| Vector clock | O(n) | exact | per-comp. |
| Hybrid Logical Clock | O(1) | total order, | physical |
| | | epsilon-bound | component |
| | | | resets it |
+====================+===========+===============+=============+
Table 1
4. Proposed Mechanism: HLC-Tagged Notifications
This document proposes augmenting the YANG-Push v2 notification
header with an HLC value in place of, or alongside, the existing
per-publisher sequence number:
augment /notification {
leaf hlc-physical {
type uint64;
description
"Physical component of a Hybrid Logical Clock, per
[Kulkarni2014], in milliseconds since the Unix epoch.";
}
leaf hlc-logical {
type uint32;
description
"Logical component of a Hybrid Logical Clock, used to
order events that tie on hlc-physical.";
}
}
A receiver orders any two records, from the same or different
publishers, by comparing (hlc-physical, hlc-logical)
lexicographically. Because hlc-physical is derived from wall-clock
time rather than from a per-publisher event count, "wraparound" in
the sense the current design has to answer for no longer arises
within any realistic subscription lifetime, and comparability
across publishers is total rather than undefined, requiring no
per-publisher-count state growth.
5. Applicability to Wraparound and Cross-Publisher Comparison
The two open questions from Section 2 are answered as follows
under this proposal: wraparound is deferred to the range of a
64-bit millisecond timestamp, which is not a practical concern
within a subscription's operational lifetime, and cross-publisher
comparison is total and requires no additional per-publisher state
at the receiver, unlike the vector-clock alternative in Section
3.2. The trade-off accepted is the bounded-skew assumption
inherent to HLC: publishers must share a loosely synchronized
physical clock (ordinary NTP synchronization is sufficient per
[Kulkarni2014]), and a receiver MUST treat any HLC pair whose
physical component is more than a configured epsilon ahead of the
receiver's own clock as suspect, following the anomaly handling
already recommended for HLC deployments.
6. Relationship to Existing Work
[I-D.ietf-nmop-yang-message-broker-message-key] independently
arrived at a byte-identical, order-sensitive key requirement for
YANG-Push records re-published through a message broker (Section
3.1.1 of that document requires XPath expressions sorted
lexicographically, deduplicated, with no trailing newline). That
requirement was derived for topic-key compaction, not for causal
ordering, but it is the same underlying problem, a third
independent instance of the field needing a well-defined
comparability primitive for YANG-Push data and arriving at an ad
hoc, document-specific answer each time. This document's HLC
proposal, if adopted at the yang-push-2 layer, would give that
document's key-compaction logic a ready-made, already-ordered
input rather than requiring it to separately define one.
7. Security Considerations
A publisher with a manipulated physical clock could advance its
HLC's physical component arbitrarily far ahead, causing its
records to always sort last (or, if adopted as a display
timestamp, to display an incorrect time). This is a bounded
variant of the same clock-trust assumption already implicit in any
NTP-synchronized system and is mitigated by the bounded-skew
rejection rule in Section 5. This is not a new attack surface
relative to the current per-publisher counter design, which
already assumes a trustworthy publisher; it does add a specific,
testable bound (epsilon) that the current design has no equivalent
of.
8. IANA Considerations
This document requests no IANA actions in this revision. A future
revision defining the augmenting YANG module will register it in
the "YANG Module Names" registry.
9. References
9.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
9.2. Informative References
[I-D.ietf-netconf-yang-push-2]
"YANG Datastore Telemetry (YANG Push version 2)", Work
in Progress.
[I-D.tgraf-netconf-notif-sequencing]
Graf, T., Quilbeuf, J., and A. Huang Feng, "Support of
Hostname and Sequencing in YANG Notifications", Work in
Progress, draft-tgraf-netconf-notif-sequencing-06 (now
superseded by draft-netana-netconf-notif-envelope), June
2024.
[I-D.ietf-netconf-distributed-notif]
"Subscription to Notifications in a Distributed
Architecture", Work in Progress.
[I-D.ietf-nmop-yang-message-broker-message-key]
"YANG Message Keys for Message Broker Integration",
Work in Progress.
[Kulkarni2014]
Kulkarni, S., Demirbas, M., Madappa, D., Avva, B., and
M. Leone, "Logical Physical Clocks and Consistent
Snapshots in Globally Distributed Databases", 2014.
Author's Address
Saumya Dikshit
Aruba Networks, HPE
Email: saumya.dikshit@hpe.com