Skip to content

InputCapability::time() requires TotalOrder and returns Option - #817

Merged
frankmcsherry merged 4 commits into
masterfrom
stamp-totalorder
Sep 11, 2026
Merged

frankmcsherry merged 4 commits into
masterfrom
stamp-totalorder

Conversation

@frankmcsherry

@frankmcsherry frankmcsherry commented Sep 11, 2026

Copy link
Copy Markdown
Member

Since #813 a message's stamp is a multiset of timestamps, and InputCapability::time() panicked unless the stamp was a singleton. This turns that runtime failure into a compile-time one.

The change. InputCapability::time() is replaced by least(), which exists only for T: TotalOrder, where every non-empty stamp has a least element, and returns Option<&T>: the least element, or None exactly when the message was stamped by no capabilities (it makes no progress claims). retain(port) is replaced by retain_least(port), returning Option<Capability<T>> likewise, and the Deref to T is gone. Partially ordered timestamps must read stamp() and forward its elements with retain_stamp(), which is unchanged. Stamp::least() is the underlying accessor. The names say what is computed, and no existing time() or retain() call site silently inherits new behavior: each one fails to compile and is pointed at least, retain_least, or retain_stamp.

An operator that needs a time asks for it with an if let, and a message with no capabilities is dropped. That is a decision, not an accident: such a message makes no progress claims and may arrive after the frontier has passed every time in its contents, so an operator whose work is keyed by time has nothing to do with it.

input.for_each_stamp(|cap, data| {
    if let Some(cap) = cap.retain_least(output.output_index()) {
        stash.entry(cap.time().clone()).or_default().extend(...);
        notificator.notify_at(cap);
    }
});

Operators. Those whose logic reads one time from the capability are bounded by TotalOrder and use the idiom: count/accumulate, aggregate, state_machine, delay, delay_batch, branch, and branch_when. For total orders they behave exactly as before; for partial orders they no longer compile, which is deliberate: they use the capability as a stand-in for a time the records do not carry, and there is no correct generalization of that to a message stamped by several times. (An earlier draft keyed their stashes by stamp and flushed once every element completed; that stalls in a loop whenever a pointwise boundary map produces comparable elements, so it was dropped.) The bound is the static property "at most one meaningful time"; it makes holding capabilities sound, and does not give these operators meaning for a multi-element stamp. They have no users in differential-dataflow or Materialize and will be removed in a follow-up PR.

Inspect is one trait with two methods, neither bounded: inspect_core observes every container with its stamp and every frontier change, and inspect observes records. inspect_batch, inspect_time, inspect_container, and the InspectCore trait are removed; each revealed one time, and each is a map inside inspect_core. reclock is rewritten to stash by stamp and notify at each clock element, releasing data once some element of its stamp is at or before the clock time; a message with no capabilities could never be released and is discarded. It needs no bound. The book is updated to the new names.

Rename. for_each_time is now for_each_stamp: containers are grouped by the stamp of their message, not by a time, and the closure receives a capability for that stamp. Closure arguments named time are renamed cap throughout the docs and examples; they were never a time. for_each_time remains as a deprecated alias.

Breaking changes. InputCapability::time() and retain() are removed in favour of least() and retain_least(), with a TotalOrder bound and Option results; Deref for InputCapability is removed, so implicit uses (*cap, cap.less_equal(..)) become explicit; the operators above gain a TotalOrder bound; Debug for InputCapability prints the stamp. Examples and doctests are adapted; the loop_variable doctest terminates by data rather than by branch_when, which now needs a total order.

Verification. All unit, integration, and 155 doctests pass. A differential-dataflow adaptation (eight files: capture, upsert, half_join, Collection::delay reimplemented without delay_batch, diagnostics, two tests) builds and passes its workspace tests against this branch and will follow as its own PR once this lands.

🤖 Generated with Claude Code

https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK

frankmcsherry and others added 2 commits September 11, 2026 13:36
A message's stamp is a multiset of timestamps, and an input capability's
time() used to panic unless the stamp was a singleton. It now exists only for
totally ordered timestamps, where every non-empty stamp has a least element,
and returns Option<&T>: the least element, or None for a message stamped by no
capabilities. retain(port) follows suit, and the Deref to T is removed, so
partially ordered timestamps must read stamp() and forward its elements with
retain_stamp(). Stamp::least() is the underlying accessor.

Operators whose logic reads one time from the capability are bounded by
TotalOrder and drop messages without a time: count/accumulate, aggregate,
state_machine, delay, delay_batch, branch, branch_when, and the inspect_batch
family. A new inspect_stamp observes each container with its stamp for any
timestamp, and inspect is built on it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
Data is released at a clock time once some element of its stamp is less or
equal to it, so reclock needs no TotalOrder bound.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
@frankmcsherry
frankmcsherry force-pushed the stamp-totalorder branch 5 times, most recently from 3eec58e to 7393168 Compare September 11, 2026 18:17
for_each_time is for_each_stamp: containers are grouped by the stamp of their
message, not by a time, and the closure receives a capability for that stamp.
Closure arguments named `time` are renamed `cap` throughout: they were never a
time. for_each_time remains as a deprecated alias.

InputCapability::time() is least() and retain(port) is retain_least(port): the
names say what is computed (the least element of a totally ordered stamp, or
None for an empty one), and no existing time() or retain() call site silently
inherits new behavior. retain_stamp() is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
…iscards timeless messages; book

inspect_core observes every container with its stamp and every frontier
change, for any timestamp; inspect observes records. inspect_batch,
inspect_time, inspect_stamp, inspect_container, and the InspectCore trait are
removed: they revealed one time, and are a map inside inspect_core.

reclock discards a message sent under no capabilities, which it could never
release. The book uses for_each_stamp, cap, and retain_least. Stamp::least
explains why it is a reduction and not first().

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
@frankmcsherry
frankmcsherry merged commit 11beaae into master Sep 11, 2026
9 checks passed
frankmcsherry added a commit that referenced this pull request Sep 11, 2026
…ntrol and Result helpers

Removed: delay/delay_batch/delay_total, count/accumulate, aggregate,
state_machine, branch, branch_when, reclock, iterator_source
(flow_controlled), and ResultStream. None has a user in differential-dataflow
or Materialize. The first seven key their logic on the capability's time, which
records do not carry; since #813 a message's stamp is a set of times, and
since #817 these operators required a total order. branch_when in particular
routes a whole container by its capability, which is wrong for any stream
whose records carry their own times.

The pingpong example carries a round counter in its data; the loop examples
in the docs and the book terminate by data; the flow-control chapter assigns
timestamps with a small unary operator and delayed capabilities.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
frankmcsherry added a commit that referenced this pull request Sep 11, 2026
…ntrol and Result helpers (#818)

Removed: delay/delay_batch/delay_total, count/accumulate, aggregate,
state_machine, branch, branch_when, reclock, iterator_source
(flow_controlled), and ResultStream. None has a user in differential-dataflow
or Materialize. The first seven key their logic on the capability's time, which
records do not carry; since #813 a message's stamp is a set of times, and
since #817 these operators required a total order. branch_when in particular
routes a whole container by its capability, which is wrong for any stream
whose records carry their own times.

The pingpong example carries a round counter in its data; the loop examples
in the docs and the book terminate by data; the flow-control chapter assigns
timestamps with a small unary operator and delayed capabilities.


Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant