InputCapability::time() requires TotalOrder and returns Option - #817
Merged
Merged
Conversation
A message's stamp is a multiset of timestamps, and an input capability's time() used to panic unless the stamp was a singleton. It now exists only for totally ordered timestamps, where every non-empty stamp has a least element, and returns Option<&T>: the least element, or None for a message stamped by no capabilities. retain(port) follows suit, and the Deref to T is removed, so partially ordered timestamps must read stamp() and forward its elements with retain_stamp(). Stamp::least() is the underlying accessor. Operators whose logic reads one time from the capability are bounded by TotalOrder and drop messages without a time: count/accumulate, aggregate, state_machine, delay, delay_batch, branch, branch_when, and the inspect_batch family. A new inspect_stamp observes each container with its stamp for any timestamp, and inspect is built on it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
Data is released at a clock time once some element of its stamp is less or equal to it, so reclock needs no TotalOrder bound. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
frankmcsherry
force-pushed
the
stamp-totalorder
branch
5 times, most recently
from
September 11, 2026 18:17
3eec58e to
7393168
Compare
for_each_time is for_each_stamp: containers are grouped by the stamp of their message, not by a time, and the closure receives a capability for that stamp. Closure arguments named `time` are renamed `cap` throughout: they were never a time. for_each_time remains as a deprecated alias. InputCapability::time() is least() and retain(port) is retain_least(port): the names say what is computed (the least element of a totally ordered stamp, or None for an empty one), and no existing time() or retain() call site silently inherits new behavior. retain_stamp() is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
frankmcsherry
force-pushed
the
stamp-totalorder
branch
from
September 11, 2026 18:28
7393168 to
9d8a3bd
Compare
…iscards timeless messages; book inspect_core observes every container with its stamp and every frontier change, for any timestamp; inspect observes records. inspect_batch, inspect_time, inspect_stamp, inspect_container, and the InspectCore trait are removed: they revealed one time, and are a map inside inspect_core. reclock discards a message sent under no capabilities, which it could never release. The book uses for_each_stamp, cap, and retain_least. Stamp::least explains why it is a reduction and not first(). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
frankmcsherry
force-pushed
the
stamp-totalorder
branch
from
September 11, 2026 20:05
05ba649 to
f0f7160
Compare
frankmcsherry
added a commit
that referenced
this pull request
Sep 11, 2026
…ntrol and Result helpers Removed: delay/delay_batch/delay_total, count/accumulate, aggregate, state_machine, branch, branch_when, reclock, iterator_source (flow_controlled), and ResultStream. None has a user in differential-dataflow or Materialize. The first seven key their logic on the capability's time, which records do not carry; since #813 a message's stamp is a set of times, and since #817 these operators required a total order. branch_when in particular routes a whole container by its capability, which is wrong for any stream whose records carry their own times. The pingpong example carries a round counter in its data; the loop examples in the docs and the book terminate by data; the flow-control chapter assigns timestamps with a small unary operator and delayed capabilities. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK
frankmcsherry
added a commit
that referenced
this pull request
Sep 11, 2026
…ntrol and Result helpers (#818) Removed: delay/delay_batch/delay_total, count/accumulate, aggregate, state_machine, branch, branch_when, reclock, iterator_source (flow_controlled), and ResultStream. None has a user in differential-dataflow or Materialize. The first seven key their logic on the capability's time, which records do not carry; since #813 a message's stamp is a set of times, and since #817 these operators required a total order. branch_when in particular routes a whole container by its capability, which is wrong for any stream whose records carry their own times. The pingpong example carries a round counter in its data; the loop examples in the docs and the book terminate by data; the flow-control chapter assigns timestamps with a small unary operator and delayed capabilities. Claude-Session: https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since #813 a message's stamp is a multiset of timestamps, and
InputCapability::time()panicked unless the stamp was a singleton. This turns that runtime failure into a compile-time one.The change.
InputCapability::time()is replaced byleast(), which exists only forT: TotalOrder, where every non-empty stamp has a least element, and returnsOption<&T>: the least element, orNoneexactly when the message was stamped by no capabilities (it makes no progress claims).retain(port)is replaced byretain_least(port), returningOption<Capability<T>>likewise, and theDereftoTis gone. Partially ordered timestamps must readstamp()and forward its elements withretain_stamp(), which is unchanged.Stamp::least()is the underlying accessor. The names say what is computed, and no existingtime()orretain()call site silently inherits new behavior: each one fails to compile and is pointed atleast,retain_least, orretain_stamp.An operator that needs a time asks for it with an
if let, and a message with no capabilities is dropped. That is a decision, not an accident: such a message makes no progress claims and may arrive after the frontier has passed every time in its contents, so an operator whose work is keyed by time has nothing to do with it.Operators. Those whose logic reads one time from the capability are bounded by
TotalOrderand use the idiom:count/accumulate,aggregate,state_machine,delay,delay_batch,branch, andbranch_when. For total orders they behave exactly as before; for partial orders they no longer compile, which is deliberate: they use the capability as a stand-in for a time the records do not carry, and there is no correct generalization of that to a message stamped by several times. (An earlier draft keyed their stashes by stamp and flushed once every element completed; that stalls in a loop whenever a pointwise boundary map produces comparable elements, so it was dropped.) The bound is the static property "at most one meaningful time"; it makes holding capabilities sound, and does not give these operators meaning for a multi-element stamp. They have no users in differential-dataflow or Materialize and will be removed in a follow-up PR.Inspect is one trait with two methods, neither bounded:
inspect_coreobserves every container with its stamp and every frontier change, andinspectobserves records.inspect_batch,inspect_time,inspect_container, and theInspectCoretrait are removed; each revealed one time, and each is a map insideinspect_core.reclockis rewritten to stash by stamp and notify at each clock element, releasing data once some element of its stamp is at or before the clock time; a message with no capabilities could never be released and is discarded. It needs no bound. The book is updated to the new names.Rename.
for_each_timeis nowfor_each_stamp: containers are grouped by the stamp of their message, not by a time, and the closure receives a capability for that stamp. Closure arguments namedtimeare renamedcapthroughout the docs and examples; they were never a time.for_each_timeremains as a deprecated alias.Breaking changes.
InputCapability::time()andretain()are removed in favour ofleast()andretain_least(), with aTotalOrderbound andOptionresults;Deref for InputCapabilityis removed, so implicit uses (*cap,cap.less_equal(..)) become explicit; the operators above gain aTotalOrderbound;Debug for InputCapabilityprints the stamp. Examples and doctests are adapted; theloop_variabledoctest terminates by data rather than bybranch_when, which now needs a total order.Verification. All unit, integration, and 155 doctests pass. A differential-dataflow adaptation (eight files: capture, upsert, half_join,
Collection::delayreimplemented withoutdelay_batch, diagnostics, two tests) builds and passes its workspace tests against this branch and will follow as its own PR once this lands.🤖 Generated with Claude Code
https://claude.ai/code/session_012k2GSwxmvD2LvckkoXi6GK