Skip to content

Publish neighbor list entries through an atomic length - #4128

Open
GuySten wants to merge 2 commits into
openmc-dev:developfrom
GuySten:claude/fix-1445-neighbor-list-race
Open

Publish neighbor list entries through an atomic length#4128
GuySten wants to merge 2 commits into
openmc-dev:developfrom
GuySten:claude/fix-1445-neighbor-list-race

Conversation

@GuySten

@GuySten GuySten commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Description

find_cell_inner() iterates a cell's neighbor list without a lock (src/geometry.cpp:134) while another thread may be inserting into it from find_cell_in_universe() (src/geometry.cpp:315). With the entries in a std::forward_list that is a data race: the reader follows node pointers that the writer publishes with no ordering, so nothing guarantees that a reader which observes a new node also observes the value stored in it. On a weakly ordered machine (aarch64, POWER) it can read an uninitialized value and use it to index model::cells, which is unchecked. ThreadSanitizer reports the race directly, between push_back() at neighbor_list.h:49 and cbegin() at neighbor_list.h:55.

This publishes the entries through an atomic length instead: the writer fills the next slot and releases the new length, and a reader acquires the length before reading any slot. Readers take one snapshot per search rather than re-reading the container on each step. The first INLINE_CAPACITY entries live in the object, so a typical cell never allocates and a search reads the entries from the same cache line as the length; beyond that they are copied into a heap block of twice the capacity, with replaced blocks retained so a reader still traversing one stays valid and chained so the destructor frees them all.

Fixes #1445.

Verification performed:

  • ThreadSanitizer, driving the real container the way find_cell_inner() does (4 threads reading while one appends): 1 warning on develop, 0 with this change.
  • A 12x12 multigroup lattice, 800k histories: tally results bitwise identical to develop at 1 thread.
  • Throughput at 1 thread over 9 interleaved repetitions is indistinguishable from develop (medians 155411 vs 154735 particles/second). At 4 threads the measurement environment was too noisy to resolve a difference of a few percent; every variant peaked between 350k and 380k particles/second. Benchmarking on a quiet machine, ideally with a continuous-energy model, is still worth doing before this is considered settled.

Trade-off to be aware of: sizeof(NeighborList) goes from 16 to 56 bytes, which is per cell rather than per entry, offset by entries costing 4 bytes instead of 16 and by no allocation at all for cells with up to 8 neighbors. INLINE_CAPACITY is the knob if the per-cell cost matters more than the per-entry saving.

Checklist

  • I have performed a self-review of my own code
  • I have run clang-format (version 18) on any C++ source files (if applicable)
  • I have followed the style guidelines for Python source files (if applicable)
  • I have made corresponding changes to the documentation (if applicable)
  • I have added tests that prove my fix is effective or that my feature works (if applicable)

No test is included: the failure is a data race with no deterministic trigger, and the suite has no ThreadSanitizer job to host a regression test.

🤖 Generated with Claude Code


Generated by Claude Code

GuySten and others added 2 commits September 12, 2026 15:22
find_cell_inner() iterates a cell's neighbor list without a lock while
another thread may be inserting into it from find_cell_in_universe(). With
the entries in a std::forward_list that is a data race: the reader follows
node pointers the writer publishes with no ordering, so nothing guarantees
a reader that observes a new node also observes the value in it. On a
weakly ordered machine it can read an uninitialized value and use it to
index model::cells. ThreadSanitizer reports the race between push_back()
and cbegin().

Publish the entries through an atomic length instead: the writer fills the
next slot and releases the new length, and a reader acquires the length
before reading any slot. Readers take one snapshot per search rather than
re-reading the container on each step.

The first entries are held in the object, so a typical cell never allocates
and a search reads them from the same cache line as the length. Beyond that
the entries are copied into a heap block of twice the capacity; a replaced
block is retained so a reader still traversing it stays valid, and because
the capacity doubles, the retained blocks hold fewer entries than the live
one. Entries cost 4 bytes against 16 for a list node, and allocation leaves
the transport loop entirely for cells with no more than INLINE_CAPACITY
neighbors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pac8jD2yokTUULiEUFwLtz
Read the published entries through NeighborList::view() rather than
re-reading the container on each step of the search.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pac8jD2yokTUULiEUFwLtz
@GuySten GuySten added the Bugs label Sep 12, 2026
@GuySten
GuySten marked this pull request as ready for review September 12, 2026 15:58
@GuySten
GuySten requested a review from paulromano September 12, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Split write bug with shared memory neighbor lists

1 participant