Conversation
d26e144 to
a8538bb
Compare
a8538bb to
d7f2b80
Compare
ShaMan123
left a comment
There was a problem hiding this comment.
cleanup types and indirection
| /** Set on answers, in either direction, despite the name. */ | ||
| toMainThread?: boolean; |
There was a problem hiding this comment.
this needs a decision - probably a rename
|
On removing Two more things before this lands. The rest of the analysis holds up well: we verified all three race claims against main, and the suite passes 129 of 129 with your changes applied. |
|
I understand that deleting the Regarding the throws. What do you suggest?
I can suggest that aborting returns a promise that resolves once completed. This will require the consumer to wait upon it - not good DX.
I assume this is exactly the async round trip being able to drop the message of a dropped model. |
|
Agreed on the direction, and thanks for laying out the reasoning. AbortSignal is the right primitive and we are happy for it to become the recommended way to cancel a load. What we cannot do is remove On the dispose-then-reload window: rather than making |
|
I will rework the PR.
I think the simplest solution would be to manage an internal id mapping. A model will get an internal id that is unique (growing counter), |
eb03100 to
d3f0b16
Compare
Every model gets a uid from FragmentsModels that is never reused, and all internal routing uses it instead of the modelId: messages to and from the workers, thread assignment, progress callbacks, the tile queue, material definitions and the worker's mesh cache. Late work of a disposed model can no longer land on a newer model loaded under the same modelId. The public `models.list` stays keyed by modelId. dispose() now takes effect before it returns: the model leaves the models list and the scene, its thread slot is freed, its in-flight load is aborted and later requests to it reject. The returned promise resolves once the worker has deleted the model. The modelId can be loaded again right away. - load() rejects as soon as it is aborted or its model is disposed, without waiting for the worker to unwind. - A DELETE_MODEL that lands mid-load aborts the load on the worker instead of disposing the model from under it. - Terminating a worker rejects the requests still waiting on it. - A FINISH from a disposed model still settles update fences. - editor.save() no longer reuses the old model's materials; they are disposed with it in finalizeDispose(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings back the abort() method and its tests that a2ca482 removed. Models are keyed by uid now, so it needs no bookkeeping of its own: it looks up the model currently loading under the ID and aborts that load the same way its signal would. An abort can't outlive its load or reach a later load of the same ID. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delta models and queued element requests belonged to a modelId, so a model loaded under a disposed model's modelId inherited them, including after editor.save(). They now belong to the model's uid and go away with it. - Disposing a model disposes its delta models. - reset() clears deltaModelId along with the delta models. - An edit or reset whose model is disposed meanwhile disposes the delta it built instead of leaving it loaded; concurrent edits no longer orphan a delta model. - save() keeps the old delta models in the scene until the new model replaces them, and does nothing if the model is disposed meanwhile. - relate(), unrelate() and deleteData() resolve their model once, and drop their work if it is disposed meanwhile. createElements() returns the elements of the model it edited. - Delta model IDs end in a counter instead of performance.now(), so two can't collide. - Queuing element requests for a model that isn't loaded throws. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Looking into the fix - I prefer first fixing modelId not being unique issue and add aborting on top of that. |
manageRequest is now the connection's handler itself, so the connection awaits it and answers a request it fails to handle with an error. The requests it receives are typed as WorkerRequest, which the workers' senders are checked against with `satisfies`, and Connection is generic over the requests it receives. The workers send these requests fire-and-forget, so they catch that error answer instead of leaving an unhandled rejection; the main thread already logs the failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FragmentsModel._invoke() only takes names of VirtualFragmentsModel's methods, with arguments matching their parameters, and resolves to their return type. A renamed or retyped worker method no longer fails at runtime only. It showed worker methods declaring narrower parameters than they handle: the highlight and visibility methods take no ids to mean every item, and getItemsData() takes GUIDs too. Their signatures now say so. Results are still cast at the call sites: they are copied between threads, so class instances in them arrive as plain objects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FragmentsModel.getItemsGeometry() and getGeometries() returned MeshData whose `transform`, typed as a THREE.Matrix4, was the plain object the worker's copy arrives as, so calling a Matrix4 method on it threw. They now rebuild it, as ItemGeometry.get() already did. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CONTRIBUTING.md gets a Design patterns section: a model's consumer-facing modelId against the internal uid everything is keyed and routed by, why async work needs the latter, and the rules for new code. Also updates the mental model's stale threads.invoke(modelId, ...) line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_invoke is typed now, and every one of these arguments matches its worker method, so the comments only switched off the argument checking. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_invoke claimed to resolve to what the worker method returns, but the
worker's answer is a structured-clone copy: class instances in it arrive
without their prototype. And nothing checked the claim, because
FragmentsConnection.fetch/invoke resolved to `any`.
Cloned<T> describes that copy: methods are dropped, so a THREE.Matrix4 is
typed as `{ elements, isMatrix4 }` and no longer passes for one. invoke<R>()
resolves to Cloned<R>, with R (default any) the method's return type, and
FragmentsConnection.fetch keeps the request's type instead of dropping it,
so the execute and box requests now declare the fields the worker answers
with.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_invoke accepted any public method of VirtualFragmentsModel, including ones that break over the wire: dispose() deleted the worker-side model behind the main thread's back, and getItemsByConfig() takes a function, which can't be copied to the worker. It now takes only the methods listed in RemoteMethods, which also documents what crosses the thread boundary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getHighlight() and getItemsMaterialDefinition() returned definitions whose `color`, typed as a THREE.Color, was the plain copy the worker's answer arrives as, so calling a THREE.Color method on it threw. resetColors() meant to rebuild it but checked `isColor`, which the copy keeps as an own property, so it never did, and getItemsMaterialDefinition() didn't call it. MaterialManager.restoreColor() rebuilds the color from a copied definition and is typed that way, so a definition can't be handed out without it. It keeps the components as they are: the worker already converted them from sRGB, and the old fallback would have converted them again. Definitions the worker transfers for rendering go through it too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…copy EditManager.restoreTransforms() takes the geometries as they arrive from the worker and returns them with their transform rebuilt, so dropping the rebuild is now a type error rather than a cast that hides it. ItemGeometry uses it instead of its own copy of the loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getBuffer() is compressed by default, and compressed it resolves to a Uint8Array, but it was cast to an ArrayBuffer, so `new DataView(buffer)` type-checked and threw. The cast couldn't simply go: the worker method's inferred type, ArrayBufferLike, absorbs Uint8Array too. The worker method now declares both, which is also what IFragmentsModel says. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Item.getAttributes() cast the worker's attributes to `type?: number`, and AttributeData and ItemAttributes.setType() took a number too, but the types are the names the importer writes, such as "IFCLABEL", and the worker declares them as strings. A numeric comparison against them never matched. They are strings now, and the cast is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getSpatialStructure(), getMetadata(), getCRS() and getSequenced() on the worker returned `any`, so the casts on the main thread were the only type their results had. They now declare what they return. That showed getSequenced() handing out results as the worker's copies: "mergedBoxes" typed as a THREE.Box3, "geometry" transforms as THREE.Matrix4 and "highlight" colors as THREE.Color were all plain objects. Each kind of result is now rebuilt from the copy, by a map keyed by the result kind, so a kind that needs rebuilding can't be left out. It resolves to null when the worker doesn't know the result kind, as its documentation says. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Most of them claimed exactly what _invoke already resolves to, each as a separate unchecked claim, so a worker method changing its result would go unnoticed. Where a result is handed out under a named type (ItemData, SpatialTreeItem, the raw data maps, ...), a return annotation keeps that name in the declarations and is checked against the worker's result. The casts that stay are the index methods' and getMetadata()'s: their type parameters are the caller's word for an index's or the metadata's shape, which the worker can't check either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The answer was posted outside the try/catch, so a result postMessage() can't copy, such as one holding a function, threw a DataCloneError as an unhandled rejection and the request never settled. The request is now answered with that error instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The main thread answers a worker's request with the request itself. RequestsManager empties a RECOMPUTE_MESHES request's tile list once it has taken the tiles, but a disposed model's request was dropped with its list in place, so every tile batch its worker still flushed was copied back, buffers and all. It is emptied there too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A call on a disposed model, e.g. through an Item or Element kept from before, was rejected by the connection as "model 7 is not loaded": 7 is the model's internal uid, which the caller has never seen. _invoke now rejects it first as 'model "arq" is disposed'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
save() disposes the model, keeping its visuals until the reloaded model is in the scene, and tore them down only after the reload. When the reload rejected (an abort, the model disposed again, FragmentsModels disposed, a worker error), the old THREE object, tiles, materials and delta models stayed in the scene for good: dispose() had already run, so nothing could clean them up. The teardown now runs either way, and the reloaded model joins the scene as soon as it is loaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
save() replaces the model with a reloaded one under a new uid, and disposing the old one dropped the element requests queued for it (with createMaterial(), setItem(), createIndex(), ...) and not applied yet, so a later applyChanges() lost them without a word. Before requests were keyed by uid they carried over under the modelId. They now move to the reloaded model, temp ids included, the way save() already passes undone requests on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each edit loads a delta model built from every request the worker had when the edit reached it, and the edit shows it once it has loaded, replacing the one shown. Loads run concurrently, so with two edits in flight the first edit's delta model could finish last and replace the second's, showing the model without the second edit until the next edit that affects rendering. A reset raced the same way: a delta model loaded after it was shown anyway. Edits and resets are now numbered in the order they reach the worker, and a delta model of an earlier one than the model shows is disposed instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getMetadata() promises an object (IFragmentsModel types it as T), but the
worker answered null for a model whose file has no metadata, which the
main thread's cast passed through. It answers {} now, so the promise
holds without every caller having to check for null. getCRS() still
answers null for a model without a CRS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getHighlight() resolved to MaterialDefinition[], but it has an entry for every item it looks up, undefined for an item without a highlight, and a highlight that preserves the item's original material (setColor(), setOpacity(), resetColor(), ...) only holds the properties it changes, so it can have no color. Both were cast away. HighlightDefinition types such a highlight, and getHighlight() now resolves to (HighlightDefinition | undefined)[], as does getSequenced() for "highlight". highlight() accepts a HighlightDefinition, so a highlight read with getHighlight() can be applied again, as the MaterialsManagement example does. The worker's material list and the definitions it sends the main thread are typed the same way, which drops their casts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A RECOMPUTE_MESHES request carried its tile requests as any[], built in four places of VirtualTilesController and read in six classes on the main thread, none of which could tell what they held. TileRequest types them, one member per TileRequestClass, and the worker side now builds, batches and prepares them for transfer against it. Typing the main side's reads comes next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Everything a connection receives from the other side, requests and answers alike, is a structured-clone copy, so ThreadHandler and fetch() now type it as Cloned. On the main thread that reaches the classes that take the tile and material requests, which read them as `any` until now: RequestsManager, MeshManager, LODManager and MaterialManager are typed against the requests, and the tile matrix and bounding box they got as copies are rebuilt instead of passed on as THREE objects. MaterialManager keeps the definitions the worker transfers as HighlightDefinition[], with one documented narrowing: a tile's own material is one of the model's, which are complete. The THROW_ERROR branch and the templateId reads go, since no worker sends either, and a tile's positions are typed as the Float32Array the worker allocates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
More work spawned out of this branch so I want to plan how to move forward. Recommended splitStacked, in merge order. Each boundary was verified green on
′ marks a commit adapted by hand during the rebase. |
|
if you don't mind that everything is on this branch - then I can review it, tidy up and hand it off |
|
Regardless the design is clear - modelId is not a uid, using a uid behind scenes is the fix. |
SPLIT_PLAN.md lays out landing ThatOpen#305 as six stacked PRs on upstream/main: which commits go where and why, what changes in the public types, what reviewers should focus on, and the test results at each boundary from replaying the stack, including a fence hang that rebasing onto upstream introduces without a conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Description
Adds a
signaloption toload()(#173) and fixes the concurrency bugs found along the way.The root cause: all model state, on both threads, was keyed by the consumer's
modelId, and amodelIdcan be reused and therefore not unique. Late async work of a model that had been disposed or replaced landed on the next model loaded under the samemodelId: worker replies, queued tiles, material transfers, progress events, an in-flight dispose.How
Internal uid. Each model gets a uid from
FragmentsModelsthat is never reused. All internal routing uses it:modelIdstays the consumer-facing name:models.list,disposeModel(),abort()and the editor API are still keyed by it. The pattern and its rules are written up inCONTRIBUTING.mdunder "Design patterns".dispose()takes effect before it returns. The model leavesmodels.listand the scene, its worker slot is freed, an in-flight load is aborted, and later requests to it reject. The returned promise resolves once the worker has deleted it. ThemodelIdcan be loaded again right away. Anything the old model's worker still sends is dropped silently.load({ signal }). Aborting rejectsload()with aLoadAbortedErrorright away, without waiting for the worker to reach its next yield point, and disposes the partial model. Disposing the model mid-load does the same.Typed messaging.
WorkerRequest, and the workers' senders are checked against it.FragmentsModel._invoke()only takes method names of the worker-side model, with matching arguments. Results are still cast at the call sites, because class instances in them arrive as plain objects.Review feedback
abort(modelId)is kept. It now shares the signal's code path: it aborts whichever load currently runs under that ID, with no bookkeeping of its own.load()no longer throws while a disposed model is still being deleted on the worker. With uid keying, that window is harmless, so themodelIdis free as soon asdispose()is called.fetchMeshCompute:Bugs fixed (all reproducible on
main)modelIdthat is already loaded or loading silently clobbered the first model. It now throws.modelId:threads.delete(modelId)could remove the new model's worker assignment.modelId. Two models with the samemodelIdon one worker read, and evicted, each other's meshes.abort()of an ID that isn't loading, a call on a disposed model) spawned a new worker. OnlyCREATE_MODELassigns one now; other requests reject.modelIdinstead of going back to the worker that asked.autoCoordinate: false, firedonModelLoadedfor the dead model;forceUpdateFinish().modelId, so a reloaded or saved model inherited the old one's delta models and queued element requests:getItemsGeometry()andgetGeometries()returnedtransformas the plain object the worker's copy arrives as, typed as aTHREE.Matrix4, so calling aMatrix4method on it threw.Behavior changes
load()of amodelIdthat is already loaded or loading throws.LoadAbortedErrorinstance (it was the worker's error string). That includes an abort that lands after the worker finished, whichmainignored.abort(modelId)for an ID that isn't loading does nothing (it spawned a worker).dispose(),disposeModel()andFragmentsModels.dispose()apply their effects synchronously and still return a promise.ElementorItem, or the old model aftereditor.save(), rejects with "model N is not loaded". Before, it went to whatever model owned themodelId.editor.save(): the new model gets its own materials instead of reusing the old one's. They leaked, since reused materials were never tracked for disposal. The old model's delta models stay in the scene until the new model replaces them.createMaterial()etc.) for a model that isn't loaded throws.<modelId>-DELTA-MODEL-<n>) instead ofperformance.now().EditUtils.getRootModelId()still parses them.model.modelIdno longer readsmodel.object.name, so renaming the object doesn't change it.uid. The worker bundle must match the library version, whichFragmentsModels.getWorker()already guarantees.Internal API changes
These are documented as "don't use directly", but they are public on their classes:
new FragmentsModel({ modelId, uid, meshManager, threads, editor, threadGroup })takes a single object.new Editor(core)no longer takes the connection.model.threads.invoke(uid, …)takes the uid. Models usemodel._invoke(…), which is typed againstVirtualFragmentsModel.MaterialManager.releaseModelSlot()is removed.RequestsManager.clean(uid)andFragmentsConnection.delete(uid)take the uid, and the latter returns a promise.Connection<TInput>andThreadHandler<T>are generic, and the newWorkerRequesttype is exported.VirtualFragmentsModelsignatures now declare what the methods already handle:getItemsData()accepts GUIDs.Code quality
Typed the message layer (
MessageBase,WorkerRequest,_invoke()) in place ofany, and collapsed some indirection inConnection. Routing is synchronous, so a request that can't be routed rejects instead of staying pending forever.Additional context
The branch history is kept for reviewers of the first version. The rework after review:
b48ef60key models by an internal uid, dispose synchronously2d759f1keepabort(modelId)(reverting its removal ina2ca482)e0fc555key editor state by the model's uid99be14btype the requests workers send to the main thread, catch the workers' fire-and-forget sends11e8e78type_invoke()against the worker model's methods6610477return geometry transforms asTHREE.Matrix4d1d3dc0document themodelIdvs uid pattern inCONTRIBUTING.md6610477stands on its own if you'd rather take it separately, and so does the editor commit.Worth a close look:
dispose()contract inFragmentsModel.dispose/DataManager.dispose/FragmentsConnection.delete.ThreadModelDeleter.Tests: 168 passed, plus 1 expected fail (the existing #300 repro). New coverage:
load-abort.test.ts;fragments-connection.test.ts;school_arq.frag:thread-model-deleter.test.ts;edit-helper.test.ts;edit-manager.test.ts;mesh-manager-fence.test.ts.I checked that the key tests fail without their fix. None of this has been exercised in a browser with real workers yet.
What is the purpose of this pull request?
Before submitting the PR, please make sure you do the following:
feat(examples): add hello-world example).fixes #123).🤖 Generated with Claude Code