Skip to content

Take ccall element pointers without a method on Base.RefValue - #1366

Open
maleadt wants to merge 1 commit into
JuliaArrays:masterfrom
maleadt:tb/ccall-invalidations
Open

maleadt wants to merge 1 commit into
JuliaArrays:masterfrom
maleadt:tb/ccall-invalidations

Conversation

@maleadt

@maleadt maleadt commented Oct 23, 2026

Copy link
Copy Markdown

Loading StaticArrays invalidates code that is already compiled for Base's unsafe_convert(::Type{Ptr{T}}, ::RefValue{T}). The cause is

Base.unsafe_convert(::Type{Ptr{T}}, a::Base.RefValue{SA}) where {S,T,D,L,SA<:SArray{S,T,D,L}}

and its FieldArray counterpart, which give ccall a pointer to the elements. T appears in both arguments. Type intersection cannot rule out an overlap between that signature and Base's method for an abstract T, so every compiled specialization of Base's method with an abstract T is invalidated. This method is the largest source of invalidations from loading StaticArrays (@snoop_invalidations using StaticArrays, Julia 1.13):

Session before using StaticArrays Invalidated, master From this method Invalidated, this PR
fresh 60 13 51
after using DataFrames, InlineStrings, HTTP, JSON3 662 124 563

With this PR, cconvert wraps the RefValue in an internal type when the argument is a pointer to the elements. The wrapper has the only unsafe_convert method, so nothing is defined on Base.RefValue. A helper decides by dispatch which arguments take this path, using the signatures of the removed methods. Every other argument keeps the RefValue that cconvert returned before, and Base converts it as it did. Neither new method roots an invalidation.

Behavior is unchanged:

  • ccall passes the same addresses as before.
  • A mutable FieldArray is still passed in place.
  • Arguments that errored before still error.
  • ccall still doesn't allocate: the decision is made at compile time and inlined.

These conversions had no tests. The new ones pass SArrays and immutable and mutable FieldVectors as Ptr{T}, Ptr{Cvoid} and a pointer to the array type, directly and as a Ref, and read and write through the pointer. They also cover Refs with an abstract element type such as Ref{SVector}, which Base converts. The tests pass on master and with this change, on Julia 1.10, 1.12 and nightly. The full suite passes on 1.12, with no new Aqua ambiguities or piracies.

`SArray` and `FieldArray` gave `ccall` a pointer to their elements with

    unsafe_convert(::Type{Ptr{T}}, a::Base.RefValue{SA}) where {S,T,D,L,SA<:SArray{S,T,D,L}}

and the `FieldArray` equivalent. These tie `T` through both arguments, so
type intersection cannot rule out an overlap with Base's
`unsafe_convert(::Type{Ptr{T}}, ::RefValue{T})` for an abstract `T`, and
loading StaticArrays invalidated every compiled specialization of that
method with an abstract `T`. It is the largest source of invalidations
from loading StaticArrays: on Julia 1.13, 13 of 60 method instances in a
fresh session, and 124 of 662 after using DataFrames, InlineStrings,
HTTP and JSON3. With this change, neither `cconvert` nor `unsafe_convert`
roots an invalidation.

`cconvert` now wraps the `RefValue` in a type of this package when the
argument is a pointer to the elements, and the only `unsafe_convert`
method is on that wrapper. Which arguments take that path is decided by
dispatch on a helper, whose signatures are those of the removed methods.
Everything else is the `RefValue` that `cconvert` returned before, which
Base converts as it did. The addresses are unchanged, `ccall` still does
not allocate, and no conversion that worked before fails or vice versa.

The conversions had no tests; the new ones cover `SArray` and immutable
and mutable `FieldVector`s passed as `Ptr{T}`, `Ptr{Cvoid}` and a pointer
to the array type, directly and as a `Ref`, including `Ref`s with an
abstract element type, reading and writing through the pointer.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant