Repository navigation
[feat] need better to_layout and from_iter design #35
Description
I'm test driving using a function from AK as a sink to UnROOT:
julia> using UnROOT, AwkwardArray
julia> t = LazyTree(UnROOT.samplefile("NanoAODv5_sample.root"), "Events", r"Muon_(pt|eta|phi)$");
julia> t[1:10]
Row │ Muon_phi Muon_pt Muon_eta
│ Vector{Float32} Vector{Float32} Vector{Float32}
─────┼───────────────────────────────────────────────────
1 │ [] [] []
2 │ [-0.305, 0.99] [19.9, 15.3] [0.53, 0.229]
3 │ [] [] []
4 │ [] [] []
5 │ [] [] []
6 │ [] [] []
7 │ [2.71, 1.37] [22.2, 4.43] [-1.13, 1.98]
8 │ [] [] []
9 │ [] [] []
10 │ [] [] []
julia> AwkwardArray.from_iter([collect(t[i]) for i =1:10])
10-element AwkwardArray.RecordArray{(:Muon_phi, :Muon_pt, :Muon_eta), Tuple{AwkwardArray.ListOffsetArray{Vector{Int64}, AwkwardArray.PrimitiveArray{Float32, Vector{Float32}, :default}, :default}, AwkwardArray.ListOffsetArray{Vector{Int64}, AwkwardArray.PrimitiveArray{Float32, Vector{Float32}, :default}, :default}, AwkwardArray.ListOffsetArray{Vector{Int64}, AwkwardArray.PrimitiveArray{Float32, Vector{Float32}, :default}, :default}}, :default}:
{Muon_phi: [], Muon_pt: [], Muon_eta: []}
{Muon_phi: [-0.30541992f0, ...], Muon_pt: [...], Muon_eta: [...]}
{Muon_phi: [], Muon_pt: [], Muon_eta: []}
{Muon_phi: [], Muon_pt: [], Muon_eta: []}
{Muon_phi: [], Muon_pt: [], Muon_eta: []}
{Muon_phi: [], Muon_pt: [], Muon_eta: []}
{Muon_phi: [2.713379f0, 1.3676758f0], Muon_pt: [...], Muon_eta: [...]}
{Muon_phi: [], Muon_pt: [], Muon_eta: []}
{Muon_phi: [], Muon_pt: [], Muon_eta: []}
{Muon_phi: [], Muon_pt: [], Muon_eta: []}This works but is not very ergonomic and also involves copying everything twice.
Because LazyTree is already iterable, I tried to get the following to work
julia> AwkwardArray.from_iter(t)
# errors because `to_layout` doesn't understand `LazyEvent`
# so I implement this to say "LazyEvent is same as NamedTuple"
julia> function AwkwardArray.layout_for(x::Type{UnROOT.LazyEvent{T}}) where T
AwkwardArray.layout_for(T)
endthen I hit a infinite recursion:
ERROR: StackOverflowError:
Stacktrace:
[1] push!(a::AwkwardArray.RecordArray{(:Muon_phi, :Muon_pt, :Muon_eta), Tuple{…}, :default}, iter::UnROOT.LazyEvent)
@ Base ./array.jl:1186
[2] append!
@ AwkwardArray ~/.julia/dev/AwkwardArray/src/AwkwardArray.jl:150 [inlined]
--- the last 2 lines are repeated 52332 more times ---
[104667] push!(a::AwkwardArray.RecordArray{(:Muon_phi, :Muon_pt, :Muon_eta), Tuple{…}, :default}, iter::UnROOT.LazyEvent)
@ Base ./array.jl:1186
[104668] from_iter(input::LazyTree with 3 branches:
Muon_phi, Muon_pt, Muon_eta
)
@ AwkwardArray ~/.julia/dev/AwkwardArray/src/AwkwardArray.jl:2358
Some type information was truncated. Use `show(err)` to see complete types.to me it all just means that:
- we can probably use something that doesn't involve
for ... push!() end - we need to change design of
to_layoutandfrom_itera little bit to allow easy custom type
What I really want to do here is to tell AwkwardArray:
- when you see
evt::LazyEvent, you can just callcollect(evt)first and you will get aNamedTuplewhich you know how to handle
Activity
[-][feat] need Tables.jl interface compatible constructor[/-][+][feat] need better `to_layout` and `from_iter` design[/+]Moelf commentedon Oct 13, 2023
A possible implementation is to define
and then define
from_iterto something like:and the user-facing interface/contract for custom type is: if you have a custom type, you need to specialize two functions:
AwkwardArray.layout_for(::Type{MyType})which must return a type that is (eventually) recognized by AwkwardAwkwardArray.akconvert(x::MyType)which must convert to the same type as returned by 1jpivarski commentedon Oct 13, 2023
I'm open to any such changes, even if they change the interface. Nothing is firmly established yet.