Repository navigation
Move the Python client to the result format API and defer row-returning statements - #650
Closed
evertlammerts wants to merge 7 commits into
Closed
evertlammerts wants to merge 7 commits into
evertlammerts wants to merge 7 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on duckdb/duckdb#25801 and duckdb/duckdb#26314.
With this PR the client submits statements through
QueryParameters::format. Arrow consumers are backed by aQueryResultStream<ArrowFormat>, so conversion runs in parallel on worker threads and arrays leave the engine as Arrow. The batch size is a maximum (arrays also end at row-group and producer boundaries); the default is 131,072, matching the C API.execute()now describes a statement (BindStatement) instead of running it. A query or DML with RETURNING runs when its result is first read, and the first consumer fixes the shape: rows or Arrow, chosen once. Running another statement withdraws an unfinished result. Statements whose effects a rollback cannot undo (DDL, CALL, PRAGMA, COPY, and so on) still complete insideexecute(). Newcomplete()andabort()make the lifecycle explicit, and a RETURNING write is all-or-nothing with its rows: abandoned unread, it never happened, and it fails an open transaction.rowcountnow reports changed rows, withexecutemanytotalling its sets.Existing bugs are fixed along the way:
execute()ignored the transaction invalidation policy..pl()leaked anArrowSchemaper call.