Repository navigation
fix: Permission Error trying to use pb.load_data_set() with duckdb on Windows #244
Description
Description
When running pb.load_dataset(dataset="small_table", tbl_type="duckdb") on my Windows machine, I get the following error:
File "C:\Users\00113294\Projects\pointblank\pointblank\validate.py", line 562, in load_dataset
with tempfile.TemporaryDirectory() as tmp, ZipFile(data_path, "r") as z:
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 946, in __exit__
self.cleanup()
~~~~~~~~~~~~^^
File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 950, in cleanup
self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 930, in _rmtree
_shutil.rmtree(name, onexc=onexc)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\shutil.py", line 790, in rmtree
return _rmtree_unsafe(path, onexc)
File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\shutil.py", line 629, in _rmtree_unsafe
onexc(os.unlink, fullname, err)
~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\00113294\AppData\Roaming\uv\python\cpython-3.13.2-windows-x86_64-none\Lib\tempfile.py", line 905, in onexc
_os.unlink(path)
~~~~~~~~~~^^^^^^
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\00113294\\AppData\\Local\\Temp\\tmps3x_936b\\small_table.ddb'According to Claude:
This is a Windows-specific issue with DuckDB file locking and temporary file cleanup. The error occurs because:
- File Handle Not Released: When
pb.load_dataset()creates a DuckDB database file in a temporary directory, the database connection isn't being properly closed before Python tries to clean up the temporary directory.- Windows File Locking: Windows is more strict about file locking than Unix systems - it won't allow deletion of files that have open handles, even if the process that opened them is trying to delete them.
Development environment
- Operating System: Windows
- Pointblank Version: 0.11.2
Activity
rich-iannone commentedon Jul 17, 2025
Hmm, this is my bad for not testing this on Windows. I'm glad you did identify this issue though; maybe there should be some Windows-specific disconnect statements applied in key places (thanks for starting the dedicated PR). Also, I guess we ought to expand the CI testing to include Windows tests?
matt-humphrey commentedon Jul 21, 2025
Yeah that sounds like a reasonable alternative. I'll look into adding some logic to handle Windows differently. I think Windows tests for CI would make sense too.
rich-iannone commentedon Oct 21, 2026
Fixed in 9241a4c. The table is now copied into in-memory DuckDB (
validate.py:707-733).