Skip to content

fix(gdrive): paginate folder listing in _download_folder - #9637

Open
pcbeingused333 wants to merge 1 commit into
OpenMined:devfrom
pcbeingused333:fix-gdrive-pagination
Open

pcbeingused333 wants to merge 1 commit into
OpenMined:devfrom
pcbeingused333:fix-gdrive-pagination

Conversation

@pcbeingused333

Copy link
Copy Markdown

Summary

_download_folder in syft/gdrive_utils.py called files().list once, requested only files(id,name,mimeType) (no nextPageToken) and never followed pagination. For folders with more results than one page, the remaining files were silently skipped while the call reported success.

Changes

  • Request nextPageToken in fields and loop, passing pageToken, until no token is returned (same pattern as DriveCachedClient.get_file_metadatas_from_folder).

Testing

  • New tests/unit/test_gdrive_utils_pagination.py: a two-page response (empty first page with nextPageToken, file on the second) must download the second-page file and resend the token. Fails before the fix (1 list call), passes after.
  • tests/unit/test_gdrive_pagination.py still passes (12 passed in total).

Asana task

Not applicable.

AI assistance was used to prepare this change and its tests.

Local validation

Base: dev 425d615 (default branch is dev, not main).
No issue exists for this finding, so there is no Fixes reference.

files().list returns at most one page; _download_folder ignored nextPageToken, so folders with more results silently lost files. Follow nextPageToken until exhausted.

Signed-off-by: Alex Castillo <alex.castillog33@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant