Skip to content

Fix missing dataset splits and EXIF dimensions - #147

Merged
glenn-jocher merged 7 commits into
ultralytics:mainfrom
wxs-lang:main
Oct 26, 2026
Merged

glenn-jocher merged 7 commits into
ultralytics:mainfrom
wxs-lang:main

Conversation

@wxs-lang

@wxs-lang wxs-lang commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Dataset splitting can silently omit images when rounded partitions leave a remainder or when a split contains only index zero. The ATH converter also passes a path object to the split writer, which fails when writing a populated split. Images with EXIF orientations 5 and 7 report dimensions in the wrong order.

Use cumulative split boundaries, write nonempty splits, handle path inputs, and account for transposed orientations. Keep focused regression coverage in the existing converter tests.

Validation: the full test suite passed on Python 3.8 and 3.11. Reproduced the split and EXIF failures on the original code and the ATH failure before its path fix.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Updated dataset splitting and EXIF dimension handling to preserve images across splits and correctly size transposed images.

📊 Key Changes

  • Split boundaries now use cumulative ratios, so rounding does not leave images unassigned.
  • Split writers check whether splits contain entries, preserving single-item splits, and split_rows_simple now accepts path inputs.
  • exif_size swaps width and height for EXIF orientations 5–8; regression tests cover splitting, ATH conversion, and these orientations.

🎯 Purpose & Impact

  • Dataset conversion workflows can write populated splits, including single-image ATH datasets, and report corrected dimensions for transposed images.

wxs-lang and others added 2 commits October 22, 2026 22:38
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

All Contributors have signed the CLA. ✅

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-22T14:43:54.440327Z
We couldn’t complete this review. Comment @codex review to try again.
5074ef3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@UltralyticsAssistant

Copy link
Copy Markdown
Member

👋 Hello @wxs-lang, thank you for submitting a ultralytics/JSON2YOLO 🚀 PR! This automated message confirms your contribution was received, and an Ultralytics engineer will assist with the review. To ensure a seamless integration of your work, please review the following checklist:

  • ✅ Define a Purpose: Clearly explain the purpose of your fix or feature in your PR description, and link to any relevant issues. Ensure your commit messages are clear, concise, and adhere to the project's conventions.
  • ✅ Synchronize with Source: Confirm your PR is synchronized with the ultralytics/JSON2YOLO main branch. If it's behind, update it by clicking the 'Update branch' button or by running git pull and git merge main locally.
  • ✅ Ensure CI Checks Pass: Verify all Ultralytics Continuous Integration (CI) checks are passing. If any checks fail, please address the issues.
  • ✅ Update Documentation: Update the relevant documentation for any new or modified features.
  • ✅ Add Tests: If applicable, include or update tests to cover your changes, and confirm that all tests are passing.
  • ✅ Sign the CLA: Please ensure you have signed our Contributor License Agreement if this is your first Ultralytics PR; the CLA check will comment with signing instructions if needed.
  • ✅ Minimize Changes: Limit your changes to the minimum necessary for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

For more guidance, please refer to our Contributing Guide. Don't hesitate to leave a comment if you have any questions. Thank you for contributing to Ultralytics! 🚀

@UltralyticsAssistant UltralyticsAssistant left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 PR Review

Made with ❤️ by Ultralytics Actions

Reviewed the four added test files. The coverage otherwise matches the existing converter behavior, but this test uses Python 3.9-only syntax despite the repository advertising Python 3.8 support.

💬 Posted 1 inline comment
  • 📝 LOW tests/test_general_json2yolo.py:325 On Python 3.8, this dictionary union raises TypeError when the test runs because dict | dict was added in Python 3.9. The README advertises Python 3.8+, so the new test suite fails on a supported version. Replace the union with a Python 3.8-compatible way to set images.

Comment thread tests/test_general_json2yolo.py Resolved
@wxs-lang

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I sign the CLA

@pderrenger

Copy link
Copy Markdown
Member

Thanks for confirming the CLA, @wxs-lang, and for contributing coverage for the converter edge cases and utility helpers.

@glenn-jocher

Copy link
Copy Markdown
Member

Thank you for the contribution. After reviewing the full diff, we are keeping this suite focused: the 961-line expansion tests many implementation details and duplicates existing test setup, and it asserts that a 50/50 split drops an input as well as preserving incorrect dimensions for EXIF orientations 5 and 7. Those assertions would lock in existing bugs rather than protect the desired behavior. We are closing this broad test-only change; a small regression test accompanying a proven fix would be more appropriate.

@glenn-jocher glenn-jocher changed the title Add pytest coverage for utils, COCO/LabelMe edge cases, and the legacy + Labelbox converters Fix dataset splits and EXIF dimensions Oct 26, 2026
@glenn-jocher

Copy link
Copy Markdown
Member

Thank you, @wxs-lang. I reopened this and used your coverage to identify and fix dataset splitting and EXIF dimension bugs, keeping focused regression tests and the Labelbox conversion test. The full suite passes locally; CI is running on the updated branch.

@glenn-jocher glenn-jocher changed the title Fix dataset splits and EXIF dimensions Fix missing dataset splits and EXIF dimensions Oct 26, 2026
@glenn-jocher

Copy link
Copy Markdown
Member

The final scope also fixes the ATH converter’s path-based split writer, found during review. I narrowed the retained tests to the split, ATH, and EXIF regressions so the PR stays focused; the existing Labelbox loader coverage remains.

@glenn-jocher glenn-jocher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the full updated diff. The fixes preserve all split indices, handle the ATH path input, and correct transposed EXIF dimensions. The full suite passes on Python 3.8 and 3.11; awaiting CI.

@glenn-jocher
glenn-jocher merged commit a376145 into ultralytics:main Oct 26, 2026
3 checks passed
@glenn-jocher

Copy link
Copy Markdown
Member

Merged—thank you, @wxs-lang, for the contribution and for surfacing the split and EXIF cases. The final change passed independent review, Python 3.8 and 3.11 validation, and CI.

@UltralyticsAssistant

Copy link
Copy Markdown
Member

🎉 This PR has been merged! Thank you @wxs-lang, with contributions from @glenn-jocher and @pderrenger.

“Alone we can do so little; together we can do so much.” — Helen Keller

Together, these changes help preserve images across dataset splits and correctly report dimensions for transposed images.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants