Skip to content

feat: add City of Beachwood, OH (beachwood_oh_us) - #7276

Open
GreenDavidA wants to merge 1 commit into
mampfes:release/3.0.0from
GreenDavidA:add-source-beachwood-oh-us
Open

feat: add City of Beachwood, OH (beachwood_oh_us)#7276
GreenDavidA wants to merge 1 commit into
mampfes:release/3.0.0from
GreenDavidA:add-source-beachwood-oh-us

Conversation

@GreenDavidA

@GreenDavidA GreenDavidA commented Aug 31, 2026

Copy link
Copy Markdown

Summary

New pipeline source for the City of Beachwood, Ohio, USA — residential rubbish + recycling.

The city publishes a "Rubbish Pick Up Days" PDF mapping each street to one fixed collection day (Mon–Thu); both carts go out that day. No address API, no ICS feed. The source fetches and parses the PDF every run via the shared PdfTextParser; all Beachwood-specific logic (table parsing, street resolution, recurrence, holiday shifting) lives in preprocess functions.

  • Config: two-level cascading dropdown (street → section) from the live PDF — no maintained street list. RAISE_ON_EMPTY. A split street raises SourceArgumentRequiredWithSuggestions on the section field so the flow points at the right dropdown.
  • Holiday rules (per the city's schedule page): OPM-observed US-federal calendar for Ohio, minus Columbus Day and Veterans Day; a collection on/after a holiday in that Mon–Sun week is delayed a day, except Thanksgiving, which moves only the Thursday collection to the Wednesday before.
  • PDF quirk: one Cedar Road section is printed without its opening parenthesis (city won't fix) — _PDF_FIXUPS repairs it on read; any new malformed entry is logged.

Shared change

recurrence.us_federal_holidays now returns {date: name} instead of set[date] so the Thanksgiving rule can name the holiday. date in holidays still works (dict keys). red_bank_tn_us (the only other caller) wraps the call in set() to keep its behaviour.

Branch target

Targets release/3.0.0 because it's a BaseSource pipeline source and master has none of the pipeline infrastructure. Happy to close this and re-open against master once 3.0.0 merges, if you want. Just let me know, happy to work with maintainers on what works for you folks.

Closes #7275

Type of change

  • New source
  • Bug fix / source fix
  • Migration of an existing source to the 3.0.0 pipeline
  • Documentation update
  • Other

Checklist

  • python -m pytest tests/test_source_components.py -q passes
  • ruff check --fix and ruff format run on changed source files
  • No generated files in diff (README.md, info.md, sources.json, translations/*.json, CI regenerates these post-merge)
  • doc/source/<name>.md created for new legacy sources — N/A, pipeline source (auto-generated post-merge)
  • TEST_CASES use real, publicly accessible addresses (not my own)

New or migrated pipeline sources

  • Cassette recorded. python tests/record_fixtures.py beachwood_oh_us — one per TEST_CASES entry (8 streets) plus the cascading-choices cassette, committed under tests/fixtures/beachwood_oh_us/.
  • Composes shared components. parse = PdfTextParser(...); no Retriever/Parser subclass, no retrieve/parse/preprocess/transform override, no module-level function issuing the provider's HTTP. All Beachwood logic is in plain preprocess functions.
  • Platform additions. The only shared change is recurrence.us_federal_holidays returning names (above) — it belongs in the shared calendar so any US source can do holiday-name-specific rules, not in this source.

Test output

Testing source beachwood_oh_us ...
  found 36 entries for Beacon Drive
  found 36 entries for East Silsby Road
  found 36 entries for Fairmount Boulevard (East bound from Sulgrave, and West bound to Richmond)
  found 36 entries for Fairmount Boulevard (West bound from 24471)
  found 36 entries for Cedar Road (East bound from Fenway to Community)
  found 36 entries for Fernwood Road
  found 36 entries for Halworth Road
  found 36 entries for Woodside Lane

Full pytest -m "not live" (7733 passed, 8 skipped) and pre-commit run --all-files pass on both the CI minimum lane (Python 3.12 / HA 2024.4.0) and the current lane.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

_observed_holidays() is cached without a year-based key, so a long-running process can apply incorrect holiday shifting after a year boundary.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a new BaseSource pipeline provider for the City of Beachwood, OH (USA), sourcing street→weekday pickup mapping from the city’s published PDF and applying Beachwood-specific holiday shifting rules. (PR number wasn’t included in the metadata—share it if you want this review tied back to the GitHub PR view.)

Changes:

  • Introduces beachwood_oh_us pipeline source with cascading street/section selection, PDF parsing, recurrence expansion, and holiday adjustments.
  • Updates recurrence.us_federal_holidays to return {date: name} (dict) to support holiday-name-specific rules; adapts red_bank_tn_us.
  • Adds offline unit tests plus recorded fixtures and ensures the new test file is included in CI via pytest.ini.
File summaries
File Description
custom_components/waste_collection_schedule/waste_collection_schedule/source/beachwood_oh_us.py New Beachwood pipeline source: PDF parsing, cascading dropdown choices, recurrence + holiday shifting.
custom_components/waste_collection_schedule/waste_collection_schedule/recurrence.py Changes us_federal_holidays return type to a date→name dict; updates docs accordingly.
custom_components/waste_collection_schedule/waste_collection_schedule/source/red_bank_tn_us.py Wraps us_federal_holidays with set() to preserve prior “set-of-dates” behavior.
tests/test_beachwood_oh_us.py New offline unit tests pinning PDF parsing behavior and holiday shifting rules.
pytest.ini Adds test_beachwood_oh_us.py to the python_files allowlist so it runs in CI.
tests/fixtures/beachwood_oh_us/beacon_drive.json Recorded fixture for Beachwood PDF fetch.
tests/fixtures/beachwood_oh_us/east_silsby_road.json Recorded fixture for Beachwood PDF fetch.
tests/fixtures/beachwood_oh_us/fairmount_boulevard_west_bound_from_24471.json Recorded fixture for Beachwood PDF fetch.
tests/fixtures/beachwood_oh_us/fernwood_road.json Recorded fixture for Beachwood PDF fetch.
tests/fixtures/beachwood_oh_us/halworth_road.json Recorded fixture for Beachwood PDF fetch.
tests/fixtures/beachwood_oh_us/woodside_lane.json Recorded fixture for Beachwood PDF fetch.
Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Residential rubbish + recycling for the City of Beachwood, Ohio, as a
BaseSource pipeline source.

The city publishes a "Rubbish Pick Up Days" PDF mapping each street to one
fixed collection day (Mon-Thu). The source fetches and parses that PDF every
run via the shared PdfTextParser; the Beachwood-specific logic (table parsing,
street resolution, recurrence, holiday shifting) lives in preprocess functions.

- Config: a two-level cascading dropdown (street, then section) whose options
  come from the live PDF, so there is no maintained street list. The street
  field binds field_terms.STREET for a localised label and help; the source
  sets RAISE_ON_EMPTY. A miss is blamed on the field that is actually wrong:
  base names for an unknown street, section names for a split one, and a
  spurious section on an unsplit street is an error, not silently dropped.
- Holiday rules per the city's schedule page: the OPM-observed US federal
  calendar for Ohio minus Columbus Day and Veterans Day (Public Works stays
  open); a collection on or after a holiday in that Mon-Sun week is delayed a
  day, except Thanksgiving, which moves only the Thursday collection to the
  Wednesday before. The observed-holiday set is cached per year so a
  long-running process keeps working across a New Year.
- The published PDF prints one section entry without its opening parenthesis
  and the city has declined to fix it; _PDF_FIXUPS repairs it on read and any
  new malformed entry is logged.

Depends on a shared change: recurrence.us_federal_holidays now returns
{date: name} so the Thanksgiving rule can name the holiday (`date in holidays`
still works); red_bank_tn_us wraps the call in set() to keep its behaviour.

Tests: tests/test_beachwood_oh_us.py — fully offline unit coverage (parsing,
street resolution, the malformed-entry repair, the split-street prompt, and
every holiday rule) that the cassette gates do not give. Not in the pytest.ini
allowlist, matching the other per-source test files and discussion mampfes#7252, so a
stray failure here cannot break the gating CI run. Plus one cassette per
TEST_CASE and the cascading-choices cassette.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@GreenDavidA
GreenDavidA force-pushed the add-source-beachwood-oh-us branch from 0367244 to d3cea81 Compare September 3, 2026 01:32
@GreenDavidA

GreenDavidA commented Sep 7, 2026

Copy link
Copy Markdown
Author

@bbr111 hope you don't mind me tagging you, as you were the one who kicked off my PR's CI. It looks like there's already a failing CI test for this branch in the upstream release/3.0.0 branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants