Skip to content

Add a stop-consolidation-url config option - #1773

Draft
aaronbrethorst wants to merge 2 commits into
ab/stop-consolidation-loadfrom
ab/stop-consolidation-import
Draft

aaronbrethorst wants to merge 2 commits into
ab/stop-consolidation-loadfrom
ab/stop-consolidation-import

Conversation

@aaronbrethorst

@aaronbrethorst aaronbrethorst commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Part of #1977

Draft. Stacked on #1776. Draft until multi-feed phases B and C land.

This PR began as the whole import draft. It's now one layer of a stack:

PR Change Base
#1771 Parser main
#1772 Validator #1771
#1774 Apply #1772
#1775 Remove consolidated stops while importing #1677 plus cherry-picks of the three above
#1776 Apply the file on static import #1775
#1773 Config option, plus the plan doc #1776
#1777 Remove trips consolidation leaves too short #1773

What it does

  • gtfs-static-feed.stop-consolidation-url names a Java OneBusAway stop consolidation file, as an http(s) URL or a local path. It's the counterpart of STOP_CONSOLIDATION_URL in OneBusAway's Docker bundle builder.
  • It's validated like gtfs-static-feed.url, which now shares the check: no file:// URLs and no path traversal.
  • -dump-config shows it when set.
  • Also adds docs/stop-consolidation-plan.md, which covers:
    • the settled behavior, and the production Puget Sound evidence behind it
    • what's done
    • the remaining PRs, in order

Root cause

Exposes #1776's Config.StopConsolidationURL to operators.

Testing

  • Test failing on main before this change: n/a (new option).
  • TestToGtfsConfigData_StopConsolidationURL, TestValidate_StopConsolidationURL and TestStopConsolidationURLReachesGTFSConfigAndDump.
  • Manual check: maglev -f config.json -dump-config shows stop-consolidation-url, and the server applies the file (results in Apply a stop consolidation file on static import #1776).
  • make test and go vet (both tag sets) pass.

Checklist

  • This PR addresses one issue. (no issue filed yet)
  • It is under ~200 changed lines, or the description says why it can't be split.
  • This PR is not a bug fix, or the new or changed test fails on main and passes on this branch.
  • Each commit is one logical change with a clear message.
  • I did not hand-edit sqlc-generated files under gtfsdb/ or testdata/openapi.yml.
  • This PR changes no handler.
  • This PR doesn't change a Java-served response.
  • This PR adds no endpoint.
  • I read every line of this diff and can explain it if asked.

@coderabbitai

coderabbitai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

The description follows the PR template.

@github-actions

Copy link
Copy Markdown

Performance Smoke Test Results

Status: PASSED

Metric Value
p(95) latency 2.1 ms
Error rate 0.00%
Total requests 344
Req/sec 11.3

Smoke test config: 5 VUs x 30s. Thresholds: p(95) < 300ms, error rate < 1%.

Full results uploaded as workflow artifact: k6-smoke-summary.

@aaronbrethorst aaronbrethorst changed the title Apply stop consolidation files on static import (draft) Add a stop-consolidation-url config option Oct 24, 2026
@aaronbrethorst
aaronbrethorst changed the base branch from ab/1659-connection-pragmas to ab/stop-consolidation-load October 24, 2026 06:46
@github-actions github-actions Bot added the follows-template The PR description follows the PR template label Nov 8, 2026
@aaronbrethorst
aaronbrethorst force-pushed the ab/stop-consolidation-import branch from 0571db8 to 7faee60 Compare November 8, 2026 17:58
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

Performance Smoke Test Results

Status: PASSED

Metric Value
p(95) latency 1.9 ms
Error rate 0.00%
Total requests 336
Req/sec 11.0

Smoke test config: 5 VUs x 30s. Thresholds: p(95) < 300ms, error rate < 1%.

Full results uploaded as workflow artifact: k6-smoke-summary.

@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

Performance Smoke Test Results

Status: PASSED

Metric Value
p(95) latency 2.1 ms
Error rate 0.00%
Total requests 324
Req/sec 10.6

Smoke test config: 5 VUs x 30s. Thresholds: p(95) < 300ms, error rate < 1%.

Full results uploaded as workflow artifact: k6-smoke-summary.

@burma-shave burma-shave left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 7faee60. The config plumbing is fine, and pulling the checks into validateFeedLocation is a real improvement: the old code returned early for an HTTP static URL, so without the refactor the new field would have skipped path-traversal validation. One scope comment inline about the plan doc.

Minor: there's no -stop-consolidation-url CLI flag to go with -gtfs-url. Fine if JSON-only is intended; worth a line in the description if so.


Generated by Claude Code

Comment thread docs/stop-consolidation-plan.md Outdated
@@ -0,0 +1,127 @@
# Stop consolidation: implementation plan

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This plan doc is a separate concern from the config option and will go stale as the stack lands (#1777 and #1808 already edit it). The behaviour table is worth keeping, but I'd move it to the tracking issue #1977 or the wiki's Implementation Decisions (which the doc itself says should record these), and drop this commit from the PR.


Generated by Claude Code

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The behavior table, production evidence, token-agency rule and open items now live in §9.4 of the multi-feed spec, and stop.md has an Implementation Decisions note for the absorbed-ID rule. I dropped the plan-doc commit here, and its follow-up edit in #1777, from the stack.

gtfs-static-feed.stop-consolidation-url names a Java OneBusAway stop
consolidation file, as an http(s) URL or a local path, and passes it to
the GTFS manager. It is the counterpart of the STOP_CONSOLIDATION_URL
setting in OneBusAway's Docker bundle builder.

It is validated like gtfs-static-feed.url, which now shares the check:
no file:// URLs and no path traversal. -dump-config shows it when set.
gtfs-static-feed.stop-consolidation-url could only be set from a JSON
config file. Add the matching flag beside -gtfs-url so a flag-only run
can load a consolidation file too. It goes through the same validation
as the JSON field, so a file:// URL is still rejected.
@aaronbrethorst
aaronbrethorst force-pushed the ab/stop-consolidation-import branch from 7faee60 to 538aa41 Compare November 9, 2026 19:04
@aaronbrethorst

Copy link
Copy Markdown
Member Author

Also added the -stop-consolidation-url flag next to -gtfs-url in 538aa41. It goes through the same validation as the JSON key, so file:// is still rejected.

@sonarqubecloud

sonarqubecloud Bot commented Nov 9, 2026

Copy link
Copy Markdown

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Performance Smoke Test Results

Status: PASSED

Metric Value
p(95) latency 2.1 ms
Error rate 0.00%
Total requests 340
Req/sec 11.2

Smoke test config: 5 VUs x 30s. Thresholds: p(95) < 300ms, error rate < 1%.

Full results uploaded as workflow artifact: k6-smoke-summary.

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Performance Smoke Test Results

Status: PASSED

Metric Value
p(95) latency 1.4 ms
Error rate 0.00%
Total requests 331
Req/sec 10.9

Smoke test config: 5 VUs x 30s. Thresholds: p(95) < 300ms, error rate < 1%.

Full results uploaded as workflow artifact: k6-smoke-summary.

@aaronbrethorst
aaronbrethorst removed this pull request from stack #1985 November 9, 2026 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

follows-template The PR description follows the PR template

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants