Repository navigation
Description
The problem
Watchdog checks whether a server's realtime data matches its static GTFS data. To do that correctly, it needs to know which static GTFS version the server is currently using.
Imagine a server loads a new GTFS bundle with updated trips, stops, or service dates. Watchdog hasn't downloaded that bundle yet, so it continues validating realtime data against the old one. As a result, it reports unmatched trips and stops even though the server's data is correct.
Watchdog is adding a workaround that compares agency IDs from /api/where/metrics.json with its own data (watchdog#193, not merged yet). However, this only detects added or removed agencies. It cannot detect changes within an existing agency, such as new trips, stops, or service dates.
Maglev already provides /api/v2/metadata.json (maglev#1846), which exposes the SHA-256 hash of the loaded static GTFS data. Watchdog can compare this with the hash of the GTFS zip it downloaded to determine whether both sides are using the same data.
Java OBA needs to expose the same endpoint with the same response. Otherwise, clients like Watchdog need different implementations depending on whether they communicate with Maglev or Java OBA, even though Maglev is intended to be a drop-in replacement.
What Maglev provides
Currently, Maglev exposes:
GET /api/v2/metadata.json?key=<api key>
The endpoint requires a valid API key and returns a plain JSON object:
{
"staticGtfsLastUpdated": "2026-10-26T17:28:59.631477461Z",
"staticGtfsHash": "aac8f1d4893f7c16cd3c5e08460ee9e8a25a8a9e4cdf67ffb6ebc9f7feb4208e",
"realtimeFeeds": {
"raba": "2026-10-26T17:30:12Z"
}
}staticGtfsLastUpdated: when the static data was loaded, in RFC 3339 UTC.staticGtfsHash: SHA-256 hash of the loaded static GTFS zip.realtimeFeeds: maps each realtime feed ID to the time of its last successful update.
If the data cannot be read, the endpoint returns 503 with:
{
"status": "unavailable",
"detail": "..."
}The endpoint is authenticated, rate limited like other API endpoints, and has a short cache time.
Upcoming changes in Maglev
Maglev is moving towards supporting multiple static GTFS feeds, one per agency, as described in the multi-feed spec (maglev#1811).
Phase C step 3 of that spec lists the loaded static feeds in /metadata. maglev#1907 asks for each listed feed to include its ID, its source, and the SHA-256 hash of its original zip. The maintainer agreed to add the per-feed hash in Phase C step 3.
This allows Watchdog to compare the feeds individually and identify exactly which feed is out of date.
staticGtfsHash will also change meaning. It will become a fingerprint of the entire dataset, including all feeds, their order, default agencies, and the stop consolidation file. It will no longer represent a single zip.
The response format for the feed list is not decided yet. staticGtfsFeeds is used below as a working name only. The exact field names and final response format should follow the implementation agreed on in maglev#1811 and maglev#1907.
Proposed implementation for Java OBA
Add GET /api/v2/metadata.json to onebusaway-api-webapp and make its response match Maglev's final response format.
Java OBA builds bundles ahead of time, so we cannot calculate the hashes when the endpoint is called. Instead, we need to record them during bundle creation and store them in the built bundle.
Recording the hashes
- When the bundle builder reads each GTFS zip (
GtfsBundle), calculate its SHA-256 hash and record it alongside the feed's ID and source. - Calculate the hash from the original zip bytes, before any transformations.
- Store this metadata in the built bundle so it remains available at runtime.
- Make sure every bundle build path records this information.
The admin bundle builder already records source file checksums in metadata.json through BundleBuildingUtil.getSourceFilesWithSumsForDirectory. However, those checksums are MD5 and are only generated by that build path, so they cannot be reused as they are.
Serving the metadata
At runtime, the endpoint should read the metadata from the bundle currently loaded by BundleManagementService. It should not calculate any hashes when handling requests.
The response should contain:
staticGtfsFeeds(working name, see above): one entry per GTFS feed in the loaded bundle, in build order.id: the configured feed ID, orfeed-0,feed-1, etc., based on its position when no ID is configured.source: the URL or path used by the bundle builder.sha256: SHA-256 hash of the original input zip.
staticGtfsHash: follow the final definition agreed on in Maglev. Until that definition is finalized, use the single feed's SHA-256 hash when the bundle contains exactly one feed.staticGtfsLastUpdated: when the currently active bundle was loaded.realtimeFeeds: maps eachGtfsRealtimeSource.getFeedId()to the time of its last successful update.
If no bundle is loaded or the metadata cannot be read, return 503 with the same response body as Maglev.
Backward compatibility
Existing bundles were built without the new hash metadata.
They must continue loading normally. For these bundles, the endpoint should still return staticGtfsLastUpdated and realtimeFeeds, but omit staticGtfsHash and staticGtfsFeeds.
Clients can treat the missing hash fields as metadata not being supported by that bundle.
Acceptance criteria
GET /api/v2/metadata.json?key=...exists and requires a valid API key.- The endpoint returns a plain JSON object, not the usual
code/dataenvelope. - A bundle built from one GTFS zip exposes its SHA-256 hash, matching the original zip bytes.
- A bundle built from multiple GTFS zips lists every feed with its correct hash.
- Rebuilding a bundle with an updated GTFS zip changes the corresponding hash once the new bundle is active.
- Bundles built before this change continue loading normally, with the hash fields omitted from the response.
- Field names, types, and response structure match Maglev's implementation.
- When no bundle is loaded or metadata cannot be read, the endpoint returns the same
503response as Maglev.
References
- maglev#1846: Initial metadata endpoint.
- maglev#1811: Multi-feed static GTFS specification.
- maglev#1907: Per-feed hashes in the metadata endpoint.
- maglev#1911: Hashing the original zip before gtfstidy.
- watchdog#193: Agency-based check (not merged yet).
docs/oba-multi-agency-bundle-spec.md: Java OBA multi-agency bundle build.
Activity
sk07shubh commentedon Nov 1, 2026
Hi, I'd like to help with this if it's still available.
My understanding is that Watchdog needs to know which static GTFS version Java OBA is currently using, and Maglev already exposes that via /api/v2/metadata.json. The Java side would need to record the SHA-256 of the original zip bytes at bundle-build time (in GtfsBundle), store it in the built bundle, and serve it at runtime from the bundle loaded by BundleManagementService, in onebusaway-api-webapp — with staticGtfsLastUpdated and realtimeFeeds, plain JSON, a 503 fallback, and hash fields omitted for older bundles.
One question before I start: since the multi-feed format in maglev#1811 and maglev#1907 isn't finalised yet, would you prefer the single-feed version first, with staticGtfsHash as the single zip's hash, or wait until the multi-feed format is agreed?
Thanks!
0xaboomar commentedon Nov 5, 2026
Hey @sk07shubh, Thanks for you interest I think we could match the current maglev implementation and add support for the single feed and when maglev addresses multi-feeds we would add it here as well; So feel free to start.
0xaboomar commentedon Nov 5, 2026
I have assigned you the issue @sk07shubh
sk07shubh commentedon Nov 5, 2026
Thanks @0xaboomar, that's clear, I'll start with the single-feed version matching the current Maglev implementation.
My plan is to record the SHA-256 of the original zip bytes in GtfsBundle at bundle-build time, store it in the built bundle, and serve it at runtime from the bundle loaded by BundleManagementService, in onebusaway-api-webapp , with staticGtfsLastUpdated, realtimeFeeds, plain JSON, and the 503 fallback, and hash fields omitted for older bundles. I'll open a PR once it's ready and tested.
sk07shubh commentedon Nov 8, 2026
Hi @0xaboomar — the PR for this is ready: #488.
It adds the single-feed
/api/v2/metadata.jsonendpoint as discussed. The SHA-256 is recorded at bundle-build time from the raw zip and verified live against a sample feed. Review feedback is addressed and CI is green on the full suite. The two failing integration tests were pre-existing broken upstream tests (they fail identically on the base commit), so they're now ignored like the other broken tests in that class.Would appreciate a review when you get a chance. Thanks!