Repository navigation
graph/db: v2 model and store support - #11306
Open
ViktorT-11 wants to merge 19 commits into
Open
ViktorT-11 wants to merge 19 commits into
ViktorT-11 wants to merge 19 commits into
Conversation
When FetchChannelEdgesByID hits a zombie edge, it constructs the partial ChannelEdgeInfo it returns alongside ErrZombieEdge with a hard-coded ChannelID of zero instead of the actual channel ID that was looked up. Callers such as the gossiper's processZombieUpdate receive this struct and may use the ChannelID field; returning zero is incorrect and could mask bugs in downstream code. Pass the looked-up chanID through to NewV1Channel / NewV2Channel so the zombie ChannelEdgeInfo carries the correct ChannelID.
Add two precomputed mapping tables that track the "best" gossip version for each unique node (pub_key) and channel (SCID): - graph_preferred_nodes: pub_key -> node_id - graph_preferred_channels: scid -> channel_id Priority for nodes: v2 announced > v1 announced > v2 shell > v1 shell. Priority for channels: v2 with policies > v1 with policies > v2 > v1. These tables enable simple indexed-join queries for cross-version traversal (ForEachNode, ForEachChannel, ForEachNodeDirectedChannel) without expensive per-row COALESCE subqueries. The tables are populated from existing data during the migration and maintained by upsert/delete queries on every write path (added in the next commit).
Add version-agnostic channel fetch helpers that choose the preferred gossip-version row for a logical channel: the highest version with policy data, falling back to the highest bare version. Implement the SQL path inside a single read transaction instead of calling SQLStore methods recursively, preserve zombie edge info for SCID lookups, and keep KV behavior as v1-only delegation. Callers that need to know whether any version of a channel exists can rely on the ErrEdgeNotFound returned by the Preferred fetch path. Add coverage for preferred selection, missing channels, missing outpoints, and zombie edge info.
Add UpsertPreferredNode and UpsertPreferredChannel calls to every Store write path so the preferred mapping tables stay consistent: - upsertSourceNode, upsertNode, maybeCreateShellNode, DeleteNode - insertChannel, updateChanEdgePolicy, DeleteChannelEdges - pruneGraphNodes CASCADE deletes on the underlying graph_nodes and graph_channels tables automatically clean up preferred entries when a version is removed, so PruneGraph and DisconnectBlockAtHeight (which delete every version of a SCID) need no explicit upsert call.
Drop the GossipVersion parameter from ForEachNode and ForEachChannel. Both methods now iterate across all gossip versions, yielding one result per unique pub_key or SCID using the preferred mapping tables for pagination. Wire ChannelGraph.FetchChannelEdgesByID and FetchChannelEdgesByOutpoint through the Preferred variants so the public ChannelGraph API loses its GossipVersion parameter and becomes version-agnostic on read.
Make FetchNodeFeatures version-agnostic by trying v2 first and falling back to v1 when v2 features are empty. The no-cache fallback for ForEachNodeDirectedChannel stays version-scoped (v1 only) because the SQL backend always runs with the in-memory cache enabled, and the cache already merges v1+v2 with v2 precedence. populateCache now skips empty v2 feature entries so they don't shadow a non-empty v1 feature set when the cache is rebuilt, matching the FetchNodeFeatures precedence rule. The cache iterates v1 then v2 so v2 data overwrites v1 on key collision.
VersionedGraph wraps ChannelGraph for callers that want to operate against a specific gossip version. After the cross-version refactor, the ForEachNode and ForEachChannel methods on VersionedGraph silently ignored c.v and iterated across all versions — a surprising behaviour for a wrapper whose whole purpose is version-scoping. Move the cross-version iteration to ChannelGraph (where it belongs) and drop the foot-gun overrides from VersionedGraph. *VersionedGraph continues to expose these via the embedded *ChannelGraph, but now they are explicitly methods of the version-agnostic type. Add ChannelGraph.ForEachNode mirroring ChannelGraph.ForEachChannel, and update the only non-embedded callsite (rpcserver describeGraph) to take ChannelGraph directly. The two remaining test helpers also switch to *ChannelGraph since they only ever wanted a node count.
SQLStore.ForEachNodeCached takes a gossip version and uses it correctly when paginating through nodes, but the inner ListChannelsForNodeIDs call hardcoded GossipVersion1 instead of forwarding the requested version. Calling ForEachNodeCached(ctx, GossipVersion2, ...) therefore returned v2 nodes paired with v1 channels — silently inconsistent data. The bug was latent because every existing caller happens to request v1, but it would surface as soon as any v2-scoped caller appears. Add a regression test that creates a v2-only channel between two nodes that exist under both versions and asserts that ForEachNodeCached(ctx, GossipVersion2, ...) reports the channel for each endpoint.
VersionedGraph.GraphSession duplicated ChannelGraph.GraphSession with no observable difference: in the cache-loaded branch, both pass themselves to the callback, and the cache's NodeTraverser surface (GetFeatures, ForEachChannel) has no version concept — so receiving a *VersionedGraph vs a *ChannelGraph routes to the same cache lookups. In the no-cache branch, both delegate to c.db.GraphSession identically. Remove the override and fold its v1-only note into ChannelGraph's docstring so the production invariant (no-cache path is KV-only, which is v1-only) is preserved at the surviving callsite. This mirrors the hygiene from the earlier ForEachNode/ForEachChannel cleanup, where VersionedGraph overrides that ignored c.v were moved off the wrapper.
sqlNodeTraverser.ForEachNodeDirectedChannel and FetchNodeFeatures both pass lnwire.GossipVersion1 to their helpers without explanation. The reason this is correct is non-obvious: sqlNodeTraverser is only ever constructed by SQLStore.GraphSession, which is only reached when the in-memory graph cache is unavailable. Since the SQL backend always runs with the cache enabled in production, this fallback never executes at runtime — the cache-backed NodeTraverser (which already merges v1+v2) is what real callers see. Only tests exercise this code, and they operate on v1 data. Capture that rationale on the type doc so a future reader doesn't mistake the hardcoding for a bug. Also switch the call sites from lnwire.GossipVersion1 to the file-local gossipV1 alias to match the convention used elsewhere in this file.
Use Go 1.22 integer ranges to satisfy the intrange linter.
Update NodeFromWireAnnouncement to accept the lnwire.NodeAnnouncement interface instead of only *NodeAnnouncement1, and return an error for unsupported types. Add a NodeAnnouncement() method that returns the version-appropriate wire message from a Node model. Also add NodeTimestamp() which returns the version-appropriate ordering timestamp (unix time for v1, block height for v2).
Add ChannelAuthProofFromWireAnnouncement and ChannelAuthProofFromAnnounceSignatures for constructing auth proofs from wire messages. Add ChannelEdgeInfoFromWireAnnouncement for building a ChannelEdgeInfo from a ChannelAnnouncement interface. Also add ChannelEdgePolicyFromWireUpdate for constructing a ChannelEdgePolicy from a ChannelUpdate interface, and update the KV and SQL stores to use the new ChannelEdgeInfo.ChanProofBytes() accessor.
Add ToNodeAnnouncement which serializes a Node model back to its version-appropriate lnwire.NodeAnnouncement wire message. This enables round-tripping between the internal model and wire format. Also add V2NodeAddrs/SetV2NodeAddrs helpers for extracting and setting the typed address fields on v2 node announcements, and comprehensive tests for node wire round-tripping.
Add MarkEdgeZombie and MarkEdgeLive convenience methods to VersionedGraph that supply the gossip version from the embedded field, matching the pattern used by other VersionedGraph wrappers.
Add SQL queries for version-aware node traversal used by the sqlNodeTraverser: ListChannelsForNodeV2, GetV2NodesByPubKeys, ListChannelsWithPoliciesForCachePaginatedV2, and ListChannelsPaginatedV2. Update the KV store's ForEachNodeDirectedChannel to return ErrVersionNotSupportedForKVDB for non-v1 instead of silently returning no results.
Filter out non-v3 onion addresses when setting v2 node addresses, since v2 gossip only supports v3 onion addresses. Also update the SQL store to handle v2 node address filtering during persistence and retrieval.
github-actions
Bot
added
the
severity-critical
🔴 PR Severity: CRITICAL
🔴 Critical (9 files)
🟠 High (12 files)
🟢 Low (5 files)
AnalysisThis PR adds a new SQL migration ( To override, add a |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #10657, and updates the PR to be aligned with the changes added with #11175 &
master.Summary
Complete the graph database's v2 gossip support by adding wire conversion
helpers, versioned graph wrappers, SQL queries for v2 node traversal, and
v3-only onion address filtering.
Depends on: #11175 (graph/db: cross-version graph Store)
Part of the Gossip 1.75 epic.
Key changes
Node model generalization:
NodeFromWireAnnouncementnow accepts thelnwire.NodeAnnouncementinterface (v1 and v2). AddNodeAnnouncement()for version-appropriate wire round-tripping,
NodeTimestamp()forversion-aware ordering, and
ToNodeAnnouncement()for full serializationback to wire format including v2 address helpers.
Channel model helpers:
ChannelAuthProofFromWireAnnouncement,ChannelAuthProofFromAnnounceSignatures,ChannelEdgeInfoFromWireAnnouncement, andChannelEdgePolicyFromWireUpdatefor constructing internal models fromversioned wire messages.
VersionedGraph zombie wrappers:
MarkEdgeZombieandMarkEdgeLiveconvenience methods that supply the gossip version from the embedded field.
SQL v2 node traversal: Add
ListChannelsForNodeV2,GetV2NodesByPubKeys,ListChannelsWithPoliciesForCachePaginatedV2, andListChannelsPaginatedV2queries for version-aware node traversal.V3-only onion addresses: Filter out non-v3 onion addresses for v2
gossip, since the v2 protocol only supports v3 onion services.