[epic]: BOLT 12 Implementation Roadmap

[bitromortac]

BOLT 12 is an alternative to BOLT 11 invoices with reusable offers, a two-step negotiation protocol where a payer decodes an offer, requests an invoice via onion messages, and pays the returned invoice through blinded payment paths. This issue tracks the progress towards that functionality. Original issue: #5594. I'll edit this issue to track more fine grained steps (the final architecture is still being worked on) and to link to corresponding PRs.

Milestones

• (A) Codec: bolt12 package for messages and data structures (PoC)
  • bolt12+lnwire: add codec foundation with Offer message #10789
  • bolt12: add InvoiceRequest codec and structural validators #10832
  • bolt12: add Invoice codec and structural validators #10941
  • bolt12: add invoice_error message for onion message replies #10958
  • bolt12: add Bech32 string codec and spec test vectors #11001
  • bolt12: add Merkle tree and BIP-340 message signatures #11061
  • bolt12: add string-codec wrappers and fuzz harnesses #11146
  • bolt12: split off review cleanups and test changes from #11191 #11237
  • bolt12: finalize the codec package #11191
• (B) Receiver side: offer store, invoice request handler
  • offers: add a SQL-backed store for BOLT 12 offers #11278
• (C) Sender side: offer decoding, invoice request construction, payment dispatch
• (D) Multi-hop: pathfinding, blinded path builder, handler upgrade
• (E) PayOffer: safe payment dispatch
• (F) RPC parity: BOLT 12 fields on existing invoice/payment RPCs

Design decisions worth knowing

• Stateless invoice settlement for DoS mitigation: the receiver does not store invoices at creation time. A signed envelope in the blinded path carries enough data to reconstruct the invoice when the HTLC arrives, avoiding per-request DB writes.
• Invoice RPC backward compatibility: the change is planned in such a way that Bolt12 invoices integrate into existing invoice RPCs, with additional Bolt12 specific fields. Hold invoices will not be supported initially.

[ziggie1984]

Related spec work: lightning/bolts#1341 tightens the allowed Unicode characters in UTF-8 fields and requires minimal feature-vector encoding. Its BOLT 12 changes affect offer_description, offer_issuer, invreq_payer_note, offer_features, invreq_features, and invoice_features, so the codec/validator work should account for it:

lightning/bolts#1341

[paulscode]

Hi, following this epic rather than interrupting the codec PRs.

Some context in case it is useful, and then three findings and no ask.

I maintain a fork of lnd that follows a different chain, and it carries an
implementation of roughly milestones B through E: an offer store, an invoice
request handler, the sender side, and blinded path construction over onion
messages. It is shipped in that fork and exercised against Core Lightning in a
regtest harness, where the two peer, open channels from either side, and pay
each other's offers. The BOLT 12 parts touch no chain hashing, so none of what
follows is specific to the chain we are on.

I am not proposing it for these milestones, and I want to be clear about that
up front rather than have you work it out. Two of your design decisions are
ones I did not make:

• I create an invoice per answered request. Your epic says the receiver does
  not store invoices at creation time and reconstructs them from a signed
  envelope in the blinded path. That is a better answer than mine and it is
  not a change I could retrofit.
• My RPC is a separate subserver behind a build tag, where your milestone F
  puts BOLT 12 fields on the existing invoice and payment RPCs.

So the code is the wrong shape for this roadmap. What I think is worth passing
on is what it cost to find out.

1. A rate limiter that refuses in silence is worse than no rate limiter.
Because I create an invoice per request, I needed a per-peer limit. I set one,
and it dropped over-limit requests silently. About one fetch in six then failed
in my own lab, and I chased it through three wrong theories before instrumenting
the limiter itself.

The limiting was right. The silence was not. A requester that gets nothing back
cannot tell rate limiting from the issuer being offline, the message being lost,
or its own reply path being broken, so it waits out its entire timeout for
something that was decided in microseconds. BOLT 12 has invoice_error for
exactly this. Answering the first over-limit request with one, and staying quiet
for the rest of a short window, turns a sixty second silent timeout into a
126ms refusal with a reason, without making the limiter an amplifier for the
traffic it exists to shed.

This is mostly an argument for your stateless design, since it removes the
reason for the limiter. It still applies wherever the receiver refuses for any
other reason: unknown offer, expired offer, amount out of range. A silent drop
there is indistinguishable from a broken path, and the payer pays for the
difference in latency.

2. Two blinded path cases make an invoice unissuable, and both are ordinary.

• The introduction node is the payer itself. A path that starts at the paying
  node needs the router to treat the first hop as already reached, or
  pathfinding fails on a route that is trivially valid.
• No peer can serve as an introduction node, because every peer's only channel
  is the one to this node, or because none of them signals route blinding. A
  small or new node hits this immediately. Falling back to a path that starts
  at the issuer always works and is worth reporting rather than silently
  producing a worse path.

3. Gating onion messages on having a channel makes offers unusable for the
case they exist to serve. A payer fetching an offer usually has no channel
with the issuer. That is the normal case, not the exception. I made the gate
configurable and defaulted it off, with global and per-peer rate limits to bound
what a peer without a channel can send, which seemed the smaller risk.

If any of that is useful I am happy to write it up properly, as issue comments
or as text you can lift. And when B and C land I would be glad to run them
against the same Core Lightning harness and report what breaks, which is
probably the most useful thing I can offer given the code itself is not a fit.

[tammyhirtle-ui]

Duplicate of #11191