rich1.0npub19helcfnqgk2jrwzjex2aflq6jwfc8zd9uzzkwlgwhve7lykv23mq5zkvn4https://nostr.ae/npub19helcfnqgk2jrwzjex2aflq6jwfc8zd9uzzkwlgwhve7lykv23mq5zkvn4njumphttps://nostr.ae📅 Original date posted:2022-06-07 📝 Original message: Hi y'all, Last week nearly 30 (!) Lightning developers and researchers gathered in Oakland, California for three day to discuss a number of matters related to the current state and evolution of the protocol. This time around, we had much better representation for all the major Lightning Node implementations compared to the last LN Dev Summit (Zurich, Oct 2021). Similar to the prior LN Dev Summit, notes were kept throughout the day that attempted on a best effort basis to capture the relevant discussions, decisions, and new relevant research or follow up areas to circle back on. Last time around, I sent out an email that summarized some key takeaways (from my PoV) of the last multi-day dev summit [1]. What follows in this email is a similar summary/recap of the three day summit. Just like last time: if you attended and felt I missed out on a key point, or inadvertently misrepresented a statement/idea, please feel free to reply, correcting or adding additional detail. The meeting notes in full can be found here: https://docs.google.com/document/d/1KHocBjlvg-XOFH5oG_HwWdvNBIvQgxwAok3ZQ6bnCW0/edit?usp=sharing # Simple Taproot Channels During the last summit, Taproot was a major discussion topic as though the soft fork had been deployed, we we're all still watching the 🟩 's stack up on the road to ultimately activation. Fast forward several months later and Taproot has now been fully activated, with ecosystem starting to progressively deploy more and more advanced systems/applications that take advantage of the new features. One key deployment model that came out of the last LN Dev summit was the concept of an iterative roadmap that progressively revamped the system to use more taprooty features, instead of a "big bang" approach that would attempt to package up as many things as possible into one larger update. At a high level the iterative roadmap proposed that we unroll an existing larger proposal [2] into more bite sized pieces that can be incrementally reviewed, implemented, and ultimately deployed (see my post on the LN Dev Summit 2021 for more details). ## Extension BOLTs Riiight before we started on the first day, I wrote up a minimal proposal that attempted to tackle the first two items of the Taproot iterative deployment schedule (musig2 funding outputs and simple tapscript mapping) [3]. I called the proposal "Simple Taproot Channels" as it set out to do a mechanical mapping of the current commitment and script structure to a more taprooty domain. Rather than edit 4 or 5 different BOLTs with a series of "if this feature bit applies" nested clauses, I instead opted to create a new standalone "extension bolt" that defines _new_ behavior on top of the existing BOLTs, referring to the BOLTs when necessary. The style of the document was inspired by the "proposals" proposal (very meta), which was popularized by cdecker and adopted by t-bast with his documents on Trampoline and Blinded Paths. If the concept catches on, extension BOLTs provide us with a new way to extend the spec: rather than insert everything in-line, we could instead create new standalone documents for larger features. Having a single self contained document makes the proposal easier to review, and also gives the author more room to provide any background knowledge, primaries, and also rationale. Overtime, as the new extensions become widespread (eg: taproot is the default channel type), we can fold in the extensions back to the main set of "core" BOLTs (or make new ones as relevant). Smaller changes to the spec like deprecating an old field or tightening up some language will likely still follow the old approach of mutating the existing BOLTs, but larger overhauls like the planned PTLC update may find the extension BOLTs to be a better tool. ## Tapscript, Musig2, and Lightning As mentioned above the Simple Taproot Channels proposal does two main things: 1. Move the existing 2-of-2 p2wsh segwit v0 funding output to a _single key_ p2tr output, with the single key actually being an aggregated musig2 key. 2. Map all our existing scripts to the tapscript domain, using the internal key (keyspend path) for things like revocations, which an potentially allow nodes to store less state for HTLCs. Of the two components #1 is by far the trickiest. Musig2 is a very elegant protocol (not to mention the spec which y'all should totally check out) but as the signatures aren't deterministic (like RFC 6979 [5]), both signers need to "protect themselves at all times" to ensure they don't ever re-use nonces, which can lead to a private key leak (!!). Rather than try to create some sort of psuedo-deterministic nonces scheme (which maaybe works until the Blockstream Research team squints vaguely in its direction), I opted to just make all nonces 100% ephemeral and tied to the lifetime of a connection. Musig2 defines something called a public nonces, which is actually two individual 33-byte nonces. This value needs to be exchanged before signing can begin (but can be sent before sides know they're aggregated keys). One important thing to note is that given that the channels today have _asymmetric_ state, we actually need a _pair_ of public nonces: one that I'll use to sign my commitment, and one I'll use to sign yours. Lightning channels w/ symmetric state like eltoo can get by w/ only exchange a single set of nonces, as there's only one message per state. Nonce exchange takes place in a few places: * During initial funding: I send my public nonce in the open_channel message, you send yours in the accept_channel message. After this exchange we can both generate signatures for the refund commitment transactions. * After the channel is "ready" we send another set of nonces, so we can sign the next state. This is similar to the existing revocation key exchange: I need your next nonce/key before I can sign a new state. * Upon channel re-establishment a _new_ set of nonces is sent, as they're 100% ephemeral. The current draft also requires that if you were re-transmitting a sig, then you use the _new_ nonces to sign again, as it's possible you went to retransmit but left off an expired/trimmed HLTC (could lead to nonce re-use and also needing to remember nonces). * Each time I revoke my channel, I send to you a single nonce, my "local nonce" (naming needs some work here), which lets you sign for a new state. * Each time I send a new sig, I also send you another nonce, my "remote" nonce", which * When I send a shutdown (co-op close) I send a single public nonce so we can sign the next co-opc close offer. * When I send a closing_signed I send another nonce so once you send your offer, we sign another set. The final flows aren't 100% yet finalized, as we'll need some implementations drafted to make sure the nonce handling and script mapping works out properly. ### Lightning Channels & Recursive Musig2 One other cool topic that came up is the concept of leveraging recursive musig2 (so musig2 within musig2) to make channels even _more_ multi-sigy. The benefit here is that Bob & Carol can each have their individual keys (which might actually be aggregated keys themselves) and make a channel w/ Alice, who only knows of them as Barol, and doesn't know there're actually another pair of keys at play. This is _really_ cool as it allows node operators, wallets, and lightning platforms to experiment with various key/signing trees that may add more security, redundancy, or flexibility. When this first came up, someone brought up the fact that while the scheme is "known" the initial paper as they weren't sure how to actually write a proof for it. During the session, someone emailed one of the musig2 authors asking for more details, and if it's safe to implement and roll out. Thankfully they quickly replied and explained that the proof recursive musig (pls someone correct me again here if I'm wrong) wasn't left out due to impossibility, but that a proof in the existing Random Oracle Model (which was used to derive a bound for the number of nonces needed) would lead to a blow up in the number of nonces required. Attempting to write the proof in some other model would likely lead to better results (proved w/ two nonces as base musig2), but would end up being pretty complicated, so hard to read and even review for correctness. Assuming everything checks out, then a useful mental model explained by the musig2 BIP author is a sort of tree structure. Assuming I'm a signer, and we assemble the other signer as a sibling leaf in a binary tree, then I just need to wait for the sibling nonce/key, before I can aggregate that into the final value. So if there're 3 signers, I wait for the regular public nonce, but the other signers sum their respective nonces into a single nonce, then send that to me. A similar operation is carried out for key aggregation, with the rest of the protocol being mostly the same. Ultimately, even if wallets/nodes aren't ready to roll something like this out today, we at least want to make sure the proposed flow is compatible with Simple Taproot Channels, and ideally we'd have a toy implementation to verify out understanding and show it's possible/sound. I volunteered to hack up a simple recursive musig2 demo, as there doesn't seem to be any code in the wild that implements it. ## Lightning Gossip # Gossip V2: Now Or Later? Another big topic related to Taproot was the question of how we should update the gossip network: the gossip protocol today has all channels validated by node, which requires that the nodes understand how to reconstruct the funding output based on the set of advertised keys. The protocol today assumes a segwit v0 p2wsh multi-sig is used. Assuming we had everything implemented today, a node wouldn't be able to advertise its new taproot channels to the rest of the public graph as they wouldn't understand how to validate it. This presents a new opportunity: we already need to rework gossip for taproot, so should we go ahead and re-design the entire thing with an eye for better privacy and future extensibility? A proposal for the "re-design the entire thing" was floated in the past by Rusty [6]. It does away with the strict coupling of channels to channel announcements, and instead moves them to the _node_ level. Each node would then advertise the set of "outputs" they have control of, which would then be mapped to the total capacity of a node, without requiring that these outputs self identify themselves on-chain as Lightning Channels. This also opens up the door to different, potentially more privacy preserving proofs-of-channel-ownership (something something zkp). On the other hand, we could just follow the path of Simple Taproot Channels and map musig2+schnorr onto the existing gossip network. This is less changes in total, with the main benefit being the ability to only send 1 sig (aggregated musig2 sig of keys) instead of 4 individual sigs. I made a very lofty proposal in this direction here [7]. Ultimately we decided to take the "just musig2 aspects" from gossip v1.5 (not the real name), and the "let's refresh all the messages w/ TLV goodness" from the gossip v2 proposal. This gives us a smaller package to implement, and lets us potentially rejigger the messages to be more extensible and remove cruft like the node color that almost nothing uses, but we all validate/store. The follow up work in this area is a more concrete proposal that updates the relevant gossip messages to be taproot aware and TLV'd and also update the set of requirements w.r.t _how_ to validate the channels in the first place (so given two keys verify that applying the keyagg method of musig2 lead to what' in the funding output). Gossip v2 will likely happen "eventually", but the rather large design space needs to be explored a bit more so we can properly analyze exactly what privacy and extensibility properties we'll get out of it. # Applying Mini Sketch to LN Gossip One issue we have today, is that other than the initial scid query mechanism added to the protocol, there isn't a great way to ensure you have all the latest updates your peer has. These days, many nodes pretty aggressively rate limit other nodes, so you might even have trouble sending out your update in the first place. A recent paper (that I haven't actually fully read yet) [8] analyzes the gossip network today to work out things like: exactly how long it takes things to propagate, total bandwidth usage, etc. Minisketch [9] (the grandchild of IBLTs ;)), is an efficient set reconciliation protocol that was designed for Bitcoin p2p mempool syncing, but can be applied to other protocols. An attendee has been working on brushing off some older work to try to see how we could apply it to the LN protocol to give nodes a more bandwidth efficient way to sync channel updates, and also achieve better update propagation. This supplements some existing investigative work done by Alex Meyers [10], with more concrete designs w.r.t: what goes into the sketch, and the various size parameters that need to be chosen. # Channel Jamming An attendee gave a talk on the various proposed solutions to channel jamming, evaluating them on several axis including: punishment/monetary, local vs global reputation, feasibility of mechanism design, UX implications, and implementation complexity. The presenter didn't present a new concrete proposal, but instead went through the various trade-offs, ultimately concluding that they factor monetary penalties wherein the funds are distributed across the route, rather than being provably burnt to miners. However they alluded to some future upcoming work that attempts a more rigorous analysis of the proposed solutions, their tradeoffs, and potential ways we can parametrize solutions to be more effective (how much should they pay, etc). For those looking to brush up on the latest state of research/mitigations in this area, I recommend this blog post by Bitmex research [11]. # Onion Messages & DoS The topic of DoS concerns related to onion messages (in isolation, so not necessarily related to things like bolt12 that take advantage of them came up. During a white boarding session some argued that DoS isn't actually much of an issue, as nodes can leverage "back propagation congestion control" to inform the source (who may not actually be the sender) that they'll start to drop or limit their packets, with each node doing this iteratively until the actual source of the spam has been clamped. A few lofty designs were thrown around, but more work needs to be done to concretely specify something so it can be properly analyzed. On the other side of the spectrum, rather than attempt to rate limit at the node level (which each node having their own policy), nodes could opt instead to forward _anything_ as long as the sender pays them enough. I proposed a lofty approach that combined AMP and Onion Messages earlier this year [12]. At a high level I make an AMP payment, which pushes extra coins to all nodes on a route, and also drops off a special identifier to them. When I send an onion message I include this identifier, with each node performing their own account w.r.t the amount of bandwidth an ID has remaining. Ultimately a few implementations are pretty close to deploying their implementation of onion messages, so no matter the intended use case, it would be good to have code deployed along side to either rate limit or price resource consumption accordingly. Otherwise, we might end up in a scenario where DoS concerns were brushed aside, but end up being a huge issue later. # Blinded Paths, QR Codes & Invoices Blinded paths [13] is a new-er proposal to solve the "last mile" privacy issue when receiving payments on LN. Today invoices to unadvertised channels contain a set of hop hints, which are anchored at public nodes in the graph, and also leak the scid of the unadvertised channel (points on-chain to the channel receiving payments). A solution for the on-chain leak, SCID channel aliases [15] are in the process of being widely rolled out. Channel aliases instead use a random value in the invoice, allowing receiving nodes to break that on-chain link and even rotate out the value periodically. With the on-chain leak addressed, it's still the case that you give away your "position" in the network, since as a sender I know that you're connected to node N with a private channel. Blinded paths address this node-level last mile privacy leak by replacing hop hints with a new cryptographically blinded path. At a high level, the receiver can construct a "hop hint" of length greater than 1, gather the public keys of each of the nodes, then blinded them such that: the sender can use them for path finding, but doesn't actually now exactly _which_ nodes they actually are. There're two type of blinded paths: those in onion messages and those used for actual payments. The latter variant was only formalized earlier this year, as before people were mainly interested in using them to fetch BOLT 12 invoice via onion messages. One issue that pops up when attempting to use blinded paths for normal payments is: the size of the resulting invoice. As blinded paths are actually fragments of publicly known paths, as a receiver, you want to stuff as many of them into the invoice as possible, since they MUST be taken in order to route towards you. Invoices are typically communicated via QR codes, which have a hard limit w.r.t the amount of information that can be packed in. On the other hand for invoice fetching, all that matters is that a path exists, so you can get by with stuffing less of then in a QR code. As a result, blinded paths aren't necessarily compatible with the widely deployed BOLT 11 based QR codes. Instead a way to fetch invoice on demand is required. Both BOLT-12 and LN-URL provide standardized ways for nodes to fetch invoices, though their transport/signalling medium of choice differs. Blinded routes are technically compatible with BOLT 11 invoices, but may be hampered by the fact that you can only include so many routes. Another consideration is that unlike hop hints, blinded paths require more maintain once, as since they traverse public route, policy changes like a fee update may invalidate an entire set set of routes. One proposed solution is that forwarding nodes should observe their older policy for a period of time (so a grace period), and also that blinded paths should have an explicit expiry (similar to the existing invoice expiry). One other implication is that the set of routes the receiver includes matters more: if they don't send enough or select them poorly, the sender may never be able to reach them even though a path exists in theory. More hands on experience is needed so the spec authors can better guide implementations and wallets w.r.t best practices. # Friend-of-a-friend Balance Sharing & Probing A presentation was given on friend-of-a-friend balance sharing [16]. The high level idea is that if we share _some_ information within a local radius, then this gives the sender more information to choose a path that's potentially more reliable. The tradeoff here ofc is that nodes will be giving away more information that can potentially be used to ascertain payment flows. In an attempt to minimize the amount of information shared, the presenter proposed that just 2 bits of information be shared. Some initial simulations showed that sharing local information actually performed better than sharing global information (?). Some were puzzled w.r.t how that's possible, but assuming the slides+methods are published others can dig further into the model/parameter used to signal the inclusion. Arguably, information like this is already available via probing, so one line of thinking is something like: "why not just share _some_ of it" that may actually lead to less internal failures? This is related to a sort of tension between probing as a tool to increase payment reliability and also as a tool to degrade privacy in the network. On the other hand, others argued that probing provides natural cover traffic, since they actually _are_ payments, though they may not be intended to succeed. On the topic of channel probing, a sort of makeshift protocol was devised to make it harder in practice, sacrificing too much on the axis of payment reliability. At a high level it proposes that: * nodes more diligently set both their max_htlc amount, as well as the max_htlc_value_in_flight amount * a 50ms (or select other value) timer should be used when sending out commitment signatures, independent of HTLC arrival * nodes leverage the max_htlc value to set a false ceiling on the max in flight parameter * for each HTLC sent/forwarded, select 2 other channels at random and reduce the "fake" in-flight ceiling for a period of time Some more details still need to be worked out, but some felt that this would kick start more research into this area, and also make balance mapping _slightly_ more difficult. From afar, it may be the case that achieving balance privacy while also achieving acceptable levels of payment reliability might be at odds with each other. # Eltoo & ANYPREVOUT One of the attendees is currently working on both fully implementing eltoo, as well as specifying the exact channel funding+update interaction were it to be rolled out align side the existing penalty based channels in the protocol. As this version of eltoo is based on Taproot, we were able to compare notes a bit to find the overlapping set of changes (nonce handling, etc), which permits cross review of the proposals. This type of work is cool, as only by fully implementing something end to end can you reaaally work out all the edge cases and nuances. ANYPREVOUT as hasn't changed significantly as of late. An attendee shared plans to create a sort of mega all-future-feasible-soft-forks fork of bitcoind, that would package up various unmerged (from bitcoind's) proposal soft fork packages into an easy to run+install binary/project attached to a signet. The hop is that by giving developers an easy way to interact with proposed soft fork proposals (vs debasing some ancient pull request), wider participation in testing/implementation/review can be facilitated. # Trampoline Routing There was a presentation on Trampoline routing explaining the motivation, history, and current state of the proposal. The two main cases we've narrowed down on are: 1. A mobile user doesn't necessarily want to sync the _entire_ graph, so they can use trampoline to maintain a subset and still be able to send payments. 2. A mobile user wants to be able to instate a payment, go offline, and return at a later time to learn about the final state of the payment. Use case #2 seems to be the most promising when combined with other proposals for holding HTLCs at an origin node (call it an "LSP") [13]. Combined together, this would allow a mobile node to send a payment, then go offline, with the LSP being able to retry the payment either continuously or only when it knows the receiver is online to accept the payment. This may potentially dramatically improve the UX for LN on mobile, as things suddenly become a lot more asynchronous: I do something go offline, and the LSP node can fulfil the payment in the background, then wait for me to come online to settle the final. hop. Trampoline can also be composed well with blinded routes (blinded route from last trampoline to receiver) and also MPP (internal nodes can split themselves with local information). One added trade-off is that since the sender doesn't know the entire route, they need to sort of overshoot w.r.t fees and CTLVs. This is something we've known for a while, but until Trampoline is more widely rolled out, we won't have a very good feel w.r.t how much extra senders will need to allocate. # Node Fee Optimization & Fee Rate Cards Over the past few years, a common thread we've seen across successful routing nodes is dynamic fee setting as a way to encourage/discourage traffic. A routing nodes can utilize the set of fees of a channel to either make it too expensive for other nodes to route through (it's already depleted don't try unless you'll give be 10 mil sats, which no one would) or very cheap, which'll incentivize flows in the other direction. If all nodes are constantly sending out updates of this nature, then it can generate a lot of traffic, and also sort of leak more balance information overtime (which some nodes are already doing: using fees/max_htlc to communicate available balances). One attendee proposed allowing nodes to express a sort of fee gradient via a static curve/bucket/function, instead of dynamically communicating what the latest state of the fee+liquidity distribution looks like. A possible manifestation could be a series of buckets, each of which with varying fee rates. If your payment consumes 50% of channel balance, then you pay this rate, otherwise if it's 5% you pay this rate, etc, etc. This might allow for nodes to capture the same dynamics as they do with more dynamic fee updates, but in a way that leaks less information and also consumes less gossip bandwidth. # The Return of Splicing Splicing is one of those things that was discussed a long time ago, but was never really fully implemented and rolled out. A few attendees have started to take a closer look at the problem, building off of the interactive-tx scheme that the dual-funding protocol extension uses. The main intricacy discussed was if concurrent splices should be allowed or not, and if so, how we would handle the various edge cases. As an example, if I propose a splice to add more funds via my input, but that turns out to already be spent, then the splicing transaction we created is invalid and can never be confirmed. However if we allow _another_ splice to take place, and another one, and another one, then ideally _one_ of them will confirm and serve as the new anchor for the channel. In a world of concurrent splices, the question of "what is my Lightning balance" becomes even more murky. Wallet and implementations will likely want to show the most pessimistic value, while also ensuring that the user is able to effectively account for where all their funds and what they can spend on/off chain. # LN-URL + BOLT 12 LN-URL and BOLT 12 are both standardized ways that answer the question of: how can I fetch an invoice from Bob? LN-URL differs from BOLT 12 in that it uses the existing BOLT 11 invoice format, and uses an HTTP based protocol for the negotiation process. BOLT 12 on the other hand is a suite of protocol additions that includes (amongst other things) a new invoice format (yay TLV!) and also a way to use onion messages to fetch an invoice _via_ the network. Assuming blinded paths is widely rolled out, then the question of how invoices are obtained becomes more important as blinded paths means that you can't fit much in the traditional QR encoding. As a result, fetching invoices on demand may become a more common place flow, with all its trade-offs. There was a group discussion on how we could sort of unifying everything either by allowing BOLT 12 to be used over LN-URL or the other way around. One proposal was to add a new query parameter to the normal LN-URL QR code contents. This would mean that when a wallet goes to scan an LN-URL QR code, if they know of the extra param, and what BOLT 12, they can just use the enclosed offer to fetch the invoice. An alternative proposal was to instead extract the BOLT 12 _invoice_ format from the greater BOLT 12 "Offers" proposal. Assuming blinded paths is only specified w.r.t BOLT 12 _invoices_, then this would mean an LN-URL extension could be rolled out that allowed returning BOLT 12 invoice rather than BOLT 11 invoices. This would allow the ecosystem to slowly transition to a shared invoice format, even if there may be fundamental disagreements w.r.t _how_ the invoices should be fetched in the first place. It's worth noting that both of these proposals can be combined: * If a wallet knows how to BOLT 12 Offers, they can take the enclosed offer and run w/ it. * If they don't know about Offers, but can send w/ the BOLT _invoice_ format, then they can fetch that and complete the payment. This might be a nice middle ground as it would tend all wallets/implementations to being able to decode and send w/ a BOLT 12 _invoice_, and leave the question of _how_ it should be fetched up to the application/wallet/service. In the end, if paths never quite intersect, then it's still possible to add route blinding to BOLT 11, with LN-URL sticking with that invoice format to take advantage of the new privacy enhancements [1]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-November/003336.html [2]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-October/003278.html [3]: https://github.com/lightning/bolts/pull/995 [4]: https://github.com/jonasnick/bips/blob/musig2/bip-musig2.mediawiki [5]: https://datatracker.ietf.org/doc/html/rfc6979 [6]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-February/003470.html [7]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-March/003526.html [8]: https://arxiv.org/abs/2205.12737 [9]: https://bitcoinops.org/en/topics/minisketch/ [10]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-April/003551.html [11]: https://blog.bitmex.com/preventing-channel-jamming/ [12]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-February/003498.html [13]: https://github.com/lightning/bolts/pull/765 [14]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-October/003307.html [15]: https://github.com/lightning/bolts/pull/910 [16]: https://github.com/lightning/bolts/pull/780 -- Laolu -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220607/eb9e8d17/attachment-0001.html>