Nostr notes by Carla Kirk-Cohen [ARCHIVE]

📅 Original date posted:2023-08-03 🗒️ Summary of this ...

2023-08-04T20:17:18Z

In reply to nevent1q…v2z8
_________________________

📅 Original date posted:2023-08-03
🗒️ Summary of this message: The sender expresses concerns about the potential re-identification of anonymized data and requests clarification on the collection period, data storage, and access. The recipient assures that data will be handled securely, anonymized, and not shared. They also mention the possibility of fuzzing timestamps.
📝 Original message:
Hi Elias,

Thanks for re-emphasizing the importance of being privacy-conscious
as we look into this work - we completely agree!

> clarify upfront whether you intend to time-box the collection period,
where the data would be stored, and who would have access to it

Our ideal collection period would be limited to a 6 month period. One
of our main aims in defining a common data format is to ensure that we
can provide node operators with tooling that they can run locally, so
that they do not need to export the data _at all_, only very aggregated
results.

In the case where folks are comfortable sharing their data with us,
we will follow best practices handling this sensitive information and
will not share the data onwards at all. Fields will also be anonymized
as described in the original email. Re your concerns around timestamps,
we can also fuzz timestamps, as only the resolution period matters to
our work (thanks for flagging!).

I hope that addresses your concerns. Research based on real world data
is always a difficult line to walk, but we believe worthwhile in this
case.

Cheers,
Carla + Clara

On Thu, Aug 3, 2023 at 4:54 AM Elias Rohrer <lnml at tnull.de> wrote:

> Hi Carla + Clara,
>
> I want to prefix this by saying that I'm very familiar with how limiting
> the lack of available real-world datasets can be for conducting significant
> simulations and empirical experiments on Lightning.
>
> However, it may be noteworthy that long-term collection of the proposed
> fields could potentially allow to re-identify the anonymized channel
> counterparties based off some heuristics correlating with the public graph
> data, especially when datasets from multiple (possibly neighbouring)
> collection points will end up being combined. Subsequently, this might
> allow to draw further conclusions on transferred amounts, channel
> liquidities at particular times, and, as HTLC settlement/failure timestamps
> are recorded in nanosecond resolution, potentially even the payment
> destination's identity (cf. 1 <https://arxiv.org/pdf/2006.12143.pdf>;).
>
> As surrendering this kind of data therefore requires a good level of trust
> in the researchers, it might be helpful (and best practise) if you could
> clarify upfront whether you intend to time-box the collection period, where
> the data would be stored, and who would have access to it. From my point of
> view clearly defining the collection period would also be mandatory as we
> don't want to incentivise node operators to collect and store HTLC data
> longer-term, especially if it's to this degree of detail.
>
> Best,
>
> Elias
>
> ### 1. Collect Anonymized Data
> We're aware that we are dealing with sensitive and private information.
> For this reason, we propose defining a common data format so that
> analysis tooling can be built around, so that node operators can run
> the analysis locally if desired. Fields marked with [P] *MUST* be
> randomized if exported to researching teams.
>
> The proposed format is a CSV file with the following fields:
> * version (uint8): set to 1, included to future-proof ourselves
> against the need to change this format.
> * channel_in (uint64)[P]: the short channel ID of the incoming channel
> that forwarded the HLTC.
> * channel_out (uint64)[P]: the short channel ID of the outgoing
> channel that forwarded the HTLC.
> * peer_in (hex string)[P]: the hex encoded pubkey of the remote peer
> for the channel_in.
> * peer_out (hex_string)[P]: the hex encoded pubkey of the remote peer
> for the channel_out.
> * fee_msat(uint64): the fee offered by the HTLC, expressed in msat.
> * outgoing_liquidity (float64): the portion of
> `max_htlc_value_in_flight` that is occupied on channel_out after the
> HTLC has been forwarded.
> * outgoing_slots (float64): the portion of `max_accepted_htlcs` that
> is occupied on channel_out after the HTLC has been forwarded.
> * ts_added_ns (uint64): the unix timestamp that the HTLC was added,
> expressed in nanoseconds.
> * ts_removed_ns (uint64): the unix timestamp that the HLTC was
> removed, expressed in nanoseconds.
> * htlc_settled (bool): set to 0 if the HTLC failed, and 1 if it was
> settled.
> * incoming_endorsed (int16): an integer indicating the endorsement
> status of the incoming HTLC (-1 if not present, otherwise set to the
> value in the incoming endorsement TLV).
> * outgoing_endorsed (int16): an integer indicating the endorsement
> status of the outgoing HTLC (-1 if not set, otherwise set to the
> value set in the outgoing endorsement TLV).
>
> Before we add endorsement signaling and setting via an experimental
> TLV, the last two values here will always be -1. The data is still
> incredibly useful in the meantime, and allows for easy update once the
>
> TLV is propagated through the network.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230803/bc25467c/attachment.html>;

📅 Original date posted:2023-08-01 🗒️ Summary of this ...

2023-08-02T22:15:37Z

In reply to nevent1q…m6kk
_________________________

📅 Original date posted:2023-08-01
🗒️ Summary of this message: The plan is to collect data on HTLC endorsement and local reputation tracking to mitigate jamming attacks. Multiple teams are involved in different phases of the plan.
📝 Original message:
Hi list,

## TL;DR
We're moving ahead with the plan discussed at the summit to "dry run"
HTLC endorsement and local reputation tracking to better inform our
efforts to mitigate jamming attacks.

Our goals are:
* To use real-world data to sanity check the "steady state" behavior
of local reputation algorithms, and to better inform creation of
synthetic data for simulating attack scenarios.
* To obtain liquidity and slot utilization data to inform sane defaults
for resource bucketing.
* To provide a common data export format to use as a common basis for
analysis.

As code takes some time to write and deploy, there are a few phases
for this plan - details at the end of the email for those who are
interested!
1. Collect anonymized forwarding data with a common format.
2. Propagate experimental `endorsement` TLV.
3. Implement local reputation and set `endorsement` values.

This is a multi-team effort:
* Eclair: Thomas is looking into collecting local reputation data
in [1].
* CLN: Vincenzo is working on experimental update to propagate the
endorsement field and a plugin that will allow us to run local
reputation scoring.
* LND: I am working on data export and HTLC endorsement via
circuitbreaker [2].
* LDK: some additional plumbing is needed, as outlined in [3].

## Research Plan

### 1. Collect Anonymized Data
We're aware that we are dealing with sensitive and private information.
For this reason, we propose defining a common data format so that
analysis tooling can be built around, so that node operators can run
the analysis locally if desired. Fields marked with [P] *MUST* be
randomized if exported to researching teams.

The proposed format is a CSV file with the following fields:
* version (uint8): set to 1, included to future-proof ourselves
against the need to change this format.
* channel_in (uint64)[P]: the short channel ID of the incoming channel
that forwarded the HLTC.
* channel_out (uint64)[P]: the short channel ID of the outgoing
channel that forwarded the HTLC.
* peer_in (hex string)[P]: the hex encoded pubkey of the remote peer
for the channel_in.
* peer_out (hex_string)[P]: the hex encoded pubkey of the remote peer
for the channel_out.
* fee_msat(uint64): the fee offered by the HTLC, expressed in msat.
* outgoing_liquidity (float64): the portion of
`max_htlc_value_in_flight` that is occupied on channel_out after the
HTLC has been forwarded.
* outgoing_slots (float64): the portion of `max_accepted_htlcs` that
is occupied on channel_out after the HTLC has been forwarded.
* ts_added_ns (uint64): the unix timestamp that the HTLC was added,
expressed in nanoseconds.
* ts_removed_ns (uint64): the unix timestamp that the HLTC was
removed, expressed in nanoseconds.
* htlc_settled (bool): set to 0 if the HTLC failed, and 1 if it was
settled.
* incoming_endorsed (int16): an integer indicating the endorsement
status of the incoming HTLC (-1 if not present, otherwise set to the
value in the incoming endorsement TLV).
* outgoing_endorsed (int16): an integer indicating the endorsement
status of the outgoing HTLC (-1 if not set, otherwise set to the
value set in the outgoing endorsement TLV).

Before we add endorsement signaling and setting via an experimental
TLV, the last two values here will always be -1. The data is still
incredibly useful in the meantime, and allows for easy update once the
TLV is propagated through the network.

### 2. Propagate Experimental Endorsement TLV
HTLC endorsement is signaled using an experimental range TLV in
`update_add_htlc` (which has been reserved in [4]):

tlv_stream: update_add_htlc_tlvs
Type: 655555
Data: byte (endorsed)

This signal should be propagated by forwarding nodes in the following
manner:
- if `endorsed` is present in the incoming `update_add_htlc`:
- set the same value for the outgoing `update_add_htlc`.
- otherwise:
- set `endorsed` = 0 for the outgoing `update_add_htlc`.

### 3. Implement Local Reputation and Set Endorsement
The final step will be to implement local reputation algorithms and
start to actively set the value of the `endorsed` TLV for outgoing
HTLCs, rather than simply copying the value presented by the sending
node. This signal will *not* be used for any purpose other than
data collection.

Experimenters are free to use the full range of bits to express
endorsement values, but should be aware that any non-zero value will
be interpreted as a positive endorsement signal by implementations
using binary endorsement (as is currently specified in [5]).

A positive endorsement signal requires that the original sender of a
HTLC sets a non-zero value, but bears the privacy risk of indicating
that they are the sending node during upgrade. We suggest that senders
choose some probability P (suggested default: 20%) with which to set
endorsed=1 for their payments.

Once we've got data collection code in place, we'll make a more
general call for node operators to start collection. In the meantime,
feel free to reach out if you have any questions or are interested in
helping out!

Cheers,
Carla + Clara

## References
[1] https://github.com/ACINQ/eclair/pull/2716
[2] https://github.com/lightningequipment/circuitbreaker/issues/77
[3] https://github.com/lightningdevkit/rust-lightning/issues/2425
[4] https://github.com/lightning/blips/pull/27
[5] https://github.com/lightning/bolts/pull/1071
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230801/9ef1baa2/attachment-0001.html>;

📅 Original date posted:2023-07-27 🗒️ Summary of this ...

2023-07-30T22:54:53Z

In reply to nevent1q…r5h7
_________________________

📅 Original date posted:2023-07-27
🗒️ Summary of this message: Transcripts of specification meetings held every other Monday are now available at https://btctranscripts.com/lightning-specification, thanks to Gurwinder at Chaincode.
📝 Original message:
Hi List,

Since the beginning of the year we've made an effort to record and
transcribe the specification meetings that we hold every other Monday.
We've managed to keep the habit up, so I'm sharing here in the hopes
that they'll be discovered by those who will find them useful.

These transcripts are available here:
https://btctranscripts.com/lightning-specification

Massive thanks to Gurwinder at Chaincode who human-checks the
transcripts to keep the AI overlords on their best behavior.

Cheers,
Carla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230727/8322a83c/attachment-0001.html>;

📅 Original date posted:2023-07-19 🗒️ Summary of this ...

2023-07-21T12:47:43Z

In reply to nevent1q…kuqe
_________________________

📅 Original date posted:2023-07-19
🗒️ Summary of this message: The text is a summary of the annual specification meeting held in NYC. It includes discussions on package relay, commitment transactions, and zero fee commitments.
📝 Original message:
Hi List,

At the end of June we got together in NYC for the annual specification
meeting. This time around we made an attempt at taking transcript-style
notes which are available here:
https://docs.google.com/document/d/1MZhAH82YLEXWz4bTnSQcdTQ03FpH4JpukK9Pm7V02bk/edit?usp=sharing
.
To decrease our dependence on my google drive I've also included the
full set of notes at the end of this email (no promises about the
formatting however).

We made a semi-successful attempt at recording larger group topics, so
these notes roughly follow the structure of the discussions that we had
at the summit (rather than being a summary). Speakers are not
attributed, and any mistakes are my own.

Thanks to everyone who traveled far, Wolf for hosting us in style in
NYC and to Michael Levin for helping out with notes <3

# LN Summit - NYC 2023

## Day One

### Package Relay
- The current proposal for package relay is ancestor package relay:
- One child can have up to 24 ancestors.
- Right now, we only score mempool transactions by ancestry anyway, so
there isn’t much point in other types of packages.
- For base package relay, commitment transactions will still need to have
the minimum relay fee.
- No batch bumping is allowed, because it can open up pinning attacks.
- With one anchor, we can package RBF.
- Once we have package relay, it will be easier to get things into the
mempool.
- Once we have V3 transactions, we can drop minimum relay fees because we
are restricted to one child pays for one parent transaction:
- The size of these transactions is limited.
- You can’t arbitrarily attach junk to pin them.
- If we want to get rid of 330 sat anchors, we will need ephemeral anchors:
- If there is an OP_TRUE output, it can be any value including zero.
- It must be spent by a child in the same package.
- Only one child can spend the anchor (because it’s one output).
- The parent must be zero fee because we never want it in a block on its
own, or relayed without the child.
- If the child is evicted, we really want the parent to be evicted as
well (there are some odd edge cases at the bottom of the mempool, so zero
ensures that we’ll definitely be evicted).
- The bigger change is with HLTCs:
- With SIGHASH_ANYONECANPAY, your counterparty can inflate the size of
your transaction - eg: HTLC success, anyone can attack junk.
- How much do we want to change here?
- So, we can get to zero fee commitment transactions and one (ephemeral)
anchor per transaction.
- With zero fee commitments, where do we put trimmed HTLCs?
- You can just drop it in an OP_TRUE, and reasonably expect the miner
to take it.
- In a commitment with to_self and to_remote and an ephemeral anchor that
must be spent, you can drop the 1 block CSV in the transaction that does
not have a revocation path (ie, you can drop it for to_remote).
- Any spend of this transaction must spend the one anchor in the same
block.
- No other output is eligible to have a tx attached to it, so we don’t
need the delay anymore.
- Theoretically, your counterparty could get hold of your signed local
copy, but then you can just RBF.
- Since these will be V3 transactions, the size of the child must be small
so you can’t pin it.
- Both parties can RBF the child spending the ephemeral anchor.
- This isn’t specifically tailored to lightning, it’s a more general
concept.
- Child transactions of V3 are implicitly RBF, so we won’t have to worry
about it not being replaceable (no inheritance bug).
- In general, when we’re changing the mempool policy we want to make the
minimal relaxation that allows the best improvement.
- We need “top of block” mempool / cluster mempool:
- The mempool can have clusters of transactions (parents/children
arranged in various topologies) and the whole mempool could even be a
single cluster (think a trellis of parents and children “zigzagging”).
- The mining algorithm will pick one “vertical” using ancestor fee rate.
- There are some situations where you add a single transaction and it
would completely change the order in which our current selection picks
things.
- Cluster mempool groups transactions into “clusters” that make this
easier to sort and reason about.
- It can be expensive (which introduces a risk of denial of service), but
if we have limits on cluster sizes then we can limit this.
- This is the only way to get package RBF.
- How far along is all of this?
- BIP331: the P2P part that allows different package types is moving
along and implementation is happening. This is done in a way that would
allow us to add different types of packages in future if we need them.
- There are some improvements being made to core’s orphan pool,
because we need to make sure that peers can’t knock orphans out of your
pool that you may later need to retrieve as package ancestors.
- We reserve some spots with tokens, and if you have a token we keep
some space for you.
- V3 transactions: implemented on top of package relay, since they don’t
really make sense without it. This is an opt-in regime where you make
things easier to RBF.
- Ephemeral Anchors: on top of package relay and V3.
- Cluster Mempool: This is further out, but there’s been some progress.
Right now people are working on the linearization algorithm.
- If these changes don’t suit lightning’s use case, now is the time to
speak because it’s all being worked on.
- In what way does it not fit LN, as it’s currently designed?
- With the V3 paradigm, to stop all pinning vectors, HTLC transactions
will need to get anchors and you will have to drop ANYONECANPAY (to use
anchors instead).
- Could we fix this with additional restrictions on V3 (or a V4)?
- You could do something where V4 means that you can have no ancestors
and no descendants (in the mempool).
- V3 restricts the child, you could also restrict the parent further.
- You could have a heuristic on the number of inputs and outputs, but
anyone can pay or add inputs.
- You could commit to the whole thing being some size by setting a
series of bits in sequence to mark maximum size but that would involve
running script (which is annoying).
- The history of V3 was to allow a parent of any size because
commitment transactions can be any size.
- What about keeping HTLCs the way they are today?
- A lot of other things are less pineapple, maybe that’s ok? It’s a step
forward.
- The long term solution is to change relay policy to top of mempool. We
shouldn’t be doing things until the long term solution is clear, and we
shouldn’t change the protocol in a way that doesn’t fit with the long term.
- We already do a lot of arbitrary things, if we were starting from day
one we wouldn’t do HTLCs with anchors (it’s too much bloat), and being
unable to RBF is more bloat because you overpay.
- If the remote commitment is broadcast, you can spen the HTLC with V2.
You can’t add a V3 child (it won’t get in the mempool).
- If you want to be consistent, even when it’s the remote commit you need
a presigned transaction, we should just use it as designed today.
- What is the “top of mempool” assumption?
- If large transactions are at the top of your mempool, you (a miner)
want transactions with higher fee rates to increase your total revenue.
- Today, you can’t really easily answer or reason about these questions.
- We want to accept transactions in our mempool in a denial of service
resistant way that will strictly increase our miner fees.
- If a transaction is at the top of your mempool, you may be more willing
to accept a replacement fee. If it’s way down in the mempool, you would
probably replace more slowly.
- It’s conceivable that we could get here, but it’s years off.
- Once we have this, pinning only happens with large transactions at
the bottom of the mempool. If you just slow roll these transactions, you
can just relay what you’ve got.
- If somebody comes and replaces the bottom with something small
that’ll be mined in the next two blocks, you clearly want to accept that.
- Does cluster mempool fix rule 3?
- No but they help.
- Today when you get a transaction you don’t know which block it’s going
in - we don’t have a preference ordering.
- Miners don’t make blocks as they go - with cluster mempool you can make
very fast block templates.
- When talking about relay, you can ignore block making and just get a
very good estimate.
- The question is: when do we jump?
- Wail until V3? Package Relay?
- Package relay will help. You still negotiate fees but they don’t
matter as much.
- We’d like to kill upate fee and have a magic number for fees, can we
do that when we get package relay?
- Package relay is the hard part, V3 should be relatively easier after
that.
- When we get V3 we can drop to zero fee.
- Is there a future where miners don’t care about policy at all?
- Say, the block that I’m mining has V3 transactions. They’re just
maximizing fees so they’ll accept anything out of band, ignoring these new
policy rules.
- Accelerators are already starting to emerge today - how are they doing
it?
- We’re carving out a small space in which mempool rules work, and it’s
okay to work around it.
- If a miner mines it, great - that’s what we want. The problem is not
getting mined (ie pinning).
- Figuring this out is not simple:
- Today there are times where we accept replacements when we should
reject them and times where we reject replacements when we should accept
them.
- There can be situations where miners will mine things that aren’t
incentive compatible (ie, not the best block template).
- What if somebody pays out of band not to mine?
- Ephemeral anchors are interesting, they never enter the UTXO set - if the
child gets evicted, you get evicted.
- The first transaction being zero is optimal, want to be sure it’ll be
evicted (there are some mempool quirks).
- It must be zero fee so that it will be evicted.
- Should we add trimmed HTLCs to the ephemeral anchor?
- We don’t want to get above min relay fee because then we could hang
around in the mempool.
- The eltoo implementation currently does this.
- You can’t keep things in OP_TRUE because they’ll be taken.
- You can also just put it in fees as before.
- More on cluster mempools:
- It can be used for block selection, but it’s currently focused on
mempool selection.
- You can simulate template selection with it.
- When you’re mining, you have to pick from an ancestor downwards.
- First we lineralize the transactions to flatten the structure
intelligently (by topology and fee rate).
- Then we figure out the best ordering of this flat structure.
- Each chunk in a cluster is less than 75 transactions, and a cluster has
chunks of transactions.
- If you woul include a transaction with the ancestor, it goes in the
same chunk. Otherwise it goes in the next chunk.
- Miners select the highest fee rate chunks, lower fee rate ones can
safely be evicted.
- For replacement, you can just check replacement for a single cluster,
rechunk it and then resort the chunks.
- It must beat the fee rate of the chunks to get in.
- You can check how much it beats the chunk by, whether it would go in
the next block(s) and then decide.
- Chunk ordering takes into account transaction size, beyond 25
transactions we just do ancestor set feerate.
- One of the limits is going away, one is sticking around. We still have
sibling eviction issues.
- Is one of the issues with chunks is that they’re limited by transaction
weight, 101 kilo-v-bytes, which is the maximum package size?
- We should be okay, these limits are higher than present.
- You can bound chunk size pretty easily.
- If we get chunk fee rate replacement then you can do batch fee bumping
(eg, a bunch of ephemeral anchors that are all batched together).
- Are there long term policy implications for privacy for ephemeral anchors?
- You’re essentially opting into transaction sponsors?
- If everyone uses V3 eventually there’s no issue.
- For right now, it would be nice if V3 is only for unilateral closes.
- It’s also useful in a custodial wallet setting, where you have one team
creating on chain transactions and the other attaching fees. This is a
common accounting headache. People can also non-interactively fee bump.

### Taproot
- Spec is still in draft right now, and the code is a bit ahead of the test
vectors.
- The biggest change that has been made is around anchors, which become
more complicated with taproot:
- The revocation path on to_local takes the script path, so now we need
to reveal data for the anchor.
- The downside is that revocation is more expensive - 32 more bytes in
the control block.
- The to_remote has a NUMS point, previously we just had multisig keys:
- If you’re doing a rescan, you won’t know those keys.
- Now you just need to know the NUMS point and you can always rescan.
- The NUMS point itself is pretty verifiable, you just start with a
string and hash it.
- It is constant, you just use it randomized with your key.
- The internal pubkey is constant on to_remote.
- Co-op close [broke into several different discussions]:
- The idea here is to remove negotiation, since we’ve had disagreement
issues in the past.
- In this version, the initiator just accepts the fee.
- Do we want negotiation?
- In the past we’ve had bugs with fee rate estimates that won’t budge.
- Why don’t we just pick our fees and send two sets of signatures?
- This would no longer be symmetric.
- What if nobody wants to close first? We’d need to work through the
game theory.
- The person who wants their funds has an incentive.
- Why is RBF so hard for co-op close?
- Closing should be marked as RBF, there’s no reason not to.
- Just pick your fee rate, pay it and then come back and RBF if you’d
like. You can bump/broadcast as much as you like.
- If we’re going to have something like that where we iterate, why
don’t we just do the simple version where we pick a fee and sign?
- If you have to pay the whole fee, you have less incentive to sign.
- Why is this linked to taproot work?
- It needs to change anyway, and we need to add nonces.
- What about, whoever wants to close sends a fee rate (paying the fees)
and the responder just sends a signature?
- If you don’t have enough balance, you can’t close. But why do you
care anyway, you have no funds?
- We can do this as many times as we want.
- Shutdown is still useful to clear the air on the channel.
- When you reconnect, you start a new interaction completely.
- TL;DR:
- Shutdown message stays.
- You send a signature with a fee rate.
- The remote party signs it.
- If they disagree, you do it again.
- It’s all RBF-able.
- You must retransmit shutdown, and you must respond with a shutdown.
- You can send nonces at any time:
- In revoke and ack.
- On channel re-establish.
- For taproot/musig2 we need nonces:
- Today we store the commitment signature from the remote party. We don’t
need to store our own signature - we can sign at time of broadcast.
- To be able to sign you need the verification nonce - you could remember
it, or you could use a counter:
- Counter based:
- We re-use shachain and then just use it to generate nonces.
- Start with a seed, derive from that, use it to generate nonces.
- This way you don’t need to remember state, since it can always be
generated from what you already have.
- Why is this safe?
- We never re-use nonces.
- The remote party never sees your partial signature.
- The message always stays the same (the dangerous re-use case is
using the same nonce for different messages).
- If we used the same nonce for different messages we could leak our
key.
- You can combine the sighash + nonce to make it unique - this also
binds more.
- Remote party will only see the full signature on chain, never your
partial one.
- Each party has sign and verify nonces, 4 total.
- Co-op close only has 2 because it’s symmetric.

### Gossip V1.5 vs V2
- How much do we care about script binding?
- It it’s loose, it can be any script - you can advertise any UTXO.
- You revel less information, just providing a full signature with the
full taproot public key.
- If it’s tight, you have to provide two keys and then use the BIP 86
tweak to check that it’s a 2-of-2 multisig.
- Should we fully bind to the script, or just allow any taproot output?
- Don’t see why we’d want additional overhead.
- Any taproot output can be a channel - let people experiment.
- We shouldn’t have cared in the first place, so it doesn’t matter what
it’s bound to.
- It’s just there for anti-DOS, just need to prove that you can sign.
- Let every taproot output be a lightning channel, amen.
- We’re going to decouple:
- You still need a UTXO but it doesn’t matter what it looks like.
- This also allows other channel types in future.
- We send:
- UTXO: unspent and in the UTXO set
- Two node pubkeys
- One signature
- How much do we care about amount binding?
- Today it is exact.
- People use it for capacity graphs.
- Graph go up.
- We can watch the chain for spends when we know which UTXO to watch
per-channel.
- Is there an impact on pathfinding if we over-advertize?
- We use capacity to pathfind.
- What’s the worst case if people lie? We don’t use them.
- If we’ve already agreed that this can be a UTXO that isn’t a channel,
then it shouldn’t matter.
- If you allow value magnification, we can use a single UTXO to claim for
multiple channels. Even in the most naive version (say 5x), you’re only
revealing 20% of your UTXOs.
- How much leverage can we allow? The only limit is denial of service.
- There’s the potential for a market for UTXOs.
- There’s a privacy trade-off:
- If you one-to-one map them, then there’s no privacy gain.
- Do we know that you get substantial privacy?
- Even if you have two UTXOs and two channels, those UTXOs are now not
linked (because you can just use the first one to advertise).
- This is only assuming that somebody implements it/ infrastructure is
built out.
- People could create more elaborate things over time, even if
implementations do the “dumb” way.
- Gossip 1.5 (ie, with amount binding) fits in the current flow, V2 (ie,
without amount binding) has a very different scope.
- It’s a big step, and you don’t truly know until you implement it.
- What about things like: a very large node and a very small node, whose
announced UTXO do you use?
- We decided not to put UTXOs in node announcement, so we’d put it in
channel announcement:
- Sometimes there’s a UTXO, sometimes there isn’t.
- You look at a node’s previous channels to see if they still have
“quota”.
- If you don’t have “quota” left, you have to include a signature TLV.
- With the goal of publicly announcing taproot channels, 1.5 gets us there
and is a much smaller code change.
- We’ve talked alot about capacity for pathfinding, but we haven’t really
touched on control valves like max HTLC:
- Currently we don’t use these valves to tune our pathfinding, people
don’t use it.
- If we get better here, we won’t need capacity.
- This value is already < 50% anyway.
- If we don’t un-bind amounts now, when will we do it?
- It’s always a lower priority and everyone is busy.
- If we allow overcommitting by some factor now, it’s not unrealistic
that it will allow some degree of privacy.
- Between these features, we have opened the door to leasing UTXOs:
- Before we do more over-commitment, let’s see if anybody uses it?
- We add channel capacity to the channel announcement with a feature bit:
- If we turn the feature off, we are on-to-one mapped.
- But a node can’t use the upgraded version until everyone is upgraded?
- Our current “get everyone upgraded” cycle is 18 months (or a CVE).
- If you’re upgrading to 2x multiplier, nobody on 1x will accept that
gossip.
- People will not pay for privacy (via lost revenue of people not
seeing their gossip).
- This is additive to defeating chain analysis.
- Private UTXO management is already complicated, we don’t know the
ideal that we’re working towards.
- What about if we just set it to 2 today?
- Is 2 qualitatively better than 1 (without script binding) today?
- Will a marketplace magically emerge if we allow over-commitment?
- We don’t know the implications of setting a global multiplier for routing
or denial of service, and we don’t have a clear view of what privacy would
look like (other than “some” improvement).
- We agree that adding a multiplier doesn’t break the network.
- People with a lower value will see a subnet when we upgrade.
- We’re going to go with gossip “1.75”:
- Bind to amount but not script.
- We include a TLV cut out that paves the way to overcommitment.

### Multi-Sig Channel Parties
- There are a few paths we could take to get multi-sig for one channel
party:
- Script: just do it on the script level for the UTXO, but it’s heavy
handed.
- FROSTy: you end up having to do a bunch of things around fault
tolerance which require a more intense setup. You also may not want the
shachain to be known by all of the parties in the setup (we have an ugly
solution for this, we think).
- Recursive musig:
- Context: you have one key in the party, but you actually want it to be
multiple keys under the hood. You don’t want any single party to know the
revocation secrets, so you have to each have a part and combine them.
- Ugliest solution: just create distinct values and store them.
- Less ugly solution uses multiple shachains:
- Right now we have a shachan, and we reveal two leaves of it.
- You have 8 shachains, and you XOR them all together.
- Why do we need 8? That’ll serve a 5-of-7.
- Maybe we need 21? 5 choose 7, we’re not sure.
- How does this work with K-of-N?
- Each party has a piece, and you can always combine them in different
combinations (of K pieces) to get to the secret you’re using.

### PTLCs
- We can do PTLCs in two ways:
- Regular musig
- Adaptor signatures
- Do they work with trampoline as defined today?
- The sender picks all the blinding factors today, so it’s fine.
- There’s a paper called: splitting locally while routing
interdimensionally:
- You can let intermediate nodes do splitting because they know all the
local values.
- Then can generate new blinding factors and split out to the next node.
- Adaptor signatures could possibly be combined for PTLCs that get fanned
out then combined.
- There are a few options for redundant overpayment (ie, “stuckless”
payments):
- Boomerang:
- Preimages are coefficients of a polynomial, and you commit to the
polynomial itself.
- If you have a degree P polynomial and you take P+1 shares, then you
can claim in the other direction.
- Quite complex.
- You have to agree on the number of splits in advance.
- Spear:
- H2TLC: there are two payment hashes per-HTLC, one if from the sender
and one is from the invoice.
- When you send a payment, the sender only reveals the right number of
sender preimages.
- This also gives us HTLC acknowledgement, which is nice.
- You can concurrently split, and then spray and pray.
- Interaction is required to get preimages.
- Do we want to add redundant overpayment with PTLCs?
- We’ll introduce a communication requirement.
- Do we need to decide that before we do PTLCs?
- Can we go for the simplest possible option first and then add it?
- For spear, yes - the intermediate nodes don’t know that it’s two
hashes.
- We could build them out without thinking about overpayment, and then
mix in a sender secret so that we can claim a subset.
- We’ll have more round trips, but you also have the ability to ACK HTLCs
that have arrived.
- Spray and pray uses the same about as you would with our current “send
/ wait / send”, it’s just concurrently not serially.
- Is it a problem that HLTCs that aren’t settled don’t pay fees?
- You’re paying the fastest routes.
- Even if it’s random, you still get a constant factor of what we should
have gotten otherwise.
- It makes sense to use onion messages if it’s available to us.
- Are we getting payment acknowledgement? Seems so!

## Day Two

### Hybrid Approach to Channel Jamming
- We’ve been talking about jamming for 8 years, back and forth on the
mailing list.
- We’d like to find a way to move forward so that we can get something
done.
- Generally when we think about jamming, there are three “classes” of
mitigations:
- Monetary: unconditional fees, implemented in various ways.
- Reputation: locally assessed (global is terrible)
- Scarce Resources: POW, stake, tokens.
- The problem is that none of these solutions work in isolation.
- Monetary: the cost that will deter an attacker is unreasonable for an
honest user, and the cost that is reasonable for an honest user is too low
for an attacker.
- Reputation: any system needs to define some threshold that is
considered good behavior, and an attacker can aim to fall just under it.
Eg: if you need a payment to resolve in 1 minute, you can fall just under
that bar.
- Scarce resources: like with monetary, pricing doesn’t work out.
- Paper: proof of work doesn’t work for email spam, as an example.
- Since scarce resources can be purchased, they could be considered a
subset of monetary.
- There is no silver bullet for jamming mitigation.
- Combination of unconditional fees and reputation:
- Good behavior grants access to more resources, bad behavior loses it.
- If you want to fall just below that threshold, we close the gap with
unconditional fees.
- Looking a these three classes implemented in isolation, are there any
unresolved questions that people have - “what about this”?
- Doesn’t POW get you around the cold start problem in reputation where
if you want to put money in to quickly bootstrap you can?
- Since POW can be rented, it’s essentially a monetary solution - just
extra steps.
- We run into the same pricing issues.
- Why these combinations?
- Since scarce resources are essentially monetary, we think that
unconditional fees are the simplest possible monetary solution.
- Unconditional Fees:
- As a sender, you’re building a route and losing money if it doesn’t go
through?
- Yes, but they only need to be trivially small compared to success
case fee budgets.
- You can also eventually succeed so long as you retry enough, even if
failure rates are very high.
- How do you know that these fees will be small? The market could decide
otherwise.
- Routing nodes still need to be competitive. If you put an
unconditional fee of 100x the success case, senders will choose to not send
through you because you have no incentive to forward.
- We could also add an in-protocol limit or sender-side advisory.
- With unconditional fees, a fast jamming attack is very clearly paid for.
- Reputation:
- The easiest way to jam somebody today is to send a bunch of HTLCs
through them and hold them for two weeks. We’re focusing on reputation to
begin with, because in this case we can quite easily identify that people
are doing something wrong (at the extremes).
- If you have a reputation score that blows up on failed attempts,
doesn’t that fix it without upfront fees?
- We have to allow some natural rate of failure in the network.
- An attacker can still aim to fall just below that failure threshold
and go through multiple channels to attack an individual channel.
- THere isn’t any way to set a bar that an attacker can’t fall just
beneath.
- Isn’t this the same for reputation? We have a suggestion for
reputation but all of them fail because they can be gamed below the bar.
- If reputation matches the regular operation of nodes on the network,
you will naturally build reputation up over time.
- If we do not match reputation accumulation to what normal nodes do,
then an attacker can take some other action to get more reputation than the
rest of the network. We don’t want attackers to be able to get ahead of
regular nodes.
- Let’s say you get one point for success and one for failure, a
normal node will always have bad reputation. An attacker could then send 1
say payments all day long, pay a fee for it and gain reputation.
- Can you define jamming? Is it stuck HTLCs or a lot of 1 sat HTLCS
spamming up your DB?
- Jamming is holding HTLCs to or streaming constant failed HTLCs to
prevent a channel from operating.
- This can be achieved with slots or liquidity.
- Does the system still work if users are playing with reputation?
- In the steady state, it doesn’t really matter whether a node has a good
reputation or not.
- If users start to set reputation in a way that doesn’t reflect normal
operation of the network, it will only affect their ability to route when
under attack.
- Isn’t reputation monetary as well, as you can buy a whole node?
- There is a connection, and yes in the extreme case you can buy an
entire identity.
- Even if you do this, the resource bucketing doesn’t give you a “golden
ticket” to consume all slots/liquidity with good reputation, so you’re
still limited in what you can do.
- Can we learn anything from research elsewhere / the way things are done
on the internet?
- A lot of our confidence that these solutions don’t work in isolation is
based on previous work looking at spam on the internet.
- Lightning is also unique because it is a monetary network - we have
money built in, so we have different tools to use.
- To me, it seems like if the scarce resource that we’re trying to allocate
is HTLC slots and upfront fees you can pay me upfront fees for the worst
case (say two weeks) and then if it settles if 5 seconds you give it back?
- The dream solution is to only pay for the amount of time that a HTLC is
held in flight.
- The problem here is that there’s no way to prove time when things go
wrong, and any solution without a universal clock will fall back on
cooperation which breaks down in the case of an attack.
- No honest user will be willing to pay the price for the worst case,
which gets us back to the pricing issue.
- There’s also an incentives issue when the “rent” we pay for these two
weeks worst case is more than the forwarding fee, so a router may be
incentivized to just hang on to that amount and bank it.
- We’ve talked about forwards and backwards fees extensively on the
mailing list:
- They’re not large enough to be enforceable, so somebody always has
to give the money back off chain.
- This means that we rely on cooperation for this refund.
- The complexity of this type of system is very high, and we start to
open up new “non-cooperation” concerns - can we be attacked using this
mechanism itself?
- Doesn’t an attacker need to be directly connected to you to steal in
the non-cooperative case?
- At the end of the day, somebody ends up getting robbed when we can’t
pull the money from the source (attacker).
- Does everybody feel resolved on the statement that we need to take this
hybrid approach to clamp down on jamming? Are there any “what about
solution X” questions left for anyone? Nothing came up.

### Reputation for Channel Jamming
- Resource bucketing allows us to limit the number of slots and amount of
liquidity that are available for nodes that do not have good reputation.
- No reputation system is perfect, and we will always have nodes that
have low-to-no activity, or are new to the network that we can’t form
reputation scores for.
- It would be a terrible outcome for lightning to just drop these HTLCs,
so we reserve some portion of resources for them.
- We have two buckets: protected and general (split 50/50 for the purposes
of explanation, but we’ll find more intelligent numbers with further
research):
- In the normal operation of the network, it doesn’t matter if you get
into the protected slots. When everyone is using the network as usual,
things clear out quickly so the general bucket won’t fill up.
- When the network comes under attack, an attacker will fill up slots and
liquidity in the general bucket. When this happens, only nodes with good
reputation will be able to use the protected slots, other HTLCs will be
dropped.
- During an attack, nodes that don’t have a good reputation will
experience lower quality of service - we’ll gradually degrade.
- What do you mean by the steady state?
- Nobody is doing anything malicious, payments are clearing out as usual
- not sitting on the channel using all 483 slots.
- We decide which bucket the HTLC goes into using two signals:
- Reputation: whether the upstream node had good reputation with our
local node.
- Endorsement: whether the upstream node has indicated that the HTLC is
expected to be honest (0 if uncertain, 1 if expected to be honest).
- If reputation && endorsement, then we’ll allow the HTLC into protected
slots and forward the HTLC on with endorsed=1.
- We need reputation to add a local viewpoint to this endorsement signal
- otherwise we can just trivially be jammed if we just copy what the
incoming peer said.
- We need endorsement to be able to propagate this signal over multiple
hops - once it drops, it’s dropped for good.
- There’s a privacy questions for when senders set endorsed:
- You can flip a coin or set the endorsed field for your payments at
the same proportion as you endorse forwards.
- We think about reputation in terms of the maximum amount of damage that
can be done by abusing it:
- Longest CLTV that we allow in the future from current height 2016
blocks (~2 weeks): this it the longest that we can be slow jammed.
- Total route length ~27 hops: this is the largest amplifying factor an
attacker can have.
- We use the two week period to calculate the node’s total routing revenue,
this is what we have to lose if we are jammed.
- We then look at a longer period, 10x the two week period to see what the
peer has forwarded us over that longer period.
- If they have forwarded us more over that longer period than what we have
to loose in the shorter period, then they have good reputation.
- This is the damage that is observable to us - there are values outside of
the protocol that are also affected by jamming:
- Business reliability, joy of running a node, etc
- We contend that these values are inherently unmeasurable to protocol
devs:
- End users can’t easily put a value on them.
- If we try to approximate them, users will likely just run the
defaults.
- One of the simplest attacks we can expect is an “about turn” where an
attacker behaves perfectly and then attacks:
- So, once you have good reputation we can’t just give you full access to
protected slots.
- We want to reward behavior that we consider to be honest, so we consider
“effective” HTLC fees - the fee value that a HTLC has given us relative to
how long it took to resolve:
- Resolution period: the amount of time that a HTLC can reasonably take
to resolve - based on MPP timeout / 1 minute.
- We calculate opportunity cost for every minute after the first
“allowed” minute as the fees that we could have earned with that
liquidity/slot.
- Reputation is only negatively affected if you endorsed the HTLC.
- If you did not endorse, then you only gain reputation for fast success
(allowing bootstrapping).
- When do I get access to protected slots?
- When you get good reputation, you can use protected slots for your
endorsed HTLCs but there is a cap on the number of in flight HLTCs that are
allowed.
- We treat every HLTC as if it will resolve with the worst possible
outcome, and temporarily dock reputation until it resolves:
- In the good case, it resolves quickly and you get your next HLTC
endorsed.
- In the bad case, you don’t get any more HTLCs endorsed and you
reputation remains docked once it resolves (slowly).
- Wouldn’t a decaying average be easier to implement, rather than a sliding
window?
- If you’re going to use large windows, then a day here or there doesn’t
matter so much.
- Have you thought about how to make this more visible to node operators?
- In the steady state we don’t expect this to have any impact on routing
operations, so they’ll only need a high level view.
- Can you elaborate on slots vs liquidity for these buckets?
- Since we have a proportional fee for HTLCs, this indirectly represents
liquidity: larger HTLCs will have larger fees so will be more “expensive”
to get endorsed.
- Where do we go from here?
- We would like to dry run with an experimental endorsement field, and
ask volunteers to gather data for us.
- Are there any objections to an experimental TLV?
- No.
- We could also test multiple endorsement signals / algorithms in
parallel.
- In your simulations, have you looked at the ability to segment off
attacks as they happen? To see how quickly an attacker's reputation drops
off, and that you have a protected path?
- Not yet, but plan to.
- Do any of these assumptions change with trampoline? Don’t think it’s
related.
- Your reputation is at stake when you endorse, when do you decide to
endorse my own payments?
- You do the things you were already doing to figure out a good route.
Paths that you think have good liquidity, and have had success with in the
past.
- What about redundant overpayment, some of your HTLCs are bound to fail?
- Provided that they fail fast, it shouldn’t be a problem.
- Is it possible that the case where general slots are perpetually filled
by attackers becomes the steady state? And we can’t tell the difference
between a regular user and attacker.
- This is where unconditional fees come in, if somebody wants to
perpetually fill up the general bucket they have to pay for it.
- Is there anything we’d like to see that will help us have move confidence
here?
- What do you think is missing from the information presented?
- We can simulate the steady state / create synthetic data, but can’t
simulate every attack. Would like to spend more time thinking through the
ways this could possibly be abused.
- Would it help to run this on signet? Or scaling lightning?
- Its a little easier to produce various profiles of activity on
regtest[.

### Simplified Commitments
- Simplified commitments makes our state machine easier to think about.
- Advertise option_simplified_commitment: once both peers upgrade, we can
just use it.
- Simplify our state machine before we make any more changes to it.
- Right now Alice and Bob can have changes in flight at the same time:
- Impossible to debug, though technically optimal.
- Everyone is afraid of touching the state machine.
- We can simplify this by introducing turn taking:
- First turn is taken by the lower pubkey.
- Alice: update / commit.
- Bob: revoke and ack.
- If alice wants to go when it’s bob’s turn, she can just send a message.
- Bob can ignore it, or yield and accept it.
- This has been implemented in CLN for LNSymmetry
- It’s less code to not have the ignore message, but for real performance
we’ll want it. Don’t want to suck up all of that latency.
- The easiest way is to re-establish is to upgrade on re-establish.
- If it wasn’t somebody’s turn, you can just do lowest pubkey.
- If it was somebody’s turn, you can just resume it.
- We could also add REVOKE and NACK:
- Right now we have no way to refuse updates.
- Why do we want a NACK?
- You currently have to express to the other side what they can put in
your channel because you can’t handle it if they give you something you
don't’ like (eg, a HTLC below min_htlc).
- You can likely force a break by trying to send things that aren't
allowed, which is a robustness issue.
- We just force close when we get things we don’t allow.
- Could possibly trigger force closes.
- There’s an older proposal called fastball where you send a HTLC and
advise that you’re going to fail it.
- If Alice gets it, she can reply with UNADD.
- If you don’t get it in time,you just go through the regular cycle.
- When you get commitment signed, you could NACK it. This could mean you’re
failing the whole commitment, or just a few HTLCs.
- You can’t fail a HTLC when you’ve sent commitment signed, so you need a
new cycle to clear it out.
- What NACK says is: I’ve ignored all of your updates and I’m progressing
to the next commitment.
- Revoke and NACK is followed by commitment signed where you clear out all
the bad HTLCs, ending that set of updates.
- You have to NACk and then wait for another commitment signature, signing
for the same revocation number.
- Bob never has to hold a HTLC that he doesn't want from Alice on his
commitment.
- This is bad for latency, good for robustness.
- Alice can send whatever she wants, and Bob has a way to reject it.
- There are a whole lot of protocol violations that Alice can force a
force close with, now they can be NACKed.
- This is good news for remote signers where policy has been violate
because we have cases where policy has been violated and our only way right
now is the close the channel.
- You still want Alice to know Bob’s limits so that you can avoid endless
invalid HTLCs.
- Simplified commitment allows us to do things more easily in the protocol.
- When we specced this all out, we didn’t foresee that update fee would
be so complicated, with this we know update fee will be correct.
- If we don’t do this, we have to change update fee?
- Sender of the HTLC adds fee.
- Or fixed fee.
- Even if we have zero fees, don’t we still have HTLC dust problems?
- You can have a bit on update add that says the HTLC is dust.
- You can’t be totally fee agnostic because you have to be able to
understand when to trim HLTCs.
- Even update fee aside, shouldn’t things just be simpler?
- Would a turn based protocol have implications for musig nonces?
- If you’re taking a turn, it’s a session.
- You’d need to have different nonces for different sessions.
- We should probably do this before we make another major change, it
simplifies things.
- Upgrade on re-establish is pretty neat because you can just tell them
what type you’d like.
- This worked very well for CLN getting rid of static remote.
- What about parameter exchange?
- There’s a version of splice that allows you to add new inputs and
outputs.
- Splice no splice which means that you can only make a new commitment
transaction, no on-chain work.
- Seems like you can get something like dynamic commitments with this,
and it’s a subset of splicing.

## Day Three

### Meta Spec Process
- Do we want to re-evaluate the concept of a “living document”?
- It’s only going to get longer.
- As we continue to update, we have two options:
- Remove old text and replace entirely.
- Do an extension and then one day replace.
- If implementing from scratch, what would you want to use?
- Nobody is currently doing this.
- By the time they finished, everything would have move.
- The protocol isn’t actually that modular:
- Except for untouched BOLT-08, which can be read in isolation.
- Other things have tentacles.
- We should endeavor for things to be as modular as possible.
- Back in Adelaide we have a version with a set of features.
- We have not re-visited that discussion.
- Is it possible for us to come up with versions and hold ourselves
accountable to them?
- If we’re going to start having different ideas of what lightning looks
like, versioning helps.
- Versioning is not tied one-to-one to the protocol:
- Features make it less clean because you have a grab bag of features
on top of any “base” version we decide on.
- Does a version imply that we’re implementing in lock step?
- If we do extraction, remove and cleanup, we could say that we’re on
version X with features A/B/C.
- How confident are we that we can pull things out? Only things that are
brand new will be easy to do this with.
- Keeping context in your head is hard, and jumping between documents
breaks up thought.
- Should we fully copy the current document and copy it?
- Not everything is a rewrite, some things are optional.
- What’s our design goal?
- To be able to more easily speak about compatibility.
- To have a readable document for implementation.
- A single living document works when we were all making a unified push,
now it makes less sense:
- You can’t be compliant “by commit” because things are implemented in
different orders.
- We can’t fix that with extensions, they’re hard to keep up to date?
- RFCs work like this, they have replacement ones.
- BOLT 9 can act as a control bolt because it defines features.
- Extensions seem helpful:
- Can contain more rationale.
- You can have spaghetti and ravioli code, these could be raviolo
extensions.
- If everything is an extension BOLT with minimal references, we avoid
the if-else-ey structure we have right now.
- For small things, we can just throw out the old stuff.
- If it were possible to modularize and have working groups, that would be
great but it seems like we’d tread on each other’s toes.
- We must avoid scenarios like vfmanprint:
- The return value says “vfprintf returns -1 on error”
- The next sentence says “this was true until version 2”
- But nobody reads the next sentence.
- Cleanup PRs won’t be looked at, and need to be maintained as new stuff
gets in.
- Eg: legacy onion - we just looked at network use and removed when it was
unused:
- Rip out how you generate them.
- Rip out how you handle them.
- One of the issues with deleting old text is that existing software that
delete the old text is annoying when you run into interop issues on the old
spec version.
- If there are real things to deal with on the network, we must keep them.
- We must reference old commits so that people at least know what was
there and can do git archeology to find out what it used to be.
- We can remove some things today!
- Static remote
- Non-zero fee anchors
- ANYSEGWIT is default (/compulsory)
- Payment secrets / basic MPP.
- Should we have regular cleanups?
- Even if we do, they need review.
- For now, let’s do wholesale replacement to avoid cleanup.
- The proposals folder is nice to know what’s touched by what changes.
- Rationale sections need improvement: sometimes they’re detailed,
sometimes vague.
- Once a feature becomes compulsory on the network we can possibly ignore
it.
- What about things that are neither bolts nor blips - like inbound fees?
- Why does it need to be merged anywhere?
- If it’s an implementation experiment, we can merge it once we’re
convinced it works.
- If we reach a stage where we all agree it should be universally done,
then it should be a bolt.
- This is a BLIP to BOLT path.
- Communication:
- We’re not really using IRC anymore - bring it back!
- We need a canonical medium, recommit to lightning-dev.

### Async Payments/ Trampoline
- Blinded payments are a nice improvement for trampoline because you don’t
know where the recipient is.
- The high level idea is:
- Light nodes only see a small part of the network that they are close
to.
- Recipients only give a few trampolines in the network that they can be
reached via.
- In the onion for the first trampoline, there will be an onion for the
second trampoline.
- You just need to give a trampoline a blinded path and they can do the
rest.
- If you only have one trampoline, they can probably make a good guess
where the payment came from (it’s in the reachable neighborhood).
- Is there a new sync mode for trampoline gossip?
- We’d now need radius-based gossip rather than block based.
- The trust version is just getting this from a LSP.
- In cold bootstrap, you’re probably going to open a channel so you ask
them for gossip.
- Can you split MPP over trampoline? Yes.
- Routing nodes can learn more about the network because they make their
own attempts.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230719/ea8a0ec2/attachment-0001.html>;

📅 Original date posted:2023-01-30 🗒️ Summary of this ...

2023-06-09T13:12:34Z

In reply to nevent1q…k7d7
_________________________

📅 Original date posted:2023-01-30
🗒️ Summary of this message: The Lightning Network community resumed their fortnightly channel jamming mitigation calls for 2023, discussing upgrade mechanisms and upfront fees.
📝 Original message:
Hi list,

On Monday last week we resumed our fortnightly channel jamming
mitigation calls for 2023.

Details for the next call:
* Monday 02/06 @ 18:00 UTC
* https://meet.jit.si/UnjammingLN
* Agenda: https://github.com/ClaraShk/LNJamming/issues/2

# Meeting Summary
This email attempts to summarize the discussion, but is of course
subject to my errors and opinions so the full transcript is available
at [1]. It is (imperfectly) AI generated, so please reach out to myself
or Clara if you'd like clarification for a specific section.

1. Upgrade Mechanisms
The first topic of discussion was around how we would go about
upgrading the network to support an anti-jamming mitigation. Most
solutions share the following characteristics:
* They will likely require a network wide upgrade: when a sender
makes a payment, the full path will need to understand the new
feature bit for it to be used.
* Sending nodes are unlikely to be incentivised to upgrade to
support a channel jamming mitigation, as jamming mitigations incur
additional cost (be it in the form of upfront fees, tokens or some
other mechanism).
* Routing nodes have no way of knowing whether a payment comes from
a node that is not upgraded, or simply doesn't want to pay the
additional cost introduced by a jamming mitigation.

It was generally agreed that the network is still in an early enough
stage that it is fair for relaying nodes to initially upgrade but not
enforce the feature, and later look at forwarding traffic to determine
whether it is appropriate to require the feature. This will require
sending nodes to opt-in to the new feature before the network can
widely enforce it, as no rational routing node will require a feature
which would reduce its ability to serve un-upgraded traffic (unless
under attack, which is not the case at present).

A note was also made that wallets tend to upgrade less frequently, in
part because some providers run forked versions of LN implementations,
so this horizon may be quite long. It was emphasized that this type of
network upgrade would require input from wallets, designers and
application developers anyway, so hopefully by the time we look to
deploy a change there is rough consensus among the Lightning community
already.

2. Upfront Fees
Next, we discussed the draft proposal for upfront fees [2] that
implements the first of a two-part jamming mitigation described in [3].
As it stands, the PR describes the simplest possible way to introduce
upfront fees to the protocol:
* Upfront fees are expressed as a ppm of a channel's success-case
fees, and charged on the outgoing link of a route.
* Nodes can advertise custom upfront fee rates if desired, but to
save gossip bandwidth we assume a network-wide default of 1%.
* Upfront fees accumulate along the route (as htlc payment amounts
do), and are simply pushed to the `to_remote` balance on
`update_add_htlc`.
* Final hop nodes advertise an upfront fee policy in bolt 11 invoices
(or bolt 12 blinded routes) which is sufficiently padded to
obfuscate their location in the route.

The interaction between upfront fees and possible future protocol
changes such as inbound fees and negative fees was briefly discussed,
specifically the case where a node sets low (or negative) fees to
attract traffic then fails payments to accumulate upfront fees. As is,
all implementations’ algorithms optimize for factors other than fees -
specifically avoiding a node once it's produced failures - and we
suspect that careful sender-side behavior will mitigate this risk.
However, it was also acknowledged that a node that attempts this attack
may be able to fool multiple individual senders and still be able to
accumulate some fees.

We then discussed the shortcomings of the "simplest possible" upfront
fees in [2], specifically focused on the following scenario where
nodes receive 1% of their success-case fees as an upfront payment:
* Alice is sending a payment to Dave, who is a popular sink node in
the network.
* Bob -> Carol has low/default routing policy: 1000 msat success
case / 10 msat upfront fee
* Carol -> Dave has high fees: 400,000 msat success case / 4000 msat
upfront fee
* Dave advertises no success case fee (is recipient) / 100 msat of
upfront fee chosen for his invoice.

Alice ------ Bob ------ Carol ------ Dave

Since we need to source all of these funds from the sender (Alice), the
forwarding of funds is as follows:
* Alice pushes 4110 msat to Bob.
* Bob _should_ push 4100 msat to Carol, claiming his 10 msat upfront
fee.
* Carol _should_ push 100 msat to Dave, claiming her 4000 msat
upfront fee.

However, Bob has no real incentive to forward the HTLC to Carol and
push her the 4100 msat because that value is more than the fees he'll
earn from successfully forwarding and settling the HTLC (10 msat
upfront fees + 1000 msat success case fees). It's worth noting that
this type of fee differential already exists in the network today - I
got these example numbers by looking at the fee rates of the Loop
node's peers [4].

There were a few points discussed around this issue:
* Whether we should look into a more complicated approach that
includes a "proof of forward" secret in the next node's onion which
must be supplied to claim the upfront fee.
* Discussion about whether these order of magnitude fee differences
will exist in an efficient market (differing opinions).
* Whether Bob _would_ steal the upfront fee by dropping the payment,
as it'll impact nodes' desire to route through them in future.
* Is there a way to deliver upfront fees to nodes along the route
*without* them adding up along the route (other than the naive
version of sending each hop a payment, which comes with its own
problems)? Seems not, but bright ideas are wanted and welcome!
* That nodes will decide where they fall in the trade-off of setting
upfront fees to cover their opportunity cost for high routing fees
and market pressure to keep these fees low for the sake of
remaining competitive.

A few people mentioned the desire to keep an upfront fee solution as
simple as possible, but generally it seemed that a solution where the
accumulated upfront fees at any point along a route exceed any single
hop's expected success case fees is unacceptable because incentives
are misaligned (even if we can properly attribute errors with [5]).

We touched briefly on a few "proof of forwarding" solutions for
upfront fees:
* Using a locking mechanism that relies on a secret that is provided
by the next node's onion.
* A taproot tree for all HTLCs where each leaf has a subset of HTLCs,
and fees could be incorporated here (summary note: apparently some
work has been done here, but I couldn't find the link).
* If we're confident that a more complicated solution will solve many
problems, then we should pursue them, otherwise a simple solution
is possibly a _good enough_ first step.

There was some discussion about how we want to go about addressing
spamming in the network:
* Is it good enough to deploy something today that just charges a
simple, capped upfront fee (eg 1 sat per hop) that solves jamming
for the current state of the network and allows us to learn
something then iterate?
* Should our first attempt at solving this issue be for a future
steady state where nodes have significant traffic and nodes face
real opportunity cost (in the form of lost revenue) if they are
targeted by a jamming attack?

3. Circuit Breaker Update
* Circuit breaker has a UI now, and some issues are being opened in
the repo so people have at minimum gone through the effort to spin
it up.
* It's still pretty manual, but it does provide node operators with
information about the payment failures on their nodes (not easily
surfaced in many LN UIs).
* Hoping to get insights into stuck HTLCs and spam HTLCs so that we
can learn from real life experience.
* Circuit breaker is generally pushing in the same direction as the
reputation scheme in [3], just a more manual one. We could
possibly use it to visualize the endorsement system in [3] before
enforcing it.
* While circuit breaker isn't necessarily *the* solution to solving
jamming on LN, it provides us with some data points and a
practical way for individual operators to address spamming of
their nodes.

3. Tokens Update
Since our last discussion an architecture document was added to the
proposal [6], details in the mailing list post [7]. The main goal is
to try to dissociate the people paying the fees from those gaining the
benefits from the credentials, so that credentials can be paid by a
LSP (for example).

4. Reputation discussions in LSP Specification
Some attendees have been working on a reputation system for the LSP
specification group [8]. This system is intended to provide standard
metrics about what makes a node good/bad to connect to in the context
of a decentralized marketplace. While this work looks at the problem
from a different perspective, there's a possibility to consolidate
the efforts. It seems particularly useful in the anti-jamming case
where a node has newly connected to you, and needs a "start state"
reputation score. The idea of defining what qualifies as good
behavior in the Lightning Network is useful across the board. The
LSP specification group also has bi-weekly calls, and welcomes
additional participants!

5. Call Cadence
Attendees agreed to continue with calls every two weeks. General
consensus was that we would aim for a 30 minute meeting with the
option to extend to an hour if discussion warrants it.

# What's next
Thanks to everyone who attended last week's meeting, we hope to see
more folks in the next one!

There's a lot to discuss - brave readers who made it all the way to
the bottom of this email will notice a fair number of question marks
scattered about - and we're going to continue to break the problem
down into smaller parts to discuss in the hope of making progress.

Any change to the Lightning Network that aims to address channel
jamming attacks will be a significant (but necessary) change to the
protocol, and it's going to need input from many different
stakeholders. Please reach out here or in private
(carla at chaincode.com / clara at chaincode.com) if you have any questions
/ concerns / comments!

Cheers until next time,
Carla and Clara
(Yes, our names are confusing. No, you won't get used to it)

[1]
https://github.com/ClaraShk/LNJamming/blob/main/meeting-transcripts/23-01-23-transcript.md
[2] https://github.com/lightning/bolts/pull/1052
[3] https://eprint.iacr.org/2022/1454.pdf
[4]
https://mempool.space/lightning/node/021c97a90a411ff2b10dc2a8e32de2f29d2fa49d41bfbb52bd416e460db0747d0d
[5] https://github.com/lightning/bolts/pull/1044
[6] https://github.com/lightning/bolts/pull/1043
[8] https://github.com/BitcoinAndLightningLayerSpecs/lsp/issues/12
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230130/95a70748/attachment.html>;

📅 Original date posted:2023-05-16 📝 Original message: Hi ...

2023-06-09T13:08:51Z

In reply to nevent1q…3dc8
_________________________

📅 Original date posted:2023-05-16
📝 Original message:
Hi all,

Pulling together a few conversation threads here. I’ve also updated
the draft spec PR [1] with a full write up of the reputation scheme
we’re proposing to help clarify open questions.

TL;DR
1. Reputation is tracked locally for each of a node’s peers, there
is *no gossip component*.
2. During a jamming attack, the less active edges of the network will
experience gradually degraded quality of service, but they will be
unaffected in times of peace.
3. Reputation is slow and expensive to build (accumulated through
payment of fees) and fast to degrade, so sudden changes in behavior
are short-lived.
4. Good reputation is always examined relative to a node’s recent
routing activity, so reputation gained cheaply in the past during
low-activity periods can’t be exploited in busier times.

Re [2]
> I'd be very interested in how many repeat interactions nodes get from
individual senders, since that also tells us how much use we can get
out of local-only reputation based systems, and I wouldn't be
surprised if, for large routing nodes, we have sufficient data for
them to make an informed decision, while the edges may be more
vulnerable, but they'd also be used by way fewer senders, and the
impact of an attack would also be proportionally smaller.

I’m unclear on what you mean by “individual senders” here? In our
scheme, nodes only track local reputation for their direct peers so
what matters is their history with all HTLCs a peer has forwarded to
them (not whether they come from repeat senders).

It’s true that nodes that forward fewer HTLCs are less likely to be
able to build a good reputation with very active routing nodes. In the
regular operation of the network, this should have low to no impact on
their activity - they don’t require much from their peers anyway.
During an attack, small and low activity nodes will temporarily be in
competition for large routing nodes’ scarce liquidity and slots, but
will still be able to interact with similar nodes where they have
better chances of building a good reputation.

Re [3]
> I think with some implementation like cln we can write an extension
> an deploy in some nodes, I need to go deeper into it but I can help
> with this. But I would love to discuss how I can help with some
> implementation details.

An experimental data gathering mechanism for CLN would be great! Seems
like lnmetrics would be a good home for it - I’ll follow up with you
when we start working on data collection.

Cheers,
Carla + Clara

[1] https://github.com/lightning/bolts/pull/1071
[2]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2023-May/003944.html
[3]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2023-May/003949.html

On Wed, May 10, 2023 at 7:58 AM Christian Decker <decker.christian at gmail.com>
wrote:

> Hi Antoine,
>
> this is an intrinsic issue with reputation systems, and the main
> reason I'm sceptical w.r.t. their usefulness in lightning.
> Fundamentally any reputation system bases their expectations for the
> future on experiences they made in the past, and they are thus always
> susceptible to sudden behavioral changes (going rogue from a prior
> clean record) and whitewashing attacks (switching identity, abusing
> any builtin bootstrapping method for new users to gain a good or
> neutral reputation before turning rogue repeatedly).
>
> This gets compounded as soon as we start gossiping about reputations,
> since now our decisions are no longer based just on information we can
> witness ourselves, or at least verify its correctness, and as such an
> attacker can most likely "earn" a positive reputation in some other
> part of the world, and then turn around and attack the nodes that
> trusted the reputation shared from those other parts.
>
> I'd be very interested in how many repeat interactions nodes get from
> individual senders, since that also tells us how much use we can get
> out of local-only reputation based systems, and I wouldn't be
> surprised if, for large routing nodes, we have sufficient data for
> them to make an informed decision, while the edges may be more
> vulnerable, but they'd also be used by way fewer senders, and the
> impact of an attack would also be proportionally smaller.
>
> Cheers,
> Christian
>
> On Mon, May 8, 2023 at 10:26 PM Antoine Riard <antoine.riard at gmail.com>
> wrote:
> >
> > Hi *,
> >
> > > Our suggestion is to start simple with a binary endorsement field. As
> > > we learn more, we will be better equipped to understand whether a
> > > more expressive value is required.
> >
> > I think the HTLC endorsement scheme as proposed is still suffering from
> a vulnerability as local reputation can be built up during periods of low
> routing fees, endorsement gained and then abused during periods of high
> routing fees. Therefore, it sounds to me this scheme should aim for some
> reputational transitivity between incoming traffic and outgoing traffic.
> Namely, the acquisition cost of the local reputation should be equal to the
> max timevalue damage that one can inflict on a routing node channel
> accessible from its local counterparty granting this high-level of
> reputation.
> >
> > I don't know if this can be fixed by ensuring permanent link-level
> "gossip" where counterparties along a payment path expose their reputation
> heuristics to guarantee this transitivity, or it's a fundamental issue with
> a point-to-point approach like HTLC endorsement.
> >
> > Opened an issue on the repository to converge on a threat model:
> > https://github.com/ClaraShk/LNJamming/pull/13
> >
> > I still think building data gathering infrastructure for Lightning is
> valuable as ultimately any jamming mitigation will have to adapt its
> upfront fees or reputation acquisition cost in function of HTLC traffic and
> market forces.
> >
> > Looking forward to giving an update on Staking Credentials [0], an
> end-to-end approach to mitigate channel jamming.
> >
> > Best,
> > Antoine
> >
> > [0]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-November/003754.html
> >
> > Le dim. 30 avr. 2023 à 03:57, Carla Kirk-Cohen <kirkcohenc at gmail.com> a
> écrit :
> >>
> >> Hi list,
> >>
> >> Some updates on channel jamming!
> >>
> >> # Next Call
> >> - Monday 01 May @ 15:00 UTC
> >> - https://meet.jit.si/UnjammingLN
> >> - Agenda: https://github.com/ClaraShk/LNJamming/issues/12
> >>
> >> # Data Gathering
> >> During these weekly calls, we've come to agreement that we would like
> >> to gather data about the use of HTLC endorsement and local reputation
> >> tracking for jamming mitigation. A reminder of the full scheme is
> >> included at the end of this email, and covered more verbosely in [1].
> >>
> >> We have a few goals in mind:
> >> - Observe the effect of endorsement in the steady state with
> >> logging-only implementation.
> >> - Gather real-world data for use in future simulation work.
> >> - Experiment with different algorithms for tracking local reputation.
> >>
> >> The minimal changes required to add HTLC endorsement are outlined in
> [2].
> >> Our suggestion is to start simple with a binary endorsement field. As
> >> we learn more, we will be better equipped to understand whether a
> >> more expressive value is required.
> >>
> >> With this infrastructure in place, we can start to experiment with
> >> various local reputation schemes and data gathering, possibly even
> >> externally to LN implementations in projects like circuitbreaker [3].
> >> We'd be interested to hear whether there's any appetite to deploy using
> >> an experimental TLV value?
> >>
> >> # Reputation Scheme
> >> - Each node locally tracks the reputation of its direct neighbors.
> >> - Each node allocates, per its risk tolerance:
> >> - A number of slots reserved for endorsed HTLCs from high reputation
> >> peers.
> >> - A portion of liquidity reserved for endorsed HTLCs from high
> >> reputation peers.
> >> - Forwarding of HTLCs:
> >> - If a HTLC is endorsed by a high reputation peer, it is forwarded
> >> as usual with endorsed = 1.
> >> - Otherwise, it is forwarded with endorsed = 0 if there are slots and
> >> liquidity available for unknown HTLCs.
> >>
> >> Endorsement and reputation are proposed as the first step in a two part
> >> scheme for mitigating channel jamming:
> >> - Reputation for slow jams which are easily detected as misbehavior.
> >> - Unconditional fees for quick jams that are difficult to detect, as
> >> they can always fall under a target threshold.
> >>
> >> Looking forward to discussing further in the upcoming call!
> >>
> >> Best,
> >> Carla and Clara
> >>
> >> [1] https://gist.github.com/carlaKC/be820bb638624253f3ae7b39dbd0e343
> >> [2] https://github.com/lightning/bolts/pull/1071
> >> [3] https://github.com/lightningequipment/circuitbreaker
> >> _______________________________________________
> >> Lightning-dev mailing list
> >> Lightning-dev at lists.linuxfoundation.org
> >> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
> >
> > _______________________________________________
> > Lightning-dev mailing list
> > Lightning-dev at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230516/90159528/attachment.html>;

📅 Original date posted:2023-02-19 📝 Original message: Hi ...

2023-06-09T13:08:12Z

In reply to nevent1q…qmp8
_________________________

📅 Original date posted:2023-02-19
📝 Original message:
Hi List,

A reminder that we've got another jamming call coming up tomorrow.

Monday 20 Feb
19:00 UTC
https://meet.jit.si/UnjammingLN

We'll be talking about local reputation scoring.
Feel free to add additional agenda items here:
https://github.com/ClaraShk/LNJamming/issues/4

Cheers,
Carla and Clara
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230219/5b1741e1/attachment.html>;

📅 Original date posted:2023-02-08 📝 Original message: Hi ...

2023-06-09T13:08:02Z

In reply to nevent1q…htnx
_________________________

📅 Original date posted:2023-02-08
📝 Original message:
Hi list,

Unfortunately we had some technical issues with the recording for
Monday's call so we're going to have to rely on my memory (a severely
corrupted data store). Thankfully, Clara jotted down some notes as well,
but please chime in if you attended and we've missed something out!

Details for next call:
* Monday 20 February
* 18:00 UTC (possibly 19:00, be confirmed in a follow up email)
* https://meet.jit.si/UnjammingLN
* Agenda: https://github.com/ClaraShk/LNJamming/issues/3

# Meeting Summary

1. Proof of Forwarding
We started the call with a discussion of the various "proof of
forwarding" schemes that have been kicked around this mailing list
in the past.

Working of the assumption that upfront fees must always be sourced
from the sender, we ran into similar issues around accumulated fees
and differential rates even in the case where we have some proof of
forward, because nodes can still choose to collaborate to "forward"
a payment.

Given the following topology and conditions:
Alice ------ Bob ------ Carol ------ Dave ----- Evelyn

* Alice is sending a payment to Evenlyn, a popular sink on the network.
* Evenlyn has zero final-hop upfront fees (for simplicity's sake).
* Bob and Carol are malicious actors, each charging 10 msat upfront.
* Dave is an honest node with 4000 msat upfront fees (which is a rational
upfront fee for a channel to a sink node that would have high success
case fees).

Using a proof of forward scheme where Alice places a secret in the
_next_ node's onion which is required to claim upfront fees:
* Bob may claim 4020 msat in upfront fees with a secret from Carol
* Carol may claim 4010 msat in upfront fees with a secret from Dave
* Dave may claim 4000 msat in upfront fees with a secret from Evelyn

In the honest case, this nets out to 10 msat for Bob, 10 msat for
Carol, and 4000 msat for Dave. In the dishonest case, Carol can
disclose the forwarding secret to Bob, allowing them to claim the
accumulated 4020 sat, then fail the payment anyway.

We also spoke about the case where the downstream peer refuses to give
up the secret, but this breakdown in cooperation is likely remedied by
closing your channel. We didn't consider the case where upfront fees
do not accumulate along the route, because this exposes us to a whole
lot of (even worse) draining attacks.

We once again discussed the complexity of this nature of jamming
mitigation, how practical it would be to implement it and whether it
is worthwhile pursuing if it will be an imperfect solution to upfront
fee theft.

2. Upfront Fees + Attributable Errors
The next point of discussion was the combination of upfront fees with
attributable errors [1] to allow senders to more severely punish
nodes that choose to steal upfront fees rather than forward the HTLC
(due to the incentive issues noted in [2] when there are large fee
differentials across the route).

This led to a discussion about how effectively senders can enforce
good upfront fee behavior - a question that remains open. We discussed:
- Strict punishment of nodes that fail payments with upfront fees.
- Sender side protections against selection of routes with incentive
incompatible upfront fees.
- The suboptimal, yet doable, solution of starting with an upper bound
on upfront fees. This has the drawback of not properly compensating
nodes that have a legitimate claim to high upfront fees because they
face high opportunity cost.

3. Experimentation with Circuit Breaker
We discussed the possibility of using circuit breaker[3] in the context
of local reputation (or HTLC endorsement) in two different ways:
(a) To begin surfacing the type of information that end users would
require to decide to endorse a payment, and possibly automating
that decision making.
(b) In conjunction with an experimental TLV to add binary HTLC
endorsement to update_add_htlc, though it was noted that this
would require wider spread adoption of circuit breaker, because
nodes that do not have it installed would not include the TLV on
forwarding. This would also require LND changes to surface TLVs
included with update_add_htlc on the interceptor API.

As always, thank you to all that attended and hope to see ya'll in the
next one!

Best,
Carla + Clara

[1] https://github.com/lightning/bolts/pull/1044
[2]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2023-January/003834.html
[3] https://github.com/lightningequipment/circuitbreaker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230208/c383d310/attachment.html>;

📅 Original date posted:2021-07-28 📝 Original message: Hi ...

2023-06-09T12:40:34Z

In reply to nevent1q…ceg0
_________________________

📅 Original date posted:2021-07-28
📝 Original message:
Hi Rusty,

Thanks for taking a look!

I like the idea of being able to reference a specific message/field
that your node finds problematic. Definitely makes the job simpler
than trying to define every error evar.

> erroneous_message is the message we're complaining about (including
2-byte type), which may be truncated (but must be at least 2 bytes)

Do you think this could be reduced to just the 2-byte message type?
If the reason for including the whole message is so that the error
recipient can match it to the original message, we could possibly just
include the field that we had a problem with as another TLV? That way
we save on having to resend the whole message, and guarantee that we'd
be delivering the problematic field value (it could be cut off if we
truncate the message).

With this approach, there's a chance that we can't match errors back
to their original message if the same channel sends the same message
with the same incorrect field more than once, but I don't see much
practical need to exactly match error to original message in this case,
since we know which field is broken.

> (We should similarly add this TLV to warnings?)

Sounds like a good idea to me!

> But it's worth noting that with the exception of timeouts, these are
all expressible in form "problem is this message, this field". Perhaps
its worth having a special TLV case for timeouts in the message?

If we're confident that the only error that we're going to need that
doesn't follow the "problem/message" structure is timeouts, then a
`timeout` tlv sounds reasonable. If we're also going to add these
fields to warning messages, I'm wondering whether it could be useful
to define an `error_code` enum, where `timeout` is one of the cases?
Perhaps overkill if we're certain that the only value will ever be
`timeout`.

Putting that all together, an update on your proposed solution:
1. `tlv_stream`: `error_tlvs`
2. types:
1. type: 1 (`erroneous_msgtype`)
2. data:
* [`u16`:`type`]
1. type: 3 (`erroneous_fieldnum`)
2. data:
* [`tu64`:`fieldnum`]
1. type: 5 (`suggested_value`)
2. data:
* [`...*byte`:`value`]
1. type: 7 (`erroneous_value`)
2. data:
* [`...*byte`:`value`]
1. type: 9 (`error_code`)
2. data:
* [`u16`:`code`]

Cheers,
- Carla

On Wed, Jul 7, 2021 at 2:36 AM Rusty Russell <rusty at rustcorp.com.au> wrote:

> Carla Kirk-Cohen <kirkcohenc at gmail.com> writes:
> > Hi all,
>
> Hi Carla,
>
> I apologize for not responding to this earlier, but it was
> raised again in the recent spec meeting
> (
> https://lightningd.github.io/meetings/ln_spec_meeting/2021/ln_spec_meeting.2021-07-05-20.06.log.html
> ).
>
> I love the idea of more specific error codes, BTW!
>
> Feedback interleaved:
>
> > Since we shouldn’t have non-ascii values in the error string itself,
> > this can most easily be achieved by adding TLV fields after the
> > data field. In terms of supporting nodes that have not upgraded,
> > we could either include the error code in the data field to cover
> > our bases, or introduce a feature bit so that we know whether
> > to backfill the data field. This gives upgraded nodes an improved
> > quality of life, while leaving older nodes unaffected.
>
> Older nodes should definitely ignore extra fields; it's in the spec and
> we've relied on this to extend messages in the past, so this part is
> easy.
>
> Technically, all defined types are now assumed to have an optional TLV
> appended, since f068dd0d (Bolt 1: Specify that extensions to existing
> messages must use TLV (#754)).
>
> > While we can’t enumerate every possible error, there are quite
> > a few cases in the spec where we can introduce explicit error
> > codes. For the sake of the skim-readers, I’ve left that list at
> > the end of the email.
> >
> > Taking the example of our node receiving an invalid signature for
> > a htlc, a new error would look like this:
>
> I think this is both too much, and not enough.
>
> Too much:
> - Many of these errors are "your implementation is broken", which is
> really not something actionable by the recipient.
> - A lot of work to fill in all these error cases, which will (because
> they're usually impossible) will be untested and broken.
>
> Not enough:
> - Look at the proposal for channel_types, where you would object to the
> channel_type if you don't like it. This would be grouped under
> "Funding params unacceptable", which is actually 99% of errors at this
> point and does not say what the problem is with specificity.
>
> I took a different approach with onion messages[1], where you (optionally)
> specify the field number, even an optional suggested value:
>
> 1. type: 1 (`erroneous_field`)
> 2. data:
> * [`tu64`:`tlv_fieldnum`]
> 1. type: 3 (`suggested_value`)
> 2. data:
> * [`...*byte`:`value`]
> 1. type: 5 (`error`)
> 2. data:
> * [`...*utf8`:`msg`]
>
> In our case, we need to refer to which message (if any) caused the
> error, and we have non-tlv fields, so it can't simply use the tlv field
> number.
>
> Here's my straw proposal:
>
> 1. `tlv_stream`: `error_tlvs`
> 2. types:
> 1. type: 1 (`erroneous_message`)
> 2. data:
> * [`...*byte`:`message`]
> 1. type: 3 (`erroneous_fieldnum`)
> 2. data:
> * [`tu64`:`fieldnum`]
> 1. type: 5 (`suggested_value`)
> 2. data:
> * [`...*byte`:`value`]
>
> erroneous_message is the message we're complaining about (including
> 2-byte type), which may be truncated (but must be at least 2 bytes).
>
> fieldnum is either the 0-based field number (for fixed fields), or the
> number of fixed fields + the tlv type (for tlv fields).
>
> suggested_value is the optional value if we have an idea if what we
> expected / prefer.
>
> > This new kind of error provides us with an error code that tells us
> >
> > exactly what has gone wrong, and metadata pointing to the htlc
> >
> > with an invalid sig. This information can be logged, or stored in a
> >
> > more permanent error store to help diagnose issues in the future.
> >
> > Right now, the spec is pretty strict on error handling [13], indicating
> >
> > that senders/recipients of errors `MUST` fail the channel referenced
> >
> > in the error.
>
> > This isn’t very practical, and I believe that the majority
> > of the impls don’t abide by this instruction.
>
> This was inevitable eventually, but c-lightning deliberately treated
> errors as fatal for a long time so people would *notice* and *report*
> these issues.
>
> To be fair, *LND* didn't treat them as fatal. As so naturally your
> engineers didn't *think* of them as a big deal (and testing didn't show
> it up), so it would send errors for cases which it clearly didn't want
> to close the channel (e.g. peer too slow to respond!).
>
> Hence this PR, which makes these less fatal, and adds warning support:
>
> https://github.com/lightningnetwork/lightning-rfc/pull/834
>
> (We should similarly add this TLV to warnings?)
>
> > Candidates for error codes:
>
> The vast majority of these are "contact your developer, peer says we did
> something illegal". Which is always nice to have more information
> about, but not critital.
>
> The exceptions are:
>
> > Funding Process:
> >
> > * Funding process timeout [2]
> >
> > * Fees out of range [3]
> >
> > * Funding tx spent [11]
> >
> > * Funding params unacceptable (eg, channel too small)
> ...
> > Channel State Machine:
> >
> > * HTLC timeout [4]
> ...
> > Fee Updates
> >
> > * Update fee to low/high [9]
> ...
>
> And BTW this one is misguided:
>
> > * Feature bit required
>
> If Alice says a feature bit is even (compulsory), and Bob doesn't offer
> it, that's on Bob! Alice's behavior here is undefined: she may do a
> courtesy message to Bob, but she may also *assume* it's agreed.
>
> But it's worth noting that with the exception of timeouts, these are all
> expressable in form "problem is this message, this field". Perhaps it's
> worth having a special TLV case for timeouts in the message?
>
> Thanks for starting this ball rolling!
>
> Cheers,
> Rusty.
>
> [1] https://github.com/lightningnetwork/lightning-rfc/pull/759
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20210728/4b5ba70c/attachment-0001.html>;