Nostr notes by

📅 Original date posted:2023-06-03 🗒️ Summary of this ...

2023-06-19T20:19:53+02:00

In reply to nevent1q…nsew
_________________________

📅 Original date posted:2023-06-03
🗒️ Summary of this message: A discussion on the incentive compatibility of a malleated wtxid with annex data stuffed to max transaction size, and the possibility of reviving a proposal to limit the annex.
📝 Original message:
On Sat, Jun 3, 2023 at 2:43 PM Greg Sanders <gsanders87 at gmail.com> wrote:

> No in this case the txid is identical. Only the wtxid is malleated, with
> annex data stuffed to max transaction size.
>

This doesn't sound incentive compatible? While gathering context, I did
find https://github.com/bitcoin/bitcoin/pull/24007. Apparently closed
because of a lack of use case. But perhaps the desire to not limit the
annex can revive that proposal?

Joost

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230603/a01c286a/attachment-0001.html>;

📅 Original date posted:2023-06-08 🗒️ Summary of this ...

2023-06-19T20:19:53+02:00

In reply to nevent1q…23ea
_________________________

📅 Original date posted:2023-06-08
🗒️ Summary of this message: Proposal suggests defining any annex starting with '0' as free-form, allowing immediate utilization and future flexibility while preserving possibility of transitioning to structured format.
📝 Original message:
Hi,

In the proposal below, any annex that begins with `0x00` is defined as
free-form. This isn't the most efficient format though because there is
always one byte lost on signalling. In a future where unstructured annex
data turns out to be the predominant use case, this may be relevant. Also
for very short annexes, the lost byte may weigh relatively heavy.

Without sacrificing future extensions to structured annex data, one could
also store the annex data as is except for the case where the data starts
with `0x21` (or any other 'uncommon' byte). If the data starts with `0x21`,
this byte needs to be repeated first and then followed by the remainder of
the data.

Examples:

Data: [01 02 03]
Encoding [01 02 03]

Data: [21 22 23]
Encoding: [21 21 22 23]

The prefixes [21 (not 21)] are available for future upgrades.

Joost

On Fri, Jun 2, 2023 at 5:00 PM Joost Jager <joost.jager at gmail.com> wrote:

> Hi,
>
> As it stands, the taproot annex is consensus valid but non-standard. The
> conversations around standardization seem to be leaning towards the
> adoption of a flexible Type-Length-Value (TLV) format [1]. There's no doubt
> that this approach has considerable potential. However, settling on an
> exact format may require a significant amount of time.
>
> In the interim, the benefits of making the annex available in a
> non-structured form are both evident and immediate. By allowing developers
> to utilize the taproot annex without delay, we can take advantage of its
> features today, without the need to wait for the finalization of a more
> lengthy standardization process.
>
> With this in view, I am proposing that we define any annex that begins
> with '0' as free-form, without any additional constraints. This strategy
> offers several distinct benefits:
>
> Immediate utilization: This opens the door for developers to make use of
> the taproot annex for a variety of applications straight away, thus
> eliminating the need to wait for the implementation of TLV or any other
> structured format.
>
> Future flexibility: Assigning '0'-beginning annexes as free-form keeps our
> options open for future developments and structure improvements. As we
> forge ahead in determining the best way to standardize the annex, this
> strategy ensures we do not limit ourselves by setting its structure in
> stone prematurely.
>
> Chainspace efficiency: Non-structured data may require fewer bytes
> compared to a probable TLV format, which would necessitate the encoding of
> length even when there's only a single field.
>
> In conclusion, adopting this approach will immediately broaden the
> utilization scope of the taproot annex while preserving the possibility of
> transitioning to a more structured format in the future. I believe this is
> a pragmatic and efficient route, one that can yield substantial benefits in
> both the short and long term.
>
> Joost
>
> [1] https://github.com/bitcoin/bips/pull/1381
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230608/9422bb77/attachment.html>;

📅 Original date posted:2023-06-03 🗒️ Summary of this ...

2023-06-19T20:19:52+02:00

In reply to nevent1q…lptt
_________________________

📅 Original date posted:2023-06-03
🗒️ Summary of this message: Bitcoin developers discuss the removal of the need for a commitment transaction, enabling data inclusion within a single transaction.
📝 Original message:
On Sat, Jun 3, 2023 at 9:49 AM Joost Jager <joost.jager at gmail.com> wrote:

> The removal of the need for a commitment transaction also enables the
> inclusion of data within a single transaction that relies on its own
> transaction identifier (txid). This is possible because the txid
> calculation does not incorporate the annex, where the data would be housed.
> This feature can be beneficial in scenarios that require the emulation of
> covenants through the use of presigned transactions involving an ephemeral
> signer.
>

I think this avoidance of a circular reference is also why LN-Symmetry uses
the annex?

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230603/defc2cc7/attachment.html>;

📅 Original date posted:2023-06-03 🗒️ Summary of this ...

2023-06-19T20:19:52+02:00

In reply to nevent1q…aa73
_________________________

📅 Original date posted:2023-06-03
🗒️ Summary of this message: A potential issue with transaction replacement in Bitcoin could mislead developers into thinking their transactions are immune to replacement. This occurs when a counter-party submits a strictly worse transaction for miners by bloating its weight, not adding fees. Mitigation methods are available.
📝 Original message:
>
> > Depending on policy to mitigate this annex malleability vector could
> mislead developers into believing their transactions are immune to
> replacement, when in fact they might not be.
>
> The issue I'm talking about is where someone's transaction is denied entry
> into the mempool entirely because a counter-party decided to put in a
> strictly worse transaction for miners by bloating the weight of it, not
> adding fees. A strictly worse "API" for paying miners for no gain seems
> like a bad trade to me, especially when there are reasonable methods for
> mitigating this.
>

Just to expand this, an example would be a transaction with inputs A' and
B' signed by two parties A and B. A has a fully signed transaction in
hands, but can't publish it because B created and published an alternative
version of it with a large annex for input B'. Wouldn't miners just accept
A's version because it's fee rate is higher? I am looking at this case
assuming the user has a direct connection to a miner, ignoring any
potential concerns related to p2p transport.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230603/08c8c4df/attachment-0001.html>;

📅 Original date posted:2023-06-03 🗒️ Summary of this ...

2023-06-19T20:19:51+02:00

In reply to nevent1q…md53
_________________________

📅 Original date posted:2023-06-03
🗒️ Summary of this message: Taproot's annex feature eliminates the need for a commit transaction, allowing for the inclusion of data within a single transaction, benefiting scenarios that require the emulation of covenants.
📝 Original message:
Hi David,

On Sat, Jun 3, 2023 at 3:08 AM David A. Harding <dave at dtrt.org> wrote:

> Out of curiosity, what features and benefits are available today? I
> know Greg Sanders wants to use annex data with LN-Symmetry[1], but
> that's dependent on a soft fork of SIGHASH_ANYPREVOUT. I also heard you
> mention that it could allow putting arbitrary data into a witness
> without having to commit to that data beforehand, but that would only
> increase the efficiency of witness stuffing like ordinal inscriptions by
> only 0.4% (~2 bytes saved per 520 bytes pushed) and it'd still be
> required to create an output in order to spend it.
>

Indeed, there's a minor efficiency gain in the reveal transaction witness,
but I think the real advantage is that it eliminates the need to publish
and pay for the commit transaction in the first place. Any spend of a
taproot UTXO can be supplemented with arbitrary data in just a single
transaction.

> Is there some other way to use the annex today that would be beneficial
> to users of Bitcoin?

The removal of the need for a commitment transaction also enables the
inclusion of data within a single transaction that relies on its own
transaction identifier (txid). This is possible because the txid
calculation does not incorporate the annex, where the data would be housed.
This feature can be beneficial in scenarios that require the emulation of
covenants through the use of presigned transactions involving an ephemeral
signer.

For instance, one can establish a time-locked vault using 2-of-2 multisig
presigned transactions in which one of the signers is ephemeral [1]. After
signing, the private key is discarded, leaving only the signature. To
ensure the signature is never lost, it can be stored as a backup in the
annex of the transaction that the presigned transaction spends. Such an
operation would not be possible with a commit/reveal inscription.

[1] https://github.com/LedgerHQ/app-bitcoin-new/issues/153
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230603/76c769ae/attachment.html>;

📅 Original date posted:2023-06-03 🗒️ Summary of this ...

2023-06-19T20:19:51+02:00

In reply to nevent1q…heg7
_________________________

📅 Original date posted:2023-06-03
🗒️ Summary of this message: The issue with BIP341/342 signature hashes not covering other inputs' annex fields in a coinjoin scenario could lead to griefing. Solutions such as making annex usage opt-in or imposing size limitations may not align with economic incentives and could mislead developers into believing their transactions are immune to replacement.
📝 Original message:
HI Greg,

On Sat, Jun 3, 2023 at 3:14 AM Greg Sanders <gsanders87 at gmail.com> wrote:

> Attempting to summarize the linked PR:
>
> I think the biggest remaining issue to this kind of idea, which is why I
> didn't propose it for mainnet,
> is the fact that BIP341/342 signature hashes do not cover *other* inputs'
> annex fields, which we
> briefly discussed here
> https://github.com/bitcoin-inquisition/bitcoin/pull/22#discussion_r1143382264
> .
>
> This means that in a coinjoin like scenario, even if the other joining
> parties prove they don't have any
> crazy script paths, a malicious party can make the signed transaction into
> a maximum sized transaction
> package, causing griefing. The mitigation in the PR I linked was to limit
> it to 126 bytes, basically punting
> on the problem by making the grief vector small. Another solution could be
> to make annex usage "opt-in"
> by requiring all inputs to commit to an annex to be relay-standard. In
> this case, you've opted into a possible
> vector, but at least current usage patterns wouldn't be unduly affected.
> For those who opt-in, perhaps the first
> order of business would be to have a field that limits the total
> transaction weight, by policy only?
>
> Some logs related to that here:
> https://gist.github.com/instagibbs/7406931d953fd96fea28f85be50fc7bb
>
> Related discussion on possible BIP118 modifications to mitigate this in
> tapscript-spending circumstances:
> https://github.com/bitcoin-inquisition/bitcoin/issues/19
>

While solutions such as making annex usage opt-in or imposing size
limitations may initially appear effective, they may also inadvertently
foster a false sense of security, as they lack alignment with economic
incentives.

Relying solely on policy enforcement merely transfers responsibility to the
miners, without necessarily aligning their incentives with the broader
network health. This situation is reminiscent of the challenges encountered
with opt-in rbf. Despite signaling for non-replaceability, miners began
accepting replacements probably due to the enticing higher fee incentives.
At least that's how I picked up this development. Businesses that relied on
zero-confirmation payments were unexpectedly affected, leading to
undesirable outcomes.

While we can define policy rules, miners will ultimately operate in a
manner that maximizes their profits. Consequently, if a miner identifies an
opportunity to bolster their fees by replacing an annex transaction,
they're likely to seize it, regardless of any policy rules. This might not
be readily apparent currently with a limited number of pools dominating
block production, but it is my hope that mining will be more decentralized
in the future.

Depending on policy to mitigate this annex malleability vector could
mislead developers into believing their transactions are immune to
replacement, when in fact they might not be. This potential misalignment
could result in developers and businesses constructing systems based on
assumptions that could be compromised in the future, mirroring the
situation that unfolded with zero-confirmation payments and rbf.

It may thus be more prudent to permit the utilization of the annex without
restrictions, inform developers of its inherent risks, and acknowledge that
Bitcoin, in its present state, might not be ideally suited for certain
types of applications?

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230603/4b064f42/attachment.html>;

📅 Original date posted:2023-06-05 🗒️ Summary of this ...

2023-06-15T02:33:40+02:00

In reply to nevent1q…enkj
_________________________

📅 Original date posted:2023-06-05
🗒️ Summary of this message: The author suggests using out-of-band relay as a more efficient way to get transaction packages to miners while p2p package relay is under development.
📝 Original message:
Hi,

Before starting, I would like to state that I do not necessarily support
the implementation of the idea I'm about to present, but I think it's worth
mentioning as it might inspire different use cases or provoke some debate.
I believe that out-of-band relay is a more preferable and efficient way to
get transaction packages to miners while p2p package relay is under
development.

Let's consider the situation where we have a parent transaction A that pays
0 sat/b (for example a lightning commitment transaction), and a fee bumping
child transaction B. These transactions currently cannot reach miners.

We can, however, conceive a workaround. Let's introduce a third transaction
C, crafted to contain the raw transactions A and B in a taproot annex. A
commit/reveal style inscription could also be used instead, but I think it
would be more complicated and less efficient.

To ensure propagation, transaction C would pay sufficient fees. Also it
would use at least one of the same fee contributing inputs as transaction
B, but obviously not any inputs from A.

Miners, upon receiving transaction C, could detect the embedded
transactions A and B in the annex and immediately submit them to their
mempool as a transaction package. This transaction package (A+B) would then
replace transaction C and could be included in a block for mining.

It's of course important to ensure that the combined package of A+B is more
attractive to miners than the C transaction. The extra weight of the
embedded transactions in C helps with this. Also it is worth noting that
the fees for C will never be paid because it has been replaced. Thus there
are no extra costs for using this package relay scheme, unless perhaps the
weight of A+B is very low and B needs to pay a higher fee rate than
necessary to ensure replacement of C.

If not all miners adopt this incentive-compatible replacement, there's a
chance transaction C ends up being mined. This is likely less probable if
the fee rate for C is kept to a minimum. If transaction C is indeed mined,
the operation can be retried with a modified B and C, though the fees paid
for the initial transaction C would be forfeited.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230605/0468684e/attachment.html>;

📅 Original date posted:2023-02-15 🗒️ Summary of this ...

2023-06-09T15:12:43+02:00

In reply to nevent1q…ddpq
_________________________

📅 Original date posted:2023-02-15
🗒️ Summary of this message: Lightning network performance depends on the type of payment flows, with fast payment being favored for end-users and cheap fees for remittance payments. Adding latency as a criteria for pathfinding construction and using forward-error-correction code on top of MPP are potential solutions. More liquidity may be used by routing nodes to serve tailored HTLC requests, but this should be rewarded by higher routing fees. Careful policy rules design and upgradeability are necessary to prevent higher fees for end-users.
📝 Original message:
>
> I think the performance question depends on the type of payment flows
> considered. If you're an
> end-user sending a payment to your local Starbucks for coffee, here fast
> payment sounds the end-goal.
> If you're doing remittance payment, cheap fees might be favored, and in
> function of those flows you're
> probably not going to select the same "performant" routing nodes. I think
> adding latency as a criteria for
> pathfinding construction has already been mentioned in the past for LDK
> [0].
>

My hopes are that eventually lightning nodes can run so efficient that in
practice there is no real trade-off anymore between cost and speed. But of
course hard to say how that's going to play out. I am all for adding
latency as an input to pathfinding. Attributable errors should help with
that too.

> Or there is the direction to build forward-error-correction code on top of
> MPP, like in traditional
> networking [1]. The rough idea, you send more payment shards than the
> requested sum, and then
> you reveal the payment secrets to the receiver after an onion
> interactivity round to finalize payment.
>

This is not very different from payment pre-probing is it? So try a larger
set of possible routes simultaneously and when one proves to be open, send
the real payment across that route. Of course a balance may have shifted in
the mean time, but seems unlikely enough to prevent the approach from being
usable. The obvious downside is that the user needs more total liquidity to
have multiple htlcs outstanding at the same time. Nevertheless an
interesting way to reduce payment latency.

> At the end of the day, we add more signal channels between HTLC senders
> and the routing
> nodes offering capital liquidity, if the signal mechanisms are efficient,
> I think they should lead
> to better allocation of the capital. So yes, I think more liquidity might
> be used by routing nodes
> to serve finely tailored HTLC requests by senders, however this liquidity
> should be rewarded
> by higher routing fees.
>

This is indeed part of the idea. By signalling HA, you may not only attract
more traffic, but also be able to command a higher fee.

> I think if we have lessons to learn on policy rules design and deployment
> on the base-layer
> (the full-rbf saga), it's to be careful in the initial set of rules, and
> how we ensure smooth
> upgradeability, from one version to another. Otherwise the re-deployment
> cost towards
> the new version might incentive the old routing node to stay on the
> non-optimal versions,
> and as we have historical buckets in routing algorithms, or preference for
> older channels,
> this might lead the end-user to pay higher fees, than they could access to.
>

I see the parallel, but also it seems that we have this situation already
today on lightning. Senders apply penalties and routing nodes need to make
assumptions about how they are penalised. Perhaps more explicit signalling
can actually help to reduce the degree of uncertainty as to how a routing
nodes is supposed to perform to keep senders happy?

> This is where the open question lies to me - "highly available" can be
> defined with multiple
> senses, like fault-tolerance, latency processing, equilibrated liquidity.
> And a routing node might
> not be able to optimize its architecture for the same end-goal (e.g more
> watchtower on remote
> host probably increases the latency processing).
>

Yes, good point. So maybe a few more bits to signal what a sender may
expect from a channel exactly?

> > Without shadow channels, it is impossible to guarantee liquidity up to
> the channel capacity. It might make sense for senders to only assume high
> > availability for amounts up to `htlc_maximum_msat`.
>
> As a note, I think "senders assumption" should be well-documented,
> otherwise there will be
> performance discrepancies between node implementations or even versions.
> E.g, an upgraded
> sender penalizing a node for the lack of shadow/parallel channels
> fulfilling HTLC amounts up to
> `htlc_maximum_msat`.
>

Well documented, or maybe even explicit in the name of the feature bit. For
example `htlc_max_guaranteed`.

> I think signal availability should be explicit rather than implicit. Even
> if it's coming with more
> gossip bandwidth data consumed. I would say for bandwidth performance
> management, relying
> on new gossip messages, where they can be filtered in function of the
> level of services required
> is interesting.
>

In terms of implementation, I think this kind of signalling is easier as an
extension of `channel_update`, but it can probably work as a separate
message too.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230215/c17c1304/attachment-0001.html>;

📅 Original date posted:2023-02-15 🗒️ Summary of this ...

2023-06-09T15:12:41+02:00

In reply to nevent1q…5w2x
_________________________

📅 Original date posted:2023-02-15
🗒️ Summary of this message: Running a high availability channel on the Lightning Network requires maintaining outbound liquidity by opening new channels, rebalancing, or swapping in. Publicly visible data may not be representative of the network as a whole.
📝 Original message:
>
> But how do you decide to set it without a credit relationship? Do I
> measure my channel and set the
>
bit because the channel is "usually" (at what threshold?) saturating in the
> inbound direction? What
> happens if this changes for an hour and I get unlucky? Did I just screw
> myself?
>

As a node setting the flag, you'll have to make sure you open new channels,
rebalance or swap-in in time to maintain outbound liquidity. That's part of
the game of running an HA channel.

> > How can you be sure about this? This isn't publicly visible data.
>
> Sure it is! https://river.com/learn/files/river-lightning-report.pdf

Some operators publish data, but are the experiences of one of the most
well connected (custodial) nodes representative for the network as a whole
when evaluating payment success rates? In the end you can't know what's
happening on the lightning network.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230215/1fb6f297/attachment.html>;

📅 Original date posted:2023-02-14 🗒️ Summary of this ...

2023-06-09T15:12:40+02:00

In reply to nevent1q…40f2
_________________________

📅 Original date posted:2023-02-14
🗒️ Summary of this message: The Lightning Network should not create two classes of nodes based on availability, according to a post on the Lightning-dev mailing list. The post argued that communicating reliability up front would not work, and penalising nodes after the fact would be ineffective. The author also said relying on third-party information to track reliability would be problematic. Instead, the post suggested using high-availability signalling to make local reliability tracking more efficient.
📝 Original message:
Hi Christian,

> And after all this rambling, let's get back to the topic at hand: I
> don't think enshrining the differences of availability in the protocol,
> thus creating two classes of nodes, is a desirable
> feature.

Yes so to be clear, the HA signaling is not on the node level but on the
channel level. So each node can decide per channel whether they want to
potentially attract additional traffic at the cost of severe penalties (or
avoidance if you want to use a different wording) if the channel can't be
used. They can still maintain a set of less reliable channels along side.

> Communicating up-front that I intend to be reliable does
> nothing, and penalizing after the fact isn't worth much due to the
> repeat interactions issue.

I think it is currently quite common for pathfinders to try another channel
of the same node for the payment at hand. Or re-attempt the same channel
for a future payment to the same destination. I understand the repeat
interactions issue, but not sure about the extent to which it applies to
lightning in practice. A think a common pattern for payments in general is
to pay to the same destinations repeatedly, for example for a daily coffee.

> It'd be even worse if now we had to rely on a
> third party to aggregate and track the reliability, in order to get
> enough repeat interactions to build a good model of their liquidity,
> since we're now back in the hearsay world, and the third party can feed
> us wrong information to maximize their profits.
>

Yes, using 3rd party info seems difficult. As mentioned in my reply to
Matt, the idea of HA signaling is to make local reliability tracking more
efficient so that it becomes less likely that senders need to rely on
external aggregators for their view on the network.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230214/a7f355e6/attachment-0001.html>;

📅 Original date posted:2023-02-14 🗒️ Summary of this ...

2023-06-09T15:12:40+02:00

In reply to nevent1q…cp2u
_________________________

📅 Original date posted:2023-02-14
🗒️ Summary of this message: The Lightning Network should not be built out of high reliability nodes to avoid centralization pressure, according to a developer. Nodes should measure the network and route over paths that are reliable for what they want, robustly over an unreliable network.
📝 Original message:
Hi Matt,

If nodes start aggressively preferring routes through nodes that reliably
> route payments (which I believe lnd already does, in effect, to some large
> extent), they should do so by measurement, not signaling.
>

The signaling is intended as a way to make measurement more efficient. If a
node signals that a particular channel is HA and it fails, no other
measurements on that same node need to be taken by the sender. They can
skip the node altogether for a longer period of time.

> In practice, many channels on the network are “high availability” today,
> but only in one direction (I.e. they aren’t regularly spliced/rebalanced
> and are regularly unbalanced). A node strongly preferring a high payment
> success rate *should* prefer such a channel, but in your scheme would not.
>

This shouldn't be a problem, because the HA signaling is also directional.
Each end can decide independently on whether to add the flag for a
particular channel.

> This ignores the myriad of “at what threshold do you signal HA” issues,
> which likely make such a signal DOA, anyway.
>

I think this is a product of sender preference for HA channels and the
severity of the penalty if an HA channel fails. Given this, routing nodes
will need to decide whether they can offer a service level that increases
their routing revenue overall if they would signal HA. It is indeed
dynamic, but I think the market is able to work it out.

> Finally, I’m very dismayed at this direction in thinking on how ln should
> work - nodes should be measuring the network and routing over paths that it
> thinks are reliable for what it wants, *robustly over an unreliable
> network*. We should absolutely not be expecting the lightning network to be
> built out of high reliability nodes, that creates strong centralization
> pressure. To truly meet a “high availability” threshold, realistically,
> you’d need to be able to JIT 0conf splice-in, which would drive lightning
> to actually being a credit network.
>

Different people can have different opinions about how ln should work, that
is fine. I see a trade-off between the reliability of the network and the
barrier of entry, and I don't think the optimum is on one of the ends of
the scale.

> With reasonable volume, lightning today is very reliable and relatively
> fast, with few retries required. I don’t think we need to change anything
> to fix it. :)
>

How can you be sure about this? This isn't publicly visible data.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230214/da656e7a/attachment.html>;

📅 Original date posted:2023-02-13 🗒️ Summary of this ...

2023-06-09T15:12:38+02:00

In reply to nevent1q…4n9v
_________________________

📅 Original date posted:2023-02-13
🗒️ Summary of this message: The Lightning Network may require routing nodes to operate flawlessly or face penalties, to ensure competitiveness with centralized payment systems. A proposed solution is to extend the `channel_update` field with a new bit to signal `highly_available` channels, allowing payers to prefer them during pathfinding, but with a stronger penalty for failure. This creates an opportunity for routing nodes to attract more traffic but also comes with the responsibility to deliver.
📝 Original message:
Hi,

For a long time I've held the expectation that eventually payers on the
lightning network will become very strict about node performance. That they
will require a routing node to operate flawlessly or else apply a hefty
penalty such as completely avoiding the node for an extended period of time
- multiple weeks. The consequence of this is that routing nodes would need
to manage their liquidity meticulously because every failure potentially
has a large impact on future routing revenue.

I think movement in this direction is important to guarantee
competitiveness with centralised payment systems and their (at least
theoretical) ability to process a payment in the blink of an eye. A
lightning wallet trying multiple paths to find one that works doesn't help
with this.

A common argument against strict penalisation is that it would lead to less
efficient use of capital. Routing nodes would need to maintain pools of
liquidity to guarantee successes all the time. My opinion on this is that
lightning is already enormously capital efficient at scale and that it is
worth sacrificing a slight part of that efficiency to also achieve the
lowest possible latency.

This brings me to the actual subject of this post. Assuming strict
penalisation is good, it may still not be ideal to flip the switch from one
day to the other. Routing nodes may not offer the required level of service
yet, causing senders to end up with no nodes to choose from.

One option is to gradually increase the strength of the penalties, so that
routing nodes are given time to adapt to the new standards. This does
require everyone to move along and leaves no space for cheap routing nodes
with less leeway in terms of liquidity.

Therefore I am proposing another way to go about it: extend the
`channel_update` field `channel_flags` with a new bit that the sender can
use to signal `highly_available`.

It's then up to payers to decide how to interpret this flag. One way could
be to prefer `highly_available` channels during pathfinding. But if the
routing node then returns a failure, a much stronger than normal penalty
will be applied. For routing nodes this creates an opportunity to attract
more traffic by marking some channels as `highly_available`, but it also
comes with the responsibility to deliver.

Without shadow channels, it is impossible to guarantee liquidity up to the
channel capacity. It might make sense for senders to only assume high
availability for amounts up to `htlc_maximum_msat`.

A variation on this scheme that requires no extension of `channel_update`
is to signal availability implicitly through routing fees. So the more
expensive a channel is, the stronger the penalty that is applied on failure
will be. It seems less ideal though, because it could disincentivize cheap
but reliable channels on high traffic links.

The effort required to implement some form of a `highly_available` flag
seem limited and it may help to get payment success rates up. Interested to
hear your thoughts.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230213/adf0c913/attachment.html>;

📅 Original date posted:2023-02-14 📝 Original message: Hi ...

2023-06-09T15:08:06+02:00

In reply to nevent1q…q4nn
_________________________

📅 Original date posted:2023-02-14
📝 Original message:
Hi Matt,

If nodes start aggressively preferring routes through nodes that reliably
> route payments (which I believe lnd already does, in effect, to some large
> extent), they should do so by measurement, not signaling.
>

The signaling is intended as a way to make measurement more efficient. If a
node signals that a particular channel is HA and it fails, no other
measurements on that same node need to be taken by the sender. They can
skip the node altogether for a longer period of time.

> In practice, many channels on the network are “high availability” today,
> but only in one direction (I.e. they aren’t regularly spliced/rebalanced
> and are regularly unbalanced). A node strongly preferring a high payment
> success rate *should* prefer such a channel, but in your scheme would not.
>

This shouldn't be a problem, because the HA signaling is also directional.
Each end can decide independently on whether to add the flag for a
particular channel.

> This ignores the myriad of “at what threshold do you signal HA” issues,
> which likely make such a signal DOA, anyway.
>

I think this is a product of sender preference for HA channels and the
severity of the penalty if an HA channel fails. Given this, routing nodes
will need to decide whether they can offer a service level that increases
their routing revenue overall if they would signal HA. It is indeed
dynamic, but I think the market is able to work it out.

> Finally, I’m very dismayed at this direction in thinking on how ln should
> work - nodes should be measuring the network and routing over paths that it
> thinks are reliable for what it wants, *robustly over an unreliable
> network*. We should absolutely not be expecting the lightning network to be
> built out of high reliability nodes, that creates strong centralization
> pressure. To truly meet a “high availability” threshold, realistically,
> you’d need to be able to JIT 0conf splice-in, which would drive lightning
> to actually being a credit network.
>

Different people can have different opinions about how ln should work, that
is fine. I see a trade-off between the reliability of the network and the
barrier of entry, and I don't think the optimum is on one of the ends of
the scale.

> With reasonable volume, lightning today is very reliable and relatively
> fast, with few retries required. I don’t think we need to change anything
> to fix it. :)
>

How can you be sure about this? This isn't publicly visible data.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230214/da656e7a/attachment.html>;

📅 Original date posted:2023-02-13 📝 Original message: Hi, ...

2023-06-09T15:08:03+02:00

In reply to nevent1q…7c44
_________________________

📅 Original date posted:2023-02-13
📝 Original message:
Hi,

For a long time I've held the expectation that eventually payers on the
lightning network will become very strict about node performance. That they
will require a routing node to operate flawlessly or else apply a hefty
penalty such as completely avoiding the node for an extended period of time
- multiple weeks. The consequence of this is that routing nodes would need
to manage their liquidity meticulously because every failure potentially
has a large impact on future routing revenue.

I think movement in this direction is important to guarantee
competitiveness with centralised payment systems and their (at least
theoretical) ability to process a payment in the blink of an eye. A
lightning wallet trying multiple paths to find one that works doesn't help
with this.

A common argument against strict penalisation is that it would lead to less
efficient use of capital. Routing nodes would need to maintain pools of
liquidity to guarantee successes all the time. My opinion on this is that
lightning is already enormously capital efficient at scale and that it is
worth sacrificing a slight part of that efficiency to also achieve the
lowest possible latency.

This brings me to the actual subject of this post. Assuming strict
penalisation is good, it may still not be ideal to flip the switch from one
day to the other. Routing nodes may not offer the required level of service
yet, causing senders to end up with no nodes to choose from.

One option is to gradually increase the strength of the penalties, so that
routing nodes are given time to adapt to the new standards. This does
require everyone to move along and leaves no space for cheap routing nodes
with less leeway in terms of liquidity.

Therefore I am proposing another way to go about it: extend the
`channel_update` field `channel_flags` with a new bit that the sender can
use to signal `highly_available`.

It's then up to payers to decide how to interpret this flag. One way could
be to prefer `highly_available` channels during pathfinding. But if the
routing node then returns a failure, a much stronger than normal penalty
will be applied. For routing nodes this creates an opportunity to attract
more traffic by marking some channels as `highly_available`, but it also
comes with the responsibility to deliver.

Without shadow channels, it is impossible to guarantee liquidity up to the
channel capacity. It might make sense for senders to only assume high
availability for amounts up to `htlc_maximum_msat`.

A variation on this scheme that requires no extension of `channel_update`
is to signal availability implicitly through routing fees. So the more
expensive a channel is, the stronger the penalty that is applied on failure
will be. It seems less ideal though, because it could disincentivize cheap
but reliable channels on high traffic links.

The effort required to implement some form of a `highly_available` flag
seem limited and it may help to get payment success rates up. Interested to
hear your thoughts.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230213/adf0c913/attachment.html>;

📅 Original date posted:2022-10-19 📝 Original message: Hi ...

2023-06-09T15:07:06+02:00

In reply to nevent1q…zqqj
_________________________

📅 Original date posted:2022-10-19
📝 Original message:
Hi list,

I wanted to get back to a long-standing issue in Lightning: gaps in error
attribution. I've posted about this before back in 2019 [1].

Error attribution is important to properly penalize nodes after a payment
failure occurs. The goal of the penalty is to give the next attempt a
better chance at succeeding. In the happy failure flow, the sender is able
to determine the origin of the failure and penalizes a single node or pair
of nodes.

Unfortunately it is possible for nodes on the route to hide themselves. If
they return random data as the failure message, the sender won't know where
the failure happened. Some senders then penalize all nodes that were part
of the route [4][5]. This may exclude perfectly reliable nodes from being
used for future payments. Other senders penalize no nodes at all [6][7],
which allows the offending node to keep the disruption going.

A special case of this is a final node sending back random data. Senders
that penalize all nodes will keep looking for alternative routes. But
because each alternative route still ends with that same final node, the
sender will ultimately penalize all of its peers and possibly a lot of the
rest of the network too.

I can think of various reasons for exploiting this weakness. One is just
plain grievance for whatever reason. Another one is to attract more traffic
by getting competing routing nodes penalized. Or the goal could be to
sufficiently mess up reputation tracking of a specific sender node to make
it hard for that node to make further payments.

Related to this are delays in the path. A node can delay propagating back a
failure message and the sender won't be able to determine which node did
it.

The link at the top of this post [1] describes a way to address both
unreadable failure messages as well as delays by letting each node on the
route append a timestamp and hmac to the failure message. The great
challenge is to do this in such a way that nodes don’t learn their position
in the path.

I'm revisiting this idea, and have prototyped various ways to implement it.
In the remainder of this post, I will describe the variant that I thought
works best (so far).

# Failure message format

The basic idea of the new format is to let each node (not just the error
source) commit to the failure message when it passes it back by adding an
hmac. The sender verifies all hmacs upon receipt of the failure message.
This makes it impossible for any of the nodes to modify the failure message
without revealing that they might have played a part in the modification.
It won’t be possible for the sender to pinpoint an exact node, because
either end of a communication channel may have modified the message.
Pinpointing a pair of nodes however is good enough, and is commonly done
for regular onion failures too.

On the highest level, the new failure message consists of three parts:

`message` (var len) | `payloads` (fixed len) | `hmacs` (fixed len)

* `message` is the standard onion failure message as described in [2], but
without the hmac. The hmac is now part of `hmacs` and doesn't need to be
repeated.

* `payloads` is a fixed length array that contains space for each node
(`hop_payload`) on the route to add data to return to the sender. Ideally
the contents and size of `hop_payload` is signaled so that future
extensions don’t require all nodes to upgrade. For now, we’ll assume the
following 9-byte format:

`is_final` (1 byte) | `duration` (8 bytes)

`is_final` indicates whether this node is the failure source. The sender
uses `is_final` to determine when to stop the decryption/verification
process.

`duration` is the time in milliseconds that the node held the htlc. By
observing the series of reported durations, the sender is able to pinpoint
a delay down to a pair of nodes.

The `hop_payload` is repeated 27 times (the maximum route length).

Every hop shifts `payloads` 9 bytes to the right and puts its own
`hop_payload` in the 9 left-most bytes.

* `hmacs` is a fixed length array where nodes add their hmacs as the
failure message travels back to the sender.

To keep things simple, I'll describe the format as if the maximum route
length was only three hops (instead of 27):

`hmac_0_2` | `hmac_0_1`| `hmac_0_0`| `hmac_1_1`| `hmac_1_0`| `hmac_2_0`

Because nodes don't know their position in the path, it's unclear to them
what part of the failure message they are supposed to include in the hmac.
They can't just include everything, because if part of that data is deleted
later (to keep the message size fixed) it opens up the possibility for
nodes to blame others.

The solution here is to provide hmacs for all possible positions. The
last node that updated `hmacs` added `hmac_0_2`, `hmac_0_1` and `hmac_0_0`
to the block. Each hmac corresponds to a presumed position in the path,
where `hmac_0_2` is for the longest path (2 downstream hops) and `hmac_0_0`
for the shortest (node is the error source).

`hmac_x_y` is the hmac added by node x (counted from the node that is
currently handling the failure message) assuming that this node is y hops
away from the final node.

Before an hop adds its hmacs, it first deletes some of the previous hmacs.
This keeps the failure message at a fixed length. The removed hmacs are the
ones that cannot be useful anymore. If node 0 adds itself, the former node
0 (now node 1) cannot be at the first position anymore. The former node 1
(now node 2) cannot be at the second position anymore. The former node 2
cannot be the source of the error anymore and isn’t represented in the
failure message any longer. The corresponding hmacs (the now non-existent
`hmac_0_2`, `hmac_1_1` and `hmac_2_0`) are deleted by node 0.

Deleting the useless data reduces the number of hmacs (and roughly the
total failure message size) to half.

The delete operation transform the fields above to:

<empty> | <empty> | <empty> | `hmac_0_1`| `hmac_0_0`| `hmac_1_0`

The exact data that is included in each hmac is:
* `message`
* the node’s own `hop_payload` and a set of downstream `hop_payload`s,
depending on assumed position
* a set of downstream node hmacs, depending on assumed position

For example `hmac_0_1` is based on:

`message` | `hop_payload[0]` | `hop_payload[1]` | `hmac_1_0`

If the node that is currently handling the failure message is one hop away
from the final node, it needs to cover its own `hop_payload[0]`, the final
node `hop_payload[1]` and the final node hmac `hmac_1_0`.

A longer path is committed to in `hmac_0_2`:

`message` | `hop_payload[0]` | `hop_payload[1]` | `hop_payload[2]` |
`hmac_1_1` | `hmac_2_0`

The current node is two hops away from the final node. It needs to cover
its own `hop_payload[0]` as well as `hop_payload[1]` and `hop_payload[2]`
for the next and final hops. Additionally it covers the next hop `hmac_1_1`
and final hop `hmac_2_0`, which correspond to the positions of those nodes
in the path that is assumed for `hmac_0_2`.

With this information, the sender is able to verify the longest chain of
hmacs until it encounters a `hop_payload` with `is_final` set.

If any of the nodes messes with any byte in the failure message, the sender
is always able to determine a pair of nodes that the offending node is part
of. This statement can be verified through reasoning, but to be sure I also
tested it with code. I’ve simulated a malicious node that modifies a byte
of the failure message at index x and observed the error source as
determined by the sender. For every x, the sender reports the same correct
pair.

# Size

The obvious downside of the scheme above is the size. Given a maximum of 27
hops, the `hmacs` block contains 27+26+25+...+1=378 hmacs of 32 bytes each.
This makes for a total size of 12 KB.

It could be the case though that it is not possible to devise a more
compact scheme that also preserves the existing privacy guarantees. I know
that smart people have spent time on this problem, but nonetheless no
better solution has come up in the past years. A proof of its non-existence
would be interesting for sure.

I personally think the size increase is justified to fix this vulnerability
in Lightning. Also if failures are expected to become more rare going
forward, size becomes less relevant to the overall operation of the network.

Another option is to reduce the maximum number of hops. It is questionable
whether 27 hops are really needed in practice, and such long routes also
contribute to latency and capital lock up. If for example the new failure
message could only be used with routes up to 10 hops, the total number of
hmacs would drop from 378 to 55. This makes for a total message size of
about 2 KB.

# Signaling

For backwards compatibility nodes need to know what algorithm they should
run to generate or transform the failure message. This can be signaled by
the sender via a tlv onion field. A failure message format signaling
mechanism is also discussed in the context of long failure messages [3].
The failure message described in this post could be just another version.

Additionally, intermediate nodes need to advertise their capability to
transform the new format through a feature bit.

# Delayed successes

It’s not just failures that can be delayed. Successes can too. In that
case, there is no failure message to improve. It could be an option to add
the same `payloads` and `hmacs` blocks to the `update_fulfill_htlc` message.

[1]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2019-June/002015.html
[2]
https://github.com/lightning/bolts/blob/master/04-onion-routing.md#returning-errors
[3] https://github.com/lightning/bolts/pull/1021
[4]
https://github.com/lightningnetwork/lnd/blob/4fbd608b734f348d7e79fbfc7feaecc5c6c33a90/routing/result_interpretation.go#L419
[5]
https://github.com/ACINQ/eclair/blob/a0433aa0c027c9be618c5afe18e7f91642a7f372/eclair-core/src/main/scala/fr/acinq/eclair/payment/PaymentEvents.scala#L221
[6]
https://github.com/ElementsProject/lightning/blob/62bfed9a8df8731be44ba4e86afb08a5d28a4442/plugins/libplugin-pay.c#L1461
[7]
https://github.com/lightningdevkit/rust-lightning/blob/e61f3a238a70cbac87209e223b7c396108a49b97/lightning-invoice/src/payment.rs#L682
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20221019/759cba0c/attachment.html>;

📅 Original date posted:2022-07-10 📝 Original message: On ...

2023-06-09T15:06:37+02:00

In reply to nevent1q…k4ge
_________________________

📅 Original date posted:2022-07-10
📝 Original message:
On Thu, Jun 30, 2022 at 4:19 AM Matt Corallo <lf-lists at mattcorallo.com>
wrote:

> Better yet, as Val points out, requiring a channel to relay onion messages
> puts a very real,
> nontrivial (in a world of msats) cost to getting an onion messaging
> channel. Better yet, with
> backpressure ability to DoS onion message links isn't denominated in
> number of messages, but instead
> in number of channels you are able to create, making the backpressure
> system equivalent to today's
> HTLC DoS considerations, whereas explicit payment allows an attacker to
> pay much less to break the
> system.
>

It can also be considered a bad thing that DoS ability is not based on a
number of messages. It means that for the one time cost of channel
open/close, the attacker can generate spam forever if they stay right below
the rate limit.

> Ultimately, paying suffers from the standard PoW-for-spam issue - you
> cannot assign a reasonable
> cost that an attacker cares about without impacting the system's usability
> due to said cost. Indeed,
> making it expensive enough to mount a months-long DDoS without impacting
> legitimate users be pretty
> easy - at 1msat per relay of a 1366 byte onion message you can only
> saturate an average home users'
> 30Mbps connection for 30 minutes before you rack up a dollar in costs, but
> if your concern is
> whether someone can reasonably trivially take out the network for minutes
> at a time to make it have
> perceptibly high failure rates, no reasonable cost scheme will work. Quite
> the opposite - the only
> reasonable way to respond is to respond to a spike in traffic while
> maintaining QoS is to rate-limit
> by inbound edge!
>

Suppose the attacker has enough channels to hit the rate limit on an
important connection some hops away from themselves. They can then sustain
that attack indefinitely, assuming that they stay below the rate limit on
the routes towards the target connection. What will the response be in that
case? Will node operators work together to try to trace back to the source
and take down the attacker? That requires operators to know each other.

Maybe this is a difference between lightning network and the internet that
is relevant for this discussion. That routers on the internet know each
other and have physical links between them, where as in lightning ties can
be much looser.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220710/d7fe084e/attachment.html>;

📅 Original date posted:2022-07-01 📝 Original message: > ...

2023-06-09T15:06:31+02:00

In reply to nevent1q…4amd
_________________________

📅 Original date posted:2022-07-01
📝 Original message:
>
> Path-finding algorithms that are currently in use generally don’t support
> negative fees. But in this case, the sum of inbound and outbound fees is
> still positive and therefore not a problem. If routing nodes set their
> policies accidentally or intentionally so that the sum of fees turns out
> negative, senders can just round up to zero and find a path as normal.
>

Correction to this:

The sum of inbound and outbound are not the fees set by one single routing
node. When path-finding considers a candidate hop, this adds the outbound
fee of the "from" node and the inbound fee of the "to" node. Because those
nodes don't necessarily coordinate fees, it may happen more often that the
fee goes negative. Rounding up to zero is still a quick fix and better than
ignoring inbound fees completely.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220701/632a50ce/attachment.html>;

📅 Original date posted:2022-07-01 📝 Original message: Hi ...

2023-06-09T15:06:31+02:00

In reply to nevent1q…se54
_________________________

📅 Original date posted:2022-07-01
📝 Original message:
Hi Bastien,

I vaguely remembered that the idea of inbound fees had been discussed
before. Before writing my post, I scanned through old ML posts and bolts
issues but couldn't find the discussion. Maybe it was part of a different
but related email or a bolts pr?

With regards to your objections, isn't it the case that it is always
possible to DoS your peer by just rejecting any forward that comes in from
them? Or indirectly affecting them negatively by setting high fees on all
outbound channels? To me it seems that there is nothing to lose by adding
inbound fees.

My thinking is that if I accept an incoming htlc, my local balance
increases on that incoming channel. My money gets locked up in a channel
that may or may not be interesting to me. Wouldn't it be fair to be
compensated for that?

Any thoughts from routing node operators would be welcome too (or links to
previous threads).

Joost

On Fri, Jul 1, 2022 at 1:19 PM Bastien TEINTURIER <bastien at acinq.fr> wrote:

> Hi Joost,
>
> As I've already stated every time this has been previously discussed, I
> believe
> this doesn't make any sense. The funds that are on the other side of the
> channel belong to your peer, not you, so they're free to use it however
> they
> want. If you're not happy with the way your peer is managing their fees,
> then
> don't open channels to them and let the network decide whether you're
> right or
> not.
>
> Moreover, you shouldn't care at all. If all the funds are on your peer's
> side,
> this isn't your problem, you used up all the money that was yours. As long
> as
> the channel is open, this is free inbound liquidity for you, so you're even
> benefiting from this.
>
> If Alice could set fees for Bob's side of the channel, Alice could
> arbitrarily
> DoS Bob's payments by setting a high fee. This is just one example of the
> many
> ways this idea completely breaks the routing incentives.
>
> Cheers,
> Bastien
>
> Le ven. 1 juil. 2022 à 13:10, Joost Jager <joost.jager at gmail.com> a
> écrit :
>
>> Path-finding algorithms that are currently in use generally don’t support
>>> negative fees. But in this case, the sum of inbound and outbound fees is
>>> still positive and therefore not a problem. If routing nodes set their
>>> policies accidentally or intentionally so that the sum of fees turns out
>>> negative, senders can just round up to zero and find a path as normal.
>>>
>>
>> Correction to this:
>>
>> The sum of inbound and outbound are not the fees set by one single
>> routing node. When path-finding considers a candidate hop, this adds the
>> outbound fee of the "from" node and the inbound fee of the "to" node.
>> Because those nodes don't necessarily coordinate fees, it may happen more
>> often that the fee goes negative. Rounding up to zero is still a quick fix
>> and better than ignoring inbound fees completely.
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220701/5b33a7c3/attachment.html>;

📅 Original date posted:2022-07-01 📝 Original message: ...

2023-06-09T15:06:30+02:00

In reply to nevent1q…jxj5
_________________________

📅 Original date posted:2022-07-01
📝 Original message:
Currently routing nodes on the lightning network charge fees based on a
policy that pertains to the outgoing channel only.

Several mentions have been made by routing node operators that this limits
the control that they can exert over the flow of traffic. The movement of
funds on all of the incoming channels is free of charge, which does not
match the reality of not all inbound liquidity being equal.

One option to fix this is to add two additional fields to the
`channel_update` message:
* `inbound_fee_base_msat`
* `inbound_fee_proportional_millionths`

With the previously introduced tlv message extensions, it should be
possible to let these fields propagate throughout the network without any
upgrades required.

Senders must pay each routing node the sum of its advertised inbound and
outbound fee for the channels used:

outbound_fee(amt_to_fwd) + inbound_fee(amt_to_fwd +
outbound_fee(amt_to_fwd))

So the inbound_fee is calculated based on the actual balance change in the
incoming channel. This includes the amount to forward as well as the
outbound fee.

An important characteristic of any solution that is to be deployed in an
existing network, is that it is backwards compatible. If routing nodes
start to require inbound fees, every sender that hasn’t upgraded their node
software will no longer be able to use that routing node. The routing node
will miss out on routing fees.

One mitigation is to charge zero inbound fees until a sufficiently large
portion of the senders has upgraded. It may be unclear though when this is
the case, and will likely take a significant amount of time. A test could
be to temporarily charge a minimal inbound fee, and watch for a reduction
in traffic and increase in `fee_insufficient` failures returned. If there
is little or no effect, then most senders have probably upgraded.

Another way to go about this is to set negative inbound fees during the
transitory phase. It is effectively a discount for using specific inbound
channels. So a routing node that charges 10 sats for forwarding today, may
in the future increase that to 13 sats and set the inbound fee to -3 sats.

Senders ignoring inbound fees will overpay (13 sats whereas 10 sats would
have been sufficient), but are still able to use the routing node. The
routing node may see a reduction in traffic though because it effectively
increased its fee for older senders only. But inbound fees could be
increased (decreased really because they are negative) gradually while
monitoring for fee over-payments. Over-payments are indicative of senders
ignoring the inbound fee discount.

Path-finding algorithms that are currently in use generally don’t support
negative fees. But in this case, the sum of inbound and outbound fees is
still positive and therefore not a problem. If routing nodes set their
policies accidentally or intentionally so that the sum of fees turns out
negative, senders can just round up to zero and find a path as normal.

Overall I think this can be a relatively compact change that may ultimately
lead to better capital placement on the network and lower routing fees.

Looking for feedback on the idea from both lightning devs and routing node
operators.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220701/c0768856/attachment-0001.html>;

📅 Original date posted:2022-02-15 📝 Original message: Hi ...

2023-06-09T15:05:18+02:00

In reply to nevent1q…cgfz
_________________________

📅 Original date posted:2022-02-15
📝 Original message:
Hi Rusty,

Nice to see the proposal in more concrete terms. Few questions:

- The total proved utxo value (not counting any utxos which are spent)
> is multiplied by 10 to give the "announcable_channel_capacity" for that
> node.
>

Could this work as a dynamic value too, similar to the minimum relay fee on
L1?

> 1. `tlv_stream`: `channel_update_v2_tlvs`
>
2. types:
> 1. type: 4 (`capacity`)
> 2. data:
> * [`tu64`:`satoshis`]
>

What does capacity mean exactly outside the context of a real channel? Will
this be reduced to that maximum htlc amount that the nodes want to route,
to save as much of the announceable budget as possible?

It is also the question of whether 10 x 10k channels should weigh as much
on the budget as a 1 x 100k channel. A spammer may be able to do more harm
with multiple smaller channels because there is more for the sender's
pathfinding algorithms to explore. Maybe it doesn't matter as long as there
is some mechanism to discourage spam.

> 1. type: 5 (`cost`)
> 2. data:
> * [`u16`:`cltv_expiry_delta`]
> * [`u32`:`fee_proportional_millionths`]
> * [`tu32`:`fee_base_msat`]
> 1. type: 6 (`min_msat`)
> 2. data:
> * [`tu64`:`min_htlc_sats`]
>
> - `channel_id_and_claimant` is a 31-bit per-node channel_id which can be
> used in onion_messages, and a one bit stolen for the `claim` flag.
>

If you'd increase the budget multiplier from 10 to 20, couldn't this be
simplified to always applying the cost to both nodes?

> - A channel is not considered to exist until both peers have sent a
> channel_update_v2, at least one of which must set the `claim` flag.
> - If a node sets `claim`, the capacity of the channel is subtracted from
> the remaining announcable_channel_capacity for that node (minimum
> 10,000 sats).
>

Same question about magic value and whether it can be dynamic.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220215/1c100bd3/attachment.html>;

📅 Original date posted:2021-11-15 📝 Original message: Hi ...

2023-06-09T15:04:24+02:00

In reply to nevent1q…7rgp
_________________________

📅 Original date posted:2021-11-15
📝 Original message:
Hi Rene,

> First I am happy that you also agree that reliability can and should be
> expressed as a probability as discussed in [0].
>

Probability based routing is not new to me. I've implemented a form of that
in lnd in march 2019: https://github.com/lightningnetwork/lnd/pull/2802,
followed by several rounds of refinement.

> The problem that you address is that of feature engineering[1]. Which
> consists of two (or even more) steps:
>
> 1.) Feature selection: That means in payment delivery we will compute a
> min cost flow [2] with a chosen cost function (historically people used
> dijkstra seach for single paths with the cost function representing the
> weights on the edges of the graph -which is what most folks currently still
> do). While [2] and I personally agree with you that the cost function
> should be a combination the two features fees and reliability (as in
> successprobability) Matt Corallo righfully pointed out [3] that other
> features might be chosen in the future to deliver more optimal results. For
> example implementations currently often use CLTV as a feature (which I
> honestly find horrible) and I am currently investigating if one could add
> latency of channels or - for known IP addresses - either the geo distance
> or IP distance.
>

I am aware that there are more candidate features, but my question is
specifically about the ones that I mentioned.

2.) Combining features: This is the question that you are asking. Often
> people use a linear weighted sum to combine features. This is what often
> happens implicitly in neural networks. While this is often good enough and
> while it is often practical to either learn the weights or give users a
> choice there are many situation where the weighted linear sum does not work
> well with the selected features. An example for the weighted sum is the
> risk-factor in c-lightning that could have been used to decide if one
> wanted the dijkstra seach to either optimize for CLTV delta or for paid
> routing fees. Also in our paper [2] in which we discuss the same two
> features that you mentioned we explain how a linear sum of two features can
> be optimal due to the lagrangian bounding principle. However in practice
> (of machine learning) it has been shown that using the harmonic mean [4]
> between features often works very well without the necessity to learn a
> weight / parameter. This has for example been done when c-lightnign
> recently switched to probabilistic path finding [5]. In this thread you
> find a long discussion and evaluation how the harmonic mean outperformed
> the linear sum.
>

Obviously features can be combined in a multitude of ways, but I am looking
for something that is anchored to some kind of understandable starting
point. What I did in lnd is to work with so called 'payment attempt cost'.
A virtual satoshi amount that represents the cost of a failed attempt. If
you put a high price on failed attempts, pathfinding will tend towards more
reliable routes even if they require a higher fee. To me, the idea of
putting a (virtual) cost on a payment attempt is tangible and ideally the
math should follow from that. I don't want zero parameters, because I think
that ultimately the fee/reliability trade-off is up to the user to decide
on.

Joost

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211115/0be67f54/attachment.html>;

📅 Original date posted:2021-11-15 📝 Original message: In ...

2023-06-09T15:04:23+02:00

In reply to nevent1q…dgyu
_________________________

📅 Original date posted:2021-11-15
📝 Original message:
In Lightning pathfinding the two main variables to optimize for are routing
fee and reliability. Routing fee is concrete. It is the sat amount that is
paid when a payment succeeds. Reliability is a property of a route that can
be expressed as a probability. The probability that a route will be
successful.

During pathfinding, route options are compared against each other. So for
example:

Route A: fee 10 sat, success probability 50%
Route B: fee 20 sat, success probability 80%

Which one is the better route? That depends on user preference. A patient
user will probably go for route A in the hope of saving on fees whereas for
a time-sensitive payment route B looks better.

It would be great to offer this trade-off to the user in a simple way.
Preferably a single [0, 1] value that controls the selection process. At 0,
the route is only optimized for fees and probabilities are ignored
completely. At 1, the route is only optimized for reliability and fees are
ignored completely.

But how to choose between the routes A and B for a value somewhere in
between 0 and 1? For example 0.5 - perfect balance between reliability and
fee. But what does that mean exactly?

Anyone got an idea on how to approach this best? I am looking for a simple
formula to decide between routes, preferably with a reasonably sound
probability-theoretical basis (whatever that means).

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211115/fffa5edd/attachment.html>;

📅 Original date posted:2021-10-21 📝 Original message: ...

2023-06-09T15:04:20+02:00

In reply to nevent1q…hpzu
_________________________

📅 Original date posted:2021-10-21
📝 Original message:
>If it is a multipart and we have the preimage, wait for all the parts to
arrive, then say yes to all of them.

Without actual reservations made in the channels, is this going to work?
For example: a 10M payment and a route that contains a channel with only 5M
balance. The sender's multi-path algorithm will try to split and send the
first 5M. Then they'll do the second 5M, but because there is no actual
reservation, the second 5M seems to be passing alright through the
bottleneck channel too. When the payment is then executed, it will fail.

Or do nodes keep track of all the unresolved probes and deduct the total
amount from the available balance? Of course only for the available balance
for probes. When a real htlc comes through, outstanding probes are ignored.
Although the problem with that could be that someone can spam you with
probes so that your available 'probe' balance is zero and you'll receive no
real traffic anymore.

Perhaps an alternative is to let senders attach a random identifier to a
probe. For multi-part probes, each probe will carry the same identifier.
Routing nodes will deduct the outstanding probe amounts from the available
balance, but only for probes within the same group (same id). That way each
probe(group) is isolated from everything else that is going on.

Joost

On Wed, Sep 29, 2021 at 5:40 AM ZmnSCPxj via Lightning-dev <
lightning-dev at lists.linuxfoundation.org> wrote:

> Good morning list,
>
> While discussing something tangentially related with aj, I wondered this:
>
> > Why do we shoot an HTLC first and then ask the question "can you
> actually resolve this?" later?
>
> Why not something like this instead?
>
> * For a payer:
> * Generate a path.
> * Ask first hop if it can resolve an HTLC with those specs (passing the
> encrypted onion).
> * If first hop says "yes", actually do the `update_add_htlc` dance.
> Otherwise try again.
> * For a forwarder:
> * If anybody asks "can you resolve this path" (getting an encrypted
> onion):
> * Decrypt one layer to learn the next hop.
> * Check if the next hop is alive and we have the capacity towards it,
> if not, answer no.
> * Ask next hop if it can resolve the next onion layer.
> * Return the response from the next hop.
> * For a payee:
> * If anybody asks "can you resolve this path":
> * If it is not a multipart and we have the preimage, say yes.
> * If it is a multipart and we have the preimage, wait for all the
> parts to arrive, then say yes to all of them.
> * Otherwise say no.
>
> Now, the most obvious reason against this, that comes to mind, is that
> this is a potential DoS vector.
> Random node can trigger a lot of network activity by asking random stuff
> of random nodes.
> Asking the question is free, after all.
>
> However, we should note that sending *actual* HTLCs is a similar DoS
> vector **today**.
> This is still "free" in that the asker has no need to pay fees for failed
> HTLCs; they just lose the opportunity cost of the amount being locked up in
> the HTLCs.
> And presumably the opportunity cost is low since Lightning forwarding
> earnings are so tiny.
>
> One way to mitigate against this is to make generating an onion costly but
> validating and decrypting it cheap.
> We could use an encryption scheme that is more computationally expensive
> to encrypt but cheap to decrypt, for example.
> Or we could require proof-of-work on the onion: each unwrapped onion
> layer, when hashed, has to have a hash that is less than some threshold
> (this scales according to the number of hops in the onion, as well).
> Ultimate askers need to grind the shared secret until the onion layer hash
> achieves the target.
>
> Obviously just because you asked a few milliseconds ago if a path is
> viable does not mean that the path *remains* viable right now when you
> actually send out an HTLC, but presumably that risk is now lessened.
> Unexpected shutdowns or loss of connectivity has to appear in a smaller
> and shorter time frame to negatively affect intermediate nodes.
>
> Another thought is: Does the forwarding node have an incentive to lie?
> Suppose the next hop is alive but the forwarding node has insufficient
> capacity towards the next hop.
> Then the forwarding node can lie and claim it can still resolve the HTLC,
> in the hope that a few milliseconds later, when the actual HTLC arrives,
> the capacity towards the next hop has changed.
> Thus, even if the capacity now is insufficient, the forwarding node has an
> incentive to lie and claim sufficient capacity.
>
> Against the above, we can mitigate this by accepting "no" from *any* node
> along the path, but only accepting "yes" from the actual payee.
> We already have a mechanism where any node along a route can report a
> forwarding or other payment error, and the sender is able to identify which
> node along the path raised it.
> Thus, the payer can identify which node along the route responded with a
> "yes", and check that it definitely reached the payee.
> Presumably, when a node receives a question, it checks if the asking node
> has sufficient capacity towards it first, and if not, fails the channel
> between them, since obviously the asking node is not behaving according to
> protocol and is buggy.
>
> Now, this can be used to probe capacities, for free, but again --- we
> already *have* probing capacities, for free, today, by just using random
> hashes.
>
>
>
> Why is this advantageous at all?
>
> One reason for doing this is that it improves payment latency.
> Some paths *will* fail, because there is no single consistent view of the
> network and its capacity (which is impossible due to others also possibly
> sending out via the same forwarding nodes you are using, and besides, even
> if such a view could be made to exist, it would be dangerously
> anti-privacy).
> This mechanism does not require that intermediate nodes respond with a
> signature and wait for a replied signature *before* they forward the onion
> to the next hop; when they are *just* asking, there is no HTLC involved, no
> updates to the channel state, and the question can be forwarded as soon as
> we can check locally.
> Further, in the current mechanism where we shoot HTLCs first and ask
> questions later, failures also require 1.5 roundtrips due to sharing
> signatures; with the "just ask first" phase there is no need for round
> trips to respond to questions.
>
> Basically, we replace multiple round trips per hop in case of a failure
> along a route, with a single large round trip from the payer to the failure
> point.
> In case of a success we just add more latency, but as we move to more
> multipath payments, perhaps it becomes more advantageous, since the
> probability of a particular sub-path failing is now higher?
>
> More importantly, by allowing to ask first, we reduce the probability that
> HTLCs made in good faith --- i.e. those that are fully intended to reach a
> destination and be resolved --- it may now be more palatable to charge for
> failing actual HTLCs.
> Since we expect that HTLC failures due to a node or channel failing along
> the way is lessened compared to today, because the sender *asks* first
> before *shooting* the HTLC, then the effect of charging for failing
> *actual* HTLCs is lessened, possibly to a more acceptable level.
>
>
> So, to lightning-dev --- is this a decent idea at all?
> Note that in particular this is something that requires a whole-network
> upgrade, as intermediate nodes have to upgrade as well.
>
>
> Regards,
> ZmnSCPxj
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211021/aac2de10/attachment-0001.html>;

📅 Original date posted:2021-10-21 📝 Original message: On ...

2023-06-09T15:04:19+02:00

In reply to nevent1q…rfmd
_________________________

📅 Original date posted:2021-10-21
📝 Original message:
On Thu, Oct 21, 2021 at 12:00 PM ZmnSCPxj <ZmnSCPxj at protonmail.com> wrote:

> Good morning Joost,
>
> > A potential downside of a dedicated probe message is that it could be
> used for free messaging on lightning by including additional data in the
> payload for the recipient. Free messaging is already possible today via
> htlcs, but a probe message would lower the cost to do so because the sender
> doesn't need to lock up liquidity for it. This probably increases the spam
> potential. I am wondering if it is possible to design the probe message so
> that it is useless for anything other than probing. I guess it is hard
> because it would still have that obfuscated 1300 bytes block with the
> remaining part of the route in it and nodes can't see whether there is
> other meaningful data at the end.
>
> For the probe, the onion max size does not *need* to be 1300, we could
> reduce the size to make it less useable for *remote* messaging.
>

Yes, maybe it can be reduced a bit. But if we want to support 27 hops like
we do for payments, there will be quite some space left for messaging on
real routes which are mostly much shorter.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211021/ac8545c8/attachment.html>;

📅 Original date posted:2021-10-21 📝 Original message: A ...

2023-06-09T15:04:19+02:00

In reply to nevent1q…cycv
_________________________

📅 Original date posted:2021-10-21
📝 Original message:
A potential downside of a dedicated probe message is that it could be used
for free messaging on lightning by including additional data in the payload
for the recipient. Free messaging is already possible today via htlcs, but
a probe message would lower the cost to do so because the sender doesn't
need to lock up liquidity for it. This probably increases the spam
potential. I am wondering if it is possible to design the probe message so
that it is useless for anything other than probing. I guess it is hard
because it would still have that obfuscated 1300 bytes block with the
remaining part of the route in it and nodes can't see whether there is
other meaningful data at the end.

On Thu, Oct 14, 2021 at 9:48 AM Joost Jager <joost.jager at gmail.com> wrote:

> A practice that is widely applied by lightning wallets is to probe routes
> with an unknown payment hash before making the actual payment. Probing
> yields an accurate routing fee that can be shown to the user before
> execution of the payment.
>
> The downside of this style of probing is that for a short period of time,
> liquidity is locked up. Not just the sender's liquidity, but also liquidity
> of nodes along the route. And if the probe gets stuck for whatever reason,
> that short period may become longer.
>
> But does this lock up serve a purpose at all? Suppose there would be a
> liquidity probing protocol message similar to `update_add_htlc`
> (`probe_htlc`?) that would skip the whole channel update machinery and is
> only forwarded to the next hop if the link would be able to carry the htlc.
> Won't this work as well as the current probing without the downsides? Nodes
> can obviously reject these probes because they are distinguishable from
> real payments (contrary to unknown hash probing where everything looks the
> same). However if they do so, senders won't use that route and the routing
> node misses out on routing fees.
>
> Another problem of the lightning network is its susceptibility to channel
> jamming. Multiple options have been proposed (see also
> https://github.com/t-bast/lightning-docs/blob/master/spam-prevention.md),
> but they all come with downsides.
>
> Personally I incline towards solutions that involve deterring the attacker
> by making them pay actual satoshis. Lightning itself is payment system and
> it seems that paying for the payments is a natural solution to the problem.
> Several iterations of this idea have been proposed. One of my own that
> builds on an earlier idea by t-bast is described in
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-February/002958.html
> .
>
> The main criticism that this proposal has received is that it deteriorates
> the user experience for honest users when multiple payment routes need to
> be attempted. Every attempt will have a cost, so the user will see its
> balance going down by only just trying to make the payment. How bad this is
> depends on the attempt failure rate. I expect this rate to become really
> low as the network matures and senders hold routing nodes to high
> standards. Others however think otherwise and consider a series of failed
> attempts part of a healthy system.
>
> Custodial wallets could probably just swallow the cost for failures. They
> typically use one pathfinding system for all their users and are therefore
> able to collect a lot of information on routing node performance. This is
> likely to decrease the payment failure rate to an acceptably low level.
>
> For non-custodial nodes, this may be different. They have to map out the
> good routing nodes all by themselves and this exploration will bear a cost.
>
> So how would things work out with a combination of both of the proposals
> described in this mail? First we make probing free (free as in no liquidity
> locked up) and then we'll require senders to pay for failed payment
> attempts too. Failed payment attempts after a successful probe should be
> extremely rate, so doesn't this fix the ux issue with upfront fees?
>
> Joost
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211021/2730142c/attachment.html>;

📅 Original date posted:2021-10-15 📝 Original message: On ...

2023-06-09T15:04:18+02:00

In reply to nevent1q…hnru
_________________________

📅 Original date posted:2021-10-15
📝 Original message:
On Fri, Oct 15, 2021 at 4:21 PM Owen Gunden <ogunden at phauna.org> wrote:

> On Thu, Oct 14, 2021 at 09:48:27AM +0200, Joost Jager wrote:
> > So how would things work out with a combination of both of the
> > proposals described in this mail? First we make probing free (free as
> > in no liquidity locked up) and then we'll require senders to pay for
> > failed payment attempts too. Failed payment attempts after a
> > successful probe should be extremely rate, so doesn't this fix the ux
> > issue with upfront fees?
>
> Why couldn't a malicious routing node (or group of colluding routing
> nodes) succeed the probe and then fail the payment in order to collect
> the failed payment fee?
>

Yes they could, but senders should be really suspicious when this happens.
It could happen occasionally because balances may have shifted in between
probe and payment. But if it keeps happening they may want to ban this
routing node for a long time. This may disincentivize the routing node
enough to respond honestly to probes.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211015/a7713880/attachment.html>;

📅 Original date posted:2021-10-19 📝 Original message: There ...

2023-06-09T15:04:18+02:00

In reply to nevent1q…rcsy
_________________________

📅 Original date posted:2021-10-19
📝 Original message:
There could be some corners where the incentives may not work out 100%, but
I doubt that any routing node would bother exploiting this. Especially
because there could always be that reputation scheme at the sender side
which may cost the routing node a lot more in lost routing fees than the
marginal gain from the upfront payment.

Another option is that nodes that don't care to be secretive about their
channel balances could include the actual balance in a probe failed
message. Related: https://github.com/lightningnetwork/lightning-rfc/pull/695

Overall it seems that htlc-less probes are an improvement to what we
currently have. Immediate advantages include a reduction of the load on
nodes by cutting out the channel update machinery, better ux (faster
probes) and no locked up liquidity. On the longer term it opens up the
option to charge for failed payments so that we finally have an answer to
channel jamming.

ZmnSCPxj, as first person to propose the idea (I think?), would you be
interested in opening a draft PR on the spec repository that outlines the
new message(s) that we'd need and continue detailing from there?

Joost

On Sat, Oct 16, 2021 at 12:51 AM ZmnSCPxj via Lightning-dev <
lightning-dev at lists.linuxfoundation.org> wrote:

> Good morning Owen,
>
> > C now notes that B is lying, but is faced with the dilemma:
> >
> > "I could either say 'no' because I can plainly see that B is lying, or
> > I could say 'yes' and get some free sats from the failed payment (or
> > via the hope of a successful payment from a capacity increase in the
> > intervening milliseconds)."
>
> Note that if B cannot forward an HTLC to C later, then C cannot have a
> failed payment and thus cannot earn any money from the upfront payment
> scheme; thus, at least that part of the incentive is impossible.
>
> On the other hand, there is still a positive incentive for continuing the
> lie --- later, maybe the capacity becomes OK and C could earn both the
> upfront fee and the success fee.
>
> > So C decides it's in his interest to keep the lie going. D, the payee,
> > can't tell that it's a lie when it reaches her.
> >
> > If C did want to tattle, it's important that he be able to do so in a
> > way that blames B instead of himself, otherwise payers will assume
> > (incorrectly, and to C's detriment) that the liquidity deficit is with C
> > rather than B.
>
> That is certainly quite possible to do.
>
> Regards,
> ZmnSCPxj
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211019/dd316d31/attachment.html>;

📅 Original date posted:2021-10-15 📝 Original message: > ...

2023-06-09T15:04:17+02:00

In reply to nevent1q…p307
_________________________

📅 Original date posted:2021-10-15
📝 Original message:
>
> > On Thu, Oct 14, 2021 at 09:48:27AM +0200, Joost Jager wrote:
> >
> > > So how would things work out with a combination of both of the
> > > proposals described in this mail? First we make probing free (free as
> > > in no liquidity locked up) and then we'll require senders to pay for
> > > failed payment attempts too. Failed payment attempts after a
> > > successful probe should be extremely rate, so doesn't this fix the ux
> > > issue with upfront fees?
> >
> > Why couldn't a malicious routing node (or group of colluding routing
> > nodes) succeed the probe and then fail the payment in order to collect
> > the failed payment fee?
>
> Good observation!
>
> I propose substantially the same thing here:
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-September/003256.html

I totally missed that thread, but it is indeed the same thing including the
notion that it may make upfront payments palatable! Contains some great
additional ideas too.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211015/342fb320/attachment.html>;

📅 Original date posted:2021-10-14 📝 Original message: A ...

2023-06-09T15:04:16+02:00

In reply to nevent1q…q7q0
_________________________

📅 Original date posted:2021-10-14
📝 Original message:
A practice that is widely applied by lightning wallets is to probe routes
with an unknown payment hash before making the actual payment. Probing
yields an accurate routing fee that can be shown to the user before
execution of the payment.

The downside of this style of probing is that for a short period of time,
liquidity is locked up. Not just the sender's liquidity, but also liquidity
of nodes along the route. And if the probe gets stuck for whatever reason,
that short period may become longer.

But does this lock up serve a purpose at all? Suppose there would be a
liquidity probing protocol message similar to `update_add_htlc`
(`probe_htlc`?) that would skip the whole channel update machinery and is
only forwarded to the next hop if the link would be able to carry the htlc.
Won't this work as well as the current probing without the downsides? Nodes
can obviously reject these probes because they are distinguishable from
real payments (contrary to unknown hash probing where everything looks the
same). However if they do so, senders won't use that route and the routing
node misses out on routing fees.

Another problem of the lightning network is its susceptibility to channel
jamming. Multiple options have been proposed (see also
https://github.com/t-bast/lightning-docs/blob/master/spam-prevention.md),
but they all come with downsides.

Personally I incline towards solutions that involve deterring the attacker
by making them pay actual satoshis. Lightning itself is payment system and
it seems that paying for the payments is a natural solution to the problem.
Several iterations of this idea have been proposed. One of my own that
builds on an earlier idea by t-bast is described in
https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-February/002958.html
.

The main criticism that this proposal has received is that it deteriorates
the user experience for honest users when multiple payment routes need to
be attempted. Every attempt will have a cost, so the user will see its
balance going down by only just trying to make the payment. How bad this is
depends on the attempt failure rate. I expect this rate to become really
low as the network matures and senders hold routing nodes to high
standards. Others however think otherwise and consider a series of failed
attempts part of a healthy system.

Custodial wallets could probably just swallow the cost for failures. They
typically use one pathfinding system for all their users and are therefore
able to collect a lot of information on routing node performance. This is
likely to decrease the payment failure rate to an acceptably low level.

For non-custodial nodes, this may be different. They have to map out the
good routing nodes all by themselves and this exploration will bear a cost.

So how would things work out with a combination of both of the proposals
described in this mail? First we make probing free (free as in no liquidity
locked up) and then we'll require senders to pay for failed payment
attempts too. Failed payment attempts after a successful probe should be
extremely rate, so doesn't this fix the ux issue with upfront fees?

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211014/04750627/attachment.html>;

📅 Original date posted:2021-09-21 📝 Original message: On ...

2023-06-09T15:03:49+02:00

In reply to nevent1q…m3z5
_________________________

📅 Original date posted:2021-09-21
📝 Original message:
On Tue, Sep 21, 2021 at 2:05 PM fiatjaf <fiatjaf at gmail.com> wrote:

> What if instead of the payer generating the preimage the payee could
> generate stateless invoices? Basically just use some secret to compute the
> preimage upon receiving the HTLC, for example:
>

Maybe my explanation wasn't clear enough, but this is exactly what I am
proposing. The payee generates a stateless invoice and gives it to the
payer.

> 1. Payer requests an invoice.
> 2. Payee computes hash = sha256(hmac(local_secret, arbitrary_invoice_id)),
> then encodes arbitrary_invoice_id into the invoice somehow.
> 3. Payer sends payment with arbitrary_invoice_id as tlv_record_a.
> 4. Upon receiving the HTLC, payee computes preimage = hmac(local_secret,
> tlv_record_a) and resolves it.
>

One way to do this that I tried to describe in the initial post is via the
payment_secret. This is already an arbitrary invoice id that is already
sent as a tlv record.

> I've implemented such a scheme on @lntxbot, but it required low level code
> in a c-lightning plugin and a hack with route hints: since TLV payloads
> were not an option (as payers wouldn't know how to send them) I've used a
> "shadow" route hint to a private channel that didn't exist, so preimage was
> generated on the payee using preimage = hmac(local_secret,
> next_channel_scid).
>

Clever workaround.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20210921/8f1a2a53/attachment.html>;

📅 Original date posted:2021-09-21 📝 Original message: > ...

2023-06-09T15:03:48+02:00

In reply to nevent1q…kwq3
_________________________

📅 Original date posted:2021-09-21
📝 Original message:
>
> > preimage = H(node_secret | payment_secret | payment_amount |
> encoded_order_details)
> > invoice_hash = H(preimage)
> >
> > The sender sends an htlc locked to invoice_hash for payment_amount and
> passes along payment_secret and encoded_order_details in a custom tlv
> record.
> >
> > When the recipient receives the htlc, they reconstruct the preimage
> according to the formula above. At this point, all data is available to do
> so. When H(preimage) indeed matches the htlc hash, they can settle the
> payment knowing that this is an order that they committed to earlier.
> Settling could be implemented as a just-in-time inserted invoice to keep
> the diff small.
> >
> > The preimage is returned to the sender and serves as a proof of payment.
>
> Does this actually work?
> How does the sender know the `invoice_hash` to lock the HTLC(s) to?

> If the sender does not know the `node_secret` (from its name, I am
> guessing it is a secret known only by the recipient?) then it cannot
> compute `invoice_hash`, the `invoice_hash` has to be somehow learned by the
> sender from the recipient.
>

So to be clear: this isn't a spontaneous payment protocol with
proof-of-payment. The sender will still request an invoice from the
recipient via an ordinary http request (think of a paywall with qr invoice
that is presented when web-browsing to a paid article). That is also how
the sender learns the invoice_hash. It is part of the bolt11 payment
request as it always is.

The goal of the scheme is to alleviate the recipient from storing the
invoices that they generate.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20210921/e9fbb17f/attachment.html>;

📅 Original date posted:2021-09-21 📝 Original message: ...

2023-06-09T15:03:47+02:00

In reply to nevent1q…utyd
_________________________

📅 Original date posted:2021-09-21
📝 Original message:
Problem

One of the qualities of lightning is that it can provide light-weight,
no-login payments with minimal friction. Games, paywalls, podcasts, etc can
immediately present a QR code that is ready for scan and pay.

Optimistically presenting payment requests does lead to many of those
payment requests going unused. A user visits a news site and decides not to
buy the article. The conventional approach is to create a lightning invoice
on a node and store the invoice together with order details in a database.
If the order then goes unfulfilled, cleaning processes remove the data from
the node and database again.

The problem with this setup is that it needs protection against unbounded
generation of payment requests. There are solutions for that such as rate
limiting, but wouldn't it be nice if invoices can be generated without the
need to keep any state at all?

Stateless invoices

What would happen if a lightning invoice is only generated and stored
nowhere on the recipient side? To the user, it won't make a difference.
They would still scan and pay the invoice. When the payment arrives at the
recipient though, two problems arise:

1. Recipient doesn't know whom or what the payment is for.

This can be solved by attaching additional custom tlv records to the htlc.
On the wire, this is all arranged for. The only missing piece is the
ability to specify additional data for that custom tlv record in a bolt11
invoice. One way would be to define a new tagged field for this in which
the recipient can encode the order details.

An alternative is to use the existing invoice description field and simply
always pass that along with the htlc as a custom tlv record.

A second alternative that already works today is to use part (for example
16 out of 32 bytes) of the payment_secret (aka payment address) to encode
the order details in. This assumes that the secret is still secret enough
with reduced entropy. Also there may not be enough space for every
application.

2. Recipient doesn't know the preimage that is needed to settle the htlc(s).

One option is to use a keysend payment or AMP payment. In that case, the
sender includes the preimage with the htlc. Unfortunately this doesn't
provide the sender with a proof of payment that they'd get with a regular
lightning payment.

An alternative solution is to use a deterministic preimage based on a
(recipient node key-derived) secret, the payment secret and other relevant
properties. This allows the recipient to derive the same preimage twice:
Once when the lightning invoice is generated and again when a payment
arrives.

It could be something like this:

payment_secret = random
preimage = H(node_secret | payment_secret | payment_amount |
encoded_order_details)
invoice_hash = H(preimage)

The sender sends an htlc locked to invoice_hash for payment_amount and
passes along payment_secret and encoded_order_details in a custom tlv
record.

When the recipient receives the htlc, they reconstruct the preimage
according to the formula above. At this point, all data is available to do
so. When H(preimage) indeed matches the htlc hash, they can settle the
payment knowing that this is an order that they committed to earlier.
Settling could be implemented as a just-in-time inserted invoice to keep
the diff small.

The preimage is returned to the sender and serves as a proof of payment.

Resilience

To me it seems that stateless invoices can be a relatively simple way to
improve the resiliency of systems that deal with lightning invoices.
Unlimited amounts of invoices can be generated without worrying about
storage or memory, no matter if the requests are due to popularity of a
service or a deliberate dos attack.

Interested to hear your thoughts.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20210921/788f449b/attachment-0001.html>;

📅 Original date posted:2020-10-13 📝 Original message: > ...

2023-06-09T15:01:11+02:00

In reply to nevent1q…jetr
_________________________

📅 Original date posted:2020-10-13
📝 Original message:
>
> > If I were LOW-REP, I'd still charge an unknown node a hold fee. I
> > would only waive the hold fee for high-reputation nodes. In that case,
> > the attacker is still paying for the attack. I may be forced to take a
> > small loss on the difference, but at least the larger part of the pain
> > is felt by the attacker. The assumption is that this is sufficient
> > enough to deter the attacker from even trying.
>
> The LOW-REP node being out of pocket is the clue here: if one party
> loses funds, even a tiny bit, another party gains some funds. In this
> case the HIGH-REP node collaborating with the ATTACKER can extract some
> funds from the intermediate node, allowing them to dime their way to all
> of LOW-REP's funds. If an attack results in even a tiny loss for an
> intermediary and can be repeated, the intermediary's funds can be
> syphoned by an attacker.
>

The assumption is that HIGH-REP nodes won't do this :) LOW-REP will see all
those failed payments and small losses and start to realize that something
strange is happening. I know the proposal isn't fully trustless, but I
think it can work in practice.

> Another attack that is a spin on ZmnSCPxj's waiting to backpropagate the
> preimage is even worse:
>
> - Attacker node `A` charging hold fees receives HTLC from victim `V`
> - `A` does not forward the HTLC, but starts charging hold fees
> - Just before the timeout for the HTLC would force us to settle onchain
> `A` just removes the HTLC without forwarding it or he can try to
> forward at the last moment, potentially blaming someone else for its
> failure to complete
>
> This results in `A` extracting the maximum hold fee from `V`, without
> the downstream hold fees cutting into their profits. By forwarding as
> late as possible `A` can cause a downstream failure and look innocent,
> and the overall payment has the worst possible outcome: we waited an
> eternity for what turns out to be a failed attempt.
>

The idea is that an attacker node is untrusted and won't be able to charge
hold fees.

- Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20201013/5b59146c/attachment.html>;

📅 Original date posted:2020-03-09 📝 Original message: On ...

2023-06-09T14:59:14+02:00

In reply to nevent1q…hyzh
_________________________

📅 Original date posted:2020-03-09
📝 Original message:
On Thu, Feb 20, 2020 at 4:22 AM Anthony Towns <aj at erisian.com.au> wrote:

> On Tue, Feb 18, 2020 at 10:23:29AM +0100, Joost Jager wrote:
> > A different way of mitigating this is to reverse the direction in which
> the
> > bond is paid. So instead of paying to offer an htlc, nodes need to pay to
> > receive an htlc. This sounds counterintuitive, but for the described
> jamming
> > attack there is also an attacker node at the end of the route. The
> attacker
> > still pays.
>
> I think this makes a lot of sense. I think the way it would end up working
> is that the further the route extends, the greater the payments are, so:
>
> A -> B : B sends A 1msat per minute
> A -> B -> C : C sends B 2msat per minute, B forwards 1msat/min to A
> A -> B -> C -> D : D sends C 3 msat, etc
> A -> B -> C -> D -> E : E sends D 4 msat, etc
>
> so each node is receiving +1 msat/minute, except for the last one, who's
> paying n msat/minute, where n is the number of hops to have gotten up to
> the last one. There's the obvious privacy issue there, with fairly
> obvious ways to fudge around it, I think.
>

Yes, that is definitely a good point. Otherwise the attacker can hold the
htlc at the end of the route and pay the hold fee to its predecessor. The
hold fee will propagate back to the first node (and increase along the
way). The first node is also owned by the attacker. Meaning there again is
no cost for the attacker to jam the channel.

In the mean time, I've been jamming channels on testnet myself. See what
pathfinding changes are needed to do it efficiently and check out the
effect. There was the expected outcome of a channel being jammed for as
long as I wanted. But I also learned something else:

Traversing a path takes time, especially if the path is optimized for
maximum length and contains loops. In particular when some of the nodes
and/or network connections are slow, the total round-trip from the sender
point of view can get seriously long. Even if the final node immediately
fails the htlc, the nodes at the start of the path still see their outgoing
htlcs being held for quite some time.

What this means is that the channel jamming attack can also be executed
without the attacker controlling the final node. The attacker can construct
long routes for which it doesn't matter where they end. Suppose it takes 1
minute for the htlc to be released again on the channel that is targeted
(the round trip from the targeted channel to the final node). The attacker
just needs to launch htlcs at a rate higher than one per minute to
(eventually) saturate the channel. In my experiment, I launched many htlcs
concurrently, which seemed to make the total latency even longer. Probably
because those htlcs then start competing for limited resources at the route
hops.

This variation does require more action from the attacker. They need to
keep refreshing htlcs that return back to them. Therefore it may be easier
to address this with some form of rate limiting, although that has its own
downsides.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20200309/d26ea327/attachment.html>;

📅 Original date posted:2020-02-18 📝 Original message: Hi ...

2023-06-09T14:58:56+02:00

In reply to nevent1q…5g7g
_________________________

📅 Original date posted:2020-02-18
📝 Original message:
Hi all,

Within our team, we've been discussing the subject of preventing
liquidity-consuming spam (aka channel jamming) further. One idea came up
that I think is worth sharing.

Previous prepay ideas were based on the sender paying something up-front in
case the htlc causes grief on the network. This however leaves the sender
vulnerable to nodes stealing that up-front payment.

Consider what is probably the worst known channel jamming attack: an
attacker sends minimum sized htlcs to fill up the limited number of
commitment slots of channels along the route. Those htlcs will be held as
long as possible by the receiving node (that is also controlled by the
attacker). The hold time per htlc doesn't even need to be very long,
because a fresh htlc can be launched to immediately re-occupy a slot after
it opens up again.

The cost to the network of this attack is mostly dependent on the capacity
of the channels used. The bigger the capacity, the more funds are locked up
if a sufficient number of minimum sized htlcs are pending. The size of the
up-front payment likely needs to be proportional to this cost.

This means that for small htlcs, the up-front payment requirements may very
well be exceeding the htlc amount and routing fees paid by far. At that
point, a routing node may decide to steal the up-front payment rather than
earn the routing fee in an honest way.

A different way of mitigating this is to reverse the direction in which the
bond is paid. So instead of paying to offer an htlc, nodes need to pay to
receive an htlc. This sounds counterintuitive, but for the described
jamming attack there is also an attacker node at the end of the route. The
attacker still pays. The advantage is that for legitimate senders, there is
no up-front payment that can be stolen.

How this would work is that channel peers charge each other for the time
that the other party holds an htlc. So if node A extends an htlc to node B,
node B will pay node A amount x per minute of hold time. If node B doesn't
pay (doesn't hold up the contract), A will close the channel. It can be a
running balance between A and B that materializes as a single htlc per
channel on the commitment transaction.

As long as node B forwards the htlc swiftly to node C, the dfiference (the
actual cost) between what B needs to pay A and what B receives from C will
be tiny. Only when the htlc reaches the attacker node, or any other node on
the network that is (unintentionally) mishaving for some reason, the delta
starts to increase quickly for that node. The cost is borne by the node
that should bear it.

This would also fix concerns that have been voiced around hodl invoices.
With the reverse bond payment as described above, hodling nodes will pay
for the cost of their actions.

Many details skipped over, but interested to hear opinions on the viability
of this variation of up-front payments.

Joost

On Tue, Nov 5, 2019 at 3:25 AM Rusty Russell <rusty at rustcorp.com.au> wrote:

> Hi all,
>
> It's been widely known that we're going to have to have up-front
> payments for msgs eventually, to avoid Type 2 spam (I think of Type 1
> link-local, Type 2 though multiple nodes, and Type 3 liquidity-using
> spam).
>
> Since both Offers and Joost's WhatSat are looking at sending
> messages, it's time to float actual proposals. I've been trying to come
> up with something for several years now, so thought I'd present the best
> I've got in the hope that others can improve on it.
>
> 1. New feature bit, extended messages, etc.
> 2. Adding an HTLC causes a *push* of a number of msat on
> commitment_signed (new field), and a hash.
> 3. Failing/succeeding an HTLC returns some of those msat, and a count
> and preimage (new fields).
>
> How many msat can you take for forwarding? That depends on you
> presenting a series of preimages (which chain into a final hash given in
> the HTLC add), which you get by decoding the onion. You get to keep 50
> msat[1] per preimage you present[2].
>
> So, how many preimages does the user have to give to have you forward
> the payment? That depends. The base rate is 16 preimages, but subtract
> one for each leading 4 zero bits of the SHA256(blockhash | hmac) of the
> onion. The blockhash is the hash of the block specified in the onion:
> reject if it's not in the last 3 blocks[3].
>
> This simply adds some payment noise, while allowing a hashcash style
> tradeoff of sats for work.
>
> The final node gets some variable number of preimages, which adds noise.
> It should take all and subtract from the minimum required invoice amount
> on success, or take some random number on failure.
>
> This leaks some forward information, and makes an explicit tradeoff for
> the sender between amount spent and privacy, but it's the best I've been
> able to come up with.
>
> Thoughts?
> Rusty.
>
> [1] If we assume $1 per GB, $10k per BTC and 64k messages, we get about
> 655msat per message. Flat pricing for simplicity; we're trying to
> prevent spam, not create a spam market.
> [2] Actually, a number and a single preimage; you can check this is
> indeed the n'th preimage.
> [3] This reduces incentive to grind the damn things in advance, though
> maybe that's dumb? We can also use a shorter hash (siphash?), or
> even truncated SHA256 (128 bits).
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20200218/59b4ba73/attachment.html>;

📅 Original date posted:2019-11-07 📝 Original message: > ...

2023-06-09T14:57:12+02:00

In reply to nevent1q…g0pj
_________________________

📅 Original date posted:2019-11-07
📝 Original message:
>
> > Isn't spam something that can also be addressed by using rate limits for
> > failures? If all relevant nodes on the network employ rate limits, they
> can
> > isolate the spammer and diminish their disruptive abilities.
>
> Sure, once the spammer has jammed up the network, he'll be stopped. So
> will everyone else. Conner had a proposal like this which didn't work,
> IIRC.
>

Do you have ref to this proposal?

Imagine the following setup: a network of nodes that trust each other (as
far as spam is concerned) applies a 100 htlc/sec rate limit to the channels
between themselves. Channels to untrusted nodes get a rate of only 1
htlc/sec. Assuming the spammer isn't a trusted node, they can only spam at
1 htlc/s and won't jam up the network?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20191107/f34b9e63/attachment.html>;

📅 Original date posted:2019-10-26 📝 Original message: We ...

2023-06-09T14:56:52+02:00

In reply to nevent1q…70cj
_________________________

📅 Original date posted:2019-10-26
📝 Original message:
We started to look at the `push_me` outputs again. Will refer to them as
`anchor` outputs from now on, to prevent confusion with `push_msat` on the
`open_channel` message.

The cpfp carve-out https://github.com/bitcoin/bitcoin/pull/15681 has been
merged and for reasons described earlier in this thread, we now need to add
a csv time lock to every non-anchor output on the commitment transaction.

To realize this, we are currently considering the following changes:

* Add `to_remote_delay OP_CHECKSEQUENCEVERIFY OP_DROP` to the `to_remote`
output. `to_remote_delay` is the csv delay that the remote party accepted
in the funding flow for their outputs. This not only ensures that the
carve-out works as intended, but also removes the incentive to game the
other party into force-closing. If desired, both parties can still agree to
have different `to_self_delay` values.

* Add `1 OP_CHECKSEQUENCEVERIFY OP_DROP` to the non-revocation clause of
the HTLC outputs.

For the anchor outputs we consider:

* Output type: normal P2WKH. At one point, an additional spending path was
proposed that was unconditional except for a 10 block csv lock. The
intention of this was to prevent utxo set pollution by allowing anyone to
clean up. This however also opens up the possibility for an attacker to
'use up' the cpfp carve-out after those 10 blocks. If the user A is offline
for that period of time, a malicious peer B may already have broadcasted
the commitment tx and pinned down user A's anchor output with a low fee
child. That way, the commitment tx could still remain unconfirmed while an
important htlc expires.

* For the keys to use for `to_remote_anchor` and `to_local_anchor`, we’d
like to introduce new addresses that both parties communicate in the
`open_channel` and `accept_channel` messages. We don’t want to reuse the
main commitment output addresses, because those may (at some point) be cold
storage addresses and the cpfp is likely to happen from a hot wallet.

* Within each version of the commitment transaction, both anchors always
have equal values and are paid for by the initiator. The value of the
anchors is the dust limit that was negotiated in the `open_channel` or
`accept_channel` message of the party that publishes the transaction. It
means that the definitive balance of an endpoint is dependent on which
version of the commitment transaction confirms. This however is nothing
new. In the current commitment format, there are always two or three valid
versions of the commitment transaction (local, remote and sometimes the not
yet revoked previous remote tx) which can have slightly different balances.
For the initiator, it is important to validate the other party's dust
limit. The initiator pays for it and doesn't want to give away more free
money than necessary.

Furthermore, there doesn’t seem to be a compelling reason anymore for
tweaking the keys (new insights into watchtower designs, encrypt by txid).
Therefore we think we can remove them entirely in this new commitment
format and require less channel state data to sweep the outputs.

Joost

On Wed, Nov 21, 2018 at 3:17 AM Rusty Russell <rusty at rustcorp.com.au> wrote:

> I'm also starting to implement this, to see what I missed!
>
> Original at https://github.com/lightningnetwork/lightning-rfc/pull/513
>
> Pasted here for your reading convenience:
>
> - Option is sticky; it set at open time, it stays with channel
> - I didn't want to have to handle penalty txs on channels which switch
> - We could, however, upgrade on splice.
> - Feerate is fixed at 253
> - `feerate_per_kw` is still in open /accept (just ignored): multifund
> may want it.
> - closing tx negotiates *upwards* not *downwards*
> - Starting from base fee of commitment tx = 282 satoshi.
> - to_remote output is always CSV delayed.
> - pushme outputs are paid for by funder, but only exist if the matching
> to_local/remote output exists.
> - After 10 blocks, they become anyone-can-spend (they need to see the
> to-local/remote witness script though).
> - remotepubkey is not rotated.
> - You must spend your pushme output; you may sweep for others.
>
> Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
>
> diff --git a/02-peer-protocol.md b/02-peer-protocol.md
> index 7cf9ebf..6ec1155 100644
> --- a/02-peer-protocol.md
> +++ b/02-peer-protocol.md
> @@ -133,7 +133,9 @@ node can offer.
> (i.e. 1/4 the more normally-used 'satoshi per 1000 vbytes') that this
> side will pay for commitment and HTLC transactions, as described in
> [BOLT #3](03-transactions.md#fee-calculation) (this can be adjusted
> -later with an `update_fee` message).
> +later with an `update_fee` message). Note that if
> +`option_simplified_commitment` is negotiated, this `feerate_per_kw`
> +is treated as 253 for all transactions.
>
> `to_self_delay` is the number of blocks that the other node's to-self
> outputs must be delayed, using `OP_CHECKSEQUENCEVERIFY` delays; this
> @@ -208,7 +210,8 @@ The receiving node MUST fail the channel if:
> - `push_msat` is greater than `funding_satoshis` * 1000.
> - `to_self_delay` is unreasonably large.
> - `max_accepted_htlcs` is greater than 483.
> - - it considers `feerate_per_kw` too small for timely processing or
> unreasonably large.
> + - if `option_simplified_commitment` is not negotiated:
> + - it considers `feerate_per_kw` too small for timely processing or
> unreasonably large.
> - `funding_pubkey`, `revocation_basepoint`, `htlc_basepoint`,
> `payment_basepoint`, or `delayed_payment_basepoint`
> are not valid DER-encoded compressed secp256k1 pubkeys.
> - `dust_limit_satoshis` is greater than `channel_reserve_satoshis`.
> @@ -228,7 +231,7 @@ The *channel reserve* is specified by the peer's
> `channel_reserve_satoshis`: 1%
>
> The sender can unconditionally give initial funds to the receiver using a
> non-zero `push_msat`, but even in this case we ensure that the funder has
> sufficient remaining funds to pay fees and that one side has some amount it
> can spend (which also implies there is at least one non-dust output). Note
> that, like any other on-chain transaction, this payment is not certain
> until the funding transaction has been confirmed sufficiently (with a
> danger of double-spend until this occurs) and may require a separate method
> to prove payment via on-chain confirmation.
>
> -The `feerate_per_kw` is generally only of concern to the sender (who pays
> the fees), but there is also the fee rate paid by HTLC transactions; thus,
> unreasonably large fee rates can also penalize the recipient.
> +The `feerate_per_kw` is generally only of concern to the sender (who pays
> the fees), but there is also the fee rate paid by HTLC transactions; thus,
> unreasonably large fee rates can also penalize the recipient. It is
> ignored for `option_simplified_commitment`.
>
> Separating the `htlc_basepoint` from the `payment_basepoint` improves
> security: a node needs the secret associated with the `htlc_basepoint` to
> produce HTLC signatures for the protocol, but the secret for the
> `payment_basepoint` can be in cold storage.
>
> @@ -340,6 +343,12 @@ This message introduces the `channel_id` to identify
> the channel. It's derived f
>
> #### Requirements
>
> +Both peers:
> + - if `option_simplified_commitment` was negotiated:
> + - `option_simplified_commitment` applies to all commitment and HTLC
> transactions
> + - otherwise:
> + - `option_simplified_commitment` does not apply to any commitment or
> HTLC transactions
> +
> The sender MUST set:
> - `channel_id` by exclusive-OR of the `funding_txid` and the
> `funding_output_index` from the `funding_created` message.
> - `signature` to the valid signature, using its `funding_pubkey` for
> the initial commitment transaction, as defined in [BOLT
> #3](03-transactions.md#commitment-transaction).
> @@ -351,6 +360,12 @@ The recipient:
> - on receipt of a valid `funding_signed`:
> - SHOULD broadcast the funding transaction.
>
> +#### Rationale
> +
> +We decide on `option_simplified_commitment` at this point when we first
> have to generate the commitment
> +transaction. Even if a later reconnection does not negotiate this
> parameter, this channel will honor it.
> +This simplifies channel state, particularly penalty transaction handling.
> +
> ### The `funding_locked` Message
>
> This message indicates that the funding transaction has reached the
> `minimum_depth` asked for in `accept_channel`. Once both nodes have sent
> this, the channel enters normal operating mode.
> @@ -508,8 +523,11 @@ The funding node:
> - SHOULD send a `closing_signed` message.
>
> The sending node:
> - - MUST set `fee_satoshis` less than or equal to the
> - base fee of the final commitment transaction, as calculated in [BOLT
> #3](03-transactions.md#fee-calculation).
> + - if `option_upfront_shutdown_script` applies to the final commitment
> transaction:
> + - MUST set `fee_satoshis` greater than or equal to 282.
> + - otherwise:
> + - MUST set `fee_satoshis` less than or equal to the
> + base fee of the final commitment transaction, as calculated in
> [BOLT #3](03-transactions.md#fee-calculation).
> - SHOULD set the initial `fee_satoshis` according to its
> estimate of cost of inclusion in a block.
> - MUST set `signature` to the Bitcoin signature of the close
> @@ -543,9 +561,18 @@ progress is made, even if only by a single satoshi at
> a time. To avoid
> keeping state and to handle the corner case, where fees have shifted
> between disconnection and reconnection, negotiation restarts on
> reconnection.
>
> -Note there is limited risk if the closing transaction is
> -delayed, but it will be broadcast very soon; so there is usually no
> -reason to pay a premium for rapid processing.
> +In the `option_simplified_commitment` case, the fees on the commitment
> +transaction itself are minimal (it is assumed that a child transaction
> will
> +supply additional fee incentive), so that forms a floor for negotiation.
> +[BOLT #3](03-transactions.md#fee-calculation), gives 282 satoshis (1116
> +weight, 254 `feerate_per_kw`).
> +
> +Otherwise, the commitment transaction usually pays a premium fee, so that
> +forms a ceiling.
> +
> +Note there is limited risk if the closing transaction is delayed, but it
> will
> +be broadcast very soon; so there is usually no reason to pay a premium for
> +rapid processing.
>
> ## Normal Operation
>
> @@ -763,7 +790,10 @@ is destined, is described in [BOLT
> #4](04-onion-routing.md).
> A sending node:
> - MUST NOT offer `amount_msat` it cannot pay for in the
> remote commitment transaction at the current `feerate_per_kw` (see
> "Updating
> -Fees") while maintaining its channel reserve.
> +Fees") while maintaining its channel reserve
> + - if `option_simplified_commitment` applies to this commitment
> transaction and the sending
> + node is the funder:
> + - MUST be able to additionally pay for `to_local_pushme` and
> `to_remote_pushme` above its reserve.
> - MUST offer `amount_msat` greater than 0.
> - MUST NOT offer `amount_msat` below the receiving node's
> `htlc_minimum_msat`
> - MUST set `cltv_expiry` less than 500000000.
> @@ -782,7 +812,7 @@ Fees") while maintaining its channel reserve.
> A receiving node:
> - receiving an `amount_msat` equal to 0, OR less than its own
> `htlc_minimum_msat`:
> - SHOULD fail the channel.
> - - receiving an `amount_msat` that the sending node cannot afford at the
> current `feerate_per_kw` (while maintaining its channel reserve):
> + - receiving an `amount_msat` that the sending node cannot afford at the
> current `feerate_per_kw` (while maintaining its channel reserve and any
> `to_local_pushme` and `to_remote_pushme` fees):
> - SHOULD fail the channel.
> - if a sending node adds more than its `max_accepted_htlcs` HTLCs to
> its local commitment transaction, OR adds more than its
> `max_htlc_value_in_flight_msat` worth of offered HTLCs to its local
> commitment transaction:
> @@ -997,6 +1027,11 @@ A node:
>
> ### Updating Fees: `update_fee`
>
> +If `option_simplified_commitment` applies to the commitment transaction,
> +`update_fee` is never used: the `feerate_per_kw` is always considered
> 253, but
> +the funder also pays 2000 satoshi for the `to_local_pushme` and
> +`to_remote_pushme` outputs.
> +
> An `update_fee` message is sent by the node which is paying the
> Bitcoin fee. Like any update, it's first committed to the receiver's
> commitment transaction and then (once acknowledged) committed to the
> @@ -1020,13 +1055,19 @@ given in [BOLT
> #3](03-transactions.md#fee-calculation).
> #### Requirements
>
> The node _responsible_ for paying the Bitcoin fee:
> - - SHOULD send `update_fee` to ensure the current fee rate is sufficient
> (by a
> + - if `option_simplified_commitment` applies to the commitment
> transaction:
> + - MUST NOT send `update_fee`.
> + - otherwise:
> + - SHOULD send `update_fee` to ensure the current fee rate is
> sufficient (by a
> significant margin) for timely processing of the commitment
> transaction.
>
> The node _not responsible_ for paying the Bitcoin fee:
> - MUST NOT send `update_fee`.
>
> A receiving node:
> + - if `option_simplified_commitment` applies to the commitment
> transaction:
> + - SHOULD fail the channel.
> + - MUST NOT update the `feerate_per_kw`.
> - if the `update_fee` is too low for timely processing, OR is
> unreasonably large:
> - SHOULD fail the channel.
> - if the sender is not responsible for paying the Bitcoin fee:
> @@ -1038,7 +1079,12 @@ A receiving node:
>
> #### Rationale
>
> -Bitcoin fees are required for unilateral closes to be effective —
> +Fee adjustments are unnecessary for `option_simplified_commitment` which
> +relies on "pushme" outputs and a child transaction which will provide
> +additional fee incentive which can be calculated at the time it is spent,
> and
> +replaced by higher-fee children if required.
> +
> +Without this option, bitcoin fees are required for unilateral closes to
> be effective —
> particularly since there is no general method for the broadcasting node
> to use
> child-pays-for-parent to increase its effective fee.
>
> diff --git a/03-transactions.md b/03-transactions.md
> index e769961..440bd0d 100644
> --- a/03-transactions.md
> +++ b/03-transactions.md
> @@ -82,6 +82,8 @@ To allow an opportunity for penalty transactions, in
> case of a revoked commitmen
> The reason for the separate transaction stage for HTLC outputs is so that
> HTLCs can timeout or be fulfilled even though they are within the
> `to_self_delay` delay.
> Otherwise, the required minimum timeout on HTLCs is lengthened by this
> delay, causing longer timeouts for HTLCs traversing the network.
>
> +If `option_simplified_commitment` applies to the commitment transaction,
> then the `to_self_delay` used for all transactions is the greater of the
> `to_self_delay` sent by each peer. Otherwise, each peer sends the
> `to_self_delay` to be used for the other peer's commitment amd HTLC
> transactions.
> +
> The amounts for each output MUST be rounded down to whole satoshis. If
> this amount, minus the fees for the HTLC transaction, is less than the
> `dust_limit_satoshis` set by the owner of the commitment transaction, the
> output MUST NOT be produced (thus the funds add to fees).
>
> #### `to_local` Output
> @@ -109,7 +111,40 @@ If a revoked commitment transaction is published, the
> other party can spend this
>
> #### `to_remote` Output
>
> -This output sends funds to the other peer and thus is a simple P2WPKH to
> `remotepubkey`.
> +This output sends funds to the other peer, thus is not encumbered by a
> +revocation private key.
> +
> +If `option_simplified_commitment` applies to the commitment transaction,
> the `to_remote` output is delayed similarly to the `to_local` output, and
> is to a fixed key:
> +
> + `to_self_delay`
> + OP_CSV
> + OP_DROP
> + <remote_pubkey>
> +
> +The output is spent by a transaction with `nSequence` field set to
> `to_self_delay` (which can only be valid after that duration has passed)
> and witness:
> +
> + <remote_sig>
> +
> +Otherwise, this output is a simple P2WPKH to `remotepubkey`.
> +
> +
> +#### `to_local_pushme` and `to_remote_pushme` Output
> (option_simplified_commitment)
> +
> +This output can be spent by the local and remote nodes respectivey to
> provide incentive to mine the transaction, using child-pays-for-parent.
> They are only added if the `to_local` and `to_remote` outputs exist,
> respectively.
> +
> + OP_DEPTH
> + OP_IF
> + <pubkey> OP_CHECKSIG
> + OP_ELSE
> + 10 OP_CSV
> + OP_ENDIF
> +
> +The `<pubkey>` is `<local_delayedpubkey>` to `to_local_pushme` and
> +`<remote_delayedpubkey>` for `to_remote_pushme`. The output amount is
> +1000 satoshi, to encourage spending of the output. Once the
> +`remote_pubkey` is revealed (by spending the `to_local` output) and
> +the commitment transaction is 10 blocks deep, anyone can spend it.
> +
>
> #### Offered HTLC Outputs
>
> @@ -294,6 +329,9 @@ The fee calculation for both commitment transactions
> and HTLC
> transactions is based on the current `feerate_per_kw` and the
> *expected weight* of the transaction.
>
> +Note that if `option_simplified_commitment` applies to the commitment
> +transaction then `feerate_per_kw` is 253.
> +
> The actual and expected weights vary for several reasons:
>
> * Bitcoin uses DER-encoded signatures, which vary in size.
> @@ -306,10 +344,12 @@ Thus, a simplified formula for *expected weight* is
> used, which assumes:
> * Signatures are 73 bytes long (the maximum length).
> * There are a small number of outputs (thus 1 byte to count them).
> * There are always both a `to_local` output and a `to_remote` output.
> +* (if `option_simplified_commitment`) there are always both a
> `to_local_pushme` and `to_remote_pushme` output.
>
> This yields the following *expected weights* (details of the computation
> in [Appendix A](#appendix-a-expected-weights)):
>
> - Commitment weight: 724 + 172 * num-untrimmed-htlc-outputs
> + Commitment weight (no option_simplified_commitment): 724 + 172 *
> num-untrimmed-htlc-outputs
> + Commitment weight (option_simplified_commitment: 1116 + 172 *
> num-untrimmed-htlc-outputs
> HTLC-timeout weight: 663
> HTLC-success weight: 703
>
> @@ -366,7 +406,7 @@ outputs) is 7140 satoshi. The final fee may be even
> higher if the
>
> ### Fee Payment
>
> -Base commitment transaction fees are extracted from the funder's amount;
> if that amount is insufficient, the entire amount of the funder's output is
> used.
> +Base commitment transaction fees and amounts for `to_local_pushme` and
> `to_remote_pushme` outputs are extracted from the funder's amount; if that
> amount is insufficient, the entire amount of the funder's output is used.
>
> Note that after the fee amount is subtracted from the to-funder output,
> that output may be below `dust_limit_satoshis`, and thus will also
> @@ -390,23 +430,29 @@ committed HTLCs:
> 2. Calculate the base [commitment transaction fee](#fee-calculation).
> 3. Subtract this base fee from the funder (either `to_local` or
> `to_remote`),
> with a floor of 0 (see [Fee Payment](#fee-payment)).
> +4. If `option_simplified_commitment` applies to the commitment
> transaction,
> + subtract 2000 satoshis from the funder (either `to_local` or
> `to_remote`).
> 3. For every offered HTLC, if it is not trimmed, add an
> [offered HTLC output](#offered-htlc-outputs).
> 4. For every received HTLC, if it is not trimmed, add an
> [received HTLC output](#received-htlc-outputs).
> 5. If the `to_local` amount is greater or equal to `dust_limit_satoshis`,
> add a [`to_local` output](#to_local-output).
> +6. If `option_simplified_commitment` applies to the commitment
> transaction,
> + and `to_local` was added, add `to_local_pushme`.
> 6. If the `to_remote` amount is greater or equal to `dust_limit_satoshis`,
> add a [`to_remote` output](#to_remote-output).
> +6. If `option_simplified_commitment` applies to the commitment
> transaction,
> + and `to_remote` was added, add `to_remote_pushme`.
> 7. Sort the outputs into [BIP 69
> order](#transaction-input-and-output-ordering).
>
> # Keys
>
> ## Key Derivation
>
> -Each commitment transaction uses a unique set of keys: `localpubkey` and
> `remotepubkey`.
> +Each commitment transaction uses a unique `localpubkey`, and a
> `remotepubkey`.
> The HTLC-success and HTLC-timeout transactions use `local_delayedpubkey`
> and `revocationpubkey`.
> -These are changed for every transaction based on the
> `per_commitment_point`.
> +These are changed for every transaction based on the
> `per_commitment_point`, with the exception of `remotepubkey` if
> `option_simplified_commitment` is negotiated.
>
> The reason for key change is so that trustless watching for revoked
> transactions can be outsourced. Such a _watcher_ should not be able to
> @@ -419,8 +465,9 @@ avoid storage of every commitment transaction, a
> _watcher_ can be given the
> the scripts required for the penalty transaction; thus, a _watcher_ need
> only be
> given (and store) the signatures for each penalty input.
>
> -Changing the `localpubkey` and `remotepubkey` every time ensures that
> commitment
> -transaction ID cannot be guessed; every commitment transaction uses an ID
> +Changing the `localpubkey` every time ensures that commitment
> +transaction ID cannot be guessed except in the trivial case where there
> is no
> +`to_local` output, as every commitment transaction uses an ID
> in its output script. Splitting the `local_delayedpubkey`, which is
> required for
> the penalty transaction, allows it to be shared with the _watcher_ without
> revealing `localpubkey`; even if both peers use the same _watcher_,
> nothing is revealed.
> @@ -434,14 +481,13 @@ For efficiency, keys are generated from a series of
> per-commitment secrets
> that are generated from a single seed, which allows the receiver to
> compactly
> store them (see [below](#efficient-per-commitment-secret-storage)).
>
> -### `localpubkey`, `remotepubkey`, `local_htlcpubkey`,
> `remote_htlcpubkey`, `local_delayedpubkey`, and `remote_delayedpubkey`
> Derivation
> +### `localpubkey``local_htlcpubkey`, `remote_htlcpubkey`,
> `local_delayedpubkey`, and `remote_delayedpubkey` Derivation
>
> These pubkeys are simply generated by addition from their base points:
>
> pubkey = basepoint + SHA256(per_commitment_point || basepoint) * G
>
> -The `localpubkey` uses the local node's `payment_basepoint`; the
> `remotepubkey`
> -uses the remote node's `payment_basepoint`; the `local_delayedpubkey`
> +The `localpubkey` uses the local node's `payment_basepoint`; the
> `local_delayedpubkey`
> uses the local node's `delayed_payment_basepoint`; the `local_htlcpubkey`
> uses the
> local node's `htlc_basepoint`; and the `remote_delayedpubkey` uses the
> remote
> node's `delayed_payment_basepoint`.
> @@ -451,6 +497,17 @@ secrets are known (i.e. the private keys
> corresponding to `localpubkey`, `local_
>
> privkey = basepoint_secret + SHA256(per_commitment_point || basepoint)
>
> +### `remotepubkey` Derivation
> +
> +If `option_simplified_commitment` is negotiated the `remotepubkey` is
> simply the remote node's `payment_basepoint`, otherwise it is calculated as
> above using the remote node's `payment_basepoint`.
> +
> +The simplified derivation means that a node can spend a commitment
> +transaction even if it has lost data and doesn't know the
> +corresponding `payment_basepoint`. A watchtower could correlate
> +transactions given to it which only have a `to_remote` output if it
> +sees one of them onchain, but such transactions do not need any
> +enforcement and should not be handed to a watchtower.
> +
> ### `revocationpubkey` Derivation
>
> The `revocationpubkey` is a blinded key: when the local node wishes to
> create a new
> @@ -636,12 +693,22 @@ The *expected weight* of a commitment transaction is
> calculated as follows:
> - var_int: 1 byte (pk_script length)
> - pk_script (p2wsh): 34 bytes
>
> - output_paying_to_remote: 31 bytes
> + output_paying_to_remote (no option_simplified_commitment): 31 bytes
> - value: 8 bytes
> - var_int: 1 byte (pk_script length)
> - pk_script (p2wpkh): 22 bytes
>
> - htlc_output: 43 bytes
> + output_paying_to_remote (option_simplified_commitment): 43 bytes
> + - value: 8 bytes
> + - var_int: 1 byte (pk_script length)
> + - pk_script (p2wsh): 34 bytes
> +
> + output_pushme (option_simplified_commitment): 43 bytes
> + - value: 8 bytes
> + - var_int: 1 byte (pk_script length)
> + - pk_script (p2wsh): 34 bytes
> +
> + htlc_output: 43 bytes
> - value: 8 bytes
> - var_int: 1 byte (pk_script length)
> - pk_script (p2wsh): 34 bytes
> @@ -650,7 +717,7 @@ The *expected weight* of a commitment transaction is
> calculated as follows:
> - flag: 1 byte
> - marker: 1 byte
>
> - commitment_transaction: 125 + 43 * num-htlc-outputs bytes
> + commitment_transaction (no option_simplified_commitment): 125 +
> 43 * num-htlc-outputs bytes
> - version: 4 bytes
> - witness_header <---- part of the witness data
> - count_tx_in: 1 byte
> @@ -663,15 +730,32 @@ The *expected weight* of a commitment transaction is
> calculated as follows:
> ....htlc_output's...
> - lock_time: 4 bytes
>
> + commitment_transaction (option_simplified_commitment): 223 + 43 *
> num-htlc-outputs bytes
> + - version: 4 bytes
> + - witness_header <---- part of the witness data
> + - count_tx_in: 1 byte
> + - tx_in: 41 bytes
> + funding_input
> + - count_tx_out: 1 byte
> + - tx_out: 172 + 43 * num-htlc-outputs bytes
> + output_paying_to_remote,
> + output_paying_to_local,
> + output_pushme,
> + output_pushme,
> + ....htlc_output's...
> + - lock_time: 4 bytes
> +
> Multiplying non-witness data by 4 results in a weight of:
>
> - // 500 + 172 * num-htlc-outputs weight
> + // 500 + 172 * num-htlc-outputs weight (no
> option_simplified_commitment)
> + // 892 + 172 * num-htlc-outputs weight
> (option_simplified_commitment)
> commitment_transaction_weight = 4 * commitment_transaction
>
> // 224 weight
> witness_weight = witness_header + witness
>
> - overall_weight = 500 + 172 * num-htlc-outputs + 224 weight
> + overall_weight (no option_simplified_commitment) = 500 + 172 *
> num-htlc-outputs + 224 weight
> + overall_weight (option_simplified_commitment) = 892 + 172 *
> num-htlc-outputs + 224 weight
>
> ## Expected Weight of HTLC-timeout and HTLC-success Transactions
>
> diff --git a/05-onchain.md b/05-onchain.md
> index 231c209..c5fb5e1 100644
> --- a/05-onchain.md
> +++ b/05-onchain.md
> @@ -89,21 +89,29 @@ trigger any action.
> # Commitment Transaction
>
> The local and remote nodes each hold a *commitment transaction*. Each of
> these
> -commitment transactions has four types of outputs:
> +commitment transactions has six types of outputs:
>
> 1. _local node's main output_: Zero or one output, to pay to the *local
> node's*
> -commitment pubkey.
> +delayed pubkey.
> 2. _remote node's main output_: Zero or one output, to pay to the *remote
> node's*
> -commitment pubkey.
> +pubkey.
> +1. _local node's push output_: Zero or one output, to pay to the *local
> node's*
> +delayed pubkey.
> +2. _remote node's push output_: Zero or one output, to pay to the *remote
> node's*
> +pubkey.
> 3. _local node's offered HTLCs_: Zero or more pending payments (*HTLCs*),
> to pay
> the *remote node* in return for a payment preimage.
> 4. _remote node's offered HTLCs_: Zero or more pending payments
> (*HTLCs*), to
> pay the *local node* in return for a payment preimage.
>
> To incentivize the local and remote nodes to cooperate, an
> `OP_CHECKSEQUENCEVERIFY`
> -relative timeout encumbers the *local node's outputs* (in the *local
> node's
> +relative timeout encumbers some outputs: the *local node's outputs* (in
> the *local node's
> commitment transaction*) and the *remote node's outputs* (in the *remote
> node's
> -commitment transaction*). So for example, if the local node publishes its
> +commitment transaction*). If `option_simplified_commitment` applies
> +to the commitment transaction, then the *to_remote* output of each
> commitment is
> +identically encumbered, for fairness.
> +
> +Without `option_simplified_commitment`, if the local node publishes its
> commitment transaction, it will have to wait to claim its own funds,
> whereas the remote node will have immediate access to its own funds. As a
> consequence, the two commitment transactions are not identical, but they
> are
> @@ -140,6 +148,11 @@ A node:
> - otherwise:
> - MUST use the *last commitment transaction*, for which it has a
> signature, to perform a *unilateral close*.
> + - MUST spend any `to_local_pushme` output, providing sufficient
> fees as incentive to include the commitment transaction in a block
> + - SHOULD use [replace-by-fee](
> https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki) or other
> mechanism on the spending transaction if it proves insufficient for timely
> inclusion in a block.
> +
> +A node:
> + - MAY monitor the blockchain for unspent `to_local_pushme` and
> `to_remote_pushme` outputs and try to spend them after 10 confirmations.
>
> ## Rationale
>
> @@ -154,7 +167,8 @@ need not consume resources monitoring the channel
> state.
> There exists a bias towards preferring mutual closes over unilateral
> closes,
> because outputs of the former are unencumbered by a delay and are directly
> spendable by wallets. In addition, mutual close fees tend to be less
> exaggerated
> -than those of commitment transactions. So, the only reason not to use the
> +than those of commitment transactions (or in the case of
> `option_simplified_commitment`,
> +the commitment transaction may require a child transaction to cause it to
> be mined). So, the only reason not to use the
> signature from `closing_signed` would be if the fee offered was too small
> for
> it to be processed.
>
> diff --git a/09-features.md b/09-features.md
> index d06fcff..caea38b 100644
> --- a/09-features.md
> +++ b/09-features.md
> @@ -26,6 +26,7 @@ These flags may only be used in the `init` message:
> | 3 | `initial_routing_sync` | Indicates that the sending node needs a
> complete routing information dump | [BOLT
> #7](07-routing-gossip.md#initial-sync) |
> | 4/5 | `option_upfront_shutdown_script` | Commits to a shutdown
> scriptpubkey when opening channel | [BOLT
> #2](02-peer-protocol.md#the-open_channel-message) |
> | 6/7 | `gossip_queries` | More sophisticated gossip control |
> [BOLT #7](07-routing-gossip.md#query-messages) |
> +| 8/9 | `option_simplified_commitment` | Simplified commitment
> transactions | [BOLT #3](03-transactions.md) |
>
> ## Assigned `globalfeatures` flags
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20191026/fe4f49f3/attachment-0001.html>;

📅 Original date posted:2019-06-12 📝 Original message: Hello ...

2023-06-09T14:55:07+02:00

In reply to nevent1q…ffv2
_________________________

📅 Original date posted:2019-06-12
📝 Original message:
Hello list,

In Lightning, the reliability of payments is dependent on the reliability
of the chosen route. Information about previous payment attempts helps to
select better routes and improve the payment experience. Therefore
implementations usually track the past performance of nodes and channels.
This can be as simple as a black list that contains previously failed
channels.

In order for this mechanism to be most effective, it is important to know
which node is to blame for a non-ideal payment attempt.

Non-ideal payment attempts are not only failed payment attempts (either
instantly failed or after a delay), but also successful payments for which
it took a long time to receive the `htlc_fulfill` message.

For non-ideal payment attempts, we are currently not always able to
determine the node that should be penalized. In particular:
* If an attempt takes long to complete (settle or fail), we have no
information that points us to the source of the delay.
* Nodes can return a corrupt failure message. When this message arrives at
the sender after a number of encryption rounds, the sender is no longer
able to pinpoint the node that failed the payment.

A potential solution is to change the failure message such that every hop
along the backward path adds an hmac to the message (currently only the
error source authenticates the message). This allows the source of a
corruption to be narrowed down to a pair of nodes, which is enough to
properly apply a penalty.

In addition to that, all hops could add two timestamps to the failure
message: the htlc add time and the htlc fail time. Using this information,
the sender of the payment can identify the source of the delay down to,
again, a pair of nodes. Those timestamps could be added to the settle
message as well, to also allow diagnostics on slow settled payments.

The challenge here is to design the failure message format in such a way
that hops cannot learn their position in the path. Just appending
timestamps and hmacs to a variable length message would reveal the distance
between a node and the error source.

A fixed length message in which hops shift some previous (unused) data out
from the message to create space to add their own data does not seem to
work. What happens is that the earlier nodes calculate their hmac over data
that is shifted out and cannot be recovered anymore by the sender. The
sender has no way to verify the hmac in that case. Regenerating the shifted
out data (similar to deterministic padding on the forward path) isn't a
solution either, because a node may modify that (unused) data before
passing the message on. This would invalidate all hmacs, denying the sender
from locating the responsible node.

One space-inefficient solution is to have every hop add hmacs for every
possible (real) message length, but this would require n^2 hmacs in total
(20*20*32 bytes). Half of these could be discarded along the way, but it
would still leave 10*20*32=6.4kb of hmacs.

Another direction might be to use a variable length message, but have the
error source add a seemingly random length padding. The actual length could
be deterministically derived from the shared secret, so that the erring
node cannot just not add padding. This obfuscates the distance to the error
source somewhat, but still reveals a bit of information. If one knows that
the padding length is somewhere between 0 and 20 blocks worth of bytes, a
message length of say 25 blocks would reveal that the err source is at
least 5 hops away. It could be a fair trade-off though.

An alternative to all of this is to try to locate bad nodes by probing with
different route lengths and coming from different angles. This will however
require more attempts and more complex processing of the outcomes. There is
also a level of indirectness because not all information is gathered in a
single roundtrip. And in addition to that, a malicious node may somehow act
differently if it manages to recognize the probes.

I'd be interested to hear your opinions about the importance of being able
to locate bad nodes irrefutably, as well as ideas around the failure
message format.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190612/f110fbb1/attachment.html>;

📅 Original date posted:2023-06-02 🗒️ Summary of this ...

2023-06-08T01:22:06+02:00

In reply to nevent1q…rtvy
_________________________

📅 Original date posted:2023-06-02
🗒️ Summary of this message: Proposal to make the taproot annex available in a non-structured form by defining any annex that begins with '0' as free-form, allowing immediate utilization and future flexibility.
📝 Original message:Hi,

As it stands, the taproot annex is consensus valid but non-standard. The
conversations around standardization seem to be leaning towards the
adoption of a flexible Type-Length-Value (TLV) format [1]. There's no doubt
that this approach has considerable potential. However, settling on an
exact format may require a significant amount of time.

In the interim, the benefits of making the annex available in a
non-structured form are both evident and immediate. By allowing developers
to utilize the taproot annex without delay, we can take advantage of its
features today, without the need to wait for the finalization of a more
lengthy standardization process.

With this in view, I am proposing that we define any annex that begins with
'0' as free-form, without any additional constraints. This strategy offers
several distinct benefits:

Immediate utilization: This opens the door for developers to make use of
the taproot annex for a variety of applications straight away, thus
eliminating the need to wait for the implementation of TLV or any other
structured format.

Future flexibility: Assigning '0'-beginning annexes as free-form keeps our
options open for future developments and structure improvements. As we
forge ahead in determining the best way to standardize the annex, this
strategy ensures we do not limit ourselves by setting its structure in
stone prematurely.

Chainspace efficiency: Non-structured data may require fewer bytes compared
to a probable TLV format, which would necessitate the encoding of length
even when there's only a single field.

In conclusion, adopting this approach will immediately broaden the
utilization scope of the taproot annex while preserving the possibility of
transitioning to a more structured format in the future. I believe this is
a pragmatic and efficient route, one that can yield substantial benefits in
both the short and long term.

Joost

[1] https://github.com/bitcoin/bips/pull/1381
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230602/4411b85b/attachment-0001.html>;

📅 Original date posted:2023-05-30 🗒️ Summary of this ...

2023-06-08T01:22:01+02:00

In reply to nevent1q…qg0j
_________________________

📅 Original date posted:2023-05-30
🗒️ Summary of this message: Block templates and weak blocks can be relayed through a more flexible Nostr relay, improving fee estimation and providing valuable insights for miners and full nodes. This could also be a playground for experimenting with various relay and mining ideas.
📝 Original message:Hi David,

> A block template is an ordered list of raw transactions that can all be
> included in the next block (with some space reserved for a coinbase
> transaction). A full node can validate those transactions and calculate
> how much fee they pay. A Nostr relay can simply relay almost[1] any
> template that pays more fees than the previous best template it saw for
> the next block. That can be more flexible than the current
> implementation of submitblock with package relay which still enforces a
> lot of the rules that helps keep a regular relay node safe from DoS and
> a miner node able to select mineable transactions quickly.
>

Interesting idea! This would also make it easy for external services to try
to do the best possible block building using advanced algorithms. Miners
would just select the best template available from various sources
including nostr.

> A weak block is a block whose header doesn't quite hash to low enough of
> a value to be included on the chain. It still takes an extraordinary
> amount of hashrate to produce, so it's inherently DoS resistant. If
> miners are producing block that include transactions not seen by typical
> relay nodes, that can reduce the efficiency and effectiveness of BIP152
> compact block relay, which hurts the profitability of miners of custom
> blocks. To compensate, miners could relay weak blocks through Nostr to
> full nodes and other miners so that they could quickly relay and accept
> complete blocks that later included the same custom transactions. This
> would also help fee estimation and provide valuable insights to those
> trying to get their transactions included into the next block.
>

I believe this would be useful right away, wouldn't it? Looking at
mempool.space's block audit, there are definitely blocks that have a
"surprising" content and might take long to download.

The anti-dos measures that you describe for both weak blocks and block
templates seem very robust, but they would require a more intelligent nostr
relay to enforce. Not sure if it is still allowed to call it nostr at that
point. Perhaps it becomes more of a specialised bitcoin relay. btcstr -
"bitcoin stuff transmitted by relays".

Regarding size, the block template and weak block could both be sent in
> BIP152 compact block format as a diff against the expected contents of a
> typical node, allowing Alice to send just a small amount of additional
> data for relay over what she'd have to send anyway for each transaction
> in a package. (Although it's quite possible that BetterHash or Stratum
> v2 have even better solutions, possibly already implemented.)
>

Sounds like a great way to repurpose what already exists to reduce resource
usage for these additional message types.

> If nothing else, I think Nostr could provide an interesting playground
> for experimenting with various relay and mining ideas we've talked about
> for years, so thanks again for working on this!
>

I think so too! The main question on my mind though is how to actually make
this work. There is a bit of a chicken-egg problem here with users and
miners possibly waiting for each other to adopt.

Joost
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230530/1f001ed5/attachment.html>;

📅 Original date posted:2023-05-23 🗒️ Summary of this ...

2023-06-08T01:22:00+02:00

In reply to nevent1q…u647
_________________________

📅 Original date posted:2023-05-23
🗒️ Summary of this message: A proposed alternative relay mechanism for Bitcoin transactions, called Nostr, could address limitations in the current P2P system and support non-standard transactions. Miners would listen for broadcasted transaction packages and insert them into their local mempool, potentially boosting the system's resilience and democratizing access to miner mempools. Nostr could also delegate responsibility for dealing with DoS threats to relays and introduce more flexibility in the system. A prototype has been developed and demonstrated in a video.
📝 Original message:Hi,

I write to get your thoughts on an alternative approach for Bitcoin
transaction relay, addressing some of the limitations in the current
peer-to-peer transaction relay system. To the best of my knowledge, the
credit for the original concept goes to Ben Carman. I felt it would be
beneficial to share the idea on this list to garner wider perspectives and
feedback.

The existing peer-to-peer (P2P) transaction relay system comes with a set
of limitations that may negatively impact applications, notably those like
Lightning that make extensive use of pre-signed transactions. A key
limitation lies in the system's inability to relay transaction packages.
This constraint can lead to HTLCs expiring before being swept, thereby
risking fund losses. In addition, the P2P system falls short in supporting
non-standard transactions, despite an established demand for such
transactions in the marketplace.

Nostr, an open and decentralized network of relays for public and ephemeral
messages between pseudonymous entities, could help address these
shortcomings. With the standards defined in NIP-89 [1], it becomes possible
to broadcast arbitrary Bitcoin transaction packages, overcoming one of the
key hurdles in the current relay system.

In this proposed alternative relay mechanism, miners would listen for these
broadcasted transaction packages and insert the packages into their local
mempool. They can take advantage of the `submitpackage` RPC, limited to
safe topologies only - specifically child and direct parents, tree only
[2]. This feature could serve as an interim solution for package relay
until it becomes available through the traditional P2P method.

A notable advantage of this approach is that it delegates the
responsibility of dealing with Denial-of-Service (DoS) threats to the
relays themselves. They could, for example, require a payment to mitigate
such concerns. There are in fact paid nostr relays already in operation.
This partitioning would result in a clear separation between the Bitcoin
transaction layer and DoS protection, introducing more flexibility in the
system and potentially boosting its resilience.

Implementing Nostr as a relay mechanism also has the potential to
democratize access to miner mempools, thus leveling the playing field in
the Bitcoin network. In the current state, those with direct connections or
certain privileges can more readily submit transactions to miners, perhaps
even through means as informal as email.

I have been working on a prototype of this concept (based on [3]) and have
captured its workings in a demonstration video [4].

Joost

[1] https://github.com/nostr-protocol/nips/pull/476

[2] https://github.com/bitcoin/bitcoin/pull/27609#issuecomment-1544414801

[3] https://github.com/benthecarman/nostr-tx-broadcast

[4] https://twitter.com/joostjgr/status/1658487013237211155
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20230523/6866548f/attachment.html>;