npub1za…uua2a

Last Notes

2023-06-09 15:01:23 CEST

- reply

📅 Original date posted:2020-10-20
📝 Original message:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi all,
Today we are writing to disclose the details of CVE-2020-26896 as a follow up to
the partial disclosure sent to lightning-dev [1].
## Abstract
Prior to v0.11.0-beta, an lnd node could be coerced into revealing an invoice
preimage for a forwarded HTLC with a colliding payment hash. This can be
exploited to a) weaken the victim's receiver privacy by confirming the
destination of an HTLC, and/or b) under certain circumstances, result in loss of
funds to the victim by believing the invoice was paid when it only received
routing fees. We have no evidence of either case being exploited in the wild.
It affects routing nodes, i.e. any lnd node that permits HTLC forwarding and
operates as any form of merchant node (accepts payment for goods & services);
nodes that use rejecthtlc=1 are not affected.
The vulnerability was reported privately to the lnd team by Antoine Riard.
## Background
When resolving incoming HTLCs on-chain, the forwarding node is expected to
supply a valid preimage via the witness in order to claim the HTLC. Preimages
can be learned in two primary ways: by generating an invoice or receiving a
preimage as the result of a successfully forwarded HTLC. Internally, lnd tracks
these differing preimage classes in two distinct places: an invoice database and
a preimage database.
For proper handling, a node should inspect the off-chain HTLC it received to
determine whether the node was an intermediary or the final hop (indicated by an
all zeros next_hop short channel id). Final hops are intended to consult the
invoice database for preimages, while intermediaries should consult the preimage
database.
Earlier versions of lnd would incorrectly fallback to the invoice database if
the preimage database did not contain the preimage. This resulted in situations
where the node would reveal an invoice preimage, even though the victim was not
the final hop in a route.
In coordination with a triggered channel closure, an attacker can construct a
malicious HTLC through the victim, divulging the target invoice preimage
on-chain when the incoming HTLCis claimed. After disclosing the preimage, an
attacker can claim a concurrent outgoing HTLC whose CLTV has not yet expired.
his results in the HTLC's value being stolen, minus routing fees, since the
victim incorrectly believes they received the payment.
## Attack Scenario
An example of the attack goes as follows:
Assume a _real HTLC_ is routed to a malicious node, Malice, that has a channel
with sufficient outgoing bandwidth to forward to the victim, Bob, the intended
receiver of the payment.
Instead of forwarding the real HTLC, Malice creates a new _malicious HTLC_ which
has the same payment hash. The first hop’s amount should be equal or slightly
larger, the CLTV can be increased as desired up to default implementation
limits.
Malice forwards the malicious HTLC in a circular route through Bob and back to
herself, where she holds the malicious HTLC.
With the malicious HTLC locked in place, Malice triggers a unilateral close of
the Malice-Bob channel. The commitment played (from Bob’s POV) has an incoming
HTLC with the target invoice’s payment hash. The unilateral closure should be
initiated well before any of the malicious HTLC’s CLTVs expire, otherwise the
last channel in the route will also go to chain.
When the unilateral close is confirmed, Bob will promptly broadcast the
HTLC-success transaction for the malicious HTLC. Due to the bug, the preimage
provided to sweep the malicious HTLC is obtained from the invoice database as a
fallback to the (forwarded) preimage database, rather than being ignored.
The victim, Bob, has already marked the invoice settled and published the
preimage on-chain. However, the malicious HTLC is still active on Bob’s
downstream channel (and the rest of the circular route), allowing Malice to
settle the malicious HTLC she holds _after_ the invoice has already been marked
paid. This results in Bob only receiving routing fees, and the Malice
redirecting the payment to herself, while simultaneously convincing Bob of
receiving the full payment.
At this point, the malicious HTLC has been successfully pulled from Malice to
herself in a circular route, making herself whole minus routing fees (in
addition to chain fees if she was the initiator of the Malice-Bob channel).
Malice then settles the real, intercepted HTLC using the same preimage to obtain
a profit.
### Caveats
For the attack to succeed, Malice must intercept a legacy HTLC or a
single-sharded MPP HTLC. If any other concurrent MPP shard has already reached
Bob before attempting to claim on-chain, the attack will fail due to additional
safety checks added to lnd [2], preventing an invoice from being settled by a
downgraded HTLC after it has already accepted an MPP shard with a valid payment
secret.
In order to directly profit from the attack, Malice must be intercepting a
payment from an unsuspecting victim, limiting control of timing and the amount
that can be siphoned. Malice must also somehow infer or guess that Bob has the
corresponding invoice being paid.
If Malice runs the same attack without intercepting a real HTLC, she pays
routing fees, and possibly chain fees, in exchange for the invoice preimage and
identity of the receiver. However, it is possible for her to indirectly profit
from this if the service provider releases tangible goods or services to anyone
with knowledge of the invoice preimage, which is not recommended in practice.
The upstream attacker does not need to be adjacent, they only need to know which
channel to target and watch for closure. Being adjacent increases the
assuredness of pulling off an exploit, but is not strictly required.
Similarly, the downstream attacker (possibly distinct from Malice) does not need
to be adjacent, they can settle the malicious HTLC further downstream to the
same effect at the cost of more routing fees.
## Patch
This vulnerability was patched in lnd v0.11.0-beta, by properly isolating the
preimage database from the invoice database according to the HTLC's next_hop
field in commit cf739f3f [3] of PR 4157 [4]. The isolation ensures that we can
only claim forwarded HTLCs as a result of learning the preimage from an outgoing
HTLC. It also fixes the privacy leak by not revealing invoice preimages unless
the node is the final destination.
Due to the complexities involved in describing vulnerabilities over textual
mediums, the full nature of the issue wasn’t fully understood until after
v0.10.0-beta had been released. Additionally, the covert fix contained in the
v0.11.0-beta release was pushed back due to a concurrent investigation into
network instabilities resulting in unexpected channel closures.
Note, although the above patch fixes the issue, this issue could have also been
avoided by having receivers require payment secrets (BOLT 11 `s` field) since
the attackers would be unable to guess the payment secret. However, left as
optional, the attacker can always downgrade to using malicious HTLCs that omit
the payment secret.
For some time we have debated flipping the switch on requiring payment secrets
across the three major implementations. This vulnerability is further evidence
to the additional safety and privacy benefits. Now almost a year since the
initial deployment of payment secrets in lnd, the upcoming v0.12.0-beta release
of lnd is likely to make payment secrets required by default. We would welcome
other implementations to do the same.
## Timeline
04/19/2020 - Initial report from Antoine Riard
04/29/2020 - lnd v0.10.0-beta released
07/07/2020 - PR 4157 merged into master
08/20/2020 - lnd v0.11.0-beta released
10/08/2020 - Partial Disclosure sent to lightning-dev and lnd mailing list [1]
10/20/2020 - Full Disclosure sent to lightning-dev and lnd mailing list
## References
[1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-October/002819.html
[2] https://github.com/lightningnetwork/lnd/blob/9f32942a90bcd91cc37a4a9c6c2fb454f534a65d/invoices/update.go#L229
[3] https://github.com/lightningnetwork/lnd/pull/4157/commits/cf739f3f87fdcb28ab45dfd48e3d18adf26e45b3
[4] https://github.com/lightningnetwork/lnd/pull/4157
[5] https://gist.github.com/ariard/6bdeb995565d1cc292753e1ee4ae402d
A big thank you to Antoine for the responsible disclosure and for helping to
make lnd more safu. More information can be found in Antoine’s disclosure [5].
Regards,
Conner Fromknecht
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEnI1hhop8SSADsnRO59c3tn+lkscFAl+PWzMACgkQ59c3tn+l
kseIJw//UwswUyh6BNgmi4D8NoC6olelW0dRmecqcZF7JBQa619kVFm/D7rixp33
J1YsXvZC2OLTpqmaJcJ3OvBKLVcW7CxheDp3Pm0JjrfVnmOl1NGX4CSymL6Zpou7
nFqh+nqOZ2n6o4OIv+mx0y2YANKjAVtAcr9LakubMn/3LgYzqvKKu39QGqrtz9vZ
lYGAAPU3zlAjIjFNv56xWpF0Pj9VE2mQB27w2QmbSuNtR21feOSJhJimEvmXhk6d
O0Ze78Fea+eaS+d1uyRkB7aaEKBRAA5WCtDKgSOwfEY+mHC7u5+LRasyegjlc8Ie
hYBNOsjEZqVjwIgr+lqMDbQ8B5RtW4LVro/LMYGCbVRnGuF16gHu/lkDnVgz/sY7
sbsPVG11wfVFH0U/TyJoBC8qOmeHMJoVsvGbY9I2XQiFw7yAbWxEdU+7mMhQZA2Z
Zd9pl0ATByLFPyg58gA6G4JV+F45DvYrG3jj6cdkUvL2nQST08IZtTjnDxAnkDTk
HwnJo0fd7vsixEyssTMuSCjbGSaPDMPCkmNQg8PAhhoIK8MeKUlylCKJuM6gMWeW
YypzGBmE6O7OtoMTOYFWysU67edVXgQTV2dD/PE6abTYOfS79gvkNekU4BvW9NDE
af0JBywXovzNVshdqpijPBleOT8/QSyTdLvI78ev+zpJMsKMugU=
=PY6W
-----END PGP SIGNATURE-----

2023-06-09 14:57:32 CEST

- reply

📅 Original date posted:2019-12-03
📝 Original message:
Good evening,
> I didn't think this was the design. The update transaction can spend any
prior, with a fixed script, due to NOINPUT.
>From my reading of the final construction, each update transaction has a
unique script to bind settlement transactions to exactly one update.
> My understanding is that this is not logically possible?
The update transaction has no fixed txid until it commits to a particular
output-to-be-spent, which is either the funding/kickoff txout, or a
lower-`nLockTime` update transaction output.
> Thus a settlement transaction *must* use `NOINPUT` as well, as it has no
txid it can spend, if it is constrained to spend a particular update
transaction.
This is also my understanding. Any presigned descendants of a NOINPUT txn
must also use NOINPUT as well. This chain must continue until a signer is
online to bind a txn to a confirmed input. The unique settlement keys thus
prevent rebinding of settlement txns since NOINPUT with a shared script
would be too liberal.
Cheers,
Conner
On Mon, Dec 2, 2019 at 18:55 ZmnSCPxj <ZmnSCPxj at protonmail.com> wrote:
> Good morning Rusty,
>
> > > Hi all,
> > > I recently revisited the eltoo paper and noticed some things related
> > > watchtowers that might affect channel construction.
> > > Due to NOINPUT, any update transaction can spend from any other, so
> > > in theory the tower only needs the most recent update txn to resolve
> > > any dispute.
> > > In order to spend, however, the tower must also produce a witness
> > > script which when hashed matches the witness program of the input. To
> > > ensure settlement txns can only spend from exactly one update txn,
> > > each update txn uses unique keys for the settlement clause, meaning
> > > that each state has a unique witness program.
> >
> > I didn't think this was the design. The update transaction can spend
> > any prior, with a fixed script, due to NOINPUT.
> >
> > The settlement transaction does not use NOINPUT, and thus can only
> > spend the matching update.
>
> My understanding is that this is not logically possible?
> The update transaction has no fixed txid until it commits to a particular
> output-to-be-spent, which is either the funding/kickoff txout, or a
> lower-`nLockTime` update transaction output.
> Thus a settlement transaction *must* use `NOINPUT` as well, as it has no
> txid it can spend, if it is constrained to spend a particular update
> transaction.
>
> Unless I misunderstand how update transactions work, or what settlement
> transactions are.
>
> Regards,
> ZmnSCPxj
>
--
—Sent from my Spaceship
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20191202/00acdf73/attachment-0001.html>

2023-06-09 14:52:41 CEST

- reply

📅 Original date posted:2018-11-13
📝 Original message:
Good morning all,
> MUST NOT forward (if an intermediate node) or claim (if the final node) unless
> it has received a total greater or equal to `intended_total_payment` in all
> incoming HTLCs for the same `payment_hash`.
I was under the impression that this would not require changes on behalf of the
intermediaries, and only need to be implemented by the sender and receiver?
If not, then nodes would need to advertise that they support this so that the
sender can be sure to route through the subset of nodes that support it.
Either way, it would seem that this constraint can only be accurately enforced
by the receiver. If any partial payments fail, then the `intended_total_payment`
through an intermediary may never arise and the payment would be held. This
would also seem to exclude the possibility of iterative path finding, since the
entire payment flow must be known up front during onion packet construction.
Seems the proposal still works without the intermediaries needing to know this?
We may want to add that the receiver:
* SHOULD fail the payment if `intended_total_payment` is less than the invoice
amount
> I'm wondering, since these payments are no longer atomic, should we name it
> accordingly?
Indeed this true. Perhaps NAMP or CPHR (Concurrent Payment Hash Re-use) are more
accurate and may avoid confusion?
Cheers,
Conner
On Tue, Nov 13, 2018 at 8:33 AM Johan Torås Halseth <johanth at gmail.com> wrote:
>
> Good evening Z and list,
>
> I'm wondering, since these payments are no longer atomic, should we name it accordingly?
>
> Cheers,
> Johan
>
> On Tue, Nov 13, 2018 at 1:28 PM ZmnSCPxj via Lightning-dev <lightning-dev at lists.linuxfoundation.org> wrote:
>>
>> Good morning list,
>>
>> I propose the below to support Base AMP.
>>
>> The below would allow arbitrary merges of paths, but not arbitrary splits. I am uncertain about the safety of arbitrary splits.
>>
>> ### The `multipath_merge_per_hop` type (`option_base_amp`)
>>
>> This indicates that payment has been split by the sender using Base AMP, and that the receiver should wait for the total intended payment before forwarding or claiming the payment.
>> In case the receiving node is not the last node in the path, then succeeding hops MUST be the same across all splits.
>>
>> 1. type: 1 (`termination_per_hop`)
>> 2. data:
>> * [`8` : `short_channel_id`]
>> * [`8` : `amt_to_forward`]
>> * [`4` : `outgoing_cltv_value`]
>> * [`8` : `intended_total_payment`]
>> * [`4` : `zeros`]
>>
>> The contents of this hop will be the same across all paths of the Base AMP.
>> The `payment_hash` of the incoming HTLCs will also be the same across all paths of the Base AMP.
>>
>> `intended_total_payment` is the total amount of money that this node should expect to receive in all incoming paths to the same `payment_hash`.
>>
>> This may be the last hop of a payment onion, in which case the `HMAC` for this hop will be `0` (the same rule as for `per_hop_type` 0).
>>
>> The receiver:
>>
>> * MUST impose a reasonable timeout for waiting to receive all component paths, and fail all incoming HTLC offers for the `payment_hash` if they have not totalled equal to `intended_total_payment`.
>> * MUST NOT forward (if an intermediate node) or claim (if the final node) unless it has received a total greater or equal to `intended_total_payment` in all incoming HTLCs for the same `payment_hash`.
>>
>> The sender:
>>
>> * MUST use the same `payment_hash` for all paths of a single multipath payment.
>>
>> Regards,
>> ZmnSCPxj
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

2023-06-09 14:51:40 CEST

- reply

📅 Original date posted:2018-10-19
📝 Original message:
Good evening all,
Thank you Rusty for starting us down this path :) and to ZmnSCPxj and Lisa
for
your thoughts. I think this narrows down the design space considerably!
In light of this, and if I'm following along, it seems our hand is forced in
splicing via a single on-chain transaction. In my book, this is preferable
anyway. I'd much rather push complexity off-chain than having to do a
mutli-stage splicing pipeline.
> To add some context to this, if you start accepting HTLC's for the new
balance
> after the parallel commitment is made, but before the re-anchor is buried,
> there's the potential for a race condition between a unilateral close (or
any
> revoked commitment transaction) and the re-anchoring commitment
transaction,
> that spends the 'pre-committed' UTXO of splicing in funds and the original
> funding transaction
Indeed, I'm not aware of any splicing mechanism that enables off-chain use
of
spliced-in funds before the new funding output confirms. Even in the async,
single-txn case, the new funds cannot be spent until the new funding output
confirms sufficiently.
>From my POV, the desired properties of a splice are:
1. non-blocking (asynchronous) usage of the channel
2. single on-chain txn
3. ability to RBF (have multiple pending splices)
Of these, it seems we've solidified 1 and 2. I understand the desire to not
tackle RBF on the first attempt given the additional complexity. However, I
do believe there are ways we can proceed in which our first attempt largely
coincides with supporting it in the future.
With that in mind, here are some thoughts on the proposals above.
## RBF and Multiple Splices
> 1. type: 132 (`commitment_signed`)
> 2. data:
> * [`32`:`channel_id`]
> * [`64`:`signature`]
> * [`2`:`num_htlcs`]
> * [`num_htlcs*64`:`htlc_signature`]
> * [`num_htlcs*64`:`htlc_splice_signature`] (`option_splice`)
This will overflow the maximum message size of 65535 bytes for num_htlcs >
511.
I would propose sending a distinct message, which references the
`active_channel_id` and a `splice_channel_id` for the pending splice:
1. type: XXX (`commitment_splice_signed`) (`option_splice`)
2. data:
* [`32`:`active_channel_id`]
* [`32`:`splice_channel_id`]
* [`64`:`signature`]
* [`2`:`num_htlcs`]
* [`num_htlcs*64`:`htlc_signature`]
This more directly addresses handling multiple pending splices, as well as
preventing us from running into any size constraints. The purpose of
including the `active_channel_id` would be to remote node locate the
spliced channel, since it may not be populated indexes containing
active channels. If we don't want to include this, the existing message
can be used without modification.
> We shouldn't allow more than one pending splice operation anyway, as
> stated in your proposal initially. We are already critically reliant on
our
> transaction being confirmed on-chain, so I don't see this as much of an
> added issue.
IMO there's no reason to limit ourselves to one pending splice at the
message
level. I think it'd be an oversight to not to plan ahead with RBF in mind,
given that funding transactions have gone unconfirmed precisely because of
improperly chosen fee rates. Arguably, funding flow should be extended to
support this as well.
CPFP works, though it's more wasteful than resigning and I'd prefer only to
do
so out of necessity, rather than relying on it. CPFP is nice because it
doesn't
require interaction, though we are already assuming the other party to be
online during the splice (unlike unilateral closes).
Adding a splice-reject message/error code should be sufficient to allow
implementations to signal that their local tolerance for number of pending
splices has been reached. It's likely we'd all start with getting one splice
working, but then the messages won't need to modified if we want to
implement
additional pending splices via RBF.
A node that wants to RBF but receives a reject can then proceed with CPFP
as a
last resort.
Are there any downsides I'm overlooking with this approach?
> | Bit Position | Name | Field
|
> | ------------- | ------------------------- |
-------------------------------- |
> | 0 | `option_channel_htlc_max` | `htlc_maximum_msat`
|
> | 1 | `option_channel_moving` | `moving_txid
|
>
> The `channel_update` gains the following field:
> * [`32`: moving_txid`] (option_channel_moving)
Do we actually need to send the `moving_txid` via a channel update? I think
it's
enough for both parties to send `channel_update`s with the
`option_channel_moving` bit set, and continue to keep the channel in our
routing
table.
If we receive later receive two `channel_update`s whose `short_channel_id`s
reference the spending transaction (and the node pubkeys are the same), we
assume the splice was successful and that this channel has been subsumed. I
think this works so long as the spending transaction doesn't contain
multiple
funding outputs, though I think the current proposal is fallible to this as
well.
To me, this proposal has the benefit of not bloating gossip bandwidth with
an
extra field that would need to parsed indefinitely, and gracefully
supporting
RBF down the road. Otherwise we'd need to gossip and store each potential
txid.
With regards to forwarding, both `short_channel_id`s would be accepted by
the
splicers for up to 100 blocks (after splice confirm?), at which point they
can
both forget the prior `short_channel_id`.
## Shachain
> I thought about restarting the revocation sequence, but it seems like
> that only saves a tiny amount since we only store log(N) entries. We
> can drop old HTLC info post-splice though, and (after some delay for
> obscurity) tell watchtowers to drop old entries I think.
I agree the additional state isn't too burdensome, and that we would still
be
able to drop watchtower state after some delay as you mentioned.
On one hand, it does seem like the opportune time to remove such state if
desired.
OTOH, it is _really_ nice from an atomicity perspective that the current
channel and (potentially) N pending channels can be revoked using a single
commitment secret and message. Doing so would mean we don't have to
modify the `revoke_and_ack` or `channel_reestablish` messages. The receiver
would just apply the commitment secrets/points to the current channel and
any
pending splices.
## Misc
> Any reason to now make the splicing_add_* messages allow one to add
several
> inputs in a single message? Given "acceptable" constraints for how large
the
> witness and pkScripts can be, we can easily enforce an upper limit on the
> number of inputs/outputs to add.
Yes, I prefer this simplification.
> Additionally, as the size of the channel is either expanding or
contracting,
> both sides should be allowed to modify things like the CSV param, reserve,
> max accepted htlc's, max htlc size, etc. Many of these parameters like the
> CSV value should scale with the size of the channel, not allowing these
> parameters to be re-negotiated could result in odd scenarios like still
> maintain a 1 week CSV when the channel size has dipped from 1 BTC to 100k
> satoshis.
Agreed!
> These all seem marginal to me. I think if we start hitting max values,
> we should discuss increasing them.
Doesn't this defeat the goal of firewalling funds against individual channel
failures?
>>> One thing that I think we should lift from the multiple funding output
>>> approach is the "pre seating of inputs". This is cool as it would allow
>>> clients to generate addresses, that others could deposit to, and then
have
>>> be spliced directly into the channel. Public derivation can be used,
along
>>> with a script template to do it non-interactively, with the clients
picking
>>> up these deposits, and initiating a splice in as needed.
>>
>> How about this restatement?
>>
>> 1. Each channel has two public-key-derivation paths (BIP32) to create
onchain
>> addresses. One for each side of the channel.
>> 2. The base of the above is actually a combined private-public keypair
of both
>> sides (e.g. created via MuSig or some other protocol). Thus the
addresses
>> require cooperation of both parties to spend.
>> 3. When somebody sends to one of the onchain addresses in the path,
their
>> client detects this.
>> 4. The client updates the current transaction state, such that the new
commit
>> transaction has two inputs ( the original channel transaction and the
new UTXO).
>>
>> The above seems unsafe without trust in the other peer, as, the other
peer can
>> simply refuse to create the new commit transaction. Since the address
requires
>> both parties to spend, the money cannot be spent and there is no backoff
>> transaction that can be used. But maybe you can describe some mechanism
to
>> ensure this, if this is what is meant instead?
>
> This could easily be solved by making the destination address a Taproot
> address, which by default is just a 2-of-2, but in the uncooperative
> case it can reveal the script it commits to, which is just a timelocked
> refund that requires a single-sig. The only problem with this is that
> the refund would be non-interactive, and so the entirety of the funds,
> that may be from a third-party, need to be claimed by one endpoint,
> i.e., there is no splitting the funds in case of an uncollaborative
> refund. Not sure how important that is though, since I don't think
> third-party funds will come from unrelated parties, e.g., most of these
> funds will come from an on-chain wallet that is under the control of
> either parties so the refund should go back to that party anyway.
This can be accomplished similarly by having either (or both) party
publishing a
static address or publicly derivable address specific to the channel,
derived
from their HD seed.
Arguably, the address should perhaps be global, so that it can outlive the
lifetime of the channel, i.e. as soon as the first person deposits and a
splice
is initiated, is the address still valid for the new channel if new keys are
used? Similarly, the channel could be closed and the funds locked until
the timeout if the peer disappears.
Regardless, both approaches can be made to have equivalent amounts of
[non-]interactivity. However, the recipient isn't burdened in spending by
1) interaction with the channel peer, or 2) an absolute timeout if 1 fails,
giving the receiver more flexibility if they wish to not commit the received
funds to a splice. It also benefits from smaller witness sizes, a larger
anonymity set, etc.
In general, using a 2-of-2+timeout to stage funds for splicing doesn't offer
that much IMO. It seems the primary purpose is to prevent the funds from
being
double spent during the splice, but observe that this is still possible if
the
timeout matures, perhaps because the splice doesn't confirm in a timely
manner.
Acknowledging this, detecting double-spent inputs is still required for full
correctness. By implementing it, either party is free to propose arbitrary
inputs for a splice, which I believe reduces complexity in the long run.
Splice out,
Conner
On Tue, Oct 16, 2018 at 10:00 PM ZmnSCPxj via Lightning-dev <
lightning-dev at lists.linuxfoundation.org> wrote:
> Good morning lisa,
>
> This is a good observation.
>
> Before, I'd already considered the rationale, for why channels have a
> single 2-of-2 UTXO as funding output. And it seems I should have
> considered this, prior to accepting the "parallel" construction as feasible.
>
> For sake of posterity, I leave the below writeup as a tangential to the
> design of splice (and to the design of Lightning having a single 2-of-2
> UTXOs):
>
> # 0-conf is Unsafe, Yet Lightning is Safe; Why?
>
> To accept a 0-conf transaction output, is known to be unsafe.
> Replace-by-fee is always a possibility, regardless of whether the
> transaction opts in to RBF or not: a rational miner will always accept the
> higher feerate, disregarding any "opt-in" flag that is set or not set on
> the transaction. Thus we reject any advice that claims that 0-conf is
> tenable, even for tiny amounts.
>
> Yet when viewed solely in terms of transactions, Lightning protocol uses
> transactions that are not on any block (are kept offchain). Since they are
> not in a block, they are indistinguishable from 0-conf transactions, which
> are accepted by the receiver, yet are also not on any block. One might
> argue the distinction, that a "real" 0-conf transaction exists on some
> mempool somewhere, and thus has a chance to be on a block in the future,
> but mempools have no consensus, and the existence of a transaction on some
> mempool is not a safe assurance of it existing in the mempool of the next
> winning miner.
>
> So why is Lightning safe, when 0-conf transactions are in general not safe?
>
> Again, we should focus on why 0-conf transactions in general are not safe:
> transaction replacement. Thus, 0-conf transactions can be made safe, if
> you are somehow able to ensure that replacement transactions cannot be made.
>
> For example, if you are part of an n-of-n federation that signs the
> transaction, you can always safely accept a 0-conf transaction from that
> federation paying only to you, because you can always veto any replacement
> (by simply refusing to sign) that is not in your interests.
>
> This is in fact how Lightning works: a 2-of-2 federation (the channel
> counterparties) are the signatories of the 0-conf transactions that are the
> commitment transactions of the Lightning protocol. Replacement of the
> commitment transactions is strictly guided by the protocol; both sides have
> veto rights, since the source transaction output is 2-of-2.
>
> Thus, Lightning, though it uses 0-conf transactions, is safe, because it
> prevents the replacement of a 0-conf transaction without the receiver
> allowing it, by the simple expedient of including the receiver in the
> 2-of-2 multisig guarding its single funding TXO.
>
> ## The Implications for Splice Proposals
>
> Some splice proposals involve creating the equivalent of multiple funding
> TXOs for a single channel. Such constructions are unsafe-by-default on
> Poon-Dryja.
>
> In reality, every commitment transaction (or update transaction in
> Decker-Osuntokun-Russell) is replaceable by any other commitment (or
> update) transaction for that channel. Under Poon-Dryja older transactions
> are revoked (and hence one side risks loss of their collateral) while under
> Decker-Osuntokun-Russell older transactions may be "gainsaid" (i.e. newer
> update transactions may be reanchored to consume the TXO of the older
> update transaction, thus preventing that update from truly being committed
> to).
>
> This is relevant since before a splice, the channel has a single funding
> TXO, while after the splice, the channel has multiple.
>
> In particular, a commitment (or update) transaction, that has multiple
> inputs (to consume the multiple funding TXOs), can be replaced with a
> commitment (or update) transaction that was created before the splice.
> Under Poon-Dryja, such a commitment transaction may be revoked, but this
> leaves the other funding TXOs unuseable. Under Decker-Osuntokun-Russell,
> as long as the sequence number is preserved across the splice, it is
> possible for a later update transaction with multiple inputs to simply
> gainsay the old single-input update with the new multiple-input update
> transaction. (I suppose, that this is another advantage that
> Decker-Osuntokun-Russell has).
>
> Regards,
> ZmnSCPxj
>
>
> Sent with ProtonMail <https://protonmail.com> Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Wednesday, October 17, 2018 9:09 AM, lisa neigut <niftynei at gmail.com>
> wrote:
>
> To add some context to this, if you start accepting HTLC's for the new
> balance after the parallel commitment is made, but before the re-anchor is
> buried, there's the potential for a race condition between a unilateral
> close (or any revoked commitment transaction) and the re-anchoring
> commitment transaction, that spends the 'pre-committed' UTXO of splicing in
> funds and the original funding transaction.
>
> You can get around this by waiting until both the pre-commitment UTXO and
> the re-anchor have cleared a minimum depth before accepting HTLC's for the
> new balance totals, but that's twice as long of a wait as the first,
> synchronized re-commitment scheme that Rusty originally proposed.
>
> It also makes leaving the original funding transaction 'exposed' (ie
> Rene's version of parallel splice) untenable, as there's always the risk of
> an old state being published to consume that input. This foobars your
> current HTLC commitments.
>
> On Tue, Oct 16, 2018 at 3:31 PM Rusty Russell <rusty at rustcorp.com.au>
> wrote:
>
>> Rusty Russell <rusty at rustcorp.com.au> writes:
>> > If we're going to do side splice-in like this, I would use a very
>> > different protocol: the reason for this protocol was to treat splice-in
>> > and splice-out the same, and inline splice-in requires wait time. Since
>> > splice-out doesn't, we don't need this at all.
>> >
>> > It would look much more like:
>> >
>> > 1. Prepare any output with script of specific form. eg:
>> > OP_DEPTH 3 OP_EQUAL OP_IF
>> > <funding_pubkey1> <funding_pubkey2> OP_CHECKMULTISIG
>> > OP_ELSE
>> > <blockheight> OP_CHECKLOCKTIMEVERIFY OP_DROP
>> > <myrescue_pubkey> OP_CHECKSIG
>> > OP_ENDIF
>> >
>> > 1. type: 40 (`splice_in`) (`option_splice`)
>> > 2. data:
>> > * [`32`:`channel_id`]
>> > * [`8`: `satoshis`]
>> > * [`32`: `txid`]
>> > * [`4`: `txoutnum`]
>> > * [`4`: `blockheight`]
>> > * [`33`: `myrescue_pubkey`]
>> >
>> > 1. type: 137 (`update_splice_in_accept`) (`option_splice`)
>> > data:
>> > * [`32`:`channel_id`]
>> > * [`32`: `txid`]
>> > * [`4`: `txoutnum`]
>> >
>> > 1. type: 138 (`update_splice_in_reject`) (`option_splice`)
>> > data:
>> > * [`32`:`channel_id`]
>> > * [`32`: `txid`]
>> > * [`2`:`len`]
>> > * [`len`:`errorstr`]
>> >
>> > The recipient of `splice_in` checks that it's happy with the
>> > `blockheight` (far enough in future). Once it sees the tx referred to
>> > buried to its own `minimum_depth`, it checks output is what they
>> > claimed, then sends `update_splice_in_accept`; it's followed up
>> > `commitment_signed` like normal, but from this point onwards, all
>> > commitment txs signatures have one extra sig.
>>
>> Lisa started asking pointed questions, and so I noticed that parallel
>> splice doesn't work with Poon-Dryja channels.
>>
>> The counterparty can spend the old funding txout with a revoked spend.
>> Sure, I can take all the money from that, but what about the spliced
>> input?
>>
>> I came up with increasingly elaborate workarounds, but nothing stuck.
>>
>> Back to Plan A...
>> Rusty.
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20181018/d57ce11c/attachment-0001.html>

2023-06-09 14:51:40 CEST

- reply

📅 Original date posted:2018-10-23
📝 Original message:
Good evening lightning-dev,
> If we receive later receive two `channel_update`s whose
`short_channel_id`s
> reference the spending transaction (and the node pubkeys are the same), we
> assume the splice was successful and that this channel has been subsumed.
I
> think this works so long as the spending transaction doesn't contain
multiple
> funding outputs, though I think the current proposal is fallible to this
as
> well.
Thought about this some more. The main difference seems to be whether the
gossiped data is forward or backward looking. By forward looking, I mean
that we
gossip where the splice will move to, and backward looking gossips where the
splice moved from.
If we want to make the original proposal work w/ multiple funding outputs on
one splice, I think it can be accomplished by sending the funding outpoint
as
opposed to just the txid. For the backward looking proposal, the
`channel_update`
could be modified to include the `short_channel_id` of the prior funding
output.
IMO we probably want to include the extra specificity even if we don't plan
to
have multiple funding outputs on a commitment implemented tomorrow, since
outputs are what we truly care about.
Of the two, it still seems like the backward looking approach results in
less
gossiped data since are able to reference a single confirmed output by
location
(8 bytes), instead of N unconfirmed outputs by outpoint (N*34 bytes).
Another advantage I see with the backward looking splice announcments is
that
they can be properly verified before forwarding to the network by examining
the
channel lineage. In contrast, one can't be sure if the outpoint in a
forward looking
announcement will ever confirm, or even if it spends from the original
channel point
unless one also has the transaction. Until a splice does confirm, a node has
to store multiple potential splice outpoints. Seeing this, it seems to me
that
backward looking announcements are less susceptible to abuse and DOS in
this regard.
Thoughts?
Cheers,
Conner
On Thu, Oct 18, 2018 at 8:04 PM Conner Fromknecht
<conner at lightning.engineering> wrote:
> Good evening all,
>
> Thank you Rusty for starting us down this path :) and to ZmnSCPxj and Lisa
> for
> your thoughts. I think this narrows down the design space considerably!
>
> In light of this, and if I'm following along, it seems our hand is forced
> in
> splicing via a single on-chain transaction. In my book, this is preferable
> anyway. I'd much rather push complexity off-chain than having to do a
> mutli-stage splicing pipeline.
>
> > To add some context to this, if you start accepting HTLC's for the new
> balance
> > after the parallel commitment is made, but before the re-anchor is
> buried,
> > there's the potential for a race condition between a unilateral close
> (or any
> > revoked commitment transaction) and the re-anchoring commitment
> transaction,
> > that spends the 'pre-committed' UTXO of splicing in funds and the
> original
> > funding transaction
>
> Indeed, I'm not aware of any splicing mechanism that enables off-chain use
> of
> spliced-in funds before the new funding output confirms. Even in the async,
> single-txn case, the new funds cannot be spent until the new funding output
> confirms sufficiently.
>
> From my POV, the desired properties of a splice are:
> 1. non-blocking (asynchronous) usage of the channel
> 2. single on-chain txn
> 3. ability to RBF (have multiple pending splices)
>
> Of these, it seems we've solidified 1 and 2. I understand the desire to not
> tackle RBF on the first attempt given the additional complexity. However,
> I
> do believe there are ways we can proceed in which our first attempt largely
> coincides with supporting it in the future.
>
> With that in mind, here are some thoughts on the proposals above.
>
> ## RBF and Multiple Splices
>
> > 1. type: 132 (`commitment_signed`)
> > 2. data:
> > * [`32`:`channel_id`]
> > * [`64`:`signature`]
> > * [`2`:`num_htlcs`]
> > * [`num_htlcs*64`:`htlc_signature`]
> > * [`num_htlcs*64`:`htlc_splice_signature`] (`option_splice`)
>
> This will overflow the maximum message size of 65535 bytes for num_htlcs >
> 511.
>
> I would propose sending a distinct message, which references the
> `active_channel_id` and a `splice_channel_id` for the pending splice:
>
> 1. type: XXX (`commitment_splice_signed`) (`option_splice`)
> 2. data:
> * [`32`:`active_channel_id`]
> * [`32`:`splice_channel_id`]
> * [`64`:`signature`]
> * [`2`:`num_htlcs`]
> * [`num_htlcs*64`:`htlc_signature`]
>
> This more directly addresses handling multiple pending splices, as well as
> preventing us from running into any size constraints. The purpose of
> including the `active_channel_id` would be to remote node locate the
> spliced channel, since it may not be populated indexes containing
> active channels. If we don't want to include this, the existing message
> can be used without modification.
>
> > We shouldn't allow more than one pending splice operation anyway, as
> > stated in your proposal initially. We are already critically reliant on
> our
> > transaction being confirmed on-chain, so I don't see this as much of an
> > added issue.
>
> IMO there's no reason to limit ourselves to one pending splice at the
> message
> level. I think it'd be an oversight to not to plan ahead with RBF in mind,
> given that funding transactions have gone unconfirmed precisely because of
> improperly chosen fee rates. Arguably, funding flow should be extended to
> support this as well.
>
> CPFP works, though it's more wasteful than resigning and I'd prefer only
> to do
> so out of necessity, rather than relying on it. CPFP is nice because it
> doesn't
> require interaction, though we are already assuming the other party to be
> online during the splice (unlike unilateral closes).
>
> Adding a splice-reject message/error code should be sufficient to allow
> implementations to signal that their local tolerance for number of pending
> splices has been reached. It's likely we'd all start with getting one
> splice
> working, but then the messages won't need to modified if we want to
> implement
> additional pending splices via RBF.
>
> A node that wants to RBF but receives a reject can then proceed with CPFP
> as a
> last resort.
>
> Are there any downsides I'm overlooking with this approach?
>
> > | Bit Position | Name | Field
> |
> > | ------------- | ------------------------- |
> -------------------------------- |
> > | 0 | `option_channel_htlc_max` | `htlc_maximum_msat`
> |
> > | 1 | `option_channel_moving` | `moving_txid
> |
> >
> > The `channel_update` gains the following field:
> > * [`32`: moving_txid`] (option_channel_moving)
>
> Do we actually need to send the `moving_txid` via a channel update? I
> think it's
> enough for both parties to send `channel_update`s with the
> `option_channel_moving` bit set, and continue to keep the channel in our
> routing
> table.
>
> If we receive later receive two `channel_update`s whose `short_channel_id`s
> reference the spending transaction (and the node pubkeys are the same), we
> assume the splice was successful and that this channel has been subsumed. I
> think this works so long as the spending transaction doesn't contain
> multiple
> funding outputs, though I think the current proposal is fallible to this as
> well.
>
> To me, this proposal has the benefit of not bloating gossip bandwidth with
> an
> extra field that would need to parsed indefinitely, and gracefully
> supporting
> RBF down the road. Otherwise we'd need to gossip and store each potential
> txid.
>
> With regards to forwarding, both `short_channel_id`s would be accepted by
> the
> splicers for up to 100 blocks (after splice confirm?), at which point they
> can
> both forget the prior `short_channel_id`.
>
> ## Shachain
>
> > I thought about restarting the revocation sequence, but it seems like
> > that only saves a tiny amount since we only store log(N) entries. We
> > can drop old HTLC info post-splice though, and (after some delay for
> > obscurity) tell watchtowers to drop old entries I think.
>
> I agree the additional state isn't too burdensome, and that we would still
> be
> able to drop watchtower state after some delay as you mentioned.
>
> On one hand, it does seem like the opportune time to remove such state if
> desired.
>
> OTOH, it is _really_ nice from an atomicity perspective that the current
> channel and (potentially) N pending channels can be revoked using a single
> commitment secret and message. Doing so would mean we don't have to
> modify the `revoke_and_ack` or `channel_reestablish` messages. The receiver
> would just apply the commitment secrets/points to the current channel and
> any
> pending splices.
>
> ## Misc
>
> > Any reason to now make the splicing_add_* messages allow one to add
> several
> > inputs in a single message? Given "acceptable" constraints for how large
> the
> > witness and pkScripts can be, we can easily enforce an upper limit on the
> > number of inputs/outputs to add.
>
> Yes, I prefer this simplification.
>
> > Additionally, as the size of the channel is either expanding or
> contracting,
> > both sides should be allowed to modify things like the CSV param,
> reserve,
> > max accepted htlc's, max htlc size, etc. Many of these parameters like
> the
> > CSV value should scale with the size of the channel, not allowing these
> > parameters to be re-negotiated could result in odd scenarios like still
> > maintain a 1 week CSV when the channel size has dipped from 1 BTC to 100k
> > satoshis.
>
> Agreed!
>
> > These all seem marginal to me. I think if we start hitting max values,
> > we should discuss increasing them.
>
> Doesn't this defeat the goal of firewalling funds against individual
> channel
> failures?
>
> >>> One thing that I think we should lift from the multiple funding output
> >>> approach is the "pre seating of inputs". This is cool as it would allow
> >>> clients to generate addresses, that others could deposit to, and then
> have
> >>> be spliced directly into the channel. Public derivation can be used,
> along
> >>> with a script template to do it non-interactively, with the clients
> picking
> >>> up these deposits, and initiating a splice in as needed.
> >>
> >> How about this restatement?
> >>
> >> 1. Each channel has two public-key-derivation paths (BIP32) to create
> onchain
> >> addresses. One for each side of the channel.
> >> 2. The base of the above is actually a combined private-public keypair
> of both
> >> sides (e.g. created via MuSig or some other protocol). Thus the
> addresses
> >> require cooperation of both parties to spend.
> >> 3. When somebody sends to one of the onchain addresses in the path,
> their
> >> client detects this.
> >> 4. The client updates the current transaction state, such that the new
> commit
> >> transaction has two inputs ( the original channel transaction and the
> new UTXO).
> >>
> >> The above seems unsafe without trust in the other peer, as, the other
> peer can
> >> simply refuse to create the new commit transaction. Since the address
> requires
> >> both parties to spend, the money cannot be spent and there is no backoff
> >> transaction that can be used. But maybe you can describe some
> mechanism to
> >> ensure this, if this is what is meant instead?
> >
> > This could easily be solved by making the destination address a Taproot
> > address, which by default is just a 2-of-2, but in the uncooperative
> > case it can reveal the script it commits to, which is just a timelocked
> > refund that requires a single-sig. The only problem with this is that
> > the refund would be non-interactive, and so the entirety of the funds,
> > that may be from a third-party, need to be claimed by one endpoint,
> > i.e., there is no splitting the funds in case of an uncollaborative
> > refund. Not sure how important that is though, since I don't think
> > third-party funds will come from unrelated parties, e.g., most of these
> > funds will come from an on-chain wallet that is under the control of
> > either parties so the refund should go back to that party anyway.
>
> This can be accomplished similarly by having either (or both) party
> publishing a
> static address or publicly derivable address specific to the channel,
> derived
> from their HD seed.
>
> Arguably, the address should perhaps be global, so that it can outlive the
> lifetime of the channel, i.e. as soon as the first person deposits and a
> splice
> is initiated, is the address still valid for the new channel if new keys
> are
> used? Similarly, the channel could be closed and the funds locked until
> the timeout if the peer disappears.
>
> Regardless, both approaches can be made to have equivalent amounts of
> [non-]interactivity. However, the recipient isn't burdened in spending by
> 1) interaction with the channel peer, or 2) an absolute timeout if 1 fails,
> giving the receiver more flexibility if they wish to not commit the
> received
> funds to a splice. It also benefits from smaller witness sizes, a larger
> anonymity set, etc.
>
> In general, using a 2-of-2+timeout to stage funds for splicing doesn't
> offer
> that much IMO. It seems the primary purpose is to prevent the funds from
> being
> double spent during the splice, but observe that this is still possible if
> the
> timeout matures, perhaps because the splice doesn't confirm in a timely
> manner.
>
> Acknowledging this, detecting double-spent inputs is still required for
> full
> correctness. By implementing it, either party is free to propose arbitrary
> inputs for a splice, which I believe reduces complexity in the long run.
>
> Splice out,
> Conner
>
> On Tue, Oct 16, 2018 at 10:00 PM ZmnSCPxj via Lightning-dev <
> lightning-dev at lists.linuxfoundation.org> wrote:
>
>> Good morning lisa,
>>
>> This is a good observation.
>>
>> Before, I'd already considered the rationale, for why channels have a
>> single 2-of-2 UTXO as funding output. And it seems I should have
>> considered this, prior to accepting the "parallel" construction as feasible.
>>
>> For sake of posterity, I leave the below writeup as a tangential to the
>> design of splice (and to the design of Lightning having a single 2-of-2
>> UTXOs):
>>
>> # 0-conf is Unsafe, Yet Lightning is Safe; Why?
>>
>> To accept a 0-conf transaction output, is known to be unsafe.
>> Replace-by-fee is always a possibility, regardless of whether the
>> transaction opts in to RBF or not: a rational miner will always accept the
>> higher feerate, disregarding any "opt-in" flag that is set or not set on
>> the transaction. Thus we reject any advice that claims that 0-conf is
>> tenable, even for tiny amounts.
>>
>> Yet when viewed solely in terms of transactions, Lightning protocol uses
>> transactions that are not on any block (are kept offchain). Since they are
>> not in a block, they are indistinguishable from 0-conf transactions, which
>> are accepted by the receiver, yet are also not on any block. One might
>> argue the distinction, that a "real" 0-conf transaction exists on some
>> mempool somewhere, and thus has a chance to be on a block in the future,
>> but mempools have no consensus, and the existence of a transaction on some
>> mempool is not a safe assurance of it existing in the mempool of the next
>> winning miner.
>>
>> So why is Lightning safe, when 0-conf transactions are in general not
>> safe?
>>
>> Again, we should focus on why 0-conf transactions in general are not
>> safe: transaction replacement. Thus, 0-conf transactions can be made safe,
>> if you are somehow able to ensure that replacement transactions cannot be
>> made.
>>
>> For example, if you are part of an n-of-n federation that signs the
>> transaction, you can always safely accept a 0-conf transaction from that
>> federation paying only to you, because you can always veto any replacement
>> (by simply refusing to sign) that is not in your interests.
>>
>> This is in fact how Lightning works: a 2-of-2 federation (the channel
>> counterparties) are the signatories of the 0-conf transactions that are the
>> commitment transactions of the Lightning protocol. Replacement of the
>> commitment transactions is strictly guided by the protocol; both sides have
>> veto rights, since the source transaction output is 2-of-2.
>>
>> Thus, Lightning, though it uses 0-conf transactions, is safe, because it
>> prevents the replacement of a 0-conf transaction without the receiver
>> allowing it, by the simple expedient of including the receiver in the
>> 2-of-2 multisig guarding its single funding TXO.
>>
>> ## The Implications for Splice Proposals
>>
>> Some splice proposals involve creating the equivalent of multiple funding
>> TXOs for a single channel. Such constructions are unsafe-by-default on
>> Poon-Dryja.
>>
>> In reality, every commitment transaction (or update transaction in
>> Decker-Osuntokun-Russell) is replaceable by any other commitment (or
>> update) transaction for that channel. Under Poon-Dryja older transactions
>> are revoked (and hence one side risks loss of their collateral) while under
>> Decker-Osuntokun-Russell older transactions may be "gainsaid" (i.e. newer
>> update transactions may be reanchored to consume the TXO of the older
>> update transaction, thus preventing that update from truly being committed
>> to).
>>
>> This is relevant since before a splice, the channel has a single funding
>> TXO, while after the splice, the channel has multiple.
>>
>> In particular, a commitment (or update) transaction, that has multiple
>> inputs (to consume the multiple funding TXOs), can be replaced with a
>> commitment (or update) transaction that was created before the splice.
>> Under Poon-Dryja, such a commitment transaction may be revoked, but this
>> leaves the other funding TXOs unuseable. Under Decker-Osuntokun-Russell,
>> as long as the sequence number is preserved across the splice, it is
>> possible for a later update transaction with multiple inputs to simply
>> gainsay the old single-input update with the new multiple-input update
>> transaction. (I suppose, that this is another advantage that
>> Decker-Osuntokun-Russell has).
>>
>> Regards,
>> ZmnSCPxj
>>
>>
>> Sent with ProtonMail <https://protonmail.com> Secure Email.
>>
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On Wednesday, October 17, 2018 9:09 AM, lisa neigut <niftynei at gmail.com>
>> wrote:
>>
>> To add some context to this, if you start accepting HTLC's for the new
>> balance after the parallel commitment is made, but before the re-anchor is
>> buried, there's the potential for a race condition between a unilateral
>> close (or any revoked commitment transaction) and the re-anchoring
>> commitment transaction, that spends the 'pre-committed' UTXO of splicing in
>> funds and the original funding transaction.
>>
>> You can get around this by waiting until both the pre-commitment UTXO and
>> the re-anchor have cleared a minimum depth before accepting HTLC's for the
>> new balance totals, but that's twice as long of a wait as the first,
>> synchronized re-commitment scheme that Rusty originally proposed.
>>
>> It also makes leaving the original funding transaction 'exposed' (ie
>> Rene's version of parallel splice) untenable, as there's always the risk of
>> an old state being published to consume that input. This foobars your
>> current HTLC commitments.
>>
>> On Tue, Oct 16, 2018 at 3:31 PM Rusty Russell <rusty at rustcorp.com.au>
>> wrote:
>>
>>> Rusty Russell <rusty at rustcorp.com.au> writes:
>>> > If we're going to do side splice-in like this, I would use a very
>>> > different protocol: the reason for this protocol was to treat splice-in
>>> > and splice-out the same, and inline splice-in requires wait time.
>>> Since
>>> > splice-out doesn't, we don't need this at all.
>>> >
>>> > It would look much more like:
>>> >
>>> > 1. Prepare any output with script of specific form. eg:
>>> > OP_DEPTH 3 OP_EQUAL OP_IF
>>> > <funding_pubkey1> <funding_pubkey2> OP_CHECKMULTISIG
>>> > OP_ELSE
>>> > <blockheight> OP_CHECKLOCKTIMEVERIFY OP_DROP
>>> > <myrescue_pubkey> OP_CHECKSIG
>>> > OP_ENDIF
>>> >
>>> > 1. type: 40 (`splice_in`) (`option_splice`)
>>> > 2. data:
>>> > * [`32`:`channel_id`]
>>> > * [`8`: `satoshis`]
>>> > * [`32`: `txid`]
>>> > * [`4`: `txoutnum`]
>>> > * [`4`: `blockheight`]
>>> > * [`33`: `myrescue_pubkey`]
>>> >
>>> > 1. type: 137 (`update_splice_in_accept`) (`option_splice`)
>>> > data:
>>> > * [`32`:`channel_id`]
>>> > * [`32`: `txid`]
>>> > * [`4`: `txoutnum`]
>>> >
>>> > 1. type: 138 (`update_splice_in_reject`) (`option_splice`)
>>> > data:
>>> > * [`32`:`channel_id`]
>>> > * [`32`: `txid`]
>>> > * [`2`:`len`]
>>> > * [`len`:`errorstr`]
>>> >
>>> > The recipient of `splice_in` checks that it's happy with the
>>> > `blockheight` (far enough in future). Once it sees the tx referred to
>>> > buried to its own `minimum_depth`, it checks output is what they
>>> > claimed, then sends `update_splice_in_accept`; it's followed up
>>> > `commitment_signed` like normal, but from this point onwards, all
>>> > commitment txs signatures have one extra sig.
>>>
>>> Lisa started asking pointed questions, and so I noticed that parallel
>>> splice doesn't work with Poon-Dryja channels.
>>>
>>> The counterparty can spend the old funding txout with a revoked spend.
>>> Sure, I can take all the money from that, but what about the spliced
>>> input?
>>>
>>> I came up with increasingly elaborate workarounds, but nothing stuck.
>>>
>>> Back to Plan A...
>>> Rusty.
>>> _______________________________________________
>>> Lightning-dev mailing list
>>> Lightning-dev at lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>>
>>
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20181022/337246c7/attachment-0001.html>

2023-06-09 14:50:10 CEST

- reply

📅 Original date posted:2018-04-17
📝 Original message:
Good evening ZmnSCPxj,
> Also, thank you for the link.
Definitely! I had to do some digging myself to recover these hidden gems.
> I understand. It would be good to know what you have, and perhaps consider
> planning a new BOLT document for such.
Yes, that is the ultimate goal. I think it might be a little to soon to
have a
full-on BOLT spec. There are still some implementation details that we would
like to address before formalizing everything. I am working to have
something
written up in the short-term documenting the approach[es] that ends up being
solidified. Hopefully that can get some eyes during development, and perhaps
serve as working document/rough draft.
> Sorry, I seem confused this idea. Can you give example for commitment
with 2x
> HTLC?
Sure thing! The confirmation of second level HTLC txns can be separated by
arbitrary delays. This is particularly true if the CLTVs have already
expired,
offering an attacker total control over when the txns appear on the
network. One
way this can happen is if the attacker iteratively broadcasts a single
second-level txn, waits for confirmation and CSV to expire, then repeat with
another second-level txn.
Since the CSVs begin ticking as soon as they are included in the chain, the
attacker could try to sweep each one immediately after its CSV expires. If
the
watchtower doesn't have the ability to sweep outputs independently, it would
have no way to intercept this behavior, and prevent the breacher from
sweeping
individual HTLCs sequentially.
> When the commitment txid is found onchain, the WatchTower creates a single
> main output claim transaction using the 1 or 2 signatures for the main
> outputs. And for each HTLC outpoint on the commitment transaction, if it
gets
> spent, the WatchTower creates one HTLC justice transaction from the
> second-stage HTLC transaction.
Yes, this is how it would work in context of what I was suggesting.
Certainly,
there are other ways to accomplish the same thing. I don't wish to claim
that
this is the best solution available, there are a lot of tradeoffs that need
to be evaluated. I'm hoping that you and others can bring any shortcomings
to
light and help us sift through them.
> 5. 0 or 1 or 2 signatures for the main outputs. These sign a single
> transaction that claims only the main outputs.
Yes, it seems necessary to separate the commitment outpoints from the HTLC
outpoints in case the commitment txn is broadcasted before the CLTVs expire.
You could try to batch these with the HTLCs, but then we get back into
exponential territory.
> Is that approximately what is needed? Have I missed anything?
Nope, I think your understanding is on point. IMO this seems to be a
reasonable
compromise of the tradeoffs at hand, and definitely something that could
serve
as an initial iteration due to its simplicity. In the future, there
are definitely
ways
to improve on this on make it even more efficient! Though having a
solid/workable v0 is important if it is to be deployed. I enjoy hearing your
thoughts on this, thank you for your responses!
Best,
Conner
On Tue, Apr 17, 2018 at 6:14 AM ZmnSCPxj <ZmnSCPxj at protonmail.com> wrote:
> Good morning Conner,
>
>
>
>
> Hi ZmnSCPxj,
> > Can you describe the "encrypted blob" approach to me? Or point me to
> > materials?
>
> There's an awesome watchtower thread on the mailing list from 2016 that
> starts
> here [1]. It covers a broader range of possibilities than just the
> encrypted
> blob approach, and also considers other revocation schemes, e.g. elkrem.
>
> Similar to what you described, one encrypted blob approached discussed in
> that thread is:
> 1. hint = tixd[:16]
> 2. blob = Enc(data, txid[16:])
> 3. Send (hint, blob) to watchtower.
>
> Whenever a new block is mined, the watchtower checks if it has an entry
> for each
> txid[:16]. If so, it decrypts using txid[16:], assembles the justice txn,
> and
> broadcasts (assuming the reward output matches what was negotiated).
>
>
> Thank you, that is indeed similar to what I was thinking given the name
> "encrypted blob".
>
> Also, thank you for the link. I have not had much time to back-read
> anything older than 2017 in the archives. I observe that neither Poon nor
> Dryja seem to strongly participate in this list from 2017 onwards.
>
>
>
> > Do you have a description of the WatchTower protocol used in lnd? It
> may be
> > useful to be intercompatible.
> We don't have anything written up formally, though what we have currently
> operates on the design above.
>
>
> I understand. It would be good to know what you have, and perhaps consider
> planning a new BOLT document for such.
>
> Nicolas Dorier mentioned plans for BTCPay to somehow host "merchant
> support networks" where merchants may expose WatchTower endpoints, which
> other merchants may post revocation information for their channels to.
>
>
>
> I'll also take this time to brain dump some recent investigations I've
> been doing on
>
>
>
> watchtowers. TL;DR @ fin.
>
>
>
> FWIW, I've been thinking about this in the context of the simple encrypted
> blob approach, though the observations can generalize to other schemes.
>
>
>
> As Laolu mentioned, the storage requirement for the watchtower is
> dominated by
> the number of HTLC signatures included in the encrypted blob. Due to
> independence of the second stage transactions, there is a combinatoric
> blowup in
> the number of signatures that would need to be pre-signed under the
> revocation
> private key _if sweeping of HTLC outputs is batched_.
>
> If we want to batch sweep without more liberal sighash flags, I think we'd
> need to
> pre-sign n*2^n signatures. There are 2^n possible ways that n HTLCs can
> straddle
> the first and second stages, and each permutation would require n distinct
> signatures
> since the set of inputs is unique to each permutation. Needless to say,
> this isn't feasible
> with the maximum number of HTLCs allowed in the protocol.
>
>
> Yes, I thought this too.
>
>
>
> However, I have some observations that might inform an efficient set of
> signatures we can choose to include in the encrypted blobs.
>
>
>
> The first is that the HTLC timeout or HTLC success transaction _must_ be
> broadcast before the attacker can move funds back into their wallet. If
> these transactions are never mined, it is actually fine to do nothing and
> leave
> those outputs in the breached state.
>
> If/when the victim comes back online, they themselves can sign and
> broadcast
> a justice transaction that executes the revocation clause of either the
> offered or
> received HTLC scripts, based on the observed spentness of the various
> commitment
> HLTC outputs at that time. So, we can save on signature data by only
> requiring the
> watchtower to act if second stage transactions are confirmed.
>
>
> This is a good observation! I initially thought that we would have to
> provide both the first-stage and second-stage revocation signatures.
>
>
> One reallyyy nice thing about not having the watchtower sweep the HTLC
> outputs
> on the commitment txn directly is that it doesn't need to know how to
> reconstruct the more complex HTLC redeem scripts. It only needs to
> reconstruct
> commitment to-local and second-stage to-local scripts and witnesses. This
> means
> the blob primarily contains:
> - 1 revocation pubkey
> - 1 local delay pubkey
> - 1 CSV delay
> - 2 commitment signatures
> - n HTLC signatures
> and we don't have to bother sending CLTVs, local/remote htlc pubkeys, or
> payment hashes at all.
>
> The storage for this ends up being something like ~100 + 64*(2+nhtlcs)
> when you
> include other things like the sweep address.
>
>
> Thank you, that seems like a start at something that can be implemented.
>
>
> The second observation is that the second stage transactions could be
> broadcast
> sequentially such that the CSV delays don't overlap at all. In this
> event, the
> watchtower needs to sweep the HTLCs iteratively to prevent the attacker
> from
> sweeping any of the outputs as the relative timelocks expire.
>
>
> Sorry, I seem confused this idea. Can you give example for commitment
> with 2x HTLC?
>
>
> One minimal solution could be to send signatures for independent sweep
> transactions, allowing the watchtower to sweep each HTLC output
> individually.
> This is nice because it permits the watchtower to sweep exactly the
> subset of
> HTLCs that ever transition into the second stage, and under any
> permutation
> wrt. ordering of confirmed second stage transactions.
>
>
> Yes, this seems like a good general idea.
>
>
> With the single transaction per HTLC approach, the total number of
> signatures that
> are sent to the watchtower remains linear in the number HTLCs on the
> commitment
> transaction. This approach does have the downside of consuming slightly
> more
> fees, since each output is swept with a distinct transaction.
>
>
>
> However, this approach is fairly efficient in preventing the attacker
> entirely from
> moving funds from the channel into their wallet wrt. to the amount of data
> stored.
> Considering that the majority of the channel balance is expected to be in
> the commitment outputs and that hypothetically on-chains fees are offset
> by the
> remote balance, this could be an acceptable tradeoff.
>
>
>
> I suspect that in practice, most second stage transactions will be valid
> by the
> time an attacker would drop to chain. Because of this, it's possible that
> they
> could be mined in the same block as the breach transaction.
>
> If everything is mined in the same block or in quick succession, it might
> be
> worthwhile to also pre-sign a justice txn that batch sweeps all HTLCs
> directly
> from the second layer, requiring one additional signature/HTLC.
> This could be a plausible scenario if the offender breached
> unintentionally, and
> their implementation tries to proceed normally. However it does require
> all of the
> CSV delays to conincide. If that doesn't happen, the watchtower can always
> resort to sweeping the outputs individually.
>
> All in all, I think the ability to sweep each HTLC independently is
> more-or-less
> a requirement just given the complexity of how the on-chain state-space can
>
> manifest, especially if CLTVs have already expired. Other scenarios may
> be worth including on a case by case basis or if we feel they are
> justified. This
> could be handled dynamically by including some bitvector or some compact
> representation of how to reconstruct the transactions for any additional,
> included
> signatures.
>
>
> Okay. So it seems, the blob contains:
>
> 1. Revocation pubkey (from our revocation basepoint and per-commitment
> basepoint)
> 2. Their delayed payment pubkey (needed in scripts)
> 3. Our imposed to_self_delay (the setting we indicate, that we impose on
> the remote side)
> 4. Our payment pubkey
> 5. 0 or 1 or 2 signatures for the main outputs. These sign a single
> transaction that claims only the main outputs.
> 6. 0 or more second-stage HTLC revocation signatures. These sign
> individual transactions (one per HTLC) that claims only the second-stage
> HTLC output.
> 7. scriptpubkey to put all the funds in.
>
> When the commitment txid is found onchain, the WatchTower creates a single
> main output claim transaction using the 1 or 2 signatures for the main
> outputs. And for each HTLC outpoint on the commitment transaction, if it
> gets spent, the WatchTower creates one HTLC justice transaction from the
> second-stage HTLC transaction.
>
> Is that approximately what is needed? Have I missed anything?
>
> Regards,
> ZmnSCPxj
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180417/873b48e1/attachment-0001.html>

2023-06-09 14:48:58 CEST

- reply

📅 Original date posted:2018-02-07
📝 Original message:
Hi ZmnSCPxj and Laolu,
> Indeed, the existence of per-hop fees (`fee_base_msat`) means, splitting
the
> payment over multiple flows will be, very likely, more expensive,
compared to
> using a single flow.
As Laolu pointed out, we have yet to see how fees evolve on mainnet or what
will
emerge as a sane, default fee schedules. I agree that if the same
proportional
fee is used across all partial payments, then it could certainly be more
expensive.
However, it could also be the case that you were paying a needlessly high
proportional fee to begin with, because paths of sufficient capacity to the
destination were scarce. In an AMP world, there will be an abundance of
channels
that can route small, partial payments, which may itself drive down the
competitive fee rate for smaller payments. Just a hypothesis, we shall see
where
supply meets demand!
At the end of the day, the user can always fall back to regular payment if
they
expect to end up paying more fees using an AMP.
> (If we want to support multiple routes converging to an intermediate node,
> then continue routing to a different final node after routes have merged
(i.e.
> A->B->C->D, and A->E->C->D, with the payment being merged by C, who
forwards
> the combination to D), then we need to follow the current hop data
format, but
> I think supporting AMP at final payees is actually enough...
I think this is an interesting idea, sounds maybe like a
recursive/hierarchical
AMP? The ability to merge the payments seems like it would result in a
decent privacy
leak, as I believe an intermediary would have enough evidence to prove that
two
payments were merged/correlated. Simple traffic analysis would also reveal a
discrepancy in the number of incoming and outgoing packets, and possibly
other
observable differences in routing (some) AMPs vs regular payments.
FWIW the current proposal allows the paths of partial payments to overlap,
in such a scenario C would just forward the HTLCs independently. One could
send
them all along the same path if they desired! I'm assuming the intent here
is to
try and reduce total fees?
Minor correction^2:
> This should actually be (H(s_0 || s_1 || ...), s_i).
This assumes the receiver knows the indexes of each share. Without this
knowledge they would have to brute force all orderings to check the
fingerprint.
To maintain order invariance on the receiving end, I would propose sending
(0, s_i) for the first n-1 partial payments, and then (n, s_i) on the final
one.
As in the description of the basic AMP scheme, the receiver maintains a
persistent count of how many partial payments have been received for ID. If
the
receiver does not get the last payment last, the receiver just waits until
all n
have been received before deciding that its reconstructed value is BP.
The receiver can verify they've received the correct BP and n by rederiving
the
partial preimages r_i = H(BP || i) and checking that there are n outstanding
payments, one for each h_i = H(r_i). This also saves the receiving node n
additional hash invocations.
-Conner
On Tue, Feb 6, 2018 at 4:04 PM Olaoluwa Osuntokun <laolu32 at gmail.com> wrote:
> Hi ZmnSCPxj,
>
> > This is excellent work!
>
> Thanks!
>
> > I think, a `globalfeatures` odd bit could be used for this. As it is
> > end-ot-end, `localfeatures` is not appropriate.
>
> Yep, it would need to be a global feature bit. In the case that we're
> sending to a destination which isn't publicly advertised, then perhaps an
> extension to BOLT-11 could be made to signal receiver support.
>
> > I believe, currently, fees have not this super-linear component
>
> Yep they don't. Arguably, we should also have a component that scales
> according to the proposed CLTV value of the outgoing HTLC. At Scaling
> Bitcoin Stanford, Aviv Zohar gave a talked titled "How to Charge Lightning"
> where the authors analyzed the possible evolution of fees on the network
> (and also suggested adding this super-linear component to extend the
> lifetime of channels). However, the talk itself focused on a very simple
> "mega super duper hub" topology. Towards the end he alluded to a
> forthcoming
> paper that had more comprehensive analysis of more complex topologies. I
> look forward to the publication of their finalized work.
>
> > Indeed, the existence of per-hop fees (`fee_base_msat`) means, splitting
> > the payment over multiple flows will be, very likely, more expensive,
> > compared to using a single flow.
>
> Well it's still to be seen how the fee structure on mainnet emerges once
> the
> network is still fully bootstrapped. AFAIK, most running on mainnet atm are
> using the default fee schedules for their respective implementations. For
> example, the default fee_base_msat for lnd is 1000 msat (1 satoshi).
>
> > I believe the `realm` byte is intended for this.
>
> The realm byte is meant to signal "forward this to the dogecoin channel".
> ATM, we just default to 0 as "Bitcoin". However, the byte itself only
> really
> need significance between the sender and the intermediate node. So there
> isn't necessarily pressure to have a globally synchronized set of realm
> bytes.
>
> > Thus, you can route over nodes that are unaware of AMP, and only provide
> > an AMP realm byte to the destination node, who, is able to reconstruct
> this
> > your AMP data as per your algorithm.
>
> Yes, the intermediate nodes don't need to be aware of the end-to-end
> protocol. For the final hop, there are actually 53 free bytes (before one
> needs to signal the existence of EOBs):
>
> * 1 byte realm
> * 8 bytes next addr (all zeroes to signal final dest)
> * 32 bytes hmac (also all zeroes for the final dest)
> * 12 bytes padding
>
> So any combo of these bytes can be used to signal more advanced protocols
> to
> the final destination.
>
>
> A correction from the prior email description:
>
> > We can further modify our usage of the per-hop payloads to send
> > (H(BP), s_i) to consume most of the EOB sent from sender to receiver.
>
> This should actually be (H(s_0 || s_1 || ...), s_i). So we still allow them
> to check this finger print to see if they have all the final shares, but
> don't allow them to preemptively pull all the payments.
>
>
> -- Laolu
>
>
> On Mon, Feb 5, 2018 at 11:12 PM ZmnSCPxj <ZmnSCPxj at protonmail.com> wrote:
>
>> Good morning Laolu,
>>
>> This is excellent work!
>>
>> Some minor comments...
>>
>>
>> (Atomic Multi-path Payments). It can be experimented with on Lightning
>> *today* with the addition of a new feature bit to gate this new
>> feature. The beauty of the scheme is that it requires no fundamental
>> changes
>> to the protocol as is now, as the negotiation is strictly *end-to-end*
>> between sender and receiver.
>>
>>
>> I think, a `globalfeatures` odd bit could be used for this. As it is
>> end-ot-end, `localfeatures` is not appropriate.
>>
>> - Potential fee savings for larger payments, contingent on there being a
>> super-linear component to routed fees. It's possible that with
>> modifications to the fee schedule, it's actually *cheaper* to send
>> payments over multiple flows rather than one giant flow.
>>
>>
>> I believe, currently, fees have not this super-linear component. Indeed,
>> the existence of per-hop fees (`fee_base_msat`) means, splitting the
>> payment over multiple flows will be, very likely, more expensive, compared
>> to using a single flow. Tiny roundoffs in computing the proportional fees
>> (`fee_proportional_millionths`) may make smaller flows give a slight fee
>> advantage, but I think the multiplication of per-hop fees will dominate.
>>
>>
>> - Using smaller payments increases the set of possible paths a partial
>> payment could have taken, which reduces the effectiveness of static
>> analysis techniques involving channel capacities and the plaintext
>> values being forwarded.
>>
>>
>> Strongly agree!
>>
>>
>> In order to include the three tuple within the per-hop payload for the
>> final
>> destination, we repurpose the _first_ byte of the un-used padding bytes in
>> the payload to signal version 0x01 of the AMP protocol (note this is a PoC
>> outline, we would need to standardize signalling of these 12 bytes to
>> support other protocols).
>>
>>
>> I believe the `realm` byte is intended for this. Intermediate nodes do
>> not need to understand realm bytes that are understood by other nodes in
>> the route, including the realm bytes understood by the final destination,
>> as intermediate nodes cannot, indeed, read the hop data of other nodes.
>> Thus, you can route over nodes that are unaware of AMP, and only provide an
>> AMP realm byte to the destination node, who, is able to reconstruct this
>> your AMP data as per your algorithm.
>>
>> Indeed, the `realm` byte controls the interpretation of the rest of the
>> 65-byte packet. If you define, instead, a separate `realm` that is
>> understood by the destination node, you can redefine the entire 64 bytes of
>> the final hop data as you wish.
>>
>> If we support AMP only at final payees, we can completely redefine the 64
>> bytes in the final hop data for the new AMP `realm`, and not consume the
>> next hop (which would reduce route length by 1).
>>
>> (If we want to support multiple routes converging to an intermediate
>> node, then continue routing to a different final node after routes have
>> merged (i.e. A->B->C->D, and A->E->C->D, with the payment being merged by
>> C, who forwards the combination to D), then we need to follow the current
>> hop data format, but I think supporting AMP at final payees is actually
>> enough... AMP at intermediate nodes might not be used often enough by
>> senders for it to matter, as taking advantage of that seems more complex
>> than just asking your routing algo to provide you multiple routes to a
>> destination, which you are probably already doing)
>>
>> ----
>>
>> Overall, good work I think.
>>
>> Regards,
>> ZmnSCPxj
>>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180207/686a561b/attachment.html>

2023-06-07 20:12:15 CEST

- reply

📅 Original date posted:2018-05-23
📝 Original message:Hi all,
Jimpo, thanks for looking into those stats! I had always imagined that there
would be a more significant savings in having all filters in one bundle, as
opposed to separate. These results are interesting, to say the least, and
definitely offer us some flexibility in options for filter sharding.
So far, the bulk of this discussion has centered around bandwidth. I am
concerned, however, that splitting up the filters is at odds with the other
goal of the proposal in offering improved privacy.
Allowing clients to choose individual filter sets trivially exposes the
type of
data that client is interested in. This alone might be enough to
fingerprint the
function of a peer and reduce anonymity set justifying their potential
behavior.
Furthermore, if a match is encountered, and block requested, full nodes have
more targeted insight into what caused a particular match. They could infer
that
the client received funds in a particular block, e.g., if they are only
requesting
output scripts.
This is above and beyond the additional complexity of now syncing,
validating,
and managing five or six distinct header/filter-header/filter/block chains.
I agree that saving on bandwidth is an important goal, but bandwidth and
privacy
are always seemingly at odds. Strictly comparing the bandwidth requirements
of
a system that heavily weighs privacy to existing ones, e.g. BIP39, that
don't is a
losing battle IMO.
I'm not fundamentally opposed to splitting the filters, I certainly see the
arguments for flexibility. However, I also want to ensure we are
considering the
second order effects that fall out of optimizing for one metric when others
exist.
Cheers,
Conner
On Wed, May 23, 2018 at 10:29 Gregory Maxwell via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Any chance you could add a graph of input-scripts (instead of input
> outpoints)?
>
> On Wed, May 23, 2018 at 7:38 AM, Jim Posen via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
> > So I checked filter sizes (as a proportion of block size) for each of the
> > sub-filters. The graph is attached.
> >
> > As interpretation, the first ~120,000 blocks are so small that the
> > Golomb-Rice coding can't compress the filters that well, which is why the
> > filter sizes are so high proportional to the block size. Except for the
> > input filter, because the coinbase input is skipped, so many of them
> have 0
> > elements. But after block 120,000 or so, the filter compression converges
> > pretty quickly to near the optimal value. The encouraging thing here is
> that
> > if you look at the ratio of the combined size of the separated filters vs
> > the size of a filter containing all of them (currently known as the basic
> > filter), they are pretty much the same size. The mean of the ratio
> between
> > them after block 150,000 is 99.4%. So basically, not much compression
> > efficiently is lost by separating the basic filter into sub-filters.
> >
> > On Tue, May 22, 2018 at 5:42 PM, Jim Posen <jim.posen at gmail.com> wrote:
> >>>
> >>> My suggestion was to advertise a bitfield for each filter type the node
> >>> serves,
> >>> where the bitfield indicates what elements are part of the filters.
> This
> >>> essentially
> >>> removes the notion of decided filter types and instead leaves the
> >>> decision to
> >>> full-nodes.
> >>
> >>
> >> I think it makes more sense to construct entirely separate filters for
> the
> >> different types of elements and allow clients to download only the ones
> they
> >> care about. If there are enough elements per filter, the compression
> ratio
> >> shouldn't be much worse by splitting them up. This prevents the
> exponential
> >> blowup in the number of filters that you mention, Johan, and it works
> nicely
> >> with service bits for advertising different filter types independently.
> >>
> >> So if we created three separate filter types, one for output scripts,
> one
> >> for input outpoints, and one for TXIDs, each signaled with a separate
> >> service bit, are people good with that? Or do you think there shouldn't
> be a
> >> TXID filter at all, Matt? I didn't include the option of a prev output
> >> script filter or rolling that into the block output script filter
> because it
> >> changes the security model (cannot be proven to be correct/incorrect
> >> succinctly).
> >>
> >> Then there's the question of whether to separate or combine the headers.
> >> I'd lean towards keeping them separate because it's simpler that way.
> >
> >
> >
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180523/b7412db9/attachment.html>