Nostr notes by

📅 Original date posted:2023-10-17 🗒️ Summary of this ...

2023-10-18T15:02:09+02:00

In reply to nevent1q…9t52
_________________________

📅 Original date posted:2023-10-17
🗒️ Summary of this message: The deployed mitigations for the transaction-relay jamming attack on lightning channels are not expected to fix the issue. Mempool scanning and transaction re-signing/re-broadcasting are discussed as possible mitigations, but they have limitations and may not effectively prevent the attack. The only real fix would be for miners to keep a history of transactions and try them again after a potential attack.
📝 Original message:
There appears to be some confusion about this issue and the mitigations. To be clear, the deployed
mitigations are not expected to fix this issue, its arguable if they provide anything more than a PR
statement.

There are two discussed mitigations here - mempool scanning and transaction re-signing/re-broadcasting.

Mempool scanning relies on regularly checking the mempool of a local node to see if we can catch the
replacement cycle mid-cycle. It only works if wee see the first transaction before the second
transaction replaces it.

Today, a large majority of lightning nodes run on machines with a Bitcoin node on the same IP
address, making it very clear what the "local node" of the lightning node is. An attacker can
trivially use this information to connect to said local node and do the replacement quickly,
preventing the victim from seeing the replacement.

More generally, however, similar discoverability is true for mining pools. An attacker performing
this attack is likely to do the replacement attack on a miner's node directly, potentially reducing
the reach of the intermediate transaction to only miners, such that the victim can never discover it
at all.

The second mitigation is similarly pathetic. Re-signing and re-broadcasting the victim's transaction
in an attempt to get it to miners even if its been removed may work, if the attacker is super lazy
and didn't finish writing their attack system. If the attacker is connected to a large majority of
hashrate (which has historically been fairly doable), they can simply do their replacement in a
cycle aggressively and arbitrarily reduce the probability that the victim's transaction gets confirmed.

Now, the above is all true in a spherical cow kinda world, and the P2P network has plenty of slow
nodes and strange behavior. Its possible that these mitigations might, by some stroke of luck,
happen to catch such an attack and prevent it, because something took longer than the attacker
intended or whatever. But, that's a far cry from any kind of material "fix" for the issue.

Ultimately the only fix for this issue will be when miners keep a history of transactions they've
seen and try them again after they may be able to enter the mempool because of an attack like this.

Matt

On 10/16/23 12:57 PM, Antoine Riard wrote:
> (cross-posting mempool issues identified are exposing lightning chan to loss of funds risks, other
> multi-party bitcoin apps might be affected)
>
> Hi,
>
> End of last year (December 2022), amid technical discussions on eltoo payment channels and
> incentives compatibility of the mempool anti-DoS rules, a new transaction-relay jamming attack
> affecting lightning channels was discovered.
>
> After careful analysis, it turns out this attack is practical and immediately exposed lightning
> routing hops carrying HTLC traffic to loss of funds security risks, both legacy and anchor output
> channels. A potential exploitation plausibly happening even without network mempools congestion.
>
> Mitigations have been designed, implemented and deployed by all major lightning implementations
> during the last months.
>
> Please find attached the release numbers, where the mitigations should be present:
> - LDK: v0.0.118 - CVE-2023 -40231
> - Eclair: v0.9.0 - CVE-2023-40232
> - LND: v.0.17.0-beta - CVE-2023-40233
> - Core-Lightning: v.23.08.01 - CVE-2023-40234
>
> While neither replacement cycling attacks have been observed or reported in the wild since the last
> ~10 months or experimented in real-world conditions on bitcoin mainet, functional test is available
> exercising the affected lightning channel against bitcoin core mempool (26.0 release cycle).
>
> It is understood that a simple replacement cycling attack does not demand privileged capabilities
> from an attacker (e.g no low-hashrate power) and only access to basic bitcoin and lightning
> software. Yet I still think executing such an attack successfully requests a fair amount of bitcoin
> technical know-how and decent preparation.
>
> From my understanding of those issues, it is yet to be determined if the mitigations deployed are
> robust enough in face of advanced replacement cycling attackers, especially ones able to combine
> different classes of transaction-relay jamming such as pinnings or vetted with more privileged
> capabilities.
>
> Please find a list of potential affected bitcoin applications in this full disclosure report using
> bitcoin script timelocks or multi-party transactions, albeit no immediate security risk exposure as
> severe as the ones affecting lightning has been identified. Only cursory review of non-lightning
> applications has been conducted so far.
>
> There is a paper published summarizing replacement cycling attacks on the lightning network:
> https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf
> <https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf>;
>
> ## Problem
>
> A lightning node allows HTLCs forwarding (in bolt3's parlance accepted HTLC on incoming link and
> offered HTLC on outgoing link) should settle the outgoing state with either a success or timeout
> before the incoming state timelock becomes final and an asymmetric defavorable settlement might
> happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" section 2.3 for a classical
> exposition of this lightning security property).
>
> Failure to satisfy this settlement requirement exposes a forwarding hop to a loss of fund risk where
> the offered HTLC is spent by the outgoing link counterparty's HTLC-preimage and the accepted HTLC is
> spent by the incoming link counterparty's HTLC-timeout.
>
> The specification mandates the incoming HTLC expiration timelock to be spaced out by an interval of
> `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this exact interval value being an
> implementation and node policy setting. As a minimal value, the specification recommends 34 blocks
> of interval. If the timelock expiration I of the inbound HTLC is equal to 100 from chain tip, the
> timelock expiration O of the outbound HTLC must be equal to 66 blocks from chain tip, giving a
> reasonable buffer of reaction to the lightning forwarding node.
>
> In the lack of cooperative off-chain settlement of the HTLC on the outgoing link negotiated with the
> counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the lightning
> node should broadcast its commitment transaction. Once the commitment is confirmed (if anchor and
> the 1 CSV encumbrance is present), the lightning node broadcasts and confirms its HTLC-timeout
> before I height is reached.
>
> Here enter a replacement cycling attack. A malicious channel counterparty can broadcast its
> HTLC-preimage transaction with a higher absolute fee and higher feerate than the honest HTLC-timeout
> of the victim lightning node and triggers a replacement. Both for legacy and anchor output channels,
> a HTLC-preimage on a counterparty commitment transaction is malleable, i.e additional inputs or
> outputs can be added. The HTLC-preimage spends an unconfirmed and unrelated to the channel parent
> transaction M and conflicts its child.
>
> As the HTLC-preimage spends an unconfirmed input that was already included in the unconfirmed and
> unrelated child transaction (rule 2), pays an absolute higher fee of at least the sum paid by the
> HTLC-timeout and child transaction (rule 3) and the HTLC-preimage feerate is greater than all
> directly conflicting transactions (rule 6), the replacement is accepted. The honest HTLC-timeout is
> evicted out of the mempool.
>
> In an ulterior move, the malicious counterparty can replace the parent transaction itself with
> another candidate N satisfying the replacement rules, triggering the eviction of the malicious
> HTLC-preimage from the mempool as it was a child of the parent T.
>
> There is no spending candidate of the offered HTLC output for the current block laying in network
> mempools.
>
> This replacement cycling tricks can be repeated for each rebroadcast attempt of the HTLC-timeout by
> the honest lightning node until expiration of the inbound HTLC timelock I. Once this height is
> reached a HTLC-timeout is broadcast by the counterparty's on the incoming link in collusion with the
> one on the outgoing link broadcasting its own HTLC-preimage.
>
> The honest Lightning node has been "double-spent" in its HTLC forwarding.
>
> As a notable factor impacting the success of the attack, a lightning node's honest HTLC-timeout
> might be included in the block template of the miner winning the block race and therefore realizes a
> spent of the offered output. In practice, a replacement cycling attack might over-connect to miners'
> mempools and public reachable nodes to succeed in a fast eviction of the HTLC-timeout by its
> HTLC-preimage. As this latter transaction can come with a better ancestor-score, it should be picked
> up on the flight by economically competitive miners.
>
> A functional test exercising a simple replacement cycling of a HTLC transaction on bitcoin core
> mempool is available:
> https://github.com/ariard/bitcoin/commits/2023-test-mempool
> <https://github.com/ariard/bitcoin/commits/2023-test-mempool>;
>
> ## Deployed LN mitigations
>
> Aggressive rebroadcasting: As the replacement cycling attacker benefits from the HTLC-timeout being
> usually broadcast by lightning nodes only once every block, or less the replacement cycling
> malicious transactions paid only equal the sum of the absolute fees paid by the HTLC, adjusted with
> the replacement penalty. Rebroadcasting randomly and multiple times before the next block increases
> the absolute fee cost for the attacker.
>
> Implemented and deployed by Eclair, Core-Lightning, LND and LDK .
>
> Local-mempool preimage monitoring: As the replacement cycling attacker in a simple setup broadcast
> the HTLC-preimage to all the network mempools, the honest lightning node is able to catch on the
> flight the unconfirmed HTLC-preimage, before its subsequent mempool replacement. The preimage can be
> extracted from the second-stage HTLC-preimage and used to fetch the off-chain inbound HTLC with a
> cooperative message or go on-chain with it to claim the accepted HTLC output.
>
> Implemented and deployed by Eclair and LND.
>
> CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid by the attacker, a risk
> of the HTLC-preimage being detected or discovered by the honest lightning node, or the HTLC-timeout
> to slip in a winning block template. Bumping the default CLTV delta hardens the odds of success of a
> simple replacement cycling attack.
>
> Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72.
>
> ## Affected Bitcoin Protocols and Applications
>
> From my understanding the following list of Bitcoin protocols and applications could be affected by
> new denial-of-service vectors under some level of network mempools congestion. Neither tests or
> advanced review of specifications (when available) has been conducted for each of them:
> - on-chain DLCs
> - coinjoins
> - payjoins
> - wallets with time-sensitive paths
> - peerswap and submarine swaps
> - batch payouts
> - transaction "accelerators"
>
> Inviting their developers, maintainers and operators to investigate how replacement cycling attacks
> might disrupt their in-mempool chain of transactions, or fee-bumping flows at the shortest delay.
> Simple flows and non-multi-party transactions should not be affected to the best of my understanding.
>
> ## Open Problems: Package Malleability
>
> Pinning attacks have been known for years as a practical vector to compromise lightning channels
> funds safety, under different scenarios (cf. current bip331's motivation section). Mitigations at
> the mempool level have been designed, discussed and are under implementation by the community
> (ancestor package relay + nverrsion=3 policy). Ideally, they should constraint a pinning attacker to
> always attach a high feerate package (commitment + CPFP) to replace the honest package, or allow a
> honest lightning node to overbid a malicious pinning package and get its time-sensitive transaction
> optimistically included in the chain.
>
> Replacement cycling attack seem to offer a new way to neutralize the design goals of package relay
> and its companion nversion=3 policy, where an attacker package RBF a honest package out of the
> mempool to subsequently double-spend its own high-fee child with a transaction unrelated to the
> channel. As the remaining commitment transaction is pre-signed with a minimal relay fee, it can be
> evicted out of the mempool.
>
> A functional test exercising a simple replacement cycling of a lightning channel commitment
> transaction on top of the nversion=3 code branch is available:
> https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2
> <https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2>;
>
> ## Discovery
>
> In 2018, the issue of static fees for pre-signed lightning transactions is made more widely known,
> the carve-out exemption in mempool rules to mitigate in-mempool package limits pinning and the
> anchor output pattern are proposed.
>
> In 2019, bitcoin core 0.19 is released with carve-out support. Continued discussion of the anchor
> output pattern as a dynamic fee-bumping method.
>
> In 2020, draft of anchor output submitted to the bolts. Initial finding of economic pinning against
> lightning commitment and second-stage HTLC transactions. Subsequent discussions of a
> preimage-overlay network or package-relay as mitigations. Public call made to inquiry more on
> potential other transaction-relay jamming attacks affecting lightning.
>
> In 2021, initial work in bitcoin core 22.0 of package acceptance. Continued discussion of the
> pinning attacks and shortcomings of current mempool rules during community-wide online workshops.
> Later the year, in light of all issues for bitcoin second-layers, a proposal is made about killing
> the mempool.
>
> In 2022, bip proposed for package relay and new proposed v3 policy design proposed for a review and
> implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and conceptual questions about
> alignment of mempool rules w.r.t miners incentives are investigated.
>
> Along this year 2022, eltoo lightning channels design are discussed, implemented and reviewed. In
> this context and after discussions on mempool anti-DoS rules, I discovered this new replacement
> cycling attack was affecting deployed lightning channels and immediately reported the finding to
> some bitcoin core developers and lightning maintainers.
>
> ## Timeline
>
> - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg Sanders and Gloria Zhao
> - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, Matt Corallo and Olaoluwa
> Osuntunkun
> - 2022-12-23: Sharing to Eugene Siegel (LND)
> - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning potential affected projects)
> - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers issuers) and initial
> proposal of an early public disclosure
> - 2022-01-19: Collection of analysis if other second-layers and multi-party applications affected.
> LN mitigations development starts.
> - 2023-05-04: Sharing to Wilmer Paulino (LDK)
> - 2023-06-20: LN mitigations implemented and progressively released. Week of the 16 october proposed
> for full disclosure.
> - 2023-08-10: CVEs assigned by MITRE
> - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling attack existence to
> security at bitcoincore.org <mailto:security at bitcoincore.org>.
> - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234
> and replacement cycling attacks
>
> ## Conclusion
>
> Despite the line of mitigations adopted and deployed by current major lightning implementations, I
> believe replacement cycling attacks are still practical for advanced attackers. Beyond this new
> attack might come as a way to partially or completely defeat some of the pinning mitigations which
> have been working for years as a community.
>
> As of today, it is uncertain to me if lightning is not affected by a more severe long-term package
> malleability critical security issue under current consensus rules, and if any other time-sensitive
> multi-party protocol, designed or deployed isn't de facto affected too (loss of funds or denial of
> service).
>
> Assuming analysis on package malleability is correct, it is unclear to me if it can be corrected by
> changes in replacement / eviction rules or mempool chain of transactions processing strategy.
> Inviting my technical peers and the bitcoin community to look more on this issue, including to
> dissent. I'll be the first one pleased if I'm fundamentally wrong on those issues, or if any element
> has not been weighted with the adequate technical accuracy it deserves.
>
> Do not trust, verify. All mistakes and opinions are my own.
>
> Antoine
>
> "meet with Triumph and Disaster. And treat those two impostors just the same" - K.
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

📅 Original date posted:2023-06-14 🗒️ Summary of this ...

2023-06-15T01:23:20+02:00

In reply to nevent1q…dj2d
_________________________

📅 Original date posted:2023-06-14
🗒️ Summary of this message: Proposing an extension to BOLT-11 for bundled payments to address prepayment of mining fees for non-custodian exchanges. However, getting a new interoperable change into BOLT-11 seems unlikely.
📝 Original message:
I think the ship has probably sailed on getting any kind of new interoperable change in to BOLT-11.

We already can't get amount-less BOLT-11 invoices broadly supported, rolling out yet another new
incompatible version of BOLT-11 and expecting the entire ecosystem to support it doesn't seem all
that likely.

If we're working towards specifying some "standard" way of doing swaps, (a) I'd be curious to
understand why the need isn't obviated by splice-out, and (b) why it shouldn't be built on OMs so
you can do it more privately.

Matt

On 6/13/23 1:10 AM, Thomas Voegtlin wrote:
> Good morning list,
>
> I would like to propose an extension to BOLT-11, where an invoice can contain two bundled payments,
> with distinct preimages and amounts.
>
> The use case is for services that require the prepayment of a mining fee in order for a
> non-custodian exchange to take place:
> - Submarine swaps
> - JIT channels
>
> In both cases, the service provider receives a HTLC for which they do not have the preimage, have to
> send funds on-chain (to the channel or submarine swap funding address), and wait for the client to
> reveal the preimage when they claim the payment. Because there is no guarantee that the client will
> actually claim the payment, the service providers need to ask prepayment of mining fees.
>
> In the case of submarine swaps, services that use dedicated client software, such as Loop by
> Lightning Labs, can ask for a prepayment, because their software can handle it (this is called "no
> show penalty" on the Loop website). However, competitors who do require a dedicated wallet, not such
> as the Boltz exchange, cannot do that. Their website shows an invoice to the user, whose wallet that
> is agnostic about the swap, and it would be unpractical for them to show two invoices to be paid
> simultaneously. This creates a situation where Boltz is vulnerable to DoS attacks, where the
> attacker forces them to pay on-chain fees.
>
> In the case of JIT channels, providers who want to protect themselves against this mining fee attack
> need to ask the preimage of the main payment before they open the channel. I believe this is what
> Phoenix does (although their pay-to-open service is not open-source, so I cannot really check). The
> issue is that a service that asks for the preimage first becomes custodian. From a legal
> perspective, it does not matter whether they open the channel immediately after receiving the
> preimage, the ordering of events makes their service custodian. In Europe, such a service will fall
> within the European MICA regulation. Competitors who refuse to offer custodian services, such as
> Electrum, are excluded from that game.
>
> In order to solve that, it would be beneficial to bundle the prepayment and the main payment in the
> same BOLT-11 invoice.
>
> The semantics of bundled payments is as follows:
> - 1. the BOLT-11 invoice contains two preimages and two amounts: prepayment and main payment.
> - 2. the receiver should wait until all the HTLCs of both payments have arrived, before they
> fulfill the HTLCs of the pre-payment. If the main payment does not arrive, they should fail the
> pre-payment with a MPP timeout.
> - 3. once the HTLCs of both payments have arrived, the receiver fulfills the HTLCs of the
> prepayment, and they broadcast their on-chain transaction. Note that the main payment can still fail
> if the sender never reveal the preimage of the main payment.
>
> Of course, nothing in my proposal prevents the service provider from stealing the pre-payment, but
> that is already the case today.
>
> I believe this proposal would level the field in terms of competition between lightning service
> providers. Currently, you need to use a dedicated client in order to use Loop, and competitors who
> do not have an established user base running a dedicated client are exposed to the mining fee
> attack. I also believe that ACINQ would benefit from this, because it would make it possible for
> them to make their pay-to-open service fully non-custodian. My understanding is that in its current
> form, the 'pay-to-open' service used by Phoenix will fall into the scope of the European MICA
> regulation, which they should consider as a serious issue.
>
> Finally, I believe that such a change should be implemented in BOLT-11, and not using BOLT-12 or
> onion messages. Indeed, my proposal does not require the exchange of new messages. Some of the
> initial feedback I received was that this is a use case for BOLT-12 or OM, but I think that this is
> making things unnecessarily complicated. We should not add new messages when things can be done in a
> non-interactive way.
>
> Cheers,
> ThomasV
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

📅 Original date posted:2023-02-15 🗒️ Summary of this ...

2023-06-09T15:12:42+02:00

In reply to nevent1q…pvpc
_________________________

📅 Original date posted:2023-02-15
🗒️ Summary of this message: The Lightning Network could use forward-error-correction code on top of MPP to send more payment shards than the requested sum, reducing the impact of failed HTLCs on payment latency.
📝 Original message:
On 2/14/23 1:42 PM, Antoine Riard wrote:
> Hi Joost,
>
> > I think movement in this direction is important to guarantee competitiveness with centralised
> payment systems and their (at least theoretical) ability to
> > process a payment in the blink of an eye. A lightning wallet trying multiple paths to find one
> that works doesn't help with this.
>
> Or there is the direction to build forward-error-correction code on top of MPP, like in traditional
> networking [1]. The rough idea, you send more payment shards than the requested sum, and then
> you reveal the payment secrets to the receiver after an onion interactivity round to finalize payment.

Ah, thank you for bringing this up! I'd thought about it and then forgot to mention it in this thread.

I think this is very important to highlight as we talk about "building a reliable lightning network
out of unreliable nodes" - this is an *incredibly* powerful feature for this.

While its much less capital-effecient, the ability to over-commit upfront and then only allow the
recipient to claim a portion of the total committed funds would substantially reduce the impact of
failed HTLCs on payment latency. Of course the extra round-trip to request the "unlock keys" for the
correct set of HTLCs adds a chunk to total latency, so senders will have to be careful about
deciding when to do this or not.

Still, now that we have onion messages, we should do (well, try) this! Its not super complicated to
implement (like everything it seems, the obvious implementation forgoes proof-of-payment, and like
everything the obvious solution is PTLCs, I think). Its not clear to me how we get good data from
trials, though, we'd need a sufficient set of the network to support this that we could actually
test it, which is hard to get for a test.

Maybe someone (anyone?) wants to do some experiments doing simulations using real probing success
rates to figure out how successful this would be and propose a concrete sender strategy that would
improve success rates.

Matt

📅 Original date posted:2023-02-14 🗒️ Summary of this ...

2023-06-09T15:12:41+02:00

In reply to nevent1q…27r7
_________________________

📅 Original date posted:2023-02-14
🗒️ Summary of this message: Developers are discussing the use of signaling to make lightning network routing more efficient, but some argue that it could lead to a binary good/bad classification of nodes and push node operators towards establishing credit relationships with peers. They suggest that routing scoring should be used instead to estimate liquidity. There are also concerns about the threshold for signaling high availability and the potential for a cargo cult mentality.
📝 Original message:
On 2/14/23 2:34 AM, Joost Jager wrote:
> Hi Matt,
>
> If nodes start aggressively preferring routes through nodes that reliably route payments (which
> I believe lnd already does, in effect, to some large extent), they should do so by measurement,
> not signaling.
>
>
> The signaling is intended as a way to make measurement more efficient. If a node signals that a
> particular channel is HA and it fails, no other measurements on that same node need to be taken by
> the sender. They can skip the node altogether for a longer period of time.

But as a lightning node I don't actually care if a node is binary good/bad. I care about what
success rate a node has. If you make the decision binary, suddenly in order for a node to be "good"
I *have* to establish a credit relationship with my peers (i.e. support 0conf splicing). I think
that is a very, very bad thing to do to the lightning network.

If someone wants to establish such a relationship with their peers, so be it, but as developers we
should strongly avoid adding features which push node operators in that direction, and part of that
is writing good routing scoring so that we aren't boxing ourselves into some binary good/bad idea of
a node but rather estimating liquidity.

Honestly this just strikes me as developers being too lazy to do things right. If we do things
carefully and we are seeing issues then we can consider breaking lightning, but until we give it a
good shot, let's not!

> In practice, many channels on the network are “high availability” today, but only in one
> direction (I.e. they aren’t regularly spliced/rebalanced and are regularly unbalanced). A node
> strongly preferring a high payment success rate *should* prefer such a channel, but in your
> scheme would not.
>
>
> This shouldn't be a problem, because the HA signaling is also directional. Each end can decide
> independently on whether to add the flag for a particular channel.

But how do you decide to set it without a credit relationship? Do I measure my channel and set the
bit because the channel is "usually" (at what threshold?) saturating in the inbound direction? What
happens if this changes for an hour and I get unlucky? Did I just screw myself?

> This ignores the myriad of “at what threshold do you signal HA” issues, which likely make such a
> signal DOA, anyway.
>
>
> I think this is a product of sender preference for HA channels and the severity of the penalty if an
> HA channel fails. Given this, routing nodes will need to decide whether they can offer a service
> level that increases their routing revenue overall if they would signal HA. It is indeed dynamic,
> but I think the market is able to work it out.

I'm afraid this is going to immediately fall into a cargo cult of "set the bit" vs "don't set the
bit" and we'll never get useful data out of it. But you may be right.

> Finally, I’m very dismayed at this direction in thinking on how ln should work - nodes should be
> measuring the network and routing over paths that it thinks are reliable for what it wants,
> *robustly over an unreliable network*. We should absolutely not be expecting the lightning
> network to be built out of high reliability nodes, that creates strong centralization pressure.
> To truly meet a “high availability” threshold, realistically, you’d need to be able to JIT 0conf
> splice-in, which would drive lightning to actually being a credit network.
>
>
> Different people can have different opinions about how ln should work, that is fine. I see a
> trade-off between the reliability of the network and the barrier of entry, and I don't think the
> optimum is on one of the ends of the scale.

My point wasn't that lightning should be unreliable, but rather a reliable network build on
unreliable hops. I'm very confident we can accomplish that without falling back to forcing nodes to
establish credit to meet "reliability requirements".

> With reasonable volume, lightning today is very reliable and relatively fast, with few retries
> required. I don’t think we need to change anything to fix it. :)
>
>
> How can you be sure about this? This isn't publicly visible data.

Sure it is! https://river.com/learn/files/river-lightning-report.pdf

I'm also quite confident we can do substantially better than this.

Matt

📅 Original date posted:2023-02-14 🗒️ Summary of this ...

2023-06-09T15:12:40+02:00

In reply to nevent1q…a4ag
_________________________

📅 Original date posted:2023-02-14
🗒️ Summary of this message: The Lightning Network faces a challenge in creating a reputation system due to the rarity of repeat interactions, but a solution could be for wallet vendors to provide live network views.
📝 Original message:
On 2/13/23 7:05 PM, ZmnSCPxj wrote:
> Good morning all,
>
>>> First of all let's see what types of reputation system exist (and yes,
>>> this is my very informal categorization):
>>>
>>> - First hand experience
>>> - Inferred experience
>>> - Hearsay
>>>
>>> The first two are likely the setup we all are comfortable with: we ourselves
>>> experienced something, and make some decisions based on that
>>> experience. This is probably what we're all doing at the moment: we
>>> attempt a payment, it fails, we back off for a bit from that channel
>>> being used again. This requires either being able to witness the issue
>>> directly (local peer) or infer from unforgeable error messages (the
>>> failing node returns an error, and it can't point the finger at someone
>>> else). Notice that this also includes some transitive constructions,
>>> such as the backpressure mechanism we were discussing for ariard's
>>> credentials proposal.
>>>
>>> Ideally we'd only rely on the first two to make decisions, but here's
>>> exactly the issue we ran into with Bittorrent: repeat interactions are
>>> too rare. In addition, our local knowledge gets out of date the longer
>>> we wait, and a previously failing channel may now be good again, and
>>> vice-versa. For us to have sufficient knowledge to make good decisions
>>> we need to repeatedly interact with the same nodes in the network, and
>>> since end-users will be very unlikely to do that, we might end up in a
>>> situation were we instinctively fall back to the hearsay method, either
>>> by sharing our local reputation with peers and then somehow combine that
>>> with our own view. To the best of my knowledge such a system has never
>>> been built successfully, and all attempts have ended in a system that
>>> was either way too simple or is gameable by rational players.
>>
>>
>> In lightning we have a trivial solution to this - your wallet vendor/LSP is already extracting a fee
>> from you for every HTLC routed through it, it has you captive and can set the fee (largely)
>> arbitrarily (up to you paying on-chain fees to switch LSPs). They can happily tell you their view of
>> the network ~live and you should generally accept it. Its by no means perfect, and there's plenty of
>> games they could play on, eg, your privacy, but its pretty damned good.
>>
>> If we care a ton about the risks here, we could have a few altruistic nodes that release similar
>> info and users can median-filter the data in one way or another to reduce risk.
>>
>> I just do not buy that this is a difficult problem for the "end user" part of the network. For
>> larger nodes its (obviously, and trivially) not a problem either, which leaves the "middle nodes"
>> stranded without good data but without an LSP they want to use for data. I believe that isn't a
>> large enough cohort to change the whole network around for, and them asking a few altruistic (let's
>> say, developer?) nodes for scoring data seems more than sufficient.
>
> But this is all ultimately hearsay.
>
> LSPs can be bought out, and developers can go rogue.
> Neither should be trusted if at all possible.

You're missing the point - if your LSP wants to "go rogue" here, at worst they can charge you more
fees. They could also do this by...charging you more fees. I'm not really sure what your concern is.

> Which is why I think forwardable peerswaps fixes this: it *creates* paths that allow payment routing, without requiring pervasive monitoring (which is horrible because eventually the network will be large enough that you will never encounter the same node twice if you're a plebeian, and if you're an aristocrat, you have every incentive to lie to the plebeians to solidify your power) of the network.

No, this is much, much worse for the network. In order to do this "live" (i.e. without failing a
payment) you have to establish trust relationships across the network (i.e. make giving your peers
credit a requirement to be considered a "robust node" and, thus, receive fee revenue).

Doing splicing/peerswap as a better way to rebalance is, of course, awesome, but it doesn't solve
the issue of "what do I do if I'm just too low on capacity *right now* to clear this HTLC".

Matt

📅 Original date posted:2023-02-13 🗒️ Summary of this ...

2023-06-09T15:12:39+02:00

In reply to nevent1q…3r6n
_________________________

📅 Original date posted:2023-02-13
🗒️ Summary of this message: Reputation systems are difficult to get right and can be easily exploited. Lightning Network's solution is to rely on wallet vendors/LSPs to provide live network views.
📝 Original message:
Thanks Christian,

On 2/13/23 7:32 AM, Christian Decker wrote:
> Hi Matt,
> Hi Joost,
>
> let me chime in here, since we seem to be slowly reinventing all the
> research on reputation systems that is already out there. First of all
> let me say that I am personally not a fan of reputation systems in
> general, just to get my own biases out of the way, now on to the why :-)
>
> Reputation systems are great when they work, but they are horrible to
> get right, and certainly the patchworky approach we see being proposed
> today will end up with a system that is easy to exploit and hard to
> understand. The last time I encountered this kind of scenario was during
> my work on Bittorrent, where the often theorized tit-for-tat approach
> failed spectacularly, and leeching (i.e., not contributing to other
> people's download) is rampant even today (BT only works because a few
> don't care about their upload bandwidth).
>
> First of all let's see what types of reputation system exist (and yes,
> this is my very informal categorization):
>
> - First hand experience
> - Inferred experience
> - Hearsay
>
> The first two are likely the setup we all are comfortable with: we ourselves
> experienced something, and make some decisions based on that
> experience. This is probably what we're all doing at the moment: we
> attempt a payment, it fails, we back off for a bit from that channel
> being used again. This requires either being able to witness the issue
> directly (local peer) or infer from unforgeable error messages (the
> failing node returns an error, and it can't point the finger at someone
> else). Notice that this also includes some transitive constructions,
> such as the backpressure mechanism we were discussing for ariard's
> credentials proposal.
>
> Ideally we'd only rely on the first two to make decisions, but here's
> exactly the issue we ran into with Bittorrent: repeat interactions are
> too rare. In addition, our local knowledge gets out of date the longer
> we wait, and a previously failing channel may now be good again, and
> vice-versa. For us to have sufficient knowledge to make good decisions
> we need to repeatedly interact with the same nodes in the network, and
> since end-users will be very unlikely to do that, we might end up in a
> situation were we instinctively fall back to the hearsay method, either
> by sharing our local reputation with peers and then somehow combine that
> with our own view. To the best of my knowledge such a system has never
> been built successfully, and all attempts have ended in a system that
> was either way too simple or is gameable by rational players.

In lightning we have a trivial solution to this - your wallet vendor/LSP is already extracting a fee
from you for every HTLC routed through it, it has you captive and can set the fee (largely)
arbitrarily (up to you paying on-chain fees to switch LSPs). They can happily tell you their view of
the network ~live and you should generally accept it. Its by no means perfect, and there's plenty of
games they could play on, eg, your privacy, but its pretty damned good.

If we care a ton about the risks here, we could have a few altruistic nodes that release similar
info and users can median-filter the data in one way or another to reduce risk.

I just do not buy that this is a difficult problem for the "end user" part of the network. For
larger nodes its (obviously, and trivially) not a problem either, which leaves the "middle nodes"
stranded without good data but without an LSP they want to use for data. I believe that isn't a
large enough cohort to change the whole network around for, and them asking a few altruistic (let's
say, developer?) nodes for scoring data seems more than sufficient.

> I also object to the wording of penalizing nodes that haven't been as
> reliable in the past. It's not penalizing them if, based on our local
> information, we decide to route over other nodes for a bit. Our goal is
> optimize the payment process, chosing the best possible routes, not
> making a judgement on the honesty or reliability of a node. When talking
> about penalizing we see node operators starting to play stupid games to
> avoid that perceived penalty, when in reality they should do their best
> to route as many payments successfully as possible (the negative fees
> for direct peers "exhausting" a balanced flow is one such example of
> premature optimization in that direction imho).

Yes! Very much yes! I hate this line of thinking.

> So I guess what I'm saying is that we need to get away from this
> patchwork mode of building the protocol, and have a much clearer model
> for a) what we want to achieve, b) how much untrustworthy information we
> want to rely on, and c) how we protect (and possibly prove security)
> against manipulation by rational players. For the last question we at
> least have one nice feature (for now), namely that the identities are
> semi-permanent, and so white-washing attacks at least are not free.
>
> And after all this rambling, let's get back to the topic at hand: I
> don't think enshrining the differences of availability in the protocol,
> thus creating two classes of nodes, is a desirable
> feature. Communicating up-front that I intend to be reliable does
> nothing, and penalizing after the fact isn't worth much due to the
> repeat interactions issue. It'd be even worse if now we had to rely on a
> third party to aggregate and track the reliability, in order to get
> enough repeat interactions to build a good model of their liquidity,
> since we're now back in the hearsay world, and the third party can feed
> us wrong information to maximize their profits.
>
> Regards,
> Christian
>
>
> Matt Corallo <lf-lists at mattcorallo.com> writes:
>> Hi Joost,
>>
>> I’m not sure I agree that lightning is “capital efficient” (or even close to it), but more generally I don’t see why this needs a signal.
>>
>> If nodes start aggressively preferring routes through nodes that reliably route payments (which I believe lnd already does, in effect, to some large extent), they should do so by measurement, not signaling.
>>
>> In practice, many channels on the network are “high availability” today, but only in one direction (I.e. they aren’t regularly spliced/rebalanced and are regularly unbalanced). A node strongly preferring a high payment success rate *should* prefer such a channel, but in your scheme would not.
>>
>> This ignores the myriad of “at what threshold do you signal HA” issues, which likely make such a signal DOA, anyway.
>>
>> Finally, I’m very dismayed at this direction in thinking on how ln should work - nodes should be measuring the network and routing over paths that it thinks are reliable for what it wants, *robustly over an unreliable network*. We should absolutely not be expecting the lightning network to be built out of high reliability nodes, that creates strong centralization pressure. To truly meet a “high availability” threshold, realistically, you’d need to be able to JIT 0conf splice-in, which would drive lightning to actually being a credit network.
>>
>> With reasonable volume, lightning today is very reliable and relatively fast, with few retries required. I don’t think we need to change anything to fix it. :)
>>
>> Matt
>>
>>> On Feb 13, 2023, at 06:46, Joost Jager <joost.jager at gmail.com> wrote:
>>>
>>>
>>> Hi,
>>>
>>> For a long time I've held the expectation that eventually payers on the lightning network will become very strict about node performance. That they will require a routing node to operate flawlessly or else apply a hefty penalty such as completely avoiding the node for an extended period of time - multiple weeks. The consequence of this is that routing nodes would need to manage their liquidity meticulously because every failure potentially has a large impact on future routing revenue.
>>>
>>> I think movement in this direction is important to guarantee competitiveness with centralised payment systems and their (at least theoretical) ability to process a payment in the blink of an eye. A lightning wallet trying multiple paths to find one that works doesn't help with this.
>>>
>>> A common argument against strict penalisation is that it would lead to less efficient use of capital. Routing nodes would need to maintain pools of liquidity to guarantee successes all the time. My opinion on this is that lightning is already enormously capital efficient at scale and that it is worth sacrificing a slight part of that efficiency to also achieve the lowest possible latency.
>>>
>>> This brings me to the actual subject of this post. Assuming strict penalisation is good, it may still not be ideal to flip the switch from one day to the other. Routing nodes may not offer the required level of service yet, causing senders to end up with no nodes to choose from.
>>>
>>> One option is to gradually increase the strength of the penalties, so that routing nodes are given time to adapt to the new standards. This does require everyone to move along and leaves no space for cheap routing nodes with less leeway in terms of liquidity.
>>>
>>> Therefore I am proposing another way to go about it: extend the `channel_update` field `channel_flags` with a new bit that the sender can use to signal `highly_available`.
>>>
>>> It's then up to payers to decide how to interpret this flag. One way could be to prefer `highly_available` channels during pathfinding. But if the routing node then returns a failure, a much stronger than normal penalty will be applied. For routing nodes this creates an opportunity to attract more traffic by marking some channels as `highly_available`, but it also comes with the responsibility to deliver.
>>>
>>> Without shadow channels, it is impossible to guarantee liquidity up to the channel capacity. It might make sense for senders to only assume high availability for amounts up to `htlc_maximum_msat`.
>>>
>>> A variation on this scheme that requires no extension of `channel_update` is to signal availability implicitly through routing fees. So the more expensive a channel is, the stronger the penalty that is applied on failure will be. It seems less ideal though, because it could disincentivize cheap but reliable channels on high traffic links.
>>>
>>> The effort required to implement some form of a `highly_available` flag seem limited and it may help to get payment success rates up. Interested to hear your thoughts.
>>>
>>> Joost
>>> _______________________________________________
>>> Lightning-dev mailing list
>>> Lightning-dev at lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

📅 Original date posted:2023-02-13 🗒️ Summary of this ...

2023-06-09T15:12:38+02:00

In reply to nevent1q…58kk
_________________________

📅 Original date posted:2023-02-13
🗒️ Summary of this message: Matt argues against the idea of signaling for high availability (HA) nodes on the Lightning Network, stating that nodes should measure the network and route payments reliably over an unreliable network. He believes that relying on HA nodes creates centralization pressure and that lightning is already efficient at scale.
📝 Original message:
Hi Joost,

I’m not sure I agree that lightning is “capital efficient” (or even close to it), but more generally I don’t see why this needs a signal.

If nodes start aggressively preferring routes through nodes that reliably route payments (which I believe lnd already does, in effect, to some large extent), they should do so by measurement, not signaling.

In practice, many channels on the network are “high availability” today, but only in one direction (I.e. they aren’t regularly spliced/rebalanced and are regularly unbalanced). A node strongly preferring a high payment success rate *should* prefer such a channel, but in your scheme would not.

This ignores the myriad of “at what threshold do you signal HA” issues, which likely make such a signal DOA, anyway.

Finally, I’m very dismayed at this direction in thinking on how ln should work - nodes should be measuring the network and routing over paths that it thinks are reliable for what it wants, *robustly over an unreliable network*. We should absolutely not be expecting the lightning network to be built out of high reliability nodes, that creates strong centralization pressure. To truly meet a “high availability” threshold, realistically, you’d need to be able to JIT 0conf splice-in, which would drive lightning to actually being a credit network.

With reasonable volume, lightning today is very reliable and relatively fast, with few retries required. I don’t think we need to change anything to fix it. :)

Matt

> On Feb 13, 2023, at 06:46, Joost Jager <joost.jager at gmail.com> wrote:
>
>
> Hi,
>
> For a long time I've held the expectation that eventually payers on the lightning network will become very strict about node performance. That they will require a routing node to operate flawlessly or else apply a hefty penalty such as completely avoiding the node for an extended period of time - multiple weeks. The consequence of this is that routing nodes would need to manage their liquidity meticulously because every failure potentially has a large impact on future routing revenue.
>
> I think movement in this direction is important to guarantee competitiveness with centralised payment systems and their (at least theoretical) ability to process a payment in the blink of an eye. A lightning wallet trying multiple paths to find one that works doesn't help with this.
>
> A common argument against strict penalisation is that it would lead to less efficient use of capital. Routing nodes would need to maintain pools of liquidity to guarantee successes all the time. My opinion on this is that lightning is already enormously capital efficient at scale and that it is worth sacrificing a slight part of that efficiency to also achieve the lowest possible latency.
>
> This brings me to the actual subject of this post. Assuming strict penalisation is good, it may still not be ideal to flip the switch from one day to the other. Routing nodes may not offer the required level of service yet, causing senders to end up with no nodes to choose from.
>
> One option is to gradually increase the strength of the penalties, so that routing nodes are given time to adapt to the new standards. This does require everyone to move along and leaves no space for cheap routing nodes with less leeway in terms of liquidity.
>
> Therefore I am proposing another way to go about it: extend the `channel_update` field `channel_flags` with a new bit that the sender can use to signal `highly_available`.
>
> It's then up to payers to decide how to interpret this flag. One way could be to prefer `highly_available` channels during pathfinding. But if the routing node then returns a failure, a much stronger than normal penalty will be applied. For routing nodes this creates an opportunity to attract more traffic by marking some channels as `highly_available`, but it also comes with the responsibility to deliver.
>
> Without shadow channels, it is impossible to guarantee liquidity up to the channel capacity. It might make sense for senders to only assume high availability for amounts up to `htlc_maximum_msat`.
>
> A variation on this scheme that requires no extension of `channel_update` is to signal availability implicitly through routing fees. So the more expensive a channel is, the stronger the penalty that is applied on failure will be. It seems less ideal though, because it could disincentivize cheap but reliable channels on high traffic links.
>
> The effort required to implement some form of a `highly_available` flag seem limited and it may help to get payment success rates up. Interested to hear your thoughts.
>
> Joost
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

📅 Original date posted:2023-06-04 📝 Original message: It ...

2023-06-09T15:08:58+02:00

In reply to nevent1q…dhxa
_________________________

📅 Original date posted:2023-06-04
📝 Original message:
It seems moderation needs to go back on to tamp down spam for a minute. See y’all Monday.

Matt

📅 Original date posted:2023-02-13 📝 Original message: ...

2023-06-09T15:08:04+02:00

In reply to nevent1q…9vzf
_________________________

📅 Original date posted:2023-02-13
📝 Original message:
Thanks Christian,

On 2/13/23 7:32 AM, Christian Decker wrote:
> Hi Matt,
> Hi Joost,
>
> let me chime in here, since we seem to be slowly reinventing all the
> research on reputation systems that is already out there. First of all
> let me say that I am personally not a fan of reputation systems in
> general, just to get my own biases out of the way, now on to the why :-)
>
> Reputation systems are great when they work, but they are horrible to
> get right, and certainly the patchworky approach we see being proposed
> today will end up with a system that is easy to exploit and hard to
> understand. The last time I encountered this kind of scenario was during
> my work on Bittorrent, where the often theorized tit-for-tat approach
> failed spectacularly, and leeching (i.e., not contributing to other
> people's download) is rampant even today (BT only works because a few
> don't care about their upload bandwidth).
>
> First of all let's see what types of reputation system exist (and yes,
> this is my very informal categorization):
>
> - First hand experience
> - Inferred experience
> - Hearsay
>
> The first two are likely the setup we all are comfortable with: we ourselves
> experienced something, and make some decisions based on that
> experience. This is probably what we're all doing at the moment: we
> attempt a payment, it fails, we back off for a bit from that channel
> being used again. This requires either being able to witness the issue
> directly (local peer) or infer from unforgeable error messages (the
> failing node returns an error, and it can't point the finger at someone
> else). Notice that this also includes some transitive constructions,
> such as the backpressure mechanism we were discussing for ariard's
> credentials proposal.
>
> Ideally we'd only rely on the first two to make decisions, but here's
> exactly the issue we ran into with Bittorrent: repeat interactions are
> too rare. In addition, our local knowledge gets out of date the longer
> we wait, and a previously failing channel may now be good again, and
> vice-versa. For us to have sufficient knowledge to make good decisions
> we need to repeatedly interact with the same nodes in the network, and
> since end-users will be very unlikely to do that, we might end up in a
> situation were we instinctively fall back to the hearsay method, either
> by sharing our local reputation with peers and then somehow combine that
> with our own view. To the best of my knowledge such a system has never
> been built successfully, and all attempts have ended in a system that
> was either way too simple or is gameable by rational players.

In lightning we have a trivial solution to this - your wallet vendor/LSP is already extracting a fee
from you for every HTLC routed through it, it has you captive and can set the fee (largely)
arbitrarily (up to you paying on-chain fees to switch LSPs). They can happily tell you their view of
the network ~live and you should generally accept it. Its by no means perfect, and there's plenty of
games they could play on, eg, your privacy, but its pretty damned good.

If we care a ton about the risks here, we could have a few altruistic nodes that release similar
info and users can median-filter the data in one way or another to reduce risk.

I just do not buy that this is a difficult problem for the "end user" part of the network. For
larger nodes its (obviously, and trivially) not a problem either, which leaves the "middle nodes"
stranded without good data but without an LSP they want to use for data. I believe that isn't a
large enough cohort to change the whole network around for, and them asking a few altruistic (let's
say, developer?) nodes for scoring data seems more than sufficient.

> I also object to the wording of penalizing nodes that haven't been as
> reliable in the past. It's not penalizing them if, based on our local
> information, we decide to route over other nodes for a bit. Our goal is
> optimize the payment process, chosing the best possible routes, not
> making a judgement on the honesty or reliability of a node. When talking
> about penalizing we see node operators starting to play stupid games to
> avoid that perceived penalty, when in reality they should do their best
> to route as many payments successfully as possible (the negative fees
> for direct peers "exhausting" a balanced flow is one such example of
> premature optimization in that direction imho).

Yes! Very much yes! I hate this line of thinking.

> So I guess what I'm saying is that we need to get away from this
> patchwork mode of building the protocol, and have a much clearer model
> for a) what we want to achieve, b) how much untrustworthy information we
> want to rely on, and c) how we protect (and possibly prove security)
> against manipulation by rational players. For the last question we at
> least have one nice feature (for now), namely that the identities are
> semi-permanent, and so white-washing attacks at least are not free.
>
> And after all this rambling, let's get back to the topic at hand: I
> don't think enshrining the differences of availability in the protocol,
> thus creating two classes of nodes, is a desirable
> feature. Communicating up-front that I intend to be reliable does
> nothing, and penalizing after the fact isn't worth much due to the
> repeat interactions issue. It'd be even worse if now we had to rely on a
> third party to aggregate and track the reliability, in order to get
> enough repeat interactions to build a good model of their liquidity,
> since we're now back in the hearsay world, and the third party can feed
> us wrong information to maximize their profits.
>
> Regards,
> Christian
>
>
> Matt Corallo <lf-lists at mattcorallo.com> writes:
>> Hi Joost,
>>
>> I’m not sure I agree that lightning is “capital efficient” (or even close to it), but more generally I don’t see why this needs a signal.
>>
>> If nodes start aggressively preferring routes through nodes that reliably route payments (which I believe lnd already does, in effect, to some large extent), they should do so by measurement, not signaling.
>>
>> In practice, many channels on the network are “high availability” today, but only in one direction (I.e. they aren’t regularly spliced/rebalanced and are regularly unbalanced). A node strongly preferring a high payment success rate *should* prefer such a channel, but in your scheme would not.
>>
>> This ignores the myriad of “at what threshold do you signal HA” issues, which likely make such a signal DOA, anyway.
>>
>> Finally, I’m very dismayed at this direction in thinking on how ln should work - nodes should be measuring the network and routing over paths that it thinks are reliable for what it wants, *robustly over an unreliable network*. We should absolutely not be expecting the lightning network to be built out of high reliability nodes, that creates strong centralization pressure. To truly meet a “high availability” threshold, realistically, you’d need to be able to JIT 0conf splice-in, which would drive lightning to actually being a credit network.
>>
>> With reasonable volume, lightning today is very reliable and relatively fast, with few retries required. I don’t think we need to change anything to fix it. :)
>>
>> Matt
>>
>>> On Feb 13, 2023, at 06:46, Joost Jager <joost.jager at gmail.com> wrote:
>>>
>>>
>>> Hi,
>>>
>>> For a long time I've held the expectation that eventually payers on the lightning network will become very strict about node performance. That they will require a routing node to operate flawlessly or else apply a hefty penalty such as completely avoiding the node for an extended period of time - multiple weeks. The consequence of this is that routing nodes would need to manage their liquidity meticulously because every failure potentially has a large impact on future routing revenue.
>>>
>>> I think movement in this direction is important to guarantee competitiveness with centralised payment systems and their (at least theoretical) ability to process a payment in the blink of an eye. A lightning wallet trying multiple paths to find one that works doesn't help with this.
>>>
>>> A common argument against strict penalisation is that it would lead to less efficient use of capital. Routing nodes would need to maintain pools of liquidity to guarantee successes all the time. My opinion on this is that lightning is already enormously capital efficient at scale and that it is worth sacrificing a slight part of that efficiency to also achieve the lowest possible latency.
>>>
>>> This brings me to the actual subject of this post. Assuming strict penalisation is good, it may still not be ideal to flip the switch from one day to the other. Routing nodes may not offer the required level of service yet, causing senders to end up with no nodes to choose from.
>>>
>>> One option is to gradually increase the strength of the penalties, so that routing nodes are given time to adapt to the new standards. This does require everyone to move along and leaves no space for cheap routing nodes with less leeway in terms of liquidity.
>>>
>>> Therefore I am proposing another way to go about it: extend the `channel_update` field `channel_flags` with a new bit that the sender can use to signal `highly_available`.
>>>
>>> It's then up to payers to decide how to interpret this flag. One way could be to prefer `highly_available` channels during pathfinding. But if the routing node then returns a failure, a much stronger than normal penalty will be applied. For routing nodes this creates an opportunity to attract more traffic by marking some channels as `highly_available`, but it also comes with the responsibility to deliver.
>>>
>>> Without shadow channels, it is impossible to guarantee liquidity up to the channel capacity. It might make sense for senders to only assume high availability for amounts up to `htlc_maximum_msat`.
>>>
>>> A variation on this scheme that requires no extension of `channel_update` is to signal availability implicitly through routing fees. So the more expensive a channel is, the stronger the penalty that is applied on failure will be. It seems less ideal though, because it could disincentivize cheap but reliable channels on high traffic links.
>>>
>>> The effort required to implement some form of a `highly_available` flag seem limited and it may help to get payment success rates up. Interested to hear your thoughts.
>>>
>>> Joost
>>> _______________________________________________
>>> Lightning-dev mailing list
>>> Lightning-dev at lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

📅 Original date posted:2022-11-01 📝 Original message: ...

2023-06-09T15:07:13+02:00

In reply to nevent1q…26xp
_________________________

📅 Original date posted:2022-11-01
📝 Original message:
Right, I kinda assume let's figure that out when we get PTLCs? There's also the channel-upgrade work
that has been around for a while. Once we do that we can probably define a new type for PTLCs in
"normal" channels.

Adding two types adds a bunch of complexity right now, and it would be effectively dead code that
we're not likely to get right unless we have an immediate use/test for it. It is a good point,
though, and something implementors should likely keep in mind when writing code.

Matt

On 10/28/22 12:35 AM, Johan Torås Halseth wrote:
> Hi, Matt.
>
> You're correct, I made the suggestion mainly because it would open up for PTLCs (or other future
> features) in today's channels.
>
> Having the existing network close and reopen new channels would really slow the adoption of new
> channel features I reckon.
>
> And I don't think it adds much complexity compared to the adapter approach.
>
> - Johan
>
> On Thu, Oct 27, 2022 at 4:54 PM Matt Corallo <lf-lists at mattcorallo.com
> <mailto:lf-lists at mattcorallo.com>> wrote:
>
> I’m not sure I understand this - is there much reason to want taproot commitment outputs? I mean
> they’re cool, and witnesses are a bit smaller, which is nice I guess, but they’re not providing
> materially new features, AFAIU. Taproot funding, on the other hand, provides a Bitcoin-wide
> privacy improvement as well the potential future ability of channel participants to use multisig
> for their own channel funds transparently.
>
> Sure, if we’re doing taproot funding outputs we should probably just do it for the commitment
> outputs as well, because why not (and it’s a prereq for PTLCs). But trying to split them up
> seems like added complexity “just because”? I suppose it tees us up for eventual PTLC support in
> todays channels, but we can also consider that separately when we get to that point, IMO.
>
> Am I missing some important utility of taproot commitment transaction outputs?
>
> Matt
>
>> On Oct 27, 2022, at 02:17, Johan Torås Halseth <johanth at gmail.com <mailto:johanth at gmail.com>>
>> wrote:
>>
>>
>> Hi, Laolu.
>>
>> I think it could be worth considering dividing the taprootyness of a channel into two:
>> 1) taproot funding output
>> 2) taproot commitment outputs
>>
>> That way we could upgrade existing channels only on the commitment level, not needing to close
>> or re-anchor the channels using an adapter in order to get many of the taproot benefits.
>>
>> New channels would use taproot multisig (musig2) for the funding output.
>>
>> This seems to be less disruptive to the existing network, and we could get features enabled by
>> taproot to larger parts of the network quicker. And to me this seems to carry less complexity
>> (and closing fees) than an adapter.
>>
>> One caveat is that this wouldn't work (I think) for Eltoo channels, as the funding output
>> would not be plain multisig anymore.
>>
>> - Johan
>>
>> On Sat, Mar 26, 2022 at 1:27 AM Antoine Riard <antoine.riard at gmail.com
>> <mailto:antoine.riard at gmail.com>> wrote:
>>
>> Hi Laolu,
>>
>> Thanks for the proposal, quick feedback.
>>
>> > It *is* still the case that _ultimately_ the two transactions to close the
>> > old segwit v0 funding output, and re-open the channel with a new segwit v1
>> > funding output are unavoidable. However this adapter commitment lets peers
>> > _defer_ these two transactions until closing time.
>>
>> I think there is one downside coming with adapter commitment, which is the uncertainty of
>> the fee overhead at the closing time. Instead of closing your segwit v0 channel _now_ with
>> known fees, when your commitment is empty of time-sensitive HTLCs, you're taking the risk
>> of closing during fees spikes, due a move triggered by your counterparty, when you might
>> have HTLCs at stake.
>>
>> It might be more economically rational for a LN node operator to pay the upgrade cost now
>> if they wish to benefit from the taproot upgrade early, especially if long-term we expect
>> block fees to increase, or wait when there is a "normal" cooperative closing.
>>
>> So it's unclear to me what the economic gain of adapter commitments ?
>>
>> > In the remainder of this mail, I'll describe an alternative
>> > approach that would allow upgrading nearly all channel/commitment related
>> > values (dust limit, max in flight, etc), which is inspired by the way the
>> > Raft consensus protocol handles configuration/member changes.
>>
>> Long-term, I think we'll likely need a consensus protocol anyway for multi-party
>> constructions (channel factories/payment pools). AFAIU this proposal doesn't aim to roll
>> out a full-fledged consensus protocol *now* though it could be wise to ensure what we're
>> building slowly moves in this direction. Less critical code to maintain across bitcoin
>> codebases/toolchains.
>>
>> > The role of the signature it to prevent "spoofing" by one of the parties
>> > (authenticate the param change), and also it serves to convince a party that
>> > they actually sent a prior commitment propose update during the
>> > retransmission phase.
>>
>> What's the purpose of data origin authentication if we assume only two-parties running
>> over Noise_XK ?
>>
>> I think it's already a security property we have. Though if we think we're going to reuse
>> these dynamic upgrades for N counterparties communicating through a coordinator, yes I
>> think it's useful.
>>
>> > In the past, when ideas like this were brought up, some were concerned that
>> > it wouldn't really be possible to do this type of updates while existing
>> > HTLCs were in flight (hence some of the ideas to clear out the commitment
>> > beforehand).
>>
>> The dynamic upgrade might serve in an emergency context where we don't have the leisury to
>> wait for the settlement of the pending HTLCs. The timing of those ones might be beyond the
>> coordination of link counterparties. Thus, we have to allow upgrade of non-empty
>> commitments (and if there are undesirable interferences between new commitment types and
>> HTLCs/PTLCs present, deal case-by-case).
>>
>> Antoine
>>
>> Le jeu. 24 mars 2022 à 18:53, Olaoluwa Osuntokun <laolu32 at gmail.com
>> <mailto:laolu32 at gmail.com>> a écrit :
>>
>> Hi y'all,
>>
>> ## Dynamic Commitments Retrospective
>>
>> Two years-ish ago I made a mailing list post on some ideas re dynamic
>> commitments [1], and how the concept can be used to allow us to upgrade
>> channel types on the fly, and also remove pesky hard coded limits like the
>> 483 HTLC in-flight limit that's present today. Back then my main target was
>> upgrading all the existing channels over to the anchor output commitment
>> variant, so the core internal routing network would be more resilient in a
>> persistent high fee environment (which hasn't really happened over the past
>> 2 years for various reasons tbh). Fast forward to today, and with taproot
>> now active on mainnet, and some initial design work/sketches for
>> taproot-native channels underway, I figure it would be good to bump this
>> concept as it gives us a way to upgrade all 80k+ public channels to taproot
>> without any on chain transactions.
>>
>> ## Updating Across Witness Versions w/ Adapter Commitments
>>
>> In my original mail, I incorrectly concluded that the dynamic commitments
>> concept would only really work within the confines of a "static" multi-sig
>> output, meaning that it couldn't be used to help channels upgrade to future
>> segwit witness versions. Thankfully this reply [2] by ZmnSCPxj, outlined a
>> way to achieve this in practice. At a high level he proposes an "adaptor
>> commitment" (similar to the kickoff transaction in eltoo/duplex), which is
>> basically an upgrade transaction that spends one witness version type, and
>> produces an output with the next (upgraded) type. In the context of
>> converting from segwit v0 to v1 (taproot), two peers would collaboratively
>> create a new adapter commitment that spends the old v0 multi-sig output, and
>> produces a _new_ v1 multi-sig output. The new commitment transaction would
>> then be anchored using this new output.
>>
>> Here's a rough sequence diagram of the before and after state to better
>> convey the concept:
>>
>> * Before: fundingOutputV0 -> commitmentTransaction
>>
>> * After fundingOutputV0 -> fundingOutputV1 (the adapter) ->
>> commitmentTransaction
>>
>> It *is* still the case that _ultimately_ the two transactions to close the
>> old segwit v0 funding output, and re-open the channel with a new segwit v1
>> funding output are unavoidable. However this adapter commitment lets peers
>> _defer_ these two transactions until closing time. When force closing two
>> transactions need to be confirmed before the commitment outputs can be
>> resolved. However, for co-op close, you can just spend the v0 output, and
>> deliver to the relevant P2TR outputs. The adapter commitment can leverage
>> sighash anyonecanpay to let both parties (assuming it's symmetric) attach
>> additional inputs for fees (to avoid introducing the old update_fee related
>> static fee issues), or alternatively inherit the anchor output pattern at
>> this level.
>>
>> ## Existing Dynamic Commitments Proposals
>>
>> Assuming this concept holds up, then we need an actual concrete protocol to
>> allow for dynamic commitment updates. Last year, Rusty made a spec PR
>> outlining a way to upgrade the commitment type (leveraging the new
>> commitment type feature bits) upon channel re-establish [3]. The proposal
>> relies on another message that both sides send (`stfu`) to clear the
>> commitment (similar to the shutdown semantics) before the switch over
>> happens. However as this is tied to the channel re-establish flow, it
>> doesn't allow both sides to do things like only allow your peer to attach N
>> HTLCs to start with, slowing increasing their allotted slots and possibly
>> reducing them (TCP AIMD style).
>>
>> ## A Two-Phase Dynamic Commitment Update Protocol
>>
>> IMO if we're adding in a way to do commitment/channel upgrades, then it may
>> be worthwhile to go with a more generalized, but slightly more involved
>> route instead. In the remainder of this mail, I'll describe an alternative
>> approach that would allow upgrading nearly all channel/commitment related
>> values (dust limit, max in flight, etc), which is inspired by the way the
>> Raft consensus protocol handles configuration/member changes.
>>
>> For those that aren't aware, Raft is a consensus protocol analogous to Paxos
>> (but isn't byzantine fault tolerant out of the box) that was designed as a
>> more understandable alternative to Paxos for a pedagogical environment.
>> Typically the algorithm is run in the context of a fixed cluster with N
>> machines, but supports adding/removing machines from the cluster with a
>> configuration update protocol. At a high level the way this works is that a
>> new config is sent to the leader, with the leader synchronizing the config
>> change with the other members of the cluster. Once a majority threshold is
>> reached, the leader then commits the config change with the acknowledged
>> parties using the new config (basically a two phase commit). I'm skipping
>> over some edge cases here that can arise if the new nodes participate
>> consensus too early, which can cause a split majority leading to two leaders
>> being elected.
>>
>> Applying this to the LN context is a bit simpler than a generalized
>> protocol, as we typically just have two parties involved. The initiator is
>> already naturally a "leader" in our context, as they're the only ones that
>> can do things like trigger fee updates.
>>
>> ### Message Structure
>>
>> At a high level I propose we introduce two new messages, with the fields
>> looking something like this for `commitment_update_propose`:
>> * type: 0 (`channel_id`)
>> * value: [`32*byte`:`chan_id`]
>> * type: 1 (`propose_sig`)
>> * value: [`64*byte`:`sig`]
>> * type: 2 (`update_payload`)
>> * value: [`*byte`:`tlv_payload`]
>>
>> and this `commitment_update_apply`:
>> * type: 0 (`channel_id`)
>> * value: [`32*byte`:`chan_id`]
>> * type: 1 (`local_propose`)
>> * value: [`*byte`:`commitment_update_propose`]
>> * type: 2 (`remote_propose`)
>> * value: [`*byte`:`commitment_update_propose`]
>>
>> ### Protocol Flow
>>
>> The core idea here is that either party can propose a commitment/channel
>> param update, but only the initiator can actually apply it. The
>> `commitment_update_propose` encodes the new set of updates, with a signature
>> covering the TLV blob for the new params (more on why that's needed later).
>> The `commitment_update_apply` includes up to _two_
>> `commitment_update_propose` messages (one for the initiator and one for the
>> responder, as nested TLV messages). The `commitment_update_propose` message
>> would be treated like any other `update_*` message, in that it takes a new
>> commitment signature to properly commit/apply it.
>>
>> The normal flow takes the form of both sides sending a
>> `commitment_update_propose` message, with the initiator finally committing
>> both by sending a `commitment_update_apply` message. In the event that only
>> the responder wants to apply a param change/update, then the initiator can
>> reply immediately with a `commitment_update_apply` message that doesn't
>> include a param change for their commitment (or they just echo the
>> parameters if they're acceptable).
>>
>> ### Handling Retransmissions
>>
>> The role of the signature it to prevent "spoofing" by one of the parties
>> (authenticate the param change), and also it serves to convince a party that
>> they actually sent a prior commitment propose update during the
>> retransmission phase. As the `commitment_update_propose` message would be
>> retransmitted like any other message, if the initiator attempts to commit
>> the update but the connection dies, they'll retransmit it as normal along
>> with their latest signature.
>>
>> ### Nested TLV Param Generality
>>
>> The messages as sketched out here just have an opaque nested TLV field which
>> makes it extensible to add in other things like tweaking the total number of
>> max HTLCs, the current dust values, min/max HTLCs, etc (all things that are
>> currently hard coded for the lifetime of the channel). An initial target
>> would likely just be a `chan_type` field, with future feature bits governing
>> _what_ type of commitment updates both parties understand in the future.
>>
>> In the past, when ideas like this were brought up, some were concerned that
>> it wouldn't really be possible to do this type of updates while existing
>> HTLCs were in flight (hence some of the ideas to clear out the commitment
>> beforehand). I don't see a reason why this fundamentally _shouldn't_ be
>> allowed, as from the point of view of the channel update state machine, all
>> updates (adds/removes) get applied as normal, but with this _new_ commitment
>> type/params. The main edge case we'll need to consider is cases where the
>> new params make older HTLCs invalid for some reason.
>>
>> ## Conclusion
>>
>> Using the adapter commitment idea combined with a protocol for updating
>> commitments on the fly, would potentially allow us to update all 80k+ segwit
>> v0 channels to the base level of taprooty channels without any on chain
>> transactions. The two transactions (open+close) must happen eventually, but
>> by holding another layer of spends off-chain we can defer them (potentially
>> indefinitely, as we have channels today that have been opened for over a
>> year).
>>
>> Deploying a generalised on-the-fly dynamic commitment update protocol gives
>> us a tool to future proof the _existing_ anchored multi-sig outputs in the
>> chain, and also a way to remove many of the hard coded parameters we have
>> today in the protocol. One overly inflexible parameter we have today in the
>> network is the 483 HTLC limit. Allowing this value to float would allow
>> peers to apply similar congestion avoidance algorithm that are used in TCP
>> today, and also give us a way to protect the network against future
>> unforeseen widespread policy changes (like a raising of the dust limit).
>>
>> -- Laolu
>>
>> [1]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-July/002763.html
>> <https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-July/002763.html>;
>> [2]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-July/002770.html
>> <https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-July/002770.html>;
>> [3]: https://github.com/lightning/bolts/pull/868
>> <https://github.com/lightning/bolts/pull/868>;
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org <mailto:Lightning-dev at lists.linuxfoundation.org>
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>> <https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev>;
>>
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org <mailto:Lightning-dev at lists.linuxfoundation.org>
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>> <https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev>;
>>
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org <mailto:Lightning-dev at lists.linuxfoundation.org>
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>> <https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev>;
>
>
> On Thu, Oct 27, 2022 at 4:54 PM Matt Corallo <lf-lists at mattcorallo.com
> <mailto:lf-lists at mattcorallo.com>> wrote:
>
> I’m not sure I understand this - is there much reason to want taproot commitment outputs? I mean
> they’re cool, and witnesses are a bit smaller, which is nice I guess, but they’re not providing
> materially new features, AFAIU. Taproot funding, on the other hand, provides a Bitcoin-wide
> privacy improvement as well the potential future ability of channel participants to use multisig
> for their own channel funds transparently.
>
> Sure, if we’re doing taproot funding outputs we should probably just do it for the commitment
> outputs as well, because why not (and it’s a prereq for PTLCs). But trying to split them up
> seems like added complexity “just because”? I suppose it tees us up for eventual PTLC support in
> todays channels, but we can also consider that separately when we get to that point, IMO.
>
> Am I missing some important utility of taproot commitment transaction outputs?
>
> Matt
>
>> On Oct 27, 2022, at 02:17, Johan Torås Halseth <johanth at gmail.com <mailto:johanth at gmail.com>>
>> wrote:
>>
>>
>> Hi, Laolu.
>>
>> I think it could be worth considering dividing the taprootyness of a channel into two:
>> 1) taproot funding output
>> 2) taproot commitment outputs
>>
>> That way we could upgrade existing channels only on the commitment level, not needing to close
>> or re-anchor the channels using an adapter in order to get many of the taproot benefits.
>>
>> New channels would use taproot multisig (musig2) for the funding output.
>>
>> This seems to be less disruptive to the existing network, and we could get features enabled by
>> taproot to larger parts of the network quicker. And to me this seems to carry less complexity
>> (and closing fees) than an adapter.
>>
>> One caveat is that this wouldn't work (I think) for Eltoo channels, as the funding output
>> would not be plain multisig anymore.
>>
>> - Johan
>>
>> On Sat, Mar 26, 2022 at 1:27 AM Antoine Riard <antoine.riard at gmail.com
>> <mailto:antoine.riard at gmail.com>> wrote:
>>
>> Hi Laolu,
>>
>> Thanks for the proposal, quick feedback.
>>
>> > It *is* still the case that _ultimately_ the two transactions to close the
>> > old segwit v0 funding output, and re-open the channel with a new segwit v1
>> > funding output are unavoidable. However this adapter commitment lets peers
>> > _defer_ these two transactions until closing time.
>>
>> I think there is one downside coming with adapter commitment, which is the uncertainty of
>> the fee overhead at the closing time. Instead of closing your segwit v0 channel _now_ with
>> known fees, when your commitment is empty of time-sensitive HTLCs, you're taking the risk
>> of closing during fees spikes, due a move triggered by your counterparty, when you might
>> have HTLCs at stake.
>>
>> It might be more economically rational for a LN node operator to pay the upgrade cost now
>> if they wish to benefit from the taproot upgrade early, especially if long-term we expect
>> block fees to increase, or wait when there is a "normal" cooperative closing.
>>
>> So it's unclear to me what the economic gain of adapter commitments ?
>>
>> > In the remainder of this mail, I'll describe an alternative
>> > approach that would allow upgrading nearly all channel/commitment related
>> > values (dust limit, max in flight, etc), which is inspired by the way the
>> > Raft consensus protocol handles configuration/member changes.
>>
>> Long-term, I think we'll likely need a consensus protocol anyway for multi-party
>> constructions (channel factories/payment pools). AFAIU this proposal doesn't aim to roll
>> out a full-fledged consensus protocol *now* though it could be wise to ensure what we're
>> building slowly moves in this direction. Less critical code to maintain across bitcoin
>> codebases/toolchains.
>>
>> > The role of the signature it to prevent "spoofing" by one of the parties
>> > (authenticate the param change), and also it serves to convince a party that
>> > they actually sent a prior commitment propose update during the
>> > retransmission phase.
>>
>> What's the purpose of data origin authentication if we assume only two-parties running
>> over Noise_XK ?
>>
>> I think it's already a security property we have. Though if we think we're going to reuse
>> these dynamic upgrades for N counterparties communicating through a coordinator, yes I
>> think it's useful.
>>
>> > In the past, when ideas like this were brought up, some were concerned that
>> > it wouldn't really be possible to do this type of updates while existing
>> > HTLCs were in flight (hence some of the ideas to clear out the commitment
>> > beforehand).
>>
>> The dynamic upgrade might serve in an emergency context where we don't have the leisury to
>> wait for the settlement of the pending HTLCs. The timing of those ones might be beyond the
>> coordination of link counterparties. Thus, we have to allow upgrade of non-empty
>> commitments (and if there are undesirable interferences between new commitment types and
>> HTLCs/PTLCs present, deal case-by-case).
>>
>> Antoine
>>
>> Le jeu. 24 mars 2022 à 18:53, Olaoluwa Osuntokun <laolu32 at gmail.com
>> <mailto:laolu32 at gmail.com>> a écrit :
>>
>> Hi y'all,
>>
>> ## Dynamic Commitments Retrospective
>>
>> Two years-ish ago I made a mailing list post on some ideas re dynamic
>> commitments [1], and how the concept can be used to allow us to upgrade
>> channel types on the fly, and also remove pesky hard coded limits like the
>> 483 HTLC in-flight limit that's present today. Back then my main target was
>> upgrading all the existing channels over to the anchor output commitment
>> variant, so the core internal routing network would be more resilient in a
>> persistent high fee environment (which hasn't really happened over the past
>> 2 years for various reasons tbh). Fast forward to today, and with taproot
>> now active on mainnet, and some initial design work/sketches for
>> taproot-native channels underway, I figure it would be good to bump this
>> concept as it gives us a way to upgrade all 80k+ public channels to taproot
>> without any on chain transactions.
>>
>> ## Updating Across Witness Versions w/ Adapter Commitments
>>
>> In my original mail, I incorrectly concluded that the dynamic commitments
>> concept would only really work within the confines of a "static" multi-sig
>> output, meaning that it couldn't be used to help channels upgrade to future
>> segwit witness versions. Thankfully this reply [2] by ZmnSCPxj, outlined a
>> way to achieve this in practice. At a high level he proposes an "adaptor
>> commitment" (similar to the kickoff transaction in eltoo/duplex), which is
>> basically an upgrade transaction that spends one witness version type, and
>> produces an output with the next (upgraded) type. In the context of
>> converting from segwit v0 to v1 (taproot), two peers would collaboratively
>> create a new adapter commitment that spends the old v0 multi-sig output, and
>> produces a _new_ v1 multi-sig output. The new commitment transaction would
>> then be anchored using this new output.
>>
>> Here's a rough sequence diagram of the before and after state to better
>> convey the concept:
>>
>> * Before: fundingOutputV0 -> commitmentTransaction
>>
>> * After fundingOutputV0 -> fundingOutputV1 (the adapter) ->
>> commitmentTransaction
>>
>> It *is* still the case that _ultimately_ the two transactions to close the
>> old segwit v0 funding output, and re-open the channel with a new segwit v1
>> funding output are unavoidable. However this adapter commitment lets peers
>> _defer_ these two transactions until closing time. When force closing two
>> transactions need to be confirmed before the commitment outputs can be
>> resolved. However, for co-op close, you can just spend the v0 output, and
>> deliver to the relevant P2TR outputs. The adapter commitment can leverage
>> sighash anyonecanpay to let both parties (assuming it's symmetric) attach
>> additional inputs for fees (to avoid introducing the old update_fee related
>> static fee issues), or alternatively inherit the anchor output pattern at
>> this level.
>>
>> ## Existing Dynamic Commitments Proposals
>>
>> Assuming this concept holds up, then we need an actual concrete protocol to
>> allow for dynamic commitment updates. Last year, Rusty made a spec PR
>> outlining a way to upgrade the commitment type (leveraging the new
>> commitment type feature bits) upon channel re-establish [3]. The proposal
>> relies on another message that both sides send (`stfu`) to clear the
>> commitment (similar to the shutdown semantics) before the switch over
>> happens. However as this is tied to the channel re-establish flow, it
>> doesn't allow both sides to do things like only allow your peer to attach N
>> HTLCs to start with, slowing increasing their allotted slots and possibly
>> reducing them (TCP AIMD style).
>>
>> ## A Two-Phase Dynamic Commitment Update Protocol
>>
>> IMO if we're adding in a way to do commitment/channel upgrades, then it may
>> be worthwhile to go with a more generalized, but slightly more involved
>> route instead. In the remainder of this mail, I'll describe an alternative
>> approach that would allow upgrading nearly all channel/commitment related
>> values (dust limit, max in flight, etc), which is inspired by the way the
>> Raft consensus protocol handles configuration/member changes.
>>
>> For those that aren't aware, Raft is a consensus protocol analogous to Paxos
>> (but isn't byzantine fault tolerant out of the box) that was designed as a
>> more understandable alternative to Paxos for a pedagogical environment.
>> Typically the algorithm is run in the context of a fixed cluster with N
>> machines, but supports adding/removing machines from the cluster with a
>> configuration update protocol. At a high level the way this works is that a
>> new config is sent to the leader, with the leader synchronizing the config
>> change with the other members of the cluster. Once a majority threshold is
>> reached, the leader then commits the config change with the acknowledged
>> parties using the new config (basically a two phase commit). I'm skipping
>> over some edge cases here that can arise if the new nodes participate
>> consensus too early, which can cause a split majority leading to two leaders
>> being elected.
>>
>> Applying this to the LN context is a bit simpler than a generalized
>> protocol, as we typically just have two parties involved. The initiator is
>> already naturally a "leader" in our context, as they're the only ones that
>> can do things like trigger fee updates.
>>
>> ### Message Structure
>>
>> At a high level I propose we introduce two new messages, with the fields
>> looking something like this for `commitment_update_propose`:
>> * type: 0 (`channel_id`)
>> * value: [`32*byte`:`chan_id`]
>> * type: 1 (`propose_sig`)
>> * value: [`64*byte`:`sig`]
>> * type: 2 (`update_payload`)
>> * value: [`*byte`:`tlv_payload`]
>>
>> and this `commitment_update_apply`:
>> * type: 0 (`channel_id`)
>> * value: [`32*byte`:`chan_id`]
>> * type: 1 (`local_propose`)
>> * value: [`*byte`:`commitment_update_propose`]
>> * type: 2 (`remote_propose`)
>> * value: [`*byte`:`commitment_update_propose`]
>>
>> ### Protocol Flow
>>
>> The core idea here is that either party can propose a commitment/channel
>> param update, but only the initiator can actually apply it. The
>> `commitment_update_propose` encodes the new set of updates, with a signature
>> covering the TLV blob for the new params (more on why that's needed later).
>> The `commitment_update_apply` includes up to _two_
>> `commitment_update_propose` messages (one for the initiator and one for the
>> responder, as nested TLV messages). The `commitment_update_propose` message
>> would be treated like any other `update_*` message, in that it takes a new
>> commitment signature to properly commit/apply it.
>>
>> The normal flow takes the form of both sides sending a
>> `commitment_update_propose` message, with the initiator finally committing
>> both by sending a `commitment_update_apply` message. In the event that only
>> the responder wants to apply a param change/update, then the initiator can
>> reply immediately with a `commitment_update_apply` message that doesn't
>> include a param change for their commitment (or they just echo the
>> parameters if they're acceptable).
>>
>> ### Handling Retransmissions
>>
>> The role of the signature it to prevent "spoofing" by one of the parties
>> (authenticate the param change), and also it serves to convince a party that
>> they actually sent a prior commitment propose update during the
>> retransmission phase. As the `commitment_update_propose` message would be
>> retransmitted like any other message, if the initiator attempts to commit
>> the update but the connection dies, they'll retransmit it as normal along
>> with their latest signature.
>>
>> ### Nested TLV Param Generality
>>
>> The messages as sketched out here just have an opaque nested TLV field which
>> makes it extensible to add in other things like tweaking the total number of
>> max HTLCs, the current dust values, min/max HTLCs, etc (all things that are
>> currently hard coded for the lifetime of the channel). An initial target
>> would likely just be a `chan_type` field, with future feature bits governing
>> _what_ type of commitment updates both parties understand in the future.
>>
>> In the past, when ideas like this were brought up, some were concerned that
>> it wouldn't really be possible to do this type of updates while existing
>> HTLCs were in flight (hence some of the ideas to clear out the commitment
>> beforehand). I don't see a reason why this fundamentally _shouldn't_ be
>> allowed, as from the point of view of the channel update state machine, all
>> updates (adds/removes) get applied as normal, but with this _new_ commitment
>> type/params. The main edge case we'll need to consider is cases where the
>> new params make older HTLCs invalid for some reason.
>>
>> ## Conclusion
>>
>> Using the adapter commitment idea combined with a protocol for updating
>> commitments on the fly, would potentially allow us to update all 80k+ segwit
>> v0 channels to the base level of taprooty channels without any on chain
>> transactions. The two transactions (open+close) must happen eventually, but
>> by holding another layer of spends off-chain we can defer them (potentially
>> indefinitely, as we have channels today that have been opened for over a
>> year).
>>
>> Deploying a generalised on-the-fly dynamic commitment update protocol gives
>> us a tool to future proof the _existing_ anchored multi-sig outputs in the
>> chain, and also a way to remove many of the hard coded parameters we have
>> today in the protocol. One overly inflexible parameter we have today in the
>> network is the 483 HTLC limit. Allowing this value to float would allow
>> peers to apply similar congestion avoidance algorithm that are used in TCP
>> today, and also give us a way to protect the network against future
>> unforeseen widespread policy changes (like a raising of the dust limit).
>>
>> -- Laolu
>>
>> [1]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-July/002763.html
>> <https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-July/002763.html>;
>> [2]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-July/002770.html
>> <https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-July/002770.html>;
>> [3]: https://github.com/lightning/bolts/pull/868
>> <https://github.com/lightning/bolts/pull/868>;
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org <mailto:Lightning-dev at lists.linuxfoundation.org>
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>> <https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev>;
>>
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org <mailto:Lightning-dev at lists.linuxfoundation.org>
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>> <https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev>;
>>
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org <mailto:Lightning-dev at lists.linuxfoundation.org>
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>> <https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev>;
>

📅 Original date posted:2022-09-23 📝 Original message: Two ...

2023-06-09T15:06:51+02:00

In reply to nevent1q…2fmh
_________________________

📅 Original date posted:2022-09-23
📝 Original message:
Two questions -
a) How much gossip overhead do you expect this type of protocol to generate/is there a useful
outcome for this type of update even if you limit gossip updates to once/twice/thrice per day?
b) What are the privacy implications of the naive "update on drained channel", and have you done any
analysis of the value of this type of gossip update at different levels of privacy?

Thanks,
Matt

On 9/22/22 2:40 AM, René Pickhardt via Lightning-dev wrote:
> Good morning fellow Lightning Developers,
>
> I am pleased to share my most recent research results [1] with you. They may (if at all) only have a
> small impact on protocol development / specification but are actually mainly of concern to node
> operators and LSPs. I still thought they may be relevant for the list.
>
> While trying to estimate the expected liquidity distribution in depleted channels due to drain via
> Markov Models I realized that we can exploit the `htlc_maxium_msat` setting to act as a control
> valve and regulate the "pressure" coming from the drain and mitigate the depletion of channels. Such
> ideas are btw not novel at all and heavily used in fluid networks [2]. Thus it seems very natural
> that we do the same on the Lightning Network.
>
> In the article we show within a theoretic model how expected payment failure rates per channel may
> drop significantly by up to an order of magnitude if channels set up proper asymmetric
> `htlc_maximum_msat` pairs.
>
> We furthermore provide in our iPython notebook [3] two experimental algorithmic ideas with which
> node operators can find decent `htlc_maximum_msat` values in a greedy fashion. One of the algorithms
> does not even require to know the drain or payment size distribution or build the Markov model but
> just looks at the liquidity distribution in the channel at the last x routing attempts and adjusts
> the `htlc_maximum_msat` value if the distribution is to far away from a uniform distribution.
>
> Looking forwards for your thoughts and feedback.
>
> with kind regards Rene
>
>
> [1]:
> https://blog.bitmex.com/the-power-of-htlc_maximum_msat-as-a-control-valve-for-better-flow-control-improved-reliability-and-lower-expected-payment-failure-rates-on-the-lightning-network/ <https://blog.bitmex.com/the-power-of-htlc_maximum_msat-as-a-control-valve-for-better-flow-control-improved-reliability-and-lower-expected-payment-failure-rates-on-the-lightning-network/>;
> [2]: https://en.wikipedia.org/wiki/Control_valve <https://en.wikipedia.org/wiki/Control_valve>;
> [3]:
> https://github.com/lnresearch/Flow-Control-on-Lightning-Network-Channels-with-Drain-via-Control-Valves/blob/main/htlc_maximum_msat%20as%20a%20valve%20for%20flow%20control%20on%20the%20Lightnig%20network.ipynb <https://github.com/lnresearch/Flow-Control-on-Lightning-Network-Channels-with-Drain-via-Control-Valves/blob/main/htlc_maximum_msat%20as%20a%20valve%20for%20flow%20control%20on%20the%20Lightnig%20network.ipynb>;
>
> --
> https://ln.rene-pickhardt.de <https://ln.rene-pickhardt.de>;
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

📅 Original date posted:2022-07-10 📝 Original message: On ...

2023-06-09T15:06:37+02:00

In reply to nevent1q…mn40
_________________________

📅 Original date posted:2022-07-10
📝 Original message:
On 7/10/22 4:43 AM, Joost Jager wrote:
> It can also be considered a bad thing that DoS ability is not based on a number of messages. It
> means that for the one time cost of channel open/close, the attacker can generate spam forever if
> they stay right below the rate limit.

I don't see why this is a problem? This seems to assume some kind of per-message cost that nodes
have to bear, but there is simply no such thing. Indeed, if message spam causes denial of service to
other network participants, this would be an issue, but an attacker generating spam from one
specific location within the network should not cause that, given some form of backpressure within
the network.

> Suppose the attacker has enough channels to hit the rate limit on an important connection some hops
> away from themselves. They can then sustain that attack indefinitely, assuming that they stay below
> the rate limit on the routes towards the target connection. What will the response be in that case?
> Will node operators work together to try to trace back to the source and take down the attacker?
> That requires operators to know each other.

No it doesn't, backpressure works totally fine and automatically applies pressure backwards until
nodes, in an automated fashion, are appropriately ratelimiting the source of the traffic.

> Maybe this is a difference between lightning network and the internet that is relevant for this
> discussion. That routers on the internet know each other and have physical links between them, where
> as in lightning ties can be much looser.

No? The internet does not work by ISPs calling each other up on the phone to apply backpressure
manually whenever someone sends a lot of traffic? If anything lightning ties between nodes are much,
much stronger than ISPs on the internet - you generally are at least loosely trusting your peer with
your money, not just your customer's customer's bits.

Matt

📅 Original date posted:2022-06-29 📝 Original message: ...

2023-06-09T15:06:26+02:00

In reply to nevent1q…myjs
_________________________

📅 Original date posted:2022-06-29
📝 Original message:
Thanks Bastien for writing this up! This is a pretty trivial and straightforward way to rate-limit
onion messages in a way that allows legitimate users to continue using the system in spite of some
bad actors trying (and failing, due to being rate-limited) to DoS the network.

I do think any spec for this shouldn't make any recommendations about willingness to relay onion
messages for anonymous no-channel third parties, if anything deliberately staying mum on it and
allowing nodes to adapt policy (and probably rate-limit no-channel third-parties before they rate
limit any peer they have a channel with). Ultimately, we have to assume that nodes will attempt to
send onion messages by routing through the existing channel graph, so there's little reason to worry
too much about ensuring ability to relay for anonymous parties.

Better yet, as Val points out, requiring a channel to relay onion messages puts a very real,
nontrivial (in a world of msats) cost to getting an onion messaging channel. Better yet, with
backpressure ability to DoS onion message links isn't denominated in number of messages, but instead
in number of channels you are able to create, making the backpressure system equivalent to today's
HTLC DoS considerations, whereas explicit payment allows an attacker to pay much less to break the
system.

As for the proposal to charge for onion messages, I'm still not at all sure where its coming from.
It seems to flow from a classic "have a hammer (a system to make micropayments for things), better
turn this problem into a nail (by making users pay for it)" approach, but it doesn't actually solve
the problem at hand.

Even if you charge for onion messages, users may legitimately want to send a bunch of payments in
bulk, and trivially overflow a home or Tor nodes' bandwidth. The only response to that, whether its
a DoS attack or a legitimate user, is to rate-limit, and to rate-limit in a way that tells the user
sending the messages to back off! Sure, you could do that by failing onion messages with an error
that updates the fee you charge, but you're ultimately doing a poor-man's (or, I suppose,
rich-man's) version of what Bastien proposes, not adding some fundamental difference.

Ultimately, paying suffers from the standard PoW-for-spam issue - you cannot assign a reasonable
cost that an attacker cares about without impacting the system's usability due to said cost. Indeed,
making it expensive enough to mount a months-long DDoS without impacting legitimate users be pretty
easy - at 1msat per relay of a 1366 byte onion message you can only saturate an average home users'
30Mbps connection for 30 minutes before you rack up a dollar in costs, but if your concern is
whether someone can reasonably trivially take out the network for minutes at a time to make it have
perceptibly high failure rates, no reasonable cost scheme will work. Quite the opposite - the only
reasonable way to respond is to respond to a spike in traffic while maintaining QoS is to rate-limit
by inbound edge!

Ultimately, what we have here is a networking problem, that has to be solved with networking
solutions, not a costing problem, which can be solved with payment. I can only assume that the
desire to add a cost to onion messages ultimately stems from a desire to ensure every possible
avenue for value extraction is given to routing nodes, but I think that desire is misplaced in this
case - the cost of bandwidth is diminutive compared to other costs of routing node operation,
especially when you consider sensible rate-limits as proposed in Bastien's email.

Indeed, if anyone were proposing rate-limits which would allow anything close to enough bandwidth
usage to cause "lightning is turning into Tor and has Tor's problems" to be a legitimate concern I'd
totally agree we should charge for its use. But no one is, nor has anyone ever seriously, to my
knowledge, proposed such a thing. If lightning messages get deployed and start eating up even single
Mbps's on a consistent basis on nodes, we can totally revisit this, its not like we are shutting the
door to any possible costing system if it becomes necessary, but rate-limiting has to happen either
way, so we should start there and see if we need costing, not jump to costing on day one, hampering
utility.

Matt

On 6/29/22 8:22 PM, Olaoluwa Osuntokun wrote:
> Hi t-bast,
>
> Happy to see this finally written up! With this, we have two classes of
> proposals for rate limiting onion messaging:
>
> 1. Back propagation based rate limiting as described here.
>
> 2. Allowing nodes to express a per-message cost for their forwarding
> services, which is described here [1].
>
> I still need to digest everything proposed here, but personally I'm more
> optimistic about the 2nd category than the 1st.
>
> One issue I see w/ the first category is that a single party can flood the
> network and cause nodes to trigger their rate limits, which then affects the
> usability of the onion messages for all other well-behaving parties. An
> example, this might mean I can't fetch invoices, give up after a period of
> time (how long?), then result to a direct connection (perceived payment
> latency accumulated along the way).
>
> With the 2nd route, if an attacker floods the network, they need to directly
> pay for the forwarding usage themselves, though they may also directly cause
> nodes to adjust their forwarding rate accordingly. However in this case, the
> attacker has incurred a concrete cost, and even if the rates rise, then
> those that really need the service (notifying an LSP that a user is online
> or w/e) can continue to pay that new rate. In other words, by _pricing_ the
> resource utilization, demand preferences can be exchanged, leading to more
> efficient long term resource allocation.
>
> W.r.t this topic, one event that imo is worth pointing out is that a very
> popular onion routing system, Tor, has been facing a severe DDoS attack that
> has lasted weeks, and isn't yet fully resolved [2]. The on going flooding
> attack on Tor has actually started to affect LN (iirc over half of all
> public routing nodes w/ an advertised address are tor-only), and other
> related systems like Umbrel that 100% rely on tor for networking traversal.
> Funnily enough, Tor developers have actually suggested adding some PoW to
> attempt to mitigate DDoS attacks [3]. In that same post they throw around
> the idea of using anonymous tokens to allow nodes to give them to "good"
> clients, which is pretty similar to my lofty Forwarding Pass idea as relates
> to onion messaging, and also general HTLC jamming mitigation.
>
> In summary, we're not the first to attempt to tackle the problem of rate
> limiting relayed message spam in an anonymous/pseudonymous network, and we
> can probably learn a lot from what is and isn't working w.r.t how Tor
> handles things. As you note near the end of your post, this might just be
> the first avenue in a long line of research to best figure out how to handle
> the spam concerns introduced by onion messaging. From my PoV, it still seems
> to be an open question if the same network can be _both_ a reliable
> micro-payment system _and_ also a reliable arbitrary message transport
> layer. I guess only time will tell...
>
> > The `shared_secret_hash` field contains a BIP 340 tagged hash
>
> Any reason to use the tagged hash here vs just a plain ol HMAC? Under the
> hood, they have a pretty similar construction [4].
>
> [1]: https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-February/003498.html
> <https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-February/003498.html>;
> [2]: https://status.torproject.org/issues/2022-06-09-network-ddos/
> <https://status.torproject.org/issues/2022-06-09-network-ddos/>;
> [3]: https://blog.torproject.org/stop-the-onion-denial/
> <https://blog.torproject.org/stop-the-onion-denial/>;
> [4]: https://datatracker.ietf.org/doc/html/rfc2104 <https://datatracker.ietf.org/doc/html/rfc2104>;
>
> -- Laolu
>
>
>
> On Wed, Jun 29, 2022 at 1:28 AM Bastien TEINTURIER <bastien at acinq.fr <mailto:bastien at acinq.fr>> wrote:
>
> During the recent Oakland Dev Summit, some lightning engineers got together to discuss DoS
> protection for onion messages. Rusty proposed a very simple rate-limiting scheme that
> statistically propagates back to the correct sender, which we describe in details below.
>
> You can also read this in gist format if that works better for you [1].
>
> Nodes apply per-peer rate limits on _incoming_ onion messages that should be relayed (e.g.
> N/seconds with some burst tolerance). It is recommended to allow more onion messages from
> peers with whom you have channels, for example 10/seconds when you have a channel and 1/second
> when you don't.
>
> When relaying an onion message, nodes keep track of where it came from (by using the `node_id` of
> the peer who sent that message). Nodes only need the last such `node_id` per outgoing connection,
> which ensures the memory footprint is very small. Also, this data doesn't need to be persisted.
>
> Let's walk through an example to illustrate this mechanism:
>
> * Bob receives an onion message from Alice that should be relayed to Carol
> * After relaying that message, Bob stores Alice's `node_id` in its per-connection state with Carol
> * Bob receives an onion message from Eve that should be relayed to Carol
> * After relaying that message, Bob replaces Alice's `node_id` with Eve's `node_id` in its
> per-connection state with Carol
> * Bob receives an onion message from Alice that should be relayed to Dave
> * After relaying that message, Bob stores Alice's `node_id` in its per-connection state with Dave
> * ...
>
> We introduce a new message that will be sent when dropping an incoming onion message because it
> reached rate limits:
>
> 1. type: 515 (`onion_message_drop`)
> 2. data:
> * [`rate_limited`:`u8`]
> * [`shared_secret_hash`:`32*byte`]
>
> Whenever an incoming onion message reaches the rate limit, the receiver sends `onion_message_drop`
> to the sender. The sender looks at its per-connection state to find where the message was coming
> from and relays `onion_message_drop` to the last sender, halving their rate limits with that peer.
>
> If the sender doesn't overflow the rate limit again, the receiver should double the rate limit
> after 30 seconds, until it reaches the default rate limit again.
>
> The flow will look like:
>
> Alice Bob Carol
> | | |
> | onion_message | |
> |------------------------>| |
> | | onion_message |
> | |------------------------>|
> | | onion_message_drop |
> | |<------------------------|
> | onion_message_drop | |
> |<------------------------| |
>
> The `shared_secret_hash` field contains a BIP 340 tagged hash of the Sphinx shared secret of the
> rate limiting peer (in the example above, Carol):
>
> * `shared_secret_hash = SHA256(SHA256("onion_message_drop") || SHA256("onion_message_drop") || sphinx_shared_secret)`
>
> This value is known by the node that created the onion message: if `onion_message_drop` propagates
> all the way back to them, it lets them know which part of the route is congested, allowing them
> to retry through a different path.
>
> Whenever there is some latency between nodes and many onion messages, `onion_message_drop` may
> be relayed to the incorrect incoming peer (since we only store the `node_id` of the _last_ incoming
> peer in our outgoing connection state). The following example highlights this:
>
> Eve Bob Carol
> | onion_message | |
> |------------------------>| onion_message |
> | onion_message |------------------------>|
> |------------------------>| onion_message |
> | onion_message |------------------------>|
> |------------------------>| onion_message |
> |------------------------>|
> Alice | onion_message_drop |
> | onion_message | +----|
> |------------------------>| onion_message | |
> | |--------------------|--->|
> | | | |
> | | | |
> | | | |
> | onion_message_drop |<-------------------+ |
> |<------------------------| |
>
> In this example, Eve is spamming but `onion_message_drop` is propagated back to Alice instead.
> However, this scheme will _statistically_ penalize the right incoming peer (with a probability
> depending on the volume of onion messages that the spamming peer is generating compared to the
> volume of legitimate onion messages).
>
> It is an interesting research problem to find formulas for those probabilities to evaluate how
> efficient this will be against various types of spam. We hope researchers on this list will be
> interested in looking into it and will come up with a good model to evaluate that scheme.
>
> To increase the accuracy of attributing `onion_message_drop`, more data could be stored in the
> future if it becomes necessary. We need more research to quantify how much accuracy would be
> gained by storing more data and making the protocol more complex.
>
> Cheers,
> Bastien
>
> [1]https://gist.github.com/t-bast/e37ee9249d9825e51d260335c94f0fcf <https://gist.github.com/t-bast/e37ee9249d9825e51d260335c94f0fcf>;
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org <mailto:Lightning-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev>;
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev

📅 Original date posted:2022-05-26 📝 Original message: Oops, ...

2023-06-09T15:06:04+02:00

In reply to nevent1q…8jvu
_________________________

📅 Original date posted:2022-05-26
📝 Original message:
Oops, sorry, I don't really monitor the dev lists but once every few months so this fell off my plate :/

On 4/28/22 6:11 PM, Rusty Russell wrote:
> Matt Corallo <lf-lists at mattcorallo.com> writes:
> OK, let's step back. Unlike Bitcoin, we can use a single sketch for
> *all* peers. This is because we *can* encode enough information that
> you can get useful info from the 64 bit id, and because it's expensive
> to create them so you can't spam.

Yep, makes sense.

> The more boutique per-peer handling we need, the further it gets from
> this ideal;.
>
>> The second potential thing I think you might have meant here I don't see as an issue at all? You can
>> simply...let the sketch include one channel update that you ignored? See above discussion of similar
>> rate-limits.
>
> No, you need to get all the ignored ones somehow? There's so much cruft
> in the sketch you can't decode it. Now you need to remember the ones
> you ratelimited, and try to match other's ratelimiting.

Right, you'd end up downloading the thing you rate-limited, but only once (possibly per-peer). If
you use the total-sync approach you'd download it on every sync, vs a "only updates" approach you'd
do it once.

>> I agree there should be *some* rough consensus, but rate-limits are a locally-enforced thing, not a
>> global one. There will always be races and updates you reject that your peers dont, no matter the
>> rate-limit, and while I agree we should have guidelines, we can't "just make them the same" - it
>> both doesn't solve the problem and means we can't change them in the future.
>
> Sure it does! It severly limits the set divergence to race conditions
> (down to block height divergence, in practice).

Huh? There's always some line you draw, if an update happens right on the line (which they almost
certainly often will because people want to update, and they'll update every X hours to whatever the
rate limit is), then ~half the network will accept the update and half won't. I don't see how you
solve this problem.
> Maybe. What's a "non-update" based sketch? Some huge percentage of
> gossip is channel_update, so it's kind of the thing we want?

Oops, maybe we're on *very* different pages, here - I mean doing sketches based on "the things that
I received since the last sync, ie all the gossip updates from the last hour" vs doing sketches
based on "the things I have, ie my full gossip store".

Matt

📅 Original date posted:2022-04-22 📝 Original message: On ...

2023-06-09T15:05:54+02:00

In reply to nevent1q…sftk
_________________________

📅 Original date posted:2022-04-22
📝 Original message:
On 4/21/22 7:20 PM, Rusty Russell wrote:
> Matt Corallo <lf-lists at mattcorallo.com> writes:
>> Sure, if you’re rejecting a large % of channel updates in total
>> you’re gonna end up hitting degenerate cases, but we can consider
>> tuning the sync frequency if that becomes an issue.
>
> Let's be clear: it's a problem.
>
> Allowing only 1 a day, ended up with 18% of channels hitting the spam
> limit. We cannot fit that many channel differences inside a set!
>
> Perhaps Alex should post his more detailed results, but it's pretty
> clear that we can't stay in sync with this many differences :(

Right, the fact that most nodes don't do any limiting at all and y'all have a *very* aggressive (by
comparison) limit is going to be an issue in any context. We could set some guidelines and improve
things, but luckily regular-update-sync bypasses some of these issues anyway - if we sync once per
block and your limit is once per block, getting 1000 updates per block for some channel doesn't
result in multiple failures in the sync. Sure, multiple peers sending different updates for that
channel can still cause some failures, but its still much better.

>> gossip queries is broken in at least five ways.
>
> Naah, it's perfect if you simply want to ask "give me updates since XXX"
> to get you close enough on reconnect to start using set reconciliation.
> This might allow us to remove some of the other features?

Sure, but that's *just* the "gossip_timestamp_filter" message, there's several other messages and a
whole query system that we can throw away if we just want that message :)

> But we might end up with a gossip2 if we want to enable taproot, and use
> blockheight as timestamps, in which case we could probably just support
> that one operation (and maybe a direct query op).
>
>> Like eclair, we don’t bother to rate limit and don’t see any issues with it, though we will skip relaying outbound updates if we’re saturating outbound connections.
>
> Yeah, we did as a trial, and in some cases it's become limiting. In
> particular, people restarting their LND nodes once a day resulting in 2
> updates per day (which, in 0.11.0, we now allow).

What do you mean "its become limiting"? As in you hit some reasonably-low CPU/disk/bandwidth limit
in doing this? We have a pretty aggressive bandwidth limit for this kinda stuff (well, indirect
bandwidth limit) and it very rarely hits in my experience (unless the peer is very overloaded and
not responding to pings, which is a somewhat separate thing...)

Matt

📅 Original date posted:2022-04-21 📝 Original message: ...

2023-06-09T15:05:53+02:00

In reply to nevent1q…ym9m
_________________________

📅 Original date posted:2022-04-21
📝 Original message:
Instead of trying to make sure everyone’s gossip acceptance matches exactly, which as you point it seems like a quagmire, why not (a) do a sync on startup and (b) do syncs of the *new* things. This way you aren’t stuck staring at the same channels every time you do a sync. Sure, if you’re rejecting a large % of channel updates in total you’re gonna end up hitting degenerate cases, but we can consider tuning the sync frequency if that becomes an issue.

Like eclair, we don’t bother to rate limit and don’t see any issues with it, though we will skip relaying outbound updates if we’re saturating outbound connections.

> On Apr 14, 2022, at 17:06, Alex Myers <alex at endothermic.dev> wrote:
>
>
> Hello lightning developers,
>
> I’ve been investigating set reconciliation as a means to reduce bandwidth and redundancy of gossip message propagation. This builds on some earlier work from Rusty using the minisketch library [1]. The idea is that each node will build a sketch representing it’s own gossip set. Alice’s node will encode and transmit this sketch to Bob’s node, where it will be merged with his own sketch, and the differences produced. These differences should ideally be exactly the latest missing gossip of both nodes. Due to size constraints, the set differences will necessarily be encoded, but Bob’s node will be able to identify which gossip Alice is missing, and may then transmit exactly those messages.
>
> This process is relatively straightforward, with the caveat that the sets must otherwise match very closely (each sketch has a maximum capacity for differences.) The difficulty here is that each node and lightning implementation may have its own rules for gossip acceptance and propagation. Depending on their gossip partners, not all gossip may propagate to the entire network.
>
> Core-lightning implements rate limiting for incoming channel updates and node announcements. The default rate limit is 1 per day, with a burst of 4. I analyzed my node’s gossip over a 14 day period, and found that, of all publicly broadcasting half-channels, 18% of them fell afoul of our spam-limiting rules at least once. [2]
>
> Picking several offending channel ids, and digging further, the majority of these appear to be flapping due to Tor or otherwise intermittent connections. Well connected nodes may be more susceptible to this due to more frequent routing attempts, and failures resulting in a returned channel update (which otherwise might not have been broadcast.) A slight relaxation of the rate limit resolves the majority of these cases.
>
> A smaller subset of channels broadcast frequent channel updates with minor adjustments to htlc_maximum_msat and fee_proportional_millionths parameters. These nodes appear to be power users, with many channels and large balances. I assume this is automated channel management at work.
>
> Core-Lightning has updated rate-limiting in the upcoming release to achieve a higher acceptance of incoming gossip, however, it seems that a broader discussion of rate limits may now be worthwhile. A few immediate ideas:
> - A common listing of current default rate limits across lightning network implementations.
> - Internal checks of RPC input to limit or warn of network propagation issues if certain rates are exceeded.
> - A commonly adopted rate-limit standard.
>
> My aim is a set reconciliation gossip type, which will use a common, simple heuristic to accept or reject a gossip message. (Think one channel update per block, or perhaps one per block_height << 5.) See my github for my current draft. [3] This solution allows tighter consensus, yet suffers from the same problem as original anti-spam measures – it remains somewhat arbitrary. I would like to start a conversation regarding gossip propagation, channel_update and node_announcement usage, and perhaps even bandwidth goals for syncing gossip in the future (how about a million channels?) This would aid in the development of gossip set reconciliation, but could also benefit current node connection and routing reliability more generally.
>
> Thanks,
> Alex
>
> [1] https://github.com/sipa/minisketch
> [2] https://github.com/endothermicdev/lnspammityspam/blob/main/sampleoutput.txt
> [3] https://github.com/endothermicdev/lightning-rfc/blob/gossip-minisketch/07-routing-gossip.md#set-reconciliation
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220421/bfb44d28/attachment.html>;

📅 Original date posted:2022-04-22 📝 Original message: On ...

2023-06-09T15:05:53+02:00

In reply to nevent1q…gzpx
_________________________

📅 Original date posted:2022-04-22
📝 Original message:
On 4/22/22 9:15 AM, Alex Myers wrote:
> Hi Matt,
>
> Appreciate your responses. Hope you'll bear with me as I'm a bit new to this.
>
> Instead of trying to make sure everyone’s gossip acceptance matches exactly, which as you point
> it seems like a quagmire, why not (a) do a sync on startup and (b) do syncs of the *new* things.
>
> I'm not opposed to this technique, and maybe it ends up as a better solution. The rationale for not
> going full Erlay approach was that it's far less overhead to maintain a single sketch than to
> maintain a per-peer sketch and associated state for every gossip peer. In this way there's very
> little cost to adding additional gossip peers, which further encourages propagation and convergence
> of the gossip network.

I'm not sure what you mean by per-node state here - I'd think you can implement it with a simple
"list of updates that happened since time X" data, instead of having to maintain per-peer state.

> IIUC Erlay's design was concerned for privacy of originating nodes. Lightning gossip is public by
> nature, so I'm not sure we should constrain ourselves to the same design route without trying the
> alternative first.

Part of the design of Erlay, especially the insight of syncing updates instead of full mempools, was
actually this precise issue - Bitcoin Core nodes differ in policy for a number of reasons
(especially across updates), and thus syncing the full mempool will result in degenerate cases of
trying over and over and over again to sync stuff your peer is rejecting. At least if I recall
correctly.

> if we're gonna add a minisketch-based sync anyway, please lets also use it for initial sync
> after restart
>
> This was out of the scope of what I had in mind, but I will give this some thought. I could see how
> a block_height reference coupled with set reconciliation could provide some better options here.
> This may not be all that difficult to shoe-horn in.
>
> Regardless of single sketch or per-peer set reconciliation, it should be easier to implement with
> tighter rules on rate-limiting. (Keep in mind, the node's graph can presumably be updated
> independently of the gossip it rebroadcasts if desired.) As a thought experiment, if we consider a
> CLN-LDK set reconciliation, and that each node is gossiping with 5 other peers in an evenly spaced
> frequency, we would currently see 42.8 commonly accepted channel_updates over an average 60s window
> along with 11 more updates which LDK accepts and CLN rejects (spam.)[1] Assuming the other 5 peers
> have shared 5/6ths of this gossip before the CLN/LDK set reconciliation, we're left with CLN seeing
> 7 updates to reconcile, while LDK sees 18. Already we've lost 60% efficiency due to lack of a
> common rate-limit heuristic.

I do not believe that we will ever form a strong agreement on exactly what the rate-limits should
be. And even if we do, we still have the issue of upgrades, where a simple change to the rate-limits
causes sync to suddenly blow up and hit degenerate cases all over the place. Unless we can make the
sync system relatively robust against slightly different policies, I think we're kinda screwed.

Worse, what happens if someone sends updates at exactly the limit of the rate-limiters? Presumably
people will do this because "that's what the limit is and I want to send updates as often as I can
becaux...". Now you'll still have similar issues, I believe.

> I understand gossip traffic is manageable now, but I'm not sure it will be that long before it
> becomes an issue. Furthermore, any particular set reconciliation technique would benefit from a
> simple common rate-limit heuristic, not to mention originating nodes, who may not currently realize
> their channel updates are being rejected by a portion of the network due to differing criteria
> across implementations.

Yes, I agree there is definitely a concern with differing criteria resulting in nodes not realizing
their gossip is not propagating. I agree guidelines would be nice, but guidelines doesn't solve the
issue for sync, sadly, I think. Luckily lightning does provide a mechanism to bypass the rejection -
send an update back with an HTLC failure. If you're trying to route an HTLC and a node has new
parameters for you, it'll helpfully let you know when you try to use the old parameters.

Matt

📅 Original date posted:2021-10-13 📝 Original message: On ...

2023-06-09T15:04:14+02:00

In reply to nevent1q…hxya
_________________________

📅 Original date posted:2021-10-13
📝 Original message:
On 10/12/21 22:08, Andrés G. Aragoneses wrote:
> Hello Matt, can you clarify what you mean with this particular paragraph?:
>
> But for some reason those pesky users keep wanting to use lightning for tips, or at least accept
> payment on their phones without keeping them unlocked with the lightning app open on the foreground
> 24/7.
>
>
> So the use case here is more narrow? You mean that the recipient is a mobile user that has his phone
> locked?
> Just so I understand better what the problem is.

Yes, but not just locked, just "doesn't have the lightning app open and in the foreground when a
payment comes in". See this paragraph:

> Several lightning apps do this today, and its somewhat of a stop-gap but does help. On
> platforms
> where the app gets some meager CPU time in response to a notification, this can even fully solve
> the
> problem by claiming the HTLC in response to the notification pushed out-of-band. Sadly, the refrain
> I've heard repeatedly is, these days, on both Android and especially iOS, you can't even rely on a
> microsecond of CPU time in response to a notification. The OS fully expects your app to run code
> only when its on and in the foreground, unless you're a VoIP app you're screwed. Relying on the
> user
> to open the app immediately when they receive a notification is...fine, I guess, absent a better
> idea it seems like the best we've got today, but I'm not sure you'd find a UX designer who would
> *suggest* this :).

📅 Original date posted:2021-08-16 📝 Original message: ...

2023-06-09T15:03:28+02:00

In reply to nevent1q…2pd8
_________________________

📅 Original date posted:2021-08-16
📝 Original message:
Dropped a number of replies to which the reply would otherwise be "see above".

On 8/16/21 00:00, Anthony Towns wrote:
> On Sun, Aug 15, 2021 at 10:21:52PM -0400, Matt Corallo wrote:
>> On 8/15/21 22:02, Anthony Towns wrote:
>>>> In
>>>> one particular class of applicable routing algorithms you could use for
>>>> lightning routing having a base fee makes the algorithm intractably slow,
>>> I don't think of that as the problem, but rather as the base fee having
>>> a multiplicative effect as you split payments.
>> Yes, matching the real-world costs of forwarding an HTLC.
>
> Actually, no, not at all.
>
> The base+proportional fees paid only on success roughly match the *value*
> of forwarding an HTLC, they don't match the costs particularly well
> at all.
>
> Why not? Because the costs are incurred on failed HTLCs as well, and
> also depend on the time a HTLC lasts, and also vary heavily depending
> on how many other simultaneous HTLCs there are.

Sure, indeed, there's some additional costs which are not covered by failed HTLCs, nor incorporate the time the HTLC
slot was used. But that wasn't my argument - my argument was that base + proportional is a much, much closer match for
the costs of a node barring clever-er solutions around HTLC-slot-time-used. Dropping base fee makes the whole situation
a good chunk *worse*.

>> Yes. You have to pay the cost of a node. If we're really worried about this,
>> we should be talking about upfront fees and/or refunds on HTLC fulfillment,
>> not removing the fees entirely.
>
> (I don't believe either of those are the right approach, but based on
> previous discussions, I don't think anyone's going to realise I'm right
> until I implement it and prove it, so *shrug*)

I think I agree, but I think they may currently be better than any *other* proposal, not that they're particularly good.

>> The cost to nodes is largely [...]
>
> The cost to nodes is almost entirely the opportunity cost of not being
> able to accept other txs that would come in afterwards and would pay
> higher fees.
>
> And all those costs can be captured equally well (or badly) by just
> setting a proportional fee and a minimum payment value. I don't know why
> you keep ignoring that point.

I didn't ignore this, I just disagree, and I'm not entirely sure why you're ignoring the points I made to that effect :).

In all seriousness, I'm entirely unsure why you think proportional is just as good? As you note, the cost for nodes is a
function of the opportunity cost of the capital, and opportunity cost of the HTLC slots. Lets say as a routing node I
decide that the opportunity cost of one of my HTLC slots is generally 1 sat per second, and the average HTLC is
fulfilled in one second. Why is it that a proportional fee captures this "equally well"?!

Yes, you could amortize it, but that doesn't make it "equally" good, and there are semi-serious proposals to start
ignoring nodes that *dont* set their fees to some particular structure in routing decisions. Sure, nodes can do what
they want, but its kinda absurd to suggest that this is a perfectly fine thing to do absent a somewhat compelling
reason. This goes doubly because deploying such things significantly will mean we cannot do future protocol changes
which may better capture the time-value of node resources!

>>> Additionally, I don't think HTLC slot usage needs to be kept as a
>>> limitation after we switch to eltoo;
>> The HTLC slot limit is to keep transactions broadcastable. I don't see why
>> this would change, you still get an output for each HTLC on the latest
>> commitment in eltoo, AFAIU.
>
> eltoo gives us the ability to have channel factories....

That doesn't solve the issue at all - you still have a ton of transactions and transaction outputs and spends thereof to
put on the chain in the case of a closure with pending HTLCs. In fact, most nodes today enforce a lower limit than the
400-some-odd HTLCs that represent the transaction standardness limit, because 100KB transactions are stupid impractical.

> (By "any time soon" I mean, I could see software defaults changing if
> over 50% of the network deliberately switched to zero base fees and found
> it worked fine; and I could see deprecating non-zero fees if that ended
> up with 90% of the network on zero base fees, no good reasons for node
> operators wanting to stick with running non-zero base fees, and the
> experimental algos that relied on zero base fees being significantly
> easier to maintain or faster/better)

What is your definition of "works fine" here? In today's nearly-entirely-altruistic-routing-node network, we could
probably entirely drop the routing fees and things would "work fine". That doesn't make it a good idea for the long-term
health of the network.

My suggestion is quite simple - that the software vendors wishing to rely on these types of algorithms *first* do the
legwork to see what other ideas can be explored before jumping to "ignore all the nodes who've decided their fees are
X", because I think *that* is pretty bad idea for the long-term health of the network. I even suggested several areas of
future research for folks to look into before we get to the point of in any way seriously relying on routing algorithms
that constrain our ability to adapt fees in the future.

Matt

📅 Original date posted:2021-08-15 📝 Original message: On ...

2023-06-09T15:03:27+02:00

In reply to nevent1q…qmpq
_________________________

📅 Original date posted:2021-08-15
📝 Original message:
On 8/15/21 22:02, Anthony Towns wrote:
>> In
>> one particular class of applicable routing algorithms you could use for
>> lightning routing having a base fee makes the algorithm intractably slow,
>
> I don't think of that as the problem, but rather as the base fee having
> a multiplicative effect as you split payments.

Yes, matching the real-world costs of forwarding an HTLC.

> If every channel has the same (base,proportional) fee pair, and send a
> payment along a single path, you're paying n*(base+k*proportional). If
> you split the payment, and send half of it one way, and half the other
> way, you're paying n*(2*base+k*proportional). If you split the payment
> four ways, you're paying n*(4*base+k*proportional). Where's the value
> to the network in penalising payment splitting?

Yes. You have to pay the cost of a node. If we're really worried about this, we should be talking about upfront fees
and/or refunds on HTLC fulfillment, not removing the fees entirely.

> Being denominated in sats, the base fee also changes in value as the
> bitcoin price changes -- c-lightning dropped the base fee to 1sat (from
> 546 sat!) in Jan 2018, but the value of 1sat has increased about 4x
> since then, and it seems unlikely the fixed costs of a successful HTLC
> payment have likewise increased 4x. Proportional fees deal with this
> factor automatically, of course.

This isn't a protocol issue, implementations can automate this without issue.

>> There's real cost to distorting the fee structures on the network away from
>> the costs of node operators,
>
> That's precisely what the base fee is already doing. Yes, we need some
> other way of charging fees to prevent using up too many slots or having
> transactions not fail in a timely manner, but the base fee does not
> do that.

Huh? For values much smaller than a node's liquidity, the cost for nodes is (mostly) a function of HTLCs, not the value.
The cost to nodes is largely (a) the forever-storage that exists, roughly, per HTLC ever on a channel, (b) the HTLC
slots which are highly limited for technical reasons per channel, (c) the disk/cpu/network/etc operations per HTLC on an
channel, (d) the liquidity required per node. I'd argue (c) is basically zero in any realistic context, (a) is pretty
low, but could be non-zero in some cases, so you really just have (b) and (d). For many HTLCs forwarded today, the
liquidity on a channel isn't much, so I'd argue for many HTLCs forwarded today per-payment costs mirror the cost to a
node much, much, much, much better than some proportional fees?

I'm really not sure where you're coming from here.

>> Imagine we find some great way to address HTLC slot flooding/DoS attacks (or
>> just chose to do it in a not-great way) by charging for HTLC slot usage, now
>> we can't fix a critical DoS issue because the routing algorithms we deployed
>> can't handle the new costing.
>
> I don't think that's true. The two things we don't charge for that can
> be abused by probing spam are HTLC slot usage and channel balance usage;
> both are problems only in proportion to the amount of time they're held
> open, and the latter is also only a problem proportional to the value
> being reserved. [0]
>
> Additionally, I don't think HTLC slot usage needs to be kept as a
> limitation after we switch to eltoo;

The HTLC slot limit is to keep transactions broadcastable. I don't see why this would change, you still get an output
for each HTLC on the latest commitment in eltoo, AFAIU.

> and in the meantime, I think it can
> be better managed via adjusting the min_htlc_amount -- at least for the
> scenario where problems are being caused by legitimate payment attempts,
> which is also the only place base fee can help.

Sure, we could also shift towards upfront fees or similar solutions, though, and that was my point - if we start
dropping absolute fee amounts now in order to make some given routing algorithm work, we box ourselves in here, and
quite needlessly given no one has (yet) done the legwork to show that we even *need* to box ourselves in.

> [0] (Well, ln-penalty's requirement to permanently store HTLC information
> in order to apply the penalty is in some sense a constant
> cost, however the impact is also proportional to value, and for
> sufficiently low value HTLCs can be ignored entirely if the HTLC
> isn't included in the channel commitment)
>
>> Instead, we should investigate how we can
>> apply the ideas here with the more complicated fee structures we have.
>
> Fee structures should be *simple* not complicated.
>
> I mean, it's kind of great that we started off complicated -- if it
> turns out base fee isn't necessary, it's easy to just set it to zero;
> if we didn't have it, but needed it, it would be much more annoying to
> add it in later.

Fee structures should also match reality, and allow node operators sufficient flexibility to capture their costs. I
think we have a design that does so quite well - its pretty simple, there's only two knobs, but the two knobs capture
exactly the two broad categories of costs a node operator has.

>> Color me an optimist, but I'm quite confident with sufficient elbow grease
>> and heuristics we can get 95% of the way there. We can and should revisit
>> these conversations if such exploration is done and we find that its not
>> possible, but until then this all feels incredibly premature.
>
> Depends; I don't think it makes sense to try to ban nodes that don't have
> a base fee of zero or anything, but random people on twitter advocating
> that node operators should set it to zero and just worry about optimising
> via the proportional fee and the min htlc amount seems fine.

Sure, the great thing about today is because the protocol exposes decent knobs operators can tune their fee structures
any way they want, providing competition and multiple paths to nodes with potentially very divergent fees depending on
the type of payment. Absent liquidity limits, this should provide better service for all types of payments - routing
them to nodes that will support those types of payments!

> For an experimental plugin that aggressively splits payments up, I think
> either ignoring channels with >0 base fee entirely, or deciding that
> you're happy to spend a total of X sats on base fees, and then ignoring
> channels whose base fee is greater than X/paths/path-length sats is fine.

Sure, experimental plugins can do whatever they want!

> But long term, I also think that the base fee is an entirely unhelpful
> complication that will eventually just be hardcoded to zero by everyone,
> and eventually channels that propose non-zero base fees won't even be
> gossiped. I don't expect that to happen any time soon though.

I very strongly disagree, as discussed, and am left highly dubious that it is a practical complication in any case.

📅 Original date posted:2021-08-15 📝 Original message: ...

2023-06-09T15:03:26+02:00

In reply to nevent1q…dh45
_________________________

📅 Original date posted:2021-08-15
📝 Original message:
Thanks, AJ, for kicking off the thread.

I'm frankly still very confused why we're having these conversations now. In one particular class of applicable routing
algorithms you could use for lightning routing having a base fee makes the algorithm intractably slow, but:

a) to my knowledge, no one has (yet) done any follow-on work to investigate pulling many of the same heuristics Rene et
al use in a Dijkstras/A* algorithm with multiple passes or generating multiple routes in the same pass to see whether
you can emulate the results in a faster algorithm without the drawbacks here,

b) to my knowledge, no one has (yet) done any follow-on work to investigate mapping the base fee to other, more
flow-based-routing-compatible numbers, eg you could convert the base fee to a minimum fee by increasing the "effective"
proportional fees. From what others have commented, this may largely "solve" the issue.

c) to my knowledge, no one has (yet) done any follow-on work to analyze where the proposed algorithm may be most optimal
in the HTLC-value<->channel liquidity ratio ranges. We may find that the proposed algorithm only provides materially
better routing when the HTLC value approaches X% of common network channel liquidity, allowing us to only use it for
large-value payments where we can almost ignore the base fees entirely.

There's real cost to distorting the fee structures on the network away from the costs of node operators, especially as
we move towards requiring and using Real (tm) amounts of capital on routing nodes. If we're relying purely on hobbyists
forever who are operating out of goodwill, we should just remove all fees. If we think Lightning is going to involve
capital with real opportunity cost, matching fees to the costs is important, or at least important enough that we
shouldn't throw it away after one (pretty great) paper and limited further analysis.

Imagine we find some great way to address HTLC slot flooding/DoS attacks (or just chose to do it in a not-great way) by
charging for HTLC slot usage, now we can't fix a critical DoS issue because the routing algorithms we deployed can't
handle the new costing. Instead, we should investigate how we can apply the ideas here with the more complicated fee
structures we have.

Color me an optimist, but I'm quite confident with sufficient elbow grease and heuristics we can get 95% of the way
there. We can and should revisit these conversations if such exploration is done and we find that its not possible, but
until then this all feels incredibly premature.

Matt

On 8/14/21 21:00, Anthony Towns wrote:
> Hey *,
>
> There's been discussions on twitter and elsewhere advocating for
> setting the BOLT#7 fee_base_msat value [0] to zero. I'm just writing
> this to summarise my understanding in a place that's able to easily be
> referenced later.
>
> Setting the base fee to zero has a couple of benefits:
>
> - it means you only have one value to optimise when trying to collect
> the most fees, and one-dimensional optimisation problems are
> obviously easier to write code for than two-dimensional optimisation
> problems
>
> - when finding a route, if all the fees on all the channels are
> proportional only, you'll never have to worry about paying more fees
> just as a result of splitting a payment; that makes routing easier
> (see [1])
>
> So what's the cost? The cost is that there's no longer a fixed minimum
> fee -- so if you try sending a 1sat payment you'll pay 0.1% of the fee
> to send a 1000sat payment, and there may be fixed costs that you have
> in routing payments that you'd like to be compensated for (eg, the
> computational work to update channel state, the bandwith to forward the
> tx, or the opportunity cost for not being able to accept another htlc if
> you've hit your max htlcs per channel limit).
>
> But there's no need to explicitly separate those costs the way we do
> now; instead of charging 1sat base fee and 0.02% proportional fee,
> you can instead just set the 0.02% proportional fee and have a minimum
> payment size of 5000 sats (htlc_minimum_msat=5e6, ~$2), since 0.02%
> of that is 1sat. Nobody will be asking you to route without offering a
> fee of at least 1sat, but all the optimisation steps are easier.
>
> You could go a step further, and have the node side accept smaller
> payments despite the htlc minimum setting: eg, accept a 3000 sat payment
> provided it pays the same fee that a 5000 sat payment would have. That is,
> treat the setting as minimum_fee=1sat, rather than minimum_amount=5000sat;
> so the advertised value is just calculated from the real settings,
> and that nodes that want to send very small values despite having to
> pay high rates can just invert the calculation.
>
> I think something like this approach also makes sense when your channel
> becomes overloaded; eg if you have x HTLC slots available, and y channel
> capacity available, setting a minimum payment size of something like
> y/2/x**2 allows you to accept small payments (good for the network)
> when you're channel is not busy, but reserves the last slots for larger
> payments so that you don't end up missing out on profits because you
> ran out of capacity due to low value spam.
>
> Two other aspects related to this:
>
> At present, I think all the fixed costs are also incurred even when
> a htlc fails, so until we have some way of charging failing txs for
> incurring those costs, it seems a bit backwards to penalise successful
> txs who at least pay a proportional fee for the same thing. Until we've
> got a way of handling that, having zero base fee seems at least fair.
>
> Lower value HTLCs don't need to be included in the commitment transaction
> (if they're below the dust level, they definitely shouldn't be included,
> and if they're less than 1sat they can't be included), and as such don't
> incur all the same fixed costs that HTLCs that are committed too do.
> Having different base fees for microtransactions that incur fewer costs
> would be annoying; so having that be "amortised" into the proportional
> fee might help there too.
>
> I think eltoo can help in two ways by reducing the fixed costs: you no
> longer need to keep HTLC information around permanently, and if you do
> a multilevel channel factory setup, you can probably remove the ~400
> HTLCs per channel at any one time limit. But there's still other fixed
> costs, so I think that would just lower the fixed costs, not remove them
> altogether and isn't a fundamental change.
>
> I think the fixed costs for forwarding a HTLC are very small; something
> like:
>
> 0.02sats -- cost of permanently storing the HTLC info
> (100 bytes, $500/TB/year, 1% discount rate)
> 0.04sats -- compute and bandwidth cost for updating an HTLC ($40/month
> at linode, 1 second of compute)
>
> The opportunity cost of having HTLC slots or Bitcoin locked up until
> the HTLC succeeds/fails could be much more significant, though.
>
> Cheers,
> aj
>
> [0] https://github.com/lightningnetwork/lightning-rfc/blob/master/07-routing-gossip.md#the-channel_update-message
> [1] https://basefee.ln.rene-pickhardt.de/
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

📅 Original date posted:2021-04-24 📝 Original message: The ...

2023-06-09T15:02:23+02:00

In reply to nevent1q…z289
_________________________

📅 Original date posted:2021-04-24
📝 Original message:
The update_fee message does not, as far as I recall, change the dust limit for outputs in a channel (though I’ve suggested making such a change).

> On Apr 23, 2021, at 12:24, Bastien TEINTURIER <bastien at acinq.fr> wrote:
>
>
> Hi Eugene,
>
> The reason dust HTLCs count for the 483 HTLC limit is because of `update_fee`.
> If you don't count them and exceed the 483 HTLC limit, you can't lower the fee anymore
> because some HTLCs that were previously dust won't be dust anymore and you may end
> up with more than 483 HTLC outputs in your commitment, which opens the door to other
> kinds of attacks.
>
> This is the first issue that comes to mind, but there may be other drawbacks if we dig into
> this enough with an attacker's mindset.
>
> Bastien
>
>> Le ven. 23 avr. 2021 à 17:58, Eugene Siegel <elzeigel at gmail.com> a écrit :
>> I propose a simple mitigation to increase the capital requirement of channel-jamming attacks. This would prevent an unsophisticated attacker with low capital from jamming a target channel. It seems to me that this is a *free* mitigation without any downsides (besides code-writing), so I'd like to hear other opinions.
>>
>> In a commitment transaction, we trim dust HTLC outputs. I believe that the reason for the 483 HTLC limit each side has in the spec is to prevent commitment tx's from growing unreasonably large, and to ensure they are still valid tx's that can be included in a block. If we don't include dust HTLCs in this calculation, since they are not on the commitment tx, we still allow 483 (x2) non-dust HTLCs to be included on the commitment tx. There could be a configurable limit on the number of outstanding dust HTLCs, but the point is that it doesn't affect the non-dust throughput of the channel. This raises the capital requirement of channel-jamming so that each HTLC must be non-dust, rather than spamming 1 sat payments.
>>
>> Interested in others' thoughts.
>>
>> Eugene (Crypt-iQ)
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20210424/b09dc276/attachment.html>;

📅 Original date posted:2020-04-23 📝 Original message: Great ...

2023-06-09T14:59:50+02:00

In reply to nevent1q…ukys
_________________________

📅 Original date posted:2020-04-23
📝 Original message:
Great summary, a few notes inline.

> On Apr 22, 2020, at 21:50, ZmnSCPxj <ZmnSCPxj at protonmail.com> wrote:
>
> Good morning lists et al,
>
> Let me try to summarize things a little:
>
> * Suppose we have a forwarding payment A->B->C.
> * Suppose B does not want to maintain a mempool and is running in `blocksonly` mode to reduce operational costs.

Quick point of clarification, due to the mempool lacking a consensus system (that’s the whole point, after all :p), there are several reasons to that just running a full node/having a mempool isn’t sufficient.

> * C triggers B somehow dropping the B<->C channel, such as by sending an `error` message, which will usually cause the other side to drop the channel onchain using its commitment transaction.
> * The dropped B<->C channel has an HTLC (that was set up during the A->B->C forwarding).
> * The HTLC, being used in a Poon-Dryja channel, actually has the following contract text:
> * The fund may be claimed by either of these clauses:
> * C can claim, if C shows the preimage of some hash H (hashlock branch).
> * B and C must agree, and claim after time L (timelock branch).
> * B holds a signature from C that can claim the timelock branch of the HTLC, for a transaction that spends to an output with an `OP_CHECKSEQUENCEVERIFY`.
> * The signature is `SIGHASH_ALL`, so the transaction has a fixed feerate.
> * C can "pin" the HTLC output by spending using the hashlock branch, and creating a large fee, low fee-rate (tree of) transactions.

Another: this is the simplest example. There are also games around the package size limits if I recall correctly.

> * As it is a low fee-rate, miners have no incentive to put this in a block, especially if unrelated higher-fee-rate transactions exist that would earn them more money.
> * Even in a full RBF universe, because of the anti-DoS mempool rules, B cannot evict this pinned transaction by just bidding up the feerate.
> * A replacing transaction cannot evict alternatives unless its absolute fee is greater than the absolute fee of the alternative.
> * The pinning transaction has a high fee, but is blockspace-wasteful, so it is:
> * Undesirable to mine (low feerate).
> * Difficult to evict (high fee).
> * Thus, B is unable to get its timelock-branch transaction in the mempools of miners.
> * C waits until the A->B HTLC times out, then:
> * C directly contacts miners with an out-of-band proposal to replace its transaction with an alternative that is much smaller and has a low fee, but much better feerate.

Or they can just wait. For example in today’s mempool it would not be strange for a transaction at 1 sat/vbyte to wait a day but eventually confirm.

> * Miners, being economically rational, accept this proposal and include this in a block.
>
> The proposal by Matt is then:
>
> * The hashlock branch should instead be:
> * B and C must agree, and show the preimage of some hash H (hashlock branch).
> * Then B and C agree that B provides a signature spending the hashlock branch, to a transaction with the outputs:
> * Normal payment to C.
> * Hook output to B, which B can use to CPFP this transaction.
> * Hook output to C, which C can use to CPFP this transaction.
> * B can still (somehow) not maintain a mempool, by:
> * B broadcasts its timelock transaction.
> * B tries to CPFP the above hashlock transaction.
> * If CPFP succeeds, it means the above hashlock transaction exists and B queries the peer for this transaction, extracting the preimage and claiming the A->B HTLC.

Note that no query is required. The problem has been solved and the preimage-containing transaction should now confirm just fine.

> Is that a fair summary?

Yep!

> --
>
> Naively, and remembering I am completely ignorant of the exact details of the mempool rules, it seems to me quite strange that we are allowing an undesirable transaction (tree) into the mempool:
>
> * Undesirable to mine (low fee-rate).
> * Difficult to evict (high fee).

As noted, such transactions today are profit in 10 hours. Just because they’re big doesn’t mean they don’t pay.

> Miners are not interested in low fee-rate transactions, as long as higher fee-rate transactions exist.
> And being difficult to evict means miners cannot get alternatives that are more lucrative for them.
>
> The reason (as I understand it) eviction is purposely made difficult here is to prevent certain DoS attacks on Bitcoin nodes, specifically:
>
> 1. Attacker sends a low fee-rate tx as a "root" transaction.
> 2 Attacker sends thousands of low fee-rate tx that build off the above root.

I believe the limit is 25, though the point stands, mostly from a total-size perspective.

> 3. Attacker sends a slightly higher fee-rate alternative to the root, evicting the above tree of txes.
> 4. Attacker sends thousands of low fee-rate tx that build off the latest root.
> 5. GOTO 3.
>
> However, it seems to me, naively, that "an ounce of prevention is worth a pound of cure".

Sadly, it’s very very easy for this to be a huge amount of CPU + bandwidth.

> As I understand it, the mempool is organized already into "packages" of transactions, and adding a transaction into the mempool involves extending and merging packages.
> Perhaps the size of a package with low fee-rate (relative to the other packages in the mempool) can be limited, so that mempools drop incoming txes that extend a low-fee-rate tree of transactions.
> This means an attacker cannot send thousands of low fee-rate tx that build off some low fee-rate root tx in the first place, so it can still be evicted easily later without much impact.

There have been several proposals before around considering a transactions position in the mempool for various similar criteria. The extreme version being simply heavily rate-limiting transaction relay at low feerates and allowing much more liberal replacement of such packages. It isn’t quite perfect for this issue, though, as it may be easy for the attacker to just fill that rate-limit bucket.

> Naively, it seems to me to prevent the DoS attack as well, as at step 2 it would be prevented from sending thousands of low fee-rate tx building off the root.
>
> As well, as I understand it, this merely tightens the mempool acceptance rules, preventing low fee-rate packages from growing (analogous to a consensus-layer softfork).
> The "cannot evict high absolute fee" rule can be retained, as the low-fee-rate package is prevented from reaching a large size.
>
> Would that be workable as a general solution to solve (what I think is) the root cause of this problem?

It’s not clear to me that tightening the acceptance rules wouldn’t break other existing uses. Historically the 25 package size limit has proven to be an issue for users doing (somewhat naïve) centralized wallet withdraws. Sadly many users want that “payment pending” notification instantly, even if we know it to be somewhat lacking in security. Wallets which have over-compacted their UTXOs are thus stuck making long chains.

To revive an old discussion, on the original thread proposing the CPFP Carve-Out, I said this:

> As an alternative proposal, at various points there have been discussions around solving the "RBF-pinning" problem by allowing transactors to mark their transactions as "likely-to-be-RBF'ed", which could enable a relay policy where children of such transactions would be rejected unless the resulting package would be "near the top of the mempool". This would theoretically imply such attacks are not possible to pull off consistently, as any "transaction-delaying" channel participant will have to place the package containing A at an effective feerate which makes confirmation to occur soon with some likelihood. It is, however, possible to pull off this attack with low probability in case of feerate spikes right after broadcast.

To which Rusty responded (and I may be paraphrasing here): “Fuck Yea”. I’m still not much of a fan of this idea as it introduces too many constants (what is “the top of the mempool”, anyway?), and it’s unclear to me what you do as the mempool prevailing feerate changes, but it seems more along the lines or what you’re looking for here.

📅 Original date posted:2020-04-22 📝 Original message: On ...

2023-06-09T14:59:48+02:00

In reply to nevent1q…kv8d
_________________________

📅 Original date posted:2020-04-22
📝 Original message:
On 4/22/20 7:27 PM, Olaoluwa Osuntokun wrote:
>
>> Indeed, that is what I’m suggesting
>
> Gotcha, if this is indeed what you're suggesting (all HTLC spends are now
> 2-of-2 multi-sig), then I think the modifications to the state machine I
> sketched out in an earlier email are required. An exact construction which
> achieves the requirements of "you can't broadcast until you have a secret
> which I can obtain from the htlc sig for your commitment transaction, and my
> secret is revealed with another swap", appears to be an open problem, atm.

Hmm, indeed, it does seem to require a change to the state machine, but I don't think a very interesting one. Because B
providing A an HTLC signature spending a commitment transaction B will broadcast does not allow A to actually broadcast
said HTLC transaction, B can be rather liberal with it. Indeed, however, it would require that B provide such a
signature before A can send the commitment_signed that exists today.

> Even if they're restricted in this fashion (must be a 1-in-1 out,
> sighashall, fees are pre agreed upon), they can still spend that with a CPFP
> (while still unconfirmed in the mempool) and create another heavy tree,
> which puts us right back at the same bidding war scenario?

Right, you'd have to use anchor outputs just like we do on the commitment transaction :).

>> There are a bunch of ways of doing pinning - just opting into RBF isn’t
>> even close to enough.
>
> Mhmm, there're other ways of doing pinning. But with anchors as is defined
> in that spec PR, they're forced to spend with an RBF-replaceable
> transaction, which means the party wishing to time things out can enter into
> a bidding war. If the party trying to impeded things participates in this
> progressive absolute fee increase, it's likely that the war terminates
> with _one_ of them getting into the block, which seems to resolve
> everything?

No? Even if we assume there are no tricks that you can play with, eg, the package limits duri eviction, which I'd be
surprised about, the "absolute fee/feerate" thing still screws you. The attacker here gets to hold something at the
bottom of the mempool and the poor honest party is going to have to pay an absurd (likely more than the HTLC value) fee
just to get it unstuck, whereas the attacker never would have had to pay said fee.

> -- Laolung
>
>
> On Wed, Apr 22, 2020 at 4:20 PM Matt Corallo <lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>> wrote:
>
>
>
>> On Apr 22, 2020, at 16:13, Olaoluwa Osuntokun <laolu32 at gmail.com <mailto:laolu32 at gmail.com>> wrote:
>>
>> > Hmm, maybe the proposal wasn't clear. The idea isn't to add signatures to
>> > braodcasted transactions, but instead to CPFP a maybe-broadcasted
>> > transaction by sending a transaction which spends it and seeing if it is
>> > accepted
>>
>> Sorry I still don't follow. By "we clearly need to go the other direction -
>> all HTLC output spends need to be pre-signed.", you don't mean that the HTLC
>> spends of the non-broadcaster also need to be an off-chain 2-of-2 multi-sig
>> covenant? If the other party isn't restricted w.r.t _how_ they can spend the
>> output (non-rbf'd, ect), then I don't see how that addresses anything.
>
> Indeed, that is what I’m suggesting. Anchor output and all. One thing we could think about is only turning it on
> over a certain threshold, and having a separate “only-kinda-enforceable-on-chain-HTLC-in-flight” limit.
>
>> Also see my mail elsewhere in the thread that the other party is actually
>> forced to spend their HTLC output using an RBF-replaceable transaction. With
>> that, I think we're all good here? In the end both sides have the ability to
>> raise the fee rate of their spending transactions with the highest winning.
>> As long as one of them confirms within the CLTV-delta, then everyone is
>> made whole.
>
> It does seem like my cached recollection of RBF opt-in was incorrect but please re-read the intro email. There are a
> bunch of ways of doing pinning - just opting into RBF isn’t even close to enough.
>
>> [1]: https://github.com/bitcoin/bitcoin/pull/18191
>>
>>
>> On Wed, Apr 22, 2020 at 9:50 AM Matt Corallo <lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>> wrote:
>>
>> A few replies inline.
>>
>> On 4/22/20 12:13 AM, Olaoluwa Osuntokun wrote:
>> > Hi Matt,
>> >
>> >
>> >> While this is somewhat unintuitive, there are any number of good anti-DoS
>> >> reasons for this, eg:
>> >
>> > None of these really strikes me as "good" reasons for this limitation, which
>> > is at the root of this issue, and will also plague any more complex Bitcoin
>> > contracts which rely on nested trees of transaction to confirm (CTV, Duplex,
>> > channel factories, etc). Regarding the various (seemingly arbitrary) package
>> > limits it's likely the case that any issues w.r.t computational complexity
>> > that may arise when trying to calculate evictions can be ameliorated with
>> > better choice of internal data structures.
>> >
>> > In the end, the simplest heuristic (accept the higher fee rate package) side
>> > steps all these issues and is also the most economically rationale from a
>> > miner's perspective. Why would one prefer a higher absolute fee package
>> > (which could be very large) over another package with a higher total _fee
>> > rate_?
>>
>> This seems like a somewhat unnecessary drive-by insult of a project you don't contribute to, but feel free to
>> start with
>> a concrete suggestion here :).
>>
>> >> You'll note that B would be just fine if they had a way to safely monitor the
>> >> global mempool, and while this seems like a prudent mitigation for
>> >> lightning implementations to deploy today, it is itself a quagmire of
>> >> complexity
>> >
>> > Is it really all that complex? Assuming we're talking about just watching
>> > for a certain script template (the HTLC scipt) in the mempool to be able to
>> > pull a pre-image as soon as possible. Early versions of lnd used the mempool
>> > for commitment broadcast detection (which turned out to be a bad idea so we
>> > removed it), but at a glance I don't see why watching the mempool is so
>> > complex.
>>
>> Because watching your own mempool is not guaranteed to work, and during upgrade cycles that include changes to the
>> policy rules an attacker could exploit your upgraded/non-upgraded status to perform the same attack.
>>
>> >> Further, this is a really obnoxious assumption to hoist onto lightning
>> >> nodes - having an active full node with an in-sync mempool is a lot more
>> >> CPU, bandwidth, and complexity than most lightning users were expecting to
>> >> face.
>> >
>> > This would only be a requirement for Lightning nodes that seek to be a part
>> > of the public routing network with a desire to _forward_ HTLCs. This isn't
>> > doesn't affect laptops or mobile phones which likely mostly have private
>> > channels and don't participate in HTLC forwarding. I think it's pretty
>> > reasonable to expect a "proper" routing node on the network to be backed by
>> > a full-node. The bandwidth concern is valid, but we'd need concrete numbers
>> > that compare the bandwidth over head of mempool awareness (assuming the
>> > latest and greatest mempool syncing) compared with the overhead of the
>> > channel update gossip and gossip queries over head which LN nodes face today
>> > as is to see how much worse off they really would be.
>>
>> If mempool-watching were practical, maybe, though there are a number of folks who are talking about designing
>> partially-offline local lightning hubs which would be rendered impractical.
>>
>> > As detailed a bit below, if nodes watch the mempool, then this class of
>> > attack assuming the anchor output format as described in the open
>> > lightning-rfc PR is mitigated. At a glance, watching the mempool seems like
>> > a far less involved process compared to modifying the state machine as its
>> > defined today. By watching the mempool and implementing the changes in
>> > #lightning-rfc/688, then this issue can be mitigated _today_. lnd 0.10
>> > doesn't yet watch the mempool (but does include anchors [1]), but unless I'm
>> > missing something it should be pretty straight forward to add which mor or less
>> > resolves this issue all together.
>> >
>> >> not fixing this issue seems to render the whole exercise somewhat useless
>> >
>> > Depends on if one considers watching the mempool a fix. But even with that a
>> > base version of anchors still resolves a number of issues including:
>> > eliminating the commitment fee guessing game, allowing users to pay less on
>> > force close, being able to coalesce 2nd level HTLC transactions with the
>> > same CLTV expiry, and actually being able to reliably enforce multi-hop HTLC
>> > resolution.
>> >
>> >> Instead of making the HTLC output spending more free-form with
>> >> SIGHASH_ANYONECAN_PAY|SIGHASH_SINGLE, we clearly need to go the other
>> >> direction - all HTLC output spends need to be pre-signed.
>> >
>> > I'm not sure this is actually immediately workable (need to think about it
>> > more). To see why, remember that the commit_sig message includes HTLC
>> > signatures for the _remote_ party's commitment transaction, so they can
>> > spend the HTLCs if they broadcast their version of the commitment (force
>> > close). If we don't somehow also _gain_ signatures (our new HTLC signatures)
>> > allowing us to spend HTLCs on _their_ version of the commitment, then if
>> > they broadcast that commitment (without revoking), then we're unable to
>> > redeem any of those HTLCs at all, possibly losing money.
>>
>> Hmm, maybe the proposal wasn't clear. The idea isn't to add signatures to braodcasted transactions, but
>> instead to CPFP
>> a maybe-broadcasted transaction by sending a transaction which spends it and seeing if it is accepted. You
>> only need to
>> know the transaction's exact format (ie txid, which we do, since we sent a signature for it long ago) to do
>> this, you
>> don't have to actually *have* the fully-signed transaction (and you don't).
>>
>> > In an attempt to counteract this, we might say ok, the revoke message also
>> > now includes HTLC signatures for their new commitment allowing us to spend
>> > our HTLCs. This resolves things in a weaker security model, but doesn't
>> > address the issue generally, as after they receive the commit_sig, they can
>> > broadcast immediately, again leaving us without a way to redeem our HTLCs.
>> >
>> > I'd need to think about it more, but it seems that following this path would
>> > require an overhaul in the channel state machine to make presenting a new
>> > commitment actually take at least _two phases_ (at least a full round trip).
>> > The first phase would tender the commitment, but render them unable to
>> > broadcast it. The second phase would then <insert something something
>> > scriptless scripts here> enter a new sub-protocol which upon conclusion,
>> > gives the commitment proposer valid HTLC signatures, and gives the responder
>> > what they need to be able to broadcast their commitment and claim their
>> > HTCLs in an atomic manner.
>> >
>> > -- Laolu
>> >
>> > [1]: https://github.com/lightningnetwork/lnd/pull/3821
>>

📅 Original date posted:2020-04-22 📝 Original message: On ...

2023-06-09T14:59:38+02:00

In reply to nevent1q…d4m6
_________________________

📅 Original date posted:2020-04-22
📝 Original message:
On 4/22/20 12:12 AM, ZmnSCPxj wrote:
> Good morning Matt, and list,
>
>
>
>> RBF Pinning HTLC Transactions (aka "Oh, wait, I can steal funds, how, now?")
>> =============================
>>
>> You'll note that in the discussion of RBF pinning we were pretty broad, and that that discussion seems to in fact cover
>> our HTLC outputs, at least when spent via (3) or (4). It does, and in fact this is a pretty severe issue in today's
>> lightning protocol [2]. A lightning counterparty (C, who received the HTLC from B, who received it from A) today could,
>> if B broadcasts the commitment transaction, spend an HTLC using the preimage with a low-fee, RBF-disabled transaction.
>> After a few blocks, A could claim the HTLC from B via the timeout mechanism, and then after a few days, C could get the
>> HTLC-claiming transaction mined via some out-of-band agreement with a small miner. This leaves B short the HTLC value.
>
> My (cached) understanding is that, since RBF is signalled using `nSequence`, any `OP_CHECKSEQUENCEVERIFY` also automatically imposes the requirement "must be RBF-enabled", including `<0> OP_CHECKSEQUENCEVERIFY`.
> Adding that clause (2 bytes in witness if my math is correct) to the hashlock branch may be sufficient to prevent C from making an RBF-disabled transaction.

Hmm, indeed, though note that (IIRC) you can break this by adding children or parents which are *not* RBF-enabled and
then the package may lose the ability to be RBF'd.

> But then you mention out-of-band agreements with miners, which basically means the transaction might not be in the mempool at all, in which case the vulnerability is not really about RBF or relay, but sheer economics.

No. The whole point of this attack is that you keep a transaction in the mempool but unconfirmed via RBF pinning, which
prevents an *alternative* transaction from being confirmed. You then have plenty of time to go get it confirmed later.

> The payment is A->B->C, and the HTLC A->B must have a larger timeout (L + 1) than the HTLC B->C (L), in abstract non-block units.
> The vulnerability you are describing means that the current time must now be L + 1 or greater ("A could claim the HTLC from B via the timeout mechanism", meaning the A->B HTLC has timed out already).
>
> If so, then the B->C transaction has already timed out in the past and can be claimed in two ways, either via B timeout branch or C hashlock branch.
> This sets up a game where B and C bid to miners to get their version of reality committed onchain.
> (We can neglect out-of-band agreements here; miners have the incentive to publicly leak such agreements so that other potential bidders can offer even higher fees for their versions of that transaction.)

Right, I think I didn't explain clearly enough. The point is that, here, B tries to broadcast the timeout transaction
but cannot because there is an in-mempool conflict.

> Before L+1, C has no incentive to bid, since placing any bid at all will leak the preimage, which B can then turn around and use to spend from A, and A and C cannot steal from B.
>
> Thus, B should ensure that *before* L+1, the HTLC-Timeout has been committed onchain, which outright prevents this bidding war from even starting.
>
> The issue then is that B is using a pre-signed HTLC-timeout, which is needed since it is its commitment tx that was broadcast.
> This prevents B from RBF-ing the HTLC-Timeout transaction.
>
> So what is needed is to allow B to add fees to HTLC-Timeout:
>
> * We can add an RBF carve-out output to HTLC-Timeout, at the cost of more blockspace.
> * With `SIGHASH_NOINPUT` we can make the C-side signature `SIGHASH_NOINPUT|SIGHASH_SINGLE` and allow B to re-sign the B-side signature for a higher-fee version of HTLC-Timeout (assuming my cached understanding of `SIGHASH_NOINPUT` still holds).

This does not solve the issue because you can add as many fees as you want, as long as the transaction is RBF-pinned,
there is not much you can do in an automated fashion.

> With this, B can exponentially increase the fee as L+1 approaches.
> If B can get HTLC-Timeout confirmed before L+1, then C cannot steal the HTLC value at all, since the UTXO it could steal from has already been spent.
>
> In particular, it does not seem to me that it is necessary to change the hashlock-branch transaction of C at all, since this mechanism is enough to sidestep the issue (as I understand it).
> But it does point to a need to make HTLC-Timeout (and possibly symmetrically, HTLC-Success) also fee-bumpable.
>
> Note as well that this does not require a mempool: B can run in `blocksonly` mode and as each block comes in from L to L+1, if HTLC-Timeout is not confirmed, feebump HTLC-Timeout.
> In particular, HTLC-Timeout comes into play only if B broadcast its own commitment transaction, and B *should* be aware that it did so --- there is still no need for mempool monitoring here.
>
>
> Now, of course this only delays the war.
> Let us now consider what C can do to ensure that the bidding war will happen eventually.
>
> * C can bribe a miner to prevent HTLC-Timeout from confirming between L and L+1.
> * Or in other words, this is a censorship attack.
> * The Bitcoin censorship-resistance model is that censored transactions can be fee-bumped, which attracts non-censoring miners to try their luck at mining and evict the censoring miner.
> * Thus, letting B bump the fee on HTLC-Timeout is precisely the mechanism we need.
> * This sets up a bidding war between C requesting miners to censor, vs. B requesting miners to confirm, but that only sets the stage for a second bidding war later between C and B, thus C is at a disadvantage: it has to bribe miners to censor continuously from L to L+1 *and* additional bribe miners to confirm its transaction after L+1, whereas B can offer its bribe as being something that miners can claim now without waiting after L+1.
>
>
>
> The issue of course is the additional output that bloats the UTXO set and requires another transaction to claim later.
> And if we have `SIGHASH_NOINPUT`, it seems to me that Decker-Russell-Osuntokun sidesteps this issue as well, as any timed-out HTLC can be claimed with a fee-bumpable transaction directly without RBF-carve-out.
> (As well, it seems to me that, if both nodes support doing so, a Poon-Dryja channel can be upgraded, without onchain activity, to a Decker-Russell-Osuntokun channel: sign a transaction spending the funding tx to a txo that has been set up as Decker-Russell-Osuntokun, do not broadcast that transaction, then revoke the latest Poon-Dryja commitment transactions, then switch the mechanism over to Decker-Russell-Osuntokun; you still need to monitor for previous Poon-Dryja commitment transactions, but HTLCs now sidestep the issue under discussion here.)
>
> Regards,
> ZmnSCPxj
>

📅 Original date posted:2020-01-20 📝 Original message: On ...

2023-06-09T14:58:06+02:00

In reply to nevent1q…vc9s
_________________________

📅 Original date posted:2020-01-20
📝 Original message:
On 11/9/19 4:31 AM, Takaya Imai wrote:
> [What I do not describe]
> * A way to detect that data is correct or not, namely zero knowledge
> proof process.

Have you come across Zero Knowledge Contingent Payments? Originally it
was designed for on-chain applications but it slots neatly into
lightning as it only requires a method to lock funds to a hash preimage.

Matt

📅 Original date posted:2019-12-15 📝 Original message: Given ...

2023-06-09T14:57:42+02:00

In reply to nevent1q…4hjh
_________________________

📅 Original date posted:2019-12-15
📝 Original message:
Given the timeline for soft forks to activate on Bitcoin, I don't know
why we'd be super conservative about using new features of the Bitcoin
consensus rules. I think obviously we'd want to rush as fast as we can
into adding real cross-hop privacy to lightning payments, given both the
number of awkward edge cases it introduces into the protocol, and the
massive privacy leak.

Changing the underlying commitment transaction structure *could* come
later, as it is, indeed, lower priority (and then the rule would be
"taproot outputs are always lightning" for some time anyway), but, if
you're changing things kinda why not.

Of course once you get privacy across routing hops, the importance of
privacy-preserving routing algorithm work and fixing some other privacy
holes is probably more important, so I could be dissuaded.

Matt

On 12/15/19 3:43 PM, ZmnSCPxj via Lightning-dev wrote:
> Good morning list,
>
> I would like to initiate some discussion on how Lightning could be updated to take advantage of the upcoming taproot update to the base layer.
>
> For now, I am assuming the continued use of the existing Poon-Dryja update mechanism.
> Decker-Russell-Osuntokun requires `SIGHASH_NOINPUT`/`SIGHASH_ANYPREVOUT`, and its details seem less settled for now than taproot details.
>
> * [ ] We could update the current Poon-Dryja mechanism to use Schnorr signatures.
> * [ ] Or we could just keep the current Poon-Dryja mechanism with SegWit v0 only, and only update to Schnorr / SegWit v1 when we can implement Decker-Russell-Osuntokun.
> * This brings up the question as to whether we will allow Poon-Dryja to host pointlocked-timelocked contracts (i.e. the Scriptless Script replacement of HTLCs that uses payment points+scalars).
> * [ ] We could constrain Poon-Dryja channels to only HTLCs.
> * This might be simpler for implementations: implementations could have a completely new module that implements Decker-Russell-Osuntokun with HTLCs and PTLCs, and not touch existing modules for Poon-Dryja with HTLCs only.
> * We could "retire" Poon-Dryja constructions at some point and only add new features to Decker-Russell-Osuntokun channels.
> * [ ] We could allow hosting PTLCs as well on Poon-Dryja channels, as nothing in the base layer prevents a transaction from providing both SegWit v0 and SegWit v1 outputs anyway.
>
> Poon-Dryja with Schnorr
> -----------------------
>
> If we decide to update the current Poon-Dryja mechanism to use Schnorr, there are some wrinkles:
>
> * [ ] We could just drop-in replace the signing algorithm with Schnorr.
> * We would define a NUMS point to be used as Taproot internal key, and use a single tapscript that is simply the current script template.
> * This is arguably not really taking advantage of the Schnorr and Taproot features, however.
> * [ ] Or we could use some sort of MuSig between the two channel participants.
>
> The latter option is probably what we want to use, as it potentially allows a channel close to be indistinguishable from an ordinary SegWit v1 spend of a UTXO.
> Even for channels that have been published via gossip, it moves the onus of storing historical data about historically-published channels from base layer fullnodes to nodes that want to surveill the network.
>
> ### Digression: 2-of-2 is Obviously Lightning
>
> Existing 2-of-2 outputs have a very high probability of being Lightning Network channels.
> Thus, someone who wishes to surveill the Lightning Network can simply query any archive fullnode for historical 2-of-2 outputs and have a good chance that those are Lightning Network channels.
>
> Consider the adage: Never go to sea with two chronometers; take one or three.
> https://en.wikipedia.org/wiki/Dual_modular_redundancy
> This implies that ordinary HODL usage of transaction outputs will either use 1-of-1, or 2-of-3.
>
> Offchain techniques, on the other hand, are only safe (trustless) if they are n-of-n, and are only usefully multi-participant if n > 1.
> https://zmnscpxj.github.io/offchain/safety.html
> Thus any n-of-n is likely to be an offchain protocol, with the most likely being the most widespread offchain protocol, Lightning Network.
>
> Thus, the hyperbole "2-of-2 is Obviously Lightning".
>
> However, we can "hide" 2-of-2 in a 2-of-3, which can be done by generating a third point from a NUMS point plus a random tweak generated by both participants.
> Better yet, MuSig allows hiding any n-of-n among 1-of-1; we expect 1-of-1 use to dominate in the foreseeable future, thus MuSig usage maximizes our anonymity set.
>
> ### End Digression
>
> A potential issue with MuSig is the increased number of communication rounds needed to generate signatures.
>
> In the current Poon-Dryja setup we have, in order to completely sign the commitment transaction held by one participant, we only require sending a completed signature from the remote participant via `commitment_signed`.
> Then the local participant issues a `revoke_and_ack` to reassure the remote side that the previous commitment it held is now revoked.
> Thus, the signing itself requires only 0.5 round trips.
>
> MuSig requires three rounds to complete the signing protocol.
> Since Poon-Dryja only require the complete set of signatures for any particular commitment transaction to be owned by only one participant (i.e. Poon-Dryja commitment transactions are asymmetrical), MuSig only requires 2.5 round trips to sign a Poon-Dryja commitment.
>
> Reducing the round trips is desirable when considering latency, which affects the experienced speed of forwarding, as well as the backwards propagation of failures (but not of successful payments).
> Currently, latency on the network is known to be very low, but I observe that most Lightning Network participants today have excellent Internet connectivity, which might not be true in the foreseeable future.
>
> We can reduce the round trips to 1.5 per commitment, except for the initial commitment transaction, by sending the `R` precommitments (the first phase of MuSig) of the *next* signing session while sending the `s` for the *current* signing session.
>
> Finally, we can consider to reduce the use of MuSig via various techniques.
> One technique is the use of fast forwards: https://lists.linuxfoundation.org/pipermail/lightning-dev/2019-April/001986.html
>
> The other involves using Taproot.
> The taproot internal pubkey is the MuSig of both participants, but we also add an explicit 2-of-2 as a tapscript (or hide the 2-of-2 in a 2-of-3), i.e. `<A> OP_CHECKSIGVERIFY <B> OP_CHECKSIG` in a tapscript.
> This allows commitment transactions to use the tapscript path and have separate signatures for both participants.
> Then, mutual close uses the MuSig keypath spend.
> This reduces the changes relative to the current Poon-Dryja implementation: we can continue to use `commitment_signed` messages (or equivalent new ones for Taproot-based Poon-Dryja channels), and only the mutual close ritual needs to be modified to use MuSig.
>
> Thus the options are below, but ***please do not select an option yet***, because further sections greatly affect the tradeoffs of the options below.
>
> * [ ] Latency is not an issue (even in the foreseeable future), just use the 2.5 round trips for MuSig and MuSig-sign every commitment.
> * [ ] Pre-send the `R` commitments with the previous `s`.
> * This increases the needed storage requirements of both nodes, and possibly complexity of node software, but reduces the round trips to 1.5.
> * This usage may need to have larger bounds than what may be naively expected, if we decide to use Multi-`R` composable MuSig (see later section).
> * Even *without* this option, at the minimum we still need to store `R` commitments and `R`, at least in memory, until we complete the MuSig ritual.
> * [ ] Use fast-forwards.
> * This removes the latency of commitment transaction signing from the forwarding and failure-reporting paths.
> * *Improved* forwarding and failure-reporting latency compared to today (revocation and signature not needed for forwarding/failure-reporting).
> * Latency is still high for commitment transaction signing, but this is less of an issue as it is no longer in the forwarding/failure-reporting path.
> * [ ] Explicit 2-of-2 for commitment, MuSig for mutual close.
> * No degradation in latency compared to today (0.5 round trips for signature, additional 1 round trip for signalling and revocation).
> * No improvement (or degradation) of privacy for unilateral closes compared to today, but improved privacy for mutual closes.
> * An argument *for* this option is that unilateral closes (commitment transactions) have special scripts on their outputs anyway, thus still leak their privacy, but see later sections for how Taproot may help reduce the uniqueness of those outputs, thus *not* taking this option may still improve privacy for unilateral closes.
>
> Composable MuSig
> ----------------
>
> As I describe here: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-November/017493.html
>
> In that write-up, I mention the use-case of nodelets.
> Another use-case is for redundant hardware signers: https://lists.ozlabs.org/pipermail/c-lightning/2019-December/000172.html
>
> We have the option of allowing for a composable MuSig, probably my multi-`R` proposal, where each participant may send one or more `R` commitments and subsequent `R`s.
>
> * [ ] Not use composable MuSig of any kind, at least for now.
> * Possibly we could defer the use of a composable MuSig when a better composable MuSig (with security proofs) has been derived.
> * [ ] Use Multi-`R` composable MuSig.
> * We need to probably impose some limit on the number of `R` commitments that can be sent: _____ `R`s maximum.
>
> The latter option above affects the previous choice regarding whether to pre-send `R` commitments with the previous `s` sending, as this increases the maximal storage requirement for the pre-sent `R` commitment(s).
>
> Pointlocked Timelocked Contracts
> --------------------------------
>
> I would have preferred to keep this in a separate discussion thread, however the exact details of PTLCs may affect our decision on how to sign commitment transactions.
>
> PTLCs are necessary to switch the Lightning Network to payment point+scalar, and indeed one can argue that the entire point of enabling Schnorr and Taproot at the base layer is to allow us to use payment point+scalar at the Lightning layer.
>
> First, I will discuss how to create a certain kind of PTLCs, which I call "purely scriptless" PTLCs.
> In particular, I would like to point out that we *actually* use in current Poon-Dryja Lightning Network channels is *revocable* HTLCs, thus we need to have *revocable* PTLCs to replace them.
>
> * First, we must have a sender A, who is buying a secret scalar, and knows the point equivalent to that scalar.
> * Second, we have a receiver B, who knows this secret scalar (or can somehow learn this secret scalar).
> * A and B agree on the specifications of the PTLC: the point, the future absolute timelock, the value.
> * A creates (but *does not* sign or broadcast) a transaction that pays to a MuSig of A and B and shares the txid and output number with the relevant MuSig output.
> * A and B create a backout transaction.
> * This backout has an `nLockTime` equal to the agreed absolute timelock.
> * It spends the above MuSig output (this input must enable `nLockTime`, e.g. by setting `nSequence` to `0xFFFFFFFE`).
> * It creates an output that is solely controlled by A.
> * A and B perform a MuSig ritual to sign the backout transaction.
> * A now signs and broadcast the first transaction, the one that has an output that represents the PTLC.
> * A and B wait for the above transaction to confirm deeply.
> This completes the setup phase for the PTLC.
> * After this point, if the agreed-upon locktime is reached, A broadcasts the backout transaction and aborts the ritual.
> * A and B create a claim transaction.
> * This has an `nLockTime` of 0, or a present or past blockheight, or disabled `nLockTime`.
> * This spends the above MuSig output.
> * This creates an output that is solely controlled by B.
> * A and B generate an adaptor signature for the claim transaction, which reveals the agreed scalar.
> * This is almost entirely a MuSig ritual, except at `s` exchange, B provides `t + r + h(R | MuSig(A,B) | m) * MuSigTweak(A, B, B) * b` first, then demands `r + h(R | MuSig(A,B) | m) * MuSigTweak(A, B, A) * a` from A, then reveals `r + h(R | MuSig(A,B) | m) * MuSigTweak(A, B, B) * b` (or the completed signature, by publishing onchain), revealing the secret scalar `t` to A.
> * A is able to learn the secret scalar from the above adaptor signature followed by the full signature, completing the ritual.
>
> Note that the above ritual is "purely scriptless".
> It is possible to take advantage of taproot to create a tapscript equivalent to `<agreed locktime> OP_CHECKLOCKTIMEVERIFY OP_DROP <A> OP_CHECKSIG`.
> This removes the need to perform a MuSig ritual to sign a backout transaction: A can generate the backout transaction all by itself.
> This use is not purely scriptless, as the timelocked branch is controlled by a tapscript that is hidden in the pointlocked branch, meaning that taking the timelocked branch reveals the contract.
>
> The tradeoff is that if we use "purely scriptless" PTLCs:
>
> * If we have purely scriptless PTLCs, then even the timelock branch is potentially indistinguishable from ordinary spends.
> * We should note that Bitcoin Core itself already uses the current blockheight plus 1 for `nLockTime` when spending, instead of a 0 or disabled `nLockTime`.
> * If the above habit were more widespread (i.e. if all wallets implemented the use of a current `nLockTime` for ordinary onchain spends) then the timelock branch has an anonymity set equal to all Schnorr keypath spends, as the `nLockTime` is always used anyway.
> * Other kinds of offchain protocols may also utilize this technique, slightly increasing the anonymity set as well.
> Thus, the use of the current blockheight plus 1 for `nLockTime` should be recommended for improved privacy in Bitcoin anyway.
> * However, purely scriptless PTLCs require a complete MuSig ritual (2.5 to 3 round trips!) to sign the timelocked branch (the initial backout transaction); this increases the latency of adding new PTLCs.
> * Of note is that in the case of Lightning, we may need to add a new PTLC while another PTLC is *still* in-flight; on the new commitment transaction, we need to redo the MuSig rituals to create new backouts and new adaptor signtures for each PTLC.
> * This can be amortized such that we redo all the rituals for all existing PTLCs in parallel.
> * The use of multi-`R` composable MuSig greatly increases the storage requirements as well!
> * Further, the privacy gained in the timelock path is degraded or lost when we consider revocable PTLCs and revocable outputs.
> * It is not clear to me as fo now if the MuSig ritual for the backout transaction must complete in full before the MuSig ritual for the claim transaction (the pointlocked branch, which includes sharing an adaptor signature during `s` exchange).
> * If we can do these two rituals in parallel, this reduces the latency disadvantage of the purely scriptless PTLC to just 1 round trip compared to the tapscripted timelock branch PTLC.
> * Possibly the adaptor signature ritual could stop until B provides the adaptor signature, then full `s` exchange is deferred to after the PTLC 2-of-2 is instantiated onchain/reified in a commitment transaction.
>
> Thus our options are the below, but again, you should defer deciding on the best option until you understand about revocable PTLCs.
>
> * [ ] We can use a purely scriptless PTLC construction.
> * Better privacy, but increased latency.
> * [ ] We can use a tapscripted timelock branch and a keypath pointlocked branch for the PTLC.
> * Lower latency, but possibly reduced privacy advantage in the timelock branch (privacy advantage still exists in the pointlocked branch, which we hope is more common anyway).
>
> Revocable Outputs and PTLCs
> ---------------------------
>
> Poon-Dryja mechanism is based on revocation of contracts instantiated from previous states.
> Thus, all outputs of Poon-Dryja must be made revocable.
>
> At a high-enough level, this only requires the addition of a `(revocation_secret && remote_secret) || (CSV && L)` to the top-level logic (`L`) of a contract.
> Of course, the details have the devil.
>
> Now, in Poon-Dryja, each side of the channel has its own commitment transaction.
> Thus, contracts on the commitment transaction refer to a "local side" and a "remote side".
> The "local side" is whoever is holding the completely-signed commitment transaction, while the "remote side" is the other side.
>
> Revocable outputs always have a CSV-requirement before the output can be spent by whoever "should" own the output, according to the contract details.
> The revocation secret is known by the local side, and then when a *new* commitment transaction has been signed, the revocation secret is given by the local side to the remote side, allowing all outputs to be spent directly by the remote side if an old, revoked commitment transaction is published onchain.
>
> There are many ways to implement revocable outputs.
> The core idea of revocable outputs, however, is that there must exist some relative-locktime branch on each output that goes to the local side.
> This relative-locktime branch can be implemented as "purely scriptless" by using a 2-of-2 of both the local and remote sides, with a relative-locktime `nSequence`, using the same technique as described in the previous section.
>
> However, unlike the absolute-locktime `nLockTime` for the timelock branches of PTLCs, ***there is no current wallet software that uses activated-relative-locktime `nSequence` for ordinary spends***.
> Thus, any use of the relative-locktime branch does *not* have a privacy advantage under purely scriptless.
>
> Thus the tradeoffs here are different, and definitely in favor of the tapscripted-relative-locktime technique.
>
> * There is no current wallet software that uses relative locktimes.
> * Bitcoin Core uses absolute locktimes with the current blockheight plus 1, and we can hide the use of absolute locktimes by imitating this Bitcoin Core behavior.
> * Any use of relative locktimes is a strong signal of some kind of complicated offchain updateable mechanism.
> We can note as well that Decker-Russell-Osuntokun also uses relative locktimes.
> * The relative-locktime branch in revocable outputs is the **desirable** case, because taking the relative locktime means that the commitment transaction published *was not* a revoked transaction (i.e. it was the latest transaction version).
> * Thus, it would be nice if this had good privacy.
> * However, this *cannot* have good privacy since there is no current common use of activated relative locktimes for ordinary spends.
>
> The latter point above also biases us towards rejecting purely-scriptless in the *previous* section, in particular:
>
> * The absolute-locktime branch in the PTLC is the **undesirable** case, where the payee B somehow stops responding to A.
> * In particular, propagating failure backwards is done by simply deleting PTLCs, thus the absolute-locktime branch does not come into play during payment failure.
> * Thus, the privacy increase of hiding absolute-locktime branches is minor, and the privacy breach of revocable outputs (i.e. it leaks the use of relative-locktime, and nearly all uses of relative locktime will be offchain updateable cryptocurrency mechanisms, such as Poon-Dryja or Decker-Russell-Osuntokun) is arguably much larger than the privacy increase of purely-scriptless absolute-locktime.
>
> Of particular not is that if a PTLC is instantiated on the commitment transaction of the payer A, then the absolute-locktime backout transaction must *not* pay to A only, but instead pays to a *revocable output* that will eventually pay to A after the relative locktime.
> Thus, the privacy boost of the purely-scriptless backout transaction would be lost by the later revelation of the use of a relative locktime after the revocable output is reclaimed by A.
> Again, that use of relative locktime (whether enforced by script, or by scriptless use of a 2-of-2 that is spent with an activated relative-locktime `nSequence`) is a strong indicator of the use of Poon-Dryja or Decker-Russell-Osuntokun or even Decker-Wattenhofer mechanisms, and the logic is that since Lightning Network is the most widespread offchain mechanism, it probably indicates the use of LN.
>
> Thus the decision in the previous section is affected by this reality that we in fact need *revocable* PTLCs, not plain PTLCs, thus the advantage of purely-scriptless backout transactions in the previous section is reduced by the need to make outputs revocable.
>
> Thus:
>
> * Revocable outputs require a relative-timelock branch.
> * That branch is the **desirable** branch, i.e. the latest commitment transaction is the one that got published and thus is *not* revoked.
> * We thus expect this relative-timelock branch to be the *common* branch.
> * No wallet uses relative-timelock for ordinary spends, unlike abolute-locktime where Bitcoin Core always uses absolute-locktime `nLockTime` for ordinary spends.
> * Thus, using purely scriptless for the relative-timelock branch of revocable outputs does not in fact hide that we are doing something special.
>
> Conclusion
> ----------
>
> Others may find further issues or alternative options for the above points.
> Further discussion may be warranted before we actually select particular options to implement and evaluate in alpha software.
>
> Regards,
> ZmnSCPxj
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

📅 Original date posted:2019-12-10 📝 Original message: Nice ...

2023-06-09T14:57:37+02:00

In reply to nevent1q…agqe
_________________________

📅 Original date posted:2019-12-10
📝 Original message:
Nice writeup!

In further in-person discussions today, it was noted that the key
last-resort fallback Bitcoin Core has to week out peers in case of an
Eclipse Attack does not protect from this style of attack. While it is
only of limited concern for most Bitcoin Core users, it very much may
expose lightning nodes to theft.

More importantly, while Bitcoin Core has some protections against
generic sybil-based Eclipse attacks, they are far from sufficient in a
model like Lightning where much more is at stake from simple no-hashrate
Eclipse attacks.

The proposed countermeasure here of "raising alarms" in case the
best-block nTime field is too far behind is compelling in a
SPV-assumption world, though it is far from sufficient if the time delay
is small (eg for lightning HTLC delays or if the to_self_delay is
gratuitously insecure, eg under 144).

Sadly, however, "raising alarms" is not sufficient to protect users, as
countermeasures likely need to be taken automatically for any kind of
compelling security.

I'd encourage Lightning node authors and operators to ensure they have
multiple, redundant, trusted methods of receiving Bitcoin block data in
a timely fashion. To shill my own bags, of late I've been thinking about
such systems and expose services to fetch header/BIP157 filter data over
DNS [1, 2], header data over arbitrary radio interfaces [3] and full
block data over HTTP. Finally, there is also a parallel P2P
implementation which attempts to make different tradeoffs from the
existing Bitcoin Core P2P implementation and is much more aggressive
about seeking additional connections if there is evidence of headers
which are better than our current chain at [5], relying on some of the
other mechanisms to seek header data.

Note that the Bitcoin Core-based work here also includes what I hope can
be used as an arbitrary easy-to-write-additional-block-fetch-methods
framework.

[1] https://github.com/bitcoin/bitcoin/pull/16834
[2] https://twitter.com/TheBlueMatt/status/1200585905163112449
[3]
https://github.com/TheBlueMatt/bitcoin/commit/2019-10-rusty-all-da-stuff%5E1
you should be able to pick up headers in the 915 MHz band over LoRa with
this patch in the New York City area already :)
[4] https://github.com/bitcoin/bitcoin/pull/16762
[5] https://github.com/bitcoin/bitcoin/pull/17376

Matt

On 12/9/19 6:04 PM, Antoine Riard wrote:
> Time-Dilation Attacks on Offchain Protocols
> ===================================
>
> Lightning works on reversing the double-spend problem to a private
> state between parties instead of being a public issue verified by every
> network peer. The security model is based on revocation of previous
> states and in case of broadcast of any of them, being aware of it to
> generate justice transactions to claim misbehaving peer onchain outputs
> before contest period expiration. This period is driven by the blockchain
> which is here the system clock.
>
> Eclipse attacks's end-goal is to monopolize a victim's incoming and
> outgoing connections, by this way isolating a node from the rest of its
> peers in the network. A successful Eclipse attacks lets the attacker
> filter the victim's view of the blockchain, i.e he controls transactions
> and blocks announcements [0].
>
> Every LN node must be tied to a bitcoin full-node or light-client to
> verify independently channels opening/closing, HTLCs expiration and
> previous/latest state broadcast. To operate securely, the view of the
> blockchain must be up-to-date with the one shared with the rest of the
> network. By considering Eclipse attacks on the base layer, this assumption
> can be broken.
>
> First scenario : Targeting the CSV security delay
> --------------------------------------------------------------
>
> Alice and Mallory are LN peers with a channel opened at state N. They
> use a CSV of 144 blocks as a security parameter for contestation period.
> Mallory is able to identify Alice full-node and start to eclipse it.
> When done, it keeps announcing blocks to Alice node but delaying them by
> 2min. Given a variance of 10min, after 6 blocks, Mallory will have a
> height ahead of Alice of 1, after 24 blocks, a lead of 4, after 144,
> a lead of 24, after 1008, a lead of 168.
>
> After maintaining the eclipse for a week, Mallory will have more than a
> day of height advance on Alice view of blockchain. The difference being
> superior at the CSV timelock, Mallory can broadcast a previous
> commitment transaction at state N - 10 with a balance far more favorable
> than the legit one at height H, the synchronized height with the rest
> of the network.
>
> At revoked commitment transaction broadcast, Alice is slow down at
> height H - 168. At H+144, Mallory can unlock its funds out of the
> commitment transaction outputs and by so closing the contestation period
> while Alice is still stuck at H-24. When Alice learn about revoked
> broadcast at H, it's already too late. Mallory may have stopped the
> Eclipse attack after H+144, or he may pursue the attack on Alice because
> of targeting multiple of her channels in parallel.
>
> Second scenario : Targeting the per-hop CLTV-delta
> -------------------------------------------------------------------
>
> Alice, Bob and Caroll are LN peers with channel Alice-Bob and Bob-Caroll.
> Bob enforce a cltv_delta of 12 blocks on incoming HTLCs. Alice and Caroll
> are both malicious and start to eclipse Alice full-node until they gain a
> lead of 15 blocks on Alice.
>
> At height N, Alice route a payment E to Caroll through Bob with a base
> delta of 24 blocks. On channel AB, HTLC will expire at height H+24, on
> channel BC, it will expire at height H+12 while Alice ticking herself
> at height H-15.
>
> When real-network blockchain height reaches H+24, Caroll provides
> preimage P to Bob and following protocol rules she gets a new
> commitment transaction with balance increased of HTLC E. At same time,
> Alice broadcast unilaterally commitment transaction for AB and claim
> back HTLC E with a HTLC-timeout transaction as its nLockTime is already
> final. Bob wrongly clocking at height H+9, HTLC on channel BC won't have
> been timeout by him and provided preimage is now useless as HTLC as
> already claimed back on mainnet blockchain.
>
> Attack difficulty
> -------------------
>
> Following Eclipse attack paper publication, multiple counter-measures
> have been implemented [1], [2]. Even if it's far harder, this kind of
> attacks may still be possible for an off-path attacker. Including recent
> changes, research should be done to ascertain it. Beyond, it still
> widely feasible for attackers controlling key infrastructure between
> the victim and the network [3]. A simple Tier 3 ISP doing deep packet
> inspection and delaying your blocks may be able to practice this class
> of attacks.
>
> The case of LN nodes using light clients, like BIP157 or Electrum, as a
> backend should be specially considered. Given the low number of servers
> providing these services today it should be trivial for an attacker to run
> swarm of them and from then control blockchain view of this class of LN
> nodes.
>
> Another difficulty of the attack, is to map the LN node to its full-node.
> This can range from being trivial if both run in clear with same ipv4
> address. It would be interesting to scan both networks and see how many
> nodes pair are in this case. Learning the mapping can still be doable if
> they are using an anonymous network like Tor with same exit node base on
> message timing.
>
> Further inter-layer mapping techniques could be developped like funding
> transaction broadcast and seeing its propagation on the base layer. Or
> knowledge of onchain utxo graph leaked by addressed reused to fund
> channels.
>
> On the LN-side, depending of the scenario, cost of the attack ranges
> between opening one or two channels and paying the related onchain fees.
> That being said, an attacker can also target victim-initiated channels
> and costs would be reduced only to operating LN nodes.
>
> Onchain counter-measures
> -----------------------------------
>
> Warnings could be triggered by full-node when the block issuance rate
> becomes weird but first it's hard to dissociate weirdness from edges of
> block variance and secondly the algo being public it should be easy for
> an attacker to stay under the radar by choosing the right announcement
> delay.
>
> Raising alarms based on nTime of block headers being to far in the past
> compare to local clock wouldn't make sense due to IBD.
>
> A practical solution would be to have multiple bitcoin full-nodes
> with multi-homing and doing reconciliation of blockchain views every
> N hour/minute. This kind of method, already deployed by some bitcoin
> infrastructures, would raise attack difficulty for off-path attackers
> but wouldn't counter in-path attackers. Nevertheless, it would already
> be a good step, the issue is the current absent of implementing this
> in open-source software for hobbyists.
>
> A defense in depth would require to have multiple data links, like
> receiving headers from space/radio or tunneling through
> some other protocols [4].
>
> Even learning discrepancies between your local view of the blockchain
> and the mainnet one, the right move to do isn't clear as an automatic
> system may be triggered by false positives to halt operation of your LN
> node, a human intervention to double-check may seem preferable. And
> it's really likely the attacker control you ability of transactions
> on the base layer. BIP324 may help there.
>
> Offchain counter-measures
> -----------------------------------
>
> Some LN errors messages may be triggered at abnormal rate like
> `expiry_too_soon` due to victim using a HTLC base in the past and may
> be used to guess oddities.
>
> A measure orthogonal to running multiple bitcoin full-nodes should be
> to have multiples watchtowers running their own blocks provider (full-node
> or light clients). Assuming attacker don't control your connection to them,
> if at least one of them isn't eclipsed, it should be able to time-out HTLC/
> justice revoked transaction efficiently.
>
> Another option would be to use the control plane as a safety mechanism.
> When a LN node learns a channel_announcement to far in the future it
> would ask to this LN node, few next headers through this communication
> channel and connect these headers to its full-node. Based on this
> information, full-node may trigger alarms more accurately. This mechanism
> would work assuming one of your gossiping LN node is honest. It may ask
> for fine-tuning to avoid there too false-positives but it seems an
> interesting topic of research to use L2 information to correct L1 blockchain
> view and even further to use L2 communications channels as an emergency
> transaction broadcast.
>
>
> Further research could be lead to investigate attacks combining
> eclipsing victim
> view of both layer, that's said the LN one may seem harder to take on as
> nodes
> have persistent identities.
>
> Eclipse attacks were already known to be very bad, offchain protocols
> relying on time for their security model make them worst at it lower the bar
> to exploit them from being a miner to being party to an offchain contract.
>
>
> After talking with LN implementations teams, they don't think funds are
> at risks,
> given than light clients have low-value channels right now and it's not
> worthy the
> attack setup, but they acknowledge the issue on the long term for
> processing nodes.
>
>
> Thanks to Gleb Naumenko for fruitful discussions/review.
>
> Cheers
>
> [0] https://eprint.iacr.org/2015/263.pdf
>
> [1] https://github.com/bitcoin/bitcoin/pull/9037
>
> [2] https://github.com/bitcoin/bitcoin/pull/8282
>
> [3] https://erebus-attack.comp.nus.edu.sg/
>
> [4] https://github.com/bitcoin/bitcoin/pull/16834
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

📅 Original date posted:2019-10-24 📝 Original message: I may ...

2023-06-09T14:56:46+02:00

In reply to nevent1q…z8w8
_________________________

📅 Original date posted:2019-10-24
📝 Original message:
I may be missing something, but I'm not sure how this changes anything?

If you have a commitment transaction, you always need at least, and
exactly, one non-CSV output per party. The fact that there is a size
limitation on the transaction that spends for carve-out purposes only
effects how many other inputs/outputs you can add, but somehow I doubt
its ever going to be a large enough number to matter.

Matt

On 10/24/19 1:49 PM, Johan Torås Halseth wrote:
> Reviving this old thread now that the recently released RC for bitcoind
> 0.19 includes the above mentioned carve-out rule.
>
> In an attempt to pave the way for more robust CPFP of on-chain contracts
> (Lightning commitment transactions), the carve-out rule was added in
> https://github.com/bitcoin/bitcoin/pull/15681. However, having worked on
> an implementation of a new commitment format for utilizing the Bring
> Your Own Fees strategy using CPFP, I’m wondering if the special case
> rule should have been relaxed a bit, to avoid the need for adding a 1
> CSV to all outputs (in case of Lightning this means HTLC scripts would
> need to be changed to add the CSV delay).
>
> Instead, what about letting the rule be
>
> The last transaction which is added to a package of dependent
> transactions in the mempool must:
> * Have no more than one unconfirmed parent.
>
> This would of course allow adding a large transaction to each output of
> the unconfirmed parent, which in effect would allow an attacker to
> exceed the MAX_PACKAGE_VIRTUAL_SIZE limit in some cases. However, is
> this a problem with the current mempool acceptance code in bitcoind? I
> would imagine evicting transactions based on feerate when the max
> mempool size is met handles this, but I’m asking since it seems like
> there has been several changes to the acceptance code and eviction
> policy since the limit was first introduced.
>
> - Johan
>
>
> On Wed, Feb 13, 2019 at 6:57 AM Rusty Russell <rusty at rustcorp.com.au
> <mailto:rusty at rustcorp.com.au>> wrote:
>
> Matt Corallo <lf-lists at mattcorallo.com
> <mailto:lf-lists at mattcorallo.com>> writes:
> >>> Thus, even if you imagine a steady-state mempool growth, unless the
> >>> "near the top of the mempool" criteria is "near the top of the next
> >>> block" (which is obviously *not* incentive-compatible)
> >>
> >> I was defining "top of mempool" as "in the first 4 MSipa", ie. next
> >> block, and assumed you'd only allow RBF if the old package wasn't
> in the
> >> top and the replacement would be. That seems incentive
> compatible; more
> >> than the current scheme?
> >
> > My point was, because of block time variance, even that criteria
> doesn't hold up. If you assume a steady flow of new transactions and
> one or two blocks come in "late", suddenly "top 4MWeight" isn't
> likely to get confirmed until a few blocks come in "early". Given
> block variance within a 12 block window, this is a relatively likely
> scenario.
>
> [ Digging through old mail. ]
>
> Doesn't really matter. Lightning close algorithm would be:
>
> 1. Give bitcoind unileratal close.
> 2. Ask bitcoind what current expidited fee is (or survey your mempool).
> 3. Give bitcoind child "push" tx at that total feerate.
> 4. If next block doesn't contain unilateral close tx, goto 2.
>
> In this case, if you allow a simpified RBF where 'you can replace if
> 1. feerate is higher, 2. new tx is in first 4Msipa of mempool, 3.
> old tx isnt',
> it works.
>
> It allows someone 100k of free tx spam, sure. But it's simple.
>
> We could further restrict it by marking the unilateral close somehow to
> say "gonna be pushed" and further limiting the child tx weight (say,
> 5kSipa?) in that case.
>
> Cheers,
> Rusty.
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> <mailto:Lightning-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

📅 Original date posted:2018-10-08 📝 Original message: ...

2023-06-09T14:51:32+02:00

In reply to nevent1q…w8px
_________________________

📅 Original date posted:2018-10-08
📝 Original message:
Resending due to ML bugs....

On a related note, it would be nice to get some clarity on appropriate
usage of the r= field here.
The way I had implemented it initially was that if an invoice had an r=
field any publicly-discovered last-hop routes would be ignored as the r=
data is most likely more up-to-date than any public route rumor information.
However, if it's only used as a hint and only one or two out of
potentially many channels are included in it, that may make little sense.

Not really sure what the appropriate guidance should be, probably
something like SHOULD prefer to use invoice-r=-provided-hints over
publicly-discovered routes however MAY use other last-hops in case a
substantially better route is known?

Matt

On 09/19/18 22:10, Rusty Russell wrote:
> Hi all,
>
> I'm considering a change to c-lightning, where `invoice` would
> automatically append an 'r' field for a channel which has sufficient
> *incoming* capacity for the amount (using a weighted probability across
> our peers).
>
> This isn't quite what 'r' was for, but it would be a useful
> hint for payment routing and also potentially for establishing an
> initial channel. This is an issue for the Blockstream Store which
> deliberately doesn't advertize an address any more to avoid
> centralization.
>
> Thoughts welcome!
> Rusty.
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

📅 Original date posted:2015-11-19 📝 Original message: Nope, ...

2023-06-09T14:45:09+02:00

In reply to nevent1q…ar37
_________________________

📅 Original date posted:2015-11-19
📝 Original message:
Nope, Luke came up with a way to do it in a soft-fork.

On 11/19/15 19:12, Tadge Dryja wrote:
> I've joked that BIP62 is the "whack-a-mole" BIP in that it addresses
> many vectors for txid malleability, but maybe there are more. And more
> importantly, it addresses 3rd party malleability. It's not helpful in
> the context of lightning channel creation because ECDSA sigs are
> inherently malleable. You can always re-sign the same message with a
> different k-value and get a different signature.
>
> The functionality that's needed is to be able to reliably spend from
> unconfirmed transactions. Segregated witness can accomplish that, but
> it quite a large hard-fork change. sighash_noinput can also accomplish
> that: as input txids are not signed, if they change, the spending
> transaction can be modified while leaving counterparty signatures intact.
>
> I'm hoping to start a new "testnet-L" similar to testnet3, with this
> sighash type so that we can test malleability mitigation out.
>
> (Oh also, hi mailing list, sorry I have not posted till now! But I will
> start posting!)
>
> -Tadge
>
> On Thu, Nov 19, 2015 at 9:56 AM, Mark Friedenbach <mark at friedenbach.org
> <mailto:mark at friedenbach.org>> wrote:
>
> The basic idea of the soft-fork plan is very simple --- have the
> scriptPubKey be just the 20-byte hash of the redeem script. The
> scriptSig of the spending input is empty. The actual scriptSig, with
> the redeem script and signatures, is contained in a separate Merkle
> tree committed to elsewhere in the block (e.g. in the last output of
> the coinbase, or the last output of the last transaction).
>
> On Thu, Nov 19, 2015 at 7:31 AM, Greg Sanders <gsanders87 at gmail.com
> <mailto:gsanders87 at gmail.com>> wrote:
>
> The hardfork variant is quite simple, if I understood it
> correctly. You just stick the signatures in another parallel
> merkle tree. So if you don't want to validate signatures, just
> don't download them, and validate everything else. TXIDs don't
> use the signature at all. Nothing to malleate, AFAIK. Not sure
> what the softfork plan is, but it will be a talk at Scaling
> Bitcoin HK.
>
> On Thu, Nov 19, 2015 at 10:28 AM, Glenn Tarbox, PhD
> <glenn at tarbox.org <mailto:glenn at tarbox.org>> wrote:
>
>
> On Thu, Nov 19, 2015 at 4:33 AM, sickpig at gmail.com
> <mailto:sickpig at gmail.com> <sickpig at gmail.com
> <mailto:sickpig at gmail.com>> wrote:
>
> Hi Pierre
>
> you could start here
>
> https://github.com/ElementsProject/elementsproject.github.io#segregated-witness
> https://people.xiph.org/~greg/blockstream.gmaxwell.elements.talk.060815.pdf
> https://github.com/ElementsProject/elements
>
>
> There was a brief blip on Reddit:
>
> https://www.reddit.com/r/Bitcoin/comments/3ngtx5/could_the_segregated_witness_part_of_the/cwnthlh
>
> Its weird how little information there is on Segregated
> Witness. I'm guessing its a simple concept and those
> working on it (sipa / gmaxwell) haven't felt the need to
> write it up.
>
> That it "apparently" can be done with a soft fork similar to
> P2SH is good news... I guess...
>
>
> --
> Glenn H. Tarbox, PhD
> =]|[=
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> <mailto:Lightning-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> <mailto:Lightning-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> <mailto:Lightning-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
>
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

📅 Original date posted:2022-04-23 📝 Original message:Still ...

2023-06-08T01:07:44+02:00

In reply to nevent1q…tdp2
_________________________

📅 Original date posted:2022-04-23
📝 Original message:Still trying to make sure I understand this concern, let me know if I get this all wrong.

On 4/22/22 10:25 AM, Russell O'Connor via bitcoin-dev wrote:
> It's not the attackers *only choice to succeed*. If an attacker steals the hot key, then they have
> the option to simply wait for the user to unvault their funds of their own accord and then race /
> outspend the users transaction with their own. Indeed, this is what we expect would happen in the
> dark forest.

Right, a key security assumption of the CTV-based vaults would be that you MUST NOT EVER withdraw
more in one go than your hot wallet risk tolerance, but given that your attack isn't any worse than
simply stealing the hot wallet key immediately after a withdraw.

It does have the drawback that if you ever get a hot wallet key stole you have to rotate all of your
CTV outputs and your CTV outputs must never be any larger than your hot wallet risk tolerance
amount, both of which are somewhat frustrating limitations, but not security limitations, only
practical ones.

> And that's not even mentioning the issues already noted by the document regarding fee management,
> which would likely also benefit from a less constrained design for covenants.

Of course I've always been in favor of a less constrained covenants design from day one for ten
reasons, but that's a whole other rabbit hole :)

📅 Original date posted:2021-04-25 📝 Original ...

2023-06-08T00:52:10+02:00

In reply to nevent1q…x3pd
_________________________

📅 Original date posted:2021-04-25
📝 Original message:Alright, let's see...

Sorting by most recently updated...
https://github.com/bitcoin/bips/pulls?page=1&q=is%3Apr+is%3Aopen+sort%3Aupdated-asc+updated%3A%3E2021-01-01

#1104 has been updated nearly daily for the past many weeks. You commented 12 days ago saying "Concept NACK" (which
isn't a thing on BIPs - huh? they're author documents, as you're well aware), and nothing further.

#1105 which is less recently updated by one on the above list has a comment from you 19 hours ago.

I'm really not sure what playing dumb gets you, here. Its really transparent and isn't helpful in any way to anything.

In general, I think its time we all agree the BIP process has simply failed and move on. Luckily its not really all that
critical and proposed protocol documents can be placed nearly anywhere with the same effect.

Matt

On 4/25/21 17:22, Luke Dashjr wrote:
> On Sunday 25 April 2021 21:14:08 Matt Corallo wrote:
>> On 4/25/21 17:00, Luke Dashjr wrote:
>>> I will not become an accomplice to this deception by giving special
>>> treatment, and will process the BIP PR neutrally according to the
>>> currently-defined BIP process.
>>
>> Again, please don't play dumb, no one watching believes this - you've been
>> active on the BIP repo on numerous PRs and this has never in the past been
>> the case.
>
> I started going through PRs a few days ago, in order of "Recently updated" on
> GitHub, starting with the least-recent following the last one I triaged a
> month ago that hasn't seen activity.. the same as I have been doing month
> after month prior to this.
>
> If you don't believe me, feel free to look through the repo history.
>
> Luke
>

📅 Original date posted:2021-04-24 📝 Original message:What ...

2023-06-08T00:52:05+02:00

In reply to nevent1q…f9ja
_________________________

📅 Original date posted:2021-04-24
📝 Original message:What is preventing the BIP maintainership role from moving to a bot? It does seem like a bot should be able to do a fine
job given the explicit criteria (though ignoring obvious spam is often nice, its by no means a requirement).

Given recent events where humans have....acted like humans, it seems a move to a bot may be warranted.

Matt

On 4/24/21 00:42, Greg Maxwell via bitcoin-dev wrote:
> I am opposed to the addition of Kalle Alm at this time.
>
> Those who believe that adding him will resolve the situation with
> Luke-jr's inappropriate behavior re: PR1104 are mistaken.
>
>
>
> 27e59ffd51ee5a95d0e0faff70e045faca10b00015e90abc1c8de48b1dfff40c
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2021-03-06 📝 Original ...

2023-06-07T20:30:34+02:00

In reply to nevent1q…a8rq
_________________________

📅 Original date posted:2021-03-06
📝 Original message:I'm really unsure that three months is a short enough time window that there wouldn't be a material effort to split the
network with divergent consensus rules. Instead, a three month window is certainly long enough to organize and make a
lot of noise around such an effort, given BIP 148 was organized and reached its peak within a similar such window.

Worse, because the obvious alternative after a three month activation failure is a significant delay prior to
activation, the vocal UASF minority may be encouraged to pursue such a route to avoid such a delay.

One alternative may be to reduce the signaling windows involved and start slightly later. Instead of the likelihood of
failure growing on the horizon, simply have two signaling windows (maybe two weeks, maybe a moth each?). In order to
ensure success remains likely, begin them somewhat later after software release to give pools and miners a chance to
configure their mining software in advance.

Matt

On 3/5/21 22:43, David A. Harding via bitcoin-dev wrote:
> On the ##taproot-activation IRC channel, Russell O'Connor recently
> proposed a modification of the "Let's see what happens" activation
> proposal.[1] The idea received significant discussion and seemed
> acceptable to several people who could not previously agree on a
> proposal (although this doesn't necessarily make it their first
> choice). The following is my attempt at a description.
>
> 1. Start soon: shortly after the release of software containing this
> proposed activation logic, nodes will begin counting blocks towards
> the 90% threshold required to lock in taproot.[2]
>
> 2. Stop soon: if the lockin threshold isn't reached within approximately
> three months, the activation attempt fails. There is no mandatory
> activation and everyone is encouraged to try again using different
> activation parameters.
>
> 2. Delayed activation: in the happy occasion where the lockin threshold
> is reached, taproot is guaranteed to eventually activate---but not
> until approximately six months after signal tracking started.
>
> ## Example timeline
>
> (All dates approximate; see the section below about BIP9 vs BIP8.)
>
> - T+0: release of one or more full nodes with activation code
> - T+14: signal tracking begins
> - T+28: earliest possible lock in
> - T+104: locked in by this date or need to try a different activation process
> - T+194: activation (if lockin occurred)
>
> ## Analysis
>
> The goal of Speedy Trial is to allow a taproot activation attempt to
> either quickly succeed or quickly fail---without compromising safety in
> either case. Details below:
>
> ### Mitigating the problems of early success
>
> New rules added in a soft fork need to be enforced by a large part of
> the economy or there's a risk that a long chain of blocks breaking the
> rules will be accepted by some users and rejected by others, causing a
> chain split that can result in large direct losses to transaction
> receivers and potentially even larger indirect losses to holders due to
> reduced confidence in the safety of the Bitcoin system.
>
> One step developers have taken in the past to ensure widespread adoption
> of new consensus rules is programming in a delay between the time software
> with those rules is expected to be released and when the software starts
> tracking which blocks signal for activation. For example:
>
> Soft fork | Release | Start | Delta
> -----------------+------------+------------+----------
> BIP68 (v0.12.1) | 2016-04-15 | 2016-05-11 | 26 days
> BIP141 (v0.13.1) | 2016-10-27 | 2016-11-18 | 24 days
>
> Sources: BitcoinCore.org, https://gist.github.com/ajtowns/1c5e3b8bdead01124c04c45f01c817bc
>
> Speedy Trial replaces most of that upfront delay with a backend delay.
> No matter how fast taproot's activation threshold is reached by miners,
> there will be six months between the time signal tracking starts and when
> nodes will begin enforcing taproot's rules. This gives the userbase even
> more time to upgrade than if we had used the most recently proposed start
> date for a BIP8 activation (~July 23rd).[2]
>
> ### Succeed, or fail fast
>
> The earlier version of this proposal was documented over 200 days ago[3]
> and taproot's underlying code was merged into Bitcoin Core over 140 days
> ago.[4] If we had started Speedy Trial at the time taproot
> was merged (which is a bit unrealistic), we would've either be less than
> two months away from having taproot or we would have moved on to the
> next activation attempt over a month ago.
>
> Instead, we've debated at length and don't appear to be any closer to
> what I think is a widely acceptable solution than when the mailing list
> began discussing post-segwit activation schemes over a year ago.[5] I
> think Speedy Trial is a way to generate fast progress that will either
> end the debate (for now, if activation is successful) or give us some
> actual data upon which to base future taproot activation proposals.
>
> Of course, for those who enjoy the debate, discussion can continue while
> waiting for the results of Speedy Trial.
>
> ### Base activation protocol
>
> The idea can be implemented on top of either Bitcoin Core's existing
> BIP9 code or its proposed BIP8 patchset.[6]
>
> - BIP9 uses two time-based[7] parameters, starttime and timeout. Using
> these values plus a time-based parameter for the minimum activation
> delay would give three months for miners to activate taproot, but some
> of that time near the start or the end might not be usable due to
> signals only being measured in full retarget periods. However, the
> six month time for users to upgrade their node would be not be
> affected by either slow or fast block production.
>
> BIP9 is already part of Bitcoin Core and I think the changes being
> proposed would be relatively small, resulting in a small patch that
> could be easy to review.
>
> - BIP8 uses two height-based parameters, startheight and timeoutheight.
> Using height values would ensure miners had a certain number of
> retarget periods (6) to lock in taproot and that there'd be a certain
> number of blocks (about 24,000) until activation, although latest lock
> in and expected activation could occur moderately earlier or later
> than the estimated three and six months.
>
> BIP8 would likely be used if Speedy Trial fails, so it could be
> advantageous to base this proposal on BIP8 so that we gain
> experience running that code in production.
>
> For additional discussion about using times versus heights, see today's
> log for ##taproot-activation.[11]
>
> ### Additional concerns
>
> - Encourages false signaling: false signaling is when miners signal
> readiness to enforce rules that their nodes don't actually support.
> This was partially responsible for a six-block reorg shortly after the
> final BIP66 activation[8] and was found to still be a problem during
> the BIP68 lockin period despite BIP9 being designed to avoid it.[9]
>
> Because Speedy Trial only gives miners a maximum of three months to
> signal support for taproot, it may encourage such false signaling. If
> taproot locks in as a result of their signaling but most of them fail
> to upgrade by the activation date several months later, unprepared
> miners could lose large amounts of money and users could see long
> reorgs (with unupgraded nodes and SPV lite clients potentially losing
> money).
>
> Compared to other activation proposals, I think the only difference is
> Speedy Trial's short timeline. False signaling is possible with any
> other proposal and the same problems can occur if miners fail to
> upgrade for any mandatory activation.
>
> ### Additional advantages
>
> - No mandatory signaling: at no time are miners required to signal by
> Speedy Trial. This includes no mandatory signaling during the
> locked_in period(s), although such signaling will be encouraged (as it
> was with BIP9[10]).
>
> - Party time: to a lesser degree, a benefit mentioned for flag day
> activation may also apply here: we could get up to six months
> advanced notice of taproot activation, allowing users, developers, and
> organizations to prepare software, announcements, and celebrations for
> that event.
>
> ## Implementation details and next steps
>
> Initial discussion about implementation may be found in today's
> ##taproot-activation log.[11] If it appears Speedy Trial may have
> traction, Russell O'Connor has offered to work on a patch against BIP8
> implementing it.
>
> ## Acknowledgments
>
> The original idea for a short-duration attempt was discussed in the
> ##taproot-activation IRC channel last July and the revised idea saw
> additional evaluation there this week. Despite growing frustration,
> discussion has been overwhelmingly constructive, for which all the
> contributors should be commended. Although this should not in any way
> imply endorsement, I'm grateful for the review and comments on a draft
> of this email by Adam Gibson, Andrew Chow, Anthony Towns, Chris Belcher,
> Jeremy Rubin, Jonas Nick, Luke Dashjr, Michael Folkson, Russell
> O'Connor, and IRC users maybehuman and proofofkeags
>
> ## Footnotes
>
> [1] https://en.bitcoin.it/wiki/Taproot_activation_proposals#Let.E2.80.99s_see_what_happens.2C_BIP8.28false.2C_3m.29
>
> [2] A threshold of 1,815/2,016 blocks (90%) in a single retarget period
> seemed to have near-universal support during the 2021-02-16 IRC
> meeting. See: https://en.bitcoin.it/wiki/Taproot_activation_proposal_202102
>
> [3] https://en.bitcoin.it/w/index.php?title=Taproot_activation_proposals&oldid=68062
>
> [4] https://github.com/bitcoin/bitcoin/pull/19953
>
> [5] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-January/017547.html
>
> [6] https://github.com/bitcoin/bitcoin/pull/19573
>
> [7] BIP9's times are based on the median of the past 11 blocks, which
> usually trails UTC by about 90 minutes but which can trail behind
> realtime significantly if miners are doing weird things.
>
> [8] https://en.bitcoin.it/wiki/July_2015_chain_forks
>
> [9] https://buildingbitcoin.org/bitcoin-core-dev/log-2016-06-21.html#l-32
>
> [10] https://github.com/bitcoin/bitcoin/blob/ed25cb58f605ba583c735f330482df0bf9348f3a/src/test/versionbits_tests.cpp#L337-L339
>
> [11] http://gnusha.org/taproot-activation/2021-03-05.log
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2021-02-28 📝 Original message:I ...

2023-06-07T20:29:18+02:00

In reply to nevent1q…25rf
_________________________

📅 Original date posted:2021-02-28
📝 Original message:I think you may have misunderstood my proposal. I'm not suggesting some people run BIP 8(true), some run BIP8(false),
and some run a client which has a flag day, I'm suggesting a flag day activation instead of any BIP8-based activation.
Replies to your further points inline.

Matt

On 2/28/21 12:20, Luke Dashjr wrote:
> On Sunday 28 February 2021 16:45:22 Matt Corallo via bitcoin-dev wrote:
> Concept NACK. This still has the same problems BIP149 would have had, as I
> just reminded in my last email to this ML:
>
> 1) Such a chain does not indicate activation at all, leaving it unresolved and
> debatable whether activation has occurred or not.
> 2) As a result, it is also impractical to intentionally reject the softfork
> should anyone decide to do so.
>
> Signalling is important to activation.

Several people responded disagreeing, including myself. I'll paste my response here in case you missed it:

Forced-signaling, or any form of signaling, does not materially change whether a soft fork can be seen to be safe to
use. Pieter wrote a great post[1] some time ago that goes into depth about the security of soft forks, but, while miners
can help to avoid the risk of forks, they aren't the determining factor in whether use of a fork should be considered
safe (ie the fork "has activated").

Not only that, but the signaling methods used in BIP 8/9 (ie the version field in the block header) do not imply
anything about whether mining pools are running full nodes which enforce the soft fork at all, only whether the pool has
configured their stratum software to signal or not.

Ultimately, forced-signaling, or signaling period, are not a substitute for having a broad set of upgraded nodes across
the network, including an overwhelming majority of economically-active nodes, enforcing the rules of a new fork. As this
can be difficult to measure, waiting some time after a fork and examining upgrade patterns across the network is important.

[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/012014.html

📅 Original date posted:2021-02-28 📝 Original message:Note ...

2023-06-07T20:29:18+02:00

In reply to nevent1q…xtr4
_________________________

📅 Original date posted:2021-02-28
📝 Original message:Note further that mandatory signaling isn't "just" a flag day - unlike a Taproot flag day (where miners running Bitcoin
Core unmodified today will not generate invalid blocks), a mandatory signaling flag day blatantly ignores goal (3) from
my original post - it results in any miner who has not taken active action (and ensured every part of their often-large
infrastructure has been correctly reconfigured) generating invalid blocks.

As for "Taproot" took too long, hey, at least if its locked in people can just build things assuming it exists. Some
already are, but once its clearly locked in, there's no reason to not continue other work at the same time.

Matt

On 2/28/21 14:43, Jeremy via bitcoin-dev wrote:
> I agree with much of the logic presented by Matt here.
>
> BIP8 was intended to be simpler to agree on to maintain consensus, yet we find ourselves in a situation where a "tiny"
> parameter has the potential to cause great network disruption and confusion (rationality is not too useful a concept
> here given differing levels of sophistication and information). It is therefore much simpler and more likely to be
> universally understood by all network participants to just have a flag day. It is easier to communicate what users
> should do and when.
>
> This is ultimately not coercive to users because the upgrade for Taproot itself is provable and analyzable on its own,
> but activation parameters based on what % of economically relevant nodes are running an upgrade by a certain date are
> not. Selecting these sorts of complicated consensus parameters may ultimately present more opportunity for a cooptable
> consensus process than something more straightforward.
>
>
> That said, a few points strike me as worth delving into.
>
>
> 1) Con: Mandatory signalling is no different than a flag day. Mandatory signaling is effectively 2 flag days -- one for
> the signaling rule, 1 for the taproot type. The reason for the 2 week gap between flag day for signaling and flag day
> for taproot rules is, more or less, so that nodes who aren't taproot ready at the 1st flag day do not end up SPV mining
> (using standardness rules in mempool prevents them from mining an invalid block on top of a valid tip, but does not
> ensure the tip is valid).
> 2) Con: Releasing a flag day without releasing the LOT=true code leading up to that flag day means that clients would
> not be fully compatible with an early activation that could be proposed before the flag day is reached. E.g., LOT=true
> is a flag day that retains the possibility of being compatible with other BIP8 releases without changing software.
> 3) Pro: BIP-8 is partially in service of "early activation" and . I'm personally skeptical that early activation is/was
> ever a good idea. A fixed activation date may be largely superior for business purposes, software engineering schedules,
> etc. I think even with signaling BIP8, it would be possibly superior to activate rules at a fixed date (or a quantized
> set of fixed dates, e.g. guaranteeing at least 3 months but maybe more).
> 4) Pro: part of the argument for BIP-8=false is that it is possible that the rule could not activate, if signaling does
> not occur, providing additional stopgap against dev collusion and bugs. But BIP-8 can activate immediately (with start
> times being proposed 1 month after release?) so we don't have certainty around how much time there is for that secondary
> review process (read -- I think it isn't that valuable) and if there *is* a deadly bug discovered, we might want to
> hard-fork to fix it even if it isn't yet signaled for (e.g., if the rule activates it enables more mining reward). So I
> think that it's a healthier mindset to release a with definite deadline and not rule out having to do a hard fork if
> there is a grave issue (we shouldn't ever release a SF if we think this is at all likely, mind you).
> 5) Con: It's already taken so long for taproot, the schedule around taproot was based on the idea it could early
> activate, 2022 is now too far away. I don't know how to defray this other than, if your preferred idea is 1 year flag
> day, to do that via LOT=true so that taproot can still have early activation if desired.
>
> Overall I agree with the point that all the contention around LOT, makes a flag day look not so bad. And something
> closer to a flag day might not be so bad either for future forks as well.
>
> However, I think given the appetite for early activation, if a flag day is desired I think LOT=true is the best option
> at this time as it allows our flag day to remain compatible with such an early activation.
>
> I think we can also clearly communicate that LOT=true for Taproot is not a precedent setting occurence for any future
> forks (hold me accountable to not using this as precedent this should I ever advocate for a SF with similar release
> parameters).
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2021-02-28 📝 Original ...

2023-06-07T20:29:12+02:00

In reply to nevent1q…x4dq
_________________________

📅 Original date posted:2021-02-28
📝 Original message:Forced-signaling, or any form of signaling, does not materially change whether a soft fork can be seen to be safe to
use. Pieter wrote a great post[1] some time ago that goes into depth about the security of soft forks, but, while miners
can help to avoid the risk of forks, they aren't the determining factor in whether use of a fork should be considered
safe (ie the fork "has activated").

Not only that, but the signaling methods used in BIP 8/9 (ie the version field in the block header) do not imply
anything about whether mining pools are running full nodes which enforce the soft fork at all, only whether the pool has
configured their stratum software to signal or not.

Ultimately, forced-signaling, or signaling period, are not a substitute for having a broad set of upgraded nodes across
the network, including an overwhelming majority of economically-active nodes, enforcing the rules of a new fork. As this
can be difficult to measure, waiting some time after a fork and examining upgrade patterns across the network is important.

Matt

[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/012014.html

On 2/27/21 12:55, Luke Dashjr via bitcoin-dev wrote:
> This has the same problems BIP149 did: since there is no signalling, it is
> ambiguous whether the softfork has activated at all. Both anti-SF and pro-SF
> nodes will remain on the same chain, with conflicting perceptions of the
> rules, and resolution (if ever) will be chaotic. Absent resolution, however,
> there is a strong incentive not to rely on the rules, and thus it may never
> get used, and therefore also never resolved.
>
> Additionally, it loses the flexibility of BIP 8 to, after the initial
> deployment, move the timeoutheight sooner.
>
> Luke
>
>
> On Thursday 25 February 2021 22:33:25 Gregorio Guidi via bitcoin-dev wrote:
>> Hello,
>>
>> I followed the debate on LOT=false / LOT=true trying to get a grasp of
>> the balance of risks and advantages. The summary by Aaron van Wirdum [1]
>> explains well the difficulties to find a good equilibrium... it
>> concludes that "perhaps, a new possibility will present itself".
>>
>> Thinking about such a "new possibility" that overcomes the
>> LOT=true/false dichotomy, I would like to offer the following proposal.
>> It could be called "decreasing threshold activation".
>>
>> Decreasing threshold activation works similarly to BIP8, with the
>> difference that the threshold that triggers the STARTED -> LOCKED_IN
>> transition starts at 100% for the first retargeting period, and then is
>> gradually reduced on each period in steps of 24 blocks (~1,2%). More
>> precisely:
>>
>> On the 1st period (starting on start_height): if 2016 out of 2016 blocks
>> signal, the state is changed to LOCKED_IN on the next period (otherwise
>> stays STARTED)
>> On the 2nd period: if 1992 out of 2016 blocks signal (~98.8%), the state
>> transitions to LOCKED_IN on the next period
>> On the 3rd period: if 1968 out of 2016 blocks signal (~97.6%), the state
>> transitions to LOCKED_IN on the next period
>> ...
>> On the 14th period (~6 months): if 1704 out of 2016 blocks signal
>> (~84.5%), the state transitions to LOCKED_IN on the next period
>> ...
>> On the 27th period (~12 months): if 1392 out of 2016 blocks signal
>> (~69.0%), the state transitions to LOCKED_IN on the next period
>> ...
>> On the 40th period (~18 months): if 1080 out of 2016 blocks signal
>> (~53.6%), the state transitions to LOCKED_IN on the next period
>> ...
>> On the 53th period (~24 months): if 768 out of 2016 blocks signal
>> (~38.1%), the state transitions to LOCKED_IN on the next period
>> ...
>> On the 66th period (~30 months): if 456 out of 2016 blocks signal
>> (~22.6%), the state transitions to LOCKED_IN on the next period
>> ...
>> On the 79th period (~36 months): if 144 out of 2016 blocks signal
>> (~7.1%), the state transitions to LOCKED_IN on the next period
>> ...
>> On the 84th and final period (~39 months): if 24 out of 2016 blocks
>> signal (~1.2%), the state transitions to LOCKED_IN on the next period,
>> otherwise goes to FAILED
>>
>> (For reference, I include below a snippet of pseudocode for the
>> decreasing thresholds in the style of BIP8 and BIP9.)
>>
>> Here are the main features and advantages of this approach:
>>
>> 1. It is relatively conservative at the beginning: for activation to
>> happen in the first year, it requires a clear majority of signaling
>> hashrate, indicating that the activation is relatively safe. Only later
>> the threshold starts to move towards "unsafe" territory, accepting the
>> tradeoff of less support from existing hashrate in exchange for ensuring
>> that the activation eventually happens.
>>
>> 2. Like LOT=true, the activation will always occur in the end (except in
>> the negligible case where less than 1.2% of hashrate supports it).
>>
>> 3. This approach is quite easy to implement, in particular it avoids the
>> extra code to deal with the MUST_SIGNAL period.
>>
>> 4. There are no parameters to set (except startheight). I am a KISS fan,
>> so this is a plus for me, making the activation mechanism robust and
>> predictable with less chance for users to shoot themselves in the foot.
>> It is also a plus for me that - if adopted as the default mechanism - it
>> would require very little discussion on how to activate future
>> soft-forks. In fact I think it would be a winning move for Core to
>> commit to such a scheme, to avoid getting lost in game-theoretic rabbit
>> holes.
>>
>> 5. Since there is no MUST_SIGNAL period, no automatic chain split occurs
>> around activation when not all miners have upgraded (so activation is
>> generally as benign as a MASF). A chain split will occur only when/if an
>> invalid block is created (and this requires dedicated effort! it can
>> only happen by circumventing the normal policy rules [2]). This
>> mitigates the risk of reorgs and involuntary forks around activation,
>> even with low miner signaling.
>>
>> 6. It removes motivation to create UASF clients that force activation.
>> While individual nodes could still try to force a quicker activation,
>> the motivation to do so is reduced since the same result is obtained
>> just by waiting a little more.
>>
>> 7. Compared to LOT=true, activation is cleaner and quicker when it is
>> relatively safe to do so (when the signaling hashrate is - let's say -
>> in the 70%-80% range). On the other hand, activation is pushed further
>> and further in time when it is less safe (when signaling hashrate is
>> <50%, meaning that there is a serious risk that users/miners that did
>> not upgrade start following an alternative chain). This gives everyone
>> time to prepare properly for such a potentially disruptive event.
>>
>> 8. If a significant number of users and miners consciously decide (for
>> whatever reasons) that they don't want to upgrade and want to fork
>> themselves off from the chain followed by Core (as is their
>> prerogative), they will have time to do so safely.
>>
>> 9. Compared to the strategy of doing LOT=false and then LOT=true if it
>> fails, using the decreasing threshold approach may not seem very
>> different. But it completely removes the need to fiddle with different
>> client releases and with the issues associated with deployed nodes with
>> different consensus parameters.
>>
>> All in all, reading the various perspectives on this mailing list and
>> outside I have the feeling that the strongest arguments against LOT=true
>> have at their core a certain uneasiness with the MUST_SIGNAL mechanism
>> and the related automatic chain split on activation, which is something
>> that greatly complicates the analysis (but please tell me if I am
>> wrong...). In this sense, this proposal achieves the big objective of
>> always ending in activation (like LOT=true) without resorting to
>> MUST_SIGNAL and chain splits.
>>
>> A final note: this proposal should be seen as somewhat independent from
>> the discussion on taproot activation. Personally I would be happy with a
>> LOT=false activation for taproot that succeeds quickly, while the
>> decreasing threshold approach could be evaluated as potential default
>> activation mechanism for the future.
>>
>> I would be happy to hear what you think about this. What are the
>> possible issues/drawbacks of using this mechanism?
>>
>> Thanks,
>>
>> Gregorio
>>
>> [1]
>> https://bitcoinmagazine.com/articles/lottrue-or-lotfalse-this-is-the-last-h
>> urdle-before-taproot-activation
>>
>> [2] This was not the case in the past for upgrades such as BIP16 (P2SH),
>> which generated frequent reorgs due to a combination of low activation
>> threshold (55%) and no policy protection. But for upgrades such as
>> taproot the normal policy rules prevent the creation of invalid blocks
>> by non-upgraded miners. See
>> https://blog.bitmex.com/the-arts-of-making-softforks-protection-by-policy-r
>> ule/
>>
>> Pseudocode:
>>
>> case STARTED:
>> int elapsed_periods = (block.height - startheight) / 2016;
>> if (elapsed_periods > 2016 / 24) {
>> return FAILED;
>> }
>> int threshold = 2016 - 24 * (elapsed_periods - 1);
>> int count = 0;
>> walk = block;
>> for (i = 0; i < 2016; i++) {
>> walk = walk.parent;
>> if (walk.nVersion & 0xE0000000 == 0x20000000 &&
>> (walk.nVersion >> bit) & 1 == 1) { ++count;
>> }
>> }
>> if (count >= threshold) {
>> return LOCKED_IN;
>> }
>> return STARTED;
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2021-02-22 📝 Original message:> ...

2023-06-07T20:28:58+02:00

In reply to nevent1q…reqn
_________________________

📅 Original date posted:2021-02-22
📝 Original message:> On Feb 22, 2021, at 05:16, Anthony Towns <aj at erisian.com.au> wrote:
>
> If a lockinontimeout=true node is requesting compact blocks from a
> lockinontimeout=false node during a chainsplit in the MUST_SIGNAL phase,
> I think that could result in a ban.
>
>> More importantly, nodes on both sides of the fork need to find each other.
>
> (If there was going to be an ongoing fork there'd be bigger things to
> worry about...)

I think it should be clear that a UASF-style command line option to allow consensus rule changes in the node in the short term, immediately before a fork carries some risk of a fork, even if I agree it may not persist over months. We can’t simply ignore that.

> I think the important specific case of this is something like "if a chain
> where taproot is impossible to activate is temporarily the most work,
> miners with lockinontimeout=true need to be well connected so they don't
> end up competing with each other while they're catching back up".

Between this and your above point, I think we probably agree - there is material technical complexity hiding behind a “change the consensus rules“ option. Given it’s not a critical feature by any means, putting resources into fixing these issues probably isn’t worth it.

Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210222/08d2cbe2/attachment.html>;

📅 Original date posted:2021-02-22 📝 Original message:Hmm, ...

2023-06-07T20:28:57+02:00

In reply to nevent1q…mhvh
_________________________

📅 Original date posted:2021-02-22
📝 Original message:Hmm, indeed, I may have missed that you can skip the headers issues by not persisting them, though there are other follow-on effects that are concerning and I think still make my point valid.

A node feeding you invalid headers (used to be) cause for a ban - is that information still persisted? More importantly, nodes on both sides of the fork need to find each other. There’s not a great way to do that without forking the address database, DNS seeds and defining a new protocol magic.

Matt

> On Feb 22, 2021, at 00:16, Anthony Towns <aj at erisian.com.au> wrote:
>
> On Fri, Feb 19, 2021 at 12:48:00PM -0500, Matt Corallo via bitcoin-dev wrote:
>> It was pointed out to me that this discussion is largely moot as the
>> software complexity for Bitcoin Core to ship an option like this is likely
>> not practical/what people would wish to see.
>> Bitcoin Core does not have infrastructure to handle switching consensus
>> rules with the same datadir - after running with uasf=true for some time,
>> valid blocks will be marked as invalid,
>
> I don't think this is true? With the current proposed bip8 code,
> lockinontimeout=true will cause headers to be marked as invalid, and
> won't process the block further. If a node running lockinontimeout=true
> accepts the header, then it will apply the same consensus rules as a
> lockinontimeout=false node.
>
> I don't think an invalid header will be added to the block index at all,
> so a node restart should always cleanly allow it to be reconsidered.
>
> The test case in
>
> https://github.com/bitcoin/bitcoin/pull/19573/commits/bd8517135fc839c3332fea4d9c8373b94c8c9de8
>
> tests that a node that had rejected a chain due to lockinontimeout=true
> will reorg to that chain after being restarted as a byproduct of the way
> it tests different cases (the nodes set a new startheight, but retain
> their lockinontimeout settings).
>
>
> (I think with the current bip8 code, if you switch from
> lockinontimeout=false to lockinontimeout=true and the tip of the current
> most work chain is after the timeoutheight and did not lockin, then you
> will continue following that chain until a taproot-invalid transaction
> is inclued, rather than immediately reorging to a shorter chain that
> complies with the lockinontimeout=true rules)
>
> Cheers,
> aj
>

📅 Original date posted:2021-02-21 📝 Original message:I ...

2023-06-07T20:28:56+02:00

In reply to nevent1q…90lc
_________________________

📅 Original date posted:2021-02-21
📝 Original message:I don’t think “some vocal users are going to threaten to fork themselves off” is good justification for technical decisions. It’s important to communicate and for everyone to agree/understand that a failed BIP 8/9 activation, in the scenario people are worried about, is not the end of the story for Taproot activation. If it is clear that Taproot has broad consensus but some miners failed to upgrade in time (as it presumably would be), a flag day activation seems merited and I’m not sure anyone has argued against this. That said, forced-signaling via a UASF/BIP8(true)-style fork carries material additional risk that a classic flag-day activation does not, so let’s not optimize for something like that.

Matt

> On Feb 21, 2021, at 08:26, Ariel Lorenzo-Luaces via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>
> What would be the tradeoffs of a BIP8(false, ∞) option? That would remove some of the concerns of having to coordinate a UASF with an approaching deadline.
>
> Cheers
> Ariel Lorenzo-Luaces
>> On Feb 19, 2021, at 6:55 PM, ZmnSCPxj via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>> Good morning list,
>>
>>> It was pointed out to me that this discussion is largely moot as the software complexity for Bitcoin Core to ship an
>>> option like this is likely not practical/what people would wish to see.
>>>
>>> Bitcoin Core does not have infrastructure to handle switching consensus rules with the same datadir - after running with
>>> uasf=true for some time, valid blocks will be marked as invalid, and additional development would need to occur to
>>> enable switching back to uasf=false. This is complex, critical code to get right, and the review and testing cycles
>>> needed seem to be not worth it.
>>
>> Without implying anything else, this can be worked around by a user maintaining two `datadir`s and running two clients.
>> This would have an "external" client running an LOT=X (where X is whatever the user prefers) and an "internal" client that is at most 0.21.0, which will not impose any LOT rules.
>> The internal client then uses `connect=` directive to connect locally to the external client and connects only to that client, using it as a firewall.
>> The external client can be run pruned in order to reduce diskspace resource usage (the internal client can remain unpruned if that is needed by the user, e.g. for LN implementation sthat need to look up arbitrary short-channel-ids).
>> Bandwidth usage should be same since the internal client only connects to the external client and the OS should optimize that case.
>> CPU usage is doubled, though.
>>
>> (the general idea came from gmax, just to be clear, though the below use is from me)
>>
>> Then the user can select LOT=C or LOT=!C (where C is whatever Bitcoin Core ultimately ships with) on the external client based on the user preferences.
>>
>> If Taproot is not MASF-activated and LOT=!U is what dominates later (where U is whatever the user decided on), the user can decide to just destroy the external node and connect the internal node directly to the network (optionally upgrading the internal node to LOT=!U) as a way to "change their mind in view of the economy".
>> The internal node will then follow the dominant chain.
>>
>>
>> Regards,
>> ZmnSCPxj
>>
>>>
>>> Instead, the only practical way to ship such an option would be to treat it as a separate chain (the same way regtest,
>>> testnet, and signet are treated), including its own separate datadir and the like.
>>>
>>> Matt
>>>
>>>> On 2/19/21 09:13, Matt Corallo via bitcoin-dev wrote:
>>>>
>>>> (Also in response to ZMN...)
>>>> Bitcoin Core has a long-standing policy of not shipping options which shoot yourself in the foot. I’d be very disappointed if that changed now. People are of course more than welcome to run such software themselves, but I anticipate the loud minority on Twitter and here aren’t processing enough transactions or throwing enough financial weight behind their decision for them to do anything but just switch back if they find themselves on a chain with no blocks.
>>>> There’s nothing we can (or should) do to prevent people from threatening to (and possibly) forking themselves off of bitcoin, but that doesn’t mean we should encourage it either. The work Bitcoin Core maintainers and developers do is to recommend courses of action which they believe have reasonable levels of consensus and are technically sound. Luckily, there’s strong historical precedent for people deciding to run other software around forks, so misinterpretation is not very common (just like there’s strong historical precedent for miners not unilaterally deciding forks in the case of Segwit).
>>>> Matt
>>>>
>>>>>> On Feb 19, 2021, at 07:08, Adam Back adam at cypherspace.org wrote:
>>>>>>
>>>>>> would dev consensus around releasing LOT=false be considered as "developers forcing their views on users"?
>>>>>
>>>>> given there are clearly people of both views, or for now don't care
>>>>> but might later, it would minimally be friendly and useful if
>>>>> bitcoin-core has a LOT=true option - and that IMO goes some way to
>>>>> avoid the assumptive control via defaults.
>>>>
>>>>> Otherwise it could be read as saying "developers on average
>>>>> disapprove, but if you, the market disagree, go figure it out for
>>>>> yourself" which is not a good message for being defensive and avoiding
>>>>> mis-interpretation of code repositories or shipped defaults as
>>>>> "control".
>>>>
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev at lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>>
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210221/fb583f75/attachment.html>;

📅 Original date posted:2021-02-19 📝 Original message:(Also ...

2023-06-07T20:28:55+02:00

In reply to nevent1q…fcgg
_________________________

📅 Original date posted:2021-02-19
📝 Original message:(Also in response to ZMN...)

Bitcoin Core has a long-standing policy of not shipping options which shoot yourself in the foot. I’d be very disappointed if that changed now. People are of course more than welcome to run such software themselves, but I anticipate the loud minority on Twitter and here aren’t processing enough transactions or throwing enough financial weight behind their decision for them to do anything but just switch back if they find themselves on a chain with no blocks.

There’s nothing we can (or should) do to prevent people from threatening to (and possibly) forking themselves off of bitcoin, but that doesn’t mean we should encourage it either. The work Bitcoin Core maintainers and developers do is to recommend courses of action which they believe have reasonable levels of consensus and are technically sound. Luckily, there’s strong historical precedent for people deciding to run other software around forks, so misinterpretation is not very common (just like there’s strong historical precedent for miners not unilaterally deciding forks in the case of Segwit).

Matt

> On Feb 19, 2021, at 07:08, Adam Back <adam at cypherspace.org> wrote:
>> would dev consensus around releasing LOT=false be considered as "developers forcing their views on users"?
>
> given there are clearly people of both views, or for now don't care
> but might later, it would minimally be friendly and useful if
> bitcoin-core has a LOT=true option - and that IMO goes some way to
> avoid the assumptive control via defaults.

> Otherwise it could be read as saying "developers on average
> disapprove, but if you, the market disagree, go figure it out for
> yourself" which is not a good message for being defensive and avoiding
> mis-interpretation of code repositories or shipped defaults as
> "control".

📅 Original date posted:2021-02-19 📝 Original message:It was ...

2023-06-07T20:28:55+02:00

In reply to nevent1q…t7yg
_________________________

📅 Original date posted:2021-02-19
📝 Original message:It was pointed out to me that this discussion is largely moot as the software complexity for Bitcoin Core to ship an
option like this is likely not practical/what people would wish to see.

Bitcoin Core does not have infrastructure to handle switching consensus rules with the same datadir - after running with
uasf=true for some time, valid blocks will be marked as invalid, and additional development would need to occur to
enable switching back to uasf=false. This is complex, critical code to get right, and the review and testing cycles
needed seem to be not worth it.

Instead, the only practical way to ship such an option would be to treat it as a separate chain (the same way regtest,
testnet, and signet are treated), including its own separate datadir and the like.

Matt

On 2/19/21 09:13, Matt Corallo via bitcoin-dev wrote:
> (Also in response to ZMN...)
>
> Bitcoin Core has a long-standing policy of not shipping options which shoot yourself in the foot. I’d be very disappointed if that changed now. People are of course more than welcome to run such software themselves, but I anticipate the loud minority on Twitter and here aren’t processing enough transactions or throwing enough financial weight behind their decision for them to do anything but just switch back if they find themselves on a chain with no blocks.
>
> There’s nothing we can (or should) do to prevent people from threatening to (and possibly) forking themselves off of bitcoin, but that doesn’t mean we should encourage it either. The work Bitcoin Core maintainers and developers do is to recommend courses of action which they believe have reasonable levels of consensus and are technically sound. Luckily, there’s strong historical precedent for people deciding to run other software around forks, so misinterpretation is not very common (just like there’s strong historical precedent for miners not unilaterally deciding forks in the case of Segwit).
>
> Matt
>
>> On Feb 19, 2021, at 07:08, Adam Back <adam at cypherspace.org> wrote:
>>> would dev consensus around releasing LOT=false be considered as "developers forcing their views on users"?
>>
>> given there are clearly people of both views, or for now don't care
>> but might later, it would minimally be friendly and useful if
>> bitcoin-core has a LOT=true option - and that IMO goes some way to
>> avoid the assumptive control via defaults.
>
>> Otherwise it could be read as saying "developers on average
>> disapprove, but if you, the market disagree, go figure it out for
>> yourself" which is not a good message for being defensive and avoiding
>> mis-interpretation of code repositories or shipped defaults as
>> "control".
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2021-02-18 📝 Original message:This ...

2023-06-07T20:28:52+02:00

In reply to nevent1q…anwn
_________________________

📅 Original date posted:2021-02-18
📝 Original message:This is absolutely the case, however note that the activation method itself is consensus code which executes as a part
of a fork, and one which deserves as much scrutiny as anything else. While taproot is a model of how a soft-fork should
be designed, this doesn't imply anything about the consensus code which represents the activation thereof.

Hence all the debate around activation - ultimately its also defining a fork, and given the politics around it, one
which almost certainly carries significantly more risk than Taproot.

Note that I don't believe anyone is advocating for "try to activate, and if it fails, move on". Various people have
various views on how conservative and timelines for what to do at that point, but I believe most in this discussion are
OK with flag-day-based activation (given some level of care) if it becomes clear Taproot is supported by a vast majority
of Bitcoin users and is only not activating due to lagging miner upgrades.

Matt

On 2/18/21 10:04, Keagan McClelland wrote:
> Hi all,
>
> I think it's important for us to consider what is actually being considered for activation here.
>
> The designation of "soft fork" is accurate but I don't think it adequately conveys how non-intrusive a change like this
> is. All that taproot does (unless I'm completely missing something) is imbue a previously undefined script version with
> actual semantics. In order for a chain reorg to take place it would mean that someone would have to have a use case for
> that script version today. This is something I think that we can easily check by digging through the UTXO set or
> history. If anyone is using that script version, we absolutely should not be using it, but that doesn't mean that we
> can't switch to a script version that no one is actually using.
>
> If no one is even attempting to use the script version, then the change has no effect on whether a chain split occurs
> because there is simply no block that contains a transaction that only some of the network will accept.
>
> Furthermore, I don't know how Bitcoin can stand the test of time if we allow developers who rely on "undefined behavior"
> (which the taproot script version presently is) to exert tremendous influence over what code does or does not get run.
> This isn't a soft fork that makes some particular UTXO's unspendable. It isn't one that bans miners from collecting
> fees. It is a change that means that certain "always accept" transactions actually have real conditions you have to
> meet. I can't imagine a less intrusive change.
>
> On the other hand, choosing to let L=F be a somewhat final call sets a very real precedent that 10% of what I estimate
> to be 1% of bitcoin users can effectively block any change from here on forward. At that point we are saying that miners
> are in control of network consensus in ways they have not been up until now. I don't think this is a more desirable
> outcome to let ~0.1% of the network get to block /non-intrusive/ changes that the rest of the network wants.
>
> I can certainly live with an L=F attempt as a way to punt on the discussion, maybe the activation happens and this will
> all be fine. But if it doesn't, I hardly think that users of Bitcoin are just going to be like "well, guess that's it
> for Taproot". I have no idea what ensues at that point, but probably another community led UASF movement.
>
> I wasn't super well educated on this stuff back in '17 when Segwit went down, as I was new at that time, so if I'm
> missing something please say so. But from my point of view, we can't treat all soft forks as equal.
>
> Keagan
>
> On Thu, Feb 18, 2021 at 7:43 AM Matt Corallo via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
> We've had several softforks in Bitcoin which, through the course of their activation, had a several-block reorg. That
> should be indication enough that we need to very carefully consider activation to ensure we reduce the risk of that as
> much as absolutely possible. Again, while I think Taproot is a huge improvement and am looking forward to being able to
> use it, getting unlucky and hitting a 4-block reorg that happens to include a double-spend and some PR around an
> exchange losing millions would be worse than having Taproot is good.
>
> Matt
>
> On 2/18/21 09:26, Michael Folkson wrote:
> > Thanks for your response Matt. It is a fair challenge. There is always going to be an element of risk with soft
> forks,
> > all we can do is attempt to minimize that risk. I would argue that risk has been minimized for Taproot.
> >
> > You know (better than I do in fact) that Bitcoin (and layers built on top of it) greatly benefit from upgrades
> such as
> > Taproot. To say we shouldn't do Taproot or any future soft forks because there is a small but real risk of chain
> splits
> > I think is shortsighted. Indeed I think even if we collectively decided not to do any future soft fork upgrades ever
> > again on this mailing list that wouldn't stop soft fork attempts from other people in future.
> >
> > I don't think there is anything else we can do to minimize that risk for the Taproot soft fork at this point
> though I'm
> > open to ideas. To reiterate that risk will never be zero. I don't think I see Bitcoin as fragile as you seem to
> (though
> > admittedly you have a much better understanding than me of what happened in 2017).
> >
> > The likely scenario for the Taproot soft fork is LOT turns out to be entirely irrelevant and miners activate Taproot
> > before it becomes relevant. And even the unlikely worst case scenario would only cause short term disruption and
> > wouldn't kill Bitcoin long term.
> >
> > On Thu, Feb 18, 2021 at 2:01 PM Matt Corallo <lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>
> <mailto:lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>>> wrote:
> >
> > If the eventual outcome is that different implementations (that have material *transaction processing* userbases,
> > and I’m not sure to what extent that’s true with Knots) ship different consensus rules, we should stop here
> and not
> > activate Taproot. Seriously.
> >
> > Bitcoin is a consensus system. The absolute worst outcome at all possible is to have it fall out of consensus.
> >
> > Matt
> >
> >> On Feb 18, 2021, at 08:11, Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> >> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>> wrote:
> >>
> >>
> >> Right, that is one option. Personally I would prefer a Bitcoin Core release sets LOT=false (based on what I have
> >> heard from Bitcoin Core contributors) and a community effort releases a version with LOT=true. I don't think
> users
> >> should be forced to choose something they may have no context on before they are allowed to use Bitcoin Core.
> >>
> >> My current understanding is that roasbeef is planning to set LOT=false on btcd (an alternative protocol
> >> implementation to Bitcoin Core) and Luke Dashjr hasn't yet decided on Bitcoin Knots.
> >>
> >>
> >>
> >> On Thu, Feb 18, 2021 at 11:52 AM ZmnSCPxj <ZmnSCPxj at protonmail.com <mailto:ZmnSCPxj at protonmail.com>
> <mailto:ZmnSCPxj at protonmail.com <mailto:ZmnSCPxj at protonmail.com>>> wrote:
> >>
> >> Good morning all,
> >>
> >> > "An activation mechanism is a consensus change like any other change, can be contentious like any other
> >> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline."
> >> >
> >> > Who's we here?
> >> >
> >> > Release both and let the network decide.
> >>
> >> A thing that could be done, without mandating either LOT=true or LOT=false, would be to have a release that
> >> requires a `taprootlot=1` or `taprootlot=0` and refuses to start if the parameter is not set.
> >>
> >> This assures everyone that neither choice is being forced on users, and instead what is being forced on
> users,
> >> is for users to make that choice themselves.
> >>
> >> Regards,
> >> ZmnSCPxj
> >>
> >> >
> >> > On Thu, Feb 18, 2021 at 3:08 AM Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> >> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>> wrote:
> >> >
> >> > > Thanks for your response Ariel. It would be useful if you responded to specific points I have made
> in the
> >> mailing list post or at least quote these ephemeral "people" you speak of. I don't know if you're responding
> >> to conversation on the IRC channel or on social media etc.
> >> > >
> >> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted
> into
> >> code. But in fact this isn't true and some voices in this discussion need to be more humble about what users
> >> must or must not run.
> >> > >
> >> > > I personally have never made this assumption. Of course users aren't forced to run any particular
> software
> >> version, quite the opposite. Defaults set in software versions matter though as many users won't change
> them.
> >> > >
> >> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be
> only a
> >> handful of people that begin running it while everyone else delays their upgrade (with the very good
> reason of
> >> not getting involved in politics) and a year later those handful of people just become stuck at the
> moment of
> >> MUST_SIGNAL, unable to mine new blocks?
> >> > >
> >> > > It is a possible outcome but the likely outcome is that miners activate Taproot before LOT is even
> >> relevant. I think it is prudent to prepare for the unlikely but possible outcome that miners fail to
> activate
> >> and hence have this discussion now rather than be unprepared for that eventuality. If LOT is set to
> false in a
> >> software release there is the possibility (T2 in
> >> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>;
> >> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>>;) of individuals or a
> >> proportion of the community changing LOT to true. In that sense setting LOT=false in a software release
> >> appears to be no more safe than LOT=true.
> >> > >
> >> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with
> miners
> >> by default.
> >> > >
> >> > > There is the (unlikely but possible) possibility of a wasted year if LOT is set to false and miners fail
> >> to activate. I'm not convinced by this perception that LOT=true is antagonistic to miners. I actually
> think it
> >> offers them clarity on what will happen over a year time period and removes the need for coordinated or
> >> uncoordinated community UASF efforts on top of LOT=false.
> >> > >
> >> > > > An activation mechanism is a consensus change like any other change, can be contentious like any other
> >> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
> >> > >
> >> > > I don't know what you are recommending here to avoid "this darkest timeline". Open discussions have
> >> occurred and are continuing and in my mailing list post that you responded to **I recommended we propose
> >> LOT=false be set in protocol implementations such as Bitcoin Core**. I do think this apocalyptic language
> >> isn't particularly helpful. In an open consensus system discussion is healthy, we should prepare for bad or
> >> worst case scenarios in advance and doing so is not antagonistic or destructive. Mining pools have pledged
> >> support for Taproot but we don't build secure systems based on pledges of support, we build them to minimize
> >> trust in any human actors. We can be grateful that people like Alejandro have worked hard on
> >> taprootactivation.com <http://taprootactivation.com>; <http://taprootactivation.com
> <http://taprootactivation.com>>; (and this effort has informed the discussion) without
> >> taking pledges of support as cast iron guarantees.
> >> > >
> >> > > TL;DR It sounds like you agree with my recommendation to set LOT=false in protocol implementations in my
> >> email :)
> >> > >
> >> > > On Thu, Feb 18, 2021 at 5:43 AM Ariel Lorenzo-Luaces <arielluaces at gmail.com
> <mailto:arielluaces at gmail.com>
> >> <mailto:arielluaces at gmail.com <mailto:arielluaces at gmail.com>>> wrote:
> >> > >
> >> > > > Something what strikes me about the conversation is the emotion surrounding the letters UASF.
> >> > > > It appears as if people discuss UASF as if it's a massive tidal wave of support that is
> inevitable, like
> >> we saw during segwit activation. But the actual definition is "any activation that is not a MASF".
> >> > > > A UASF can consist of a single node, ten nodes, a thousand, half of all nodes, all business' nodes, or
> >> even all the non mining nodes. On another dimension it can have zero mining support, 51% support, 49%
> support,
> >> or any support right up against a miner activation threshold.
> >> > > > Hell a UASF doesn't even need code or even a single node running as long as it exists as a possibility
> >> in people's minds.
> >> > > > The only thing a UASF doesn't have is miner support above an agreed activation threshold (some number
> >> above %51).
> >> > > > I say this because it strikes me when people say that they are for LOT=true with the logic that
> since a
> >> UASF is guaranteed to happen then it's better to just make it default from the beginning. Words like
> >> coordination and safety are sometimes sprinkled into the argument.
> >> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted
> into
> >> code. But in fact this isn't true and some voices in this discussion need to be more humble about what users
> >> must or must not run.
> >> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be
> only a
> >> handful of people that begin running it while everyone else delays their upgrade (with the very good
> reason of
> >> not getting involved in politics) and a year later those handful of people just become stuck at the
> moment of
> >> MUST_SIGNAL, unable to mine new blocks? Or attracting a minority of miners, activating, and forking off
> into a
> >> minority fork. Then a lot=false could be started that ends up activating the feature now that the stubborn
> >> option has ran its course.
> >> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with
> miners
> >> by default. The chains could be called BitcoinLenient and BitcoinStubborn.
> >> > > > How is that strictly safer or more coordinated?
> >> > > > I may be in the minority, or maybe a silent majority, or maybe a majority that just hasn't considered
> >> this as a choice but honestly if there is contention about whether we're going to be stubborn or lenient
> with
> >> miners for Taproot and in the future then I prefer to just not activate anything at all. I'm fine for
> calling
> >> bitcoin ossified, accepting that segwit is Bitcoin's last network upgrade. Taproot is amazing but no new
> >> feature is worth a network split down the middle.
> >> > > > Maybe in 10 or 20 years, when other blockchains implement features like Taproot and many more, we will
> >> become envious enough to put aside our differences on how to behave towards miners and finally activate
> Taproot.
> >> > > > An activation mechanism is a consensus change like any other change, can be contentious like any other
> >> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
> >> > > > Cheers
> >> > > > Ariel Lorenzo-Luaces
> >> > > > On Feb 17, 2021, at 7:05 AM, Michael Folkson via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
> >> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>> wrote:
> >> > > >
> >> > > > > Yesterday (February 16th) we held a second meeting on Taproot
> >> > > > > activation on IRC which again was open to all. Despite what appeared
> >> > > > > to be majority support for LOT=false over LOT=true in the first
> >> > > > > meeting I (and others) thought the arguments had not been explored in
> >> > > > > depth and that we should have a follow up meeting almost entirely
> >> > > > > focused on whether LOT (lockinontimeout) should be set to true or
> >> > > > > false.
> >> > > > >
> >> > > > > The meeting was announced here:
> >> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>;
> >> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>>;
> >> > > > >
> >> > > > > In that mailing list post I outlined the arguments for LOT=true (T1 to
> >> > > > > T6) and arguments for LOT=false (F1 to F6) in their strongest form I
> >> > > > > could. David Harding responded with an additional argument for
> >> > > > > LOT=false (F7) here:
> >> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html>;
> >> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html>>;
> >> > > > >
> >> > > > > These meetings are very challenging given they are open to all, you
> >> > > > > don’t know who will attend and you don’t know most people’s views in
> >> > > > > advance. I tried to give time for both the LOT=true arguments and the
> >> > > > > LOT=false arguments to be discussed as I knew there was support for
> >> > > > > both. We only tried evaluating which had more support and which had
> >> > > > > more strong opposition towards the end of the meeting.
> >> > > > >
> >> > > > > The conversation log is here:
> >> > > > > http://gnusha.org/taproot-activation/2021-02-16.log
> <http://gnusha.org/taproot-activation/2021-02-16.log>; <http://gnusha.org/taproot-activation/2021-02-16.log
> <http://gnusha.org/taproot-activation/2021-02-16.log>>;
> >> > > > >
> >> > > > > (If you are so inclined you can watch a video of the meeting here.
> >> > > > > Thanks to the YouTube account “Bitcoin” for setting up the livestream:
> >> > > > > https://www.youtube.com/watch?v=vpl5q1ovMLM <https://www.youtube.com/watch?v=vpl5q1ovMLM>;
> <https://www.youtube.com/watch?v=vpl5q1ovMLM <https://www.youtube.com/watch?v=vpl5q1ovMLM>>;)
> >> > > > >
> >> > > > > A summary of the meeting was provided by Luke Dashjr on Mastodon here:
> >> > > > > https://bitcoinhackers.org/@lukedashjr/105742918779234566
> <https://bitcoinhackers.org/@lukedashjr/105742918779234566>;
> >> <https://bitcoinhackers.org/@lukedashjr/105742918779234566
> <https://bitcoinhackers.org/@lukedashjr/105742918779234566>>;
> >> > > > >
> >> > > > > Today's #Bitcoin #Taproot meeting was IMO largely unproductive, but we
> >> > > > > did manage to come to consensus on everything but LockinOnTimeout.
> >> > > > >
> >> > > > > Activation height range: 693504-745920
> >> > > > >
> >> > > > > MASF threshold: 1815/2016 blocks (90%)
> >> > > > >
> >> > > > > Keep in mind only ~100 people showed for the meetings, hardly
> >> > > > > representative of the entire community.
> >> > > > >
> >> > > > > So, these details remain JUST a proposal for now.
> >> > > > >
> >> > > > > It seems inevitable that there won't be consensus on LOT.
> >> > > > >
> >> > > > > Everyone will have to choose for himself. :/
> >> > > > >
> >> > > > > Personally I agree with most of this. I agree that there wasn’t
> >> > > > > overwhelming consensus for either LOT=true or LOT=false. However, from
> >> > > > > my perspective there was clearly more strong opposition (what would
> >> > > > > usually be deemed a NACK in Bitcoin Core review terminology) from
> >> > > > > Bitcoin Core contributors, Lightning developers and other community
> >> > > > > members against LOT=true than there was for LOT=false. Andrew Chow
> >> > > > > tried to summarize views from the meeting in this analysis:
> >> > > > > https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c
> <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c>;
> >> <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c
> <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c>>;
> >> > > > >
> >> > > > > I am also aware of other current and previous Bitcoin Core
> >> > > > > contributors and Lightning developers who didn’t attend the meeting in
> >> > > > > person who are opposed to LOT=true. I don’t want to put them in the
> >> > > > > spotlight for no reason but if you go through the conversation logs of
> >> > > > > not only the meeting but the weeks of discussion prior to this meeting
> >> > > > > you will see their views evaluated on the ##taproot-activation
> >> > > > > channel. In addition, on taprootactivation.com <http://taprootactivation.com>;
> <http://taprootactivation.com <http://taprootactivation.com>>; some mining pools
> >> > > > > expressed a preference for lot=false though I don’t know how strong
> >> > > > > that preference was.
> >> > > > >
> >> > > > > I am only one voice but it is my current assessment that if we are to
> >> > > > > attempt to finalize Taproot activation parameters and propose them to
> >> > > > > the community at this time our only option is to propose LOT=false.
> >> > > > > Any further delay appears to me counterproductive in our collective
> >> > > > > aim to get the Taproot soft fork activated as early as possible.
> >> > > > >
> >> > > > > Obviously others are free to disagree with that assessment and
> >> > > > > continue discussions but personally I will be attempting to avoid
> >> > > > > those discussions unless prominent new information comes to light or
> >> > > > > various specific individuals change their minds.
> >> > > > >
> >> > > > > Next week we are planning a code review of the Bitcoin Core PR #19573
> >> > > > > which was initially delayed because of this LOT discussion. As I’ve
> >> > > > > said previously that will be loosely following the format of the
> >> > > > > Bitcoin Core PR review club and will be lower level and more
> >> > > > > technical. That is planned for Tuesday February 23rd at 19:00 UTC on
> >> > > > > the IRC channel ##taproot-activation.
> >> > > > >
> >> > > > > Thanks to the meeting participants (and those who joined the
> >> > > > > discussion on the channel prior and post the meeting) for engaging
> >> > > > > productively and in good faith.
> >> > >
> >> > > --
> >> > > Michael Folkson
> >> > > Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com> <mailto:michaelfolkson at gmail.com
> <mailto:michaelfolkson at gmail.com>>
> >> > > Keybase: michaelfolkson
> >> > > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
> >> > > _______________________________________________
> >> > > bitcoin-dev mailing list
> >> > > bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>
> >> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
> >> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>>;
> >>
> >>
> >>
> >>
> >> --
> >> Michael Folkson
> >> Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com> <mailto:michaelfolkson at gmail.com
> <mailto:michaelfolkson at gmail.com>>
> >> Keybase: michaelfolkson
> >> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
> >> _______________________________________________
> >> bitcoin-dev mailing list
> >> bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>
> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
> >> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>>;
> >
> >
> >
> > --
> > Michael Folkson
> > Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com> <mailto:michaelfolkson at gmail.com
> <mailto:michaelfolkson at gmail.com>>
> > Keybase: michaelfolkson
> > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
>

📅 Original date posted:2021-02-18 📝 Original message:To ...

2023-06-07T20:28:51+02:00

In reply to nevent1q…s9me
_________________________

📅 Original date posted:2021-02-18
📝 Original message:To ensure we're on the same page, here - I'm not advocating we give up on Taproot. Indeed, without having dug deep into
the issue, my overall impression is that Knots has a tiny transaction-processing userbase and it likely isn't worth
giving deep thought to whether it forks itself off from the network or not. My point is that, if it were the case that
various implementations of Bitcoin's consensus that have material userbases were to release either a configurable
consensus mechanism (without incredible care being given to it, not just a "we can't decide, whatever" argument) or a
different consensus, we'd be much, much better off not having Taproot at all.

Matt

On 2/18/21 09:53, Matt Corallo via bitcoin-dev wrote:
> You say "short term PR", I say "risking millions of user dollars".
>
> On 2/18/21 09:51, Michael Folkson wrote:
>> > getting unlucky and hitting a 4-block reorg that happens to include a double-spend and some PR around an exchange
>> losing millions would be worse than having Taproot is good.
>>
>> We are at the point where an upgrade that confers significant long term benefits for the whole ecosystem is not as
>> important as bad short term PR? That is a depressing outlook if that is what you believe.
>>
>> Even in that worst case scenario exchanges should not lose money if they are competent and are able to manage that risk.
>>
>> On Thu, Feb 18, 2021 at 2:42 PM Matt Corallo <lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>> wrote:
>>
>>     We've had several softforks in Bitcoin which, through the course of their activation, had a several-block reorg. That
>>     should be indication enough that we need to very carefully consider activation to ensure we reduce the risk of
>> that as
>>     much as absolutely possible. Again, while I think Taproot is a huge improvement and am looking forward to being
>> able to
>>     use it, getting unlucky and hitting a 4-block reorg that happens to include a double-spend and some PR around an
>>     exchange losing millions would be worse than having Taproot is good.
>>
>>     Matt
>>
>>     On 2/18/21 09:26, Michael Folkson wrote:
>>      > Thanks for your response Matt. It is a fair challenge. There is always going to be an element of risk with soft
>>     forks,
>>      > all we can do is attempt to minimize that risk. I would argue that risk has been minimized for Taproot.
>>      >
>>      > You know (better than I do in fact) that Bitcoin (and layers built on top of it) greatly benefit from upgrades
>>     such as
>>      > Taproot. To say we shouldn't do Taproot or any future soft forks because there is a small but real risk of chain
>>     splits
>>      > I think is shortsighted. Indeed I think even if we collectively decided not to do any future soft fork upgrades
>> ever
>>      > again on this mailing list that wouldn't stop soft fork attempts from other people in future.
>>      >
>>      > I don't think there is anything else we can do to minimize that risk for the Taproot soft fork at this point
>>     though I'm
>>      > open to ideas. To reiterate that risk will never be zero. I don't think I see Bitcoin as fragile as you seem to
>>     (though
>>      > admittedly you have a much better understanding than me of what happened in 2017).
>>      >
>>      > The likely scenario for the Taproot soft fork is LOT turns out to be entirely irrelevant and miners activate
>> Taproot
>>      > before it becomes relevant. And even the unlikely worst case scenario would only cause short term disruption and
>>      > wouldn't kill Bitcoin long term.
>>      >
>>      > On Thu, Feb 18, 2021 at 2:01 PM Matt Corallo <lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>
>>     <mailto:lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>>> wrote:
>>      >
>>      > If the eventual outcome is that different implementations (that have material *transaction processing*
>> userbases,
>>      > and I’m not sure to what extent that’s true with Knots) ship different consensus rules, we should stop here
>>     and not
>>      > activate Taproot. Seriously.
>>      >
>>      > Bitcoin is a consensus system. The absolute worst outcome at all possible is to have it fall out of consensus.
>>      >
>>      > Matt
>>      >
>>      >> On Feb 18, 2021, at 08:11, Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
>>     <mailto:bitcoin-dev at lists.linuxfoundation.org>
>>      >> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>> wrote:
>>      >>
>>      >>
>>      >> Right, that is one option. Personally I would prefer a Bitcoin Core release sets LOT=false (based on what
>> I have
>>      >> heard from Bitcoin Core contributors) and a community effort releases a version with LOT=true. I don't think
>>     users
>>      >> should be forced to choose something they may have no context on before they are allowed to use Bitcoin Core.
>>      >>
>>      >> My current understanding is that roasbeef is planning to set LOT=false on btcd (an alternative protocol
>>      >> implementation to Bitcoin Core) and Luke Dashjr hasn't yet decided on Bitcoin Knots.
>>      >>
>>      >>
>>      >>
>>      >> On Thu, Feb 18, 2021 at 11:52 AM ZmnSCPxj <ZmnSCPxj at protonmail.com <mailto:ZmnSCPxj at protonmail.com>
>>     <mailto:ZmnSCPxj at protonmail.com <mailto:ZmnSCPxj at protonmail.com>>> wrote:
>>      >>
>>      >> Good morning all,
>>      >>
>>      >> > "An activation mechanism is a consensus change like any other change, can be contentious like any other
>>      >> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest
>> timeline."
>>      >> >
>>      >> > Who's we here?
>>      >> >
>>      >> > Release both and let the network decide.
>>      >>
>>      >> A thing that could be done, without mandating either LOT=true or LOT=false, would be to have a release
>> that
>>      >> requires a `taprootlot=1` or `taprootlot=0` and refuses to start if the parameter is not set.
>>      >>
>>      >> This assures everyone that neither choice is being forced on users, and instead what is being forced on
>>     users,
>>      >> is for users to make that choice themselves.
>>      >>
>>      >> Regards,
>>      >> ZmnSCPxj
>>      >>
>>      >> >
>>      >> > On Thu, Feb 18, 2021 at 3:08 AM Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
>>     <mailto:bitcoin-dev at lists.linuxfoundation.org>
>>      >> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>> wrote:
>>      >> >
>>      >> > > Thanks for your response Ariel. It would be useful if you responded to specific points I have made
>>     in the
>>      >> mailing list post or at least quote these ephemeral "people" you speak of. I don't know if you're
>> responding
>>      >> to conversation on the IRC channel or on social media etc.
>>      >> > >
>>      >> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted
>>     into
>>      >> code. But in fact this isn't true and some voices in this discussion need to be more humble about what
>> users
>>      >> must or must not run.
>>      >> > >
>>      >> > > I personally have never made this assumption. Of course users aren't forced to run any particular
>>     software
>>      >> version, quite the opposite. Defaults set in software versions matter though as many users won't change
>>     them.
>>      >> > >
>>      >> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be
>>     only a
>>      >> handful of people that begin running it while everyone else delays their upgrade (with the very good
>>     reason of
>>      >> not getting involved in politics) and a year later those handful of people just become stuck at the
>>     moment of
>>      >> MUST_SIGNAL, unable to mine new blocks?
>>      >> > >
>>      >> > > It is a possible outcome but the likely outcome is that miners activate Taproot before LOT is even
>>      >> relevant. I think it is prudent to prepare for the unlikely but possible outcome that miners fail to
>>     activate
>>      >> and hence have this discussion now rather than be unprepared for that eventuality. If LOT is set to
>>     false in a
>>      >> software release there is the possibility (T2 in
>>      >> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
>>     <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>;
>>      >> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
>>     <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>>;) of individuals or a
>>      >> proportion of the community changing LOT to true. In that sense setting LOT=false in a software release
>>      >> appears to be no more safe than LOT=true.
>>      >> > >
>>      >> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with
>>     miners
>>      >> by default.
>>      >> > >
>>      >> > > There is the (unlikely but possible) possibility of a wasted year if LOT is set to false and
>> miners fail
>>      >> to activate. I'm not convinced by this perception that LOT=true is antagonistic to miners. I actually
>>     think it
>>      >> offers them clarity on what will happen over a year time period and removes the need for coordinated or
>>      >> uncoordinated community UASF efforts on top of LOT=false.
>>      >> > >
>>      >> > > > An activation mechanism is a consensus change like any other change, can be contentious like any
>> other
>>      >> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
>>      >> > >
>>      >> > > I don't know what you are recommending here to avoid "this darkest timeline". Open discussions have
>>      >> occurred and are continuing and in my mailing list post that you responded to **I recommended we propose
>>      >> LOT=false be set in protocol implementations such as Bitcoin Core**. I do think this apocalyptic language
>>      >> isn't particularly helpful. In an open consensus system discussion is healthy, we should prepare for
>> bad or
>>      >> worst case scenarios in advance and doing so is not antagonistic or destructive. Mining pools have
>> pledged
>>      >> support for Taproot but we don't build secure systems based on pledges of support, we build them to
>> minimize
>>      >> trust in any human actors. We can be grateful that people like Alejandro have worked hard on
>>      >> taprootactivation.com <http://taprootactivation.com>; <http://taprootactivation.com
>>     <http://taprootactivation.com>>; (and this effort has informed the discussion) without
>>      >> taking pledges of support as cast iron guarantees.
>>      >> > >
>>      >> > > TL;DR It sounds like you agree with my recommendation to set LOT=false in protocol implementations
>> in my
>>      >> email :)
>>      >> > >
>>      >> > > On Thu, Feb 18, 2021 at 5:43 AM Ariel Lorenzo-Luaces <arielluaces at gmail.com
>>     <mailto:arielluaces at gmail.com>
>>      >> <mailto:arielluaces at gmail.com <mailto:arielluaces at gmail.com>>> wrote:
>>      >> > >
>>      >> > > > Something what strikes me about the conversation is the emotion surrounding the letters UASF.
>>      >> > > > It appears as if people discuss UASF as if it's a massive tidal wave of support that is
>>     inevitable, like
>>      >> we saw during segwit activation. But the actual definition is "any activation that is not a MASF".
>>      >> > > > A UASF can consist of a single node, ten nodes, a thousand, half of all nodes, all business'
>> nodes, or
>>      >> even all the non mining nodes. On another dimension it can have zero mining support, 51% support, 49%
>>     support,
>>      >> or any support right up against a miner activation threshold.
>>      >> > > > Hell a UASF doesn't even need code or even a single node running as long as it exists as a
>> possibility
>>      >> in people's minds.
>>      >> > > > The only thing a UASF doesn't have is miner support above an agreed activation threshold (some
>> number
>>      >> above %51).
>>      >> > > > I say this because it strikes me when people say that they are for LOT=true with the logic that
>>     since a
>>      >> UASF is guaranteed to happen then it's better to just make it default from the beginning. Words like
>>      >> coordination and safety are sometimes sprinkled into the argument.
>>      >> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted
>>     into
>>      >> code. But in fact this isn't true and some voices in this discussion need to be more humble about what
>> users
>>      >> must or must not run.
>>      >> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be
>>     only a
>>      >> handful of people that begin running it while everyone else delays their upgrade (with the very good
>>     reason of
>>      >> not getting involved in politics) and a year later those handful of people just become stuck at the
>>     moment of
>>      >> MUST_SIGNAL, unable to mine new blocks? Or attracting a minority of miners, activating, and forking off
>>     into a
>>      >> minority fork. Then a lot=false could be started that ends up activating the feature now that the
>> stubborn
>>      >> option has ran its course.
>>      >> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with
>>     miners
>>      >> by default. The chains could be called BitcoinLenient and BitcoinStubborn.
>>      >> > > > How is that strictly safer or more coordinated?
>>      >> > > > I may be in the minority, or maybe a silent majority, or maybe a majority that just hasn't
>> considered
>>      >> this as a choice but honestly if there is contention about whether we're going to be stubborn or lenient
>>     with
>>      >> miners for Taproot and in the future then I prefer to just not activate anything at all. I'm fine for
>>     calling
>>      >> bitcoin ossified, accepting that segwit is Bitcoin's last network upgrade. Taproot is amazing but no new
>>      >> feature is worth a network split down the middle.
>>      >> > > > Maybe in 10 or 20 years, when other blockchains implement features like Taproot and many more,
>> we will
>>      >> become envious enough to put aside our differences on how to behave towards miners and finally activate
>>     Taproot.
>>      >> > > > An activation mechanism is a consensus change like any other change, can be contentious like any
>> other
>>      >> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
>>      >> > > > Cheers
>>      >> > > > Ariel Lorenzo-Luaces
>>      >> > > > On Feb 17, 2021, at 7:05 AM, Michael Folkson via bitcoin-dev
>>     <bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
>>      >> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>> wrote:
>>      >> > > >
>>      >> > > > > Yesterday (February 16th) we held a second meeting on Taproot
>>      >> > > > > activation on IRC which again was open to all. Despite what appeared
>>      >> > > > > to be majority support for LOT=false over LOT=true in the first
>>      >> > > > > meeting I (and others) thought the arguments had not been explored in
>>      >> > > > > depth and that we should have a follow up meeting almost entirely
>>      >> > > > > focused on whether LOT (lockinontimeout) should be set to true or
>>      >> > > > > false.
>>      >> > > > >
>>      >> > > > > The meeting was announced here:
>>      >> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
>>     <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>;
>>      >> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
>>     <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>>;
>>      >> > > > >
>>      >> > > > > In that mailing list post I outlined the arguments for LOT=true (T1 to
>>      >> > > > > T6) and arguments for LOT=false (F1 to F6) in their strongest form I
>>      >> > > > > could. David Harding responded with an additional argument for
>>      >> > > > > LOT=false (F7) here:
>>      >> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html
>>     <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html>;
>>      >> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html
>>     <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html>>;
>>      >> > > > >
>>      >> > > > > These meetings are very challenging given they are open to all, you
>>      >> > > > > don’t know who will attend and you don’t know most people’s views in
>>      >> > > > > advance. I tried to give time for both the LOT=true arguments and the
>>      >> > > > > LOT=false arguments to be discussed as I knew there was support for
>>      >> > > > > both. We only tried evaluating which had more support and which had
>>      >> > > > > more strong opposition towards the end of the meeting.
>>      >> > > > >
>>      >> > > > > The conversation log is here:
>>      >> > > > > http://gnusha.org/taproot-activation/2021-02-16.log
>>     <http://gnusha.org/taproot-activation/2021-02-16.log>; <http://gnusha.org/taproot-activation/2021-02-16.log
>>     <http://gnusha.org/taproot-activation/2021-02-16.log>>;
>>      >> > > > >
>>      >> > > > > (If you are so inclined you can watch a video of the meeting here.
>>      >> > > > > Thanks to the YouTube account “Bitcoin” for setting up the livestream:
>>      >> > > > > https://www.youtube.com/watch?v=vpl5q1ovMLM <https://www.youtube.com/watch?v=vpl5q1ovMLM>;
>>     <https://www.youtube.com/watch?v=vpl5q1ovMLM <https://www.youtube.com/watch?v=vpl5q1ovMLM>>;)
>>      >> > > > >
>>      >> > > > > A summary of the meeting was provided by Luke Dashjr on Mastodon here:
>>      >> > > > > https://bitcoinhackers.org/@lukedashjr/105742918779234566
>>     <https://bitcoinhackers.org/@lukedashjr/105742918779234566>;
>>      >> <https://bitcoinhackers.org/@lukedashjr/105742918779234566
>>     <https://bitcoinhackers.org/@lukedashjr/105742918779234566>>;
>>      >> > > > >
>>      >> > > > > Today's #Bitcoin #Taproot meeting was IMO largely unproductive, but we
>>      >> > > > > did manage to come to consensus on everything but LockinOnTimeout.
>>      >> > > > >
>>      >> > > > > Activation height range: 693504-745920
>>      >> > > > >
>>      >> > > > > MASF threshold: 1815/2016 blocks (90%)
>>      >> > > > >
>>      >> > > > > Keep in mind only ~100 people showed for the meetings, hardly
>>      >> > > > > representative of the entire community.
>>      >> > > > >
>>      >> > > > > So, these details remain JUST a proposal for now.
>>      >> > > > >
>>      >> > > > > It seems inevitable that there won't be consensus on LOT.
>>      >> > > > >
>>      >> > > > > Everyone will have to choose for himself. :/
>>      >> > > > >
>>      >> > > > > Personally I agree with most of this. I agree that there wasn’t
>>      >> > > > > overwhelming consensus for either LOT=true or LOT=false. However, from
>>      >> > > > > my perspective there was clearly more strong opposition (what would
>>      >> > > > > usually be deemed a NACK in Bitcoin Core review terminology) from
>>      >> > > > > Bitcoin Core contributors, Lightning developers and other community
>>      >> > > > > members against LOT=true than there was for LOT=false. Andrew Chow
>>      >> > > > > tried to summarize views from the meeting in this analysis:
>>      >> > > > > https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c
>>     <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c>;
>>      >> <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c
>>     <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c>>;
>>      >> > > > >
>>      >> > > > > I am also aware of other current and previous Bitcoin Core
>>      >> > > > > contributors and Lightning developers who didn’t attend the meeting in
>>      >> > > > > person who are opposed to LOT=true. I don’t want to put them in the
>>      >> > > > > spotlight for no reason but if you go through the conversation logs of
>>      >> > > > > not only the meeting but the weeks of discussion prior to this meeting
>>      >> > > > > you will see their views evaluated on the ##taproot-activation
>>      >> > > > > channel. In addition, on taprootactivation.com <http://taprootactivation.com>;
>>     <http://taprootactivation.com <http://taprootactivation.com>>; some mining pools
>>      >> > > > > expressed a preference for lot=false though I don’t know how strong
>>      >> > > > > that preference was.
>>      >> > > > >
>>      >> > > > > I am only one voice but it is my current assessment that if we are to
>>      >> > > > > attempt to finalize Taproot activation parameters and propose them to
>>      >> > > > > the community at this time our only option is to propose LOT=false.
>>      >> > > > > Any further delay appears to me counterproductive in our collective
>>      >> > > > > aim to get the Taproot soft fork activated as early as possible.
>>      >> > > > >
>>      >> > > > > Obviously others are free to disagree with that assessment and
>>      >> > > > > continue discussions but personally I will be attempting to avoid
>>      >> > > > > those discussions unless prominent new information comes to light or
>>      >> > > > > various specific individuals change their minds.
>>      >> > > > >
>>      >> > > > > Next week we are planning a code review of the Bitcoin Core PR #19573
>>      >> > > > > which was initially delayed because of this LOT discussion. As I’ve
>>      >> > > > > said previously that will be loosely following the format of the
>>      >> > > > > Bitcoin Core PR review club and will be lower level and more
>>      >> > > > > technical. That is planned for Tuesday February 23rd at 19:00 UTC on
>>      >> > > > > the IRC channel ##taproot-activation.
>>      >> > > > >
>>      >> > > > > Thanks to the meeting participants (and those who joined the
>>      >> > > > > discussion on the channel prior and post the meeting) for engaging
>>      >> > > > > productively and in good faith.
>>      >> > >
>>      >> > > --
>>      >> > > Michael Folkson
>>      >> > > Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com> <mailto:michaelfolkson at gmail.com
>>     <mailto:michaelfolkson at gmail.com>>
>>      >> > > Keybase: michaelfolkson
>>      >> > > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
>>      >> > > _______________________________________________
>>      >> > > bitcoin-dev mailing list
>>      >> > > bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
>>     <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>
>>      >> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>     <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
>>      >> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>     <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>>;
>>      >>
>>      >>
>>      >>
>>      >>
>>      >> --
>>      >> Michael Folkson
>>      >> Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com> <mailto:michaelfolkson at gmail.com
>>     <mailto:michaelfolkson at gmail.com>>
>>      >> Keybase: michaelfolkson
>>      >> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
>>      >> _______________________________________________
>>      >> bitcoin-dev mailing list
>>      >> bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
>>     <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>
>>      >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>     <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
>>      >> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>     <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>>;
>>      >
>>      >
>>      >
>>      > --
>>      > Michael Folkson
>>      > Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com> <mailto:michaelfolkson at gmail.com
>>     <mailto:michaelfolkson at gmail.com>>
>>      > Keybase: michaelfolkson
>>      > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
>>
>>
>>
>> --
>> Michael Folkson
>> Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com>
>> Keybase: michaelfolkson
>> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

📅 Original date posted:2021-02-18 📝 Original message:You ...

2023-06-07T20:28:50+02:00

In reply to nevent1q…f6cr
_________________________

📅 Original date posted:2021-02-18
📝 Original message:You say "short term PR", I say "risking millions of user dollars".

On 2/18/21 09:51, Michael Folkson wrote:
> > getting unlucky and hitting a 4-block reorg that happens to include a double-spend and some PR around an exchange
> losing millions would be worse than having Taproot is good.
>
> We are at the point where an upgrade that confers significant long term benefits for the whole ecosystem is not as
> important as bad short term PR? That is a depressing outlook if that is what you believe.
>
> Even in that worst case scenario exchanges should not lose money if they are competent and are able to manage that risk.
>
> On Thu, Feb 18, 2021 at 2:42 PM Matt Corallo <lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>> wrote:
>
> We've had several softforks in Bitcoin which, through the course of their activation, had a several-block reorg. That
> should be indication enough that we need to very carefully consider activation to ensure we reduce the risk of that as
> much as absolutely possible. Again, while I think Taproot is a huge improvement and am looking forward to being able to
> use it, getting unlucky and hitting a 4-block reorg that happens to include a double-spend and some PR around an
> exchange losing millions would be worse than having Taproot is good.
>
> Matt
>
> On 2/18/21 09:26, Michael Folkson wrote:
> > Thanks for your response Matt. It is a fair challenge. There is always going to be an element of risk with soft
> forks,
> > all we can do is attempt to minimize that risk. I would argue that risk has been minimized for Taproot.
> >
> > You know (better than I do in fact) that Bitcoin (and layers built on top of it) greatly benefit from upgrades
> such as
> > Taproot. To say we shouldn't do Taproot or any future soft forks because there is a small but real risk of chain
> splits
> > I think is shortsighted. Indeed I think even if we collectively decided not to do any future soft fork upgrades ever
> > again on this mailing list that wouldn't stop soft fork attempts from other people in future.
> >
> > I don't think there is anything else we can do to minimize that risk for the Taproot soft fork at this point
> though I'm
> > open to ideas. To reiterate that risk will never be zero. I don't think I see Bitcoin as fragile as you seem to
> (though
> > admittedly you have a much better understanding than me of what happened in 2017).
> >
> > The likely scenario for the Taproot soft fork is LOT turns out to be entirely irrelevant and miners activate Taproot
> > before it becomes relevant. And even the unlikely worst case scenario would only cause short term disruption and
> > wouldn't kill Bitcoin long term.
> >
> > On Thu, Feb 18, 2021 at 2:01 PM Matt Corallo <lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>
> <mailto:lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>>> wrote:
> >
> > If the eventual outcome is that different implementations (that have material *transaction processing* userbases,
> > and I’m not sure to what extent that’s true with Knots) ship different consensus rules, we should stop here
> and not
> > activate Taproot. Seriously.
> >
> > Bitcoin is a consensus system. The absolute worst outcome at all possible is to have it fall out of consensus.
> >
> > Matt
> >
> >> On Feb 18, 2021, at 08:11, Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> >> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>> wrote:
> >>
> >>
> >> Right, that is one option. Personally I would prefer a Bitcoin Core release sets LOT=false (based on what I have
> >> heard from Bitcoin Core contributors) and a community effort releases a version with LOT=true. I don't think
> users
> >> should be forced to choose something they may have no context on before they are allowed to use Bitcoin Core.
> >>
> >> My current understanding is that roasbeef is planning to set LOT=false on btcd (an alternative protocol
> >> implementation to Bitcoin Core) and Luke Dashjr hasn't yet decided on Bitcoin Knots.
> >>
> >>
> >>
> >> On Thu, Feb 18, 2021 at 11:52 AM ZmnSCPxj <ZmnSCPxj at protonmail.com <mailto:ZmnSCPxj at protonmail.com>
> <mailto:ZmnSCPxj at protonmail.com <mailto:ZmnSCPxj at protonmail.com>>> wrote:
> >>
> >> Good morning all,
> >>
> >> > "An activation mechanism is a consensus change like any other change, can be contentious like any other
> >> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline."
> >> >
> >> > Who's we here?
> >> >
> >> > Release both and let the network decide.
> >>
> >> A thing that could be done, without mandating either LOT=true or LOT=false, would be to have a release that
> >> requires a `taprootlot=1` or `taprootlot=0` and refuses to start if the parameter is not set.
> >>
> >> This assures everyone that neither choice is being forced on users, and instead what is being forced on
> users,
> >> is for users to make that choice themselves.
> >>
> >> Regards,
> >> ZmnSCPxj
> >>
> >> >
> >> > On Thu, Feb 18, 2021 at 3:08 AM Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> >> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>> wrote:
> >> >
> >> > > Thanks for your response Ariel. It would be useful if you responded to specific points I have made
> in the
> >> mailing list post or at least quote these ephemeral "people" you speak of. I don't know if you're responding
> >> to conversation on the IRC channel or on social media etc.
> >> > >
> >> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted
> into
> >> code. But in fact this isn't true and some voices in this discussion need to be more humble about what users
> >> must or must not run.
> >> > >
> >> > > I personally have never made this assumption. Of course users aren't forced to run any particular
> software
> >> version, quite the opposite. Defaults set in software versions matter though as many users won't change
> them.
> >> > >
> >> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be
> only a
> >> handful of people that begin running it while everyone else delays their upgrade (with the very good
> reason of
> >> not getting involved in politics) and a year later those handful of people just become stuck at the
> moment of
> >> MUST_SIGNAL, unable to mine new blocks?
> >> > >
> >> > > It is a possible outcome but the likely outcome is that miners activate Taproot before LOT is even
> >> relevant. I think it is prudent to prepare for the unlikely but possible outcome that miners fail to
> activate
> >> and hence have this discussion now rather than be unprepared for that eventuality. If LOT is set to
> false in a
> >> software release there is the possibility (T2 in
> >> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>;
> >> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>>;) of individuals or a
> >> proportion of the community changing LOT to true. In that sense setting LOT=false in a software release
> >> appears to be no more safe than LOT=true.
> >> > >
> >> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with
> miners
> >> by default.
> >> > >
> >> > > There is the (unlikely but possible) possibility of a wasted year if LOT is set to false and miners fail
> >> to activate. I'm not convinced by this perception that LOT=true is antagonistic to miners. I actually
> think it
> >> offers them clarity on what will happen over a year time period and removes the need for coordinated or
> >> uncoordinated community UASF efforts on top of LOT=false.
> >> > >
> >> > > > An activation mechanism is a consensus change like any other change, can be contentious like any other
> >> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
> >> > >
> >> > > I don't know what you are recommending here to avoid "this darkest timeline". Open discussions have
> >> occurred and are continuing and in my mailing list post that you responded to **I recommended we propose
> >> LOT=false be set in protocol implementations such as Bitcoin Core**. I do think this apocalyptic language
> >> isn't particularly helpful. In an open consensus system discussion is healthy, we should prepare for bad or
> >> worst case scenarios in advance and doing so is not antagonistic or destructive. Mining pools have pledged
> >> support for Taproot but we don't build secure systems based on pledges of support, we build them to minimize
> >> trust in any human actors. We can be grateful that people like Alejandro have worked hard on
> >> taprootactivation.com <http://taprootactivation.com>; <http://taprootactivation.com
> <http://taprootactivation.com>>; (and this effort has informed the discussion) without
> >> taking pledges of support as cast iron guarantees.
> >> > >
> >> > > TL;DR It sounds like you agree with my recommendation to set LOT=false in protocol implementations in my
> >> email :)
> >> > >
> >> > > On Thu, Feb 18, 2021 at 5:43 AM Ariel Lorenzo-Luaces <arielluaces at gmail.com
> <mailto:arielluaces at gmail.com>
> >> <mailto:arielluaces at gmail.com <mailto:arielluaces at gmail.com>>> wrote:
> >> > >
> >> > > > Something what strikes me about the conversation is the emotion surrounding the letters UASF.
> >> > > > It appears as if people discuss UASF as if it's a massive tidal wave of support that is
> inevitable, like
> >> we saw during segwit activation. But the actual definition is "any activation that is not a MASF".
> >> > > > A UASF can consist of a single node, ten nodes, a thousand, half of all nodes, all business' nodes, or
> >> even all the non mining nodes. On another dimension it can have zero mining support, 51% support, 49%
> support,
> >> or any support right up against a miner activation threshold.
> >> > > > Hell a UASF doesn't even need code or even a single node running as long as it exists as a possibility
> >> in people's minds.
> >> > > > The only thing a UASF doesn't have is miner support above an agreed activation threshold (some number
> >> above %51).
> >> > > > I say this because it strikes me when people say that they are for LOT=true with the logic that
> since a
> >> UASF is guaranteed to happen then it's better to just make it default from the beginning. Words like
> >> coordination and safety are sometimes sprinkled into the argument.
> >> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted
> into
> >> code. But in fact this isn't true and some voices in this discussion need to be more humble about what users
> >> must or must not run.
> >> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be
> only a
> >> handful of people that begin running it while everyone else delays their upgrade (with the very good
> reason of
> >> not getting involved in politics) and a year later those handful of people just become stuck at the
> moment of
> >> MUST_SIGNAL, unable to mine new blocks? Or attracting a minority of miners, activating, and forking off
> into a
> >> minority fork. Then a lot=false could be started that ends up activating the feature now that the stubborn
> >> option has ran its course.
> >> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with
> miners
> >> by default. The chains could be called BitcoinLenient and BitcoinStubborn.
> >> > > > How is that strictly safer or more coordinated?
> >> > > > I may be in the minority, or maybe a silent majority, or maybe a majority that just hasn't considered
> >> this as a choice but honestly if there is contention about whether we're going to be stubborn or lenient
> with
> >> miners for Taproot and in the future then I prefer to just not activate anything at all. I'm fine for
> calling
> >> bitcoin ossified, accepting that segwit is Bitcoin's last network upgrade. Taproot is amazing but no new
> >> feature is worth a network split down the middle.
> >> > > > Maybe in 10 or 20 years, when other blockchains implement features like Taproot and many more, we will
> >> become envious enough to put aside our differences on how to behave towards miners and finally activate
> Taproot.
> >> > > > An activation mechanism is a consensus change like any other change, can be contentious like any other
> >> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
> >> > > > Cheers
> >> > > > Ariel Lorenzo-Luaces
> >> > > > On Feb 17, 2021, at 7:05 AM, Michael Folkson via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
> >> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>> wrote:
> >> > > >
> >> > > > > Yesterday (February 16th) we held a second meeting on Taproot
> >> > > > > activation on IRC which again was open to all. Despite what appeared
> >> > > > > to be majority support for LOT=false over LOT=true in the first
> >> > > > > meeting I (and others) thought the arguments had not been explored in
> >> > > > > depth and that we should have a follow up meeting almost entirely
> >> > > > > focused on whether LOT (lockinontimeout) should be set to true or
> >> > > > > false.
> >> > > > >
> >> > > > > The meeting was announced here:
> >> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>;
> >> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>>;
> >> > > > >
> >> > > > > In that mailing list post I outlined the arguments for LOT=true (T1 to
> >> > > > > T6) and arguments for LOT=false (F1 to F6) in their strongest form I
> >> > > > > could. David Harding responded with an additional argument for
> >> > > > > LOT=false (F7) here:
> >> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html>;
> >> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html
> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html>>;
> >> > > > >
> >> > > > > These meetings are very challenging given they are open to all, you
> >> > > > > don’t know who will attend and you don’t know most people’s views in
> >> > > > > advance. I tried to give time for both the LOT=true arguments and the
> >> > > > > LOT=false arguments to be discussed as I knew there was support for
> >> > > > > both. We only tried evaluating which had more support and which had
> >> > > > > more strong opposition towards the end of the meeting.
> >> > > > >
> >> > > > > The conversation log is here:
> >> > > > > http://gnusha.org/taproot-activation/2021-02-16.log
> <http://gnusha.org/taproot-activation/2021-02-16.log>; <http://gnusha.org/taproot-activation/2021-02-16.log
> <http://gnusha.org/taproot-activation/2021-02-16.log>>;
> >> > > > >
> >> > > > > (If you are so inclined you can watch a video of the meeting here.
> >> > > > > Thanks to the YouTube account “Bitcoin” for setting up the livestream:
> >> > > > > https://www.youtube.com/watch?v=vpl5q1ovMLM <https://www.youtube.com/watch?v=vpl5q1ovMLM>;
> <https://www.youtube.com/watch?v=vpl5q1ovMLM <https://www.youtube.com/watch?v=vpl5q1ovMLM>>;)
> >> > > > >
> >> > > > > A summary of the meeting was provided by Luke Dashjr on Mastodon here:
> >> > > > > https://bitcoinhackers.org/@lukedashjr/105742918779234566
> <https://bitcoinhackers.org/@lukedashjr/105742918779234566>;
> >> <https://bitcoinhackers.org/@lukedashjr/105742918779234566
> <https://bitcoinhackers.org/@lukedashjr/105742918779234566>>;
> >> > > > >
> >> > > > > Today's #Bitcoin #Taproot meeting was IMO largely unproductive, but we
> >> > > > > did manage to come to consensus on everything but LockinOnTimeout.
> >> > > > >
> >> > > > > Activation height range: 693504-745920
> >> > > > >
> >> > > > > MASF threshold: 1815/2016 blocks (90%)
> >> > > > >
> >> > > > > Keep in mind only ~100 people showed for the meetings, hardly
> >> > > > > representative of the entire community.
> >> > > > >
> >> > > > > So, these details remain JUST a proposal for now.
> >> > > > >
> >> > > > > It seems inevitable that there won't be consensus on LOT.
> >> > > > >
> >> > > > > Everyone will have to choose for himself. :/
> >> > > > >
> >> > > > > Personally I agree with most of this. I agree that there wasn’t
> >> > > > > overwhelming consensus for either LOT=true or LOT=false. However, from
> >> > > > > my perspective there was clearly more strong opposition (what would
> >> > > > > usually be deemed a NACK in Bitcoin Core review terminology) from
> >> > > > > Bitcoin Core contributors, Lightning developers and other community
> >> > > > > members against LOT=true than there was for LOT=false. Andrew Chow
> >> > > > > tried to summarize views from the meeting in this analysis:
> >> > > > > https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c
> <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c>;
> >> <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c
> <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c>>;
> >> > > > >
> >> > > > > I am also aware of other current and previous Bitcoin Core
> >> > > > > contributors and Lightning developers who didn’t attend the meeting in
> >> > > > > person who are opposed to LOT=true. I don’t want to put them in the
> >> > > > > spotlight for no reason but if you go through the conversation logs of
> >> > > > > not only the meeting but the weeks of discussion prior to this meeting
> >> > > > > you will see their views evaluated on the ##taproot-activation
> >> > > > > channel. In addition, on taprootactivation.com <http://taprootactivation.com>;
> <http://taprootactivation.com <http://taprootactivation.com>>; some mining pools
> >> > > > > expressed a preference for lot=false though I don’t know how strong
> >> > > > > that preference was.
> >> > > > >
> >> > > > > I am only one voice but it is my current assessment that if we are to
> >> > > > > attempt to finalize Taproot activation parameters and propose them to
> >> > > > > the community at this time our only option is to propose LOT=false.
> >> > > > > Any further delay appears to me counterproductive in our collective
> >> > > > > aim to get the Taproot soft fork activated as early as possible.
> >> > > > >
> >> > > > > Obviously others are free to disagree with that assessment and
> >> > > > > continue discussions but personally I will be attempting to avoid
> >> > > > > those discussions unless prominent new information comes to light or
> >> > > > > various specific individuals change their minds.
> >> > > > >
> >> > > > > Next week we are planning a code review of the Bitcoin Core PR #19573
> >> > > > > which was initially delayed because of this LOT discussion. As I’ve
> >> > > > > said previously that will be loosely following the format of the
> >> > > > > Bitcoin Core PR review club and will be lower level and more
> >> > > > > technical. That is planned for Tuesday February 23rd at 19:00 UTC on
> >> > > > > the IRC channel ##taproot-activation.
> >> > > > >
> >> > > > > Thanks to the meeting participants (and those who joined the
> >> > > > > discussion on the channel prior and post the meeting) for engaging
> >> > > > > productively and in good faith.
> >> > >
> >> > > --
> >> > > Michael Folkson
> >> > > Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com> <mailto:michaelfolkson at gmail.com
> <mailto:michaelfolkson at gmail.com>>
> >> > > Keybase: michaelfolkson
> >> > > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
> >> > > _______________________________________________
> >> > > bitcoin-dev mailing list
> >> > > bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>
> >> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
> >> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>>;
> >>
> >>
> >>
> >>
> >> --
> >> Michael Folkson
> >> Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com> <mailto:michaelfolkson at gmail.com
> <mailto:michaelfolkson at gmail.com>>
> >> Keybase: michaelfolkson
> >> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
> >> _______________________________________________
> >> bitcoin-dev mailing list
> >> bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
> <mailto:bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>>
> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
> >> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>>;
> >
> >
> >
> > --
> > Michael Folkson
> > Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com> <mailto:michaelfolkson at gmail.com
> <mailto:michaelfolkson at gmail.com>>
> > Keybase: michaelfolkson
> > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
>
>
>
> --
> Michael Folkson
> Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com>
> Keybase: michaelfolkson
> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3

📅 Original date posted:2021-02-18 📝 Original ...

2023-06-07T20:28:49+02:00

In reply to nevent1q…95ym
_________________________

📅 Original date posted:2021-02-18
📝 Original message:We've had several softforks in Bitcoin which, through the course of their activation, had a several-block reorg. That
should be indication enough that we need to very carefully consider activation to ensure we reduce the risk of that as
much as absolutely possible. Again, while I think Taproot is a huge improvement and am looking forward to being able to
use it, getting unlucky and hitting a 4-block reorg that happens to include a double-spend and some PR around an
exchange losing millions would be worse than having Taproot is good.

Matt

On 2/18/21 09:26, Michael Folkson wrote:
> Thanks for your response Matt. It is a fair challenge. There is always going to be an element of risk with soft forks,
> all we can do is attempt to minimize that risk. I would argue that risk has been minimized for Taproot.
>
> You know (better than I do in fact) that Bitcoin (and layers built on top of it) greatly benefit from upgrades such as
> Taproot. To say we shouldn't do Taproot or any future soft forks because there is a small but real risk of chain splits
> I think is shortsighted. Indeed I think even if we collectively decided not to do any future soft fork upgrades ever
> again on this mailing list that wouldn't stop soft fork attempts from other people in future.
>
> I don't think there is anything else we can do to minimize that risk for the Taproot soft fork at this point though I'm
> open to ideas. To reiterate that risk will never be zero. I don't think I see Bitcoin as fragile as you seem to (though
> admittedly you have a much better understanding than me of what happened in 2017).
>
> The likely scenario for the Taproot soft fork is LOT turns out to be entirely irrelevant and miners activate Taproot
> before it becomes relevant. And even the unlikely worst case scenario would only cause short term disruption and
> wouldn't kill Bitcoin long term.
>
> On Thu, Feb 18, 2021 at 2:01 PM Matt Corallo <lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>> wrote:
>
> If the eventual outcome is that different implementations (that have material *transaction processing* userbases,
> and I’m not sure to what extent that’s true with Knots) ship different consensus rules, we should stop here and not
> activate Taproot. Seriously.
>
> Bitcoin is a consensus system. The absolute worst outcome at all possible is to have it fall out of consensus.
>
> Matt
>
>> On Feb 18, 2021, at 08:11, Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
>> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>>
>>
>> Right, that is one option. Personally I would prefer a Bitcoin Core release sets LOT=false (based on what I have
>> heard from Bitcoin Core contributors) and a community effort releases a version with LOT=true. I don't think users
>> should be forced to choose something they may have no context on before they are allowed to use Bitcoin Core.
>>
>> My current understanding is that roasbeef is planning to set LOT=false on btcd (an alternative protocol
>> implementation to Bitcoin Core) and Luke Dashjr hasn't yet decided on Bitcoin Knots.
>>
>>
>>
>> On Thu, Feb 18, 2021 at 11:52 AM ZmnSCPxj <ZmnSCPxj at protonmail.com <mailto:ZmnSCPxj at protonmail.com>> wrote:
>>
>> Good morning all,
>>
>> > "An activation mechanism is a consensus change like any other change, can be contentious like any other
>> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline."
>> >
>> > Who's we here?
>> >
>> > Release both and let the network decide.
>>
>> A thing that could be done, without mandating either LOT=true or LOT=false, would be to have a release that
>> requires a `taprootlot=1` or `taprootlot=0` and refuses to start if the parameter is not set.
>>
>> This assures everyone that neither choice is being forced on users, and instead what is being forced on users,
>> is for users to make that choice themselves.
>>
>> Regards,
>> ZmnSCPxj
>>
>> >
>> > On Thu, Feb 18, 2021 at 3:08 AM Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
>> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>> >
>> > > Thanks for your response Ariel. It would be useful if you responded to specific points I have made in the
>> mailing list post or at least quote these ephemeral "people" you speak of. I don't know if you're responding
>> to conversation on the IRC channel or on social media etc.
>> > >
>> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted into
>> code. But in fact this isn't true and some voices in this discussion need to be more humble about what users
>> must or must not run.
>> > >
>> > > I personally have never made this assumption. Of course users aren't forced to run any particular software
>> version, quite the opposite. Defaults set in software versions matter though as many users won't change them.
>> > >
>> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be only a
>> handful of people that begin running it while everyone else delays their upgrade (with the very good reason of
>> not getting involved in politics) and a year later those handful of people just become stuck at the moment of
>> MUST_SIGNAL, unable to mine new blocks?
>> > >
>> > > It is a possible outcome but the likely outcome is that miners activate Taproot before LOT is even
>> relevant. I think it is prudent to prepare for the unlikely but possible outcome that miners fail to activate
>> and hence have this discussion now rather than be unprepared for that eventuality. If LOT is set to false in a
>> software release there is the possibility (T2 in
>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
>> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>;) of individuals or a
>> proportion of the community changing LOT to true. In that sense setting LOT=false in a software release
>> appears to be no more safe than LOT=true.
>> > >
>> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with miners
>> by default.
>> > >
>> > > There is the (unlikely but possible) possibility of a wasted year if LOT is set to false and miners fail
>> to activate. I'm not convinced by this perception that LOT=true is antagonistic to miners. I actually think it
>> offers them clarity on what will happen over a year time period and removes the need for coordinated or
>> uncoordinated community UASF efforts on top of LOT=false.
>> > >
>> > > > An activation mechanism is a consensus change like any other change, can be contentious like any other
>> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
>> > >
>> > > I don't know what you are recommending here to avoid "this darkest timeline". Open discussions have
>> occurred and are continuing and in my mailing list post that you responded to **I recommended we propose
>> LOT=false be set in protocol implementations such as Bitcoin Core**. I do think this apocalyptic language
>> isn't particularly helpful. In an open consensus system discussion is healthy, we should prepare for bad or
>> worst case scenarios in advance and doing so is not antagonistic or destructive. Mining pools have pledged
>> support for Taproot but we don't build secure systems based on pledges of support, we build them to minimize
>> trust in any human actors. We can be grateful that people like Alejandro have worked hard on
>> taprootactivation.com <http://taprootactivation.com>; (and this effort has informed the discussion) without
>> taking pledges of support as cast iron guarantees.
>> > >
>> > > TL;DR It sounds like you agree with my recommendation to set LOT=false in protocol implementations in my
>> email :)
>> > >
>> > > On Thu, Feb 18, 2021 at 5:43 AM Ariel Lorenzo-Luaces <arielluaces at gmail.com
>> <mailto:arielluaces at gmail.com>> wrote:
>> > >
>> > > > Something what strikes me about the conversation is the emotion surrounding the letters UASF.
>> > > > It appears as if people discuss UASF as if it's a massive tidal wave of support that is inevitable, like
>> we saw during segwit activation. But the actual definition is "any activation that is not a MASF".
>> > > > A UASF can consist of a single node, ten nodes, a thousand, half of all nodes, all business' nodes, or
>> even all the non mining nodes. On another dimension it can have zero mining support, 51% support, 49% support,
>> or any support right up against a miner activation threshold.
>> > > > Hell a UASF doesn't even need code or even a single node running as long as it exists as a possibility
>> in people's minds.
>> > > > The only thing a UASF doesn't have is miner support above an agreed activation threshold (some number
>> above %51).
>> > > > I say this because it strikes me when people say that they are for LOT=true with the logic that since a
>> UASF is guaranteed to happen then it's better to just make it default from the beginning. Words like
>> coordination and safety are sometimes sprinkled into the argument.
>> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted into
>> code. But in fact this isn't true and some voices in this discussion need to be more humble about what users
>> must or must not run.
>> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be only a
>> handful of people that begin running it while everyone else delays their upgrade (with the very good reason of
>> not getting involved in politics) and a year later those handful of people just become stuck at the moment of
>> MUST_SIGNAL, unable to mine new blocks? Or attracting a minority of miners, activating, and forking off into a
>> minority fork. Then a lot=false could be started that ends up activating the feature now that the stubborn
>> option has ran its course.
>> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with miners
>> by default. The chains could be called BitcoinLenient and BitcoinStubborn.
>> > > > How is that strictly safer or more coordinated?
>> > > > I may be in the minority, or maybe a silent majority, or maybe a majority that just hasn't considered
>> this as a choice but honestly if there is contention about whether we're going to be stubborn or lenient with
>> miners for Taproot and in the future then I prefer to just not activate anything at all. I'm fine for calling
>> bitcoin ossified, accepting that segwit is Bitcoin's last network upgrade. Taproot is amazing but no new
>> feature is worth a network split down the middle.
>> > > > Maybe in 10 or 20 years, when other blockchains implement features like Taproot and many more, we will
>> become envious enough to put aside our differences on how to behave towards miners and finally activate Taproot.
>> > > > An activation mechanism is a consensus change like any other change, can be contentious like any other
>> change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
>> > > > Cheers
>> > > > Ariel Lorenzo-Luaces
>> > > > On Feb 17, 2021, at 7:05 AM, Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org
>> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>> > > >
>> > > > > Yesterday (February 16th) we held a second meeting on Taproot
>> > > > > activation on IRC which again was open to all. Despite what appeared
>> > > > > to be majority support for LOT=false over LOT=true in the first
>> > > > > meeting I (and others) thought the arguments had not been explored in
>> > > > > depth and that we should have a follow up meeting almost entirely
>> > > > > focused on whether LOT (lockinontimeout) should be set to true or
>> > > > > false.
>> > > > >
>> > > > > The meeting was announced here:
>> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
>> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html>;
>> > > > >
>> > > > > In that mailing list post I outlined the arguments for LOT=true (T1 to
>> > > > > T6) and arguments for LOT=false (F1 to F6) in their strongest form I
>> > > > > could. David Harding responded with an additional argument for
>> > > > > LOT=false (F7) here:
>> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html
>> <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html>;
>> > > > >
>> > > > > These meetings are very challenging given they are open to all, you
>> > > > > don’t know who will attend and you don’t know most people’s views in
>> > > > > advance. I tried to give time for both the LOT=true arguments and the
>> > > > > LOT=false arguments to be discussed as I knew there was support for
>> > > > > both. We only tried evaluating which had more support and which had
>> > > > > more strong opposition towards the end of the meeting.
>> > > > >
>> > > > > The conversation log is here:
>> > > > > http://gnusha.org/taproot-activation/2021-02-16.log <http://gnusha.org/taproot-activation/2021-02-16.log>;
>> > > > >
>> > > > > (If you are so inclined you can watch a video of the meeting here.
>> > > > > Thanks to the YouTube account “Bitcoin” for setting up the livestream:
>> > > > > https://www.youtube.com/watch?v=vpl5q1ovMLM <https://www.youtube.com/watch?v=vpl5q1ovMLM>;)
>> > > > >
>> > > > > A summary of the meeting was provided by Luke Dashjr on Mastodon here:
>> > > > > https://bitcoinhackers.org/@lukedashjr/105742918779234566
>> <https://bitcoinhackers.org/@lukedashjr/105742918779234566>;
>> > > > >
>> > > > > Today's #Bitcoin #Taproot meeting was IMO largely unproductive, but we
>> > > > > did manage to come to consensus on everything but LockinOnTimeout.
>> > > > >
>> > > > > Activation height range: 693504-745920
>> > > > >
>> > > > > MASF threshold: 1815/2016 blocks (90%)
>> > > > >
>> > > > > Keep in mind only ~100 people showed for the meetings, hardly
>> > > > > representative of the entire community.
>> > > > >
>> > > > > So, these details remain JUST a proposal for now.
>> > > > >
>> > > > > It seems inevitable that there won't be consensus on LOT.
>> > > > >
>> > > > > Everyone will have to choose for himself. :/
>> > > > >
>> > > > > Personally I agree with most of this. I agree that there wasn’t
>> > > > > overwhelming consensus for either LOT=true or LOT=false. However, from
>> > > > > my perspective there was clearly more strong opposition (what would
>> > > > > usually be deemed a NACK in Bitcoin Core review terminology) from
>> > > > > Bitcoin Core contributors, Lightning developers and other community
>> > > > > members against LOT=true than there was for LOT=false. Andrew Chow
>> > > > > tried to summarize views from the meeting in this analysis:
>> > > > > https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c
>> <https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c>;
>> > > > >
>> > > > > I am also aware of other current and previous Bitcoin Core
>> > > > > contributors and Lightning developers who didn’t attend the meeting in
>> > > > > person who are opposed to LOT=true. I don’t want to put them in the
>> > > > > spotlight for no reason but if you go through the conversation logs of
>> > > > > not only the meeting but the weeks of discussion prior to this meeting
>> > > > > you will see their views evaluated on the ##taproot-activation
>> > > > > channel. In addition, on taprootactivation.com <http://taprootactivation.com>; some mining pools
>> > > > > expressed a preference for lot=false though I don’t know how strong
>> > > > > that preference was.
>> > > > >
>> > > > > I am only one voice but it is my current assessment that if we are to
>> > > > > attempt to finalize Taproot activation parameters and propose them to
>> > > > > the community at this time our only option is to propose LOT=false.
>> > > > > Any further delay appears to me counterproductive in our collective
>> > > > > aim to get the Taproot soft fork activated as early as possible.
>> > > > >
>> > > > > Obviously others are free to disagree with that assessment and
>> > > > > continue discussions but personally I will be attempting to avoid
>> > > > > those discussions unless prominent new information comes to light or
>> > > > > various specific individuals change their minds.
>> > > > >
>> > > > > Next week we are planning a code review of the Bitcoin Core PR #19573
>> > > > > which was initially delayed because of this LOT discussion. As I’ve
>> > > > > said previously that will be loosely following the format of the
>> > > > > Bitcoin Core PR review club and will be lower level and more
>> > > > > technical. That is planned for Tuesday February 23rd at 19:00 UTC on
>> > > > > the IRC channel ##taproot-activation.
>> > > > >
>> > > > > Thanks to the meeting participants (and those who joined the
>> > > > > discussion on the channel prior and post the meeting) for engaging
>> > > > > productively and in good faith.
>> > >
>> > > --
>> > > Michael Folkson
>> > > Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com>
>> > > Keybase: michaelfolkson
>> > > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
>> > > _______________________________________________
>> > > bitcoin-dev mailing list
>> > > bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
>> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
>>
>>
>>
>>
>> --
>> Michael Folkson
>> Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com>
>> Keybase: michaelfolkson
>> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org <mailto:bitcoin-dev at lists.linuxfoundation.org>
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
>
>
>
> --
> Michael Folkson
> Email: michaelfolkson at gmail.com <mailto:michaelfolkson at gmail.com>
> Keybase: michaelfolkson
> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3

📅 Original date posted:2021-02-18 📝 Original message:If the ...

2023-06-07T20:28:48+02:00

In reply to nevent1q…lvsd
_________________________

📅 Original date posted:2021-02-18
📝 Original message:If the eventual outcome is that different implementations (that have material *transaction processing* userbases, and I’m not sure to what extent that’s true with Knots) ship different consensus rules, we should stop here and not activate Taproot. Seriously.

Bitcoin is a consensus system. The absolute worst outcome at all possible is to have it fall out of consensus.

Matt

> On Feb 18, 2021, at 08:11, Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>
> Right, that is one option. Personally I would prefer a Bitcoin Core release sets LOT=false (based on what I have heard from Bitcoin Core contributors) and a community effort releases a version with LOT=true. I don't think users should be forced to choose something they may have no context on before they are allowed to use Bitcoin Core.
>
> My current understanding is that roasbeef is planning to set LOT=false on btcd (an alternative protocol implementation to Bitcoin Core) and Luke Dashjr hasn't yet decided on Bitcoin Knots.
>
>
>
>> On Thu, Feb 18, 2021 at 11:52 AM ZmnSCPxj <ZmnSCPxj at protonmail.com> wrote:
>> Good morning all,
>>
>> > "An activation mechanism is a consensus change like any other change, can be contentious like any other change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline."
>> >
>> > Who's we here?
>> >
>> > Release both and let the network decide.
>>
>> A thing that could be done, without mandating either LOT=true or LOT=false, would be to have a release that requires a `taprootlot=1` or `taprootlot=0` and refuses to start if the parameter is not set.
>>
>> This assures everyone that neither choice is being forced on users, and instead what is being forced on users, is for users to make that choice themselves.
>>
>> Regards,
>> ZmnSCPxj
>>
>> >
>> > On Thu, Feb 18, 2021 at 3:08 AM Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>> >
>> > > Thanks for your response Ariel. It would be useful if you responded to specific points I have made in the mailing list post or at least quote these ephemeral "people" you speak of. I don't know if you're responding to conversation on the IRC channel or on social media etc.
>> > >
>> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted into code. But in fact this isn't true and some voices in this discussion need to be more humble about what users must or must not run.
>> > >
>> > > I personally have never made this assumption. Of course users aren't forced to run any particular software version, quite the opposite. Defaults set in software versions matter though as many users won't change them.
>> > >
>> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be only a handful of people that begin running it while everyone else delays their upgrade (with the very good reason of not getting involved in politics) and a year later those handful of people just become stuck at the moment of MUST_SIGNAL, unable to mine new blocks?
>> > >
>> > > It is a possible outcome but the likely outcome is that miners activate Taproot before LOT is even relevant. I think it is prudent to prepare for the unlikely but possible outcome that miners fail to activate and hence have this discussion now rather than be unprepared for that eventuality. If LOT is set to false in a software release there is the possibility (T2 in https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html) of individuals or a proportion of the community changing LOT to true. In that sense setting LOT=false in a software release appears to be no more safe than LOT=true.
>> > >
>> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with miners by default.
>> > >
>> > > There is the (unlikely but possible) possibility of a wasted year if LOT is set to false and miners fail to activate. I'm not convinced by this perception that LOT=true is antagonistic to miners. I actually think it offers them clarity on what will happen over a year time period and removes the need for coordinated or uncoordinated community UASF efforts on top of LOT=false.
>> > >
>> > > > An activation mechanism is a consensus change like any other change, can be contentious like any other change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
>> > >
>> > > I don't know what you are recommending here to avoid "this darkest timeline". Open discussions have occurred and are continuing and in my mailing list post that you responded to **I recommended we propose LOT=false be set in protocol implementations such as Bitcoin Core**. I do think this apocalyptic language isn't particularly helpful. In an open consensus system discussion is healthy, we should prepare for bad or worst case scenarios in advance and doing so is not antagonistic or destructive. Mining pools have pledged support for Taproot but we don't build secure systems based on pledges of support, we build them to minimize trust in any human actors. We can be grateful that people like Alejandro have worked hard on taprootactivation.com (and this effort has informed the discussion) without taking pledges of support as cast iron guarantees.
>> > >
>> > > TL;DR It sounds like you agree with my recommendation to set LOT=false in protocol implementations in my email :)
>> > >
>> > > On Thu, Feb 18, 2021 at 5:43 AM Ariel Lorenzo-Luaces <arielluaces at gmail.com> wrote:
>> > >
>> > > > Something what strikes me about the conversation is the emotion surrounding the letters UASF.
>> > > > It appears as if people discuss UASF as if it's a massive tidal wave of support that is inevitable, like we saw during segwit activation. But the actual definition is "any activation that is not a MASF".
>> > > > A UASF can consist of a single node, ten nodes, a thousand, half of all nodes, all business' nodes, or even all the non mining nodes. On another dimension it can have zero mining support, 51% support, 49% support, or any support right up against a miner activation threshold.
>> > > > Hell a UASF doesn't even need code or even a single node running as long as it exists as a possibility in people's minds.
>> > > > The only thing a UASF doesn't have is miner support above an agreed activation threshold (some number above %51).
>> > > > I say this because it strikes me when people say that they are for LOT=true with the logic that since a UASF is guaranteed to happen then it's better to just make it default from the beginning. Words like coordination and safety are sometimes sprinkled into the argument.
>> > > > The argument comes from a naive assumption that users MUST upgrade to the choice that is submitted into code. But in fact this isn't true and some voices in this discussion need to be more humble about what users must or must not run.
>> > > > Does no one realize that it is a very possible outcome that if LOT=true is released there may be only a handful of people that begin running it while everyone else delays their upgrade (with the very good reason of not getting involved in politics) and a year later those handful of people just become stuck at the moment of MUST_SIGNAL, unable to mine new blocks? Or attracting a minority of miners, activating, and forking off into a minority fork. Then a lot=false could be started that ends up activating the feature now that the stubborn option has ran its course.
>> > > > The result: a wasted year of waiting and a minority of people who didn't want to be lenient with miners by default. The chains could be called BitcoinLenient and BitcoinStubborn.
>> > > > How is that strictly safer or more coordinated?
>> > > > I may be in the minority, or maybe a silent majority, or maybe a majority that just hasn't considered this as a choice but honestly if there is contention about whether we're going to be stubborn or lenient with miners for Taproot and in the future then I prefer to just not activate anything at all. I'm fine for calling bitcoin ossified, accepting that segwit is Bitcoin's last network upgrade. Taproot is amazing but no new feature is worth a network split down the middle.
>> > > > Maybe in 10 or 20 years, when other blockchains implement features like Taproot and many more, we will become envious enough to put aside our differences on how to behave towards miners and finally activate Taproot.
>> > > > An activation mechanism is a consensus change like any other change, can be contentious like any other change, and we must resolve it like any other change. Otherwise we risk arriving at the darkest timeline.
>> > > > Cheers
>> > > > Ariel Lorenzo-Luaces
>> > > > On Feb 17, 2021, at 7:05 AM, Michael Folkson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>> > > >
>> > > > > Yesterday (February 16th) we held a second meeting on Taproot
>> > > > > activation on IRC which again was open to all. Despite what appeared
>> > > > > to be majority support for LOT=false over LOT=true in the first
>> > > > > meeting I (and others) thought the arguments had not been explored in
>> > > > > depth and that we should have a follow up meeting almost entirely
>> > > > > focused on whether LOT (lockinontimeout) should be set to true or
>> > > > > false.
>> > > > >
>> > > > > The meeting was announced here:
>> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018380.html
>> > > > >
>> > > > > In that mailing list post I outlined the arguments for LOT=true (T1 to
>> > > > > T6) and arguments for LOT=false (F1 to F6) in their strongest form I
>> > > > > could. David Harding responded with an additional argument for
>> > > > > LOT=false (F7) here:
>> > > > > https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-February/018415.html
>> > > > >
>> > > > > These meetings are very challenging given they are open to all, you
>> > > > > don’t know who will attend and you don’t know most people’s views in
>> > > > > advance. I tried to give time for both the LOT=true arguments and the
>> > > > > LOT=false arguments to be discussed as I knew there was support for
>> > > > > both. We only tried evaluating which had more support and which had
>> > > > > more strong opposition towards the end of the meeting.
>> > > > >
>> > > > > The conversation log is here:
>> > > > > http://gnusha.org/taproot-activation/2021-02-16.log
>> > > > >
>> > > > > (If you are so inclined you can watch a video of the meeting here.
>> > > > > Thanks to the YouTube account “Bitcoin” for setting up the livestream:
>> > > > > https://www.youtube.com/watch?v=vpl5q1ovMLM)
>> > > > >
>> > > > > A summary of the meeting was provided by Luke Dashjr on Mastodon here:
>> > > > > https://bitcoinhackers.org/@lukedashjr/105742918779234566
>> > > > >
>> > > > > Today's #Bitcoin #Taproot meeting was IMO largely unproductive, but we
>> > > > > did manage to come to consensus on everything but LockinOnTimeout.
>> > > > >
>> > > > > Activation height range: 693504-745920
>> > > > >
>> > > > > MASF threshold: 1815/2016 blocks (90%)
>> > > > >
>> > > > > Keep in mind only ~100 people showed for the meetings, hardly
>> > > > > representative of the entire community.
>> > > > >
>> > > > > So, these details remain JUST a proposal for now.
>> > > > >
>> > > > > It seems inevitable that there won't be consensus on LOT.
>> > > > >
>> > > > > Everyone will have to choose for himself. :/
>> > > > >
>> > > > > Personally I agree with most of this. I agree that there wasn’t
>> > > > > overwhelming consensus for either LOT=true or LOT=false. However, from
>> > > > > my perspective there was clearly more strong opposition (what would
>> > > > > usually be deemed a NACK in Bitcoin Core review terminology) from
>> > > > > Bitcoin Core contributors, Lightning developers and other community
>> > > > > members against LOT=true than there was for LOT=false. Andrew Chow
>> > > > > tried to summarize views from the meeting in this analysis:
>> > > > > https://gist.github.com/achow101/3e179501290abb7049de198d46894c7c
>> > > > >
>> > > > > I am also aware of other current and previous Bitcoin Core
>> > > > > contributors and Lightning developers who didn’t attend the meeting in
>> > > > > person who are opposed to LOT=true. I don’t want to put them in the
>> > > > > spotlight for no reason but if you go through the conversation logs of
>> > > > > not only the meeting but the weeks of discussion prior to this meeting
>> > > > > you will see their views evaluated on the ##taproot-activation
>> > > > > channel. In addition, on taprootactivation.com some mining pools
>> > > > > expressed a preference for lot=false though I don’t know how strong
>> > > > > that preference was.
>> > > > >
>> > > > > I am only one voice but it is my current assessment that if we are to
>> > > > > attempt to finalize Taproot activation parameters and propose them to
>> > > > > the community at this time our only option is to propose LOT=false.
>> > > > > Any further delay appears to me counterproductive in our collective
>> > > > > aim to get the Taproot soft fork activated as early as possible.
>> > > > >
>> > > > > Obviously others are free to disagree with that assessment and
>> > > > > continue discussions but personally I will be attempting to avoid
>> > > > > those discussions unless prominent new information comes to light or
>> > > > > various specific individuals change their minds.
>> > > > >
>> > > > > Next week we are planning a code review of the Bitcoin Core PR #19573
>> > > > > which was initially delayed because of this LOT discussion. As I’ve
>> > > > > said previously that will be loosely following the format of the
>> > > > > Bitcoin Core PR review club and will be lower level and more
>> > > > > technical. That is planned for Tuesday February 23rd at 19:00 UTC on
>> > > > > the IRC channel ##taproot-activation.
>> > > > >
>> > > > > Thanks to the meeting participants (and those who joined the
>> > > > > discussion on the channel prior and post the meeting) for engaging
>> > > > > productively and in good faith.
>> > >
>> > > --
>> > > Michael Folkson
>> > > Email: michaelfolkson at gmail.com
>> > > Keybase: michaelfolkson
>> > > PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
>> > > _______________________________________________
>> > > bitcoin-dev mailing list
>> > > bitcoin-dev at lists.linuxfoundation.org
>> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>
>
> --
> Michael Folkson
> Email: michaelfolkson at gmail.com
> Keybase: michaelfolkson
> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210218/3aeb6764/attachment-0001.html>;

📅 Original date posted:2019-03-08 📝 Original message:To ...

2023-06-07T20:16:55+02:00

In reply to nevent1q…j0kh
_________________________

📅 Original date posted:2019-03-08
📝 Original message:To make testing easier, it may make sense to keep the existing block header format (and PoW) and instead apply the signature rules to some field in the coinbase transaction. This means SPV clients (assuming they only connect to honest/trusted nodes) work as-is.

A previous idea regarding reorgs (that I believe Greg came up with) is to allow multiple keys to sign blocks, with one signing no reorgs and one signing a reorg every few blocks, allowing users to choose the behavior they want.

> On Mar 8, 2019, at 00:54, Karl-Johan Alm via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Hello,
>
> As some of you already know, I've been working on a network called "signet", which is bascially a complement to the already existing testnet, except it is completely centralized, and blocks are signed by a specific key rather than using proof of work.
>
> Benefits of this:
>
> 1. It is more predictable than testnet. Miners appear and disappear regularly, causing irregular block generation.
>
> 2. Since it is centrally controlled, it is easy to perform global testing, such as reorgs (e.g. the network performs a 4 block reorg by request, or as scheduled).
>
> 3. It is more stable than testnet, which occasionally sees several thousand block reorgs.
>
> 4. It is trivial to spin up (and shut down) new signets to make public tests where anyone can participate.
>
> Anyone can create a signet at any time, simply by creating a key pair and creating a challenge (scriptPubKey). The network can then be used globally by anyone, assuming the creator sends some coins to the other participants.
>
> Having a persistent signet would be beneficial in particular to services which need a stable place to test features over an extended period of time. My own company implements protocols on top of Bitcoin with sidechains. We need multi-node test frameworks to behave in a predictable manner (unlike testnet) and with the same standardness relay policy as mainnet.
>
> Signets consist of 2 parameters: the challenge script (scriptPubKey) and the solution length. (The latter is needed to retain fixed length block headers, despite having an additional payload.)
>
> I propose that a default persistent "signet1" is created, which can be replaced in future versions e.g. if the coins are unwisely used as real money, similarly to what happened to previous testnets. This signet is picked by default if a user includes -signet without providing any of the parameters mentioned above. The key holder would be someone sufficiently trusted in the community, who would be willing to run the system (block generation code, faucet, etc). It could be made a little more sturdy by using 1-of-N multisig as the challenge, in case 1 <= x < N of the signers disappear. If people oppose this, it can be skipped, but will mean people can't just jump onto signet without first tracking down parameters from somewhere.
>
> Implementation-wise, the code adds an std::map with block hash to block signature. This is serialized/deserialized as appropriate (Segwit witness style), which means block headers in p2p messages are (80 + solution_length) bytes. Block header non-contextual check goes from checking if block header hash < target to checking if the payload is a valid signature for the block header hash instead.
>
> Single commit with code (will split into commits and make PR later, but just to give an idea what it looks like): https://github.com/kallewoof/bitcoin/pull/4
>
> I don't think this PR is overly intrusive, and I'm hoping to be able to get signet code into Bitcoin Core eventually, and am equally hopeful that devs of other (wallet etc) implementations will consider supporting it.
>
> Feedback requested on this.
>
> Attribution: parts of the signet code (in particular signblock and getnewblockhex) were adapted from the ElementsProject/elements repository. When PR is split into atomic commits, I will put appropriate attribution there.
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190308/96e6fa83/attachment.html>;

📅 Original date posted:2019-03-08 📝 Original message:Aside ...

2023-06-07T20:16:49+02:00

In reply to nevent1q…pmak
_________________________

📅 Original date posted:2019-03-08
📝 Original message:Aside from the complexity issues here, note that for a user to be adversely affect, they probably have to have pre-signed lock-timed transactions. Otherwise, in the crazy case that such a user exists, they should have no problem claiming the funds before activation of a soft-fork (and just switching to the swgwit equivalent, or some other equivalent scheme). Thus, adding additional restrictions like tx size limits will equally break txn.

> On Mar 8, 2019, at 14:12, Sjors Provoost <sjors at sprovoost.nl> wrote:
>
>
>> (1) It has been well documented again and again that there is desire to remove OP_CODESEPARATOR, (2) it is well-documented OP_CODESEPARATOR in non-segwit scripts represents a rather significant vulnerability in Bitcoin today, and (3) lots of effort has gone into attempting to find practical use-cases for OP_CODESEPARATOR's specific construction, with no successes as of yet. I strongly, strongly disagree that the highly-unlikely remote possibility that someone created something before which could be rendered unspendable is sufficient reason to not fix a vulnerability in Bitcoin today.
>>
>>> I suggest an alternative whereby the execution of OP_CODESEPARATOR increases the transactions weight suitably as to temper the vulnerability caused by it. Alternatively there could be some sort of limit (maybe 1) on the maximum number of OP_CODESEPARATORs allowed to be executed per script, but that would require an argument as to why exceeding that limit isn't reasonable.
>>
>> You could equally argue, however, that any such limit could render some moderately-large transaction unspendable, so I'm somewhat skeptical of this argument. Note that OP_CODESEPARATOR is non-standard, so getting them mined is rather difficult in any case.
>
> Although I'm not a fan of extra complicity, just to explore these two ideas a bit further.
>
> What if such a transaction:
>
> 1. must have one input; and
> 2. must be smaller than 400 vbytes; and
> 3. must spend from a UTXO older than fork activation
>
> Adding such a contextual check seems rather painful, perhaps comparable to nLockTime. Anything more specific than the above, e.g. counting the number of OP_CODESEPARATOR calls, seems like guess work.
>
> Transaction weight currently doesn't consider OP codes, it only considers if bytes are part of the witness. Changing that to something more akin to Ethereums gas pricing sounds too complicated to even consider.
>
>
> I would also like to believe that whoever went through the trouble of using OP_CODESEPARATOR reads this list.
>
> Sjors
>

📅 Original date posted:2019-03-12 📝 Original message:Note ...

2023-06-07T20:16:47+02:00

In reply to nevent1q…8jhf
_________________________

📅 Original date posted:2019-03-12
📝 Original message:Note that even your carve-outs for OP_NOP is not sufficient here - if you were using nSequence to tag different pre-signed transactions into categories (roughly as you suggest people may want to do with extra sighash bits) then their transactions could very easily have become un-realistically-spendable. The whole point of soft forks is that we invalidate otherwise-unused bits of the protocol. This does not seem inconsistent with the proposal here.

> On Mar 9, 2019, at 13:29, Russell O'Connor <roconnor at blockstream.io> wrote:
> Bitcoin has *never* made a soft-fork, since the time of Satoishi, that invalidated transactions that send secured inputs to secured outputs (excluding uses of OP_NOP1-OP_NOP10).

📅 Original date posted:2019-03-07 📝 Original ...

2023-06-07T20:16:45+02:00

In reply to nevent1q…80y5
_________________________

📅 Original date posted:2019-03-07
📝 Original message:Replies inline.

Matt

On 3/7/19 3:03 PM, Russell O'Connor wrote:
>
> * OP_CODESEPARATOR in non-BIP 143 scripts fails the script validation.
> This includes OP_CODESEPARATORs in unexecuted branches of if
> statements,
> similar to other disabled opcodes, but unlike OP_RETURN.
>
>
> OP_CODESEPARATOR is the only mechanism available that allows users to
> sign which particular branch they are authorizing for within scripts
> that have multiple possible conditions that reuse the same public key.

This is true, and yet it does not appear to actually be practically
usable. Thus far, despite a ton of effort, I have not yet seen a
practical use-case for OP_CODESEPARATOR (except for one example of it
being used to make SegWit scripts ever-so-slightly more effecient in
TumbleBit, hence why this BIP does not propose disabling it for SegWit).

> Because of P2SH you cannot know that no one is currently using this
> feature. Activating a soft-fork as describe above means these sorts of
> funds would be permanently lost. It is not acceptable to risk people's
> money like this.

(1) It has been well documented again and again that there is desire to
remove OP_CODESEPARATOR, (2) it is well-documented OP_CODESEPARATOR in
non-segwit scripts represents a rather significant vulnerability in
Bitcoin today, and (3) lots of effort has gone into attempting to find
practical use-cases for OP_CODESEPARATOR's specific construction, with
no successes as of yet. I strongly, strongly disagree that the
highly-unlikely remote possibility that someone created something before
which could be rendered unspendable is sufficient reason to not fix a
vulnerability in Bitcoin today.

> I suggest an alternative whereby the execution of OP_CODESEPARATOR
> increases the transactions weight suitably as to temper the
> vulnerability caused by it. Alternatively there could be some sort of
> limit (maybe 1) on the maximum number of OP_CODESEPARATORs allowed to be
> executed per script, but that would require an argument as to why
> exceeding that limit isn't reasonable.

You could equally argue, however, that any such limit could render some
moderately-large transaction unspendable, so I'm somewhat skeptical of
this argument. Note that OP_CODESEPARATOR is non-standard, so getting
them mined is rather difficult in any case.

📅 Original date posted:2019-03-08 📝 Original ...

2023-06-07T20:16:45+02:00

In reply to nevent1q…t6uz
_________________________

📅 Original date posted:2019-03-08
📝 Original message:Replies inline.

On 3/8/19 3:57 PM, Russell O'Connor wrote:
> On Thu, Mar 7, 2019 at 2:50 PM Matt Corallo <lf-lists at mattcorallo.com
> <mailto:lf-lists at mattcorallo.com>> wrote:
> It's very easy to construct a practical script using OP_CODESEPARATOR.
>
> IF <2> <ALICEPUBKEY> <BOBPUBKEY> <2> CHECKMULTISIGVERIFY ELSE
> CODESEPARATOR <ALICEPUBKEY> CHECKSIGVERFY ENDIF
>
> Now when someone hands Alice, the CFO of XYZ corp., some transaction,
> she has the option of either signing it unilaterally herself, or
> creating a partial signature such that the transaction additionally
> needs Bob, the CEOs signature as well, and Alice's choice is committed
> to the blockchain for auditing purposes later.
>
> Now, there are many things you might object about this scheme, but my
> point is that (A) regardless of what you think about this scheme, it, or
> similar schemes, may have been devised by users, and (B) users may have
> already committed funds to such schemes, and due to P2SH you cannot know
> that this is not the case.

The common way to set that up is to have a separate key, but, ok, fair
enough. That said, the argument that "it may be hidden by P2SH!" isn't
sufficient here. It has to *both* be hidden by P2SH and have never been
spent from (either on mainnet or testnet) or be lock-timed a year in the
future. I'm seriously skeptical that someone is using a highly esoteric
scheme and has just been pouring money into it without ever having
tested it or having withdrawn any money from it whatsoever. This is just
a weird argument.

> Please don't strawman my position. I am not suggesting we don't fix a
> vulnerability in Bitcoin. I am suggesting we find another way. One
> that limits the of risk destroying other people's money.
>
> Here is a more concrete proposal: No matter how bad OP_CODESEPARATOR
> is, it cannot be worse than instead including another input that spends
> another identically sized UTXO. So how about we soft-fork in a rule
> that says that an input's weight is increased by an amount equal to the
> number of OP_CODESEPARATORs executed times the sum of weight of the UTXO
> being spent and 40 bytes, the weight of a stripped input. The risk of
> destroying other people's money is limited and AFAIU it would completely
> address the vulnerabilities caused by OP_CODESEPARATOR.

You're already arguing that someone has such an esoteric use of script,
suggesting they aren't *also* creating pre-signed, long-locktimed
transactions with many inputs isn't much of a further stretch
(especially since this may result in the fee being non-standardly low if
you artificially increase its weight).

Note that "just limit number of OP_CODESEPARATOR calls" results in a ton
of complexity and reduces the simple analysis that fees (almost) have
today vs just removing it allows us to also remove a ton of code.

Further note that if you don't remove it getting the efficiency wins
right is even harder because instead of being able to cache sighashes
you now have to (at a minimum) wipe the cache between each
OP_CODESEPARATOR call, which results in a ton of additional
implementation complexity.

>
> > I suggest an alternative whereby the execution of OP_CODESEPARATOR
> > increases the transactions weight suitably as to temper the
> > vulnerability caused by it. Alternatively there could be some
> sort of
> > limit (maybe 1) on the maximum number of OP_CODESEPARATORs
> allowed to be
> > executed per script, but that would require an argument as to why
> > exceeding that limit isn't reasonable.
>
> You could equally argue, however, that any such limit could render some
> moderately-large transaction unspendable, so I'm somewhat skeptical of
> this argument. Note that OP_CODESEPARATOR is non-standard, so getting
> them mined is rather difficult in any case.
>
>
> I already know of people who's funds are tied up due to in other changes
> to Bitcoin Core's default relay policy. Non-standardness is not an
> excuse to take other people's tied up funds and destroy them permanently.

Huh?! The whole point of non-standardness in this context is to (a) make
soft-forking something out safer by derisking miners not upgrading right
away and (b) signal something that may be a candidate for soft-forking
out so that we get feedback. Who is getting things disabled who isn't
bothering to *tell* people that their use-case is being hurt?!

📅 Original date posted:2019-02-05 📝 Original message:On ...

2023-06-07T20:16:22+02:00

In reply to nevent1q…0zgn
_________________________

📅 Original date posted:2019-02-05
📝 Original message:On 2/4/19 8:18 PM, Jim Posen via bitcoin-dev wrote:
- snip -
> 1) Introduce a new P2P message to retrieve all prev-outputs for a given
> block (essentially the undo data in Core), and verify the scripts
> against the block by executing them. While this permits some forms of
> input script malleability (and thus cannot discriminate between all
> valid and invalid filters), it restricts what an attacker can do. This
> was proposed by Laolu AFAIK, and I believe this is how btcd is
proceeding.

I'm somewhat confused by this - how does the undo data help you without
seeing the full (mistate compressed) transaction? In (the realistic)
thread model where an attacker is trying to blind you from some output,
they can simply give you "undo data" where scriptPubKeys are OP_TRUE
instead of the real script and you'd be none the wiser.

On 2/5/19 1:42 AM, Olaoluwa Osuntokun via bitcoin-dev wrote:
- snip -
> I think it's too late into the current deployment of the BIPs to change
> things around yet again. Instead, the BIP already has measures in place for
> adding _new_ filter types in the future. This along with a few other filter
> types may be worthwhile additions as new filter types.
- snip -

Huh? I don't think we should seriously consider
only-one-codebase-has-deployed-anything-with-very-limited-in-the-wild-use
as "too late into the current deployment"?

Matt

📅 Original date posted:2018-05-17 📝 Original message:(1) ...

2023-06-07T20:12:11+02:00

In reply to nevent1q…xr4r
_________________________

📅 Original date posted:2018-05-17
📝 Original message:(1) can be accomplished by filtering for the set of outputs in the transaction you created. I agree (2) would ideally be done to avoid issues with a copied wallet (theft or not), but I am worried about the size of the filters themselves, not just the size of the blocks downloaded after a match.

On May 17, 2018 3:43:15 PM UTC, Peter Todd <pete at petertodd.org> wrote:
>On Thu, May 17, 2018 at 11:25:12AM -0400, Matt Corallo via bitcoin-dev
>wrote:
>> BIP 158 currently includes the following in the "basic" filter: 1)
>> txids, 2) output scripts, 3) input prevouts.
>>
>> I believe (1) could be skipped entirely - there is almost no reason
>why
>> you'd not be able to filter for, eg, the set of output scripts in a
>> transaction you know about and (2) and (3) may want to be split out -
>> many wallets may wish to just find transactions paying to them, as
>> transactions spending from their outputs should generally be things
>> they've created.
>
>So I think we have two cases where wallets want to find txs spending
>from their
>outputs:
>
>1) Waiting for a confirmation
>2) Detecting theft
>
>The former can be turned off once there are no expected unconfirmed
>transactions.
>
>As for the latter, this is probably a valuable thing for wallets to do.
>Modulo
>reorgs, reducing the frequency that you check for stolen funds doesn't
>decrease
>total bandwidth cost - it's one filter match per block regardless - but
>perhaps
>the real-world bandwidth cost can be reduced by, say, waiting for a
>wifi
>connection rather than using cellular data.
>
>--
>https://petertodd.org 'peter'[:-1]@petertodd.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180517/d64f3e3b/attachment.html>;

📅 Original date posted:2018-05-17 📝 Original message:Yea I ...

2023-06-07T20:12:11+02:00

In reply to nevent1q…e92t
_________________________

📅 Original date posted:2018-05-17
📝 Original message:Yea I generally would really prefer something like that but it
significantly complicates the download logic - currently clients can
easily cross-check a filter in case they differ between providers by
downloading the block. If we instead went with the script being spent
they would have to be provided all previous transactions (potentially
compressed via midstate) as well, making it potentially infeasible to
identify the offending node while remaining a lightweight client. Maybe
there is some other reasonable download logic to replace it with, however.

Matt

On 05/17/18 12:36, Gregory Maxwell wrote:
> On Thu, May 17, 2018 at 3:25 PM, Matt Corallo via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>> I believe (1) could be skipped entirely - there is almost no reason why
>> you'd not be able to filter for, eg, the set of output scripts in a
>> transaction you know about
>
> I think this is convincing for the txids themselves.
>
> What about also making input prevouts filter based on the scriptpubkey
> being _spent_? Layering wise in the processing it's a bit ugly, but
> if you validated the block you have the data needed.
>
> This would eliminate the multiple data type mixing entirely.
>

📅 Original date posted:2018-05-17 📝 Original message:BIP ...

2023-06-07T20:12:10+02:00

In reply to nevent1q…aa0u
_________________________

📅 Original date posted:2018-05-17
📝 Original message:BIP 158 currently includes the following in the "basic" filter: 1)
txids, 2) output scripts, 3) input prevouts.

I believe (1) could be skipped entirely - there is almost no reason why
you'd not be able to filter for, eg, the set of output scripts in a
transaction you know about and (2) and (3) may want to be split out -
many wallets may wish to just find transactions paying to them, as
transactions spending from their outputs should generally be things
they've created.

In general, I'm concerned about the size of the filters making existing
SPV clients less willing to adopt BIP 158 instead of the existing bloom
filter garbage and would like to see a further exploration of ways to
split out filters to make them less bandwidth intensive. Some further
ideas we should probably play with before finalizing moving forward is
providing filters for certain script templates, eg being able to only
get outputs that are segwit version X or other similar ideas.

Matt

📅 Original date posted:2018-01-27 📝 Original message:Gah, ...

2023-06-07T20:10:11+02:00

In reply to nevent1q…gygd
_________________________

📅 Original date posted:2018-01-27
📝 Original message:Gah, please no. I see no material reason why cross-input signature aggregation shouldn't have the signatures in the first n-1 inputs replaced with something like a single-byte push where a signature is required to indicate aggregation, and the combined signature in the last input at whatever position the signature is required.

On January 27, 2018 5:07:25 PM UTC, Russell O'Connor via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
-snip-
>Cross-input signature aggregation probably requires a new field to be
>added
>to the P2P transaction structure to hold the aggregated signature,
>since
>there isn't really a good place to put it in the existing structure
>(there
>are games you can play to make it fit, but I think it is worthwhile).
>The
>obvious way add block commitments to a new tx field is via the witness
>reserved value mechanism present in BIP 141. At this point I think
>there
>will be some leeway to adjust the discount on the weight of this new
>aggregated signature tx field so that even a single input taproot using
>the
>aggregated signature system (here an aggregation of 1 signature) ends
>up no
>more expensive than a single input segwit P2WPKH.

📅 Original date posted:2018-01-23 📝 Original message:The ...

2023-06-07T20:10:07+02:00

In reply to nevent1q…mpnj
_________________________

📅 Original date posted:2018-01-23
📝 Original message:The issue with that approach without support for the privacy-encouraging wrapper proposed by Greg here is that it encourages adoption halfway and destroys a lot of the value of the apparent-script monoculture for privacy preservation. Greg's proposal here doesn't change the format of any specific MAST implementation, but instead adds the privacy wrapper that I always felt was missing in existing proposals, without any real additional overhead in many use-cases!

Indeed, permissionless innovation is important, but the huge advantage of providing the privacy wrapper by default here is absolutely massive to the ecosystem and should not be handwaved away for vague possibly-advantages.

Matt

On January 23, 2018 2:39:37 PM UTC, Mark Friedenbach <mark at friedenbach.org> wrote:
>I had the opposite response in private, which I will share here. As
>recently as Jan 9th feedback on BIP 117 was shared on this list by
>Pieter Wuille and others suggesting we adopt native MAST template
>instead of the user programmable combination of BIPs 116 and 117. Part
>of my response then was, I quote:
>
>I havent the hubris to suggest that we know exactly what a templated
>MAST *should* look like. It's not used in production anywhere. Even if
>we did have the foresight, the tail-call semantics allow for other
>constructions besides MAST and for the sake of the future we should
>allow such permission-less innovation. The proper sequence of events
>should be to enable features in a generic way, and then to create
>specialized templates to save space for common constructions. Not the
>other way around. [1]
>
>I take this advance as further evidence in favor of this view. As
>recently as 24 hours ago if you had asked what a native-MAST template
>would have looked like, the answer would have been something like
>Johnson Lau’s BIP 114, with some quibbling over details. Taproot is a
>clearly superior approach. But is it optimal? I don’t think we can
>claim that now. Optimality of these constructs isn’t something easily
>proven, with the nearest substitute being unchanging consensus over
>extended periods of time.
>
>Every time we add an output type specialization, we introduce a new
>codepath in the core of the script consensus that must be maintained
>forever. Take P2SH: from this point forward there is no reason to use
>it in new applications, ever. But it must be forever supported. In an
>alternate universe we could have deployed a native MAST proposal, like
>BIP 114, only to have Taproot-like schemes discovered after activation.
>That would have been a sucky outcome. It is still the case that we
>could go for Taproot right now, and then in six months or a year’s time
>we find an important tweak or a different approach entirely that is
>even better, but the activation process had already started. That would
>be a sucky outcome we haven’t avoided yet.
>
>This is not an argument against template specialization for common code
>paths, especially those which increase fungibility of coins. I do think
>we should have a native MAST template eventually, using Taproot or
>something better. However if I may be allowed I will make an educated
>guess about the origin of Taproot: I think it’s no coincidence that
>Greg had this insight and/or wrote it up simultaneous with a push by
>myself and others for getting MAST features into bitcoin via BIPs 98,
>116, and 117, or 114. Cryptographers tend to only think up solutions to
>problems that are on their minds. And the problems on most people’s
>minds are primarily those that are deployable today, or otherwise
>near-term applicable.
>
>BIPS 116 and 117 each provide a reusable component that together
>happens to enable a generic form of MAST. Even without the workarounds
>required to avoid CLEANSTACK violations, the resulting MAST template is
>larger than what is possible with specialization. However let’s not
>forget that (1) they also enable other applications like honeypots, key
>trees, and script delegation; and relevant to this conversation (2)
>they get the MAST feature available for use in production by the wider
>community. I don’t think I’d personally be willing to bet that we found
>the optimal MAST structure in Greg’s Taproot until we have people doing
>interesting production work like multisig wallets, lightning protocol,
>and the next set of consensus features start putting it into production
>and exploring edge cases. We may find ways Taproot can be tweaked to
>enable other applications (like encoding a hash preimage as well) or
>simplify obscure corner cases.
>
>I feel quite strongly that the correct approach is to add support for
>generic features to accomplish the underlying goal in a user
>programmable way, and THEN after activation and some usage consider
>ways in which common use cases can be made more efficient through
>output specialization. To take a more obvious example, lightning
>protocol is still an active area or research and I think it is
>abundantly clear that we don’t know yet what the globally optimal
>layer-2 caching protocol will be, even if we have educated guesses as
>to its broad structure. A proposal right now to standardize a more
>compact lightning script type would be rightly rejected. It is less
>obvious but just as true that the same should hold for MAST.
>
>I have argued these points before in favor of permission less
>innovation first, then application specialization later, in [1] and at
>the end of the rather long email [2]. I hope you can take the time to
>read those if you still feel we should take a specialized template
>approach instead of the user programmable BIPSs 116 and 117.
>
>[1]
>https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-January/015537.html
><https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-January/015537.html>;
>[2]
>https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015029.html
><https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015029.html>;
>
>> On Jan 22, 2018, at 6:51 PM, Matt Corallo via bitcoin-dev
><bitcoin-dev at lists.linuxfoundation.org> wrote:
>>
>> Thanks Greg!
>>
>> I'd be hesitant to deploy a MAST proposal without this clever
>application of pay-to-contract-hash now! Looks like the overhead over a
>more-naive MAST construction is rather trivial, too!
>>
>> Matt
>>
>> On January 23, 2018 12:30:06 AM UTC, Gregory Maxwell via bitcoin-dev
><bitcoin-dev at lists.linuxfoundation.org> wrote:
>> Interest in merkelized scriptPubKeys (e.g. MAST) is driven by two
>main
>> areas: efficiency and privacy. Efficiency because unexecuted forks of
>> a script can avoid ever hitting the chain, and privacy because hiding
>> unexecuted code leaves scripts indistinguishable to the extent that
>> their only differences are in the unexecuted parts.
>>
>> As Mark Friedenbach and others have pointed out before it is almost
>> always the case that interesting scripts have a logical top level
>> branch which allows satisfaction of the contract with nothing other
>> than a signature by all parties. Other branches would only be used
>> where some participant is failing to cooperate. More strongly stated,
>> I believe that _any_ contract with a fixed finite participant set
>> upfront can be and should be represented as an OR between an N-of-N
>> and whatever more complex contract you might want to represent.
>>
>> One point that comes up while talking about merkelized scripts is can
>> we go about making fancier contract use cases as indistinguishable as
>> possible from the most common and boring payments. Otherwise, if the
>> anonymity set of fancy usage is only other fancy usage it may not be
>> very large in practice. One suggestion has been that ordinary
>> checksig-only scripts should include a dummy branch for the rest of
>> the tree (e.g. a random value hash), making it look like there are
>> potentially alternative rules when there aren't really. The negative
>> side of this is an additional 32-byte overhead for the overwhelmingly
>> common case which doesn't need it. I think the privacy gains are
>> worth doing such a thing, but different people reason differently
>> about these trade-offs.
>>
>> It turns out, however, that there is no need to make a trade-off.
>The
>> special case of a top level "threshold-signature OR
>> arbitrary-conditions" can be made indistinguishable from a normal
>> one-party signature, with no overhead at all, with a special
>> delegating CHECKSIG which I call Taproot.
>>
>> Let's say we want to create a coin that can be redeemed by either
>> Alice && Bob or by CSV-timelock && Bob.
>>
>> Alice has public A, Bob has pubkey B.
>>
>> We compute the 2-of-2 aggregate key C = A + B. (Simplified; to
>> protect against rogue key attacks you may want to use the MuSig key
>> aggregation function [1])
>>
>> We form our timelock script S = "<timeout> OP_CSV OP_DROP B
>OP_CHECKSIGVERIFY"
>>
>> Now we tweak C to produce P which is the key we'll publish: P = C +
>H(C||S)G.
>>
>> (This is the attack hardened pay-to-contract construction described
>in [2])
>>
>> Then we pay to a scriptPubKey of [Taproot supporting version] [EC
>point P].
>>
>> Now Alice and Bob-- assuming they are both online and agree about the
>> resolution of their contract-- can jointly form a 2 of 2 signature
>for
>> P, and spend as if it were a payment to a single party (one of them
>> just needs to add H(C||S) to their private key).
>>
>> Alternatively, the Taproot consensus rules would allow this script to
>> be satisfied by someone who provides the network with C (the original
>> combined pubkey), S, and does whatever S requires-- e.g. passes the
>> CSV check and provides Bob's signature. With this information the
>> network can verify that C + H(C||S) == P.
>>
>> So in the all-sign case there is zero overhead; and no one can tell
>> that the contract alternative exists. In the alternative redemption
>> branch the only overhead is revealing the original combined pubkey
>> and, of course, the existence of the contract is made public.
>>
>> This composes just fine with whatever other merkelized script system
>> we might care to use, as the S can be whatever kind of data we want,
>> including the root of some tree.
>>
>> My example shows 2-of-2 but it works the same for any number of
>> participants (and with setup interaction any threshold of
>> participants, so long as you don't mind an inability to tell which
>> members signed off).
>>
>> The verification computational complexity of signature path is
>> obviously the same as any other plain signature (since its
>> indistinguishable). Verification of the branch redemption requires a
>> hash and a multiplication with a constant point which is strictly
>more
>> efficient than a signature verification and could be efficiently
>fused
>> into batch signature validation.
>>
>> The nearest competitor to this idea that I can come up with would
>> supporting a simple delegation where the output can be spent by the
>> named key, or a spending transaction could provide a script along
>with
>> a signature of that script by the named key, delegating control to
>the
>> signed script. Before paying into that escrow Alice/Bob would
>> construct this signature. This idea is equally efficient in the
>common
>> case, but larger and slower to verify in the alternative spend case.
>> Setting up the signature requires additional interaction between
>> participants and the resulting signature must be durably stored and
>> couldn't just be recomputed using single-party information.
>>
>> I believe this construction will allow the largest possible anonymity
>> set for fixed party smart contracts by making them look like the
>> simplest possible payments. It accomplishes this without any overhead
>> in the common case, invoking any sketchy or impractical techniques,
>> requiring extra rounds of interaction between contract participants,
>> and without requiring the durable storage of other data.
>>
>>
>> [1] https://eprint.iacr.org/2018/068
><https://eprint.iacr.org/2018/068>;
>> [2] https://blockstream.com/sidechains.pdf
><https://blockstream.com/sidechains.pdf>; Appendix A
>>
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
><https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>;
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180123/d1e58bc1/attachment-0001.html>;

📅 Original date posted:2018-01-23 📝 Original message:Thanks ...

2023-06-07T20:10:06+02:00

In reply to nevent1q…mner
_________________________

📅 Original date posted:2018-01-23
📝 Original message:Thanks Greg!

I'd be hesitant to deploy a MAST proposal without this clever application of pay-to-contract-hash now! Looks like the overhead over a more-naive MAST construction is rather trivial, too!

Matt

On January 23, 2018 12:30:06 AM UTC, Gregory Maxwell via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>Interest in merkelized scriptPubKeys (e.g. MAST) is driven by two main
>areas: efficiency and privacy. Efficiency because unexecuted forks of
>a script can avoid ever hitting the chain, and privacy because hiding
>unexecuted code leaves scripts indistinguishable to the extent that
>their only differences are in the unexecuted parts.
>
>As Mark Friedenbach and others have pointed out before it is almost
>always the case that interesting scripts have a logical top level
>branch which allows satisfaction of the contract with nothing other
>than a signature by all parties. Other branches would only be used
>where some participant is failing to cooperate. More strongly stated,
>I believe that _any_ contract with a fixed finite participant set
>upfront can be and should be represented as an OR between an N-of-N
>and whatever more complex contract you might want to represent.
>
>One point that comes up while talking about merkelized scripts is can
>we go about making fancier contract use cases as indistinguishable as
>possible from the most common and boring payments. Otherwise, if the
>anonymity set of fancy usage is only other fancy usage it may not be
>very large in practice. One suggestion has been that ordinary
>checksig-only scripts should include a dummy branch for the rest of
>the tree (e.g. a random value hash), making it look like there are
>potentially alternative rules when there aren't really. The negative
>side of this is an additional 32-byte overhead for the overwhelmingly
>common case which doesn't need it. I think the privacy gains are
>worth doing such a thing, but different people reason differently
>about these trade-offs.
>
>It turns out, however, that there is no need to make a trade-off. The
>special case of a top level "threshold-signature OR
>arbitrary-conditions" can be made indistinguishable from a normal
>one-party signature, with no overhead at all, with a special
>delegating CHECKSIG which I call Taproot.
>
>Let's say we want to create a coin that can be redeemed by either
>Alice && Bob or by CSV-timelock && Bob.
>
>Alice has public A, Bob has pubkey B.
>
>We compute the 2-of-2 aggregate key C = A + B. (Simplified; to
>protect against rogue key attacks you may want to use the MuSig key
>aggregation function [1])
>
>We form our timelock script S = "<timeout> OP_CSV OP_DROP B
>OP_CHECKSIGVERIFY"
>
>Now we tweak C to produce P which is the key we'll publish: P = C +
>H(C||S)G.
>
>(This is the attack hardened pay-to-contract construction described in
>[2])
>
>Then we pay to a scriptPubKey of [Taproot supporting version] [EC point
>P].
>
>Now Alice and Bob-- assuming they are both online and agree about the
>resolution of their contract-- can jointly form a 2 of 2 signature for
>P, and spend as if it were a payment to a single party (one of them
>just needs to add H(C||S) to their private key).
>
>Alternatively, the Taproot consensus rules would allow this script to
>be satisfied by someone who provides the network with C (the original
>combined pubkey), S, and does whatever S requires-- e.g. passes the
>CSV check and provides Bob's signature. With this information the
>network can verify that C + H(C||S) == P.
>
>So in the all-sign case there is zero overhead; and no one can tell
>that the contract alternative exists. In the alternative redemption
>branch the only overhead is revealing the original combined pubkey
>and, of course, the existence of the contract is made public.
>
>This composes just fine with whatever other merkelized script system
>we might care to use, as the S can be whatever kind of data we want,
>including the root of some tree.
>
>My example shows 2-of-2 but it works the same for any number of
>participants (and with setup interaction any threshold of
>participants, so long as you don't mind an inability to tell which
>members signed off).
>
>The verification computational complexity of signature path is
>obviously the same as any other plain signature (since its
>indistinguishable). Verification of the branch redemption requires a
>hash and a multiplication with a constant point which is strictly more
>efficient than a signature verification and could be efficiently fused
>into batch signature validation.
>
>The nearest competitor to this idea that I can come up with would
>supporting a simple delegation where the output can be spent by the
>named key, or a spending transaction could provide a script along with
>a signature of that script by the named key, delegating control to the
>signed script. Before paying into that escrow Alice/Bob would
>construct this signature. This idea is equally efficient in the common
>case, but larger and slower to verify in the alternative spend case.
>Setting up the signature requires additional interaction between
>participants and the resulting signature must be durably stored and
>couldn't just be recomputed using single-party information.
>
>I believe this construction will allow the largest possible anonymity
>set for fixed party smart contracts by making them look like the
>simplest possible payments. It accomplishes this without any overhead
>in the common case, invoking any sketchy or impractical techniques,
>requiring extra rounds of interaction between contract participants,
>and without requiring the durable storage of other data.
>
>
>[1] https://eprint.iacr.org/2018/068
>[2] https://blockstream.com/sidechains.pdf Appendix A
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev at lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180123/2ff10ec6/attachment-0001.html>;

📅 Original date posted:2017-09-10 📝 Original message:I ...

2023-06-07T20:05:50+02:00

In reply to nevent1q…t8s5
_________________________

📅 Original date posted:2017-09-10
📝 Original message:I believe there continues to be concern over a number of altcoins which
are running old, unpatched forks of Bitcoin Core, making it rather
difficult to disclose issues without putting people at risk (see, eg,
some of the dos issues which are preventing release of the alert key).
I'd encourage the list to have a discussion about what reasonable
approaches could be taken there.

On 09/10/17 18:03, Simon Liu via bitcoin-dev wrote:
> Hi,
>
> Given today's presentation by Chris Jeffrey at the Breaking Bitcoin
> conference, and the subsequent discussion around responsible disclosure
> and industry practice, perhaps now would be a good time to discuss
> "Bitcoin and CVEs" which has gone unanswered for 6 months.
>
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013751.html
>
> To quote:
>
> "Are there are any vulnerabilities in Bitcoin which have been fixed but
> not yet publicly disclosed? Is the following list of Bitcoin CVEs
> up-to-date?
>
> https://en.bitcoin.it/wiki/Common_Vulnerabilities_and_Exposures
>
> There have been no new CVEs posted for almost three years, except for
> CVE-2015-3641, but there appears to be no information publicly available
> for that issue:
>
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-3641
>
> It would be of great benefit to end users if the community of clients
> and altcoins derived from Bitcoin Core could be patched for any known
> vulnerabilities.
>
> Does anyone keep track of security related bugs and patches, where the
> defect severity is similar to those found on the CVE list above? If
> yes, can that list be shared with other developers?"
>
> Best Regards,
> Simon
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2017-07-07 📝 Original message:This ...

2023-06-07T20:04:02+02:00

In reply to nevent1q…zr6s
_________________________

📅 Original date posted:2017-07-07
📝 Original message:This is horribly under-specified (ie not possible to implement from what
you've written, and your implementation doesn't match at all, last I heard).

> Specification

> The plain block size is defined as the serialized block size without
> witness programs.
> Deploy a modified BIP91 to activate Segwit. The only modification is
> that the signal "segsignal" is replaced by "segwit2x".

This is not a protocol change. I have no idea why you included it in the
"specification" section.

> If segwit2x (BIP91 signal) activates at block N, then block N+12960
> activates a new plain block size limit of 2 MB (2,000,000 bytes). In
> this case, at block N+12960 a hard-fork occurs.

This is not a hard fork, simply adding a new limit is a soft fork. You
appear to be confused - as originally written, AFAIR, Jeff's btc1 branch
did not increase the block size, your specification here matches that
original change, and does not increase the block size.

> The block that activates the hard-fork must have a plain block size
> greater than 1 MB.

There is no hard fork, and this would violate consensus rules. Not sure
what you mean. If you do add a hard fork to this BIP, you really need to
flip the hard fork bit.

> Any transaction with a non-witness serialized size exceeding 1,000,000
> is invalid.

This is far from sufficient to protect from DoS attacks, you really
should take a look through the mailing list archives and read some of
the old discussions on the issues here.

Matt

On 07/07/17 18:25, Sergio Demian Lerner via bitcoin-dev wrote:
> Hello,
>
> Here is a BIP that matches the reference code that the Segwit2x group
> has built and published a week ago.
>
> This BIP and code satisfies the requests of a large part of the Bitcoin
> community for a moderate increase in the Bitcoin non-witness block space
> coupled with the activation of Segwit.
>
> You can find the BIP draft in the following link:
>
> https://github.com/SergioDemianLerner/BIPs/blob/master/BIP-draft-sergiolerner-segwit2x.mediawiki
>
> Reference source was kindly provided by the Segwit2x group.
>
> Best regards,
> Sergio.
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2017-03-28 📝 Original message:Not ...

2023-06-07T19:58:01+02:00

In reply to nevent1q…5tkv
_________________________

📅 Original date posted:2017-03-28
📝 Original message:Not sure what "last week's meeting" is in reference to?

Agreed that the hard fork should be well-prepared, but I think its
dangerous to think that a hard fork as agreed upon would be a simple
relaxation of the block size. For example, Johnson Lau's previous
proposal, Spoonnet, which I think is probably one of the better ones,
would be incompatible with these rules.

I, of course, worry about what happens if we cannot come to consensus on
a number to soft fork down to, potentially significantly risking miner
profits (and, thus, the security of Bitcoin) if a group is able to keep
things "at the status quo". That said, for that to be alleviated we
could simply do something based on historical transaction growth (which
is somewhat linear, with a few inflection points), but that number ends
up being super low (eg somewhere around 2MB at the next halving, which
SegWit itself already provides :/.

We could, of course, focus on designing a hard fork's activation and
technical details, with a very large block size increase in it (ie
closer to 4/6MB at the next halving or so, something we at least could
be confident we could develop software for), with intention to soft fork
it back down if miner profits are suffering.

Matt

On 03/28/17 16:59, Wang Chun via bitcoin-dev wrote:
> I've proposed this hard fork approach last year in Hong Kong Consensus
> but immediately rejected by coredevs at that meeting, after more than
> one year it seems that lots of people haven't heard of it. So I would
> post this here again for comment.
>
> The basic idea is, as many of us agree, hard fork is risky and should
> be well prepared. We need a long time to deploy it.
>
> Despite spam tx on the network, the block capacity is approaching its
> limit, and we must think ahead. Shall we code a patch right now, to
> remove the block size limit of 1MB, but not activate it until far in
> the future. I would propose to remove the 1MB limit at the next block
> halving in spring 2020, only limit the block size to 32MiB which is
> the maximum size the current p2p protocol allows. This patch must be
> in the immediate next release of Bitcoin Core.
>
> With this patch in core's next release, Bitcoin works just as before,
> no fork will ever occur, until spring 2020. But everyone knows there
> will be a fork scheduled. Third party services, libraries, wallets and
> exchanges will have enough time to prepare for it over the next three
> years.
>
> We don't yet have an agreement on how to increase the block size
> limit. There have been many proposals over the past years, like
> BIP100, 101, 102, 103, 104, 105, 106, 107, 109, 148, 248, BU, and so
> on. These hard fork proposals, with this patch already in Core's
> release, they all become soft fork. We'll have enough time to discuss
> all these proposals and decide which one to go. Take an example, if we
> choose to fork to only 2MB, since 32MiB already scheduled, reduce it
> from 32MiB to 2MB will be a soft fork.
>
> Anyway, we must code something right now, before it becomes too late.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2017-03-23 📝 Original message:I ...

2023-06-07T19:57:42+02:00

In reply to nevent1q…ru7f
_________________________

📅 Original date posted:2017-03-23
📝 Original message:I haven't investigated, but you may be seeing segwit-invalid blocks...0.13.0+ nodes will enforce segwit as it activated some time ago on testnet, 0.12.X nodes will not.

On March 23, 2017 3:37:34 PM PDT, Juan Garavaglia via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>We notice some reorgs in Bitcoin testnet, while reorgs in testnet are
>common and may be part of different tests and experiments, it seems the
>forks are not created by a single user and multiple blocks were mined
>by different users in each chain. My first impression was that the
>problem was related to network issues but some Bitcoin explorers were
>following one chain while others follow the other one. Nonetheless,
>well established explorers like blocktrail.com or blockr.io were
>following different chains at different heights which led to me to
>believe that it was not a network issue. After some time, a reorg
>occurs and it all comes to normal state as a single chain.
>We started investigating more and we identified that the fork occurs
>with nodes 0.12; in some situations, nodes 0.12 has longer/different
>chains. The blocks in both chains are valid so something must be
>occurring in the communication between nodes but not related with the
>network itself.
>Long story short, when nodes 0.13+ receive blocks from 0.13+ nodes all
>is ok, and those blocks propagate to older nodes with no issues. But
>when a block tries to be propagated from bitcoind 0.12.+ to newer ones
>those blocks are NOT being propagated to the peers with newer versions
>while these newer blocks are being propagated to peers with older
>versions with no issues.
>My conclusion is that we have a backward compatibility issue between
>0.13.X+ and older versions.
>The issue is simple to replicate, first, get latest version of
>bitcoind, complete the IBD after is at current height, then force it to
>use exclusively one or more peers of versions 0.12.X and older, and you
>will notice that the latest version node will never receive a new
>block.
>Probably some alternative bitcoin implementations act as bridges
>between these two versions and facilitate the chain reorgs.
>I have not yet found any way where/how it can be used in a malicious
>way or be exploited by a miner but in theory Bitcoin 0.13.X+ should
>remain compatible with older ones, but a 0.13+ node may become isolated
>by 0.12 peers, and there is not notice for the node owner.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170323/f7be920b/attachment.html>;

📅 Original date posted:2016-05-11 📝 Original ...

2023-06-07T19:50:31+02:00

In reply to nevent1q…ax3v
_________________________

📅 Original date posted:2016-05-11
📝 Original message:Indeed, I think the "ASICs are bad, because 1-CPU-1-vote" arguments
mostly died out long ago, and, indeed, the goal that many making those
arguments had of building "unoptimizeable" ASICs failed with them.

I think everyone understands that there will always be some ability to
iterate on ASIC designs, however, a patented optimization breaks that
assumption. Instead of being freely able to optimize their ASIC design,
patented optimizations require that people who discover such
optimizations themselves do not use them, giving one
manufacturer/licenser a huge influence in who is successful in a market
that we're all relying on remaining rather flat. Indeed, with AsicBoost,
we saw Spondoolies independently discover the same optimization, but
with the current legal system they would not have been able to sell such
systems without licensing AsicBoost.

Matt

On 05/11/16 13:08, Marek Palatinus via bitcoin-dev wrote:
> Ehm, I though those discussions about "ASICs are bad, because X" ended
> years ago by starting "ASIC unfriendly" altcoins. ASIC industry is
> twisted even without AsicBoost. I don't see any particular reason why to
> change rules just because of 10% edge.
>
> This is opening Pandora box and it is potentially extremely dangerous
> for the health of the network. You cannot know in advance what you'll
> break by changing the rules.
>
> Disclaimer: I don't have any stake in any ASIC company/facility.
>
> slush
>
> On Wed, May 11, 2016 at 2:20 PM, Sergio Demian Lerner via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
>
>
> On Tue, May 10, 2016 at 6:43 PM, Sergio Demian Lerner
> <sergio.d.lerner at gmail.com <mailto:sergio.d.lerner at gmail.com>> wrote:
>
>
>
> You can find it here:
> https://bitslog.wordpress.com/2014/03/18/the-re-design-of-the-bitcoin-block-header/
>
> Basically, the idea is to put in the first 64 bytes a 4 byte
> hash of the second 64-byte chunk. That design also allows
> increased nonce space in the first 64 bytes.
>
> My mistake here. I didn't recalled correctly my own idea. The idea
> is to include in the second 64-byte chunk a 4-byte hash of the first
> chunk, not the opposite.
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2016-05-10 📝 Original ...

2023-06-07T19:50:29+02:00

In reply to nevent1q…z40r
_________________________

📅 Original date posted:2016-05-10
📝 Original message:Replies inline.

On 05/10/16 21:43, Sergio Demian Lerner via bitcoin-dev wrote:
-snip-

> But some ASIC companies already have cores that are better (on power,
> cost, rate, temperature, etc.) than competing companies ASICs. Why do
> you think a 10% improvement from AsicBoost is different from many of
> other improvements they already have (secretly) added? Maybe we (?)
> should only allow ASICs that have a 100% open source designs?

One is patented and requires paying a license fee to a group, or more
likely, ends up with it being impossible to import hardware from other
jurisdictions into the US/western world. The other requires more
investment in R&D, and over the long run, there is no guaranteed
advantage to such groups.

> If we change the protocol then the message to the ecosystem is that ASIC
> optimizations should be kept secret.

To some extent, this is the case, but there is a strong difference
between a guaranteed advantage enforced by the legal system and one that
is true due to intellectual superiority. In the long run, I am confident
the second will not remain the case. For example, AsicBoost was
independently discovered by at least two companies/individuals within a
year or two.

> It is fair to change the protocol
> because we don't like that certain ASIC manufacturer has better chips,
> if the chips are sold in the market and anyone can buy them? And what
> about using approximate adders (30% improvement), or dual rail
> asynchronous adders (also more than 10% improvement) ? How do we repair
> those?

As far as I'm aware neither of these are patented. Is this not the case?

> Disclaimer: I have stake in AsicBoost, but I'm not sure about this.
>
>
> On Tue, May 10, 2016 at 5:27 PM, Tier Nolan via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
> The various chunks in the double SHA256 are
>
> Chunk 1: 64 bytes
> version
> previous_block_digest
> merkle_root[31:4]
>
> Chunk 2: 64 bytes
> merkle_root[3:0]
> nonce
> timestamp
> target
>
> Chunk 3: 64 bytes
> digest from first sha pass
>
> Their improvement requires that all data in Chunk 2 is identical
> except for the nonce. With 4 bytes, the birthday paradox means
> collisions can be found reasonable easily.
>
> If hard forks are allowed, then moving more of the merkle root into
> the 2nd chunk would make things harder. The timestamp and target
> could be moved into chunk 1. This increases the merkle root to 12
> bytes in the 2nd chunk. Finding collisions would be made much more
> difficult.
>
> If ASIC limitations mean that the nonce must stay where it is, this
> would mean that the merkle root would be split into two pieces.
>
> On Tue, May 10, 2016 at 7:57 PM, Peter Todd via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
> As part of the hard-fork proposed in the HK agreement(1) we'd
> like to make the
> patented AsicBoost optimisation useless, and hopefully make
> further similar
> optimizations useless as well.
>
> What's the best way to do this? Ideally this would be SPV
> compatible, but if it
> requires changes from SPV clients that's ok too. Also the fix
> this should be
> compatible with existing mining hardware.
>
>
> 1)
> https://medium.com/@bitcoinroundtable/bitcoin-roundtable-consensus-266d475a61ff
>
> 2)
> http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-April/012596.html
>
> --
> https://petertodd.org 'peter'[:-1]@petertodd.org
> <http://petertodd.org>;
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2016-05-10 📝 Original message:Yea, I ...

2023-06-07T19:50:29+02:00

In reply to nevent1q…r6d5
_________________________

📅 Original date posted:2016-05-10
📝 Original message:Yea, I think in any hardfork that we should be talking about, a part of
it should include 1) fix the version field so its a static constant, 2)
the merkle root becomes hash of the real block header 3) swap first 2
bytes of the merkle root with the timestamp's two high-order bits, 4)
swap the next 4 bytes of the merkle root with the difficulty field.

I believe this should be compatible with all existing ASICs, with the
exception, possibly, of some 21 Inc hardware. I believe this fixes
AsicBoost (without thinking about it tooo much, so please critique).

While this is somewhat nasty, the risks of AsicBoost and the precedent
that should be set necessitates a response, and it should be included in
any hardfork.

Matt

On 05/10/16 20:27, Tier Nolan via bitcoin-dev wrote:
> The various chunks in the double SHA256 are
>
> Chunk 1: 64 bytes
> version
> previous_block_digest
> merkle_root[31:4]
>
> Chunk 2: 64 bytes
> merkle_root[3:0]
> nonce
> timestamp
> target
>
> Chunk 3: 64 bytes
> digest from first sha pass
>
> Their improvement requires that all data in Chunk 2 is identical except
> for the nonce. With 4 bytes, the birthday paradox means collisions can
> be found reasonable easily.
>
> If hard forks are allowed, then moving more of the merkle root into the
> 2nd chunk would make things harder. The timestamp and target could be
> moved into chunk 1. This increases the merkle root to 12 bytes in the
> 2nd chunk. Finding collisions would be made much more difficult.
>
> If ASIC limitations mean that the nonce must stay where it is, this
> would mean that the merkle root would be split into two pieces.
>
> On Tue, May 10, 2016 at 7:57 PM, Peter Todd via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
> As part of the hard-fork proposed in the HK agreement(1) we'd like
> to make the
> patented AsicBoost optimisation useless, and hopefully make further
> similar
> optimizations useless as well.
>
> What's the best way to do this? Ideally this would be SPV
> compatible, but if it
> requires changes from SPV clients that's ok too. Also the fix this
> should be
> compatible with existing mining hardware.
>
>
> 1)
> https://medium.com/@bitcoinroundtable/bitcoin-roundtable-consensus-266d475a61ff
>
> 2)
> http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-April/012596.html
>
> --
> https://petertodd.org 'peter'[:-1]@petertodd.org <http://petertodd.org>;
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2016-05-10 📝 Original ...

2023-06-07T19:50:22+02:00

In reply to nevent1q…ldxk
_________________________

📅 Original date posted:2016-05-10
📝 Original message:Replies inline.

On May 10, 2016 5:23:55 PM EDT, Rusty Russell via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>Gregory Maxwell <greg at xiph.org> writes:
>> On Tue, May 10, 2016 at 5:28 AM, Rusty Russell via bitcoin-dev
>> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>>> I used variable-length bit encodings, and used the shortest encoding
>>> which is unique to you (including mempool). It's a little more
>work,
>>> but for an average node transmitting a block with 1300 txs and
>another
>>> ~3000 in the mempool, you expect about 12 bits per transaction.
>IOW,
>>> about 1/5 of your current size. Critically, we might be able to fit
>in
>>> two or three TCP packets.
>>
>> Hm. 12 bits sounds very small even giving those figures. Why failure
>> rate were you targeting?
>
>That's a good question; I was assuming a best-case in which we have
>mempool set reconciliation (handwave) thus know they are close. But
>there's also an alterior motive: any later more sophisticated approach
>will want variable-length IDs, and I'd like Matt to do the work :)

Yea, there's already an ongoing discussion of that, and the UDP stuff will definitely want something different than the current proposals.

>In particular, you can significantly narrow the possibilities for a
>block by sending the min-fee-per-kb and a list of "txs in my mempool
>which didn't get in" and "txs which did despite not making the
>fee-per-kb". Those turn out to be tiny, and often make set
>reconciliation trivial. That's best done with variable-length IDs.
>
>> (*Not interesting because it mostly reduces exposure to loss and the
>> gods of TCP, but since those are the long poles in the latency tent,
>> it's best to escape them entirely, see Matt's udp_wip branch.)
>
>I'm not convinced on UDP; it always looks impressive, but then ends up
>reimplementing TCP in practice. We should be well within a TCP window
>for these, so it's hard to see where we'd win.

Not at all. The goal with the UDP stuff I've been working on is not to provide reliable transport. Like the relay network, it is assumed some percent of blocks will fail to transit properly, and you will use some other transport to figure out how to get the block. Indeed, a big part of my desire for diversity in network protocols is to enable them to make tradeoffs in reliability/privacy/etc.

>>> I would also avoid the nonce to save recalculating for each node,
>and
>>> instead define an id as:
>>
>> Doing this would greatly increase the cost of a collision though, as
>> it would happen in many places in the network at once over the on the
>> network at once, rather than just happening on a single link, thus
>> hardly impacting overall propagation.
>
>"Greatly increase"? I don't see that.
>
>Let's assume an attacker grinds out 10,000 txs with 128 bits of the
>same
>TXID, and gets them all in a block. They then win the lottery and get
>a
>collision. Now we have to transmit ~48 bytes more than expected.

I assume what Greg was referring to the idea that if there is a conflict, a given block will require an extra round trip when being broadcast between roughly each peer, compounding the effect across each hop.

>> Using the same nonce means you also would not get a recovery gain
>from
>> jointly decoding using compact blocks sent from multiple peers (which
>> you'll have anyways in high bandwidth mode).
>
>Not quite true, since if their mempools differ they'll use different
>encoding lengths, but yes, you'll get less of this.

... Assuming different encoding lengths aren't just truncated, but ok :).

>> With a nonce a sender does have the option of reusing what they got--
>> but the actual encoding cost is negligible, for a 2500 transaction
>> block its 27 microseconds (once per block, shared across all peers)
>> using Pieter's suggestion of siphash 1-3 instead of the cheaper
>> construct in the current draft.
>>
>> Of course, if you're going to check your whole mempool to reroll the
>> nonce, thats another matter-- but that seems wasteful compared to
>just
>> using a table driven size with a known negligible failure rate.
>
>I'm not worried about the sender: The recipient needs to encode all the
>mempool.
>
>>> As Peter R points out, we could later enhance receiver to brute
>force
>>> collisions (you could speed that by sending a XOR of all the txids,
>but
>>> really if there are more than a few collisions, give up).
>>
>> The band between "no collisions" and "infeasible many" is fairly
>> narrow. You can add a small amount more space to the ids and
>> immediately be in the no collision zone.
>
>Indeed, I would be adding extra bits in the sender and not implementing
>brute force in the receiver. But I welcome someone else to do so.
>
>Cheers,
>Rusty.
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev at lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

📅 Original date posted:2016-05-17 📝 Original ...

2023-06-07T19:50:22+02:00

In reply to nevent1q…m7cp
_________________________

📅 Original date posted:2016-05-17
📝 Original message:Implemented a few of your suggestions.

Also opened a formal pull request for the BIP at
https://github.com/bitcoin/bips/pull/389 and the code at
https://github.com/bitcoin/bitcoin/pull/8068.

On 05/09/16 17:06, Pieter Wuille via bitcoin-dev wrote:
> On 05/03/2016 12:13 AM, lf-lists at mattcorallo.com (Matt Corallo) wrote:
>> Hi all,
>>
>> The following is a BIP-formatted design spec for compact block relay
>> designed to limit on wire bytes during block relay. You can find the
>> latest version of this document at
>> https://github.com/TheBlueMatt/bips/blob/master/bip-TODO.mediawiki.
>
> Hi Matt,
>
> thank you for working on this!
>
>> ===New data structures===
>> Several new data structures are added to the P2P network to relay
>> compact blocks: PrefilledTransaction, HeaderAndShortIDs,
>> BlockTransactionsRequest, and BlockTransactions. Additionally, we
>> introduce a new variable-length integer encoding for use in these data
>> structures.
>>
>> For the purposes of this section, CompactSize refers to the
>> variable-length integer encoding used across the existing P2P protocol
>> to encode array lengths, among other things, in 1, 3, 5 or 9 bytes.
>
> This is a not, but I think it's a bit strange to have two separate
> variable length integers in the same specification. I understand is one
> is already the default for variable-length integers currently, and there
> are reasons to use the other one for efficiency reasons in some places,
> but perhaps we should aim to get everything using the latter?

Fixed, the whole thing now uses New Varints.

>> ====New VarInt====
>> Variable-length integers: bytes are a MSB base-128 encoding of the number.
>> The high bit in each byte signifies whether another digit follows. To make
>> sure the encoding is one-to-one, one is subtracted from all but the last
>> digit.
>
> Maybe it's worth mentioning that it is based on ASN.1 BER's compressed
> integer format (see
> https://www.itu.int/ITU-T/studygroups/com17/languages/X.690-0207.pdf
> section 8.1.3.5), though with a small modification to make every integer
> have a single unique encoding.
>
>> ====HeaderAndShortIDs====
>> A HeaderAndShortIDs structure is used to relay a block header, the short
>> transactions IDs used for matching already-available transactions, and a
>> select few transactions which we expect a peer may be missing.
>>
>> |shortids||List of uint64_ts||8*shortids_length bytes||Little
>> Endian||The short transaction IDs calculated from the transactions which
>> were not provided explicitly in prefilledtxn
>
> I tried to derive what length of short ids is actually necessary (some
> write-up is on
> https://gist.github.com/sipa/b2eb2e486156b5509ac711edd16153ed but it's
> incomplete).
>
> For any reasonable numbers I can come up with (in a very wide range),
> the number of bits needed is very well approximated by:
>
> log2(#receiver_mempool_txn * #block_txn_not_in_receiver_mempool /
> acceptable_per_block_failure_rate)
>
> For example, with 20000 mempool transactions, 2500 transactions in a
> block, 95% hitrate, and a chance of 1 in 10000 blocks to fail to
> reconstruct, needed_bits = log2(20000 * 2500 * (1 - 0.95) / 0.0001) =
> 34.54, or 5 byte txids would suffice.
>
> Note that 1 in 10000 failures may sound like a lot, but this is for each
> individual connection, and since every transmission uses separately
> salted identifiers, occasional failures should not affect global
> propagation. Given that transmission failures due to timeouts, network
> connectivity, ... already occur much more frequently than once every few
> gigabytes (what 10000 blocks corresponds to), that's probably already
> more than enough.
>
> In short: I believe 5 or 6 byte txids should be enough, but perhaps it
> makes sense to allow the sender to choose (so he can weigh trying
> multiple nonces against increasing the short txid length).

I switched to 6-byte short txids.

>> ====Short transaction IDs====
>> Short transaction IDs are used to represent a transaction without
>> sending a full 256-bit hash. They are calculated by:
>> # single-SHA256 hashing the block header with the nonce appended (in
>> little-endian)
>> # XORing each 8-byte chunk of the double-SHA256 transaction hash with
>> each corresponding 8-byte chunk of the hash from the previous step
>> # Adding each of the XORed 8-byte chunks together (in little-endian)
>> iteratively to find the short transaction ID
>
> An alternative would be using SipHash-1-3 (a form of SipHash with
> reduced iteration counts; the default is SipHash-2-4). SipHash was
> designed as a Message Authentication Code, where the security
> requirements are much stronger than in our case (in particular, we don't
> care about observers being able to finding the key, as the key is just
> public knowledge here). One of the designers of SipHash has commented
> that SipHash-1-3 for collision resistance in hash tables may be enough:
> https://github.com/rust-lang/rust/issues/29754#issuecomment-156073946
>
> Using SipHash-1-3 on modern hardware would take ~32 CPU cycles per txid.

Switched to SipHash2-4.

>> ===Implementation Notes===
>
> There are a few more heuristics that MAY be used to improve performance:
>
> * Receivers should treat short txids in blocks that match multiple
> mempool transactions as non-matches, and request the transactions. This
> significantly reduces the failure to reconstruct.

Done.

> * When constructing a compact block to send, the sender can verify it
> against its own mempool to check for collisions, and if so, choose to
> either try another nonce, or increase the short txid length.

Additionally we should compare to the orphan pool (which apparently
helps a lot).

📅 Original date posted:2016-05-08 📝 Original message:(This ...

2023-06-07T19:50:17+02:00

In reply to nevent1q…saxj
_________________________

📅 Original date posted:2016-05-08
📝 Original message:(This response was originally off-list as moderators were still
deciding, here it is for those interested).

Hi Tom,

Thanks for reading the draft text and commenting! Replies inline.

Matt

On 05/08/16 00:40, Johnathan Corgan wrote:
> ---------- Forwarded message ----------
> From: Tom <tomz at freedommail.ch <mailto:tomz at freedommail.ch>>
> To: bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>, Matt Corallo <lf-lists at mattcorallo.com <mailto:lf-lists at mattcorallo.com>>
> Cc:
> Date: Fri, 06 May 2016 13:31:15 +0100
> Subject: Re: [bitcoin-dev] Compact Block Relay BIP
> On Monday 02 May 2016 22:13:22 Matt Corallo via bitcoin-dev wrote:
>
> Thanks for putting in the time to make a spec!
>
> It looks good already, but I do think some more improvements can be made.
>
>
>> ===Intended Protocol Flow===
> I'm not a fan of the solution that a CNode should keep state and talk to
> its remote nodes differently while announcing new blocks.
> Its too complicated and ultimately counter-productive.
>
> The problem is that an individual node needs to predict network behaviour in
> advance. With the downside that if it guesses wrong that both nodes end up
> paying for the wrong guess.
> This is not a good way to design a p2p layer.

Nodes don't need to predict much in advance, and the cost for predicting
wrong is 0 if your peers receive blocks with a few hundred ms between
them (as we should expect) and you haven't set the announce bit on more
than a few peers (as the spec requires for this reason). As for
complexity of keeping state, think of it as a version flag in much the
same way sendheaders operates.

It seems I forgot to add a suggested peer-preforwarding-selection
algorithm in the text, but the intended use-case is to set the bit on
peers which recently provided you blocks faster than other peers, up to
only one or three peers. This is both simple and should be incredibly
effective.

[This has now been clarified in the BIP text]

> I would suggest that a new block is announced to all nodes equally and then
> individual nodes can respond with a request of either a 'compact' or a
> normal block.
> This is much more in line with the current design as well.
>
> Detection if remote nodes support compact blocks, for the purpose of
> requesting a compact-block, can be done either via a network-bit or just a
> protocol version. Or something else entirely, if you have better
> suggestions.

In line with recent trends, neither service bits nor protocol versions
are particularly well-suited for this purpose. Protocol versions are
impossible to handle sanely across different nodes on the network, as
they cannot indicate optional features. Service bits, while somewhat
more appropriate for this purpose, are a very limited resource which is
generally better suited to indicating significant new features which
nodes might need for correct operation, and thus might wish to actively
seek out when making connections. I'm not sure anyone is suggesting that
here, and absent that recent agreement preferred message-based feature
indication instead of version-message-extension.

>> Variable-length integers: bytes are a MSB base-128 encoding of the
>> number.
>> The high bit in each byte signifies whether another digit follows.
>> [snip bitwise spec]
>
> I suggest just referring to UTF-8 which describes this just fine.
> it is good practice to refer to existing specs when possible and not copy
> the details.

Hmm? There is no UTF anywhere in this protocol. Indeed this section
needs to be rewritten, as indicated. I'd recommend you read the code
until I update the section with better text if you're confused.

>> ====Short transaction IDs====
>> Short transaction IDs are used to represent a transaction without
>> sending a full 256-bit hash. They are calculated by:
>> # single-SHA256 hashing the block header with the nonce appended (in
>> little-endian)
>> # XORing each 8-byte chunk of the double-SHA256 transaction hash with
>> each corresponding 8-byte chunk of the hash from the previous step
>> # Adding each of the XORed 8-byte chunks together (in little-endian)
>> iteratively to find the short transaction ID
>
> I don't think this is needed. Just use the first 8 bytes.
> The reason to do xor-ing doesn't hold up and extra complexity is unneeded.
> Especially since you mention some lines down;
>
>> The short transaction ID calculation is designed to take absolutely
>> minimal processing time during block compaction to avoid introducing
>> serious DoS vulnerabilities

I'm confused as to what, specifically, you're proposing this be changed
to. I'm pretty sure the proposed protocol is about as simple as you can
get while retaining some reasonable collision resistance. I might,
however, decide to switch to siphash with a very low round count, given
that it's probably faster than the cache-fill-time taken by just
iterating over the mempool. Needs a bit further investigation.

> ==Acknowledgements==
>
> I think you need to acknowledge some more people, or just remove this
> paragraph.
>
> Cheers

Greg was the only large contributor to the document (and was a very
large contributor, as mentioned - the work is based hugely on a protocol
recommendation he wrote up several years ago) don't see why this should
mean he doesn't get credit.

[For those interested, I'm referring here to
https://en.bitcoin.it/wiki/User:Gmaxwell/block_network_coding. This
BIP/the implementation is a precursor to an implementation that looks
similar to what Greg proposes there which can be found on my udp-wip
branch, which is based on and uses the data structures involved here.]

📅 Original date posted:2016-05-06 📝 Original message:Thanks ...

2023-06-07T19:50:16+02:00

In reply to nevent1q…m04g
_________________________

📅 Original date posted:2016-05-06
📝 Original message:Thanks Greg for the testing!

Note that to those who are reviewing the doc, a few minor tweaks to
wording and clarification have been made to the git version, so please
review there.

On 05/03/16 05:02, Gregory Maxwell wrote:
> On Mon, May 2, 2016 at 10:13 PM, Matt Corallo via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>> Hi all,
>>
>> The following is a BIP-formatted design spec for compact block relay
>> designed to limit on wire bytes during block relay. You can find the
>> latest version of this document at
>> https://github.com/TheBlueMatt/bips/blob/master/bip-TODO.mediawiki.
>
> Thanks Matt!
>
> I've been testing this for a couple weeks (in various forms). I've
> been getting over 96% reduction in block-bytes sent. I don't have a
> good metric for it, but bandwidth spikes are greatly reduced. The
> largest blocktxn message I've seen on a node that has been up for at
> least a day is 475736 bytes. 94% of the blocks less than 100kb must be
> sent in total.
>
> In the opportunistic mode my measurements are showing 73% of blocks
> transferred with 0.5 RTT even without prediction, 87% if up to 4
> additional transactions are predicted, and 91% for 30 transactions (my
> rough estimate for the 10k maximum prediction suggested in the BIP.
>

📅 Original date posted:2016-01-08 📝 Original ...

2023-06-07T19:47:43+02:00

In reply to nevent1q…g87c
_________________________

📅 Original date posted:2016-01-08
📝 Original message:Indeed, anything which uses P2SH is obviously vulnerable if there is an attack on RIPEMD160 which reduces it's security only marginally. While no one thought hard about these attacks when P2SH was designed, we realized later this was not such a good idea to reuse the structure from P2PKH. Hence why this discussion came up.

On January 7, 2016 7:30:11 PM PST, Rusty Russell via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>Pieter Wuille via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org>
>writes:
>> Yes, this is what I worry about. We're constructing a 2-of-2 multisig
>> escrow in a contract. I reveal my public key A, you do a 80-bit
>search for
>> B and C such that H(A and B) = H(B and C). You tell me your keys B,
>and I
>> happily send to H(A and B), which you steal with H(B and C).
>
>FWIW, this attack would effect the current lightning-network
>"deployable
>lightning" design at channel establishment; we reveal our pubkey in the
>opening packet (which is used to redeem a P2SH using normal 2of2).
>
>At least you need to grind before replying (which will presumably time
>out), rather than being able to do it once the channel is open.
>
>We could pre-commit by exchanging hashes of pubkeys first, but
>contracts
>on bitcoin are hard enough to get right that I'm reluctant to add more
>hoops.
>
>Cheers,
>Rusty.
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev at lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

📅 Original date posted:2016-01-07 📝 Original message:So ...

2023-06-07T19:47:40+02:00

In reply to nevent1q…vt0y
_________________________

📅 Original date posted:2016-01-07
📝 Original message:So just because other attacks are possible we should weaken the crypto
we use? You may feel comfortable weakening crypto used to protect a few
billion dollars of other peoples' money, but I dont.

On 01/07/16 23:39, Gavin Andresen via bitcoin-dev wrote:
> Thanks, Ethan, that's helpful and I'll stop thinking that collision
> attacks require 2^(n/2) memory...
>
> So can we quantify the incremental increase in security of
> SHA256(SHA256) over RIPEMD160(SHA256) versus the incremental increase in
> security of having a simpler implementation of segwitness?
>
> I'm going to claim that the difference in the first case is very, very,
> very small-- the risk of an implementation error caused by having
> multiple ways of interpreting the segwitness hash in the scriptPubKey is
> much, much greater.
>
> And even if there IS some risk of collision attack now or at some point
> in the future, I claim that it is easy for wallets to mitigate that
> risk. In fact, the principle of security in depth means wallets that
> don't completely control the scriptPubKeys they're creating on behalf of
> users SHOULD be coded to mitigate that risk (e.g. not allowing arbitrary
> data around a user's public key in a Script so targeted substring
> attacks are eliminated entirely).
>
> Purely from a security point of view, I think a single 20-byte
> segwitness in the scriptPubKey is the best design.
> "Keep the design as simple and small as possible"
> https://www.securecoding.cert.org/confluence/plugins/servlet/mobile#content/view/2426
>
> Add in the implied capacity increase of smaller scriptPubKeys and I
> still think it is a no-brainer.
>
>
> On Thu, Jan 7, 2016 at 5:56 PM, Ethan Heilman <eth3rs at gmail.com
> <mailto:eth3rs at gmail.com>> wrote:
>
> >Ethan: your algorithm will find two arbitrary values that collide. That isn't useful as an attack in the context we're talking about here (both of those values will be useless as coin destinations with overwhelming probability).
>
> I'm not sure exactly the properties you want here and determining
> these properties is not an easy task, but the case is far worse than
> just two random values. For instance: (a). with a small modification
> my algorithm can also find collisions containing targeted substrings,
> (b). length extension attacks are possible with RIPEMD160.
>
> (a). targeted cycles:
>
> target1 = "str to prepend"
> target2 = "str to end with"
>
> seed = {0,1}^160
> x = hash(seed)
>
> for i in 2^80:
> ....x = hash(target1||x||target2)
> x_final = x
>
> y = hash(tartget1||x_final||target2)
>
> for j in 2^80:
> ....if y == x_final:
> ........print "cycle len: "+j
> ........break
> ....y = hash(target1||y||target2)
>
> If a collision is found, the two colliding inputs must both start with
> "str to prepend" and end with the phrase "str to end with". As before
> this only requires 2^81.5 computations and no real memory. For an
> additional 2**80 an adversary has an good change of finding two
> different targeted substrings which collide. Consider the case where
> the attacker mixes the targeted strings with the hash output:
>
> hash("my name is=0x329482039483204324423"+x[1]+", my favorite number
> is="+x) where x[1] is the first bit of x.
>
> (b). length extension attacks
>
> Even if all the adversary can do is create two random values that
> collide, you can append substrings to the input and get collisions.
> Once you find two random values hash(x) = hash(y), you could use a
> length extension attack on RIPEMD-160 to find hash(x||z) = hash(y||z).
>
> Now the bitcoin wiki says:
> "The padding scheme is identical to MD4 using Merkle–Damgård
> strengthening to prevent length extension attacks."[1]
>
> Which is confusing to me because:
>
> 1. MD4 is vulnerable to length extension attacks
> 2. Merkle–Damgård strengthening does not protect against length
> extension: "Indeed, we already pointed out that none of the 64
> variants above can withstand the 'extension' attack on the MAC
> application, even with the Merkle-Damgard strengthening" [2]
> 3. RIPEMD-160 is vulnerable to length extension attacks, is Bitcoin
> using a non-standard version of RIPEMD-160.
>
> RIPEMD160(SHA256()) does not protect against length extension attacks
> on SHA256, but should protect RIPEMD-160 against length extension
> attacks as RIPEMD-160 uses 512-bit message blocks. That being said we
> should be very careful here. Research has been done that shows that
> cascading the same hash function twice is weaker than using HMAC[3]. I
> can't find results on cascading RIPEMD160(SHA256()).
>
> RIPEMD160(SHA256()) seems better than RIPEMD160() though, but security
> should not rest on the notion that an attacker requires 2**80 memory,
> many targeted collision attacks can work without much memory.
>
> [1]: https://en.bitcoin.it/wiki/RIPEMD-160
> [2]: "Merkle-Damgard Revisited: How to Construct a Hash Function"
> https://www.cs.nyu.edu/~puniya/papers/merkle.pdf
> [3]: https://www.cs.nyu.edu/~dodis/ps/h-of-h.pdf
>
> On Thu, Jan 7, 2016 at 4:06 PM, Gavin Andresen via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
> > Maybe I'm asking this question on the wrong mailing list:
> >
> > Matt/Adam: do you have some reason to think that RIPEMD160 will be
> broken
> > before SHA256?
> > And do you have some reason to think that they will be so broken
> that the
> > nested hash construction RIPEMD160(SHA256()) will be vulnerable?
> >
> > Adam: re: "where to stop" : I'm suggesting we stop exactly at
> the current
> > status quo, where we use RIPEMD160 for P2SH and P2PKH.
> >
> > Ethan: your algorithm will find two arbitrary values that
> collide. That
> > isn't useful as an attack in the context we're talking about here
> (both of
> > those values will be useless as coin destinations with overwhelming
> > probability).
> >
> > Dave: you described a first preimage attack, which is 2**160 cpu
> time and no
> > storage.
> >
> >
> > --
> > --
> > Gavin Andresen
> >
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >
>
>
>
>
> --
> --
> Gavin Andresen
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2016-01-07 📝 Original message:We ...

2023-06-07T19:47:37+02:00

In reply to nevent1q…dnd9
_________________________

📅 Original date posted:2016-01-07
📝 Original message:We absolutely should be worried about 80-bit collision resistance.
Collisions only take 2**80 work if the hash is theoretically perfect,
which is never the case, not to mention that collision resistance is
almost always the first thing to go for hash functions, and often starts
to get easier slowly long, long before anyone is truly worried about the
security of the hash function.

I would never assume RIPEMD160's collision resistance is 2**80, and
would definitely never wager a significant amount of money that this
remains true for, say, five years.

Matt

On 01/07/16 19:02, Gavin Andresen via bitcoin-dev wrote:
> I'm hoisting this from some private feedback I sent on the segregated
> witness BIP:
>
> I said:
>
> "I'd also use RIPEMD160(SHA256()) as the hash function and save the 12
> bytes-- a successful preimage attack against that ain't gonna happen
> before we're all dead. I'm probably being dense, but I just don't see
> how a collision attack is relevant here."
>
> Pieter responded:
>
> "The problem case is where someone in a contract setup shows you a
> script, which you accept as being a payment to yourself. An attacker
> could use a collision attack to construct scripts with identical hashes,
> only one of which does have the property you want, and steal coins.
>
> So you really want collision security, and I don't think 80 bits is
> something we should encourage for that. Normal pubkey hashes don't have
> that problem, as they can't be constructed to pay to you."
>
> ... but I'm unconvinced:
>
> "But it is trivial for contract wallets to protect against collision
> attacks-- if you give me a script that is "gavin_pubkey CHECKSIG
> arbitrary_data OP_DROP" with "I promise I'm not trying to rip you off,
> just ignore that arbitrary data" a wallet can just refuse. Even more
> likely, a contract wallet won't even recognize that as a pay-to-gavin
> transaction.
>
> I suppose it could be looking for some form of "gavin_pubkey
> somebody_else_pubkey CHECKMULTISIG ... with the attacker using
> somebody_else_pubkey to force the collision, but, again, trivial
> contract protocol tweaks ("send along a proof you have the private key
> corresponding to the public key" or "everybody pre-commits pubkeys
> they'll use at protocol start") would protect against that.
>
> Adding an extra 12 bytes to every segwit to prevent an attack that takes
> 2^80 computation and 2^80 storage, is unlikely to be a problem in
> practice, and is trivial to protect against is the wrong tradeoff to make."
>
> 20 bytes instead of 32 bytes is a savings of almost 40%, which is
> significant.
>
> The general question I'd like to raise on this list is:
>
> Should we be worried, today, about collision attacks against RIPEMD160
> (our 160-bit hash)?
>
> Mounting a successful brute-force collision attack would require at
> least O(2^80) CPU, which is kinda-sorta feasible (Pieter pointed out
> that Bitcoin POW has computed more SHA256 hashes than that). But it also
> requires O(2^80) storage, which is utterly infeasible (there is
> something on the order of 2^35 bytes of storage in the entire world).
> Even assuming doubling every single year (faster than Moore's Law),
> we're four decades away from an attacker with THE ENTIRE WORLD's storage
> capacity being able to mount a collision attack.
>
>
> References:
>
> https://en.wikipedia.org/wiki/Collision_attack
>
> https://vsatglobalseriesblog.wordpress.com/2013/06/21/in-2013-the-amount-of-data-generated-worldwide-will-reach-four-zettabytes/
>
>
> --
> --
> Gavin Andresen
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2015-12-16 📝 Original message:A ...

2023-06-07T19:46:30+02:00

In reply to nevent1q…qveg
_________________________

📅 Original date posted:2015-12-16
📝 Original message:A large part of your argument is that SW will take longer to deploy than a hard fork, but I completely disagree. Though I do not agree with some people claiming we can deploy SW significantly faster than a hard fork, once the code is ready (probably a six month affair) we can get it deployed very quickly. It's true the ecosystem may take some time to upgrade, but I see that as a feature, not a bug - we can build up some fee pressure with an immediate release valve available for people to use if they want to pay fewer fees.

On the other hand, a hard fork, while simpler for the ecosystem to upgrade to, is a 1-2 year affair (after the code is shipped, so at least 1.5-2.5 from today if we all put off heads down and work). One thing that has concerned me greatly through this whole debate is how quickly people seem to think we can roll out a hard fork. Go look at the distribution of node versions on the network today and work backwards to get nearly every node upgraded... Even with a year between fork-version-release and fork-activation, we'd still kill a bunch of nodes and instead of reducing their security model, lead them to be outright robbed.

On December 16, 2015 12:38:30 PM PST, Jeff Garzik via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>1. Summary
>
>Segregated Witness (SegWitness, SW) is being presented in the context
>of
>Scaling Bitcoin. It has useful attributes, notably addressing a major
>malleability vector, but is not a short term scaling solution.
>
>
>2. Definitions
>
>Import Fee Event, ECE, TFM, FFM from previous email.
>
>Older clients - Any software not upgraded to SW
>
>Newer clients - Upgraded, SW aware software
>
>
>Block size - refers to the core block economic resource limited by
>MAX_BLOCK_SIZE. Witness data (or extension block data) is excluded.
>Requires a hard fork to change.
>
>Core block - Current bitcoin block, with upper bound MAX_BLOCK_SIZE.
>Not
>changed by SW.
>
>
>Extended transaction - Newer, upgraded version of transaction data
>format.
>
>Extended block - Newer, upgraded version of block data format.
>
>
>EBS - Extended block size. Block size seen by newer clients.
>
>
>3. Context of analysis
>
>One proposal presents SW *in lieu of* a hard fork block size increase.
>This email focuses directly on that.
>
>Useful features outside block size context, such as anti-malleability
>or
>fraud proof features, are not covered in depth.
>
>
>4.1. Observations on data structure formats and views
>
>SW creates two *views* of each transaction and block. SW has blocks
>and
>extended blocks. Similarly, there exists transactions and extended
>transactions.
>
>This view is rendered to clients depending on compatibility level.
>Newer
>clients see extended blocks and extended transactions. Older clients
>see
>blocks (limit 1M), and do not see extended blocks. Older clients see
>upgraded transactions as unsigned, anyone-can-pay transactions.
>
>Each extended transaction exists in two states, one unsigned and one
>signed, each of which passes validation as a valid bitcoin transaction.
>
>
>4.2. Observations on behavior of older transaction creation
>
>Transactions created by older clients will not use the extended
>transaction
>format. All data is stored the standard 1M block as today.
>
>
>4.3. Observations on new block economic model
>
>SW complicates block economics by creating two separate, supply limited
>resources.
>
>The core block economic resource is heavily contended. Older clients
>use
>core blocks exclusively. Newer clients use core blocks more
>conservatively, storing as much data as possible in extended blocks.
>
>The extended block economic resource is less heavily contended, though
>that
>of course grows over time as clients upgrade.
>
>Because core blocks are more heavily contended, it is presumed that
>older
>clients will pay a higher fee than newer clients (subject to elasticity
>etc.).
>
>
>5.1. Problem: Pace of roll-out will be slow - Whole Ecosystem must be
>considered.
>
>The current apparent proposal is to roll out Segregated Witness as a
>soft
>fork, and keep block size at 1M.
>
>The roll-out pace cannot simply be judged by soft fork speed - which is
>months at best. Analysis must the layers above: Updating bitcoin-core
>(JS) and bitcoinj (Java), and then the timelines to roll out those
>updates
>to apps, and then the timeline to update those apps to create extended
>transactions.
>
>Overall, wallet software and programmer libraries must be upgraded to
>make
>use of this new format, adding many more months (12+ in some stacks) to
>the
>roll out timeline. In the meantime, clients continue to contend
>entirely
>for core block space.
>
>
>5.2. Problem: Hard fork to bigger block size Just Works(tm) with
>most
>software, unlike SW.
>
>A simple hard fork such as BIP 102 is automatically compatible with the
>vast range of today's ecosystem software.
>
>SW requires merchants to upgrade almost immediately, requires wallet
>and
>other peripheral software upgrades to make use of. Other updates are
>opt-in and occur more slowly. BIP 70 processors need some updates.
>
>The number of LOC that must change for BIP 102 is very small, and the
>problem domain well known, versus SW.
>
>
>5.3. Problem: Due to pace, Fee Event not forestalled.
>
>Even presuming SW is merged into Bitcoin Core tomorrow, this does not
>address the risk of a Fee Event and associated Economic Change in the
>coming months.
>
>
>5.4. Problem: More complex economic policy, new game theory, new
>bidding
>structure risks.
>
>Splitting blocks into two pieces, each with separate and distinct
>behaviors
>and resource values, creates *two fee markets.*
>
>Having two pricing strata within each block has certainly feasible -
>that
>is the current mining policy of (1) fee/KB followed by (2)
>priority/age.
>
>Valuable or not - e.g. incentivizing older clients to upgrade - the
>fact
>remains that SW creates a more-complex bidding structure by creating a
>second economic resource.
>
>*This is clearly a change to a new economic policy* with standard risks
>associated with that. Will that induce an Economic Change Event (see
>def
>last email)? *Unlikely*, due to slow rollout pace.
>
>
>5.5. Problem: Current SW mining algorithm needs improvement
>
>Current SW block template maker does a reasonable job, but makes some
>naive
>assumptions about the fee market across an entire extended block. This
>is
>a mismatch with the economic reality (just described).
>
>5.6. Problem: New, under-analyzed attack surfaces
>
>Less significant and fundamental but still worth noting.
>
>This is not a fundamental SW problem, but simply standard complexity
>risk
>factors: splitting the signatures away from transactions, and creating
>a
>new apparently-unsigned version of the transaction opens the
>possibility of
>some network attacks which cause some clients to degrade down from
>extended
>block to core block mode temporarily.
>
>There is a chance of a failure mode that fools older clients into
>thinking
>fraudulent data is valid (judgement: unlikely vis hashpower but not
>impossible)
>
>6. Conclusions and recommendations
>
>It seems unlikely that SW provides scaling in the short term, and SW
>introduces new economics complexities.
>
>A "short term bump" hard fork block size increase addresses economic
>and
>ecosystem risks that SW does not.
>
>Bump + SW should proceed in parallel, independent tracks, as orthogonal
>issues.
>
>
>7. Appendix - Other SW comments
>
>Hard forks provide much stronger validation, and ensure the network
>operates at a fully trustless level.
>
>SW hard fork is preferred, versus soft fork. Soft forking SW places a
>huge
>amount of trust on miners to validate transaction signatures, versus
>the
>rest of the network, as the network slowly upgrades to newer clients.
>
>An SW hard fork could also add several zero-filled placeholders in a
>merkle
>tree for future use.
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev at lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20151216/4c7fc0c2/attachment.html>;

📅 Original date posted:2015-12-04 📝 Original message:On ...

2023-06-07T19:45:25+02:00

In reply to nevent1q…pvr0
_________________________

📅 Original date posted:2015-12-04
📝 Original message:On December 3, 2015 7:02:20 AM GMT+08:00, Peter Tschipper <peter.tschipper at gmail.com> wrote:
>On 02/12/2015 2:23 PM, Matt Corallo wrote:
>> My issue is more that its additional complexity and attack surface,
>> and for a very minor gain
>What is a minor gain? 15 to 27% compression sounds good to me and the
>larger the data the better the compression. And although there is a
>decent peformance gain in proportion to the % of compression, the
>original motivation of the BIP was to reduce bandwidth for users in
>regions where they are subject to caps.

Ok. It wasn't clear to me that you weren't also claiming at latency reduction as a result. In any case, the point I was making is that the p2p protocol isn't for every use-case. Indeed, I agree (as noted previously) that we should support people who have very restrictive data usage limits, but I don't think we need to do this in the p2p protocol. Considering we're in desperate need of more ways to sync, supporting syncing over slow and/or very restrictive connections is something maybe better addressed by a sync-over-http-via-cdn protocol than the p2p protocol.

>> which should disappear with further optimization elsewhere
>Why would the benefit of compressing data disappear with further
>optimizations elsewhere, I'm not following you?. The compression of
>data mainly has benefit in the sending of packets over the network. I
>would think the performance gain would be cumulative. Why would this
>go
>away by optimizing elsewhere?

My point is that, with limited further optimization, and especially after the first hundred thousand blocks, block download should nearly never be the thing limiting IBD speed.

>> and less that we absolutely shouldn't add compression because we're
>> definitely gonna have issues.
>It's not that difficult to add compression. Even if there was an
>issue,
>the compression feature can be completely turned off.

No matter how easily you can implement something, complexity always has cost. This is especially true in complicated, incredibly security critical applications exposed to the internet.

>>
>> On 12/02/15 20:16, Peter Tschipper via bitcoin-dev wrote:
>>> Building a compressor from scratch may yeild some better compression
>>> ratios, or not, but having trust and faith in whether it will stand
>up
>>> against attack vectors another matter. LZO has been around for 20
>years
>>> with very few problems and no current issues. Maybe something
>better
>>> can be built, but when and how much testing will need to be done
>before
>>> it can be trusted? Right now there is something that provides a
>benefit
>>> and in the future if something better is found it's not that
>difficult
>>> to add it. We could easily support multiple compression libraries.
>>>
>>>
>>> On 02/12/2015 10:57 AM, Emin Gün Sirer wrote:
>>>> Thanks Peter for the careful, quantitative work.
>>>>
>>>> I want to bring one additional issue to everyone's consideration,
>>>> related to the choice of the Lempel-Ziv family of compressors.
>>>>
>>>> While I'm not familiar with every single compression engine tested,
>>>> the Lempel-Ziv family of compressors are generally based on
>>>> "compression tables." Essentially, they assign a short unique
>number
>>>> to every new subsequence they encounter, and when they re-encounter
>a
>>>> sequence like "ab" in "abcdfdcdabcdfabcdf" they replace it with
>that
>>>> short integer (say, in this case, 9-bit constant 256). So this
>example
>>>> sequence may turn into "abcdfd<258 for cd><256 for ab><258 for
>>>> cd>f<261 for abc><259 for df>" which is slightly shorter than the
>>>> original (I'm doing this off the top of my head so the counts may
>be
>>>> off, but it's meant to be illustrative). Note that the sequence
>"abc"
>>>> got added into the table only after it was encountered twice in the
>>>> input.
>>>>
>>>> This is nice and generic and works well for English text where
>certain
>>>> letter sequences (e.g. "it" "th" "the" "this" "are" "there" etc)
>are
>>>> repeated often, but it is nowhere as compact as it could possibly
>be
>>>> for mostly binary data -- there are opportunities for much better
>>>> compression, made possible by the structured reuse of certain byte
>>>> sequences in the Bitcoin wire protocol.
>>>>
>>>> On a Bitcoin wire connection, we might see several related
>>>> transactions reorganizing cash in a set of addresses, and
>therefore,
>>>> several reuses of a 20-byte address. Or we might see a 200-byte
>>>> transaction get transmitted, followed by the same transaction,
>>>> repeated in a block. Ideally, we'd learn the sequence that may be
>>>> repeated later on, all at once (e.g. a Bitcoin address or a
>>>> transaction), and replace it with a short number, referring back to
>>>> the long sequence. In the example above, if we knew that "abcdf"
>was a
>>>> UNIT that would likely be repeated, we would put it into the
>>>> compression table as a whole, instead of relying on repetition to
>get
>>>> it into the table one extra byte at a time. That may let us
>compress
>>>> the original sequence down to "abcdfd<257 for cd><256 for
>abcdf><256
>>>> for abcdf>" from the get go.
>>>>
>>>> Yet the LZ variants I know of will need to see a 200-byte sequence
>>>> repeated **199 times** in order to develop a single, reusable,
>>>> 200-byte long subsequence in the compression table.
>>>>
>>>> So, a Bitcoin-specific compressor can perhaps do significantly
>better,
>>>> but is it a good idea? Let's argue both sides.
>>>>
>>>> Cons:
>>>>
>>>> On the one hand, Bitcoin-specific compressors will be closely tied
>to
>>>> the contents of messages, which might make it difficult to change
>the
>>>> wire format later on -- changes to the wire format may need
>>>> corresponding changes to the compressor. If the compressor cannot
>be
>>>> implemented cleanly, then the protocol-agnostic, off-the-shelf
>>>> compressors have a maintainability edge, which comes at the expense
>of
>>>> the compression ratio.
>>>>
>>>> Another argument is that compression algorithms of any kind should
>be
>>>> tested thoroughly before inclusion, and brand new code may lack the
>>>> maturity required. While this argument has some merit, all outputs
>are
>>>> verified separately later on during processing, so
>>>> compression/decompression errors can potentially be detected. If
>the
>>>> compressor/decompressor can be structured in a way that isolates
>>>> bitcoind from failure (e.g. as a separate process for starters),
>this
>>>> concern can be remedied.
>>>>
>>>> Pros:
>>>>
>>>> The nature of LZ compressors leads me to believe that much higher
>>>> compression ratios are possible by building a custom, Bitcoin-aware
>>>> compressor. If I had to guess, I would venture that compression
>ratios
>>>> of 2X or more are possible in some cases. In some sense, the "O(1)
>>>> block propagation" idea that Gavin proposed a while ago can be seen
>as
>>>> extreme example of a Bitcoin-specific compressor, albeit one that
>>>> constrains the order of transactions in a block.
>>>>
>>>> Compression can buy us some additional throughput at zero cost,
>modulo
>>>> code complexity.
>>>> Given the amount of acrimonious debate over the block size we have
>all
>>>> had to endure, it seems
>>>> criminal to leave potentially free improvements on the table. Even
>if
>>>> the resulting code is
>>>> deemed too complex to include in the production client right now,
>it
>>>> would be good to understand
>>>> the potential for improvement.
>>>>
>>>> How to Do It
>>>>
>>>> If we want to compress Bitcoin, a programming challenge/contest
>would
>>>> be one of the best ways to find the best possible, Bitcoin-specific
>>>> compressor. This is the kind of self-contained exercise that bright
>>>> young hackers love to tackle. It'd bring in new programmers into
>the
>>>> ecosystem, and many of us would love to discover the limits of
>>>> compressibility for Bitcoin bits on a wire. And the results would
>be
>>>> interesting even if the final compression engine is not enabled by
>>>> default, or not even merged.
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev at lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>

📅 Original date posted:2015-12-02 📝 Original message:My ...

2023-06-07T19:45:24+02:00

In reply to nevent1q…rn6g
_________________________

📅 Original date posted:2015-12-02
📝 Original message:My issue is more that its additional complexity and attack surface, and
for a very minor gain which should disappear with further optimization
elsewhere and less that we absolutely shouldn't add compression because
we're definitely gonna have issues.

On 12/02/15 20:16, Peter Tschipper via bitcoin-dev wrote:
> Building a compressor from scratch may yeild some better compression
> ratios, or not, but having trust and faith in whether it will stand up
> against attack vectors another matter. LZO has been around for 20 years
> with very few problems and no current issues. Maybe something better
> can be built, but when and how much testing will need to be done before
> it can be trusted? Right now there is something that provides a benefit
> and in the future if something better is found it's not that difficult
> to add it. We could easily support multiple compression libraries.
>
>
> On 02/12/2015 10:57 AM, Emin Gün Sirer wrote:
>> Thanks Peter for the careful, quantitative work.
>>
>> I want to bring one additional issue to everyone's consideration,
>> related to the choice of the Lempel-Ziv family of compressors.
>>
>> While I'm not familiar with every single compression engine tested,
>> the Lempel-Ziv family of compressors are generally based on
>> "compression tables." Essentially, they assign a short unique number
>> to every new subsequence they encounter, and when they re-encounter a
>> sequence like "ab" in "abcdfdcdabcdfabcdf" they replace it with that
>> short integer (say, in this case, 9-bit constant 256). So this example
>> sequence may turn into "abcdfd<258 for cd><256 for ab><258 for
>> cd>f<261 for abc><259 for df>" which is slightly shorter than the
>> original (I'm doing this off the top of my head so the counts may be
>> off, but it's meant to be illustrative). Note that the sequence "abc"
>> got added into the table only after it was encountered twice in the
>> input.
>>
>> This is nice and generic and works well for English text where certain
>> letter sequences (e.g. "it" "th" "the" "this" "are" "there" etc) are
>> repeated often, but it is nowhere as compact as it could possibly be
>> for mostly binary data -- there are opportunities for much better
>> compression, made possible by the structured reuse of certain byte
>> sequences in the Bitcoin wire protocol.
>>
>> On a Bitcoin wire connection, we might see several related
>> transactions reorganizing cash in a set of addresses, and therefore,
>> several reuses of a 20-byte address. Or we might see a 200-byte
>> transaction get transmitted, followed by the same transaction,
>> repeated in a block. Ideally, we'd learn the sequence that may be
>> repeated later on, all at once (e.g. a Bitcoin address or a
>> transaction), and replace it with a short number, referring back to
>> the long sequence. In the example above, if we knew that "abcdf" was a
>> UNIT that would likely be repeated, we would put it into the
>> compression table as a whole, instead of relying on repetition to get
>> it into the table one extra byte at a time. That may let us compress
>> the original sequence down to "abcdfd<257 for cd><256 for abcdf><256
>> for abcdf>" from the get go.
>>
>> Yet the LZ variants I know of will need to see a 200-byte sequence
>> repeated **199 times** in order to develop a single, reusable,
>> 200-byte long subsequence in the compression table.
>>
>> So, a Bitcoin-specific compressor can perhaps do significantly better,
>> but is it a good idea? Let's argue both sides.
>>
>> Cons:
>>
>> On the one hand, Bitcoin-specific compressors will be closely tied to
>> the contents of messages, which might make it difficult to change the
>> wire format later on -- changes to the wire format may need
>> corresponding changes to the compressor. If the compressor cannot be
>> implemented cleanly, then the protocol-agnostic, off-the-shelf
>> compressors have a maintainability edge, which comes at the expense of
>> the compression ratio.
>>
>> Another argument is that compression algorithms of any kind should be
>> tested thoroughly before inclusion, and brand new code may lack the
>> maturity required. While this argument has some merit, all outputs are
>> verified separately later on during processing, so
>> compression/decompression errors can potentially be detected. If the
>> compressor/decompressor can be structured in a way that isolates
>> bitcoind from failure (e.g. as a separate process for starters), this
>> concern can be remedied.
>>
>> Pros:
>>
>> The nature of LZ compressors leads me to believe that much higher
>> compression ratios are possible by building a custom, Bitcoin-aware
>> compressor. If I had to guess, I would venture that compression ratios
>> of 2X or more are possible in some cases. In some sense, the "O(1)
>> block propagation" idea that Gavin proposed a while ago can be seen as
>> extreme example of a Bitcoin-specific compressor, albeit one that
>> constrains the order of transactions in a block.
>>
>> Compression can buy us some additional throughput at zero cost, modulo
>> code complexity.
>> Given the amount of acrimonious debate over the block size we have all
>> had to endure, it seems
>> criminal to leave potentially free improvements on the table. Even if
>> the resulting code is
>> deemed too complex to include in the production client right now, it
>> would be good to understand
>> the potential for improvement.
>>
>> How to Do It
>>
>> If we want to compress Bitcoin, a programming challenge/contest would
>> be one of the best ways to find the best possible, Bitcoin-specific
>> compressor. This is the kind of self-contained exercise that bright
>> young hackers love to tackle. It'd bring in new programmers into the
>> ecosystem, and many of us would love to discover the limits of
>> compressibility for Bitcoin bits on a wire. And the results would be
>> interesting even if the final compression engine is not enabled by
>> default, or not even merged.
>>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2015-12-01 📝 Original ...

2023-06-07T19:45:22+02:00

In reply to nevent1q…lgm0
_________________________

📅 Original date posted:2015-12-01
📝 Original message:I'm really not a fan of this at all. To start with, adding a compression library that is directly accessible to the network on financial software is a really, really scary idea. If there were a massive improvement, I'd find it acceptable, but the improvement you've shown really isn't all that much. The numbers you recently posted show it improving the very beginning of IBD somewhat over high-latency connections, but if we're throughput-limited after the very beginning of IBD, we should fix that, not compress the blocks. Additionally, I'd be very surprised if this had any significant effect on the speed at which new blocks traverse the network (do you have any simulations or other thoughts on this?).

All that said, I'd love a proposal that allows clients to download compressed blocks via an external daemon, especially during IBD. This could help people with very restrictive data caps do IBD instead of being pushed to revert to SPV. Additionally, I think we need more chain sync protocols so that the current P2P protocol isn't consensus-critical anymore.

On November 30, 2015 4:12:24 PM MST, Peter Tschipper via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>@gmaxwell Bip Editor, and the Bitcoin Dev Community,
>
>After several weeks of experimenting and testing with various
>compression libraries I think there is enough evidence to show that
>compressing blocks and transactions is not only beneficial in reducing
>network bandwidth but is also provides a small performance boost when
>there is latency on the network.
>
>The following is a BIP Draft document for your review.
>(The alignment of the columns in the tables doesn't come out looking
>right in this email but if you cut and paste into a text document they
>are just fine)
>
>
><pre>
> BIP: ?
> Title: Datastream compression of Blocks and Tx's
> Author: Peter Tschipper <peter.tschipper at gmail.com>
> Status: Draft
> Type: Standards Track
> Created: 2015-11-30
></pre>
>
>==Abstract==
>
>To compress blocks and transactions, and to concatenate them together
>when possible, before sending.
>
>==Motivation==
>
>Bandwidth is an issue for users that run nodes in regions where
>bandwidth is expensive and subject to caps, in addition network latency
>in some regions can also be quite high. By compressing data we can
>reduce daily bandwidth used in a significant way while at the same time
>speed up the transmission of data throughout the network. This should
>encourage users to keep their nodes running longer and allow for more
>peer connections with less need for bandwidth throttling and in
>addition, may also encourage users in areas of marginal internet
>connectivity to run nodes where in the past they would not have been
>able to.
>
>==Specification==
>
>Advertise compression using a service bit. Both peers must have
>compression turned on in order for data to be compressed, sent, and
>decompressed.
>
>Blocks will be sent compressed.
>
>Transactions will be sent compressed with the exception of those less
>than 500 bytes.
>
>Blocks will be concatenated when possible.
>
>Transactions will be concatenated when possible or when a
>MSG_FILTERED_BLOCK is requested.
>
>Compression levels to be specified in "bitcoin.conf".
>
>Compression and decompression can be completely turned off.
>
>Although unlikely, if compression should fail then data will be sent
>uncompressed.
>
>The code for compressing and decompressing will be located in class
>CDataStream.
>
>Compression library LZO1x will be used.
>
>==Rationale==
>
>By using a service bit, compression and decompression can be turned
>on/off completely at both ends with a simple configuration setting. It
>is important to be able to easily turn off compression/decompression as
>a fall back mechanism. Using a service bit also makes the code fully
>compatible with any node that does not currently support compression. A
>node that do not present the correct service bit will simply receive
>data in standard uncompressed format.
>
>All blocks will be compressed. Even small blocks have been found to
>benefit from compression.
>
>Multiple block requests that are in queue will be concatenated together
>when possible to increase compressibility of smaller blocks.
>Concatenation will happen only if there are multiple block requests
>from
>the same remote peer. For example, if peer1 is requesting two blocks
>and they are both in queue then those two blocks will be concatenated.
>However, if peer1 is requesting 1 block and peer2 also one block, and
>they are both in queue, then each peer is sent only its block and no
>concatenation will occur. Up to 16 blocks (the max blocks in flight)
>can
>be concatenated but not exceeding the MAX_PROTOCOL_MESSAGE_LENGTH.
>Concatenated blocks compress better and further reduce bandwidth.
>
>Transactions below 500 bytes do not compress well and will be sent
>uncompressed unless they can be concatenated (see Table 3).
>
>Multiple transaction requests that are in queue will be concatenated
>when possible. This further reduces bandwidth needs and speeds the
>transfer of large requests for many transactions, such as with
>MSG_FILTERED_BLOCK requests, or when the system gets busy and is
>flooded
>with transactions. Concatenation happens in the same way as for
>blocks,
>described above.
>
>By allowing for differing compression levels which can be specified in
>the bitcoin.conf file, a node operator can tailor their compression to
>a
>level suitable for their system.
>
>Although unlikely, if compression fails for any reason then blocks and
>transactions will be sent uncompressed. Therefore, even with
>compression turned on, a node will be able to handle both compressed
>and
>uncompressed data from another peer.
>
>By Abstracting the compression/decompression code into class
>"CDataStream", compression can be easily applied to any datastream.
>
>The compression library LZO1x-1 does not compress to the extent that
>Zlib does but it is clearly the better performer (particularly as file
>sizes get larger), while at the same time providing very good
>compression (see Tables 1 and 2). Furthermore, LZO1x-999 can provide
>and almost Zlib like compression for those who wish to have more
>compression, although at a cost.
>
>==Test Results==
>
>With the LZO library, current test results show up to a 20% compression
>using LZO1x-1 and up to 27% when using LZO1x-999. In addition there is
>a marked performance improvement when there is latency on the network.
>From the test results, with a latency of 60ms there is an almost 30%
>improvement in performance when comparing LZO1x-1 compressed blocks
>with
>uncompressed blocks (see Table 5).
>
>The following table shows the percentage that blocks were compressed,
>using two different Zlib and LZO1x compression level settings.
>
>TABLE 1:
>range = data size range
>range Zlib-1 Zlib-6 LZO1x-1 LZO1x-999
>----------- ------ ------ ------- --------
>0-250 12.44 12.86 10.79 14.34
>250-500 19.33 12.97 10.34 11.11
>600-700 16.72 n/a 12.91 17.25
>700-800 6.37 7.65 4.83 8.07
>900-1KB 6.54 6.95 5.64 7.9
>1KB-10KB 25.08 25.65 21.21 22.65
>10KB-100KB 19.77 21.57 4.37 19.02
>100KB-200KB 21.49 23.56 15.37 21.55
>200KB-300KB 23.66 24.18 16.91 22.76
>300KB-400KB 23.4 23.7 16.5 21.38
>400KB-500KB 24.6 24.85 17.56 22.43
>500KB-600KB 25.51 26.55 18.51 23.4
>600KB-700KB 27.25 28.41 19.91 25.46
>700KB-800KB 27.58 29.18 20.26 27.17
>800KB-900KB 27 29.11 20 27.4
>900KB-1MB 28.19 29.38 21.15 26.43
>1MB -2MB 27.41 29.46 21.33 27.73
>
>The following table shows the time in seconds that a block of data
>takes
>to compress using different compression levels. One can clearly see
>that LZO1x-1 is the fastest and is not as affected when data sizes get
>larger.
>
>TABLE 2:
>range = data size range
>range Zlib-1 Zlib-6 LZO1x-1 LZO1x-999
>----------- ------ ------ ------- ---------
>0-250 0.001 0 0 0
>250-500 0 0 0 0.001
>500-1KB 0 0 0 0.001
>1KB-10KB 0.001 0.001 0 0.002
>10KB-100KB 0.004 0.006 0.001 0.017
>100KB-200KB 0.012 0.017 0.002 0.054
>200KB-300KB 0.018 0.024 0.003 0.087
>300KB-400KB 0.022 0.03 0.003 0.121
>400KB-500KB 0.027 0.037 0.004 0.151
>500KB-600KB 0.031 0.044 0.004 0.184
>600KB-700KB 0.035 0.051 0.006 0.211
>700KB-800KB 0.039 0.057 0.006 0.243
>800KB-900KB 0.045 0.064 0.006 0.27
>900KB-1MB 0.049 0.072 0.006 0.307
>
>TABLE 3:
>Compression of Transactions (without concatenation)
>range = block size range
>ubytes = average size of uncompressed transactions
>cbytes = average size of compressed transactions
>cmp% = the percentage amount that the transaction was compressed
>datapoints = number of datapoints taken
>
>range ubytes cbytes cmp% datapoints
>---------- ------ ------ ------ ----------
>0-250 220 227 -3.16 23780
>250-500 356 354 0.68 20882
>500-600 534 505 5.29 2772
>600-700 653 608 6.95 1853
>700-800 757 649 14.22 578
>800-900 822 758 7.77 661
>900-1KB 954 862 9.69 906
>1KB-10KB 2698 2222 17.64 3370
>10KB-100KB 15463 12092 21.80 15429
>
>The above table shows that transactions don't compress well below 500
>bytes but do very well beyond 1KB where there are a great deal of those
>large spam type transactions. However, most transactions happen to be
>in the < 500 byte range. So the next step was to appy concatenation
>for
>those smaller transactions. Doing that yielded some very good
>compression results. Some examples as follows:
>
>The best one that was seen was when 175 transactions were concatenated
>before being compressed. That yielded a 20% compression ratio, but
>that
>doesn't take into account the savings from the unneeded 174 message
>headers (24 bytes each) as well as 174 TCP ACKs of 52 bytes each which
>yields and additional 76*174 = 13224 byte savings, making for an
>overall
>bandwidth savings of 32%:
>
> 2015-11-18 01:09:09.002061 compressed data from 79890 to 67426
>txcount:175
>
>However, that was an extreme example. Most transaction aggregates were
>in the 2 to 10 transaction range. Such as the following:
>
>2015-11-17 21:08:28.469313 compressed data from 3199 to 2876 txcount:10
>
>But even here the savings of 10% was far better than the "nothing" we
>would get without concatenation, but add to that the 76 byte * 9
>transaction savings and we have a total 20% savings in bandwidth for
>transactions that otherwise would not be compressible. Therefore the
>concatenation of small transactions can also save bandwidth and speed
>up
>the transmission of those transactions through the network while
>keeping
>network and message queue chatter to a minimum.
>
>==Choice of Compression library==
>
>LZO was chosen over Zlib. LZO is the fastest most scalable option when
>used at the lowest compression setting which will be a performance
>boost
>for users that prefer performance over bandwidth savings. And at the
>higher end, LZO provides good compression (although at a higher cost)
>which approaches that of Zlib.
>
>Other compression libraries investigated were Snappy, LZOf, fastZlib
>and
>LZ4 however none of these were found to be suitable, either because
>they
>were not portable, lacked the flexibility to set compression levels or
>did not provide a useful compression ratio.
>
>The following two tables show results in seconds for syncing the first
>200,000 blocks. Tests were run on a high-speed wireless LAN with very
>little latency, and also run with a 60ms latency which was induced with
>"Netbalancer".
>
>TABLE 4:
>Results shown in seconds on highspeed wireless LAN (no induced latency)
>Num blks sync'd Uncmp Zlib-1 Zlib-6 LZO1x-1 LZO1x-999
>--------------- ----- ------ ------ ------- ---------
>10000 255 232 233 231 257
>20000 464 414 420 407 453
>30000 677 594 611 585 650
>40000 887 787 795 760 849
>50000 1099 961 977 933 1048
>60000 1310 1145 1167 1110 1259
>70000 1512 1330 1362 1291 1470
>80000 1714 1519 1552 1469 1679
>90000 1917 1707 1747 1650 1882
>100000 2122 1905 1950 1843 2111
>110000 2333 2107 2151 2038 2329
>120000 2560 2333 2376 2256 2580
>130000 2835 2656 2679 2558 2921
>140000 3274 3259 3161 3051 3466
>150000 3662 3793 3547 3440 3919
>160000 4040 4172 3937 3767 4416
>170000 4425 4625 4379 4215 4958
>180000 4860 5149 4895 4781 5560
>190000 5855 6160 5898 5805 6557
>200000 7004 7234 7051 6983 7770
>
>TABLE 5:
>Results shown in seconds with 60ms of induced latency
>Num blks sync'd Uncmp Zlib-1 Zlib-6 LZO1x-1 LZO1x-999
>--------------- ----- ------ ------ ------- ---------
>10000 219 299 296 294 291
>20000 432 568 565 558 548
>30000 652 835 836 819 811
>40000 866 1106 1107 1081 1071
>50000 1082 1372 1381 1341 1333
>60000 1309 1644 1654 1605 1600
>70000 1535 1917 1936 1873 1875
>80000 1762 2191 2210 2141 2141
>90000 1992 2463 2486 2411 2411
>100000 2257 2748 2780 2694 2697
>110000 2627 3034 3076 2970 2983
>120000 3226 3416 3397 3266 3302
>130000 4010 3983 3773 3625 3703
>140000 4914 4503 4292 4127 4287
>150000 5806 4928 4719 4529 4821
>160000 6674 5249 5164 4840 5314
>170000 7563 5603 5669 5289 6002
>180000 8477 6054 6268 5858 6638
>190000 9843 7085 7278 6868 7679
>200000 11338 8215 8433 8044 8795
>
>==Backward compatibility==
>
>Being unable to present the correct service bit, older clients will
>continue to receive standard uncompressed data and will be fully
>compatible with this change.
>
>==Fallback==
>
>It is important to be able to entirely and easily turn off compression
>and decompression as a fall back mechanism. This can be done with a
>simple bitcoin.conf setting of "compressionlevel=0". Only one of the
>two
>connected peers need to set compressionlevel=0 in order to turn off
>compression and decompression completely.
>
>==Deployment==
>
>This enhancement does not require a hard or soft fork.
>
>==Service Bit==
>
>During the testing of this implementation, service bit 28 was used,
>however this enhancement will require a permanently assigned service
>bit.
>
>==Implementation==
>
>This implementation depends on the LZO compression library: lzo-2.09
>
> https://github.com/ptschip/bitcoin/tree/compress
>
>==Copyright==
>
>This document is placed in the public domain.
>
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev at lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

📅 Original date posted:2015-10-28 📝 Original message:Oops, ...

2023-06-07T19:43:28+02:00

In reply to nevent1q…4n7q
_________________________

📅 Original date posted:2015-10-28
📝 Original message:Oops, just realized I never responded to this...

On 10/15/15 15:09, Ittay wrote:
> Thanks, Matt. Response inline.
>
> On Wed, Oct 14, 2015 at 2:57 PM, Matt Corallo <lf-lists at mattcorallo.com
> <mailto:lf-lists at mattcorallo.com>> wrote:
>
> That conversation missed a second issue. Namely that there is no way
> to punish people if there is a double spend in a micro block that
> happens in key block which reorg'd away the first transaction. eg
> one miner mines a transaction in a micro block, another miner
> (either by not having seen the first yet, or being malicious -
> potentially the same miner) mines a key block which reorgs away the
> first micro block and then, in their first micro block, mines a
> double spend. This can happen at any time, so you end up having to
> fall back to regular full blocks for confirmation times :(.
>
>
> If NG is to be used efficiently, microblocks are going to be very
> frequent, and so such forks should occur at almost every key-block
> publication. Short reorgs as you described are the norm. A user should
> wait before accepting a transaction to make sure there was no key-block
> she missed. The wait time is chosen according to the network propagation
> delay (+as much slack as the user feels necessary). This is similar to
> the situation in Bitcoin when you receive a block. To be confident that
> you have one confirmation you should wait for the propagation time of
> the network to make sure there is no branch you missed.

I think you're overstating how short the wait times can be. They need to
be much longer than the network propagation delay.

> As for the malicious case: the attacker has to win the key-block, have
> the to-be-inverted transaction in the previous epoch, and withhold his
> key-block for a while. That being said, indeed our fraud proof scheme
> doesn't catch such an event, as it is indistinguishable from benign
> behavior.

The attacker does not need to withold their keyblock at all. All the
attacker does is, for every transaction they ever send, after it is
included in a microblock, set their hashpower to start mining a keyblock
immediately prior to this microblock. When they find a keyblock, they
immediately announce it and start creating microblocks, the first of
which double-spends the previous transaction. If they dont win the key
block, oh well, their payment went through normally and they couldn't
double-spend.

In chatting with Glenn about this, we roughly agreed that the
confirmation time for microblocks possibly doesn't need to be a full
key-block, but it needs to be a reasonable period after which such an
attacker would lose more in fees than the value of their double-spend
(ie because the key-block afterwards gets 20% more in fees than the
key-block before hand). In any case, the game theory here starts to get
rather complicated and it doesn't make me want to suggest accepting
microblocks as confirmations is safe.

> Also, Greg Slepak brought up a good point on twitter at
> https://twitter.com/taoeffect/status/654358023138209792. Noting that
> this model means users could no longer pick transactions in a mining
> pool which was set up in such a way (it could be tweaked to do so
> with separate rewards and pubkeys, but now the user can commit fraud
> at a much lower cost - their own pool reward, not the block's total
> reward).
>
>
> Agreed x3: This is a good point, it is correct, and the tweak is dangerous.
> Do you perceive this as a significant practical issue?

It is not a practical issue today because no one does it, but it is a
massive issue in that the splitting of pool rewards and transaction
selection is one of the few easy wins we have left in the fight against
mining centralization. Mining centralization today is absolutely awful,
and closing off our only big win would be tragic.

> On October 14, 2015 11:28:51 AM PDT, Ittay via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
>
> On Wed, Oct 14, 2015 at 2:12 PM, Bryan Bishop <kanzure at gmail.com
> <mailto:kanzure at gmail.com>> wrote:
>
> On Wed, Oct 14, 2015 at 1:02 PM, Emin Gün Sirer
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
> > while the whitepaper has all the nitty gritty details:
> > http://arxiv.org/abs/1510.02037
>
> Taking reward compensation back by fraud proofs is not
> enough to fix
> the problems associated with double spending (such as,
> everyone has to
> wait for the "real" confirmations instead of the "possibly
> double-spend" confirmations). Some of this was discussed in
> -wizards
> recently:
> http://gnusha.org/bitcoin-wizards/2015-09-19.log
>
>
> Fraud proof removes all the attacker's revenue. It's like the
> attacker sacrifices an entire block for double spending in the
> current system. I think Luke-Jr got it right at that discussion.
>
> Best,
> Ittay
>
> ------------------------------------------------------------------------
>
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

📅 Original date posted:2015-10-14 📝 Original message:That ...

2023-06-07T19:43:28+02:00

In reply to nevent1q…3wgs
_________________________

📅 Original date posted:2015-10-14
📝 Original message:That conversation missed a second issue. Namely that there is no way to punish people if there is a double spend in a micro block that happens in key block which reorg'd away the first transaction. eg one miner mines a transaction in a micro block, another miner (either by not having seen the first yet, or being malicious - potentially the same miner) mines a key block which reorgs away the first micro block and then, in their first micro block, mines a double spend. This can happen at any time, so you end up having to fall back to regular full blocks for confirmation times :(.

Also, Greg Slepak brought up a good point on twitter at https://twitter.com/taoeffect/status/654358023138209792. Noting that this model means users could no longer pick transactions in a mining pool which was set up in such a way (it could be tweaked to do so with separate rewards and pubkeys, but now the user can commit fraud at a much lower cost - their own pool reward, not the block's total reward).

On October 14, 2015 11:28:51 AM PDT, Ittay via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>On Wed, Oct 14, 2015 at 2:12 PM, Bryan Bishop <kanzure at gmail.com>
>wrote:
>
>> On Wed, Oct 14, 2015 at 1:02 PM, Emin Gün Sirer
>> <bitcoin-dev at lists.linuxfoundation.org> wrote:
>> > while the whitepaper has all the nitty gritty details:
>> > http://arxiv.org/abs/1510.02037
>>
>> Taking reward compensation back by fraud proofs is not enough to fix
>> the problems associated with double spending (such as, everyone has
>to
>> wait for the "real" confirmations instead of the "possibly
>> double-spend" confirmations). Some of this was discussed in -wizards
>> recently:
>> http://gnusha.org/bitcoin-wizards/2015-09-19.log
>
>
>Fraud proof removes all the attacker's revenue. It's like the attacker
>sacrifices an entire block for double spending in the current system. I
>think Luke-Jr got it right at that discussion.
>
>Best,
>Ittay
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev at lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20151014/84d65583/attachment-0001.html>;

📅 Original date posted:2015-09-18 📝 Original message:I ...

2023-06-07T19:40:56+02:00

In reply to nevent1q…xnnc
_________________________

📅 Original date posted:2015-09-18
📝 Original message:I believe the discussion here is on improving initial-sync time by
simply skipping initial-sync and getting a committed-to utxo set. This
is obviously a new security model in between SPV and full-node (I would
call it SPV with future validation). Still, I'm not convinced it buys us
anything, we really should just tweak Bitcoin Core to do spv mode at
startup and validate backwards in the background. I think this would
alleviate most of the concerns raised, given the chain growth is not
entirely unreasonable going forward.

On 09/18/15 20:07, Alex Morcos via bitcoin-dev wrote:
> I guess I always assumed that UTXO set commitments were an alternative
> security model (between SPV and full-node), not that they would cause
> the existing security model to be deprecated.
>
>
> On Fri, Sep 18, 2015 at 3:43 PM, Patrick Strateman via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
> Full nodes using UTXO set commitments is a change to the bitcoin
> security model.
>
> Currently an attacker with >50% of the network hashrate can rewrite
> history.
>
> If full nodes rely on UTXO set commitments such an attacker could create
> an infinite number of bitcoins (as in many times more than the current
> 21 million bitcoin limit).
>
> Before we consider mechanisms for UTXO set commitments, we should
> seriously discuss whether the security model reduction is reasonable.
>
> On 09/18/2015 12:05 PM, Rune Kjær Svendsen via bitcoin-dev wrote:
> > Currently, when a new node wants to join the network, it needs to
> retrieve the entire blockchain history, starting from January 2009
> and up until now, in order to derive a UTXO set that it can verify
> new blocks/transactions against. With a blockchain size of 40GB and
> a UTXO size of around 1GB, the extra bandwidth required is
> significant, and will keep increasing indefinitely. If a newly mined
> block were to include the UTXO set hash of the chain up until the
> previous block — the hash of the UTXO set on top of which this block
> builds — then new nodes, who want to know whether a transaction is
> valid, would be able to acquire the UTXO set in a trustless manner,
> by only verifying proof-of-work headers, and knowing that a block
> with an invalid UTXO set hash would be rejected.
> >
> > I’m not talking about calculating a complicated tree structure
> from the UTXO set, which would put further burden on already
> burdened Bitcoin Core nodes. We simply include the hash of the
> current UTXO set in a newly created block, such that the
> transactions in the new block build *on top* of the UTXO set whose
> hash is specified. This actually alleviates Bitcoin Core nodes, as
> it will now become possible for nodes without the entire blockchain
> to answer SPV queries (by retrieving the UTXO set trustlessly and
> using this to answer queries). It also saves bandwidth for Bitcore
> Core nodes, who only need to send roughly 1GB of data, in order to
> synchronise a node, rather than 40GB+. I will continue to run a full
> Bitcoin Core node, saving the entire blockchain history, but it
> shouldn’t be a requirement to hold the entire transaction history in
> order to start verifying new transactions.
> >
> > As far as I can see, this also forces miners to actually maintain
> an UTXO set, rather than just build on top of the chain with the
> most proof-of-work. Producing a UTXO set and verifying a block
> against a chain is the same thing, so by including the hash of the
> UTXO set we force miners to verify the block that they want to build
> on top of.
> >
> > Am I missing something obvious, because as far as I can see, this
> solves the problem of quadratic time complexity for initial sync:
> http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s
> >
> > The only added step to verifying a block is to hash the UTXO set.
> So it does require additional computation, but most modern CPUs have
> a SHA256 throughput of around 500 MB/s, which means it takes only
> two seconds to hash the UTXO set. And this can be improved further
> (GPUs can do 2-3 GB/s). A small sacrifice for the added ease of
> initial syncing, in my opinion.
> >
> > /Rune
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2015-09-18 📝 Original message:Yes, ...

2023-06-07T19:40:53+02:00

In reply to nevent1q…0xqd
_________________________

📅 Original date posted:2015-09-18
📝 Original message:Yes, I'm aware, however they are closer to each other than UTC is to either :p.

On September 18, 2015 4:31:28 PM EDT, Gregory Maxwell <gmaxwell at gmail.com> wrote:
>On Fri, Sep 18, 2015 at 8:27 PM, Matt Corallo via bitcoin-dev
><bitcoin-dev at lists.linuxfoundation.org> wrote:
>> Google Calendar is localized, but has an option to change the
>timezone
>> of an event, it just doesnt have UTC in its options. So, yes, we
>should
>> use something that observes DST in roughly the same way as everyone
>else
>> - CEST/PDT/EST/etc.
>
>uh. There is fairly little global consistency in DST usage. Lots of
>places do dst on different dates.
>
>So if it's in some DST timezone it's likely to move twice each change
>for some subset of the people who do it.
>
>E.g. europe and US end DST one week apart.

📅 Original date posted:2015-09-18 📝 Original message:I ...

2023-06-07T19:40:52+02:00

In reply to nevent1q…trtt
_________________________

📅 Original date posted:2015-09-18
📝 Original message:I believe that is out of date. I see neither UTC nor GMT on the website nor on Android.

Matt

On September 18, 2015 4:30:23 PM EDT, Jeffrey Paul <jp at eeqj.com> wrote:
>
>> On 18 Sep 2015, at 22:27, Matt Corallo via bitcoin-dev
><bitcoin-dev at lists.linuxfoundation.org> wrote:
>>
>> Google Calendar is localized, but has an option to change the
>timezone
>> of an event, it just doesnt have UTC in its options. So, yes, we
>should
>> use something that observes DST in roughly the same way as everyone
>else
>> - CEST/PDT/EST/etc.
>
>According to this post, it is possible to configure Google Calendar to
>show an additional time zone to your localized one, and GMT/UTC is
>indeed in the list.
>
> https://productforums.google.com/forum/#!topic/calendar/y6uaiAdZrIg
>
>-jp
>
>--
>Jeffrey Paul EEQJ
>jp at eeqj.com https://eeqj.com
>+1-800-403-1126 (America) +1-312-361-0355 (Worldwide)
>5539 AD00 DE4C 42F3 AFE1 1575 0524 43F4 DF2A 55C2

📅 Original date posted:2015-09-18 📝 Original message:Google ...

2023-06-07T19:40:51+02:00

In reply to nevent1q…judh
_________________________

📅 Original date posted:2015-09-18
📝 Original message:Google Calendar is localized, but has an option to change the timezone
of an event, it just doesnt have UTC in its options. So, yes, we should
use something that observes DST in roughly the same way as everyone else
- CEST/PDT/EST/etc.

On 09/18/15 20:24, Btc Drak wrote:
> Google calendar is localised, so it doesn't matter. The problem with
> quoting UTC anyway it the meeting times are going to change for those
> that observe DST. It would be much better to quote an actual timezone of
> an actual area so it will remain constant, like 1700 CEST, or 0900AM PDT
> for example. Otherwise when the clocks change, what was a convenient
> meeting time will become inconvenient for some.
>
> On Fri, Sep 18, 2015 at 9:14 PM, Matt Corallo via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
> Generally in favor, but for practical purposes can we select a timezone
> that is available in Google Calendar? It appears it does not directly
> support UTC...
>
> On 09/18/15 01:07, Wladimir J. van der Laan via bitcoin-dev wrote:
> > Hello,
> >
> > At Monday's code sprint we had a good idea to schedule a regular
> developer meeting in #bitcoin-dev.
> >
> > Attendance is of course voluntary, but it may be good to have a
> time that many people are expected to be present and current issues
> can be discussed.
> >
> > Any preference for days/times?
> >
> > What about e.g. every week 15:00-16:00 UTC on Thursday?
> >
> > Wladimir
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

📅 Original date posted:2015-09-18 📝 Original ...

2023-06-07T19:40:50+02:00

In reply to nevent1q…5w9x
_________________________

📅 Original date posted:2015-09-18
📝 Original message:Generally in favor, but for practical purposes can we select a timezone
that is available in Google Calendar? It appears it does not directly
support UTC...

On 09/18/15 01:07, Wladimir J. van der Laan via bitcoin-dev wrote:
> Hello,
>
> At Monday's code sprint we had a good idea to schedule a regular developer meeting in #bitcoin-dev.
>
> Attendance is of course voluntary, but it may be good to have a time that many people are expected to be present and current issues can be discussed.
>
> Any preference for days/times?
>
> What about e.g. every week 15:00-16:00 UTC on Thursday?
>
> Wladimir
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2015-09-18 📝 Original message:I did ...

2023-06-07T19:40:28+02:00

In reply to nevent1q…h6a9
_________________________

📅 Original date posted:2015-09-18
📝 Original message:I did not intend to imply that there was agreement on a desire to
schedule a second hardfork. My wording may have been a bit too loose.
Instead, I believe there was much agreement that doing a short-term
hardfork now, with many agreeing that a second would hopefully be
entirely unnecessary/impossible, while others thought that a second
would be necessary and would have to happen. While this may set up a
similar controversy again in several years, I think everyone agreed that
we cannot predict the future and I, personally, think none of us should
be committing to a viewpoint for what should be done at that time.

Personally, I think it is also critical that there be no messaging that
people should rely on or assume there will be a future increase after a
short-term bump (which I also do not believe people should be relying on
now).

Matt

On 09/18/15 05:55, Mark Friedenbach wrote:
> Correction of a correction, in-line:
>
> On Wed, Sep 16, 2015 at 5:51 PM, Matt Corallo via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
> > - Many interested or at least willing to accept a "short term bump", a
> > hard fork to modify block size limit regime to be cost-based via
> > "net-utxo" rather than a simple static hard limit. 2-4-8 and 17%/year
> > were debated and seemed "in range" with what might work as a short term
> > bump - net after applying the new cost metric.
>
> I would be careful to point out that hard numbers were deliberately NOT
> discussed. Though some general things were thrown out, they were not
> extensively discussed nor agreed to. I personally think 2-4 is "in
> range", though 8 maybe not so much. Of course it depends on exactly how
> the non-blocksize limit accounting/adjusting is done.
>
> Still, the "greatest common denominator" agreement did not seem to be
> agreeing to an increase which continues over time, but which instead
> limits itself to a set, smooth increase for X time and then requires a
> second hardfork if there is agreement on a need for more blocksize at
> that point.
>
>
> Perhaps it is accurate to say that there wasn't consensus at all except
> that (1) we think we can work together on resolving this impasse (yay!),
> and (2) it is conceivable that changing from block size to some other
> metric might provide the basis for a compromise on near-term numbers.
>
> As an example, I do not think the net-UTXO metric provides any benefit
> with respect to scalability, and in some ways makes the situation worse
> (even though it helpfully solves an unrelated problem of spammy dust
> outputs). But there are other possible metrics and I maintain hope that
> data will show the benefit of another metric or other metrics combined
> with net-UTXO in a way that will allow us to reach consensus.
>
> As a further example, I also am quite concerned about 2-4-8MB with
> either block size or net-UTXO as the base metric. As you say, it depends
> on how the non-blocksize limit accounting/adjusting is done... But if a
> metric were chosen that addressed my concerns (worst case propagation
> and validation time), then I could be in favor of an initial bump that
> allowed a larger number of typical transactions in a block.
>
> But where I really need to disagree is on the requirement for a 2nd hard
> fork. I will go on record as being definitively against this. While
> being conservative with respect to exponentials, I would very much like
> to make sure that there is a long-term growth curve as part of any
> proposal. I am willing to accept a hard-fork if the adopted plan is too
> conservative, but I do not want to be kicking the can down the road to a
> scheduled 2nd hard fork that absolutely must occur. That, I feel, could
> be a more dangerous outcome than an exponential that outlasts
> conservative historical trends.
>
> I commend Jeff for writing a Chatham-rules summary of the outcome of
> some hallway conversations that occurred. On the whole I think his
> summary does represent the majority view of the opinions expressed by
> core developers at the workshop. I will caution though that on nearly
> every issue there were those expressed disagreement but did not fight
> the issue, and those who said nothing and left unpolled opinions.
> Nevertheless this summary is informative as it feeds forwards into the
> design of proposals that will be made prior to the Hong Kong workshop in
> December, in order that they have a higher likelihood of success.

📅 Original date posted:2015-09-16 📝 Original message:I only ...

2023-06-07T19:40:26+02:00

In reply to nevent1q…vsx4
_________________________

📅 Original date posted:2015-09-16
📝 Original message:I only have one "correction", included inline.

On 09/16/15 21:32, Jeff Garzik via bitcoin-dev wrote:
>
> During Scaling Bitcoin, Bitcoin Core committers and notable contributors
> got together and chatted about where a "greatest common denominator"
> type consensus might be. The following is a without-attribution
> (Chatham House) summary. This is my own personal summary of the chat;
> any errors are my own; this is _not_ a consensus statement or anything
> formal.
>
> - Background (pre-conference, was on public IRC): "net-utxo",
> calculating transaction size within block by applying a delta to
> transaction size based on the amount of data added, or removed, from the
> UTXO set. Fee is then evaluated after the delta is applied. This
> aligns user incentives with UTXO resource usage/cost. Original idea by
> gmaxwell (and others??).
>
> - Many interested or at least willing to accept a "short term bump", a
> hard fork to modify block size limit regime to be cost-based via
> "net-utxo" rather than a simple static hard limit. 2-4-8 and 17%/year
> were debated and seemed "in range" with what might work as a short term
> bump - net after applying the new cost metric.

I would be careful to point out that hard numbers were deliberately NOT
discussed. Though some general things were thrown out, they were not
extensively discussed nor agreed to. I personally think 2-4 is "in
range", though 8 maybe not so much. Of course it depends on exactly how
the non-blocksize limit accounting/adjusting is done.

Still, the "greatest common denominator" agreement did not seem to be
agreeing to an increase which continues over time, but which instead
limits itself to a set, smooth increase for X time and then requires a
second hardfork if there is agreement on a need for more blocksize at
that point.

> - Hard fork method: Leaning towards "if (timestamp > X)" flag day hard
> fork Y months in the future. Set high bit in version, resulting in a
> negative number, to more cleanly fork away. "miner advisement" -
> miners, as they've done recently, signal non-binding (Bitcoin Core does
> not examine the value) engineering readiness for a hard fork via
> coinbase moniker. Some fork cancellation method is useful, if
> unsuccessful after Z time elapses.
>
> - As discussed publicly elsewhere, other forks may be signaled via
> setting a bit in version, and then triggering a fork'ing change once a
> threshold is reached.
>
> Chat participants are invited to reply to this message with their own
> corrections and comments and summary in their view.
>
> For the wider community, take this as one of many "inputs" described at
> Scaling Bitcoin. Over the next few months developers and the community
> should evaluate everything discussed and work towards some concrete
> proposal(s) that are implemented, tested and simulated in December in
> Hong Kong.

📅 Original date posted:2015-08-24 📝 Original message:Its ...

2023-06-07T19:37:40+02:00

In reply to nevent1q…6j4c
_________________________

📅 Original date posted:2015-08-24
📝 Original message:Its more of a statement of "in the future, we expect things to happen
which would make this an interesting thing to do, so we state here that
it is not against spec to do so". Could reword it as "NODE_BLOOM is
distinct from NODE_NETWORK, and it is legal to advertise NODE_BLOOM but
not NODE_NETWORK (though there is little reason to do so now, some
proposals may make this more useful in the future)"?

Matt

On 08/24/15 15:29, Wladimir J. van der Laan wrote:
>> NODE_BLOOM is distinct from NODE_NETWORK, and it is legal to advertise
>> NODE_BLOOM but not NODE_NETWORK (eg for nodes running in pruned mode
>> which, nonetheless, provide filtered access to the data which they do have).
>
> But is this useful without having decided on a way to signal which blocks pruned nodes do have?
>
> It looks like the part between paranthesis is speculation and should be left to a future BIP.
>
> Wladimir
>

📅 Original date posted:2015-08-21 📝 Original message:The ...

2023-06-07T19:37:39+02:00

In reply to nevent1q…sn2k
_________________________

📅 Original date posted:2015-08-21
📝 Original message:The proposal will not break any existing clients in the first release.
After sufficient time to upgrade SPV clients, a new version will be
released which will result in older SPV clients finding themselves
disconnected from peers when they send filter* commands, so they can go
find other peers which do support bloom filtering.

On 08/21/15 05:48, Jeff Garzik wrote:
> If this is widely deployed + enabled, what is the impact to current
> wallets in use?
>
>
> On Fri, Aug 21, 2015 at 12:46 AM, Matt Corallo via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
> Peter: Since I stole most of this text from your old BIP, should I leave
> you as an author?
>
> BIP: ?
> Title: NODE_BLOOM service bit
> Author: Matt Corallo <bip at bluematt.me <mailto:bip at bluematt.me>>,
> Peter Todd <pete at petertodd.org <mailto:pete at petertodd.org>>
> Type: Standards Track (draft)
> Created: 20-08-2015
>
> Abstract
> ========
>
> This BIP extends BIP 37, Connection Bloom filtering, by defining a
> service bit to allow peers to advertise that they support bloom filters
> explicitly. It also bumps the protocol version to allow peers to
> identify old nodes which allow bloom filtering of the connection despite
> lacking the new service bit.
>
>
> Motivation
> ==========
>
> BIP 37 did not specify a service bit for the bloom filter service, thus
> implicitly assuming that all nodes that serve peers data support it.
> However, the connection filtering algorithm proposed in BIP 37, and
> implemented in several clients today, has been shown to provide little
> to no privacy, as well as being a large DoS risk on some nodes. Thus,
> allowing node operators to disable connection bloom filtering is a
> much-needed feature.
>
>
> Specification
> =============
>
> The following protocol bit is added:
>
> NODE_BLOOM = (1 << 2)
>
> Nodes which support bloom filters should set that protocol bit.
> Otherwise it should remain unset. In addition the protocol version is
> increased from 70002 to 70011 in the reference implementation. It is
> often the case that nodes which have a protocol version smaller than
> 70011, but larger than 70000 support bloom filtered connections without
> the NODE_BLOOM bit set, however clients which require bloom filtered
> connections should avoid making this assumption.
>
> NODE_BLOOM is distinct from NODE_NETWORK, and it is legal to advertise
> NODE_BLOOM but not NODE_NETWORK (eg for nodes running in pruned mode
> which, nonetheless, provide filtered access to the data which they
> do have).
>
> If a node does not support bloom filters but receives a "filterload",
> "filteradd", or "filterclear" message from a peer the node should
> disconnect that peer immediately. For backwards compatibility, in
> initial implementations, nodes may choose to only disconnect nodes which
> have the new protocol version set and attempt to send a filter command.
>
> While outside the scope of this BIP it is suggested that DNS seeds and
> other peer discovery mechanisms support the ability to specify the
> services required; current implementations simply check only that
> NODE_NETWORK is set.
>
>
> Design rational
> ===============
>
> A service bit was chosen as applying a bloom filter is a service.
>
> The increase in protocol version is for backwards compatibility. In
> initial implementations, old nodes which are not yet aware of NODE_BLOOM
> and use a protocol version < 70011 may still send filter* messages to a
> node without NODE_BLOOM. This feature may be removed after there are
> sufficient NODE_BLOOM nodes available and SPV clients have upgraded,
> allowing node operators to fully close the bloom-related DoS vectors.
>
>
> Reference Implementation
> ========================
>
> https://github.com/bitcoin/bitcoin/pull/6579
>
>
> Copyright
> =========
>
> This document is placed in the public domain.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

📅 Original date posted:2015-08-24 📝 Original ...

2023-06-07T19:37:36+02:00

In reply to nevent1q…hlnc
_________________________

📅 Original date posted:2015-08-24
📝 Original message:I'll just quote what I said on github:

Neither this pull nor the BIP has any stated intention of phasing out
bloom filtering support in the protocol. As much as I'd love to, I 100%
agree with @mikehearn here, that would break any ability of SPV clients
to operate on the P2P network (either as a way to double-check
centralized servers, or otherwise), and that is really not a good idea
without a replacement in place. This pull/BIP DOES suggest we phase out
REQUIRED bloom filtering support in the protocol - thereby fixing the
peer selection of SPV clients in the face of btcd with some flags/many
patched versions of Core/etc peers, providing a remedy for a potential
DoS attack, etc.

Matt

On 08/24/15 15:19, Tom Harding via bitcoin-dev wrote:
> On 8/21/2015 3:06 PM, Peter Todd via bitcoin-dev wrote:
>> On Fri, Aug 21, 2015 at 05:55:58PM +0000, Matt Corallo wrote:
>>> Anyone have the best reference for the DoS issues?
>> Well actually, we can reference the DoS attacks that Bitcoin XT nodes
>> are undergoing right now - part of the attack is repeated Bloom filter
>> requests to soak up disk IO bandwidth.
>
> So, to summarize, someone is attacking Mike Hearn's bitcoin fork.
> Therefore, now is the perfect time to write a BIP and author changes
> that begin the process of dropping support for the most broadly
> successful class of wallets, which Mike Hearn's SPV client library enables.
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2015-08-21 📝 Original message:BIP ...

2023-06-07T19:37:36+02:00

In reply to nevent1q…yjja
_________________________

📅 Original date posted:2015-08-21
📝 Original message:BIP Editor: Can I get a BIP # for this?

On 08/21/15 17:55, Matt Corallo via bitcoin-dev wrote:
> Revised copy follows. re: mentioning the HTTP seeding stuff, I'm not
> sure we want to encourage more people aside from bitcoinj to use
> that...I thought about adding a DNS seed section to this bip, but
> decided against it...still, I think we should add the option to select
> service bits to DNS seeds ASAP.
>
> Re: need to "shard" the blockchain: not sure what you're referring to
> here. The bloom filter stuff requires you to download the chain
> in-order, sure, but you have to do that for headers anyway, and
> hopefully your total data isnt too much more than headers alone.
>
> Anyone have the best reference for the DoS issues?
>
> BIP: ?
> Title: NODE_BLOOM service bit
> Author: Matt Corallo <bip at bluematt.me>, Peter Todd <pete at petertodd.org>
> Type: Standards Track (draft)
> Created: 20-08-2015
>
> Abstract
> ========
>
> This BIP extends BIP 37, Connection Bloom filtering, by defining a
> service bit to allow peers to advertise that they support bloom filters
> explicitly. It also bumps the protocol version to allow peers to
> identify old nodes which allow bloom filtering of the connection despite
> lacking the new service bit.
>
>
> Motivation
> ==========
>
> BIP 37 did not specify a service bit for the bloom filter service, thus
> implicitly assuming that all nodes that serve peers data support it.
> However, the connection filtering algorithm proposed in BIP 37, and
> implemented in several clients today, has been shown to provide little
> to no privacy[1], as well as being a large DoS risk on some nodes[2].
> Thus, allowing node operators to disable connection bloom filtering is a
> much-needed feature.
>
>
> Specification
> =============
>
> The following protocol bit is added:
>
> NODE_BLOOM = (1 << 2)
>
> Nodes which support bloom filters should set that protocol bit.
> Otherwise it should remain unset. In addition the protocol version is
> increased from 70002 to 70011 in the reference implementation. It is
> often the case that nodes which have a protocol version smaller than
> 70011, but larger than 70000 support bloom filtered connections without
> the NODE_BLOOM bit set, however clients which require bloom filtered
> connections should avoid making this assumption.
>
> NODE_BLOOM is distinct from NODE_NETWORK, and it is legal to advertise
> NODE_BLOOM but not NODE_NETWORK (eg for nodes running in pruned mode
> which, nonetheless, provide filtered access to the data which they do have).
>
> If a node does not support bloom filters but receives a "filterload",
> "filteradd", or "filterclear" message from a peer the node should
> disconnect that peer immediately. For backwards compatibility, in
> initial implementations, nodes may choose to only disconnect nodes which
> have the new protocol version set and attempt to send a filter command.
>
> While outside the scope of this BIP it is suggested that DNS seeds and
> other peer discovery mechanisms support the ability to specify the
> services required; current implementations simply check only that
> NODE_NETWORK is set.
>
>
> Design rational
> ===============
>
> A service bit was chosen as applying a bloom filter is a service.
>
> The increase in protocol version is for backwards compatibility. In
> initial implementations, old nodes which are not yet aware of NODE_BLOOM
> and use a protocol version < 70011 may still send filter* messages to a
> node without NODE_BLOOM. This feature may be removed after there are
> sufficient NODE_BLOOM nodes available and SPV clients have upgraded,
> allowing node operators to fully close the bloom-related DoS vectors.
>
>
> Reference Implementation
> ========================
>
> https://github.com/bitcoin/bitcoin/pull/6579
>
>
> Copyright
> =========
>
> This document is placed in the public domain.
>
>
> References
> ==========
>
> [1] http://eprint.iacr.org/2014/763
> [2] ???? is one example where the issues were found, though others
> independently discovered issues as well. Sample DoS exploit code
> available at https://github.com/petertodd/bloom-io-attack.
>
>
>
> On 08/21/15 05:42, Peter Todd wrote:
>> On Thu, Aug 20, 2015 at 10:38:19PM -0700, Peter Todd via bitcoin-dev wrote:
>>>> Motivation
>>>> ==========
>>>>
>>>> BIP 37 did not specify a service bit for the bloom filter service, thus
>>>> implicitly assuming that all nodes that serve peers data support it.
>>>> However, the connection filtering algorithm proposed in BIP 37, and
>>>> implemented in several clients today, has been shown to provide little
>>>> to no privacy, as well as being a large DoS risk on some nodes. Thus,
>>>> allowing node operators to disable connection bloom filtering is a
>>>> much-needed feature.
>>>
>>> I'd reference that paper on bloom filters re: the "little to no privacy"
>>> issue. There's also a post in the bitcoinj mailing list somewhere IIRC
>>> talking about the default settings, and how they don't provide any
>>> privacy.
>>
>> Oh, and we should also point out that Bloom filters have scaling issues,
>> as each application of the filter has to scan the whole blockchain -
>> with future blocksize increases these issues increase, in some proposals
>> quite dramatically. The underlying idea also conflicts with some
>> proposals to "shard" the blockchain, again suggesting that we need a bit
>> to handle future upgrades to more scalable designs.
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2015-08-21 📝 Original message:On ...

2023-06-07T19:37:34+02:00

In reply to nevent1q…fwmx
_________________________

📅 Original date posted:2015-08-21
📝 Original message:On 08/21/15 22:06, Peter Todd wrote:
> On Fri, Aug 21, 2015 at 05:55:58PM +0000, Matt Corallo wrote:
>> Revised copy follows. re: mentioning the HTTP seeding stuff, I'm not
>> sure we want to encourage more people aside from bitcoinj to use
>> that...I thought about adding a DNS seed section to this bip, but
>> decided against it...still, I think we should add the option to select
>> service bits to DNS seeds ASAP.
>
> Well, in general relying on seeds every time you start your node is a
> really bad idea; doing so much be carefully weighed against the
> downsides and should be used only as a last resort. Nodes should be
> doing caching and proper gossip protocol participation whenever
> possible. (note how bitcoinj nodes *do* rely on centralized servers,
> implemented with an unauthenticated, unencrypted, protocol - the worst
> of all possible solutions with many possible MITM vectors and privacy
> security holes)
>
> To that end, I'd be inclined to leave the DNS seed protocol as it is and
> let others solve the centralized server use-case, for which Cartographer
> isn't all that bad of a load balancing mechanism. Also as gmaxwell noted
> on IRC, adding flag bits does have privacy implications.

Had a discussion on IRC and with Pieter, and I kinda agree that the more
optimal way is for DNS seeds to, instead of returning NODE_NETWORK
nodes, return any node which responds to getaddr, allowing clients to
connect to a few DNS seeds by name, do a getaddr, then disconnect (like
Bitcoin Core does now if you're using Tor). They can then select the
peers they want based on nServices.

>> Re: need to "shard" the blockchain: not sure what you're referring to
>> here. The bloom filter stuff requires you to download the chain
>> in-order, sure, but you have to do that for headers anyway, and
>> hopefully your total data isnt too much more than headers alone.
>
> Any protocol change that would split blocks themselves into multiples.
> Not an easy problem to solve, but given the inherent O(n^2) scaling of
> global consensus blockchains, it's the only kind of solution that could
> in the future make the blockchain itself have reasonable scalability.

Meh, whatever, justification is already provided well enough without
having to go into "but if we did this long into the future" arguments.

>> Anyone have the best reference for the DoS issues?
>
> Well actually, we can reference the DoS attacks that Bitcoin XT nodes
> are undergoing right now - part of the attack is repeated Bloom filter
> requests to soak up disk IO bandwidth. I've CC'd Gavin and Mike - as far
> as I know they haven't published details of those attacks - a write-up
> would be very helpful.
>
> While so far those are being directed only at XT nodes, obviously this
> is a potential issue for Core nodes as well. Like I mentioned last time
> around, it's critical that miners aren't affected by these attacks -
> nodes simply serving SPV wallet clients are much less latency sensitive,
> so a good DoS attack mitigation strategy would be to have the two
> classes of nodes out there "in the wild"

Ehh, I was going more for the oldest mention.

>> BIP: ?
>> Title: NODE_BLOOM service bit
>> Author: Matt Corallo <bip at bluematt.me>, Peter Todd <pete at petertodd.org>
>> Type: Standards Track (draft)
>> Created: 20-08-2015
>>
>> Abstract
>> ========
>>
>> This BIP extends BIP 37, Connection Bloom filtering, by defining a
>> service bit to allow peers to advertise that they support bloom filters
>> explicitly. It also bumps the protocol version to allow peers to
>> identify old nodes which allow bloom filtering of the connection despite
>> lacking the new service bit.
>>
>>
>> Motivation
>> ==========
>>
>> BIP 37 did not specify a service bit for the bloom filter service, thus
>> implicitly assuming that all nodes that serve peers data support it.
>> However, the connection filtering algorithm proposed in BIP 37, and
>> implemented in several clients today, has been shown to provide little
>> to no privacy[1], as well as being a large DoS risk on some nodes[2].
>> Thus, allowing node operators to disable connection bloom filtering is a
>> much-needed feature.
>>
>>
>> Specification
>> =============
>>
>> The following protocol bit is added:
>>
>> NODE_BLOOM = (1 << 2)
>>
>> Nodes which support bloom filters should set that protocol bit.
>> Otherwise it should remain unset. In addition the protocol version is
>> increased from 70002 to 70011 in the reference implementation. It is
>> often the case that nodes which have a protocol version smaller than
>> 70011, but larger than 70000 support bloom filtered connections without
>> the NODE_BLOOM bit set, however clients which require bloom filtered
>> connections should avoid making this assumption.
>>
>> NODE_BLOOM is distinct from NODE_NETWORK, and it is legal to advertise
>> NODE_BLOOM but not NODE_NETWORK (eg for nodes running in pruned mode
>> which, nonetheless, provide filtered access to the data which they do have).
>>
>> If a node does not support bloom filters but receives a "filterload",
>> "filteradd", or "filterclear" message from a peer the node should
>> disconnect that peer immediately. For backwards compatibility, in
>> initial implementations, nodes may choose to only disconnect nodes which
>> have the new protocol version set and attempt to send a filter command.
>>
>> While outside the scope of this BIP it is suggested that DNS seeds and
>> other peer discovery mechanisms support the ability to specify the
>> services required; current implementations simply check only that
>> NODE_NETWORK is set.
>>
>>
>> Design rational
>> ===============
>>
>> A service bit was chosen as applying a bloom filter is a service.
>>
>> The increase in protocol version is for backwards compatibility. In
>> initial implementations, old nodes which are not yet aware of NODE_BLOOM
>> and use a protocol version < 70011 may still send filter* messages to a
>> node without NODE_BLOOM. This feature may be removed after there are
>> sufficient NODE_BLOOM nodes available and SPV clients have upgraded,
>> allowing node operators to fully close the bloom-related DoS vectors.
>>
>>
>> Reference Implementation
>> ========================
>>
>> https://github.com/bitcoin/bitcoin/pull/6579
>>
>>
>> Copyright
>> =========
>>
>> This document is placed in the public domain.
>>
>>
>> References
>> ==========
>>
>> [1] http://eprint.iacr.org/2014/763
>> [2] ???? is one example where the issues were found, though others
>> independently discovered issues as well. Sample DoS exploit code
>> available at https://github.com/petertodd/bloom-io-attack.
>

📅 Original date posted:2015-08-21 📝 Original ...

2023-06-07T19:37:33+02:00

In reply to nevent1q…p3cp
_________________________

📅 Original date posted:2015-08-21
📝 Original message:Revised copy follows. re: mentioning the HTTP seeding stuff, I'm not
sure we want to encourage more people aside from bitcoinj to use
that...I thought about adding a DNS seed section to this bip, but
decided against it...still, I think we should add the option to select
service bits to DNS seeds ASAP.

Re: need to "shard" the blockchain: not sure what you're referring to
here. The bloom filter stuff requires you to download the chain
in-order, sure, but you have to do that for headers anyway, and
hopefully your total data isnt too much more than headers alone.

Anyone have the best reference for the DoS issues?

BIP: ?
Title: NODE_BLOOM service bit
Author: Matt Corallo <bip at bluematt.me>, Peter Todd <pete at petertodd.org>
Type: Standards Track (draft)
Created: 20-08-2015

Abstract
========

This BIP extends BIP 37, Connection Bloom filtering, by defining a
service bit to allow peers to advertise that they support bloom filters
explicitly. It also bumps the protocol version to allow peers to
identify old nodes which allow bloom filtering of the connection despite
lacking the new service bit.

Motivation
==========

BIP 37 did not specify a service bit for the bloom filter service, thus
implicitly assuming that all nodes that serve peers data support it.
However, the connection filtering algorithm proposed in BIP 37, and
implemented in several clients today, has been shown to provide little
to no privacy[1], as well as being a large DoS risk on some nodes[2].
Thus, allowing node operators to disable connection bloom filtering is a
much-needed feature.

Specification
=============

The following protocol bit is added:

NODE_BLOOM = (1 << 2)

Nodes which support bloom filters should set that protocol bit.
Otherwise it should remain unset. In addition the protocol version is
increased from 70002 to 70011 in the reference implementation. It is
often the case that nodes which have a protocol version smaller than
70011, but larger than 70000 support bloom filtered connections without
the NODE_BLOOM bit set, however clients which require bloom filtered
connections should avoid making this assumption.

NODE_BLOOM is distinct from NODE_NETWORK, and it is legal to advertise
NODE_BLOOM but not NODE_NETWORK (eg for nodes running in pruned mode
which, nonetheless, provide filtered access to the data which they do have).

If a node does not support bloom filters but receives a "filterload",
"filteradd", or "filterclear" message from a peer the node should
disconnect that peer immediately. For backwards compatibility, in
initial implementations, nodes may choose to only disconnect nodes which
have the new protocol version set and attempt to send a filter command.

While outside the scope of this BIP it is suggested that DNS seeds and
other peer discovery mechanisms support the ability to specify the
services required; current implementations simply check only that
NODE_NETWORK is set.

Design rational
===============

A service bit was chosen as applying a bloom filter is a service.

The increase in protocol version is for backwards compatibility. In
initial implementations, old nodes which are not yet aware of NODE_BLOOM
and use a protocol version < 70011 may still send filter* messages to a
node without NODE_BLOOM. This feature may be removed after there are
sufficient NODE_BLOOM nodes available and SPV clients have upgraded,
allowing node operators to fully close the bloom-related DoS vectors.

Reference Implementation
========================

https://github.com/bitcoin/bitcoin/pull/6579

Copyright
=========

This document is placed in the public domain.

References
==========

[1] http://eprint.iacr.org/2014/763
[2] ???? is one example where the issues were found, though others
independently discovered issues as well. Sample DoS exploit code
available at https://github.com/petertodd/bloom-io-attack.

On 08/21/15 05:42, Peter Todd wrote:
> On Thu, Aug 20, 2015 at 10:38:19PM -0700, Peter Todd via bitcoin-dev wrote:
>>> Motivation
>>> ==========
>>>
>>> BIP 37 did not specify a service bit for the bloom filter service, thus
>>> implicitly assuming that all nodes that serve peers data support it.
>>> However, the connection filtering algorithm proposed in BIP 37, and
>>> implemented in several clients today, has been shown to provide little
>>> to no privacy, as well as being a large DoS risk on some nodes. Thus,
>>> allowing node operators to disable connection bloom filtering is a
>>> much-needed feature.
>>
>> I'd reference that paper on bloom filters re: the "little to no privacy"
>> issue. There's also a post in the bitcoinj mailing list somewhere IIRC
>> talking about the default settings, and how they don't provide any
>> privacy.
>
> Oh, and we should also point out that Bloom filters have scaling issues,
> as each application of the filter has to scan the whole blockchain -
> with future blocksize increases these issues increase, in some proposals
> quite dramatically. The underlying idea also conflicts with some
> proposals to "shard" the blockchain, again suggesting that we need a bit
> to handle future upgrades to more scalable designs.
>

📅 Original date posted:2015-08-14 📝 Original message:On ...

2023-06-07T19:34:55+02:00

In reply to nevent1q…w0da
_________________________

📅 Original date posted:2015-08-14
📝 Original message:On 08/14/15 00:47, Mark Friedenbach via bitcoin-dev wrote:
> On Thu, Aug 13, 2015 at 4:42 PM, Joseph Poon via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
>
> I haven't tested the details of this, but is there another bit available
> for use in the future for the relative blockheight?
>
> I strongly believe that Lightning needs mitigations for a systemic
> supervillan attack which attemps to flood the network with transactions,
> which can hypothetically be mitigated with something like a timestop
> bit (as originally suggested by gmaxwell).
>
>
> This proposal includes no such provision.
>
> Since we talked about it, I spent considerable time thinking about the
> supposed risk and proposed mitigations. I'm frankly not convinced that
> it is a risk of high enough credibility to worry about, or if it is that
> a protocol-level complication is worth doing.
>
> The scenario as I understand it is a hub turns evil and tries to cheat
> every single one of its users out of their bonds. Normally a lightning
> user is protected form such behavior because they have time to broadcast
> their own transactions spending part or all of the balance as fees.

My concern is how the hell do you automate this? Having a threat of
"well, everyone could update their software to a new version which will
destroy all coins right now" is kinda useless, and trying to come up
with a reasonable set of metrics as to how much and when you move from
just paying the fee to destroying coins is really hard, especially if
you assume the attacker is a miner with, say, enough hashrate (maybe
rented) to get one or three blocks in the next day (the timeout period).

> Therefore because of the threat of mutually assured destruction, the
> optimal outcome is to be an honest participant.
>
> But, the argument goes, the hub has many channels with many different
> people closing at the same time. So if the hub tries to cheat all of
> them at once by DoS'ing the network, it can do so and spend more in fees
> than any one participant stands to lose. My issue with this is that
> users don't act alone -- users can be assured that other users will
> react, and all of them together have enough coins to burn to make the
> attack unprofitable.

Now users are coordinating quickly in an attack scenario?

> The hub-cheats-many-users case really is the same
> as the hub-cheats-one-user case if the users act out their role in
> unison, which they don't have to coordinate to do.
>
> Other than that, even if you are still concerned about that scenario,
> I'm not sure timestop is the appropriate solution. A timestop is a
> protocol-level complication that is not trivial to implement, indeed I'm
> not even sure there is a way to implement it at all -- how do you
> differentiate in consensus code a DoS attack from regular old blocks
> filling up? And if you could, why add further complication to the
> consensus protocol?

Yea, implementation is really tricky here. I do not at all think we
should be thinking about implementing this any time soon, and should
assume Lightning will have to stand reasonably on its own without it
first, and only if it gains a lot of traction will there be enough
motivation for making such a change at the Bitcoin protocol level for
Lightning.

> A simpler solution to me seems to be outsourcing the response to an
> attack to a third party

Doesnt that defeat the purpose of Lightning?

> or otherwise engineering ways for users to
> respond-by-default even if their wallet is offline, or otherwise
> assuring sufficient coordination in the event of a bad hub.

I'm not even sure if sufficient coordination is a sufficient solution.
If you assume a hub just shut down, and everyone is trying to flush to
the chain, with a backlog of a few days worth of transactions (with
timeouts of a day or so), and users are even paying huge fees (99% of
what they'd get back), if the former-hub is a miner, it can claim that
last 1% of many of the transactions that take longer than a day to confirm.

📅 Original date posted:2015-08-05 📝 Original ...

2023-06-07T19:33:16+02:00

In reply to nevent1q…9hyp
_________________________

📅 Original date posted:2015-08-05
📝 Original message:See-also: Bitcoinrelaynetwork.org. It's already in use my the majority of large miners, is publicly available to anyone, and the protocol is rather simple and the client could be tweaked easily to keep exactly it's block ready to quickly relay to the nearest server (ie only have to relay the header, the coinbase transaction, and only small other data... Experience shows this is really easy to fit into one packet on the wire). It's not nearly as complicated as your suggestion, but may still marginally favor well-connected miners, but hopefully not much (when you're taking about single packets, it should all be latency, and the servers are well distributed). If you feel so inclined, there are some todos to make it really meet is efficiency limits filled on github.com/TheBlueMatt/RelayNode, feel free to rewrite the protocol if you really want :).

Matt

On August 5, 2015 9:07:44 PM GMT+02:00, Arnoud Kouwenhoven - Pukaki Corp via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>Hello all.
>
>We’d like to share an idea we have to dramatically increase the bitcoin
>block propagation speed after a new block has been mined for the first
>time.
>
>Efficient bitcoin block propagation
>A proposed solution to provide near-instantaneous block propagation on
>the
>bitcoin network, even with slow network connections or large block
>sizes.
>Increasing mining efficiency for everyone while decreasing transaction
>confirmation times and strengthening the distributed nature of bitcoin.
>
>Short summary: we propose to introduce bitcoin-backed guarantees
>(“Guarantee Messages”) between miners. This would allow miners to mine
>on
>blocks that are not yet fully transmitted. This reduces the effect of
>slow
>internet connections, leveling the playing field between the 1st world
>fiberoptic datacenter miners and the rest of the world. We also believe
>it
>strengthens the bitcoin network by using existing processing power that
>is
>currently wasted into further securing the blockchain, and it reduces
>the
>likelihood of transactions becoming confirmed, then unconfirmed and
>then
>-hopefully- confirmed again (due to different miners finding competing
>blocks with different transactions at approx the same time).
>
>It is possible to implement our idea as a fork of bitcoind, or as layer
>between the standard bitcoind and the mining equipment. In the future
>it
>could be incorporated in the bitcoin core if and when that becomes a
>priority, but that step would not make sense until it becomes a
>priority.
>
>There are a lot of nuances in this idea, and the first reaction is
>quite
>probably that this is a crazy idea. We have attempted to address the
>most
>important nuances in our proposal, which is currently at v.0.2.
>
>We cannot guarantee that there are no ‘hidden devils in the details’
>and we
>invite you to be critical in a friendly and constructive manner. We
>will do
>our best to answer all questions that arise.
>
>The ‘official’ proposal is at:
>PDF: http://pukaki.bz/efficient-bitcoin-block-propagation-v.0.2.pdf
>HTML: http://pukaki.bz/efficient-bitcoin-block-propagation-v.0.2.html
>
>-- Arnoud Kouwenhoven
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev at lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20150805/fc69da36/attachment.html>;

📅 Original date posted:2016-01-07 📝 Original message:So ...

2023-06-07T19:31:31+02:00

In reply to nevent1q…8f05
_________________________

📅 Original date posted:2016-01-07
📝 Original message:So just because other attacks are possible we should weaken the crypto
we use? You may feel comfortable weakening crypto used to protect a few
billion dollars of other peoples' money, but I dont.

On 01/07/16 23:39, Gavin Andresen via bitcoin-dev wrote:
> Thanks, Ethan, that's helpful and I'll stop thinking that collision
> attacks require 2^(n/2) memory...
>
> So can we quantify the incremental increase in security of
> SHA256(SHA256) over RIPEMD160(SHA256) versus the incremental increase in
> security of having a simpler implementation of segwitness?
>
> I'm going to claim that the difference in the first case is very, very,
> very small-- the risk of an implementation error caused by having
> multiple ways of interpreting the segwitness hash in the scriptPubKey is
> much, much greater.
>
> And even if there IS some risk of collision attack now or at some point
> in the future, I claim that it is easy for wallets to mitigate that
> risk. In fact, the principle of security in depth means wallets that
> don't completely control the scriptPubKeys they're creating on behalf of
> users SHOULD be coded to mitigate that risk (e.g. not allowing arbitrary
> data around a user's public key in a Script so targeted substring
> attacks are eliminated entirely).
>
> Purely from a security point of view, I think a single 20-byte
> segwitness in the scriptPubKey is the best design.
> "Keep the design as simple and small as possible"
> https://www.securecoding.cert.org/confluence/plugins/servlet/mobile#content/view/2426
>
> Add in the implied capacity increase of smaller scriptPubKeys and I
> still think it is a no-brainer.
>
>
> On Thu, Jan 7, 2016 at 5:56 PM, Ethan Heilman <eth3rs at gmail.com
> <mailto:eth3rs at gmail.com>> wrote:
>
> >Ethan: your algorithm will find two arbitrary values that collide. That isn't useful as an attack in the context we're talking about here (both of those values will be useless as coin destinations with overwhelming probability).
>
> I'm not sure exactly the properties you want here and determining
> these properties is not an easy task, but the case is far worse than
> just two random values. For instance: (a). with a small modification
> my algorithm can also find collisions containing targeted substrings,
> (b). length extension attacks are possible with RIPEMD160.
>
> (a). targeted cycles:
>
> target1 = "str to prepend"
> target2 = "str to end with"
>
> seed = {0,1}^160
> x = hash(seed)
>
> for i in 2^80:
> ....x = hash(target1||x||target2)
> x_final = x
>
> y = hash(tartget1||x_final||target2)
>
> for j in 2^80:
> ....if y == x_final:
> ........print "cycle len: "+j
> ........break
> ....y = hash(target1||y||target2)
>
> If a collision is found, the two colliding inputs must both start with
> "str to prepend" and end with the phrase "str to end with". As before
> this only requires 2^81.5 computations and no real memory. For an
> additional 2**80 an adversary has an good change of finding two
> different targeted substrings which collide. Consider the case where
> the attacker mixes the targeted strings with the hash output:
>
> hash("my name is=0x329482039483204324423"+x[1]+", my favorite number
> is="+x) where x[1] is the first bit of x.
>
> (b). length extension attacks
>
> Even if all the adversary can do is create two random values that
> collide, you can append substrings to the input and get collisions.
> Once you find two random values hash(x) = hash(y), you could use a
> length extension attack on RIPEMD-160 to find hash(x||z) = hash(y||z).
>
> Now the bitcoin wiki says:
> "The padding scheme is identical to MD4 using Merkle–Damgård
> strengthening to prevent length extension attacks."[1]
>
> Which is confusing to me because:
>
> 1. MD4 is vulnerable to length extension attacks
> 2. Merkle–Damgård strengthening does not protect against length
> extension: "Indeed, we already pointed out that none of the 64
> variants above can withstand the 'extension' attack on the MAC
> application, even with the Merkle-Damgard strengthening" [2]
> 3. RIPEMD-160 is vulnerable to length extension attacks, is Bitcoin
> using a non-standard version of RIPEMD-160.
>
> RIPEMD160(SHA256()) does not protect against length extension attacks
> on SHA256, but should protect RIPEMD-160 against length extension
> attacks as RIPEMD-160 uses 512-bit message blocks. That being said we
> should be very careful here. Research has been done that shows that
> cascading the same hash function twice is weaker than using HMAC[3]. I
> can't find results on cascading RIPEMD160(SHA256()).
>
> RIPEMD160(SHA256()) seems better than RIPEMD160() though, but security
> should not rest on the notion that an attacker requires 2**80 memory,
> many targeted collision attacks can work without much memory.
>
> [1]: https://en.bitcoin.it/wiki/RIPEMD-160
> [2]: "Merkle-Damgard Revisited: How to Construct a Hash Function"
> https://www.cs.nyu.edu/~puniya/papers/merkle.pdf
> [3]: https://www.cs.nyu.edu/~dodis/ps/h-of-h.pdf
>
> On Thu, Jan 7, 2016 at 4:06 PM, Gavin Andresen via bitcoin-dev
> <bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>> wrote:
> > Maybe I'm asking this question on the wrong mailing list:
> >
> > Matt/Adam: do you have some reason to think that RIPEMD160 will be
> broken
> > before SHA256?
> > And do you have some reason to think that they will be so broken
> that the
> > nested hash construction RIPEMD160(SHA256()) will be vulnerable?
> >
> > Adam: re: "where to stop" : I'm suggesting we stop exactly at
> the current
> > status quo, where we use RIPEMD160 for P2SH and P2PKH.
> >
> > Ethan: your algorithm will find two arbitrary values that
> collide. That
> > isn't useful as an attack in the context we're talking about here
> (both of
> > those values will be useless as coin destinations with overwhelming
> > probability).
> >
> > Dave: you described a first preimage attack, which is 2**160 cpu
> time and no
> > storage.
> >
> >
> > --
> > --
> > Gavin Andresen
> >
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev at lists.linuxfoundation.org
> <mailto:bitcoin-dev at lists.linuxfoundation.org>
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >
>
>
>
>
> --
> --
> Gavin Andresen
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2016-01-07 📝 Original message:We ...

2023-06-07T19:31:28+02:00

In reply to nevent1q…ae6r
_________________________

📅 Original date posted:2016-01-07
📝 Original message:We absolutely should be worried about 80-bit collision resistance.
Collisions only take 2**80 work if the hash is theoretically perfect,
which is never the case, not to mention that collision resistance is
almost always the first thing to go for hash functions, and often starts
to get easier slowly long, long before anyone is truly worried about the
security of the hash function.

I would never assume RIPEMD160's collision resistance is 2**80, and
would definitely never wager a significant amount of money that this
remains true for, say, five years.

Matt

On 01/07/16 19:02, Gavin Andresen via bitcoin-dev wrote:
> I'm hoisting this from some private feedback I sent on the segregated
> witness BIP:
>
> I said:
>
> "I'd also use RIPEMD160(SHA256()) as the hash function and save the 12
> bytes-- a successful preimage attack against that ain't gonna happen
> before we're all dead. I'm probably being dense, but I just don't see
> how a collision attack is relevant here."
>
> Pieter responded:
>
> "The problem case is where someone in a contract setup shows you a
> script, which you accept as being a payment to yourself. An attacker
> could use a collision attack to construct scripts with identical hashes,
> only one of which does have the property you want, and steal coins.
>
> So you really want collision security, and I don't think 80 bits is
> something we should encourage for that. Normal pubkey hashes don't have
> that problem, as they can't be constructed to pay to you."
>
> ... but I'm unconvinced:
>
> "But it is trivial for contract wallets to protect against collision
> attacks-- if you give me a script that is "gavin_pubkey CHECKSIG
> arbitrary_data OP_DROP" with "I promise I'm not trying to rip you off,
> just ignore that arbitrary data" a wallet can just refuse. Even more
> likely, a contract wallet won't even recognize that as a pay-to-gavin
> transaction.
>
> I suppose it could be looking for some form of "gavin_pubkey
> somebody_else_pubkey CHECKMULTISIG ... with the attacker using
> somebody_else_pubkey to force the collision, but, again, trivial
> contract protocol tweaks ("send along a proof you have the private key
> corresponding to the public key" or "everybody pre-commits pubkeys
> they'll use at protocol start") would protect against that.
>
> Adding an extra 12 bytes to every segwit to prevent an attack that takes
> 2^80 computation and 2^80 storage, is unlikely to be a problem in
> practice, and is trivial to protect against is the wrong tradeoff to make."
>
> 20 bytes instead of 32 bytes is a savings of almost 40%, which is
> significant.
>
> The general question I'd like to raise on this list is:
>
> Should we be worried, today, about collision attacks against RIPEMD160
> (our 160-bit hash)?
>
> Mounting a successful brute-force collision attack would require at
> least O(2^80) CPU, which is kinda-sorta feasible (Pieter pointed out
> that Bitcoin POW has computed more SHA256 hashes than that). But it also
> requires O(2^80) storage, which is utterly infeasible (there is
> something on the order of 2^35 bytes of storage in the entire world).
> Even assuming doubling every single year (faster than Moore's Law),
> we're four decades away from an attacker with THE ENTIRE WORLD's storage
> capacity being able to mount a collision attack.
>
>
> References:
>
> https://en.wikipedia.org/wiki/Collision_attack
>
> https://vsatglobalseriesblog.wordpress.com/2013/06/21/in-2013-the-amount-of-data-generated-worldwide-will-reach-four-zettabytes/
>
>
> --
> --
> Gavin Andresen
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2015-08-21 📝 Original message:BIP ...

2023-06-07T17:49:02+02:00

In reply to nevent1q…f059
_________________________

📅 Original date posted:2015-08-21
📝 Original message:BIP Editor: Can I get a BIP # for this?

On 08/21/15 17:55, Matt Corallo via bitcoin-dev wrote:
> Revised copy follows. re: mentioning the HTTP seeding stuff, I'm not
> sure we want to encourage more people aside from bitcoinj to use
> that...I thought about adding a DNS seed section to this bip, but
> decided against it...still, I think we should add the option to select
> service bits to DNS seeds ASAP.
>
> Re: need to "shard" the blockchain: not sure what you're referring to
> here. The bloom filter stuff requires you to download the chain
> in-order, sure, but you have to do that for headers anyway, and
> hopefully your total data isnt too much more than headers alone.
>
> Anyone have the best reference for the DoS issues?
>
> BIP: ?
> Title: NODE_BLOOM service bit
> Author: Matt Corallo <bip at bluematt.me>, Peter Todd <pete at petertodd.org>
> Type: Standards Track (draft)
> Created: 20-08-2015
>
> Abstract
> ========
>
> This BIP extends BIP 37, Connection Bloom filtering, by defining a
> service bit to allow peers to advertise that they support bloom filters
> explicitly. It also bumps the protocol version to allow peers to
> identify old nodes which allow bloom filtering of the connection despite
> lacking the new service bit.
>
>
> Motivation
> ==========
>
> BIP 37 did not specify a service bit for the bloom filter service, thus
> implicitly assuming that all nodes that serve peers data support it.
> However, the connection filtering algorithm proposed in BIP 37, and
> implemented in several clients today, has been shown to provide little
> to no privacy[1], as well as being a large DoS risk on some nodes[2].
> Thus, allowing node operators to disable connection bloom filtering is a
> much-needed feature.
>
>
> Specification
> =============
>
> The following protocol bit is added:
>
> NODE_BLOOM = (1 << 2)
>
> Nodes which support bloom filters should set that protocol bit.
> Otherwise it should remain unset. In addition the protocol version is
> increased from 70002 to 70011 in the reference implementation. It is
> often the case that nodes which have a protocol version smaller than
> 70011, but larger than 70000 support bloom filtered connections without
> the NODE_BLOOM bit set, however clients which require bloom filtered
> connections should avoid making this assumption.
>
> NODE_BLOOM is distinct from NODE_NETWORK, and it is legal to advertise
> NODE_BLOOM but not NODE_NETWORK (eg for nodes running in pruned mode
> which, nonetheless, provide filtered access to the data which they do have).
>
> If a node does not support bloom filters but receives a "filterload",
> "filteradd", or "filterclear" message from a peer the node should
> disconnect that peer immediately. For backwards compatibility, in
> initial implementations, nodes may choose to only disconnect nodes which
> have the new protocol version set and attempt to send a filter command.
>
> While outside the scope of this BIP it is suggested that DNS seeds and
> other peer discovery mechanisms support the ability to specify the
> services required; current implementations simply check only that
> NODE_NETWORK is set.
>
>
> Design rational
> ===============
>
> A service bit was chosen as applying a bloom filter is a service.
>
> The increase in protocol version is for backwards compatibility. In
> initial implementations, old nodes which are not yet aware of NODE_BLOOM
> and use a protocol version < 70011 may still send filter* messages to a
> node without NODE_BLOOM. This feature may be removed after there are
> sufficient NODE_BLOOM nodes available and SPV clients have upgraded,
> allowing node operators to fully close the bloom-related DoS vectors.
>
>
> Reference Implementation
> ========================
>
> https://github.com/bitcoin/bitcoin/pull/6579
>
>
> Copyright
> =========
>
> This document is placed in the public domain.
>
>
> References
> ==========
>
> [1] http://eprint.iacr.org/2014/763
> [2] ???? is one example where the issues were found, though others
> independently discovered issues as well. Sample DoS exploit code
> available at https://github.com/petertodd/bloom-io-attack.
>
>
>
> On 08/21/15 05:42, Peter Todd wrote:
>> On Thu, Aug 20, 2015 at 10:38:19PM -0700, Peter Todd via bitcoin-dev wrote:
>>>> Motivation
>>>> ==========
>>>>
>>>> BIP 37 did not specify a service bit for the bloom filter service, thus
>>>> implicitly assuming that all nodes that serve peers data support it.
>>>> However, the connection filtering algorithm proposed in BIP 37, and
>>>> implemented in several clients today, has been shown to provide little
>>>> to no privacy, as well as being a large DoS risk on some nodes. Thus,
>>>> allowing node operators to disable connection bloom filtering is a
>>>> much-needed feature.
>>>
>>> I'd reference that paper on bloom filters re: the "little to no privacy"
>>> issue. There's also a post in the bitcoinj mailing list somewhere IIRC
>>> talking about the default settings, and how they don't provide any
>>> privacy.
>>
>> Oh, and we should also point out that Bloom filters have scaling issues,
>> as each application of the filter has to scan the whole blockchain -
>> with future blocksize increases these issues increase, in some proposals
>> quite dramatically. The underlying idea also conflicts with some
>> proposals to "shard" the blockchain, again suggesting that we need a bit
>> to handle future upgrades to more scalable designs.
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2015-08-24 📝 Original ...

2023-06-07T17:49:01+02:00

In reply to nevent1q…fy6j
_________________________

📅 Original date posted:2015-08-24
📝 Original message:I'll just quote what I said on github:

Neither this pull nor the BIP has any stated intention of phasing out
bloom filtering support in the protocol. As much as I'd love to, I 100%
agree with @mikehearn here, that would break any ability of SPV clients
to operate on the P2P network (either as a way to double-check
centralized servers, or otherwise), and that is really not a good idea
without a replacement in place. This pull/BIP DOES suggest we phase out
REQUIRED bloom filtering support in the protocol - thereby fixing the
peer selection of SPV clients in the face of btcd with some flags/many
patched versions of Core/etc peers, providing a remedy for a potential
DoS attack, etc.

Matt

On 08/24/15 15:19, Tom Harding via bitcoin-dev wrote:
> On 8/21/2015 3:06 PM, Peter Todd via bitcoin-dev wrote:
>> On Fri, Aug 21, 2015 at 05:55:58PM +0000, Matt Corallo wrote:
>>> Anyone have the best reference for the DoS issues?
>> Well actually, we can reference the DoS attacks that Bitcoin XT nodes
>> are undergoing right now - part of the attack is repeated Bloom filter
>> requests to soak up disk IO bandwidth.
>
> So, to summarize, someone is attacking Mike Hearn's bitcoin fork.
> Therefore, now is the perfect time to write a BIP and author changes
> that begin the process of dropping support for the most broadly
> successful class of wallets, which Mike Hearn's SPV client library enables.
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

📅 Original date posted:2015-08-20 📝 Original message:I dont ...

2023-06-07T17:48:45+02:00

In reply to nevent1q…jkrh
_________________________

📅 Original date posted:2015-08-20
📝 Original message:I dont think a libconsensus would have any kind of networking layer, nor
is C++ an antique tool set (hopefully libconsensus can avoid a boost
dependency, though thats not antique either). Ideally it would have a
simple API to give it blocks and a simple API for it to inform you of
what the current chain is. If you really want to get fancy maybe it has
pluggable block storage, too, but I dont see why you couldnt use this in
~any client?

On 08/20/15 08:35, Tamas Blummer via bitcoin-dev wrote:
> Every re-implementation, re-factoring even copy-paste introduces a risk of disagreement,
> but also open the chance of doing the work better, in the sense of software engineering.
>
>> On Aug 20, 2015, at 10:06, Jorge Timón <jtimon at jtimon.cc> wrote:
>>
>>
>> But the goal is not reimplementing the consensus rules but rather
>> extract them from Bitcoin Core so that nobody needs to re-implement
>> them again.
>
>
>
> My goal is different. Compatibility with Bitcoin is important as I also want to deal with Bitcoins,
> but it is also imperative to be able to create and serve other block chains with other rules and for those
> I do not want to carry on the legacy of an antique tool set and a spaghetti style.
>
> Bits of Proof uses scala (akka networking), java (api service), c++ (leveledb and now libconsensus)
> and I am eager to integrate secp256k1 (c) as soon as part of consensus. The choices were
> made because each piece appears best in what they do.
>
> Tamas Blummer
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>