npub14c…0m739

Last Notes

2023-06-07 23:16:10 UTC

- reply

📅 Original date posted:2022-11-01
📝 Original message:Hi everyone,
I've made some significant changes to my package relay proposal based on
observations while implementing, feedback on this thread, and offline
discussions [1].
The new proposal is called Ancestor Package Relay, BIP331, and PR'd at
https://github.com/bitcoin/bips/pull/1382
The major changes to the proposal are:
1. Scope reduction to receiver-initiated only
2. Scope reduction to ancestor packages only
3. Removal of block hash from package information
1. Scope reduction to receiver-initiated only
Receiver-intiated package relay enables a node to ask for more information
when they suspect they are missing something (i.e. in this case to resolve
a missing parent tx). Sender-initiated package relay should, theoretically,
save a round trip by notifying the receiver ahead of time that "hey, this
is going to be a package, so make sure you download and submit these
transactions together." As with any proactive communication, there is a
chance that the node already knows this information, so this network
bandwidth was wasted. The logic used to decide _when_ to announce a package
proactively determines whether it is a net increase or decrease for overall
bandwidth usage. However, it's difficult to design anything to save
bandwidth without any idea of what its bandwidth usage actually looks like
in practice. We'll want to design the sender-initiated protocol carefully,
and inform the design decisions using data collected from the mainnet p2p
network. However, there is no historical transaction data to use because
the goal is to enable currently-rejected transactions to propagate. In
order to get this right, I propose we hold off on sender-initiated for now,
deploy receiver-initiated package relay, observe its usage and figure out
where we can save a round trip, and then produce a well-researched
sender-initiated package relay proposal.
2. Scope reduction to ancestor packages only
The proposal now only includes ancestor packages (previously called
tx-with-unconfirmed-ancestors or "v2" packages). The
child-with-unconfirmed-parents (previously called "v1") package has been
removed since it is a subset of ancestor packages and sender-initiated
relay has been removed. It may be relevant again in the future with
sender-initiated packages. If you were reviewing the previous proposal,
"pkginfo2" message has been renamed to "ancpkginfo" and "MSG_PKGINFO2" inv
type to "MSG_ANCPKGINFO".
3. Removal of block hash from package information
Most of the rationale is already on this thread. The block hash was an
attempt to enforce topology when chainstates differ, but isn't worth it. It
does not make much sense to drop or delay transaction data requests due to
mismatched chainstates, and the chainstate may change again between package
information and transaction data rounds. Instead, differences in chainstate
should be handled internally at the mempool validation level. The node
should de-duplicate recently-confirmed transactions and make a best effort
to validate the transactions it has already downloaded.
Thanks,
Gloria
[1]
https://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2022-10-11-package-relay/
On Fri, Jun 17, 2022 at 9:08 PM Antoine Riard <antoine.riard at gmail.com>
wrote:
> Hi Gloria,
>
> Thanks for working on that,
>
> > Always overestimating fees may sidestep this issue temporarily (while
> mempool
> > traffic is low and predictable), but this solution is not foolproof
> > and wastes users' money. The feerate market can change due to sudden
> > spikes in traffic (e.g. huge 12sat/vB dump a few days ago [9]) or
> > sustained, high volume of Bitcoin payments (e.g. April 2021 and
> > December 2017).
>
> Even if the LN implementations started to overestimate fees based on the
> historical worst-case of block inclusion feerates, there is still room for
> exploitation due to bip125 rule#3. Indeed, as long as the adversary is able
> to stick in the mempool a higher fee package while the feerate is not
> compelling enough to get it mined, your "honest" LN package should be
> bounced off.
>
> Considering Core's `MAX_STANDARD_TX_WEIGHT` of 400000 WU, I think it's
> practical for an attacker to succeed with this pinning tactic in periods of
> traffic spikes. Of course, LN implementation could overestimate fees with a
> target like `MAX_STANDARD_WEIGHT` * `worst_case_block_inclusion_feerate` to
> mitigate. However, assuming a value of 20sat for the latter, it would
> require from any LN user a minimal channel value of 2000000 satoshis to be
> theoretically secure against this type of pinning.
>
> So package relay is required to mitigate efficiently and realistically
> against pinning attacks, while conserving the same level of "economic"
> openness for Lightning. Beyond, it should be also noted that package relay
> is only building block of the full set of mitigations, and there should be
> a yet to-find-consensus-as-of-today other policy change such as
> user-elected package limits or replace-by-feerate.
>
> Anyway, I think it would be beneficial to document the design trade-offs
> of pinning mitigations in the `Rationale` subsection, at the attention of
> future L2s devs and users ?
>
> > {|
> > | Field Name || Type || Size || Purpose
> > |-
> > |version || uint32_t || 4 || Denotes a package version supported by the
> > node.
> > |-
> > |max_count || uint32_t || 4 ||Specifies the maximum number of
> transactions
> > per package this node is
> > willing to accept.
> > |-
> > |max_weight || uint32_t || 4 ||Specifies the maximum total weight per
> > package this node is willing
> > to accept.
> > |-
> > |}
>
> It's unclear to me what's the purpose of `max_count` and `max_weight` in
> the overall package relay flow, if they are intended to be exposed as
> configurable settings to node operators. If those fields are present to
> allow DoS protection increase of low-performance host, I believe it would
> be better to restrain the number of consumed UTXOs or executed sigops per
> package, as DoS vectors are more likely to be CPU-based, rather than
> memory-based as package size already bounded at acceptance by
> `MAX_PACKAGE_COUNT`.
>
> Thinking more we might introduce a `MAX_SIGOPS_PER_PACKAGR` limit, as
> otherwise if we naively grant one package announcement as equal to one
> transaction announcement in our tx-request logic, we might increase our DoS
> surface, node ressources staying equivalent ?
>
> > {|
> > | Field Name || Type || Size || Purpose
> > |-
> > |txns_length||CompactSize||1 or 3 bytes|| The number of transactions
> > requested.
>
> I'm not sure if we'll ever allow 3-bytes of package size, that would be
> ~32k of transactions.
>
> > |-
> > |txns||List of wtxids||txns_length * 32|| The wtxids of each transaction
> in
> > the package.
> > |}
>
> I think there is a bandwidth consumption trade-off to be aware of in the
> function of the package-relay usage. Let's consider a single issuer
> broadcasting the package to spend a shared-utxo, after the first shot the
> parent component should be spread across the network mempools. At each
> fee-bump, only the bumped CPFP will propagate on the network, the parent
> wtxid is reannounced in `pckginfo1` though there is no need to fetch it
> redundantly and waste bandwidth.
>
> However, I think the bandwidth saving does not hold in case of competing
> transaction issuers to spend a shared-utxo. In that case, the parent might
> differ at each broadcast and the list of wtxid is dissemblable at every
> claim of the shared-utxo. We could save the 32 bytes * number of packages
> elements by announcing a package_id, computed from the list of wtxids.
>
> I don't know about the occurrence of competing broadcasts among LN
> non-cooperative closes, where bandwidth could be potentially saved. I would
> say it's likely low because IIRC there is nothing in the LN protocol where
> the counterparties signal to each other they're going on-chain to introduce
> a competing broadcast synchronizing event. That said, it might increase in
> the future in a post-eltoo, multi-party contracting protocol world.
>
> So it might be interesting to document this design trade-off, if we seek
> bandwidth optimizations in function of a changing landscape in the type of
> transaction issuers in the future.
>
> > 3. The sender provides package information using "pckginfo1",
> > including the blockhash of the sender's best block, the wtxids of
> > the transactions in the package, their total fees and total weight.
>
> It's unclear to me how the `pckinfo1` receiver should proceed if the
> sender's best block is not in sync with the local chain tip.
>
> If the package isn't processed further, that's annoying for all the
> low-performance LN mobile clients, their chain tips might be always behind
> by few blocks from the p2p network nodes. It sounds like their packages
> won't propagate at all.
>
> If the package is processed further whatever the sender-receiver sync on
> chain tip, what's the purpose of including the blockhash ?
>
> > A child-with-unconfirmed-parents package for a transaction should be
> > announced when it meets the peer's fee filter but one or more of its
> > parents don't; a "inv(MSG_PCKG1)" instead of "inv(WTX)" should be sent
> > for the child. Each of the parents which meet the peer's fee filter
> > should still be announced normally.
>
> I believe we might have concerns of package-feerate downgrades attacks.
> E.g, in the LN context, where your channel counterparty is aiming to jam
> the propagation of the best-feerate version of the package.
>
> Let's say you have :
> - Alice's commitment_tx, at 1s/vB
> - package A + child B, at 3s/vB
> - package A + child C, at 10s/vB
> - block inclusion feerate at 10s/vB
> - Alice and Mallory are LN channel counterparties
> - commitment_tx is using LN's anchor outputs
>
> Alice's LN node broadcasts A+C to her mempool.
> Bob's feefilter is at 3s/vB.
> Mallory broadcasts her child B in Alice's mempool.
> LN commitment does not meet Bob's feefilter.
> Package A+child B at 3s/vB meets Bob's feefilter and is announced to Bob.
> Mallory broadcasts her own commitment_tx at 4s/vB in Bob's mempool.
> When Alice's child C is relayed to Bob, it's bounced off Bob's mempool.
>
> Do you think this situation is plausible ? Of course, it might be heavily
> dependent on package-relay yet-not-implemented internal p2p logic.
> I think it could be fixable if LN removes the counterparty's
> `anchor_output` on the local node's version of the commitment transaction,
> once package relay is deployed.
>
> Another question, at the next fee-bump iteration, Alice rebroadcasts
> A+child D, at 12 s/vB. Her node has already marked Alice's commitment_tx as
> known in Bob's `m_tx_inventory_known_filter`. So when a new higher fee
> child is discovered, should a `child-with-unconfirmed-parents` be
> announced between Alice and Bob ?
>
> Anyway, I think it would be interesting to pseudo-specify the
> package-assemblage algorithm (or if there is code already available) to see
> if it's robust against adversarial or unlucky situations ?
>
> > In fact, a package
> > of transactions may be announced using both Erlay and package relay.
> > After reconciliation, if the initiator would have announced a
> > transaction by wtxid but also has package information for it, they may
> > send "inv(MSG_PCKG)" instead of "inv(WTX)".
>
> Yes, I think this holds. Note, we might have to add to the reconciliation
> set low-fee parents succeeding the feefilter check due to a child. When the
> reconcildiff, we might have to bifucarte again on feefilter to decide to
> announce missing wtixds either as `inv(MSG_PCKG)` or `inv(WTX)`.
>
> (IIRC, I've already made few feedbacks offline though good to get them in
> the public space and think more)
>
> Antoine
>
> Le mar. 17 mai 2022 à 12:09, Gloria Zhao via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> a écrit :
>
>> Hi everybody,
>>
>> I’m writing to propose a set of p2p protocol changes to enable package
>> relay, soliciting feedback on the design and approach. Here is a link
>> to the most up-to-date proposal:
>>
>> https://github.com/bitcoin/bips/pull/1324
>>
>> If you have concept or approach feedback, *please respond on the
>> mailing list* to allow everybody to view and participate in the
>> discussion. If you find a typo or inaccurate wording, please feel free
>> to leave suggestions on the PR.
>>
>> I’m also working on an implementation for Bitcoin Core.
>>
>>
>> The rest of this post will include the same contents as the proposal,
>> with a bit of reordering and additional context. If you are not 100%
>> up-to-date on package relay and find the proposal hard to follow, I
>> hope you find this format more informative and persuasive.
>>
>>
>> ==Background and Motivation==
>>
>> Users may create and broadcast transactions that depend upon, i.e.
>> spend outputs of, unconfirmed transactions. A “package” is the
>> widely-used term for a group of transactions representable by a
>> connected Directed Acyclic Graph (where a directed edge exists between
>> a transaction that spends the output of another transaction).
>>
>> Incentive-compatible mempool and miner policies help create a fair,
>> fee-based market for block space. While miners maximize transaction
>> fees in order to earn higher block rewards, non-mining users
>> participating in transaction relay reap many benefits from employing
>> policies that result in a mempool with the same contents, including
>> faster compact block relay and more accurate fee estimation.
>> Additionally, users may take advantage of mempool and miner policy to
>> bump the priority of their transactions by attaching high-fee
>> descendants (Child Pays for Parent or CPFP). Only considering
>> transactions one at a time for submission to the mempool creates a
>> limitation in the node's ability to determine which transactions have
>> the highest feerates, since it cannot take into account descendants
>> until all the transactions are in the mempool. Similarly, it cannot
>> use a transaction's descendants when considering which of two
>> conflicting transactions to keep (Replace by Fee or RBF).
>>
>> When a user's transaction does not meet a mempool's minimum feerate
>> and they cannot create a replacement transaction directly, their
>> transaction will simply be rejected by this mempool. They also cannot
>> attach a descendant to pay for replacing a conflicting transaction.
>> This limitation harms users' ability to fee-bump their transactions.
>> Further, it presents a security issue in contracting protocols which
>> rely on **presigned**, time-sensitive transactions to prevent cheating
>> (HTLC-Timeout in LN Penalty [1] [2] [3], Unvault Cancel in Revault
>> [4], Refund Transaction in Discreet Log Contracts [5], Updates in
>> eltoo [6]). In other words, a key security assumption of many
>> contracting protocols is that all parties can propagate and confirm
>> transactions in a timely manner.
>>
>> In the past few years, increasing attention [0][1][2][3][6] has been
>> brought to **pinning attacks**, a type of censorship in which the
>> attacker uses mempool policy restrictions to prevent a transaction
>> from being relayed or getting mined. TLDR: revocation transactions
>> must meet a certain confirmation target to be effective, but their
>> feerates are negotiated well ahead of broadcast time. If the
>> forecasted feerate was too low and no fee-bumping options are
>> available, attackers can steal money from their counterparties. I walk
>> through a concrete example for stealing Lightning HTLC outputs at
>> ~23:58 in this talk [7][8]. Note that most attacks are only possible
>> when the market for blockspace at broadcast time demands much higher
>> feerates than originally anticipated at signing time. Always
>> overestimating fees may sidestep this issue temporarily (while mempool
>> traffic is low and predictable), but this solution is not foolproof
>> and wastes users' money. The feerate market can change due to sudden
>> spikes in traffic (e.g. huge 12sat/vB dump a few days ago [9]) or
>> sustained, high volume of Bitcoin payments (e.g. April 2021 and
>> December 2017).
>>
>> The best solution is to enable nodes to consider packages of
>> transactions as a unit, e.g. one or more low-fee parent transactions
>> with a high-fee child, instead of separately. A package-aware mempool
>> policy can help determine if it would actually be economically
>> rational to accept a transaction to the mempool if it doesn't meet fee
>> requirements individually. Network-wide adoption of these policies
>> would create a more purely-feerate-based market for block space and
>> allow contracting protocols to adjust fees (and therefore mining
>> priority) at broadcast time. Some support for packages has existed in
>> Bitcoin Core for years. Since v0.13, Bitcoin Core has used ancestor
>> packages instead of individual transactions to evaluate the incentive
>> compatibility of transactions in the mempool [10] and select them for
>> inclusion in blocks [11].
>>
>> Package Relay, the concept of {announcing, requesting, downloading}
>> packages between nodes on the p2p network, has also been discussed for
>> many years. The earliest public mention I can find is from 2015 [12].
>> The two most common use cases for package relay are fee-bumping
>> otherwise-too-low-fee transactions and reducing the amount of orphans.
>> It seems uncontroversial to say that everybody desires package relay
>> conceptually, with varying degrees of urgency. Lots of work has been
>> done by others over the past few years, from which I've taken
>> inspiration from [13][14][15][16].
>>
>> My approach has been to split the project into two components: (1) Package
>> Mempool Accept, which includes validation logic and mempool policy.
>> (3) Package Relay, which includes the p2p protocol changes.
>>
>> Progress so far:
>> After discussions with various developers of contracting protocols
>> (with heavier emphasis towards LN), it was determined that a
>> package containing a child with all of its unconfirmed parents
>> (child-with-unconfirmed-parents or 1-child-multi-parent package) would
>> be sufficient for their use case, i.e. fee-bumping presigned
>> transactions. A child-with-unconfirmed-parents package has several
>> properties that make many things easier to reason about.
>>
>> A few months ago, I proposed a set of policies for safe package
>> validation and fee assessment for packages of this restricted
>> topology [17]. A series of PRs implementing this proposal have
>> been merged into Bitcoin Core [18].
>>
>> Theoretically, developing a safe and incentive-compatible package
>> mempool acceptance policy is sufficient to solve this issue. Nodes
>> could opportunistically accept packages (e.g. by trying combinations
>> of transactions rejected from their mempools), but this practice would
>> likely be inefficient at best and open new Denial of Service attacks
>> at worst. Additional p2p messages may enable nodes to request and
>> share package validation-related information with one another in a
>> more communication-efficient way.
>>
>> Given that only package RBF remains for package mempool accept, and we
>> can make progress on p2p and mempool in parallel, I think it’s
>> appropriate to put forward a package relay proposal.
>>
>> ==Proposal==
>>
>> This proposal contains 2 components: a “generic” package relay
>> protocol and an extension of it, child-with-unconfirmed-parents
>> packages, as version 1 package relay. Another version of packages,
>> “tx-with-unconfirmed-ancestors” can be created to extend package relay
>> for eliminating orphans.
>>
>> ===Generic Package Relay===
>>
>> Two main ideas are introduced:
>>
>> Download and validate packages of transactions together.
>>
>> Provide information to help peers decide whether to request and/or how
>> to validate transactions which are part of a package.
>>
>> ====Intended Protocol Flow====
>>
>> Due to the asynchronous nature of a distributed transaction relay
>> network, nodes may not receive all of the information needed to
>> validate a transaction at once. For example, after a node completes
>> Initial Block Download (IBD) and first starts participating in
>> transaction relay with an empty mempool, it is common to receive
>> orphans. In such scenarios where a node is aware that it is missing
>> information, a ''receiver-initiated'' dialogue is appropriate:
>>
>> 1. Receiver requests package information.
>>
>> 2. The sender provides package information, including the wtxids of
>> the transactions in the package and anything else that might be
>> relevant (e.g. total fees and size).
>>
>> 3. The reciever uses the package information to decide how to request
>> and validate the transactions.
>>
>> Sometimes, no matter what order transactions are received by a node,
>> validating them individually is insufficient. When the sender is aware
>> of additional information that the receiver needs to accept a package,
>> a proactive ''sender-initiated'' dialogue should be enabled:
>>
>> 1. Sender announces they have package information pertaining to a
>> transaction that might otherwise be undesired on its own.
>>
>> 2. The receiver requests package information.
>>
>> 3. The sender provides package information, including the wtxids of
>> the transactions in the package and anything else that might be
>> relevant (e.g. total fees and size).
>>
>> 4. The reciever uses the package information to decide how to request
>> and validate the transactions.
>>
>> Package relay is negotiated between two peers during the version
>> handshake. Package relay requires both peers to support wtxid-based
>> relay because package transactions are referenced by their wtxid.
>>
>> ====New Messages====
>>
>> Three new protocol messages are added for use in any version of
>> package relay. Additionally, each version of package relay must define
>> its own inv type and "pckginfo" message version, referred to in this
>> document as "MSG_PCKG" and "pckginfo" respectively. See
>> BIP-v1-packages for a concrete example.
>>
>> =====sendpackages=====
>>
>> {|
>> | Field Name || Type || Size || Purpose
>> |-
>> |version || uint32_t || 4 || Denotes a package version supported by the
>> node.
>> |-
>> |max_count || uint32_t || 4 ||Specifies the maximum number of
>> transactions per package this node is
>> willing to accept.
>> |-
>> |max_weight || uint32_t || 4 ||Specifies the maximum total weight per
>> package this node is willing
>> to accept.
>> |-
>> |}
>>
>> 1. The "sendpackages" message has the structure defined above, with
>> pchCommand == "sendpackages".
>>
>> 2. During version handshake, nodes should send a "sendpackages"
>> message indicate they support package relay and may request
>> packages.
>>
>> 3. The message should contain a version supported by the node. Nodes
>> should send a "sendpackages" message for each version they support.
>>
>> 4. The "sendpackages" message MUST be sent before sending a "verack"
>> message. If a "sendpackages" message is received afer "verack", the
>> sender should be disconnected.
>>
>> 5. If 'fRelay==false' in a peer's version message, the node must not
>> send "sendpackages" to them. If a "sendpackages" message is
>> received by a peer after sending `fRelay==false` in their version
>> message, the sender should be disconnected.
>>
>> 6.. Upon receipt of a "sendpackages" message with a version that is
>> not supported, a node must treat the peer as if it never received the
>> message.
>>
>> 7. If both peers send "wtxidrelay" and "sendpackages" with the same
>> version, the peers should announce, request, and send package
>> information to each other.
>>
>> =====getpckgtxns=====
>>
>> {|
>> | Field Name || Type || Size || Purpose
>> |-
>> |txns_length||CompactSize||1 or 3 bytes|| The number of transactions
>> requested.
>> |-
>> |txns||List of wtxids||txns_length * 32|| The wtxids of each transaction
>> in the package.
>> |}
>>
>> 1. The "getpckgtxns" message has the structure defined above, with
>> pchCommand == "getpckgtxns".
>>
>> 2. A "getpckgtxns" message should be used to request all or some of
>> the transactions previously announced in a "pckginfo" message,
>> specified by witness transactiosome id.
>>
>> 3. Upon receipt of a "getpckgtxns" message, a node must respond with
>> either a "pckgtxns" containing the requested transactions or a
>> "notfound" message indicating one or more of the transactions is
>> unavailable. This allows the receiver to avoid downloading and storing
>> transactions that cannot be validated immediately.
>>
>> 4. A "getpckgtxns" message should only be sent if both peers agreed to
>> send packages in the version handshake. If a "getpckgtxns" message
>> is received from a peer with which package relay was not negotiated,
>> the sender should be disconnected.
>>
>> =====pckgtxns=====
>>
>> {|
>> | Field Name || Type || Size || Purpose
>> |-
>> |txns_length||CompactSize||1 or 3 bytes|| The number of transactions
>> provided.
>> |-
>> |txns||List of transactions||variable|| The transactions in the package.
>> |}
>>
>> 1. The "pckgtxns" message has the structure defined above, with
>> pchCommand == "pckgtxns".
>>
>> 2. A "pckgtxns" message should contain the transaction data requested
>> using "getpckgtxns".
>>
>> 3. A "pckgtxns" message should only be sent to a peer that requested
>> the package using "getpckgtxns". If a node receives an unsolicited
>> package, the sender should be disconnected.
>>
>> 4. A "pckgtxns" message should only be sent if both peers agreed to
>> send packages in the version handshake. If a "pckgtxns" message is
>> received from a peer with which package relay was not negotiated, the
>> sender should be disconnected.
>>
>> ===Version 1 Packages: child-with-unconfirmed-parents===
>>
>> This extends package relay for packages consisting of one transaction
>> and all of its unconfirmed parents,by defining version 1 packages, a
>> pckginfo1 message, and a MSG_PCKG1 inv type. It enables the use case
>> in which a child pays for its otherwise-too-low-fee parents and their
>> mempool conflict(s).
>>
>> ====Intended Protocol Flow====
>>
>> When relaying a package of low-fee parent(s) and high-fee child, the
>> sender and receiver do the following:
>>
>> 1. Sender announces they have a child-with-unconfirmed-parents package
>> for a child that pays for otherwise-too-low-fee parent(s) using
>> "inv(MSG_PCKG1)".
>>
>> 2. The receiver requests package information using
>> "getdata(MSG_PCKG1)".
>>
>> 3. The sender provides package information using "pckginfo1",
>> including the blockhash of the sender's best block, the wtxids of
>> the transactions in the package, their total fees and total weight.
>>
>> 4. The reciever uses the package information to decide how to request
>> the transactions. For example, if the receiver already has some of
>> the transactions in their mempool, they only request the missing ones.
>> They could also decide not to request the package at all based on the
>> fee information provided.
>>
>> 5. Upon receiving a "pckgtxns", the receiver submits the transactions
>> together as a package.
>>
>> ====New Messages====
>>
>> A new inv type, "MSG_PCKG1", and new protocol message, "PCKGINFO1",
>> are added.
>>
>> =====pckginfo1=====
>>
>> {|
>> | Field Name || Type || Size || Purpose
>> |-
>> |blockhash || uint256 || 32 || The chain tip at which this package is
>> defined.
>> |-
>> |pckg_fee||CAmount||4|| The sum total fees paid by all transactions in
>> the package.
>> |-
>> |pckg_weight||int64_t||8|| The sum total weight of all transactions in
>> the package.
>> |-
>> |txns_length||CompactSize||1 or 3 bytes|| The number of transactions
>> provided.
>> |-
>> |txns||List of wtxids||txns_length * 32|| The wtxids of each transaction
>> in the package.
>> |}
>>
>>
>> 1. The "pckginfo1" message has the structure defined above, with
>> pchCommand == "pckginfo1".
>>
>> 2. A "pckginfo1" message contains information about a version 1
>> package (defined below), referenced by the wtxid of the transaction
>> it pertains to and the current blockhash.
>>
>> 3. Upon receipt of a "pckginfo1" message, the node should decide if it
>> wants to validate the package, request transaction data if
>> necessary, etc.
>>
>> 4. Upon receipt of a malformed "pckginfo1" message or package that
>> does not abide by the max_count, max_weight, or other rules
>> specified by the version agreed upon in the initial negotiation, the
>> sender should be disconnected. If a node receives a "pckginfo1"
>> message for which the "pckg_fee" or "pckg_weight" do not reflect the
>> true total fees and weight, respectively, or the transactions in the
>> package, the message is malformed.
>>
>> 5. A node MUST NOT send a "pckginfo1" message that has not been
>> requested by the recipient. Upon receipt of an unsolicited
>> "pckginfo1", a node should disconnect the sender.
>>
>> 6. A "pckginfo1" message should only be sent if both peers agreed to
>> send version 1 packages in the version handshake. If a "pckginfo1"
>> message is received from a peer with which package relay was not
>> negotiated, the sender should be disconnected.
>>
>> =====MSG_PCKG1=====
>>
>> 1. A new inv type (MSG_PCKG1 == 0x6) is added, for use in inv messages
>> and getdata requests pertaining to version 1 packages.
>>
>> 2. As an inv type, it indicates that both transaction data and version
>> 1 package information are available for the transaction. The
>> transaction is referenced by its wtxid. As a getdata request type, it
>> indicates that the sender wants package information for the
>> transaction.
>>
>> 3. Upon receipt of a "getdata" request for "MSG_PCKG1", the node
>> should respond with the version 1 package corresponding to the
>> requested transaction and its current chain tip, or with NOTFOUND.
>> The node should not assume that the sender is requesting the
>> transaction data as well.
>>
>> ====Child With Parent Packages Rules====
>>
>> A child-with-unconfirmed-parents package sent between nodes must abide
>> by the rules below, otherwise the package is malformed and the sender
>> should be disconnected.
>>
>> A version 1 or ''child-with-unconfirmed-parents'' package can be
>> defined for any transaction that spends unconfirmed inputs. The child
>> can be thought of as the "representative" of the package. This package
>> can be uniquely identified by the transaction's wtxid and the current
>> chain tip block hash.
>>
>> A ''child-with-unconfirmed-parents'' package MUST be:
>>
>> 1. ''Sorted topologically.'' For every transaction t in the package,
>> if any of t's parents are present in the package, the parent must
>> appear somewhere in the list before t. In other words, the
>> transactions must be sorted in ascending order of the number of
>> ancestors present in the package.
>>
>> 2. ''Only 1 child with unconfirmed parents.'' The package must consist
>> of one transaction and its unconfirmed parents. There must not be
>> any other transactions in the package. Other dependency relationships
>> may exist within the package (e.g. one parent may spend the output of
>> another parent) provided that topological order is respected.
>>
>> 3. ''All unconfirmed parents.'' All of the child's unconfirmed parents
>> must be present.
>>
>> 4. ''No conflicts.'' None of the transactions in the package may
>> conflict with each other (i.e. spend the same prevout).
>>
>> 5. ''Total fees and weight.'' The 'total_fee' and 'total_weight'
>> fields must accurately represent the sum total of all transactions'
>> fees and weights as defined in BIP141, respectively.
>>
>> Not all of the child's parents must be present; the child transaction
>> may also spend confirmed inputs. However, if the child has confirmed
>> parents, they must not be in the package.
>>
>> While a child-with-unconfirmed-parents package is perhaps most
>> relevant when the child has a higher feerate than its parents, this
>> property is not required to construct a valid package.
>>
>> ====Clarifications====
>>
>> ''Q: Under what circumstances should a sender announce a
>> child-with-unconfirmed-parents package?''
>>
>> A child-with-unconfirmed-parents package for a transaction should be
>> announced when it meets the peer's fee filter but one or more of its
>> parents don't; a "inv(MSG_PCKG1)" instead of "inv(WTX)" should be sent
>> for the child. Each of the parents which meet the peer's fee filter
>> should still be announced normally.
>>
>> ''Q: What if a new block arrives in between messages?''
>>
>> A child-with-unconfirmed-parents package is defined for a transaction
>> based on the current chain state. As such, a new block extending the
>> tip may decrease the number of transactions in the package (i.e. if
>> any of the transaction's parents were included in the block). In a
>> reorg, the number of transactions in the package may decrease or
>> increase (i.e. if any of the transaction's parents were included in a
>> block in the previous chain but not the new one).
>>
>> If the new block arrives before the "getdata" or "pckginfo1", nothing
>> needs to change.
>>
>> If the new block arrives before "getpckgtxns" or before "pckgtxns",
>> the receiver may need to re-request package information if the block
>> contained a transaction in the package. If the block doesn't contain
>> any transactions in the package, whether it extends the previous tip
>> or causes a reorg, nothing needs to change.
>>
>> ''Q: Can "getpckgtxns" and "pckgtxns" messages contain only one
>> transaction?''
>>
>> Yes.
>>
>> ===Further Protocol Extensions===
>>
>> When introducing a new type of package, assign it a version number "n"
>> and use an additional "sendpackages" message during version handshake
>> to negotiate support for it. An additional package information message
>> "pckginfon" and inv type "MSG_PCKGn" should be defined for the type of
>> package. However, "getpckgtxns" and "pckgtxns" do not need to be
>> changed.
>>
>> Example proposal for tx-with-unconfirmed-ancestors package relay: [19]
>>
>> ===Compatibility===
>>
>> Older clients remain fully compatible and interoperable after this
>> change. Clients implementing this protocol will only attempt to send
>> and request packages if agreed upon during the version handshake.
>>
>> ===Package Erlay===
>>
>> Clients using BIP330 reconciliation-based transaction relay (Erlay)
>> are able to use package relay without interference. In fact, a package
>> of transactions may be announced using both Erlay and package relay.
>> After reconciliation, if the initiator would have announced a
>> transaction by wtxid but also has package information for it, they may
>> send "inv(MSG_PCKG)" instead of "inv(WTX)".
>>
>> ===Rationale===
>>
>> ====P2P Message Design====
>>
>> These p2p messages are added for communication efficiency and, as
>> such, one should measure alternative solutions based on the resources
>> used to communicate (not necessarily trustworthy) information: We
>> would like to minimize network bandwidth, avoid downloading a
>> transaction more than once, avoid downloading transactions that are
>> eventually rejected, and minimize storage allocated for
>> not-yet-validated transactions.
>>
>> Consider these (plausible) scenarios in transaction relay:
>>
>> Alice (the "sender") is relaying transactions to Bob (the "receiver").
>> Alice's mempool has a minimum feerate of 1sat/vB and Bob's has a
>> minimum feerate of 3sat/vB. For simplicity, all transactions are
>> 1600Wu in virtual size and 500 bytes in serialized size. Apart from
>> the spending relationships specified, all other inputs are from
>> confirmed UTXOs.
>>
>> 1. Package {A, B} where A pays 0 satoshis and B pays 8000 satoshis in
>> fees.
>>
>> 2. Package {C, D} where C pays 0 satoshis and D pays 1200 satoshis in
>> fees.
>>
>> 3. Package {E, F, G, H, J} that pays 4000, 8000, 0, 2000, and 4000
>> satoshis in fees, respectively.
>>
>> ====Alternative Designs Considered====
>>
>> ''Package Information Only:'' Just having "pckginfo" gives enough
>> information for the receiver to accept the package. Omit the
>> "getpckgtxns" and "pckgtxns" messages. While this option is a good
>> fallback if batched transaction download fails for some reason, it
>> shouldn't be used as the default because it 'always' requires storage
>> of unvalidated transactions.
>>
>> ''No Package Information Round:'' Instead of having a package
>> information round, just use the child's wtxid to refer to the package
>> and always send the entire package together. This would cause nodes to
>> redownload duplicate transactions.
>>
>> I have also created a slidedeck exploring various alternative designs
>> and some examples in which they fall flat [20]. Please feel free to
>> suggest other alternatives.
>>
>> ====Versioning System====
>>
>> This protocol should be extensible to support multiple types of
>> packages based on future desired use cases. Two "flavors" of
>> versioning were considered:
>>
>> 1. When package mempool acceptance is upgraded to support more types
>> of packages, increment the version number (similar to Erlay).
>> During version handshake, peers negotiate which version of package
>> relay they will use by each sending one "sendpackages" message.
>>
>> 2. When introducing another type of package, assign a version number
>> to it and announce it as an additional supported version (similar
>> to Compact Block Relay). During version handshake, peers send one
>> "sendpackages" message for each version supported.
>>
>> The second option was favored because it allows different parameters
>> for different versions. For example, it should be possible to support
>> both "arbitrary topology but maximum 3-transaction" package as well as
>> "child-with-unconfirmed-parents with default mempool ancestor limits"
>> packages simultaneously.
>>
>> ==Acknowledgements==
>>
>> I hope to have made it abundantly clear that this proposal isn’t
>> inventing the concept of package relay, and in fact builds upon years
>> of work by many others, including Suhas Daftuar and Antoine Riard.
>>
>> Thank you to John Newbery and Martin Zumsande for input on the design.
>>
>> Thank you to Matt Corallo, Christian Decker, David Harding, Antoine
>> Poinsot, Antoine Riard, Gregory Sanders, Chris Stewart, Bastien
>> Teinturier, and others for input on the desired interface for
>> contracting protocols.
>>
>> Looking forward to hearing your thoughts!
>>
>> Best,
>> Gloria
>>
>> [0]:
>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
>> [1]:
>> https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-April/002639.html
>> [2]:
>> https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-June/002758.html
>> [3]:
>> https://github.com/t-bast/lightning-docs/blob/master/pinning-attacks.md
>> [4]:
>> https://github.com/revault/practical-revault/blob/master/transactions.md#cancel_tx
>> [5]:
>> https://github.com/discreetlogcontracts/dlcspecs/blob/master/Transactions.md#refund-transaction
>> [6]: https://gist.github.com/instagibbs/60264606e181451e977e439a49f69fe1
>> [7]:
>> https://btctranscripts.com/adopting-bitcoin/2021/2021-11-16-gloria-zhao-transaction-relay-policy/#lightning-attacks
>> [8]: https://youtu.be/fbWSQvJjKFs?t=1438
>> [9]:
>> https://www.reddit.com/r/Bitcoin/comments/unew4e/looks_like_70_mvb_of_transactions_just_got_dumped/
>> [10]: https://github.com/bitcoin/bitcoin/pull/7594
>> [11]: https://github.com/bitcoin/bitcoin/pull/7600
>> [12]: https://github.com/bitcoin/bitcoin/pull/6455#issuecomment-122716820
>> [13]: https://gist.github.com/sdaftuar/8756699bfcad4d3806ba9f3396d4e66a
>> [14]: https://github.com/bitcoin/bitcoin/issues/14895
>> [15]: https://github.com/bitcoin/bitcoin/pull/16401
>> [16]: https://github.com/bitcoin/bitcoin/pull/19621
>> [17]:
>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html
>> [18]: https://github.com/users/glozow/projects/5/views/4?layout=board
>> [19]: https://gist.github.com/glozow/9b321cd3ef6505135c763112033ff2a7
>> [20]:
>> https://docs.google.com/presentation/d/1B__KlZO1VzxJGx-0DYChlWawaEmGJ9EGApEzrHqZpQc/edit?usp=sharing
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20221101/d77fa967/attachment-0001.html>

2023-06-07 23:15:55 UTC

- reply

📅 Original date posted:2022-10-27
📝 Original message:Hi AJ,
Not going to comment on what Bitcoin Core's philosophy on mempol policy is
or should be. I want to note that I think this:
> It's also possible that this is something of a one time thing: full rbf
> has been controversial for ages, but widely liked by devs, and other
> attempts (eg making it available in knots) haven't actually achieved
> much of a result in practice. So maybe this is just a special case
is true.
> The second thing is that whatever your relay policy is, you still
> need a path all the way to miners through nodes that will accept your
> transaction at every step. If you're making your mempool more restrictive
> (eg -permitbaremultisig=0, -datacarrier=0), that's easy for you (though
> you're making life more difficult for people who do create those sorts
> of txs); but if you want a more permissive policy (package relay,
> version-3-rbf, full-rbf), you might need to do some work.
> The cutoff for that is probably something like "do 30% of listening
> nodes have a compatible policy"? If they do, then you'll have about a
> 95% chance of having at least one of your outbound peers accept your tx,
> just by random chance.
Yes, in most cases, whether Bitcoin Core is restricting or loosening
policy, the user in question is fine as long as they have a path from their
node to a miner that will accept it. This is the case for something like
-datacarriersize if the use case is putting stuff into OP_RETURN outputs,
or if they're LN and using CPFP carveout, v3, package relay, etc. But
replacement is not only a question of "will my transaction propagate" but
also, "will someone else's transaction propagate, invalidating mine" or, in
other words, "can I prevent someone else's transaction from propagating." A
zeroconf user relies on there *not* being a path from someone else's full
RBF node to a full RBF miner. This is why I think RBF is so controversial
in general, why -mempoolfullrbf on someone else's node is considered more
significant than another policy option, and why full RBF shouldn't be
compared with something like datacarriersize. I don't think past patterns
can be easily applied here, and I don't think this necessarily shows a
different "direction" in thinking about mempool policy in general.
Best,
Gloria
On Thu, Oct 27, 2022 at 12:52 AM Anthony Towns via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Hi *,
>
> TLDR: Yes, this post is too long, and there's no TLDR. If it's any
> consolation, it took longer to write than it does to read?
>
> On Tue, Jun 15, 2021 at 12:55:14PM -0400, Antoine Riard via bitcoin-dev
> wrote:
> > Subject: Re: [bitcoin-dev] Proposal: Full-RBF in Bitcoin Core 24.0
> > I'm writing to propose deprecation of opt-in RBF in favor of full-RBF
>
> > If there is ecosystem agreement on switching to full-RBF, but 0.24 sounds
> > too early, let's defer it to 0.25 or 0.26. I don't think Core has a
> > consistent deprecation process w.r.t to policy rules heavily relied-on by
> > Bitcoin users, if we do so let sets a precedent satisfying as many folks
> as
> > we can.
>
> One precedent that seems to be being set here, which to me seems fairly
> novel for bitcoin core, is that we're about to start supporting and
> encouraging nodes to have meaningfully different mempool policies. From
> what I've seen, the baseline expectation has always been that while
> certainly mempools can and will differ, policies will be largely the same:
>
> Firstly, there is no "the mempool". There is no global mempool. Rather
> each node maintains its own mempool and accepts and rejects transaction
> to that mempool using their own internal policies. Most nodes have
> the same policies, but due to different start times, relay delays,
> and other factors, not every node has the same mempool, although they
> may be very similar.
>
> -
> https://bitcoin.stackexchange.com/questions/98585/how-to-find-if-two-transactions-in-mempool-are-conflicting
>
> Up until now, the differences between node policies supported by different
> nodes running core have been quite small, with essentially the following
> options available:
>
> -minrelaytxfee, -maxmempool - changes the lowest fee rate you'll accept
>
> -mempoolexpiry - how long to keep txs in the mempool
>
> -datacarrier - reject txs creating OP_RETURN outputs
>
> -datacarriersize - maximum size of OP_RETURN data
>
> -permitbaremultisig - prevent relay of bare multisig
>
> -bytespersigop - changes how SIGOP accounting works for relay and
> mining prioritisation
>
> as well as these, marked as "debug only" options (only shown with
> -help-debug):
>
> -incrementalrelayfee - make it easier/harder to spam txs by only
> slightly bumping the fee; marked as a "debug only" option
>
> -dustrelayfee - make it easier/harder to create uneconomic utxos;
> marked as a "debug only" option
>
> -limit{descendant,ancestor}{count,size} - changes how large the
> transaction chains can be; marked as a "debug only" option
>
> and in theory, but not available on mainnet:
>
> -acceptnonstdtxn - relay/mine non standard transactions
>
> There's also the "prioritisetransaction" rpc, which can cause you to keep
> a low feerate transaction in your mempool longer than you might otherwise.
>
> I think that -minrelaytxfee, -maxmempool and -mempoolexpiry are the only
> ones of those options commonly set, and those only rarely result in any
> differences in the txs at the top of the mempool.
>
> There are also quite a few parameters that aren't even runtime
> configurable:
>
> - MAX_STANDARD_TX_WEIGHT
> - MIN_STANDARD_TX_NONWITNESS_SIZE (see also #26265)
> - MAX_P2SH_SIGOPS (see also #26348)
> - MAX_STANDARD_TX_SIGOPS_COST
> - MAX_STANDARD_P2WSH_STACK_ITEMS
> - MAX_STANDARD_P2WSH_STACK_ITEM_SIZE
> - MAX_STANDARD_TAPSCRIPT_STACK_ITEM_SIZE
> - MAX_STANDARD_P2WSH_SCRIPT_SIZE
> - MAX_STANDARD_SCRIPTSIG_SIZE
> - EXTRA_DESCENDANT_TX_SIZE_LIMIT
> - MAX_REPLACEMENT_CANDIDATES
>
> And other plausible options aren't configurable even at compile time
> -- eg, core doesn't implement BIP 125's inherited signalling rule so
> there's no way to enable it; core doesn't allow opting out of BIP 125
> rule 3 ratchet on absolute fee; core doesn't allow CPFP carveout with
> more than 1 ancestor; core doesn't allow opting out of LOW_S checks
> (even via -acceptnonstdtxn); etc.
>
> We also naturally have different mempool policies between different
> releases: eg, expansions of policy, such as allowing OP_RETURN or
> expanding it from 40 to 80 bytes or new soft forks where old nodes won't
> relay transactions that use the new; and also occassional restrictions
> in policy, such as the LOW_S requirement.
>
>
> While supporting and encouraging different mempool polices might be new
> for core, it's not new for knots: knots changes some of these defaults
> (-permitbaremultisig defaults to false, -datacarriersize is reduced to
> 42), allows the use of -acceptnonstdtxn on mainnet, and introduces new
> options including -spkreuse and -mempoolreplacement (giving the latter
> full rbf behaviour by default). Knots also includes a `-corepolicy`
> option to make it easy to get a configuration matching core's defaults.
>
>
> I think gmaxwell's take from Feb 2015 (in the context of how restrictive
> policy on OP_RETURN data should be) was a reasonable description for
> core's approach up until now:
>
> There is also a matter of driving competent design rather than lazy
> first thing that works. E.g. In stealth addresses the early proposals
> use highly inefficient single ECDH point per output instead of simply
> pooling them. Network behavior is one of the few bits of friction
> driving good technical design rather than "move fast, break things, and
> force everyone else onto my way of doing thing rather than discussing
> the design in public". No one wants to be an outright gatekeeper,
> but the network is a shared resource and it's perfectly reasonable
> node behavior to be stingy about the perpetual storage impact of the
> transactions they're willing to process, especially when it comes to
> neutral technical criteria like the amount of network irrelevant data
> stuffed in transactions.
>
> There is also a very clear pattern we've seen in the past where
> people take anything the system lets them do as strong evidence that
> they have a irrevocable right to use the system in that way, and that
> their only responsibility-- and if their usage harms the system it's
> the responsibility of the system to not permit it. [...
> ...] For mitigating these risks it's optimal if transactions
> seem as uniform and indistinguishable as reasonably possible.
>
> - https://github.com/bitcoin/bitcoin/pull/5286#issuecomment-72564175
>
> Perhaps see also sdaftuar in Nov 2015,
>
> To me the most important question is, is priority something that miners
> want to use?
>
> If a non-negligible amount of hashpower intends to use it in their
> transaction selection, then I think it makes sense for nodes to use it
> too, because it's generally helpful to have your mempool predict the
> UTXO as much as possible, and for nodes to be able to have reasonable
> fee and priority estimates (which won't happen unless they track the
> priority transactions somehow -- I'm presuming that miners run with
> much bigger mempools than regular nodes).
>
> If the answer is no, then that's fine and I don't see a reason to push
> in this direction. I sort of assumed there was enough hashpower mining
> with priority, since last time I checked estimatepriority was still
> giving meaningful results for low-ish blockheights, but I haven't done
> any kind of real analysis.
>
> - https://github.com/bitcoin/bitcoin/pull/6992#issuecomment-155969455
>
> or in June 2019,
>
> What this PR is proposing is to get rid of a command-line option that is
> (a) a footgun for users and (b) does not reflect what I believe to be
> the understanding most users have, which is that [X txs] are expected
> to propagate well on the network.
>
> ..
>
> I don't think this rises to the level that Luke is concerned about,
> namely a prelude to forcing a common relay policy on all nodes. In
> particular I do agree it makes sense that we offer some ways of
> customizing policy parameters (eg the mempool size, min relay fee,
> etc). Instead, I think the justification for this change is that we
> should not support behaviors we think are harmful to the ecosystem
> overall and have no legitimate use-case, and we should eliminate ways
> that users might inadvertently shoot themselves in the foot.
>
> - https://github.com/bitcoin/bitcoin/pull/16171#issuecomment-500393271
>
> (or see discussion in https://github.com/bitcoin/bitcoin/pull/7219)
>
> I don't mean to imply the above are saying "there's one way to do
> things and it's this way", or that the old way of doing things should
> necessarily be the way we keep doing things. Just that previously core
> has tended towards designing a single policy that works as well as it
> can for everyone and the ecosystem as a whole. (I'm also not saying that
> fullrbf can't work well for everyone or the ecosystem as a whole)
>
>
> By contrast, I think the most common response to pushback against the
> full rbf option has been along the lines of "it's just an option, we
> don't want to force people", eg:
>
> Blaming the default false -mempoolfullrbf option for a full RBF network
> would be holding Bitcoin Core developers responsible for the decisions
> of individual node operators and miners. I don't think having the
> option (again, default false) can directly cause a full RBF network,
> and likewise, I don't think removing this option removes the "risk"
> of a full RBF network.
> - glozow
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274949400
>
> NACK. This is a default false option.
> - achow101
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1274953204
>
> Erecting artificial barriers to prevent or make it difficult for users
> to do what they want to do, is not appropriate behaviour.
> - luke-jr
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1290721905
>
> I'm in general against removing options.
> - instagibbs
> https://github.com/bitcoin/bitcoin/pull/26287#issuecomment-1292030700
>
> I think this differs from what core has done in the past, in that
> previously we've tried to ensure a new policy is good for everyone (or as
> nearly as it can be), and then enabled it as soon as it's implemented.
> Any options that have been added have either been to control resource
> usage in ways that don't significantly effect tx propagation, to
> allow people to revert to the old behaviour when the new behaviour is
> controversial (eg the -mempoolreplacement=0 option from 0.12 to 0.18),
> and to make it easier to test/debug the implementation.
>
> Giving people a new relay behaviour they can opt-in to when we aren't
> confident enough to turn on by default doesn't match the approach I've
> seen core take in the past.
>
>
> If this is going to be an ongoing shift in how core sees relay/mempool
> policy, I think that's significant and worth paying attention to.
>
> I don't think it's necessary to have that shift to roll out full rbf.
> The other approach would be either:
>
> * set -mempoolfullrbf=true as the default for 24.0, and just have the
> command line param there in case people want to do a
> "UserRejectedMempoolPolicy" campaign to get everyone to opt-out
>
> * revert it for now because we don't think mainnet is ready for fullrbf
> yet, and introduce it as default true for 25.0 or 26.0 or 27.0 or
> to activate at some scheduled date in that timeframe (potentially
> backporting it to previous releases to help with adoption too,
> whatever). same effect as the previous option, just with a bit more
> advanced notice and time to prepare
>
> I don't think anyone's proposed the first (which I interpret as "most of
> us don't think mainnet is ready for fullrbf today"), but the comments
> above are all pushback by people arguing against (the first step of)
> the second approach, and they seem to be winning the day.
>
> It's also possible that this is something of a one time thing: full rbf
> has been controversial for ages, but widely liked by devs, and other
> attempts (eg making it available in knots) haven't actually achieved
> much of a result in practice. So maybe this is just a special case and
> not a precedent, and when people propose other default false options,
> there will be substantially more resistance to them being merged,
> despite all the talk about users having options that's going on right now.
>
>
> Assuming it is the change of direction it appears to be -- and all of
> the above is really just justification for that assumption -- then like
> I said, I think it's worth seriously considering what it means for people
> to choose their own relay/mempool policies and for you to expect to have
> different mempool policies to many or most of your potential peers.
>
>
> One thing maybe worth noting is that is that you can still only choose
> your policy from options that people write code for -- if it wasn't
> something you could get by running knots or compiling a rejected PR
> yourself, it won't magically become more possible now. Presumably it
> would mean that once a PR is written, it might get better review (rather
> than being dismissed as not suitable for everyone), and there would be
> less maintenance burden than if it had to be manually rebased every
> release, though (or at least the maintenance burden would be shared
> across everyone working on the codebase).
>
>
> The second thing is that whatever your relay policy is, you still
> need a path all the way to miners through nodes that will accept your
> transaction at every step. If you're making your mempool more restrictive
> (eg -permitbaremultisig=0, -datacarrier=0), that's easy for you (though
> you're making life more difficult for people who do create those sorts
> of txs); but if you want a more permissive policy (package relay,
> version-3-rbf, full-rbf), you might need to do some work.
>
> The cutoff for that is probably something like "do 30% of listening
> nodes have a compatible policy"? If they do, then you'll have about a
> 95% chance of having at least one of your outbound peers accept your tx,
> just by random chance. If erlay allows increasing your outbound count to
> 12 connections instead of 8; that might reduce down to needing just 20%
> of listening nodes (~93%).
>
> But for cases where less than 30% (20%) of network supports your preferred
> policy, you probably need to do something cleverer.
>
> One approach is to set a service bit and preferentially peer with other
> nodes that advertise that service bit; knots does the first half of this
> for fullrbf, and both halves have been proposed for core in #25600.
> Preferential peering was previously done for the segwit deployment,
> though in that case it was necessary not just for tx propogation but
> also for ensuring block propogation, making it effectively a consensus
> critical issue.
>
> Another approach is having a separate relay network -- eg, lightning nodes
> already have a gossip network, and might want to help their own ecosystem
> by ensuring unilateral channel closes and justice transactions are quickly
> relayed. Using their own gossip network to relay the transaction around,
> and each lightning node adding it to their local bitcoind's mempool and
> allowing it to propogate (or not) from there as normal, would also be a
> way of allowing transactions to propogate well. It does mean that miners
> would either need to also participate in lightning gossip directly, or
> that miners would need to connect to *many* peers to be confident of
> seeing those transactions (eg, if only 2% of the network would see a
> tx, you'd need to make 228 connections to have a 99% chance of seeing
> the tx). You can't currently do something like this, because all the
> relay policies are also applied when adding txs to the mempool via RPC,
> and there's no convenient way to remove txs from the mempool.
>
> A case where something like that might occur is in preventing L2
> transactions from pinning attacks -- so you might have a high-fee,
> low-feerate transaction that's been widely propogated, sitting in the
> bottom of people's mempools, and you want to replace it with a smaller,
> higher-feerate transaction, but don't want to pay a higher absolute fee,
> and are thus blocked by BIP 125 rule 3. Perhaps 98% of the network is
> unwilling to deviate from BIP 125 rule 3 for you; because that would
> make it easy for random griefers to spam their mempool with large txs
> then delete them while only paying a small fee; but your L2 peers may be
> able to decode your replacement transaction and be sure that you aren't
> going to spam them, and thus will happily relay it.
>
> From a technical point-of-view, that's largely fine; the downside is it
> increases the centralisation pressure on mining: whether that's by having
> to connect to substantially more nodes, or having to parse through more
> spam, you can't just run your mining operation off a standard install
> of bitcoin core anymore, but need to actively opt-in to find all the
> weird unusual ways people are sending transactions around in order to
> actually collect as much in fees as your competitors are.
>
> That's probably moderately bad for privacy as well -- if lightning or
> coinjoins need special relay rules that most nodes haven't opted into,
> it's potentially easy to use that to find the bitcoin nodes on the
> network that are participating in those protocols, and from there to
> either identify the operator, or run a DoS attack to make it hard for you
> to keep doing what you want. Obviously if you're setting a service bit to
> get better routing, you've given up that privacy already. Likewise if the
> government or random vandals are opposed to bitcoin mining, and miners
> have to have special configuration on their nodes that distinguish them
> from regular users, then perhaps that makes it easier to find or shut
> down their operations.
>
> There are a few efficiencies to be gained from similar mempool policies as
> well: more reliable compact block reconstruction (if you're not missing
> any transactions, you avoid a round-trip) and presumably more efficient
> set reconstruction with erlay. You'll also waste less bandwidth sending
> transactions that the other node is only going to reject. Both those
> depend on how many transactions are going to rely on unusual mempool
> policies in the first place though.
>
> ariard wrote:
>
> I know I've advocated in the past to turn RBF support by default in
> the past. Though after gathering a lot of feedbacks, this approach
> of offering the policy flexiblity to the interested users only and
> favoring a full-rbf gradual deployment sounds better to me.
>
> - https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1157137026
>
> I guess all the above leads me to think that gradual deployments of
> mempool policies are likely the worse approach: even when they're not
> hurting anyone, it makes them hard to use during the gradual phase,
> and getting around that comes with worrying compromises on privacy and
> centralisation; and when they are problematic for some, the indeterminate
> nature of a gradual deployment means it's hard to plan for when that
> risk is going to eventuate.
>
>
> Theoretically, one way to recover the good parts of core deciding on
> what's good for the network might be for people outside of core to
> recommend a mempool configuration; then core can just have an option
> to make that easy, similar to "-std=c++17" for a C++ compiler, and much
> the same as knots' "-corepolicy" option.
>
> Presuming anyone actually wants to take on that job, and listen to the
> concerns of zeroconf businesses, lightning and coinjoin devs, miners, etc;
> and can come up with something that keeps most of them happy, and that
> 70% or 90% of the network ends up just following those recommendations
> because it's easy, it works, and it's recommended by all the apps they
> want to use, then that could work great:
>
> * miners don't need to do anything special, so there's no new
> mining centralisation pressure
> * miners and users don't reveal what they're doing with bitcoin by the way
> they configure their nodes, so there's no privacy problems
> * devs can be fairly confident in how they have to design their apps
> in order to get their transactions to most hashpower
> * devs don't have to add new p2p layers to make it happen
> * at least there's someone to talk to when you're trying to figure out
> how to make some new project possible when it's inhibited by current
> relay policies and you don't have to try to convince everyone to
> upgrade on your own
> * core devs just provide options, and don't have to worry about being
> seen as gatekeepers
>
> The "downside" in that scenario is that users/dev aren't making much
> actual use of all the choices core is offering by making different
> options available; but the upside is that that choice is at least readily
> available should whoever is coming up with these policy become out of
> step with what people actually want.
>
> One thing that might make an approach like that difficult is that core
> has historically been happy to remove options that don't seem useful
> anymore: eg the ability to turn of BIP 125 support (#16171), and priority
> transactions (#9602). Perhaps that's fine if you're trying to actively
> craft a single mempool/relay policy that's good enough for almost everyone
> (after all, it makes the code simpler and more efficient, and reduces
> the number of footguns); all you're doing is leaving a minority of people
> who want weird things to run a fork, and that's going to happen anyway.
>
> But if people are following policy developed outside of core, core
> might well disagree with them and decide "no that's a stupid policy,
> no one should do that" and remove some feature that others thing should
> continue to be normal. Beyond the examples above, there's already talk of
> removing the ability to disable fullrbf support in #26305, for instance.
> If that happens, then the people maintaining the policy will instead
> end up maintaining an entire fork of bitcoin core, and all we've done
> is transition to people running software from a different repo, and a
> different set of maintainers.
>
> If we're really going to a world where core's eager to add new options,
> and reluctant to remove them, at least if anyone at all finds them
> interesting, that's presumably a non-issue, though.
>
> Cheers,
> aj
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20221027/1d0aaf6c/attachment-0001.html>

2023-06-07 23:13:47 UTC

- reply

📅 Original date posted:2022-09-26
📝 Original message:Hi Greg, Antoine, Bastien,
Thanks very much for the feedback! I interpret most of the discussion
around limitations as ideas for future improvements rather than criticisms
of the proposal (please correct me if I'm wrong). I'll try to respond to as
much as possible.
Also I realize that I didn't contextualize this proposal clearly enough; it
is very tailored for LN Penalty and definitely doesn't close all pinning
attacks possible (sorry for confusing anyone). I also agree that some bits
can be a little ugly or tack-on; I would definitely prefer a comprehensive
RBF revamp to fix all our problems and enable other fee-bumping strategies
such as
sign-ANYONECANPAY-then-bring-your-own-fees-by-adding-inputs-at-broadcast. I
was hoping to get some ideas with the "RBF Improvements" post in January,
but it doesn't seem like we're much closer to a workable proposal. I think
this is a minimally-invasive step that works for Lightning today, a small
fix similar to CPFP carve out.
> As you likely know from previous discussions the biggest scenario this
does not fix in my estimation is ANYONECANPAY situations. If the parent
transaction can be "inflated" by tacking on additional inputs, this means
the total weight of the parent tx lowers the effective feerate of the
package.
(For more context to other readers I wrote an explanation for this in
"SIGHASH_ANYONECANPAY Pinning" section of RBF ML post). Yes, this
unfortunately doesn't fix any of the existing pinning attacks for single
transaction RBF but also doesn't make them worse. This boils down to adding
an incentive compatibility rule that ensures you can't replace a
transaction with something that will confirm slower. Package RBF has an
ancestor feerate-based rule for this (note it is quite conservative and not
perfect).
So in the scenario above with the "inflated" parent that was signed ACP,
the replacement would be rejected because the package ancestor feerate is
lower than the feerate of what is being replaced. But it is imperfect
(explained below) and thus I wouldn't recommend it for single transaction
replacement. So that attack still exists for single transactions, yes.
The strategy of using ACP to bring-your-own-fees has its own challenges but
hopefully has no current use cases as you say. AFAIK LN Penalty is not
affected by this since it doesn't use ACP, though obviously I agree we
should fix it for the future.
So when I said "this is intended for fee-bumping presigned txns in
contracting protocols," I should have said "this is intended for
fee-bumping presigned txns specifically using CPFP and anchor outputs."
Apologies for forgetting to contextualize, I've been sitting on this for
too long.
> The other scenario it doesn't really fix is where HTLC/commitment-like
transactions are being resolved in a batch, but due to relative time
constraints, you may want to accelerate some and not others. Now you must
pay higher rates to replace all of the transaction bumps. This is a
"self-pin" and "get good at utxos noob" type problem, but it's something
that axing rule#3 in favor of a Replace-by-ancestor-feerate system would
get us.
I understand you to mean "if you don't have enough UTXOs and you're forced
to batch-bump, you over-pay because you need to bring them all to the
highest target feerate." Isn't this kind of separate, wallet-related
problem? Contracting or not, surely every wallet needs to have enough UTXOs
to not batch transactions that shouldn't be batched... I don't see how a
replace-by-ancestor-feerate policy would make any difference for this?
Also in general I'd like to reiterate that ancestor feerate is not a
panacea to all our RBF incentive compatibility concerns. Like individual
feerate, unless we run the mining algorithm, it cannot tell us exactly how
quickly this transaction would be mined.
We're estimating the incentive compatibility of the original transaction(s)
and replacement transaction(s), with the goal of not letting a transaction
replace something that would have been more incentive compatible to mine.
As such, we don't want to overestimate how good the replacement is, and we
don't want to underestimate how good the original transactions are. This
rule "The minimum between package feerate and ancestor feerate of the child
is not lower than the individual feerates of all directly conflicting
transactions and the ancestor feerates of all original transactions" is a
conservative estimate.
> Would kind of be nice if package RBF would detect a "sibling output
spend" conflict, and knock it out of the mempool via the other replacement
rules? Getting rid of the requirement to 1 block csv lock every output
would be quite nice from a smart contracting composability point of view.
Interesting, so when a transaction hits a mempool tx's descendant limit, we
consider evicting one of its descendants in favor of this transaction,
based on the RBF rules.
Cool idea! After chewing on this for a bit, I think this *also* just boils
down to the fact that RBF should require replacements to be better mining
candidates. As in, if we added this policy and it can make us evict the
sibling and accept a transaction with a bunch of low-feerate ancestor junk,
it would be a new pinning vector.
> If you're a miner and you receive a non-V3, second descendant of an
unconfirmed V3 transaction, if the offered fee is in the top mempool
backlog, I think you would have an interest to accept such a transaction.
> So I'm not sure if those two rules are compatible with miners
incentives...
The same argument can be made for the 26th descendant of a mempool
transaction; it's also not entirely incentive-compatible to reject it, but
that is not the *only* design goal in mempool policy. Of course, the
difference here is that the 25-descendant limit rule is a sensible DoS
protection, while this 1-descendant limit rule is more of a "help the
Bitcoin ecosystem" policy, just like CPFP carve-out, dust limit, etc. I can
of course understand why not everyone would be in favor of this, but I do
think it's worth it.
> > 4. A V3 transaction that has an unconfirmed V3 ancestor cannot be
> > larger than 1000 virtual bytes.
> If I understand correctly the 1000 vb upper bound rational, it would be
to constraint the pinning counterparty to attach a high fee to a child due
to the limited size, if they would like this transaction to be stuck in the
network mempools. By doing so this child has high odds to confirm.
Yeah exactly, the "Rule 3 pin" is done by adding a child that's high-fee
(so you have to pay that much to evict it). Because they *don't* want this
tx to confirm, normally, this child would be really large. If they only
have 1000vB for the child, they can't increase the replacement cost without
also fee-bumping the transaction to make it confirm faster.
> As of today, I think yes you can already fingerprint LN transactions on
the spec-defined amount value of the anchor outputs, 330 sats. There is
always one of them on post-anchor commitment transactions. And sadly I
would say we'll always have tricky fingerprints leaking from unilateral LN
closures such as HTLC/PTLC timelocks...
> I agree with you, this isn't worse than today, unilateral closes will
probably always be identifiable on-chain.
Great to hear that there is no privacy worsening!
Best,
Gloria
On Mon, Sep 26, 2022 at 5:02 PM Greg Sanders <gsanders87 at gmail.com> wrote:
> Bastien,
>
> > This may be already covered by the current package RBF logic, in that
> scenario we are simply replacing [ParentTx, ChildTx1] with
> [ParentTx, ChildTx2] that pays more fees, right?
>
> For clarification, package RBF is ParentTx*s*(plural), and
> ChildTx(singular), so it might be a bit more complicated than we're
> thinking, and currently the V3 proposal would first de-duplicate the
> ParentTx based on what is in the mempool, then look at the "rest" of the
> transactions as a package, then individually. Not the same, not sure how
> different. I'll defer to experts.
>
> Best,
> Greg
>
> On Mon, Sep 26, 2022 at 11:48 AM Bastien TEINTURIER via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>> Thanks Gloria for this great post.
>>
>> This is very valuable work for L2 contracts, and will greatly improve
>> their security model.
>>
>> > "Only 1 anchor output? What if I need to bump counterparty's commitment
>> tx in mempool?"
>> > You won't need to fee-bump a counterparty's commitment tx using CPFP.
>> > You would just package RBF it by attaching a high-feerate child to
>> > your commitment tx.
>>
>> Note that we can also very easily make that single anchor spendable by
>> both participants (or even anyone), so if you see your counterparty's
>> commitment in your mempool, you can bump it without publishing your
>> own commitment, which is quite desirable (your own commitment tx has
>> CSV delays on your outputs, whereas your counterparty's commitment tx
>> doesn't).
>>
>> > "Is this a privacy issue, i.e. doesn't it allow fingerprinting LN
>> transactions based on nVersion?"
>>
>> I agree with you, this isn't worse than today, unilateral closes will
>> probably always be identifiable on-chain.
>>
>> > Would kind of be nice if package RBF would detect a "sibling output
>> spend"
>> > conflict, and knock it out of the mempool via the other replacement
>> rules?
>> > Getting rid of the requirement to 1 block csv lock every output would be
>> > quite nice from a smart contracting composability point of view.
>>
>> +1, that would be very neat!
>>
>> This may be already covered by the current package RBF logic, in that
>> scenario we are simply replacing [ParentTx, ChildTx1] with
>> [ParentTx, ChildTx2] that pays more fees, right?
>>
>> > 1) I do think that we should seriously consider allowing OP_TRUE to
>> become
>> > a standard script type as part of this policy update. If pinning is
>> solved,
>> > then there's no reason to require all those extra bytes for "binding" an
>> > anchor to a specific wallet/user. We can save quite a few bytes by
>> having
>> > the input be empty of witness data.
>> > 2) If we allow for a single dust-value(0 on up) output which is
>> immediately
>> > spent by the package, anchors become even easier to to design. No value
>> has
>> > to be "sapped" from contract participants to make an anchor output.
>> There's
>> > more complications for this, such as making sure the parent transaction
>> is
>> > dropped if the child spend is dropped, but maybe it's worth the squeeze.
>>
>> I also think both of these could be quite useful. This would probably
>> always
>> be used in combination with a parent transaction that pays 0 fees, so the
>> 0-value output would always be spent in the same block.
>>
>> But this means we could end up with 0-value outputs in the utxo set, if
>> for
>> some reason the parent tx is CPFP-ed via another output than the 0-value
>> one,
>> which would be a utxo set bloat issue. But I'd argue that we're probably
>> already creating utxo set bloat with the 330 sat anchor outputs
>> (especially
>> since we use two of them, but only one is usually spent), so it would
>> probably be *better* than what we're doing today.
>>
>> Thanks,
>> Bastien
>>
>> Le lun. 26 sept. 2022 à 03:22, Antoine Riard via bitcoin-dev <
>> bitcoin-dev at lists.linuxfoundation.org> a écrit :
>>
>>> Hi Gloria,
>>>
>>> Thanks for the progress on package RBF, few early questions.
>>>
>>> > 2. Any descendant of an unconfirmed V3 transaction must also be V3.
>>>
>>> > 3. An unconfirmed V3 transaction cannot have more than 1 descendant.
>>>
>>> If you're a miner and you receive a non-V3, second descendant of an
>>> unconfirmed V3 transaction, if the offered fee is in the top mempool
>>> backlog, I think you would have an interest to accept such a transaction.
>>>
>>> So I'm not sure if those two rules are compatible with miners
>>> incentives...
>>>
>>> > 4. A V3 transaction that has an unconfirmed V3 ancestor cannot be
>>> > larger than 1000 virtual bytes.
>>>
>>> If I understand correctly the 1000 vb upper bound rational, it would be
>>> to constraint the pinning counterparty to attach a high fee to a child due
>>> to the limited size, if they would like this transaction to be stuck in the
>>> network mempools. By doing so this child has high odds to confirm.
>>>
>>> I still wonder if this compatible with miner incentives in period of
>>> empty mempools, in the sense that if you've already a V3 transaction of
>>> size 100Kvb offering 2 sat/vb, it's more interesting than a V3 replacement
>>> candidate of size 1000 vb offering 10 sat/vb. It could be argued the former
>>> should be conserved.
>>>
>>> (That said, the hard thing with any replacement strategy we might evict
>>> a parent transaction *now* to which is attached a high-feerate child
>>> *latter* making for a utxo considered the best ancestor set. Maybe in the
>>> long-term miners should keep every transaction ever accepted...)
>>>
>>> > (Lower bound) the smaller this limit, the fewer UTXOs a child may use
>>> > to fund this fee-bump. For example, only allowing the V3 child to have
>>> > 2 inputs would require L2 protocols to manage a wallet with high-value
>>> > UTXOs and make batched fee-bumping impossible. However, as the
>>> > fee-bumping child only needs to fund fees (as opposed to payments),
>>> > just a few UTXOs should suffice.
>>>
>>> Reminder for L2 devs, batched fee-bumping of time-sensitive
>>> confirmations of commitment transactions is unsafe, as the counterparty
>>> could enter in a "cat-and-mouse" game to replace one of the batch element
>>> at each block to delay confirmation of the remaining elements in the batch,
>>> I think.
>>>
>>> On the other hand, I wonder if we wouldn't want a higher bound. LN
>>> wallets are likely to have one big UTXO in their fee-bumping reserve pool,
>>> as the cost of acquiring UTXO is non-null and in the optimistic case, you
>>> don't need to do unilateral closure. Let's say you close dozens of channels
>>> at the same time, a UTXO pool management strategy might be to fan-out the
>>> first spends UTXOs in N fan-out outputs ready to feed the remaining
>>> in-flight channels.
>>>
>>> > 1. The rule around unconfirmed inputs was
>>> > originally "A package may include new unconfirmed inputs, but the
>>> > ancestor feerate of the child must be at least as high as the ancestor
>>> > feerates of every transaction being replaced."
>>>
>>> Note, I think we would like this new RBF rule to also apply to single
>>> transaction package, e.g second-stage HTLC transactions, where a
>>> counterparty pins a HTLC-preimage by abusing rule 3. In that case, the
>>> honest LN node should be able to broadcast a "at least as high ancestor
>>> feerate" HTLC-timeout transaction. With `option_anchor_outputs" there is no
>>> unconfirmed ancestor to replace, as the commitment transaction, whatever
>>> the party it is originating from, should already be confirmed.
>>>
>>> > "Is this a privacy issue, i.e. doesn't it allow fingerprinting LN
>>> transactions based on nVersion?"
>>>
>>> As of today, I think yes you can already fingerprint LN transactions on
>>> the spec-defined amount value of the anchor outputs, 330 sats. There is
>>> always one of them on post-anchor commitment transactions. And sadly I
>>> would say we'll always have tricky fingerprints leaking from unilateral LN
>>> closures such as HTLC/PTLC timelocks...
>>>
>>> > "Can a V2 transaction replace a V3 transaction and vice versa?"
>>>
>>> IIUC, a V3 package could replace a V2 package, with the benefit of the
>>> new package RBF rules applied. I think this would be a significant
>>> advantage for LN, as for the current ~85k of opened channels, the old V2
>>> states shouldn't be pinning vectors. Currently, commitment transactions
>>> signal replaceability.
>>>
>>> Le ven. 23 sept. 2022 à 11:26, Gloria Zhao via bitcoin-dev <
>>> bitcoin-dev at lists.linuxfoundation.org> a écrit :
>>>
>>>> Hi everyone,
>>>>
>>>> I'm writing to propose a very simple set of mempool/transaction relay
>>>> policies intended to aid L2/contract protocols. I realized that
>>>> the previously proposed Package Mempool Accept package RBF [1]
>>>> had a few remaining problems after digging into the RBF logic more [2].
>>>> This additional set of policies solves them without requiring a huge
>>>> RBF overhaul.
>>>>
>>>> I've written an implementation (and docs) for Bitcoin Core:
>>>> https://github.com/bitcoin/bitcoin/pull/25038
>>>>
>>>> (You may notice that this proposal incorporates feedback on the PR -
>>>> thanks Suhas Daftuar, Gregory Sanders, Bastien Teinturier, Anthony Towns,
>>>> and others.)
>>>>
>>>> If you are interested in using package RBF/relay to bump presigned
>>>> transactions, I think you may be interested in reviewing this proposal.
>>>> This should solve Rule 3 pinning and perhaps allow us
>>>> to get rid of CPFP carve-out (yay!). I'm keen to hear if people find
>>>> the 1-anchor-output, 1000vB child limit too restrictive. Also, if you
>>>> find a
>>>> pinning attack or something that makes it unusable for you, I would
>>>> really really like to know.
>>>>
>>>> Note that transactions with nVersion=3 ("V3 transactions") are
>>>> currently non-standard in Bitcoin Core. That means **anything that was
>>>> standard before this policy change would still be standard
>>>> afterwards.** If you don't want your transactions to be subject to
>>>> these rules, just continue whatever you're doing and don't use
>>>> nVersion=3. AFAICT this shouldn't break anything, but let me know if
>>>> this would be disruptive for you?
>>>>
>>>> **New Policies:**
>>>>
>>>> This includes:
>>>> - a set of additional policy rules applying to V3 transactions
>>>> - modifications to package RBF rules
>>>>
>>>> **V3 transactions:**
>>>>
>>>> Existing standardness rules apply to V3 (e.g. min/max tx weight,
>>>> standard output types, cleanstack, etc.). The following additional
>>>> rules apply to V3:
>>>>
>>>> 1. A V3 transaction can be replaced, even if it does not signal BIP125
>>>> replaceability. (It must also meet the other RBF rules around fees,
>>>> etc. for replacement to happen).
>>>>
>>>> 2. Any descendant of an unconfirmed V3 transaction must also be V3.
>>>>
>>>> *Rationale*: Combined with Rule 1, this gives us the property of
>>>> "inherited" replaceability signaling when descendants of unconfirmed
>>>> transactions are created. Additionally, checking whether a transaction
>>>> signals replaceability this way does not require mempool traversal,
>>>> and does not change based on what transactions are mined. It also
>>>> makes subsequent rules about descendant limits much easier to check.
>>>>
>>>> *Note*: The descendant of a *confirmed* V3 transaction does not need to
>>>> be V3.
>>>>
>>>> 3. An unconfirmed V3 transaction cannot have more than 1 descendant.
>>>>
>>>> *Rationale*: (Upper bound) the larger the descendant limit, the more
>>>> transactions may need to be replaced. This is a problematic pinning
>>>> attack, i.e., a malicious counterparty prevents the transaction from
>>>> being replaced by adding many descendant transactions that aren't
>>>> fee-bumping.
>>>>
>>>> (Lower bound) at least 1 descendant is required to allow CPFP of the
>>>> presigned transaction. The contract protocol can create presigned
>>>> transactions paying 0 fees and 1 output for attaching a CPFP at
>>>> broadcast time ("anchor output"). Without package RBF, multiple anchor
>>>> outputs would be required to allow each counterparty to fee-bump any
>>>> presigned transaction. With package RBF, since the presigned
>>>> transactions can replace each other, 1 anchor output is sufficient.
>>>>
>>>> 4. A V3 transaction that has an unconfirmed V3 ancestor cannot be
>>>> larger than 1000 virtual bytes.
>>>>
>>>> *Rationale*: (Upper bound) the larger the descendant size limit, the
>>>> more vbytes may need to be replaced. With default limits, if the child
>>>> is e.g. 100,000vB, that might be an additional 100,000sats (at
>>>> 1sat/vbyte) or more, depending on the feerate.
>>>>
>>>> (Lower bound) the smaller this limit, the fewer UTXOs a child may use
>>>> to fund this fee-bump. For example, only allowing the V3 child to have
>>>> 2 inputs would require L2 protocols to manage a wallet with high-value
>>>> UTXOs and make batched fee-bumping impossible. However, as the
>>>> fee-bumping child only needs to fund fees (as opposed to payments),
>>>> just a few UTXOs should suffice.
>>>>
>>>> With a limit of 1000 virtual bytes, depending on the output types, the
>>>> child can have 6-15 UTXOs, which should be enough to fund a fee-bump
>>>> without requiring a carefully-managed UTXO pool. With 1000 virtual
>>>> bytes as the descendant limit, the cost to replace a V3 transaction
>>>> has much lower variance.
>>>>
>>>> *Rationale*: This makes the rule very easily "tacked on" to existing
>>>> logic for policy and wallets. A transaction may be up to 100KvB on its
>>>> own (`MAX_STANDARD_TX_WEIGHT`) and 101KvB with descendants
>>>> (`DEFAULT_DESCENDANT_SIZE_LIMIT_KVB`). If an existing V3 transaction
>>>> in the mempool is 100KvB, its descendant can only be 1000vB, even if
>>>> the policy is 10KvB.
>>>>
>>>> **Package RBF modifications:**
>>>>
>>>> 1. The rule around unconfirmed inputs was
>>>> originally "A package may include new unconfirmed inputs, but the
>>>> ancestor feerate of the child must be at least as high as the ancestor
>>>> feerates of every transaction being replaced."
>>>>
>>>> The package may still include new unconfirmed inputs. However,
>>>> the new rule is modified to be "The minimum between package feerate
>>>> and ancestor feerate of the child is not lower than the individual
>>>> feerates of all directly conflicting transactions and the ancestor
>>>> feerates of all original transactions."
>>>>
>>>> *Rationale*: We are attempting to ensure that the replacement
>>>> transactions are not less incentive-compatible to mine. However, a
>>>> package/transaction's ancestor feerate is not perfectly representative
>>>> of its incentive compatibility; it may overestimate (some subset of
>>>> the ancestors could be included by itself if it has other high-feerate
>>>> descendants or are themselves higher feerate than this
>>>> package/transaction). Instead, we use the minimum between the package
>>>> feerate and ancestor feerate of the child as a more conservative value
>>>> than what was proposed originally.
>>>>
>>>> 2. A new rule is added, requiring that all package transactions with
>>>> mempool conflicts to be V3. This also means the "sponsoring"
>>>> child transaction must be V3.
>>>>
>>>> *Note*: Combined with the V3 rules, this means the package must be
>>>> a child-with-parents package. Since package validation is only
>>>> attempted if the transactions do not pay sufficient fees to be
>>>> accepted on their own, this effectively means that only V3
>>>> transactions can pay to replace their ancestors' conflicts, and only
>>>> V3 transactions' replacements may be paid for by a descendant.
>>>>
>>>> *Rationale*: The fee-related rules are economically rational for
>>>> ancestor packages, but not necessarily other types of packages.
>>>> A child-with-parents package is a type of ancestor package. It
>>>> may be fine to allow any ancestor package, but it's more difficult
>>>> to account for all of the possibilities. For example, it gets much
>>>> harder to see that we're applying the descendant limits correctly if
>>>> the package has a gnarly, many-generation, non-tree shape. I'm also
>>>> not sure if this policy is 100% incentive-compatible if the sponsor
>>>> is not a direct descendant of the sponsee.
>>>>
>>>> Please see doc/policy/version3_transactions.md and
>>>> doc/policy/packages.md in the PR for the full set of rules.
>>>>
>>>> **Intended usage for LN:**
>>>>
>>>> Commitment transactions should be V3 and have 1 anchor output. They
>>>> can be signed with 0 fees (or 1sat/vbyte) once package relay is deployed
>>>> on a significant portion of the network. If the commitment tx must
>>>> be broadcast, determine the desired feerate at broadcast time and
>>>> spend the anchor output in a high feerate transaction. I'm going to
>>>> call the broadcasted commitment tx "the parent" and the attached
>>>> fee-bumping tx "the child."
>>>>
>>>> - This child must be V3.
>>>> - This child must be at most 1000vB. Note this restricts the
>>>> number of inputs you can use to fund the fee bump. Depending
>>>> on the output types, this is around 6-15.
>>>> - One child may fund fees for multiple commitment tx ("batched
>>>> fee-bumping").
>>>> - To do a second fee-bump to add more fees, replace the
>>>> *child* with a higher-feerate tx. Do not try to attach a grandchild.
>>>>
>>>> Otherwise, never try to spend from an unconfirmed V3 transaction. The
>>>> descendant limits for V3 transactions are very restrictive.
>>>>
>>>> **Expected Questions:**
>>>>
>>>> "Does this fix Rule 3 Pinning?"
>>>> Yes. The V3 descendant limit restricts both you and your counterparty.
>>>> Assuming nodes adopted this policy, you may reasonably assume that you
>>>> only need to replace the commitment transaction + up to 1000vB.
>>>>
>>>> "Only 1 anchor output? What if I need to bump counterparty's commitment
>>>> tx in mempool?"
>>>> You won't need to fee-bump a counterparty's commitment tx using CPFP.
>>>> You would just package RBF it by attaching a high-feerate child to
>>>> your commitment tx.
>>>>
>>>> "Is this a privacy issue, i.e. doesn't it allow fingerprinting LN
>>>> transactions based on nVersion?"
>>>> Indeed it may be unrealistic to assume V3 transactions will be in
>>>> widespread use outside of L2. IIUC, unilateral closes are already
>>>> obvious LN transactions because of the HTLC inputs. For e.g.
>>>> cooperative closes and opens, I think it makes sense to continue using
>>>> V2. So, unless I'm missing something, this shouldn't make it worse.
>>>>
>>>> "So a V3 transaction that doesn't signal BIP125 replaceability is
>>>> replaceable? Is that a backward compatibility issue?"
>>>> Yes it's replaceable. It's not an issue AFAICT because,
>>>> under previous policy, the V3 transaction wouldn't have been
>>>> in the mempool in the first place.
>>>>
>>>> "Can a V2 transaction replace a V3 transaction and vice versa?"
>>>> Yes, otherwise someone can use V3 transactions to censor V2
>>>> transactions spending shared inputs. Note if the
>>>> original V3 transaction has an unconfirmed V3 parent, this would
>>>> violate the "inherited V3" rule and would be rejected.
>>>>
>>>> Thanks for reading! Feedback and review would be much appreciated.
>>>>
>>>> [1]:
>>>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html
>>>> [2]:
>>>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
>>>>
>>>> Best,
>>>> Gloria
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev at lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev at lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220926/f053868f/attachment-0001.html>

2023-06-07 23:13:42 UTC

- reply

📅 Original date posted:2022-09-23
📝 Original message:Hi everyone,
I'm writing to propose a very simple set of mempool/transaction relay
policies intended to aid L2/contract protocols. I realized that
the previously proposed Package Mempool Accept package RBF [1]
had a few remaining problems after digging into the RBF logic more [2].
This additional set of policies solves them without requiring a huge RBF
overhaul.
I've written an implementation (and docs) for Bitcoin Core:
https://github.com/bitcoin/bitcoin/pull/25038
(You may notice that this proposal incorporates feedback on the PR - thanks
Suhas Daftuar, Gregory Sanders, Bastien Teinturier, Anthony Towns, and
others.)
If you are interested in using package RBF/relay to bump presigned
transactions, I think you may be interested in reviewing this proposal.
This should solve Rule 3 pinning and perhaps allow us
to get rid of CPFP carve-out (yay!). I'm keen to hear if people find
the 1-anchor-output, 1000vB child limit too restrictive. Also, if you find a
pinning attack or something that makes it unusable for you, I would
really really like to know.
Note that transactions with nVersion=3 ("V3 transactions") are
currently non-standard in Bitcoin Core. That means **anything that was
standard before this policy change would still be standard
afterwards.** If you don't want your transactions to be subject to
these rules, just continue whatever you're doing and don't use
nVersion=3. AFAICT this shouldn't break anything, but let me know if
this would be disruptive for you?
**New Policies:**
This includes:
- a set of additional policy rules applying to V3 transactions
- modifications to package RBF rules
**V3 transactions:**
Existing standardness rules apply to V3 (e.g. min/max tx weight,
standard output types, cleanstack, etc.). The following additional
rules apply to V3:
1. A V3 transaction can be replaced, even if it does not signal BIP125
replaceability. (It must also meet the other RBF rules around fees,
etc. for replacement to happen).
2. Any descendant of an unconfirmed V3 transaction must also be V3.
*Rationale*: Combined with Rule 1, this gives us the property of
"inherited" replaceability signaling when descendants of unconfirmed
transactions are created. Additionally, checking whether a transaction
signals replaceability this way does not require mempool traversal,
and does not change based on what transactions are mined. It also
makes subsequent rules about descendant limits much easier to check.
*Note*: The descendant of a *confirmed* V3 transaction does not need to be
V3.
3. An unconfirmed V3 transaction cannot have more than 1 descendant.
*Rationale*: (Upper bound) the larger the descendant limit, the more
transactions may need to be replaced. This is a problematic pinning
attack, i.e., a malicious counterparty prevents the transaction from
being replaced by adding many descendant transactions that aren't
fee-bumping.
(Lower bound) at least 1 descendant is required to allow CPFP of the
presigned transaction. The contract protocol can create presigned
transactions paying 0 fees and 1 output for attaching a CPFP at
broadcast time ("anchor output"). Without package RBF, multiple anchor
outputs would be required to allow each counterparty to fee-bump any
presigned transaction. With package RBF, since the presigned
transactions can replace each other, 1 anchor output is sufficient.
4. A V3 transaction that has an unconfirmed V3 ancestor cannot be
larger than 1000 virtual bytes.
*Rationale*: (Upper bound) the larger the descendant size limit, the
more vbytes may need to be replaced. With default limits, if the child
is e.g. 100,000vB, that might be an additional 100,000sats (at
1sat/vbyte) or more, depending on the feerate.
(Lower bound) the smaller this limit, the fewer UTXOs a child may use
to fund this fee-bump. For example, only allowing the V3 child to have
2 inputs would require L2 protocols to manage a wallet with high-value
UTXOs and make batched fee-bumping impossible. However, as the
fee-bumping child only needs to fund fees (as opposed to payments),
just a few UTXOs should suffice.
With a limit of 1000 virtual bytes, depending on the output types, the
child can have 6-15 UTXOs, which should be enough to fund a fee-bump
without requiring a carefully-managed UTXO pool. With 1000 virtual
bytes as the descendant limit, the cost to replace a V3 transaction
has much lower variance.
*Rationale*: This makes the rule very easily "tacked on" to existing
logic for policy and wallets. A transaction may be up to 100KvB on its
own (`MAX_STANDARD_TX_WEIGHT`) and 101KvB with descendants
(`DEFAULT_DESCENDANT_SIZE_LIMIT_KVB`). If an existing V3 transaction
in the mempool is 100KvB, its descendant can only be 1000vB, even if
the policy is 10KvB.
**Package RBF modifications:**
1. The rule around unconfirmed inputs was
originally "A package may include new unconfirmed inputs, but the
ancestor feerate of the child must be at least as high as the ancestor
feerates of every transaction being replaced."
The package may still include new unconfirmed inputs. However,
the new rule is modified to be "The minimum between package feerate
and ancestor feerate of the child is not lower than the individual
feerates of all directly conflicting transactions and the ancestor
feerates of all original transactions."
*Rationale*: We are attempting to ensure that the replacement
transactions are not less incentive-compatible to mine. However, a
package/transaction's ancestor feerate is not perfectly representative
of its incentive compatibility; it may overestimate (some subset of
the ancestors could be included by itself if it has other high-feerate
descendants or are themselves higher feerate than this
package/transaction). Instead, we use the minimum between the package
feerate and ancestor feerate of the child as a more conservative value
than what was proposed originally.
2. A new rule is added, requiring that all package transactions with
mempool conflicts to be V3. This also means the "sponsoring"
child transaction must be V3.
*Note*: Combined with the V3 rules, this means the package must be
a child-with-parents package. Since package validation is only
attempted if the transactions do not pay sufficient fees to be
accepted on their own, this effectively means that only V3
transactions can pay to replace their ancestors' conflicts, and only
V3 transactions' replacements may be paid for by a descendant.
*Rationale*: The fee-related rules are economically rational for
ancestor packages, but not necessarily other types of packages.
A child-with-parents package is a type of ancestor package. It
may be fine to allow any ancestor package, but it's more difficult
to account for all of the possibilities. For example, it gets much
harder to see that we're applying the descendant limits correctly if
the package has a gnarly, many-generation, non-tree shape. I'm also
not sure if this policy is 100% incentive-compatible if the sponsor
is not a direct descendant of the sponsee.
Please see doc/policy/version3_transactions.md and
doc/policy/packages.md in the PR for the full set of rules.
**Intended usage for LN:**
Commitment transactions should be V3 and have 1 anchor output. They
can be signed with 0 fees (or 1sat/vbyte) once package relay is deployed
on a significant portion of the network. If the commitment tx must
be broadcast, determine the desired feerate at broadcast time and
spend the anchor output in a high feerate transaction. I'm going to
call the broadcasted commitment tx "the parent" and the attached
fee-bumping tx "the child."
- This child must be V3.
- This child must be at most 1000vB. Note this restricts the
number of inputs you can use to fund the fee bump. Depending
on the output types, this is around 6-15.
- One child may fund fees for multiple commitment tx ("batched
fee-bumping").
- To do a second fee-bump to add more fees, replace the
*child* with a higher-feerate tx. Do not try to attach a grandchild.
Otherwise, never try to spend from an unconfirmed V3 transaction. The
descendant limits for V3 transactions are very restrictive.
**Expected Questions:**
"Does this fix Rule 3 Pinning?"
Yes. The V3 descendant limit restricts both you and your counterparty.
Assuming nodes adopted this policy, you may reasonably assume that you
only need to replace the commitment transaction + up to 1000vB.
"Only 1 anchor output? What if I need to bump counterparty's commitment tx
in mempool?"
You won't need to fee-bump a counterparty's commitment tx using CPFP.
You would just package RBF it by attaching a high-feerate child to
your commitment tx.
"Is this a privacy issue, i.e. doesn't it allow fingerprinting LN
transactions based on nVersion?"
Indeed it may be unrealistic to assume V3 transactions will be in
widespread use outside of L2. IIUC, unilateral closes are already
obvious LN transactions because of the HTLC inputs. For e.g.
cooperative closes and opens, I think it makes sense to continue using
V2. So, unless I'm missing something, this shouldn't make it worse.
"So a V3 transaction that doesn't signal BIP125 replaceability is
replaceable? Is that a backward compatibility issue?"
Yes it's replaceable. It's not an issue AFAICT because,
under previous policy, the V3 transaction wouldn't have been
in the mempool in the first place.
"Can a V2 transaction replace a V3 transaction and vice versa?"
Yes, otherwise someone can use V3 transactions to censor V2
transactions spending shared inputs. Note if the
original V3 transaction has an unconfirmed V3 parent, this would
violate the "inherited V3" rule and would be rejected.
Thanks for reading! Feedback and review would be much appreciated.
[1]:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html
[2]:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
Best,
Gloria
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220923/9b52c0c4/attachment.html>

2023-06-07 23:10:18 UTC

- reply

📅 Original date posted:2022-06-07
📝 Original message:Hi Eric, aj, all,
Sorry for the delayed response. @aj I'm including some paraphrased points
from our offline discussion (thanks).
> Other idea: what if you encode the parent txs as a short hash of the
wtxid (something like bip152 short ids? perhaps seeded per peer so
collisions will be different per peer?) and include that in the inv
announcement? Would that work to avoid a round trip almost all of the time,
while still giving you enough info to save bw by deduping parents?
> As I suggested earlier, a package is fundamentally a compact block (or
> block) announcement without the header. Compact block (BIP152)
announcement
> is already well-defined and widely implemented...
> Let us not reinvent the wheel and/or introduce accidental complexity. I
see
> no reason why packaging is not simply BIP152 without the 'header' field,
an
> updated protocol version, and the following sort of changes to names
Interestingly, "why not use BIP 152 shortids to save bandwidth?" is by far
the most common suggestion I hear (including offline feedback). Here's a
full explanation:
BIP 152 shortens transaction hashes (32 bytes) to shortids (6 bytes) to
save a significant amount of network bandwidth, which is extremely
important in block relay. However, this comes at the expense of
computational complexity. There is no way to directly calculate a
transaction hash from a shortid; upon receipt of a compact block, a node is
expected to calculate the shortids of every unconfirmed transaction it
knows about to find the matches (BIP 152: [1], Bitcoin Core: [2]). This is
expensive but appropriate for block relay, since the block must have a
valid Proof of Work and new blocks only come every ~10 minutes. On the
other hand, if we require nodes to calculate shortids for every transaction
in their mempools every time they receive a package, we are creating a DoS
vector. Unconfirmed transactions don't need PoW and, to have a live
transaction relay network, we should expect nodes to handle transactions at
a high-ish rate (i.e. at least 1000s of times more transactions than
blocks). We can't pre-calculate or cache shortids for mempool transactions,
since the SipHash key depends on the block hash and a per-connection salt.
Additionally, shortid calculation is not designed to prevent intentional
individual collisions. If we were to use these shortids to deduplicate
transactions we've supposedly already seen, we may have a censorship
vector. Again, these tradeoffs make sense for compact block relay (see
shortid section in BIP 152 [3]), but not package relay.
TLDR: DoSy if we calculate shortids on every package and censorship vector
if we use shortids for deduplication.
> Given this message there is no reason
> to send a (potentially bogus) fee rate with every package. It can only be
> validated by obtaining the full set of txs, and the only recourse is
> dropping (etc.) the peer, as is the case with single txs.
Yeah, I agree with this. Combined with the previous discussion with aj
(i.e. we can't accurately communicate the incentive compatibility of a
package without sending the full graph, and this whole dance is to avoid
downloading a few low-fee transactions in uncommon edge cases), I've
realized I should remove the fee + weight information from pkginfo. Yay for
less complexity!
Also, this might be pedantic, but I said something incorrect earlier and
would like to correct myself:
>> In theory, yes, but maybe it was announced earlier (while our node was
down?) or had dropped from our mempool or similar, either way we don't have
those txs yet.
I said "It's fine if they have Erlay, since a sender would know in advance
that B is missing and announce it as a package." But this isn't true since
we're only using reconciliation in place of flooding to announce
transactions as they arrive, not for rebroadcast, and we're not doing full
mempool set reconciliation. In any case, making sure a node receives the
transactions announced when it was offline is not something we guarantee,
not an intended use case for package relay, and not worsened by this.
Thanks for your feedback!
Best,
Gloria
[1]:
https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki#cmpctblock
[2]:
https://github.com/bitcoin/bitcoin/blob/master/src/blockencodings.cpp#L49
[3]:
https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki#short-transaction-id-calculation
On Thu, May 26, 2022 at 3:59 AM <eric at voskuil.org> wrote:
> Given that packages have no header, the package requires identity in a
> BIP152 scheme. For example 'header' and 'blockhash' fields can be replaced
> with a Merkle root (e.g. "identity" field) for the package, uniquely
> identifying the partially-ordered set of txs. And use of 'getdata' (to
> obtain a package by hash) can be eliminated (not a use case).
>
> e
>
> > -----Original Message-----
> > From: eric at voskuil.org <eric at voskuil.org>
> > Sent: Wednesday, May 25, 2022 1:52 PM
> > To: 'Anthony Towns' <aj at erisian.com.au>; 'Bitcoin Protocol Discussion'
> > <bitcoin-dev at lists.linuxfoundation.org>; 'Gloria Zhao'
> > <gloriajzhao at gmail.com>
> > Subject: RE: [bitcoin-dev] Package Relay Proposal
> >
> > > From: bitcoin-dev <bitcoin-dev-bounces at lists.linuxfoundation.org> On
> > Behalf
> > > Of Anthony Towns via bitcoin-dev
> > > Sent: Wednesday, May 25, 2022 11:56 AM
> >
> > > So the other thing is what happens if the peer announcing packages to
> us
> > is
> > > dishonest?
> > >
> > > They announce pkg X, say X has parents A B C and the fee rate is
> garbage.
> > But
> > > actually X has parent D and the fee rate is excellent. Do we request
> the
> > > package from another peer, or every peer, to double check? Otherwise
> > we're
> > > allowing the first peer we ask about a package to censor that tx from
> us?
> > >
> > > I think the fix for that is just to provide the fee and weight when
> > announcing
> > > the package rather than only being asked for its info? Then if one peer
> > makes
> > > it sound like a good deal you ask for the parent txids from them,
> dedupe,
> > > request, and verify they were honest about the parents.
> >
> > Single tx broadcasts do not carry an advertised fee rate, however the'
> > feefilter' message (BIP133) provides this distinction. This should be
> > interpreted as applicable to packages. Given this message there is no
> reason
> > to send a (potentially bogus) fee rate with every package. It can only be
> > validated by obtaining the full set of txs, and the only recourse is
> > dropping (etc.) the peer, as is the case with single txs. Relying on the
> > existing message is simpler, more consistent, and more efficient.
> >
> > > >> Is it plausible to add the graph in?
> > >
> > > Likewise, I think you'd have to have the graph info from many nodes if
> > you're
> > > going to make decisions based on it and don't want hostile peers to be
> > able to
> > > trick you into ignoring txs.
> > >
> > > Other idea: what if you encode the parent txs as a short hash of the
> wtxid
> > > (something like bip152 short ids? perhaps seeded per peer so collisions
> > will
> > > be different per peer?) and include that in the inv announcement? Would
> > > that work to avoid a round trip almost all of the time, while still
> giving
> > you
> > > enough info to save bw by deduping parents?
> >
> > As I suggested earlier, a package is fundamentally a compact block (or
> > block) announcement without the header. Compact block (BIP152)
> > announcement
> > is already well-defined and widely implemented. A node should never be
> > required to retain an orphan, and BIP152 ensures this is not required.
> >
> > Once a validated set of txs within the package has been obtained with
> > sufficient fee, a fee-optimal node would accept the largest subgraph of
> the
> > package that conforms to fee constraints and drop any peer that provides
> a
> > package for which the full graph does not.
> >
> > Let us not reinvent the wheel and/or introduce accidental complexity. I
> see
> > no reason why packaging is not simply BIP152 without the 'header' field,
> an
> > updated protocol version, and the following sort of changes to names:
> >
> > sendpkg
> > MSG_CMPCT_PKG
> > cmpctpkg
> > getpkgtxn
> > pkgtxn
> >
> > > > For a maximum 25 transactions,
> > > >23*24/2 = 276, seems like 36 bytes for a child-with-parents package.
> > >
> > > If you're doing short ids that's maybe 25*4B=100B already, then the
> above
> > is
> > > up to 36% overhead, I guess. Might be worth thinking more about, but
> > maybe
> > > more interesting with ancestors than just parents.
> > >
> > > >Also side note, since there are no size/count params,
> >
> > Size is restricted in the same manner as block and transaction
> broadcasts,
> > by consensus. If the fee rate is sufficient there would be no reason to
> > preclude any valid size up to what can be mined in one block (packaging
> > across blocks is not economically rational under the assumption that one
> > miner cannot expect to mine multiple blocks in a row). Count is
> incorporated
> > into BIP152 as 'shortids_length'.
> >
> > > > wondering if we
> > > >should just have "version" in "sendpackages" be a bit field instead of
> > > >sending a message for each version. 32 versions should be enough
> right?
> >
> > Adding versioning to individual protocols is just a reflection of the
> > insufficiency of the initial protocol versioning design, and that of the
> > various ad-hoc changes to it (including yet another approach in this
> > proposal) that have been introduced to compensate for it, though I'll
> > address this in an independent post at some point.
> >
> > Best,
> > e
> >
> > > Maybe but a couple of messages per connection doesn't really seem worth
> > > arguing about?
> > >
> > > Cheers,
> > > aj
> > >
> > >
> > > --
> > > Sent from my phone.
> > > _______________________________________________
> > > bitcoin-dev mailing list
> > > bitcoin-dev at lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220607/d32c46dd/attachment.html>

2023-06-07 23:09:49 UTC

- reply

📅 Original date posted:2022-05-24
📝 Original message:Hi aj,
> If you've got (A,B,C,X) where B spends A and X spends A,B,C where X+C is
below fee floor while A+B and A+B+C+X are above fee floor you have the
problem though.
To clarify, in this situation, I'm imagining something like
A: 0 sat, 100vB
B: 1500 sat, 100vB
C: 0 sat, 100vB
X: 500 sat, 100vB
feerate floor is 3sat/vB
With the algo:
> * is X alone above my fee rate? no, then forget it
> * otherwise, s := X.size, f := X.fees, R := [X]
> * for P = P1..Pn:
> * do I already have P? then skip to the next parent
> * s += P.size, f += P.fees, R += [P]
> * if f/s above my fee rate floor? if so, request all the txs in R
We'd erroneously ask for A+B+C+X, but really we should only take A+B.
But wouldn't A+B also be a package that was announced for B?
Please lmk if you were imagining something different. I think I may be
missing something.
> Is it plausible to add the graph in?
Fun to think about. Most basic design would be to represent {spends,
doesn’t spend} for a previous transaction in the package as a bit. Can
think of it as a matrix where row i, column j tells you whether Tx j
(directly) spends Tx i.
But of course you can omit the last row, since the child spends all of
them. And since topological ordering is a requirement, you only need as
many bits as there are transactions preceding this one in the package.
If you have up to 24 parents, you need 1 + 2 + ... + 23 bits to codify
spending for the 2nd ... 24th parent. For a maximum 25 transactions,
23*24/2 = 276, seems like 36 bytes for a child-with-parents package. A few
more for tx-with-ancestors.
Then you can split it up into sub-packages and everything. Still not sure
if we really need to.
Also side note, since there are no size/count params, wondering if we
should just have "version" in "sendpackages" be a bit field instead of
sending a message for each version. 32 versions should be enough right?
Best,
Gloria
On Tue, 24 May 2022 at 12:48 Anthony Towns <aj at erisian.com.au> wrote:
> On 23 May 2022 9:13:43 pm GMT-04:00, Gloria Zhao <gloriajzhao at gmail.com>
> wrote:
> >> If you're asking for the package for "D", would a response telling you:
> >> txid_D (500 sat, 100vB)
> >> txid_A (0 sat, 100vB)
> >> txid_B (2000 sat, 100 vB)
> >> be better, in that case? Then the receiver can maybe do the logic
> >> themselves to figure out that they already have A in their mempool
> >> so it's fine, or not?
> >Right, I also considered giving the fees and sizes of each transaction in
> >the package in “pckginfo1”. But I don’t think that information provides
> >additional meaning unless you know the exact topology, i.e. also know if
> >the parents have dependency relationships between them. For instance, in
> >the {A, B, D} package there, even if you have the information listed, your
> >decision should be different depending on whether B spends from A.
>
> I don't think that's true? We already know D is above our fee floor so if
> B with A is also above the floor, we want them all, but also if B isn't
> above the floor, but all of them combined are, then we also do?
>
> If you've got (A,B,C,X) where B spends A and X spends A,B,C where X+C is
> below fee floor while A+B and A+B+C+X are above fee floor you have the
> problem though.
>
> Is it plausible to add the graph in?
>
> Cheers,
> aj
>
>
>
> --
> Sent from my phone.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220524/056c9996/attachment-0001.html>

2023-06-07 23:09:47 UTC

- reply

📅 Original date posted:2022-05-23
📝 Original message:Hi aj,
> if you're writing a protocol that's
> dependent on people seeing that a package as a whole pays a competitive
> feerate, don't you want to know in advance what conditions the network
> is going to impose on your transactions in order to consider them as a
> package?
I do think unifying the size/count constraints would result in a more
stable/easier to reason about interface for L2 devs. Then the requirement
for propagation is just a path of nodes that support v1 package relay, and
it’s implied their mempool policy supports it as well. Also seems like it
could be a fingerprinting problem for nodes to give very specific
count/size limits.
> (… maybe core's defaults should be reconsidered rather than standardised
as-is)
> Worst case, you could presumably do a new package relay version with
> different constraints, if needed.
Maybe this was my actual concern. I think the defaults are safe but it’s
not like they’ve been proven to be optimal. This creates an obstacle to
changing them, especially if we want to make them smaller. But I think it’s
unlikely we’ll do that, and adding another version for new constraints
doesn’t seem too bad.
(Agreed with everything here, thanks for the feedback and clarifications!)
TLDR, making these changes:
- Count and size are implied by the version. Version 1 is specifically
child-with-unconfirmed-parents, where the whole package is at most 25
transactions and 101KvB.
- Announce sendpackages based on our own state. It’s ok to send
“sendpackages” if they sent fRelay=false.
- At verack, require fRelay=true and wtxidrelay if they sent sendpackages,
otherwise disconnect.
- If we get “getpckgtxns” or “pckgtxns” without having negotiated
“sendpackages” ahead of time, ignore, don’t disconnect. Emphasize that the
intention is to validate all of the transactions received through
“pckgtxns” together.
> If you're asking for the package for "D", would a response telling you:
> txid_D (500 sat, 100vB)
> txid_A (0 sat, 100vB)
> txid_B (2000 sat, 100 vB)
> be better, in that case? Then the receiver can maybe do the logic
> themselves to figure out that they already have A in their mempool
> so it's fine, or not?
Right, I also considered giving the fees and sizes of each transaction in
the package in “pckginfo1”. But I don’t think that information provides
additional meaning unless you know the exact topology, i.e. also know if
the parents have dependency relationships between them. For instance, in
the {A, B, D} package there, even if you have the information listed, your
decision should be different depending on whether B spends from A. The only
thing you know for sure about a child with direct parents is: if the
aggregate feerate is too low, you won’t want the child since it depends on
everyone else. If there’s a good-feerate transaction in there that doesn’t
have a dependency, you’re fine as long as someone sends it to you
individually.
Best,
Gloria
On Mon, May 23, 2022 at 2:34 PM Anthony Towns via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> On Wed, May 18, 2022 at 02:40:58PM -0400, Gloria Zhao via bitcoin-dev
> wrote:
> > > Does it make sense for these to be configurable, rather than implied
> > > by the version?
> > > … would it be better to either just not do sendpackages
> > > at all if you're limiting ancestors in the mempool incompatibly
> > Effectively: if you’re setting your ancestor/descendant limits lower than
> > the default, you can’t do package relay. I wonder if this might be
> > controversial, since it adds pressure to adhere to Bitcoin Core’s current
> > mempool policy? I would be happy to do it this way, though - makes things
> > easier to implement.
>
> How about looking at it the other way: if you're writing a protocol that's
> dependent on people seeing that a package as a whole pays a competitive
> feerate, don't you want to know in advance what conditions the network
> is going to impose on your transactions in order to consider them as a
> package? In that case, aren't the "depth" and "size" constraints things
> we should specify in a standard?
>
> (The above's not a rhetorical question; I'm not sure what the answer is.
> And even if it's "yes", maybe core's defaults should be reconsidered
> rather than standardised as-is)
>
> Worst case, you could presumably do a new package relay version with
> different constraints, if needed.
>
> > > > 5. If 'fRelay==false' in a peer's version message, the node must not
> > > > send "sendpackages" to them. If a "sendpackages" message is
> > > > received by a peer after sending `fRelay==false` in their version
> > > > message, the sender should be disconnected.
> > > Seems better to just say "if you set fRelay=false in your version
> > > message, you must not send sendpackages"? You already won't do packages
> > > with the peer if they don't also announce sendpackages.
> > I guess, theoretically, if you allow bloom filters with this peer, it’s
> > plausible they’re saying “fRelay=false, I’ll send you a bloom filter
> later,
> > and I’ll also want to talk about packages.”
>
> I was just meaning "it's okay to send VERSION fRelay=true then immediately
> send WTXIDRELAY then immediately send SENDPACKAGES" without having to
> first verify what the other guy's fRelay was set to. On the other hand,
> you do already have to verify the other guy's version is high enough,
> but it would be kind-of nice to move towards just announcing the features
> you support, and not having to make it a multistep negotiation...
>
> > > Maybe: "You must not send sendpackages unless you also send
> wtxidrelay" ?
> > Do you mean if we get a verack, and the peer sent “sendpackages” but not
> > “wtxidrelay,” we should disconnect them?
>
> Yes.
>
> > I have it as: we send a PCKG INV when this transaction’s feerate is above
> > the fee filter, but one or more of its parents don’t. I don’t think using
> > ancestor feerate is better.
> > See this counterexample:
> >
> https://raw.githubusercontent.com/glozow/bitcoin-notes/master/mempool_garden/abc_1parent_2kids.png
> > A (0fee) has 2 kids, B (3sat/vB) and C (20sat/vB), everything’s the same
> > vsize. Let’s say the fee filter is 3sat/vB.
> > If we do it based on ancestor feerate, we won’t send B. But B is actually
> > fine; C is paying for A.
>
> But that only works if the receiver also has C, in which case they also
> have A, and you don't need package relay to do anything with B? If they
> didn't have C already, then relaying {A,B} would be a waste of time,
> because {A,B} would be rejected as only paying 1.5sat/vB or whatever..
>
> If you switch it to being:
>
> A (0 sats, 200vB)
> B (2000 sats, 200vB, spends A:0)
> C (200 sats, 200vB)
> D (1000 sats, 200vB, sepnds A:1, C:0)
>
> then you get:
>
> A alone = 0s/vB
> B+A = 5s/vB
>
> C alone = 1s/vB
> D+C+A = 2s/vB
> D+C = 3s/vB (B+A already at 5s/vB)
>
> which I think recovers your point, while also having all the details
> only be dealing with direct parents.
>
> > > Are "getpckgtxns" / "pcktxns" really limited to packages, or are they
> > > just a general way to request a batch of transactions?
> > > Maybe call those messages "getbatchtxns" and "batchtxns" and allow
> them to
> > > be used more generally, potentially in ways unrelated to packages/cpfp?
> > Indeed, it’s a general way to request a batch of transactions. I’ll
> > highlight that it is “all or nothing,” i.e. if the sender is missing any
> of
> > them, they’ll just send a notfound.
> > The idea here was to avoid downloading any transactions that can’t be
> > validated right away.
>
> Right; maybe I should just be calling a "batch of packages to be validated
> together" a "tx package" in the first place.
>
> Maybe it would be worth emphasising that you should be expecting to
> validate all the txs you receive as a response to getpckgtxns (getpkgtxs
> :) all at the same time, and immediately upon receiving them?
>
> > > The "only be sent if both peers agreed to do package relay" rule could
> > > simply be dropped, I think.
> > Wouldn’t we need some way of saying “hey I support batchtxns?” Otherwise
> > you would have to guess by sending a request and waiting to see if it’s
> > ignored?
>
> Sure, perhaps I should have said leave that rule, but drop the following
> "should be disconnected" rule, so that other BIPs could add in other
> ways of negotiating the connection in future? *shrug*
>
> > > Shouldn't the sender only be sending package announcements when they
> know
> > > the recipient will be interested in the package, based on their
> feefilter?
> > I think there are cases where the sender doesn’t necessarily know.
> > Consider this example:
> >
> https://raw.githubusercontent.com/glozow/bitcoin-notes/master/mempool_garden/rich_parent_bad_cpfp.png
> > D (5sat/vB) has 2 parents, A (0sat/vB) and B (20sat/vB). All same size.
> > Feefilter is 3sat/vB.
> > If the receiver already has B, they’ll know they can just reject the
> > package already based on the pckginfo.
> > But the sender doesn’t really know that. The sender just knows A is below
> > feerate and D is above. D is above the fee filter, and its ancestor
> feerate
> > is above the fee filter.
>
> The sender would also need to know whether or not there's some other
> child E that pays for A sufficiently?
>
> If you're asking for the package for "D", would a response telling you:
>
> txid_D (500 sat, 100vB)
> txid_A (0 sat, 100vB)
> txid_B (2000 sat, 100 vB)
>
> be better, in that case? Then the receiver can maybe do the logic
> themselves to figure out that they already have A in their mempool
> so it's fine, or not?
>
> If you've got a package for X, and its direct parents P1..Pn, then
> I think the logic would be:
>
> * is X alone above my fee rate? no, then forget it
> * otherwise, s := X.size, f := X.fees, R := [X]
> * for P = P1..Pn:
> * do I already have P? then skip to the next parent
> * s += P.size, f += P.fees, R += [P]
> * if f/s above my fee rate floor? if so, request all the txs in R
>
> and you'd request txs if-and-only-if they're a match for you mempool rate?
>
> If you have a tx with 20 in-mempool parents, then the pkginfo1 message
> as proposed would be 737 bytes; including all the fee/size info would be
> 957 bytes, maybe a 30% increase. Might be worth it though?
>
> Cheers,
> aj
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220523/c502fd7c/attachment-0001.html>

2023-06-07 23:09:46 UTC

- reply

📅 Original date posted:2022-05-18
📝 Original message:(To everyone):
I should have made it much clearer that version 1 is only supposed to solve
1 of the 2 use cases. I was a lot more focused on the fee-bumping use case,
since it’s more important. Orphan-fetching was added to the motivation
section last-minute because John Newbery mentioned to me “hey you could
deal with orphans really easily with this.” Of course,
child-with-unconfirmed-parents packages aren’t very useful for
orphan-fetching since non-parent ancestors are quite common.
Maybe a version 2 package for orphan-fetching could look like this:
“pckginfo2” message contains a tx with all of its ancestors
“MSG_PCKG2” inv type refers to a “pckginfo2” for a tx. You don’t send
inv(MSG_PCKG2), but a node can request getdata(MSG_PCKG2) for a transaction
they want the ancestors for, provided they sent sendpackages(version=2)
ahead of time. It seems to me that orphan-fetching only ever needs to be
receiver-initiated.
Protocol flow would look like this:
https://user-images.githubusercontent.com/25183001/168891185-1630f583-de47-4937-86b1-2652cf8852f2.png
We don’t have a policy for dealing with anything more than a child with its
direct parents, but I also don’t think anybody is relying on fee-bumping
more than 2 generations, so the validation logic here could probably just
submit them all individually. Maybe they can request a pckginfo1 if they
see something that’s too-low-fee, and/or use the
child-with-unconfirmed-parents logic opportunistically.
Thanks aj for the feedback! Responding:
> The "PCKG" abbreviation threw me for a loop; isn't the usual
> abbreviation "PKG" ?
Oh I didn't know that. I could change it if people feel strongly.
> Does it make sense for these to be configurable, rather than implied
> by the version?
> … would it be better to either just not do sendpackages
> at all if you're limiting ancestors in the mempool incompatibly
Effectively: if you’re setting your ancestor/descendant limits lower than
the default, you can’t do package relay. I wonder if this might be
controversial, since it adds pressure to adhere to Bitcoin Core’s current
mempool policy? I would be happy to do it this way, though - makes things
easier to implement.
> > 5. If 'fRelay==false' in a peer's version message, the node must not
> > send "sendpackages" to them. If a "sendpackages" message is
> > received by a peer after sending `fRelay==false` in their version
> > message, the sender should be disconnected.
> Seems better to just say "if you set fRelay=false in your version
> message, you must not send sendpackages"? You already won't do packages
> with the peer if they don't also announce sendpackages.
I guess, theoretically, if you allow bloom filters with this peer, it’s
plausible they’re saying “fRelay=false, I’ll send you a bloom filter later,
and I’ll also want to talk about packages.”
I don’t know if that’s a use case we want to support - my gut reaction is
no.
> Maybe: "You must not send sendpackages unless you also send wtxidrelay" ?
Do you mean if we get a verack, and the peer sent “sendpackages” but not
“wtxidrelay,” we should disconnect them?
> As I understand it, the two cases for the protocol flow are "I received
> an orphan, and I'd like its ancestors please" which seems simple enough,
> and "here's a child you may be interested in, even though you possibly
> weren't interested in the parents of that child".
(Btw, please see my notes at the top of this email about separating those
two use cases. sorry for the confusion).
> I think the logic for the latter is: […]
I have it as: we send a PCKG INV when this transaction’s feerate is above
the fee filter, but one or more of its parents don’t. I don’t think using
ancestor feerate is better.
See this counterexample:
https://raw.githubusercontent.com/glozow/bitcoin-notes/master/mempool_garden/abc_1parent_2kids.png
A (0fee) has 2 kids, B (3sat/vB) and C (20sat/vB), everything’s the same
vsize. Let’s say the fee filter is 3sat/vB.
If we do it based on ancestor feerate, we won’t send B. But B is actually
fine; C is paying for A.
> Are "getpckgtxns" / "pcktxns" really limited to packages, or are they
> just a general way to request a batch of transactions?
> Maybe call those messages "getbatchtxns" and "batchtxns" and allow them to
> be used more generally, potentially in ways unrelated to packages/cpfp?
Indeed, it’s a general way to request a batch of transactions. I’ll
highlight that it is “all or nothing,” i.e. if the sender is missing any of
them, they’ll just send a notfound.
The idea here was to avoid downloading any transactions that can’t be
validated right away. With packages, this makes sense, because there are
dependency relationships. But if you’re requesting multiple unrelated
transactions, for example, it’s unnecessary. You might end up with even
more transaction data that’s just sitting around waiting to be validated.
> The "only be sent if both peers agreed to do package relay" rule could
> simply be dropped, I think.
Wouldn’t we need some way of saying “hey I support batchtxns?” Otherwise
you would have to guess by sending a request and waiting to see if it’s
ignored?
> Shouldn't the sender only be sending package announcements when they know
> the recipient will be interested in the package, based on their feefilter?
I think there are cases where the sender doesn’t necessarily know.
Consider this example:
https://raw.githubusercontent.com/glozow/bitcoin-notes/master/mempool_garden/rich_parent_bad_cpfp.png
D (5sat/vB) has 2 parents, A (0sat/vB) and B (20sat/vB). All same size.
Feefilter is 3sat/vB.
If the receiver already has B, they’ll know they can just reject the
package already based on the pckginfo.
But the sender doesn’t really know that. The sender just knows A is below
feerate and D is above. D is above the fee filter, and its ancestor feerate
is above the fee filter.
> CAmount in consensus/amount.h is a int64_t
> The maximum block weight is 4M…
Oops yes. I think we just usually use int64_t for vsizes afaik. Agree that
it should be 8 bytes for fee, and 4 bytes is enough for vsize.
> I guess tx relay is low priority enough that it wouldn't be worth tagging
> some peers as "high bandwidth" and having them immediately announce the
> PCKGINFO1 message, and skip the INV/GETDATA step?
I had the same idea as well, but seemed unnecessary. It would reduce the
number of round trips, but I don’t think an extra round trip is that big of
a deal for transaction relay. Block relay, yes of course, but I don’t think
we care that much if it takes an extra second to send a transaction?
Best,
Gloria
On Tue, May 17, 2022 at 8:35 PM Anthony Towns <aj at erisian.com.au> wrote:
> On Tue, May 17, 2022 at 12:01:04PM -0400, Gloria Zhao via bitcoin-dev
> wrote:
> > ====New Messages====
> > Three new protocol messages are added for use in any version of
> > package relay. Additionally, each version of package relay must define
> > its own inv type and "pckginfo" message version, referred to in this
> > document as "MSG_PCKG" and "pckginfo" respectively. See
> > BIP-v1-packages for a concrete example.
>
> The "PCKG" abbreviation threw me for a loop; isn't the usual
> abbreviation "PKG" ?
>
> > =====sendpackages=====
> > |version || uint32_t || 4 || Denotes a package version supported by the
> > node.
> > |max_count || uint32_t || 4 ||Specifies the maximum number of
> transactions
> > per package this node is
> > willing to accept.
> > |max_weight || uint32_t || 4 ||Specifies the maximum total weight per
> > package this node is willing
> > to accept.
>
> Does it make sense for these to be configurable, rather than implied
> by the version?
>
> I presume the idea is to cope with people specifying different values for
> -limitancestorcount or -limitancestorsize, but if people are regularly
> relaying packages around, it seems like it becomes hard to have those
> values really be configurable while being compatible with that?
>
> I guess I'm asking: would it be better to either just not do sendpackages
> at all if you're limiting ancestors in the mempool incompatibly; or
> alternatively, would it be better to do the package relay, then reject
> the particular package if it turns out too big, and log that you've
> dropped it so that the node operator has some way of realising "whoops,
> I'm not relaying packages properly because of how I configured my node"?
>
> > 5. If 'fRelay==false' in a peer's version message, the node must not
> > send "sendpackages" to them. If a "sendpackages" message is
> > received by a peer after sending `fRelay==false` in their version
> > message, the sender should be disconnected.
>
> Seems better to just say "if you set fRelay=false in your version
> message, you must not send sendpackages"? You already won't do packages
> with the peer if they don't also announce sendpackages.
>
> > 7. If both peers send "wtxidrelay" and "sendpackages" with the same
> > version, the peers should announce, request, and send package
> > information to each other.
>
> Maybe: "You must not send sendpackages unless you also send wtxidrelay" ?
>
>
> As I understand it, the two cases for the protocol flow are "I received
> an orphan, and I'd like its ancestors please" which seems simple enough,
> and "here's a child you may be interested in, even though you possibly
> weren't interested in the parents of that child". I think the logic for
> the latter is:
>
> * if tx C's fee rate is less than the peer's feefilter, skip it
> (will maybe treat it as a parent in some package later though)
> * if tx C's ancestor fee rate is less than the peer's feefilter, skip
> it?
> * look at the lowest ancestor fee rate for any of C's in-mempool
> parents
> * if that is higher than the peer's fee filter, send a normal INV
> * if it's lower than the peer's fee filter, send a PCKG INV
>
> Are "getpckgtxns" / "pcktxns" really limited to packages, or are they
> just a general way to request a batch of transactions? Particularly in
> the case of requesting the parents of an orphan tx you already have,
> it seems hard for the node receiving getpckgtxns to validate that the
> txs are related in some way; but also it doesn't seem very necessary?
>
> Maybe call those messages "getbatchtxns" and "batchtxns" and allow them to
> be used more generally, potentially in ways unrelated to packages/cpfp?
> The "only be sent if both peers agreed to do package relay" rule could
> simply be dropped, I think.
>
> > 4. The reciever uses the package information to decide how to request
> > the transactions. For example, if the receiver already has some of
> > the transactions in their mempool, they only request the missing ones.
> > They could also decide not to request the package at all based on the
> > fee information provided.
>
> Shouldn't the sender only be sending package announcements when they know
> the recipient will be interested in the package, based on their feefilter?
>
> > =====pckginfo1=====
> > {|
> > | Field Name || Type || Size || Purpose
> > |-
> > |blockhash || uint256 || 32 || The chain tip at which this package is
> > defined.
> > |-
> > |pckg_fee||CAmount||4|| The sum total fees paid by all transactions in
> the
> > package.
>
> CAmount in consensus/amount.h is a int64_t so shouldn't this be 8
> bytes? If you limit a package to 101kvB, an int32_t is enough to cover
> any package with a fee rate of about 212 BTC/block or lower, though.
>
> > |pckg_weight||int64_t||8|| The sum total weight of all transactions in
> the
> > package.
>
> The maximum block weight is 4M, and the default -limitancestorsize
> presumably implies a max package weight of 404k; seems odd to provide
> a uint64_t rather than an int32_t here, which easily allows either of
> those values?
>
> > 2. ''Only 1 child with unconfirmed parents.'' The package must consist
> > of one transaction and its unconfirmed parents. There must not be
> > any other transactions in the package. Other dependency relationships
> > may exist within the package (e.g. one parent may spend the output of
> > another parent) provided that topological order is respected.
>
> I think this means that some of the parents could also have unconfirmed
> parents, but they won't be included in the package, and must be requested
> via the recipient-initiated approach?
>
> > 5. ''Total fees and weight.'' The 'total_fee' and 'total_weight'
> > fields must accurately represent the sum total of all transactions'
> > fees and weights as defined in BIP141, respectively.
>
> Presumably this excludes any unconfirmed grandparents and earlier
> ancestors since they aren't part of the package, in this approach? Doesn't
> that make this both harder to calculate (assuming we already have
> ancestor summaries) and less useful, in the case where those ancestors
> have a lower fee rate?
>
> > ''Q: Can "getpckgtxns" and "pckgtxns" messages contain only one
> > transaction?''
> > Yes.
>
> This would be normal if you're requesting a single missing parent for
> an orphan you've received, I think?
>
> I'm slightly surprised the process is:
>
> -> INV PCKG1 C
> <- GETDATA PCKG1 C
> -> PCKGINFO1 blockhash A B C fee weight
>
> rather than announcing the package fee info in the first message.
> But if the sender is already applying the feefilter to the package before
> announcing it, it probably doesn't matter, and means you're only getting
> a 32B INV from every peer, rather than a 32*(n+2) PCKGINFO1 message from
> every peer.
>
> I guess tx relay is low priority enough that it wouldn't be worth tagging
> some peers as "high bandwidth" and having them immediately announce the
> PCKGINFO1 message, and skip the INV/GETDATA step?
>
> Cheers,
> aj
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220518/f022b65c/attachment-0001.html>

2023-06-07 23:09:43 UTC

- reply

📅 Original date posted:2022-05-17
📝 Original message:Hi Greg,
Thanks for reading!
>> A child-with-unconfirmed-parents package sent between nodes must abide by
the rules below, otherwise the package is malformed and the sender should
be disconnected.
>> However, if the child has confirmed parents, they must not be in the
package.
> If my naive understanding is correct, this means things like otherwise
common situations such as a new block will result in disconnects, say when
> the sender doesn't hear about a new block which makes the relay package
superfluous/irrelevant. Similar would be disconnection
> when confirmed gets turned into unconfirmed, but those situations are
extremely uncommon. The other rules are entirely under the control
> of the sender, which leads me to wonder if it's appropriate.
This is why the "pckginfo1" message includes the blockhash at which the
package was defined.
Also please see Clarifications - "Q: What if a new block arrives in between
messages?'' section in the v1-packages portion. It covers both cases, i.e.
a transaction going from unconfirmed->confirmed and confirmed->unconfirmed
in a reorg.
In case anybody is wondering "why don't we just allow confirmed parents?":
Since we validate based on the UTXO set, when we see a recently-confirmed
transaction, it just looks like it spends nonexistent inputs. In these
cases, we don't really know if the input was recently spent in a block or
just never existed, unless we plan on looking up transactions in past
blocks. We do some guesswork when we deal with new blocks in normal
transaction relay (e.g. we requested the tx before a block arrived):
https://github.com/bitcoin/bitcoin/blob/d5d40d59f8d12cf53c5ad1ce9710f3f108cec386/src/validation.cpp#L780-L784
I believe it's cleaner to just explicitly say which blockhash you're on to
avoid confusion.
Thanks,
Gloria
On Tue, May 17, 2022 at 1:56 PM Greg Sanders <gsanders87 at gmail.com> wrote:
> Hi Gloria,
>
> Thanks for working on this important proposal!
>
> Still a lot to digest, but I just had on area of comment/question:
>
> > A child-with-unconfirmed-parents package sent between nodes must abide by
> the rules below, otherwise the package is malformed and the sender should
> be disconnected.
>
> > However, if the child has confirmed parents, they must not be in the
> package.
>
> If my naive understanding is correct, this means things like otherwise
> common situations such as a new block will result in disconnects, say when
> the sender doesn't hear about a new block which makes the relay package
> superfluous/irrelevant. Similar would be disconnection
> when confirmed gets turned into unconfirmed, but those situations are
> extremely uncommon. The other rules are entirely under the control
> of the sender, which leads me to wonder if it's appropriate.
>
> Cheers,
> Greg
>
> On Tue, May 17, 2022 at 12:09 PM Gloria Zhao via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>> Hi everybody,
>>
>> I’m writing to propose a set of p2p protocol changes to enable package
>> relay, soliciting feedback on the design and approach. Here is a link
>> to the most up-to-date proposal:
>>
>> https://github.com/bitcoin/bips/pull/1324
>>
>> If you have concept or approach feedback, *please respond on the
>> mailing list* to allow everybody to view and participate in the
>> discussion. If you find a typo or inaccurate wording, please feel free
>> to leave suggestions on the PR.
>>
>> I’m also working on an implementation for Bitcoin Core.
>>
>>
>> The rest of this post will include the same contents as the proposal,
>> with a bit of reordering and additional context. If you are not 100%
>> up-to-date on package relay and find the proposal hard to follow, I
>> hope you find this format more informative and persuasive.
>>
>>
>> ==Background and Motivation==
>>
>> Users may create and broadcast transactions that depend upon, i.e.
>> spend outputs of, unconfirmed transactions. A “package” is the
>> widely-used term for a group of transactions representable by a
>> connected Directed Acyclic Graph (where a directed edge exists between
>> a transaction that spends the output of another transaction).
>>
>> Incentive-compatible mempool and miner policies help create a fair,
>> fee-based market for block space. While miners maximize transaction
>> fees in order to earn higher block rewards, non-mining users
>> participating in transaction relay reap many benefits from employing
>> policies that result in a mempool with the same contents, including
>> faster compact block relay and more accurate fee estimation.
>> Additionally, users may take advantage of mempool and miner policy to
>> bump the priority of their transactions by attaching high-fee
>> descendants (Child Pays for Parent or CPFP). Only considering
>> transactions one at a time for submission to the mempool creates a
>> limitation in the node's ability to determine which transactions have
>> the highest feerates, since it cannot take into account descendants
>> until all the transactions are in the mempool. Similarly, it cannot
>> use a transaction's descendants when considering which of two
>> conflicting transactions to keep (Replace by Fee or RBF).
>>
>> When a user's transaction does not meet a mempool's minimum feerate
>> and they cannot create a replacement transaction directly, their
>> transaction will simply be rejected by this mempool. They also cannot
>> attach a descendant to pay for replacing a conflicting transaction.
>> This limitation harms users' ability to fee-bump their transactions.
>> Further, it presents a security issue in contracting protocols which
>> rely on **presigned**, time-sensitive transactions to prevent cheating
>> (HTLC-Timeout in LN Penalty [1] [2] [3], Unvault Cancel in Revault
>> [4], Refund Transaction in Discreet Log Contracts [5], Updates in
>> eltoo [6]). In other words, a key security assumption of many
>> contracting protocols is that all parties can propagate and confirm
>> transactions in a timely manner.
>>
>> In the past few years, increasing attention [0][1][2][3][6] has been
>> brought to **pinning attacks**, a type of censorship in which the
>> attacker uses mempool policy restrictions to prevent a transaction
>> from being relayed or getting mined. TLDR: revocation transactions
>> must meet a certain confirmation target to be effective, but their
>> feerates are negotiated well ahead of broadcast time. If the
>> forecasted feerate was too low and no fee-bumping options are
>> available, attackers can steal money from their counterparties. I walk
>> through a concrete example for stealing Lightning HTLC outputs at
>> ~23:58 in this talk [7][8]. Note that most attacks are only possible
>> when the market for blockspace at broadcast time demands much higher
>> feerates than originally anticipated at signing time. Always
>> overestimating fees may sidestep this issue temporarily (while mempool
>> traffic is low and predictable), but this solution is not foolproof
>> and wastes users' money. The feerate market can change due to sudden
>> spikes in traffic (e.g. huge 12sat/vB dump a few days ago [9]) or
>> sustained, high volume of Bitcoin payments (e.g. April 2021 and
>> December 2017).
>>
>> The best solution is to enable nodes to consider packages of
>> transactions as a unit, e.g. one or more low-fee parent transactions
>> with a high-fee child, instead of separately. A package-aware mempool
>> policy can help determine if it would actually be economically
>> rational to accept a transaction to the mempool if it doesn't meet fee
>> requirements individually. Network-wide adoption of these policies
>> would create a more purely-feerate-based market for block space and
>> allow contracting protocols to adjust fees (and therefore mining
>> priority) at broadcast time. Some support for packages has existed in
>> Bitcoin Core for years. Since v0.13, Bitcoin Core has used ancestor
>> packages instead of individual transactions to evaluate the incentive
>> compatibility of transactions in the mempool [10] and select them for
>> inclusion in blocks [11].
>>
>> Package Relay, the concept of {announcing, requesting, downloading}
>> packages between nodes on the p2p network, has also been discussed for
>> many years. The earliest public mention I can find is from 2015 [12].
>> The two most common use cases for package relay are fee-bumping
>> otherwise-too-low-fee transactions and reducing the amount of orphans.
>> It seems uncontroversial to say that everybody desires package relay
>> conceptually, with varying degrees of urgency. Lots of work has been
>> done by others over the past few years, from which I've taken
>> inspiration from [13][14][15][16].
>>
>> My approach has been to split the project into two components: (1) Package
>> Mempool Accept, which includes validation logic and mempool policy.
>> (3) Package Relay, which includes the p2p protocol changes.
>>
>> Progress so far:
>> After discussions with various developers of contracting protocols
>> (with heavier emphasis towards LN), it was determined that a
>> package containing a child with all of its unconfirmed parents
>> (child-with-unconfirmed-parents or 1-child-multi-parent package) would
>> be sufficient for their use case, i.e. fee-bumping presigned
>> transactions. A child-with-unconfirmed-parents package has several
>> properties that make many things easier to reason about.
>>
>> A few months ago, I proposed a set of policies for safe package
>> validation and fee assessment for packages of this restricted
>> topology [17]. A series of PRs implementing this proposal have
>> been merged into Bitcoin Core [18].
>>
>> Theoretically, developing a safe and incentive-compatible package
>> mempool acceptance policy is sufficient to solve this issue. Nodes
>> could opportunistically accept packages (e.g. by trying combinations
>> of transactions rejected from their mempools), but this practice would
>> likely be inefficient at best and open new Denial of Service attacks
>> at worst. Additional p2p messages may enable nodes to request and
>> share package validation-related information with one another in a
>> more communication-efficient way.
>>
>> Given that only package RBF remains for package mempool accept, and we
>> can make progress on p2p and mempool in parallel, I think it’s
>> appropriate to put forward a package relay proposal.
>>
>> ==Proposal==
>>
>> This proposal contains 2 components: a “generic” package relay
>> protocol and an extension of it, child-with-unconfirmed-parents
>> packages, as version 1 package relay. Another version of packages,
>> “tx-with-unconfirmed-ancestors” can be created to extend package relay
>> for eliminating orphans.
>>
>> ===Generic Package Relay===
>>
>> Two main ideas are introduced:
>>
>> Download and validate packages of transactions together.
>>
>> Provide information to help peers decide whether to request and/or how
>> to validate transactions which are part of a package.
>>
>> ====Intended Protocol Flow====
>>
>> Due to the asynchronous nature of a distributed transaction relay
>> network, nodes may not receive all of the information needed to
>> validate a transaction at once. For example, after a node completes
>> Initial Block Download (IBD) and first starts participating in
>> transaction relay with an empty mempool, it is common to receive
>> orphans. In such scenarios where a node is aware that it is missing
>> information, a ''receiver-initiated'' dialogue is appropriate:
>>
>> 1. Receiver requests package information.
>>
>> 2. The sender provides package information, including the wtxids of
>> the transactions in the package and anything else that might be
>> relevant (e.g. total fees and size).
>>
>> 3. The reciever uses the package information to decide how to request
>> and validate the transactions.
>>
>> Sometimes, no matter what order transactions are received by a node,
>> validating them individually is insufficient. When the sender is aware
>> of additional information that the receiver needs to accept a package,
>> a proactive ''sender-initiated'' dialogue should be enabled:
>>
>> 1. Sender announces they have package information pertaining to a
>> transaction that might otherwise be undesired on its own.
>>
>> 2. The receiver requests package information.
>>
>> 3. The sender provides package information, including the wtxids of
>> the transactions in the package and anything else that might be
>> relevant (e.g. total fees and size).
>>
>> 4. The reciever uses the package information to decide how to request
>> and validate the transactions.
>>
>> Package relay is negotiated between two peers during the version
>> handshake. Package relay requires both peers to support wtxid-based
>> relay because package transactions are referenced by their wtxid.
>>
>> ====New Messages====
>>
>> Three new protocol messages are added for use in any version of
>> package relay. Additionally, each version of package relay must define
>> its own inv type and "pckginfo" message version, referred to in this
>> document as "MSG_PCKG" and "pckginfo" respectively. See
>> BIP-v1-packages for a concrete example.
>>
>> =====sendpackages=====
>>
>> {|
>> | Field Name || Type || Size || Purpose
>> |-
>> |version || uint32_t || 4 || Denotes a package version supported by the
>> node.
>> |-
>> |max_count || uint32_t || 4 ||Specifies the maximum number of
>> transactions per package this node is
>> willing to accept.
>> |-
>> |max_weight || uint32_t || 4 ||Specifies the maximum total weight per
>> package this node is willing
>> to accept.
>> |-
>> |}
>>
>> 1. The "sendpackages" message has the structure defined above, with
>> pchCommand == "sendpackages".
>>
>> 2. During version handshake, nodes should send a "sendpackages"
>> message indicate they support package relay and may request
>> packages.
>>
>> 3. The message should contain a version supported by the node. Nodes
>> should send a "sendpackages" message for each version they support.
>>
>> 4. The "sendpackages" message MUST be sent before sending a "verack"
>> message. If a "sendpackages" message is received afer "verack", the
>> sender should be disconnected.
>>
>> 5. If 'fRelay==false' in a peer's version message, the node must not
>> send "sendpackages" to them. If a "sendpackages" message is
>> received by a peer after sending `fRelay==false` in their version
>> message, the sender should be disconnected.
>>
>> 6.. Upon receipt of a "sendpackages" message with a version that is
>> not supported, a node must treat the peer as if it never received the
>> message.
>>
>> 7. If both peers send "wtxidrelay" and "sendpackages" with the same
>> version, the peers should announce, request, and send package
>> information to each other.
>>
>> =====getpckgtxns=====
>>
>> {|
>> | Field Name || Type || Size || Purpose
>> |-
>> |txns_length||CompactSize||1 or 3 bytes|| The number of transactions
>> requested.
>> |-
>> |txns||List of wtxids||txns_length * 32|| The wtxids of each transaction
>> in the package.
>> |}
>>
>> 1. The "getpckgtxns" message has the structure defined above, with
>> pchCommand == "getpckgtxns".
>>
>> 2. A "getpckgtxns" message should be used to request all or some of
>> the transactions previously announced in a "pckginfo" message,
>> specified by witness transactiosome id.
>>
>> 3. Upon receipt of a "getpckgtxns" message, a node must respond with
>> either a "pckgtxns" containing the requested transactions or a
>> "notfound" message indicating one or more of the transactions is
>> unavailable. This allows the receiver to avoid downloading and storing
>> transactions that cannot be validated immediately.
>>
>> 4. A "getpckgtxns" message should only be sent if both peers agreed to
>> send packages in the version handshake. If a "getpckgtxns" message
>> is received from a peer with which package relay was not negotiated,
>> the sender should be disconnected.
>>
>> =====pckgtxns=====
>>
>> {|
>> | Field Name || Type || Size || Purpose
>> |-
>> |txns_length||CompactSize||1 or 3 bytes|| The number of transactions
>> provided.
>> |-
>> |txns||List of transactions||variable|| The transactions in the package.
>> |}
>>
>> 1. The "pckgtxns" message has the structure defined above, with
>> pchCommand == "pckgtxns".
>>
>> 2. A "pckgtxns" message should contain the transaction data requested
>> using "getpckgtxns".
>>
>> 3. A "pckgtxns" message should only be sent to a peer that requested
>> the package using "getpckgtxns". If a node receives an unsolicited
>> package, the sender should be disconnected.
>>
>> 4. A "pckgtxns" message should only be sent if both peers agreed to
>> send packages in the version handshake. If a "pckgtxns" message is
>> received from a peer with which package relay was not negotiated, the
>> sender should be disconnected.
>>
>> ===Version 1 Packages: child-with-unconfirmed-parents===
>>
>> This extends package relay for packages consisting of one transaction
>> and all of its unconfirmed parents,by defining version 1 packages, a
>> pckginfo1 message, and a MSG_PCKG1 inv type. It enables the use case
>> in which a child pays for its otherwise-too-low-fee parents and their
>> mempool conflict(s).
>>
>> ====Intended Protocol Flow====
>>
>> When relaying a package of low-fee parent(s) and high-fee child, the
>> sender and receiver do the following:
>>
>> 1. Sender announces they have a child-with-unconfirmed-parents package
>> for a child that pays for otherwise-too-low-fee parent(s) using
>> "inv(MSG_PCKG1)".
>>
>> 2. The receiver requests package information using
>> "getdata(MSG_PCKG1)".
>>
>> 3. The sender provides package information using "pckginfo1",
>> including the blockhash of the sender's best block, the wtxids of
>> the transactions in the package, their total fees and total weight.
>>
>> 4. The reciever uses the package information to decide how to request
>> the transactions. For example, if the receiver already has some of
>> the transactions in their mempool, they only request the missing ones.
>> They could also decide not to request the package at all based on the
>> fee information provided.
>>
>> 5. Upon receiving a "pckgtxns", the receiver submits the transactions
>> together as a package.
>>
>> ====New Messages====
>>
>> A new inv type, "MSG_PCKG1", and new protocol message, "PCKGINFO1",
>> are added.
>>
>> =====pckginfo1=====
>>
>> {|
>> | Field Name || Type || Size || Purpose
>> |-
>> |blockhash || uint256 || 32 || The chain tip at which this package is
>> defined.
>> |-
>> |pckg_fee||CAmount||4|| The sum total fees paid by all transactions in
>> the package.
>> |-
>> |pckg_weight||int64_t||8|| The sum total weight of all transactions in
>> the package.
>> |-
>> |txns_length||CompactSize||1 or 3 bytes|| The number of transactions
>> provided.
>> |-
>> |txns||List of wtxids||txns_length * 32|| The wtxids of each transaction
>> in the package.
>> |}
>>
>>
>> 1. The "pckginfo1" message has the structure defined above, with
>> pchCommand == "pckginfo1".
>>
>> 2. A "pckginfo1" message contains information about a version 1
>> package (defined below), referenced by the wtxid of the transaction
>> it pertains to and the current blockhash.
>>
>> 3. Upon receipt of a "pckginfo1" message, the node should decide if it
>> wants to validate the package, request transaction data if
>> necessary, etc.
>>
>> 4. Upon receipt of a malformed "pckginfo1" message or package that
>> does not abide by the max_count, max_weight, or other rules
>> specified by the version agreed upon in the initial negotiation, the
>> sender should be disconnected. If a node receives a "pckginfo1"
>> message for which the "pckg_fee" or "pckg_weight" do not reflect the
>> true total fees and weight, respectively, or the transactions in the
>> package, the message is malformed.
>>
>> 5. A node MUST NOT send a "pckginfo1" message that has not been
>> requested by the recipient. Upon receipt of an unsolicited
>> "pckginfo1", a node should disconnect the sender.
>>
>> 6. A "pckginfo1" message should only be sent if both peers agreed to
>> send version 1 packages in the version handshake. If a "pckginfo1"
>> message is received from a peer with which package relay was not
>> negotiated, the sender should be disconnected.
>>
>> =====MSG_PCKG1=====
>>
>> 1. A new inv type (MSG_PCKG1 == 0x6) is added, for use in inv messages
>> and getdata requests pertaining to version 1 packages.
>>
>> 2. As an inv type, it indicates that both transaction data and version
>> 1 package information are available for the transaction. The
>> transaction is referenced by its wtxid. As a getdata request type, it
>> indicates that the sender wants package information for the
>> transaction.
>>
>> 3. Upon receipt of a "getdata" request for "MSG_PCKG1", the node
>> should respond with the version 1 package corresponding to the
>> requested transaction and its current chain tip, or with NOTFOUND.
>> The node should not assume that the sender is requesting the
>> transaction data as well.
>>
>> ====Child With Parent Packages Rules====
>>
>> A child-with-unconfirmed-parents package sent between nodes must abide
>> by the rules below, otherwise the package is malformed and the sender
>> should be disconnected.
>>
>> A version 1 or ''child-with-unconfirmed-parents'' package can be
>> defined for any transaction that spends unconfirmed inputs. The child
>> can be thought of as the "representative" of the package. This package
>> can be uniquely identified by the transaction's wtxid and the current
>> chain tip block hash.
>>
>> A ''child-with-unconfirmed-parents'' package MUST be:
>>
>> 1. ''Sorted topologically.'' For every transaction t in the package,
>> if any of t's parents are present in the package, the parent must
>> appear somewhere in the list before t. In other words, the
>> transactions must be sorted in ascending order of the number of
>> ancestors present in the package.
>>
>> 2. ''Only 1 child with unconfirmed parents.'' The package must consist
>> of one transaction and its unconfirmed parents. There must not be
>> any other transactions in the package. Other dependency relationships
>> may exist within the package (e.g. one parent may spend the output of
>> another parent) provided that topological order is respected.
>>
>> 3. ''All unconfirmed parents.'' All of the child's unconfirmed parents
>> must be present.
>>
>> 4. ''No conflicts.'' None of the transactions in the package may
>> conflict with each other (i.e. spend the same prevout).
>>
>> 5. ''Total fees and weight.'' The 'total_fee' and 'total_weight'
>> fields must accurately represent the sum total of all transactions'
>> fees and weights as defined in BIP141, respectively.
>>
>> Not all of the child's parents must be present; the child transaction
>> may also spend confirmed inputs. However, if the child has confirmed
>> parents, they must not be in the package.
>>
>> While a child-with-unconfirmed-parents package is perhaps most
>> relevant when the child has a higher feerate than its parents, this
>> property is not required to construct a valid package.
>>
>> ====Clarifications====
>>
>> ''Q: Under what circumstances should a sender announce a
>> child-with-unconfirmed-parents package?''
>>
>> A child-with-unconfirmed-parents package for a transaction should be
>> announced when it meets the peer's fee filter but one or more of its
>> parents don't; a "inv(MSG_PCKG1)" instead of "inv(WTX)" should be sent
>> for the child. Each of the parents which meet the peer's fee filter
>> should still be announced normally.
>>
>> ''Q: What if a new block arrives in between messages?''
>>
>> A child-with-unconfirmed-parents package is defined for a transaction
>> based on the current chain state. As such, a new block extending the
>> tip may decrease the number of transactions in the package (i.e. if
>> any of the transaction's parents were included in the block). In a
>> reorg, the number of transactions in the package may decrease or
>> increase (i.e. if any of the transaction's parents were included in a
>> block in the previous chain but not the new one).
>>
>> If the new block arrives before the "getdata" or "pckginfo1", nothing
>> needs to change.
>>
>> If the new block arrives before "getpckgtxns" or before "pckgtxns",
>> the receiver may need to re-request package information if the block
>> contained a transaction in the package. If the block doesn't contain
>> any transactions in the package, whether it extends the previous tip
>> or causes a reorg, nothing needs to change.
>>
>> ''Q: Can "getpckgtxns" and "pckgtxns" messages contain only one
>> transaction?''
>>
>> Yes.
>>
>> ===Further Protocol Extensions===
>>
>> When introducing a new type of package, assign it a version number "n"
>> and use an additional "sendpackages" message during version handshake
>> to negotiate support for it. An additional package information message
>> "pckginfon" and inv type "MSG_PCKGn" should be defined for the type of
>> package. However, "getpckgtxns" and "pckgtxns" do not need to be
>> changed.
>>
>> Example proposal for tx-with-unconfirmed-ancestors package relay: [19]
>>
>> ===Compatibility===
>>
>> Older clients remain fully compatible and interoperable after this
>> change. Clients implementing this protocol will only attempt to send
>> and request packages if agreed upon during the version handshake.
>>
>> ===Package Erlay===
>>
>> Clients using BIP330 reconciliation-based transaction relay (Erlay)
>> are able to use package relay without interference. In fact, a package
>> of transactions may be announced using both Erlay and package relay.
>> After reconciliation, if the initiator would have announced a
>> transaction by wtxid but also has package information for it, they may
>> send "inv(MSG_PCKG)" instead of "inv(WTX)".
>>
>> ===Rationale===
>>
>> ====P2P Message Design====
>>
>> These p2p messages are added for communication efficiency and, as
>> such, one should measure alternative solutions based on the resources
>> used to communicate (not necessarily trustworthy) information: We
>> would like to minimize network bandwidth, avoid downloading a
>> transaction more than once, avoid downloading transactions that are
>> eventually rejected, and minimize storage allocated for
>> not-yet-validated transactions.
>>
>> Consider these (plausible) scenarios in transaction relay:
>>
>> Alice (the "sender") is relaying transactions to Bob (the "receiver").
>> Alice's mempool has a minimum feerate of 1sat/vB and Bob's has a
>> minimum feerate of 3sat/vB. For simplicity, all transactions are
>> 1600Wu in virtual size and 500 bytes in serialized size. Apart from
>> the spending relationships specified, all other inputs are from
>> confirmed UTXOs.
>>
>> 1. Package {A, B} where A pays 0 satoshis and B pays 8000 satoshis in
>> fees.
>>
>> 2. Package {C, D} where C pays 0 satoshis and D pays 1200 satoshis in
>> fees.
>>
>> 3. Package {E, F, G, H, J} that pays 4000, 8000, 0, 2000, and 4000
>> satoshis in fees, respectively.
>>
>> ====Alternative Designs Considered====
>>
>> ''Package Information Only:'' Just having "pckginfo" gives enough
>> information for the receiver to accept the package. Omit the
>> "getpckgtxns" and "pckgtxns" messages. While this option is a good
>> fallback if batched transaction download fails for some reason, it
>> shouldn't be used as the default because it 'always' requires storage
>> of unvalidated transactions.
>>
>> ''No Package Information Round:'' Instead of having a package
>> information round, just use the child's wtxid to refer to the package
>> and always send the entire package together. This would cause nodes to
>> redownload duplicate transactions.
>>
>> I have also created a slidedeck exploring various alternative designs
>> and some examples in which they fall flat [20]. Please feel free to
>> suggest other alternatives.
>>
>> ====Versioning System====
>>
>> This protocol should be extensible to support multiple types of
>> packages based on future desired use cases. Two "flavors" of
>> versioning were considered:
>>
>> 1. When package mempool acceptance is upgraded to support more types
>> of packages, increment the version number (similar to Erlay).
>> During version handshake, peers negotiate which version of package
>> relay they will use by each sending one "sendpackages" message.
>>
>> 2. When introducing another type of package, assign a version number
>> to it and announce it as an additional supported version (similar
>> to Compact Block Relay). During version handshake, peers send one
>> "sendpackages" message for each version supported.
>>
>> The second option was favored because it allows different parameters
>> for different versions. For example, it should be possible to support
>> both "arbitrary topology but maximum 3-transaction" package as well as
>> "child-with-unconfirmed-parents with default mempool ancestor limits"
>> packages simultaneously.
>>
>> ==Acknowledgements==
>>
>> I hope to have made it abundantly clear that this proposal isn’t
>> inventing the concept of package relay, and in fact builds upon years
>> of work by many others, including Suhas Daftuar and Antoine Riard.
>>
>> Thank you to John Newbery and Martin Zumsande for input on the design.
>>
>> Thank you to Matt Corallo, Christian Decker, David Harding, Antoine
>> Poinsot, Antoine Riard, Gregory Sanders, Chris Stewart, Bastien
>> Teinturier, and others for input on the desired interface for
>> contracting protocols.
>>
>> Looking forward to hearing your thoughts!
>>
>> Best,
>> Gloria
>>
>> [0]:
>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
>> [1]:
>> https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-April/002639.html
>> [2]:
>> https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-June/002758.html
>> [3]:
>> https://github.com/t-bast/lightning-docs/blob/master/pinning-attacks.md
>> [4]:
>> https://github.com/revault/practical-revault/blob/master/transactions.md#cancel_tx
>> [5]:
>> https://github.com/discreetlogcontracts/dlcspecs/blob/master/Transactions.md#refund-transaction
>> [6]: https://gist.github.com/instagibbs/60264606e181451e977e439a49f69fe1
>> [7]:
>> https://btctranscripts.com/adopting-bitcoin/2021/2021-11-16-gloria-zhao-transaction-relay-policy/#lightning-attacks
>> [8]: https://youtu.be/fbWSQvJjKFs?t=1438
>> [9]:
>> https://www.reddit.com/r/Bitcoin/comments/unew4e/looks_like_70_mvb_of_transactions_just_got_dumped/
>> [10]: https://github.com/bitcoin/bitcoin/pull/7594
>> [11]: https://github.com/bitcoin/bitcoin/pull/7600
>> [12]: https://github.com/bitcoin/bitcoin/pull/6455#issuecomment-122716820
>> [13]: https://gist.github.com/sdaftuar/8756699bfcad4d3806ba9f3396d4e66a
>> [14]: https://github.com/bitcoin/bitcoin/issues/14895
>> [15]: https://github.com/bitcoin/bitcoin/pull/16401
>> [16]: https://github.com/bitcoin/bitcoin/pull/19621
>> [17]:
>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html
>> [18]: https://github.com/users/glozow/projects/5/views/4?layout=board
>> [19]: https://gist.github.com/glozow/9b321cd3ef6505135c763112033ff2a7
>> [20]:
>> https://docs.google.com/presentation/d/1B__KlZO1VzxJGx-0DYChlWawaEmGJ9EGApEzrHqZpQc/edit?usp=sharing
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220517/321c2029/attachment-0001.html>

2023-06-07 23:09:41 UTC

- reply

📅 Original date posted:2022-05-17
📝 Original message:Hi everybody,
I’m writing to propose a set of p2p protocol changes to enable package
relay, soliciting feedback on the design and approach. Here is a link
to the most up-to-date proposal:
https://github.com/bitcoin/bips/pull/1324
If you have concept or approach feedback, *please respond on the
mailing list* to allow everybody to view and participate in the
discussion. If you find a typo or inaccurate wording, please feel free
to leave suggestions on the PR.
I’m also working on an implementation for Bitcoin Core.
The rest of this post will include the same contents as the proposal,
with a bit of reordering and additional context. If you are not 100%
up-to-date on package relay and find the proposal hard to follow, I
hope you find this format more informative and persuasive.
==Background and Motivation==
Users may create and broadcast transactions that depend upon, i.e.
spend outputs of, unconfirmed transactions. A “package” is the
widely-used term for a group of transactions representable by a
connected Directed Acyclic Graph (where a directed edge exists between
a transaction that spends the output of another transaction).
Incentive-compatible mempool and miner policies help create a fair,
fee-based market for block space. While miners maximize transaction
fees in order to earn higher block rewards, non-mining users
participating in transaction relay reap many benefits from employing
policies that result in a mempool with the same contents, including
faster compact block relay and more accurate fee estimation.
Additionally, users may take advantage of mempool and miner policy to
bump the priority of their transactions by attaching high-fee
descendants (Child Pays for Parent or CPFP). Only considering
transactions one at a time for submission to the mempool creates a
limitation in the node's ability to determine which transactions have
the highest feerates, since it cannot take into account descendants
until all the transactions are in the mempool. Similarly, it cannot
use a transaction's descendants when considering which of two
conflicting transactions to keep (Replace by Fee or RBF).
When a user's transaction does not meet a mempool's minimum feerate
and they cannot create a replacement transaction directly, their
transaction will simply be rejected by this mempool. They also cannot
attach a descendant to pay for replacing a conflicting transaction.
This limitation harms users' ability to fee-bump their transactions.
Further, it presents a security issue in contracting protocols which
rely on **presigned**, time-sensitive transactions to prevent cheating
(HTLC-Timeout in LN Penalty [1] [2] [3], Unvault Cancel in Revault
[4], Refund Transaction in Discreet Log Contracts [5], Updates in
eltoo [6]). In other words, a key security assumption of many
contracting protocols is that all parties can propagate and confirm
transactions in a timely manner.
In the past few years, increasing attention [0][1][2][3][6] has been
brought to **pinning attacks**, a type of censorship in which the
attacker uses mempool policy restrictions to prevent a transaction
from being relayed or getting mined. TLDR: revocation transactions
must meet a certain confirmation target to be effective, but their
feerates are negotiated well ahead of broadcast time. If the
forecasted feerate was too low and no fee-bumping options are
available, attackers can steal money from their counterparties. I walk
through a concrete example for stealing Lightning HTLC outputs at
~23:58 in this talk [7][8]. Note that most attacks are only possible
when the market for blockspace at broadcast time demands much higher
feerates than originally anticipated at signing time. Always
overestimating fees may sidestep this issue temporarily (while mempool
traffic is low and predictable), but this solution is not foolproof
and wastes users' money. The feerate market can change due to sudden
spikes in traffic (e.g. huge 12sat/vB dump a few days ago [9]) or
sustained, high volume of Bitcoin payments (e.g. April 2021 and
December 2017).
The best solution is to enable nodes to consider packages of
transactions as a unit, e.g. one or more low-fee parent transactions
with a high-fee child, instead of separately. A package-aware mempool
policy can help determine if it would actually be economically
rational to accept a transaction to the mempool if it doesn't meet fee
requirements individually. Network-wide adoption of these policies
would create a more purely-feerate-based market for block space and
allow contracting protocols to adjust fees (and therefore mining
priority) at broadcast time. Some support for packages has existed in
Bitcoin Core for years. Since v0.13, Bitcoin Core has used ancestor
packages instead of individual transactions to evaluate the incentive
compatibility of transactions in the mempool [10] and select them for
inclusion in blocks [11].
Package Relay, the concept of {announcing, requesting, downloading}
packages between nodes on the p2p network, has also been discussed for
many years. The earliest public mention I can find is from 2015 [12].
The two most common use cases for package relay are fee-bumping
otherwise-too-low-fee transactions and reducing the amount of orphans.
It seems uncontroversial to say that everybody desires package relay
conceptually, with varying degrees of urgency. Lots of work has been
done by others over the past few years, from which I've taken
inspiration from [13][14][15][16].
My approach has been to split the project into two components: (1) Package
Mempool Accept, which includes validation logic and mempool policy.
(3) Package Relay, which includes the p2p protocol changes.
Progress so far:
After discussions with various developers of contracting protocols
(with heavier emphasis towards LN), it was determined that a
package containing a child with all of its unconfirmed parents
(child-with-unconfirmed-parents or 1-child-multi-parent package) would
be sufficient for their use case, i.e. fee-bumping presigned
transactions. A child-with-unconfirmed-parents package has several
properties that make many things easier to reason about.
A few months ago, I proposed a set of policies for safe package
validation and fee assessment for packages of this restricted
topology [17]. A series of PRs implementing this proposal have
been merged into Bitcoin Core [18].
Theoretically, developing a safe and incentive-compatible package
mempool acceptance policy is sufficient to solve this issue. Nodes
could opportunistically accept packages (e.g. by trying combinations
of transactions rejected from their mempools), but this practice would
likely be inefficient at best and open new Denial of Service attacks
at worst. Additional p2p messages may enable nodes to request and
share package validation-related information with one another in a
more communication-efficient way.
Given that only package RBF remains for package mempool accept, and we
can make progress on p2p and mempool in parallel, I think it’s
appropriate to put forward a package relay proposal.
==Proposal==
This proposal contains 2 components: a “generic” package relay
protocol and an extension of it, child-with-unconfirmed-parents
packages, as version 1 package relay. Another version of packages,
“tx-with-unconfirmed-ancestors” can be created to extend package relay
for eliminating orphans.
===Generic Package Relay===
Two main ideas are introduced:
Download and validate packages of transactions together.
Provide information to help peers decide whether to request and/or how
to validate transactions which are part of a package.
====Intended Protocol Flow====
Due to the asynchronous nature of a distributed transaction relay
network, nodes may not receive all of the information needed to
validate a transaction at once. For example, after a node completes
Initial Block Download (IBD) and first starts participating in
transaction relay with an empty mempool, it is common to receive
orphans. In such scenarios where a node is aware that it is missing
information, a ''receiver-initiated'' dialogue is appropriate:
1. Receiver requests package information.
2. The sender provides package information, including the wtxids of
the transactions in the package and anything else that might be
relevant (e.g. total fees and size).
3. The reciever uses the package information to decide how to request
and validate the transactions.
Sometimes, no matter what order transactions are received by a node,
validating them individually is insufficient. When the sender is aware
of additional information that the receiver needs to accept a package,
a proactive ''sender-initiated'' dialogue should be enabled:
1. Sender announces they have package information pertaining to a
transaction that might otherwise be undesired on its own.
2. The receiver requests package information.
3. The sender provides package information, including the wtxids of
the transactions in the package and anything else that might be
relevant (e.g. total fees and size).
4. The reciever uses the package information to decide how to request
and validate the transactions.
Package relay is negotiated between two peers during the version
handshake. Package relay requires both peers to support wtxid-based
relay because package transactions are referenced by their wtxid.
====New Messages====
Three new protocol messages are added for use in any version of
package relay. Additionally, each version of package relay must define
its own inv type and "pckginfo" message version, referred to in this
document as "MSG_PCKG" and "pckginfo" respectively. See
BIP-v1-packages for a concrete example.
=====sendpackages=====
{|
| Field Name || Type || Size || Purpose
|-
|version || uint32_t || 4 || Denotes a package version supported by the
node.
|-
|max_count || uint32_t || 4 ||Specifies the maximum number of transactions
per package this node is
willing to accept.
|-
|max_weight || uint32_t || 4 ||Specifies the maximum total weight per
package this node is willing
to accept.
|-
|}
1. The "sendpackages" message has the structure defined above, with
pchCommand == "sendpackages".
2. During version handshake, nodes should send a "sendpackages"
message indicate they support package relay and may request
packages.
3. The message should contain a version supported by the node. Nodes
should send a "sendpackages" message for each version they support.
4. The "sendpackages" message MUST be sent before sending a "verack"
message. If a "sendpackages" message is received afer "verack", the
sender should be disconnected.
5. If 'fRelay==false' in a peer's version message, the node must not
send "sendpackages" to them. If a "sendpackages" message is
received by a peer after sending `fRelay==false` in their version
message, the sender should be disconnected.
6.. Upon receipt of a "sendpackages" message with a version that is
not supported, a node must treat the peer as if it never received the
message.
7. If both peers send "wtxidrelay" and "sendpackages" with the same
version, the peers should announce, request, and send package
information to each other.
=====getpckgtxns=====
{|
| Field Name || Type || Size || Purpose
|-
|txns_length||CompactSize||1 or 3 bytes|| The number of transactions
requested.
|-
|txns||List of wtxids||txns_length * 32|| The wtxids of each transaction in
the package.
|}
1. The "getpckgtxns" message has the structure defined above, with
pchCommand == "getpckgtxns".
2. A "getpckgtxns" message should be used to request all or some of
the transactions previously announced in a "pckginfo" message,
specified by witness transactiosome id.
3. Upon receipt of a "getpckgtxns" message, a node must respond with
either a "pckgtxns" containing the requested transactions or a
"notfound" message indicating one or more of the transactions is
unavailable. This allows the receiver to avoid downloading and storing
transactions that cannot be validated immediately.
4. A "getpckgtxns" message should only be sent if both peers agreed to
send packages in the version handshake. If a "getpckgtxns" message
is received from a peer with which package relay was not negotiated,
the sender should be disconnected.
=====pckgtxns=====
{|
| Field Name || Type || Size || Purpose
|-
|txns_length||CompactSize||1 or 3 bytes|| The number of transactions
provided.
|-
|txns||List of transactions||variable|| The transactions in the package.
|}
1. The "pckgtxns" message has the structure defined above, with
pchCommand == "pckgtxns".
2. A "pckgtxns" message should contain the transaction data requested
using "getpckgtxns".
3. A "pckgtxns" message should only be sent to a peer that requested
the package using "getpckgtxns". If a node receives an unsolicited
package, the sender should be disconnected.
4. A "pckgtxns" message should only be sent if both peers agreed to
send packages in the version handshake. If a "pckgtxns" message is
received from a peer with which package relay was not negotiated, the
sender should be disconnected.
===Version 1 Packages: child-with-unconfirmed-parents===
This extends package relay for packages consisting of one transaction
and all of its unconfirmed parents,by defining version 1 packages, a
pckginfo1 message, and a MSG_PCKG1 inv type. It enables the use case
in which a child pays for its otherwise-too-low-fee parents and their
mempool conflict(s).
====Intended Protocol Flow====
When relaying a package of low-fee parent(s) and high-fee child, the
sender and receiver do the following:
1. Sender announces they have a child-with-unconfirmed-parents package
for a child that pays for otherwise-too-low-fee parent(s) using
"inv(MSG_PCKG1)".
2. The receiver requests package information using
"getdata(MSG_PCKG1)".
3. The sender provides package information using "pckginfo1",
including the blockhash of the sender's best block, the wtxids of
the transactions in the package, their total fees and total weight.
4. The reciever uses the package information to decide how to request
the transactions. For example, if the receiver already has some of
the transactions in their mempool, they only request the missing ones.
They could also decide not to request the package at all based on the
fee information provided.
5. Upon receiving a "pckgtxns", the receiver submits the transactions
together as a package.
====New Messages====
A new inv type, "MSG_PCKG1", and new protocol message, "PCKGINFO1",
are added.
=====pckginfo1=====
{|
| Field Name || Type || Size || Purpose
|-
|blockhash || uint256 || 32 || The chain tip at which this package is
defined.
|-
|pckg_fee||CAmount||4|| The sum total fees paid by all transactions in the
package.
|-
|pckg_weight||int64_t||8|| The sum total weight of all transactions in the
package.
|-
|txns_length||CompactSize||1 or 3 bytes|| The number of transactions
provided.
|-
|txns||List of wtxids||txns_length * 32|| The wtxids of each transaction in
the package.
|}
1. The "pckginfo1" message has the structure defined above, with
pchCommand == "pckginfo1".
2. A "pckginfo1" message contains information about a version 1
package (defined below), referenced by the wtxid of the transaction
it pertains to and the current blockhash.
3. Upon receipt of a "pckginfo1" message, the node should decide if it
wants to validate the package, request transaction data if
necessary, etc.
4. Upon receipt of a malformed "pckginfo1" message or package that
does not abide by the max_count, max_weight, or other rules
specified by the version agreed upon in the initial negotiation, the
sender should be disconnected. If a node receives a "pckginfo1"
message for which the "pckg_fee" or "pckg_weight" do not reflect the
true total fees and weight, respectively, or the transactions in the
package, the message is malformed.
5. A node MUST NOT send a "pckginfo1" message that has not been
requested by the recipient. Upon receipt of an unsolicited
"pckginfo1", a node should disconnect the sender.
6. A "pckginfo1" message should only be sent if both peers agreed to
send version 1 packages in the version handshake. If a "pckginfo1"
message is received from a peer with which package relay was not
negotiated, the sender should be disconnected.
=====MSG_PCKG1=====
1. A new inv type (MSG_PCKG1 == 0x6) is added, for use in inv messages
and getdata requests pertaining to version 1 packages.
2. As an inv type, it indicates that both transaction data and version
1 package information are available for the transaction. The
transaction is referenced by its wtxid. As a getdata request type, it
indicates that the sender wants package information for the
transaction.
3. Upon receipt of a "getdata" request for "MSG_PCKG1", the node
should respond with the version 1 package corresponding to the
requested transaction and its current chain tip, or with NOTFOUND.
The node should not assume that the sender is requesting the
transaction data as well.
====Child With Parent Packages Rules====
A child-with-unconfirmed-parents package sent between nodes must abide
by the rules below, otherwise the package is malformed and the sender
should be disconnected.
A version 1 or ''child-with-unconfirmed-parents'' package can be
defined for any transaction that spends unconfirmed inputs. The child
can be thought of as the "representative" of the package. This package
can be uniquely identified by the transaction's wtxid and the current
chain tip block hash.
A ''child-with-unconfirmed-parents'' package MUST be:
1. ''Sorted topologically.'' For every transaction t in the package,
if any of t's parents are present in the package, the parent must
appear somewhere in the list before t. In other words, the
transactions must be sorted in ascending order of the number of
ancestors present in the package.
2. ''Only 1 child with unconfirmed parents.'' The package must consist
of one transaction and its unconfirmed parents. There must not be
any other transactions in the package. Other dependency relationships
may exist within the package (e.g. one parent may spend the output of
another parent) provided that topological order is respected.
3. ''All unconfirmed parents.'' All of the child's unconfirmed parents
must be present.
4. ''No conflicts.'' None of the transactions in the package may
conflict with each other (i.e. spend the same prevout).
5. ''Total fees and weight.'' The 'total_fee' and 'total_weight'
fields must accurately represent the sum total of all transactions'
fees and weights as defined in BIP141, respectively.
Not all of the child's parents must be present; the child transaction
may also spend confirmed inputs. However, if the child has confirmed
parents, they must not be in the package.
While a child-with-unconfirmed-parents package is perhaps most
relevant when the child has a higher feerate than its parents, this
property is not required to construct a valid package.
====Clarifications====
''Q: Under what circumstances should a sender announce a
child-with-unconfirmed-parents package?''
A child-with-unconfirmed-parents package for a transaction should be
announced when it meets the peer's fee filter but one or more of its
parents don't; a "inv(MSG_PCKG1)" instead of "inv(WTX)" should be sent
for the child. Each of the parents which meet the peer's fee filter
should still be announced normally.
''Q: What if a new block arrives in between messages?''
A child-with-unconfirmed-parents package is defined for a transaction
based on the current chain state. As such, a new block extending the
tip may decrease the number of transactions in the package (i.e. if
any of the transaction's parents were included in the block). In a
reorg, the number of transactions in the package may decrease or
increase (i.e. if any of the transaction's parents were included in a
block in the previous chain but not the new one).
If the new block arrives before the "getdata" or "pckginfo1", nothing
needs to change.
If the new block arrives before "getpckgtxns" or before "pckgtxns",
the receiver may need to re-request package information if the block
contained a transaction in the package. If the block doesn't contain
any transactions in the package, whether it extends the previous tip
or causes a reorg, nothing needs to change.
''Q: Can "getpckgtxns" and "pckgtxns" messages contain only one
transaction?''
Yes.
===Further Protocol Extensions===
When introducing a new type of package, assign it a version number "n"
and use an additional "sendpackages" message during version handshake
to negotiate support for it. An additional package information message
"pckginfon" and inv type "MSG_PCKGn" should be defined for the type of
package. However, "getpckgtxns" and "pckgtxns" do not need to be
changed.
Example proposal for tx-with-unconfirmed-ancestors package relay: [19]
===Compatibility===
Older clients remain fully compatible and interoperable after this
change. Clients implementing this protocol will only attempt to send
and request packages if agreed upon during the version handshake.
===Package Erlay===
Clients using BIP330 reconciliation-based transaction relay (Erlay)
are able to use package relay without interference. In fact, a package
of transactions may be announced using both Erlay and package relay.
After reconciliation, if the initiator would have announced a
transaction by wtxid but also has package information for it, they may
send "inv(MSG_PCKG)" instead of "inv(WTX)".
===Rationale===
====P2P Message Design====
These p2p messages are added for communication efficiency and, as
such, one should measure alternative solutions based on the resources
used to communicate (not necessarily trustworthy) information: We
would like to minimize network bandwidth, avoid downloading a
transaction more than once, avoid downloading transactions that are
eventually rejected, and minimize storage allocated for
not-yet-validated transactions.
Consider these (plausible) scenarios in transaction relay:
Alice (the "sender") is relaying transactions to Bob (the "receiver").
Alice's mempool has a minimum feerate of 1sat/vB and Bob's has a
minimum feerate of 3sat/vB. For simplicity, all transactions are
1600Wu in virtual size and 500 bytes in serialized size. Apart from
the spending relationships specified, all other inputs are from
confirmed UTXOs.
1. Package {A, B} where A pays 0 satoshis and B pays 8000 satoshis in
fees.
2. Package {C, D} where C pays 0 satoshis and D pays 1200 satoshis in
fees.
3. Package {E, F, G, H, J} that pays 4000, 8000, 0, 2000, and 4000
satoshis in fees, respectively.
====Alternative Designs Considered====
''Package Information Only:'' Just having "pckginfo" gives enough
information for the receiver to accept the package. Omit the
"getpckgtxns" and "pckgtxns" messages. While this option is a good
fallback if batched transaction download fails for some reason, it
shouldn't be used as the default because it 'always' requires storage
of unvalidated transactions.
''No Package Information Round:'' Instead of having a package
information round, just use the child's wtxid to refer to the package
and always send the entire package together. This would cause nodes to
redownload duplicate transactions.
I have also created a slidedeck exploring various alternative designs
and some examples in which they fall flat [20]. Please feel free to
suggest other alternatives.
====Versioning System====
This protocol should be extensible to support multiple types of
packages based on future desired use cases. Two "flavors" of
versioning were considered:
1. When package mempool acceptance is upgraded to support more types
of packages, increment the version number (similar to Erlay).
During version handshake, peers negotiate which version of package
relay they will use by each sending one "sendpackages" message.
2. When introducing another type of package, assign a version number
to it and announce it as an additional supported version (similar
to Compact Block Relay). During version handshake, peers send one
"sendpackages" message for each version supported.
The second option was favored because it allows different parameters
for different versions. For example, it should be possible to support
both "arbitrary topology but maximum 3-transaction" package as well as
"child-with-unconfirmed-parents with default mempool ancestor limits"
packages simultaneously.
==Acknowledgements==
I hope to have made it abundantly clear that this proposal isn’t
inventing the concept of package relay, and in fact builds upon years
of work by many others, including Suhas Daftuar and Antoine Riard.
Thank you to John Newbery and Martin Zumsande for input on the design.
Thank you to Matt Corallo, Christian Decker, David Harding, Antoine
Poinsot, Antoine Riard, Gregory Sanders, Chris Stewart, Bastien
Teinturier, and others for input on the desired interface for
contracting protocols.
Looking forward to hearing your thoughts!
Best,
Gloria
[0]:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
[1]:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-April/002639.html
[2]:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-June/002758.html
[3]: https://github.com/t-bast/lightning-docs/blob/master/pinning-attacks.md
[4]:
https://github.com/revault/practical-revault/blob/master/transactions.md#cancel_tx
[5]:
https://github.com/discreetlogcontracts/dlcspecs/blob/master/Transactions.md#refund-transaction
[6]: https://gist.github.com/instagibbs/60264606e181451e977e439a49f69fe1
[7]:
https://btctranscripts.com/adopting-bitcoin/2021/2021-11-16-gloria-zhao-transaction-relay-policy/#lightning-attacks
[8]: https://youtu.be/fbWSQvJjKFs?t=1438
[9]:
https://www.reddit.com/r/Bitcoin/comments/unew4e/looks_like_70_mvb_of_transactions_just_got_dumped/
[10]: https://github.com/bitcoin/bitcoin/pull/7594
[11]: https://github.com/bitcoin/bitcoin/pull/7600
[12]: https://github.com/bitcoin/bitcoin/pull/6455#issuecomment-122716820
[13]: https://gist.github.com/sdaftuar/8756699bfcad4d3806ba9f3396d4e66a
[14]: https://github.com/bitcoin/bitcoin/issues/14895
[15]: https://github.com/bitcoin/bitcoin/pull/16401
[16]: https://github.com/bitcoin/bitcoin/pull/19621
[17]:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html
[18]: https://github.com/users/glozow/projects/5/views/4?layout=board
[19]: https://gist.github.com/glozow/9b321cd3ef6505135c763112033ff2a7
[20]:
https://docs.google.com/presentation/d/1B__KlZO1VzxJGx-0DYChlWawaEmGJ9EGApEzrHqZpQc/edit?usp=sharing
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220517/f2e7c128/attachment-0001.html>

2023-06-07 23:03:15 UTC

- reply

📅 Original date posted:2022-02-07
📝 Original message:Hi everyone,
Thanks for giving your attention to the post! I haven't had time to write
responses to everything, but sending my thoughts about what has been most
noteworthy to me:
@jeremy:
> A final point is that a verifiable delay function could be used over,
e.g., each of the N COutpoints individually to rate-limit transaction
replacement. The VDF period can be made shorter / eliminated depending on
the feerate increase.
Thanks for the suggestion! In general, I don't think rate limiting by
outpoint/prevout is a safe option, as it is particularly dangerous for L2
applications with shared prevouts. For example, the prevout that LN channel
counterparties conflict on is the output from their shared funding tx. Any
kind of limit on spending this prevout can be monopolized by a spammy
attacker. For example, if you only allow 1 per minute, the attacker will
just race to take up that slot every minute to prevent the honest party's
transaction from being accepted.
This is similar to the pinning attack based on monopolizing the
transaction's descendant limit, except we can't carve out an exemption
because we wouldn't know whose replacement we're looking at.
@tbast:
> The way I understand it, limiting the impact on descendant transactions
is only important for DoS protection, not for incentive compatibility.
> I believe it's completely ok to require increasing both the fees and
feerate if we don't take descendants into account, because you control your
ancestor set - whereas the descendant set may be completely out of your
control.
Ignoring descendants of direct conflicts would certainly make our lives
much easier! Unfortunately, I don't think we can do this since they can be
fee bumps, i.e., in AJ's example. Considering descendants is important for
both incentive compatibility and DoS.
If the replacement transaction has a higher feerate than its direct
conflict, but the direct conflict also has high feerate descendants, we
might end up with lower fees and/or feerates by accepting the replacement.
@aj:
> I wonder sometimes if it could be sufficient to just have a relay rate
limit and prioritise by ancestor feerate though. Maybe something like:
>
> - instead of adding txs to each peers setInventoryTxToSend immediately,
> set a mempool flag "relayed=false"
>
> - on a time delay, add the top N (by fee rate) "relayed=false" txs to
> each peer's setInventoryTxToSend and mark them as "relayed=true";
> calculate how much kB those txs were, and do this again after
> SIZE/RATELIMIT seconds
>
> - don't include "relayed=false" txs when building blocks?
Wow cool! I think outbound tx relay size-based rate-limiting and
prioritizing tx relay by feerate are great ideas for preventing spammers
from wasting bandwidth network-wide. I agree, this would slow the low
feerate spam down, preventing a huge network-wide bandwidth spike. And it
would allow high feerate transactions to propagate as they should,
regardless of how busy traffic is. Combined with inbound tx request
rate-limiting, might this be sufficient to prevent DoS regardless of the
fee-based replacement policies?
One point that I'm not 100% clear on: is it ok to prioritize the
transactions by ancestor feerate in this scheme? As I described in the
original post, this can be quite different from the actual feerate we would
consider a transaction in a block for. The transaction could have a high
feerate sibling bumping its ancestor.
For example, A (1sat/vB) has 2 children: B (49sat/vB) and C (5sat/vB). If
we just received C, it would be incorrect to give it a priority equal to
its ancestor feerate (3sat/vB) because if we constructed a block template
now, B would bump A, and C's new ancestor feerate is 5sat/vB.
Then, if we imagine that top N is >5sat/vB, we're not relaying C. If we
also exclude C when building blocks, we're missing out on good fees.
> - keep high-feerate evicted txs around for a while in case they get
> mined by someone else to improve compact block relay, a la the
> orphan pool?
Replaced transactions are already added to vExtraTxnForCompact :D
@ariard
> Deployment of Taproot opens interesting possibilities in the
vaults/payment channels design space, where the tapscripts can commit to
different set of timelocks/quorum of keys. Even if the pre-signed states
stay symmetric, whoever is the publisher, the feerate cost to spend can
fluctuate.
Indeed, perhaps with taproot we may legitimately have
same-txid-different-witness transactions as a normal thing rather than rare
edge case. But as with everything enabled by taproot, I wouldn't count our
tapscript eggs until a concrete use case hatches and/or an application
actually implements it.
> How this new replacement rule would behave if you have a parent in the
"replace-by-feerate" half but the child is in the "replace-by-fee" one ?
Thanks for considering my suggestion! This particular scenario is not
possible, since a child cannot be considered for the next block without its
parent. But if the original transactions are found both in and outside the
next block, I think it would be fine to just require both are met.
> Overall, I think there is the deployment issue to warn of. Moving to a
new set of RBF rules implies for a lot of Bitcoin applications to rewrite
their RBF logics.
I agree that transitioning as painlessly as possible would be a huge
priority in any kind of upgrade to mempool policy. I'm very interested in
hearing wallet devs' feedback on this.
I'm also not actually clear on what backwards compatibility in this
scenario would look like. I imagine it to mean we run both sets of RBF
rules and accept the replacement if it passes either one. Or do we only
accept the replacement if it passes both?
For wallets, AJ's "All you need is for there to be *a* path that follows
the new relay rules and gets from your node/wallet to perhaps 10% of
hashpower" makes sense to me (which would be the former). For merchants who
care more about making sure the original transaction isn't replaceable,
would they prefer that either policy is sufficient to prevent a replacement
(more in line with the latter)? Or is that covered by signaling / am I
overthinking this?
Thanks,
Gloria
On Mon, Feb 7, 2022 at 10:24 AM Bastien TEINTURIER via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Good morning,
>
> > The tricky question is what happens when X arrives on its own and it
> > might be that no one ever sends a replacement for B,C,D)
>
> It feels ok to me, but this is definitely arguable.
>
> It covers the fact that B,C,D could have been fake transactions whose
> sole purpose was to do a pinning attack: in that case the attacker would
> have found a way to ensure these transactions don't confirm anyway (or
> pay minimal/negligible fees).
>
> If these transactions were legitimate, I believe that their owners would
> remake them at some point (because these transactions reflect a business
> relationship that needed to happen, so it should very likely still
> happen). It's probably hard to verify because the new corresponding
> transactions may have nothing in common with the first, but I think the
> simplifications it offers for wallets is worth it (which is just my
> opinion and needs more scrutiny/feedback).
>
> > But if your backlog's feerate does drop off, *and* that matters, then
> > I don't think you can ignore the impact of the descendent transactions
> > that you might not get a replacement for.
>
> That is because you're only taking into account the current backlog, and
> not taking into account the fact that new items will be added to it soon
> to replace the evicted descendants. But I agree that this is a bet: we
> can't predict the future and guarantee these replacements will come.
>
> It is really a trade-off, ignoring descendents provides a much simpler
> contract that doesn't vary from one mempool to another, but when your
> backlog isn't full enough, you may lose some future profits if
> transactions don't come in later.
>
> > I think "Y% higher" rather than just "higher" is only useful for
> > rate-limiting, not incentive compatibility. (Though maybe it helps
> > stabilise a greedy algorithm in some cases?)
>
> That's true. I claimed these policies only address incentives, but using
> a percentage increase addresses rate-limiting a bit as well (I couldn't
> resist trying to do at least something for it!). I find it a very easy
> mechanism to implement, while choosing an absolute value is hard (it's
> always easier to think in relatives than absolutes).
>
> > This is why I think it is important to understand the rationales for
> introducing the rules in the first place
>
> I completely agree. As you mentioned, we are still in brainstorming
> phase, once (if?) we start to converge on what could be better policies,
> we do need to clearly explain each policy's expected goal. That will let
> future Bastien writing code in 2030 clearly highlight why the 2022 rules
> don't make sense anymore!
>
> Cheers,
> Bastien
>
> Le sam. 5 févr. 2022 à 14:22, Michael Folkson <
> michaelfolkson at protonmail.com> a écrit :
>
>> Thanks for this Bastien (and Gloria for initially posting about this).
>>
>> I sympathetically skimmed the eclair PR (
>> https://github.com/ACINQ/eclair/pull/2113) dealing with replaceable
>> transactions fee bumping.
>>
>> There will continue to be a (hopefully) friendly tug of war on this
>> probably for the rest of Bitcoin's existence. I am sure people like Luke,
>> Prayank etc will (rightfully) continue to raise that Lightning and other
>> second layer protocols shouldn't demand that policy rules be changed if
>> there is a reason (e.g. DoS vector) for those rules on the base network.
>> But if there are rules that have no upside, introduce unnecessary
>> complexity for no reason and make Lightning implementers like Bastien's
>> life miserable attempting to deal with them I really hope we can make
>> progress on removing or simplifying them.
>>
>> This is why I think it is important to understand the rationales for
>> introducing the rules in the first place (and why it is safe to remove them
>> if indeed it is) and being as rigorous as possible on the rationales for
>> introducing additional rules. It sounds like from Gloria's initial post we
>> are still at a brainstorming phase (which is fine) but knowing what we know
>> today I really hope we can learn from the mistakes of the original BIP 125,
>> namely the Core implementation not matching the BIP and the sparse
>> rationales for the rules. As Bastien says this is not criticizing the
>> original BIP 125 authors, 7 years is a long time especially in Bitcoin
>> world and they probably weren't thinking about Bastien sitting down to
>> write an eclair PR in late 2021 (and reviewers of that PR) when they wrote
>> the BIP in 2015.
>>
>> --
>> Michael Folkson
>> Email: michaelfolkson at protonmail.com
>> Keybase: michaelfolkson
>> PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3
>>
>>
>>
>> ------- Original Message -------
>> On Monday, January 31st, 2022 at 3:57 PM, Bastien TEINTURIER via
>> bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>>
>> Hi Gloria,
>>
>> Many thanks for raising awareness on these issues and constantly pushing
>> towards finding a better model. This work will highly improve the
>> security of any multi-party contract trying to build on top of bitcoin
>> (because most multi-party contracts will need to have timeout conditions
>> and participants will need to make some transactions confirm before a
>> timeout happens - otherwise they may lose funds).
>>
>> For starters, let me quickly explain why the current rules are hard to
>> work with in the context of lightning (but I believe most L2 protocols
>> will have the same issues). Feel free to skip this part if you are
>> already convinced.
>>
>> ## Motivation
>>
>> The biggest pain point is BIP 125 rule 2.
>> If I need to increase the fees of a time-sensitive transaction because
>> the feerate has been rising since I broadcast it, I may need to also pay
>> high fees just to produce a confirmed utxo that I can use. I'm actually
>> paying a high fee twice instead of once (and needlessly using on-chain
>> space, our scarcest asset, because we could have avoided that additional
>> transaction!).
>>
>> It also has some annoying "non-determinism".
>> Imagine that my transaction has been evicted from my mempool because its
>> feerate was too low. I could think "Great, that means I don't have to
>> apply BIP 125 restrictions, I can just fund this transaction as if it
>> were a new one!". But actually I do, because my transaction could still
>> be in miner's mempools and I have no way of knowing it...this means that
>> whenever I have broadcast a transaction, I must assume that I will
>> always need to abide by whatever replacement rules the network applies.
>>
>> Fortunately, as far as I understand it, this rule only exists because of
>> a previous implementation detail of bitcoin core, so there's simply no
>> good reason to keep it.
>>
>> The second biggest pain point is rule 3. It prevents me from efficiently
>> using my capital while it's unconfirmed. Whenever I'm using a big utxo
>> to fund a transaction, I will get a big change output, and it would
>> really be a waste to be unable to use that change output to fund other
>> transactions. In order to be capital-efficient, I will end up creating
>> descendant trees for my time-sensitive transactions. But as Gloria
>> explained, replacing all my children will cost me an absurdly large
>> amount of fees. So what I'm actually planning to do instead is to RBF
>> one of the descendants high enough to get the whole tree confirmed.
>> But if those descendants' timeouts were far in the future, that's a
>> waste, I paid a lot more fees for them than I should have. I'd like to
>> just replace my transaction and republish the invalidated children
>> independently.
>>
>> Rule 4 doesn't hurt as much as the two previous ones, I don't have too
>> much to say about it.
>>
>> To be fair to the BIP 125 authors, all of these scenarios were very hard
>> to forecast at the time this BIP was created. We needed years to build
>> on those rules to get a better understanding of their limitations and if
>> the rationale behind them made sense in the long term.
>>
>> ## Proposals
>>
>> I believe that now is a good time to re-think those, and I really like
>> Gloria's categorization of the design constraints.
>>
>> I'd like to propose a different way of looking at descendants that makes
>> it easier to design the new rules. The way I understand it, limiting the
>> impact on descendant transactions is only important for DoS protection,
>> not for incentive compatibility. I would argue that after evictions,
>> descendant transactions will be submitted again (because they represent
>> transactions that people actually want to make), so evicting them does
>> not have a negative impact on mining incentives (in a world where blocks
>> are full most of the time).
>>
>> I'm curious to hear other people's thoughts on that. If it makes sense,
>> I would propose the following very simple rules:
>>
>> 1. The transaction's ancestor absolute fees must be X% higher than the
>> previous transaction's ancestor fees
>> 2. The transaction's ancestor feerate must be Y% higher than the
>> previous transaction's ancestor feerate
>>
>> I believe it's completely ok to require increasing both the fees and
>> feerate if we don't take descendants into account, because you control
>> your ancestor set - whereas the descendant set may be completely out of
>> your control.
>>
>> This is very easy to use by wallets, because the ancestor set is easy to
>> obtain. And an important point is that the ancestor set is the same in
>> every mempool, whereas the descendant set is not (your mempool may have
>> rejected the last descendants, while other people's mempools may still
>> contain them).
>>
>> Because of that reason, I'd like to avoid having a rule that relies on
>> some size of the replaced descendant set: it may be valid in your
>> mempool but invalid in someone else's, which makes it exploitable for
>> pinning attacks.
>>
>> I believe these rules are incentive compatible (again, if you accept
>> the fact that the descendants will be re-submitted and mined as well,
>> so their fees aren't lost).
>>
>> Can we choose X and Y so that these two rules are also DoS-resistant?
>> Unfortunately I'm not sure, so maybe we'll need to add a third rule to
>> address that. But before we do, can someone detail what it costs for a
>> node to evict a descendant tree? Given that bitcoin core doesn't allow
>> chains of more than 25 transactions, the maximum number of transactions
>> being replaced will be bounded by 25 * N (where N is the number of
>> outputs of the transaction being replaced). If it's just O(n) pruning of
>> a graph, maybe that's ok? Or maybe we make X or Y depend on the number
>> of outputs of the transaction being replaced (this would need very
>> careful thoughts)?
>>
>> If you made it this far, thanks for reading!
>> A couple of comments on the previous messages:
>>
>> > Currently, if we see a transaction
>> > that has the same txid as one in the mempool, we reject it as a
>> > duplicate, even if the feerate is much higher. It's unclear to me if
>> > we have a very strong reason to change this, but noting it as a
>> > limitation of our current replacement policy.
>>
>> I don't see a strong reason from an L2 protocol's point of view yet, but
>> there are many unkown unknowns. But from a miner incentive's point of
>> view, we should keep the transaction with the higher feerate, shouldn't
>> we? In that case it's also a more efficient use of on-chain space, which
>> is a win, right?
>>
>> > We might have a more-or-less long transition period during which we
>> support both...
>>
>> Yes, this is a long term thing.
>> Even if bitcoin core releases a new version with updated RBF rules, as a
>> wallet you'll need to keep using the old rules for a long time if you
>> want to be safe.
>>
>> But it's all the more reason to try to ship this as soon as possible,
>> this way maybe our grand-children will be able to benefit from it ;)
>> (just kidding on the timespan obviously).
>>
>> Cheers,
>> Bastien
>>
>> Le lun. 31 janv. 2022 à 00:11, Antoine Riard via bitcoin-dev <
>> bitcoin-dev at lists.linuxfoundation.org> a écrit :
>>
>>> Hi Gloria,
>>>
>>> Thanks for this RBF sum up. Few thoughts and more context comments if it
>>> can help other readers.
>>>
>>> > For starters, the absolute fee pinning attack is especially
>>> > problematic if we apply the same rules (i.e. Rule #3 and #4) in
>>> > Package RBF. Imagine that Alice (honest) and Bob (adversary) share a
>>> > LN channel. The mempool is rather full, so their pre-negotiated
>>> > commitment transactions' feerates would not be considered high
>>> > priority by miners. Bob broadcasts his commitment transaction and
>>> > attaches a very large child (100KvB with 100,000sat in fees) to his
>>> > anchor output. Alice broadcasts her commitment transaction with a
>>> > fee-bumping child (200vB with 50,000sat fees which is a generous
>>> > 250sat/vB), but this does not meet the absolute fee requirement. She
>>> > would need to add another 50,000sat to replace Bob's commitment
>>> > transaction.
>>>
>>> Solving LN pinning attacks, what we're aiming for is enabling a fair
>>> feerate bid between the counterparties, thus either forcing the adversary
>>> to overbid or to disengage from the confirmation competition. If the
>>> replace-by-feerate rule is adopted, there shouldn't be an incentive for Bob
>>> to
>>> pick up the first option. Though if he does, that's a winning outcome
>>> for Alice, as one of the commitment transactions confirms and her
>>> time-sensitive second-stage HTLC can be subsequently confirmed.
>>>
>>> > It's unclear to me if
>>> > we have a very strong reason to change this, but noting it as a
>>> > limitation of our current replacement policy. See [#24007][12].
>>>
>>> Deployment of Taproot opens interesting possibilities in the
>>> vaults/payment channels design space, where the tapscripts can commit to
>>> different set of timelocks/quorum of keys. Even if the pre-signed states
>>> stay symmetric, whoever is the publisher, the feerate cost to spend can
>>> fluctuate.
>>>
>>> > While this isn't completely broken, and the user interface is
>>> > secondary to the safety of the mempool policy
>>>
>>> I think with L2s transaction broadcast backend, the stability and
>>> clarity of the RBF user interface is primary. What we could be worried
>>> about is a too-much complex interface easing the way for an attacker to
>>> trigger your L2 node to issue policy-invalid chain of transactions.
>>> Especially, when we consider that an attacker might have leverage on chain
>>> of transactions composition ("force broadcast of commitment A then
>>> commitment B, knowing they will share a CPFP") or even transactions size
>>> ("overload commitment A with HTLCs").
>>>
>>> > * If the original transaction is in the top {0.75MvB, 1MvB} of the
>>> > mempool, apply the current rules (absolute fees must increase and
>>> > pay for the replacement transaction's new bandwidth). Otherwise, use a
>>> > feerate-only rule.
>>>
>>> How this new replacement rule would behave if you have a parent in the
>>> "replace-by-feerate" half but the child is in the "replace-by-fee" one ?
>>>
>>> If we allow the replacement of the parent based on the feerate, we might
>>> decrease the top block absolute fees.
>>>
>>> If we block the replacement of the parent based on the feerate because
>>> the replacement absolute fees aren't above the replaced package, we still
>>> preclude a pinning vector. The child might be low-feerate junk and even
>>> attached to a low ancestor-score branch.
>>>
>>> If I'm correct on this limitation, maybe we could turn off the
>>> "replace-by-fee" behavior as soon as the mempool is fulfilled with a few
>>> blocks ?
>>>
>>> > * Rate-limit how many replacements we allow per prevout.
>>>
>>> Depending on how it is implemented, though I would be concerned it
>>> introduces a new pinning vector in the context of shared-utxo. If it's a
>>> hardcoded constant, it could be exhausted by an adversary starting at the
>>> lowest acceptable feerate then slowly increasing while still not reaching
>>> the top of the mempool. Same if it's time-based or block-based, no
>>> guarantee the replacement slot is honestly used by your counterparty.
>>>
>>> Further, an above-the-average replacement frequency might just be the
>>> reflection of your confirmation strategy reacting to block schedule or
>>> mempools historical data. As long as the feerate penalty is paid, I lean to
>>> allow replacement.
>>>
>>> (One solution could be to associate per-user "tag" to the LN
>>> transactions, where each "tag" would have its own replacement slots, but
>>> privacy?)
>>>
>>> > * Rate-limit transaction validation in general, per peer.
>>>
>>> I think we could improve on the Core's new transaction requester logic.
>>> Maybe we could bind the peer announced flow based on the feerate score
>>> (modulo validation time) of the previously validated transactions from that
>>> peer ? That said, while related to RBF, it sounds to me that enhancing
>>> Core's rate-limiting transaction strategy is a whole discussion in itself
>>> [0]. Especially ensuring it's tolerant to the specific requirements of LN &
>>> consorts.
>>>
>>> > What should they be? We can do some arithmetic to see what happens if
>>> > you start with the biggest/lowest feerate transaction and do a bunch
>>> > of replacements. Maybe we end up with values that are high enough to
>>> > prevent abuse and make sense for applications/users that do RBF.
>>>
>>> That's a good question.
>>>
>>> One observation is that the attacker can always renew the set of DoSy
>>> utxos to pursue the attack. So maybe we could pick up constants scaled on
>>> the block size ? That way an attacker would have to burn fees, thus
>>> deterring them from launching an attack. Even if the attackers are miners,
>>> they have to renounce their income to acquire new DoSy utxos. If a low-fee
>>> period, we could scale up the constants ?
>>>
>>>
>>> Overall, I think there is the deployment issue to warn of. Moving to a
>>> new set of RBF rules implies for a lot of Bitcoin applications to rewrite
>>> their RBF logics. We might have a more-or-less long transition period
>>> during which we support both...
>>>
>>> Cheers,
>>> Antoine
>>>
>>> [0] https://github.com/bitcoin/bitcoin/pull/21224
>>>
>>> Le jeu. 27 janv. 2022 à 09:10, Gloria Zhao via bitcoin-dev <
>>> bitcoin-dev at lists.linuxfoundation.org> a écrit :
>>>
>>>> Hi everyone,
>>>>
>>>> This post discusses limitations of current Bitcoin Core RBF policy and
>>>> attempts to start a conversation about how we can improve it,
>>>> summarizing some ideas that have been discussed. Please reply if you
>>>> have any new input on issues to be solved and ideas for improvement!
>>>>
>>>> Just in case I've screwed up the text wrapping again, another copy can
>>>> be
>>>> found here:
>>>> https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff
>>>>
>>>> ## Background
>>>>
>>>> Please feel free to skip this section if you are already familiar
>>>> with RBF.
>>>>
>>>> Nodes may receive *conflicting* unconfirmed transactions, aka
>>>> "double spends" of the same inputs. Instead of always keeping the
>>>> first transaction, since v0.12, Bitcoin Core mempool policy has
>>>> included a set of Replace-by-Fee (RBF) criteria that allows the second
>>>> transaction to replace the first one and any descendants it may have.
>>>>
>>>> Bitcoin Core RBF policy was previously documented as BIP 125.
>>>> The current RBF policy is documented [here][1]. In summary:
>>>>
>>>> 1. The directly conflicting transactions all signal replaceability
>>>> explicitly.
>>>>
>>>> 2. The replacement transaction only includes an unconfirmed input if
>>>> that input was included in one of the directly conflicting
>>>> transactions.
>>>>
>>>> 3. The replacement transaction pays an absolute fee of at least the
>>>> sum paid by the original transactions.
>>>>
>>>> 4. The additional fees pays for the replacement transaction's
>>>> bandwidth at or above the rate set by the node's *incremental relay
>>>> feerate*.
>>>>
>>>> 5. The sum of all directly conflicting transactions' descendant counts
>>>> (number of transactions inclusive of itself and its descendants)
>>>> does not exceed 100.
>>>>
>>>> We can split these rules into 3 categories/goals:
>>>>
>>>> - **Allow Opting Out**: Some applications/businesses are unable to
>>>> handle transactions that are replaceable (e.g. merchants that use
>>>> zero-confirmation transactions). We (try to) help these businesses by
>>>> honoring BIP125 signaling; we won't replace transactions that have not
>>>> opted in.
>>>>
>>>> - **Incentive Compatibility**: Ensure that our RBF policy would not
>>>> accept replacement transactions which would decrease fee profits
>>>> of a miner. In general, if our mempool policy deviates from what is
>>>> economically rational, it's likely that the transactions in our
>>>> mempool will not match the ones in miners' mempools, making our
>>>> fee estimation, compact block relay, and other mempool-dependent
>>>> functions unreliable. Incentive-incompatible policy may also
>>>> encourage transaction submission through routes other than the p2p
>>>> network, harming censorship-resistance and privacy of Bitcoin payments.
>>>>
>>>> - **DoS Protection**: Limit two types of DoS attacks on the node's
>>>> mempool: (1) the number of times a transaction can be replaced and
>>>> (2) the volume of transactions that can be evicted during a
>>>> replacement.
>>>>
>>>> Even more abstract: our goal is to make a replacement policy that
>>>> results in a useful interface for users and safe policy for
>>>> node operators.
>>>>
>>>> ## Motivation
>>>>
>>>> There are a number of known problems with the current RBF policy.
>>>> Many of these shortcomings exist due to mempool limitations at the
>>>> time RBF was implemented or result from new types of Bitcoin usage;
>>>> they are not criticisms of the original design.
>>>>
>>>> ### Pinning Attacks
>>>>
>>>> The most pressing concern is that attackers may take advantage of
>>>> limitations in RBF policy to prevent other users' transactions from
>>>> being mined or getting accepted as a replacement.
>>>>
>>>> #### SIGHASH_ANYONECANPAY Pinning
>>>>
>>>> BIP125#2 can be bypassed by creating intermediary transactions to be
>>>> replaced together. Anyone can simply split a 1-input 1-output
>>>> transaction off from the replacement transaction, then broadcast the
>>>> transaction as is. This can always be done, and quite cheaply. More
>>>> details in [this comment][2].
>>>>
>>>> In general, if a transaction is signed with SIGHASH\_ANYONECANPAY,
>>>> anybody can just attach a low feerate parent to this transaction and
>>>> lower its ancestor feerate. Even if you require SIGHASH\_ALL which
>>>> prevents an attacker from changing any outputs, the input can be a
>>>> very low amount (e.g. just above the dust limit) from a low-fee
>>>> ancestor and still bring down the ancestor feerate of the transaction.
>>>>
>>>> TLDR: if your transaction is signed with SIGHASH\_ANYONECANPAY and
>>>> signals replaceability, regardless of the feerate you broadcast at, an
>>>> attacker can lower its mining priority by adding an ancestor.
>>>>
>>>> #### Absolute Fee
>>>>
>>>> The restriction of requiring replacement transactions to increase the
>>>> absolute fee of the mempool has been described as "bonkers." If the
>>>> original transaction has a very large descendant that pays a large
>>>> amount of fees, even if it has a low feerate, the replacement
>>>> transaction must now pay those fees in order to meet Rule #3.
>>>>
>>>> #### Package RBF
>>>>
>>>> There are a number of reasons why, in order to enable Package RBF, we
>>>> cannot use the same criteria.
>>>>
>>>> For starters, the absolute fee pinning attack is especially
>>>> problematic if we apply the same rules (i.e. Rule #3 and #4) in
>>>> Package RBF. Imagine that Alice (honest) and Bob (adversary) share a
>>>> LN channel. The mempool is rather full, so their pre-negotiated
>>>> commitment transactions' feerates would not be considered high
>>>> priority by miners. Bob broadcasts his commitment transaction and
>>>> attaches a very large child (100KvB with 100,000sat in fees) to his
>>>> anchor output. Alice broadcasts her commitment transaction with a
>>>> fee-bumping child (200vB with 50,000sat fees which is a generous
>>>> 250sat/vB), but this does not meet the absolute fee requirement. She
>>>> would need to add another 50,000sat to replace Bob's commitment
>>>> transaction.
>>>>
>>>> Disallowing new unconfirmed inputs (Rule #2) in Package RBF would be
>>>> broken for packages containing transactions already in the mempool,
>>>> explained [here][7].
>>>>
>>>> Note: I originally [proposed][6] Package RBF using the same Rule #3
>>>> and #4 before I realized how significant this pinning attack is. I'm
>>>> retracting that proposal, and a new set of Package RBF rules would
>>>> follow from whatever the new individual RBF rules end up being.
>>>>
>>>> #### Same Txid Different Witness
>>>>
>>>> Two transactions with the same non-witness data but different
>>>> witnesses have the same txid but different wtxid, and the same fee but
>>>> not necessarily the same feerate. Currently, if we see a transaction
>>>> that has the same txid as one in the mempool, we reject it as a
>>>> duplicate, even if the feerate is much higher. It's unclear to me if
>>>> we have a very strong reason to change this, but noting it as a
>>>> limitation of our current replacement policy. See [#24007][12].
>>>>
>>>> ### User Interface
>>>>
>>>> #### Using Unconfirmed UTXOs to Fund Replacements
>>>>
>>>> The restriction of only allowing confirmed UTXOs for funding a
>>>> fee-bump (Rule #2) can hurt users trying to fee-bump their
>>>> transactions and complicate wallet implementations. If the original
>>>> transaction's output value isn't sufficient to fund a fee-bump and/or
>>>> all of the user's other UTXOs are unconfirmed, they might not be able
>>>> to fund a replacement transaction. Wallet developers also need to
>>>> treat self-owned unconfirmed UTXOs as unusable for fee-bumping, which
>>>> adds complexity to wallet logic. For example, see BDK issues [#144][4]
>>>> and [#414][5].
>>>>
>>>> #### Interface Not Suitable for Coin Selection
>>>>
>>>> Currently, a user cannot simply create a replacement transaction
>>>> targeting a specific feerate or meeting a minimum fee amount and
>>>> expect to meet the RBF criteria. The fee amount depends on the size of
>>>> the replacement transaction, and feerate is almost irrelevant.
>>>>
>>>> Bitcoin Core's `bumpfee` doesn't use the RBF rules when funding the
>>>> replacement. It [estimates][13] a feerate which is "wallet incremental
>>>> relay fee" (a conservative overestimation of the node's incremental
>>>> relay fee) higher than the original transaction, selects coins for
>>>> that feerate, and hopes that it meets the RBF rules. It never fails
>>>> Rule #3 and #4 because it uses all original inputs and refuses to
>>>> bump a transaction with mempool descendants.
>>>>
>>>> This is suboptimal, but is designed to work with the coin selection
>>>> engine: select a feerate first, and then add fees to cover it.
>>>> Following the exact RBF rules would require working the other way
>>>> around: based on how much fees we've added to the transaction and its
>>>> current size, calculate the feerate to see if we meet Rule #4.
>>>>
>>>> While this isn't completely broken, and the user interface is
>>>> secondary to the safety of the mempool policy, we can do much better.
>>>> A much more user-friendly interface would depend *only* on the
>>>> fee and size of the original transactions.
>>>>
>>>> ### Updates to Mempool and Mining
>>>>
>>>> Since RBF was first implemented, a number of improvements have been
>>>> made to mempool and mining logic. For example, we now use ancestor
>>>> feerates in mining (allowing CPFP), and keep track of ancestor
>>>> packages in the mempool.
>>>>
>>>> ## Ideas for Improvements
>>>>
>>>> ### Goals
>>>>
>>>> To summarize, these seem to be desired changes, in order of priority:
>>>>
>>>> 1. Remove Rule #3. The replacement should not be *required* to pay
>>>> higher absolute fees.
>>>>
>>>> 2. Make it impossible for a replacement transaction to have a lower
>>>> mining score than the original transaction(s). This would eliminate
>>>> the `SIGHASH\_ANYONECANPAY` pinning attack.
>>>>
>>>> 3. Remove Rule #2. Adding new unconfirmed inputs should be allowed.
>>>>
>>>> 4. Create a more helpful interface that helps wallet fund replacement
>>>> transactions that aim for a feerate and fee.
>>>>
>>>> ### A Different Model for Fees
>>>>
>>>> For incentive compatibility, I believe there are different
>>>> formulations we should consider. Most importantly, if we want to get
>>>> rid of the absolute fee rule, we can no longer think of it as "the
>>>> transaction needs to pay for its own bandwidth," since we won't always
>>>> be getting additional fees. That means we need a new method of
>>>> rate-limiting replacements that doesn't require additional fees every
>>>> time.
>>>>
>>>> While it makes sense to think about monetary costs when launching a
>>>> specific type of attack, given that the fees are paid to the miner and
>>>> not to the mempool operators, maybe it doesn't make much sense to
>>>> think about "paying for bandwidth". Maybe we should implement
>>>> transaction validation rate-limiting differently, e.g. building it
>>>> into the P2P layer instead of the mempool policy layer.
>>>>
>>>> Recently, Suhas gave a [formulation][8] for incentive compatibility
>>>> that made sense to me: "are the fees expected to be paid in the next
>>>> (N?) blocks higher or lower if we process this transaction?"
>>>>
>>>> I started by thinking about this where N=1 or `1 + p`.
>>>> Here, a rational miner is looking at what fees they would
>>>> collect in the next block, and then some proportion `p` of the rest of
>>>> the blocks based on their hashrate. We're assuming `p` isn't *so high*
>>>> that they would be okay with lower absolute fees in the next 1 block.
>>>> We're also assuming `p` isn't *so low* that the miner doesn't care
>>>> about what's left of the mempool after this block.
>>>>
>>>> A tweak to this formulation is "if we process this transaction, would
>>>> the fees in the next 1 block higher or lower, and is the feerate
>>>> density of the rest of the mempool higher or lower?" This is pretty
>>>> similar, where N=1, but we consider the rest of the mempool by feerate
>>>> rather than fees.
>>>>
>>>> ### Mining Score of a Mempool Transaction
>>>>
>>>> We are often interested in finding out what
>>>> the "mining score" of a transaction in the mempool is. That is, when
>>>> the transaction is considered in block template building, what is the
>>>> feerate it is considered at?
>>>>
>>>> Obviously, it's not the transaction's individual feerate. Bitcoin Core
>>>> [mining code sorts][14] transactions by their ancestor feerate and
>>>> includes them packages at a time, keeping track of how this affects the
>>>> package feerates of remaining transactions in the mempool.
>>>>
>>>> *ancestor feerate*: Ancestor feerate is easily accessible information,
>>>> but it's not accurate either, because it doesn't take into account the
>>>> fact that subsets of a transaction's ancestor set can be included
>>>> without it. For example, ancestors may have high feerates on their own
>>>> or we may have [high feerate siblings][8].
>>>>
>>>> TLDR: *Looking at the current ancestor feerate of a transaction is
>>>> insufficient to tell us what feerate it will be considered at when
>>>> building a block template in the future.*
>>>>
>>>> *min(individual feerate, ancestor feerate)*: Another
>>>> heuristic that is simple to calculate based on current mempool tooling
>>>> is to use the [minimum of a transaction's individual score and its
>>>> ancestor score][10] as a conservative measure. But this can
>>>> overestimate as well (see the example below).
>>>>
>>>> *min ancestor feerate(tx + possible ancestor subsets)* We can also
>>>> take the minimum of every possible ancestor subset, but this can be
>>>> computationally expensive since there can be lots and lots of ancestor
>>>> subsets.
>>>>
>>>> *max ancestor feerate(tx + possible descendant subsets)*: Another idea
>>>> is to use the [maximum ancestor score of the transaction + each of its
>>>> descendants][9]. This doesn't work either; it has the same blindspot
>>>> of ancestor subsets being mined on their own.
>>>>
>>>> #### Mining Score Example
>>>>
>>>> Here's an example illustrating why mining score is tricky to
>>>> efficiently calculate for mempool transactions:
>>>>
>>>> Let's say you have same-size transactions A (21sat/vB), B (1sat/vB),
>>>> C(9sat/vB), D(5sat/vB).
>>>> The layout is: grandparent A, parent B, and two children C and D.
>>>>
>>>> ```
>>>> A
>>>> ^
>>>> B
>>>> ^ ^
>>>> C D
>>>> ```
>>>>
>>>> A miner using ancestor packages to build block templates will first
>>>> include A with a mining score of 21. Next, the miner will include B and
>>>> C with a mining score of 6. This leaves D, with a mining score of 5.
>>>>
>>>> Note: in this case, mining by ancestor feerate results in the most
>>>> rational decisions, but [a candidate set-based approach][10] which
>>>> makes ancestor feerate much less relevant could
>>>> be more advantageous in other situations.
>>>>
>>>> Here is a chart showing the "true" mining score alongside the values
>>>> calculating using imperfect heuristics described above. All of them
>>>> can overestimate or underestimate.
>>>>
>>>> ```
>>>> A B C D
>>>> mining score | 21 | 6 | 6 | 5 |
>>>> ancestor feerate | 21 | 11 | 10.3 | 9 |
>>>> min(individual, ancestor) | 21 | 1 | 9 | 5 |
>>>> min(tx + ancestor subsets) | 21 | 1 | 5 | 3 |
>>>> max(tx + descendants subsets) | 21 | 9 | 9 | 5 |
>>>>
>>>> ```
>>>>
>>>> Possibly the best solution for finding the "mining score" of a
>>>> transaction is to build a block template, see what feerate each
>>>> package is included at. Perhaps at some cutoff, remaining mempool
>>>> transactions can be estimated using some heuristic that leans
>>>> {overestimating, underestimating} depending on the situation.
>>>>
>>>> Mining score seems to be relevant in multiple places: Murch and I
>>>> recently [found][3] that it would be very important in
>>>> "ancestor-aware" funding of transactions (the wallet doesn't
>>>> incorporate ancestor fees when using unconfirmed transactions in coin
>>>> selection, which is a bug we want to fix).
>>>>
>>>> In general, it would be nice to know the exact mining priority of
>>>> one's unconfirmed transaction is. I can think of a few block/mempool
>>>> explorers who might want to display this information for users.
>>>>
>>>> ### RBF Improvement Proposals
>>>>
>>>> After speaking to quite a few people, here are some suggestions
>>>> for improvements that I have heard:
>>>>
>>>> * The ancestor score of the replacement must be {5, 10, N}% higher
>>>> than that of every original transaction.
>>>>
>>>> * The ancestor score of the replacement must be 1sat/vB higher than
>>>> that of every original transaction.
>>>>
>>>> * If the original transaction is in the top {0.75MvB, 1MvB} of the
>>>> mempool, apply the current rules (absolute fees must increase and
>>>> pay for the replacement transaction's new bandwidth). Otherwise, use a
>>>> feerate-only rule.
>>>>
>>>> * If fees don't increase, the size of the replacement transaction must
>>>> decrease by at least N%.
>>>>
>>>> * Rate-limit how many replacements we allow per prevout.
>>>>
>>>> * Rate-limit transaction validation in general, per peer.
>>>>
>>>> Perhaps some others on the mailing list can chime in to throw other
>>>> ideas into the ring and/or combine some of these rules into a sensible
>>>> policy.
>>>>
>>>> #### Replace by Feerate Only
>>>>
>>>> I don't think there's going to be a single-line feerate-based
>>>> rule that can incorporate everything we need.
>>>> On one hand, a feerate-only approach helps eliminate the issues
>>>> associated with Rule #3. On the other hand, I believe the main concern
>>>> with a feerate-only approach is how to rate limit replacements. We
>>>> don't want to enable an attack such as:
>>>>
>>>> 1. Attacker broadcasts large, low-feerate transaction, and attaches a
>>>> chain of descendants.
>>>>
>>>> 2. The attacker replaces the transaction with a smaller but higher
>>>> feerate transaction, attaching a new chain of descendants.
>>>>
>>>> 3. Repeat 1000 times.
>>>>
>>>> #### Fees in Next Block and Feerate for the Rest of the Mempool
>>>>
>>>> Perhaps we can look at replacements like this:
>>>>
>>>> 1. Calculate the directly conflicting transactions and, with their
>>>> descendants, the original transactions. Check signaling. Limit the
>>>> total volume (e.g. can't be more than 100 total or 1MvB or something).
>>>>
>>>> 2. Find which original transactions would be in the next ~1 block. The
>>>> replacement must pay at least this amount + X% in absolute fees. This
>>>> guarantees that the fees of the next block doesn't decrease.
>>>>
>>>> 3. Find which transactions would be left in the mempool after that ~1
>>>> block. The replacement's feerate must be Y% higher than the maximum
>>>> mining score of these transactions. This guarantees that you now have
>>>> only *better* candidates in your after-this-block mempool than you did
>>>> before, even if the size and fees the transactions decrease.
>>>>
>>>> 4. Now you have two numbers: a minimum absolute fee amount and a
>>>> minimum feerate. Check to see if the replacement(s) meet these
>>>> minimums. Also, a wallet would be able to ask the node "What fee and
>>>> feerate would I need to put on a transaction replacing this?" and use
>>>> this information to fund a replacement transaction, without needing to
>>>> guess or overshoot.
>>>>
>>>> Obviously, there are some magic numbers missing here. X and Y are
>>>> TBD constants to ensure we have some kind of rate limiting for the
>>>> number of replacements allowed using some set of fees.
>>>>
>>>> What should they be? We can do some arithmetic to see what happens if
>>>> you start with the biggest/lowest feerate transaction and do a bunch
>>>> of replacements. Maybe we end up with values that are high enough to
>>>> prevent abuse and make sense for applications/users that do RBF.
>>>>
>>>> ### Mempool Changes Need for Implementation
>>>>
>>>> As described in the mining score section above,
>>>> we may want additional tooling to more accurately assess
>>>> the economic gain of replacing transactions in our mempool.
>>>>
>>>> A few options have been discussed:
>>>>
>>>> * Calculate block templates on the fly when we need to consider a
>>>> replacement. However, since replacements are [quite common][11]
>>>> and the information might be useful for other things as well,
>>>> it may be worth it to cache a block template.
>>>>
>>>> * Keep a persistent block template so that we know what transactions
>>>> we would put in the next block. We need to remember the feerate
>>>> at which each transaction was included in the template, because an
>>>> ancestor package may be included in the same block template in
>>>> multiple subsets. Transactions included earlier alter the ancestor
>>>> feerate of the remaining transactions in the package. We also need
>>>> to keep track of the new feerates of transactions left over.
>>>>
>>>> * Divide the mempool into two layers, "high feerate" and "low
>>>> feerate." The high feerate layer contains ~1 block of packages with
>>>> the highest ancestor feerates, and the low feerate layer contains
>>>> everything else. At the edge of a block, we have a Knapsacky problem
>>>> where the next highest ancestor feerate package might not fit, so we
>>>> would probably want the high feerate layer ~2MvB or something to avoid
>>>> underestimating the fees.
>>>>
>>>> ## Acknowledgements
>>>>
>>>> Thank you to everyone whose RBF-related suggestions, grievances,
>>>> criticisms and ideas were incorporated in this document:
>>>> Andrew Chow, Matt Corallo, Suhas Daftuar, Christian Decker,
>>>> Mark Erhardt, Lloyd Fournier, Lisa Neigut, John Newbery,
>>>> Antoine Poinsot, Antoine Riard, Larry Ruane,
>>>> S3RK and Bastien Teinturier.
>>>>
>>>> Thanks for reading!
>>>>
>>>> Best,
>>>> Gloria
>>>>
>>>> [1]:
>>>> https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md
>>>> [2]:
>>>> https://github.com/bitcoin/bitcoin/pull/23121#issuecomment-929475999
>>>> [3]:
>>>> https://github.com/Xekyo/bitcoin/commit/d754b0242ec69d42c570418aebf9c1335af0b8ea
>>>> [4]: https://github.com/bitcoindevkit/bdk/issues/144
>>>> [5]: https://github.com/bitcoindevkit/bdk/issues/414
>>>> [6]:
>>>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html
>>>> [7]:
>>>> https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a#new-unconfirmed-inputs-rule-2
>>>> [8]:
>>>> https://github.com/bitcoin/bitcoin/pull/23121#discussion_r777131366
>>>> [9]:
>>>> https://github.com/bitcoin/bitcoin/pull/22290#issuecomment-865887922
>>>> [10]:
>>>> https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md
>>>> [11]:
>>>> https://github.com/bitcoin/bitcoin/pull/22539#issuecomment-885763670
>>>> [12]: https://github.com/bitcoin/bitcoin/pull/24007
>>>> [13]:
>>>> https://github.com/bitcoin/bitcoin/blob/1a369f006fd0bec373b95001ed84b480e852f191/src/wallet/feebumper.cpp#L114
>>>> [14]:
>>>> https://github.com/bitcoin/bitcoin/blob/cf5bb048e80d4cde8828787b266b7f5f2e3b6d7b/src/node/miner.cpp#L310-L320
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev at lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev at lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>
>> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220207/34141dc7/attachment-0001.html>

2023-06-07 23:03:00 UTC

- reply

📅 Original date posted:2022-01-27
📝 Original message:Hi everyone,
This post discusses limitations of current Bitcoin Core RBF policy and
attempts to start a conversation about how we can improve it,
summarizing some ideas that have been discussed. Please reply if you
have any new input on issues to be solved and ideas for improvement!
Just in case I've screwed up the text wrapping again, another copy can be
found here: https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff
## Background
Please feel free to skip this section if you are already familiar
with RBF.
Nodes may receive *conflicting* unconfirmed transactions, aka
"double spends" of the same inputs. Instead of always keeping the
first transaction, since v0.12, Bitcoin Core mempool policy has
included a set of Replace-by-Fee (RBF) criteria that allows the second
transaction to replace the first one and any descendants it may have.
Bitcoin Core RBF policy was previously documented as BIP 125.
The current RBF policy is documented [here][1]. In summary:
1. The directly conflicting transactions all signal replaceability
explicitly.
2. The replacement transaction only includes an unconfirmed input if
that input was included in one of the directly conflicting
transactions.
3. The replacement transaction pays an absolute fee of at least the
sum paid by the original transactions.
4. The additional fees pays for the replacement transaction's
bandwidth at or above the rate set by the node's *incremental relay
feerate*.
5. The sum of all directly conflicting transactions' descendant counts
(number of transactions inclusive of itself and its descendants)
does not exceed 100.
We can split these rules into 3 categories/goals:
- **Allow Opting Out**: Some applications/businesses are unable to
handle transactions that are replaceable (e.g. merchants that use
zero-confirmation transactions). We (try to) help these businesses by
honoring BIP125 signaling; we won't replace transactions that have not
opted in.
- **Incentive Compatibility**: Ensure that our RBF policy would not
accept replacement transactions which would decrease fee profits
of a miner. In general, if our mempool policy deviates from what is
economically rational, it's likely that the transactions in our
mempool will not match the ones in miners' mempools, making our
fee estimation, compact block relay, and other mempool-dependent
functions unreliable. Incentive-incompatible policy may also
encourage transaction submission through routes other than the p2p
network, harming censorship-resistance and privacy of Bitcoin payments.
- **DoS Protection**: Limit two types of DoS attacks on the node's
mempool: (1) the number of times a transaction can be replaced and
(2) the volume of transactions that can be evicted during a
replacement.
Even more abstract: our goal is to make a replacement policy that
results in a useful interface for users and safe policy for
node operators.
## Motivation
There are a number of known problems with the current RBF policy.
Many of these shortcomings exist due to mempool limitations at the
time RBF was implemented or result from new types of Bitcoin usage;
they are not criticisms of the original design.
### Pinning Attacks
The most pressing concern is that attackers may take advantage of
limitations in RBF policy to prevent other users' transactions from
being mined or getting accepted as a replacement.
#### SIGHASH_ANYONECANPAY Pinning
BIP125#2 can be bypassed by creating intermediary transactions to be
replaced together. Anyone can simply split a 1-input 1-output
transaction off from the replacement transaction, then broadcast the
transaction as is. This can always be done, and quite cheaply. More
details in [this comment][2].
In general, if a transaction is signed with SIGHASH\_ANYONECANPAY,
anybody can just attach a low feerate parent to this transaction and
lower its ancestor feerate. Even if you require SIGHASH\_ALL which
prevents an attacker from changing any outputs, the input can be a
very low amount (e.g. just above the dust limit) from a low-fee
ancestor and still bring down the ancestor feerate of the transaction.
TLDR: if your transaction is signed with SIGHASH\_ANYONECANPAY and
signals replaceability, regardless of the feerate you broadcast at, an
attacker can lower its mining priority by adding an ancestor.
#### Absolute Fee
The restriction of requiring replacement transactions to increase the
absolute fee of the mempool has been described as "bonkers." If the
original transaction has a very large descendant that pays a large
amount of fees, even if it has a low feerate, the replacement
transaction must now pay those fees in order to meet Rule #3.
#### Package RBF
There are a number of reasons why, in order to enable Package RBF, we
cannot use the same criteria.
For starters, the absolute fee pinning attack is especially
problematic if we apply the same rules (i.e. Rule #3 and #4) in
Package RBF. Imagine that Alice (honest) and Bob (adversary) share a
LN channel. The mempool is rather full, so their pre-negotiated
commitment transactions' feerates would not be considered high
priority by miners. Bob broadcasts his commitment transaction and
attaches a very large child (100KvB with 100,000sat in fees) to his
anchor output. Alice broadcasts her commitment transaction with a
fee-bumping child (200vB with 50,000sat fees which is a generous
250sat/vB), but this does not meet the absolute fee requirement. She
would need to add another 50,000sat to replace Bob's commitment
transaction.
Disallowing new unconfirmed inputs (Rule #2) in Package RBF would be
broken for packages containing transactions already in the mempool,
explained [here][7].
Note: I originally [proposed][6] Package RBF using the same Rule #3
and #4 before I realized how significant this pinning attack is. I'm
retracting that proposal, and a new set of Package RBF rules would
follow from whatever the new individual RBF rules end up being.
#### Same Txid Different Witness
Two transactions with the same non-witness data but different
witnesses have the same txid but different wtxid, and the same fee but
not necessarily the same feerate. Currently, if we see a transaction
that has the same txid as one in the mempool, we reject it as a
duplicate, even if the feerate is much higher. It's unclear to me if
we have a very strong reason to change this, but noting it as a
limitation of our current replacement policy. See [#24007][12].
### User Interface
#### Using Unconfirmed UTXOs to Fund Replacements
The restriction of only allowing confirmed UTXOs for funding a
fee-bump (Rule #2) can hurt users trying to fee-bump their
transactions and complicate wallet implementations. If the original
transaction's output value isn't sufficient to fund a fee-bump and/or
all of the user's other UTXOs are unconfirmed, they might not be able
to fund a replacement transaction. Wallet developers also need to
treat self-owned unconfirmed UTXOs as unusable for fee-bumping, which
adds complexity to wallet logic. For example, see BDK issues [#144][4]
and [#414][5].
#### Interface Not Suitable for Coin Selection
Currently, a user cannot simply create a replacement transaction
targeting a specific feerate or meeting a minimum fee amount and
expect to meet the RBF criteria. The fee amount depends on the size of
the replacement transaction, and feerate is almost irrelevant.
Bitcoin Core's `bumpfee` doesn't use the RBF rules when funding the
replacement. It [estimates][13] a feerate which is "wallet incremental
relay fee" (a conservative overestimation of the node's incremental
relay fee) higher than the original transaction, selects coins for
that feerate, and hopes that it meets the RBF rules. It never fails
Rule #3 and #4 because it uses all original inputs and refuses to
bump a transaction with mempool descendants.
This is suboptimal, but is designed to work with the coin selection
engine: select a feerate first, and then add fees to cover it.
Following the exact RBF rules would require working the other way
around: based on how much fees we've added to the transaction and its
current size, calculate the feerate to see if we meet Rule #4.
While this isn't completely broken, and the user interface is
secondary to the safety of the mempool policy, we can do much better.
A much more user-friendly interface would depend *only* on the
fee and size of the original transactions.
### Updates to Mempool and Mining
Since RBF was first implemented, a number of improvements have been
made to mempool and mining logic. For example, we now use ancestor
feerates in mining (allowing CPFP), and keep track of ancestor
packages in the mempool.
## Ideas for Improvements
### Goals
To summarize, these seem to be desired changes, in order of priority:
1. Remove Rule #3. The replacement should not be *required* to pay
higher absolute fees.
2. Make it impossible for a replacement transaction to have a lower
mining score than the original transaction(s). This would eliminate
the `SIGHASH\_ANYONECANPAY` pinning attack.
3. Remove Rule #2. Adding new unconfirmed inputs should be allowed.
4. Create a more helpful interface that helps wallet fund replacement
transactions that aim for a feerate and fee.
### A Different Model for Fees
For incentive compatibility, I believe there are different
formulations we should consider. Most importantly, if we want to get
rid of the absolute fee rule, we can no longer think of it as "the
transaction needs to pay for its own bandwidth," since we won't always
be getting additional fees. That means we need a new method of
rate-limiting replacements that doesn't require additional fees every
time.
While it makes sense to think about monetary costs when launching a
specific type of attack, given that the fees are paid to the miner and
not to the mempool operators, maybe it doesn't make much sense to
think about "paying for bandwidth". Maybe we should implement
transaction validation rate-limiting differently, e.g. building it
into the P2P layer instead of the mempool policy layer.
Recently, Suhas gave a [formulation][8] for incentive compatibility
that made sense to me: "are the fees expected to be paid in the next
(N?) blocks higher or lower if we process this transaction?"
I started by thinking about this where N=1 or `1 + p`.
Here, a rational miner is looking at what fees they would
collect in the next block, and then some proportion `p` of the rest of
the blocks based on their hashrate. We're assuming `p` isn't *so high*
that they would be okay with lower absolute fees in the next 1 block.
We're also assuming `p` isn't *so low* that the miner doesn't care
about what's left of the mempool after this block.
A tweak to this formulation is "if we process this transaction, would
the fees in the next 1 block higher or lower, and is the feerate
density of the rest of the mempool higher or lower?" This is pretty
similar, where N=1, but we consider the rest of the mempool by feerate
rather than fees.
### Mining Score of a Mempool Transaction
We are often interested in finding out what
the "mining score" of a transaction in the mempool is. That is, when
the transaction is considered in block template building, what is the
feerate it is considered at?
Obviously, it's not the transaction's individual feerate. Bitcoin Core
[mining code sorts][14] transactions by their ancestor feerate and
includes them packages at a time, keeping track of how this affects the
package feerates of remaining transactions in the mempool.
*ancestor feerate*: Ancestor feerate is easily accessible information,
but it's not accurate either, because it doesn't take into account the
fact that subsets of a transaction's ancestor set can be included
without it. For example, ancestors may have high feerates on their own
or we may have [high feerate siblings][8].
TLDR: *Looking at the current ancestor feerate of a transaction is
insufficient to tell us what feerate it will be considered at when
building a block template in the future.*
*min(individual feerate, ancestor feerate)*: Another
heuristic that is simple to calculate based on current mempool tooling
is to use the [minimum of a transaction's individual score and its
ancestor score][10] as a conservative measure. But this can
overestimate as well (see the example below).
*min ancestor feerate(tx + possible ancestor subsets)* We can also
take the minimum of every possible ancestor subset, but this can be
computationally expensive since there can be lots and lots of ancestor
subsets.
*max ancestor feerate(tx + possible descendant subsets)*: Another idea
is to use the [maximum ancestor score of the transaction + each of its
descendants][9]. This doesn't work either; it has the same blindspot
of ancestor subsets being mined on their own.
#### Mining Score Example
Here's an example illustrating why mining score is tricky to
efficiently calculate for mempool transactions:
Let's say you have same-size transactions A (21sat/vB), B (1sat/vB),
C(9sat/vB), D(5sat/vB).
The layout is: grandparent A, parent B, and two children C and D.
```
A
^
B
^ ^
C D
```
A miner using ancestor packages to build block templates will first
include A with a mining score of 21. Next, the miner will include B and
C with a mining score of 6. This leaves D, with a mining score of 5.
Note: in this case, mining by ancestor feerate results in the most
rational decisions, but [a candidate set-based approach][10] which
makes ancestor feerate much less relevant could
be more advantageous in other situations.
Here is a chart showing the "true" mining score alongside the values
calculating using imperfect heuristics described above. All of them
can overestimate or underestimate.
```
A B C D
mining score | 21 | 6 | 6 | 5 |
ancestor feerate | 21 | 11 | 10.3 | 9 |
min(individual, ancestor) | 21 | 1 | 9 | 5 |
min(tx + ancestor subsets) | 21 | 1 | 5 | 3 |
max(tx + descendants subsets) | 21 | 9 | 9 | 5 |
```
Possibly the best solution for finding the "mining score" of a
transaction is to build a block template, see what feerate each
package is included at. Perhaps at some cutoff, remaining mempool
transactions can be estimated using some heuristic that leans
{overestimating, underestimating} depending on the situation.
Mining score seems to be relevant in multiple places: Murch and I
recently [found][3] that it would be very important in
"ancestor-aware" funding of transactions (the wallet doesn't
incorporate ancestor fees when using unconfirmed transactions in coin
selection, which is a bug we want to fix).
In general, it would be nice to know the exact mining priority of
one's unconfirmed transaction is. I can think of a few block/mempool
explorers who might want to display this information for users.
### RBF Improvement Proposals
After speaking to quite a few people, here are some suggestions
for improvements that I have heard:
* The ancestor score of the replacement must be {5, 10, N}% higher
than that of every original transaction.
* The ancestor score of the replacement must be 1sat/vB higher than
that of every original transaction.
* If the original transaction is in the top {0.75MvB, 1MvB} of the
mempool, apply the current rules (absolute fees must increase and
pay for the replacement transaction's new bandwidth). Otherwise, use a
feerate-only rule.
* If fees don't increase, the size of the replacement transaction must
decrease by at least N%.
* Rate-limit how many replacements we allow per prevout.
* Rate-limit transaction validation in general, per peer.
Perhaps some others on the mailing list can chime in to throw other
ideas into the ring and/or combine some of these rules into a sensible
policy.
#### Replace by Feerate Only
I don't think there's going to be a single-line feerate-based
rule that can incorporate everything we need.
On one hand, a feerate-only approach helps eliminate the issues
associated with Rule #3. On the other hand, I believe the main concern
with a feerate-only approach is how to rate limit replacements. We
don't want to enable an attack such as:
1. Attacker broadcasts large, low-feerate transaction, and attaches a
chain of descendants.
2. The attacker replaces the transaction with a smaller but higher
feerate transaction, attaching a new chain of descendants.
3. Repeat 1000 times.
#### Fees in Next Block and Feerate for the Rest of the Mempool
Perhaps we can look at replacements like this:
1. Calculate the directly conflicting transactions and, with their
descendants, the original transactions. Check signaling. Limit the
total volume (e.g. can't be more than 100 total or 1MvB or something).
2. Find which original transactions would be in the next ~1 block. The
replacement must pay at least this amount + X% in absolute fees. This
guarantees that the fees of the next block doesn't decrease.
3. Find which transactions would be left in the mempool after that ~1
block. The replacement's feerate must be Y% higher than the maximum
mining score of these transactions. This guarantees that you now have
only *better* candidates in your after-this-block mempool than you did
before, even if the size and fees the transactions decrease.
4. Now you have two numbers: a minimum absolute fee amount and a
minimum feerate. Check to see if the replacement(s) meet these
minimums. Also, a wallet would be able to ask the node "What fee and
feerate would I need to put on a transaction replacing this?" and use
this information to fund a replacement transaction, without needing to
guess or overshoot.
Obviously, there are some magic numbers missing here. X and Y are
TBD constants to ensure we have some kind of rate limiting for the
number of replacements allowed using some set of fees.
What should they be? We can do some arithmetic to see what happens if
you start with the biggest/lowest feerate transaction and do a bunch
of replacements. Maybe we end up with values that are high enough to
prevent abuse and make sense for applications/users that do RBF.
### Mempool Changes Need for Implementation
As described in the mining score section above,
we may want additional tooling to more accurately assess
the economic gain of replacing transactions in our mempool.
A few options have been discussed:
* Calculate block templates on the fly when we need to consider a
replacement. However, since replacements are [quite common][11]
and the information might be useful for other things as well,
it may be worth it to cache a block template.
* Keep a persistent block template so that we know what transactions
we would put in the next block. We need to remember the feerate
at which each transaction was included in the template, because an
ancestor package may be included in the same block template in
multiple subsets. Transactions included earlier alter the ancestor
feerate of the remaining transactions in the package. We also need
to keep track of the new feerates of transactions left over.
* Divide the mempool into two layers, "high feerate" and "low
feerate." The high feerate layer contains ~1 block of packages with
the highest ancestor feerates, and the low feerate layer contains
everything else. At the edge of a block, we have a Knapsacky problem
where the next highest ancestor feerate package might not fit, so we
would probably want the high feerate layer ~2MvB or something to avoid
underestimating the fees.
## Acknowledgements
Thank you to everyone whose RBF-related suggestions, grievances,
criticisms and ideas were incorporated in this document:
Andrew Chow, Matt Corallo, Suhas Daftuar, Christian Decker,
Mark Erhardt, Lloyd Fournier, Lisa Neigut, John Newbery,
Antoine Poinsot, Antoine Riard, Larry Ruane,
S3RK and Bastien Teinturier.
Thanks for reading!
Best,
Gloria
[1]:
https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md
[2]: https://github.com/bitcoin/bitcoin/pull/23121#issuecomment-929475999
[3]:
https://github.com/Xekyo/bitcoin/commit/d754b0242ec69d42c570418aebf9c1335af0b8ea
[4]: https://github.com/bitcoindevkit/bdk/issues/144
[5]: https://github.com/bitcoindevkit/bdk/issues/414
[6]:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html
[7]:
https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a#new-unconfirmed-inputs-rule-2
[8]: https://github.com/bitcoin/bitcoin/pull/23121#discussion_r777131366
[9]: https://github.com/bitcoin/bitcoin/pull/22290#issuecomment-865887922
[10]:
https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md
[11]: https://github.com/bitcoin/bitcoin/pull/22539#issuecomment-885763670
[12]: https://github.com/bitcoin/bitcoin/pull/24007
[13]:
https://github.com/bitcoin/bitcoin/blob/1a369f006fd0bec373b95001ed84b480e852f191/src/wallet/feebumper.cpp#L114
[14]:
https://github.com/bitcoin/bitcoin/blob/cf5bb048e80d4cde8828787b266b7f5f2e3b6d7b/src/node/miner.cpp#L310-L320
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220127/ec8ff818/attachment-0001.html>

2023-06-07 23:00:54 UTC

- reply

📅 Original date posted:2021-12-07
📝 Original message:Hi Darosior and Ariard,
Thank you for your work looking into fee-bumping so thoroughly, and for
sharing your results. I agree about fee-bumping's importance in contract
security and feel that it's often under-prioritized. In general, what
you've described in this post, to me, is strong motivation for some of the
proposed changes to RBF we've been discussing. Mostly, I have some
questions.
> The part of Revault we are interested in for this study is the delegation
process, and more
> specifically the application of spending policies by network monitors
(watchtowers).
I'd like to better understand how fee-bumping would be used, i.e. how the
watchtower model works:
- Do all of the vault parties both deposit to the vault and a refill/fee to
the watchtower, is there a reward the watchtower collects for a successful
Cancel, or something else? (Apologies if there's a thorough explanation
somewhere that I haven't already seen).
- Do we expect watchtowers tracking multiple vaults to be batching multiple
Cancel transaction fee-bumps?
- Do we expect vault users to be using multiple watchtowers for a better
trust model? If so, and we're expecting batched fee-bumps, won't those
conflict?
> For Revault we can afford to introduce malleability in the Cancel
transaction since there is no
> second-stage transaction depending on its txid. Therefore it is
pre-signed with ANYONECANPAY. We
> can't use ANYONECANPAY|SINGLE since it would open a pinning vector [3].
Note how we can't leverage
> the carve out rule, and neither can any other more-than-two-parties
contract.
We've already talked about this offline, but I'd like to point out here
that even transactions signed with ANYONECANPAY|ALL can be pinned by RBF
unless we add an ancestor score rule. [0], [1] (numbers are inaccurate,
Cancel Tx feerates wouldn't be that low, but just to illustrate what the
attack would look like)
[0]:
https://user-images.githubusercontent.com/25183001/135104603-9e775062-5c8d-4d55-9bc9-6e9db92cfe6d.png
[1]:
https://user-images.githubusercontent.com/25183001/145044333-2f85da4a-af71-44a1-bc21-30c388713a0d.png
> can't use ANYONECANPAY|SINGLE since it would open a pinning vector [3].
Note how we can't leverage
> the carve out rule, and neither can any other more-than-two-parties
contract.
Well stated about CPFP carve out. I suppose the generalization is that
allowing n extra ancestorcount=2 descendants to a transaction means it can
help contracts with <=n+1 parties (more accurately, outputs)? I wonder if
it's possible to devise a different approach for limiting
ancestors/descendants, e.g. by height/width/branching factor of the family
instead of count... :shrug:
> You could keep a single large UTxO and peel it as you need to sponsor
transactions. But this means
> that you need to create a coin of a specific value according to your need
at the current feerate
> estimation, hope to have it confirmed in a few blocks (at least for now!
[5]), and hope that the
> value won't be obsolete by the time it confirmed.
IIUC, a Cancel transaction can be generalized as a 1-in-1-out where the
input is presigned with counterparties, SIGHASH_ANYONECANPAY. The fan-out
UTXO pool approach is a clever solution. I also think this smells like a
case where improving lower-level RBF rules is more appropriate than
requiring applications to write workarounds and generate extra
transactions. Seeing that the BIP125#2 (no new unconfirmed inputs)
restriction really hurts in this case, if that rule were removed, would you
be able to simply keep the 1 big UTXO per vault and cut out the exact
nValue you need to fee-bump Cancel transactions? Would that feel less like
"burning" for the sake of fee-bumping?
> First of all, when to fee-bump? At fixed time intervals? At each block
connection? It sounds like,
> given a large enough timelock, you could try to greed by "trying your
luck" at a lower feerate and
> only re-bumping every N blocks. You would then start aggressively bumping
at every block after M
> blocks have passed.
I'm wondering if you also considered other questions like:
- Should a fee-bumping strategy be dependent upon the rate of incoming
transactions? To me, it seems like the two components are (1) what's in the
mempool and (2) what's going to trickle into the mempool between now and
the target block. The first component is best-effort keeping
incentive-compatible mempool; historical data and crystal ball look like
the only options for incorporating the 2nd component.
- Should the fee-bumping strategy depend on how close you are to your
timelock expiry? (though this seems like a potential privacy leak, and the
game theory could get weird as you mentioned).
- As long as you have a good fee estimator (i.e. given a current mempool,
can get an accurate feerate given a % probability of getting into target
block n), is there any reason to devise a fee-bumping strategy beyond
picking a time interval?
It would be interesting to see stats on the spread of feerates in blocks
during periods of fee fluctuation.
> > In the event that you notice a consequent portion of the block is
filled with transactions paying
> > less than your own, you might want to start panicking and bump your
transaction fees by a certain
> > percentage with no consideration for your fee estimator. You might skew
miners incentives in doing
> > so: if you increase the fees by a factor of N, any miner with a
fraction larger than 1/N of the
> > network hashrate now has an incentive to censor your transaction at
first to get you to panic.
> Yes I think miner-harvesting attacks should be weighed carefully in the
design of offchain contracts fee-bumping strategies, at least in the future
when the mining reward exhausts further.
Miner-harvesting (such cool naming!) is interesting, but I want to clarify
the value of N - I don't think it's the factor by which you increase the
fees on just your transaction.
To codify: your transaction pays a fee of `f1` right now and might pay a
fee of `f2` in a later block that the miner expects to mine with 1/N
probability. The economically rational miner isn't incentivized if simply
`f2 = N * f1` unless their mempool is otherwise empty.
By omitting your transaction in this block, the miner can include another
transaction/package paying `g1` fees instead, so they lose `f1-g1` in fees
right now. In the future block, they have the choice between collecting
`f2` or `g2` (from another transaction/package) in fees, so their gain is
`max(f2-g2, 0)`.
So the equation is more like: a miner with 1/N of the hashrate, employing
this censorship strategy, gains only if `max(f2-g2, 0) > N * (f1-g1)`. More
broadly, the miner only profits if `f2` is significantly higher than `g2`
and `f1` is about the same feerate as everything else in your mempool: it
seems like they're betting on how much you _overshoot_, not how much you
bump.
In general, I agree it would really suck to inadvertently create a game
where miners can drive feerates up by triggering desperation-driven
fee-bumping procedures. I guess this is a reason to avoid
increasingly-aggressive feebumping, or strategies where we predictably
overshoot.
Slightly related question: in contracts, generally, the timelock deadline
is revealed in the script, so the miner knows how "desperate" we are right?
Is that a problem? For Revault, if your Cancel transaction is a keypath
spend (I think I remember reading that somewhere?) and you don't reveal the
script, they don't see your timelock deadline yes?
Again, thanks for the digging and sharing. :)
Best,
Gloria
On Tue, Nov 30, 2021 at 3:27 PM darosior via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Hi Antoine,
>
> Thanks for your comment. I believe for Lightning it's simpler with regard
> to the management of the UTxO pool, but harder with regard to choosing
> a threat model.
> Responses inline.
>
>
> For any opened channel, ensure the confirmation of a Commitment
> transaction and the children HTLC-Success/HTLC-Timeout transactions. Note,
> in the Lightning security game you have to consider (at least) 4 types of
> players moves and incentives : your node, your channel counterparties, the
> miners, the crowd of bitcoin users. The number of the last type of players
> is unknown from your node, however it should not be forgotten you're in
> competition for block space, therefore their block demands bids should be
> anticipated and reacted to in consequence. With that remark in mind,
> implications for your LN fee-bumping strategy will be raised afterwards.
>
> For a LN service provider, on-chain overpayments are bearing on your
> operational costs, thus downgrading your economic competitiveness. For the
> average LN user, overpayment might price out outside a LN non-custodial
> deployment, as you don't have the minimal security budget to be on your own.
>
>
> I think this problem statement can be easily generalised to any offchain
> contract. And your points stand for all of them.
> "For any opened contract, ensure at any point the confirmation of a (set
> of) transaction(s) in a given number of blocks"
>
>
> Same issue with Lightning, we can be pinned today on the basis of
> replace-by-fee rule 3. We can be also blinded by network mempool
> partitions, a pinning counterparty can segregate all the full-nodes in as
> many subsets by broadcasting a revoked Commitment transaction different for
> each. For Revault, I think you can also do unlimited partitions by mutating
> the ANYONECANPAY-input of the Cancel.
>
>
> Well you can already do unlimited partitions by adding different inputs to
> it. You could malleate the witness, but since we are using Miniscript i'm
> confident you would only be able in a marginal way.
>
>
> That said, if you have a distributed towers deployment, spread across the
> p2p network topology, and they can't be clustered together through
> cross-layers or intra-layer heuristics, you should be able to reliably
> observe such partitions. I think such distributed monitors are deployed by
> few L1 merchants accepting 0-conf to detect naive double-spend.
>
>
> We should aim to more than 0-conf (in)security level..
> It seems to me the only policy-level mitigation for RBF pinning around the
> "don't decrease the abolute fees of a less-than-a-block mempool" would be
> to drop the requirement on increasing absolute fees if the mempool is "full
> enough" (and the feerate increases exponentially, of course).
> Another approach could be by introducing new consensus rules as proposed
> by Jeremy last year [0]. If we go in the realm of new consensus rules, then
> i think that simply committing to a maximum tx size would fix pinning by
> RBF rule 3. Could be in the annex, or in the unused sequence bits (although
> they currently are by Lightning, meh). You could also check in the output
> script that the input commits to this.
>
> [0]
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-September/018168.html
>
>
> Have we already discussed a fee-bumping "shared cache", a CPFP variation ?
> Strawman idea: Alice and Bob commit collateral inputs to a separate UTXO
> from the main "offchain contract" one. This UTXO is locked by a multi-sig.
> For any Commitment transaction pre-signed, also counter-sign a CPFP with
> top mempool feerate included, spending a Commitment anchor output and the
> shared-cache UTXO. If the fees spike, you can re-sign a high-feerate CPFP,
> assuming interactivity. As the CPFP is counter-signed by everyone, the
> outputs can be CSV-1 encumbered to prevent pinnings. If the share-cache is
> feeded at parity, there shouldn't be an incentive to waste or maliciously
> inflate the feerate. I think this solution can be easily generalized to
> more than 2 counterparties by using a multi-signature scheme. Big issue, if
> the feerate is short due to fee spikes and you need to re-sign a
> higher-feerate CPFP, you're trusting your counterparty to interact, though
> arguably not worse than the current update fee mechanism.
>
>
> It really looks just like `update_fee`. Except maybe with the property
> that you have the channel liquidity not depend on the onchain feerate.
> In any case, for Lightning i think it's a bad idea to re-introduce trust
> on this side post anchor outputs. For Revault it's clearly out of the
> question to introduce trust in your counterparties (why would you bother
> having a fee-bumping mechanism in the first place then?). Probably the same
> holds for all offchain contracts.
>
>
> > For Lightning, it'd mean keeping an equivalent amount of funds as the
> sum of all your
> channels balances sitting there unallocated "just in case". This is not
> reasonable.
>
> Agree, game-theory wise, you would like to keep a full fee-bumping
> reserve, ready to burn as much in fees as the contested HTLC value, as it's
> the maximum gain of your counterparty. Though perfect equilibrium is hard
> to achieve because your malicious counterparty might have an edge pushing
> you to broadcast your Commitment first by witholding HTLC resolution.
>
> Fractional fee-bumping reserves are much more realistic to expect in the
> LN network. Lower fee-bumping reserve, higher liquidity deployed, in theory
> higher routing fees. By observing historical feerates, average offchain
> balances at risk and routing fees expected gains, you should be able to
> discover an equilibrium where higher levels of reserve aren't worth the
> opportunity cost. I guess this equilibrium could be your LN fee-bumping
> reserve max feerate.
>
> Note, I think the LN approach is a bit different from what suits a custody
> protocol like Revault, as you compute a direct return of the frozen
> fee-bumping liquidity. With Revault, if you have numerous bitcoins
> protected, it's might be more interesting to adopt a "buy the mempool,
> stupid" strategy than risking fund safety for few percentages of interest
> returns.
>
>
> True for routing nodes. For wallets (if receiving funds), it's not about
> an investment: just users expectations to being able to transact without
> risking to lose their funds (ie being able to enforce their contract
> onchain). Although wallets they are much less at risk.
>
>
> This is where the "anticipate the crowd of bitcoin users move" point can
> be laid out. As the crowd of bitcoin users' fee-bumping reserves are
> ultimately unknown from your node knowledge, you should be ready to be a
> bit more conservative than the vanilla fee-bumping strategies shipped by
> default. In case of massive mempool congestion, your additional
> conservatism might get your time-sensitive transactions and game on the
> crowd of bitcoin users. First Problem: if all offchain bitcoin software
> adopt that strategy we might inflate the worst-case feerate rate at the
> benefit of the miners, without holistically improving block throughput.
> Second problem : your class of offchain bitcoin softwares might have
> ridiculous fee-bumping reserve compared
> to other classes of offchain bitcoin softwares (Revault > Lightning) and
> just be priced out bydesign in case of mempool congestion. Third problem :
> as the number of offchain bitcoin applications should go up with time, your
> fee-bumping reserve levels based from historical data might be always late
> by one "bank-run" scenario.
>
>
> Black swan event 2.0? Just rule n°3 is inherent to any kind of fee
> estimation.
>
> For Lightning, if you're short in fee-bumping reserves you might still do
> preemptive channel closures, either cooperatively or unilaterally and get
> back the off-chain liquidity to protect the more economically interesting
> channels. Though again, that kind of automatic behavior might be compelling
> at the individual node-level, but make the mempol congestion worse
> holistically.
>
>
> Yeah so we are back to the "fractional reserve" model: you can only
> enforce X% of the offchain contracts your participate in.. Actually it's
> even an added assumption: that you still have operating contracts, with
> honest counterparties.
>
>
> In case of massive mempool congestion, you might try to front-run the
> crowd of bitcoin users relying on block connections for fee-bumping, and
> thus start your fee-bumping as soon as you observe feerate groups
> fluctuations in your local mempool(s).
>
>
> I don't think any kind of mempool-based estimate generalizes well, since
> at any point the expected time before the next block is 10 minutes (and a
> lot can happen in 10min).
>
> Also you might proceed your fee-bumping ticks on a local clock instead of
> block connections in case of time-dilation or deeper eclipse attacks of
> your local node. Your view of the chain might be compromised but not your
> ability to broadcast transactions thanks to emergency channels (in the
> non-LN sense...though in fact quid of txn wrapped in onions ?) of
> communication.
>
>
> Oh, yeah, i didn't explicit "not getting eclipsed" (or more generally
> "data availability") as an assumption since it's generally one made by
> participants of any offchain contract. In this case you can't even have
> decent fee estimation, so you are screwed anyways.
>
>
> Yes, stay open the question on how you enforce this block insurance
> market. Reputation, which might be to avoid due to the latent
> centralization effect, might be hard to stack and audit reliably for an
> emergency mechanism running, hopefully, once in a halvening period. Maybe
> maybe some cryptographic or economically based mechanism on slashing or
> swaps could be found...
>
>
> Unfortunately, given current mining centralisation, pools are in a very
> good position to offer pretty decent SLAs around that. With a block space
> insurance, you of course don't need all these convoluted fee-bumping hacks.
> I'm very concerned that large stakeholders of the "offchain contracts
> ecosystem" would just go this (easier) way and further increase mining
> centralisation pressure.
>
> I agree that a cryptography-based scheme around this type of insurance
> services would be the best way out.
>
>
> Antoine
>
> Le lun. 29 nov. 2021 à 09:34, darosior via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> a écrit :
>
>> Hi everyone,
>>
>> Fee-bumping is paramount to the security of many protocols building on
>> Bitcoin, as they require the
>> confirmation of a transaction (which might be presigned) before the
>> expiration of a timelock at any
>> point after the establishment of the contract.
>>
>> The part of Revault using presigned transactions (the delegation from a
>> large to a smaller multisig)
>> is no exception. We have been working on how to approach this for a while
>> now and i'd like to share
>> what we have in order to open a discussion on this problem so central to
>> what seem to be The Right
>> Way [0] to build on Bitcoin but which has yet to be discussed in details
>> (at least publicly).
>>
>> I'll discuss what we came up with for Revault (at least for what will be
>> its first iteration) but my
>> intent with posting to the mailing list is more to frame the questions to
>> this problem we are all
>> going to face rather than present the results of our study tailored to
>> the Revault usecase.
>> The discussion is still pretty Revault-centric (as it's the case study)
>> but hopefully this can help
>> future protocol designers and/or start a discussion around what
>> everyone's doing for existing ones.
>>
>>
>> ## 1. Reminder about Revault
>>
>> The part of Revault we are interested in for this study is the delegation
>> process, and more
>> specifically the application of spending policies by network monitors
>> (watchtowers).
>> Coins are received on a large multisig. Participants of this large
>> multisig create 2 [1]
>> transactions. The Unvault, spending a deposit UTxO, creates an output
>> paying either to the small
>> multisig after a timelock or to the large multisig immediately. The
>> Cancel, spending the Unvault
>> output through the non-timelocked path, creates a new deposit UTxO.
>> Participants regularly exchange the Cancel transaction signatures for
>> each deposit, sharing the
>> signatures with the watchtowers they operate. They then optionally [2]
>> sign the Unvault transaction
>> and share the signatures with the small multisig participants who can in
>> turn use them to proceed
>> with a spending. Watchtowers can enforce spending policies (say, can't
>> Unvault outside of business
>> hours) by having the Cancel transaction be confirmed before the
>> expiration of the timelock.
>>
>>
>> ## 2. Problem statement
>>
>> For any delegated vault, ensure the confirmation of a Cancel transaction
>> in a configured number of
>> blocks at any point. In so doing, minimize the overpayments and the UTxO
>> set footprint. Overpayments
>> increase the burden on the watchtower operator by increasing the required
>> frequency of refills of the
>> fee-bumping wallet, which is already the worst user experience. You are
>> likely to manage a number of
>> UTxOs with your number of vaults, which comes at a cost for you as well
>> as everyone running a full
>> node.
>>
>> Note that this assumes miners are economically rationale, are
>> incentivized by *public* fees and that
>> you have a way to propagate your fee-bumped transaction to them. We also
>> don't consider the block
>> space bounds.
>>
>> In the previous paragraph and the following text, "vault" can generally
>> be replaced with "offchain
>> contract".
>>
>>
>> ## 3. With presigned transactions
>>
>> As you all know, the first difficulty is to get to be able to
>> unilaterally enforce your contract
>> onchain. That is, any participant must be able to unilaterally bump the
>> fees of a transaction even
>> if it was co-signed by other participants.
>>
>> For Revault we can afford to introduce malleability in the Cancel
>> transaction since there is no
>> second-stage transaction depending on its txid. Therefore it is
>> pre-signed with ANYONECANPAY. We
>> can't use ANYONECANPAY|SINGLE since it would open a pinning vector [3].
>> Note how we can't leverage
>> the carve out rule, and neither can any other more-than-two-parties
>> contract.
>> This has a significant implication for the rest, as we are entirely
>> burning fee-bumping UTxOs.
>>
>> This opens up a pinning vector, or at least a significant nuisance: any
>> other party can largely
>> increase the absolute fee without increasing the feerate, leveraging the
>> RBF rules to prevent you
>> from replacing it without paying an insane fee. And you might not see it
>> in your own mempool and
>> could only suppose it's happening by receiving non-full blocks or with
>> transactions paying a lower
>> feerate.
>> Unfortunately i know of no other primitive that can be used by
>> multi-party (i mean, >2) presigned
>> transactions protocols for fee-bumping that aren't (more) vulnerable to
>> pinning.
>>
>>
>> ## 4. We are still betting on future feerate
>>
>> The problem is still missing one more constraint. "Ensuring confirmation
>> at any time" involves ensuring
>> confirmation at *any* feerate, which you *cannot* do. So what's the
>> limit? In theory you should be ready
>> to burn as much in fees as the value of the funds you want to get out of
>> the contract. So... For us
>> it'd mean keeping for each vault an equivalent amount of funds sitting
>> there on the watchtower's hot
>> wallet. For Lightning, it'd mean keeping an equivalent amount of funds as
>> the sum of all your
>> channels balances sitting there unallocated "just in case". This is not
>> reasonable.
>>
>> So you need to keep a maximum feerate, above which you won't be able to
>> ensure the enforcement of
>> all your contracts onchain at the same time. We call that the "reserve
>> feerate" and you can have
>> different strategies for choosing it, for instance:
>> - The 85th percentile over the last year of transactions feerates
>> - The maximum historical feerate
>> - The maximum historical feerate adjusted in dollars (makes more sense
>> but introduces a (set of?)
>> trusted oracle(s) in a security-critical component)
>> - Picking a random high feerate (why not? It's an arbitrary assumption
>> anyways)
>>
>> Therefore, even if we don't have to bet on the broadcast-time feerate
>> market at signing time anymore
>> (since we can unilaterally bump), we still need some kind of prediction
>> in preparation of making
>> funds available to bump the fees at broadcast time.
>> Apart from judging that 500sat/vb is probably more reasonable than
>> 10sat/vbyte, this unfortunately
>> sounds pretty much crystal-ball-driven.
>>
>> We currently use the maximum of the 95th percentiles over 90-days windows
>> over historical block chain
>> feerates. [4]
>>
>>
>> ## 5. How much funds does my watchtower need?
>>
>> That's what we call the "reserve". Depending on your reserve feerate
>> strategy it might vary over
>> time. This is easier to reason about with a per-contract reserve. For
>> Revault it's pretty
>> straightforward since the Cancel transaction size is static:
>> `reserve_feerate * cancel_size`. For
>> other protocols with dynamic transaction sizes (or even packages of
>> transactions) it's less so. For
>> your Lightning channel you would probably take the maximum size of your
>> commitment transaction
>> according to your HTLC exposure settings + the size of as many
>> `htlc_success` transaction?
>>
>> Then you either have your software or your user guesstimate how many
>> offchain contracts the
>> watchtower will have to watch, time that by the per-contract reserve and
>> refill this amount (plus
>> some slack in practice). Once again, a UX tradeoff (not even mentioning
>> the guesstimation UX):
>> overestimating leads to too many unallocated funds sitting on a hot
>> wallet, underestimating means
>> (at best) inability to participate in new contracts or being "at risk"
>> (not being able to enforce
>> all your contracts onchain at your reserve feerate) before a new refill.
>>
>> For vaults you likely have large-value UTxOs and small transactions (the
>> Cancel is one-in one-out in
>> Revault). For some other applications with large transactions and
>> lower-value UTxOs on average it's
>> likely that only part of the offchain contracts might be enforceable at a
>> reasonable feerate. Is it
>> reasonable?
>>
>>
>> ## 6. UTxO pool layout
>>
>> Now that you somehow managed to settle on a refill amount, how are you
>> going to use these funds?
>> Also, you'll need to manage your pool across time (consolidating small
>> coins, and probably fanning
>> out large ones).
>>
>> You could keep a single large UTxO and peel it as you need to sponsor
>> transactions. But this means
>> that you need to create a coin of a specific value according to your need
>> at the current feerate
>> estimation, hope to have it confirmed in a few blocks (at least for now!
>> [5]), and hope that the
>> value won't be obsolete by the time it confirmed. Also, you'd have to do
>> that for any number of
>> Cancel, chaining feebump coin creation transactions off the change of the
>> previous ones or replacing
>> them with more outputs. Both seem to become really un-manageable (and
>> expensive) in many edge-cases,
>> shortening the time you have to confirm the actual Cancel transaction and
>> creating uncertainty about
>> the reserve (how much is my just-in-time fanout going to cost me in fees
>> that i need to refill in
>> advance on my watchtower wallet?).
>> This is less of a concern for protocols using CPFP to sponsor
>> transactions, but they rely on a
>> policy rule specific to 2-parties contracts.
>>
>> Therefore for Revault we fan-out the coins per-vault in advance. We do so
>> at refill time so the
>> refiller can give an excess to pay for the fees of the fanout transaction
>> (which is reasonable since
>> it will occur just after the refilling transaction confirms). When the
>> watchtower is asked to watch
>> for a new delegated vault it will allocate coins from the pool of
>> fanned-out UTxOs to it (failing
>> that, it would refuse the delegation).
>> What is a good distribution of UTxOs amounts per vault? We want to
>> minimize the number of coins,
>> still have coins small enough to not overpay (remember, we can't have
>> change) and be able to bump a
>> Cancel up to the reserve feerate using these coins. The two latter
>> constraints are directly in
>> contradiction as the minimal value of a coin usable at the reserve
>> feerate (paying for its own input
>> fee + bumping the feerate by, say, 5sat/vb) is already pretty high.
>> Therefore we decided to go with
>> two distributions per vault. The "reserve distribution" alone ensures
>> that we can bump up to the
>> reserve feerate and is usable for high feerates. The "bonus distribution"
>> is not, but contains
>> smaller coins useful to prevent overpayments during low and medium fee
>> periods (which is most of the
>> time).
>> Both distributions are based on a basic geometric suite [6]. Each value
>> is half the previous one.
>> This exponentially decreases the value, limiting the number of coins. But
>> this also allows for
>> pretty small coins to exist and each coin's value is equal to the sum of
>> the smaller coins,
>> or smaller by at most the value of the smallest coin. Therefore bounding
>> the maximum overpayment to
>> the smallest coin's value [7].
>>
>> For the management of the UTxO pool across time we merged the
>> consolidation with the fanout. When
>> fanning out a refilled UTxO, we scan the pool for coins that need to be
>> consolidated according to a
>> heuristic. An instance of a heuristic is "the coin isn't allocated and
>> would not have been able to
>> increase the fee at the median feerate over the past 90 days of blocks".
>> We had this assumption that feerate would tend to go up with time and
>> therefore discarded having to
>> split some UTxOs from the pool. We however overlooked that a large
>> increase in the exchange price of
>> BTC as we've seen during the past year could invalidate this assumption
>> and that should arguably be
>> reconsidered.
>>
>>
>> ## 7. Bumping and re-bumping
>>
>> First of all, when to fee-bump? At fixed time intervals? At each block
>> connection? It sounds like,
>> given a large enough timelock, you could try to greed by "trying your
>> luck" at a lower feerate and
>> only re-bumping every N blocks. You would then start aggressively bumping
>> at every block after M
>> blocks have passed. But that's actually a bet (in disguised?) that the
>> next block feerate in M blocks
>> will be lower than the current one. In the absence of any predictive
>> model it is more reasonable to
>> just start being aggressive immediately.
>> You probably want to base your estimates on `estimatesmartfee` and as a
>> consequence you would re-bump
>> (if needed )after each block connection, when your estimates get updated
>> and you notice your
>> transaction was not included in the block.
>>
>> In the event that you notice a consequent portion of the block is filled
>> with transactions paying
>> less than your own, you might want to start panicking and bump your
>> transaction fees by a certain
>> percentage with no consideration for your fee estimator. You might skew
>> miners incentives in doing
>> so: if you increase the fees by a factor of N, any miner with a fraction
>> larger than 1/N of the
>> network hashrate now has an incentive to censor your transaction at first
>> to get you to panic. Also
>> note this can happen if you want to pay the absolute fees for the
>> 'pinning' attack mentioned in
>> section #2, and that might actually incentivize miners to perform it
>> themselves..
>>
>> The gist is that the most effective way to bump and rebump (RBF the
>> Cancel tx) seems to just be to
>> consider the `estimatesmartfee 2 CONSERVATIVE` feerate at every block
>> your tx isn't included in, and
>> to RBF it if the feerate is higher.
>> In addition, we fallback to a block chain based estimation when estimates
>> aren't available (eg if
>> the user stopped their WT for say a hour and we come back up): we use the
>> 85th percentile over the
>> feerates in the last 6 blocks. Sure, miners can try to have an influence
>> on that by stuffing their
>> blocks with large fee self-paying transactions, but they would need to:
>> 1. Be sure to catch a significant portion of the 6 blocks (at least 2,
>> actually)
>> 2. Give up on 25% of the highest fee-paying transactions (assuming they
>> got the 6 blocks, it's
>> proportionally larger and incertain as they get less of them)
>> 3. Hope that our estimator will fail and we need to fall back to the
>> chain-based estimation
>>
>>
>> ## 8. Our study
>>
>> We essentially replayed the historical data with different deployment
>> configurations (number of
>> participants and timelock) and probability of an event occurring (event
>> being say an Unvault, an
>> invalid Unvault, a new delegation, ..). We then observed different
>> metrics such as the time at risk
>> (when we can't enforce all our contracts at the reserve feerate at the
>> same time), or the
>> operational cost.
>> We got the historical fee estimates data from Statoshi [9], Txstats [10]
>> and the historical chain
>> data from Riccardo Casatta's `blocks_iterator` [11]. Thanks!
>>
>> The (research-quality..) code can be found at
>> https://github.com/revault/research under the section
>> "Fee bumping". Again it's very Revault specific, but at least the data
>> can probably be reused for
>> studying other protocols.
>>
>>
>> ## 9. Insurances
>>
>> Of course, given it's all hacks and workarounds and there is no good
>> answer to "what is a reasonable
>> feerate up to which we need to make contracts enforceable onchain?",
>> there is definitely room for an
>> insurance market. But this enters the realm of opinions. Although i do
>> have some (having discussed
>> this topic for the past years with different people), i would like to
>> keep this post focused on the
>> technical aspects of this problem.
>>
>>
>>
>> [0] As far as i can tell, having offchain contracts be enforceable
>> onchain by confirming a
>> transaction before the expiration of a timelock is a widely agreed-upon
>> approach. And i don't think
>> we can opt for any other fundamentally different one, as you want to know
>> you can claim back your
>> coins from a contract after a deadline before taking part in it.
>>
>> [1] The Real Revault (tm) involves more transactions, but for the sake of
>> conciseness i only
>> detailed a minimum instance of the problem.
>>
>> [2] Only presigning part of the Unvault transactions allows to only
>> delegate part of the coins,
>> which can be abstracted as "delegate x% of your stash" in the user
>> interface.
>>
>> [3]
>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-May/017835.html
>>
>> [4]
>> https://github.com/revault/research/blob/1df953813708287c32a15e771ba74957ec44f354/feebumping/model/statemachine.py#L323-L329
>>
>> [5] https://github.com/bitcoin/bitcoin/pull/23121
>>
>> [6]
>> https://github.com/revault/research/blob/1df953813708287c32a15e771ba74957ec44f354/feebumping/model/statemachine.py#L494-L507
>>
>> [7] Of course this assumes a combinatorial coin selection, but i believe
>> it's ok given we limit the
>> number of coins beforehand.
>>
>> [8] Although there is the argument to outbid a censorship, anyone
>> censoring you isn't necessarily a
>> miner.
>>
>> [9] https://www.statoshi.info/
>>
>> [10] https://www.statoshi.info/
>>
>> [11] https://github.com/RCasatta/blocks_iterator
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20211207/79ed7638/attachment-0001.html>

2023-06-07 23:00:20 UTC

- reply

📅 Original date posted:2021-10-26
📝 Original message:Hi Lisa,
Some background for people who are not familiar with mempool:
The mempool is a cache of unconfirmed transactions, designed in a way
to help miners efficiently pick the highest feerate packages to
include in new blocks. It stores more than a block's worth of
transactions because transaction volume fluctuates and any rational
miner would try to maximize the fees in their blocks; in a reorg, we
also don't want to completely forget what transactions were in the
now-stale tip.
In Bitcoin Core, full nodes keep a mempool by default. The additional
requirements for keeping a mempool are minimal (300MB storage, can be
configured to be lower) because anyone, anywhere in the world, should
be able to run a node and broadcast a Bitcoin payment without special
connectivity to some specific set of people or expensive/inaccessible
hardware. Perhaps connecting directly to miners can be a solution for
some people, but I don't think it's healthy for the network.
Some benefits of keeping a mempool as a non-mining node include:
- Fee estimation based on your node's knowledge of unconfirmed
transactions + historical data.
- Dramatically increased block validation (and thus propagation)
speed, since you cache signature and script results of transactions
before they are confirmed.
- Reduced block relay bandwidth usage (Bitcoin Core nodes use BIP152
compact block relay), as you don't need to re-request the block
transactions you already have in your mempool.
- Wallet ability to send/receive transactions that spend unconfirmed outputs.
> I had the realization that the mempool is obsolete and should be eliminated.
I assume you mean that the mempool should still exist but be turned
off for non-mining nodes. A block template producer needs to keep
unconfirmed transactions somewhere.
On Bitcoin Core today, you can use the -blocksonly config option to
ignore incoming transactions (effectively switching off your mempool),
but there are strong reasons not to do this:
- It is trivial for your peers to detect that all transactions
broadcasted by your node = from your wallet. Linking your node to your
transactions is a very bad privacy leak.
- You must query someone else for what feerate to put on your transaction.
- You can't use BIP152 compact block relay, so your network bandwidth
usage spikes at every block. You also can't cache validation results,
so your block validation speed slows down.
> Removing the mempool would greatly reduce the bandwidth requirement for running a node...
If you're having problems with your individual node's bandwidth usage,
you can also decrease the number of connections you make or turn off
incoming connections. There are efforts to reduce transaction relay
bandwidth usage network-wide [1].
> Removing the mempool would... keep intentionality of transactions private until confirmed/irrevocable...
I'm confused - what is the purpose of keeping a transaction private
until it confirms? Don't miners still see your transaction? A
confirmed transaction is not irrevocable; reorgs happen.
> Removing the mempool would... naturally resolve all current issues inherent in package relay and rbf rules.
Removing the mempool does not help with this. How does a miner decide
whether a conflicting transaction is an economically-advantageous
replacement or a DoS attack? How do you submit your CPFP if the parent
is below the mempool minimum feerate? Do they already have a different
relay/mempool implementation handling all of these problems but don't
aren't sharing it with the rest of the community?
> Initial feerate estimation would need to be based on published blocks, not pending transactions (as this information would no longer be available), or from direct interactions with block producers.
There are many reasons why using only published blocks for fee
estimates is a flawed design, including:
- The miner of a block can artificially inflate the feerate of the
transactions in their mempool simply by including a few of their own
transactions that pay extremely high feerates. This costs them
nothing, as they collect the fees.
- A miner constructs a block based on the transactions in their
mempool. Your transaction's feerate may have been enough to be
included 2 blocks ago or a week ago, but it will be compared to the
other unconfirmed transactions available to the miner now. They can
tell you what's in their mempool or what the next-block feerate is,
but you would be a fool to believe them.
See also [2],[3].
> Provided the number of block template producing actors remains beneath, say 1000, it’d be quite feasible to publish a list of tor endpoints that nodes can independently + directly submit their transactions to. In fact, merely allowing users to select their own list of endpoints to use alternatively to the mempool would be a low effort starting point for the eventual replacement.
As a thought experiment, let's imagine we have some public registry of
mining nodes' tor endpoints and we use it for this secondary
direct-to-miner transaction relay network. If the registry is
maintained (by who?) and accurate (based on whose word?), it is a
point of failure for transaction censorship and deanonymization, as
well as an additional barrier to becoming a miner, encouraging
centralization.
The other possibility is that the registry is not accurate. In fact,
unless the registry requires miners to identify themselves (which
others on this thread have already pointed out is ill-advised), this
should be treated similarly to regular addr gossip. We would never
automatically trust that the entity behind the endpoint provides the
service it advertises, is an honest node that won't simply blackhole
our transaction, or even belongs to a Bitcoin node at all.
Best,
Gloria
[1]: https://arxiv.org/pdf/1905.10518.pdf
[2]: https://bitcointechtalk.com/an-introduction-to-bitcoin-core-fee-estimation-27920880ad0
[3]: https://gist.github.com/morcos/d3637f015bc4e607e1fd10d8351e9f41
On Tue, Oct 26, 2021 at 8:38 AM lisa neigut via bitcoin-dev <
bitcoin-dev at lists.linuxfoundation.org> wrote:
> Hi all,
>
> In a recent conversation with @glozow, I had the realization that the
> mempool is obsolete and should be eliminated.
>
> Instead, users should submit their transactions directly to mining pools,
> preferably over an anonymous communication network such as tor. This can
> easily be achieved by mining pools running a tor onion node for this
> express purpose (or via a lightning network extension etc)
>
> Mempools make sense in a world where mining is done by a large number of
> participating nodes, eg where the block template is constructed by a
> majority of the participants on the network. In this case, it is necessary
> to socialize pending transaction data to all participants, as you don’t
> know which participant will be constructing the winning block template.
>
> In reality however, mempool relay is unnecessary where the majority of
> hashpower and thus block template creation is concentrated in a
> semi-restricted set.
>
> Removing the mempool would greatly reduce the bandwidth requirement for
> running a node, keep intentionality of transactions private until
> confirmed/irrevocable, and naturally resolve all current issues inherent in
> package relay and rbf rules. It also resolves the recent minimum relay
> questions, as relay is no longer a concern for unmined transactions.
>
> Provided the number of block template producing actors remains beneath,
> say 1000, it’d be quite feasible to publish a list of tor endpoints that
> nodes can independently + directly submit their transactions to. In fact,
> merely allowing users to select their own list of endpoints to use
> alternatively to the mempool would be a low effort starting point for the
> eventual replacement.
>
> On the other hand, removing the mempool would greatly complicate solo
> mining and would also make BetterHash proposals, which move the block
> template construction away from a centralized mining pool back to the
> individual miner, much more difficult. It also makes explicit the target
> for DoS attacks.
>
> A direct communication channel between block template construction venues
> and transaction proposers also provides a venue for direct feedback wrt
> acceptable feerates at the time, which both makes transaction confirmation
> timelines less variable as well as provides block producers a mechanism for
> (independently) enforcing their own minimum security budget. In other
> words, expressing a minimum acceptable feerate for continued operation.
>
> Initial feerate estimation would need to be based on published blocks, not
> pending transactions (as this information would no longer be available), or
> from direct interactions with block producers.
>
>
> ~niftynei
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20211026/9477b0b5/attachment.html>

2023-06-07 22:59:26 UTC

- reply

📅 Original date posted:2021-09-21
📝 Original message:Hi Bastien,
Thank you for your feedback!
> In your example we have a parent transaction A already in the mempool
> and an unrelated child B. We submit a package C + D where C spends
> another of A's inputs. You're highlighting that this package may be
> rejected because of the unrelated transaction(s) B.
> The way I see this, an attacker can abuse this rule to ensure
> transaction A stays pinned in the mempool without confirming by
> broadcasting a set of child transactions that reach these limits
> and pay low fees (where A would be a commit tx in LN).
I believe you are describing a pinning attack in which your adversarial
counterparty attempts to monopolize the mempool descendant limit of the
shared transaction A in order to prevent you from submitting a fee-bumping
child C; I've tried to illustrate this as diagram A here:
https://user-images.githubusercontent.com/25183001/134159860-068080d0-bbb6-4356-ae74-00df00644c74.png
(please let me know if I'm misunderstanding).
I believe this attack is mitigated as long as we attempt to submit
transactions individually (and thus take advantage of CPFP carve out)
before attempting package validation. So, in scenario A2, even if the
mempool receives a package with A+C, it would deduplicate A, submit C as an
individual transaction, and allow it due to the CPFP carve out exemption. A
more general goal is: if a transaction would propagate successfully on its
own now, it should still propagate regardless of whether it is included in
a package. The best way to ensure this, as far as I can tell, is to always
try to submit them individually first.
I would note that this proposal doesn't accommodate something like diagram
B, where C is getting CPFP carve out and wants to bring a +1 (e.g. C has
very low fees and is bumped by D). I don't think this is a use case since C
should be the one fee-bumping A, but since we're talking about limitations
around the CPFP carve out, this is it.
Let me know if this addresses your concerns?
Thanks,
Gloria
On Mon, Sep 20, 2021 at 10:19 AM Bastien TEINTURIER <bastien at acinq.fr>
wrote:
> Hi Gloria,
>
> Thanks for this detailed post!
>
> The illustrations you provided are very useful for this kind of graph
> topology problems.
>
> The rules you lay out for package RBF look good to me at first glance
> as there are some subtle improvements compared to BIP 125.
>
> > 1. A package cannot exceed `MAX_PACKAGE_COUNT=25` count and
> > `MAX_PACKAGE_SIZE=101KvB` total size [8]
>
> I have a question regarding this rule, as your example 2C could be
> concerning for LN (unless I didn't understand it correctly).
>
> This also touches on the package RBF rule 5 ("The package cannot
> replace more than 100 mempool transactions.")
>
> In your example we have a parent transaction A already in the mempool
> and an unrelated child B. We submit a package C + D where C spends
> another of A's inputs. You're highlighting that this package may be
> rejected because of the unrelated transaction(s) B.
>
> The way I see this, an attacker can abuse this rule to ensure
> transaction A stays pinned in the mempool without confirming by
> broadcasting a set of child transactions that reach these limits
> and pay low fees (where A would be a commit tx in LN).
>
> We had to create the CPFP carve-out rule explicitly to work around
> this limitation, and I think it would be necessary for package RBF
> as well, because in such cases we do want to be able to submit a
> package A + C where C pays high fees to speed up A's confirmation,
> regardless of unrelated unconfirmed children of A...
>
> We could submit only C to benefit from the existing CPFP carve-out
> rule, but that wouldn't work if our local mempool doesn't have A yet,
> but other remote mempools do.
>
> Is my concern justified? Is this something that we should dig into a
> bit deeper?
>
> Thanks,
> Bastien
>
> Le jeu. 16 sept. 2021 à 09:55, Gloria Zhao via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> a écrit :
>
>> Hi there,
>>
>> I'm writing to propose a set of mempool policy changes to enable package
>> validation (in preparation for package relay) in Bitcoin Core. These
>> would not
>> be consensus or P2P protocol changes. However, since mempool policy
>> significantly affects transaction propagation, I believe this is relevant
>> for
>> the mailing list.
>>
>> My proposal enables packages consisting of multiple parents and 1 child.
>> If you
>> develop software that relies on specific transaction relay assumptions
>> and/or
>> are interested in using package relay in the future, I'm very interested
>> to hear
>> your feedback on the utility or restrictiveness of these package policies
>> for
>> your use cases.
>>
>> A draft implementation of this proposal can be found in [Bitcoin Core
>> PR#22290][1].
>>
>> An illustrated version of this post can be found at
>> https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a.
>> I have also linked the images below.
>>
>> ## Background
>>
>> Feel free to skip this section if you are already familiar with mempool
>> policy
>> and package relay terminology.
>>
>> ### Terminology Clarifications
>>
>> * Package = an ordered list of related transactions, representable by a
>> Directed
>> Acyclic Graph.
>> * Package Feerate = the total modified fees divided by the total virtual
>> size of
>> all transactions in the package.
>> - Modified fees = a transaction's base fees + fee delta applied by
>> the user
>> with `prioritisetransaction`. As such, we expect this to vary across
>> mempools.
>> - Virtual Size = the maximum of virtual sizes calculated using [BIP141
>> virtual size][2] and sigop weight. [Implemented here in Bitcoin
>> Core][3].
>> - Note that feerate is not necessarily based on the base fees and
>> serialized
>> size.
>>
>> * Fee-Bumping = user/wallet actions that take advantage of miner
>> incentives to
>> boost a transaction's candidacy for inclusion in a block, including
>> Child Pays
>> for Parent (CPFP) and [BIP125][12] Replace-by-Fee (RBF). Our intention in
>> mempool policy is to recognize when the new transaction is more
>> economical to
>> mine than the original one(s) but not open DoS vectors, so there are some
>> limitations.
>>
>> ### Policy
>>
>> The purpose of the mempool is to store the best (to be most
>> incentive-compatible
>> with miners, highest feerate) candidates for inclusion in a block. Miners
>> use
>> the mempool to build block templates. The mempool is also useful as a
>> cache for
>> boosting block relay and validation performance, aiding transaction
>> relay, and
>> generating feerate estimations.
>>
>> Ideally, all consensus-valid transactions paying reasonable fees should
>> make it
>> to miners through normal transaction relay, without any special
>> connectivity or
>> relationships with miners. On the other hand, nodes do not have unlimited
>> resources, and a P2P network designed to let any honest node broadcast
>> their
>> transactions also exposes the transaction validation engine to DoS
>> attacks from
>> malicious peers.
>>
>> As such, for unconfirmed transactions we are considering for our mempool,
>> we
>> apply a set of validation rules in addition to consensus, primarily to
>> protect
>> us from resource exhaustion and aid our efforts to keep the highest fee
>> transactions. We call this mempool _policy_: a set of (configurable,
>> node-specific) rules that transactions must abide by in order to be
>> accepted
>> into our mempool. Transaction "Standardness" rules and mempool
>> restrictions such
>> as "too-long-mempool-chain" are both examples of policy.
>>
>> ### Package Relay and Package Mempool Accept
>>
>> In transaction relay, we currently consider transactions one at a time for
>> submission to the mempool. This creates a limitation in the node's
>> ability to
>> determine which transactions have the highest feerates, since we cannot
>> take
>> into account descendants (i.e. cannot use CPFP) until all the
>> transactions are
>> in the mempool. Similarly, we cannot use a transaction's descendants when
>> considering it for RBF. When an individual transaction does not meet the
>> mempool
>> minimum feerate and the user isn't able to create a replacement
>> transaction
>> directly, it will not be accepted by mempools.
>>
>> This limitation presents a security issue for applications and users
>> relying on
>> time-sensitive transactions. For example, Lightning and other protocols
>> create
>> UTXOs with multiple spending paths, where one counterparty's spending
>> path opens
>> up after a timelock, and users are protected from cheating scenarios as
>> long as
>> they redeem on-chain in time. A key security assumption is that all
>> parties'
>> transactions will propagate and confirm in a timely manner. This
>> assumption can
>> be broken if fee-bumping does not work as intended.
>>
>> The end goal for Package Relay is to consider multiple transactions at
>> the same
>> time, e.g. a transaction with its high-fee child. This may help us better
>> determine whether transactions should be accepted to our mempool,
>> especially if
>> they don't meet fee requirements individually or are better RBF
>> candidates as a
>> package. A combination of changes to mempool validation logic, policy, and
>> transaction relay allows us to better propagate the transactions with the
>> highest package feerates to miners, and makes fee-bumping tools more
>> powerful
>> for users.
>>
>> The "relay" part of Package Relay suggests P2P messaging changes, but a
>> large
>> part of the changes are in the mempool's package validation logic. We
>> call this
>> *Package Mempool Accept*.
>>
>> ### Previous Work
>>
>> * Given that mempool validation is DoS-sensitive and complex, it would be
>> dangerous to haphazardly tack on package validation logic. Many efforts
>> have
>> been made to make mempool validation less opaque (see [#16400][4],
>> [#21062][5],
>> [#22675][6], [#22796][7]).
>> * [#20833][8] Added basic capabilities for package validation, test
>> accepts only
>> (no submission to mempool).
>> * [#21800][9] Implemented package ancestor/descendant limit checks for
>> arbitrary
>> packages. Still test accepts only.
>> * Previous package relay proposals (see [#16401][10], [#19621][11]).
>>
>> ### Existing Package Rules
>>
>> These are in master as introduced in [#20833][8] and [#21800][9]. I'll
>> consider
>> them as "given" in the rest of this document, though they can be changed,
>> since
>> package validation is test-accept only right now.
>>
>> 1. A package cannot exceed `MAX_PACKAGE_COUNT=25` count and
>> `MAX_PACKAGE_SIZE=101KvB` total size [8]
>>
>> *Rationale*: This is already enforced as mempool ancestor/descendant
>> limits.
>> Presumably, transactions in a package are all related, so exceeding this
>> limit
>> would mean that the package can either be split up or it wouldn't pass
>> this
>> mempool policy.
>>
>> 2. Packages must be topologically sorted: if any dependencies exist
>> between
>> transactions, parents must appear somewhere before children. [8]
>>
>> 3. A package cannot have conflicting transactions, i.e. none of them can
>> spend
>> the same inputs. This also means there cannot be duplicate transactions.
>> [8]
>>
>> 4. When packages are evaluated against ancestor/descendant limits in a
>> test
>> accept, the union of all of their descendants and ancestors is
>> considered. This
>> is essentially a "worst case" heuristic where every transaction in the
>> package
>> is treated as each other's ancestor and descendant. [8]
>> Packages for which ancestor/descendant limits are accurately captured by
>> this
>> heuristic: [19]
>>
>> There are also limitations such as the fact that CPFP carve out is not
>> applied
>> to package transactions. #20833 also disables RBF in package validation;
>> this
>> proposal overrides that to allow packages to use RBF.
>>
>> ## Proposed Changes
>>
>> The next step in the Package Mempool Accept project is to implement
>> submission
>> to mempool, initially through RPC only. This allows us to test the
>> submission
>> logic before exposing it on P2P.
>>
>> ### Summary
>>
>> - Packages may contain already-in-mempool transactions.
>> - Packages are 2 generations, Multi-Parent-1-Child.
>> - Fee-related checks use the package feerate. This means that wallets can
>> create a package that utilizes CPFP.
>> - Parents are allowed to RBF mempool transactions with a set of rules
>> similar
>> to BIP125. This enables a combination of CPFP and RBF, where a
>> transaction's descendant fees pay for replacing mempool conflicts.
>>
>> There is a draft implementation in [#22290][1]. It is WIP, but feedback is
>> always welcome.
>>
>> ### Details
>>
>> #### Packages May Contain Already-in-Mempool Transactions
>>
>> A package may contain transactions that are already in the mempool. We
>> remove
>> ("deduplicate") those transactions from the package for the purposes of
>> package
>> mempool acceptance. If a package is empty after deduplication, we do
>> nothing.
>>
>> *Rationale*: Mempools vary across the network. It's possible for a parent
>> to be
>> accepted to the mempool of a peer on its own due to differences in policy
>> and
>> fee market fluctuations. We should not reject or penalize the entire
>> package for
>> an individual transaction as that could be a censorship vector.
>>
>> #### Packages Are Multi-Parent-1-Child
>>
>> Only packages of a specific topology are permitted. Namely, a package is
>> exactly
>> 1 child with all of its unconfirmed parents. After deduplication, the
>> package
>> may be exactly the same, empty, 1 child, 1 child with just some of its
>> unconfirmed parents, etc. Note that it's possible for the parents to be
>> indirect
>> descendants/ancestors of one another, or for parent and child to share a
>> parent,
>> so we cannot make any other topology assumptions.
>>
>> *Rationale*: This allows for fee-bumping by CPFP. Allowing multiple
>> parents
>> makes it possible to fee-bump a batch of transactions. Restricting
>> packages to a
>> defined topology is also easier to reason about and simplifies the
>> validation
>> logic greatly. Multi-parent-1-child allows us to think of the package as
>> one big
>> transaction, where:
>>
>> - Inputs = all the inputs of parents + inputs of the child that come from
>> confirmed UTXOs
>> - Outputs = all the outputs of the child + all outputs of the parents that
>> aren't spent by other transactions in the package
>>
>> Examples of packages that follow this rule (variations of example A show
>> some
>> possibilities after deduplication): ![image][15]
>>
>> #### Fee-Related Checks Use Package Feerate
>>
>> Package Feerate = the total modified fees divided by the total virtual
>> size of
>> all transactions in the package.
>>
>> To meet the two feerate requirements of a mempool, i.e., the
>> pre-configured
>> minimum relay feerate (`minRelayTxFee`) and dynamic mempool minimum
>> feerate, the
>> total package feerate is used instead of the individual feerate. The
>> individual
>> transactions are allowed to be below feerate requirements if the package
>> meets
>> the feerate requirements. For example, the parent(s) in the package can
>> have 0
>> fees but be paid for by the child.
>>
>> *Rationale*: This can be thought of as "CPFP within a package," solving
>> the
>> issue of a parent not meeting minimum fees on its own. This allows L2
>> applications to adjust their fees at broadcast time instead of
>> overshooting or
>> risking getting stuck/pinned.
>>
>> We use the package feerate of the package *after deduplication*.
>>
>> *Rationale*: It would be incorrect to use the fees of transactions that
>> are
>> already in the mempool, as we do not want a transaction's fees to be
>> double-counted for both its individual RBF and package RBF.
>>
>> Examples F and G [14] show the same package, but P1 is submitted
>> individually before
>> the package in example G. In example F, we can see that the 300vB package
>> pays
>> an additional 200sat in fees, which is not enough to pay for its own
>> bandwidth
>> (BIP125#4). In example G, we can see that P1 pays enough to replace M1,
>> but
>> using P1's fees again during package submission would make it look like a
>> 300sat
>> increase for a 200vB package. Even including its fees and size would not
>> be
>> sufficient in this example, since the 300sat looks like enough for the
>> 300vB
>> package. The calculcation after deduplication is 100sat increase for a
>> package
>> of size 200vB, which correctly fails BIP125#4. Assume all transactions
>> have a
>> size of 100vB.
>>
>> #### Package RBF
>>
>> If a package meets feerate requirements as a package, the parents in the
>> transaction are allowed to replace-by-fee mempool transactions. The child
>> cannot
>> replace mempool transactions. Multiple transactions can replace the same
>> transaction, but in order to be valid, none of the transactions can try to
>> replace an ancestor of another transaction in the same package (which
>> would thus
>> make its inputs unavailable).
>>
>> *Rationale*: Even if we are using package feerate, a package will not
>> propagate
>> as intended if RBF still requires each individual transaction to meet the
>> feerate requirements.
>>
>> We use a set of rules slightly modified from BIP125 as follows:
>>
>> ##### Signaling (Rule #1)
>>
>> All mempool transactions to be replaced must signal replaceability.
>>
>> *Rationale*: Package RBF signaling logic should be the same for package
>> RBF and
>> single transaction acceptance. This would be updated if single transaction
>> validation moves to full RBF.
>>
>> ##### New Unconfirmed Inputs (Rule #2)
>>
>> A package may include new unconfirmed inputs, but the ancestor feerate of
>> the
>> child must be at least as high as the ancestor feerates of every
>> transaction
>> being replaced. This is contrary to BIP125#2, which states "The
>> replacement
>> transaction may only include an unconfirmed input if that input was
>> included in
>> one of the original transactions. (An unconfirmed input spends an output
>> from a
>> currently-unconfirmed transaction.)"
>>
>> *Rationale*: The purpose of BIP125#2 is to ensure that the replacement
>> transaction has a higher ancestor score than the original transaction(s)
>> (see
>> [comment][13]). Example H [16] shows how adding a new unconfirmed input
>> can lower the
>> ancestor score of the replacement transaction. P1 is trying to replace
>> M1, and
>> spends an unconfirmed output of M2. P1 pays 800sat, M1 pays 600sat, and
>> M2 pays
>> 100sat. Assume all transactions have a size of 100vB. While, in
>> isolation, P1
>> looks like a better mining candidate than M1, it must be mined with M2,
>> so its
>> ancestor feerate is actually 4.5sat/vB. This is lower than M1's ancestor
>> feerate, which is 6sat/vB.
>>
>> In package RBF, the rule analogous to BIP125#2 would be "none of the
>> transactions in the package can spend new unconfirmed inputs." Example J
>> [17] shows
>> why, if any of the package transactions have ancestors, package feerate
>> is no
>> longer accurate. Even though M2 and M3 are not ancestors of P1 (which is
>> the
>> replacement transaction in an RBF), we're actually interested in the
>> entire
>> package. A miner should mine M1 which is 5sat/vB instead of M2, M3, P1,
>> P2, and
>> P3, which is only 4sat/vB. The Package RBF rule cannot be loosened to
>> only allow
>> the child to have new unconfirmed inputs, either, because it can still
>> cause us
>> to overestimate the package's ancestor score.
>>
>> However, enforcing a rule analogous to BIP125#2 would not only make
>> Package RBF
>> less useful, but would also break Package RBF for packages with parents
>> already
>> in the mempool: if a package parent has already been submitted, it would
>> look
>> like the child is spending a "new" unconfirmed input. In example K [18],
>> we're
>> looking to replace M1 with the entire package including P1, P2, and P3.
>> We must
>> consider the case where one of the parents is already in the mempool (in
>> this
>> case, P2), which means we must allow P3 to have new unconfirmed inputs.
>> However,
>> M2 lowers the ancestor score of P3 to 4.3sat/vB, so we should not replace
>> M1
>> with this package.
>>
>> Thus, the package RBF rule regarding new unconfirmed inputs is less
>> strict than
>> BIP125#2. However, we still achieve the same goal of requiring the
>> replacement
>> transactions to have a ancestor score at least as high as the original
>> ones. As
>> a result, the entire package is required to be a higher feerate mining
>> candidate
>> than each of the replaced transactions.
>>
>> Another note: the [comment][13] above the BIP125#2 code in the original
>> RBF
>> implementation suggests that the rule was intended to be temporary.
>>
>> ##### Absolute Fee (Rule #3)
>>
>> The package must increase the absolute fee of the mempool, i.e. the total
>> fees
>> of the package must be higher than the absolute fees of the mempool
>> transactions
>> it replaces. Combined with the CPFP rule above, this differs from BIP125
>> Rule #3
>> - an individual transaction in the package may have lower fees than the
>> transaction(s) it is replacing. In fact, it may have 0 fees, and the
>> child
>> pays for RBF.
>>
>> ##### Feerate (Rule #4)
>>
>> The package must pay for its own bandwidth; the package feerate must be
>> higher
>> than the replaced transactions by at least minimum relay feerate
>> (`incrementalRelayFee`). Combined with the CPFP rule above, this differs
>> from
>> BIP125 Rule #4 - an individual transaction in the package can have a lower
>> feerate than the transaction(s) it is replacing. In fact, it may have 0
>> fees,
>> and the child pays for RBF.
>>
>> ##### Total Number of Replaced Transactions (Rule #5)
>>
>> The package cannot replace more than 100 mempool transactions. This is
>> identical
>> to BIP125 Rule #5.
>>
>> ### Expected FAQs
>>
>> 1. Is it possible for only some of the package to make it into the
>> mempool?
>>
>> Yes, it is. However, since we evict transactions from the mempool by
>> descendant score and the package child is supposed to be sponsoring the
>> fees of
>> its parents, the most common scenario would be all-or-nothing. This is
>> incentive-compatible. In fact, to be conservative, package validation
>> should
>> begin by trying to submit all of the transactions individually, and only
>> use the
>> package mempool acceptance logic if the parents fail due to low feerate.
>>
>> 2. Should we allow packages to contain already-confirmed transactions?
>>
>> No, for practical reasons. In mempool validation, we actually aren't
>> able to
>> tell with 100% confidence if we are looking at a transaction that has
>> already
>> confirmed, because we look up inputs using a UTXO set. If we have
>> historical
>> block data, it's possible to look for it, but this is inefficient, not
>> always
>> possible for pruning nodes, and unnecessary because we're not going to do
>> anything with the transaction anyway. As such, we already have the
>> expectation
>> that transaction relay is somewhat "stateful" i.e. nobody should be
>> relaying
>> transactions that have already been confirmed. Similarly, we shouldn't be
>> relaying packages that contain already-confirmed transactions.
>>
>> [1]: https://github.com/bitcoin/bitcoin/pull/22290
>> [2]:
>> https://github.com/bitcoin/bips/blob/1f0b563738199ca60d32b4ba779797fc97d040fe/bip-0141.mediawiki#transaction-size-calculations
>> [3]:
>> https://github.com/bitcoin/bitcoin/blob/94f83534e4b771944af7d9ed0f40746f392eb75e/src/policy/policy.cpp#L282
>> [4]: https://github.com/bitcoin/bitcoin/pull/16400
>> [5]: https://github.com/bitcoin/bitcoin/pull/21062
>> [6]: https://github.com/bitcoin/bitcoin/pull/22675
>> [7]: https://github.com/bitcoin/bitcoin/pull/22796
>> [8]: https://github.com/bitcoin/bitcoin/pull/20833
>> [9]: https://github.com/bitcoin/bitcoin/pull/21800
>> [10]: https://github.com/bitcoin/bitcoin/pull/16401
>> [11]: https://github.com/bitcoin/bitcoin/pull/19621
>> [12]: https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki
>> [13]:
>> https://github.com/bitcoin/bitcoin/pull/6871/files#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2R1101-R1104
>> [14]:
>> https://user-images.githubusercontent.com/25183001/133567078-075a971c-0619-4339-9168-b41fd2b90c28.png
>> [15]:
>> https://user-images.githubusercontent.com/25183001/132856734-fc17da75-f875-44bb-b954-cb7a1725cc0d.png
>> [16]:
>> https://user-images.githubusercontent.com/25183001/133567347-a3e2e4a8-ae9c-49f8-abb9-81e8e0aba224.png
>> [17]:
>> https://user-images.githubusercontent.com/25183001/133567370-21566d0e-36c8-4831-b1a8-706634540af3.png
>> [18]:
>> https://user-images.githubusercontent.com/25183001/133567444-bfff1142-439f-4547-800a-2ba2b0242bcb.png
>> [19]:
>> https://user-images.githubusercontent.com/25183001/133456219-0bb447cb-dcb4-4a31-b9c1-7d86205b68bc.png
>> [20]:
>> https://user-images.githubusercontent.com/25183001/132857787-7b7c6f56-af96-44c8-8d78-983719888c19.png
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210921/505919a9/attachment-0001.html>

2023-06-07 22:59:19 UTC

- reply

📅 Original date posted:2021-09-23
📝 Original message:Hi Antoine,
Thanks as always for your input. I'm glad we agree on so much!
In summary, it seems that the decisions that might still need
attention/input from devs on this mailing list are:
1. Whether we should start with multiple-parent-1-child or 1-parent-1-child.
2. Whether it's ok to require that the child not have conflicts with
mempool transactions.
Responding to your comments...
> IIUC, you have package A+B, during the dedup phase early in
`AcceptMultipleTransactions` if you observe same-txid-different-wtixd A'
and A' is higher feerate than A, you trim A and replace by A' ?
> I think this approach is safe, the one who appears unsafe to me is when
A' has a _lower_ feerate, even if A' is already accepted by our mempool ?
In that case iirc that would be a pinning.
Right, the fact that we essentially always choose the first-seen witness is
an unfortunate limitation that exists already. Adding package mempool
accept doesn't worsen this, but the procedure in the future is to replace
the witness when it makes sense economically. We can also add logic to
allow package feerate to pay for witness replacements as well. This is
pretty far into the future, though.
> It sounds uneconomical for an attacker but I think it's not when you
consider than you can "batch" attack against multiple honest
counterparties. E.g, Mallory broadcast A' + B' + C' + D' where A' conflicts
with Alice's honest package P1, B' conflicts with Bob's honest package P2,
C' conflicts with Caroll's honest package P3. And D' is a high-fee child of
A' + B' + C'.
> If D' is higher-fee than P1 or P2 or P3 but inferior to the sum of HTLCs
confirmed by P1+P2+P3, I think it's lucrative for the attacker ?
I could be misunderstanding, but an attacker wouldn't be able to
batch-attack like this. Alice's package only conflicts with A' + D', not A'
+ B' + C' + D'. She only needs to pay for evicting 2 transactions.
> Do we assume that broadcasted packages are "honest" by default and that
the parent(s) always need the child to pass the fee checks, that way saving
the processing of individual transactions which are expected to fail in 99%
of cases or more ad hoc composition of packages at relay ?
> I think this point is quite dependent on the p2p packages format/logic
we'll end up on and that we should feel free to revisit it later ?
I think it's the opposite; there's no way for us to assume that p2p
packages will be "honest." I'd like to have two things before we expose on
P2P: (1) ensure that the amount of resources potentially allocated for
package validation isn't disproportionately higher than that of single
transaction validation and (2) only use package validation when we're
unsatisifed with the single validation result, e.g. we might get better
fees.
Yes, let's revisit this later :)
> Yes, if you receive A+B, and A is already in-mempoo, I agree you can
discard its feerate as B should pay for all fees checked on its own. Where
I'm unclear is when you have in-mempool A+B and receive A+B'. Should B'
have a fee high enough to cover the bandwidth penalty replacement
(`PaysForRBF`, 2nd check) of both A+B' or only B' ?
B' only needs to pay for itself in this case.
> > Do we want the child to be able to replace mempool transactions as well?
> If we mean when you have replaceable A+B then A'+B' try to replace with a
higher-feerate ? I think that's exactly the case we need for Lightning as
A+B is coming from Alice and A'+B' is coming from Bob :/
Let me clarify this because I can see that my wording was ambiguous, and
then please let me know if it fits Lightning's needs?
In my proposal, I wrote "If a package meets feerate requirements as a
package, the parents in the transaction are allowed to replace-by-fee
mempool transactions. The child cannot replace mempool transactions." What
I meant was: the package can replace mempool transactions if any of the
parents conflict with mempool transactions. The child cannot not conflict
with any mempool transactions.
The Lightning use case this attempts to address is: Alice and Mallory are
LN counterparties, and have packages A+B and A'+B', respectively. A and A'
are their commitment transactions and conflict with each other; they have
shared inputs and different txids.
B spends Alice's anchor output from A. B' spends Mallory's anchor output
from A'. Thus, B and B' do not conflict with each other.
Alice can broadcast her package, A+B, to replace Mallory's package, A'+B',
since B doesn't conflict with the mempool.
Would this be ok?
> The second option, a child of A', In the LN case I think the CPFP is
attached on one's anchor output.
While it would be nice to have full RBF, malleability of the child won't
block RBF here. If we're trying to replace A', we only require that A'
signals replaceability, and don't mind if its child doesn't.
> > B has an ancestor score of 10sat/vb and D has an
> > ancestor score of ~2.9sat/vb. Since D's ancestor score is lower than
B's,
> > it fails the proposed package RBF Rule #2, so this package would be
> > rejected. Does this meet your expectations?
> Well what sounds odd to me, in my example, we fail D even if it has a
higher-fee than B. Like A+B absolute fees are 2000 sats and A+C+D absolute
fees are 4500 sats ?
Yes, A+C+D pays 2500sat more in fees, but it is also 1000vB larger. A miner
should prefer to utilize their block space more effectively.
> Is this compatible with a model where a miner prioritizes absolute fees
over ancestor score, in the case that mempools aren't full-enough to
fulfill a block ?
No, because we don't use that model.
Thanks,
Gloria
On Thu, Sep 23, 2021 at 5:29 AM Antoine Riard <antoine.riard at gmail.com>
wrote:
> > Correct, if B+C is too low feerate to be accepted, we will reject it. I
> > prefer this because it is incentive compatible: A can be mined by itself,
> > so there's no reason to prefer A+B+C instead of A.
> > As another way of looking at this, consider the case where we do accept
> > A+B+C and it sits at the "bottom" of our mempool. If our mempool reaches
> > capacity, we evict the lowest descendant feerate transactions, which are
> > B+C in this case. This gives us the same resulting mempool, with A and
> not
> > B+C.
>
> I agree here. Doing otherwise, we might evict other transactions mempool
> in `MempoolAccept::Finalize` with a higher-feerate than B+C while those
> evicted transactions are the most compelling for block construction.
>
> I thought at first missing this acceptance requirement would break a
> fee-bumping scheme like Parent-Pay-For-Child where a high-fee parent is
> attached to a child signed with SIGHASH_ANYONECANPAY but in this case the
> child fee is capturing the parent value. I can't think of other fee-bumping
> schemes potentially affected. If they do exist I would say they're wrong in
> their design assumptions.
>
> > If or when we have witness replacement, the logic is: if the individual
> > transaction is enough to replace the mempool one, the replacement will
> > happen during the preceding individual transaction acceptance, and
> > deduplication logic will work. Otherwise, we will try to deduplicate by
> > wtxid, see that we need a package witness replacement, and use the
> package
> > feerate to evaluate whether this is economically rational.
>
> IIUC, you have package A+B, during the dedup phase early in
> `AcceptMultipleTransactions` if you observe same-txid-different-wtixd A'
> and A' is higher feerate than A, you trim A and replace by A' ?
>
> I think this approach is safe, the one who appears unsafe to me is when A'
> has a _lower_ feerate, even if A' is already accepted by our mempool ? In
> that case iirc that would be a pinning.
>
> Good to see progress on witness replacement before we see usage of Taproot
> tree in the context of multi-party, where a malicious counterparty inflates
> its witness to jam a honest spending.
>
> (Note, the commit linked currently points nowhere :))
>
>
> > Please note that A may replace A' even if A' has higher fees than A
> > individually, because the proposed package RBF utilizes the fees and size
> > of the entire package. This just requires E to pay enough fees, although
> > this can be pretty high if there are also potential B' and C' competing
> > commitment transactions that we don't know about.
>
> Ah right, if the package acceptance waives `PaysMoreThanConflicts` for the
> individual check on A, the honest package should replace the pinning
> attempt. I've not fully parsed the proposed implementation yet.
>
> Though note, I think it's still unsafe for a Lightning
> multi-commitment-broadcast-as-one-package as a malicious A' might have an
> absolute fee higher than E. It sounds uneconomical for
> an attacker but I think it's not when you consider than you can "batch"
> attack against multiple honest counterparties. E.g, Mallory broadcast A' +
> B' + C' + D' where A' conflicts with Alice's honest package P1, B'
> conflicts with Bob's honest package P2, C' conflicts with Caroll's honest
> package P3. And D' is a high-fee child of A' + B' + C'.
>
> If D' is higher-fee than P1 or P2 or P3 but inferior to the sum of HTLCs
> confirmed by P1+P2+P3, I think it's lucrative for the attacker ?
>
> > So far, my understanding is that multi-parent-1-child is desired for
> > batched fee-bumping (
> > https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-897951289)
> and
> > I've also seen your response which I have less context on (
> > https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-900352202).
> That
> > being said, I am happy to create a new proposal for 1 parent + 1 child
> > (which would be slightly simpler) and plan for moving to
> > multi-parent-1-child later if that is preferred. I am very interested in
> > hearing feedback on that approach.
>
> I think batched fee-bumping is okay as long as you don't have
> time-sensitive outputs encumbering your commitment transactions. For the
> reasons mentioned above, I think that's unsafe.
>
> What I'm worried about is L2 developers, potentially not aware about all
> the mempool subtleties blurring the difference and always batching their
> broadcast by default.
>
> IMO, a good thing by restraining to 1-parent + 1 child, we artificially
> constraint L2 design space for now and minimize risks of unsafe usage of
> the package API :)
>
> I think that's a point where it would be relevant to have the opinion of
> more L2 devs.
>
> > I think there is a misunderstanding here - let me describe what I'm
> > proposing we'd do in this situation: we'll try individual submission for
> A,
> > see that it fails due to "insufficient fees." Then, we'll try package
> > validation for A+B and use package RBF. If A+B pays enough, it can still
> > replace A'. If A fails for a bad signature, we won't look at B or A+B.
> Does
> > this meet your expectations?
>
> Yes there was a misunderstanding, I think this approach is correct, it's
> more a question of performance. Do we assume that broadcasted packages are
> "honest" by default and that the parent(s) always need the child to pass
> the fee checks, that way saving the processing of individual transactions
> which are expected to fail in 99% of cases or more ad hoc composition of
> packages at relay ?
>
> I think this point is quite dependent on the p2p packages format/logic
> we'll end up on and that we should feel free to revisit it later ?
>
>
> > What problem are you trying to solve by the package feerate *after* dedup
> rule ?
> > My understanding is that an in-package transaction might be already in
> the mempool. Therefore, to compute a correct RBF penalty replacement, the
> vsize of this transaction could be discarded lowering the cost of package
> RBF.
>
> > I'm proposing that, when a transaction has already been submitted to
> > mempool, we would ignore both its fees and vsize when calculating package
> > feerate.
>
> Yes, if you receive A+B, and A is already in-mempoo, I agree you can
> discard its feerate as B should pay for all fees checked on its own. Where
> I'm unclear is when you have in-mempool A+B and receive A+B'. Should B'
> have a fee high enough to cover the bandwidth penalty replacement
> (`PaysForRBF`, 2nd check) of both A+B' or only B' ?
>
> If you have a second-layer like current Lightning, you might have a
> counterparty commitment to replace and should always expect to have to pay
> for parent replacement bandwidth.
>
> Where a potential discount sounds interesting is when you have an univoque
> state on the first-stage of transactions. E.g DLC's funding transaction
> which might be CPFP by any participant iirc.
>
> > Note that, if C' conflicts with C, it also conflicts with D, since D is a
> > descendant of C and would thus need to be evicted along with it.
>
> Ah once again I think it's a misunderstanding without the code under my
> eyes! If we do C' `PreChecks`, solve the conflicts provoked by it, i.e mark
> for potential eviction D and don't consider it for future conflicts in the
> rest of the package, I think D' `PreChecks` should be good ?
>
> > More generally, this example is surprising to me because I didn't think
> > packages would be used to fee-bump replaceable transactions. Do we want
> the
> > child to be able to replace mempool transactions as well?
>
> If we mean when you have replaceable A+B then A'+B' try to replace with a
> higher-feerate ? I think that's exactly the case we need for Lightning as
> A+B is coming from Alice and A'+B' is coming from Bob :/
>
> > I'm not sure what you mean? Let's say we have a package of parent A +
> child
> > B, where A is supposed to replace a mempool transaction A'. Are you
> saying
> > that counterparties are able to malleate the package child B, or a child
> of
> > A'?
>
> The second option, a child of A', In the LN case I think the CPFP is
> attached on one's anchor output.
>
> I think it's good if we assume the
> solve-conflicts-after-parent's`'PreChecks` mentioned above or fixing
> inherited signaling or full-rbf ?
>
> > Sorry, I don't understand what you mean by "preserve the package
> > integrity?" Could you elaborate?
>
> After thinking the relaxation about the "new" unconfirmed input is not
> linked to trimming but I would say more to the multi-parent support.
>
> Let's say you have A+B trying to replace C+D where B is also spending
> already in-mempool E. To succeed, you need to waive the no-new-unconfirmed
> input as D isn't spending E.
>
> So good, I think we agree on the problem description here.
>
> > I am in agreement with your calculations but unsure if we disagree on the
> > expected outcome. Yes, B has an ancestor score of 10sat/vb and D has an
> > ancestor score of ~2.9sat/vb. Since D's ancestor score is lower than B's,
> > it fails the proposed package RBF Rule #2, so this package would be
> > rejected. Does this meet your expectations?
>
> Well what sounds odd to me, in my example, we fail D even if it has a
> higher-fee than B. Like A+B absolute fees are 2000 sats and A+C+D absolute
> fees are 4500 sats ?
>
> Is this compatible with a model where a miner prioritizes absolute fees
> over ancestor score, in the case that mempools aren't full-enough to
> fulfill a block ?
>
> Let me know if I can clarify a point.
>
> Antoine
>
> Le lun. 20 sept. 2021 à 11:10, Gloria Zhao <gloriajzhao at gmail.com> a
> écrit :
>
>>
>> Hi Antoine,
>>
>> First of all, thank you for the thorough review. I appreciate your
>> insight on LN requirements.
>>
>> > IIUC, you have a package A+B+C submitted for acceptance and A is
>> already in your mempool. You trim out A from the package and then evaluate
>> B+C.
>>
>> > I think this might be an issue if A is the higher-fee element of the
>> ABC package. B+C package fees might be under the mempool min fee and will
>> be rejected, potentially breaking the acceptance expectations of the
>> package issuer ?
>>
>> Correct, if B+C is too low feerate to be accepted, we will reject it. I
>> prefer this because it is incentive compatible: A can be mined by itself,
>> so there's no reason to prefer A+B+C instead of A.
>> As another way of looking at this, consider the case where we do accept
>> A+B+C and it sits at the "bottom" of our mempool. If our mempool reaches
>> capacity, we evict the lowest descendant feerate transactions, which are
>> B+C in this case. This gives us the same resulting mempool, with A and not
>> B+C.
>>
>>
>> > Further, I think the dedup should be done on wtxid, as you might have
>> multiple valid witnesses. Though with varying vsizes and as such offering
>> different feerates.
>>
>> I agree that variations of the same package with different witnesses is a
>> case that must be handled. I consider witness replacement to be a project
>> that can be done in parallel to package mempool acceptance because being
>> able to accept packages does not worsen the problem of a
>> same-txid-different-witness "pinning" attack.
>>
>> If or when we have witness replacement, the logic is: if the individual
>> transaction is enough to replace the mempool one, the replacement will
>> happen during the preceding individual transaction acceptance, and
>> deduplication logic will work. Otherwise, we will try to deduplicate by
>> wtxid, see that we need a package witness replacement, and use the package
>> feerate to evaluate whether this is economically rational.
>>
>> See the #22290 "handle package transactions already in mempool" commit (
>> https://github.com/bitcoin/bitcoin/pull/22290/commits/fea75a2237b46cf76145242fecad7e274bfcb5ff),
>> which handles the case of same-txid-different-witness by simply using the
>> transaction in the mempool for now, with TODOs for what I just described.
>>
>>
>> > I'm not clearly understanding the accepted topologies. By "parent and
>> child to share a parent", do you mean the set of transactions A, B, C,
>> where B is spending A and C is spending A and B would be correct ?
>>
>> Yes, that is what I meant. Yes, that would a valid package under these
>> rules.
>>
>> > If yes, is there a width-limit introduced or we fallback on
>> MAX_PACKAGE_COUNT=25 ?
>>
>> No, there is no limit on connectivity other than "child with all
>> unconfirmed parents." We will enforce MAX_PACKAGE_COUNT=25 and child's
>> in-mempool + in-package ancestor limits.
>>
>>
>> > Considering the current Core's mempool acceptance rules, I think CPFP
>> batching is unsafe for LN time-sensitive closure. A malicious tx-relay
>> jamming successful on one channel commitment transaction would contamine
>> the remaining commitments sharing the same package.
>>
>> > E.g, you broadcast the package A+B+C+D+E where A,B,C,D are commitment
>> transactions and E a shared CPFP. If a malicious A' transaction has a
>> better feerate than A, the whole package acceptance will fail. Even if A'
>> confirms in the following block,
>> the propagation and confirmation of B+C+D have been delayed. This could
>> carry on a loss of funds.
>>
>> Please note that A may replace A' even if A' has higher fees than A
>> individually, because the proposed package RBF utilizes the fees and size
>> of the entire package. This just requires E to pay enough fees, although
>> this can be pretty high if there are also potential B' and C' competing
>> commitment transactions that we don't know about.
>>
>>
>> > IMHO, I'm leaning towards deploying during a first phase
>> 1-parent/1-child. I think it's the most conservative step still improving
>> second-layer safety.
>>
>> So far, my understanding is that multi-parent-1-child is desired for
>> batched fee-bumping (
>> https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-897951289)
>> and I've also seen your response which I have less context on (
>> https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-900352202).
>> That being said, I am happy to create a new proposal for 1 parent + 1 child
>> (which would be slightly simpler) and plan for moving to
>> multi-parent-1-child later if that is preferred. I am very interested in
>> hearing feedback on that approach.
>>
>>
>> > If A+B is submitted to replace A', where A pays 0 sats, B pays 200 sats
>> and A' pays 100 sats. If we apply the individual RBF on A, A+B acceptance
>> fails. For this reason I think the individual RBF should be bypassed and
>> only the package RBF apply ?
>>
>> I think there is a misunderstanding here - let me describe what I'm
>> proposing we'd do in this situation: we'll try individual submission for A,
>> see that it fails due to "insufficient fees." Then, we'll try package
>> validation for A+B and use package RBF. If A+B pays enough, it can still
>> replace A'. If A fails for a bad signature, we won't look at B or A+B. Does
>> this meet your expectations?
>>
>>
>> > What problem are you trying to solve by the package feerate *after*
>> dedup rule ?
>> > My understanding is that an in-package transaction might be already in
>> the mempool. Therefore, to compute a correct RBF penalty replacement, the
>> vsize of this transaction could be discarded lowering the cost of package
>> RBF.
>>
>> I'm proposing that, when a transaction has already been submitted to
>> mempool, we would ignore both its fees and vsize when calculating package
>> feerate. In example G2, we shouldn't count M1 fees after its submission to
>> mempool, since M1's fees have already been used to pay for its individual
>> bandwidth, and it shouldn't be used again to pay for P2 and P3's bandwidth.
>> We also shouldn't count its vsize, since it has already been paid for.
>>
>>
>> > I think this is a footgunish API, as if a package issuer send the
>> multiple-parent-one-child package A,B,C,D where D is the child of A,B,C.
>> Then try to broadcast the higher-feerate C'+D' package, it should be
>> rejected. So it's breaking the naive broadcaster assumption that a
>> higher-feerate/higher-fee package always replaces ?
>>
>> Note that, if C' conflicts with C, it also conflicts with D, since D is a
>> descendant of C and would thus need to be evicted along with it.
>> Implicitly, D' would not be in conflict with D.
>> More generally, this example is surprising to me because I didn't think
>> packages would be used to fee-bump replaceable transactions. Do we want the
>> child to be able to replace mempool transactions as well? This can be
>> implemented with a bit of additional logic.
>>
>> > I think this is unsafe for L2s if counterparties have malleability of
>> the child transaction. They can block your package replacement by
>> opting-out from RBF signaling. IIRC, LN's "anchor output" presents such an
>> ability.
>>
>> I'm not sure what you mean? Let's say we have a package of parent A +
>> child B, where A is supposed to replace a mempool transaction A'. Are you
>> saying that counterparties are able to malleate the package child B, or a
>> child of A'? If they can malleate a child of A', that shouldn't matter as
>> long as A' is signaling replacement. This would be handled identically with
>> full RBF and what Core currently implements.
>>
>> > I think this is an issue brought by the trimming during the dedup
>> phase. If we preserve the package integrity, only re-using the tx-level
>> checks results of already in-mempool transactions to gain in CPU time we
>> won't have this issue. Package childs can add unconfirmed inputs as long as
>> they're in-package, the bip125 rule2 is only evaluated against parents ?
>>
>> Sorry, I don't understand what you mean by "preserve the package
>> integrity?" Could you elaborate?
>>
>> > Let's say you have in-mempool A, B where A pays 10 sat/vb for 100
>> vbytes and B pays 10 sat/vb for 100 vbytes. You have the candidate
>> replacement D spending both A and C where D pays 15sat/vb for 100 vbytes
>> and C pays 1 sat/vb for 1000 vbytes.
>>
>> > Package A + B ancestor score is 10 sat/vb.
>>
>> > D has a higher feerate/absolute fee than B.
>>
>> > Package A + C + D ancestor score is ~ 3 sat/vb ((A's 1000 sats + C's
>> 1000 sats + D's 1500 sats) / A's 100 vb + C's 1000 vb + D's 100 vb)
>>
>> I am in agreement with your calculations but unsure if we disagree on the
>> expected outcome. Yes, B has an ancestor score of 10sat/vb and D has an
>> ancestor score of ~2.9sat/vb. Since D's ancestor score is lower than B's,
>> it fails the proposed package RBF Rule #2, so this package would be
>> rejected. Does this meet your expectations?
>>
>> Thank you for linking to projects that might be interested in package
>> relay :)
>>
>> Thanks,
>> Gloria
>>
>> On Mon, Sep 20, 2021 at 12:16 AM Antoine Riard <antoine.riard at gmail.com>
>> wrote:
>>
>>> Hi Gloria,
>>>
>>> > A package may contain transactions that are already in the mempool. We
>>> > remove
>>> > ("deduplicate") those transactions from the package for the purposes of
>>> > package
>>> > mempool acceptance. If a package is empty after deduplication, we do
>>> > nothing.
>>>
>>> IIUC, you have a package A+B+C submitted for acceptance and A is already
>>> in your mempool. You trim out A from the package and then evaluate B+C.
>>>
>>> I think this might be an issue if A is the higher-fee element of the ABC
>>> package. B+C package fees might be under the mempool min fee and will be
>>> rejected, potentially breaking the acceptance expectations of the package
>>> issuer ?
>>>
>>> Further, I think the dedup should be done on wtxid, as you might have
>>> multiple valid witnesses. Though with varying vsizes and as such offering
>>> different feerates.
>>>
>>> E.g you're going to evaluate the package A+B and A' is already in your
>>> mempool with a bigger valid witness. You trim A based on txid, then you
>>> evaluate A'+B, which fails the fee checks. However, evaluating A+B would
>>> have been a success.
>>>
>>> AFAICT, the dedup rationale would be to save on CPU time/IO disk, to
>>> avoid repeated signatures verification and parent UTXOs fetches ? Can we
>>> achieve the same goal by bypassing tx-level checks for already-in txn while
>>> conserving the package integrity for package-level checks ?
>>>
>>> > Note that it's possible for the parents to be
>>> > indirect
>>> > descendants/ancestors of one another, or for parent and child to share
>>> a
>>> > parent,
>>> > so we cannot make any other topology assumptions.
>>>
>>> I'm not clearly understanding the accepted topologies. By "parent and
>>> child to share a parent", do you mean the set of transactions A, B, C,
>>> where B is spending A and C is spending A and B would be correct ?
>>>
>>> If yes, is there a width-limit introduced or we fallback on
>>> MAX_PACKAGE_COUNT=25 ?
>>>
>>> IIRC, one rationale to come with this topology limitation was to lower
>>> the DoS risks when potentially deploying p2p packages.
>>>
>>> Considering the current Core's mempool acceptance rules, I think CPFP
>>> batching is unsafe for LN time-sensitive closure. A malicious tx-relay
>>> jamming successful on one channel commitment transaction would contamine
>>> the remaining commitments sharing the same package.
>>>
>>> E.g, you broadcast the package A+B+C+D+E where A,B,C,D are commitment
>>> transactions and E a shared CPFP. If a malicious A' transaction has a
>>> better feerate than A, the whole package acceptance will fail. Even if A'
>>> confirms in the following block,
>>> the propagation and confirmation of B+C+D have been delayed. This could
>>> carry on a loss of funds.
>>>
>>> That said, if you're broadcasting commitment transactions without
>>> time-sensitive HTLC outputs, I think the batching is effectively a fee
>>> saving as you don't have to duplicate the CPFP.
>>>
>>> IMHO, I'm leaning towards deploying during a first phase
>>> 1-parent/1-child. I think it's the most conservative step still improving
>>> second-layer safety.
>>>
>>> > *Rationale*: It would be incorrect to use the fees of transactions
>>> that are
>>> > already in the mempool, as we do not want a transaction's fees to be
>>> > double-counted for both its individual RBF and package RBF.
>>>
>>> I'm unsure about the logical order of the checks proposed.
>>>
>>> If A+B is submitted to replace A', where A pays 0 sats, B pays 200 sats
>>> and A' pays 100 sats. If we apply the individual RBF on A, A+B acceptance
>>> fails. For this reason I think the individual RBF should be bypassed and
>>> only the package RBF apply ?
>>>
>>> Note this situation is plausible, with current LN design, your
>>> counterparty can have a commitment transaction with a better fee just by
>>> selecting a higher `dust_limit_satoshis` than yours.
>>>
>>> > Examples F and G [14] show the same package, but P1 is submitted
>>> > individually before
>>> > the package in example G. In example F, we can see that the 300vB
>>> package
>>> > pays
>>> > an additional 200sat in fees, which is not enough to pay for its own
>>> > bandwidth
>>> > (BIP125#4). In example G, we can see that P1 pays enough to replace
>>> M1, but
>>> > using P1's fees again during package submission would make it look
>>> like a
>>> > 300sat
>>> > increase for a 200vB package. Even including its fees and size would
>>> not be
>>> > sufficient in this example, since the 300sat looks like enough for the
>>> 300vB
>>> > package. The calculcation after deduplication is 100sat increase for a
>>> > package
>>> > of size 200vB, which correctly fails BIP125#4. Assume all transactions
>>> have
>>> > a
>>> > size of 100vB.
>>>
>>> What problem are you trying to solve by the package feerate *after*
>>> dedup rule ?
>>>
>>> My understanding is that an in-package transaction might be already in
>>> the mempool. Therefore, to compute a correct RBF penalty replacement, the
>>> vsize of this transaction could be discarded lowering the cost of package
>>> RBF.
>>>
>>> If we keep a "safe" dedup mechanism (see my point above), I think this
>>> discount is justified, as the validation cost of node operators is paid for
>>> ?
>>>
>>> > The child cannot replace mempool transactions.
>>>
>>> Let's say you issue package A+B, then package C+B', where B' is a child
>>> of both A and C. This rule fails the acceptance of C+B' ?
>>>
>>> I think this is a footgunish API, as if a package issuer send the
>>> multiple-parent-one-child package A,B,C,D where D is the child of A,B,C.
>>> Then try to broadcast the higher-feerate C'+D' package, it should be
>>> rejected. So it's breaking the naive broadcaster assumption that a
>>> higher-feerate/higher-fee package always replaces ? And it might be unsafe
>>> in protocols where states are symmetric. E.g a malicious counterparty
>>> broadcasts first S+A, then you honestly broadcast S+B, where B pays better
>>> fees.
>>>
>>> > All mempool transactions to be replaced must signal replaceability.
>>>
>>> I think this is unsafe for L2s if counterparties have malleability of
>>> the child transaction. They can block your package replacement by
>>> opting-out from RBF signaling. IIRC, LN's "anchor output" presents such an
>>> ability.
>>>
>>> I think it's better to either fix inherited signaling or move towards
>>> full-rbf.
>>>
>>> > if a package parent has already been submitted, it would
>>> > look
>>> >like the child is spending a "new" unconfirmed input.
>>>
>>> I think this is an issue brought by the trimming during the dedup phase.
>>> If we preserve the package integrity, only re-using the tx-level checks
>>> results of already in-mempool transactions to gain in CPU time we won't
>>> have this issue. Package childs can add unconfirmed inputs as long as
>>> they're in-package, the bip125 rule2 is only evaluated against parents ?
>>>
>>> > However, we still achieve the same goal of requiring the
>>> > replacement
>>> > transactions to have a ancestor score at least as high as the original
>>> > ones.
>>>
>>> I'm not sure if this holds...
>>>
>>> Let's say you have in-mempool A, B where A pays 10 sat/vb for 100 vbytes
>>> and B pays 10 sat/vb for 100 vbytes. You have the candidate replacement D
>>> spending both A and C where D pays 15sat/vb for 100 vbytes and C pays 1
>>> sat/vb for 1000 vbytes.
>>>
>>> Package A + B ancestor score is 10 sat/vb.
>>>
>>> D has a higher feerate/absolute fee than B.
>>>
>>> Package A + C + D ancestor score is ~ 3 sat/vb ((A's 1000 sats + C's
>>> 1000 sats + D's 1500 sats) /
>>> A's 100 vb + C's 1000 vb + D's 100 vb)
>>>
>>> Overall, this is a review through the lenses of LN requirements. I think
>>> other L2 protocols/applications
>>> could be candidates to using package accept/relay such as:
>>> * https://github.com/lightninglabs/pool
>>> * https://github.com/discreetlogcontracts/dlcspecs
>>> * https://github.com/bitcoin-teleport/teleport-transactions/
>>> * https://github.com/sapio-lang/sapio
>>> *
>>> https://github.com/commerceblock/mercury/blob/master/doc/statechains.md
>>> * https://github.com/revault/practical-revault
>>>
>>> Thanks for rolling forward the ball on this subject.
>>>
>>> Antoine
>>>
>>> Le jeu. 16 sept. 2021 à 03:55, Gloria Zhao via bitcoin-dev <
>>> bitcoin-dev at lists.linuxfoundation.org> a écrit :
>>>
>>>> Hi there,
>>>>
>>>> I'm writing to propose a set of mempool policy changes to enable package
>>>> validation (in preparation for package relay) in Bitcoin Core. These
>>>> would not
>>>> be consensus or P2P protocol changes. However, since mempool policy
>>>> significantly affects transaction propagation, I believe this is
>>>> relevant for
>>>> the mailing list.
>>>>
>>>> My proposal enables packages consisting of multiple parents and 1
>>>> child. If you
>>>> develop software that relies on specific transaction relay assumptions
>>>> and/or
>>>> are interested in using package relay in the future, I'm very
>>>> interested to hear
>>>> your feedback on the utility or restrictiveness of these package
>>>> policies for
>>>> your use cases.
>>>>
>>>> A draft implementation of this proposal can be found in [Bitcoin Core
>>>> PR#22290][1].
>>>>
>>>> An illustrated version of this post can be found at
>>>> https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a.
>>>> I have also linked the images below.
>>>>
>>>> ## Background
>>>>
>>>> Feel free to skip this section if you are already familiar with mempool
>>>> policy
>>>> and package relay terminology.
>>>>
>>>> ### Terminology Clarifications
>>>>
>>>> * Package = an ordered list of related transactions, representable by a
>>>> Directed
>>>> Acyclic Graph.
>>>> * Package Feerate = the total modified fees divided by the total
>>>> virtual size of
>>>> all transactions in the package.
>>>> - Modified fees = a transaction's base fees + fee delta applied by
>>>> the user
>>>> with `prioritisetransaction`. As such, we expect this to vary
>>>> across
>>>> mempools.
>>>> - Virtual Size = the maximum of virtual sizes calculated using
>>>> [BIP141
>>>> virtual size][2] and sigop weight. [Implemented here in Bitcoin
>>>> Core][3].
>>>> - Note that feerate is not necessarily based on the base fees and
>>>> serialized
>>>> size.
>>>>
>>>> * Fee-Bumping = user/wallet actions that take advantage of miner
>>>> incentives to
>>>> boost a transaction's candidacy for inclusion in a block, including
>>>> Child Pays
>>>> for Parent (CPFP) and [BIP125][12] Replace-by-Fee (RBF). Our intention
>>>> in
>>>> mempool policy is to recognize when the new transaction is more
>>>> economical to
>>>> mine than the original one(s) but not open DoS vectors, so there are
>>>> some
>>>> limitations.
>>>>
>>>> ### Policy
>>>>
>>>> The purpose of the mempool is to store the best (to be most
>>>> incentive-compatible
>>>> with miners, highest feerate) candidates for inclusion in a block.
>>>> Miners use
>>>> the mempool to build block templates. The mempool is also useful as a
>>>> cache for
>>>> boosting block relay and validation performance, aiding transaction
>>>> relay, and
>>>> generating feerate estimations.
>>>>
>>>> Ideally, all consensus-valid transactions paying reasonable fees should
>>>> make it
>>>> to miners through normal transaction relay, without any special
>>>> connectivity or
>>>> relationships with miners. On the other hand, nodes do not have
>>>> unlimited
>>>> resources, and a P2P network designed to let any honest node broadcast
>>>> their
>>>> transactions also exposes the transaction validation engine to DoS
>>>> attacks from
>>>> malicious peers.
>>>>
>>>> As such, for unconfirmed transactions we are considering for our
>>>> mempool, we
>>>> apply a set of validation rules in addition to consensus, primarily to
>>>> protect
>>>> us from resource exhaustion and aid our efforts to keep the highest fee
>>>> transactions. We call this mempool _policy_: a set of (configurable,
>>>> node-specific) rules that transactions must abide by in order to be
>>>> accepted
>>>> into our mempool. Transaction "Standardness" rules and mempool
>>>> restrictions such
>>>> as "too-long-mempool-chain" are both examples of policy.
>>>>
>>>> ### Package Relay and Package Mempool Accept
>>>>
>>>> In transaction relay, we currently consider transactions one at a time
>>>> for
>>>> submission to the mempool. This creates a limitation in the node's
>>>> ability to
>>>> determine which transactions have the highest feerates, since we cannot
>>>> take
>>>> into account descendants (i.e. cannot use CPFP) until all the
>>>> transactions are
>>>> in the mempool. Similarly, we cannot use a transaction's descendants
>>>> when
>>>> considering it for RBF. When an individual transaction does not meet
>>>> the mempool
>>>> minimum feerate and the user isn't able to create a replacement
>>>> transaction
>>>> directly, it will not be accepted by mempools.
>>>>
>>>> This limitation presents a security issue for applications and users
>>>> relying on
>>>> time-sensitive transactions. For example, Lightning and other protocols
>>>> create
>>>> UTXOs with multiple spending paths, where one counterparty's spending
>>>> path opens
>>>> up after a timelock, and users are protected from cheating scenarios as
>>>> long as
>>>> they redeem on-chain in time. A key security assumption is that all
>>>> parties'
>>>> transactions will propagate and confirm in a timely manner. This
>>>> assumption can
>>>> be broken if fee-bumping does not work as intended.
>>>>
>>>> The end goal for Package Relay is to consider multiple transactions at
>>>> the same
>>>> time, e.g. a transaction with its high-fee child. This may help us
>>>> better
>>>> determine whether transactions should be accepted to our mempool,
>>>> especially if
>>>> they don't meet fee requirements individually or are better RBF
>>>> candidates as a
>>>> package. A combination of changes to mempool validation logic, policy,
>>>> and
>>>> transaction relay allows us to better propagate the transactions with
>>>> the
>>>> highest package feerates to miners, and makes fee-bumping tools more
>>>> powerful
>>>> for users.
>>>>
>>>> The "relay" part of Package Relay suggests P2P messaging changes, but a
>>>> large
>>>> part of the changes are in the mempool's package validation logic. We
>>>> call this
>>>> *Package Mempool Accept*.
>>>>
>>>> ### Previous Work
>>>>
>>>> * Given that mempool validation is DoS-sensitive and complex, it would
>>>> be
>>>> dangerous to haphazardly tack on package validation logic. Many
>>>> efforts have
>>>> been made to make mempool validation less opaque (see [#16400][4],
>>>> [#21062][5],
>>>> [#22675][6], [#22796][7]).
>>>> * [#20833][8] Added basic capabilities for package validation, test
>>>> accepts only
>>>> (no submission to mempool).
>>>> * [#21800][9] Implemented package ancestor/descendant limit checks for
>>>> arbitrary
>>>> packages. Still test accepts only.
>>>> * Previous package relay proposals (see [#16401][10], [#19621][11]).
>>>>
>>>> ### Existing Package Rules
>>>>
>>>> These are in master as introduced in [#20833][8] and [#21800][9]. I'll
>>>> consider
>>>> them as "given" in the rest of this document, though they can be
>>>> changed, since
>>>> package validation is test-accept only right now.
>>>>
>>>> 1. A package cannot exceed `MAX_PACKAGE_COUNT=25` count and
>>>> `MAX_PACKAGE_SIZE=101KvB` total size [8]
>>>>
>>>> *Rationale*: This is already enforced as mempool ancestor/descendant
>>>> limits.
>>>> Presumably, transactions in a package are all related, so exceeding
>>>> this limit
>>>> would mean that the package can either be split up or it wouldn't pass
>>>> this
>>>> mempool policy.
>>>>
>>>> 2. Packages must be topologically sorted: if any dependencies exist
>>>> between
>>>> transactions, parents must appear somewhere before children. [8]
>>>>
>>>> 3. A package cannot have conflicting transactions, i.e. none of them
>>>> can spend
>>>> the same inputs. This also means there cannot be duplicate
>>>> transactions. [8]
>>>>
>>>> 4. When packages are evaluated against ancestor/descendant limits in a
>>>> test
>>>> accept, the union of all of their descendants and ancestors is
>>>> considered. This
>>>> is essentially a "worst case" heuristic where every transaction in the
>>>> package
>>>> is treated as each other's ancestor and descendant. [8]
>>>> Packages for which ancestor/descendant limits are accurately captured
>>>> by this
>>>> heuristic: [19]
>>>>
>>>> There are also limitations such as the fact that CPFP carve out is not
>>>> applied
>>>> to package transactions. #20833 also disables RBF in package
>>>> validation; this
>>>> proposal overrides that to allow packages to use RBF.
>>>>
>>>> ## Proposed Changes
>>>>
>>>> The next step in the Package Mempool Accept project is to implement
>>>> submission
>>>> to mempool, initially through RPC only. This allows us to test the
>>>> submission
>>>> logic before exposing it on P2P.
>>>>
>>>> ### Summary
>>>>
>>>> - Packages may contain already-in-mempool transactions.
>>>> - Packages are 2 generations, Multi-Parent-1-Child.
>>>> - Fee-related checks use the package feerate. This means that wallets
>>>> can
>>>> create a package that utilizes CPFP.
>>>> - Parents are allowed to RBF mempool transactions with a set of rules
>>>> similar
>>>> to BIP125. This enables a combination of CPFP and RBF, where a
>>>> transaction's descendant fees pay for replacing mempool conflicts.
>>>>
>>>> There is a draft implementation in [#22290][1]. It is WIP, but feedback
>>>> is
>>>> always welcome.
>>>>
>>>> ### Details
>>>>
>>>> #### Packages May Contain Already-in-Mempool Transactions
>>>>
>>>> A package may contain transactions that are already in the mempool. We
>>>> remove
>>>> ("deduplicate") those transactions from the package for the purposes of
>>>> package
>>>> mempool acceptance. If a package is empty after deduplication, we do
>>>> nothing.
>>>>
>>>> *Rationale*: Mempools vary across the network. It's possible for a
>>>> parent to be
>>>> accepted to the mempool of a peer on its own due to differences in
>>>> policy and
>>>> fee market fluctuations. We should not reject or penalize the entire
>>>> package for
>>>> an individual transaction as that could be a censorship vector.
>>>>
>>>> #### Packages Are Multi-Parent-1-Child
>>>>
>>>> Only packages of a specific topology are permitted. Namely, a package
>>>> is exactly
>>>> 1 child with all of its unconfirmed parents. After deduplication, the
>>>> package
>>>> may be exactly the same, empty, 1 child, 1 child with just some of its
>>>> unconfirmed parents, etc. Note that it's possible for the parents to be
>>>> indirect
>>>> descendants/ancestors of one another, or for parent and child to share
>>>> a parent,
>>>> so we cannot make any other topology assumptions.
>>>>
>>>> *Rationale*: This allows for fee-bumping by CPFP. Allowing multiple
>>>> parents
>>>> makes it possible to fee-bump a batch of transactions. Restricting
>>>> packages to a
>>>> defined topology is also easier to reason about and simplifies the
>>>> validation
>>>> logic greatly. Multi-parent-1-child allows us to think of the package
>>>> as one big
>>>> transaction, where:
>>>>
>>>> - Inputs = all the inputs of parents + inputs of the child that come
>>>> from
>>>> confirmed UTXOs
>>>> - Outputs = all the outputs of the child + all outputs of the parents
>>>> that
>>>> aren't spent by other transactions in the package
>>>>
>>>> Examples of packages that follow this rule (variations of example A
>>>> show some
>>>> possibilities after deduplication): ![image][15]
>>>>
>>>> #### Fee-Related Checks Use Package Feerate
>>>>
>>>> Package Feerate = the total modified fees divided by the total virtual
>>>> size of
>>>> all transactions in the package.
>>>>
>>>> To meet the two feerate requirements of a mempool, i.e., the
>>>> pre-configured
>>>> minimum relay feerate (`minRelayTxFee`) and dynamic mempool minimum
>>>> feerate, the
>>>> total package feerate is used instead of the individual feerate. The
>>>> individual
>>>> transactions are allowed to be below feerate requirements if the
>>>> package meets
>>>> the feerate requirements. For example, the parent(s) in the package can
>>>> have 0
>>>> fees but be paid for by the child.
>>>>
>>>> *Rationale*: This can be thought of as "CPFP within a package," solving
>>>> the
>>>> issue of a parent not meeting minimum fees on its own. This allows L2
>>>> applications to adjust their fees at broadcast time instead of
>>>> overshooting or
>>>> risking getting stuck/pinned.
>>>>
>>>> We use the package feerate of the package *after deduplication*.
>>>>
>>>> *Rationale*: It would be incorrect to use the fees of transactions
>>>> that are
>>>> already in the mempool, as we do not want a transaction's fees to be
>>>> double-counted for both its individual RBF and package RBF.
>>>>
>>>> Examples F and G [14] show the same package, but P1 is submitted
>>>> individually before
>>>> the package in example G. In example F, we can see that the 300vB
>>>> package pays
>>>> an additional 200sat in fees, which is not enough to pay for its own
>>>> bandwidth
>>>> (BIP125#4). In example G, we can see that P1 pays enough to replace M1,
>>>> but
>>>> using P1's fees again during package submission would make it look like
>>>> a 300sat
>>>> increase for a 200vB package. Even including its fees and size would
>>>> not be
>>>> sufficient in this example, since the 300sat looks like enough for the
>>>> 300vB
>>>> package. The calculcation after deduplication is 100sat increase for a
>>>> package
>>>> of size 200vB, which correctly fails BIP125#4. Assume all transactions
>>>> have a
>>>> size of 100vB.
>>>>
>>>> #### Package RBF
>>>>
>>>> If a package meets feerate requirements as a package, the parents in the
>>>> transaction are allowed to replace-by-fee mempool transactions. The
>>>> child cannot
>>>> replace mempool transactions. Multiple transactions can replace the same
>>>> transaction, but in order to be valid, none of the transactions can try
>>>> to
>>>> replace an ancestor of another transaction in the same package (which
>>>> would thus
>>>> make its inputs unavailable).
>>>>
>>>> *Rationale*: Even if we are using package feerate, a package will not
>>>> propagate
>>>> as intended if RBF still requires each individual transaction to meet
>>>> the
>>>> feerate requirements.
>>>>
>>>> We use a set of rules slightly modified from BIP125 as follows:
>>>>
>>>> ##### Signaling (Rule #1)
>>>>
>>>> All mempool transactions to be replaced must signal replaceability.
>>>>
>>>> *Rationale*: Package RBF signaling logic should be the same for package
>>>> RBF and
>>>> single transaction acceptance. This would be updated if single
>>>> transaction
>>>> validation moves to full RBF.
>>>>
>>>> ##### New Unconfirmed Inputs (Rule #2)
>>>>
>>>> A package may include new unconfirmed inputs, but the ancestor feerate
>>>> of the
>>>> child must be at least as high as the ancestor feerates of every
>>>> transaction
>>>> being replaced. This is contrary to BIP125#2, which states "The
>>>> replacement
>>>> transaction may only include an unconfirmed input if that input was
>>>> included in
>>>> one of the original transactions. (An unconfirmed input spends an
>>>> output from a
>>>> currently-unconfirmed transaction.)"
>>>>
>>>> *Rationale*: The purpose of BIP125#2 is to ensure that the replacement
>>>> transaction has a higher ancestor score than the original
>>>> transaction(s) (see
>>>> [comment][13]). Example H [16] shows how adding a new unconfirmed input
>>>> can lower the
>>>> ancestor score of the replacement transaction. P1 is trying to replace
>>>> M1, and
>>>> spends an unconfirmed output of M2. P1 pays 800sat, M1 pays 600sat, and
>>>> M2 pays
>>>> 100sat. Assume all transactions have a size of 100vB. While, in
>>>> isolation, P1
>>>> looks like a better mining candidate than M1, it must be mined with M2,
>>>> so its
>>>> ancestor feerate is actually 4.5sat/vB. This is lower than M1's
>>>> ancestor
>>>> feerate, which is 6sat/vB.
>>>>
>>>> In package RBF, the rule analogous to BIP125#2 would be "none of the
>>>> transactions in the package can spend new unconfirmed inputs." Example
>>>> J [17] shows
>>>> why, if any of the package transactions have ancestors, package feerate
>>>> is no
>>>> longer accurate. Even though M2 and M3 are not ancestors of P1 (which
>>>> is the
>>>> replacement transaction in an RBF), we're actually interested in the
>>>> entire
>>>> package. A miner should mine M1 which is 5sat/vB instead of M2, M3, P1,
>>>> P2, and
>>>> P3, which is only 4sat/vB. The Package RBF rule cannot be loosened to
>>>> only allow
>>>> the child to have new unconfirmed inputs, either, because it can still
>>>> cause us
>>>> to overestimate the package's ancestor score.
>>>>
>>>> However, enforcing a rule analogous to BIP125#2 would not only make
>>>> Package RBF
>>>> less useful, but would also break Package RBF for packages with parents
>>>> already
>>>> in the mempool: if a package parent has already been submitted, it
>>>> would look
>>>> like the child is spending a "new" unconfirmed input. In example K
>>>> [18], we're
>>>> looking to replace M1 with the entire package including P1, P2, and P3.
>>>> We must
>>>> consider the case where one of the parents is already in the mempool
>>>> (in this
>>>> case, P2), which means we must allow P3 to have new unconfirmed inputs.
>>>> However,
>>>> M2 lowers the ancestor score of P3 to 4.3sat/vB, so we should not
>>>> replace M1
>>>> with this package.
>>>>
>>>> Thus, the package RBF rule regarding new unconfirmed inputs is less
>>>> strict than
>>>> BIP125#2. However, we still achieve the same goal of requiring the
>>>> replacement
>>>> transactions to have a ancestor score at least as high as the original
>>>> ones. As
>>>> a result, the entire package is required to be a higher feerate mining
>>>> candidate
>>>> than each of the replaced transactions.
>>>>
>>>> Another note: the [comment][13] above the BIP125#2 code in the original
>>>> RBF
>>>> implementation suggests that the rule was intended to be temporary.
>>>>
>>>> ##### Absolute Fee (Rule #3)
>>>>
>>>> The package must increase the absolute fee of the mempool, i.e. the
>>>> total fees
>>>> of the package must be higher than the absolute fees of the mempool
>>>> transactions
>>>> it replaces. Combined with the CPFP rule above, this differs from
>>>> BIP125 Rule #3
>>>> - an individual transaction in the package may have lower fees than the
>>>> transaction(s) it is replacing. In fact, it may have 0 fees, and the
>>>> child
>>>> pays for RBF.
>>>>
>>>> ##### Feerate (Rule #4)
>>>>
>>>> The package must pay for its own bandwidth; the package feerate must be
>>>> higher
>>>> than the replaced transactions by at least minimum relay feerate
>>>> (`incrementalRelayFee`). Combined with the CPFP rule above, this
>>>> differs from
>>>> BIP125 Rule #4 - an individual transaction in the package can have a
>>>> lower
>>>> feerate than the transaction(s) it is replacing. In fact, it may have 0
>>>> fees,
>>>> and the child pays for RBF.
>>>>
>>>> ##### Total Number of Replaced Transactions (Rule #5)
>>>>
>>>> The package cannot replace more than 100 mempool transactions. This is
>>>> identical
>>>> to BIP125 Rule #5.
>>>>
>>>> ### Expected FAQs
>>>>
>>>> 1. Is it possible for only some of the package to make it into the
>>>> mempool?
>>>>
>>>> Yes, it is. However, since we evict transactions from the mempool by
>>>> descendant score and the package child is supposed to be sponsoring the
>>>> fees of
>>>> its parents, the most common scenario would be all-or-nothing. This is
>>>> incentive-compatible. In fact, to be conservative, package validation
>>>> should
>>>> begin by trying to submit all of the transactions individually, and
>>>> only use the
>>>> package mempool acceptance logic if the parents fail due to low feerate.
>>>>
>>>> 2. Should we allow packages to contain already-confirmed transactions?
>>>>
>>>> No, for practical reasons. In mempool validation, we actually
>>>> aren't able to
>>>> tell with 100% confidence if we are looking at a transaction that has
>>>> already
>>>> confirmed, because we look up inputs using a UTXO set. If we have
>>>> historical
>>>> block data, it's possible to look for it, but this is inefficient, not
>>>> always
>>>> possible for pruning nodes, and unnecessary because we're not going to
>>>> do
>>>> anything with the transaction anyway. As such, we already have the
>>>> expectation
>>>> that transaction relay is somewhat "stateful" i.e. nobody should be
>>>> relaying
>>>> transactions that have already been confirmed. Similarly, we shouldn't
>>>> be
>>>> relaying packages that contain already-confirmed transactions.
>>>>
>>>> [1]: https://github.com/bitcoin/bitcoin/pull/22290
>>>> [2]:
>>>> https://github.com/bitcoin/bips/blob/1f0b563738199ca60d32b4ba779797fc97d040fe/bip-0141.mediawiki#transaction-size-calculations
>>>> [3]:
>>>> https://github.com/bitcoin/bitcoin/blob/94f83534e4b771944af7d9ed0f40746f392eb75e/src/policy/policy.cpp#L282
>>>> [4]: https://github.com/bitcoin/bitcoin/pull/16400
>>>> [5]: https://github.com/bitcoin/bitcoin/pull/21062
>>>> [6]: https://github.com/bitcoin/bitcoin/pull/22675
>>>> [7]: https://github.com/bitcoin/bitcoin/pull/22796
>>>> [8]: https://github.com/bitcoin/bitcoin/pull/20833
>>>> [9]: https://github.com/bitcoin/bitcoin/pull/21800
>>>> [10]: https://github.com/bitcoin/bitcoin/pull/16401
>>>> [11]: https://github.com/bitcoin/bitcoin/pull/19621
>>>> [12]: https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki
>>>> [13]:
>>>> https://github.com/bitcoin/bitcoin/pull/6871/files#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2R1101-R1104
>>>> [14]:
>>>> https://user-images.githubusercontent.com/25183001/133567078-075a971c-0619-4339-9168-b41fd2b90c28.png
>>>> [15]:
>>>> https://user-images.githubusercontent.com/25183001/132856734-fc17da75-f875-44bb-b954-cb7a1725cc0d.png
>>>> [16]:
>>>> https://user-images.githubusercontent.com/25183001/133567347-a3e2e4a8-ae9c-49f8-abb9-81e8e0aba224.png
>>>> [17]:
>>>> https://user-images.githubusercontent.com/25183001/133567370-21566d0e-36c8-4831-b1a8-706634540af3.png
>>>> [18]:
>>>> https://user-images.githubusercontent.com/25183001/133567444-bfff1142-439f-4547-800a-2ba2b0242bcb.png
>>>> [19]:
>>>> https://user-images.githubusercontent.com/25183001/133456219-0bb447cb-dcb4-4a31-b9c1-7d86205b68bc.png
>>>> [20]:
>>>> https://user-images.githubusercontent.com/25183001/132857787-7b7c6f56-af96-44c8-8d78-983719888c19.png
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev at lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210923/d1576a8f/attachment-0001.html>

2023-06-07 22:59:16 UTC

- reply

📅 Original date posted:2021-09-20
📝 Original message:Hi Antoine,
First of all, thank you for the thorough review. I appreciate your insight
on LN requirements.
> IIUC, you have a package A+B+C submitted for acceptance and A is already
in your mempool. You trim out A from the package and then evaluate B+C.
> I think this might be an issue if A is the higher-fee element of the ABC
package. B+C package fees might be under the mempool min fee and will be
rejected, potentially breaking the acceptance expectations of the package
issuer ?
Correct, if B+C is too low feerate to be accepted, we will reject it. I
prefer this because it is incentive compatible: A can be mined by itself,
so there's no reason to prefer A+B+C instead of A.
As another way of looking at this, consider the case where we do accept
A+B+C and it sits at the "bottom" of our mempool. If our mempool reaches
capacity, we evict the lowest descendant feerate transactions, which are
B+C in this case. This gives us the same resulting mempool, with A and not
B+C.
> Further, I think the dedup should be done on wtxid, as you might have
multiple valid witnesses. Though with varying vsizes and as such offering
different feerates.
I agree that variations of the same package with different witnesses is a
case that must be handled. I consider witness replacement to be a project
that can be done in parallel to package mempool acceptance because being
able to accept packages does not worsen the problem of a
same-txid-different-witness "pinning" attack.
If or when we have witness replacement, the logic is: if the individual
transaction is enough to replace the mempool one, the replacement will
happen during the preceding individual transaction acceptance, and
deduplication logic will work. Otherwise, we will try to deduplicate by
wtxid, see that we need a package witness replacement, and use the package
feerate to evaluate whether this is economically rational.
See the #22290 "handle package transactions already in mempool" commit (
https://github.com/bitcoin/bitcoin/pull/22290/commits/fea75a2237b46cf76145242fecad7e274bfcb5ff),
which handles the case of same-txid-different-witness by simply using the
transaction in the mempool for now, with TODOs for what I just described.
> I'm not clearly understanding the accepted topologies. By "parent and
child to share a parent", do you mean the set of transactions A, B, C,
where B is spending A and C is spending A and B would be correct ?
Yes, that is what I meant. Yes, that would a valid package under these
rules.
> If yes, is there a width-limit introduced or we fallback on
MAX_PACKAGE_COUNT=25 ?
No, there is no limit on connectivity other than "child with all
unconfirmed parents." We will enforce MAX_PACKAGE_COUNT=25 and child's
in-mempool + in-package ancestor limits.
> Considering the current Core's mempool acceptance rules, I think CPFP
batching is unsafe for LN time-sensitive closure. A malicious tx-relay
jamming successful on one channel commitment transaction would contamine
the remaining commitments sharing the same package.
> E.g, you broadcast the package A+B+C+D+E where A,B,C,D are commitment
transactions and E a shared CPFP. If a malicious A' transaction has a
better feerate than A, the whole package acceptance will fail. Even if A'
confirms in the following block,
the propagation and confirmation of B+C+D have been delayed. This could
carry on a loss of funds.
Please note that A may replace A' even if A' has higher fees than A
individually, because the proposed package RBF utilizes the fees and size
of the entire package. This just requires E to pay enough fees, although
this can be pretty high if there are also potential B' and C' competing
commitment transactions that we don't know about.
> IMHO, I'm leaning towards deploying during a first phase
1-parent/1-child. I think it's the most conservative step still improving
second-layer safety.
So far, my understanding is that multi-parent-1-child is desired for
batched fee-bumping (
https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-897951289) and
I've also seen your response which I have less context on (
https://github.com/bitcoin/bitcoin/pull/22674#issuecomment-900352202). That
being said, I am happy to create a new proposal for 1 parent + 1 child
(which would be slightly simpler) and plan for moving to
multi-parent-1-child later if that is preferred. I am very interested in
hearing feedback on that approach.
> If A+B is submitted to replace A', where A pays 0 sats, B pays 200 sats
and A' pays 100 sats. If we apply the individual RBF on A, A+B acceptance
fails. For this reason I think the individual RBF should be bypassed and
only the package RBF apply ?
I think there is a misunderstanding here - let me describe what I'm
proposing we'd do in this situation: we'll try individual submission for A,
see that it fails due to "insufficient fees." Then, we'll try package
validation for A+B and use package RBF. If A+B pays enough, it can still
replace A'. If A fails for a bad signature, we won't look at B or A+B. Does
this meet your expectations?
> What problem are you trying to solve by the package feerate *after* dedup
rule ?
> My understanding is that an in-package transaction might be already in
the mempool. Therefore, to compute a correct RBF penalty replacement, the
vsize of this transaction could be discarded lowering the cost of package
RBF.
I'm proposing that, when a transaction has already been submitted to
mempool, we would ignore both its fees and vsize when calculating package
feerate. In example G2, we shouldn't count M1 fees after its submission to
mempool, since M1's fees have already been used to pay for its individual
bandwidth, and it shouldn't be used again to pay for P2 and P3's bandwidth.
We also shouldn't count its vsize, since it has already been paid for.
> I think this is a footgunish API, as if a package issuer send the
multiple-parent-one-child package A,B,C,D where D is the child of A,B,C.
Then try to broadcast the higher-feerate C'+D' package, it should be
rejected. So it's breaking the naive broadcaster assumption that a
higher-feerate/higher-fee package always replaces ?
Note that, if C' conflicts with C, it also conflicts with D, since D is a
descendant of C and would thus need to be evicted along with it.
Implicitly, D' would not be in conflict with D.
More generally, this example is surprising to me because I didn't think
packages would be used to fee-bump replaceable transactions. Do we want the
child to be able to replace mempool transactions as well? This can be
implemented with a bit of additional logic.
> I think this is unsafe for L2s if counterparties have malleability of the
child transaction. They can block your package replacement by opting-out
from RBF signaling. IIRC, LN's "anchor output" presents such an ability.
I'm not sure what you mean? Let's say we have a package of parent A + child
B, where A is supposed to replace a mempool transaction A'. Are you saying
that counterparties are able to malleate the package child B, or a child of
A'? If they can malleate a child of A', that shouldn't matter as long as A'
is signaling replacement. This would be handled identically with full RBF
and what Core currently implements.
> I think this is an issue brought by the trimming during the dedup phase.
If we preserve the package integrity, only re-using the tx-level checks
results of already in-mempool transactions to gain in CPU time we won't
have this issue. Package childs can add unconfirmed inputs as long as
they're in-package, the bip125 rule2 is only evaluated against parents ?
Sorry, I don't understand what you mean by "preserve the package
integrity?" Could you elaborate?
> Let's say you have in-mempool A, B where A pays 10 sat/vb for 100 vbytes
and B pays 10 sat/vb for 100 vbytes. You have the candidate replacement D
spending both A and C where D pays 15sat/vb for 100 vbytes and C pays 1
sat/vb for 1000 vbytes.
> Package A + B ancestor score is 10 sat/vb.
> D has a higher feerate/absolute fee than B.
> Package A + C + D ancestor score is ~ 3 sat/vb ((A's 1000 sats + C's 1000
sats + D's 1500 sats) / A's 100 vb + C's 1000 vb + D's 100 vb)
I am in agreement with your calculations but unsure if we disagree on the
expected outcome. Yes, B has an ancestor score of 10sat/vb and D has an
ancestor score of ~2.9sat/vb. Since D's ancestor score is lower than B's,
it fails the proposed package RBF Rule #2, so this package would be
rejected. Does this meet your expectations?
Thank you for linking to projects that might be interested in package relay
:)
Thanks,
Gloria
On Mon, Sep 20, 2021 at 12:16 AM Antoine Riard <antoine.riard at gmail.com>
wrote:
> Hi Gloria,
>
> > A package may contain transactions that are already in the mempool. We
> > remove
> > ("deduplicate") those transactions from the package for the purposes of
> > package
> > mempool acceptance. If a package is empty after deduplication, we do
> > nothing.
>
> IIUC, you have a package A+B+C submitted for acceptance and A is already
> in your mempool. You trim out A from the package and then evaluate B+C.
>
> I think this might be an issue if A is the higher-fee element of the ABC
> package. B+C package fees might be under the mempool min fee and will be
> rejected, potentially breaking the acceptance expectations of the package
> issuer ?
>
> Further, I think the dedup should be done on wtxid, as you might have
> multiple valid witnesses. Though with varying vsizes and as such offering
> different feerates.
>
> E.g you're going to evaluate the package A+B and A' is already in your
> mempool with a bigger valid witness. You trim A based on txid, then you
> evaluate A'+B, which fails the fee checks. However, evaluating A+B would
> have been a success.
>
> AFAICT, the dedup rationale would be to save on CPU time/IO disk, to avoid
> repeated signatures verification and parent UTXOs fetches ? Can we achieve
> the same goal by bypassing tx-level checks for already-in txn while
> conserving the package integrity for package-level checks ?
>
> > Note that it's possible for the parents to be
> > indirect
> > descendants/ancestors of one another, or for parent and child to share a
> > parent,
> > so we cannot make any other topology assumptions.
>
> I'm not clearly understanding the accepted topologies. By "parent and
> child to share a parent", do you mean the set of transactions A, B, C,
> where B is spending A and C is spending A and B would be correct ?
>
> If yes, is there a width-limit introduced or we fallback on
> MAX_PACKAGE_COUNT=25 ?
>
> IIRC, one rationale to come with this topology limitation was to lower the
> DoS risks when potentially deploying p2p packages.
>
> Considering the current Core's mempool acceptance rules, I think CPFP
> batching is unsafe for LN time-sensitive closure. A malicious tx-relay
> jamming successful on one channel commitment transaction would contamine
> the remaining commitments sharing the same package.
>
> E.g, you broadcast the package A+B+C+D+E where A,B,C,D are commitment
> transactions and E a shared CPFP. If a malicious A' transaction has a
> better feerate than A, the whole package acceptance will fail. Even if A'
> confirms in the following block,
> the propagation and confirmation of B+C+D have been delayed. This could
> carry on a loss of funds.
>
> That said, if you're broadcasting commitment transactions without
> time-sensitive HTLC outputs, I think the batching is effectively a fee
> saving as you don't have to duplicate the CPFP.
>
> IMHO, I'm leaning towards deploying during a first phase 1-parent/1-child.
> I think it's the most conservative step still improving second-layer safety.
>
> > *Rationale*: It would be incorrect to use the fees of transactions that
> are
> > already in the mempool, as we do not want a transaction's fees to be
> > double-counted for both its individual RBF and package RBF.
>
> I'm unsure about the logical order of the checks proposed.
>
> If A+B is submitted to replace A', where A pays 0 sats, B pays 200 sats
> and A' pays 100 sats. If we apply the individual RBF on A, A+B acceptance
> fails. For this reason I think the individual RBF should be bypassed and
> only the package RBF apply ?
>
> Note this situation is plausible, with current LN design, your
> counterparty can have a commitment transaction with a better fee just by
> selecting a higher `dust_limit_satoshis` than yours.
>
> > Examples F and G [14] show the same package, but P1 is submitted
> > individually before
> > the package in example G. In example F, we can see that the 300vB package
> > pays
> > an additional 200sat in fees, which is not enough to pay for its own
> > bandwidth
> > (BIP125#4). In example G, we can see that P1 pays enough to replace M1,
> but
> > using P1's fees again during package submission would make it look like a
> > 300sat
> > increase for a 200vB package. Even including its fees and size would not
> be
> > sufficient in this example, since the 300sat looks like enough for the
> 300vB
> > package. The calculcation after deduplication is 100sat increase for a
> > package
> > of size 200vB, which correctly fails BIP125#4. Assume all transactions
> have
> > a
> > size of 100vB.
>
> What problem are you trying to solve by the package feerate *after* dedup
> rule ?
>
> My understanding is that an in-package transaction might be already in the
> mempool. Therefore, to compute a correct RBF penalty replacement, the vsize
> of this transaction could be discarded lowering the cost of package RBF.
>
> If we keep a "safe" dedup mechanism (see my point above), I think this
> discount is justified, as the validation cost of node operators is paid for
> ?
>
> > The child cannot replace mempool transactions.
>
> Let's say you issue package A+B, then package C+B', where B' is a child of
> both A and C. This rule fails the acceptance of C+B' ?
>
> I think this is a footgunish API, as if a package issuer send the
> multiple-parent-one-child package A,B,C,D where D is the child of A,B,C.
> Then try to broadcast the higher-feerate C'+D' package, it should be
> rejected. So it's breaking the naive broadcaster assumption that a
> higher-feerate/higher-fee package always replaces ? And it might be unsafe
> in protocols where states are symmetric. E.g a malicious counterparty
> broadcasts first S+A, then you honestly broadcast S+B, where B pays better
> fees.
>
> > All mempool transactions to be replaced must signal replaceability.
>
> I think this is unsafe for L2s if counterparties have malleability of the
> child transaction. They can block your package replacement by opting-out
> from RBF signaling. IIRC, LN's "anchor output" presents such an ability.
>
> I think it's better to either fix inherited signaling or move towards
> full-rbf.
>
> > if a package parent has already been submitted, it would
> > look
> >like the child is spending a "new" unconfirmed input.
>
> I think this is an issue brought by the trimming during the dedup phase.
> If we preserve the package integrity, only re-using the tx-level checks
> results of already in-mempool transactions to gain in CPU time we won't
> have this issue. Package childs can add unconfirmed inputs as long as
> they're in-package, the bip125 rule2 is only evaluated against parents ?
>
> > However, we still achieve the same goal of requiring the
> > replacement
> > transactions to have a ancestor score at least as high as the original
> > ones.
>
> I'm not sure if this holds...
>
> Let's say you have in-mempool A, B where A pays 10 sat/vb for 100 vbytes
> and B pays 10 sat/vb for 100 vbytes. You have the candidate replacement D
> spending both A and C where D pays 15sat/vb for 100 vbytes and C pays 1
> sat/vb for 1000 vbytes.
>
> Package A + B ancestor score is 10 sat/vb.
>
> D has a higher feerate/absolute fee than B.
>
> Package A + C + D ancestor score is ~ 3 sat/vb ((A's 1000 sats + C's 1000
> sats + D's 1500 sats) /
> A's 100 vb + C's 1000 vb + D's 100 vb)
>
> Overall, this is a review through the lenses of LN requirements. I think
> other L2 protocols/applications
> could be candidates to using package accept/relay such as:
> * https://github.com/lightninglabs/pool
> * https://github.com/discreetlogcontracts/dlcspecs
> * https://github.com/bitcoin-teleport/teleport-transactions/
> * https://github.com/sapio-lang/sapio
> * https://github.com/commerceblock/mercury/blob/master/doc/statechains.md
> * https://github.com/revault/practical-revault
>
> Thanks for rolling forward the ball on this subject.
>
> Antoine
>
> Le jeu. 16 sept. 2021 à 03:55, Gloria Zhao via bitcoin-dev <
> bitcoin-dev at lists.linuxfoundation.org> a écrit :
>
>> Hi there,
>>
>> I'm writing to propose a set of mempool policy changes to enable package
>> validation (in preparation for package relay) in Bitcoin Core. These
>> would not
>> be consensus or P2P protocol changes. However, since mempool policy
>> significantly affects transaction propagation, I believe this is relevant
>> for
>> the mailing list.
>>
>> My proposal enables packages consisting of multiple parents and 1 child.
>> If you
>> develop software that relies on specific transaction relay assumptions
>> and/or
>> are interested in using package relay in the future, I'm very interested
>> to hear
>> your feedback on the utility or restrictiveness of these package policies
>> for
>> your use cases.
>>
>> A draft implementation of this proposal can be found in [Bitcoin Core
>> PR#22290][1].
>>
>> An illustrated version of this post can be found at
>> https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a.
>> I have also linked the images below.
>>
>> ## Background
>>
>> Feel free to skip this section if you are already familiar with mempool
>> policy
>> and package relay terminology.
>>
>> ### Terminology Clarifications
>>
>> * Package = an ordered list of related transactions, representable by a
>> Directed
>> Acyclic Graph.
>> * Package Feerate = the total modified fees divided by the total virtual
>> size of
>> all transactions in the package.
>> - Modified fees = a transaction's base fees + fee delta applied by
>> the user
>> with `prioritisetransaction`. As such, we expect this to vary across
>> mempools.
>> - Virtual Size = the maximum of virtual sizes calculated using [BIP141
>> virtual size][2] and sigop weight. [Implemented here in Bitcoin
>> Core][3].
>> - Note that feerate is not necessarily based on the base fees and
>> serialized
>> size.
>>
>> * Fee-Bumping = user/wallet actions that take advantage of miner
>> incentives to
>> boost a transaction's candidacy for inclusion in a block, including
>> Child Pays
>> for Parent (CPFP) and [BIP125][12] Replace-by-Fee (RBF). Our intention in
>> mempool policy is to recognize when the new transaction is more
>> economical to
>> mine than the original one(s) but not open DoS vectors, so there are some
>> limitations.
>>
>> ### Policy
>>
>> The purpose of the mempool is to store the best (to be most
>> incentive-compatible
>> with miners, highest feerate) candidates for inclusion in a block. Miners
>> use
>> the mempool to build block templates. The mempool is also useful as a
>> cache for
>> boosting block relay and validation performance, aiding transaction
>> relay, and
>> generating feerate estimations.
>>
>> Ideally, all consensus-valid transactions paying reasonable fees should
>> make it
>> to miners through normal transaction relay, without any special
>> connectivity or
>> relationships with miners. On the other hand, nodes do not have unlimited
>> resources, and a P2P network designed to let any honest node broadcast
>> their
>> transactions also exposes the transaction validation engine to DoS
>> attacks from
>> malicious peers.
>>
>> As such, for unconfirmed transactions we are considering for our mempool,
>> we
>> apply a set of validation rules in addition to consensus, primarily to
>> protect
>> us from resource exhaustion and aid our efforts to keep the highest fee
>> transactions. We call this mempool _policy_: a set of (configurable,
>> node-specific) rules that transactions must abide by in order to be
>> accepted
>> into our mempool. Transaction "Standardness" rules and mempool
>> restrictions such
>> as "too-long-mempool-chain" are both examples of policy.
>>
>> ### Package Relay and Package Mempool Accept
>>
>> In transaction relay, we currently consider transactions one at a time for
>> submission to the mempool. This creates a limitation in the node's
>> ability to
>> determine which transactions have the highest feerates, since we cannot
>> take
>> into account descendants (i.e. cannot use CPFP) until all the
>> transactions are
>> in the mempool. Similarly, we cannot use a transaction's descendants when
>> considering it for RBF. When an individual transaction does not meet the
>> mempool
>> minimum feerate and the user isn't able to create a replacement
>> transaction
>> directly, it will not be accepted by mempools.
>>
>> This limitation presents a security issue for applications and users
>> relying on
>> time-sensitive transactions. For example, Lightning and other protocols
>> create
>> UTXOs with multiple spending paths, where one counterparty's spending
>> path opens
>> up after a timelock, and users are protected from cheating scenarios as
>> long as
>> they redeem on-chain in time. A key security assumption is that all
>> parties'
>> transactions will propagate and confirm in a timely manner. This
>> assumption can
>> be broken if fee-bumping does not work as intended.
>>
>> The end goal for Package Relay is to consider multiple transactions at
>> the same
>> time, e.g. a transaction with its high-fee child. This may help us better
>> determine whether transactions should be accepted to our mempool,
>> especially if
>> they don't meet fee requirements individually or are better RBF
>> candidates as a
>> package. A combination of changes to mempool validation logic, policy, and
>> transaction relay allows us to better propagate the transactions with the
>> highest package feerates to miners, and makes fee-bumping tools more
>> powerful
>> for users.
>>
>> The "relay" part of Package Relay suggests P2P messaging changes, but a
>> large
>> part of the changes are in the mempool's package validation logic. We
>> call this
>> *Package Mempool Accept*.
>>
>> ### Previous Work
>>
>> * Given that mempool validation is DoS-sensitive and complex, it would be
>> dangerous to haphazardly tack on package validation logic. Many efforts
>> have
>> been made to make mempool validation less opaque (see [#16400][4],
>> [#21062][5],
>> [#22675][6], [#22796][7]).
>> * [#20833][8] Added basic capabilities for package validation, test
>> accepts only
>> (no submission to mempool).
>> * [#21800][9] Implemented package ancestor/descendant limit checks for
>> arbitrary
>> packages. Still test accepts only.
>> * Previous package relay proposals (see [#16401][10], [#19621][11]).
>>
>> ### Existing Package Rules
>>
>> These are in master as introduced in [#20833][8] and [#21800][9]. I'll
>> consider
>> them as "given" in the rest of this document, though they can be changed,
>> since
>> package validation is test-accept only right now.
>>
>> 1. A package cannot exceed `MAX_PACKAGE_COUNT=25` count and
>> `MAX_PACKAGE_SIZE=101KvB` total size [8]
>>
>> *Rationale*: This is already enforced as mempool ancestor/descendant
>> limits.
>> Presumably, transactions in a package are all related, so exceeding this
>> limit
>> would mean that the package can either be split up or it wouldn't pass
>> this
>> mempool policy.
>>
>> 2. Packages must be topologically sorted: if any dependencies exist
>> between
>> transactions, parents must appear somewhere before children. [8]
>>
>> 3. A package cannot have conflicting transactions, i.e. none of them can
>> spend
>> the same inputs. This also means there cannot be duplicate transactions.
>> [8]
>>
>> 4. When packages are evaluated against ancestor/descendant limits in a
>> test
>> accept, the union of all of their descendants and ancestors is
>> considered. This
>> is essentially a "worst case" heuristic where every transaction in the
>> package
>> is treated as each other's ancestor and descendant. [8]
>> Packages for which ancestor/descendant limits are accurately captured by
>> this
>> heuristic: [19]
>>
>> There are also limitations such as the fact that CPFP carve out is not
>> applied
>> to package transactions. #20833 also disables RBF in package validation;
>> this
>> proposal overrides that to allow packages to use RBF.
>>
>> ## Proposed Changes
>>
>> The next step in the Package Mempool Accept project is to implement
>> submission
>> to mempool, initially through RPC only. This allows us to test the
>> submission
>> logic before exposing it on P2P.
>>
>> ### Summary
>>
>> - Packages may contain already-in-mempool transactions.
>> - Packages are 2 generations, Multi-Parent-1-Child.
>> - Fee-related checks use the package feerate. This means that wallets can
>> create a package that utilizes CPFP.
>> - Parents are allowed to RBF mempool transactions with a set of rules
>> similar
>> to BIP125. This enables a combination of CPFP and RBF, where a
>> transaction's descendant fees pay for replacing mempool conflicts.
>>
>> There is a draft implementation in [#22290][1]. It is WIP, but feedback is
>> always welcome.
>>
>> ### Details
>>
>> #### Packages May Contain Already-in-Mempool Transactions
>>
>> A package may contain transactions that are already in the mempool. We
>> remove
>> ("deduplicate") those transactions from the package for the purposes of
>> package
>> mempool acceptance. If a package is empty after deduplication, we do
>> nothing.
>>
>> *Rationale*: Mempools vary across the network. It's possible for a parent
>> to be
>> accepted to the mempool of a peer on its own due to differences in policy
>> and
>> fee market fluctuations. We should not reject or penalize the entire
>> package for
>> an individual transaction as that could be a censorship vector.
>>
>> #### Packages Are Multi-Parent-1-Child
>>
>> Only packages of a specific topology are permitted. Namely, a package is
>> exactly
>> 1 child with all of its unconfirmed parents. After deduplication, the
>> package
>> may be exactly the same, empty, 1 child, 1 child with just some of its
>> unconfirmed parents, etc. Note that it's possible for the parents to be
>> indirect
>> descendants/ancestors of one another, or for parent and child to share a
>> parent,
>> so we cannot make any other topology assumptions.
>>
>> *Rationale*: This allows for fee-bumping by CPFP. Allowing multiple
>> parents
>> makes it possible to fee-bump a batch of transactions. Restricting
>> packages to a
>> defined topology is also easier to reason about and simplifies the
>> validation
>> logic greatly. Multi-parent-1-child allows us to think of the package as
>> one big
>> transaction, where:
>>
>> - Inputs = all the inputs of parents + inputs of the child that come from
>> confirmed UTXOs
>> - Outputs = all the outputs of the child + all outputs of the parents that
>> aren't spent by other transactions in the package
>>
>> Examples of packages that follow this rule (variations of example A show
>> some
>> possibilities after deduplication): ![image][15]
>>
>> #### Fee-Related Checks Use Package Feerate
>>
>> Package Feerate = the total modified fees divided by the total virtual
>> size of
>> all transactions in the package.
>>
>> To meet the two feerate requirements of a mempool, i.e., the
>> pre-configured
>> minimum relay feerate (`minRelayTxFee`) and dynamic mempool minimum
>> feerate, the
>> total package feerate is used instead of the individual feerate. The
>> individual
>> transactions are allowed to be below feerate requirements if the package
>> meets
>> the feerate requirements. For example, the parent(s) in the package can
>> have 0
>> fees but be paid for by the child.
>>
>> *Rationale*: This can be thought of as "CPFP within a package," solving
>> the
>> issue of a parent not meeting minimum fees on its own. This allows L2
>> applications to adjust their fees at broadcast time instead of
>> overshooting or
>> risking getting stuck/pinned.
>>
>> We use the package feerate of the package *after deduplication*.
>>
>> *Rationale*: It would be incorrect to use the fees of transactions that
>> are
>> already in the mempool, as we do not want a transaction's fees to be
>> double-counted for both its individual RBF and package RBF.
>>
>> Examples F and G [14] show the same package, but P1 is submitted
>> individually before
>> the package in example G. In example F, we can see that the 300vB package
>> pays
>> an additional 200sat in fees, which is not enough to pay for its own
>> bandwidth
>> (BIP125#4). In example G, we can see that P1 pays enough to replace M1,
>> but
>> using P1's fees again during package submission would make it look like a
>> 300sat
>> increase for a 200vB package. Even including its fees and size would not
>> be
>> sufficient in this example, since the 300sat looks like enough for the
>> 300vB
>> package. The calculcation after deduplication is 100sat increase for a
>> package
>> of size 200vB, which correctly fails BIP125#4. Assume all transactions
>> have a
>> size of 100vB.
>>
>> #### Package RBF
>>
>> If a package meets feerate requirements as a package, the parents in the
>> transaction are allowed to replace-by-fee mempool transactions. The child
>> cannot
>> replace mempool transactions. Multiple transactions can replace the same
>> transaction, but in order to be valid, none of the transactions can try to
>> replace an ancestor of another transaction in the same package (which
>> would thus
>> make its inputs unavailable).
>>
>> *Rationale*: Even if we are using package feerate, a package will not
>> propagate
>> as intended if RBF still requires each individual transaction to meet the
>> feerate requirements.
>>
>> We use a set of rules slightly modified from BIP125 as follows:
>>
>> ##### Signaling (Rule #1)
>>
>> All mempool transactions to be replaced must signal replaceability.
>>
>> *Rationale*: Package RBF signaling logic should be the same for package
>> RBF and
>> single transaction acceptance. This would be updated if single transaction
>> validation moves to full RBF.
>>
>> ##### New Unconfirmed Inputs (Rule #2)
>>
>> A package may include new unconfirmed inputs, but the ancestor feerate of
>> the
>> child must be at least as high as the ancestor feerates of every
>> transaction
>> being replaced. This is contrary to BIP125#2, which states "The
>> replacement
>> transaction may only include an unconfirmed input if that input was
>> included in
>> one of the original transactions. (An unconfirmed input spends an output
>> from a
>> currently-unconfirmed transaction.)"
>>
>> *Rationale*: The purpose of BIP125#2 is to ensure that the replacement
>> transaction has a higher ancestor score than the original transaction(s)
>> (see
>> [comment][13]). Example H [16] shows how adding a new unconfirmed input
>> can lower the
>> ancestor score of the replacement transaction. P1 is trying to replace
>> M1, and
>> spends an unconfirmed output of M2. P1 pays 800sat, M1 pays 600sat, and
>> M2 pays
>> 100sat. Assume all transactions have a size of 100vB. While, in
>> isolation, P1
>> looks like a better mining candidate than M1, it must be mined with M2,
>> so its
>> ancestor feerate is actually 4.5sat/vB. This is lower than M1's ancestor
>> feerate, which is 6sat/vB.
>>
>> In package RBF, the rule analogous to BIP125#2 would be "none of the
>> transactions in the package can spend new unconfirmed inputs." Example J
>> [17] shows
>> why, if any of the package transactions have ancestors, package feerate
>> is no
>> longer accurate. Even though M2 and M3 are not ancestors of P1 (which is
>> the
>> replacement transaction in an RBF), we're actually interested in the
>> entire
>> package. A miner should mine M1 which is 5sat/vB instead of M2, M3, P1,
>> P2, and
>> P3, which is only 4sat/vB. The Package RBF rule cannot be loosened to
>> only allow
>> the child to have new unconfirmed inputs, either, because it can still
>> cause us
>> to overestimate the package's ancestor score.
>>
>> However, enforcing a rule analogous to BIP125#2 would not only make
>> Package RBF
>> less useful, but would also break Package RBF for packages with parents
>> already
>> in the mempool: if a package parent has already been submitted, it would
>> look
>> like the child is spending a "new" unconfirmed input. In example K [18],
>> we're
>> looking to replace M1 with the entire package including P1, P2, and P3.
>> We must
>> consider the case where one of the parents is already in the mempool (in
>> this
>> case, P2), which means we must allow P3 to have new unconfirmed inputs.
>> However,
>> M2 lowers the ancestor score of P3 to 4.3sat/vB, so we should not replace
>> M1
>> with this package.
>>
>> Thus, the package RBF rule regarding new unconfirmed inputs is less
>> strict than
>> BIP125#2. However, we still achieve the same goal of requiring the
>> replacement
>> transactions to have a ancestor score at least as high as the original
>> ones. As
>> a result, the entire package is required to be a higher feerate mining
>> candidate
>> than each of the replaced transactions.
>>
>> Another note: the [comment][13] above the BIP125#2 code in the original
>> RBF
>> implementation suggests that the rule was intended to be temporary.
>>
>> ##### Absolute Fee (Rule #3)
>>
>> The package must increase the absolute fee of the mempool, i.e. the total
>> fees
>> of the package must be higher than the absolute fees of the mempool
>> transactions
>> it replaces. Combined with the CPFP rule above, this differs from BIP125
>> Rule #3
>> - an individual transaction in the package may have lower fees than the
>> transaction(s) it is replacing. In fact, it may have 0 fees, and the
>> child
>> pays for RBF.
>>
>> ##### Feerate (Rule #4)
>>
>> The package must pay for its own bandwidth; the package feerate must be
>> higher
>> than the replaced transactions by at least minimum relay feerate
>> (`incrementalRelayFee`). Combined with the CPFP rule above, this differs
>> from
>> BIP125 Rule #4 - an individual transaction in the package can have a lower
>> feerate than the transaction(s) it is replacing. In fact, it may have 0
>> fees,
>> and the child pays for RBF.
>>
>> ##### Total Number of Replaced Transactions (Rule #5)
>>
>> The package cannot replace more than 100 mempool transactions. This is
>> identical
>> to BIP125 Rule #5.
>>
>> ### Expected FAQs
>>
>> 1. Is it possible for only some of the package to make it into the
>> mempool?
>>
>> Yes, it is. However, since we evict transactions from the mempool by
>> descendant score and the package child is supposed to be sponsoring the
>> fees of
>> its parents, the most common scenario would be all-or-nothing. This is
>> incentive-compatible. In fact, to be conservative, package validation
>> should
>> begin by trying to submit all of the transactions individually, and only
>> use the
>> package mempool acceptance logic if the parents fail due to low feerate.
>>
>> 2. Should we allow packages to contain already-confirmed transactions?
>>
>> No, for practical reasons. In mempool validation, we actually aren't
>> able to
>> tell with 100% confidence if we are looking at a transaction that has
>> already
>> confirmed, because we look up inputs using a UTXO set. If we have
>> historical
>> block data, it's possible to look for it, but this is inefficient, not
>> always
>> possible for pruning nodes, and unnecessary because we're not going to do
>> anything with the transaction anyway. As such, we already have the
>> expectation
>> that transaction relay is somewhat "stateful" i.e. nobody should be
>> relaying
>> transactions that have already been confirmed. Similarly, we shouldn't be
>> relaying packages that contain already-confirmed transactions.
>>
>> [1]: https://github.com/bitcoin/bitcoin/pull/22290
>> [2]:
>> https://github.com/bitcoin/bips/blob/1f0b563738199ca60d32b4ba779797fc97d040fe/bip-0141.mediawiki#transaction-size-calculations
>> [3]:
>> https://github.com/bitcoin/bitcoin/blob/94f83534e4b771944af7d9ed0f40746f392eb75e/src/policy/policy.cpp#L282
>> [4]: https://github.com/bitcoin/bitcoin/pull/16400
>> [5]: https://github.com/bitcoin/bitcoin/pull/21062
>> [6]: https://github.com/bitcoin/bitcoin/pull/22675
>> [7]: https://github.com/bitcoin/bitcoin/pull/22796
>> [8]: https://github.com/bitcoin/bitcoin/pull/20833
>> [9]: https://github.com/bitcoin/bitcoin/pull/21800
>> [10]: https://github.com/bitcoin/bitcoin/pull/16401
>> [11]: https://github.com/bitcoin/bitcoin/pull/19621
>> [12]: https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki
>> [13]:
>> https://github.com/bitcoin/bitcoin/pull/6871/files#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2R1101-R1104
>> [14]:
>> https://user-images.githubusercontent.com/25183001/133567078-075a971c-0619-4339-9168-b41fd2b90c28.png
>> [15]:
>> https://user-images.githubusercontent.com/25183001/132856734-fc17da75-f875-44bb-b954-cb7a1725cc0d.png
>> [16]:
>> https://user-images.githubusercontent.com/25183001/133567347-a3e2e4a8-ae9c-49f8-abb9-81e8e0aba224.png
>> [17]:
>> https://user-images.githubusercontent.com/25183001/133567370-21566d0e-36c8-4831-b1a8-706634540af3.png
>> [18]:
>> https://user-images.githubusercontent.com/25183001/133567444-bfff1142-439f-4547-800a-2ba2b0242bcb.png
>> [19]:
>> https://user-images.githubusercontent.com/25183001/133456219-0bb447cb-dcb4-4a31-b9c1-7d86205b68bc.png
>> [20]:
>> https://user-images.githubusercontent.com/25183001/132857787-7b7c6f56-af96-44c8-8d78-983719888c19.png
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210920/09a51edf/attachment-0001.html>

2023-06-07 22:59:15 UTC

- reply

📅 Original date posted:2021-09-16
📝 Original message:Hi there,
I'm writing to propose a set of mempool policy changes to enable package
validation (in preparation for package relay) in Bitcoin Core. These would
not
be consensus or P2P protocol changes. However, since mempool policy
significantly affects transaction propagation, I believe this is relevant
for
the mailing list.
My proposal enables packages consisting of multiple parents and 1 child. If
you
develop software that relies on specific transaction relay assumptions
and/or
are interested in using package relay in the future, I'm very interested to
hear
your feedback on the utility or restrictiveness of these package policies
for
your use cases.
A draft implementation of this proposal can be found in [Bitcoin Core
PR#22290][1].
An illustrated version of this post can be found at
https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a.
I have also linked the images below.
## Background
Feel free to skip this section if you are already familiar with mempool
policy
and package relay terminology.
### Terminology Clarifications
* Package = an ordered list of related transactions, representable by a
Directed
Acyclic Graph.
* Package Feerate = the total modified fees divided by the total virtual
size of
all transactions in the package.
- Modified fees = a transaction's base fees + fee delta applied by the
user
with `prioritisetransaction`. As such, we expect this to vary across
mempools.
- Virtual Size = the maximum of virtual sizes calculated using [BIP141
virtual size][2] and sigop weight. [Implemented here in Bitcoin
Core][3].
- Note that feerate is not necessarily based on the base fees and
serialized
size.
* Fee-Bumping = user/wallet actions that take advantage of miner incentives
to
boost a transaction's candidacy for inclusion in a block, including Child
Pays
for Parent (CPFP) and [BIP125][12] Replace-by-Fee (RBF). Our intention in
mempool policy is to recognize when the new transaction is more economical
to
mine than the original one(s) but not open DoS vectors, so there are some
limitations.
### Policy
The purpose of the mempool is to store the best (to be most
incentive-compatible
with miners, highest feerate) candidates for inclusion in a block. Miners
use
the mempool to build block templates. The mempool is also useful as a cache
for
boosting block relay and validation performance, aiding transaction relay,
and
generating feerate estimations.
Ideally, all consensus-valid transactions paying reasonable fees should
make it
to miners through normal transaction relay, without any special
connectivity or
relationships with miners. On the other hand, nodes do not have unlimited
resources, and a P2P network designed to let any honest node broadcast their
transactions also exposes the transaction validation engine to DoS attacks
from
malicious peers.
As such, for unconfirmed transactions we are considering for our mempool, we
apply a set of validation rules in addition to consensus, primarily to
protect
us from resource exhaustion and aid our efforts to keep the highest fee
transactions. We call this mempool _policy_: a set of (configurable,
node-specific) rules that transactions must abide by in order to be accepted
into our mempool. Transaction "Standardness" rules and mempool restrictions
such
as "too-long-mempool-chain" are both examples of policy.
### Package Relay and Package Mempool Accept
In transaction relay, we currently consider transactions one at a time for
submission to the mempool. This creates a limitation in the node's ability
to
determine which transactions have the highest feerates, since we cannot take
into account descendants (i.e. cannot use CPFP) until all the transactions
are
in the mempool. Similarly, we cannot use a transaction's descendants when
considering it for RBF. When an individual transaction does not meet the
mempool
minimum feerate and the user isn't able to create a replacement transaction
directly, it will not be accepted by mempools.
This limitation presents a security issue for applications and users
relying on
time-sensitive transactions. For example, Lightning and other protocols
create
UTXOs with multiple spending paths, where one counterparty's spending path
opens
up after a timelock, and users are protected from cheating scenarios as
long as
they redeem on-chain in time. A key security assumption is that all parties'
transactions will propagate and confirm in a timely manner. This assumption
can
be broken if fee-bumping does not work as intended.
The end goal for Package Relay is to consider multiple transactions at the
same
time, e.g. a transaction with its high-fee child. This may help us better
determine whether transactions should be accepted to our mempool,
especially if
they don't meet fee requirements individually or are better RBF candidates
as a
package. A combination of changes to mempool validation logic, policy, and
transaction relay allows us to better propagate the transactions with the
highest package feerates to miners, and makes fee-bumping tools more
powerful
for users.
The "relay" part of Package Relay suggests P2P messaging changes, but a
large
part of the changes are in the mempool's package validation logic. We call
this
*Package Mempool Accept*.
### Previous Work
* Given that mempool validation is DoS-sensitive and complex, it would be
dangerous to haphazardly tack on package validation logic. Many efforts
have
been made to make mempool validation less opaque (see [#16400][4],
[#21062][5],
[#22675][6], [#22796][7]).
* [#20833][8] Added basic capabilities for package validation, test accepts
only
(no submission to mempool).
* [#21800][9] Implemented package ancestor/descendant limit checks for
arbitrary
packages. Still test accepts only.
* Previous package relay proposals (see [#16401][10], [#19621][11]).
### Existing Package Rules
These are in master as introduced in [#20833][8] and [#21800][9]. I'll
consider
them as "given" in the rest of this document, though they can be changed,
since
package validation is test-accept only right now.
1. A package cannot exceed `MAX_PACKAGE_COUNT=25` count and
`MAX_PACKAGE_SIZE=101KvB` total size [8]
*Rationale*: This is already enforced as mempool ancestor/descendant
limits.
Presumably, transactions in a package are all related, so exceeding this
limit
would mean that the package can either be split up or it wouldn't pass this
mempool policy.
2. Packages must be topologically sorted: if any dependencies exist between
transactions, parents must appear somewhere before children. [8]
3. A package cannot have conflicting transactions, i.e. none of them can
spend
the same inputs. This also means there cannot be duplicate transactions. [8]
4. When packages are evaluated against ancestor/descendant limits in a test
accept, the union of all of their descendants and ancestors is considered.
This
is essentially a "worst case" heuristic where every transaction in the
package
is treated as each other's ancestor and descendant. [8]
Packages for which ancestor/descendant limits are accurately captured by
this
heuristic: [19]
There are also limitations such as the fact that CPFP carve out is not
applied
to package transactions. #20833 also disables RBF in package validation;
this
proposal overrides that to allow packages to use RBF.
## Proposed Changes
The next step in the Package Mempool Accept project is to implement
submission
to mempool, initially through RPC only. This allows us to test the
submission
logic before exposing it on P2P.
### Summary
- Packages may contain already-in-mempool transactions.
- Packages are 2 generations, Multi-Parent-1-Child.
- Fee-related checks use the package feerate. This means that wallets can
create a package that utilizes CPFP.
- Parents are allowed to RBF mempool transactions with a set of rules
similar
to BIP125. This enables a combination of CPFP and RBF, where a
transaction's descendant fees pay for replacing mempool conflicts.
There is a draft implementation in [#22290][1]. It is WIP, but feedback is
always welcome.
### Details
#### Packages May Contain Already-in-Mempool Transactions
A package may contain transactions that are already in the mempool. We
remove
("deduplicate") those transactions from the package for the purposes of
package
mempool acceptance. If a package is empty after deduplication, we do
nothing.
*Rationale*: Mempools vary across the network. It's possible for a parent
to be
accepted to the mempool of a peer on its own due to differences in policy
and
fee market fluctuations. We should not reject or penalize the entire
package for
an individual transaction as that could be a censorship vector.
#### Packages Are Multi-Parent-1-Child
Only packages of a specific topology are permitted. Namely, a package is
exactly
1 child with all of its unconfirmed parents. After deduplication, the
package
may be exactly the same, empty, 1 child, 1 child with just some of its
unconfirmed parents, etc. Note that it's possible for the parents to be
indirect
descendants/ancestors of one another, or for parent and child to share a
parent,
so we cannot make any other topology assumptions.
*Rationale*: This allows for fee-bumping by CPFP. Allowing multiple parents
makes it possible to fee-bump a batch of transactions. Restricting packages
to a
defined topology is also easier to reason about and simplifies the
validation
logic greatly. Multi-parent-1-child allows us to think of the package as
one big
transaction, where:
- Inputs = all the inputs of parents + inputs of the child that come from
confirmed UTXOs
- Outputs = all the outputs of the child + all outputs of the parents that
aren't spent by other transactions in the package
Examples of packages that follow this rule (variations of example A show
some
possibilities after deduplication): ![image][15]
#### Fee-Related Checks Use Package Feerate
Package Feerate = the total modified fees divided by the total virtual size
of
all transactions in the package.
To meet the two feerate requirements of a mempool, i.e., the pre-configured
minimum relay feerate (`minRelayTxFee`) and dynamic mempool minimum
feerate, the
total package feerate is used instead of the individual feerate. The
individual
transactions are allowed to be below feerate requirements if the package
meets
the feerate requirements. For example, the parent(s) in the package can
have 0
fees but be paid for by the child.
*Rationale*: This can be thought of as "CPFP within a package," solving the
issue of a parent not meeting minimum fees on its own. This allows L2
applications to adjust their fees at broadcast time instead of overshooting
or
risking getting stuck/pinned.
We use the package feerate of the package *after deduplication*.
*Rationale*: It would be incorrect to use the fees of transactions that are
already in the mempool, as we do not want a transaction's fees to be
double-counted for both its individual RBF and package RBF.
Examples F and G [14] show the same package, but P1 is submitted
individually before
the package in example G. In example F, we can see that the 300vB package
pays
an additional 200sat in fees, which is not enough to pay for its own
bandwidth
(BIP125#4). In example G, we can see that P1 pays enough to replace M1, but
using P1's fees again during package submission would make it look like a
300sat
increase for a 200vB package. Even including its fees and size would not be
sufficient in this example, since the 300sat looks like enough for the 300vB
package. The calculcation after deduplication is 100sat increase for a
package
of size 200vB, which correctly fails BIP125#4. Assume all transactions have
a
size of 100vB.
#### Package RBF
If a package meets feerate requirements as a package, the parents in the
transaction are allowed to replace-by-fee mempool transactions. The child
cannot
replace mempool transactions. Multiple transactions can replace the same
transaction, but in order to be valid, none of the transactions can try to
replace an ancestor of another transaction in the same package (which would
thus
make its inputs unavailable).
*Rationale*: Even if we are using package feerate, a package will not
propagate
as intended if RBF still requires each individual transaction to meet the
feerate requirements.
We use a set of rules slightly modified from BIP125 as follows:
##### Signaling (Rule #1)
All mempool transactions to be replaced must signal replaceability.
*Rationale*: Package RBF signaling logic should be the same for package RBF
and
single transaction acceptance. This would be updated if single transaction
validation moves to full RBF.
##### New Unconfirmed Inputs (Rule #2)
A package may include new unconfirmed inputs, but the ancestor feerate of
the
child must be at least as high as the ancestor feerates of every transaction
being replaced. This is contrary to BIP125#2, which states "The replacement
transaction may only include an unconfirmed input if that input was
included in
one of the original transactions. (An unconfirmed input spends an output
from a
currently-unconfirmed transaction.)"
*Rationale*: The purpose of BIP125#2 is to ensure that the replacement
transaction has a higher ancestor score than the original transaction(s)
(see
[comment][13]). Example H [16] shows how adding a new unconfirmed input can
lower the
ancestor score of the replacement transaction. P1 is trying to replace M1,
and
spends an unconfirmed output of M2. P1 pays 800sat, M1 pays 600sat, and M2
pays
100sat. Assume all transactions have a size of 100vB. While, in isolation,
P1
looks like a better mining candidate than M1, it must be mined with M2, so
its
ancestor feerate is actually 4.5sat/vB. This is lower than M1's ancestor
feerate, which is 6sat/vB.
In package RBF, the rule analogous to BIP125#2 would be "none of the
transactions in the package can spend new unconfirmed inputs." Example J
[17] shows
why, if any of the package transactions have ancestors, package feerate is
no
longer accurate. Even though M2 and M3 are not ancestors of P1 (which is the
replacement transaction in an RBF), we're actually interested in the entire
package. A miner should mine M1 which is 5sat/vB instead of M2, M3, P1, P2,
and
P3, which is only 4sat/vB. The Package RBF rule cannot be loosened to only
allow
the child to have new unconfirmed inputs, either, because it can still
cause us
to overestimate the package's ancestor score.
However, enforcing a rule analogous to BIP125#2 would not only make Package
RBF
less useful, but would also break Package RBF for packages with parents
already
in the mempool: if a package parent has already been submitted, it would
look
like the child is spending a "new" unconfirmed input. In example K [18],
we're
looking to replace M1 with the entire package including P1, P2, and P3. We
must
consider the case where one of the parents is already in the mempool (in
this
case, P2), which means we must allow P3 to have new unconfirmed inputs.
However,
M2 lowers the ancestor score of P3 to 4.3sat/vB, so we should not replace M1
with this package.
Thus, the package RBF rule regarding new unconfirmed inputs is less strict
than
BIP125#2. However, we still achieve the same goal of requiring the
replacement
transactions to have a ancestor score at least as high as the original
ones. As
a result, the entire package is required to be a higher feerate mining
candidate
than each of the replaced transactions.
Another note: the [comment][13] above the BIP125#2 code in the original RBF
implementation suggests that the rule was intended to be temporary.
##### Absolute Fee (Rule #3)
The package must increase the absolute fee of the mempool, i.e. the total
fees
of the package must be higher than the absolute fees of the mempool
transactions
it replaces. Combined with the CPFP rule above, this differs from BIP125
Rule #3
- an individual transaction in the package may have lower fees than the
transaction(s) it is replacing. In fact, it may have 0 fees, and the child
pays for RBF.
##### Feerate (Rule #4)
The package must pay for its own bandwidth; the package feerate must be
higher
than the replaced transactions by at least minimum relay feerate
(`incrementalRelayFee`). Combined with the CPFP rule above, this differs
from
BIP125 Rule #4 - an individual transaction in the package can have a lower
feerate than the transaction(s) it is replacing. In fact, it may have 0
fees,
and the child pays for RBF.
##### Total Number of Replaced Transactions (Rule #5)
The package cannot replace more than 100 mempool transactions. This is
identical
to BIP125 Rule #5.
### Expected FAQs
1. Is it possible for only some of the package to make it into the mempool?
Yes, it is. However, since we evict transactions from the mempool by
descendant score and the package child is supposed to be sponsoring the
fees of
its parents, the most common scenario would be all-or-nothing. This is
incentive-compatible. In fact, to be conservative, package validation should
begin by trying to submit all of the transactions individually, and only
use the
package mempool acceptance logic if the parents fail due to low feerate.
2. Should we allow packages to contain already-confirmed transactions?
No, for practical reasons. In mempool validation, we actually aren't
able to
tell with 100% confidence if we are looking at a transaction that has
already
confirmed, because we look up inputs using a UTXO set. If we have historical
block data, it's possible to look for it, but this is inefficient, not
always
possible for pruning nodes, and unnecessary because we're not going to do
anything with the transaction anyway. As such, we already have the
expectation
that transaction relay is somewhat "stateful" i.e. nobody should be relaying
transactions that have already been confirmed. Similarly, we shouldn't be
relaying packages that contain already-confirmed transactions.
[1]: https://github.com/bitcoin/bitcoin/pull/22290
[2]:
https://github.com/bitcoin/bips/blob/1f0b563738199ca60d32b4ba779797fc97d040fe/bip-0141.mediawiki#transaction-size-calculations
[3]:
https://github.com/bitcoin/bitcoin/blob/94f83534e4b771944af7d9ed0f40746f392eb75e/src/policy/policy.cpp#L282
[4]: https://github.com/bitcoin/bitcoin/pull/16400
[5]: https://github.com/bitcoin/bitcoin/pull/21062
[6]: https://github.com/bitcoin/bitcoin/pull/22675
[7]: https://github.com/bitcoin/bitcoin/pull/22796
[8]: https://github.com/bitcoin/bitcoin/pull/20833
[9]: https://github.com/bitcoin/bitcoin/pull/21800
[10]: https://github.com/bitcoin/bitcoin/pull/16401
[11]: https://github.com/bitcoin/bitcoin/pull/19621
[12]: https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki
[13]:
https://github.com/bitcoin/bitcoin/pull/6871/files#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2R1101-R1104
[14]:
https://user-images.githubusercontent.com/25183001/133567078-075a971c-0619-4339-9168-b41fd2b90c28.png
[15]:
https://user-images.githubusercontent.com/25183001/132856734-fc17da75-f875-44bb-b954-cb7a1725cc0d.png
[16]:
https://user-images.githubusercontent.com/25183001/133567347-a3e2e4a8-ae9c-49f8-abb9-81e8e0aba224.png
[17]:
https://user-images.githubusercontent.com/25183001/133567370-21566d0e-36c8-4831-b1a8-706634540af3.png
[18]:
https://user-images.githubusercontent.com/25183001/133567444-bfff1142-439f-4547-800a-2ba2b0242bcb.png
[19]:
https://user-images.githubusercontent.com/25183001/133456219-0bb447cb-dcb4-4a31-b9c1-7d86205b68bc.png
[20]:
https://user-images.githubusercontent.com/25183001/132857787-7b7c6f56-af96-44c8-8d78-983719888c19.png
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210916/d71208a2/attachment-0001.html>