npub1gl…l5ays

Last Notes

2023-06-07 23:13:06 UTC

- reply

📅 Original date posted:2022-08-25
📝 Original message:> Not only is JSON limited to editing only through specific software or text editors, but (in the latter case) it is fragile enough that a single missing character can cause an entire file to fail parsing. CSV is more forgiving in this regard.
I think quite simply: A forgiving format is not appropriate for a standard.
It'd be hard to understate how much extra and pointless effort it creates for everyone, and every implementation ends up creating its own defacto standard for what it produces and accepts. Even doing something as simple as adding an extra column will not be possible in the future because it'll break comparability with previous parsers.
I've literally worked on projects where the csv parser has evolved into scan-ahead to use heuristics to understand "rules" of a csv file, and then do line-by-line heuristics to override those rules in pathological cases. Makes a bit of sense when you're trying to achieve 30 years of backwards compatibility. Doesn't make sense for much else..
If your application users really like csv, then introduce an application-specific import-from-csv and export-to-csv with your own rules.
-Ryan
------- Original Message -------
On Thursday, August 25th, 2022 at 1:59 AM, Craig Raw <craigraw at gmail.com> wrote:
> Thanks for your thoughts Ryan.
>
> Without reference to the quality feedback on this proposal, I was aware when submitting it for review that it provides an excellent opportunity for bike shedding. As developers, we have all experienced frustration with data formats. One thing that I did not perhaps make clear enough is that this format is not solely intended for developers, but general users who are probably not well represented on this list.
>
> While doing research for this proposal I spoke to several professional users of Sparrow Wallet (who are not developers). They all expressed a desire for the format to integrate with their business processes, which are driven by business tools such as Excel. Labelling provides an important function in UTXO and address management in these scenarios, and needs to be accessible and manageable outside of wallet software.
>
> If this is to be achieved, it immediately rules out JSON as a data format. Not only is JSON limited to editing only through specific software or text editors, but (in the latter case) it is fragile enough that a single missing character can cause an entire file to fail parsing. CSV is more forgiving in this regard. With respect to your comments on escaping, my expectation would be that developers will be using a mature CSV library rather than handling character escaping themselves. I would rather propose a format that is generally usable, even if occasionally a label is escaped incorrectly.
>
> Finally, I'll note that CSV files are already common and uncontroversial in Bitcoin wallet software. Bitcoin Core, Electrum, Sparrow (and no doubt many others) already export addresses and/or transactions with their labels as CSV files. This proposal simply attempts to create a standard for importing and exporting all the labels in a wallet.
>
> Craig
>
> On Wed, Aug 24, 2022 at 9:01 PM <rhavar at protonmail.com> wrote:
>
>> I'd strongly suggest not using CSV. Especially for a standard. I've worked with it as an interchange format many a times, and it's always been a clusterfuck.
>>
>> Right off the bat, you have stuff like "The fields may be quoted, but this is unnecessary as the first comma in the line will always be the delimiter" which invariably leads to some implementations doing it, some implementations not doing it, and others that are intolerant of the other way.
>>
>> And you have also made the classic mistake of not strictly defining escape rules. So everyone will pick their own (e.g. some will \, escape commas, others will not cause it's quoted and escape quotes, and others will assume no escaping is required since its the last column in a csv).
>>
>> Over time it morphs into its own mini-monster that introduces so much pain.
>>
>> On a similar note, allowing alternatives (like: txid>index vs txid:index) provides no benefit, but creates additional work for implementations (who quite likely only test formats they produce) and future incompatibilities.
>>
>> I know everyone loves to hate on it, but really (line-separated?) json is the way to go.
>>
>> { "tx": "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎", "label": "wow, such label" }
>> { "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b", "txout": 4, "label": "omg this is so easy to parse" }
>> { "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b", "txin": 0, "label": "wow this is going to be extensible as well" }
>>
>> -Ryan
>>
>> ------- Original Message -------
>> On Wednesday, August 24th, 2022 at 2:18 AM, Craig Raw via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>>
>>> Hi all,
>>>
>>> I would like to propose a BIP that specifies a format for the export and import of labels from a wallet. While transferring access to funds across wallet applications has been made simple through standards such as BIP39, wallet labels remain siloed and difficult to extract despite their value, particularly in a privacy context.
>>>
>>> The proposed format is a simple two column CSV file, with the reference to a transaction, address, input or output in the first column, and the label in the second column. CSV was chosen for its wide accessibility, especially to users without specific technical expertise. Similarly, the CSV file may be compressed using the ZIP format, and optionally encrypted using AES.
>>>
>>> The full text of the BIP can be found at https://github.com/craigraw/bips/blob/master/bip-wallet-labels.mediawiki and also copied below.
>>>
>>> Feedback is appreciated.
>>>
>>> Thanks,
>>> Craig Raw
>>>
>>> ---
>>>
>>> <pre>
>>> BIP: wallet-labels
>>> Layer: Applications
>>> Title: Wallet Labels Export Format
>>> Author: Craig Raw <craig at sparrowwallet.com>
>>> Comments-Summary: No comments yet.
>>> Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-wallet-labels
>>> Status: Draft
>>> Type: Informational
>>> Created: 2022-08-23
>>> License: BSD-2-Clause
>>> </pre>
>>>
>>> ==Abstract==
>>>
>>> This document specifies a format for the export of labels that may be attached to the transactions, addresses, input and outputs in a wallet.
>>>
>>> ==Copyright==
>>>
>>> This BIP is licensed under the BSD 2-clause license.
>>>
>>> ==Motivation==
>>>
>>> The export and import of funds across different Bitcoin wallet applications is well defined through standards such as BIP39, BIP32, BIP44 etc.
>>> These standards are well supported and allow users to move easily between different wallets.
>>> There is, however, no defined standard to transfer any labels the user may have applied to the transactions, addresses, inputs or outputs in their wallet.
>>> The UTXO model that Bitcoin uses makes these labels particularly valuable as they may indicate the source of funds, whether received externally or as a result of change from a prior transaction.
>>> In both cases, care must be taken when spending to avoid undesirable leaks of private information.
>>> Labels provide valuable guidance in this regard, and have even become mandatory when spending in several Bitcoin wallets.
>>> Allowing users to export their labels in a standardized way ensures that they do not experience lock-in to a particular wallet application.
>>> In addition, by using common formats, this BIP seeks to make manual or bulk management of labels accessible to users without specific technical expertise.
>>>
>>> ==Specification==
>>>
>>> In order to make the import and export of labels as widely accessible as possible, this BIP uses the comma separated values (CSV) format, which is widely supported by consumer, business, and scientific applications.
>>> Although the technical specification of CSV in RFC4180 is not always followed, the application of the format in this BIP is simple enough that compatibility should not present a problem.
>>> Moreover, the simplicity and forgiving nature of CSV (over for example JSON) lends itself well to bulk label editing using spreadsheet and text editing tools.
>>>
>>> A CSV export of labels from a wallet must be a UTF-8 encoded text file, containing one record per line, with records containing two fields delimited by a comma.
>>> The fields may be quoted, but this is unnecessary, as the first comma in the line will always be the delimiter.
>>> The first line in the file is a header, and should be ignored on import.
>>> Thereafter, each line represents a record that refers to a label applied in the wallet.
>>> The order in which these records appear is not defined.
>>>
>>> The first field in the record contains a reference to the transaction, address, input or output in the wallet.
>>> This is specified as one of the following:
>>> * Transaction ID (<tt>txid</tt>)
>>> * Address
>>> * Input (rendered as <tt>txid<index</tt>)
>>> * Output (rendered as <tt>txid>index</tt> or <tt>txid:index</tt>)
>>>
>>> The second field contains the label applied to the reference.
>>> Exporting applications may omit records with no labels or labels of zero length.
>>> Files exported should use the <tt>.csv</tt> file extension.
>>>
>>> In order to reduce file size while retaining wide accessibility, the CSV file may be compressed using the ZIP file format, using the <tt>.zip</tt> file extension.
>>> This <tt>.zip</tt> file may optionally be encrypted using either AES-128 or AES-256 encryption, which is supported by numerous applications including Winzip and 7-zip.
>>> In order to ensure that weak encryption does not proliferate, importers following this standard must refuse to import <tt>.zip</tt> files encrypted with the weaker Zip 2.0 standard.
>>> The textual representation of the wallet's extended public key (as defined by BIP32, with an <tt>xpub</tt> header) should be used as the password.
>>>
>>> ==Importing==
>>>
>>> When importing, a naive algorithm may simply match against any reference, but it is possible to disambiguate between transactions, addresses, inputs and outputs.
>>> For example in the following pseudocode:
>>> <pre>
>>> if reference length < 64
>>> Set address label
>>> else if reference length == 64
>>> Set transaction label
>>> else if reference contains '<'
>>> Set input label
>>> else
>>> Set output label
>>> </pre>
>>>
>>> Importing applications may truncate labels if necessary.
>>>
>>> ==Test Vectors==
>>>
>>> The following fragment represents a wallet label export:
>>> <pre>
>>> Reference,Label
>>> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎,Transaction
>>> 1A69TXnEM2ms9fMaY9UuiJ7415X7xZaUSg,Address
>>> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎<0,Input
>>> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎>0,Output
>>> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎:0,Output (alternative)
>>> </pre>
>>>
>>> ==Reference Implementation==
>>>
>>> TBD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220825/b57b0503/attachment-0001.html>

2023-06-07 23:13:03 UTC

- reply

📅 Original date posted:2022-08-24
📝 Original message:I'd strongly suggest not using CSV. Especially for a standard. I've worked with it as an interchange format many a times, and it's always been a clusterfuck.
Right off the bat, you have stuff like "The fields may be quoted, but this is unnecessary as the first comma in the line will always be the delimiter" which invariably leads to some implementations doing it, some implementations not doing it, and others that are intolerant of the other way.
And you have also made the classic mistake of not strictly defining escape rules. So everyone will pick their own (e.g. some will \, escape commas, others will not cause it's quoted and escape quotes, and others will assume no escaping is required since its the last column in a csv).
Over time it morphs into its own mini-monster that introduces so much pain.
On a similar note, allowing alternatives (like: txid>index vs txid:index) provides no benefit, but creates additional work for implementations (who quite likely only test formats they produce) and future incompatibilities.
I know everyone loves to hate on it, but really (line-separated?) json is the way to go.
{ "tx": "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎", "label": "wow, such label" }
{ "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b", "txout": 4, "label": "omg this is so easy to parse" }
{ "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b", "txin": 0, "label": "wow this is going to be extensible as well" }
-Ryan
------- Original Message -------
On Wednesday, August 24th, 2022 at 2:18 AM, Craig Raw via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
> Hi all,
>
> I would like to propose a BIP that specifies a format for the export and import of labels from a wallet. While transferring access to funds across wallet applications has been made simple through standards such as BIP39, wallet labels remain siloed and difficult to extract despite their value, particularly in a privacy context.
>
> The proposed format is a simple two column CSV file, with the reference to a transaction, address, input or output in the first column, and the label in the second column. CSV was chosen for its wide accessibility, especially to users without specific technical expertise. Similarly, the CSV file may be compressed using the ZIP format, and optionally encrypted using AES.
>
> The full text of the BIP can be found at https://github.com/craigraw/bips/blob/master/bip-wallet-labels.mediawiki and also copied below.
>
> Feedback is appreciated.
>
> Thanks,
> Craig Raw
>
> ---
>
> <pre>
> BIP: wallet-labels
> Layer: Applications
> Title: Wallet Labels Export Format
> Author: Craig Raw <craig at sparrowwallet.com>
> Comments-Summary: No comments yet.
> Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-wallet-labels
> Status: Draft
> Type: Informational
> Created: 2022-08-23
> License: BSD-2-Clause
> </pre>
>
> ==Abstract==
>
> This document specifies a format for the export of labels that may be attached to the transactions, addresses, input and outputs in a wallet.
>
> ==Copyright==
>
> This BIP is licensed under the BSD 2-clause license.
>
> ==Motivation==
>
> The export and import of funds across different Bitcoin wallet applications is well defined through standards such as BIP39, BIP32, BIP44 etc.
> These standards are well supported and allow users to move easily between different wallets.
> There is, however, no defined standard to transfer any labels the user may have applied to the transactions, addresses, inputs or outputs in their wallet.
> The UTXO model that Bitcoin uses makes these labels particularly valuable as they may indicate the source of funds, whether received externally or as a result of change from a prior transaction.
> In both cases, care must be taken when spending to avoid undesirable leaks of private information.
> Labels provide valuable guidance in this regard, and have even become mandatory when spending in several Bitcoin wallets.
> Allowing users to export their labels in a standardized way ensures that they do not experience lock-in to a particular wallet application.
> In addition, by using common formats, this BIP seeks to make manual or bulk management of labels accessible to users without specific technical expertise.
>
> ==Specification==
>
> In order to make the import and export of labels as widely accessible as possible, this BIP uses the comma separated values (CSV) format, which is widely supported by consumer, business, and scientific applications.
> Although the technical specification of CSV in RFC4180 is not always followed, the application of the format in this BIP is simple enough that compatibility should not present a problem.
> Moreover, the simplicity and forgiving nature of CSV (over for example JSON) lends itself well to bulk label editing using spreadsheet and text editing tools.
>
> A CSV export of labels from a wallet must be a UTF-8 encoded text file, containing one record per line, with records containing two fields delimited by a comma.
> The fields may be quoted, but this is unnecessary, as the first comma in the line will always be the delimiter.
> The first line in the file is a header, and should be ignored on import.
> Thereafter, each line represents a record that refers to a label applied in the wallet.
> The order in which these records appear is not defined.
>
> The first field in the record contains a reference to the transaction, address, input or output in the wallet.
> This is specified as one of the following:
> * Transaction ID (<tt>txid</tt>)
> * Address
> * Input (rendered as <tt>txid<index</tt>)
> * Output (rendered as <tt>txid>index</tt> or <tt>txid:index</tt>)
>
> The second field contains the label applied to the reference.
> Exporting applications may omit records with no labels or labels of zero length.
> Files exported should use the <tt>.csv</tt> file extension.
>
> In order to reduce file size while retaining wide accessibility, the CSV file may be compressed using the ZIP file format, using the <tt>.zip</tt> file extension.
> This <tt>.zip</tt> file may optionally be encrypted using either AES-128 or AES-256 encryption, which is supported by numerous applications including Winzip and 7-zip.
> In order to ensure that weak encryption does not proliferate, importers following this standard must refuse to import <tt>.zip</tt> files encrypted with the weaker Zip 2.0 standard.
> The textual representation of the wallet's extended public key (as defined by BIP32, with an <tt>xpub</tt> header) should be used as the password.
>
> ==Importing==
>
> When importing, a naive algorithm may simply match against any reference, but it is possible to disambiguate between transactions, addresses, inputs and outputs.
> For example in the following pseudocode:
> <pre>
> if reference length < 64
> Set address label
> else if reference length == 64
> Set transaction label
> else if reference contains '<'
> Set input label
> else
> Set output label
> </pre>
>
> Importing applications may truncate labels if necessary.
>
> ==Test Vectors==
>
> The following fragment represents a wallet label export:
> <pre>
> Reference,Label
> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎,Transaction
> 1A69TXnEM2ms9fMaY9UuiJ7415X7xZaUSg,Address
> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎<0,Input
> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎>0,Output
> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b‎:0,Output (alternative)
> </pre>
>
> ==Reference Implementation==
>
> TBD
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220824/f1fcaed1/attachment-0001.html>

2023-06-07 18:16:17 UTC

- reply

📅 Original date posted:2019-02-02
📝 Original message:More of a shower-thought than a BIP, but it's something I've long wish (hardware) wallets supported:
---
Abstract: Bitcoin Wallets generally ask us to trust their seed generation is both correct and honest. Especially for hardware and air gapped wallets, this is both a big ask and more or less impossible to practically verify. So we propose a bring-your-own-entropy approach in which the wallet can function completely deterministically. Our method is based on shuffling physical deck of cards. There are 52! (2^219.88) different shuffle order, which is a big enough space to be secure against collision and brute force attacks. Conveniently a shuffled deck of cards also can serve as a physical backup which is easy to hide in plain sight with great plausible deniability.
Representation:
Each card has a suit which can be represented by one of SCHD (spades, clubs, hearts, diamonds) and a value of one of 23456789TJQKA where the numbers are obvious and (T=ten, J=jack, Q=queen, K=king, A=ace) so "7 of clubs" would be represented by "7C" and a "Ten of Hearts" would be represented with "TH".
An deck of cards looks like:
2S,3S,4S,5S,6S,7S,8S,9S,TS,JS,QS,KS,AS,2C,3C,4C,5C,6C,7C,8C,9C,TC,JC,QC,KC,AC,2H,3H,4H,5H,6H,7H,8H,9H,TH,JH,QH,KH,AH,2D,3D,4D,5D,6D,7D,8D,9D,TD,JD,QD,KD,AD
And can be verified by making sure that every one of the 52 cards appears exactly once.
Step 1. Shuffle your deck of cards
This is a lot harder than you'd imagine, so do it quite a few times, with quite a few different techniques. It is advised to do at *least* 7 good quality shuffles to achieve a true cryptographically secure shuffle. Do not look at the cards while shuffling (to avoid biasing) and don't be afraid to also shuffle them face down on the table. Err on the side over over-shuffling.
See also: https://en.wikipedia.org/wiki/Shuffling#Sufficient_number_of_shuffles
Step 2. Write out the order (comma separated)
And example shuffle is:
5C,7C,4C,AS,3C,KC,AD,QS,7S,2S,5H,4D,AC,9C,3H,6H,9D,4S,8D,TD,2H,7H,JD,QD,2D,JC,KH,9S,9H,4H,6C,7D,3D,6S,2C,AH,QC,TH,TC,JS,6D,8H,8C,JH,8S,KD,QH,5D,5S,KS,TS,3S
Step 3. Sha512 it to create a seed
In the example above you should get:
dc04e4c331b1bd347581d4361841335fe0b090d39dfe5e1c258c547255cd5cf1545e2387d8a7c4dc53e03cacca049a414a9269a2ac6954429955476c56038498
Step 4. Interpret it
e.g. For bip32 you would treat the first 32 bytes as the private key, and the second 32 bytes as as the extension code.
-Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190202/5489fb15/attachment.html>

2023-06-07 18:16:04 UTC

- reply

📅 Original date posted:2019-01-27
📝 Original message:> Is there a missing word. "by giving a person.."? Not actually sure what
> you're getting at here but I suspect it's again tangential to this BIP
> discussion.
Correct on both points. I meant to say "giving a (txid:vout:privkey)" to a person as means of payment.
> So I think the limiting factor is in fact creating a standard that a reasonable number
> of people could agree with (and I like operational definitions, so
> subjective as it is, I like the goal of "good/clear enough that it could
> be incorporated into something like BtcPayServer")
The problem with BtcPayServer (and a lot of similar software), is that it's not very unsuitable for any sender/receiver coinjoin due to it not having its own wallet. As I understand the basic architecture is just a fancy wrapper around bip32 address generation and monitoring the payment to those addresses. This means that adding support is not only a large code change, but it also entails a substantial change for merchants (you can't just have your payments flow into your trezor, but need to run a hot wallet)
But I strongly believe that bustapay is good enough _right now_ for BtcPayServer integration (which I'd happily contribute myself, if it wasn't for my unfamiliarity of the project and c#)
> But this relates back to my first "generic" point that you haven't
> addressed here - protocol versioning and the possibility of more than
> one option. Perhaps more realistic (debatable): have the current version
> be non-PSBT but with a plan to have a version bump with PSBT in future.
> Stuff like that. It seems crazy to actually long term reject it.
Adding backwards-compatible versioning at a later stage would be pretty trivial through either the URL or HTTP header (e.g version=2) and if breaking backwards compatible is desirable it can also easily been done (e.g. bump the bip21 or send an incompatible request/response). I don't see this as a problem at all, and I'm not rejecting it long-term, I just don't think it's particularly helpful to bikeshed now, when adoption is pretty much zero.
> I don't want to be that guy, but this was a central part of the proposal
> that came of the meetup last summer and is in Haywood's blogpost. I mean
> if you came up it with separately, then cool :) But I was there, that
> was established immediately as the right way of doing this to avoid a
> trivial attack.
Oh wow. had no idea. I saw the part about the receiver spamming the sender with a bunch of transactions, where only 1 of them are real and thought "ewww" and "came up" with the idea of a "template transaction" instead. I was always wondering why no one came up it, but now it makes sense. The transaction-spam stage was just an _additional_ layer of protection.
Ok, now I feel like an idiot =) Thanks for letting me know.
> The counterargument is Laurent's statistics which I previously linked,
> suggesting that maybe 30% of txs violate this anyway, today. I'm not
> sure about that, will need more analysis; Core's SRD algo may be one
> reason, but anyway ... better to make things look like payments.
I think it's interesting, but I don't think it particularly matters. Avoiding UIH1 I think is pretty much irrelevant, as it'll likely just confuse any analysis into thinking the payment is the reverse of what it actually is. And wallets already don't care about violating UIH2(as a way to do implicit consolidation). If 30% of tx's are violating it, you can be pretty sure it means the _vast majority_ of wallets run coin selection in such a way that can violation UIH2. Most wallets use a coin selection algorithm that you can approximate with:
while !enoughMoney {
inputs += getAnotherInput();
}
and don't run a final pass that would prune superfluous inputs. Even coinsayer (shill alert) which I believe runs the most advanced coin selection algorithms, will routinely and intentionally violate UIH2 when it's ideal (e.g. most classic case: when `consolidationFeeRate >= minFeeRate`).
I'm not trying to dismiss your analysis, as I find it interesting. I'm just against increasing the cognitive burden on implementations by mentioning all this stuff, when the truth is it (barely) matters. If wallets routinely avoided UIH2 and making a UIH2 payment would "out" the transaction as much more likely to be a bustapay, then I'd definitely reconsider and provide a basic suggestion into how to try avoid it.
And like I said, I also think there's much more important things that go into "picking a contributed input" than just this.
> A last point, you also don't see value in being more explicit about
> simple things like transaction version and locktime? Even if you think
> these things should not be controlled, e.g. the protocol should allow
> either transaction version, then it'd be better to explicitly say so.
My intention was that wallets create a transaction exactly like they normally would do, and use that as the template transaction. The only time I wanted to be prescriptive was when it would increase the implementation complexity (e.g. being non-segwit compatible is a pain in the ass. So I'd rather just be "pure segwit only" transactions). But something like locktime makes no difference as long as the transaction is mempool eligible, so I'd rather just wallets do what they do anyway.
Although I think there should be a separate discussion on improving the uniformity of bitcoin transactions in general. The current state of affairs is really atrocious.
---
P.S. I know I come across as obstinate, but it's not really so. If you can come up with an alternative to bustapay with some traction, I'd love to get behind it and deprecate bustapay in favor of it. I just am pretty happy with the state of bustapay and it's status a sort of "MVP of pay2endpoint", and unless the argument is in the form: "We'd love to support it, but in order to do so we'd need X" I'm probably going to disagree.

2023-06-07 18:16:02 UTC

- reply

📅 Original date posted:2019-01-27
📝 Original message:Thanks Adam,
I have fixed the mistakes you have pointed out: https://github.com/bitcoin/bips/pull/754
Thanks for the detailed look!
> but its virtue of steganographic hiding means only minimal uptake
> is still enormously interesting and worth pursuing; that's my current feeling.
I very much agree =) I really think anything that (silently) breaks the assumption of common ownership of transaction inputs offers outsized benefits for the whole ecosystem.
One other idea I have is (way) better support for moving utxo's between wallets. The least controversial use case is moving funds between wallets you own. Like I might want to move *specific* utxo's from/to my joinmarket wallet, but not create a (privacy losing / expensive) transaction. Both core and joinmarket fail at this at a practical point of view.
Like imho it'd be pretty cool having a standardized format for (txid:vout:privatekey) with wallets showing it as "External UTXO" and preferentially spending it (and wallet not automatically importing any other utxo from that address).
Taken a bit further (this is the part which everyone hates) you could send someone money (or withdraw it from a service) by giving a person. It's not generally useful (for obvious reasons), but there's a lot of cases I think it's super cool.
---
Getting back on topic, without trying to do a point-by-point reply, I agree with pretty much everything you said but I am reluctant to make any changes.
I don't meant to be obtuse or anything, but I strongly believe the limiting factor to adoption to all these protocols is actually just getting people to implement it. I made multiple implementations of bustapay from both the sending/receiving end, so I could try develop the easiest to implement system that is still practical.
For instance I like PSBT and it's nice in theory. I actually had an original implementation using it, which is how I found some bugs in the core and golang version of PSBT). But in practice it's hugely overkill and significantly increases the implementation complexity complexity and is poorly supported. Switching to just a raw transaction actually made everything easier. (And that's not to criticise PSBT, I would definitely want to use it in other contexts).
Anyway, a big motivation for me even writing it as a BIP was to formalize my little anti-DOS trick of privately creating a "template transaction" which can just be dumped on the network as punishment. So if nothing else, hopefully I'll have demonstrated it's a pretty practical way of doing things.
--
Also your analysis on "Unnecessary Input Heuristic" is pretty cool, but I also don't like telling people to "avoid the UIH2" without providing the actual algo they should use. But really I think it's better off in a sort of article "how to pick contributed inputs" or something, as while it's nice it's not a huge deal and there's a lot of debatable tradeoffs that can/should be used. For instance the implementation I wrote for bustabit.com currently just heavily biases tainted inputs (e.g. ones associated with address reuse).
-Ryan
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Friday, January 25, 2019 6:47 AM, Adam Gibson via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
> Ryan and list,
> I want to add some commentary to this (BIP79) to see if we can get
> further in standardizing this idea.
>
> When I first mulled it over I thought it too impractical, but its virtue
> of steganographic hiding means only minimal uptake is still enormously
> interesting and worth pursuing; that's my current feeling. I've offered
> more detailed thoughts in my blog post[1] (def not required reading here).
>
> Both Joinmarket and Samourai have started implementing this kind of
> transaction. And while that's interesting experimentally, some kind of
> cross-wallet standard would be helpful, albeit there some differences
> between that and the merchant/centralized service use-case.
>
> We might imagine as a concrete goal for this BIP to create something
> that would be acceptable for inclusion into a project like BTCPayServer,
> so that it could be used in a realistic use case by smaller bitcoin
> accepting merchants.
>
> Comments to the BIP[2] as follows, with generic comments first, and then
> specific comments for existing points in the BIP:
>
> [1] https://joinmarket.me/blog/blog/payjoin
> [2] https://github.com/bitcoin/bips/blob/master/bip-0079.mediawiki
>
> Generic comments
>
> ==================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
>
> - Protocol versioning. Since inevitably (even if only merchants), this
> must be implemented by multiple wallets to be useful, the communication
> protocol will need versioning (for example i have in my
> simple/experimental Joinmarket PayJoin that sender sends min and max
> supported version and receiver responds with a chosen protocol version
> so we can update). I do understand that as a client-server model can
> apply here, we can ditch a lot of the complexities around network/p2p
> interaction, but this much at least seems necessary.
>
> - Although it has its logic, I don't think "Bustapay" is a good name for
> this protocol. I prefer "PayJoin" which is neutral sounding and
> self-descriptive. Needless to say this is not a hill I intend to die on.
>
> - PSBT/BIP174. I realise this has already been discussed, but this is a
> good example of what this standardisation was designed for, so I'd be
> against not including it, even given the reality that, as you correctly
> observe, it is not yet implemented in the majority of wallets and
> libraries. One way round that is to make it optional (possibly combined
> with above point about versioning). Note that for example you were
> observing the necessity to check the sequence number was unchanged; that
> would be encapsulated by checking equality of PSBT Input objects/fields.
> While one can make such software architecture arguments, the really
> fundamental point is the need for standards for x-wallet compatibility.
>
> - Version, Locktime: Perhaps this is not needed; in a peer to peer
> wallet scenario I think there might be logic in trying to get cover
> traffic of (Core, Electrum, others), say, by using
> last-block-locktime-mostly, as they do. Version should be 2 and sequence
> is a function of your suggestion to use BIP125. Worth noting that BIP125
> is not currently widely used on the network, though (see
> https://p2sh.info/dashboard/db/replace-by-fee?orgId=1). For this reason
> it should perhaps be explicitly only optional.
>
> - Avoidance of non-payment "Unnecessary Input Heuristic" (1, 2). For
> reference, see the definition here
> https://gist.github.com/AdamISZ/4551b947789d3216bacfcb7af25e029e#gistcomment-2796539
> and some data here
> https://gist.github.com/AdamISZ/4551b947789d3216bacfcb7af25e029e#gistcomment-2800791
> (whole comment thread may be of interest) - note this UIH name is afaik
> Chris Belcher's invention, it seems useful as a categorisation.
> So, it seems that UIH2 is more important to avoid; while some more
> sophisticated wallet coin selection algorithms may occasionally pick
> an input set where one input is larger than any output, most won't, and
> some in particular never will. So I think the text here should indicate
> that *the receiver's contributed input(s) SHOULD be chosen to avoid
> triggering the UIH2 heuristic where possible, so that the final payjoin
> transaction is maximally plausible as an ordinary payment" or similar.
> UIH1 is a nice-to-have (meaning the plausibility extends to two
> different (both wrong) payment amounts, but it may not be necessary to
> mention it in the BIP.
>
> Specific comments
> =================
>
>
> > > ====Step 4. Receiver validates, re-signs, and propagates on the
>
> bitcoin network====
>
> I believe this should say "Sender" not Receiver. Also for the next
> sentence, s/receiver/sender/:
>
> > > The receiver MUST validate the ''partial transaction'' was changed
>
> correctly and non-maliciously (to allow using potentially untrusted
> communication channels), re-sign its original inputs and propagate the
> final transaction over the bitcoin network.
>
> Your very correct highlighting of the attack vector of "receiver gives
> sender other inputs belonging to sender to unwittingly sign (described
> below), should be highlighted here, perhaps with the phrase "re-sign its
> ORIGINAL inputs" (only!)".
>
> > > When the sender is creating a "template transaction" it is done
>
> almost identically to creating a normal send, with the exception that
> only segwit inputs may be used. The sender is also encouraged to use a
> slightly more aggressive feerate than usual as well as BIP125 (Opt-in
> Full Replace-by-Fee Signaling), but neither is strictly required.
>
> "slightly more aggressive feerate than usual" - this I understand is to
> make up for receiver contributed utxo, OK.
>
> "only segwit inputs" - it certainly makes things simpler. One can work
> with non-segwit inputs but especially considering (as mentioned below)
> we really ought to "MUST" the part about matching input types, I tend to
> agree that non-segwit should be disallowed.
>
> > > The receiver must add at least one input to the transaction (the
>
> "contributed inputs"). If the receiver has no inputs, it should use a
> 500 internal server error, so the client can send the transaction as per
> normal (or try again later).
>
> Would it not be much simpler for the server to return a different
> (non-error) response indicating that it will broadcast the template tx
> in this case?
>
> > > Its generally advised to only add a single contributed input, however
>
> they are circumstances where adding more than a single input can be useful.
>
> I don't see a good reason to advise the use of only 1 input? (but this
> will also connect with the above generic comment about "UIH"). I guess
> it's because of your approach to fees. I'd prefer not to create a
> limitation here.
>
> > > To prevent an attack where a receiver is continually sent variations
>
> of the same transaction to enumerate the receivers utxo set, it is
> essential that the receiver always returns the same contributed inputs
> when it's seen the same inputs.
>
> This is an approach to avoiding this problem which has the virtue of
> simplicity, but it seems a little problematic. (1) You must keep a
> mapping of proposed payment utxos to one's proposed contributed input
> utxos, but (2) how should this be updated if you need to spend the
> contribution mentioned in (1)? Ironically use of payjoin exacerbates
> this issue, because it results in a smaller number of utxos being held
> by the receiver at any one time :) All this considered, I still see the
> value in your approach, but it might end up with a re-attempted payment
> being rejected. Certainly the more complex suggested solutions coming
> out of the summer 2018 coinjoin workshop aren't as practical as this,
> and may be overkill for small merchants/receivers.
>
> > > It is strongly preferable that the receiver makes an effort to pick a
>
> contributed input of the same type as the other transaction inputs if
> possible.
>
> I have also thought about this and you could reasonably argue this
> should be a MUST section in the BIP, that is, if the receiver cannot use
> inputs of the same type, he should fall back to the template
> transaction. A mixed-input payjoin/coinjoin is essentially
> near-perfectly identifiable as such (there is almost zero other usage of
> multi-type-input transactions), which is a very different thing than a
> non-identifiable payjoin transaction. That may or may not be OK to the
> sender. This is debatable though, for sure.
>
> > > After adding inputs to the transaction, the receiver generally will
>
> want to adjust the output that pays himself by increasing it by the sum
> of the contributed input amounts (minus any fees he wants to
> contribute). However the only strict requirement is that the receiver
> must never add or remove inputs, and must not ever decrease any
> output amount.
>
> "must never add or remove inputs" - did you mean "must never remove
> inputs"? he surely has to add one! Or, perhaps you mean he must not
> alter the list of inputs provided by the sender (in which case it should
> be clarified).
>
> "must not decrease any output amount" - I initally disagreed with this
> but it is a better option than the one I currently chose in Joinmarket
> payjoin (sender pays all fee as long as receiver utxos are not too
> much). So this means that the receiver either consciously chooses to not
> increase the fee, meaning the fee rate may be a bit low (hence your
> earlier comment about being generous, got it), or contributes via the
> payout amount. I guess the latter might break merchant software
> expecting to have amount output fixed and fees determined by change.
>
> Regards,
> Adam Gibson/waxwing
>
> On 30. 08. 18 22:24, Ryan Havar via bitcoin-dev wrote:
>
> > I've just finished writing an implementing of this, and extremely happy
> > with how it turned out. So I'd like to go and try go down the path of
> > more formally describing it and getting some comments and ultimately
> > encourage its wide-spread use.
> > ==Abstract==
> > The way bitcoin transactions are overwhelming used is known to leak more
> > information than desirable. This has lead to fungibility concerns in bitcoin
> > and a raise of unreasonably effective blockchain analysis.
> > Bustapay proposes a simple, practical way to bust these assumptions to
> > immediate
> > benefit of the sender and recievers. Furthermore it does so in such a
> > way that
> > helps recievers avoid utxo bloat, a constant problem for bitcoin merchants.
> > ==Copyright==
> > This BIP is in the public domain.
> > ==Motivation==
> > One of the most powerful heuristic's employed by those whose goal is to
> > undermine
> > bitcoin's fungiblity has been to assume all inputs of a transaction are
> > signed by
> > a single party. In the few cases this assumption does not hold, it is
> > generally
> > readibly recognizable (e.g. traditional coinjoins have a very obvious
> > structure,
> > or multisig outputs are most frequently validated onchain).
> > Bustapay requires no changes to bitcoin and creates bitcoin transactions
> > that are
> > indistinguishable from normal ones.
> > It is worth noting that this specification has been intentionally kept
> > as simple
> > as possible to encourage adoption. There are almost an endless amount of
> > extensions
> > possible but the harder the implementation of clients/server the less
> > likely it
> > will ever be done. Should bustapay enjoy widespread adoption, a "v2"
> > specification
> > will be created with desired extensions.
> > ==Specification==
> > A bustapay payment is made from a sender to a receiver.
> > Step 1. Sender creates a bitcoin transaction paying the receiver
> > This transaction must be fully valid, signed and all inputs must use
> > segwit. This transaction is known as the "template transaction". This
> > transaction must not be propagated on the bitcoin network.
> > Step 2. Sender gives the "template transaction" to the receiver
> > This would generally be done as an HTTP POST. The exact URL to submit it
> > to could be specified with a bip21 encoded address. Such as
> > bitcoin:2NABbUr9yeRCp1oUCtVmgJF8HGRCo3ifpTT?bustapay=https://bp.bustabit.com/submit
> > and the HTTP body should be the raw transaction hex encoded as text.
> > Step 3. Receiver processes the transaction and returns a partially
> > signed coinjoin
> > The receiver validates the transaction is valid, pays himself and is
> > eligible for propation. The receiver then adds one of his own inputs
> > (known as the "contributed input") and increase the output that pays
> > himself by the contributed input amount. Doing so will invalidate the
> > "template transaction"'s original input signatures, so the sender needs
> > to return this "partial transaction" back to the receiver to sign. This
> > is returned as a hex-encoded raw transaction a response to the original
> > HTTP POST request.
> > Step 4. Receiver validates, re-signs, and propagates on the bitcoin network
> > The receiver is responsible in making sure the "partial transaction"
> > returned by the sender was changed correctly (it should assume the
> > connection has been MITM'd and act accordingly), resign its original
> > inputs and propagates this transaction over the bitcoin network. The
> > client must be aware that the server can reorder inputs and outputs.
> > Step 5. Receiver observes the finalized transaction on the bitcoin network
> > Once the receiver has seen the finalized transactions on the network
> > (and has enough confirmations) it can process it like a normal payment
> > for the sent amount (as opposed to the amount that it looks like on the
> > network). If the receiver does not see the finalized transaction after a
> > timeout will propagate the original "template transaction" to ensure the
> > payment happens and function a strong anti-DoS mechanism.
> > === Implementation Notes ===
> > For anyone wanting to implement bustapay payments, here are some notes
> > for receivers:
> >
> > - A transaction can easily be checked if it's suitable for the mempool
> > with testmempoolaccept in bitcoin core 0.17
> >
> > - Tracking transactions by txid is precarious. To keep your sanity make
> > sure all inputs are segwit. But remember segwit does not prevent txid
> > malleability unless you validate the transaction. So really make sure
> > you're using testmempoolaccept at the very least
> >
> > - Bustapay could be abused by a malicious party to query if you own a
> > deposit address or not. So never accept a bustapay transaction that pays
> > an already used deposit address
> >
> > - You will need to keep a mapping of which utxos people have showed you
> > and which you revealed. So if you see them again, you can reveal the
> > same one of your own
> >
> > - Check if the transaction was already sorted according to BIP69, if so
> > ensure the result stays that way. Otherwise probably just shuffle the
> > inputs/outpus
> >
> >
> > Notes for sending applications:
> >
> > - The HTTP response must not be trusted. It should be fully validated
> > that no unexpected changes have been made to the transaction.
> >
> > - The sender should be aware the original "template transaction" may be
> > propagated at any time, and in fact can intentionally be
> > done so for the purpose of RBF as it should have a slightly higher fee
> > rate.
> >
> >
> > == Credits ==
> > The idea is obviously based upon Dr. Maxwell's seminal CoinJoin
> > proposal, and reduced scope inspired by a simplification of the "pay 2
> > endpoint" (now offline) blog post by blockstream.
> > -Ryan
> >
> > bitcoin-dev mailing list
> > bitcoin-dev at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

2023-06-07 18:10:54 UTC

- reply

📅 Original date posted:2018-02-18
📝 Original message:No, you are misunderstanding. The block size limit (1MB) has been replaced in favor of a block weight limit (4M weight). Bytes which must be sent to old clients are weighted at 4 units each which is what allows it to be a soft fork.
So as such, there's not two separate limits or anything.
P.S. what's up with your signature lol
-Ryan

-------- Original Message --------
On February 18, 2018 11:26 AM, CANNON via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> I have a question in reference to the increased blockspace enabled
> by the segregated witness upgrade. Is this extra blockspace beyond
> the legacy 1 MB limit limited to just witness data?
>
>
>
>Cannon
> PGP Fingerprint: 2BB5 15CD 66E7 4E28 45DC 6494 A5A2 2879 3F06 E832
> Email: cannon at cannon-ciota.info
>
> NOTICE: ALL EMAIL CORRESPONDENCE NOT SIGNED/ENCRYPTED WITH PGP SHOULD
> BE CONSIDERED POTENTIALLY FORGED, AND NOT PRIVATE.
> -----BEGIN PGP SIGNATURE-----
>
> iQIcBAEBCgAGBQJaiajLAAoJEAYDai9lH2mwDbMQAKgJseZG9oOoP9WJlESFdAzm
> jLM69KLF4RVWDQjMWCtyVayyolXMLavW6fL4GL7/ztLYSl9Pz+FDlH3Qgo1GLDx4
> fRUkmb0ApLBSAjmdqM+kWjgq3s6/oZaIkNxGeyK5SYY4QhF80Z42PWLBPKjAc7vn
> pe7Im13NwtyJiRuCmq4e1z6GmW86fFOKOsR4rlWcftHOFmH8Gz5d9KK9LbPqU6Ca
> +rde8yb+A9uXEv3O9MqeLId7FYSJixgzFGJosidVkAVQI7YZ0TAfG4HSwKjtqDz7
> SW2Bs3nDeiCr2QN7Su9TUaoSN/yKTBOw5jnql9cNOYuf4GAG0MXWinmYP8MWCiqh
> bIrzvOnlZGapX/36Fpab67VDcFnXJJc8ofqYmn+oUoW/q7geQpu/V1oz+AR9nD/d
> n8wFxvZdRlTbq2XDXySaontxNH0rd80fSG5SJtO5Js9hK/vNG+Xa7Zc+76gEtvlF
> 5G1F6MOcsoXUDCnMteuNxaZx6TPML6RuWVmbR1wXOaX4qZ01p7AsjQlTIcrlVDsB
> LVGW70wjfETBCffn1JQvFmIK7NzggIUuYfLF0IrfM4BrTa/01RnmfBOpWGiIvyIG
> qZqKLYDbW7BDkc/HMtAonR/0t6bTUv/388USSnbMakO9bvih0xxIw4NWTyoSoef/
> I+W6ny7qXt+pegLF4cYL
> =YqEM
> -----END PGP SIGNATURE-----
>
>bitcoin-dev mailing list
>bitcoin-dev at lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>