Nostr notes by

📅 Original date posted:2015-09-19 📝 Original message:We ...

2023-06-07T17:40:59Z

In reply to nevent1q…478j
_________________________

📅 Original date posted:2015-09-19
📝 Original message:We need to distinguish between two different things here:

1) A 51% attack, where the majority of mining power is *malicious* (hence “attack”)

and

2) A fork that exists because of a disagreement in the network, with total mining power split in two camps, each camp mining peacefully on their own chain

These are two very different scenarios.

Some claim that including the UTXO set hash in the block creates a vulnerability where miners can include the wrong UTXO hash, and mine on that, but this is only possible if a 51% attack is in effect. And if a 51% attack is in effect, it’s a moot point, because Bitcoin is useless anyway, because that 51% malicious mining power could just as well be used for mining empty blocks on top of the official chain. Or blocks containing randomly generated transactions, without confirming any legitimate transactions. This is why we say that a majority of honest miners is a hard requirement for Bitcoin. A majority of dishonest miners can only be circumvented through centralisation, and then we don’t have Bitcoin any longer.

Scenario 2 is unproblematic regardless of whether we include the UTXO hash in the block (and make it consensus-critical) or not, since the majority mining power on either chain isn’t malicious.

It is correct that if you’re a full node, and a 51% attack is in effect, you are able to verify that miners are honest (ie. you know whether a 51% attack is in effect or not). But this doesn’t change the fact that the Bitcoin network is unreliable, at best, when a majority of mining power is used for malicious purposes.

/Rune

> On 19 Sep 2015, at 04:30, Justus Ranvier via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> On 18/09/15 15:17, Rune Kjær Svendsen via bitcoin-dev wrote:
>> Bitcoin does not function if the majority of mining power is dishonest. There is no way around that. It’s how proof-of-work functions.
>
>
> None of those statements are true.
>
> If a majority of Bitcoin miners are mining invalid blocks, then they
> aren't Bitcoin miners any more and are no longer relevant to the Bitcoin
> consensus.
>
> There does exist a problem that light clients aren't always able to tell
> the difference between chains that are valid and chains that are not
> valid, but it's is possible to create simple proofs that would do so:
>
> https://gist.github.com/justusranvier/451616fa4697b5f25f60
>
>
> If those changes would be implemented, then any node that knew a chain
> was invalid could produce a compact proof that anyone else in the
> network could verify, regardless of how much proof of work was used to
> create the invalid chain.
>
> Committed UTXO sets would need safe to rely upon if a similar set of
> proofs that a particular set was invalid existed.
>
> --
> Justus Ranvier
> Open Bitcoin Privacy Project
> http://www.openbitcoinprivacyproject.org/
> justus at openbitcoinprivacyproject.org
> E7AD 8215 8497 3673 6D9E 61C4 2A5F DA70 EAD9 E623
> <0xEAD9E623.asc>_______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

📅 Original date posted:2015-09-18 📝 Original message:There ...

2023-06-07T17:40:56Z

In reply to nevent1q…lvxk
_________________________

📅 Original date posted:2015-09-18
📝 Original message:There are a couple of points I’d like to address.

Firstly, yes, >50% attacks are a problem for Bitcoin. Bitcoin does not function if the majority of mining power is dishonest. There is no way around that. It’s how proof-of-work functions. And if we lose proof-of-work, we lose Bitcoin.

Secondly, I’m not suggesting that UTXO set hashes *replace* block hashes, or even that it should be in the block header (probably in the coinbase somewhere). I suggest it as an *addition* to the existing consensus rules. Full nodes can still verify the chain with the added step of hashing the UTXO set for every block. Of course, this can easily be deferred to after proof-of-work has been verified already, such that no work is wasted. Unless a 51% attack is in effect. But I argue that this is a moot point, since Bitcoin is useless anyway under such circumstances.

Lastly, I’m not suggesting miners discard the blockchain history. A miner has an incentive to be absolutely sure that the chain he’s building on is the right one. If he’s wrong, he loses money/income. There’s simply no reason for a professional miner *not* to do the full initial sync, which only needs to be done once. Non-miners, who just want to check the balance of their wallet, however, really don’t need to retrieve information about Hal Finney sending bitcoins to Satoshi in 2010. In any case, this practice isn’t sustainable.

In the end, it isn’t possible to control whether a miner verifies the entire blockchain anyway (anyone can send the UTXO set over the wire). Not letting the proof-of-work cover the UTXO hash doesn’t solve this problem, it only makes it impossible to know whether a given UTXO set is the one that the majority is mining on without retrieving the entire blockchain, and doing the verification yourself. People can choose to skip that regardless of what we do.

Furthermore, all nodes have the option of deciding which level of security they want. We’re not lessening security of the protocol, we’re strengthening the security of something that’s already possible to do (build on top of an unverified blockchain), but we’d rather want that people not do.

/Rune

> On 18 Sep 2015, at 21:43, Patrick Strateman via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> wrote:
>
> Full nodes using UTXO set commitments is a change to the bitcoin
> security model.
>
> Currently an attacker with >50% of the network hashrate can rewrite history.
>
> If full nodes rely on UTXO set commitments such an attacker could create
> an infinite number of bitcoins (as in many times more than the current
> 21 million bitcoin limit).
>
> Before we consider mechanisms for UTXO set commitments, we should
> seriously discuss whether the security model reduction is reasonable.
>
> On 09/18/2015 12:05 PM, Rune Kjær Svendsen via bitcoin-dev wrote:
>> Currently, when a new node wants to join the network, it needs to retrieve the entire blockchain history, starting from January 2009 and up until now, in order to derive a UTXO set that it can verify new blocks/transactions against. With a blockchain size of 40GB and a UTXO size of around 1GB, the extra bandwidth required is significant, and will keep increasing indefinitely. If a newly mined block were to include the UTXO set hash of the chain up until the previous block — the hash of the UTXO set on top of which this block builds — then new nodes, who want to know whether a transaction is valid, would be able to acquire the UTXO set in a trustless manner, by only verifying proof-of-work headers, and knowing that a block with an invalid UTXO set hash would be rejected.
>>
>> I’m not talking about calculating a complicated tree structure from the UTXO set, which would put further burden on already burdened Bitcoin Core nodes. We simply include the hash of the current UTXO set in a newly created block, such that the transactions in the new block build *on top* of the UTXO set whose hash is specified. This actually alleviates Bitcoin Core nodes, as it will now become possible for nodes without the entire blockchain to answer SPV queries (by retrieving the UTXO set trustlessly and using this to answer queries). It also saves bandwidth for Bitcore Core nodes, who only need to send roughly 1GB of data, in order to synchronise a node, rather than 40GB+. I will continue to run a full Bitcoin Core node, saving the entire blockchain history, but it shouldn’t be a requirement to hold the entire transaction history in order to start verifying new transactions.
>>
>> As far as I can see, this also forces miners to actually maintain an UTXO set, rather than just build on top of the chain with the most proof-of-work. Producing a UTXO set and verifying a block against a chain is the same thing, so by including the hash of the UTXO set we force miners to verify the block that they want to build on top of.
>>
>> Am I missing something obvious, because as far as I can see, this solves the problem of quadratic time complexity for initial sync: http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s
>>
>> The only added step to verifying a block is to hash the UTXO set. So it does require additional computation, but most modern CPUs have a SHA256 throughput of around 500 MB/s, which means it takes only two seconds to hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). A small sacrifice for the added ease of initial syncing, in my opinion.
>>
>> /Rune
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

📅 Original date posted:2015-09-18 📝 Original ...

2023-06-07T17:40:55Z

In reply to nevent1q…cp5q
_________________________

📅 Original date posted:2015-09-18
📝 Original message:Currently, when a new node wants to join the network, it needs to retrieve the entire blockchain history, starting from January 2009 and up until now, in order to derive a UTXO set that it can verify new blocks/transactions against. With a blockchain size of 40GB and a UTXO size of around 1GB, the extra bandwidth required is significant, and will keep increasing indefinitely. If a newly mined block were to include the UTXO set hash of the chain up until the previous block — the hash of the UTXO set on top of which this block builds — then new nodes, who want to know whether a transaction is valid, would be able to acquire the UTXO set in a trustless manner, by only verifying proof-of-work headers, and knowing that a block with an invalid UTXO set hash would be rejected.

I’m not talking about calculating a complicated tree structure from the UTXO set, which would put further burden on already burdened Bitcoin Core nodes. We simply include the hash of the current UTXO set in a newly created block, such that the transactions in the new block build *on top* of the UTXO set whose hash is specified. This actually alleviates Bitcoin Core nodes, as it will now become possible for nodes without the entire blockchain to answer SPV queries (by retrieving the UTXO set trustlessly and using this to answer queries). It also saves bandwidth for Bitcore Core nodes, who only need to send roughly 1GB of data, in order to synchronise a node, rather than 40GB+. I will continue to run a full Bitcoin Core node, saving the entire blockchain history, but it shouldn’t be a requirement to hold the entire transaction history in order to start verifying new transactions.

As far as I can see, this also forces miners to actually maintain an UTXO set, rather than just build on top of the chain with the most proof-of-work. Producing a UTXO set and verifying a block against a chain is the same thing, so by including the hash of the UTXO set we force miners to verify the block that they want to build on top of.

Am I missing something obvious, because as far as I can see, this solves the problem of quadratic time complexity for initial sync: http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s

The only added step to verifying a block is to hash the UTXO set. So it does require additional computation, but most modern CPUs have a SHA256 throughput of around 500 MB/s, which means it takes only two seconds to hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). A small sacrifice for the added ease of initial syncing, in my opinion.

/Rune

📅 Original date posted:2014-02-12 📝 Original ...

2023-06-07T15:13:21Z

In reply to nevent1q…lzv8
_________________________

📅 Original date posted:2014-02-12
📝 Original message:Instead of trying to remove the possibility of transaction
malleability, would it make sense to define a new, "canonical
transaction hash/ID" (cTxID), which would be a hash of the part of the
transaction data which we know is not malleable, and have clients use
this cTxID internally, thus making the traditional transaction hash
irrelevant for a client to function correctly?

We already have a non-malleable transaction hash: the hash that is
signed, ie. the transaction with each scriptSig replaced by the
scriptPubKey it redeems. This could be the cTxID.

Or is this simply a too fundamental change to the way bitcoin-qt (and
all other clients) work in order to be feasible?

As far as I can see, it completely solves the issue of not having a
canonical ID for a transaction, but it also increases the
computational requirements for a node. For one, as far as I can see,
it requires the node to index all transactions, because in order to
calculate a cTxID, it would be necessary to fetch all transactions
referred to by the transaction in question, in order to pull in the
scriptPubKeys that are redeemed.

On Mon, Feb 10, 2014 at 4:00 AM, Peter Todd <pete at petertodd.org> wrote:
> On Mon, Feb 10, 2014 at 12:33:02AM +0100, Pieter Wuille wrote:
>> Hello all,
>>
>> it was something I planned to do since a long time, but with the
>> recent related issues popping up, I finally got around to writing a
>> BIP about how we can get rid of transaction malleability over time.
>>
>> The proposed document is here: https://gist.github.com/sipa/8907691
>>
>> I expect most rules to not be controversial. Maybe rules 1 and 3, as
>> they require modifications to wallet software (Bitcoin Core 0.9 and
>> BitcoinJ already implement it, though) and potentially invalidate some
>> script functionality. However, these new rules remain optional and
>> controlled by an nVersion increase.
>>
>> Comments please!
>
> You should probably add making CHECKMULTISIG require the dummy value to
> be exactly equal to OP_FALSE; verifying that in the transaction itself is
> laborious. A more subtle example is we may want both CHECKSIG and
> CHECKMULTISIG to fail the transaction if the signature is invalid but
> not exactly equal to OP_FALSE; some transaction forms are significantly
> more compact if you can have failed signatures, but that's a source of
> malleability. (are there counter examples people can think of?)
>
>
> But as I said on IRC, I'm a bit hesitant to bake in assumptions about
> malleability when we have no solid idea if ECC signatures are or are not
> malleable on a fundemental level; if "whack-a-mole" anti-malleability is
> all we've got it could be ugly if a break is found. Similarly, we may
> find we missed something, or some needed change makes the malleability
> rules difficult to work with for some new script type that is required.
>
> I'd rather see a new CHECKSIG mode for the case where malleability
> absolutely must be eliminated - certain multi-party protocols - and fix
> wallet software instead. (the malleability problems people see are
> closely related to inability to handle double-spends and reorgs) But I
> can easily see that being an impossible goal engineering wise...
>
> --
> 'peter'[:-1]@petertodd.org
> 0000000000000001465bc2730ffed7493d166d18d288f6cf15e8cdb5d4a3c7b1
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

📅 Original date posted:2013-05-31 📝 Original message:On ...

2023-06-07T15:02:33Z

In reply to nevent1q…x6m8
_________________________

📅 Original date posted:2013-05-31
📝 Original message:On Fri, May 31, 2013 at 2:10 PM, Michael Hendricks <michael at ndrix.org>wrote:

> On Fri, May 31, 2013 at 5:56 AM, Rune Kjær Svendsen <runesvend at gmail.com>wrote:
>
>> I have an application that wants to keep up with new blocks as they come
>> in. For that I can use the -blocknotify option with bitcoind, which will
>> execute my application for each new block.
>>
>> The problem is that my app isn't necessarily quick enough to finish its
>> work before a new block comes in and the app is executed again.
>>
>
> In a similar circumstance, I changed my -blocknotify script to quickly
> append necessary information to a queue and immediately exit. A separate
> script runs at all times monitoring this queue for work and performs the
> labor intensive calculations.
>

I've thought about this as well. It just seems somewhat clunky to me. I'd
really prefer having bitcoind put out messages in batches, if it's doable,
that is.

I'd run into a lot of concurrency issues, as far as I can see, where I
can't be sure that the queue isn't written to while, for example, it is
opened by the program that needs to process the queue items.

What if a disk operation takes a long time to finish, and a two queue
operations want to add to the queue simultaneously? This really brings
forward all the horrors of concurrent programming.

On Fri, May 31, 2013 at 2:17 PM, Jeremy Spilman <jeremy at taplink.co> wrote:

> Would it work to just block the bitcoind thread until your process exits?
>

I don't think that's optimal, no. That would slow down synchronization
drastically.

It would be really nimble for bitcoind to be able to synchronize at full
speed, and only send out events when necessary, batching together
previously queued items.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20130531/56c9f354/attachment.html>;

📅 Original date posted:2013-05-31 📝 Original message:Hello ...

2023-06-07T15:02:32Z

In reply to nevent1q…jvpy
_________________________

📅 Original date posted:2013-05-31
📝 Original message:Hello dear list

I have an application that wants to keep up with new blocks as they come
in. For that I can use the -blocknotify option with bitcoind, which will
execute my application for each new block.

The problem is that my app isn't necessarily quick enough to finish its
work before a new block comes in and the app is executed again. This means
the that bitcoind might keep executing my application even though the
previous instance hasn't finished, and that's fairly inefficient
resource-wise, as many instances of the application will be running
simultaneously.

I've discussed this with wumpus on bitcoin-dev, and we figured out a
solution that might be better. It could replace -blocknotify or we could
put it in a new function called -batchblocknotify

The idea is that when bitcoind is executed with the -batchblocknotify
option, and it's missing a lot of blocks, upon the first incoming block,
the command specified by -batchblocknotify is executed, and if additional
blocks come in while this command is still running, we add the block hashes
to a list instead of executing the command again. When the previous command
finishes, we execute it again and pass two parameters to it: 1. the first
block hash in the list of queued blocks, and 2. the number of blocks that
have come in while the last command was executing.

This prevents bitcoind from "fork bombing" the system, and allows the
command to handle incoming blocks in batches.

Would this make sense as an approach?

I've been looking at the code and I'm not sure how to implement it.

As far as I can see, I need to pass an object - whose state is retained
between calls - to the thread function (runCommand) that runs the command,
which contains a variable that keeps track of whether a previously executed
command is still running, and that contains a list of block hashes that
haven't been processed. And I'm not sure how to do this.

The runCommand thread is started in SetBestChain() in
main.cpp. SetBestChain() is executed by ConnectBestBlock() in main.cpp.
ConnectBestBlock() is executed by CBlock::AddToBlockIndex() in main.cpp.
CBlock::AddToBlockIndex() is executed by CBlock::AcceptBlock() in main.cpp.
CBlock::AcceptBlock() is executed by ProcessBlock() in main.cpp.
ProcessBlock() is executed by ProcessMessage() in main.cpp. And so on, and
so forth.

What's the right way to create an object that can be passed to the
runCommand thread, whose state is retained, so it can hold information
about whether the -batchblocknotify command is still executing, and contain
a list of blocks that are waiting to be passed to the -batchblocknotify
command?

I assume I shouldn't add a new parameter to the ProcessMessage() function,
which passes it to ProcessBlock(), which passes it to AcceptBlock() which
passes it to AddToBlockIndex()... and so on.

Would it be appropriate to store this object inside the CValidationState
class that is passed to SetBestChain()?

I'm not quite so how to go about this.

/Rune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20130531/73ed57be/attachment.html>;