{"type":"rich","version":"1.0","author_name":"npub17ty4mumkv43w8wtt0xsz2jypck0gvw0j8xrcg6tpea25z2nh7meqf4qgyd","author_url":"https://nostr.ae/npub17ty4mumkv43w8wtt0xsz2jypck0gvw0j8xrcg6tpea25z2nh7meqf4qgyd","provider_name":"njump","provider_url":"https://nostr.ae","html":"📅 Original date posted:2013-07-17\n📝 Original message:Partial UTXO sets is a neat idea. Unfortunately my intuition is that many\nSPV wallets only remain open for \u003c1 minute at a time because the user wants\nto see they received money, or to send it. It'd be neat to get some\ntelemetry from the Android wallet for this - I will ask Andreas to let\nusers opt in to usage statistics.\n\nSo for anti-DoS I think smart prioritisation heuristics are the way to go\nagain. Perhaps by letting clients have an \"identity\" that they provide to a\nnode when it's load shedding. Clients that have been seen before, have a\ntrack record of not being abusive etc get priority and new clients that\nwere never seen before get dropped. Coming up with a way to do that whilst\npreserving privacy sounds like an interesting cryptographic challenge.\n\n\nOn Wed, Jul 17, 2013 at 12:58 PM, Peter Todd \u003cpete at petertodd.org\u003e wrote:\n\n\u003e On Tue, Jul 16, 2013 at 04:16:23PM +0200, Wendell wrote:\n\u003e \u003e Hello everyone,\n\u003e \u003e\n\u003e \u003e In the previous thread, I expressed interest in seeing an SPV bitcoind,\n\u003e further stating that I would fund such work. Mike Hearn followed up with\n\u003e some of Satoshi's old code for this, which is now quite broken. The offer\n\u003e and interest on my side still stand, as more diversity in SPV options seems\n\u003e like the right way to go.\n\u003e \u003e\n\u003e \u003e Time-permitting, I would really appreciate feedback from knowledgable\n\u003e parties about the possible approaches to an SPV bitcoind. We at Hive\n\u003e ideally want to see something that could one be merge into master, rather\n\u003e than a fork.\n\u003e\n\u003e Keep in mind that SPV mode is newer than many realize: bloom filters are\n\u003e a 0.8 feature, itself released only last Febuary. As John Dillon posted\n\u003e earlier this week in \"Protecting Bitcoin against network-wide DoS\n\u003e attack\" the Bitcoin codebase will have to implement much better anti-DoS\n\u003e attack defences soon, and in a decentralized system there aren't any\n\u003e options other than requiring peers to either do work (useful or not) or\n\u003e sacrifice something of value. SPV peers can't do useful work, leaving\n\u003e only sacrifice - to what extent and how much is unknown. In addition SPV\n\u003e nodes have serious privacy issues because their peers know that any\n\u003e transaction sent to them by the SPV node is guaranteed to be from the\n\u003e node rather than relayed; bloom filters are only really helpful with\n\u003e payment protocols that don't exist yet and don't apply to merchants.\n\u003e Then you have MITM problems, vulnerability to fake blocks etc.\n\u003e\n\u003e It'll be awhile before we know how serious these issues are in practice,\n\u003e and we're likely to find new issues we didn't think of too. In any case\n\u003e Bitcoin is far better off if we make it easy to run a full node,\n\u003e donating whatever resources you can. Fortunately there's a whole\n\u003e continuum between SPV and full nodes.\n\u003e\n\u003e The way you do this is by maintaining partial UTXO sets. The trick is\n\u003e that if you have verified every block in some range i to j, every time\n\u003e you see a txout created by a transaction, and not subsequently spent,\n\u003e you can be sure that at height j the txout existed. If height j is the\n\u003e current block, you can be sure the txout exists provided that the chain\n\u003e itself is valid. Any transaction that only spends txouts in this partial\n\u003e set is a transaction you can fully verify and safely relay; for other\n\u003e transactions you just don't know and have to wait until you see them in\n\u003e a block.\n\u003e\n\u003e So what's useful about that? Basically it means your node starts with\n\u003e the same security level, and usefulness to the network, as a SPV node.\n\u003e But over time you keep downloading blocks as they are created, and with\n\u003e whatever bandwidth you have left (out of some user-configurable\n\u003e allocation) you download additional blocks going further and further\n\u003e back in time. Gradually your UTXO set becomes more complete, and over\n\u003e time you can verify a higher and higher % of all valid transactions.\n\u003e Eventually your node becomes a full node, but in the meantime it was\n\u003e still useful for the user, and still contributed to the network by\n\u003e relaying blocks and an increasingly large subset of all transactions.\n\u003e (optionally you can store a subset of the chain history too for other\n\u003e nodes to bootstrap from) You've also got better security because you\n\u003e *are* validating blocks, starting off incompletely, and increasingly\n\u003e completely until your finally validating fully. Privacy is improved, for\n\u003e both you and others, by mixing your transactions with others and adding\n\u003e to the overall anonymity set.\n\u003e\n\u003e In the future we'll have miners commit a hash of the UTXO set, and that\n\u003e gives us even more options to, for instance, have relayed transactions\n\u003e include proof that their inputs were valid, allowing all nodes to relay\n\u003e them safely.\n\u003e\n\u003e\n\u003e As for specifics, you need to maintain a UTXO set, and in addition a set\n\u003e of spent txouts (the STXO set) for which you haven't seen the\n\u003e transaction that created the txout. As download newer blocks you update\n\u003e the UTXO set; as you download older blocks you update the UTXO set and\n\u003e STXO set.\n\u003e\n\u003e Nodes now advertise this new variable to their peers:\n\u003e\n\u003e nOldestBlock - The oldest block that we've validated. (and all\n\u003e subsequent blocks)\n\u003e\n\u003e We'll also want the ability to advertise what sub-ranges of the\n\u003e blockchain data we have on hand:\n\u003e\n\u003e listArchivedBlockRanges - lists of (begin, end pairs)\n\u003e\n\u003e Nodes should drop all but the largest n pairs, say 5 or something. The\n\u003e index -1 is reserved to indicate the last block to make it easy to\n\u003e advertise that you have every block starting at some height to the most\n\u003e recent. (reserving -n with n as the last block might be a better choice\n\u003e to show intent, but still allow for specific proofs when we get node\n\u003e identities)\n\u003e\n\u003e We probably want to define a NODE_PARTIAL service bit or something; I'll\n\u003e have to re-read Pieter Wuille's proposal and think about it. Nodes\n\u003e should NOT advertize NODE_NETWORK unless they have the full chain and\n\u003e have verified it.\n\u003e\n\u003e Nodes with partial peers should only relay transactions to those peers\n\u003e if the transactions spend inputs the peers know about - remember how\n\u003e even an SPV node has that information if it's not spending unconfirmed\n\u003e inputs it didn't create. Nodes will have to update their peers\n\u003e periodically as nOldestBlock changes. That said it may also be\n\u003e worthwhile to simply relay all transactions in some cases too - a\n\u003e reasonable way to approach this might be to set a bloom filter for tx's\n\u003e that you *definitely* want, and if you are interested in everything,\n\u003e just set the filter to all 1's. If someone comes up with a reasonable\n\u003e micropayment or proof-of-work system even relaying txs that you haven't\n\u003e validated is fine - the proof-of-work and prioritization will prevent\n\u003e DoS attacks just fine.\n\u003e\n\u003e Remember that if you're running a partial node, it can get new blocks\n\u003e from any partial node, and it can retrieve historic blockchain data from\n\u003e any partial node that has archived the sequence of blocks you need next.\n\u003e On a large scale this is similar to how in BitTorrent you can serve data\n\u003e to your peers the moment you get it - a significant scalability\n\u003e improvement for the network as a whole. Even if a large % of the network\n\u003e was partial nodes running for just a few hours a day the whole system\n\u003e would work fine due to how partial nodes can serve each other the data\n\u003e they need.\n\u003e\n\u003e On startup you can act as a SPV node temporarily, grabbing asking for\n\u003e filtered blocks matching your wallet, and then go back and get the full\n\u003e blocks, or just download the full blocks right away. That's a tradeoff\n\u003e on how long the node has been off.\n\u003e\n\u003e Anyway, it's a bit more code compared to pure-SPV, but it results in a\n\u003e much more scalable Bitcoin, and if you can spare the modest bandwidth\n\u003e requirements to keep up with the blockchain it'll result in much better\n\u003e robustness against DoS attacks for you and Bitcoin in general.\n\u003e\n\u003e --\n\u003e 'peter'[:-1]@petertodd.org\n\u003e\n\u003e\n\u003e ------------------------------------------------------------------------------\n\u003e See everything from the browser to the database with AppDynamics\n\u003e Get end-to-end visibility with application monitoring from AppDynamics\n\u003e Isolate bottlenecks and diagnose root cause in seconds.\n\u003e Start your free trial of AppDynamics Pro today!\n\u003e http://pubads.g.doubleclick.net/gampad/clk?id=48808831\u0026iu=/4140/ostg.clktrk\n\u003e _______________________________________________\n\u003e Bitcoin-development mailing list\n\u003e Bitcoin-development at lists.sourceforge.net\n\u003e https://lists.sourceforge.net/lists/listinfo/bitcoin-development\n\u003e\n\u003e\n-------------- next part --------------\nAn HTML attachment was scrubbed...\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20130717/98b856c4/attachment.html\u003e"}