{"type":"rich","version":"1.0","author_name":"npub16cadfx8ew8z4qmsmsd9kndvntt4uad7vsdmkjer4jd8wjq4swxrqp2gy5d","author_url":"https://nostr.ae/npub16cadfx8ew8z4qmsmsd9kndvntt4uad7vsdmkjer4jd8wjq4swxrqp2gy5d","provider_name":"njump","provider_url":"https://nostr.ae","html":"📅 Original date posted:2012-07-25\n📝 Original message:On 25/07/2012 10:45, Michael Grønager wrote:\n\u003e Hi Steve,\n\u003e \n\u003e I see dramatic differences in performance on virtual machines vs \n\u003e running directly on the iron. I am not an expert in virtual\n\u003e machines,\n\nThey can be, it depends on how they are set up.  For reference, these\nVM's used to test network stacks and file format bugs.  They do this\nvia debug tracing, trace into, not over.  They then dump this data in\nfiles and it can keep up with a core2 duo laptop for file io. however\nI moved to using an in ram database (4gb chunks, that then get dumped\nover the network port to a db on a seperate machine)\n\nI am not sure I have the skills to instrament this into bitcoind\n\n\u003e \n\u003e I would like to do a test keeping database log files in memory. It \n\u003e should not matter for durability of the wallet, as it flushes at\n\u003e each write anyway. As for the blockindex, it will remain\n\u003e consistent, but might be lagging some blocks behind at startup,\n\u003e which shouldn't really matter (except that the same block could end\n\u003e up appearing twice in the block00X files, inelegant, but not really\n\u003e a problem).\n\u003e \n\u003e Otherwise the system you describe (raid0 over 6 disks) should\n\u003e perform like crazy wrt disk i/o, at least on par with SSD. It is\n\u003e your virtualization I am worried about.\n\nI will test that when I get back :)\n\n\u003e \n\u003e Have a safe trip to down under!\n\nwill do, thanks for the response. :)\n\n\u003e \n\u003e /M\n\u003e \n\u003e On 24/07/2012, at 21:56, steve wrote:\n\u003e \n\u003e\u003e Hi Michael,\n\u003e\u003e \n\u003e\u003e from what I have noticed, bitcoin blockchain\n\u003e\u003e download/verfication all happens in 1 thread.  (so multicores\n\u003e\u003e doesnt really help)\n\u003e\u003e \n\u003e\u003e That said, I have never tried on an ssd.\n\u003e\u003e \n\u003e\u003e What I do have is 6 SATA 6gbs configed as RAID0 Drives. 32gb of \n\u003e\u003e ram. ubuntu 64 (yeah I know), this runs upto 16 VM's (I have 4\n\u003e\u003e of these)\n\u003e\u003e \n\u003e\u003e However I have not tried to download the blockchain on the\n\u003e\u003e master os, just in virtulisation.  However, the dedicated\n\u003e\u003e machines that I have been using for benchmarking the VM's against\n\u003e\u003e is a q6600 8gb ram sata2 hdd - Win 7 (seems faster than\n\u003e\u003e slackware...) to me it has always felt like network bandwidth was\n\u003e\u003e the issue.  I might instrument the bitcoin-qt exe to only pick\n\u003e\u003e low ping nodes (has someone already done this?)\n\u003e\u003e \n\u003e\u003e I guess it is time to start some benchmarking (like the gpu \n\u003e\u003e comparison page)\n\u003e\u003e \n\u003e\u003e hte verification for the 5 past 5 days was negliglable. I am off\n\u003e\u003e on a flight to australia tomorrrow, so I will set some\n\u003e\u003e breakpoints and do some timings in a debugger.\n\u003e\u003e \n\u003e\u003e This will all happen on an e-450 (wonderful machine!)\n\u003e\u003e \n\u003e\u003e Thanks very much for your response. it would seem that I am\n\u003e\u003e 'doing it wrong' :/\n\u003e\u003e \n\u003e\u003e cheers mate,\n\u003e\u003e \n\u003e\u003e steve\n\u003e\u003e \n\u003e\u003e (this message isnt signed because I have forgotten my password.)\n\u003e\u003e \n\u003e\u003e On 24/07/2012 09:25, Michael Grønager wrote:\n\u003e\u003e\u003e Hi Steve,\n\u003e\u003e\u003e \n\u003e\u003e\u003e 45-90 minutes - note that its numbers from March/April, so a\n\u003e\u003e\u003e bit longer today, but far, far away from the 12 hours.\n\u003e\u003e\u003e \n\u003e\u003e\u003e I am using libcoin and the bitcoind build based on this.\n\u003e\u003e\u003e Libcoin is based on the Satoshi client, but refactured to use\n\u003e\u003e\u003e an async concurrency model. I also did a minor tweeks to the\n\u003e\u003e\u003e db parameters. It has earlier been tested up against Satoshi\n\u003e\u003e\u003e bitcoin where on some OS'es it performs similarly (at least on\n\u003e\u003e\u003e some linuxes) and on some faster (e.g. mac).\n\u003e\u003e\u003e \n\u003e\u003e\u003e What is your CPU load during a block download ? (both \n\u003e\u003e\u003e initially/up to the point where verification sets in and\n\u003e\u003e\u003e after). The initial download is typically disk I/O bound, the \n\u003e\u003e\u003e verification stage CPU bound, though I lean to believe that\n\u003e\u003e\u003e even there it is disk I/O bound (at least on my system ~50% CPU\n\u003e\u003e\u003e load). What should be better in libcoin is the concurrency\n\u003e\u003e\u003e model. The Satoshi client uses a pure reentrant mutexes model,\n\u003e\u003e\u003e that is not generally believed to motivate the best coding\n\u003e\u003e\u003e practice nor performance, you might end up without the\n\u003e\u003e\u003e concurrency you initially strived for *). As mentioned earlier\n\u003e\u003e\u003e libcoin uses a pure async concurrency model (and so does\n\u003e\u003e\u003e libbitcoin btw).\n\u003e\u003e\u003e \n\u003e\u003e\u003e I would like to stress again that these numbers will depend \n\u003e\u003e\u003e largely on the system running the test - I would call my laptop\n\u003e\u003e\u003e a bit over the average today (MB Pro, 2.66Ghz i7 dual core,\n\u003e\u003e\u003e 8GBRAM, 512GB SSD). But again 12 hours - I only reach such\n\u003e\u003e\u003e numbers on some of my VPS'es (linode 1024) that are known for\n\u003e\u003e\u003e notoriously slow disk I/O. (here I have a few % CPU load during\n\u003e\u003e\u003e the verification indicating indeed that the disk i/o is the \n\u003e\u003e\u003e culprit).\n\u003e\u003e\u003e \n\u003e\u003e\u003e Cheers,\n\u003e\u003e\u003e \n\u003e\u003e\u003e Michael\n\u003e\u003e\u003e \n\u003e\u003e\u003e \n\u003e\u003e\u003e *) I like this Dave Butenhof quote: \"The biggest of all the big\n\u003e\u003e\u003e  problems with recursive mutexes is that they encourage you to\n\u003e\u003e\u003e  completely lose track of your locking scheme and scope. This\n\u003e\u003e\u003e is deadly. Evil. It's the \"thread eater\". You hold locks for\n\u003e\u003e\u003e the absolutely shortest possible time. Period. Always. If\n\u003e\u003e\u003e you're calling something with a lock held simply because you\n\u003e\u003e\u003e don't know it's held, or because you don't know whether the\n\u003e\u003e\u003e callee needs the mutex, then you're holding it too long. You're\n\u003e\u003e\u003e aiming a shotgun at your application and pulling the trigger.\n\u003e\u003e\u003e You presumably started using threads to get concurrency; but\n\u003e\u003e\u003e you've just PREVENTED concurrency.\"\n\u003e\u003e\u003e \n\u003e\u003e\u003e \n\u003e\u003e\u003e \n\u003e\u003e\u003e \n\u003e\u003e\u003e On 23/07/2012, at 17:54, steve wrote:\n\u003e\u003e\u003e \n\u003e\u003e\u003e Hi Michael,\n\u003e\u003e\u003e \n\u003e\u003e\u003e On 23/07/2012 10:00, Michael Grønager wrote:\n\u003e\u003e\u003e\u003e\u003e\u003e I get a full blockchain from scratch in 45 minutes on my\n\u003e\u003e\u003e\u003e\u003e\u003e  laptop, /M\n\u003e\u003e\u003e\u003e\u003e\u003e \n\u003e\u003e\u003e Hang on a sec, in 45 minutes you can download the entire chain \n\u003e\u003e\u003e from the genesis block?\n\u003e\u003e\u003e \n\u003e\u003e\u003e I have been doing extensive testing in this area and would\n\u003e\u003e\u003e love to know what is special about your setup (I have never had\n\u003e\u003e\u003e the entire chain in under 12 hours, infact it is normally\n\u003e\u003e\u003e closerto 24.) I have an extensive setup of test machines,\n\u003e\u003e\u003e everything from e4300 to phenom2x6 to i5's.\n\u003e\u003e\u003e \n\u003e\u003e\u003e as an example on an amd e-450 with 4gb ram, and approx 3gb/s \n\u003e\u003e\u003e internet connection it took 2 hours to sync the last 5 days.\n\u003e\u003e\u003e \n\u003e\u003e\u003e Maybe i am missing something important...\n\u003e\u003e\u003e \n\u003e\u003e\u003e Any additional information that you could provide to help me\n\u003e\u003e\u003e with testing would be really appreciated.\n\u003e\u003e\u003e \n\u003e\u003e\u003e cheers,\n\u003e\u003e\u003e \n\u003e\u003e\u003e steve\n\u003e\u003e\u003e \n\u003e\u003e\u003e\u003e \n\u003e\u003e\u003e\u003e ------------------------------------------------------------------------------\n\u003e\u003e\u003e\u003e\n\u003e\u003e\u003e\u003e\n\u003e\u003e\n\u003e\u003e\u003e\u003e\n\u003e\u003e\u003e\u003e \nLive Security Virtual Conference\n\u003e\u003e\u003e\u003e Exclusive live event will cover all the ways today's\n\u003e\u003e\u003e\u003e security and threat landscape has changed and how IT managers\n\u003e\u003e\u003e\u003e can respond. Discussions will include endpoint security,\n\u003e\u003e\u003e\u003e mobile security and the latest in malware threats. \n\u003e\u003e\u003e\u003e http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ \n\u003e\u003e\u003e\u003e _______________________________________________ \n\u003e\u003e\u003e\u003e Bitcoin-development mailing list \n\u003e\u003e\u003e\u003e Bitcoin-development at lists.sourceforge.net \n\u003e\u003e\u003e\u003e https://lists.sourceforge.net/lists/listinfo/bitcoin-development\n\u003e\u003e\u003e\n\u003e\u003e\u003e\n\u003e\u003e\u003e\n\u003e\u003e\u003e\n\u003e\u003e\u003e\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e\n\u003e\u003e\u003e\u003e\n\u003e\u003e\u003e\u003e \n------------------------------------------------------------------------------\n\u003e\u003e Live Security Virtual Conference Exclusive live event will cover \n\u003e\u003e all the ways today's security and threat landscape has changed\n\u003e\u003e and how IT managers can respond. Discussions will include\n\u003e\u003e endpoint security, mobile security and the latest in malware\n\u003e\u003e threats. \n\u003e\u003e http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ \n\u003e\u003e _______________________________________________\n\u003e\u003e Bitcoin-development mailing list\n\u003e\u003e Bitcoin-development at lists.sourceforge.net \n\u003e\u003e https://lists.sourceforge.net/lists/listinfo/bitcoin-development\n\u003e"}