* [Bitcoin-development] Synchronization: 19.5 % orphaned blocks at height 197'324 @ 2014-08-10 13:42 mbde 2014-08-10 14:07 ` Bob McElrath 2014-08-10 14:20 ` Jeff Garzik 0 siblings, 2 replies; 4+ messages in thread From: mbde @ 2014-08-10 13:42 UTC (permalink / raw) To: Bitcoin-development Hello all, I'm currently synchronizing a new node and right now, at a progress of a height of 197'324 blocks, I count in my debug.log an aweful amount of 38'447 orphaned blocks which is about 19.5 %. It has been I while since I watched the synchronization process closely, but this number seems pretty high to me. I'm wondering about the following: would it be possible for a malicious party to generate chains of blocks with low difficulity which are not part of the main chain to slow down the sync process? Build and version information: https://github.com/jmcorgan/bitcoin/tree/026686c7de76dfde6fcacfc3d667fb3418a946a7 (sipa/jmcorgan address index) Rebased with: https://github.com/bitcoin/bitcoin/tree/94e1b9e05b96e4fe639e5b07b7a53ea216170962 (almost up-to-date mainline) Compressed debug.log attached: https://www.dropbox.com/s/uvtd91xiwmdmun7/debug.7z?m= (filesize: 7.67 MB, uncompressed: 41.3 MB) ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Bitcoin-development] Synchronization: 19.5 % orphaned blocks at height 197'324 2014-08-10 13:42 [Bitcoin-development] Synchronization: 19.5 % orphaned blocks at height 197'324 mbde @ 2014-08-10 14:07 ` Bob McElrath 2014-08-10 18:07 ` Pieter Wuille 2014-08-10 14:20 ` Jeff Garzik 1 sibling, 1 reply; 4+ messages in thread From: Bob McElrath @ 2014-08-10 14:07 UTC (permalink / raw) To: mbde; +Cc: Bitcoin-development [-- Attachment #1.1: Type: text/plain, Size: 2046 bytes --] I had the same problem (repeatedly) which came down a hardware problem. Bitcoin more than other applications is very sensitive to single bit flips in memory or during computation. (In the end I under-clocked my CPU and RAM to fix the problem) Attached is a small python script which will run sha256 on random data repeatedly and will print a message if a mismatch is found. For me it took many hours of running before a sha256 mismatch, but one is enough to fork the blockchain. mbde@bitwatch•co [mbde@bitwatch•co] wrote: > Hello all, > > I'm currently synchronizing a new node and right now, at a progress of a > height of 197'324 blocks, I count in my debug.log an aweful amount of > 38'447 orphaned blocks which is about 19.5 %. > > It has been I while since I watched the synchronization process closely, > but this number seems pretty high to me. > > I'm wondering about the following: would it be possible for a malicious > party to generate chains of blocks with low difficulity which are not > part of the main chain to slow down the sync process? > > > Build and version information: > https://github.com/jmcorgan/bitcoin/tree/026686c7de76dfde6fcacfc3d667fb3418a946a7 > (sipa/jmcorgan address index) > Rebased with: > https://github.com/bitcoin/bitcoin/tree/94e1b9e05b96e4fe639e5b07b7a53ea216170962 > (almost up-to-date mainline) > > Compressed debug.log attached: > https://www.dropbox.com/s/uvtd91xiwmdmun7/debug.7z?m= > (filesize: 7.67 MB, uncompressed: 41.3 MB) > > ------------------------------------------------------------------------------ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Cheers, Bob McElrath "The individual has always had to struggle to keep from being overwhelmed by the tribe. If you try it, you will be lonely often, and sometimes frightened. But no price is too high to pay for the privilege of owning yourself." -- Friedrich Nietzsche [-- Attachment #1.2: sha256check.py --] [-- Type: text/x-python, Size: 792 bytes --] #!/usr/bin/python # Repeatedly run a sha256 on random data. Keeps a rolling buffer of the last # <buflen> hashes and re-checks them. Prints an error ONLY if a mismatch is # found. If a mismatch is found, you have a hardware problem. from hashlib import sha256 from collections import deque import random buflen = 100000 hashbuf = deque(maxlen=buflen) for i in range(buflen): hashbuf.append([str(i), sha256(str(i)).hexdigest()]) while True: k, khash = hashbuf.popleft() pophash = sha256(k).hexdigest() if pophash != khash: print "ERROR: sha256(%s) = %s does not match:"%(k, khash) print " sha256(%s) = %s"%(k, pophash) k = str(random.getrandbits(1000)) khash = sha256(k).hexdigest() hashbuf.append([k, khash]) [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Bitcoin-development] Synchronization: 19.5 % orphaned blocks at height 197'324 2014-08-10 14:07 ` Bob McElrath @ 2014-08-10 18:07 ` Pieter Wuille 0 siblings, 0 replies; 4+ messages in thread From: Pieter Wuille @ 2014-08-10 18:07 UTC (permalink / raw) To: Bob McElrath; +Cc: Bitcoin Dev On Sun, Aug 10, 2014 at 4:07 PM, Bob McElrath <bob_bitcoin@mcelrath•org> wrote: > I had the same problem (repeatedly) which came down a hardware problem. This is actually an independent problem (though something to be aware of). Flaky hardware can make synchronization fail completely - as it relies on being able to exactly assess the validity of everything in the blockchain. Stilll... > mbde@bitwatch•co [mbde@bitwatch•co] wrote: >> Hello all, >> >> I'm currently synchronizing a new node and right now, at a progress of a >> height of 197'324 blocks, I count in my debug.log an aweful amount of >> 38'447 orphaned blocks which is about 19.5 %. >> >> It has been I while since I watched the synchronization process closely, >> but this number seems pretty high to me. Orphan blocks during synchronization are unfortunately very common, and the result of a mostly broken download logic in the client. They are blocks that are further ahead in the chain than the point where you're currently synchronized to, and thus can't be validated yet. Note that 'orphan' here means 'we do not know the parent'; it doesn't just mean 'not in the main chain'. They are blocks that are received out of order. As Jeff mentions, headers-first synchronization fixes this problem (and many other download-logic related things), by first verifying the headers in the chain (thus already having partially validated everything), and then downloading the blocks (in not necessarily the right order) anymore, from multiple peers in parallel. There is currently a pull request for it, but it's not production ready (#4468). >> I'm wondering about the following: would it be possible for a malicious >> party to generate chains of blocks with low difficulity which are not >> part of the main chain to slow down the sync process? Yes and no. While you're still synchronization, and don't actually know the best chain, a peer could send you stale branches (with valid proof of work), which you would accept, store and process. But it has to be done very early, as once you learn of a good-enough chain, a branch with more proof of work would be requires due to some heuristics designed to exactly prevent such an attack. -- Pieter ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Bitcoin-development] Synchronization: 19.5 % orphaned blocks at height 197'324 2014-08-10 13:42 [Bitcoin-development] Synchronization: 19.5 % orphaned blocks at height 197'324 mbde 2014-08-10 14:07 ` Bob McElrath @ 2014-08-10 14:20 ` Jeff Garzik 1 sibling, 0 replies; 4+ messages in thread From: Jeff Garzik @ 2014-08-10 14:20 UTC (permalink / raw) To: mbde; +Cc: Bitcoin Dev This issue is being worked on, under the category of "headers first synchronization." Until that is finished, it is recommended that you download bootstrap.dat via torrent: https://bitcointalk.org/index.php?topic=145386.0 On Sun, Aug 10, 2014 at 9:42 AM, mbde@bitwatch•co <mbde@bitwatch•co> wrote: > Hello all, > > I'm currently synchronizing a new node and right now, at a progress of a > height of 197'324 blocks, I count in my debug.log an aweful amount of > 38'447 orphaned blocks which is about 19.5 %. > > It has been I while since I watched the synchronization process closely, > but this number seems pretty high to me. > > I'm wondering about the following: would it be possible for a malicious > party to generate chains of blocks with low difficulity which are not > part of the main chain to slow down the sync process? > > > Build and version information: > https://github.com/jmcorgan/bitcoin/tree/026686c7de76dfde6fcacfc3d667fb3418a946a7 > (sipa/jmcorgan address index) > Rebased with: > https://github.com/bitcoin/bitcoin/tree/94e1b9e05b96e4fe639e5b07b7a53ea216170962 > (almost up-to-date mainline) > > Compressed debug.log attached: > https://www.dropbox.com/s/uvtd91xiwmdmun7/debug.7z?m= > (filesize: 7.67 MB, uncompressed: 41.3 MB) > > ------------------------------------------------------------------------------ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-08-10 18:08 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-08-10 13:42 [Bitcoin-development] Synchronization: 19.5 % orphaned blocks at height 197'324 mbde 2014-08-10 14:07 ` Bob McElrath 2014-08-10 18:07 ` Pieter Wuille 2014-08-10 14:20 ` Jeff Garzik
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox