--- Day changed Sun Nov 15 2015
00:03 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
00:03 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
00:06 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
00:06 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
00:07 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
00:08 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
00:13 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
00:13 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
00:37 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-dqoeuasnutjtugwz] has joined #bitcoin-core-dev
00:44 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
00:45 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
00:53 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
01:07 -!- ParadoxSpiral [~ParadoxSp@p508B9C40.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
01:23 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]
01:54 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
02:36 -!- ParadoxSpiral [~ParadoxSp@p508B9C40.dip0.t-ipconnect.de] has quit [Remote host closed the connection]
03:04 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev
03:06 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has quit [Remote host closed the connection]
04:27 -!- CodeShark_ [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has quit [Ping timeout: 240 seconds]
04:50 -!- Squidicc [~squid@pool-173-48-117-206.bstnma.fios.verizon.net] has joined #bitcoin-core-dev
04:52 -!- Squidicuz [~squid@pool-173-48-117-206.bstnma.fios.verizon.net] has quit [Ping timeout: 250 seconds]
05:44 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev
06:04 < morcos> Luke-Jr: oh interesting. i thought i'd run all the RPC tests, but hadn't noticed that the CLTV tests were not added to the extended tests list.  we should probably add them.
06:04 < morcos> (for 0.11 that is)
06:42 -!- bsm117532 [~mcelrath@static-108-21-236-13.nycmny.fios.verizon.net] has joined #bitcoin-core-dev
06:51 -!- jl2012_ [~jl2012@119246245241.ctinets.com] has quit [Read error: Connection reset by peer]
06:52 -!- jl2012 [~jl2012@unaffiliated/jl2012] has joined #bitcoin-core-dev
06:56 -!- zooko [~user@71-218-216-6.hlrn.qwest.net] has joined #bitcoin-core-dev
07:01 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
07:01 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
07:57 -!- MarcoFalke [8af6026a@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.106] has joined #bitcoin-core-dev
08:24 -!- zooko [~user@71-218-216-6.hlrn.qwest.net] has quit [Ping timeout: 250 seconds]
08:35 -!- JackH [~Jack@host-80-43-142-236.as13285.net] has joined #bitcoin-core-dev
10:21 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
10:21 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Max SendQ exceeded]
10:22 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
10:36 -!- arubi [~ese168@unaffiliated/arubi] has joined #bitcoin-core-dev
10:53 -!- ParadoxSpiral [~ParadoxSp@p508B9C40.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
11:31 -!- MarcoFalke [8af6026a@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.106] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]
11:34 -!- Thireus1 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
11:35 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Ping timeout: 244 seconds]
11:44 -!- evoskuil [~evoskuil@c-73-225-134-208.hsd1.wa.comcast.net] has quit [Ping timeout: 240 seconds]
12:08 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
12:09 < morcos> oh heh, that really is annoying.  turns out this was already broken once in the opposite direction, and sdaftuar fixed it in master 3 weeks ago.
12:10 < morcos> but Luke-Jr the version of the cltv rpc test in 0.11 should work with 0.11?  did you manually run the new one against it instead of the one in the branch/tag
12:11 < Luke-Jr> morcos: IIRC I was backporting a newer test
12:12 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
12:12 < morcos> well either way, at least as of now, the version in 0.11 has the old meaning of the arguments and works, and the version in master has the new meaning and works.
12:22 < morcos> Oh so I think I was confused.  sendFreeTransactions is defaulted off for both QT and bitcoind?  those damn QT config settings always confuse me.
12:31 -!- jgarzik [~jgarzik@unaffiliated/jgarzik] has quit [Quit: Leaving]
12:33 -!- jgarzik [~jgarzik@104-178-201-106.lightspeed.tukrga.sbcglobal.net] has joined #bitcoin-core-dev
12:33 -!- jgarzik [~jgarzik@104-178-201-106.lightspeed.tukrga.sbcglobal.net] has quit [Changing host]
12:33 -!- jgarzik [~jgarzik@unaffiliated/jgarzik] has joined #bitcoin-core-dev
12:53 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
13:03 < GitHub64> [bitcoin] morcos opened pull request #7021: add bip65 tests to rpc-tests.sh -extended (in 0.11 branch) (0.11...11rpcfixups) https://github.com/bitcoin/bitcoin/pull/7021
13:10 < GitHub84> [bitcoin] morcos opened pull request #7022: Change default block priority size to 0 (master...defaultPrioritySize) https://github.com/bitcoin/bitcoin/pull/7022
13:14 -!- evoskuil [~evoskuil@c-73-225-134-208.hsd1.wa.comcast.net] has joined #bitcoin-core-dev
13:17 < BlueMatt> gmaxwell: heh, funny, I hadnt even read scrollback and did a similar calculation for the relay network:
13:17 < BlueMatt> if you model the relay network as a pipe from the first node to receive any given block to the last node which will receive that block, it has an effective throughput of ~512kbps in the best case
13:17 < BlueMatt> so thats about right
14:22 -!- ParadoxSpiral [~ParadoxSp@p508B9C40.dip0.t-ipconnect.de] has quit [Remote host closed the connection]
14:38 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has quit [Ping timeout: 250 seconds]
14:50 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Quit: Leaving]
14:54 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]
15:09 -!- CodeShark_ [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has joined #bitcoin-core-dev
15:17 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
15:34 -!- d4de [~d4de@41.234.188.168] has quit [Ping timeout: 240 seconds]
15:40 -!- PaulCape_ [~PaulCapes@204.28.124.82] has quit [Quit: .]
15:41 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
15:55 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has joined #bitcoin-core-dev
16:19 -!- d4de [~d4de@41.234.213.218] has joined #bitcoin-core-dev
16:27 < dcousens> the *.dat files, they are assumed to be in order yeah?  I know the blocks in them may be out of order, but the files themselves are in order yeah?
16:28 < sipa> what does order for files mean, if the blocks are not in order?
16:28 < dcousens> sequential*
16:29 < dcousens> in that, data from blk001.dat "flows" into blk002.dat
16:29 < sipa> yes
16:29 < sipa> :)
16:29 < sipa> though that has been broken in the past
16:30 < dcousens> sipa: oh?
16:30 < dcousens> (having troubles with a parser, it makes it 370k blocks in before blowing up with the magic header being wrong)
16:30 < sipa> it would at startup rescan through all block files the first time a block was written, and if there was space in one, it would go there
16:30 < dcousens> (running a re-index now)
16:33 < dcousens> sipa: is there a separate index for the blocks in the .dat files?
16:33 < sipa> that's called the block index :p
16:33 < sipa> $DATADIR/blocks/indx
16:33 < sipa> $DATADIR/blocks/index
16:34 < dcousens> haha
16:34 < dcousens> Sure, is that referencing file pointers?
16:34 < dcousens> like, file+offset pointers
16:34 < sipa> indeed
16:35 < sipa> file number, byte offset
16:35 < dcousens> and even then, the blocks would still be back to back right, there shouldn't be empty space out of order data etc?
16:35 < dcousens> like,  I wouldn't expect a block to flow from 001.dat to 003.dat,
16:35 < dcousens> (that is, skipping 002.dat)
16:35 < sipa> that's certainly possible
16:35 < sipa> there are really no guarantees
16:36 < sipa> we try to put blocks in order
16:36 < sipa> to the extent possible
16:36 < sipa> but only by limiting how much out-of-order blocks we fetch
16:36 < dcousens> sipa: wouldn't that mean the block index would have to maintain multiple file number/offsets?
16:36 < sipa> how so?
16:36 < dcousens> per block
16:37 < sipa> oh! a block is always in only exactly one file
16:37 < sipa> continuously
16:37 < dcousens> with a possibility to flow into the next file, right?
16:37 < sipa> how do you mean 'flow into' ?
16:37 < dcousens> half in blk001.dat, half in blk002.dat
16:37 < sipa> no
16:37 < sipa> always continuously in one file
16:37 < sipa> if file N can't fit it, it goes into file N+!
16:38 < sipa> which is why the block files aren't exactly the same size :)
16:40 < dcousens> ok! easy
16:41 < dcousens> but the block files themselves could contain complete garbage data mixed with actual data
16:41 < dcousens> conceptually
16:41 < sipa> yes
16:41 < sipa> though that too is something we try to prevent :)
16:42 < dcousens> well, crap. Haha, I'll let you know how I go, but, considered I got 360k blocks in... might be easier to just trash the ?garbage? blocks and see how it goes after
16:43 < sipa> what is the problem?
16:43 < dcousens> "(having troubles with a parser, it makes it ~365k blocks in before blowing up with the magic header being wrong)"
16:44 < sipa> well scan for the first valid header :)
16:44 < dcousens> but the parser makes the assumption the data is contigous with no gaps (aka, no garbage)
16:44 < dcousens> sipa: might do, just wasn't aware that was necessary
16:45 < sipa> i think it's very unlikely to happen in 0.11 or later
16:45 < sipa> would require block index corruption
16:45 < dcousens> is a fresh build and IBD from master
16:45 < sipa> weird
16:45 < sipa> maybe i'm wrong
16:45 < sipa> maybe your block.data is just actually corrupted? :)
16:45 < dcousens> maybe I am, like I said, trying a re-index now
16:45 < dcousens> see how bitcoind goes, if it gets past it
16:45 < dcousens> well
16:46 < dcousens> i know exactly where its blowing up, so I can hex dump and have a looksy :)
16:46 < dcousens> thankfully the parser only takes 23s to run
16:47 < sipa> that's impressive
16:47 < sipa> wait
16:48 < dcousens> no script validation, obviously
16:48 < sipa> you parse 2 Gbyte per second?
16:48 < dcousens> that sounds a bit majestic doesn't it
16:49 < dcousens> and not right...
16:49 < dcousens> but, its a full parse.  I've got a 100MB buffer and the disk IO is going hard
16:49 < dcousens> its on AWS
16:50 < dcousens> so reading 100MB at a time, lazy loads the block in terms of what I want out of it (in this case, for a script index)
16:50 < sipa> well how else do you parse 50 Gbyte of block data in 23s?
16:50 < sipa> oh, you don't actually read the block data, just the headers?
16:50 < dcousens> sipa: technically I do though, like, I'm not doing an ftell...
16:51 < sipa> not following
16:51 < dcousens> in terms of IO...
16:51 < dcousens> sipa: 2GB seems a bit fast for what IO should be, AFAIK the AWS SSD's are only about 600mb/s
16:51 < dcousens> remembering, this is 365k blocks
16:52 < sipa> ah, a bit less
16:52 < dcousens> assuming 0.5mb for the remaining 20k?
16:52 < dcousens> thats like, 10GB less
16:52 < dcousens> still... 1.7GB/s ?
17:13 < GitHub18> [bitcoin] CodeShark opened pull request #7023: Fixed integer comparison warning. (master...signed_int_comparison_fix) https://github.com/bitcoin/bitcoin/pull/7023
17:28 -!- challisto [~challisto@c-76-16-149-33.hsd1.il.comcast.net] has joined #bitcoin-core-dev
17:28 -!- challisto [~challisto@c-76-16-149-33.hsd1.il.comcast.net] has quit [Changing host]
17:28 -!- challisto [~challisto@unaffiliated/challisto] has joined #bitcoin-core-dev
17:33 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has quit [Ping timeout: 246 seconds]
17:33 -!- cfields [~quassel@unaffiliated/cfields] has quit [Remote host closed the connection]
17:34 -!- lightningbot [supybot@2400:8900::f03c:91ff:fedf:3a06] has quit [Remote host closed the connection]
17:34 -!- lightningbot [supybot@2400:8900::f03c:91ff:fedf:3a06] has joined #bitcoin-core-dev
17:34 -!- luke-jr_ [~luke-jr@unaffiliated/luke-jr] has joined #bitcoin-core-dev
17:34 -!- Luke-Jr [~luke-jr@unaffiliated/luke-jr] has quit [Ping timeout: 246 seconds]
17:37 -!- cfields [~quassel@unaffiliated/cfields] has joined #bitcoin-core-dev
17:38 -!- jonasschnelli [~jonasschn@2a01:4f8:200:7025::2] has joined #bitcoin-core-dev
17:44 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-dqoeuasnutjtugwz] has quit [Quit: Connection closed for inactivity]
17:54 -!- luke-jr_ is now known as Luke-Jr
18:19 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
18:25 -!- CodeShark_ [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has quit [Ping timeout: 250 seconds]
18:36 < phantomcircuit> jgarzik, did you ever benchmark sqlite against leveldb with the WAL settings?
18:36 < jgarzik> phantomcircuit, yes and no - was in the 'obviously too slow' category
18:36 < jgarzik> sqlite4 storage engine looks interesting
18:37 < phantomcircuit> jgarzik, that's what i guessed
18:37 < phantomcircuit> the python data node i wrote in ~2010 had a super optimized sqlite3 db and even then it was comically slow
18:38 < phantomcircuit> even with a proper db like postgres it was pretty horrendously slow
19:20 < dcousens> jgarzik: even with prepared statements etc?
19:20 < jgarzik> dcousens, yes
19:20 -!- droark [~droark@c-24-22-36-12.hsd1.or.comcast.net] has joined #bitcoin-core-dev
19:29 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
19:39 -!- zooko [~user@2601:281:8001:26aa:8874:a77d:6fe6:611e] has joined #bitcoin-core-dev
19:40 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
19:54 -!- arowser [~quassel@106.120.101.38] has joined #bitcoin-core-dev
20:01 -!- go1111111 [~go1111111@162.244.138.37] has quit [Ping timeout: 240 seconds]
20:03 < gmaxwell> it's not really that sqllite is "slow", it's pretty fast for what it does; it's just that dedicated transactional key value stores are really quite fast.
20:05 < gmaxwell> for me bitcoin core sync runs at around 8000tx/s (with secp256k1), even though every tx is involving a non-cachable missing record check for bip30 (in addition to all the cached stuff)... thats really quite remarkable IMO.
20:15 -!- go1111111 [~go1111111@104.200.154.41] has joined #bitcoin-core-dev
20:20 -!- zooko [~user@2601:281:8001:26aa:8874:a77d:6fe6:611e] has quit [Ping timeout: 240 seconds]
20:22 -!- challisto [~challisto@unaffiliated/challisto] has quit [Quit: Leaving]
20:44 -!- Anduck [~anduck@unaffiliated/anduck] has quit [Ping timeout: 255 seconds]
20:45 -!- Anduck [~anduck@unaffiliated/anduck] has joined #bitcoin-core-dev
20:49 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
21:18 -!- zarathustra [zanoni@unaffiliated/tolstoi] has quit [Quit: do what thou wilt shall be the whole of the law]
21:39 -!- Anduck [~anduck@unaffiliated/anduck] has quit [Ping timeout: 276 seconds]
21:46 -!- Anduck [~anduck@unaffiliated/anduck] has joined #bitcoin-core-dev
21:48 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
22:41 -!- Thireus1 [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
22:41 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
22:45 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-aimtdiiuklyzfhvg] has joined #bitcoin-core-dev
23:01 -!- guest234234 [~guest2342@171.5.157.75] has joined #bitcoin-core-dev
23:03 -!- Guest23423 [~guest2342@171.5.157.75] has joined #bitcoin-core-dev
23:06 -!- guest234234 [~guest2342@171.5.157.75] has quit [Ping timeout: 264 seconds]
23:11 -!- gavinand1esen [~gavin@unaffiliated/gavinandresen] has joined #bitcoin-core-dev
23:11 -!- btcdrak_ [uid115429@gateway/web/irccloud.com/x-yvkcjomrigurxopx] has joined #bitcoin-core-dev
23:12 -!- Anduck [~anduck@unaffiliated/anduck] has quit [Ping timeout: 240 seconds]
23:13 -!- Anduck [~anduck@unaffiliated/anduck] has joined #bitcoin-core-dev
23:14 -!- CodeShark_ [uid126576@gateway/web/irccloud.com/x-qjfqykpchqoysfzj] has joined #bitcoin-core-dev
23:15 -!- morcos_ [~morcos@static-100-38-11-146.nycmny.fios.verizon.net] has joined #bitcoin-core-dev
23:15 -!- teward- [teward@ubuntu/member/teward] has joined #bitcoin-core-dev
23:15 -!- btcdrak [uid115429@gateway/web/irccloud.com/x-feqmqxzmdxuavzvt] has quit [Ping timeout: 448 seconds]
23:15 -!- teward [teward@ubuntu/member/teward] has quit [Ping timeout: 448 seconds]
23:15 -!- CodeShark [uid126576@gateway/web/irccloud.com/x-gylzfgvtfqnzsvyc] has quit [Ping timeout: 448 seconds]
23:15 -!- morcos [~morcos@static-100-38-11-146.nycmny.fios.verizon.net] has quit [Remote host closed the connection]
23:15 -!- gavinandresen [~gavin@unaffiliated/gavinandresen] has quit [Read error: Connection reset by peer]
23:15 -!- teward- is now known as teward
23:15 -!- CodeShark_ is now known as CodeShark
23:16 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
23:16 -!- btcdrak_ is now known as btcdrak
23:16 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
23:28 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
23:30 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
23:32 -!- s1w [~s1w@unaffiliated/someoneweird] has quit [Ping timeout: 240 seconds]
23:38 -!- s1w [~s1w@2400:6180:0:d0::4e1:5001] has joined #bitcoin-core-dev
23:38 -!- jonasschnelli [~jonasschn@2a01:4f8:200:7025::2] has quit [Changing host]
23:38 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has joined #bitcoin-core-dev
23:39 -!- s1w is now known as Guest16267
23:40 < GitHub184> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/b632145edeb3...814697c5569c
23:40 < GitHub184> bitcoin/master 773ae46 Jonas Schnelli: [Qt] add shortcurts for debug-/console-window
23:40 < GitHub184> bitcoin/master 814697c Jonas Schnelli: Merge pull request #7000...
23:40 < GitHub19> [bitcoin] jonasschnelli closed pull request #7000: [Qt] add shortcurts for debug-/console-window (master...2015/11/qt_shortcuts) https://github.com/bitcoin/bitcoin/pull/7000
23:40 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
23:41 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
23:42  * btcdrak claps. jonas' first merge :)
23:43 < jonasschnelli> hah. Thanks btcdrak 
23:47 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
23:47 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
23:49 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
23:50 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev