--- Day changed Thu Feb 23 2017 00:00 -!- norotartagen [~norotarta@71-89-76-184.dhcp.bycy.mi.charter.com] has quit [Remote host closed the connection] 00:00 < midnightmagic> auto-EULA might not be strong enough. 00:01 < wumpus> so you imagine something where you have to read an EULA and click for every node you connect to? 00:02 < wumpus> uhm, no, I'll pass 00:02 -!- norotartagen [~norotarta@71-89-76-184.dhcp.bycy.mi.charter.com] has joined #bitcoin-core-dev 00:02 < gmaxwell> midnightmagic: proposed previously was an alternative protocol for transaction relay that has standard requirements like that as just part of the protocol spec. 00:03 < gmaxwell> midnightmagic: anyone is free to create such a thing. 00:03 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 00:03 < midnightmagic> well, triangulation of an idea is nice to hear anyway. 00:03 < gmaxwell> Because so much of the abusive behavior we can detect is commercially motivated-- its an intresting question. 00:03 < midnightmagic> commercial motivation implies pockets, and corporation behavioural laws. 00:04 < wumpus> yes, separating transaction submission/relay would make sense 00:04 < gmaxwell> But it's not something that would make sense as the general Bitcoin protocol I think. Or at least it would be too easily trolled. But people are free to try it out and I hope they do. 00:04 < wumpus> that's the only thing people care to spy on anyway 00:04 < wumpus> for relaying blocks, it doesn't matter nearly as much 00:05 < midnightmagic> I was thinking just connecting at all. I wonder if it would devolve into horros e.g. to authorize your client to agree to EULA semi-automatically on your behalf based on acceptable terms. 00:06 < gmaxwell> midnightmagic: I think it only makes sense if the terms are basically just part of the protocol. 00:06 < midnightmagic> hrm. 00:07 < wumpus> making it harder to spy on transactions by using onion routing / some kind of deaddrop system, would do a lot against spying 00:07 < wumpus> better than getting involved inthe vagaries of international law 00:13 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 00:17 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 00:18 < wumpus> what are the exact specifications of the travis VMs? 00:19 < wumpus> ubuntu trusty at least, will try with that locally 00:25 < bitcoin-git> [bitcoin] benma opened pull request #9834: qt: clean up initialize/shutdown signals (master...exitcode) https://github.com/bitcoin/bitcoin/pull/9834 00:44 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 00:46 < wumpus> of course, it all passes perfectly in my own trusty vm... 00:47 < gmaxwell> make your vm slower? :P 00:48 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 00:48 < wumpus> I guess running a few instances of test_bitcoin in parallel could simulate that 00:54 < wumpus> looks like test_bitcoin doesn't clean up its UUID directories (/tmp/92bc-0a4a-f496-6b40) 00:54 < wumpus> while the /tmp/test_bitcoin_1487840020_6012 are being cleaned up 00:55 < wumpus> (yes, the tests finish succesfully) 01:02 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 260 seconds] 01:11 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-nwetehhdrcnhmqtu] has joined #bitcoin-core-dev 01:15 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 01:19 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 01:35 -!- ryankung [~ryankung@14.152.49.250] has quit [Remote host closed the connection] 01:39 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/bed5b30a5622...d14555de3de9 01:39 < bitcoin-git> bitcoin/master 874c736 Russell Yanofsky: Fix pruning test broken by 2 hour manual prune window... 01:39 < bitcoin-git> bitcoin/master d14555d Wladimir J. van der Laan: Merge #9820: Fix pruning test broken by 2 hour manual prune window... 01:39 < bitcoin-git> [bitcoin] laanwj closed pull request #9820: Fix pruning test broken by 2 hour manual prune window (master...pr/prunewind) https://github.com/bitcoin/bitcoin/pull/9820 01:39 < bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/599c69abe3101512eeee13733c0f58eb3e363eae 01:39 < bitcoin-git> bitcoin/0.14 599c69a Russell Yanofsky: Fix pruning test broken by 2 hour manual prune window... 01:41 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/d14555de3de9...1d2a57e9fd2c 01:41 < bitcoin-git> bitcoin/master fa4cd2e MarcoFalke: qa: Check return code when stopping nodes... 01:41 < bitcoin-git> bitcoin/master 1d2a57e Wladimir J. van der Laan: Merge #9824: qa: Check return code when stopping nodes... 01:41 < bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/260c71cbb857d12540a2f53372282d11f2b401a8 01:41 < bitcoin-git> bitcoin/0.14 260c71c MarcoFalke: qa: Check return code when stopping nodes... 01:41 < bitcoin-git> [bitcoin] laanwj closed pull request #9824: qa: Check return code when stopping nodes (master...Mf1702-qaRet) https://github.com/bitcoin/bitcoin/pull/9824 01:47 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 01:49 < bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/1d2a57e9fd2c...e68c266f3d56 01:49 < bitcoin-git> bitcoin/master b602fe0 Cory Fields: build: warn about variable length arrays 01:49 < bitcoin-git> bitcoin/master 205830a Cory Fields: build: add --enable-werror option... 01:49 < bitcoin-git> bitcoin/master e68c266 Wladimir J. van der Laan: Merge #9789: build: add --enable-werror and warn on vla's... 01:49 < bitcoin-git> [bitcoin] laanwj closed pull request #9789: build: add --enable-werror and warn on vla's (master...no-vla) https://github.com/bitcoin/bitcoin/pull/9789 01:49 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to 0.14: https://github.com/bitcoin/bitcoin/compare/260c71cbb857...05e906dbc68e 01:49 < bitcoin-git> bitcoin/0.14 749fe95 Cory Fields: build: warn about variable length arrays... 01:49 < bitcoin-git> bitcoin/0.14 05e906d Cory Fields: build: add --enable-werror option... 01:51 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 01:55 -!- Alina-malina [~Alina-mal@unaffiliated/alina-malina] has quit [Ping timeout: 260 seconds] 01:56 -!- rafalcpp [~racalcppp@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev 02:03 -!- vicenteH [~user@96.235.15.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 02:08 -!- vicenteH [~user@96.235.15.37.dynamic.jazztel.es] has quit [Ping timeout: 240 seconds] 02:10 -!- vicenteH [~user@96.235.15.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 02:18 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 02:20 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev 02:22 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 03:01 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 240 seconds] 03:08 -!- lclc [~lclc@unaffiliated/lclc] has quit [Ping timeout: 255 seconds] 03:09 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 03:15 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has joined #bitcoin-core-dev 03:22 < wumpus> I've been running test_bitcoin with five threads, for three hours on a trusty VM - no single failure 03:46 -!- lclc [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 03:48 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 260 seconds] 04:16 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 04:19 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 04:23 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 04:40 < warren> Hmm, the blk*.dat files exported by linearize-data.py is slightly modified by bitcoind if you drop them in your blocks/ directory. http://pastebin.com/PH99QYdm diff of two hexdumps shows a tiny bit at the end of each file is truncated. 04:49 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 04:54 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 05:21 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 05:26 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 05:34 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 268 seconds] 05:35 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 05:38 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 05:38 < wumpus> warren: hm that;s a bug 05:39 < wumpus> only the last blk.dat file should be modified 05:39 < wumpus> oh, of course, it scans them again from the beginning, so every file is 'last file' for a while 05:40 < wumpus> so the truncating behavior makes some sense - don't think it necessarily needs to do that during reindexing, but it's not unexpected 05:41 < wumpus> this is different from the bootstrap.dat file which was really an external file which was imported 05:49 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 06:05 -!- goksinen [~goksinen@cpe-74-71-4-175.nyc.res.rr.com] has joined #bitcoin-core-dev 06:06 -!- goksinen [~goksinen@cpe-74-71-4-175.nyc.res.rr.com] has quit [Read error: Connection reset by peer] 06:07 -!- goksinen [~goksinen@2604:2000:c591:8400:1140:1b7b:9932:1c90] has joined #bitcoin-core-dev 06:08 -!- goksinen_ [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 06:11 -!- cannon-c [ccc23f04@gateway/web/freenode/ip.204.194.63.4] has quit [Quit: Page closed] 06:12 -!- goksinen_ [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 06:14 -!- nickler [~nickler@185.12.46.130] has quit [Ping timeout: 260 seconds] 06:20 -!- goksinen_ [~goksinen@2604:2000:c591:8400:24db:6ba6:2d93:b634] has joined #bitcoin-core-dev 06:22 -!- goksinen [~goksinen@2604:2000:c591:8400:1140:1b7b:9932:1c90] has quit [Ping timeout: 255 seconds] 06:26 -!- nickler [~nickler@185.12.46.130] has joined #bitcoin-core-dev 06:29 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 06:33 -!- jnewbery_ [~john@static-100-38-11-146.nycmny.fios.verizon.net] has joined #bitcoin-core-dev 06:35 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 06:43 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 06:50 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev 07:03 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 07:07 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 07:08 -!- city22 [~textual@221.220.183.38] has joined #bitcoin-core-dev 07:09 -!- city22 [~textual@221.220.183.38] has quit [Client Quit] 07:16 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 07:21 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 07:23 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 260 seconds] 07:27 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 07:29 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 07:35 < bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e68c266f3d56...7146d96de3e1 07:35 < bitcoin-git> bitcoin/master c578408 John Newbery: Add exclude option to rpc-tests.py 07:35 < bitcoin-git> bitcoin/master 7146d96 MarcoFalke: Merge #9766: Add --exclude option to rpc-tests.py... 07:35 < bitcoin-git> [bitcoin] MarcoFalke closed pull request #9766: Add --exclude option to rpc-tests.py (master...rpctestexclude) https://github.com/bitcoin/bitcoin/pull/9766 07:38 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev 07:40 < bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/7146d96de3e1...d6064a89ac97 07:40 < bitcoin-git> bitcoin/master 3f95a80 John Newbery: Fix docstrings in qa tests... 07:40 < bitcoin-git> bitcoin/master d6064a8 MarcoFalke: Merge #9577: Fix docstrings in qa tests... 07:40 < bitcoin-git> [bitcoin] MarcoFalke closed pull request #9577: Fix docstrings in qa tests (master...docstrings) https://github.com/bitcoin/bitcoin/pull/9577 07:40 < MarcoFalke> Great work on the test framework jnewbery! 07:41 < jnewbery> Thanks MarcoFalke! 07:46 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Quit: Leaving.] 07:46 -!- Giszmo [~leo@ip-146-233.219.201.nextelmovil.cl] has joined #bitcoin-core-dev 07:55 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 08:02 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt] 08:15 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection] 08:15 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 08:20 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 260 seconds] 08:34 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 08:37 < bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/d6064a89ac97...a13a417cdcfd 08:37 < bitcoin-git> bitcoin/master 3333ad0 MarcoFalke: qa: Set correct path for binaries in rpc tests 08:37 < bitcoin-git> bitcoin/master a13a417 MarcoFalke: Merge #9823: qa: Set correct path for binaries in rpc tests... 08:37 < bitcoin-git> [bitcoin] MarcoFalke closed pull request #9823: qa: Set correct path for binaries in rpc tests (master...Mf1702-qaPath) https://github.com/bitcoin/bitcoin/pull/9823 08:37 -!- kyletorpey [~kyle@pool-71-176-227-116.rcmdva.fios.verizon.net] has joined #bitcoin-core-dev 08:38 -!- kyletorpey [~kyle@pool-71-176-227-116.rcmdva.fios.verizon.net] has left #bitcoin-core-dev [] 08:47 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev 08:50 -!- abpa [~abpa@96-82-80-25-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 08:57 < bitcoin-git> [bitcoin] DannyHamilton opened pull request #9838: Update COPYING (master...patch-1) https://github.com/bitcoin/bitcoin/pull/9838 08:58 < bitcoin-git> [bitcoin] MarcoFalke closed pull request #9838: Update COPYING (master...patch-1) https://github.com/bitcoin/bitcoin/pull/9838 09:01 < gmaxwell> https://tradeblock.com/bitcoin/tx/8d31992805518fd62daa3bdd2a5c4fd2cd3054c9b3dca1d78055e9528cff6adc > sha1 bounty redeemed. 09:03 < jouke_> :) 09:07 < BlueMatt> gmaxwell: is that a second sha1 collision, then? 09:10 < BlueMatt> gmaxwell: indeed, a second sha1 collision 09:11 < gmaxwell> smartbits is much nicer in its display: https://www.smartbit.com.au/tx/8d31992805518fd62daa3bdd2a5c4fd2cd3054c9b3dca1d78055e9528cff6adc 09:12 < BlueMatt> gmaxwell: oh, nvm, same sha1 collision 09:12 < BlueMatt> same prefix, just dropped the postfix 09:20 < Chris_Stewart_5> all of the inputs on that tx correspond to outputs that had SHA1 bounties? 09:23 < gmaxwell> Yes. 09:32 -!- niska [~niska@68.ip-149-56-14.net] has quit [Quit: Leaving] 09:39 -!- atroxes [~atroxes@unaffiliated/atroxes] has quit [Quit: bye] 09:41 -!- atroxes [~atroxes@unaffiliated/atroxes] has joined #bitcoin-core-dev 09:55 -!- niska [~niska@68.ip-149-56-14.net] has joined #bitcoin-core-dev 09:57 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Read error: Connection reset by peer] 09:59 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 10:03 < warren> wumpus: I'd like behavior where during -reindex it can handle any existing blk*.dat files to be read-only, do not truncate during scan, and create new after it scanned whatever already exists there. That way I can hardlink shared files between multiple full archival node instances on the same machine. 10:04 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/a13a417cdcfd...692c9eddba67 10:04 < bitcoin-git> bitcoin/master 9829c54 Cory Fields: build: force a c++ standard to be specified... 10:04 < bitcoin-git> bitcoin/master 692c9ed Wladimir J. van der Laan: Merge #9831: build: force a c++ standard to be specified... 10:04 < bitcoin-git> [bitcoin] laanwj closed pull request #9831: build: force a c++ standard to be specified (master...no-default-std) https://github.com/bitcoin/bitcoin/pull/9831 10:04 < wumpus> warren: heh yes, hardlinking block files works as long as you don't reindex, apparently 10:05 < wumpus> warren: I tend to hardlink both the block/utxo database files and the block files, so have never noticed that 10:05 < wumpus> warren: though: truncating the files to not contain any zero space at the end shouldn't affect any other users 10:05 < warren> which are the "block/utxo database files" 10:06 < wumpus> the ldb files: https://gist.github.com/laanwj/3c4614a23e072cbb3d39090da1834a68 10:07 < warren> wumpus: currently during *any* startup it fails if it cannot open a blk*.dat file writable, which is unnecessary. If they can be read it should be adequate as long as new blk.dat files can be written after the read-only files. 10:07 < bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/99fd85cb44fe9983c64cfa376299e67b18568304 10:07 < bitcoin-git> bitcoin/0.14 99fd85c Cory Fields: build: force a c++ standard to be specified... 10:07 < wumpus> I agree, but that's not the case right now 10:07 < gmaxwell> wumpus: sounds like a trivial change to make it not open them writable. 10:08 < gmaxwell> er warren 10:08 < sipa> we could mark files in the block index as unwritable as well 10:08 < sipa> and then skip them when looking for files to append new blocks to 10:09 < wumpus> I remember luke-jr has an issue open about this https://github.com/bitcoin/bitcoin/issues/2039 10:09 < warren> I know. currently it goes through a generic abstraction to open files 10:09 -!- lclc [~lclc@unaffiliated/lclc] has quit [Ping timeout: 268 seconds] 10:10 < wumpus> sipa: the problem in this case is not appending, but truncating. But yeah that'd help too 10:10 -!- rndguy [4e2b2923@gateway/web/freenode/ip.78.43.41.35] has joined #bitcoin-core-dev 10:11 < wumpus> I like how pruning at least works with hardlinked block files, by pruning a block file at a time instead of messing with the contents 10:11 < warren> fopen failing on read-only and truncation are two related problems, following by marking which files should be not writable. Anything else? 10:11 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has quit [Ping timeout: 260 seconds] 10:12 < warren> where should that read-only marking be written? 10:12 < wumpus> in the block index db 10:19 < bitcoin-git> [bitcoin] ryanofsky opened pull request #9839: [qa] Make import-rescan.py watchonly check reliable (master...pr/travis-rescan) https://github.com/bitcoin/bitcoin/pull/9839 10:29 -!- mturquette [sid66043@gateway/web/irccloud.com/x-lhpqxmzfmunelasz] has quit [Ping timeout: 240 seconds] 10:31 -!- mturquette [sid66043@gateway/web/irccloud.com/x-mhwhvqovdklpxfrv] has joined #bitcoin-core-dev 10:32 -!- Sosumi [~Leon@bl10-113-190.dsl.telepac.pt] has joined #bitcoin-core-dev 10:43 -!- jnewbery_ [~john@static-100-38-11-146.nycmny.fios.verizon.net] has quit [Read error: Connection reset by peer] 10:43 -!- jnewbery [~Thunderbi@static-100-38-11-146.nycmny.fios.verizon.net] has quit [Read error: Connection reset by peer] 10:49 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 10:49 < wumpus> almost time to tag rc2? 10:49 < wumpus> well I guess it can wait until the meeting :) 10:49 < bitcoin-git> [bitcoin] ryanofsky opened pull request #9840: Update sendfrom RPC help to correct coin selection misconception (master...pr/fromacct) https://github.com/bitcoin/bitcoin/pull/9840 10:49 < BlueMatt> heh 10:49 < BlueMatt> wumpus: i think so 10:52 < BlueMatt> cfields: where are we on #9787 10:52 < gribble> https://github.com/bitcoin/bitcoin/issues/9787 | release: add a few performance-related notes by theuni · Pull Request #9787 · bitcoin/bitcoin · GitHub 10:52 < BlueMatt> oh, wait, missed you updated part of it 10:52 < sdaftuar> #9814 looks potentially concerning? unclear if its reproducible i guess 10:52 < gribble> https://github.com/bitcoin/bitcoin/issues/9814 | 0.14rc1 coredump in QScreen::handle() · Issue #9814 · bitcoin/bitcoin · GitHub 10:53 < BlueMatt> oh, did we not fix that yet? ugh, yea, probably need a fix for that before rc2 10:53 < wumpus> sdaftuar: I don't know. haven't heard any reports about that from anyone else 10:53 < wumpus> also no one seems to be able to reproduce it 10:54 < wumpus> so no, I don't think it should block rc2 10:54 < cfields> BlueMatt: yea, i need some clarification from you. Also, I assume there are some more obvious user-facing perf improvements that should be added, i just can't come up with any 10:54 < BlueMatt> wumpus: hmmm....I mean I'd tend to trust dooglus to not have some obviously-broken shit 10:54 < cfields> BlueMatt: we could do a quick call for additions in the meeting 10:54 < BlueMatt> wumpus: but, ok 10:54 < BlueMatt> wumpus: (I dont see much of a different option) 10:55 < wumpus> BlueMatt: also it seems like a qt issue 10:55 < BlueMatt> yes 10:55 < wumpus> and seems it's a custom built binary, not our static one from bitcoin.org 10:56 < BlueMatt> well i hope we support custom binaries with non-static qt :p 10:56 < BlueMatt> cfields: i think just the spelling issue sdaftuar mentioned and we can get it in for rc2? 10:56 < wumpus> but we can't support all broken configurations out there 10:56 < cfields> BlueMatt: sure, if you're good with the rest 10:56 < wumpus> well at least I can't. Maybe you want to take that up, fine with me 10:56 < wumpus> :) 10:57 -!- lclc [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 10:57 < BlueMatt> wumpus: heh, indeed, if no one can figure out the issue then it shouldnt block rc2, I just want to make sure someone who actually knows qt has given it a super good look-over, because i certainliy cant help there :/ 10:57 < cfields> wumpus: did you see my comment the other day about that smelling like an old touch/wacom bug? Not sure if you dug that up, but I'll have a look now if not 10:58 < wumpus> cfields: I vaguely remember that one, yes. 10:58 < gmaxwell> if it's an issue in an older QT we could disable building with versions that old. 10:58 < wumpus> cfields: it needed a qt patch 10:58 < cfields> wumpus: right. My thought was that because the crash comes from older system libs, that bug may be present 10:58 < cfields> looking it up to see if it could be the same thing 10:59 < wumpus> looks like even the filer cannot reproduce it 10:59 < wumpus> "Edit: I've tried repeating the same steps again and wasn't able to get the crash to happen again." 11:00 < wumpus> #startmeeting 11:00 < wumpus> #meetingstart 11:00 < lightningbot> Meeting started Thu Feb 23 19:00:31 2017 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot. 11:00 < lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic. 11:01 < gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier 11:01 < jonasschnelli> hi 11:01 < wumpus> cfields: we could ask if dooglus is using a tablet/touch screen, or was using it at that time. 11:01 < cfields> wumpus: oh, haha, i didn't notice it was the same reporter 11:01 < kanzure> hi. 11:02 < sipa> present 11:02 < petertodd> hi 11:02 < jonasschnelli> wumpus: did you reproduce 9814 with the same setup? Ubuntu and Qt 5.3? 11:02 < cfields> oh, nm 11:02 < petertodd> so quick note re: git and SHA1 collisions: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-February/013600.html 11:02 < wumpus> jonasschnelli: no, I did not reproduce it 11:02 < sipa> petertodd: i wanted to bring that up 11:03 < wumpus> #topic git and sha collisions 11:03 < petertodd> so while many people will say git isn't vulnerable if you trust a git repo's maintainers, that is *not* true as a pull-req could add a colliding file 11:03 < luke-jr> sounds like MD5 breakage 11:03 < achow101> does git and github only support sha1? 11:03 < sipa> achow101: yes 11:03 < sipa> i only read about this new sha1 attack an hour ago... how hard is it to create a collision? 11:04 < petertodd> probably only relevant with binary files, but we don't know 100% yet because details of the attack haven't been published 11:04 -!- MarcoFalke [~marco@2001:4ca0:0:f226:a063:75b:7f88:d780] has joined #bitcoin-core-dev 11:04 < luke-jr> achow101: IIRC, git is designed in such a way that changing SHA1 is very difficult 11:04 < achow101> sipa: still very hard 11:04 < sipa> i see 11:04 < gmaxwell> sipa: it's not a new attack its the one published from a coule years ago with about 2^64 complexity. There were some estimates of a cost on EC2 of about $110k. 11:04 < wumpus> 110 GPU-years or so 11:04 < gmaxwell> It's just actually been executed now. 11:04 < sipa> gmaxwell: no, it seems new research was done that simplifies it further 11:04 < gmaxwell> (at least thats my understanding) 11:04 < petertodd> gmaxwell: is it confirmed that google's efforts don't include any advances on what was already known to be possible? 11:04 < gmaxwell> oh. I missed that, okay! 11:05 < luke-jr> is the git project taking any action now? 11:05 < BlueMatt> luke-jr: doesnt look like it 11:05 < wumpus> Google and CWI, the dutch center for mathetmatics and informatics 11:05 < BlueMatt> luke-jr: (at least no movement as of this morning) 11:05 < gmaxwell> I think they already said they wouldn't before? 11:06 < sipa> i wonder how hard it is to create an overlay... that goes back in history, computes a sha256 for every tree and commit, and then include gpg signatures on those? 11:06 < jonasschnelli> sipa: great idea 11:06 < petertodd> sipa: I have code for something very similar to that in OpenTimestamps actually, and people have written tools to do that as well other than opentimestamps 11:06 < btcdrak> The exploit has a fancy name as usual http://shattered.io/ 11:06 -!- jtimon [~quassel@garage-jp.static.monkeybrains.net] has joined #bitcoin-core-dev 11:06 < gmaxwell> http://marc.info/?l=git&m=115678778717621&w=2 "The only _dangerous_ kind of collision is the inadvertent kind, 11:06 < gmaxwell> but that's obviously also the very very unlikely kind." 11:06 < BlueMatt> sipa: I was looking this morning to see if you could reasonably patch git to do so directly, looks hard.....still, we can update our dev scripts to do a sha256sum of all committed files and sign that as well 11:07 < sipa> BlueMatt: signing just the trees i guess is an easy first step 11:07 < btcdrak> according to the site "How is GIT affected? 11:07 < btcdrak> GIT strongly relies on SHA-1 for the identification and integrity checking of all file objects and commits. It is essentially possible to create two GIT repositories with the same head commit hash and different contents, say a benign source code and a backdoored one. An attacker could potentially selectively serve either repository to targeted users. This 11:07 < btcdrak> will require attackers to compute their own collision." 11:07 < wumpus> but the tree is a sha1 hash too, isn't it? 11:07 < BlueMatt> gmaxwell: yup, great, linus and associated kernel folks continue to willfully ignore that security matters even the slightest bit 11:07 < gmaxwell> what does the signed commit stuff sign? if it signs the commit message, then we can include a sha256 in it and have the verify check that too. 11:07 < BlueMatt> wumpus: yes, it would have to be a separate thing 11:07 < wumpus> if you don't trust SHA1 anymore, that rabbit hole goes deep 11:07 * luke-jr votes banning binary files in any case :p 11:07 < sdaftuar> is it worth periodically running https://github.com/cr-marcstevens/sha1collisiondetection on commits in our repo? 11:07 < BlueMatt> wumpus: eg the merge scripts could include a hash of sha256sum * in the commit message 11:08 < petertodd> gmaxwell: the signed commit stuff signs a SHA1 hash, but it's easy to extend that with a gnupg wrapper; I could take the code from OpenTimestamps and do that 11:08 < BlueMatt> sdaftuar: I have a patch for git to use that as the hash function 11:08 < BlueMatt> and abort() if it detects a potentially-bad hash 11:08 < sipa> BlueMatt: sounds like a great idea 11:08 < kanzure> off-topic, but i wonder what git could change to make upgrading backwards-compatible in the future. for now i think it must be an incompatible upgrade to switch to another hash function? 11:08 < sipa> (including the sha256sum * in the merge commit message) 11:08 < wumpus> BlueMatt: if bad hashes are rare, that makes sense 11:08 < petertodd> gmaxwell: the one thing OpenTimestamps doesn't already do is recalculate hashes of *prior* commits with SHA256, but that'd be easy to add 11:09 < kanzure> i would also like the gh-meta repository maintainer to consider timestamping with opentimestamps at some point, i dunno who he is and i haven't pestered him yet 11:09 < BlueMatt> wumpus: the authors of that hash code claim 2**-90 11:09 < kanzure> er, the bitcoin-gh-meta repository person 11:09 < BlueMatt> (for random hits) 11:09 < luke-jr> detecting it after the fact seems like it would still be irrepairable? 11:10 < wumpus> kanzure: iwilcox on IRC 11:10 < petertodd> kanzure: git could easily have git commits simultaneously generate and sign two different hash functions; quite feasible to implement. I'll bet you I could pull that off in a week or two of work. 11:10 < kanzure> oh good, he is easily pesterable. 11:10 < kanzure> petertodd: yea but needs to be backwards compatible; and the bloat from two commits does not sound ideal. are they considering that btw? 11:10 < wumpus> BlueMatt: if the chance of an inadvertent match is that low, in that case, abort() /reject is a great solution 11:11 < luke-jr> kanzure: I'd guess you simply list the parent commit hashes twice? 11:11 < petertodd> kanzure: no bloat involved - the data can be shared across both commits 11:12 < kanzure> ah okay. well, i hadn't heard that proposal today, someone should check if git upstream is thinking about that. 11:12 < gmaxwell> probably said enough on this for now. 11:12 < btcdrak> There's a conversation on the git mailing list https://public-inbox.org/git/xmqqk28g92h7.fsf@gitster.mtv.corp.google.com/T/#t 11:12 < petertodd> gmaxwell: ack 11:12 < wumpus> yes, agreed 11:12 < petertodd> next topic 11:13 < wumpus> #topic help cfields with adding performance-related release notes 11:13 < wumpus> https://github.com/bitcoin/bitcoin/pull/9787 11:13 < cfields> quick call for 0.14 perf improvements on 9897 11:13 < cfields> heh, thanks :) 11:13 < cfields> err... #9787 11:13 < gribble> https://github.com/bitcoin/bitcoin/issues/9787 | release: add a few performance-related notes by theuni · Pull Request #9787 · bitcoin/bitcoin · GitHub 11:13 < jtimon> kanzure: yeah I assume changing the hash function would be a hardfork for old repositories :p 11:14 < cfields> if anyone added a big user-facing performance improvement for 0.14, please speak up 11:14 < gmaxwell> jtimon: please don't use bitcoin terms for other things especially non-consensus systems. The classic words "backwards incompatible" are fine. :P 11:15 < jtimon> gmaxwell: yep, sorry, was a bad joke 11:15 < wumpus> jtimon: dependso n how close the new hash function is to SHA-1. If it gets the same output 1-2**90 of the time, most repositories won't be affected 11:15 < gmaxwell> cfields: no measurements in the notes? 11:16 < cfields> gmaxwell: see the pr description. I've taken some measurements on the net stuff, but they're highly localized. 11:17 < gmaxwell> wish I realized no one was going to benchmark I could have started one last night. :P 11:17 < jeremyrubin> I would also note that the cuckoocache means nodes wanting to use more cores but had performance degrade should try more cores again 11:18 < sipa> right, cuckoocache has no effect at low parallellism, i think? 11:18 < morcos> sipa: no it's smarter about keepign the right signatures in the cache due to the generations and lazy eviction 11:18 < cfields> gmaxwell: i've done lots of benchmarking. But as I said, I'm not sure how to translate that into helpful figures 11:19 < gmaxwell> sipa: e.g. it will make reorgs much faster! 11:19 < cfields> well the alternative is to use a reference system/environment for each improvement 11:19 < gmaxwell> cfields: users don't care about each improvement. 11:20 < wumpus> it also doesn't need to be super precise 11:20 < gmaxwell> cfields: "Sync with least release takes N hours now, sync with new release takes Y now." would be nice. 11:20 < jonasschnelli> or syncs are rougle XYZ% faster... 11:20 -!- lclc_ [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 11:21 < jonasschnelli> use the ~ and nobody will blame you afterwards. :) 11:21 < jeremyrubin> use two ~~ to be extra aproximate 11:21 -!- lclc [~lclc@unaffiliated/lclc] has quit [Ping timeout: 255 seconds] 11:21 < wumpus> it's marketing not science :p hehe 11:21 < gmaxwell> but ~~ will just give you the same number you put in! 11:22 < jeremyrubin> The is-approximately operator is non-involutive ;) 11:22 < gmaxwell> Well people just have no general idea of the impact. Marketing would be saying that it's 2x faster rathern the 3x faster because 2x is more plausable. :P 11:22 < jeremyrubin> anyways 11:23 < cfields> ok, i'll add a vague 0.13 vs 0.14 sync time. The 0.13 will take time though, I haven't had the patience to finish one yet 11:23 < jeremyrubin> Could be cool to spin up a few different EC2 instances to compare... 11:23 < wumpus> EC is not a good comparison environment 11:23 < wumpus> sloooow i/o 11:23 < sipa> i'm syncing on a RPi3 11:24 < sipa> sloooow 11:24 < wumpus> what storage? 11:24 < luke-jr> lol 11:24 < sipa> microsd card 11:24 < jonasschnelli> uh 11:24 < cfields> i used EC for my sync benching because i figured it represented a very typical use-case 11:24 < wumpus> that's the cause of the slowness, likely 11:24 < jeremyrubin> Actually I like that. We should publish the worst system that can sync :p 11:24 < sipa> wumpus: absolutely 11:24 * luke-jr ponders trying on his USB Armory again 11:25 < jtimon> other topics? 11:25 < wumpus> sipa: if it's really CPU bound, don't forget to use the experimental ARM assembly secp256k1 :) 11:25 < wumpus> sipa: right, as I expected 11:25 < cfields> For reference, 0.14 sync took roughly 3 hours on ec2 w/ 4 cpu cores 11:25 < sipa> wumpus: i enabled it, but it's not nearly CPU bound 11:25 < achow101> topic: rc2 status? 11:26 < wumpus> #topic rc2 status 11:26 < wumpus> I think it should be ready to tag 11:26 < wumpus> well, need to update the translations and release notes to include changes since rc1, but I don't think there's anything that still needs to be solved 11:27 < achow101> there are still open prs in the milestone though 11:27 < achow101> (well 1 that actually does something) 11:27 < wumpus> achow101: one is release notes - that can go in any time before final 11:27 < wumpus> #9829 would be nice to get in, but it's breaking travis 11:28 < gribble> https://github.com/bitcoin/bitcoin/issues/9829 | Fix importmulti returning rescan errors for wrong keys by ryanofsky · Pull Request #9829 · bitcoin/bitcoin · GitHub 11:28 < jtimon> ryanofsky: ping 11:29 < ryanofsky> should i do something? bump travis? 11:29 < wumpus> (I've already tried to retrigger it so it's not one of today's intermittent problems) 11:29 < wumpus> ryanofsky: I don't think that helps 11:30 < gmaxwell> Does it pass locally? 11:30 < ryanofsky> yeah, the errors are the same "__pthread_mutex_lock: Assertion `mutex->__data.__owner == 0' failed" things i've seen on other prs 11:31 < wumpus> ryanofsky: oh, https://github.com/bitcoin/bitcoin/issues/9825 11:31 < ryanofsky> i had to bump one of my prs last week 5-6 times to make travis pass 11:31 < wumpus> ryanofsky: okay, will respin 11:32 < bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/847e3753a6d6a6ab4dd081e2945cb200faf730d4 11:32 < bitcoin-git> bitcoin/0.14 847e375 Wladimir J. van der Laan: qt: pre-rc2 translations update 11:33 < morcos> Can we figure out when these travis errors started? Do we get them on personal travis instances? Can we bisect? Seems likely somethign changed on our end right? 11:33 < wumpus> #topic travis issues 11:33 < gmaxwell> do we have any idea whats causing that? I assume no one has hit it locally? I left test bitcoin running in a loop when I first saw it on the off chance it was an ASLR mediated thing that was only hitting travis sometimes. 11:33 < gmaxwell> (no results, of course) 11:33 < jonasschnelli> We recently added a missing ecc_start to bitcoin-tx... but I can't see any relation 11:33 < gmaxwell> jonasschnelli: I don't even see why you mention that? 11:33 < wumpus> I tried to reproduce #9825 for hours on a trusty vm, with five threads for a few hours... but no dice 11:33 < gribble> https://github.com/bitcoin/bitcoin/issues/9825 | Intermittent FAIL: test/test_bitcoin in Travis · Issue #9825 · bitcoin/bitcoin · GitHub 11:34 < wumpus> to bitcoin-tx? yea, that won't make a difference 11:34 < jonasschnelli> gmaxwell: becuase of that "test_bitcoin: key.cpp:300: void ECC_Start(): Assertion `secp256k1_context_sign == __null' failed."? 11:34 < gmaxwell> It looks like test_bitcoin fails before it even really starts. So global constructors or somehting in the C libraries. 11:34 < wumpus> jonasschnelli: that's only a effect of the failure 11:34 < jeremyrubin> I noticed that one of the builds seems to have some additional bounds checks enabled -- might be nice to enable those across travis tests? Hopefully no code relies on bounds checks/not 11:34 < wumpus> jonasschnelli: *everything* after the initial failure fails 11:34 < wumpus> jonasschnelli: secp256k1 is innocent 11:35 < jonasschnelli> ah.. I see. 11:35 < jtimon> perhaps we're forgetting some EEC_stop() somewhere? 11:35 < gmaxwell> no. geesh 11:35 < wumpus> no, I'm fairly sure it doesn't have to do with secp256k1 11:35 < wumpus> it's something done with mutexes before mutexes are initialized 11:36 < jtimon> ok, I really have no idea what's happening 11:36 < wumpus> looks like some kind of race condition, but I have no idea where or what 11:36 < wumpus> if it happens it *always* happens in test_bitcoin, never in bitcoind, bitcoin-cli, bitcoin-tx 11:37 < gmaxwell> I wonder if we could make our travis build script rerun any failing case under gdb so it would print a backtrace? 11:37 < BlueMatt> gmaxwell: probably using coredumps? 11:37 < wumpus> if you rerun it it will probably pass 11:37 < jtimon> perhaps some of the globals initialized in test_bitcoin.cpp ? 11:37 < gmaxwell> or that. 11:37 < jeremyrubin> in theory boost test runner can start gdb 11:37 < jtimon> just thinking out loud again... 11:37 < jonasschnelli> Could it be related to the boost test framework? 11:37 < BlueMatt> eg if it crashes coredump and gdb print all backtraces 11:38 -!- jnewbery [~Thunderbi@static-100-38-11-146.nycmny.fios.verizon.net] has joined #bitcoin-core-dev 11:38 < wumpus> but yes, getting a core file would be useful 11:38 < jeremyrubin> boost test runner can start gdb. I wonder if it reads .gdbinit 11:38 < gmaxwell> BlueMatt: thats probably the right way to go there. 11:38 < wumpus> although you need the executable too, to debug it 11:38 < gmaxwell> jeremyrubin: yes, but if it crashes its probably not in a good state to do so. :P 11:38 < wumpus> and uploading from travis :( 11:38 < BlueMatt> wumpus: i mean we can at least print stack traces 11:38 < gmaxwell> wumpus: just detect the crash in the script and run gdb. 11:38 < gmaxwell> and print the backtraces. 11:39 < wumpus> ok, yes, backtrace would help 11:40 -!- lclc_ [~lclc@unaffiliated/lclc] has quit [Read error: Connection reset by peer] 11:40 < gmaxwell> do we know when the first of these errors was? 11:40 < wumpus> jtimon: yes, it sounds ilke a global initialisation order fiasco 11:40 < jeremyrubin> could be nice to detect the failing case, and then re-run it under say, valgrind 11:40 -!- lclc [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 11:40 < wumpus> no, I don't know. I started getting annoyed by them today, but it's been going on yesterday at least as well 11:41 < wumpus> bisecting this with travis would take *a lot* of patience 11:41 < gmaxwell> that kind of suggests that it's a change on travis' end, no? we haven't had any changes on 0.14 in the past two days that could have caused this? 11:42 < wumpus> I don't think so either. 11:42 < wumpus> the big locking changes are longer ago 11:42 < wumpus> and it didn't start after that 11:42 < jeremyrubin> topic suggest -- code reorganizing (renaming rpc tests, etc) 11:43 < gmaxwell> does something in the travis log report what hardware it ran on? 11:43 < wumpus> still, the question is whether it is a change at travis's end that exposes something in our code 11:43 < gmaxwell> e.g. so we could correlate failures with a specific host? 11:43 < wumpus> or something that is completely broken on their end 11:43 < wumpus> gmaxwell: I'm afraid not. Wasn't able to find it 11:43 -!- jtimon [~quassel@garage-jp.static.monkeybrains.net] has quit [Ping timeout: 268 seconds] 11:43 < wumpus> I don't think they give access to that information 11:43 < jnewbery> wumpus: why is uploading from travis :( ? is it something you've tried before? 11:44 < wumpus> jnewbery: it's a pit of snakes, security-wise 11:44 < ryanofsky> this is older than 2 days, i first noticed these last friday on 9773: https://github.com/bitcoin/bitcoin/pull/9773#issuecomment-280684263 11:44 -!- jtimon [~quassel@garage-jp.static.monkeybrains.net] has joined #bitcoin-core-dev 11:44 < wumpus> jnewbery: we could temporary have it submit files somewhere to debug an issue, I guess 11:45 < jnewbery> It can be configured it to upload artifacts to S3: https://docs.travis-ci.com/user/uploading-artifacts/ 11:45 < gmaxwell> jeremyrubin: the issue there is just in terms of provisioning travis with secrets. 11:45 < kanzure> no private environment variables? 11:45 < gmaxwell> er darnit, too many js in here. 11:45 < wumpus> private environment variables don't work for pull requests, the test script could be replaced with anything 11:45 < jeremyrubin> gmaxwell: i was wat-ing :) 11:46 < gmaxwell> kanzure: PR's can steal them. 11:46 < jeremyrubin> Can you get a shell to travis 11:46 < wumpus> including echo $SECRET_CODE or wget https://host/secretcode?$SECRETCODE 11:46 < kanzure> cool. hm. 11:46 < jeremyrubin> (probably against TOS) 11:46 < MarcoFalke> I think you can specify secrets that are valid on branches only, but I might be wrong 11:46 < gmaxwell> jeremyrubin: A assume just open a PR that connects back to you. :P 11:47 < wumpus> lol a reverse shell in a PR 11:47 < BlueMatt> I assume you can, but, yea ToS (not to mention monopolizing their service....) 11:47 < gmaxwell> wait, you're all not mining bitcoins there already? 11:47 < jeremyrubin> I heard someone did that recently right? 11:47 -!- harrymm [~wayne@104.222.140.93] has quit [Remote host closed the connection] 11:48 < gmaxwell> in any case, we've wasted most of a perfectly good hour on this. :P 11:48 < wumpus> I like the idea of uploading the executable and core files more. They could be pushed to a private server, no need to openly host them, that meanst here's less reason for people up to no good to inject anything 11:48 < wumpus> the biggest security worry was with hosting the built binaries 11:49 < jtimon> topic code reorganizing? 11:49 < wumpus> bleh 11:49 < wumpus> okay :p 11:49 < wumpus> #topic code reorganizing 11:49 < jnewbery> suggested (marginally related) topic: code coverage and benchmark tracking 11:49 < gmaxwell> we should ask travis for a feature that sets an enviroment variable to H(project secret || commit hash) ... and then we could have something whitelist uploads from specific PRs (e.g. by known people) 11:49 < jtimon> it's 10 mins, so no time for a big topic I think 11:50 < jeremyrubin> I have a POC branch which moves most of the pure data structures to a datastructures dir. 11:50 < gmaxwell> Rename rpctests to tests rename test_bitcoin to useless_thing_that_crashes_in_travis. Done. 11:50 < wumpus> gmaxwell: yep 11:50 < jtimon> jeremyrubin: you mean including things like CTransaction and CBlockHeader? 11:50 < luke-jr> jeremyrubin: that doesn't sound like the right direction 11:50 < jeremyrubin> no 11:50 < jeremyrubin> non-bitcoin specific ones 11:50 < gmaxwell> jeremyrubin: really? datastructures? shall we have a file called functions.cpp and a file called variables.cpp too? :P 11:50 < jeremyrubin> E.g., prevector 11:50 < luke-jr> XD 11:51 -!- echonaut1 [~echonaut@46.101.192.134] has quit [Remote host closed the connection] 11:51 < luke-jr> jeremyrubin: oh, okay, that sounds less bad 11:51 < gmaxwell> oh you mean things like effective language extensions. 11:51 < jtimon> I would prefer to move prevector to the consensus dir 11:51 -!- echonaut [~echonaut@46.101.192.134] has joined #bitcoin-core-dev 11:51 < wumpus> yea, prevector is consensus critical 11:51 < gmaxwell> suggests that the name still isn't good in that luke and I misunderstood it immediately. :P 11:51 < jeremyrubin> so is secp256k1 ;) 11:51 < sipa> so is libc 11:51 -!- echonaut [~echonaut@46.101.192.134] has quit [Remote host closed the connection] 11:51 < jtimon> like in https://github.com/bitcoin/bitcoin/pull/8328 11:51 -!- harrymm [~wayne@104.222.140.110] has joined #bitcoin-core-dev 11:51 < wumpus> yes, we're not renaming secp256k1 are we 11:52 < luke-jr> move prevector to boost 11:52 * luke-jr hides 11:52 < gmaxwell> Okay, so step one we move libc under consensus/ ... 11:52 < jeremyrubin> Anyways, I think it would be nice to move things like that, which could later be made upstream repos 11:52 < wumpus> I don't think this is going anywhere, everyone has a different opinoin on what file to put where 11:52 < gmaxwell> lpad.js. 11:52 -!- echonaut [~echonaut@46.101.192.134] has joined #bitcoin-core-dev 11:53 < luke-jr> Does prevector have any usage outside consensus? 11:53 < jeremyrubin> I think the reason I'd want that is it makes it easier to have more exhaustive testing and also allows other users to more easily use such datastructures 11:53 < sipa> luke-jr: it probably should :) 11:53 < wumpus> do we have any unique data structures that anyone else inthe world would want to use? 11:53 < jtimon> jeremyrubin: yeah I would like libconsensus to be an "upstream repo" like libsecp256k1, but we would need to complete it first 11:53 < gmaxwell> jeremyrubin: I don't think any of us care to maintain things like prevector for other usage. Making a good library takes a tremendous amount of work. 11:54 < wumpus> I don't think a bitcoin-datastructures library makes sense 11:54 < wumpus> right 11:54 < wumpus> if we offer a library it should be something useful to bitcoin users 11:54 < gmaxwell> Obviously if the author of something like that wants to create a library for other usage, thats fine! but not a project priority. 11:54 < wumpus> like a wallet library, or extend libconsensus, ... 11:55 < wumpus> sure, you can do whatever you want with the files in the repository, if you need it in your project you can make a library out of it for your own use, or just copy the file, etc 11:55 < gmaxwell> from a code org perspective I don't see a problem with moving things like prevector, limitedmap into a utility function directory.. With just the understanding that many of those are used in consensus code too. 11:55 < wumpus> not everything needs to be maintained as a library 11:55 < jtimon> but since each dev seems to have a different idea of what a "complete libconsensus" should look like... 11:56 < wumpus> gmaxwell: me neither 11:56 < kallewoof> Compartmentalizing bitcoin into separate modules seemed like a plan but maybe not shared by everyone. If it is a shared plan restructuring file places seems important. 11:56 < luke-jr> I don't mind it, but IMO more important to work toward more useful library splitting 11:56 < sipa> gmaxwell: agree, some things are generic enough that they can be seen as extensions of the standard libraries 11:56 < gmaxwell> sipa: e.g. someday libstdc++ could get something that generalizes prevector, if it did, we'd drop prevector and use that. 11:56 < sipa> c++23 11:57 < jonasschnelli> heh 11:57 < gmaxwell> (well STL technically, I guess, point remains) 11:57 < gmaxwell> sipa: C++23 will just integrate libconsensus of course. template cryprocurrency. 11:57 < wumpus> hehe 11:57 < jeremyrubin> Well that's my point kinda 11:57 < jnewbery> suggested topic: code coverage and benchmark tracking 11:57 < jeremyrubin> the consensus things that aren't overly bitcoin-specific 11:57 < gmaxwell> Except I was making it as a joke. :P 11:58 < wumpus> jnewbery: next week 11:58 < jeremyrubin> Facebook's folly lib is kinda like that 11:58 < wumpus> the meeting is about to close 11:58 < sipa> anyone have any last words? 11:58 < gmaxwell> jnewbery: we can talk about about it after the meeting and discuss further next week. :) 11:58 < jnewbery> gmaxwell: sounds good 11:58 < wumpus> #endmeeting 11:58 < lightningbot> Meeting ended Thu Feb 23 19:58:51 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) 11:58 < lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-23-19.00.html 11:58 < lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-23-19.00.txt 11:58 < lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-23-19.00.log.html 11:58 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt] 11:59 < BlueMatt> ok, #topic code coverage and benchmark tracking 11:59 < jeremyrubin> I wrote a benchdiff tool a while back 11:59 < MarcoFalke> ACK on benchmark tracking 11:59 < gmaxwell> jeremyrubin: the history of corporate soup libraries isn't really good, a lot of them just become abandoneware... and we have some many things to worry about that cooking up more libraries is really ... not the project's priority even if it would be good in an abstract sense. 11:59 -!- lclc_ [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 11:59 < gmaxwell> In some cases like libsecp256k1 it can be strategic to build something for general use. But thats case by case. 12:00 < wumpus> folly is unfocused 12:00 < jnewbery> is anyone doing coverage? Anyone tried using coveralls or codecov or ... for tracking code coverage? 12:00 < jeremyrubin> Sure, the point was not really about having it be a separate library 12:00 < jeremyrubin> but for increasing testing 12:00 < wumpus> 'random grabbag of data structures' libraries don't tend to be used a lot 12:00 < kanzure> i thought it was already known that code coverage is low 12:00 < gmaxwell> jnewbery: we have configure script setup for lcov. It works. 12:00 < wumpus> even if the data structures in it are genius 12:01 < luke-jr> wumpus: we'd use it 12:01 < wumpus> jeremyrubin: the problem is that you also get lots of issues for unrelated uses 12:01 < jnewbery> gmaxwell: it works is a good start. Which way does code coverage go over time? 12:01 < gmaxwell> And our expirence from libsecp256k1 suggests that there has been relatively little project value from making it generally usable. :( basically many things that really should use it simply do not. And what does use it almost never reports interesting feedback or provides useful testing. :( 12:01 < luke-jr> wumpus: if nothing else, libraries can reduce the binary download size by not duplicating the code between our 4 or 5 programs 12:01 < kanzure> jnewbery: there's a lot of missing unit tests. they don't exist; go write them. 12:01 < wumpus> jeremyrubin: right now the code only supports our use case, it doens't have to balloon to everyone's features and favorite other things 12:01 < gmaxwell> jnewbery: it's been going up. well at least the 'rpc tests' improvements increased it considerably. 12:02 < luke-jr> gmaxwell: some things which really should use it seem to? 12:02 < cfields> kanzure: he has been :) 12:02 -!- lclc [~lclc@unaffiliated/lclc] has quit [Ping timeout: 268 seconds] 12:02 < kanzure> cfields: shows what i know 12:02 < jnewbery> cfields: :) 12:02 < wumpus> luke-jr:sure, internal libraries for organization without exposing them to the outside or making them official APIs 12:02 < wumpus> luke-jr: we already have those 12:03 < cfields> kanzure: you're right to point out that we're missing lots of coverage, ofc. 12:03 < gmaxwell> luke-jr: e.g. the horrific python stuff is used much more commonly.. and various embedded devices reinvented the wheel creating big sidechannel vulnerablities along the way. 12:03 < kanzure> cfields: prolly not a good excuse to not do constant coverage reports, hehe 12:03 < wumpus> luke-jr: turn them to dlls and you could reduce download size a bit 12:03 < cfields> heh 12:03 < jnewbery> kanzure: I'd like to do more. Doing coverage once is good. Tracking it across releases is better 12:03 < luke-jr> gmaxwell: eg JoinMarket seems to use it? 12:03 < wumpus> luke-jr: not by much I think as most of the duplication will tend to get compressed out 12:03 < luke-jr> or was it Electrum 12:03 < jnewbery> I'd also like to have historic data about benchmarks 12:03 < cfields> jnewbery: i'd be very curious to see if something like coveralls could be hooked up easily-ish 12:03 -!- jtimon [~quassel@garage-jp.static.monkeybrains.net] has quit [Ping timeout: 255 seconds] 12:04 < gmaxwell> jnewbery: ideally we'd be at a point where travis could fail builds that drop coverage too much, but the existing tests do not have high enough coverage reliably to make the workthwhile. 12:04 < luke-jr> wumpus: IMO libconsensus will be more significant 12:04 < jeremyrubin> Benchdiff tool is here: https://github.com/JeremyRubin/bitcoin/tree/benchdiff 12:04 < gmaxwell> luke-jr: yes, after I nagged them to move off some crazily vulnerable python code that was found on the internet. 12:04 < luke-jr> XD 12:04 < gmaxwell> luke-jr: e.g. code they were using would accept the signature 0,0 as valid for any pubkey+message. 12:04 < jnewbery> gmaxwell: I'm not aiming for ideal just yet. Just a general sense of which direction we're moving in 12:05 < wumpus> comparing benchmarks would need a machine dedicated to just that 12:05 < jnewbery> cfields: do you know if anyone has tried hooking up coveralls? 12:05 < gmaxwell> wumpus: we could run benchmarks in a cpu simulator... at glacial speed but get consistent results. :P (this is actually reasonable to do for things like in the benchmark tool) 12:05 < BlueMatt> benchmarks would have to happen on custom infrastructure, but coveralls or something would be nice 12:05 < wumpus> comparing results from different machines or from VMs would be pointless 12:06 < BlueMatt> if its easy to hook up (and doesnt require stupid shit like write permissions on your repo), i dont see why not? 12:06 < cfields> jnewbery: not that i'm aware of. I think the gcov stuff on our side needs a bit of love first, I'd be happy to help out there. 12:06 < wumpus> gmaxwell: ah yes a cycle correct simulator 12:06 < gmaxwell> I've been trying to get a simulator restup for libsecp256k1, where we can use it to verify constant-timeness of the compiled result. 12:07 < sipa> restup? 12:07 < jnewbery> ok, I'll try to hook up coveralls on my repo before next week's meeting 12:07 < gmaxwell> (I have the test working locally but haven't figured out how to automate it, will likely need to modify the simulator) 12:07 < gmaxwell> sipa: setup. 12:07 < jeremyrubin> diff 12:07 < wumpus> riscv has a cycle-accurate simulator 12:07 < jeremyrubin> crap sorry 12:08 < gmaxwell> wumpus: yes, though it doesn't simulate dram so I found it wasn't good enough for my libsecp256k1 testing. :P there is a 'cycle accurate' simulator of abstract x86_64 which doesn't exactly match any particular cpu but it close enough. 12:08 < Chris_Stewart_5> jnewbery: would love to see that on our repo 12:08 < jeremyrubin> here's the POC of the move datastructures/utils to a separate dir btw https://github.com/JeremyRubin/bitcoin/tree/move-to-libraries (may need rebase) 12:08 < gmaxwell> (I also tried the riscv one though before realizing it didn't do dram... it's crazy, it's a seperate compile target for the HDL) 12:09 < wumpus> gmaxwell: I'll shut up, seems you did much more research into this :) 12:09 < Chris_Stewart_5> jnewbery: I know isle2983 is interested in this as well -- not sure how much free time he has but might be worth collaborating 12:09 < jeremyrubin> i only did a few things just as an example -- editing lots of files isn't super fun :p 12:09 < gmaxwell> wumpus: well my research is mostly for a weird thing, trying to setup a CI test that catches the compiler undermining the constant time behavior of libsecp256k1's private key operations. 12:10 < jnewbery> Chris_Stewart_5: thanks. I'll ping him 12:10 < wumpus> in any case +1 for hooking up coveralls, seems like a small thing to do with potentially large payoff 12:10 < Chris_Stewart_5> wumpus: strongly agree. 12:11 < jeremyrubin> gmaxwell: isn't that pretty tough to do given modern pipelining? 12:12 < wumpus> gmaxwell: that HDL tool is pretty neat in itself, allowing generating customized riscv cores on demand, but yes using it to generate c++ seems a bit crazy 12:13 < wumpus> jeremyrubin: and don't forget branch prediction 12:14 < jeremyrubin> wumpus: that's a part of the pipeline 12:14 < gmaxwell> jeremyrubin: libsecp256k1 private key operations are constant time on current intel cpus. It was not that hard. The code just has to be completely free of data dependant branches, data dependant memory accesses, etc. 12:15 < gmaxwell> for example, sha256's compression function is a function that is naturally constant time, even a trivial implementation gets that right. 12:15 < gmaxwell> Just write all your code to look like that. :P 12:16 < gmaxwell> Though the compiler could still screw you over if it gets too smart, which is why I want ci support for testing it. 12:17 < bitcoin-git> [bitcoin] jnewbery opened pull request #9842: Fix RPC failure testing (continuation of #9707) (master...rpctestassert2) https://github.com/bitcoin/bitcoin/pull/9842 12:18 < MarcoFalke> BlueMatt: Is coveralls the thing that spams a comment on every pull request? 12:18 < BlueMatt> MarcoFalke: I sure hope not? 12:18 < wumpus> I don't think that's necessary 12:18 < BlueMatt> I'd hope it can integrate with github's ci status thing 12:18 < wumpus> it can integrate into github 12:19 < MarcoFalke> Sound good when this is possible. 12:19 < MarcoFalke> re: test_bitcoin failures 12:20 < MarcoFalke> It is an uninitialized read 12:20 < MarcoFalke> because g_conman is not set up in the unit tests, I assume 12:20 < MarcoFalke> *g_connman 12:23 < wumpus> MarcoFalke: mh that could be it 12:24 < gmaxwell> perhaps someday bitcoin core can have coverage like: https://people.xiph.org/~greg/libsecp256k1-coverage/src/index.html 12:25 < wumpus> could you add it? :) 12:26 < wumpus> that's pretty similar to coveralls output 12:26 < Chris_Stewart_5> MarcoFalke: I think coverall is what you are thinking of wrt comments on pull requests, see: https://github.com/bitcoin-s/bitcoin-s-core/pull/60 12:26 < BlueMatt> wumpus: I think he meant the mostly-100% coverage indicators there :p 12:26 < Chris_Stewart_5> not sure if it is configurable or not, honestly haven't played with it too much 12:26 < wumpus> BlueMatt: oh okay 12:26 < Chris_Stewart_5> but I think it is extremely helpful for beginners to figure out where to contribute too 12:26 < MarcoFalke> Chris_Stewart_5: Yes, this is terrible 12:26 < jeremyrubin> Can implement the trump policy: every line of new code must test 2 other lines of code 12:27 < BlueMatt> jeremyrubin: do we, then, need to test tests? 12:27 < gmaxwell> the code is the test of the tests. 12:27 < gmaxwell> go break the code, tests should fail. 12:27 < BlueMatt> gmaxwell: yes, we should have test harnesses for that :P 12:28 < gmaxwell> I still have not found any good tools for C. I have something that belongs in a lovecraftian horror novel... 12:28 < jeremyrubin> can we get rid of the code 12:28 < jeremyrubin> and just gen it from tests 12:29 < wumpus> mauling the source code in unspeakable ways to make tests fail 12:29 < jnewbery> gmaxwell: that report is a beautiful thing. So much green 12:29 < gmaxwell> (A script that systemtatically makes 1 byte changes to the source code, then sees if it compiles, and if it does checks if the stripped binary matches the original, and skips it.. and otherwise checks if tests fail....) 12:29 < BlueMatt> gmaxwell: yea.......... 12:29 < gmaxwell> jnewbery: it's actually better than the report shows.. that report is only with running the tests at the defaults, they get a bit more coverage when cranked up. 12:35 < jeremyrubin> Is there any coverage tools that let you see if you execute a branch with multiple values? 12:35 < jeremyrubin> E.g., doing concolic execution of some kind would be cool 12:35 < jeremyrubin> klee? 12:38 -!- jtimon [~quassel@199.116.72.155] has joined #bitcoin-core-dev 12:40 < bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.14: https://github.com/bitcoin/bitcoin/commit/3b2f7fdcaea36c8c5c93b21030d4e2327d2b6e8c 12:40 < bitcoin-git> bitcoin/0.14 3b2f7fd Wladimir J. van der Laan: doc: Add authors and changes since rc1 to release notes 12:41 < wumpus> * [new tag] v0.14.0rc2 -> v0.14.0rc2 12:42 < BlueMatt> hey-o 12:42 < achow101> yay 12:42 < sipa> wee 12:43 < achow101> I hope gitian works this time (but it probably won't) 12:43 < wumpus> well there were no changes to gitian since 0.14.0rc1, so if it didn't work for you it probably still won't 12:44 < wumpus> but you can try 12:44 < sipa> gmaxwell: it looks like the sha1 collision that google produced took 2^63 sha invocations 12:44 < cfields> damn, just pushed release notes spelling fix :( 12:45 < wumpus> cfields: there's no hurry ,release notes changes can go in until final 12:45 * luke-jr tries his gitian automation script again :p 12:45 < wumpus> and between the last rc and final 12:45 < cfields> ok 12:47 -!- lclc [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 12:47 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to 0.14: https://github.com/bitcoin/bitcoin/compare/3b2f7fdcaea3...f00429666c60 12:47 < bitcoin-git> bitcoin/0.14 95e68df Cory Fields: release: add a few performance-related notes 12:47 < cfields> I'm doing a vanilla, all defaults, public p2p sync of 0.13/0.14 nodes on ec2 with 4cores/16gb ram. I think that's a common enough use-case. If anyone would like to bench under different conditions, please go for it 12:47 < bitcoin-git> bitcoin/0.14 f004296 Wladimir J. van der Laan: Merge #9787: release: add a few performance-related notes... 12:49 -!- lclc_ [~lclc@unaffiliated/lclc] has quit [Ping timeout: 260 seconds] 12:50 < wumpus> cfields: at least it's kind of standardized 12:50 < achow101> cfields: I think 8gb ram is a more likely scenario 12:51 < luke-jr> note 32-bit builds can't use much RAM, but I don't think we care so much about that 12:54 < wumpus> let's not start arguing his choice of benchmark machine 12:54 < wumpus> if you want to benchmark on something else, go ahead 12:58 -!- lclc [~lclc@unaffiliated/lclc] has quit [Read error: Connection reset by peer] 12:59 -!- lclc [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 13:06 -!- pavel_ [~paveljani@79.98.72.176] has joined #bitcoin-core-dev 13:06 -!- pavel_ [~paveljani@79.98.72.176] has quit [Remote host closed the connection] 13:08 -!- lclc [~lclc@unaffiliated/lclc] has quit [Ping timeout: 255 seconds] 13:09 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Ping timeout: 260 seconds] 13:25 < achow101> woohoo! First again! (but there's probably an issue with osx) 13:47 -!- jtimon [~quassel@199.116.72.155] has quit [Ping timeout: 255 seconds] 13:47 -!- jtimon [~quassel@garage-jp.static.monkeybrains.net] has joined #bitcoin-core-dev 13:49 < jtimon> re style, I wish I could uncomment ;; (add-hook 'before-save-hook 'delete-trailing-whitespace) 14:05 -!- Madars [~null@unaffiliated/madars] has quit [Quit: Leaving.] 14:13 -!- Madars [~null@unaffiliated/madars] has joined #bitcoin-core-dev 14:20 -!- rndguy [4e2b2923@gateway/web/freenode/ip.78.43.41.35] has quit [Quit: Page closed] 14:22 < bitcoin-git> [bitcoin] jnewbery opened pull request #9843: Fix segwit getblocktemplate test (master...fixsegwitgetblocktemplate) https://github.com/bitcoin/bitcoin/pull/9843 14:26 < gmaxwell> I think we should not have that behavior. We should not make mining catch fire when segwit activates. 14:33 < luke-jr> gmaxwell: in other words, we should support non-segwit mining after it activates? 14:33 < luke-jr> (that'd be somewhat non-trivial FWIW) 14:34 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)] 14:34 -!- marcoagner [~marcoagne@191.32.199.160] has joined #bitcoin-core-dev 14:35 < luke-jr> basically we'd need to teach CreateNewBlock to filter out segwit txs, and then do some magic with GBT caching 14:38 < luke-jr> IIRC last time it was discussed, the conclusion was to not support it; I forget where that convo was tho 14:44 -!- MarcoFalke [~marco@2001:4ca0:0:f226:a063:75b:7f88:d780] has quit [Quit: MarcoFalke] 14:44 -!- MarcoFalke [~marco@host244-2.natpool.mwn.de] has joined #bitcoin-core-dev 14:47 -!- jtimon [~quassel@garage-jp.static.monkeybrains.net] has quit [Ping timeout: 255 seconds] 14:50 -!- jnewbery [~Thunderbi@static-100-38-11-146.nycmny.fios.verizon.net] has quit [Ping timeout: 240 seconds] 14:50 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Remote host closed the connection] 14:54 -!- jtimon [~quassel@garage-jp.static.monkeybrains.net] has joined #bitcoin-core-dev 14:57 < cfields> achow101: no match :( 15:02 < achow101> cfields: on osx? again? 15:02 < cfields> achow101: yep :( 15:02 < cfields> still building the rest 15:03 < achow101> *sigh* it's still a problem 15:05 < achow101> I'll see if it does the alternating thing like last time 15:11 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 15:13 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has joined #bitcoin-core-dev 15:15 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds] 15:18 < gmaxwell> luke-jr: yes, we should. 15:18 < gmaxwell> luke-jr: the only harm is you won't include segwit transactions... and we can whine at miners later to fix themselves. 15:18 < gmaxwell> better to have some segwit txn rather than none. 15:19 < gmaxwell> We should also set the version bit, even if you're not sending the flag. 15:19 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 15:20 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Remote host closed the connection] 15:20 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 15:25 < bitcoin-git> [bitcoin] jtimon opened pull request #9845: RPC: cleanups in rpc/server (master...0.15-extern-rpc-server) https://github.com/bitcoin/bitcoin/pull/9845 15:26 < jtimon> jonasschnelli: am I missing something about the wallet in ^^, I hope not 15:40 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [] 15:55 -!- neha [~narula@tbilisi.csail.mit.edu] has quit [Ping timeout: 260 seconds] 15:55 -!- fanquake [~fanquake@unaffiliated/fanquake] has joined #bitcoin-core-dev 15:56 < fanquake> Speaking of benchmarking, can anyone tell me on "average" how much slower reindexing is compared to sync? 15:56 -!- neha [~narula@tbilisi.csail.mit.edu] has joined #bitcoin-core-dev 15:57 < fanquake> Ive managed to get qt reindexing at ~3%/hr, which seems very slow. This machine has performed a -reindex-chainstate in <3 hrs before. 15:57 < fanquake> * less than 3 hrs 15:58 -!- vicenteH [~user@96.235.15.37.dynamic.jazztel.es] has quit [Ping timeout: 260 seconds] 15:59 < sipa> fanquake: reindex-chainstate should be strictly faster than sync 16:00 < sipa> but the progress estimation code was changed significantly in 0.14 16:02 < gmaxwell> reindexing spends something like 20 minutes up front scanning for headers, which might be distorting your numbers. 16:05 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has quit [Read error: Connection reset by peer] 16:05 -!- PRab [~chatzilla@c-68-62-95-247.hsd1.mi.comcast.net] has quit [Quit: ChatZilla 0.9.93 [Firefox 51.0.1/20170125094131]] 16:05 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has joined #bitcoin-core-dev 16:14 -!- handlex [~handlex@2804:14c:658f:4dc7:95b:d955:8008:1898] has joined #bitcoin-core-dev 16:15 -!- alpalp [~alpalp@unaffiliated/alpalp] has joined #bitcoin-core-dev 16:32 -!- marcoagner [~marcoagne@191.32.199.160] has quit [Ping timeout: 268 seconds] 16:34 -!- pfeerpedr [68c89a3b@gateway/web/freenode/ip.104.200.154.59] has joined #bitcoin-core-dev 16:35 < pfeerpedr> who do i need to talk to in order to speed up my transaction? 16:38 -!- pfeerpedr [68c89a3b@gateway/web/freenode/ip.104.200.154.59] has quit [Client Quit] 16:39 < bitcoin-git> [bitcoin] MarcoFalke opened pull request #9846: doc: Small release notes fixups in the list of pulls (0.14...Mf1702-014doc) https://github.com/bitcoin/bitcoin/pull/9846 16:40 -!- handlex [~handlex@2804:14c:658f:4dc7:95b:d955:8008:1898] has quit [Quit: handlex] 16:58 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has quit [Read error: Connection reset by peer] 17:01 -!- abpa [~abpa@96-82-80-25-static.hfc.comcastbusiness.net] has quit [Quit: Textual IRC Client: www.textualapp.com] 17:05 -!- Giszmo [~leo@ip-146-233.219.201.nextelmovil.cl] has quit [Ping timeout: 260 seconds] 17:14 -!- alpalp [~alpalp@unaffiliated/alpalp] has quit [Ping timeout: 240 seconds] 17:21 -!- Giszmo [~leo@ip-75-233.219.201.nextelmovil.cl] has joined #bitcoin-core-dev 17:24 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Read error: Connection reset by peer] 17:26 -!- goksinen [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #bitcoin-core-dev 17:34 -!- dodomojo [~goksinen@2604:2000:c591:8400:3d65:ad68:578:aaff] has joined #bitcoin-core-dev 17:37 -!- goksinen_ [~goksinen@2604:2000:c591:8400:24db:6ba6:2d93:b634] has quit [Ping timeout: 255 seconds] 17:45 -!- MarcoFalke [~marco@host244-2.natpool.mwn.de] has quit [Quit: MarcoFalke] 17:45 -!- handlex [~handlex@2804:14c:658f:4dc7:60bd:a855:cebd:c298] has joined #bitcoin-core-dev 17:54 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-nwetehhdrcnhmqtu] has quit [Quit: Connection closed for inactivity] 18:02 < bitcoin-git> [bitcoin] sipa opened pull request #9847: Extra test vector for BIP32 (master...bip32up) https://github.com/bitcoin/bitcoin/pull/9847 18:15 -!- alpalp [~alpalp@unaffiliated/alpalp] has joined #bitcoin-core-dev 18:24 -!- Giszmo [~leo@ip-75-233.219.201.nextelmovil.cl] has quit [Quit: Leaving.] 18:40 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has joined #bitcoin-core-dev 18:42 < achow101> cfields: just reset my gitian and got 8d4bb27b5ab1916f04b74a2bcdccf8781c46fea96a3d5eb4a4a7f587577df64c bitcoin-0.14.0-osx-unsigned.dmg 18:42 < achow101> does that match yours? 18:42 -!- jtimon [~quassel@garage-jp.static.monkeybrains.net] has quit [Remote host closed the connection] 18:42 < achow101> It's probably doing the alternating thing again 18:54 -!- go1111111 [~go1111111@2601:602:8802:78c0:ed70:9491:8420:5f8d] has joined #bitcoin-core-dev 18:55 -!- praxeology [~dmgores@unaffiliated/praxeology] has joined #bitcoin-core-dev 18:56 < fanquake> achow101 looks like it does match 18:56 < fanquake> So you've got the alternating builds again? I'm just about to finish mine. 19:08 -!- dodomojo [~goksinen@2604:2000:c591:8400:3d65:ad68:578:aaff] has quit [Remote host closed the connection] 19:11 -!- echonaut [~echonaut@46.101.192.134] has quit [Remote host closed the connection] 19:11 < bitcoin-git> [bitcoin] appop opened pull request #9848: update (master...master) https://github.com/bitcoin/bitcoin/pull/9848 19:11 -!- echonaut [~echonaut@46.101.192.134] has joined #bitcoin-core-dev 19:12 < bitcoin-git> [bitcoin] fanquake closed pull request #9848: update (master...master) https://github.com/bitcoin/bitcoin/pull/9848 19:16 -!- dodomojo [~goksinen@2604:2000:c591:8400:c35:ae55:525c:155f] has joined #bitcoin-core-dev 19:21 < fanquake> achow101 Interestingly, my osx gitian results now match cfields. Which is weird, because nothings changes since rc1 that could have fixed gitian issues. 19:29 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 19:30 -!- handlex [~handlex@2804:14c:658f:4dc7:60bd:a855:cebd:c298] has quit [Quit: handlex] 19:32 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 240 seconds] 19:36 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has quit [Quit: MarcoFalke] 19:37 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has joined #bitcoin-core-dev 19:42 < achow101> actually, just ran gitian again and it got cfields's results. I'll run it a few more times to make sure it is deterministic 20:01 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 255 seconds] 20:01 < bitcoin-git> [bitcoin] luke-jr opened pull request #9849: Qt: Network Watch tool (master...gui_netwatch) https://github.com/bitcoin/bitcoin/pull/9849 20:02 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 20:03 < cfields> achow101: it'd be really helpful if you could upload the .o files from a non-matching build 20:03 < achow101> I think I can give you the kvm image of the non-matching build. I just need to make sure it is the right one 20:04 < cfields> achow101: "on-target" after the build gives you a shell 20:04 -!- alpalp [~alpalp@unaffiliated/alpalp] has quit [Ping timeout: 240 seconds] 20:09 < achow101> cfields: well that build ended a while ago and I have since done other builds. right now I am trying to start the vm with that image of the mismatching build which I saved and then ssh'ing into it, but it doesn't seem to be working now 20:12 -!- dodomojo [~goksinen@2604:2000:c591:8400:c35:ae55:525c:155f] has quit [Remote host closed the connection] 20:16 < cfields> achow101: ok, let me know if you manage to get them. I'll check back in the morning 20:22 -!- fanquake [~fanquake@unaffiliated/fanquake] has quit [Quit: Leaving.] 20:28 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has quit [Read error: Connection reset by peer] 20:39 < achow101> cfields: I got all of the build stuff off of the vm and tar'ed it. It should contain all of the .o files. Download: https://drive.google.com/file/d/0Bxw3ip9QfNOUVzkwUnlhMTExYjg/view?usp=sharing 20:40 < achow101> also I can give you the vm which contains all of that stuff too. I'm waiting for the upload of that to finish 20:46 < achow101> cfields: vm with the mismatching build: https://drive.google.com/file/d/0Bxw3ip9QfNOUN0E2aDZZQU1Pd2s/view?usp=sharing 21:00 -!- dermoth [~thomas@dsl-216-221-55-141.mtl.contact.net] has quit [Read error: Connection reset by peer] 21:00 -!- dermoth [~thomas@dsl-216-221-55-141.mtl.contact.net] has joined #bitcoin-core-dev 21:05 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Ping timeout: 240 seconds] 21:06 -!- PaulCape_ [~PaulCapes@2604:5500:17:2ea:e1f0:f5b8:653b:5575] has joined #bitcoin-core-dev 21:20 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has joined #bitcoin-core-dev 21:25 < cfields> achow101: er, you sure that's a broken build? 21:25 -!- praxeology [~dmgores@unaffiliated/praxeology] has left #bitcoin-core-dev [] 21:26 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 260 seconds] 21:30 < luke-jr> (we have 3 sigs on rc2) 21:30 < luke-jr> oh, but not all matching 21:31 < cfields> luke-jr: yea, i think i'll delay signing until morning once a few more are in 21:31 < cfields> now that we have achow101's objects for comparison, I'm hoping it'll point to the culprit 21:31 < achow101> cfields: I'm pretty sure that's the broken build 21:35 < achow101> luke-jr: my matching osx ones are pr'ed 21:36 < cfields> achow101: are you positive? All of my object files are identical as far as i can tell 21:37 < achow101> yes. 21:37 < achow101> you can fire up the vm image I gave you to check as well 21:38 -!- whphhg [whphhg@gateway/vpn/mullvad/x-jpgdxnhncnkcyckv] has quit [Read error: Connection reset by peer] 21:44 -!- go1111111 [~go1111111@2601:602:8802:78c0:ed70:9491:8420:5f8d] has quit [Ping timeout: 240 seconds] 21:44 < cfields> achow101: ok nm, got the diff now 21:48 < achow101> cool 21:53 < cfields> achow101: mmm, they're different kernels 21:53 < cfields> that's the only obvious thing i see 21:54 < cfields> maybe qt embeds uname output? 21:56 < achow101> but why would it only affect osx? 21:56 < achow101> also, how are they different kernels? I thought the vms were built exactly the same 21:56 < cfields> they should be 21:57 -!- lclc [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 21:57 < achow101> oh, maybe the upgrade that happens every time was failing some of the time? 21:57 < cfields> -uname -r = 3.13.0-108-generic 21:57 < cfields> +uname -r = 3.13.0-77-generic 21:57 < luke-jr> LXC uses the host's kernel 21:57 < luke-jr> so no matter what, we can't rely on kernels to match 21:57 < achow101> luke-jr: I'm using kvm 21:59 < cfields> luke-jr: well the fact that the kernels don't match is indicative that they're not using the same base 21:59 < cfields> in which case glibc (or something) may be different 21:59 < luke-jr> hm 22:00 < cfields> so it seems to be some kind of gitian issue 22:03 -!- go1111111 [~go1111111@c-24-18-220-198.hsd1.wa.comcast.net] has joined #bitcoin-core-dev 22:08 -!- instagibbs [~instagibb@pool-100-15-114-3.washdc.fios.verizon.net] has quit [Ping timeout: 240 seconds] 22:13 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has quit [Read error: Connection reset by peer] 22:14 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has joined #bitcoin-core-dev 22:18 -!- instagibbs [~instagibb@pool-100-15-117-236.washdc.fios.verizon.net] has joined #bitcoin-core-dev 22:26 -!- lclc [~lclc@unaffiliated/lclc] has quit [Ping timeout: 240 seconds] 22:32 -!- paveljanik [~paveljani@79.98.72.176] has joined #bitcoin-core-dev 22:32 -!- paveljanik [~paveljani@79.98.72.176] has quit [Changing host] 22:32 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev 23:06 -!- e4xit [~textual@cpc1-cmbg20-2-0-cust188.5-4.cable.virginm.net] has quit [Read error: Connection reset by peer] 23:42 < luke-jr> jonasschnelli: what kind of locking issues? can you elaborate? 23:43 < jonasschnelli> luke-jr: the app is unresponsive. I had to force shut down... will take a closer look 23:43 < jonasschnelli> luke-jr: but I like the PR 23:44 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection] 23:44 -!- lclc [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 23:48 < wumpus> so it looks like someone had the test_bitcoin issue outside of travis: #9850 23:48 < gribble> https://github.com/bitcoin/bitcoin/issues/9850 | test_bitcoin: /usr/include/boost/thread/pthread/recursive_mutex.hpp:104: boost::recursive_mutex::~recursive_mutex(): Assertion `!pthread_mutex_destroy() failed. · Issue #9850 · bitcoin/bitcoin · GitHub 23:50 < jonasschnelli> yes 23:51 < jonasschnelli> I tried to reproduce in ubuntu 14.04. but did not had the issue 23:51 < wumpus> same here. 23:51 < wumpus> did a depends build, just like travis, on 14.04, just like travis 23:51 < wumpus> so that means the same version of boost, gcc, etc 23:52 < wumpus> this is really strange 23:52 < jonasschnelli> Oh. Even that. 23:52 -!- lclc_ [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev 23:53 < gmaxwell> hurray! (?) 23:53 < jonasschnelli> I ran test_bitcoin in valgrind and I could see some uninitialised value 23:53 -!- fanquake [~fanquake@unaffiliated/fanquake] has joined #bitcoin-core-dev 23:54 < jonasschnelli> invoked by the toggle_network RPC tests 23:55 -!- lclc [~lclc@unaffiliated/lclc] has quit [Ping timeout: 260 seconds] 23:59 -!- lclc [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev