--- Day changed Thu Jun 08 2017
00:00 < gribble> https://github.com/bitcoin/bitcoin/issues/10477 | Use C++ initializer to initialze map and implement map comparator as const by cg31 · Pull Request #10477 · bitcoin/bitcoin · GitHub
00:07 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has joined #bitcoin-core-dev     
00:10 < wumpus> hhm thinking whether to add statistics such as number of incoming/outgoing connections, connections per bind point, to getnetworkinfo or just make a statistics script based on getpeerinfo on top for myself
00:10 < wumpus> the # in/out connections is in the GUI, so in principle it's make sense to add it to RPC
00:13 -!- btcdrak [uid230524@gateway/web/irccloud.com/x-kncikaaolpzfngpr] has quit [Quit: Connection closed for inactivity]     
00:14 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has quit [Ping timeout: 260 seconds]     
00:15 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev     
00:15 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has joined #bitcoin-core-dev     
00:17 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
00:17 -!- fanquake [~fanquake@unaffiliated/fanquake] has joined #bitcoin-core-dev     
00:18 < fanquake> kallewoof oops, corrected.
00:37 -!- coredump__ [~quassel@vpn-qld171.vpnsolutions.com.au] has quit [Ping timeout: 246 seconds]     
00:39 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has quit [Read error: Connection reset by peer]     
00:39 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev     
00:43 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e801084decf4...6c2d81f34dc4
00:43 < bitcoin-git> bitcoin/master 0abc588 practicalswift: [tests] Remove printf(...)
00:43 < bitcoin-git> bitcoin/master 6c2d81f Wladimir J. van der Laan: Merge #10524: [tests] Remove printf(...)...
00:44 < bitcoin-git> [bitcoin] laanwj closed pull request #10524: [tests] Remove printf(...) (master...u-for-unsigned-int) https://github.com/bitcoin/bitcoin/pull/10524
01:19 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Quit: Leaving.]     
01:21 -!- timothy [tredaelli@redhat/timothy] has joined #bitcoin-core-dev     
01:28 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Ping timeout: 246 seconds]     
01:38 -!- comboy [~quassel@tesuji.pl] has quit [Read error: Connection reset by peer]     
01:38 -!- comboy_ [~quassel@tesuji.pl] has joined #bitcoin-core-dev     
01:41 -!- justan0theruser is now known as justanotheruser     
01:49 < bitcoin-git> [bitcoin] practicalswift opened pull request #10553: Simplify "bool x = y ? true : false". Remove unused function and trailing semicolon. (master...minor-cleanups) https://github.com/bitcoin/bitcoin/pull/10553
02:08 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-cjbcyjevcckcgxbv] has quit [Quit: Connection closed for inactivity]     
02:23 -!- fanquake [~fanquake@unaffiliated/fanquake] has quit [Quit: Leaving.]     
02:25 -!- btcdrak [uid230524@gateway/web/irccloud.com/x-fghpzazjpottbtcj] has joined #bitcoin-core-dev     
02:51 -!- tunafizz [~tuna@c-71-207-56-72.hsd1.pa.comcast.net] has quit [Ping timeout: 246 seconds]     
02:51 -!- tunafizz [~tuna@c-71-207-56-72.hsd1.pa.comcast.net] has joined #bitcoin-core-dev     
03:18 -!- coredump_ [~quassel@101.165.141.152] has joined #bitcoin-core-dev     
03:25 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
03:29 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:f5e0:364b:9b9d:28f7] has joined #bitcoin-core-dev     
03:33 -!- jtimon [~quassel@117.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev     
03:33 -!- goatturner [~Beatrootg@2a02:c7d:12e:100:8936:bf9f:89a5:2cde] has quit [Ping timeout: 255 seconds]     
03:35 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Ping timeout: 246 seconds]     
03:40 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/6c2d81f34dc4...71ab6e553856
03:40 < bitcoin-git> bitcoin/master 227ae9b practicalswift: [tests] Use FastRandomContext instead of boost::random::{mt19937,uniform_int_distribution}
03:40 < bitcoin-git> bitcoin/master 71ab6e5 Wladimir J. van der Laan: Merge #10547: [tests] Use FastRandomContext instead of boost::random::{mt19937,uniform_int_distribution}...
03:40 < bitcoin-git> [bitcoin] laanwj closed pull request #10547: [tests] Use FastRandomContext instead of boost::random::{mt19937,uniform_int_distribution} (master...remove-boost-random-dependency) https://github.com/bitcoin/bitcoin/pull/10547
03:41 -!- coredump_ [~quassel@101.165.141.152] has quit [Ping timeout: 246 seconds]     
03:45 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/71ab6e553856...35e7f13f68f9
03:45 < bitcoin-git> bitcoin/master 246a02f practicalswift: Use std::unordered_{map,set} (C++11) instead of boost::unordered_{map,set}
03:45 < bitcoin-git> bitcoin/master 35e7f13 Wladimir J. van der Laan: Merge #10548: Use std::unordered_{map,set} (C++11) instead of boost::unordered_{map,set}...
03:46 < bitcoin-git> [bitcoin] laanwj closed pull request #10548: Use std::unordered_{map,set} (C++11) instead of boost::unordered_{map,set} (master...unordered_map) https://github.com/bitcoin/bitcoin/pull/10548
04:17 -!- cryptapus_afk [~cryptapus@rrcs-97-77-228-114.sw.biz.rr.com] has joined #bitcoin-core-dev     
04:17 -!- cryptapus_afk [~cryptapus@rrcs-97-77-228-114.sw.biz.rr.com] has quit [Changing host]     
04:17 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
04:27 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Remote host closed the connection]     
04:29 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
04:31 -!- JackH [~laptop@79-73-186-168.dynamic.dsl.as9105.com] has joined #bitcoin-core-dev     
04:33 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Remote host closed the connection]     
04:37 < bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/35e7f13f68f9...9c248e39f280
04:37 < bitcoin-git> bitcoin/master 5b75c47 Andrew Chow: Add a valid opcode sanity check to CScript...
04:37 < bitcoin-git> bitcoin/master ac4e438 Andrew Chow: Sanity check transaction scripts in DecodeHexTx...
04:37 < bitcoin-git> bitcoin/master 9c248e3 Wladimir J. van der Laan: Merge #10481: Decodehextx scripts sanity check...
04:37 < bitcoin-git> [bitcoin] laanwj closed pull request #10481: Decodehextx scripts sanity check  (master...decodehextx-sanity) https://github.com/bitcoin/bitcoin/pull/10481
04:47 -!- dermoth [~thomas@dsl-66-36-135-165.mtl.aei.ca] has quit [Read error: Connection reset by peer]     
04:48 -!- dermoth [~thomas@dsl-66-36-135-165.mtl.aei.ca] has joined #bitcoin-core-dev     
04:58 -!- btcdrak [uid230524@gateway/web/irccloud.com/x-fghpzazjpottbtcj] has quit [Quit: Connection closed for inactivity]     
05:13 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:c0d:8205:7075:a1ed] has joined #bitcoin-core-dev     
05:17 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:f5e0:364b:9b9d:28f7] has quit [Ping timeout: 246 seconds]     
05:18 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has quit [Ping timeout: 260 seconds]     
05:23 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has joined #bitcoin-core-dev     
05:30 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has quit [Ping timeout: 240 seconds]     
05:34 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev     
05:37 -!- Aaronvan_ [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev     
05:40 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 258 seconds]     
06:03 -!- nickler_ is now known as nickler     
06:03 -!- btcdrak [uid230524@gateway/web/irccloud.com/x-cwozluzovrkfcovk] has joined #bitcoin-core-dev     
06:17 -!- Chris_Stewart_5 [~chris@173-31-39-168.client.mchsi.com] has joined #bitcoin-core-dev     
06:22 < bitcoin-git> [bitcoin] somdoron opened pull request #10554: ZMQ: add publishers for wallet transactions. (master...zmq_wallet_tx) https://github.com/bitcoin/bitcoin/pull/10554
06:31 -!- cryptapus_afk [~cryptapus@rrcs-97-77-228-114.sw.biz.rr.com] has joined #bitcoin-core-dev     
06:31 -!- cryptapus_afk [~cryptapus@rrcs-97-77-228-114.sw.biz.rr.com] has quit [Changing host]     
06:31 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
06:48 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 245 seconds]     
06:50 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev     
06:55 -!- GAit [~GAit@unaffiliated/gait] has quit [Ping timeout: 245 seconds]     
06:58 -!- GAit [~GAit@unaffiliated/gait] has joined #bitcoin-core-dev     
07:05 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev     
07:08 -!- tunafizz [~tuna@c-71-207-56-72.hsd1.pa.comcast.net] has quit [Ping timeout: 240 seconds]     
07:08 -!- tunafizz [~tuna@c-71-207-56-72.hsd1.pa.comcast.net] has joined #bitcoin-core-dev     
07:46 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Remote host closed the connection]     
07:47 < bitcoin-git> [bitcoin] jnewbery opened pull request #10555: [tests] various improvements to zmq_test.py (master...zmqtestimprovements) https://github.com/bitcoin/bitcoin/pull/10555
08:01 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has quit [Quit: quit]     
08:02 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has quit [Quit: Leaving]     
08:04 < bitcoin-git> [bitcoin] jnewbery opened pull request #10556: Move stop/start functions from utils.py into BitcoinTestFramework (master...testframeworkstopstart) https://github.com/bitcoin/bitcoin/pull/10556
08:08 -!- cryptapus_afk [~cryptapus@jupiter.osmus.org] has joined #bitcoin-core-dev     
08:08 -!- cryptapus_afk [~cryptapus@jupiter.osmus.org] has quit [Changing host]     
08:08 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
08:24 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Remote host closed the connection]     
08:24 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
08:28 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Remote host closed the connection]     
08:29 -!- cryptapus_afk [~cryptapus@jupiter.osmus.org] has joined #bitcoin-core-dev     
08:29 -!- cryptapus_afk [~cryptapus@jupiter.osmus.org] has quit [Changing host]     
08:29 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
08:32 < bitcoin-git> [bitcoin] morcos opened pull request #10557: Make check to distinguish between orphan txs and old txs more efficient. (master...dontcheckoutputs) https://github.com/bitcoin/bitcoin/pull/10557
08:42 -!- Gnof [~Gnof@CPE00fc8d7da303-CM00fc8d7da300.cpe.net.cable.rogers.com] has joined #bitcoin-core-dev     
08:48 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev     
08:49 -!- Aaronvan_ [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 255 seconds]     
08:56 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 246 seconds]     
09:16 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has joined #bitcoin-core-dev     
09:16 < bitcoin-git> [bitcoin] morcos opened pull request #10558: Address nits from per-utxo change (master...10195nits) https://github.com/bitcoin/bitcoin/pull/10558
09:19 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
09:22 < jtimon> travis didn't run for https://github.com/bitcoin/bitcoin/pull/9176
09:31 -!- timothy [tredaelli@redhat/timothy] has quit [Quit: Konversation terminated!]     
09:37 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Ping timeout: 260 seconds]     
09:51 -!- jtimon [~quassel@117.29.134.37.dynamic.jazztel.es] has quit [Remote host closed the connection]     
10:14 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 240 seconds]     
10:18 -!- Gnof [~Gnof@CPE00fc8d7da303-CM00fc8d7da300.cpe.net.cable.rogers.com] has quit [Ping timeout: 255 seconds]     
10:19 -!- cryptapus_afk [~cryptapus@jupiter.osmus.org] has joined #bitcoin-core-dev     
10:19 -!- cryptapus_afk [~cryptapus@jupiter.osmus.org] has quit [Changing host]     
10:19 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
10:25 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev     
10:28 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev     
10:45 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 240 seconds]     
10:45 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev     
10:52 -!- ula [~kvirc@b2b-78-94-11-194.unitymedia.biz] has quit [Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/]     
10:53 -!- nakaluna [~nakaluna@gateway/vpn/privateinternetaccess/nakaluna] has joined #bitcoin-core-dev     
10:56 -!- ula [~kvirc@b2b-78-94-11-194.unitymedia.biz] has joined #bitcoin-core-dev     
10:57 < bitcoin-git> [bitcoin] achow101 closed pull request #9522: [RPC] Fix decoderawtransaction decoding of segwit txs (master...fix-decoderawtx) https://github.com/bitcoin/bitcoin/pull/9522
11:02 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev     
11:02 -!- ndr76 [5aae0202@gateway/web/freenode/ip.90.174.2.2] has joined #bitcoin-core-dev     
11:03 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit]     
11:06 -!- nakaluna [~nakaluna@gateway/vpn/privateinternetaccess/nakaluna] has quit [Quit: -]     
11:07 -!- nakaluna [~nakaluna@gateway/vpn/privateinternetaccess/nakaluna] has joined #bitcoin-core-dev     
11:09 < arubi> vindicated at last :) https://github.com/bitcoin/bitcoin/pull/8837#issuecomment-250542708
11:11 < arubi> luke-jr made a comment about changing that 0x00 to 0xff.  in case segwit is retried, this is very good input imo..
11:12 -!- goatturner [~Beatrootg@2a02:c7d:12e:100:b42b:5d5a:c350:ff7f] has joined #bitcoin-core-dev     
11:13 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:bdaa:5de1:b10d:1512] has joined #bitcoin-core-dev     
11:13 < sipa> yeah we should have used 0xff
11:13 -!- sdfkjs23 [~steven@173.239.11.108] has quit [Quit: Leaving]     
11:13 < sipa> or even better... we should never have used normal transaction encoding for partial transactions
11:15 < arubi> right, I'd love for a blob of tx data to tell me what it wants to me to do with it
11:15  * sipa proposes ASN.1
11:15 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:c0d:8205:7075:a1ed] has quit [Ping timeout: 260 seconds]     
11:16  * sipa hides
11:16 < arubi> haha, make the wallet even fatter
11:16 < Chris_Stewart_5> arubi: so some sort of marker that says 'needs signature'? 
11:16 < arubi> the scriptsig area itself can describe what it wants without hurting current layout
11:17 < arubi> "oops, this is unsigned input" or maybe a prevtxid "oops, not funded..."
11:17 -!- goatturner [~Beatrootg@2a02:c7d:12e:100:b42b:5d5a:c350:ff7f] has quit [Ping timeout: 260 seconds]     
11:17 < arubi> I mean, it depends at what stage it's parsed..  hard to tell for "decoderawtransaction" :\
11:18 -!- ajd_ [~Anthony@91.239.96.138] has joined #bitcoin-core-dev     
11:19 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-odjefsqsnwswkqil] has joined #bitcoin-core-dev     
11:23 -!- jtimon [~quassel@117.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev     
11:27 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/9c248e39f280...29f80cd230c3
11:27 < bitcoin-git> bitcoin/master 3fb81a8 practicalswift: Use list initialization (C++11) for maps/vectors instead of boost::assign::map_list_of/list_of
11:27 < bitcoin-git> bitcoin/master 29f80cd Wladimir J. van der Laan: Merge #10545: Use list initialization (C++11) for maps/vectors instead of boost::assign::map_list_of/list_of...
11:27 < bitcoin-git> [bitcoin] laanwj closed pull request #10545: Use list initialization (C++11) for maps/vectors instead of boost::assign::map_list_of/list_of (master...list_of) https://github.com/bitcoin/bitcoin/pull/10545
11:30 -!- achow101 [~achow101@unaffiliated/achow101] has left #bitcoin-core-dev ["Leaving"]     
11:30 -!- achow101 [~achow101@unaffiliated/achow101] has joined #bitcoin-core-dev     
11:50 < bitcoin-git> [bitcoin] sdaftuar reopened pull request #10357: Allow setting nMinimumChainWork on command line (master...2017-05-chainwork-commandline) https://github.com/bitcoin/bitcoin/pull/10357
11:50 < gmaxwell> Meeting in ten minutes.
12:00 < sipa> DOINK
12:00 < wumpus> #startmeeting
12:00 < lightningbot> Meeting started Thu Jun  8 19:00:04 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:00 < lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
12:00 < wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 instagibbs
12:00 < jonasschnelli> hi     
12:00 < sdaftuar> hi     
12:00 < sipa> hi     
12:00 < achow101> hi     
12:00 < wumpus> proposed topics?
12:00 < kanzure> hi.     
12:00 < sipa> UI interaction with pertxout upgrade?
12:00 < gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
12:01 < jonasschnelli> sipa: ack
12:01 < wumpus> ok, let's do high priority for review first, then that
12:01 < wumpus> #topic high priority for review
12:01 < sipa> #10148
12:01 < gribble> https://github.com/bitcoin/bitcoin/issues/10148 | Use non-atomic flushing with block replay by sipa · Pull Request #10148 · bitcoin/bitcoin · GitHub
12:01 < luke-jr> I've rebased multiwallet etc
12:02 < wumpus> luke-jr: great!
12:02 < sipa> (^ will double our effectively available dbcache)
12:02 < wumpus> luke-jr: sae there were some new review comments, did you address them yet?
12:03 < luke-jr> wumpus: I believe all review comments are addressed or answered
12:03 < wumpus> added 10148
12:03 < sipa> thanks!
12:03 < jtimon> still waiting on my current one, thanks wumpus for benchmarking
12:03 < jonasschnelli> #8694 now passes gitian, will re-utack soon
12:04 < gribble> https://github.com/bitcoin/bitcoin/issues/8694 | Basic multiwallet support by luke-jr · Pull Request #8694 · bitcoin/bitcoin · GitHub
12:04 < wumpus> jtimon: a very nice gain!
12:04 < luke-jr> oh, there was a comment asking for tests..
12:04 < luke-jr> I didn't write any, although I agree they would be nice
12:04 < wumpus> jtimon: unfortunately bitcoind nuked my log so I can't get the timings, but 26% savings in block hash operations is noce
12:05 < jtimon> wumpus: nice
12:05 < jtimon> we're talking about #10339 in case someone is lost
12:05 < gribble> https://github.com/bitcoin/bitcoin/issues/10339 | Optimization: Calculate block hash less times by jtimon · Pull Request #10339 · bitcoin/bitcoin · GitHub
12:05 < wumpus> luke-jr: well I'd say add those later, this is only the first in a series towards multiwallet after all
12:06 < sipa> wumpus: status on account to label conversion?
12:06 < luke-jr> indeed, it might not be possible to test right now since it's not exposed to RPC
12:06 < BlueMatt> wumpus: "in block hash operations" hmm? can you be more specific?
12:06 < gmaxwell> Related to hashing, does anyone here have AMD ryzen yet?
12:06 < BlueMatt> whats that compared to total runtime?
12:06 < wumpus> BlueMatt: I don't know about time, I just instrumented the block header hash function
12:06 < gmaxwell> BlueMatt: it's tiny vs total runtime, I assume-- but it should impact latency on block handling somewhat.
12:06 < luke-jr> gmaxwell: no, but I'd been holding out for SHA2 acceleration before upgrading, so I might get one if AMD is competing with Intel again
12:06 < sipa> related topic: sha2/crc32 hw accel?
12:07 < wumpus> as I said, I lost the timings (blame bitcoind nuking the log if it's too large)
12:07 < gmaxwell> luke-jr: Ryzen has the sha2 instructions... so I'm asking to find out if anyone will be able to test support for them. :)
12:07 < luke-jr> gmaxwell: right, that's why ;)
12:07 < luke-jr> I just haven't had time to investigate the pros/cons otherwise yet
12:07 < wumpus> sipa: no progress on that yet
12:08 < gmaxwell> In any case, 98% of the work needed to support sha2 instructions is the same as using SSE4 SHA2 which will itself be a non-trival speedup.
12:08 < wumpus> crc speedup should be possible after the leveldb upgrade that sipa PRed
12:08 < luke-jr> gmaxwell: we removed SSE4 stuff years ago, but I'm not sure if it was used for non-mining
12:08 < sipa> luke-jr: it was only used for mining at the time
12:08 < gmaxwell> BlueMatt: IIRC I instrumented and measured accepting a block at tip hashed the header ~6 times or so.
12:08 < gmaxwell> luke-jr: it was mining only.
12:09 < wumpus> luke-jr: that was a different one, the N-way-parallel
12:09 < sipa> yup 4-way parallel SSE2, iirc
12:09 < gmaxwell> and it was a vector SHA2 that had to work on 4 elements at a time.
12:09 < jtimon> gmaxwell: that's before or after the patch?
12:09 < gmaxwell> What we should be implementing now is the one at a time SIMD sha2.  I believe matt uses it in fibre but without autodetection.
12:09 < sipa> it seems we've strayed from the topic?
12:09 < gmaxwell> jtimon: before.
12:09 < luke-jr> I wonder if it'd be worth using 4-way for merkle trees etc
12:09 < BlueMatt> gmaxwell: yea, but if its not measurable on the total, unlikely to be worth the effort and complexity.....an alternative - do we have CBlockIndex*es in most of those places? pblockindex->GetBlockHash() is free and simpler than passing hashes around
12:09 < jonasschnelli> Will have access to a Ryzen in a couple of days via Hetzner
12:09 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Quit: Leaving.]     
12:10  * luke-jr wonders if the GCC compile farm has a Ryzen
12:10 < morcos> Without doing a thorough review of 10339, is the speedup worth it as opposed to the tradeoff on making the code a little more complicated/involved.  Who was it that said the primary point of source code is to communicate between developers
12:10 < jonasschnelli> #10339
12:10 < morcos> It just seems weird to pass a block and separately it's hash into a bunch of functions
12:10 < gribble> https://github.com/bitcoin/bitcoin/issues/10339 | Optimization: Calculate block hash less times by jtimon · Pull Request #10339 · bitcoin/bitcoin · GitHub
12:11 < jtimon> BlueMatt: what's wrong with passing hashes around?
12:11 < BlueMatt> DoPartOfConnectBlock(42 args)
12:11 < wumpus> #topic Optimization: Calculate block hash less times by jtimon
12:11 < BlueMatt> we already have too many args in many of those functions
12:11 < instagibbs> "fewer" :P
12:11 < wumpus> well if hashing is a bottleneck, the obvious optimization is to just to it less
12:11 < sipa> instagibbs: i didn't want to say anything :)
12:12 < BlueMatt> wumpus: is it?
12:12 < wumpus> if hashing is not a bottleneck, then going after SSE and such makes no sense either
12:12 < sipa> BlueMatt: i believe that in all places where we already have CBlockIndex, no new block hashes are computed
12:12 < sipa> all of the cases in 10339 are before having a CBlockIndex
12:12 < wumpus> so if #10339 is not an improvement then we can leave hashing alone
12:12 < gribble> https://github.com/bitcoin/bitcoin/issues/10339 | Optimization: Calculate block hash less times by jtimon · Pull Request #10339 · bitcoin/bitcoin · GitHub
12:12 < gmaxwell> Well my suggestion on that hash thing was just to cache the hash in the block object, but Pieter said he prefered this.
12:12 < jtimon> instagibbs: calculate fewer times? thanks
12:12 < gmaxwell> (I also implemented that, though in a way that wasn't correct for the mining code.)
12:13 < BlueMatt> gmaxwell: I highly prefer that, though making CBlock const ala CTransaction is a bit more involved
12:13 < sipa> gmaxwell: if we create another CBlock for the cases where that's not needed
12:13 < sipa> (like GetTransaction and mining code)
12:13 < jtimon> Node also that we're sometimes getting the hash from the index and sometimes from the CBlock
12:13 < wumpus> right, calculating it eagerly can also result in overhead
12:13 < morcos> I'd just like to see some evidence, even a little bit, that this optimization is worth it.  26% fewer hashing operations results in what savings?  Or do people not agree it makes the code a bit more cumbersome
12:14 < sipa> i don't see much problem in passing a hash along with a block if you know it
12:14 < BlueMatt> passing in the hash as a part of ProcessNewBlock? yuck
12:14 < sipa> it's certainly a bit of complication, but a trivial one
12:14 < wumpus> morcos: sure, my point is just that if 26% fewer hashing operations isn't worth it, then using special intstructions to gain a few % is netiher worth it
12:14 < CodeShark> seems like there are probably lower hanging fruit in optimizations but meh
12:14 < BlueMatt> I just spent a bunch of time getting fewer args there
12:14 < morcos> wumpus: well hashing occurs a lot more often than in the calculations of these block hashes
12:14 < BlueMatt> to separate the consensus code from net_processing, now we're adding more coupling :(
12:14 < jtimon> it seems to me that the reserve is mostly a question of style
12:14 < gmaxwell> morcos: My reason for bringing it up is that I believe the repeated hashing is on the latency critical path for block propagation to the tune of maybe a millisecond-ish. 
12:15 < jtimon> I mean calculating it eagerly can penalize in some cases, right
12:15 < gmaxwell> wumpus: these optimizations just prevent repeated hashing of the headers, in terms of total bytes hashed while syncing thats pretty small even though we do have a high repetition factor.
12:15 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev     
12:15 < gmaxwell> jtimon: what case would computing it early peanalize? 
12:15 < wumpus> gmaxwell: ok...
12:15 < CodeShark> BlueMatt: my preference is to sacrifice a little performance for better architecture ;)
12:15 < sdaftuar> i will try to benchmark the effect on validation speed for this
12:15 < wumpus> CodeShark: I think that's a fair argument
12:15 < sipa> wumpus: transaction hashing accounts for far more than block header hashing, but not on the critical path
12:16 < jtimon> gmaxwell: when the validation fails for some other reason before you need the hash?
12:16  * BlueMatt would be more ok if we calculated it at the top of ProcessNewBlock and pass that one around, but still not ideal
12:16 < BlueMatt> passing it in to ProcessNewBlock is.....ugh
12:16 < gmaxwell> jtimon: I don't think there is any validation failure path where we don't hash... after all, we'll want to know _what_ block hash failed to validate.
12:16 < jtimon> BlueMatt: you could have said that when each function had a separated commit, but I can leave that out
12:16 < BlueMatt> or, really, chnging the signatures of stuff in validation.h for this without a benchmark showing its a real win :(
12:16 < jtimon> gmaxwell: yeah, fair enough
12:17  * morcos has said his piece and is willing to let the people who spent more time thinking about this decide
12:17 < wumpus> ok, clear, better to close it I guess, would have made sense to have this discussion earlier
12:17 < jtimon> so we come back to only the "ugh" argument
12:17 < gmaxwell> But I still don't understand why sipa prefered to do this jtimon patch instead of making the object cache it.   FWIW, just monkey patching it 'works' and the only obvious issue is mining.
12:17 < sipa> gmaxwell: and every other read from disk
12:18 < jtimon> I preferred this because I think it's simpler than the cache, even if it's "ugh"
12:18 < gmaxwell> sipa: I don't understand your comment.
12:18 < sipa> i consider adding more arguments to validation-specific functions less invasively than changing a primitive datastructure
12:18 < sipa> we can do both, if needed
12:19 < wumpus> sipa: I tend to agree
12:19 < sipa> gmaxwell: i believe we often read blocks from disk without caring about its hash, because we already know it
12:19 < wumpus> sipa: just passing extra arguments is easier to reason about than extending primitive datastructures
12:19 < gmaxwell> sipa: the read funcion computes it!
12:19 < sipa> gmaxwell: ?
12:20 < sipa> currently, yes, as a checksum
12:20 < sipa> if we care about the checksum functionality, we should add a crc
12:20 < gmaxwell>     // Check the header
12:20 < gmaxwell>     if (!CheckProofOfWork(block.GetHash(), block.nBits, consensusParams))
12:20 < wumpus> jtimon: I agree, caching is always somewhat risky and error prone, this is more explicit and clear
12:20 < gmaxwell>         return error("ReadBlockFromDisk: Errors in block header at %s", pos.ToString());
12:20 < wumpus> then again if it's not worth it, performanec wise, we shouldn't do it at all
12:20 < wumpus> not in another way either
12:21 < sipa> making the block hash part of CBlock makes us unable to skip computing that hash
12:21 < sipa> even in places where we don't care about it
12:21 < gmaxwell> I think I previously gave figures on the latency impact of caching.
12:21 < jtimon> how can I benchmark this to convince morcos?
12:21 < gmaxwell> sipa: Where are those places?
12:21 < wumpus> I don't know, I tried
12:22 < sipa> gmaxwell: every time we read a block from disk, we already have its hash (because that's how we look it up!)
12:22 < sdaftuar> sipa: don't we also hash all the transactions when we read a block from disk?
12:22 < sipa> gmaxwell: recomputing it there serves as nothing more than a checksum
12:22 < sdaftuar> surely that dominates all this
12:22 < morcos> sipa: i know i said i'd shut up, but you just read from DISK, who cares if you hash 80 bytes?
12:22 < wumpus> might be time for next topic, if this is not a perf issue, might be better to discuss it after meeting
12:22 < sipa> sdaftuar: good point.
12:23 < gmaxwell> sdaftuar: we don't. I used to think we did but we do not. 
12:23 < sipa> yes, we do
12:23 < gmaxwell> sipa: where? 
12:23 < sipa> deserializing a transaction computes its hash
12:23 < gmaxwell> oh right, what we don't do is validate the hashroot.
12:23 < sipa> let's discuss this after the meeting
12:23 < gmaxwell> k     
12:24 < sipa> next topic?
12:24 < wumpus> #topic UI interaction with pertxout upgrade
12:25 < jtimon> in any case, BlueMatt morcos if you already knew you didn't liked this, I would have appreaciated knowing it earlier, maybe a provisional concept NACK or something
12:25 < sipa> so, since 10195, we'll have a possibly lengthy (minutes on decent hardware, possibly much longer elsewhere) upgrade process from the old per-txid utxo db to the per-txout one
12:25 < morcos> jtimon: yes sorry, i only looked at the PR during todays meeting, also why i said i'd stop arguing about it
12:26 < sipa> that needs some GUI interaction, i believe
12:26 < wumpus> jtimon: same here, this should have been clear sooner, otherwise I wouldn't have spent as much time on it
12:26 < sipa> or perhaps even in bitcoind
12:26 < morcos> wumpus: in general perf improvements shoudl include data in the PR request when possible I think
12:26 < wumpus> jtimon: it's kind of disappointing to discover this does what it says on the tin, save ~1/4th of block hash operations, just to discover that's not even important
12:26 < jonasschnelli> Can you not just use uiInterface.Progress() like in VerifyDB?
12:26 < wumpus> jtimon: that kind of annoys me
12:26 < sipa> jonasschnelli: that doesn't let you interrupt
12:27 < morcos> sipa: yeah i noticed that it doesn't even output to debug log that its doing an upgrade right?
12:27 < sipa> it should log
12:27 < sipa> "Upgrading database..."
12:27 < luke-jr> jonasschnelli: I'd think users should be able to send txs during the upgrade.
12:27 < morcos> oh     
12:27 < jonasschnelli> luke-jr: but RPC is also not ready in this case? RIght?
12:27 < gmaxwell> morcos: it logs, but you won't see it if you have leveldb logging turned on. :P
12:27 < sipa> nothing works during the upgrade
12:27 < sipa> there is no full node available
12:28 < jonasschnelli> why would you then need interruption?
12:28 < wumpus> it should at least show a progress indicator and be interruptible indeed, ideally
12:28 < sipa> because maybe it takes too long
12:28 < sipa> and they want to continue another time
12:28 < wumpus> yes, because the user may want to do something else with the computer, having not expected to have to spend a long time upgrading
12:28 < gmaxwell> by interruption this means 'crash', not continue running without the upgrade.
12:28 < jtimon> anyway, I would still like to benchmark it if anyone can help me (not sure what to do)
12:28 < jonasschnelli> Ah.. the update can be done in multiple steps..
12:28 < sipa> yes     
12:28 < jonasschnelli> well that would also need communication then
12:28 < wumpus> gmaxwell: well 'exit' not crash
12:29 < gmaxwell> same difference
12:29 < morcos> sipa: yeah i missed that somehow, sorry
12:29 < sipa> crashing "should" be fine
12:29 < gmaxwell> it better be! :) (also, I've tested it a fair bit)
12:29 < jonasschnelli> [Updating 0%] \n You can shutdown and continue later [Shutdown-Button] <-- something like that?
12:29 < wumpus> jtimon: benchmarking the reindex chainstate maybe?
12:29 < gmaxwell> jonasschnelli: like that.
12:29 < luke-jr> what if you crash, run the old version, and then the new one again?
12:29 < wumpus> jtimon: I still wonder why -stopatblock doesn't work
12:30 < gmaxwell> luke-jr: you get to keep both pieces? (I believe the old version will tell you the database is corrupt and stop there.... but I haven't tested that, so it's a good question.)
12:30 < sipa> wumpus: no; that's dominated by tx validation/deserialization... a good benchmark tests block validation given fully populated mempool and sigcache etc
12:30 < jonasschnelli> How about logging? Can we log similar to the VerifyDB [the non-newline % steps]?
12:30 < sipa> jonasschnelli: sure
12:30 < wumpus> sipa: ok...
12:30 < jonasschnelli> I can work on that. Seems to fit my skillset. :)
12:30 < luke-jr> gmaxwell: IMO users may get tired of waiting and prefer to start the upgrade over in exchange for being able to use the old version in the meantime
12:30 < jtimon> sipa: perhaps it could be as simple as using -upgradeutxodb=1 once or something of the sort?
12:30 < wumpus> sipa: good luck doing that in a deterministic way, though
12:30 < sipa> maybe VerifyDB or something like that can pass a bool saying "interruptible" ?
12:31 < gmaxwell> luke-jr: sounds like a case we should handle.
12:31 < sipa> jtimon: you can't not do it
12:31 < sipa> jtimon: i mean, -upgradeutxodb=0 would mean just exiting immediately :)
12:31 < wumpus> no, the upgrade needs to be done
12:31 < luke-jr> we could prompt before starting to warn the user, but IMO we'd probably want to begin ASAP, even if we hold off destroying backward compat until the user OKs it
12:31 < gmaxwell> meh.
12:32 < wumpus> sure, you could warn...
12:32 < gmaxwell> Keep in mind that on a reasonably fast computer the upgrade does not take long.
12:32 < jtimon> no, I mean once you set -upgradeutxodb=1 once it doesn't matter what you set anymore
12:32 < sipa> it's a new major release, i don't care about backward compatibility; the user can always reindex if they really need to
12:32 < morcos> so just to be clear, right now if someone went back to an old version, there is some chance they'd screw themselves right?
12:32 < morcos> (and need to reindex)
12:32 < wumpus> morcos: yes, it simply doesn't work. We should warn in the release notes.
12:32 < sipa> morcos: i believe that with very high probability the startup sanity check will fail
12:32 < luke-jr> pruned nodes cant reindex
12:32 < gmaxwell> morcos: old version rejects with a "your database is corrupted"  though luke was asking an interesting question about what happens if you do this mid-conversion.
12:33 < wumpus> we coudl have added logic to 0.14.2 to detect the new format and give a more specific error
12:33 < morcos> sipa: yeah that was my question, how far do you have to get along in the upgrade before your sanity check would fail with high probability
12:33 < gmaxwell> sipa: I did test that case, how could it not fail?
12:33 < sdaftuar> we only go back 6 blocks now in the startup sanity check i think
12:33 < luke-jr> wumpus: IMO it'd be better to destroy the upgrade so it starts over, and work (for incomplete upgrades)
12:33 < sdaftuar> isn't it possible you don't come across any bad entries?
12:33 < gmaxwell> wumpus: hm? I don't think that would be the right way, the right way would be to make the new version more incompatible.
12:33 < wumpus> gmaxwell: ok...
12:34 < sipa> there is a trivial change we can make that guarantees old versions will treat it as an empty db
12:34 < luke-jr> change the dir name?
12:34 < sipa> haha
12:34 < wumpus> gmaxwell: I just mean a specific error like 'the utxo database is in a newer format than supported by this version' would be clearer than a general sanity check error
12:34 < sipa> yes, i've considered that too
12:34 < morcos> empty db == corrupt and leave it so the new version can continue
12:34 < gmaxwell> no there is just the height index entry. 
12:34 < morcos> wumpus +1
12:34 < wumpus> but I don't know ,sorry for suggesting it
12:34 < gmaxwell> wumpus: oh, sure that would have been okay!
12:34 < morcos> thats at least somethign we shoudl do  for future changes
12:34 < sipa> wumpus: unfortunately it doesn't have a version number
12:34 < jtimon> luke-jr: we could move the main datadir to ./bitcoin/main maybe
12:34 < sipa> agree
12:34 < luke-jr> using a new db would also mean users from pre-obfuscation get that enabled..
12:35 < sipa> a new db is a possibility
12:35 < wumpus> sipa: we could have added one! but yes, too late.
12:35 < sipa> though i'd prefer to not need 2x disk storage during the upgrade
12:35 < sipa> not too late
12:35 < morcos> sipa: what was the trivial change you were referring to
12:35 < sipa> morcos: change the record name for the 'best block hash' entry
12:35 < gmaxwell> the database stores a record to indicate the current tip; we can just change that.
12:35 < sipa> and set the old one to something invalid
12:36 < luke-jr> [19:35:19] <sipa> though i'd prefer to not need 2x disk storage during the upgrade <-- IMO unavoidable if the "user gets tired of waiting and runs old version" is desired to work okay
12:36 < gmaxwell> FWIW the non-atomic flushing change already changes those records around.
12:36 < sipa> yup     
12:36 < sipa> well, in a backward compatible way
12:36 < gmaxwell> I don't think we should support user gets tired of waiting.
12:36 < gmaxwell> too high a cost for too low a benefit.
12:36 < sipa> it's even easier if it doesn't need to be backward compatible
12:37 < morcos> gmaxwell: yes we shouldn't support it, but it might be nice to support they tried to do it and didn't corrupt their database by trying so they now have to reindex
12:37 < gmaxwell> morcos: yes, in practice that is the case but it isn't guarenteed. We could make it guarenteed.
12:37 < wumpus> gmaxwell: well just exiting and continuing the upgrade later would be ok
12:38 < gmaxwell> wumpus: yea, that works except we don't have an exit button other than kill -9 :) 
12:38 < sipa> wumpus: also, i don't understand why the CompactRange doesn't work for you
12:38 < sipa> if it's not guaranteed (or even not likely) to work, maybe the 2x disk storage is inevitable
12:38 < sipa> and we're better off explicitly creating a new db
12:38 < wumpus> sipa: I don't know either, could try it again if you think it's a fluke...
12:38 < sipa> i'll test it myself again too
12:38 < gmaxwell> I tested an earlier range compacting patch and it did work.
12:39 < gmaxwell> I'll also test.
12:39 < sipa> monitor disk usage during upgrade, with and without compactrange patch
12:39 < sipa> would be nice
12:39 < sipa> and maybe this discussion depends on the outcome there
12:39 < gmaxwell> if indeed compacting isn't reliable we should change to create a new database. IMO
12:40 < sipa> agree
12:40 < wumpus> it might have to do with my test framework, I'll copy the database directory next time instead of using my usual hardlink hack
12:41 < gmaxwell> (FWIW, the reason I checked compacting before is because I was going to suggest changing to a new database if it didn't work.)
12:41 < wumpus> other topics?
12:41 < sipa> crc32 leveldb 1.20?
12:41 < gmaxwell> ^     
12:41 < wumpus> #topic crc32 leveldb 1.20
12:41 < jtimon> gmaxwell: agreed on cost/benefit for supporting interruption, much easier to require confirmation with a warning "this may take a while and cannot be interrupted" or something
12:42 < gmaxwell> sipa: Why is travis failing?
12:42 < sipa> i personally really dislike the approach they're using (which seems to be the advised way even), which requires a separate object compiled with different flags
12:42 < sipa> gmaxwell: i assume incompatibility with our own build system integration of leveldb
12:42 < wumpus> compiling a separate object with special flags is  pretty much the standard way of doing it
12:42 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 255 seconds]     
12:42 < sipa> they're even abusing it... they're calling the new object without knowing that the CPU supports it
12:43 < wumpus> ok, that's not good
12:43 < gmaxwell> Compiling a new object with different flags is standard and correct,  calling it without knowing, is wrong.
12:43 < wumpus> yes     
12:43 < wumpus> the detection function should not be in that object
12:43 < sipa> the approach in #10377 is so much easier 
12:43 < gribble> https://github.com/bitcoin/bitcoin/issues/10377 | Use rdrand as entropy source on supported platforms by sipa · Pull Request #10377 · bitcoin/bitcoin · GitHub
12:43 < wumpus> simple as that
12:44 < wumpus> that only works because you don't spell out the instruction but put it in binary, if you spelled out the instruction it would have to be compiled with a special flag
12:44 < wumpus> you can't use e.g. sse4.2 instructions if the object isn't compiled with that
12:45 < gmaxwell> IIRC MSVC has banned inline ASM, if we don't care about MSVC then we're free to do other things.  However, you need to do the object thing if you want to use code that accesses simd via intrensics, for something like rdrand or clmul it's no big deal to use asm.
12:45 < sipa> ok ok
12:45 < wumpus> gmaxwell: yes, intrinsics are more compatible
12:45 < cfields_> here!
12:46 < gmaxwell> so we need to fix leveldb to now call into that object without having done the detection. Anything else needed there?
12:46 < wumpus> I don't think leveldb's way is wrong, except for putting the detection function in the instruction-set specific object
12:47 < BlueMatt> oh, i was supposed to mention cfields_ would be late
12:47 < cfields_> heh, thanks
12:47 < jtimon> maybe we can open an issue on their repo or something?
12:47 < BlueMatt> you're welcome :)
12:47 < sipa> jtimon: the issue is in our own build system integration, i'm sure
12:47 < sipa> not upstream
12:48 < sipa> oh, you mean for calling the machine specific object? yeah
12:48 < wumpus> wouldn't upstream have the same problem then?
12:48 < gmaxwell> jtimon might have been referring to the detection in the wrong object, thats just wrong.
12:48 < sipa> yeah, agree
12:48 < sipa> if they ever look at issues...
12:48 < jtimon> gmaxwell: sipa right
12:48 < gmaxwell> we should submit a fix, it should be trivial.
12:48 < cfields_> that's done already: https://github.com/theuni/bitcoin/commit/2cb7dda13884e44105f33c16e7e2c1a9aed46295
12:48 < wumpus> yes better to submit a fix, submitting an issue likely won't help
12:48 < sipa> cfields_: oh!
12:48 < cfields_> or are you guys talking about something else?
12:48 < sipa> probably not
12:48 < sipa> let's try
12:49 < gmaxwell> okay, we have actions here.
12:49 < wumpus> lol <long discussion> oh, cfields did it already
12:49 < cfields_> that enables the runtime detection and separate object, but iirc there's still a little generic code that may end up with the wrong instructions
12:49 < sipa> cfields_: yup...
12:50 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 258 seconds]     
12:50 < gmaxwell> that just needs to be moved into another file, I expect.
12:50 < sipa> yup     
12:50 < sipa> let's do that independently
12:50 < wumpus> yes, move all the code out of it except for the function itself
12:50 < gmaxwell> should be a trivial patch we can take locally and send upstream.
12:50 < cfields_> i even linked it on the PR! :)
12:50 < cfields_> heh, sorry I was late. Movers showed up at 3:00 exactly :\
12:51 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev     
12:51 < gmaxwell> We'll need to do the same thing for SSE4 (and sha2 instruction) sha2.
12:51 < jtimon> cfields_: that doesn't seem related the meeting started at 21:00 :p
12:51 < wumpus> from what I remember that's actual assembly code from intel, in .s format, not using intrinsics
12:52 < gmaxwell> ah. okay!
12:52 < luke-jr> jtimon: 3:00 is meeting time for east USA
12:52 < jtimon> luke-jr: sure, bad joke, sorry
12:52 < cfields_> i suspect I missed this discussion too, but I have intrinsics done for sha2
12:52 < gmaxwell> I usually expect corporate code to be intrinsics because having good performance is not enterprise enough. :P
12:52 < wumpus> https://github.com/laanwj/bitcoin/commit/7e3e717892d96cae7f05dd1b6742425a298cc12e
12:53 < wumpus> it's even in yasm format
12:53 -!- ajd_ [~Anthony@91.239.96.138] has quit [Ping timeout: 240 seconds]     
12:53 < cfields_> wumpus: ah, i'll shut up :)
12:53 < gmaxwell> I'm very exicited about getting the faster hash in, with all our other improvements, I'm now expecting a 8% IBD speedup. 
12:53 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has quit [Remote host closed the connection]     
12:53 < gmaxwell> if not more.
12:53 < wumpus> cfields_: you mean sha-specific instructions? the stuff I quote is from before that
12:54  * jtimon was used to the other notation and used assemble with nasm, but it's pretty sure we don't want that
12:54 < cfields_> wumpus: yea, intrinsics for intel sha1/2
12:54 < wumpus> it just implements a SHA using SSE4/AVX, so it works on older architectures..
12:54 < sipa> jtimon: ha, cool
12:54 < sipa> wumpus: no license issues?
12:54 < jtimon> sipa: not cool, I should have learned with the at & t notation, no?
12:55 < gmaxwell> sipa: it's permissively licensed IIRC.
12:55 < cfields_> wumpus: ah, nice. I suspect that's where the leveldb discussion came from. We can re-use the build-time logic for sure.
12:55 < wumpus> sipa: it's either MIT or BSD
12:55 < sipa> jtimon: just swap the argument order :)
12:55 < wumpus> spot-the-license https://github.com/laanwj/bitcoin/blob/7e3e717892d96cae7f05dd1b6742425a298cc12e/src/crypto/sha256_avx1.asm#L10
12:55 < gmaxwell> Before the metting closes, I'd like to remind people that Matt's caching change #10192 is a 31% block connection speedup.  It's done, rebased current, and needs review.
12:55 < gribble> https://github.com/bitcoin/bitcoin/issues/10192 | Cache full script execution results in addition to signatures by TheBlueMatt · Pull Request #10192 · bitcoin/bitcoin · GitHub
12:56 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev     
12:56 < wumpus> #topic cache full script execution
12:56 < jtimon> sipa: yeah, I think that's mostly it, and some minor things I think, but I was too lazzy to translate the programs I had
12:57 < gmaxwell> I don't know that there is too much to say about 10192. It saves a lot of operations when checking if an already validated tx is cached.  It's fairly straight forward.
12:58 < sipa> yeah, let's do it
12:58 < wumpus> seems it had a lot of review, no (ut)ACKs though
12:58 < gmaxwell> I'm working on an ACK now. Just want other people to look, it sat for a long time needing rebase.
12:59 < wumpus> right
12:59 < morcos> on the list
12:59 < sdaftuar> +1     
12:59 < wumpus> it's already on the high priority for review list
12:59 < gmaxwell> The non-atomic flush stuff is also still outstanding, but I think it might get slightly changed based on todays per txo discussion.
12:59 < sipa> cfields_: yours commit (after trivial cherry pick) works
13:00 < wumpus> #endmeeting
13:00 < lightningbot> Meeting ended Thu Jun  8 20:00:33 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
13:00 < lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-08-19.00.html
13:00 < lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-08-19.00.txt
13:00 < lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-08-19.00.log.html
13:00 < gmaxwell> wumpus: thanks!
13:01 < luke-jr> #10512's travis failure is being weird :/
13:01 < gribble> https://github.com/bitcoin/bitcoin/issues/10512 | Rework same-chain from abusing DoS banning, to explicit checks by luke-jr · Pull Request #10512 · bitcoin/bitcoin · GitHub
13:01 < luke-jr> it seems like adding debugging logging is fixing the test
13:02 < sipa> ah, a heisenbu
13:02 < sipa> +g     
13:02 < kanzure> heisentypu
13:02 < gmaxwell> aside, I have a fair amount of distaste for intrensics; they're slightly more portable but for a long time they had poor performance because they left register allocation up to the compliler and it usually did it wrong.    But newer compilers have been much better about register allocation around simd code. 
13:02 < cfields_> luke-jr: double the debug logging, close as fixed.
13:04 < luke-jr> oh wait, I'm wrong. *rubs eyes*
13:04 < luke-jr> forgot I had started a bisect last night
13:04 < jtimon> I have the feeling that some who didn't like #10339 may not like #8498 either for similar reasons
13:04 < gribble> https://github.com/bitcoin/bitcoin/issues/10339 | Optimization: Calculate block hash less times by jtimon · Pull Request #10339 · bitcoin/bitcoin · GitHub
13:04 < wumpus> for ARM32, intrinsics were pretty bad, apparently there were some changes in ARM64 arch making them better suited for optimization by compilers
13:04 < gribble> https://github.com/bitcoin/bitcoin/issues/8498 | Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
13:05 < jtimon> BlueMatt: morcos ^^
13:05 < gmaxwell> Sorry about earlier with jtimon caching patch,  I thought I had expressed the view that I thought the change was adding complexity-- on reread of my comment I can see that my effort to be more constructive (in the form of offering "why not just cache") backfired.
13:06 < gmaxwell> I don't really think the complexity it adds is that big a deal. 
13:06 < wumpus> but if reducing the number of header hashing operations isn't worth it in the bigger scheme of things, it's a boondoggle
13:06 < jtimon> gmaxwell: I think you were clear: you preferred a cache, but since there were more comments and you didn't say anything else, I assumed it wasn't a strong preference
13:06 < sipa> wumpus: i'm not convinced it's not worth it
13:07 < cfields_> sipa: i can patch up leveldb to move all ops to that file, if you're not already working on it
13:07 < jtimon> is there a benchmarking tutorial somewhere ?
13:07 < sipa> wumpus: but it will (if at all) only be relevant for blocks with pre-cached utxos, pre-checked sigs, already loaded...
13:07 < sipa> which is what matters for relay, not reindex
13:08 < wumpus> sipa: well for what it's worth it also helped with the number of hashes in reindex
13:08 < sipa> wumpus: i very much doubt it will be significant if you could all SHA256 operations (as opposed to just block header hashes)
13:08 < sipa> *count
13:08 < jtimon> yeah, it should reduce the number of hashes when validating a block
13:09 < jnewbery> luke-jr: I've noticed an issue with zmq_test.py not exiting correctly. #10555 fixes that and also adds a timeout for individual tests (which means travis should show more useful output if one of the tests hangs). Try rebasing your PR on that and see if it helps.
13:09 < gribble> https://github.com/bitcoin/bitcoin/issues/10555 | [tests] various improvements to zmq_test.py by jnewbery · Pull Request #10555 · bitcoin/bitcoin · GitHub
13:09 < wumpus> well the point of the pull was to reduce header hashes, so that's what I counted
13:09 < morcos> jtimon: I
13:09 < sipa> i'm sure it reduces header hashes (and thanks for the concrete evidence for that)... but there are probably 2 orders of magnitude more hashes in computing merkle trees
13:09 < wumpus> but I don't care, won't spend any more time on it at least
13:10 < sipa> :)     
13:10 < sipa> cfields_: that should go in our leveldb repo then
13:10 < sipa> ... or upstream
13:10 < morcos> jtimon: I'd need to look at 8498 more closely, but that one doesn't not seem to me to add that much complication.. it might be a nicer layout..  but i need to spend more time thinking it through
13:10 < cfields_> sipa: sure, first one, then the other
13:11 < gmaxwell> jtimon: it wasn't a strong preference, I'm okay with that PR if pieter is okay with it.  Though morcos seems to have stronger views.
13:11 < jtimon> morcos: thank you for the fast look, that already helps
13:12 < jtimon> gmaxwell: yeah, that's why I should learn to benchmark these things, I'm sure it isn't hard and it's just that I've never tried
13:12 < morcos> like i said, i personally don't love the direction that 10339 goes, but if other people have already done the review and like it, i don't think its worth further debate to stop it
13:13 < morcos> it seems like premature optimization that makes the code less easy to use to me, but not something terrible
13:13 < sipa> right - perhaps there is no noticable performance gain, in which case it's obviously not worth it
13:14 < sipa> i soft of assumed there would be, and if that's the case, i think passing along a hash of an object with the object itself is a very reasonable thing to do
13:14 < sipa> but perhaps all of this is premature
13:14 < wumpus> right
13:14 < wumpus> it's all up to finding a way to benchmark it then
13:15 < sipa> i believe we should perhaps postpone this until after #10192
13:15 < gribble> https://github.com/bitcoin/bitcoin/issues/10192 | Cache full script execution results in addition to signatures by TheBlueMatt · Pull Request #10192 · bitcoin/bitcoin · GitHub
13:15 < sipa> and then use the benchmark strategy that BlueMatt used there
13:16 < sipa> or use some replay that feeds transactions and blocks into bitcoind over p2p...
13:18 < jtimon> I'll go look that benchmarking stratedy
13:25 < gmaxwell> I think there is a simpler way to argue the performance: benchmark hashing a header, and then count how many hashes the change saves.  Total saved time is no more than the product. 
13:25 < wumpus> the playback would work using morcos's data
13:26 < jtimon> so how did you counted it was about 6 ?
13:27 < wumpus> jtimon: https://github.com/laanwj/bitcoin/commit/fcfd20ea01f413dd2aa19b0013fe2b7aa2e47ad8
13:28 -!- echonaut1 [~echonaut@46.101.192.134] has quit [Remote host closed the connection]     
13:32 -!- trippysa1mon [~trippy@cyberdynesys.org] has joined #bitcoin-core-dev     
13:33 -!- trippysalmon [~trippy@cyberdynesys.org] has quit [Remote host closed the connection]     
13:33 -!- echonaut [~echonaut@46.101.192.134] has joined #bitcoin-core-dev     
13:40 -!- nakaluna [~nakaluna@gateway/vpn/privateinternetaccess/nakaluna] has quit [Quit: -]     
13:45 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Quit: Leaving.]     
13:45 < jtimon> wumpus: thanks
13:51 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Remote host closed the connection]     
13:51 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
13:55 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Ping timeout: 240 seconds]     
14:05 < bitcoin-git> [bitcoin] morcos opened pull request #10559: Change semantics of HaveCoinInCache to match HaveCoin (master...haveunspentincache) https://github.com/bitcoin/bitcoin/pull/10559
14:10 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 240 seconds]     
14:16 -!- owowo [~ovovo@185.183.104.83] has joined #bitcoin-core-dev     
14:16 -!- owowo [~ovovo@185.183.104.83] has quit [Changing host]     
14:16 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev     
14:18 -!- Chris_Stewart_5 [~chris@173-31-39-168.client.mchsi.com] has quit [Ping timeout: 240 seconds]     
14:21 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:5514:bad8:6bd9:240e] has joined #bitcoin-core-dev     
14:25 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:bdaa:5de1:b10d:1512] has quit [Ping timeout: 246 seconds]     
14:26 < gmaxwell> Am I alone in thinking we should have more comments in the code and commit message vs PRs?   E.g. when I wanted to cite the multiply/shift hash trick for the mailing list, I first looked at the source-- no explination of what its doing,  then the commit message that added the change,  all it said was allow non-power-of-two cache sizes, -- the PR text had the citation I wanted https://github.com/
14:26 < gmaxwell> bitcoin/bitcoin/pull/9533
14:28 < wumpus> no, more (useful) source code comments would be nice
14:28 < wumpus> I'm always happy if people add them, or other documentation for that matter
14:29 < morcos> agreed, i've at least started to incororate most of my PR messages in the commits 
14:29 < morcos> but more source code commentary would be good too, although it sometimes gets a bit stale when a major PR changes a lot of pieces
14:30 < wumpus> yes that's the drawback, keeping comments up to date is extra maintenance work
14:30 < wumpus> ideal would be self-illustrating code
14:30 < wumpus> but uh, yeah
14:30 < luke-jr> gmaxwell: I agree
14:31 < luke-jr> instead of answering questions on PRs, we should interpret it as a bug that it's underdocumented ;)
14:31 < wumpus> I've tried, there's many times I've encouraged people to add a code comment instead of just mentioning the rationale in a PR/commit message
14:32 < wumpus> in any case being able to find it in the commit message or even PR is still better than if it was some discussion on IRC :)
14:34 < luke-jr> heh     
15:03 -!- so [~so@unaffiliated/so] has quit [Ping timeout: 240 seconds]     
15:04 -!- so [~so@unaffiliated/so] has joined #bitcoin-core-dev     
15:04 -!- haakonn [~haakonn@pdpc/supporter/active/haakonn] has quit [Ping timeout: 260 seconds]     
15:04 -!- haakonn [~haakonn@146.185.155.218] has joined #bitcoin-core-dev     
15:04 -!- haakonn is now known as Guest56170     
15:05 -!- vicenteH [~user@135.234.15.37.dynamic.jazztel.es] has quit [Ping timeout: 260 seconds]     
15:08 < bitcoin-git> [bitcoin] practicalswift closed pull request #10343: Remove redundant on-the-same-line-repetition of type names (DRY): RepeatedTypeName foo = static_cast<RepeatedTypeName>(bar) (master...auto) https://github.com/bitcoin/bitcoin/pull/10343
15:09 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]     
15:54 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 245 seconds]     
15:57 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev     
16:03 -!- ndr76 [5aae0202@gateway/web/freenode/ip.90.174.2.2] has quit [Quit: Page closed]     
16:11 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 260 seconds]     
16:21 < bitcoin-git> [bitcoin] practicalswift opened pull request #10560: Remove non-existing "force safe mode" (-testsafemode). Remove unused constants. (master...noop-modes) https://github.com/bitcoin/bitcoin/pull/10560
16:27 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:f9f2:acd6:1d53:dc03] has joined #bitcoin-core-dev     
16:30 -!- coredump_ [~quassel@vpn-qld171.vpnsolutions.com.au] has joined #bitcoin-core-dev     
16:30 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:5514:bad8:6bd9:240e] has quit [Ping timeout: 260 seconds]     
16:35 < bitcoin-git> [bitcoin] practicalswift opened pull request #10561: Remove duplicate includes (master...remove-duplicate-includes) https://github.com/bitcoin/bitcoin/pull/10561
16:50 -!- echonaut [~echonaut@46.101.192.134] has quit [Remote host closed the connection]     
16:50 -!- echonaut2 [~echonaut@46.101.192.134] has joined #bitcoin-core-dev     
16:53 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:ac15:9f56:15df:35de] has joined #bitcoin-core-dev     
16:57 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:f9f2:acd6:1d53:dc03] has quit [Ping timeout: 244 seconds]     
16:58 -!- cryptapus [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev     
16:58 -!- cryptapus is now known as cryptapus_afk     
16:59 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Textual IRC Client: www.textualapp.com]     
17:48 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:7c5b:aa13:d3a2:8f9f] has joined #bitcoin-core-dev     
17:52 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:ac15:9f56:15df:35de] has quit [Ping timeout: 246 seconds]     
18:07 < bitcoin-git> [bitcoin] achow101 opened pull request #10563: Remove safe mode (master...rm-safemode) https://github.com/bitcoin/bitcoin/pull/10563
18:20 -!- tunafizz [~tuna@c-71-207-56-72.hsd1.pa.comcast.net] has quit [Ping timeout: 260 seconds]     
18:20 -!- tunafizz [~tuna@c-71-207-56-72.hsd1.pa.comcast.net] has joined #bitcoin-core-dev     
18:35 -!- Chris_Stewart_5 [~chris@173-31-39-168.client.mchsi.com] has joined #bitcoin-core-dev     
18:35 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-odjefsqsnwswkqil] has quit [Quit: Connection closed for inactivity]     
18:43 -!- jtimon [~quassel@117.29.134.37.dynamic.jazztel.es] has quit [Ping timeout: 240 seconds]     
19:01 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev     
19:02 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 255 seconds]     
19:09 < bitcoin-git> [bitcoin] gmaxwell opened pull request #10564: Return early in IsBanned. (master...banned-early-term) https://github.com/bitcoin/bitcoin/pull/10564
19:13 -!- Chris_Stewart_5 [~chris@173-31-39-168.client.mchsi.com] has quit [Quit: WeeChat 1.4]     
19:14 < bitcoin-git> [bitcoin] achow101 opened pull request #10565: [coverage] Remove leveldb, univalue, and benchmarks from coverage report (master...lcov-remove-extra) https://github.com/bitcoin/bitcoin/pull/10565
19:31 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:9090:b4fa:8c23:c2e7] has joined #bitcoin-core-dev     
19:33 -!- goatturner [~Beatrootg@2a02:c7d:12e:100:a95f:7228:9d85:f3af] has joined #bitcoin-core-dev     
19:35 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:7c5b:aa13:d3a2:8f9f] has quit [Ping timeout: 245 seconds]     
19:35 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has joined #bitcoin-core-dev     
19:37 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:9090:b4fa:8c23:c2e7] has quit [Ping timeout: 260 seconds]     
19:46 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:2404:48f0:43d3:10b] has joined #bitcoin-core-dev     
19:50 -!- goatturner [~Beatrootg@2a02:c7d:12e:100:a95f:7228:9d85:f3af] has quit [Ping timeout: 260 seconds]     
20:06 -!- goatturner [~Beatrootg@2a02:c7d:12e:100:f596:b2c9:4774:2a60] has joined #bitcoin-core-dev     
20:09 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:2404:48f0:43d3:10b] has quit [Ping timeout: 260 seconds]     
20:21 -!- jamesob [~jamesob@c-73-241-180-136.hsd1.ca.comcast.net] has joined #bitcoin-core-dev     
21:16 -!- unholymachine [~quassel@2601:8c:c003:9f16:99b6:dc1b:fa94:4db8] has joined #bitcoin-core-dev     
21:39 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:a085:fcd6:3155:c01] has joined #bitcoin-core-dev     
21:43 -!- goatturner [~Beatrootg@2a02:c7d:12e:100:f596:b2c9:4774:2a60] has quit [Ping timeout: 245 seconds]     
21:53 -!- kadoban [~mud@unaffiliated/kadoban] has quit [Quit: bye]     
21:54 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:4069:8b5c:ff86:139d] has joined #bitcoin-core-dev     
21:57 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:a085:fcd6:3155:c01] has quit [Ping timeout: 246 seconds]     
22:26 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:a1cb:276e:d9c9:58c3] has joined #bitcoin-core-dev     
22:29 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:4069:8b5c:ff86:139d] has quit [Ping timeout: 260 seconds]     
22:39 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-lgpovuskkatafauh] has joined #bitcoin-core-dev     
22:41 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:fdc3:3701:4758:1d48] has joined #bitcoin-core-dev     
22:45 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:a1cb:276e:d9c9:58c3] has quit [Ping timeout: 255 seconds]     
23:00 -!- dermoth [~thomas@dsl-66-36-135-165.mtl.aei.ca] has quit [Read error: Connection reset by peer]     
23:01 -!- dermoth [~thomas@dsl-66-36-135-165.mtl.aei.ca] has joined #bitcoin-core-dev     
23:36 -!- [Author] [~Author]@2401:a400:3200:5600:bad:d09:15:90d] has quit [Ping timeout: 276 seconds]     
23:37 -!- [Author] [~Author]@2401:a400:3200:5600:bad:d09:15:90d] has joined #bitcoin-core-dev     
23:40 -!- coredump_ [~quassel@vpn-qld171.vpnsolutions.com.au] has quit [Ping timeout: 246 seconds]     
23:49 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 245 seconds]     
23:53 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Quit: Leaving]