--- Day changed Fri Nov 06 2015
00:00 < wumpus> it shouldn't - I think the timings are always done, just not logged normally
00:00 < wumpus> so any performance overhead will be in generating more log data
00:01 < gmaxwell> really chatty log messages do, but the debug bench log data is not that chatty.
00:01 < jonasschnelli> okay. thanks... i do no compare the IBD time of the secp256k1 branch with master (with standard parameters)
00:06 < jonasschnelli> *now
00:07 < phantomcircuit> are there windows builds for 6954 ?
00:09 < gmaxwell> As far as protecting all the data goes, --- assuming it could be compatible it would be useful, though otoh, it would be nice to keep the checksums for pubkeys around in memory so we could test against at point of use to reduce the window for corruption further.
00:12 < jonasschnelli> phantomcircuit: sure: https://bitcoin.jonasschnelli.ch/pulls/6954/
00:20 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
00:30 < gmaxwell> phantomcircuit: 3h 16m for the reindex on that host. 
00:30 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has quit [Quit: Leaving.]
00:33 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has quit [Ping timeout: 260 seconds]
00:42 -!- NLNico [~NLNico@unaffiliated/nlnico] has joined #bitcoin-core-dev
00:54 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has joined #bitcoin-core-dev
01:15 < arowser> gmaxwell: I got the the error: ‘EVENT_LOG_WARN’ when try to build master on ubuntu12.04, the libevent version is 2.0-5, and I found you got the same error in 10-21 https://botbot.me/freenode/bitcoin-core-dev/2015-10-21/?msg=52437191&page=2
01:16 < arowser> Is that a known issue
01:18 < wumpus> yes, that is a known issue, should be easy to solve by adding some ifdefs
01:20 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has quit [Ping timeout: 260 seconds]
01:48 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has joined #bitcoin-core-dev
02:11 -!- guest234234 [~guest2342@223.207.239.102] has quit [Ping timeout: 246 seconds]
02:16 -!- arowser_ [~quassel@45.32.248.113] has joined #bitcoin-core-dev
02:16 -!- arowser_ [~quassel@45.32.248.113] has quit [Remote host closed the connection]
02:21 -!- arowser_ [~quassel@45.32.248.113] has joined #bitcoin-core-dev
02:26 -!- CodeShark_ [~CodeShark@2600:380:4b43:bae5:fd25:a062:1823:9dda] has quit [Remote host closed the connection]
02:28 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Quit: Leaving]
02:37 -!- arowser_ [~quassel@45.32.248.113] has quit [Remote host closed the connection]
02:57 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev
02:57 -!- rubensayshi [~ruben@c89225.upc-c.chello.nl] has joined #bitcoin-core-dev
03:19 -!- JackH [~Jack@host-80-43-141-3.as13285.net] has joined #bitcoin-core-dev
03:28 < GitHub98> [bitcoin] MarcoFalke opened pull request #6961: luke-jr constants (master...luke-jr-const) https://github.com/bitcoin/bitcoin/pull/6961
04:13 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 252 seconds]
04:20 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev
04:26 < GitHub13> [bitcoin] MarcoFalke opened pull request #6962: translations: Don't translate markup or force English grammar (master...MarcoFalke-2015-translations) https://github.com/bitcoin/bitcoin/pull/6962
04:28 -!- arowser_ [~quassel@45.32.248.113] has joined #bitcoin-core-dev
04:30 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 252 seconds]
04:51 -!- ParadoxSpiral [~ParadoxSp@p508B8497.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
04:57 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev
04:58 -!- aburan28 [~xypher@static-108-45-93-78.washdc.fios.verizon.net] has quit [Ping timeout: 252 seconds]
05:01 -!- guest234234 [~guest2342@223.207.239.102] has joined #bitcoin-core-dev
05:08 < GitHub153> [bitcoin] laanwj closed pull request #6825: Backport bugfixes to 0.11 (2015-10-22 / f2c869a) (0.11...backport-bugfixes-to-0.11-20151014) https://github.com/bitcoin/bitcoin/pull/6825
05:08 < GitHub120> [bitcoin] laanwj pushed 18 new commits to 0.11: https://github.com/bitcoin/bitcoin/compare/df616ae43ed4...6c31ac019f1b
05:08 < GitHub120> bitcoin/0.11 9b9acc2 Diego Viola: Fix spelling of Qt
05:08 < GitHub120> bitcoin/0.11 01878c9 Alex Morcos: Fix locking in GetTransaction....
05:08 < GitHub120> bitcoin/0.11 b3eaa30 MarcoFalke: [Qt] Raise debug window when requested...
05:18 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Quit: Lost terminal]
05:33 < GitHub30> [bitcoin] laanwj pushed 1 new commit to 0.11: https://github.com/bitcoin/bitcoin/commit/4e895b08da5dc2dfa94ccefb24632062481a8d97
05:33 < GitHub30> bitcoin/0.11 4e895b0 Pieter Wuille: Always flush block and undo when switching to new file...
05:37 < sdaftuar> gmaxwell: sipa: off the top of your head, do you have a sense of the relative speed of calculating a sha256 hash versus performing a signature validation using secp?
05:38 < jgarzik> 256x2 itym?
05:39 < sdaftuar> yeah probably that is what i mean
05:41 < sdaftuar> actually it might not be what i mean, i think #6918 (which i'm analyzing performance on now) does a single sha256 if i'm reading the code right
05:52 -!- arowser_ [~quassel@45.32.248.113] has quit [Ping timeout: 246 seconds]
05:57 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
05:58 -!- MarcoFalke [8af602bb@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.187] has joined #bitcoin-core-dev
06:09 -!- treehug88 [~textual@static-108-30-103-59.nycmny.fios.verizon.net] has joined #bitcoin-core-dev
06:15 -!- molly [~molly@unaffiliated/molly] has quit [Read error: Connection reset by peer]
06:16 -!- molly [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
06:26 -!- MarcoFalke [8af602bb@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.187] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]
06:29 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
06:41 < morcos> wumpus: re my comment on flushing undo files.  i think i didn't realize that a reindex could pick up from where it left off.  so it still reads through all the block files, but if the hash is in mapBlockIndex it skips processing?
06:41 < wumpus> yes
06:41 < morcos> i was assuming the corruption happened after a reindex finished, but before the flush
06:41 < morcos> ok, thx
06:42 < wumpus> it used to remember where it was, but this is complicated by block files being possibly out of order, so now it scans from the beginning again after restart (but this goes pretty fast)
06:43 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Quit: Leaving.]
07:12 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
07:37 -!- dixson3 [~dixson3@cpe-72-182-110-15.austin.res.rr.com] has joined #bitcoin-core-dev
08:00 -!- danielsocials [~quassel@45.32.248.113] has quit [Remote host closed the connection]
08:24 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has quit [Ping timeout: 260 seconds]
08:27 -!- zooko [~user@2601:281:8001:26aa:2108:c69:413e:8d37] has joined #bitcoin-core-dev
08:28 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
08:34 -!- Naphex [~naphex@unaffiliated/naphex] has quit [Quit: leaving]
08:46 < jonasschnelli> IBD up to 382324 with secp256k1 branch took 9 hours... 
08:46 < jonasschnelli> standard parameters
08:47 < jonasschnelli> Quad Core Intel(R) Xeon(R) CPU E31245 @ 3.30GHz
08:47 < jonasschnelli> 16GB ram (but i did not change -dbcache)
08:50 -!- guest234234 [~guest2342@223.207.239.102] has quit [Ping timeout: 260 seconds]
08:50 < jonasschnelli> (now doing the same with current master [openssl], just curios how the performance benefit are with standard args on a standard system)
08:54 -!- danielsocials [~quassel@45.32.248.113] has quit [Ping timeout: 246 seconds]
09:03 -!- rubensayshi [~ruben@c89225.upc-c.chello.nl] has quit [Ping timeout: 252 seconds]
09:05 < gmaxwell> sdaftuar: a verify with libsecp256k1 as it's used in bitcoin core is 280 times slower than a single run of the sha256 compression function.  That PR ends up calling the sha256 compression function twice.
09:08 < gmaxwell> sdaftuar: I think at some point we should add a faster cryptgraphic hash to the source code base for these internal non-normative uses. (E.g. blake2 is a bit over 5x faster than sha2-256; though if the user has a not-yet-released cpu with a sha256 instruction, sha256 will be faster again).
09:10 < sdaftuar> gmaxwell: thanks.  i was investigating whether the use of sha256 in the PR might cause an unexpected slowdown...  after some more investigation i think my concern was misplaced, it looks like any "slowdown" is pretty minuscule, and i think i wasn't hitting the conditions under which the PR would provide expected speedups.  will keep testing...
09:10 < gmaxwell> sdaftuar: figuring out the if there was any tradeoff there is part of why pieter was measuring the cache performance with many sizes (because a small amount of hitrate increase offsets any constant cost).  Also, the remove operation in the current code ends up being pretty slow compared to the hashing, in fact.
09:11 < sdaftuar> because of map versus unordered_map?
09:13 < gmaxwell> Yes.
09:16 < gmaxwell> sipa: 6954 results--- so reindex with libsecp256k1,  without 6954 was 3hr 16 minutes, reindex with it was _2 hours_ 7 minutes.  Going to benchmark without signature validation.
09:17 < gmaxwell> UTXO in memory at the end was 4012 MB. 
09:35 < sipa> gmaxwell: so from 5486 MB to 4012 MB, because of 6914?
09:35 < sipa> gmaxwell: 6954 is libsecp validation, so i guess you mean 6914 instead?
09:38 < morcos> sipa: is 6914 something you think should be considered for 0.12.  i haven't really looked at it much, but i'd happily test it out if you think its a near term consideration?
09:38 < gmaxwell> sipa: oops yes. 
09:39 < sipa> morcos: i'm not sure what i can do more for it than get review
09:40 < morcos> gmaxwell's test seems to indicate there is a HUGE speed improvment, thats from allocation overhead? i think that would be something nice to test in my Connect Block benching
09:40 < morcos> sipa: understood, but it wasn't clear to me from your previous conversation with gmaxwell whether you even wanted to do 6914 or wanted to wait and do a bigger change
09:40 < sipa> morcos: as far as my reading of the specification is concerned, i think it's fully compatible with std::vector
09:40 < sipa> morcos: i don't think a bigger change is in scope 
09:41 < sipa> i mean... i'd like to change allocation of scripts to some pooled storage per block and tx
09:41 < gmaxwell> One third less sync time and 25% less memory consumption is pretty nice.  These results don't surprise me -- well, so I'm a little surprised that _just_ fixing the scripts did this, I would be less surprised if the whole cache entry were made a single allocation.
09:42 < morcos> sipa: yes thats what i was referring to, but that doesnt mean you don't want 6914 for now is what i'm clarifying, you still do want 6914
09:43 < morcos> gmaxwell, i was seeing a significant chunk of ConnectBlock time being just destroying the temporary cache at the end.  
09:44 < morcos> while i have you both here.  do you have any thoughts on how we could get rid of the 1 remaining database lookup for the coinbase in 6932
09:44 < morcos> i hate that you have to access the DB for every block just in case you're processing the 2 historical violations of BIP 30
09:46 < sipa> looking
09:46 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection]
09:46 < jgarzik> morcos, agreed
09:47 -!- fkhan [weechat@gateway/vpn/mullvad/x-etquhaflyhyuyfgu] has quit [Ping timeout: 260 seconds]
09:47 < jgarzik> that wants fixing (as you've proposed)
09:48 < morcos> maybe i'm trying to overoptimize, but i wasn't able to eliminate checking the database for preexisting coins when you are creating the new outputs for a new coinbase
09:53 < sipa> morcos: how about writing them and just not marking them as fresh?
09:53 < sipa> make ModifyNewCoins take a boolean fPossiblePreexisting or something
09:53 < morcos> that was my first plan, but it assert fails
09:54 < morcos> the code assumes that if you're not fresh you must exist in the parent
09:54 < sipa> i see
09:54 < sipa> maybe that can be relaxed
09:54 < morcos> i think you coudl remove that assert, but i didn't like making that change
09:57 < morcos> so the idea i've been hesitant to suggest is to actually check if its one of those two hashes
09:57 < sipa> yuck
09:57 < morcos> hence the hesitancy, but its really more clear as to why you're doing what you're doing
09:58 < morcos> otherwise you have to have a long winded explanation of why you treat a coinbase differently anyway
09:58 < morcos> also i assume way faster than having to not mark all coinbases as fresh or first check the database for them
09:59 < morcos> i don't know maybe not faster that not FRESH marking i guess
10:00 < morcos> anyway, lunch..  i'm happy to do whatever you guys are most comfortable with
10:00 < sipa> morcos: the only advantage of fresh is that it's removed from memory altogether before hitting disk, if it's spent before flush
10:01 -!- fkhan [weechat@gateway/vpn/mullvad/x-ghdxmcfiadssqrnd] has joined #bitcoin-core-dev
10:02 < morcos> sipa: right. so i guess that doesn't happen unless you have a big cache, with coinbases...   but even if it happens a small fraction of the time, a DB write has to be orders of magnitude more expensive than checking 2 hashes on every coinbase
10:03 -!- CodeShark [~androirc@40.135.239.174] has joined #bitcoin-core-dev
10:05 -!- Guest57961 [~pigeons@94.242.209.214] has quit [Ping timeout: 246 seconds]
10:06 -!- PaulCape_ [~PaulCapes@204.28.124.82] has quit [Quit: .]
10:08 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
10:09 < sipa> morcos: i think removing the assertion is fine
10:11 < sipa> gmaxwell: my sigcache performance at 320 MB (with validating everything) keeps improving (it's an exponentially decaying average with 0.99 factor per block, so ~half a day halflife), down to 6% now
10:12 -!- pigeons [~pigeons@94.242.209.214] has joined #bitcoin-core-dev
10:12 -!- pigeons is now known as Guest91590
10:14 < sipa> gmaxwell: feel like commenting on 6914?
10:16 < sipa> morcos: i guess in 6914 i can add more comments that refer to the specification and unit tests for the iterators
10:23 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev
10:36 < morcos> sipa: what size in MB would be equivalent to a 1M sigcache of the old version
10:37 < morcos> i tried 80MB which was my guess from the pull, and verification seemed to get a little faster, but overall run time slowed down
10:37 < sipa> morcos: it should be around 3 times smaller
10:37 < sipa> between 70 and 85 bytes per entry now
10:37 < sipa> around 220 before
10:44 < gmaxwell> sipa: will comment when I've through testing things with it.
10:45 < gmaxwell> reindex with 6914 and no signature checks, 1h 17m. 
10:48 -!- NLNico [~NLNico@unaffiliated/nlnico] has quit [Quit: Leaving]
10:50 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
10:51 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev
10:52 < morcos> sipa: did you benchmark adding things to the sigcache, or only looking them up?  I'm wondering if calling GetRand() that often is too expensive.  something slowed down for me for sure...
10:52 -!- JackH [~Jack@host-80-43-141-3.as13285.net] has quit [Ping timeout: 265 seconds]
10:55 -!- danielsocials [~quassel@45.32.248.113] has quit [Remote host closed the connection]
10:55 < gmaxwell> morcos: it GetRand's on evicition (and the initial initilization) but not otherwise.
10:56 < morcos> yeah i guess in the steady state it shouldn't be more than a couple evictions per new tx..   hmm.
11:01 -!- zooko [~user@2601:281:8001:26aa:2108:c69:413e:8d37] has quit [Ping timeout: 246 seconds]
11:01 < sipa> morcos: the eviction takes like a microsecond
11:02 < morcos> yeah i misremembered GetRand speed, but its a tad under a micro, i thought it was a few micros.  still maybe better to use insecure_rand(), but doesn't explain my performance issue
11:02 < sipa> morcos: in any case, the getrand is certainly the largest cpu time offender in the sigcache behaviour
11:02 < sipa> more than the hash
11:03 < morcos> i'm checking to see if i didn't match cache sizes, but i don't think thats the problem, since verification did speed up
11:03 < morcos> its the rest of the run time that slowed down... 
11:03 < sipa> do you measure this by running two nodes in parallel?
11:03 < sipa> or just long period of time averaging?
11:04 -!- zooko [~user@c-73-217-96-13.hsd1.co.comcast.net] has joined #bitcoin-core-dev
11:05 < sipa> morcos: where exactly are you seeing slowdown?
11:05 < morcos> total time to run a simulation over 7 days, increased by about 5%
11:06 < sipa> interesting!
11:06 < morcos> time spent in Connect Block had about a 10% decrease (but this number i've found has a lot of noise, i think due to the vagaries of the level db cache and/or whatever else is happening with my hard disk)
11:07 < morcos> btw, have you thought about the effect on reorgs
11:07 < morcos> reorgs are already incredibly slow, and now you've just lost the signature cache for every tx in the block
11:08 -!- zooko` [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
11:09 < morcos> maybe we can skip signature checking on blocks added back from a reorg?
11:09 < morcos> txs from the disconnected block added to the mempool that is
11:10 < morcos> eh, that won't help since a bunch of them will need to be checked if they are included inthe replacement block
11:10 -!- zooko [~user@c-73-217-96-13.hsd1.co.comcast.net] has quit [Ping timeout: 240 seconds]
11:12 -!- zooko [~user@50.141.118.162] has joined #bitcoin-core-dev
11:15 -!- zooko` [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Ping timeout: 264 seconds]
11:15 < morcos> sipa: looks like it might be GetRand().  just ran with insecure_rand and it was a 5% overall speedup instead.  i'm going to try again removing a bunch of extraneous debugging i have.
11:18 < sipa> morcos: oh, that's an optimization i've thought about before
11:18 < sipa> morcos: we actually store in the block index whether a block was verified already
11:19 < morcos> sipa: are you talking about something different?  a re-re-org?
11:19 < sipa> right
11:19 < sipa> morcos: it's strange that getrand makes things slower... the earlier version used way more entropy
11:19 < morcos> i'm just talking about regular reorg, where you add back all the txs from the disconnected block.   so now you have to process on the order of a few thousand txs with literally none of their sigs in the cahce
11:20 < sipa> by using GetRandBytes()
11:20 < sipa> morcos: one way to mitigate that would be to not remove sigcache entries during validation, but instead keep a list of txids added in each block in the past, and remove entries a few blocks deep
11:21 < sipa> eh, nevermind
11:21 < sipa> that would mean keeping a list of all signature checks
11:21 < morcos> wait why wouldnt' that work
11:22 < morcos> just keep a list of entries to remove
11:22 < sipa> it would work, but it's ugly, large, and a layer violation :)
11:22 < morcos> every new block add to the back of the list and remove from the front
11:22 < sipa> we could use greg's idea of annotating the sigcache entries with a sequence number
11:22 < sipa> and then after every block wipe things with a sequence number a few before
11:22 < morcos> as for GetRand().  you're right, that doesn't make sense that its slower
11:23 < morcos> so maybe the speedup from insecure_rand is real, but that still doesn't explain why it was slower in the first place
11:23 < sipa> can you try with a 5 MB cache?
11:23 < sipa> maybe the slowdown is just from larger working set
11:24 < cfields> morcos: something you might try, i discovered this last week when profiling new network code...
11:24 < morcos> the base case i was comparing against was a 1M entry cache
11:24 < sipa> 1M entries with old code?
11:24 < morcos> yes
11:25 < sipa> does it get filled?
11:25 < cfields> morcos: the secure_delete openssl allocator was taking ~25% of the time to process a transaction. Commenting that out made a massive difference in the profile
11:25 < cfields> not sure if that's artificial or if it actually surfaces in the real-world
11:25 < cfields> s/transaction/message/
11:25 < morcos> i didn't check, but over 7 days it accepts 1.3M txs to the mempool, so i would assume thats several million sigs
11:25 < sipa> cfields: wut
11:25 < cfields> sipa: take that with a grain of salt, it was a very artificial benchmark
11:26 < sipa> morcos: so an interesting thing i saw is that with master and a 300 M mempool, and delete-on-use-in-block sigcache, i never got higher than 944k sigcache entries
11:26 < sipa> cfields: what kind of profiling?
11:27 < morcos> sipa: that seems plausible, but the old code doesn't have delete on use
11:27 < cfields> sipa: gprof of echoing messages back/forth between nodes
11:27 < sipa> morcos: in fact, it stayed between 936k and 944k very consistently
11:27 < cfields> sipa: CDataStream is backed by the zeroafterfree alloc, so i suspect it'd be an issue in many other places
11:28 < sipa> cfields: yeah, that may be unnecessarily much
11:28 < cfields> i'd prefer if we were more conservative with where that was used. for non-sensitive data, it just seems wasteful
11:29 < sipa> indeed
11:29 < sipa> and network data should never be sensitive
11:30 < cfields> sipa: right. very likely that that particular case was highly inflated though. It was a worst-case of bouncing thousands of tiny messages around very quickly. Still goes to show the trend, though.
11:39 -!- PaulCape_ [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
11:41 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Ping timeout: 240 seconds]
11:50 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 246 seconds]
11:53 -!- BashCo [BashCo@gateway/vpn/mullvad/x-hjcgqioyaqmubkbf] has joined #bitcoin-core-dev
11:56 -!- zooko [~user@50.141.118.162] has quit [Ping timeout: 255 seconds]
12:05 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
12:07 -!- PaulCape_ [~PaulCapes@204.28.124.82] has quit [Ping timeout: 260 seconds]
12:29 -!- CodeShark_ [~androirc@40.135.239.174] has joined #bitcoin-core-dev
12:29 -!- CodeShark [~androirc@40.135.239.174] has quit [Read error: Connection reset by peer]
12:30 < morcos> sipa: apologies. that slowdown was an artifact.  i've run it a couple times now, and it is faster over all with 6918.  i'm still seeing 10% speedup in ConnectBlock, but then also an an overall speedup
12:30 -!- zooko [~user@2601:281:8001:26aa:2108:c69:413e:8d37] has joined #bitcoin-core-dev
12:30 < morcos> switching to insecure_rand offered incremental improvement over that, but nothing like what i thought i saw before
12:35 -!- CodeShark_ [~androirc@40.135.239.174] has quit [Ping timeout: 250 seconds]
12:39 < sipa> morcos: ok, good to know!
12:50 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Quit: Leaving]
12:52 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
13:01 -!- danielsocials [~quassel@45.32.248.113] has quit [Remote host closed the connection]
13:06 -!- ParadoxSpiral [~ParadoxSp@p508B8497.dip0.t-ipconnect.de] has quit [Remote host closed the connection]
13:09 -!- CodeShark_ [~androirc@38.104.224.174] has joined #bitcoin-core-dev
13:14 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]
13:15 -!- CodeShark_ [~androirc@38.104.224.174] has quit [Ping timeout: 260 seconds]
13:22 -!- CodeShark_ [~androirc@71-6-103-72.static-ip.telepacific.net] has joined #bitcoin-core-dev
13:36 -!- treehug88 [~textual@static-108-30-103-59.nycmny.fios.verizon.net] has quit [Quit: Textual IRC Client: www.textualapp.com]
13:59 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
14:21 -!- CodeShark_ [~androirc@71-6-103-72.static-ip.telepacific.net] has quit [Ping timeout: 265 seconds]
14:32 -!- CodeShark_ [~androirc@184-23-239-227.dedicated.static.sonic.net] has joined #bitcoin-core-dev
14:38 -!- zooko [~user@2601:281:8001:26aa:2108:c69:413e:8d37] has quit [Ping timeout: 246 seconds]
14:47 -!- danielsocials [~quassel@45.32.248.113] has quit [Ping timeout: 240 seconds]
15:13 < gmaxwell> 18 of the last 101 blocks have been v4.
15:14 < phantomcircuit> gmaxwell, oon mainnet?
15:15 < phantomcircuit> that was fast
15:16 -!- belcher [~user@unaffiliated/belcher] has joined #bitcoin-core-dev
15:18 < gmaxwell> cfields: the zeroafterfree allocator has the benefit of killing making some use after free or use-of-uninitilized memory vulnerabilities unexploitable.
15:19 < cfields> gmaxwell: mmm
15:21 < GitHub14> [bitcoin] TheBlueMatt opened pull request #6964: Benchmark sanity checks and fork checks in ConnectBlock (master...verify-commits-fixes) https://github.com/bitcoin/bitcoin/pull/6964
15:23 < gmaxwell> I dunno if its worth the cost, of course-- but it's not totally pointless.
15:24 < jgarzik> perhaps makes valgrind happier
15:25 < GitHub124> [bitcoin] TheBlueMatt opened pull request #6965: Benchmark sanity checks and fork checks in ConnectBlock (master...bench) https://github.com/bitcoin/bitcoin/pull/6965
15:28 < gmaxwell> nah. it's on free, won't make valgrind happier.
15:37 < phantomcircuit> gmaxwell, what's the cost?
15:52 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
15:55 < gmaxwell> phantomcircuit: memory bandwidth.
15:57 -!- danielsocials [~quassel@45.32.248.113] has quit [Ping timeout: 272 seconds]
16:01 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
16:48 -!- danielsocials [~quassel@45.32.248.113] has quit [Remote host closed the connection]
16:49 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
17:10 -!- danielsocials [~quassel@45.32.248.113] has quit [Ping timeout: 272 seconds]
17:14 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
17:24 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-dipefamoeiqhqgcr] has quit [Quit: Connection closed for inactivity]
17:28 < phantomcircuit> gmaxwell, well yes i was wondering if you had a % ?
17:28 < phantomcircuit> (of course this changes with the hardware, but a ballpark)
17:31 < gmaxwell> phantomcircuit: no clue, actually depends on other stuff going on. it wastes memory/tlb bandwidth. 
17:31 < gmaxwell> e.g. it's free if the next thing you're going to do is immediate run in a tight loop already in cache. :)
17:32 < gmaxwell> obvious thing to try would be to drop the zeroizing from the code and benchmark, but I've not done that (well not recently at least).
17:32 < gmaxwell> I don't think this has yet ever saved our bacon, but I've dealt with vulnerabilities elsewhere that would be made unexploitable due to zeroizing network buffers. 
17:33 < sipa> i'm generally in favor of saving bacon
17:35 < gmaxwell> Bitcoin Core used to have a number of other adhoc additional security measures which have been dropped.
17:36 < gmaxwell> E.g. there eas stack randomization via recursion for each thread.
17:37 < sipa> which was actually optimized out by modern compilers...
17:38 < gmaxwell> the zeroize on free could end up optimized out too (though I doubt it is currently!)
17:40 -!- baldur [~baldur@pool-173-52-43-219.nycmny.fios.verizon.net] has quit [Remote host closed the connection]
17:40 -!- baldur [~baldur@pool-173-52-43-219.nycmny.fios.verizon.net] has joined #bitcoin-core-dev
17:45 -!- guest234234 [~guest2342@171.5.150.155] has joined #bitcoin-core-dev
17:45 < phantomcircuit> it would be nice if compilers supported "dont optimize this block"
17:46 < sipa> phantomcircuit: not optimizing doesn't mean they suddenly have to respect stronger semantics
17:47 < gmaxwell> phantomcircuit: C(++) is _not_ a fancy macro assembler.
17:49 < gmaxwell> The language only promises to obey certian observable semantics of a particular abstract machine.  The language cannot be directly mapped to your instruction set without _some_ transformation, (though not much if you're targeting a PDP-11 :) ) ... so there will always be some difference between what the code explicitly says and what the machine does.
17:51 < sipa> PDP-11... didn't the y use braindead  byte ordering?
17:52 < sipa> Ah, wikipedia calls it "Middle-endian". Boring.
17:58 -!- danielsocials [~quassel@45.32.248.113] has quit [Remote host closed the connection]
18:00 -!- sipa [~pw@2a02:348:86:3011::1] has quit [Ping timeout: 264 seconds]
18:10 -!- Guest23423 [~guest2342@171.4.236.18] has joined #bitcoin-core-dev
18:12 -!- guest234234 [~guest2342@171.5.150.155] has quit [Ping timeout: 265 seconds]
18:23 -!- Guest23423 [~guest2342@171.4.236.18] has quit [Read error: Connection reset by peer]
18:26 -!- belcher [~user@unaffiliated/belcher] has quit [Quit: Leaving]
19:00 < Luke-Jr> [01:51:02] <sipa> PDP-11… didn't the y use braindead  byte ordering? <-- what are those symbols supposed to be?
19:02 < Luke-Jr> oh, it's a unicode DELETE :x
19:03 < jgarzik> speaking of unicode, thanks for the JSON issues wumpus 
19:23 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev
20:04 -!- challisto [~challisto@c-76-16-149-33.hsd1.il.comcast.net] has joined #bitcoin-core-dev
20:04 -!- challisto [~challisto@c-76-16-149-33.hsd1.il.comcast.net] has quit [Changing host]
20:04 -!- challisto [~challisto@unaffiliated/challisto] has joined #bitcoin-core-dev
20:25 < GitHub189> [bitcoin] pstratem opened pull request #6966: Cache CWalletDB pointer in CWallet to improve performance (master...wallet_speedup) https://github.com/bitcoin/bitcoin/pull/6966
20:31 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has quit [Ping timeout: 240 seconds]
20:36 -!- grubles [~freebsd@unaffiliated/grubles] has quit [Ping timeout: 272 seconds]
20:47 -!- CodeShark_ [~androirc@184-23-239-227.dedicated.static.sonic.net] has quit [Ping timeout: 240 seconds]
21:00 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Remote host closed the connection]
21:15 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-core-dev
23:12 < btcdrak> gmaxwell: sipa: petertodd: what do you think of this patch for the sequence numbers PR https://github.com/NicolasDorier/bitcoin/commit/5f24b6603407c78ae112ae82fd293ac24fbefb52
23:27 -!- ParadoxSpiral [~ParadoxSp@p508B9645.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
23:41 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Quit: .]
23:49 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-wdyhlvsnieowecnh] has joined #bitcoin-core-dev
23:50 -!- guest234234 [~guest2342@171.4.237.85] has joined #bitcoin-core-dev