--- Day changed Thu Aug 04 2016 00:13 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 00:19 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Excess Flood] 00:19 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #bitcoin-core-dev 00:23 -!- laurentmt [~Thunderbi@80.215.234.124] has joined #bitcoin-core-dev 00:27 -!- d_t [~textual@83-244-233-136.cust-83.exponential-e.net] has quit [Ping timeout: 258 seconds] 00:36 -!- jgarzik [~jgarzik@unaffiliated/jgarzik] has joined #bitcoin-core-dev 01:13 -!- d_t [~textual@83-244-233-135.cust-83.exponential-e.net] has joined #bitcoin-core-dev 01:25 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has joined #bitcoin-core-dev 01:50 < wumpus> phantomcircuit: nice! 01:53 < wumpus> sipa: the last update of secp256k1 has been ~9 months ago (6c527ec), is it maybe time to update the subtree again? At least the (experimental) ARM assembly has been merged since 01:54 < sipa> wumpus: good idea 01:55 < sipa> there are certainly no regressions known that would make the current master less appropriate than the subtree in bitcoin currently 01:56 < wumpus> exactly, I did mean for master, not 0.13 01:57 < sipa> sure 01:58 -!- Ayhan [4d24983c@gateway/web/freenode/ip.77.36.152.60] has joined #bitcoin-core-dev 02:03 -!- Giszmo [~leo@ppp-188-174-93-152.dynamic.mnet-online.de] has joined #bitcoin-core-dev 02:04 -!- Ayhan [4d24983c@gateway/web/freenode/ip.77.36.152.60] has quit [Ping timeout: 250 seconds] 02:08 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 02:18 -!- Lauda [~Lauda@2a06:8ec0:3::1:b224] has quit [Quit: I'm out!] 02:19 -!- Lauda [~Lauda@185.122.59.11] has joined #bitcoin-core-dev 02:25 < GitHub30> [bitcoin] laanwj opened pull request #8453: Bring secp256k1 subtree up to date with master (master...2016_08_update_secp256k1) https://github.com/bitcoin/bitcoin/pull/8453 02:43 -!- berndj [~berndj@197.242.93.84] has quit [Ping timeout: 260 seconds] 02:44 -!- NLNico [~NLNico@unaffiliated/nlnico] has quit [Quit: Leaving] 02:48 -!- aureianimus_ [~quassel@s55963df3.adsl.online.nl] has joined #bitcoin-core-dev 03:07 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has joined #bitcoin-core-dev 03:21 < GitHub18> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/5c7a5e1f66d8...37d83bb0a980 03:21 < GitHub18> bitcoin/master 122786d NicolasDorier: Consensus: Remove ISM 03:21 < GitHub18> bitcoin/master 37d83bb Wladimir J. van der Laan: Merge #8391: Consensus: Remove ISM... 03:21 < GitHub105> [bitcoin] laanwj closed pull request #8391: Consensus: Remove ISM (master...remove-ism) https://github.com/bitcoin/bitcoin/pull/8391 03:22 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has quit [Quit: Leaving.] 03:26 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has left #bitcoin-core-dev [] 03:34 < GitHub46> [bitcoin] laanwj pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/37d83bb0a980...f97d335942ae 03:34 < GitHub46> bitcoin/master 0fd2a33 Pieter Wuille: Use a signal to continue init after genesis activation 03:34 < GitHub46> bitcoin/master aa59f2e Pieter Wuille: Add extra message to avoid a long 'Loading banlist' 03:34 < GitHub46> bitcoin/master 9d4eb9a Pieter Wuille: Do diskspace check before import thread is started 03:34 < GitHub154> [bitcoin] laanwj closed pull request #8392: Fix several node initialization issues (master...fixactivatewait) https://github.com/bitcoin/bitcoin/pull/8392 03:42 -!- d_t [~textual@83-244-233-135.cust-83.exponential-e.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 03:46 -!- rubensayshi [~ruben@82.201.93.169] has joined #bitcoin-core-dev 03:49 -!- laurentmt [~Thunderbi@80.215.234.124] has quit [Read error: Connection reset by peer] 03:52 -!- Cory [~C@unaffiliated/cory] has quit [Ping timeout: 265 seconds] 03:53 < paveljanik> wumpus, do you use gmp? string.h is included in num_gmp_impl.h... This could be difference. 03:54 < wumpus> yes, that could be it 03:55 < wumpus> will try with bignum=no 03:55 < wumpus> yes, that does it 03:56 -!- laurentmt [~Thunderbi@80.215.234.124] has joined #bitcoin-core-dev 03:57 -!- Cory [~C@unaffiliated/cory] has joined #bitcoin-core-dev 04:02 < wumpus> paveljanik: https://github.com/bitcoin-core/secp256k1/pull/410 04:12 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has quit [Quit: Leaving.] 04:14 -!- laurentmt [~Thunderbi@80.215.234.124] has quit [Remote host closed the connection] 04:15 -!- laurentmt [~Thunderbi@80.215.234.124] has joined #bitcoin-core-dev 04:17 < paveljanik> thanks! 04:31 < GitHub189> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f97d335942ae...6e6ab2c32382 04:31 < GitHub189> bitcoin/master 2c517b3 Suhas Daftuar: Fix p2p-feefilter.py for changed tx relay behavior 04:31 < GitHub189> bitcoin/master 6e6ab2c Wladimir J. van der Laan: Merge #8444: Fix p2p-feefilter.py for changed tx relay behavior... 04:31 < GitHub87> [bitcoin] laanwj closed pull request #8444: Fix p2p-feefilter.py for changed tx relay behavior (master...fix-feefilter-test) https://github.com/bitcoin/bitcoin/pull/8444 04:35 < paveljanik> wumpus, in fact, the fix is not upstream, but side-stream ;-) 04:35 < wumpus> well I hope it can be merged soon, I'll update #8453 after that 04:36 < wumpus> bah I'd really feel better without expat in the depends :) 04:37 -!- cjcj [2e3b026a@gateway/web/freenode/ip.46.59.2.106] has joined #bitcoin-core-dev 04:43 -!- cryptapus [~cryptapus@87.254.202.210] has joined #bitcoin-core-dev 04:43 -!- cryptapus [~cryptapus@87.254.202.210] has quit [Changing host] 04:43 -!- cryptapus [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev 04:54 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 05:02 -!- berndj [~berndj@197.242.93.84] has joined #bitcoin-core-dev 05:26 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 05:26 < GitHub129> [bitcoin] MarcoFalke opened pull request #8454: [0.13.1] Fix p2p-feefilter.py for changed tx relay behavior (0.13...Mf1608-qaFeeFilter013) https://github.com/bitcoin/bitcoin/pull/8454 05:40 < GitHub71> [bitcoin] laanwj pushed 1 new commit to 0.13: https://github.com/bitcoin/bitcoin/commit/d485a6c5a8d238cac5ad732ce9e744f4b121143c 05:40 < GitHub71> bitcoin/0.13 d485a6c Wladimir J. van der Laan: doc: Add list of new and removed RPC commands to release notes... 05:41 < wumpus> sipa: do you still plan on writing something for the release notes about -reindex-chainstate? 05:45 < wumpus> I've added a list of added and removed and changed RPC calls, which means that #7678 can almost be closed, leaving -maxtimeadjust and -reindex-chainstate, of which I think reindex-chainstate (and the changed reindexing in general) is most relevant 05:56 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)] 05:59 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 05:59 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 250 seconds] 06:04 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 06:06 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 06:06 -!- BashCo_ [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 06:29 -!- zooko [~user@2601:281:8000:8387:e144:a18c:9f1:1fb] has joined #bitcoin-core-dev 06:33 -!- laurentmt1 [~Thunderbi@80.215.138.80] has joined #bitcoin-core-dev 06:34 -!- laurentmt [~Thunderbi@80.215.234.124] has quit [Ping timeout: 258 seconds] 06:34 -!- laurentmt1 is now known as laurentmt 06:52 < GitHub198> [bitcoin] mrbandrews opened pull request #8456: [RPC] Simplified bumpfee command. (master...ba-rpcbumpfee) https://github.com/bitcoin/bitcoin/pull/8456 07:00 -!- mkarrer [~mkarrer@142.red-83-47-107.dynamicip.rima-tde.net] has quit [Remote host closed the connection] 07:00 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 07:01 -!- mkarrer [~mkarrer@142.red-83-47-107.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 07:05 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 07:20 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has joined #bitcoin-core-dev 07:33 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 08:00 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 08:02 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 08:07 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 08:35 -!- aalex [~aalex@64.187.177.58] has joined #bitcoin-core-dev 08:53 -!- BashCo_ [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 08:54 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 09:02 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 250 seconds] 09:03 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 09:04 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection] 09:08 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 09:17 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 09:24 < GitHub160> [bitcoin] pedrobranco opened pull request #8457: Add block height support in rpc call getblock (master...feature/add-get-block-by-number) https://github.com/bitcoin/bitcoin/pull/8457 09:30 -!- rubensayshi [~ruben@82.201.93.169] has quit [Remote host closed the connection] 09:30 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 09:32 -!- saurb [6c1d5fa4@gateway/web/freenode/ip.108.29.95.164] has joined #bitcoin-core-dev 09:33 -!- saurb [6c1d5fa4@gateway/web/freenode/ip.108.29.95.164] has quit [] 09:41 -!- jtimon [~quassel@55.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 09:45 < sipa> wumpus: ack, will write 09:46 -!- laurentmt [~Thunderbi@80.215.138.80] has quit [Quit: laurentmt] 10:03 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Remote host closed the connection] 10:05 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 10:06 -!- zooko [~user@2601:281:8000:8387:e144:a18c:9f1:1fb] has quit [Ping timeout: 250 seconds] 10:08 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 10:10 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 10:26 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 10:27 -!- shaiguit1r [~rosenfs@shairosenfeld.com] has joined #bitcoin-core-dev 10:30 -!- shaiguitar [~rosenfs@shairosenfeld.com] has quit [Ping timeout: 260 seconds] 10:46 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 264 seconds] 10:52 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 11:07 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 11:08 < instagibbs> during salvage on an unbroken wallet.dat file I'm seeing keys getting skipped 11:08 < instagibbs> on master 11:08 < instagibbs> 0-4 keys on a fresh wallet with keypool 100, randomly 11:09 < gmaxwell> I've said before that salvagewallet should be called "savagewallet". 11:09 < gmaxwell> instagibbs: can you figure out why? 11:09 < instagibbs> no ive been trying to chase it down all day 11:10 < instagibbs> it's very random, and only happens once every 25+ keys 11:10 < gmaxwell> presumably you've filled up the salvage code with instrumentation to see where they're getting dropped? (e.g. does bdb just never hand them to us?) 11:11 -!- gabridome [~gabridome@host254-183-static.123-81-b.business.telecomitalia.it] has joined #bitcoin-core-dev 11:11 < instagibbs> it's an error thrown when doing >> for private key 11:11 < instagibbs> "key" type, trying to deserialize the key 11:11 < instagibbs> priv* 11:11 < sipa> is it trying to deserialize master key data as the wrong type? 11:11 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 11:13 < instagibbs> I don't think so, as keypool=1 almost never has it happen 11:14 < instagibbs> I've never had it happen with keypool 1, that is to say 11:16 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 244 seconds] 11:19 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 11:20 -!- jtimon [~quassel@55.31.134.37.dynamic.jazztel.es] has quit [Ping timeout: 260 seconds] 11:20 -!- gabridome [~gabridome@host254-183-static.123-81-b.business.telecomitalia.it] has quit [Read error: Connection reset by peer] 11:20 -!- telelvis [~Adium@2a02:810d:a840:884:b016:fae6:585b:13e3] has joined #bitcoin-core-dev 11:22 < wumpus> instagibbs: can you upload the wallet somewhere that it happens with? 11:23 < instagibbs> it's literally a fresh wallet, start up bitcoind in regtest, stop, start back up with salvagewallet. Some not small % of time i get those messages 11:23 < instagibbs> but yes I can upload one 11:23 < wumpus> it's a known issue btw: salvagewallet drops valid keys #7379 but I've never had it happen to me 11:24 < wumpus> another similar problem is salvagewallet fails verification #7463 , salvagewallet on an uncorrupted wallet sometimes makes bdb return errors 11:24 < wumpus> so what you are seeing may be similar, I noticed that problem, just never saw keys being dropped 11:25 < instagibbs> 2016-08-04 18:07:42 Salvage(aggressive) found 439 records 11:25 < instagibbs> 2016-08-04 18:07:42 WARNING: CWalletDB::Recover skipping key: 11:25 < wumpus> I would not be surprised if bdb's "aggressive" salvage sometimes returns crap or misses records 11:25 < instagibbs> oh is it an actual bdb mode? That might explain the difference 11:27 < gmaxwell> maybe it should run twice, once normally without any bdb magig, and then again in agressive mode? 11:27 < gmaxwell> "never do worse than simply reading it." 11:27 < wumpus> I don't know if that makes a difference, but worth a try 11:27 < instagibbs> i can try that, if i figure out the settings 11:27 < gmaxwell> well presumably it must, after all we don't miss those keys during normal reads without salvaging. 11:29 < wumpus> ooh! those may be ghost keys instagibbs 11:29 < wumpus> "Output all the key/data pairs in the file that can be found. By default, DB->verify() does not assume corruption. For example, if a key/data pair on a page is marked as deleted, it is not then written to the output file. When DB_AGGRESSIVE is specified, corruption is assumed, and any key/data pair that can be found is written. In this case, key/data pairs that are corrupted or have been deleted may appear in the output (even 11:29 < wumpus> if the file being salvaged is in no way corrupt), and the output will almost certainly require editing before being loaded into a database." 11:29 < instagibbs> i aint afraid of no ghost 11:29 < wumpus> aggressive should return *more* records, but some of them may be deleted, or not really exist at all 11:29 < cfields> sipa: mm, I just noticed that 0.13 doesn't enable asm for osx due to secp256k1 #373 :\ 11:30 < wumpus> *even if the file being salvaged is in no way corrupt* is the important part there 11:30 < instagibbs> so how can we test that 11:31 < wumpus> so yes, trying to salvage without aggressive makes sense first 11:32 < wumpus> but how can we detect if keys have been dropped due to corruption, which would appear with salvage aggressive? I don't know 11:32 < wumpus> yes you don't really know, that's the problem 11:32 < wumpus> was it a real key or just a corrupted echo? 11:33 < wumpus> cfields: maybe do the secp256k1 update for 0.13.1 too then 11:34 < instagibbs> so worst case with "ghost" keys would be having keys you didn't actually generate? or? 11:34 < wumpus> instagibbs: or partial old records 11:34 -!- telelvis1 [~Adium@corp.inetgate.hzo.de.adidas-group.com] has joined #bitcoin-core-dev 11:34 < cfields> wumpus: probably makes sense, yes. I'm running a bench with/without atm. 11:34 < wumpus> instagibbs: it does a linear scan over the file and everything that looks like a key/value is returned 11:35 < instagibbs> so aggressive will indeed return more, but we toss anything that looks wrong 11:35 < paveljanik> Trivial -Wshadow PRs for review: 8109, 8163, 8191, 8449. I'd be glad to get some ACKs. Thank you. 11:35 < wumpus> instagibbs: yes, we do, we're fairly robust there. But it means possible false positives inthe log 11:35 -!- telelvis [~Adium@2a02:810d:a840:884:b016:fae6:585b:13e3] has quit [Ping timeout: 260 seconds] 11:36 < instagibbs> wumpus, ok, well that's less scary, thanks 11:36 < wumpus> instagibbs: I hope that's it! but you can only find out by analysing that wallet\ 11:37 < wumpus> see if the keys that are in the original wallet and post-salvage wallet are the same, despite the WARNING 11:37 < sipa> wumpus: i'll merge the strings include in secp 11:37 < wumpus> sipa: thanks 11:38 < sipa> so that can be included in the subtree 11:38 < wumpus> right, I'll update the pr 11:38 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has quit [Quit: Leaving.] 11:38 < cfields> 116us vs 124us. not the end of the world. 11:38 -!- telelvis [~Adium@ip5f5add75.dynamic.kabel-deutschland.de] has joined #bitcoin-core-dev 11:39 -!- telelvis1 [~Adium@corp.inetgate.hzo.de.adidas-group.com] has quit [Ping timeout: 258 seconds] 11:39 < wumpus> don't think that'd be worth it for any random warning, but missing prototypes are pretty serious business 11:40 < sipa> cfields: that's the output of bench_verify ? 11:40 < gmaxwell> cfields: I'm surprised by that. 11:40 < gmaxwell> the difference is smaller than I expected. 11:40 < sipa> likewise 11:40 < cfields> sipa: yes 11:41 < sipa> what compilation flags? 11:41 < cfields> sipa: default passed down from bitcoin 11:42 < wumpus> 116 versus 124 seems in the error margin, are you sure something changed at all? 11:42 < cfields> sipa: http://pastebin.com/raw/VbmBERA8 11:43 < cfields> ran a few times of each, didn't deviate much. double-checking. 11:43 < wumpus> okay 11:44 < gmaxwell> cfields: what cpu is this? 11:46 < cfields> gmaxwell: i5, 2.4ghz. there's no /proc/cpuinfo in osx, i can dig around and find it if you'd like the gen 11:47 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has joined #bitcoin-core-dev 11:48 < sipa> cfields: reproduced 11:48 < cfields> there we go. 4258U. haswell. 11:48 < sipa> 112us vs 117us here 11:48 < sipa> ecdsa_verify: min 117us / avg 117us / max 117us 11:48 < sipa> ecdsa_verify: min 112us / avg 113us / max 113us 11:49 < jonasschnelli> cfields: OSX: use "sysctl -n machdep.cpu.brand_string" instead cat /proc/cpuinfo 11:49 < cfields> sipa: in linux? or reproduced on mac? 11:49 < sipa> linux 11:49 < sipa> with cpu clock pinned to a single frequency 11:49 < GitHub127> [bitcoin] MarcoFalke pushed 2 new commits to 0.13: https://github.com/bitcoin/bitcoin/compare/d485a6c5a8d2...114f7e944b1c 11:49 < GitHub127> bitcoin/0.13 cd0910b Suhas Daftuar: Fix p2p-feefilter.py for changed tx relay behavior... 11:49 < GitHub127> bitcoin/0.13 114f7e9 MarcoFalke: Merge #8454: [0.13.1] Fix p2p-feefilter.py for changed tx relay behavior... 11:49 < GitHub182> [bitcoin] MarcoFalke closed pull request #8454: [0.13.1] Fix p2p-feefilter.py for changed tx relay behavior (0.13...Mf1608-qaFeeFilter013) https://github.com/bitcoin/bitcoin/pull/8454 11:50 < cfields> huh. 11:50 < gmaxwell> sipa: that looks broken, those numbers are much higher than I expect. 11:50 < gmaxwell> (for both of you) 11:50 < sipa> that's without gmp or endomorphism, at 2.6 GHz 11:50 < cfields> gmaxwell: remember no bignum or endo 11:50 < gmaxwell> I normally expect a verify to be more like 70us. but maybe thats all gmp. 11:51 < cfields> jonasschnelli: thanks 11:52 < wumpus> MarcoFalke: please don't merge anything for 0.13 now, we have to decide on the meeting whether to release final, this is a test change but I don't want to have to trigger a new rc without good reason 11:52 < MarcoFalke> This should not be enough for a new rc 11:52 < wumpus> I know, but you named it 0.13.1, and it's not time to merge for 0.13.1 yet, just clarifying 11:53 < MarcoFalke> oops 11:53 * gmaxwell predicts a build system problem. 11:53 < gmaxwell> (in libsecp256k1) 11:53 < wumpus> (people may still want to do e.g. changelog changes before tagging final) 11:53 < gmaxwell> cd ~ 11:54 < cfields> gmaxwell: for which? 11:54 < wumpus> bitcoin is always without gmp 11:54 < sipa> gmaxwell: sounds likely... with endo and gmp enabled is see no statistically significant performance difference between asm enabled and disabled 11:54 < sipa> 71.2us vs 71.4us 11:55 < sipa> (with variance between tests exceeding 0.2us) 11:57 < gmaxwell> Aside, I think in the GUI and logs we should display the verification performance... so people will better realize how radically difference in speed different machines are. :) 11:58 < wumpus> yes, a graph of verification performance in the gui would be nice 11:59 < wumpus> similar to the bandwidth one 11:59 < wumpus> with part spent in i/o secp256k1 etc 11:59 < jonasschnelli> heh.. currently working on a mempool graph 11:59 < wumpus> nice 12:00 < jonasschnelli> http://i.imgur.com/G6yi9UR.png (very basic atm) 12:00 < gmaxwell> wumpus: Well I meant as a static this computer x faster than that computer. not usage. 12:00 < gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier 12:00 < wumpus> #startmeeting 12:00 < lightningbot> Meeting started Thu Aug 4 19:00:43 2016 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:00 < lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic. 12:01 < cfields> gmaxwell: mm, i just stuck some "#error foo" in the asm paths, it blows up as expected. 12:01 < cfields> whoops, will continue after meeting 12:01 < MarcoFalke> topic 0.13.0, I guess? 12:01 < wumpus> #topic 0.13.0 final? 12:01 < wumpus> did anyone hear of any critical problems with rc2? 12:01 < gmaxwell> I haven't heard much on it. Been quiet. 12:02 < sipa> we forgot to backport 8438/8365 into 0.13... 12:02 < wumpus> sipa something for 0.13.1? 12:02 < kanzure> hi. greetings. 12:02 < luke-jr> wumpus: the release notes are still inappropriate AFAIK 12:03 < luke-jr> also gmaxwell was going to add a section I believe, covering the new relay policy influences (maybe I missed that being added) 12:03 < jeremyrubin> hi 12:03 < gmaxwell> I think I owed some release note text which I haven't done yet re max blocksize settings. 12:03 < kanzure> this isn't short-term development related but it's high signal-noise crypto content that a few developers recently participated in, http://diyhpl.us/wiki/transcripts/2016-july-bitcoin-developers-miners-meeting/dan-boneh/ 12:03 < MarcoFalke> Wouldn't 8438 require rc3? 12:03 < wumpus> luke-jr: then update them, sipa also still wants to add something 12:03 < wumpus> MarcoFalke: yes 12:03 < luke-jr> wumpus: my PR to update them is still open. 12:03 < wumpus> luke-jr: it also updates code 12:03 < jonasschnelli> luke-jr: does the PR still contain code changes? 12:03 -!- telelvis [~Adium@ip5f5add75.dynamic.kabel-deutschland.de] has quit [Quit: Leaving.] 12:03 < luke-jr> so you want a version without code changes? 12:04 < wumpus> luke-jr: if you make a PR that just updates the changelog, and it's accepted by others, it would have been merged a long time ago 12:04 < luke-jr> wumpus: the current language was not accepted. 12:04 < jonasschnelli> sipa: do you think https://github.com/bitcoin/bitcoin/pull/8438 can wait for 0.13.1? 12:05 < luke-jr> I find this double-standard somewhat annoying. 12:05 < morcos> sigh, if this is another meeting of us all arguing against luke-jr then i'm going to find something else to do 12:05 -!- telelvis1 [~Adium@ip5f5add75.dynamic.kabel-deutschland.de] has joined #bitcoin-core-dev 12:05 < jonasschnelli> heh. Yes. Finish this. luke-jr can open a pr (action) 12:05 < wumpus> I'm not arguing against luke-jr 12:06 < wumpus> anyhow that derailed my question - so no one had any critical problems with 0.13.0rc2? 12:06 < wumpus> and I don't mean with the release notes 12:06 < sipa> is there any fear of instagibbs' problem being due to a code error? 12:06 < wumpus> what problem? 12:06 < instagibbs> sipa, fwiw it's showing up on older fork... 12:06 < instagibbs> it's not HD-related, or anything fairly recent 12:07 < gmaxwell> instagibbs: so without the bip32 changes? 12:07 < instagibbs> correct 12:07 < wumpus> you mean the salvagewallet problem? that's old hat, there's two issues open with salvagewallet problems 12:07 < sipa> instagibbs: ok, good to know 12:07 < wumpus> as I mentioned 12:07 < sipa> sorry, i didn't follow the whole discussion 12:07 < wumpus> this is not a last minute problem 12:07 < luke-jr> wumpus: earlier we were discussing 0.13.0's failure mode downgrading from 0.13.1; I am not clear what the status of that is, or if we need changes for it 12:07 < instagibbs> yes I'll double-check that keys aren't actually going away 12:07 < instagibbs> but not worried now 12:07 < gmaxwell> OK. 12:07 < instagibbs> LoadWallet works anyhoo :P 12:07 < wumpus> yes 12:08 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 12:08 < wumpus> so, all agree that it is time to release 0.13.0 final? 12:08 < wumpus> (after say, a day of updating the release notes) 12:09 < sipa> do we care about what happens when someone downgrades 0.13.1 to 0.13.0 after segwit activates? 12:09 < jonasschnelli> ACK, I can live with https://github.com/bitcoin/bitcoin/pull/8438 for 0.13.1 but don't know about others opinions. 12:09 < sdaftuar> i think we can just tell users who want to downgrade to 0.13.0 after segwit activates to do a reindex? 12:09 < sdaftuar> they'll end up redownloading blocks, without witnesses, but that seems fine 12:09 < luke-jr> sipa: I care only to the extent that it shouldn't make a "working" node that doesn't match consensus 12:09 < sdaftuar> kind of a weird case to try to optimize 12:09 < wumpus> agree with luke-jr - it should give a clear error 12:09 < luke-jr> ie, it should give a hard error or reindex or something 12:09 < wumpus> it should not seem to work, but if it requires a reindex that's fine 12:10 < jonasschnelli> sdaftuar: Yes. I think this would be good. I don't think we need to cover the downgrade case. 12:10 < sipa> luke-jr: i think the only risk (but i haven't tried) is that it either works fine or crashes 12:10 < gmaxwell> Loss of #8438 is unfortunate but if including it would mean delaying the release I don't think it would be good to do so. 12:10 < sipa> luke-jr: the UTXO set is identical across the two 12:10 < luke-jr> I suggest someone should try it before final 0.13.0 12:10 < luke-jr> probably not difficult using testnet? 12:10 < sdaftuar> sipa: i guess its somewhat unfortunate that it will seem to work fine. 12:10 < sipa> luke-jr: ack 12:11 < gmaxwell> it can be tested by changing the testnet parameters. 12:11 < sipa> detecting this situation is not hard, btw 12:11 < sdaftuar> oh because 0.13.1 will use OPT_WITNESS? 12:11 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has quit [Ping timeout: 276 seconds] 12:11 < sipa> sdaftuar: yes 12:12 < sdaftuar> hm so maybe we should add code then to look for that, hmm 12:12 < sipa> it's very last minute though... 12:12 < sdaftuar> i agree. and seemingly not that important 12:12 < sipa> i wish we thought of this before 12:12 < sipa> but i don't think it's worth holding up the release for 12:12 < luke-jr> it's trivial to test I think; if everyone else is too busy, I will try to do it today 12:13 < wumpus> gmaxwell: it would mean doing another rc, we could do final next week 12:13 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 12:13 < gmaxwell> that woudl also allow adding segwit downgrade protection sipa is currently discussing. (should be 3loc or so) 12:14 < wumpus> I don't know if it is that critical though, at some point we should just make the cut and stop slipping 12:14 < wumpus> yes that is true 12:15 * sipa notes that we're only 3 days past the date in https://github.com/bitcoin/bitcoin/issues/7679 12:15 < sipa> ideally deadlines don't slip, but iirc we're much more on track than for earlier major releases 12:15 < jonasschnelli> Yes. Maybe https://github.com/bitcoin/bitcoin/pull/8438 + SW0.13 downgrade compatibility is worth a week delay 12:15 < wumpus> yes, we've caught up a bit with the rc1 slip by doing a fast rc phase 12:15 < GitHub107> [bitcoin] luke-jr opened pull request #8458: [0.13] release notes: remove bad advice to switch to blockmaxweight prematurely (master...0.13_relnotes_remove_bad_advice) https://github.com/bitcoin/bitcoin/pull/8458 12:15 < btcdrak> 0.13.1 shouldnt be too far behind at least, but my preference is to include it and do another rc 12:15 < wumpus> note that I planned amonth for the rc1 phase 12:15 < wumpus> eh for the rc phase 12:16 < wumpus> well the downgrade protection cna't be postponed to 0.13.1 12:16 < wumpus> that'd defeat the point 12:16 < wumpus> otherwise that'd be my preference 12:16 < luke-jr> 8458 also mentions an idea I had for a 3-liner to make blockmaxsize perform identical to blockmaxweight, if we do another rc.. if that's desired, someone please say so and I'll do it now 12:17 < sipa> luke-jr: i think you included a few commits too many 12:17 < wumpus> yes now everyone starts with last minute ideas 12:17 < wumpus> great 12:17 < luke-jr> oh crap, I made it against master 12:17 < GitHub168> [bitcoin] luke-jr closed pull request #8458: [0.13] release notes: remove bad advice to switch to blockmaxweight prematurely (master...0.13_relnotes_remove_bad_advice) https://github.com/bitcoin/bitcoin/pull/8458 12:17 < gmaxwell> luke-jr: nah, something changing transaction selection.. not a good thing now. 12:17 < luke-jr> wumpus: well, it's trivial and probably unnecessary; fine if we don't do it 12:18 < wumpus> #action check if downgrade protection is really needed, or whether it already fails 12:18 < GitHub10> [bitcoin] luke-jr opened pull request #8459: [0.13] release-notes: Do not encourage changing blockmaxsize to blockmaxweight (0.13...0.13_relnotes_remove_bad_advice) https://github.com/bitcoin/bitcoin/pull/8459 12:19 < wumpus> other topics? 12:19 < cfields> oh 12:20 < sipa> segwit mempool malleability dos (#8279) 12:20 < cfields> (sec, someone else feel free to go ahead) 12:20 < wumpus> #topic segwit mempool malleability dos 12:21 < wumpus> wasn't that supposed to be solved in #8312? 12:21 < sdaftuar> no, it was only solved for 0.13.0 12:22 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has joined #bitcoin-core-dev 12:22 < sdaftuar> we need to decide what to do in our first segwit release 12:22 < sipa> so i suggested removing the "transaction failure purely due to witness does not cause rejection cache entry" 12:22 < sipa> with the rationale that all it does it prevent an attacker from hiding a valid transaction from you in some cases 12:23 < sipa> but it doesn't prevent it entirely (they can announce and just never send the transaction) 12:23 < sipa> sdaftuar notes that the mempool malleability attack only needs a short lived connection, while the never send tx data attack needs a persistent one 12:23 < sdaftuar> and it needs more than one persistent one -- you need several, as we retry tx download from other peers 12:24 < sipa> sdaftuar: rejection cache is also temporary, but a node won't just re-request... 12:24 < morcos> i think i'm coming around to the idea of full verification of all txs 12:24 < gmaxwell> morcos: hah 12:24 < BlueMatt> was there rationale to inv'ing with txid instead of wtxid for segwit nodes? 12:25 < BlueMatt> (just noting that wtxid would be Keep The Same Behavior As Pre SegWit (tm)) 12:26 < sipa> BlueMatt: duplicating a lot of logic (mempool, orphan, caches, ...), and causes at least a potential doubling anyway (you could be inved the same tx from a pre-segwit and post-segwit node once with txid and once with wtxid, without being able to tell they're the same) 12:27 < gmaxwell> also in the long term, the high malleability of witnesses would let an attacker waste lots of bandwidth. with nodes relaying different witness versions of the same txn among each other. 12:27 < BlueMatt> sipa: if you're inved a tx with a witness from a pre-segwit node we get a double anyway since its garbage to us 12:28 < BlueMatt> gmaxwell: same is true for scriptSigs...up to IsStandard limits (which could, in fact, be more easily enforced on witnesses) 12:28 < sdaftuar> BlueMatt: that's true but shouldn't happen, as pre-segwit nodes shouldn't be accepting segwit transactions unless they're running with an unusual policy 12:28 < sipa> also, if you have a tx with malleable witness (say, op_drop argument), nodes on the network could modify the witness without invalidating, and you wouldn't even be able to tell you already had the transaction before download 12:28 < sipa> and you couldn't ban them for it, because they're all valid transactions 12:30 < gmaxwell> BlueMatt: the kind of isstandard restriction against malleability only works great for limited classes of transactions, at least in the long term that isn't great. 12:30 < BlueMatt> I'm aware 12:30 < sipa> a solution is inving with both txid and wtxid... but if we go that way, we should also adds resource information to all invs (fees, size, sigops, ...) 12:30 < BlueMatt> sipa: I was about to say that...seems like a real solution is inving both... 12:30 < morcos> aren't we approachign the size of the tx at that point 12:31 < sipa> morcos: certainly within an order of magnitude 12:31 < BlueMatt> gmaxwell: but IsStandard is allowed to serve the same purpose as always - enforce reasonable limits to ensure code sanity until we've explored edge cases thouroughly 12:31 < gmaxwell> the txid is within an order of magnitude. :P 12:31 < sdaftuar> sipa: i think we might we just end up overfitting current policy rules by adding resource information 12:31 < gmaxwell> (for the median size) 12:31 < sipa> sdaftuar: 'overfitting' ? 12:31 < gmaxwell> but future mempool sync these sizes will matter less. 12:32 < gmaxwell> sdaftuar: feerate is pretty fundimental, however. 12:32 < sdaftuar> ie policy will change, and the information will be insufficient 12:32 < gmaxwell> I think including sigops would be dumb. 12:32 < sdaftuar> gmaxwell: yes, but already handled with less bandwidth by feefilter 12:32 < gmaxwell> if sigops really matter, something is wrong. 12:32 < gmaxwell> sdaftuar: indeed. feefilter makes it implicit. 12:32 < sipa> well, ok... we could send size and fee 12:33 < sipa> but that's not much better than making feefilter mandatory 12:33 < sipa> except slightly more flexible 12:33 < sipa> and sending two hashes is annoying 12:33 < gmaxwell> we're in the weeds for this right now. 12:33 < sipa> agree 12:33 < sipa> i think there is no simple clear cut solution for this 12:34 < gmaxwell> if we're doing set recon it doesn't really matter that much how long the identifiers are. 12:34 < sipa> but that's much further out 12:34 < BlueMatt> if inv'ing both hashes werent stupid for bandwidth, Id say that was a pretty great solution 12:34 < BlueMatt> true 12:34 < gmaxwell> sipa: depends on how long it takes me to convince you to implement the relevant number theory code. :P 12:35 < gmaxwell> In any case, the witnessID really doesn't matter for much of anything except rejection here. 12:35 < sipa> alternative idea: make feefilter mandatory for segwit, and just disable rejectioncache... 12:36 < sipa> rejectioncache only helps in practice against repeated announcements of transactions below your threshold 12:36 < sipa> it's not a protection against attacks 12:38 < sdaftuar> hm, not a bad idea 12:38 < luke-jr> feefilter isn't even used by default last I knew? (because there is no min fee until the mempool fills) 12:38 < sipa> luke-jr: good point 12:38 < BlueMatt> damn, mem^H^H^Hdiskpool was too late :( 12:38 < morcos> i'm all for making fee filter mandatory, if i'd known people would have liked it this much, we should have designed it that way from teh beginning 12:39 < morcos> i'm a bit worried about removing the rejection cache entirely 12:39 < gmaxwell> luke-jr: there is a minrelayfee. 12:39 < gmaxwell> Do we not feefilter on that? 12:39 < sdaftuar> gmaxwell: but we let in free transactions still 12:39 < gmaxwell> oh right. 12:39 < morcos> anytime there is any policy difference between nodes (such as a change in minrelay fee change isDust) then you get txs bouncing aroudn the network between the sets of nodes with different policies 12:40 < gmaxwell> full validation and distinguishing why rejection happened would help. 12:40 < morcos> i think its nice to have a sort of protection against that 12:42 < gmaxwell> we could just have one rejection filter per peer type. 12:42 < gmaxwell> e.g. rejection filter for non-sw peers and a rejection filter for sw peers. 12:43 < sdaftuar> with sw peers using wtxid invs? 12:43 < sipa> sdaftuar: still not enough... you need to be able to tell whether you already have another version of the same tx 12:43 < sipa> even with only sw peers 12:43 < gmaxwell> +full validation. 12:44 -!- jtimon [~quassel@55.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 12:44 < sdaftuar> sipa: i don't see why that's needed, any more than we have today with not knowing a tx is a double-spend before processing 12:45 < sipa> sdaftuar: someone connects to a 1000 nodes on the network, and relays a different version of the same valid transaction to each 12:45 < sipa> sdaftuar: now you potentially need to download 1000 transactions 12:46 < sipa> only one of which pays fees 12:46 < morcos> proposal: make feefilter mandatory. fully validate txs so we can punish peers who send us invalid stuff. don't put any witness tx in the rejection cache. then evaluate how useful rejection cache continues to be or whether we have policy violating segwit txs bouncing around 12:46 < sdaftuar> so you mean a 3rd party uses signature malleability to do this? 12:46 < sdaftuar> because the tx originator can already do that 12:46 < sipa> sdaftuar: right, the distinction isn't all that big 12:47 < sipa> morcos: i like that... but i think it's a big change for 0.13.1 12:47 < gmaxwell> the distinction though is that an attacker doing it pays fees, eventually runs out of funds, vs an attacker doing it to other peoples' txn does not ever run out of funds. 12:48 -!- d_t [~textual@83-244-233-135.cust-83.exponential-e.net] has joined #bitcoin-core-dev 12:48 < morcos> sipa: i think if we start with " don't put any witness tx in the rejection cache" then we'll be ok 12:48 < sipa> morcos: ack 12:48 < morcos> we can see how easy and short he other 2 are 12:50 < wumpus> that concludes the topic for this meeting? 12:50 < wumpus> any other topic proposals? cfields? 12:50 < NicolasDorier> for information, I created ##libconsensus bikeshedding room about how to handle libconsensus for anybody interested. 12:50 < cfields> NicolasDorier: haha, i was just about to paste a blurb for that 12:51 < NicolasDorier> aha I just woke up for that, will go back bed :D 12:51 < instagibbs> I have to leave now, but just letting you know salvage is finding lots of "extra" keys vs keys actually in wallet. *shrug* 12:52 < cfields> NicolasDorier and I discussed libconsensus a bit this weekend. We're hoping to discuss amongst ourselves, come up with some goals, and present them clearly for easier review 12:52 < morcos> +100 12:52 < cfields> jtimon: ^^ 12:52 < wumpus> #topic libconsensus 12:52 < cfields> that was it :) 12:52 < NicolasDorier> wumpus: we have not started the bikeshedding yet, expect longer fight about it soon :p 12:53 < luke-jr> NicolasDorier: IMO just discuss it here 12:53 < luke-jr> way too many channels already 12:53 < NicolasDorier> the reason I prefer a separate channel is so we can keep history and review it more easily 12:53 < wumpus> it's not like this channel is very busy\ 12:53 < NicolasDorier> things said here get lost quickly enough in the sea of discussion 12:54 < wumpus> that always happens on IRC, if you want discussions with good history/memory then it may be better to use a different discussion mechanism, or keep summaries or such 12:54 < sipa> i think having a separate place to 'form' a proposal makes sense 12:55 < luke-jr> #bitcoin-dev is also fairly quiet 12:55 < sipa> involving the whole world in a design rarely leads to something useful 12:55 -!- cryptapus [~cryptapus@unaffiliated/cryptapus] has quit [Ping timeout: 260 seconds] 12:55 < jtimon> from what I talked with NicolasDorier in previous times, the current goal is to complete a verifyBlock without necessarily exposing it in libconsensus at first 12:55 < wumpus> that's true, as long as you can get the people you want to pay attention to join it's fine 12:55 < wumpus> sometimes it's good to not have too many people there 12:55 < wumpus> but speaking for myself, I'm already in so many channels, I have a hard time following more 12:55 < morcos> to be clear, the design will be presented to the bigger group for feedback and not set in stone 12:55 < btcdrak> yeah ack on limiting channel proliferation, also it is more open in a logged channel 12:56 < morcos> i don't plan on joining the libconsensus subgroup, but i think having a focused small group will make progress actually happen 12:56 < BlueMatt> I'm with wumpus...too many channels these days (not that I've been paying enough attention to have much of a voice, but still) 12:56 < wumpus> then again that's up to you, doesn't need to be a discussion topic here to decide onthat. 12:56 < jtimon> I don't think creating a new channel or not is going to be important either way... 12:56 < cfields> morcos: sure. doesn't matter to me where the discussion takes place, just hoping to organize the discussion/planning a bit. 12:57 < wumpus> any other topics? 4 minutes to go 12:57 < sipa> i'm in favor of having it in a separate channel because i prefer not to see all discussions around it before there is clean design :) 12:57 < NicolasDorier> bikeshedding about the creation of the channel libconsensus announce the color of the future bikeshedding on the actual code ;) 12:57 < wumpus> NicolasDorier: exactly 12:57 < wumpus> let's stop that 12:57 < cfields> heh 12:57 < sipa> haha 12:58 < sipa> 3 minutes 12:58 < jtimon> cfields: NicolasDorier if you can share the goals 12:58 < wumpus> looks like we're done 12:58 < wumpus> #endmeeting 12:58 < lightningbot> Meeting ended Thu Aug 4 19:58:24 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) 12:58 < lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-08-04-19.00.html 12:58 < lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-08-04-19.00.txt 12:58 < lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-08-04-19.00.log.html 12:59 < NicolasDorier> I have a code question: I'm working on a commit similar to one from jtimon, he is removing one check here https://github.com/jtimon/bitcoin/compare/remove-ism...consensus-post-remove-ism#diff-7ec3c68a81efff79b6ca22ac1f1eabbaL2353 about BIP30, does somebody know if it is actually safe to do so ? 13:00 < cfields> jtimon: i think everyone has different goals in mind. the point of organizing is to settle on something and work towards it. i can catch you up on our discussions this weekend, though ofc you've spent more time on it than anyone 13:01 < jtimon> cfields: well, a small summary of the "goals in common" or "easy to achieve" goals or something would be something 13:02 < NicolasDorier> jtimon: I think we still have also to discover what the goals in common are. Which is why I think we can discuss it in the channel. I guess if the three of us agree (plus anybody in the channel), there is good chance the rest agree too 13:02 -!- telelvis1 [~Adium@ip5f5add75.dynamic.kabel-deutschland.de] has quit [Quit: Leaving.] 13:02 < morcos> NicolasDorier: See this comment from sdaftuar: https://github.com/bitcoin/bitcoin/pull/8391#issuecomment-237584871 13:03 < NicolasDorier> correct, I'll write a bip 13:04 < jtimon> NicolasDorier: I may not be so optimist, but fair enough, I mean, great. do you guys have time now to give me a little summary of your discussions so far now (in the other channel)? 13:04 < morcos> It's related. The point is once you are enforcing BIP34 then BIP30 is unnecessary except for historical violations. We're changing the rule to have BIP34 be enforced as of a certain height, however, it kind of requires that the chain be the same otherwise we dont' know what the historical violations are or might be 13:04 < NicolasDorier> yes, I will jtimon, but later I'll go back bed soon (5am here) 13:05 < jtimon> yeah, I say so in my commit, it is unnecessary because we're not using ISM anymore 13:06 < jtimon> oh, no worries, another time will be fine 13:06 -!- gabridome [~gabridome@37.159.28.15] has joined #bitcoin-core-dev 13:07 < cfields> jtimon: same, in the middle of several things atm. maybe some time tomorrow? 13:07 < jtimon> sure, no problem 13:07 < NicolasDorier> morcos: so basically we could remove BIP30 altogether as this is unlikely another chain manage to catch up the amount of work of the current one 13:08 < sipa> isn't there some interaction between BIP30 and the no-overwrite optimization in coinscache? 13:08 < sipa> which needs a BIP30 check during IBD 13:09 < NicolasDorier> mmh I think I won't make any change about it after all... I'm not confident of this thing 13:09 < morcos> NicolasDorier: yeah we need it on IBD 13:09 < morcos> hmm this brings up a kind of good point 13:09 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 13:09 < morcos> you can't remove the BIP34Hash unless you are going to checkpoint the chain at that point 13:10 < NicolasDorier> ok, so I will not touch it 13:10 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has quit [Ping timeout: 252 seconds] 13:11 < jtimon> you cannot remove it completely, but when you use bip34 you don't need to do it (this is current behaviour) 13:11 < morcos> jtimon: no, when you use BIP34 on the existing BIP34 hash then you dont' need to do it 13:11 < morcos> if you activated BIP34 at the same height on a different chain, it would be unclear whether yous till needed to do BIP30 13:11 < jtimon> morcos: only one block? 13:11 < morcos> depends on the status of historical violations on that chain 13:12 < morcos> no on all future blocks 13:13 < jtimon> https://github.com/jtimon/bitcoin/commit/273e27bd0c086f5ba20cebc2f5ec9e24f9d79015 13:13 < morcos> i can't remember off the top of my head, but if in an alternate chain you spent coinbase A before you generate coinbase A' then you activated BIP34, then you still need BIP30 to make sure the spend of A' doesn't conflict with the spend of A 13:13 < morcos> but in the BIP34hash chain we know that didn't happen 13:13 < jtimon> do you think that commit is incorrect after removing ISM? 13:14 < morcos> jtimon: its incorrect in that it'll be broken code on a mainnet chain that doesn't contain BIP34hash 13:14 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 13:14 < morcos> so who knows what happens if someone feeds you such a chain on startup 13:15 < jtimon> I think it will only fail if we reorg below BIP34Height (227931) 13:15 < morcos> oh wait 13:15 < morcos> maybe because BIP30 is enforced for everythign except those two exceptions you're ok 13:15 < jtimon> and what we have now, if it reorgs that long, without code for ISM...what? bip34 is going to be activated at that height no matter what 13:15 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has left #bitcoin-core-dev [] 13:16 < jtimon> well, I wouldn't call them exceptions 13:17 < jtimon> pindexBIP34height will only be NULL if pindex->pprev->GetAncestor(227931) returns null 13:17 -!- JackH_ [~Jack@79-73-188-45.dynamic.dsl.as9105.com] has joined #bitcoin-core-dev 13:18 < jtimon> what is doing now is that if you are above that height and the block you activated bip34 in is 227931, then don't enforce bip30 since bip34 is enforced 13:19 < jtimon> in a reorg that deep, if it was activated with another block, then bip30 will be checked always regardless of bip34 activation (no optimization, never) 13:19 < morcos> jtimon: i honestly don't think its worth doing. but if you really want to, i will take the time to think about it. there are a lot of issues, down to when you flush your cache. but please make an actual PR for me to review that other people also want to merge 13:19 < jtimon> with my suggested change, it would always do the optimization after bip34 activation height, even after such a long reorg 13:20 < jtimon> morcos: absolutely, no problem 13:20 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-mgiptbgavvdxnhwq] has joined #bitcoin-core-dev 13:21 < morcos> jtimon: ok, i think i got the problem 13:21 < jtimon> well, if you tell me now I won't open the PR :) 13:21 < morcos> imagine that you reorg to height 90000 (or are just fed an alternate chain on startup) 13:22 < morcos> ah shoot.... 13:22 < morcos> nm 13:22 < morcos> wait 13:22 < morcos> yes 13:22 < morcos> thats it 13:22 < morcos> so confusing 13:22 < jtimon> let's do one thing at a time, yep, you reorg to 9000 or you fed an alternate chain? 13:23 < morcos> either, same affect you start processing different blocks starting at 90000 13:23 < jtimon> think the worse case: you reorg to block 1 13:23 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has joined #bitcoin-core-dev 13:23 < morcos> thats after coinbase A was created but before its duplicate A' was created 13:23 < morcos> now you spend coinbase A 13:24 < jtimon> bip 30 is checked before bip34 always except the exceptions 13:24 < jtimon> after bip34 activation, it is no longer necessary 13:24 < morcos> then you create coinbase A' (allowed b/c it doesn't violate BIP30 , A is spent, and BIP34 is not active yet) 13:24 < morcos> now you activate BIP34 at height 227931 13:25 -!- JackH_ [~Jack@79-73-188-45.dynamic.dsl.as9105.com] has quit [Quit: Leaving] 13:25 < morcos> then you can spend A' and create the same txid as your spend of A b/c you aren't enforcing BIP30 any more 13:25 < jtimon> yes, from now on you can stop checking bip30 in new blocks 13:25 < morcos> no you can't 13:25 < morcos> spending A' would be allowed and would create a duplicate txid of the spend of A 13:27 < jtimon> mhmm, whay they cannot today? 13:27 < jtimon> why 13:27 < morcos> because A' the coinbase was created before A was spent 13:27 < morcos> so A' overwrote A already 13:28 < morcos> but if in an alternate history you spent A to B first 13:28 < jtimon> doesn't include the height make it very difficult to collide with coinbases before 227931 ? 13:28 < morcos> then you have to worry about B' and B 13:28 < jtimon> instead of A' 13:28 < jtimon> think of all the coinbases before 227931 13:29 < jtimon> no? you don't want the same as neither of those 13:29 < morcos> you can try this on regtest.. enforce BIP30 only up to height 100 and BIP34 only starting at height 100. create some coinbases, spend them and then create duplicates before 100. 13:29 < jtimon> that's with or without a reorg 13:29 < morcos> then after 100 make them colide 13:29 < Chris_Stewart_5> Is there a simple way to serialize a CKey? Doesn't seem to have the SERIALIZE_METHODS macro 13:30 < jtimon> morcos: yeah, you could "attack regtest" that way... 13:31 < morcos> well if you change the code, someone could feed you a messed up mainnet chain that way 13:31 < sipa> Chris_Stewart_5: historically, it's converted from/to a CPrivKey for serialization 13:31 < jtimon> I thought we were talking about mainnet 13:32 < morcos> jtimon: yes so on mainnet, you'd have to have a REALLY deep reorg, below 227931 13:32 < Chris_Stewart_5> sipa: Is there a simple utility function serialize CPrivKey? Can it just be passed to HexStr? 13:33 < morcos> or if somoene just fed you an alternate low work chain then i'm not really sure what would happen, its possible that your utxo could get into a messed up state than you wouldn't be able to fix when you found out about the real chain 13:33 < jtimon> and then a node without my change would always check bip30, even after bip34 activation, is that the desired behaviour? 13:33 < sipa> Chris_Stewart_5: CKey also has begin() and end(), to it can be passed to HexStr directly 13:33 < sipa> Chris_Stewart_5: and has a constructor that takes a byte array 13:33 < jtimon> my node would not check bip30 after bip34, even after the reorg 13:34 < morcos> jtimon: yes you can either always check BIP30 even after BIP34, or you can examine the chain you're on pre BIP34 and decide whether you can now skip BIP30 checks (which is what we decided to do with mainnet, but requires manual intervention) 13:34 < morcos> are you opposed to leaving in the BIP34 hash 13:35 < morcos> checking BIP30 is exteremely slow 13:35 < morcos> that's why we put in the ability to skip it 13:36 < jtimon> so, yes, theoretically always checking bip30 after bip34 has certain risk if there's that reorg...but aren't we screwed if that happens anyway? 13:37 < jtimon> like, really screwed 13:37 < jtimon> I don't know, maybe it's not worth it 13:38 < Chris_Stewart_5> sipa: Thanks 13:39 < jtimon> the advantages are code simplification and a slight optimization (really no need to call GetAncestor if you don't care about the hash), and keep the optimization in case of pre-bip34 reorg, which is arguably a disadvantage 13:39 < jtimon> no, not opposed, removing it is just a suggestion 13:40 < jtimon> if we knew we weren't doing it, we could have made https://github.com/jtimon/bitcoin/commit/6f98197634026e740a9172405386b15c3a79b5a5 without bip34 directly in #8391... 13:41 < jtimon> now I'm afraid the right time to do that will be...never 13:41 < morcos> jtimon: i think the issue is less a reorg than that someone might feed you an alternate chain on startup and get your utxo into a permanently messed up state b/c you don't know how to reorg back 13:41 < jtimon> oh, I see 13:41 < morcos> i'm not positive that would happen, but seems more likely than not, and also seems like it could be affected by future changes to the way coins work that we might not anticipate 13:41 < jtimon> yeah, makes sense, thanks 13:44 < jtimon> yeah, I wasn't properly thinking about an isolated node doing initial synchornization, that's what you meant by fed up with the wrong chain from the beginning, sorry 13:44 < jtimon> nah, better to avoid that risk, definitely not worth it 13:44 < jtimon> thansk 13:44 < morcos> jtimon: yeah 13:44 < morcos> sure 13:47 < jtimon> well, maybe the time for https://github.com/jtimon/bitcoin/commit/6f98197634026e740a9172405386b15c3a79b5a5 is whenever bip113 activation is simplified from bip9 in the same way 13:47 -!- gabridome [~gabridome@37.159.28.15] has quit [Quit: gabridome] 13:48 < jtimon> or when/if we expose consensus::Params in libconsensus 13:49 < jtimon> anyway, that's another topic 13:49 < Chris_Stewart_5> sipa: When I serialize CPriv/CPub I get something that is larger than 32 bytes? 13:50 < sipa> Chris_Stewart_5: CPrivKey uses OpenSSL DER serialization... which is 300 bytes or so for a private key 13:50 < sipa> CPubKeys are 33 or 65 bytes 13:50 < sipa> CKey is internally 32 bytes + compression flag 13:52 < Chris_Stewart_5> ahh ok, that makes more sense. So basically CKey is the interface exposed inside of core... except for serialization. Something that could be implemented in a pull? 13:52 < sipa> well there isn't one single obvious serialization for CKey 13:55 < Chris_Stewart_5> Seems like the 32 byte serialization would be the intuitive one.. at least to me. Am I missing something here? 13:56 < Chris_Stewart_5> oh nvm i see what you are talking about now, since CKeys can be both priv and pubs right? 13:57 < sipa> no 13:57 < sipa> CKey is a private key 13:57 < sipa> but you need the compression flag too 13:58 < sipa> and there is no deserialize because we use secure memory for private keys where possible 14:02 -!- TomMc [~tom@unaffiliated/tommc] has joined #bitcoin-core-dev 14:08 < NicolasDorier> jtimon, I will soon make a PR similar to https://github.com/jtimon/bitcoin/commit/6f98197634026e740a9172405386b15c3a79b5a5 along with another change to parametize buriedsf height for regtest 14:10 < jtimon> NicolasDorier: cool, but I believe the right time was in your PR, at least with the new ones if you didn't wanted to touch bip34 14:11 < NicolasDorier> jtimon: I disagree, my PR was making a "big" consensus change so I wanted to keep it as simple as possible to review 14:11 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 14:11 < NicolasDorier> I wanted to be it merged the quickier I could so I can continue my work on libconsensus 14:11 < jtimon> although I won't oppose the refactor, of course, just worry that it will be less acceptable 14:12 < jtimon> how would using an array instead of two variables have been more complicated? 14:12 < jtimon> anyway, as said I won't oppose 14:12 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 250 seconds] 14:13 < NicolasDorier> jtimon: I don't think it wil lbe less acceptable, this buried deployment stuff will be a BIP. In the future other SF will end up using the same structure 14:13 < NicolasDorier> anyway, I made two differents commit 14:13 < NicolasDorier> we can take one and not the other 14:13 < jtimon> NicolasDorier: yeah I guess the BIP is an opportunity to reorganize the struct 14:15 < jtimon> anyway, I'm suffering from premature worry about controversy, let's not discuss about whether something will be acceptable or not when we both want it 14:15 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has joined #bitcoin-core-dev 14:16 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 14:20 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has quit [Quit: gabridome] 14:21 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has joined #bitcoin-core-dev 14:27 < GitHub193> [bitcoin] NicolasDorier opened pull request #8460: parametrize buried soft fork in regtest and refactor (master...buriedsf) https://github.com/bitcoin/bitcoin/pull/8460 14:27 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has joined #bitcoin-core-dev 14:27 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 14:27 < NicolasDorier> jtimon: worrying about controversy create a meta controversy :D 14:28 < NicolasDorier> just created the PR 14:31 -!- TomMc [~tom@unaffiliated/tommc] has quit [Ping timeout: 244 seconds] 14:33 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has quit [Quit: gabridome] 14:33 < jtimon> yeah, I'm revieweing it 14:34 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has joined #bitcoin-core-dev 14:35 < jtimon> and looking at 8418 too 14:36 < jtimon> https://github.com/bitcoin/bitcoin/pull/8418/files#diff-c865a8939105e6350a50af02766291b7R981 14:36 < jtimon> .MineBlocksOnDemand() is starting too look like IsRegtest() 14:37 < GitHub92> [bitcoin] jlopp opened pull request #8461: document return value of networkhashps for getmininginfo RPC endpoint (master...rpcMiningHelp) https://github.com/bitcoin/bitcoin/pull/8461 14:37 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has quit [Client Quit] 14:38 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has joined #bitcoin-core-dev 14:43 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has quit [Quit: gabridome] 14:43 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has joined #bitcoin-core-dev 14:44 -!- TomMc [~tom@gateway/vpn/privateinternetaccess/tommc] has joined #bitcoin-core-dev 14:44 < jtimon> oh, it was RegTest() or Params().NetworkID() == CChainParams::REGTEST directly, never mind, it's been a while since 3824, thinking out loud 14:46 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has quit [Client Quit] 14:50 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has joined #bitcoin-core-dev 14:52 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has quit [Client Quit] 14:55 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has joined #bitcoin-core-dev 14:59 < jtimon> NicolasDorier: in #8460, how do you modify a couple of them? -buriedsfparams=bip65:1351,bip66:1251 ? 14:59 < NicolasDorier> you repeat 14:59 < NicolasDorier> -buriedsfparams=bip65:1351 -buriedsfparams=bip66:1251 14:59 < jtimon> ok, thanks 15:02 < jtimon> https://github.com/bitcoin/bitcoin/pull/8418/files#r73610819 15:03 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has left #bitcoin-core-dev [] 15:05 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt] 15:06 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has quit [Ping timeout: 252 seconds] 15:09 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has joined #bitcoin-core-dev 15:09 -!- Lauda_ [~quassel@2a06:8ec0:3::1:b224] has joined #bitcoin-core-dev 15:10 -!- Lauda_ [~quassel@2a06:8ec0:3::1:b224] has quit [Client Quit] 15:11 -!- TomMc [~tom@gateway/vpn/privateinternetaccess/tommc] has quit [Ping timeout: 244 seconds] 15:12 -!- Lauda_ [~quassel@2a06:8ec0:3::1:b224] has joined #bitcoin-core-dev 15:12 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 15:14 -!- Lauda [~Lauda@185.122.59.11] has quit [Disconnected by services] 15:14 -!- Lauda_ is now known as Lauda 15:15 -!- LaudaM [~Lauda@2a06:8ec0:3::1:b224] has joined #bitcoin-core-dev 15:17 -!- LaudaM [~Lauda@2a06:8ec0:3::1:b224] has quit [Disconnected by services] 15:17 -!- LaudaM [~Lauda@185.122.59.11] has joined #bitcoin-core-dev 15:17 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 15:20 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Remote host closed the connection] 15:23 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has joined #bitcoin-core-dev 15:25 < NicolasDorier> jtimon: I think one bool is enough. fAllowOverwriteSoftforks which both BIP9 and buried SF use 15:27 -!- TomMc [~tom@unaffiliated/tommc] has joined #bitcoin-core-dev 15:27 < jtimon> NicolasDorier: fine with me, sounds reasonable 15:27 < NicolasDorier> will add commit to my PR later 15:28 < jtimon> great 15:37 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has quit [Excess Flood] 15:37 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has joined #bitcoin-core-dev 15:39 < jtimon> NicolasDorier: you are not removing BIP34Height BIP65Height and BIP66Height in Consensus::Params 15:41 < jtimon> you can leave: 15:41 < jtimon> /** Block hash at which BIP34 becomes active (for BIP30 optimization)*/ 15:41 < jtimon> uint256 BIP34Hash; 15:42 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)] 15:44 < NicolasDorier> oh 15:51 -!- LaudaM is now known as random4343 15:51 -!- random4343 is now known as LaudaM 15:52 -!- LaudaM is now known as CatWoman 15:57 -!- CatWoman [~Lauda@185.122.59.11] has quit [Disconnected by services] 15:57 -!- LaudaM [~Lauda@2a06:8ec0:3::1:b224] has joined #bitcoin-core-dev 16:01 -!- Giszmo [~leo@ppp-188-174-93-152.dynamic.mnet-online.de] has quit [Ping timeout: 276 seconds] 16:04 < sipa> does anyone know offhand where the last segwit transaction in the chain is? 16:04 < sipa> *testnet chain 16:04 -!- molly [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 16:05 < sipa> i've reset the testnet chainparams for segwit to 0, and reconsiderblock/invalidateblock 100000 blocks, and it works fine 16:07 -!- molz [~molly@unaffiliated/molly] has quit [Ping timeout: 240 seconds] 16:13 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 16:18 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 16:27 -!- molz [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 16:30 -!- molly [~molly@unaffiliated/molly] has quit [Ping timeout: 258 seconds] 16:39 -!- d_t [~textual@83-244-233-135.cust-83.exponential-e.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 16:45 < NicolasDorier> jtimon: I fixed the nit as well as added the commit with the allowSfOverride 16:48 -!- belcher [~user@unaffiliated/belcher] has joined #bitcoin-core-dev 16:50 < jtimon> NicolasDorier: tiny last nit https://github.com/bitcoin/bitcoin/pull/8460/files#r73622457 16:52 < NicolasDorier> jtimon: this is removed in later commits 16:52 < NicolasDorier> oups 16:52 < NicolasDorier> nop 16:54 < NicolasDorier> removed :) 16:54 < NicolasDorier> I did quite a lot of rebase to clean up the history. But now it should be stable 16:55 < jtimon> good work 16:56 < NicolasDorier> thanks, going to bed for real now (9am) :D 16:58 < jtimon> oh, sorry about that 17:00 < sipa> sdaftuar, luke-jr, gmaxwell, wumpus: i believe that downgrading from 0.13.1 to 0.13.0 will just work 17:00 < sipa> sdaftuar: even in case of a reorg that takes you back into witness-enabled code 17:00 < sipa> s/code/blocks/ 17:01 < luke-jr> well, that's good, I think 17:01 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 252 seconds] 17:01 < sipa> the test that no witness data is present in non-witness blocks is only done in AcceptBlock, not in ConnectBlock 17:02 < sipa> so we enforce it when receiving the block from somewhere, but not afterwards 17:02 < sipa> which i think is exactly right 17:02 < gmaxwell> sipa: what happens if you downgrad then upgrade again? 17:03 < sipa> gmaxwell: newly added blocks will be rewinded 17:15 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 17:20 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 17:33 -!- jtimon [~quassel@55.31.134.37.dynamic.jazztel.es] has quit [Ping timeout: 240 seconds] 17:41 -!- TomMc [~tom@unaffiliated/tommc] has quit [Ping timeout: 276 seconds] 17:49 < morcos> sipa: luke-jr: sorry, but i'm not sure i understand sipa's suggested compromise in 8459. sorry i don't remember exactly what the first version was, but i think it makes sense to explain that you don't NEED to specify both size and weight, otherwise its just confusing as to how you are supposed to set things up 17:50 < morcos> as to making a recommendation whether or not to set size... i'm not sure why you say its not worse than 0.12, we don't know that. my guess is that it isn't significantly worse now, but it certainly could be post segwit, when you are trying to fit the last tx in a block and you keep trying EVERYTHING b/c the limit thats stopping you isn't the consensus limit 17:51 < morcos> i do apologize for not testing at least the serialization effect (the other would be tricky to test), but i've been too caught up in trying to figure out why waking up threads is so abysmally slow 17:52 < luke-jr> it doesn't serialise any more than without maxsize AFAIK. 17:52 < luke-jr> GetSerializeSize doesn't serialize 17:54 < morcos> ah so its expected that GetSerializeSize is fast? well that explains why anecdotally its not slower 17:54 < luke-jr> morcos: I thought it was obvious the options were optional.. but if you disagree, I can add perhaps: "Both of these are optional, but at least blockmaxsize ought to be assigned by miners, to determine the largest block they are willing to create." 17:54 < morcos> luke-jr: i assume you make suggestions like that on purpose 17:54 < luke-jr> morcos: GetSerializeSize is literally just adding the sizes of each field 17:54 < luke-jr> morcos: ? 17:55 < sipa> i expect getserializesize time to be dominated by just memory access of iterating all vectors 17:56 < morcos> luke-jr: you are against what you see as advice in the notes to use weight, but surely you understand that all of us are against advice to limit based on bytes? if any of us believed that was an issue, we'd be opposed to segwit 17:56 < luke-jr> morcos: it is clearly an issue today. I am shocked that anyone would disagree. 17:56 < luke-jr> 0.13 will hopefully change that with CB, but it isn't deployed yet. 17:58 < sipa> luke-jr: my view is that at worst it may make block propagation slightly worse, but at the same time it improves utxo storage incentives, which i'm much more concerned about long term 17:58 < morcos> how about language along the lines of "If a miner wishes to limit the actual size of a block in bytes (as opposed to the consensus weight limit) then it is necessary to specify a blockmaxsize explicitly. If a miner does not wish to do that, it would be better to only specify a blockmaxweight" 17:58 < sipa> a small constant factor was never something i opposed for block size. an expectation of it being increased under whatever perceived pressure is 17:59 < luke-jr> morcos: that suggests miners ought not to limit size 17:59 < sipa> i think miners ought not to limit size 17:59 < morcos> something like that, not exactly "as opposed to" b/c that sounds like you're not enforcing the consensus limit 17:59 < sipa> because it's unreasonable that all of them will 17:59 < morcos> luke-jr: why does that language suggest they ought not limit size. its just saying that if you dont' care about a limit on actual size, then its better to not set blockmaxsize. 17:59 < morcos> which is TRUE 18:00 < luke-jr> if they don't care, they should set it based on someone's recommendation, but not necessarily the big blocker recommendation 18:01 < luke-jr> if they care about Bitcoin (but not per se enough to research their own opinion on block size limits), they should follow recommendations to set it lower. 18:01 < sipa> luke-jr: as a collective, perhaps 18:01 < luke-jr> brb 18:01 < sipa> luke-jr: there could be a gentlemen's agreement not to set it to the maximum immediately 18:01 -!- jtimon [~quassel@55.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 18:01 < sipa> but i believe that's unrealistic at this point 18:02 < morcos> "If a miner wishes to limit the actual size of a block in bytes then it is necessary to specify a blockmaxsize explicitly. If a miner wishes only to limit block creation by weight (BIP141) regardless of how large in bytes the block is they should only specify a blockmaxweight" 18:02 < luke-jr> sipa: every time I brought this up before, I was asked to "wait for the segwit release notes"; now I'm told it's too late? 18:02 < morcos> i'm trying to make it as neutral as possible, while explaining the difference 18:02 < luke-jr> (still brb mode) 18:03 < morcos> what i don't want is a bunch of miners trying to mine max blocks and having a blockmaxsize set (especially after segwit). and we have to explain in detail enough for that to happen. i have no objection if they actually want to limit bytes 18:03 < sipa> luke-jr: i can't remember ever recommending to wait for release notes. i have told you many times that if you disagree that weight is the limiting factor on the network, you should oppose segwit 18:05 < morcos> sipa: want to think about something different an arcane for a second? in those checkqueue graphs i presented you, the scriptthreads were busyspinning waiting for work 18:05 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-mgiptbgavvdxnhwq] has quit [Quit: Connection closed for inactivity] 18:05 < sipa> morcos: i heard 18:06 < morcos> it slows down the whole thing quite a bit (about 5ms per block) but still a big improvement, if you wake up the scripthreads only when there is work 18:06 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 18:06 < morcos> even though it takes only about 100us for them all to wake up, and you can make that happen as early as the start of ConnectTip so they are all awake and busy spinning well before work comes in 18:07 < morcos> i'm assuming its the fact that their caches have been blown away, but doesn't 5ms seem like a lot? 18:08 < morcos> just wondering if you had any thoughts.. we're still working on it (mostly jeremy, i'm mostly just testing) 18:09 < sipa> morcos: is the slowdown as significant when you have fewer threads? 18:09 < morcos> sadly i didn't test that 18:10 < morcos> yet 18:11 < sipa> and this is 5ms clock time, not cpu time? 18:13 < morcos> yes, 5ms clock time. it appears from debugging that the scriptthreads are just slower to run. 18:13 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has quit [Quit: Leaving.] 18:16 < sipa> i guess if there are many memory locations to access one by one for script validation to run, the effect of not running could be many ram latencies 18:16 < sipa> but i'm not sure whether than makes sense 18:18 < morcos> ok, don't worry about it.. its mostly just bugging me. i realize that it doesn't make sense to micro optimize for one architecture anyway, but just trying to understand the effect 18:22 -!- belcher [~user@unaffiliated/belcher] has quit [Quit: Leaving] 18:41 -!- arowser [~arowser@106.120.101.38] has joined #bitcoin-core-dev 18:42 -!- arowser [~arowser@106.120.101.38] has left #bitcoin-core-dev [] 18:43 -!- arowser [~quassel@106.120.101.38] has joined #bitcoin-core-dev 18:57 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 19:22 -!- arowser [~quassel@106.120.101.38] has quit [Ping timeout: 252 seconds] 19:28 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 264 seconds] 20:09 -!- Alopex [~bitcoin@cyber.dealing.ninja] has quit [Remote host closed the connection] 20:10 -!- Alopex [~bitcoin@cyber.dealing.ninja] has joined #bitcoin-core-dev 20:10 < PatBoy> thx dev for all ur hard work ! 20:11 < luke-jr> sipa: I do not believe that opposing segwit is a serious suggestion. 20:12 < luke-jr> morcos: then in the segwit announcement, we can re-evaluate and make recommendations that make sense given the new network conditions 20:13 < luke-jr> (which might or might not include removing blockmaxsize depending on CB effectiveness and real-world testing) 20:15 < sipa> luke-jr: miners will set block and weight likely to the maximum... that may be suboptimal, but it's inevitable that this is what will happen if segwit goes through 20:15 < sipa> people can suggest to not do that... but for centralization risk (which is a long term effect), the important question is now what people do, but what they could do 20:16 < sipa> *not 20:16 < luke-jr> sipa: we don't know that yet, and our recommendations should always be what is sane even if they get ignored. 20:18 < sipa> luke-jr: that's a reasonable position... but the code is written from a viewpoint that we will get weight-limited block construction 20:18 < sipa> luke-jr: and the release notes should describe the code 20:19 < luke-jr> then the code is broken (sabotaged, it sounds like) and fixing it should be considered a blocker for any release. 20:19 < sipa> if that is your viewpoint, then it is segwit that is sabotaged 20:20 < sipa> i disagree strongly with that 20:20 < luke-jr> sipa: do you disagree strongly with 29000050e575a26b7f7c853cbccdb6bc60ddfdd9 for 0.13? 20:21 < sipa> i think it's pointless 20:21 < sipa> we're doing busy work to avoid stating reality in release notes 20:21 < sipa> the reality is that segwit will mean that blocks go over 1 MB 20:22 < luke-jr> reality is that blockmaxsize should be used for today and the near future 20:22 < sipa> i disagree with that 20:22 < gmaxwell> luke-jr: I think you are not paying attention to reality. Virtually all miners make blocks as large as they can. If at any point they make a block smaller than maximum, angry mobs show up and demand them to make maximum size blocks. Not wanting to deal with such nonsense, they simply do it. 20:22 < sipa> or rather, i would agree with that if there was a reasonable chance that the entire ecosystem will self limit in that way 20:22 < sipa> it won't 20:22 < gmaxwell> this is the existing behavior in the network, and has been for a long time-- more or less since we made it configurable at all. 20:23 < luke-jr> sipa: the entire ecosystem doesn't need to; why is it all or nothing? 20:23 < luke-jr> gmaxwell: that's with 1 MB blocks. who knows if it will work out the same with 3 MB 20:24 < gmaxwell> it especially will work out that way, with CB and relay mitigating orphaning effects. 20:24 < luke-jr> I know that if miners start doing >1 MB blocks before the network is ready, I will intentionally NOT use a segwit wallet, to do my small part in preventing them from bloating the blocks. I think a lot of the community could be convinced to do the same. 20:24 < gmaxwell> luke-jr: and if you want to posit a change in how things work you should suggest how. 20:25 < gmaxwell> luke-jr: nothing held back blocksizes from growing from 250k to 1mb... not even the defaults in bitcoin core-- or large increases in orphaning rates. 20:25 < sipa> luke-jr: smaller blocks will always be better for propagation relay effects, and larger blocks will always be worse 20:25 < sipa> luke-jr: there is no 'ready' here; just a compromise that is acceptable 20:25 < sipa> segwit shifts the relay effects one way, but shifts other incentives the other way 20:27 < luke-jr> ugh, even IF miners will end up doing terrible things, it doesn't mean we need to sabotage miners who WANT to do what is best for the network. 20:28 < gmaxwell> its not a question of "want to do what is best"; most people don't want to decide and do not have the time to do so in any case. 20:28 < gmaxwell> You're thinking in terms of pixie unicorn miners. 20:28 < sipa> the only way miners can do something that is good for the network is by making themselves replacable 20:28 < gmaxwell> Go look at the actual blocks. The miners you're expecting just don't exist at a meaningful level. 20:30 < sipa> luke-jr: moreover, we're not removing the ability to limit the block size 20:31 < sipa> reality is that the code is optimized for limiting by weight, and segwit is designed with the assumption that that is what will happen... your suggestions seem to be that we should hide that 20:31 < sipa> block size limiting is supported, and we should explain how to use it for those who want to 20:32 < luke-jr> gmaxwell: yes, pixie unicorn miners who actually exist, despite being ignored by Core 20:32 < luke-jr> sipa: either blockmaxsize has a notable cost or it doesn't. if it doesn't, there is no reason to claim it does. 20:32 < sipa> luke-jr: so, benchmark 20:32 < luke-jr> if it does, then it's sabotaging miners who use it 20:33 < gmaxwell> luke-jr: they don't exist in a meaningful sense. a miner that gets a block a day is not making a meaningful dent in the systems' survivability. 20:33 < sipa> luke-jr: limiting by block size is already taking a cut. fee income will be suboptimal if you limit by size 20:33 < sipa> luke-jr: that's as much sabotaging 20:33 < luke-jr> gmaxwell: so that justifies ignoring the requests of such miners, and sabotaging them with (allegedly) bad performance? 20:34 < sipa> limiting by weight is a design decision which has costs 20:34 < gmaxwell> 'sabotaging' after I had to jump up and down and yell for over a month to get eligius to upgrade to 0.12 even though it made a _tens of fold_ increase in createnewblocktime? 20:34 < gmaxwell> This is such bullshit. 20:34 -!- gmaxwell [greg@wikimedia/KatWalsh/x-0001] has left #bitcoin-core-dev [] 20:35 < luke-jr> ignoring that 0.12 broke numerous things Eligius provided that were left unmerged in Core since 0.6.. 20:36 < sipa> such as? 20:36 < luke-jr> CPFP 20:37 < sipa> i never trusted that code enough to merge 20:37 -!- gmaxwell [greg@wikimedia/KatWalsh/x-0001] has joined #bitcoin-core-dev 20:37 < sipa> and i think i was not alone 20:37 < gmaxwell> I am fed up with this. 20:38 < luke-jr> same here. 20:38 < gmaxwell> luke-jr: you are abusive towards me and the other contributors. 20:38 < gmaxwell> you are obsessing over minutia on top of minutia. 20:38 < gmaxwell> You are wasting countless hours exhausting all patience. 20:39 < gmaxwell> Over matters which _do_ _not_ _matter_. The few obscure miners which will set non-defaults even though they get abusive and threatening contact from users (which drives away their hashpower); can _still_ do so. If it's slightly slower? so what--- the latest software is dozens of times faster to creates blocks than older software and they hardly cared to upgrade. 20:40 < gmaxwell> it litterally makes no difference in the world, and yet you force people to spend hours and hours debating these things. 20:41 < gmaxwell> and I get to spend my time asking others to not leave the project because they are exhausted by you; but it even exhausts me too. 20:45 < gmaxwell> The last block from eligius was 64 hours ago. It contained _NO_ transactions. I would say that createnew block being merely 29.5 times faster than the old code it was running until recently instead of 30x faster won't matter. ... except it won't even see that difference when it mines empty blocks with no transactions at all. 20:47 < gmaxwell> When it does actually include transactions-- it appears to produce maximum size blocks just like everyone else: https://blockchain.info/block/0000000000000000050631b932ed81eb9de6267f1cb8b8a353dd59365fd07fd5 21:53 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 22:22 -!- jtimon [~quassel@55.31.134.37.dynamic.jazztel.es] has quit [Ping timeout: 244 seconds] 22:30 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 22:42 -!- d_t [~textual@83-244-233-135.cust-83.exponential-e.net] has joined #bitcoin-core-dev 22:49 -!- fengling [~fengling@58.135.95.136] has quit [Ping timeout: 240 seconds] 23:17 -!- zooko [~user@73.95.137.83] has joined #bitcoin-core-dev 23:19 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has joined #bitcoin-core-dev 23:27 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has quit [Quit: Leaving.] 23:29 -!- fengling [~fengling@58.135.95.136] has joined #bitcoin-core-dev 23:32 -!- gabridome [~gabridome@net-93-145-200-45.cust.vodafonedsl.it] has quit [Quit: gabridome] 23:33 -!- Silence_ [jb@ip.fi] has joined #bitcoin-core-dev 23:35 -!- d_t [~textual@83-244-233-135.cust-83.exponential-e.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 23:35 -!- d_t [~textual@83-244-233-135.cust-83.exponential-e.net] has joined #bitcoin-core-dev 23:45 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection] 23:45 -!- zooko [~user@73.95.137.83] has quit [Ping timeout: 244 seconds]