--- Day changed Mon Jul 11 2016 00:07 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has left #bitcoin-core-dev [] 00:11 -!- GreenIsMyPepper [~GreenIsMy@2605:6400:20:11aa:189e:28a5:52ed:8948] has quit [Ping timeout: 250 seconds] 00:13 -!- GreenIsMyPepper [~GreenIsMy@2605:6400:20:11aa:189e:28a5:52ed:8948] has joined #bitcoin-core-dev 00:17 -!- GreenIsMyPepper [~GreenIsMy@2605:6400:20:11aa:189e:28a5:52ed:8948] has quit [Ping timeout: 250 seconds] 00:19 -!- GreenIsMyPepper [~GreenIsMy@2605:6400:20:11aa:189e:28a5:52ed:8948] has joined #bitcoin-core-dev 00:31 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-wfhnncmhoswhwwwt] has joined #bitcoin-core-dev 00:32 -!- GreenIsMyPepper [~GreenIsMy@2605:6400:20:11aa:189e:28a5:52ed:8948] has quit [Ping timeout: 250 seconds] 00:34 -!- GreenIsMyPepper [~GreenIsMy@2605:6400:20:11aa:189e:28a5:52ed:8948] has joined #bitcoin-core-dev 01:03 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 01:03 -!- jannes [~jannes@178.132.211.90] has joined #bitcoin-core-dev 01:05 -!- fengling [~fengling@58.135.95.137] has joined #bitcoin-core-dev 01:12 -!- assder [82eb8818@gateway/web/freenode/ip.130.235.136.24] has joined #bitcoin-core-dev 01:19 -!- JackH [~Jack@79-73-186-51.dynamic.dsl.as9105.com] has joined #bitcoin-core-dev 01:29 -!- gabridome [~gabridome@host120-205-dynamic.9-87-r.retail.telecomitalia.it] has quit [Quit: gabridome] 01:37 -!- arubi [~ese168@unaffiliated/arubi] has quit [Ping timeout: 272 seconds] 01:41 -!- arubi [~ese168@unaffiliated/arubi] has joined #bitcoin-core-dev 02:03 < GitHub82> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/67caef673089...26316ffa7dc5 02:03 < GitHub82> bitcoin/master 1ba3db6 Christian von Roques: bash-completion: Adapt for 0.12 and 0.13... 02:03 < GitHub82> bitcoin/master 26316ff Wladimir J. van der Laan: Merge #8289: bash-completion: Adapt for 0.12 and 0.13... 02:03 < GitHub140> [bitcoin] laanwj closed pull request #8289: bash-completion: Adapt for 0.12 and 0.13 (master...completion) https://github.com/bitcoin/bitcoin/pull/8289 02:10 < assder> When is the first release candidate for 0.13 expected? 02:13 < gmaxwell> "Soon" 02:22 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)] 02:52 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 02:53 < GitHub143> [bitcoin] laanwj pushed 3 new commits to 0.12: https://github.com/bitcoin/bitcoin/compare/080457c4ee97...5c8438207661 02:53 < GitHub78> [bitcoin] laanwj closed pull request #8318: [0.12] Backport "Rename OP_NOP3 to OP_CHECKSEQUENCEVERIFY #7540" (0.12...rename_nop3_0.12) https://github.com/bitcoin/bitcoin/pull/8318 02:53 < GitHub143> bitcoin/0.12 ac5577b BtcDrak: Rename OP_NOP3 to OP_CHECKSEQUENCEVERIFY 02:53 < GitHub143> bitcoin/0.12 c4e5688 BtcDrak: Rename NOP3 to CHECSEQUENCEVERIFY in rpc tests 02:53 < GitHub143> bitcoin/0.12 5c84382 Wladimir J. van der Laan: Merge #8318: [0.12] Backport "Rename OP_NOP3 to OP_CHECKSEQUENCEVERIFY #7540"... 02:54 -!- molz [~molly@unaffiliated/molly] has quit [Ping timeout: 246 seconds] 02:54 -!- kadoban [~mud@unaffiliated/kadoban] has quit [Quit: bye] 03:06 -!- kekstone [~dn@c-73-14-252-164.hsd1.co.comcast.net] has left #bitcoin-core-dev ["Closing Window"] 03:09 -!- YOU-JI [~youyouyou@aa036079.dynamic.ppp.asahi-net.or.jp] has joined #bitcoin-core-dev 03:12 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has joined #bitcoin-core-dev 03:14 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has left #bitcoin-core-dev [] 03:16 -!- rrt [49a546fd@gateway/web/freenode/ip.73.165.70.253] has quit [Quit: Page closed] 03:20 -!- mkarrer [~mkarrer@48.red-83-47-111.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 03:29 < GitHub157> [bitcoin] laanwj pushed 2 new commits to 0.12: https://github.com/bitcoin/bitcoin/compare/5c8438207661...1233cb42dde9 03:29 < GitHub157> bitcoin/0.12 fe98533 Jonas Schnelli: [Qt] Disable some menu items during splashscreen/verification state... 03:29 < GitHub27> [bitcoin] laanwj closed pull request #8302: 0.12.2: [Qt] Disable some menu items during splashscreen/verification state (0.12...Mf1607-012qtDebugSplash) https://github.com/bitcoin/bitcoin/pull/8302 03:29 < GitHub157> bitcoin/0.12 1233cb4 Wladimir J. van der Laan: Merge #8302: 0.12.2: [Qt] Disable some menu items during splashscreen/verification state... 03:43 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 03:44 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit] 03:51 < GitHub118> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/26316ffa7dc5...304eff3c614a 03:51 < GitHub118> bitcoin/master 477777f MarcoFalke: [rpcwallet] Don't use floating point 03:51 < GitHub118> bitcoin/master 304eff3 Wladimir J. van der Laan: Merge #8317: [rpcwallet] Don't use floating point... 03:51 < GitHub31> [bitcoin] laanwj closed pull request #8317: [rpcwallet] Don't use floating point (master...Mf1607-rpcFloat) https://github.com/bitcoin/bitcoin/pull/8317 04:12 -!- MarcoFalke [~marco@host235-2.natpool.mwn.de] has joined #bitcoin-core-dev 04:25 -!- jtimon [~quassel@55.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 04:40 -!- arubi [~ese168@unaffiliated/arubi] has quit [Quit: Leaving] 04:41 -!- arubi [~ese168@unaffiliated/arubi] has joined #bitcoin-core-dev 04:54 -!- arubi [~ese168@unaffiliated/arubi] has quit [Quit: Leaving] 04:55 -!- arubi [~ese168@unaffiliated/arubi] has joined #bitcoin-core-dev 04:58 < MarcoFalke> Who guessed it 04:58 < sipa> ? 04:58 < MarcoFalke> Of course windows test are timing out, ow 04:59 < MarcoFalke> I am not sure why they don't work in parallel "out of the box" 04:59 < MarcoFalke> Maybe wine can't handle it? 05:00 < sipa> reduce their parallelism? 05:00 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 05:01 < MarcoFalke> There is already an overhead of about 2 for just using wine. Then, disabling parallel is another slowdown by 4. 05:01 < MarcoFalke> So about 8 times longer rpc test for windows 05:01 < sipa> fun. 05:02 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit] 05:06 < luke-jr> what? why would WINE have any overhead? 05:23 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 05:25 -!- MarcoFalke [~marco@host235-2.natpool.mwn.de] has quit [Quit: MarcoFalke] 06:06 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has joined #bitcoin-core-dev 06:10 <@wumpus> luke-jr: that's not really clear, I'd expect an overhead at startup/shutdown due to setting up the 'virtual windows environment', but not on networking 06:10 -!- harrymm [~wayne@178.162.199.47] has quit [Remote host closed the connection] 06:12 < GitHub175> [bitcoin] jtimon opened pull request #8328: Consensus: Rename: Move consensus code to the consensus directory (master...0.12.99-consensus-rename) https://github.com/bitcoin/bitcoin/pull/8328 06:14 < jtimon> ping kanzure ^^ 06:15 < jtimon> quite trivial to review, just touches the makefile and a bunch of includes 06:16 < kanzure> thanks for the ping 06:16 < jtimon> no problem, thanks for asking for it 06:19 -!- harrymm [~wayne@37.58.59.76] has joined #bitcoin-core-dev 06:25 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 244 seconds] 06:27 -!- YOU-JI [~youyouyou@aa036079.dynamic.ppp.asahi-net.or.jp] has quit [Quit: Leaving...] 06:30 -!- harrymm [~wayne@37.58.59.76] has quit [Quit: Leaving.] 06:31 -!- harrymm [~wayne@37.58.59.76] has joined #bitcoin-core-dev 06:33 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 06:34 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 06:35 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit] 06:36 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has left #bitcoin-core-dev [] 06:41 -!- assder [82eb8818@gateway/web/freenode/ip.130.235.136.24] has quit [Quit: Page closed] 06:42 < jtimon> sipa: what's the advantage of validating this https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L3558 in ContextualCheckBlock() instead of ConnectBlock() ? 06:42 < jtimon> and the same question kind of applies to the other 2 consensus checks in there 06:43 < jtimon> in other words would it be crazy to move all those checks to just connectblock? why? 06:45 < jtimon> or in other words, is it necessary or advantageous to call ContextualCheckBlock() in other places other than ConnectBlock() in some way I'm missing? 06:45 < jtimon> answers from anyone welcomed 06:46 -!- GreenIsMyPepper [~GreenIsMy@2605:6400:20:11aa:189e:28a5:52ed:8948] has quit [Ping timeout: 250 seconds] 06:48 < sipa> jtimon: contextualcheckblock runs before storing a block on disk, connectblock after 06:48 -!- molz [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 06:48 < sipa> so everything that can be in contextualcheckblock should be, as it make attacks that cause disk space wasting harder 06:48 -!- GreenIsMyPepper [~GreenIsMy@209.141.33.28] has joined #bitcoin-core-dev 06:49 < jtimon> I see, so the advantage is to discard offending blocks before storing them on disk for checks that are relatively cheap 06:49 < jtimon> I didn't knew we stored invalid blocks at all 06:49 < jtimon> why do we store invalid blocks? 06:50 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 252 seconds] 06:51 < sipa> because we can't know until we reorg the chainstate 06:51 < sipa> we may accept a block on a side branch that does not overtake the best chai yet 06:51 < sipa> *chain 06:51 < jtimon> but it's still valid, no? 06:52 < sipa> we can't know that without reorg 06:52 < sipa> because we need the utxo set for its parent 06:52 < jtimon> we can know whether a block is valid in its chain without knowing whether it belongs to the "longest" chain or not, right? 06:52 < sipa> in theory, yes 06:52 < sipa> in practice, no 06:52 < sipa> because we don't have its utxoset we can't validate 06:53 < jtimon> oh, I see, we never build incompatible utxo histories 06:53 < sipa> right, we just have a single utxo set for the tip 06:54 < sipa> and delay full validation until we're forced to switch tips 06:54 < jtimon> I thought that was part of the CCoinsViewCache purpose... 06:55 < jtimon> but I guess yeah, it doesn't make sense to fully validate until you're forced to reorg by the accumulated pow, storing that block is cheaper I guess 06:55 < sipa> no, that's just so we don't have to write everything to disk immediately 06:55 < sipa> now, pow is the most expensive thing for an attacker 06:56 < sipa> so everything that comes after pow validation and corruption checks does not matter very much 06:56 < sipa> why are you asking, btw? 06:56 < sipa> ah, i know 06:56 < sipa> iswitnessenabled depends on versionbits 06:56 < jtimon> mhmm, why do we validate any part of a block that we don't see as belonging to the longest chain at all? 06:57 < sipa> because we at least need to know whether it is actually the right block 06:57 < sipa> a peer could change one transaction before relay 06:57 < jtimon> sipa: it was part of my latest consensus refactor branch to get rid of ContextualCheckBlock() 06:58 < sipa> and in that case we must punish the peer, but not mark the block as invalid 06:58 < sipa> so an invariant is that all blocks on disk are known to actually match the blocks whose headers we know 06:59 < jtimon> if we know it's not making a longer chain, why bother validating BIP113 ? 07:00 < sipa> validating bip113 is not necessary i think 07:00 < sipa> but the segwit commitment check is 07:01 < sipa> before storing on disk 07:01 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 252 seconds] 07:02 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 07:03 < jtimon> that's useful, what about the bip34 check in ContextualCheckBlock(), can it be moved out? 07:04 < sipa> i believe so 07:04 < jtimon> ok, thanks 07:04 < jtimon> but not the new segwit one, fine 07:06 < jtimon> well, I would prefer to pass the consensus validation flags instead of calling IsWitnessEnabled() from there 07:08 < jtimon> but I should try to write that first before discussing it further 07:11 -!- Sosumi [~Leon@bl10-113-190.dsl.telepac.pt] has joined #bitcoin-core-dev 07:32 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 250 seconds] 07:32 -!- achow101 [~achow101@pool-108-2-58-173.phlapa.fios.verizon.net] has quit [Quit: Leaving] 07:36 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 07:37 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit] 07:42 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 07:46 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 08:24 < sdaftuar> reviewers, please review some open PRs for 0.13.0: #8295, #8305, #8312 08:38 -!- harrymm [~wayne@37.58.59.76] has quit [Quit: Leaving.] 08:38 -!- harrymm [~wayne@37.58.59.76] has joined #bitcoin-core-dev 08:42 -!- harrymm [~wayne@37.58.59.76] has quit [Client Quit] 08:42 -!- harrymm [~wayne@37.58.59.76] has joined #bitcoin-core-dev 08:43 -!- bsm117532 [~mcelrath@38.121.165.30] has quit [Read error: Connection reset by peer] 08:43 -!- bsm117532 [~mcelrath@38.121.165.30] has joined #bitcoin-core-dev 08:44 -!- bsm117532 [~mcelrath@38.121.165.30] has quit [Remote host closed the connection] 08:44 -!- bsm117532 [~mcelrath@38.121.165.30] has joined #bitcoin-core-dev 08:45 -!- harrymm [~wayne@37.58.59.76] has quit [Read error: Connection reset by peer] 08:45 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Quit: WeeChat 0.4.2] 08:46 -!- achow101 [~achow101@pool-108-2-58-173.phlapa.fios.verizon.net] has joined #bitcoin-core-dev 09:04 -!- a87ry5 [~a87ry5@cpe-66-108-174-124.nyc.res.rr.com] has joined #bitcoin-core-dev 09:07 -!- GreenIsMyPepper [~GreenIsMy@209.141.33.28] has quit [Ping timeout: 252 seconds] 09:12 -!- a87ry5 [~a87ry5@cpe-66-108-174-124.nyc.res.rr.com] has quit [] 09:13 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has joined #bitcoin-core-dev 09:16 -!- harrymm [~wayne@37.58.59.76] has joined #bitcoin-core-dev 09:17 -!- GreenIsMyPepper [~GreenIsMy@2605:6400:20:11aa:189e:28a5:52ed:8948] has joined #bitcoin-core-dev 09:34 -!- BTCking [~wowperson@c-67-183-151-251.hsd1.wa.comcast.net] has joined #bitcoin-core-dev 09:34 < BTCking> AMAZING! My bitcoin-grower is now ready. Send me some bitcoin (1btc, 2btc, etc.) and get MUCH more back INSTANTLY. This uses Block-chain explanding technology! Proven safe & secure. You cannot lose. PM me to begin!! 09:35 < sipa> BTCking: fuck off 09:35 * BTCking slaps sipa around a bit with a large trout 09:35 < BTCking> AMAZING! My bitcoin-grower is now ready. Send me some bitcoin (1btc, 2btc, etc.) and get MUCH more back INSTANTLY. This uses Block-chain explanding technology! Proven safe & secure. You cannot lose. PM me to begin!! 09:36 -!- sipa [~pw@2a02:348:86:3011::1] has quit [Changing host] 09:36 -!- sipa [~pw@unaffiliated/sipa1024] has joined #bitcoin-core-dev 09:36 -!- mode/#bitcoin-core-dev [+o sipa] by ChanServ 09:37 -!- mode/#bitcoin-core-dev [+b *!*wowperson@*.hsd1.wa.comcast.net] by sipa 09:37 -!- BTCking was kicked from #bitcoin-core-dev by sipa [BTCking] 09:37 -!- LeMiner2 [LeMiner@5ED1AFBF.cm-7-2c.dynamic.ziggo.nl] has joined #bitcoin-core-dev 09:39 -!- LeMiner [LeMiner@unaffiliated/leminer] has quit [Ping timeout: 252 seconds] 09:39 -!- LeMiner2 is now known as LeMiner 09:47 < GitHub65> [bitcoin] jtimon opened pull request #8329: Consensus: MOVEONLY: Move functions for tx verification (master...0.12.99-consensus-moveonly-tx) https://github.com/bitcoin/bitcoin/pull/8329 09:49 < jtimon> kanzure: ping again, still quite simple to review and re-write 10:04 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has quit [Quit: Leaving] 10:09 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has joined #bitcoin-core-dev 10:10 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has quit [Quit: Leaving.] 10:13 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 250 seconds] 10:14 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 10:16 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 10:28 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 264 seconds] 10:30 -!- droark [~droark@c-24-22-36-12.hsd1.or.comcast.net] has joined #bitcoin-core-dev 10:35 -!- BCBot [~BCBot@46.101.246.115] has quit [Remote host closed the connection] 10:35 -!- BCBot [~BCBot@46.101.246.115] has joined #bitcoin-core-dev 10:38 -!- molz [~molly@unaffiliated/molly] has quit [Ping timeout: 240 seconds] 10:39 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 10:46 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has joined #bitcoin-core-dev 10:46 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has quit [Remote host closed the connection] 10:48 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has joined #bitcoin-core-dev 10:49 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has quit [Remote host closed the connection] 10:50 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has joined #bitcoin-core-dev 10:51 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has quit [Remote host closed the connection] 10:52 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has joined #bitcoin-core-dev 10:54 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has quit [Remote host closed the connection] 10:55 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has joined #bitcoin-core-dev 10:55 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has quit [Remote host closed the connection] 10:57 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has joined #bitcoin-core-dev 10:57 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has quit [Remote host closed the connection] 11:00 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has joined #bitcoin-core-dev 11:00 -!- BCBot2 [~BCBot2@201.158.62.81.dynamic.wline.res.cust.swisscom.ch] has quit [Remote host closed the connection] 11:03 -!- mkarrer [~mkarrer@48.red-83-47-111.dynamicip.rima-tde.net] has quit [] 11:10 -!- kadoban [~mud@unaffiliated/kadoban] has joined #bitcoin-core-dev 11:10 <@sipa> sdaftuar: done 11:10 -!- mode/#bitcoin-core-dev [-o sipa] by sipa 11:10 < sdaftuar> sipa: thanks! 11:13 < sipa> github.com down? 11:15 < eragmus> sipa: http://www.isup.me/github.com 11:15 < eragmus> http://isitdownorjust.me/github-com/ 11:16 < sipa> thanks 11:16 < sipa> it also works from my vps... 11:16 < tadasv> working fine for me 11:19 -!- mkarrer [~mkarrer@48.red-83-47-111.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 11:19 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has joined #bitcoin-core-dev 11:33 < gmaxwell> sdaftuar: maybe it doesn't matter because its a corner case, but #8305 makes it look like it will effectively discard the dangling header-- it pushmessages a getheaders and returns, without AcceptBlockHeader or UpdateBlockAvailability. 11:33 < sdaftuar> oh, UpdateBlocKAvailability, hmm 11:34 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 11:34 < gmaxwell> Is the expectation that it'll come back in via the getheaders, and that the remote end won't do anything smart like not send it because it already did? (I guess thats reasonable?) though the lack of updateavailablity probably means we'll send a redundant announcement in that direction. 11:35 < sdaftuar> well there's no way to avoid getting the headers message back to you 11:35 < sdaftuar> if you set hashstop equal to the hash of that header, it'll still appear in the response 11:35 < sdaftuar> i don't think the remote end is allowed to not re-send, though i suppose lack of a spec makes this a difficult statement to concretely make 11:35 < sdaftuar> but we expect the contents of a headers message to be a linear connecting chain 11:36 < gmaxwell> K. That makes sense, I thought that was the assumption but wanted to be sure (I think it's correct.) 11:37 < sdaftuar> i guess there could be an advantage to setting hashLastUnknownBlock if a header doesn't connect... that would mean that if a different peer supplied the missing headers, we could download the block from the peer who made the announcement 11:37 < sdaftuar> but that would only matter if the peer didn't respond to the getheaders being pushed in this patch 11:38 < gmaxwell> so now let me ask, if I send you a header that doesn't connect because of timestamp stupidity. And I'm your only peer. If there are two blocks in a row during the time when the prior header will be rejected... When will it start connecting again? I'll get the flag false on me, then it'll never be sent to true, since all further headers I send will also not connect. 11:38 < sdaftuar> which is a weird corner case 11:38 < gmaxwell> On the update front, I was mostly think in terms of not sending surpflous compact block messages. 11:38 < sdaftuar> when you respond to the getheaders, the bit is cleared 11:39 < midnightmagic> ;;isitdown github.com 11:39 < gribble> github.com is up 11:40 < sdaftuar> and the headers will start connecting again 11:40 < sdaftuar> well, 11:40 < sdaftuar> unless the time stamp is still broken 11:40 -!- xinxi [74569cde@gateway/web/freenode/ip.116.86.156.222] has joined #bitcoin-core-dev 11:40 < gmaxwell> hm. I was thinking that it wouldn't be set to true if the getheaders reply also didn't connect due to bad timestamps. 11:41 < xinxi> Hi guys 11:41 < xinxi> Anybody here? 11:42 < sdaftuar> gmaxwell: yeah, i can't solve the problem of a bunch of blocks in a row all being invalid to one peer but not another 11:42 < sdaftuar> but typically, the first block is too far ahead 11:42 < sdaftuar> and then by the time the next block comes in, you would be able to accept both 11:42 < sdaftuar> if you'd be willing to try 11:42 < sdaftuar> but we have no "try" mechanism currently 11:43 < sdaftuar> i guess the question is, what should we do if a peer is relaying many blocks in a row that are all actually too far in the future? 11:43 < sdaftuar> i just fall back to the current behavior of failing/disconnecting 11:45 < sdaftuar> i did briefly think about whether it would be feasible to somehow accept headers that are too far in the future, but i'm pretty sure that's a major rewrite to our consensus logic 11:45 < sdaftuar> (accept them, so that you could re-try them later) 11:50 < sdaftuar> gmaxwell: perhaps we could always send a getheaders if the peer is whitelisted? 11:51 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has quit [Quit: Leaving.] 11:52 < gmaxwell> I don't think that would be an awful thing, but I don't think it addresses the concern-- you shouldn't only fail to fail if you have whitelisted peers. 11:53 < sdaftuar> i guess i could do the thing you imply -- actually just check to see if the headers connect, and use that to re-set the bit 11:55 < gmaxwell> maybe I'm misunderstanding, but that situation I'm thinking about is if you're slow on time by a few seconds, rejected a block, then the next comes in, and it still doesn't connect... you're going to stop sending getheaders after that. Then no others will connect from that peer. The time is a bit freaky, since it's a temporary failure. 11:55 < sdaftuar> yes, i agree your example is a correct one 11:55 < sdaftuar> i'm just trying to distinguish that from an example where an attacker sends you a chain that's far into the future 11:56 < gmaxwell> I think we award a little bit of misbehaving there. Ironically, for sync purposes it might be better to immediately disconnect: at least that will clear the flag! :) 11:58 < gmaxwell> relying on DOS disconnect to make progress is weird though, but I think thats the answer here: after some number non-connecting blocks I think you'll disconnect the peer. When you reconnect, the flag will be true and you'll try to sync again. 11:59 < gmaxwell> sdaftuar: to prevent looping, why not store the most recent ID. And don't getheader again if it's the same? 12:00 < gmaxwell> e.g. block announce, doesn't connect, store the most recent ID and getheader. Get headers, still doesn't connect and most recent ID is the same. Stop. 12:00 < sdaftuar> sorry, what is most recent ID? 12:01 < gmaxwell> when the headers response comes in take the ID (hash) of the latest block in the reply. And do not keep requesting headers when that value hasn't changed for this peer. 12:02 < gmaxwell> I announce X, X doesn't connect. Remember X, request headers. Headers come in, they tell you a bunch of junk ending at X. Still doesn't connect. Stop. 12:02 < sdaftuar> gmaxwell: i guess i just worry about assuming too much about the broken implementations out there 12:02 -!- xinxi [74569cde@gateway/web/freenode/ip.116.86.156.222] has quit [Quit: Page closed] 12:02 < sdaftuar> eg maybe the peer who was sending me 160kb of junk was sending me random junk 12:04 < gmaxwell> Well thats what misbehaving is for. 12:04 -!- xinxi [~xinxi@116.86.156.222] has joined #bitcoin-core-dev 12:04 < sdaftuar> but in your example, wouldn't that behavior result in infinite loop? 12:05 < gmaxwell> The headers reply would end with the block X, which we already had remembered, so we would stop there. 12:05 < gmaxwell> no loop. then they later announce block y, and we'd getheaders again. and hopefully connect. 12:05 < gmaxwell> So basically one headers retry per novel header they send. 12:05 < sdaftuar> i'm specifically referring to peers that are totally broken and on initial getheaders sync, they send us 2000 headers that don't connect 12:06 < sdaftuar> i don't want to assume too much about how that peer is doing that 12:06 < gmaxwell> sure, whatever, testnet node on mainnet or whatnot. 12:06 < sdaftuar> if getting it wrong means 160kb over and over again... 12:07 < gmaxwell> just don't assume it's malicious, since there are better ways to waste our bandwidth for malicious peers. 12:08 < sdaftuar> well you're suggesting we assume it's deterministic i think. which is hopefully true, but if it's not, then sadness 12:08 < gmaxwell> yes, indeed, and actually the 2000 limit gets in the way too. 12:09 < sdaftuar> i mean, i think in practice your suggestion probably works, i was just hoping for something more robust to accidental disaster 12:10 < gmaxwell> speaking of 2000, I'm not seeing how what you suggest doesn't actually break sync. 12:10 < sipa> can i get a summary of the problem? 12:11 < gmaxwell> oh nevermind last comment, I was mistaking the order. 12:11 < sdaftuar> sipa: i noticed on testnet that i had a node that would fall out of sync, because its one peer would occasionally send a block header (using headers announcements) that was too far in the future 12:11 < gmaxwell> (I was thinking that we'd request headers, if there was more than 2000 missing, that they sent the later headers first) 12:11 < sdaftuar> so the peer thought the header was valid, but the local node would reject 12:11 < sdaftuar> this is a case i had never thought of when working on the headers announcements stuff 12:12 < sdaftuar> since the announcing peer doesn't realize the recipient rejects the header, they continue to announce blocks on top of the header 12:12 < sipa> sdaftuar: yes, i get that 12:12 < sipa> but you have a patch for that 12:12 < sdaftuar> resulting in headers announcements that don't connect for the recipient, and then eventually disconnect 12:13 < xinxi> hey guys, I want to make Bitcoin provably correct. 12:13 < sdaftuar> oh right. so gmaxwell wonders if we can avoid failure in the case where two blocks come in quickly, where the first block is still invalid when the second one is announced 12:13 < sdaftuar> my patch doesn't address that 12:13 < xinxi> I am serious. 12:13 < sipa> xinxi: you'll need to be a bit more specific 12:13 < sipa> what part do you want to prove? 12:14 < gmaxwell> sipa: and sdaftuar is concerned that relaxations will leave it in a state where its constantly requesting headers over and over again. 12:14 < xinxi> sipa: thank you for your attention. Yeah, basically, I think bugs are still in the C++ implementation. And I want to make sure it's absolutely correct. 12:14 < xinxi> so there will be no bugs. 12:14 < sipa> xinxi: that implies you have a specification for correct to compare to 12:14 < gmaxwell> I think this could be addressed by just adding a small amount of DOS each time. 12:15 < gmaxwell> Correct with respect to a specification may well be meaningless if the specification is not "correct" in a broader sense. 12:15 < xinxi> sipa: yeah, I want to work out a formal specification as well. 12:15 < sipa> xinxi: for the consensus logic, the code implementation is the specification, so it is by definition correct 12:15 < xinxi> something like mathematical theorems about the code. 12:15 < sipa> xinxi: however, being able to verify that the behaviour of that code does not change between releases would be incredibly valuable 12:16 < sipa> unfortunately, due to historical reasons mostly, the consensus code is not well separated from the rest 12:16 < gmaxwell> sdaftuar: e.g. I think we could just trigger a getheaders on any header message with only a single header in it that doesn't connect, and add a small dos penalty. 12:16 < xinxi> sipa: I understand what you mean. The code, as a specification, may not be consistent with itself. 12:16 < gmaxwell> sdaftuar: then every new block that shows up on the network will cause us to retry to connect. 12:16 < morcos> i know this sounds kind of janky, but it seems to me that a lot of this sync logic is stuff that to a human is kind of obviously stupid. i wonder if putting in a bit of belt and suspender check on things getting stuck might make sense. 12:17 < sipa> xinxi: indeed. not consistent with itself, or not consistent with other implementations/versions 12:17 < gmaxwell> sdaftuar: and eventually it will either connect, or disconnect the peer. 12:17 < xinxi> sipa: also, you may not want to treat bugs as part of the specification like the heartbleed bug in OpenSSL should not be part of OpenSSL. 12:17 < sipa> xinxi: we have to 12:17 < btcdrak> xinxi: there is a slow and arduous process of extracting the consensus code into a separate library which will make things a lot easier in the long run. 12:18 < morcos> for instance very 60 secs you clear out some state such as fLastHeadersConnected and perhaps nSyncStarted (maybe only if we're not actively receiving headers) 12:18 < morcos> i'm sure we can imagine other things that kind of get stuck 12:18 < xinxi> btcdrak: I've heard of that. It's libconsensus, isn't it? 12:18 < gmaxwell> morcos: you mean like a periodic, pick a random peer and restart our efforts to sync with it? and loudly logging that the sync state machine has failed if that accomplishes anything? 12:18 < sipa> xinxi: even if we would write down a specification for the behaviour, and every one would agree that that spec is indeed the behaviour we want 12:18 < sipa> xinxi: what if we then find a bug, and the code does something slightly different? 12:18 < sdaftuar> gmaxwell: i think that sounds pretty good, though the DoS score should decay back down once headers that do connect are received, right? otherwise, over a long enough period of time, you eventually disconnect your peer 12:19 < morcos> gmaxwell: yes something along those lines, i like the idea of loudly announcing it so we try to make stuff work in general, but that next time we miss something, hopefully it kind of autorecovers (like power cycling) 12:19 < btcdrak> xinxi: the problem is it's quite disruptive to the codebase and causes everyone's pull-requests to need rebasing so it tends to get done slowly and in bursts. yes it's called libconsensus 12:19 < gmaxwell> sdaftuar: yea, :( the whole dos score thing is kinda lame. 12:19 < xinxi> btcdrak: but still, that probably reduces bugs in the consensus part, but still it does not guarantee the correctness. 12:19 < sipa> xinxi: the code would have a bug, arguably. but we can't tell everyone to change their code, so it will need to be the document that had to be changed 12:19 < gmaxwell> morcos: I agree with that in principle. We don't however, want to end up with a web of redundant mechenisms, potentially interacting. 12:20 < sipa> xinxi: so we've usually said that a specification for bitcoin would be descriptive but not prescriptive... the laws of the network are those implemented by the code that people choose to run, not by an abstract descriptio 12:20 < morcos> a bigger web you mean? 12:20 < gmaxwell> morcos: hah 12:20 < sdaftuar> unfortunately some of the bugs we've encountered (eg that hasHeaders thing that got reverted) wouldn't be fixed on restart either 12:20 < sipa> for 0.14 i think we need to refactor the sync code 12:21 * sdaftuar ducks 12:21 < sipa> many parallel mechanisms and duplicated code 12:21 < gmaxwell> sdaftuar: you could instead have a simple counter, set to zero whenever you connect, incremented on every non-connecting header response. And dos score only when it hits a threshold to avoid that effect. 12:21 < sipa> no blame anywhere, but we've accumulated too much technical debt 12:21 < sdaftuar> gmaxwell: yep that's what i was thinking too 12:21 < gmaxwell> sdaftuar: I believe thats the first sync stuck bug I've ever seen that wasn't fixed on a restart. 12:22 < sdaftuar> i think i've seen 0.9 nodes not be able to recover from restart 12:22 < xinxi> sipa: we don't have to be like that. I was motivated by the project CompCert and SEL4. SEL4 is a OS kernel that's proved to be 100% consistent with the specification. 12:22 < sipa> xinxi: that implies you have a specification 12:22 < sipa> as i've said before, we can't have one 12:22 < sipa> we can describe the current behaviour 12:22 < gmaxwell> sdaftuar: well there was also the "stops on too many orphans" stuff between ~0.8 and 0.10. 12:22 < sipa> and then formally prove that future code matches that behaviour 12:23 < gmaxwell> (where it would start fetching blocks backwards, run into a 750 block orphan limit then stop, but even that could recover with enough restarts) 12:24 < xinxi> sipa: I just don't understand the factor that stops us from driving a formal specification from the code. 12:24 < gmaxwell> sdaftuar: in any case, I think doing get headers triggered on single header responses, and counting the failures, and DOSing on too many failures... would address both our concerns here. 12:25 < sipa> xinxi: deriving? yes, we could 12:25 < sdaftuar> gmaxwell: sounds right to me as well. for 0.13.0? 12:25 < sipa> xinxi: i guess my point is that the only way to write a formal specification is by extracting it from the actually implemented and deployed code 12:25 < xinxi> sipa: yeah, that's where we can start from. 12:25 < sipa> xinxi: and not from behaviour that humans write 12:25 < btcdrak> xinxi: the difficulty is if the code differs from the written specification the code wins. Even if you class it as a bug, changing it is extremely hard because you have to change the network consensus. There are examples where this has happened and we've just had to live with the bug. 12:26 < gmaxwell> Yes, I think it wouldn't be more complex than that patch you currently have. (this is also not just a testnet problem-- we're just fortunate that miners have lately not be excessively rolling their timestamps forward...) 12:26 < btcdrak> xinxi: the other problem is unknown bugs which we might not be aware of, rare edge cases say, are actually part of the consensus rules even if we dont know it. 12:26 < sdaftuar> ok i'll see if i can whip up an improved patch 12:26 < gmaxwell> btcdrak: the kind of "formal specification" in the sense of SEL4 can be proven to agree with the code (effectively, the specification is at least as complex as the implementation) 12:26 < btcdrak> xinxi: so it's hard to document things you dont know 12:27 < xinxi> @btcdrak those rare edge bugs can be eliminated by formal proofs I think. 12:27 < btcdrak> xinxi: seems like a very worthwhile pursuit though. 12:27 < xinxi> so the motivation of this is to avoid catastrophic failures of Bitcoin. 12:28 < sipa> xinxi: for example, for a long time, bitcoin relied on OpenSSL's signature parsing code 12:28 < sipa> xinxi: people thought they knew what this code did 12:28 < sipa> xinxi: and reimplemented the logic in other places 12:28 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 12:28 < sipa> but nobody checked that what they thought actually matched what openssl was doing 12:28 < sipa> turns out, it didn't 12:29 < sipa> so now bitcoin's "specification" implicitly depended on OpenSSL's implementation 12:29 < xinxi> sipa: this kind of inconsistencies can be eliminated by formal proofs as well. 12:29 < sipa> xinxi: absolutely 12:29 < sipa> xinxi: but only if they would take OpenSSL's code into account and analyse it, and not treat it as a black box 12:30 < gmaxwell> xinxi: I'm not sure if you have a grasp of the difficulty of what you're thinking about, go look at the size of the SEL4 kernel. I believe it's about 4000 lines of C. The isabelle specification for it is something like a half million lines, and though it proves many useful things about it, it falls far short of proving so much that agreement with the spec would be equivilent to correct in a hum 12:30 < gmaxwell> an sense. 12:30 < sipa> and this cross-layer effect on consensus makes it IMHO nearly impossible 12:30 < xinxi> gmaxwell: it's about 9000 lines of code. 12:30 < sipa> but i am not an expert on the state of the art of formal code proofs 12:30 < gmaxwell> xinxi: k, my figures are from memory thus approximate. 12:31 < xinxi> gmaxwell: SEL4 has to guarantee high efficiency. So the code has to be written in C which is difficult to prove. 12:32 < gmaxwell> xinxi: yes, and Bitcoin has to achieve high efficiency, efficiency is a security parameter for us. 12:32 < xinxi> For Bitcoin, the programming language is not a big issue, and using languages like Coq, which enables a unified way for both coding and proofs could make it a lot easier. 12:32 < gmaxwell> No. incorrect. 12:32 < gmaxwell> If nodes are not fast enough the system will stop converging, at some point this results in a total consensus failure. 12:32 < xinxi> I think OCaml is quite decent in terms of efficiency. Coq can generate OCaml code. 12:32 < sipa> well we're not going to rewrite the consensus code in another language! 12:33 < gmaxwell> Coq can generate C code via the compcert formalization (see VST). 12:33 < xinxi> gmaxwell: here we go. 12:33 < sipa> (or at least not without proving that the existing code matches the new code) 12:33 < xinxi> how many lines of code are there for the consensus part? 12:34 < xinxi> rewriting the consensus part is not infeasible after you separate it out. 12:34 < sipa> do you include the cryptography library? the c++ standard library? the kernel? 12:34 < gmaxwell> Far more than SEL4, the crypto part of it alone is that big. 12:34 < sipa> changes in any of those can affect consensus 12:34 < sipa> at some point you have to make an abstraction 12:34 < kanzure> "formal proofs" cannot eliminate edge cases, they can only prove that the edge case exists 12:34 < xinxi> sipa: we can leave cryptography part there first. C++ is just hopeless for verification. It's too complicated. 12:35 < sipa> xinxi: then i think there is nothing we can do 12:35 < kanzure> xinxi: how are you migrating the network precisely? 12:35 < kanzure> wtf? 12:35 < gmaxwell> chill chill. 12:35 < sipa> these are intesting things to discuss 12:35 < Chris_Stewart_5> xinxi: I think someone wrote a libsecp256k1 for ocaml if you are serious about implementing 12:35 < Chris_Stewart_5> using ctypes 12:36 < sipa> Chris_Stewart_5: what if the ocaml implementation doesn't match the C one? 12:36 < sipa> we won't know without analysing the one that is actually used 12:36 < sipa> (libsecp256k1, by the way, does have formal proofs for part of its operation) 12:36 < xinxi> I mean we can separate out libconsensus first, and then use Coq to rewrite it and generate a provably correct version of libconsensus. 12:36 < kanzure> xinxi: there are a lot of subtle side effects from the actual running network, but everyone in here still thinks it would be valuable to pursue formal verification anyway 12:36 -!- gmaxwell [greg@wikimedia/KatWalsh/x-0001] has left #bitcoin-core-dev [] 12:37 < sipa> xinxi: how will you prove that the existing libconsensus (which only does script validation, not full block validation) matches that Coq specification? 12:37 < kanzure> libconsensus imho should be a priority similar to segwit 12:37 < sipa> kanzure: i agree. but we first need a plan for what 'libconsensus' means. people have been talking about it for ages, and have meant very different (and conflicting) things with it 12:38 < xinxi> sipa: we don't prove the existing libconsensus. We derive the formal specification based on the existing libconsensus, and then write another libconsensus in Coq. 12:38 < kanzure> sipa: maybe we can hijack the milan conference for these purposes (scaling bitcoin 3) 12:38 < GitHub72> [bitcoin] JeremyRubin opened pull request #8330: WIP: Structure Packing Optimizations in CTransaction and CTxMemPoolEntry (master...structurepacking) https://github.com/bitcoin/bitcoin/pull/8330 12:38 < Chris_Stewart_5> sipa: It just binds to the C library, so unless there is a bug in ctypes you should be ok 12:38 < sipa> xinxi: does you derivation prove that the specification matches the current libconsensus? 12:39 < sipa> sorry; does you derivation result in a specification that provably captures the current behaviour? 12:39 < xinxi> sipa: no. do they have to match exactly? 12:39 < sipa> Yes. 12:39 < xinxi> we just take off there. 12:39 < kanzure> can coq do outputs re: "behavior is definitely undefined here"? 12:40 < sipa> xinxi: the specification for the network is the actually deployed code 12:40 < sipa> any divergence from it can result in a network fork 12:40 < xinxi> sipa: I think there can be slight difference? 12:40 < sipa> xinxi: such as? 12:40 < sipa> if you can't describe the current code exactly, we're done 12:40 < kanzure> once you announce a difference, soeone will probably try to trigger those circumstances anyway :) 12:41 < sipa> we won't rewrite the consensus layer in another language without certainty that it does not introduce any changes compared to the current code 12:42 < xinxi> have you guys talked about this before? 12:42 < kanzure> endlessly 12:42 < sipa> yes 12:42 < xinxi> many times? 12:42 < sipa> xinxi: i don't think you're understanding what i'm saying 12:42 < sipa> we do not prescribe the rules of the network 12:42 < kanzure> xinxi: there is high interest in our community in formal verification and libconsensus and coq. 12:43 < Chris_Stewart_5> ^^^ 12:43 < petertodd> sipa: along with the pragmatic requirement that the new language have a sufficiently large community of programmers to be maintainable - unusual languages don't need that criteria 12:43 < sipa> petertodd: agree, but i wasn't going to bring that up 12:43 < sipa> the first point is that our compatibility concerns are way beyond almost any other system 12:44 < xinxi> @petertodd that may not be a problem. Later the code will be proof driven. And the core team writes the specification, and the committers need to prove the correctness of their code, which makes them impossible to make any mistakes. 12:44 < kanzure> xinxi: this seems like something you would like to work on? 12:44 < sipa> xinxi: i'm getting annoyed 12:44 < sipa> xinxi: the core team does not write the specification 12:45 < xinxi> @kanzure yes, it is. And two professors in my school, i.e. National University of Singapore, are willing to work in it as well. They are experts on software verification. 12:45 < petertodd> ...and the specification will change over time as new features are soft-forked in, and specification flaws are fixed 12:45 < sipa> xinxi: that's very interesting 12:45 < xinxi> and we may have big funding support. 12:45 < xinxi> the purpose is simple. we want to make bitcoin reliable. 12:46 < sipa> xinxi: but i first want to understand why you say "< xinxi> sipa: I think there can be slight difference?" 12:46 < Chris_Stewart_5> petertodd: Specifications can be changed overtime, and theoretically the proofs should reflect those changes 12:46 < xinxi> the impact could be huge. 12:46 < kanzure> xinxi: there are many bitcoin developers who want that as well. i think you could easily find collaborators from this conversation. 12:46 < petertodd> xinxi: it's not going to be huge - the reliability of the consensus specification/implementation hasn't been a major problem for bitcoin - other problems are far higher-impact (scaling) 12:47 < petertodd> xinxi: I'd suggest you focus your efforts on smart-contract use-cases, where specification flaws have been a huge problem (ethereum dao) 12:47 < sipa> hey 12:47 < kanzure> petertodd: nooo we do need some people thinking about libconsensus things 12:47 < xinxi> petertodd: I think we are just lucky that Bitcoin has not yet experienced any catastrophic attacks? 12:47 < Chris_Stewart_5> petertodd: I disagree, the consensus layer is the bedrock of bitcoin, if we screw that up in a major way we are done 12:48 < kanzure> smart contract stuff is already getting formal verification attention, i woudn't worry about that, it was fairly high profile and papers have been published already 12:48 < petertodd> kanzure: I'm not saying we don't, it's just that from the point of view of a researcher, they're going to get better return on their time by doing other things 12:48 < kanzure> disagree 12:48 < xinxi> petertodd: not all researchers work for papers. 12:48 < petertodd> xinxi: it's not luck... it's how we make changes - turns out that intense review works 12:48 < xinxi> impact is much more important. 12:48 < Chris_Stewart_5> petertodd: Until it doesn't 12:49 < Chris_Stewart_5> Bitcoin has already had issues with this in the past 12:49 < xinxi> petertodd: OpenSSL has been used by so many people. Still it has serious bugs. 12:49 -!- jtimon [~quassel@55.31.134.37.dynamic.jazztel.es] has quit [Ping timeout: 276 seconds] 12:49 < Chris_Stewart_5> Bip66 (or was it 62) comes to mind 12:49 < sipa> how many consensus failures in bitcoin can you name? :) 12:49 < sipa> yes, that's one 12:49 < sipa> the march 2013 fork was another one 12:49 < btcdrak> xinxi: you should probably also have a conversation with jtimon (who just pinged out). he's going a lot of the libconsensus stuff. 12:50 < petertodd> Chris_Stewart_5: there's no reason to think yet that formal proofs actually do any better in practice, because the pool of talent that can work on them is much smaller 12:50 < sipa> and bitcoin's scripting language before the removal of OP_VER was also a consensus problem 12:50 < sipa> i don't know of any others tbh 12:50 < petertodd> xinxi: indeed, which is why OpenSSL got replaced by libsecp256k1 12:50 < sipa> OP_VER was in 2010 12:50 < sipa> and all the others were due to supporting libraries 12:50 < sipa> xinxi: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-July/009697.html <- read that 12:51 < xinxi> btcdrak: Yeah some one mentioned him to me. 12:51 < sipa> (shameless plug) 12:51 < sipa> xinxi: it gives you an idea of how complicated it was to find some of the actually known consensus failures in bitcoin 12:51 < sipa> the fact that no easier ones have been found may give an indication 12:52 < Chris_Stewart_5> petertodd: I think there actually has been some considerable interest in the formal verification community in bitcoin. I've talked to a handful of researchers myself and I don't think directing them away from the core protocol is a good idea 12:52 < xinxi> I mean the provably correct version of libconsensus may not be consistent with the existing one. The most serious result would be a hard fork, rigth? 12:52 -!- TomMc [~tom@unaffiliated/tommc] has joined #bitcoin-core-dev 12:52 < xinxi> But after that, we will take off. And Bitcoin will be much more secure than before. 12:52 < sipa> xinxi: but we'll need a hard fork to get there? 12:53 < xinxi> if the provably correct version is done, is it worth to have a hard fork? 12:53 < sipa> maybe 12:53 < petertodd> Chris_Stewart_5: I'm not saying there isn't interest, I'm saying that if you want to make a high impact with formal verification deployed in production, Bitcoin isn't interesting because the risks of deploying formal verification in production are higher than the theoretical benefits 12:53 < sipa> xinxi: but there are requirements beyond just correctness 12:54 < petertodd> Chris_Stewart_5: whereas with smart-contract systems, the risks of the existing approaches are known to be very high - likely too high to be practical - so the risks of alternate approaches are less in comparison 12:54 < sipa> xinxi: such as performance, and ease of maintenance 12:54 < btcdrak> xinxi: there are other things than correctness which can cause consensus failure. 12:54 < xinxi> petertodd: sorry, peter, I think current smart contract platforms are trying to solve some hypothetical problems. But this is a bit irrelevant. 12:55 < petertodd> xinxi: speaking of performance, will your formal verification techniques be able to guarantee bounds on execution time/space? 12:55 < kanzure> is jl2012 in singapore? 12:55 < btcdrak> hk 12:55 < kanzure> damn 12:55 < sipa> jl2012 is in hong kong 12:55 < sipa> what petertodd said 12:56 < petertodd> xinxi: *ethereum* is trying to solve a hypothetical problem; systems like R3's Corda are solving very real problems. Yes, they may be problems in centralized systems, but that doesn't make those problems any less real. 12:56 < sipa> i think proving time and space bounds (ideally on the current code!) is probably more useful in practice than a formal specification (which i expect to be much more complicated than you expect, if it is to have production value) 12:56 < xinxi> petertodd: I think most smart contracts should run on AWS. That's much more efficient and easy to use. 12:57 < xinxi> sipa: why do we need to prove time and space bounds? 12:57 < petertodd> xinxi: smart contracts are about verification, not computation; "should run on AWS" makes no sense 12:58 < sipa> xinxi: because if an attack exists that can make validation slow on some nodes, you can take advantage of that (a miner could knock out competition, or get a propagation advantage) 12:58 < petertodd> xinxi: if you exceed the time/space bounds allowed by the actual implementation, the system fails to reach consensus 12:58 < sipa> xinxi: if an attack exists that can make memory grow unbounded, you can knock out validation 12:58 < petertodd> xinxi: in fact, it's fair to say that much of the current development effort is ultimately about making the upper-bound time cost of a block lower 12:58 < sipa> xinxi: for bitcoin's security assumptions to hold, verification of blocks must be negligable in time compared to the block production rate 12:59 < xinxi> sipa: that sounds like the scalability issues that you guys are trying to solve. 13:00 < sipa> well, it's much more a problem today than fear of consensus failure 13:01 < xinxi> petertodd: a simpler way to do smart contracts is to have trustworthy organization to run a platform on AWS and then people can run their smart contracts on the platform. It's scalable, efficient, and can also be verified. 13:02 < petertodd> xinxi: the whole point of smart contracts in banking is to get away from that model 13:02 < btcdrak> xinxi: but we are interested in decentralised systems; there's special about yet another centralised system. 13:02 < sipa> petertodd, xinxi: i think centralized smart contract design is a bit off topic here 13:02 < petertodd> xinxi: verifying computaitons are done correctly is not easy 13:02 < xinxi> petertodd: yeah, we can use Bitcoin in that AWS based smart contract platform. It does not have to be decentralized. 13:03 < petertodd> sipa: point is, I think xinxi would be much better off solving problems in that problem space first, as they're easier to solve with more impact, and that'll give them tools to tackle bitcoin later; misunderstandings about that problem space are indicative of serious misunderstandings with how bitcoin works 13:04 < petertodd> xinxi: again, do you understand my point about verification vs. computation? 13:06 < xinxi> petertodd: you are right. A third party's platform cannot be fully trusted and thus full verification is impossible. But most people's smart contracts don't need that kind of strong verification. 13:06 -!- gmaxwell [greg@wikimedia/KatWalsh/x-0001] has joined #bitcoin-core-dev 13:08 < xinxi> To me, Bitcoin is great because it is solving a real problem. 13:08 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 276 seconds] 13:08 < Chris_Stewart_5> xinxi: Having a formally verified library of smart contracts would be useful. As in any programming language you end up having design patterns, you will see the emergence of patterns in smart contracts imo 13:09 < xinxi> Chris_Stewart_5: it is truly useful in some extreme cases. 13:09 < petertodd> xinxi: I suspect you're unclear as to what smart contracts even do... the sane use-cases I'm talking about, are to formalize legal contracts, allowing adherence to them to be verified - that only works because you can take the state you're in - legally speaking - and verify that it matches the smart contracts (aka protocol definitions) 13:09 < xinxi> And the biggest concern of Bitcoin to me is not the lack of functions but its security. Many people still think it is too young and not reliable enough and could fail completely and thus don't want to adopt it. 13:10 < petertodd> xinxi: bitcoin is exactly the same, except that on top of that, PoW allows our state to converge to a single consensus history 13:10 < petertodd> xinxi: the most likely way bitcoin will fail right now is a failure of decentralization, which is very closely tied to scalability 13:11 < sipa> petertodd: agree, but i still think smart contracts is off topic here. 13:11 < sipa> of the type you're describing 13:11 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 13:11 < petertodd> sipa: again, I'm trying to explain to xinxi how this stuff works - not make smart contracts the topic... 13:12 < sipa> ok, great 13:12 < sipa> xinxi: i think it's unlikely that people will accept a hard fork to switch to a more formally provable consensus implementation 13:13 < xinxi> petertodd: to me, smart contracts are just contracts in code. 13:13 < sipa> i think you're talking about different things 13:13 < petertodd> xinxi: that's not any different from what I said... legal contracts are verification, not computation - they're about conditions that need to be met for something to be valid - aka a protocol specification 13:14 < petertodd> xinxi: equally, the definition of the bitcoin protocol is what the bitcoin implementation accepts as valid - the fact the code that does that is somewhat intermixed with code that records state is just a historical mistake 13:14 < xinxi> yeah, a coded contracts can be run on AWS and still legally binding. 13:15 < xinxi> you just write the legal part in terms of conditions. 13:15 < sipa> xinxi: but if it has very clear benefits (like performance/resource bounds), is easy to work with... maybe, but that's not something for us to decide 13:15 < btcdrak> xinxi: in this space smart contracts are self enforcing contract 13:15 < sipa> xinxi: i don't think people should ever expect a future hard forking change to succeed 13:15 < petertodd> xinxi: why do you keep saying AWS here? that's completely missing the point: what you want is to be able to show to someone else (e.g a judge, but hopefully it doesn't go that far) that the smart contract hasn't been upheld 13:16 < sipa> xinxi: so imho for formal verification to be practical in the short term, it has to be about the currently deployed code 13:16 < petertodd> btcdrak: self-enforcing because of the underlying proof-of-work layers ensuring consensus, and the fact that people widely choose to accept the outputs of the system - bitcoin being a perfect - albeit inflexible - example 13:16 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 264 seconds] 13:16 < xinxi> My point is current smart contract platforms are mostly solving hypothetical problems. 13:16 < petertodd> sipa: yeah, pragmatically speaking, replacing the entire codebase in a hardfork is going to be incredibly controversial at best 13:16 < sipa> petertodd: indeed 13:17 < sipa> petertodd, xinxi: again, please take that discussion elsewhere. yes, there is some overlap that is relevant here. but discussion about what problems smart contracts are solving is not 13:17 -!- mode/#bitcoin-core-dev [+o sipa] by ChanServ 13:17 < petertodd> sipa: ack 13:17 < xinxi> sipa: yeah. My focus is still formal verification. 13:18 -!- mode/#bitcoin-core-dev [-o sipa] by sipa 13:18 < sipa> thanks 13:18 < sipa> xinxi: have you read my BIP66 writeup above? 13:18 < xinxi> So you are Pieter? 13:18 < sipa> yes 13:19 < xinxi> cool 13:19 < sipa> i think it is useful to illustrate the difficulty of consensus problems 13:19 < sipa> of finding 13:19 < kanzure> btcdrak: bitcoincore.org/en/team/ is lacking usernames 13:20 < btcdrak> kanzure: really could do with a bio page. people's user handles are not always consistent across platforms 13:20 -!- go1111111 [~go1111111@104.200.154.24] has joined #bitcoin-core-dev 13:21 < xinxi> sipa: please let met read it first. 13:21 < petertodd> xinxi: we'll, lets continue the discussion in another hour or two after you read it :) 13:21 < petertodd> bbl! 13:21 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has joined #bitcoin-core-dev 13:23 < sipa> xinxi: (i'll only mention this once): bitcoin core's cryptography library (libsecp256k1) does have some modest attempts at formally verifying pieces of the code, and help there would also be very welcome. It's much less ambitious than proving properties about bitcoin's consensus code, but also much easier to accomplish something in my opinion 13:23 < kanzure> sipa: don't you guys have that libsecp256k1 paper at some point? :D 13:23 < sipa> kanzure: at some point. 13:24 < btcdrak> xinxi: see https://github.com/bitcoin-core/secp256k1 13:25 < xinxi> btcdrak: thanks. 13:25 < xinxi> sipa: that's understandable. It takes huge amount of money to get started. 13:26 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 250 seconds] 13:27 < sipa> also, #secp256k1 for that 13:29 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 13:29 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 13:31 < Chris_Stewart_5> sipa: I just read the email, however the hard fork last July wasn't related to the implementation of BIP66 , it was just spv mining correct? 13:32 < sipa> that was not a hard fork 13:32 < sipa> just miners creating invalid blocks 13:32 < sipa> and it triggered when bip65 activated 13:32 < sipa> not bip66, which was a year earlier 13:35 < Chris_Stewart_5> Seems that OP_CLTV was activated ~7 months ago? 13:35 < Chris_Stewart_5> I distinctly remember the issue being July 4th of last year 13:35 < btcdrak> yes, he means BIP66 13:35 < Chris_Stewart_5> oh ok 13:36 < sipa> oh, i'm misremembering 13:36 < btcdrak> CLTV went without a hitch. 13:36 < sipa> apologies 13:36 < Chris_Stewart_5> np 13:36 < btcdrak> I dont know, I think we must give a punishment. 13:38 < Chris_Stewart_5> Seven more years of reading openssl! 13:38 < btcdrak> too harsh. maybe no computer! and go to bed early! 13:46 < sipa> hah 13:46 < xinxi> sipa: so you wrote most of secp256k1? 13:46 -!- BCBot [~BCBot@46.101.246.115] has quit [Remote host closed the connection] 13:47 -!- BCBot [~BCBot@46.101.246.115] has joined #bitcoin-core-dev 13:48 < xinxi> The email seems fantastic. 13:48 < xinxi> It's truly a tough journey. 13:49 < sipa> xinxi: i started it; most of the fancier optimizations and verifications/tests were proposed or implemented by others 13:50 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 250 seconds] 13:53 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 252 seconds] 13:56 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 13:57 < xinxi> sipa: that's interesting. I saw it's written in C intead of C++. I thought you deliberately did that for verification? 13:58 < sipa> xinxi: it started as C++, actually, but gmaxwell suggested converting it to C to aide with formal verification 13:58 < sipa> as there are more analysis tools available for C than C++ 13:58 < sipa> this was very early on 13:58 < xinxi> sipa: fantastic! I am so glad to see it's formally verified! 13:59 < sipa> i wouldn't say it's formally verified 13:59 < sipa> just some pieces 13:59 < xinxi> C++ is just too complicated to verify. 13:59 < sipa> i can discuss more on #secp256k1 13:59 < xinxi> Is it a channel? 13:59 < sipa> yes 13:59 < xinxi> sipa: let's talk there. 14:04 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 276 seconds] 14:06 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 14:08 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 14:14 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 14:14 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit] 14:23 -!- justanotheruser [~Justan@unaffiliated/justanotheruser] has quit [Read error: Connection reset by peer] 14:23 -!- justanotheruser [~Justan@24.219.72.136] has joined #bitcoin-core-dev 14:24 -!- justanotheruser [~Justan@24.219.72.136] has quit [Client Quit] 14:24 -!- justanotheruser [~Justan@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 14:32 -!- frankenmint [~frankenmi@174-25-9-165.ptld.qwest.net] has joined #bitcoin-core-dev 14:34 -!- xinxi [~xinxi@116.86.156.222] has quit [Remote host closed the connection] 14:38 -!- arubi [~ese168@unaffiliated/arubi] has quit [Ping timeout: 240 seconds] 14:42 -!- arubi [~ese168@unaffiliated/arubi] has joined #bitcoin-core-dev 14:43 -!- xinxi [~xinxi@116.86.156.222] has joined #bitcoin-core-dev 14:43 -!- jannes [~jannes@178.132.211.90] has quit [Quit: Leaving] 14:44 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 240 seconds] 14:47 -!- xinxi [~xinxi@116.86.156.222] has quit [Ping timeout: 250 seconds] 14:51 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has quit [Excess Flood] 14:52 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has joined #bitcoin-core-dev 15:13 -!- frankenmint [~frankenmi@174-25-9-165.ptld.qwest.net] has quit [Remote host closed the connection] 15:14 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)] 15:15 -!- proslogion [~proslogio@2.127.106.111] has joined #bitcoin-core-dev 15:15 -!- frankenmint [~frankenmi@174-25-9-165.ptld.qwest.net] has joined #bitcoin-core-dev 15:15 -!- proslogion [~proslogio@2.127.106.111] has left #bitcoin-core-dev ["离开"] 15:17 -!- TomMc [~tom@unaffiliated/tommc] has quit [Ping timeout: 252 seconds] 15:28 -!- belcher [~user@unaffiliated/belcher] has joined #bitcoin-core-dev 15:30 -!- TomMc [~tom@unaffiliated/tommc] has joined #bitcoin-core-dev 15:35 < GitHub44> [bitcoin] dooglus opened pull request #8331: Fix three 'comparison between signed and unsigned integer expressions' warnings. (master...fix-compilation-warnings) https://github.com/bitcoin/bitcoin/pull/8331 15:50 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 16:06 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 276 seconds] 16:10 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 16:16 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 252 seconds] 16:25 -!- spudowiar [~spudowiar@unaffiliated/spudowiar] has quit [Quit: Leaving.] 16:40 -!- mkarrer [~mkarrer@48.red-83-47-111.dynamicip.rima-tde.net] has quit [Ping timeout: 252 seconds] 16:42 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 16:49 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 244 seconds] 17:01 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 17:04 -!- TomMc [~tom@unaffiliated/tommc] has quit [Ping timeout: 240 seconds] 17:05 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 264 seconds] 17:06 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 17:28 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Ping timeout: 276 seconds] 17:35 -!- TomMc [~tom@unaffiliated/tommc] has joined #bitcoin-core-dev 17:53 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 244 seconds] 17:56 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 18:02 -!- belcher [~user@unaffiliated/belcher] has quit [Quit: Leaving] 18:08 -!- Giszmo1 [~leo@80.31.9.48] has joined #bitcoin-core-dev 18:09 -!- Giszmo [~leo@112.red-79-153-18.dynamicip.rima-tde.net] has quit [Ping timeout: 276 seconds] 18:10 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 276 seconds] 18:15 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-wfhnncmhoswhwwwt] has quit [Quit: Connection closed for inactivity] 18:30 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 18:31 -!- Lysander1 [~Lysanders@unaffiliated/lysanders] has joined #bitcoin-core-dev 18:34 -!- Lysanders [~Lysanders@unaffiliated/lysanders] has quit [Ping timeout: 252 seconds] 18:39 -!- xinxi [~xinxi@116.86.156.222] has joined #bitcoin-core-dev 18:40 -!- Giszmo1 [~leo@80.31.9.48] has quit [Ping timeout: 240 seconds] 18:40 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Read error: Connection reset by peer] 18:41 -!- Giszmo [~leo@80.31.9.48] has joined #bitcoin-core-dev 18:43 -!- xinxi [~xinxi@116.86.156.222] has quit [Ping timeout: 240 seconds] 18:48 -!- TomMc [~tom@unaffiliated/tommc] has quit [Ping timeout: 244 seconds] 18:51 -!- Giszmo [~leo@80.31.9.48] has quit [Ping timeout: 264 seconds] 18:51 < phantomcircuit> fyi there's someone repeatedly connecting and disconnecting from a narrow ip range 18:51 < phantomcircuit> 2016-07-12 01:47:04.455787 receive version message: /bitcoinj:0.14.1/: version 70001, blocks=0, us=127.0.0.1:8333, peer=40824, peeraddr=37.97.164.230:60893 18:52 < phantomcircuit> 37.97.164.159 18:52 < phantomcircuit> 37.97.164.160 18:52 < phantomcircuit> 37.97.164.230 18:52 < phantomcircuit> 37.97.164.231 18:55 -!- afk11 [~afk11@unaffiliated/afk11] has quit [Ping timeout: 246 seconds] 18:56 -!- afk11 [~afk11@109.255.154.81] has joined #bitcoin-core-dev 18:56 -!- afk11 [~afk11@109.255.154.81] has quit [Changing host] 18:56 -!- afk11 [~afk11@unaffiliated/afk11] has joined #bitcoin-core-dev 18:58 -!- harrymm [~wayne@37.58.59.76] has quit [Ping timeout: 276 seconds] 19:12 -!- harrymm [~wayne@223.204.247.243] has joined #bitcoin-core-dev 19:15 -!- frankenmint [~frankenmi@174-25-9-165.ptld.qwest.net] has quit [Remote host closed the connection] 19:19 < Chris_Stewart_5> Can you receive messages on the p2p network from a non standard port? ie not 8333/18333 19:19 < Chris_Stewart_5> if I send a version message with the non standard port? 19:29 < sipa> yes 19:30 < sipa> but nodes strongly prefer connecting to nodes on the standard port 19:30 < sipa> they will pretty much only choose nodes with nonstandard ports if they have no other option 19:33 < dgenr8> good news. we don't have to worry too much that time-locked transactions could be delayed by 30 days by evil miners https://github.com/bitcoinxt/bitcoinxt/pull/142#issuecomment-231918519 19:34 -!- frankenmint [~frankenmi@174-25-9-165.ptld.qwest.net] has joined #bitcoin-core-dev 19:39 -!- fengling [~fengling@58.135.95.137] has quit [Quit: WeeChat 1.4] 19:40 -!- fengling [~fengling@58.135.95.137] has joined #bitcoin-core-dev 19:44 -!- whphhg [whphhg@gateway/vpn/mullvad/x-zisyjxwpfaztlach] has quit [Ping timeout: 264 seconds] 19:53 -!- justanotheruser [~Justan@unaffiliated/justanotheruser] has quit [Ping timeout: 264 seconds] 19:55 -!- justanotheruser [~Justan@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 20:01 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 260 seconds] 20:26 -!- xinxi [~xinxi@116.86.156.222] has joined #bitcoin-core-dev 20:29 < GitHub136> [bitcoin] dcousens opened pull request #8332: semi trivial: clarify witness branches in transaction.h serialization (master...patch-1) https://github.com/bitcoin/bitcoin/pull/8332 20:31 -!- xinxi [~xinxi@116.86.156.222] has quit [Ping timeout: 252 seconds] 21:02 -!- Alopex [~bitcoin@cyber.dealing.ninja] has quit [Remote host closed the connection] 21:03 -!- Alopex [~bitcoin@cyber.dealing.ninja] has joined #bitcoin-core-dev 21:09 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has joined #bitcoin-core-dev 21:11 < gmaxwell> dgenr8: uh. you realize that when you have .5 or more non-honest miners sustained, the existance of an honest blocks is purely their choice, right? So that graph doesn't make a lot of sense. 21:12 < gmaxwell> yet again you have another negligently wrong security analysis, where you report 50% as absolutely safe, when in fact it's a guarenteed failure (assuming dedicated attackers) 21:16 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #bitcoin-core-dev 21:16 < dgenr8> indeed, a 51% attack is a far greater threat than a blocktime slowdown, you get it 21:18 < gmaxwell> what? your analysis is just obviously broken on its face. 21:19 < gmaxwell> it claims that 80% dishonest mining is unable to slew the time. 21:20 < gmaxwell> This is obviously untrue, so you've make some obvious mistake-- if you have any result that isn't 100% at 50% hashpower. 21:23 < gmaxwell> And, what the heck is "a 51% attack" -- Bitcoin exists as a confluence of incentives, one part of it is that even a user who buys up large amounts of hashpower for a brief period only gains the ability to reorganize recent transactions, which is difficult to profitably exploit at any scale-- which users can protect themselves from by waiting for additional confirmations. Under the security mod 21:23 < gmaxwell> el change that you propose (and incorportated into your software without an adequate disclosure because you (based on the prior conversation) weren't actually aware of the implications, miners that achieve a large amount (unclear how much but a majority is clearly sufficient) can add huge amounts of coins to themselves without any chain reorg and without even making a purchase. 21:25 < gmaxwell> (re obviously didn't understand, I'm referring to your prior claim that 100% hashpower and control over the user's clock was needed.) 21:26 < dgenr8> you read that statement completely wrong 21:28 < gmaxwell> K. Did I misunderstand that this was shipped out in XT without a release note pointing out the security model change; to a user community that obsesses over even minor softforks as taking away validation? 21:34 < gmaxwell> am I misreading your graph showing that a miner with 79% hashpower has a non-unity probablity of success at moving the timestamp back? 21:34 < dgenr8> XT, unlike core, would warn loudly as that 51% attack slowed the generation rate 21:35 < gmaxwell> Did I miss the fact that this whole design is just an insecure rip off of the proposal from this channel to use 30 days of _work_ at the current tip, as a gate on signature checks? 21:36 < gmaxwell> dgenr8: there is no need for the generation rate to slow significantly enough to trigger any warning that doesn't false positive constantly. 21:38 -!- fengling [~fengling@58.135.95.137] has quit [Ping timeout: 240 seconds] 21:41 < dgenr8> the reason XT exists is that a few people cannot abide the choices you are making. it's too bad. for a while, folks worked together pretty well. 21:41 < dgenr8> you view the loss of talented people like gavin, jgarzik, and hearn as positve, and seek to utterly discredit others. but i'm OT 21:42 < gmaxwell> dgenr8: You are going to get some people robbed blind with this kind of sloppyness; it's a risk to our entire industry. 21:45 < gmaxwell> I don't have to do anything to discredit you (not to mention other people)-- you do so _yourself_; by shipping software to users that you don't even understand and being continually antagonistic to people who would otherwise help you avoid danger. 21:47 < dgenr8> Perhaps try again to manipulate the timestamp on testnet. Any hashing that sets timestamps correctly has a huge advantage though. 21:47 < gmaxwell> dgenr8: you didn't even notice the successful attack on testnet against classic then, I guess. 21:48 < dgenr8> i did see it was pushed it back a few days 21:49 < gmaxwell> A few days ago you seemed to be arguing that using the _miner provided_ timestamps on blocks to decide to bother validating the block or not would require 100% hashpower and control of the users local clock to exploit-- functionality you added without even adding a _release note_ to Bitcoin XT. Your response to this being pointed out was to again repeat accusations that I backdoored your softwa 21:49 < gmaxwell> re. After having your mistaken understanding thrust on you; you've gone away and come back with another obviously broken misunderstanding; again radically overstating the security of this 'optimization'. 21:50 < gmaxwell> The only reason I even knew that XT and classic had incorporated this ill advised security change is because of people bragging about how much faster it synced. Yea, it's faster when you don't check 95% of the signautures, blindly... 21:51 < gmaxwell> So if you think you're going to make a war for node operation a war about who can make the most insecure software-- the stuff with the most tail risk of total failure, then hell yes I'm going to call that out. And you should be thankful that I've taken the concerns directly to you, and not-- like other people in your community-- gone straight to the press with them. 21:51 < gmaxwell> (Especially when it's so easy to demonstrate the exploit on a live network) 21:53 < dgenr8> hmm. i think this whole display is for the benefit of third parties. anyhow i disagree on the degree of risk, but that's not to say the regular checkpoints couldn't come back or that the default selection could even change. but that belongs to a rational discussion we're not having 21:53 < gmaxwell> Fair enough. 21:55 < dgenr8> my concern with your thin blocks change was not that it made it to XT briefly. it was that you removed the thin blocks functionality from core users. 21:56 < gmaxwell> I'm sorry for being such a dick on this. I do really think that the blockheader based skipping validation is a race to the bottom-- who can provide the worst security (for the fastest performance), to hell with the consequences (and no one adequately disclosing it). It doesn't help that a much better way of doing the same thing was already proposed in Core; and to me it seems like pure NIH to i 21:56 < gmaxwell> mplement a much riskier version. 21:58 < gmaxwell> dgenr8: I had no idea that it had any impact there, hell, the change was actually motivated by a complaint you made about the behavior (which, since you didn't disclose, I didn't know it was due to the prior limit of 1000 breaking mike's thinblocks). I didn't even initially use the bloom filter until wumpus and pieter beat me down on the memory usage. And ultimately since there was _no_ reliabl 21:58 < gmaxwell> e mechenism to request transactions that were already in a block (an observation we made on that approach in 2013), mike's way of doing it couldn't have been reliable. 21:58 -!- xinxi [~xinxi@116.86.156.222] has joined #bitcoin-core-dev 21:59 < gmaxwell> I also explained this previously, but you seem even more insistant on assuming bad faith on my part than I am on seeing you as a fool. :) 22:00 < gmaxwell> Seriously, if I wanted to 'backdoor' your software, the avenues available are far more profound-- your response has made me profoundly uncomfortable with the continued existance of Bitcoin XT-- you'll continue to copy code from Core, and when the inevitable errors that happen from interactions which I could never have anticiapted; you've shown that you'll accuse me of unethical or even criminal 22:00 < gmaxwell> conduct. 22:02 < gmaxwell> s/blockheader based/blockheader timestamp based/ to be more clear. the earlier proposal of using total work instead of timestamps, is much more appealing. 22:03 < dgenr8> whoa slow down ... the map size problem I found had nothing to do with thin blocks, which is why i was eager to copy your fix for it 22:04 < gmaxwell> dgenr8: I thought (after trying to figure out why you even copied the change) the reason you commented on the map size is because with thinblocks the small map meant the far end had no idea that you had most of the transactions you already had, then sent redundant stuff. 22:04 < dgenr8> no, i found it while looking at some other core PR 22:05 < gmaxwell> then why did you even assume I was trying to screw up your thinblocks stuff? I don't believe I had _any_ idea XT was even screwing with that at that time. 22:06 < gmaxwell> (I wouldn't have assumed it because the patch mike had was so obviously junk-- handling it in the ping handler, no way to cope with missing transactions except praying that they were still in the relay cache) 22:06 < dgenr8> if you recall that effect was found independently weeks later by Peter Tschipper 22:06 < dgenr8> its all irrelevant now anyway 22:07 -!- xinxi [~xinxi@116.86.156.222] has quit [Ping timeout: 276 seconds] 22:07 < gmaxwell> in any case, indeed it broke the thin block stuff completely; then you merged it, and released a new version with it and the initial release of that thinblock stuff--- meaning you hadn't even tested it for basic functionality; which is basically terrifying from my perspective. 22:07 < gmaxwell> You can't deny that the commit message was scream loud totally clear about _exactly_ what the change did. 22:08 < gmaxwell> and so how the hell can I defend myself against bad testing on your part? I made a very very clearly described change, which you didn't comment on, integrated so it broke your stuff totally, then claimed misconduct on my part. Thats seriously bad mojo. 22:08 < dgenr8> it went to master, not a release. please don't put 'backdoor' in quotes as I never said that word and as I said, that was not my concern 22:08 -!- xinxi [~xinxi@116.86.156.222] has joined #bitcoin-core-dev 22:11 < gmaxwell> that was certantly what people extracted from your comments, https://www.reddit.com/r/bitcoinxt/comments/43jgtt/blockstream_engineers_maxwell_and_wuille_caught/ 22:12 < gmaxwell> Though indeed, I guess you only use the words "your sneak attack" and not "backdoor" yourself. 22:16 -!- kadoban [~mud@unaffiliated/kadoban] has quit [Quit: bye] 22:16 -!- kadoban [~mud@unaffiliated/kadoban] has joined #bitcoin-core-dev 22:19 < gmaxwell> and "Here we have Blockstream actively fighting scaling improvements" (and not to mention misrepresentation the that change was to avoid a one in a million false positive, when the chance happens for every txn indpendantly, so after seeing just 10k txn the chance is 1%.) 22:27 -!- xinxi_ [~xinxi@116.86.156.222] has joined #bitcoin-core-dev 22:27 -!- xinxi [~xinxi@116.86.156.222] has quit [Read error: Connection reset by peer] 22:29 < gmaxwell> XT v0.11E appears to contain the setinventory known and 'thinblock' change; but now looking at it I see you left it probablistically screwing over spv clients. 22:30 < eragmus> gmaxwell: Why are you wasting your precious time with a dgenr8 like him? :) Or, at least, maybe charge him and the others by the minute for this consulting. 22:30 < gmaxwell> I wonder how many bitcoins will be lost by people throwing out wallets that look empty but arn't. :( 22:31 < eragmus> dgenr8: Btw, you should be careful before making everyone laugh so much -- "the loss of talented people like gavin, jgarzik, and hearn" -- I nearly choked. I might have to bill you for any potential hospital expenses. 22:31 -!- fengling [~fengling@58.135.95.137] has joined #bitcoin-core-dev 22:32 < gmaxwell> eragmus: (1) because software that silently downgrades security then promotes itself as faster is a risk to the continued viability and value of Bitcoin, which I care about. .. and because he attacks me by claiming that I've acted unethically, which if left unchallenged ends up recycled as accepted truth in places like the NYT where it can cause me lifelong harm. 22:32 -!- cryptapus_ [~cryptapus@199.47.67.56] has joined #bitcoin-core-dev 22:32 -!- cryptapus_ [~cryptapus@199.47.67.56] has quit [Changing host] 22:32 -!- cryptapus_ [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev 22:33 <@wumpus> PSA: this channel is about bitcoin core development, XT and classic and certainly non-technical drama around them are very off-topic 22:37 < phantomcircuit> indeed wumpus is right, possibly this could move to #bitcoin 22:38 < phantomcircuit> wumpus, fyi you might find this interesting https://github.com/bitcoin/bitcoin/issues/8333 22:38 < dgenr8> gmaxwell: breadwallet already detects the condition and corrects it by reconnecting 22:41 <@wumpus> phantomcircuit: in what world is 31.87ms slow, I mean, I've seen logs in where validation took that *per txin* :) 22:43 < phantomcircuit> wumpus, in a world where i can mess with settings and have an average block validate in 30ms 22:43 < phantomcircuit> thanks to my 8GiB sigcache and 16GiB dbcache 22:43 < phantomcircuit> (and a hack that loads the entire utxo into cache on start) 22:44 < gmaxwell> wumpus: he's optimizing mining overheads, some of them are harder to address than bitcoind behavior. :) (e.g. cannot decrease the radius of the earth :P ) 22:44 <@wumpus> phantomcircuit: anyhow to say more about it we 22:45 <@wumpus> 'd need detailed profiling of that step 22:46 < phantomcircuit> gmaxwell, im working on that, i have a plan to drill down and around the mantle 22:46 < phantomcircuit> i only need a few trillion dollars 22:47 -!- xinxi_ [~xinxi@116.86.156.222] has quit [Remote host closed the connection] 22:47 < gmaxwell> /msg phantomcircuit has the patent issued yet for the neutrion based transciever yet? 22:49 <@wumpus> gmaxwell: yes can only get as close as possible to the physical limit 22:50 -!- xinxi [~xinxi@116.86.156.222] has joined #bitcoin-core-dev 22:52 < gmaxwell> wumpus: there are a number of things that aren't the physical limits but are still harder to speedup than core, also at least as long as validation is in the relay critical path, any part of validation adds up as the node count increases. Though thats probably better fixed by getting validation out of the critcial path for relay to other full nodes. 22:55 -!- xinxi [~xinxi@116.86.156.222] has quit [Ping timeout: 260 seconds] 23:02 <@wumpus> gmaxwell: what kind of things, optimizing the P2P protocol? network infrastructure? 23:04 <@wumpus> as long as any part of validation is the bottleneck, optimizing that is still part of 'speeding up core' 23:07 <@wumpus> if validation is out of the critical part all that is left is the network, where the random P2P network isn't the most efficient structure for propagating something around the world quickly 23:09 <@wumpus> (but that's what the relay network is for) 23:21 -!- Yv7trNY [~IUTYVExrY@86.121.131.196] has joined #bitcoin-core-dev 23:24 -!- Yv7trNY [~IUTYVExrY@86.121.131.196] has quit [Client Quit] 23:26 -!- xinxi [~xinxi@116.86.156.222] has joined #bitcoin-core-dev 23:27 -!- davidlj95 [~davidlj95@deic-dyn-232.uab.es] has joined #bitcoin-core-dev 23:37 < sipa> phantomcircuit: it's good to see that part of the logic becoming measurable :) 23:38 < sipa> phantomcircuit: it would be nice if you could see if the two PRs i referenced on your issue affect that number 23:47 -!- xinxi [~xinxi@116.86.156.222] has quit [Remote host closed the connection] 23:48 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-rsvvmawbpoovtsay] has joined #bitcoin-core-dev 23:53 -!- cryptapus_ [~cryptapus@unaffiliated/cryptapus] has quit [Ping timeout: 264 seconds] 23:57 -!- xinxi [~xinxi@116.86.156.222] has joined #bitcoin-core-dev