--- Day changed Fri Jun 30 2017 00:00 -!- dermoth [~dermoth@gateway/tor-sasl/dermoth] has quit [Remote host closed the connection] 00:00 -!- dermoth [~dermoth@gateway/tor-sasl/dermoth] has joined #bitcoin-core-dev 00:05 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Quit: Leaving.] 00:11 < wumpus> whoa, validation really became a lot quicker with recent changes 00:20 -!- Yogaqueef [~textual@dsl-hkibng42-5673c3-32.dhcp.inet.fi] has joined #bitcoin-core-dev 00:21 < sipa> wumpus: IBD or at tip 00:21 < sipa> ? 00:22 < wumpus> at tip 00:22 < wumpus> (a node catching up a few days) 00:24 -!- arowser [~quassel@45.32.248.113] has quit [Remote host closed the connection] 00:24 < sipa> ah, i mean while in full sync processing newly mined blocks 00:25 < wumpus> it used to bring this particular system to a halt while slowly syncing blocks, now the blocks breeze by 00:25 < sipa> if you have slow I/O, pertxout likely helps a lot 00:25 < sipa> because it causes far fewer flushes 00:26 < wumpus> definitely slow i/o 00:26 < luke-jr> I need to kill my btrfs.. 00:27 -!- timothy [~tredaelli@redhat/timothy] has joined #bitcoin-core-dev 00:28 < wumpus> but that's great, a lot of users, esp windows users, tend to have slow i/o laptops 00:28 < wumpus> so that's a decent thing to optimize for 00:29 < wumpus> luke-jr: why? 00:29 < luke-jr> wumpus: slow I/O. very slow. :/ 00:30 < luke-jr> even Chromium gets slow on btrfs home :/ 00:30 -!- arowser [~quassel@45.32.248.113] has joined #bitcoin-core-dev 00:30 < wumpus> which reminds me I should probably do some window testing again pre-0.15 *sigh* 00:31 < wumpus> luke-jr: so btrfs is better in everything except performance? 00:31 < luke-jr> wumpus: and reliability, I guess. had some kernel panics in btrfs code a few times several months ago 00:31 < luke-jr> and someone in #btrfs told me to add a mount option to avoid a very rare data corruption bug 00:32 < luke-jr> oh, and ioctls don't work in 32-bit mode, but I use 64-bit now 00:32 < wumpus> I've never tried btrfs yet, just sticking with extN on linux, I guess I'm extremely conservative with regard to file systems 00:33 < sipa> i tend to use ext4 as root fs, and zfs for all the rest 00:33 < sipa> (where "all the rest" is not much) 00:33 < luke-jr> hm, it probably doesn't help I/O that I have btrfs compression enabled 00:34 < luke-jr> although you'd think modern CPUs would be fast enough it didn't matter 00:34 < wumpus> sipa: doesn't zfs on linux require custom patches? 00:35 < sipa> not on ubuntu at least 00:35 < luke-jr> O.o 00:36 < wumpus> luke-jr: for encryption that's mostly true, for compression (especially when writing) I'd expect a reasonable perf loss with compression, for reading you might get a perf gain in some cases (but only if you store files that are well-compressible on a slow medium) 00:37 < luke-jr> tempting to hack btrfs to notice when files get modified often and not compress those.. but I don't really want to hack on my filesystem, so not happening ☺ 00:37 < wumpus> also, even if throughput higher, latency might be worse because of the extra decompression step 00:38 < luke-jr> oh, ZFS is the filesystem that's GPL-infringing XD 00:39 < luke-jr> prob just go back to ext4 00:41 < wumpus> disk compression seems a waste of time in the 201x's, most large files (video, images, music) are already compressed so I'd dare say it helps very little in practice 00:42 < wumpus> even if you store raw video/music then generic gzip compression is not very suited 00:43 < luke-jr> maybe I should try just turning it off before giving up on it 00:44 < wumpus> thinking, something it might help with is compiler object files, those tend to be well-compressible 00:44 < wumpus> and large (in case of c++) 00:45 * luke-jr changes it to use LZO compression rather than gzip 00:46 < luke-jr> (supposedly it's faster) 00:46 < sipa> wumpus: i use lzo compression for the blockchain 00:47 < sipa> lzo is much faster 00:52 < gmaxwell> wumpus: until recently the gap between disk IO speed and the rest of the system was making it so that compression (at least fast compression like LZO) actually increased performance... 00:54 < luke-jr> recently = SSD? 00:55 < luke-jr> I also made the mistake of moving my home to 5400 RPM.. >_< 01:21 < gmaxwell> not just SSD but newer pcie ssds. 01:24 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 01:33 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 01:35 -!- jannes [~jannes@178.132.211.90] has joined #bitcoin-core-dev 01:36 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 268 seconds] 01:37 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt] 01:42 -!- vicenteH [~user@195.235.96.150] has joined #bitcoin-core-dev 01:44 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 01:45 -!- Aaronvan_ [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 01:47 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has quit [Quit: WeeChat 1.7.1] 01:47 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 01:49 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 240 seconds] 01:53 -!- ivan [~ivan@unaffiliated/ivan/x-000001] has quit [Quit: lp0 on fire] 02:06 < wumpus> gmaxwell: also for writing? 02:07 < wumpus> I guess it depends on the kind of data 02:09 < wumpus> for reading performance it can be quite obviously true 02:12 < wumpus> sipa: custom patch, or at the file system level? 03:20 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev 03:23 -!- marcoagner [~user@177.99.123.164] has quit [Ping timeout: 260 seconds] 03:35 -!- marcoagner [~user@177.99.124.100] has joined #bitcoin-core-dev 03:41 -!- Char0n [~Char0n@unaffiliated/char0n] has quit [Quit: ZNC 1.6.3+deb1+xenial0 - http://znc.in] 03:41 -!- Char0n [~Char0n@unaffiliated/char0n] has joined #bitcoin-core-dev 03:48 -!- emzy [~quassel@unaffiliated/emzy] has quit [Remote host closed the connection] 03:51 -!- emzy [~quassel@raspberry.emzy.de] has joined #bitcoin-core-dev 03:55 -!- emzy [~quassel@raspberry.emzy.de] has quit [Changing host] 03:55 -!- emzy [~quassel@unaffiliated/emzy] has joined #bitcoin-core-dev 03:57 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has joined #bitcoin-core-dev 04:18 < bitcoin-git> [bitcoin] Mirobit opened pull request #10710: REST/RPC example update (master...docupt) https://github.com/bitcoin/bitcoin/pull/10710 04:27 -!- Soligor [~Soligor@unaffiliated/soligor] has quit [Ping timeout: 240 seconds] 04:30 -!- Soligor [~Soligor@unaffiliated/soligor] has joined #bitcoin-core-dev 04:52 -!- EarlyGrey [~lepton@209.107.210.197] has joined #bitcoin-core-dev 04:55 -!- Guyver2 [~Guyver@guyver2.xs4all.nl] has joined #bitcoin-core-dev 05:06 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has quit [Read error: Connection reset by peer] 05:26 -!- Aaronvan_ is now known as AaronvanW 05:45 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev 05:59 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Ping timeout: 240 seconds] 06:13 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 06:19 -!- To7 [~theo@2604:2000:1382:b7:5935:8771:8a1c:12bd] has quit [Ping timeout: 246 seconds] 06:33 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 06:33 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit] 06:46 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 260 seconds] 06:46 < morcos> instagibbs: I'm trying to review #10333 and I was trying to understand the ReturnKey() behavior. How come it is not a bug (also in master) that we call ReturnKey() even in the case where CoinControl gave us a destination address (so we never reserved a new key) 06:46 < gribble> https://github.com/bitcoin/bitcoin/issues/10333 | [wallet] fee fixes: always create change, adjust value, and p… by instagibbs · Pull Request #10333 · bitcoin/bitcoin · GitHub 06:48 < instagibbs> calling return is a nop if you haven't marked a key as reserved IIRC 06:48 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev 06:50 < instagibbs> oh i see what you mean, lemme continue reading... 06:51 < morcos> instagibbs: i just figured it out 06:51 < morcos> i didn't know how reservekey's work 06:51 < morcos> but yeah the reservekey object has nIndex -1 until you ask it to actually get a reserved key 06:51 < instagibbs> yep 06:51 < morcos> so calling return on it is a no-op 06:51 < instagibbs> so yeah i think the logic is right 06:52 < morcos> while i have you, aren't you missing a continue at the bottom of your loop in the subtractFeeFromAmount > 0 case 06:52 < instagibbs> it's safe to call returnkey unless you actively are saving a usage 06:52 < instagibbs> checking 06:52 < morcos> instagibbs: yes agreed on the returnkey 06:52 < morcos> after line 2378 06:54 < instagibbs> wallet.cpp? That's in SelectCoins on my end 06:55 < bitcoin-git> [bitcoin] jnewbery opened pull request #10711: [tests] Introduce TestNode (master...testnode2) https://github.com/bitcoin/bitcoin/pull/10711 06:55 < instagibbs> it's just going to loop, right? 06:56 < instagibbs> if subtractFeeFromAmount != 0 and you don't have enough fee, it just hits the end and loops 06:58 < morcos> but it'll sign first? 06:59 < morcos> also you seem to have a regression in the subtractFeeFromAmount > 0 case where you have isDust change. Previously that change was getting deleted and added to fees, now I think you're not doing that 06:59 < instagibbs> ah yeah i wrote this previous to that behavior, will fix 06:59 < instagibbs> not sure what you mean about signing first, I'm probably looking at the wrong line 07:00 < morcos> oh maybe i am, i was looking without whitespace, hold on 07:03 < morcos> yeah my fault on that, seems fine 07:06 < morcos> We ought to be able to avoid looping completely when subtractFeeFromAmount > 0 though... 07:07 < morcos> If we changed the logic when subtractFeeFromAmount > 0 to first check for dust change, and if that exists move it to fee, then reduce the recipient amounts as necessary to get to the fee. (unless the recipient amounts aren't big enough, which is a failure anyway) 07:08 < instagibbs> I mean it can still be considered at the end in the "Fee adjustment" stage right? 07:08 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 07:09 < morcos> I think it would be better to eliminate dust change to fee first, b/c then you can just subtract less from the recipients (only in the case where subtractFeeFroAmount > 0) 07:09 < instagibbs> mm right 07:11 < morcos> If you have a minute, indulge me for a second while we walk through exactly what scenario 10333 is trying to solve 07:12 < instagibbs> so it's the same situation as the previous fix you made, where it grabs a lot of inputs, fails, then next time overpays in fees. Your fix adds fees to existing change output 07:12 < instagibbs> This is to make it also cover the "no change output" case 07:12 < instagibbs> meaning we calculate what the change would look like every time, adjust as necessary 07:13 < morcos> In the no change output case though, doesn't that by definition mean we've tried via the knapsack algorithm to get inputs that add up as closely as possible to the desired target 07:13 < instagibbs> for that crazy fee level, yes 07:13 < instagibbs> nFeeRet in pathological case can be many times higher than required 07:14 < instagibbs> change is only calculated via valuein-valuetoselect, nFeeRet is part of the latter term 07:16 < morcos> Yeah maybe it's not made worse by your PR, but it seems like the algorithm that tries to find an exact match is always going to try to find an exact match assuming a change ouput, and then therefore will always overpay fees by the feerate*1 extra output in the case where we do find a match which doesn't require change 07:16 -!- haakonn [~haakonn@pdpc/supporter/active/haakonn] has quit [Ping timeout: 260 seconds] 07:16 -!- so [~so@unaffiliated/so] has quit [Ping timeout: 240 seconds] 07:17 < morcos> but if that amount is > dust, which i think it always is, then we'll always revert to the second step of trying to find MIN_CHANGE? 07:17 < morcos> i don't know, i guess it just seems like in some ways its changing the coin selection algorithm 07:18 < morcos> maybe not for the worse, but its a weird way to change it 07:18 < instagibbs> the real fix is to do effective value selection 07:18 < morcos> Yes, that combined with not trying to find an EXACT match, but trying to find a match within some tolerance that you are willing to lose to extra fees 07:18 < morcos> EXACT match finding is stupid 07:19 < Murch> morcos: BranchAndBound assumes no change output. 07:19 < morcos> that tolerance already kidn of happens via the remove dust output to fees , but we could make it explicit and do better 07:20 < instagibbs> Yeah imo the weakness of 10333 is that it's only needed to avoid stupidity, and will likely poorly interact with real fixes 07:20 < instagibbs> right now we're just so bad at estimating, we almost never exact match 07:20 < morcos> Murch: yeah i need to review that, but I think it shoudl be aware of the amount its wiling to discard in excess fees, and return success if it finds a result under that tolerance, and if not, assume a change output and aim for TARGET_CHANGE (probably just MIN_CHANGE) 07:20 < Murch> instagibbs: If it is only a minuscule amount missing for the fees, wouldn't it perhaps be acceptable to take that from the change output? 07:20 < instagibbs> Murch, it already does 07:21 < instagibbs> it's the case of it not having a fee output to make larger in which is currently just SFYL 07:21 < morcos> instagibbs: what would you think about another approach, that perhaps changed the logic less 07:21 < Murch> morcos: It allows for excess up to the cost of creating a change output. In my original proposal also the cost of creating an input (spending the output later), but with the high fee rate volatility, maybe only the change is a better measure. 07:21 < morcos> i'm wary of unintended consequences... 07:21 < instagibbs> morcos, very open to it, if you have the idea 07:22 < morcos> but suppose in the no-change case if we overpay the fee by too much, we just be willing to loop again (to some limit so its not unbounded) 07:22 < Murch> morcos: BranchAndBound is only for the "no-change" case. If you're going to create a change output anyway, I don't understand why you're going to work so hard to minimize it to an arbitrary number. ;) 07:23 < morcos> Murch: ok, good that makes sense. Yes. Agreed with that too! 07:23 < instagibbs> morcos, maybe keep the caching logic, and just loop? 07:23 < instagibbs> with some anti-exhaustion param? 07:23 < Murch> I mean, you don't want to have all of your money in transit, but besides that, why is 0.01 BTC a great size for output? 07:23 < morcos> actually the caching logic isn't somehting i love 07:23 < morcos> it just seems to had logistical complexity 07:23 -!- so [~so@unaffiliated/so] has joined #bitcoin-core-dev 07:23 < morcos> s/had/add/ 07:24 < instagibbs> so any looping change will have to guarantee efficient convergence to a valid tx 07:26 < morcos> instagibbs: well in the dumbest implementation... just have a bool, triedAgain, and if nFeeRet < nFeeNeeded OR (nFeeRet > nFeeNeeded + too_much_extra AND !tried_again) then you loop again 07:26 < Murch> morcos: BranchAndBound does an exhaustive search, so looping again will yield no other results. We should just do a stricter limit if we feel that we're overpaying. 07:26 < morcos> Murch: we're talking about for 0.15 before BAB 07:27 < Murch> ah, I'm sorry. 07:27 < instagibbs> morcos, heh, that's basically what I tried earlier 07:27 < morcos> instagibbs: do we have a sense for how rare these overpayments are? my gut is they are extremely rare, and just being willing to discard it one time will make them all but disappear 07:27 < instagibbs> well, with more complex logic on top 07:27 < morcos> instagibbs: and what happened? 07:28 < morcos> sorry btw, for engaging on this so late, was too caught up in the fee estimate stuff (so many edge cases to get right there too, but probably this should have been more important) 07:28 < instagibbs> appreciate the feedback 07:30 < instagibbs> so to happen twice you'd basically have to oscillate twice, and get exact matches twice 07:30 < Murch> Here's an idea: How about we just raise the target for the knapsack a little, and use that adjustment to buffer missing fees but remain over MIN_CHANGE? 07:31 < morcos> Murch: we allow reducing MIN_CHANGE to MIN_CHANGE/2 to achieve that affect, its only the case where we found an exact match but used fewer inputs that we have a problem 07:31 -!- cryptapus_afk is now known as cryptapus 07:32 < Murch> ah okay 07:32 < Murch> sorry, I'm late to the conversation, maybe I'm not helping :-I 07:32 -!- haakonn [~haakonn@146.185.155.218] has joined #bitcoin-core-dev 07:32 -!- haakonn is now known as Guest5641 07:34 < morcos> instagibbs: ok here is an even more obviously correct idea i think 07:34 < instagibbs> yeah I'm not quite convincing myself on previous idea 07:34 < instagibbs> I'd like to say "no worse" 07:35 < morcos> what if we put an additional loop around the section that starts at: const CAmount nChange = nValueIn - nValueToSelect; 07:36 < morcos> where we only repeat that section if nFeeRet - nFeeNeeded is too high.. and if it was, then we change nValueToSelect to reflect the new fee, but we don't redo the coinselction 07:36 < morcos> This will result in just calculating that now we do have positive change, and creating the change output for us as expected 07:37 < morcos> I like avoiding the unintended consequences in either of the other two approaches of redoing coin selection if some criteria happens 07:38 < instagibbs> as long as that redefinition doesn't do anything weird, sounds reasonable 07:38 < morcos> In this case we're only running coin selection exactly as many times as we currently do, we're just adding a change output if we didn't have one and the fee is way too high 07:38 < morcos> you don't have to reuse the same variable 07:38 < morcos> structure that little loop to make sense 07:38 -!- cryptapus is now known as cryptapus_afk 07:39 < morcos> also i'm happy to write up that idea if you want to take a look 07:43 < instagibbs> let me take a look and give it a shot 07:49 < morcos> ok, i already started, i'll shoot you a link in a few mins, but i'm happy to let you do in your PR, i just wanted to see if it would work out easily. it might still have issues 07:52 < instagibbs> actually go ahead and work on it, I've got to wrap up work stuff before 4 day weekend 07:52 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev 07:52 < instagibbs> thanks 07:57 < morcos> heh, damn... there are few details that need to be worked out that i was hoping you'd do. 07:57 < morcos> instagibbs: anywhere here is what i started. https://github.com/morcos/bitcoin/commit/d180deabc9a6cfd94814546c48931dfb06eacc3b?w=1 07:58 < instagibbs> hah, just dont tell gmaxwell I'm working on it 07:58 < instagibbs> looking 07:59 < morcos> if thats right , i think we just need to smartly determine the threshold, and we need to convince ourselves or add logic so it can't get stuck in some infinite loop, i don't knwo if the pick_new_inputs bool needs to be reset or its too confusing to just be confident it'll complete next time or what 08:04 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev 08:07 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 08:08 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 08:09 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Client Quit] 08:11 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 08:11 < instagibbs> smells right, leaning towards "no need to reset" but will need to think more 08:12 < morcos> i don't think reseting could hurt... just in an else to if (pick_new_inputs) you reset it to true... but not sure 08:12 < morcos> also not sure what to do about the threshold, but i think that problem is no different than 10333, it's just more explicit 08:13 < instagibbs> sure, I think reset would be more "default" behavior, easier to review 08:14 < morcos> I think something simplish like GetMinimumFee(Approx size of change output) + Min_Change_size (Dust Threshold of change output) is about right 08:14 < instagibbs> re:threshhold, I think you should also be adding the current nChange in the calc 08:14 < morcos> it's maybe a bit hacky, but could be cleaned up once we have effective value machinery in the future 08:14 < morcos> huh? 08:14 < morcos> there is no current nChange 08:14 < morcos> that section is only it if changeouput position == -1 08:15 < morcos> only hit 08:15 < morcos> anyway, afk for a few 08:15 < instagibbs> right, but that is going to fees, and you may not need to after 08:15 < instagibbs> later 08:15 < morcos> wait what 08:16 < instagibbs> ill annotate on github... 08:16 < morcos> ok 08:17 < instagibbs> nevermind, was wrong 08:17 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has quit [Quit: Leaving] 08:25 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Read error: Connection reset by peer] 08:33 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev 08:39 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-snztfnuyjppnjgen] has joined #bitcoin-core-dev 08:43 < sipa> wumpus: zfs 08:47 -!- EarlyGrey [~lepton@209.107.210.197] has quit [Ping timeout: 260 seconds] 08:47 -!- Aaronvan_ [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 08:49 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 260 seconds] 09:04 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Ping timeout: 260 seconds] 09:06 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 258 seconds] 09:09 -!- cryptapus_afk is now known as cryptapus 09:13 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has quit [Ping timeout: 260 seconds] 09:19 -!- chjj [~chjj@2603:3024:1c04:eae0:813b:f4a4:d908:58df] has joined #bitcoin-core-dev 09:20 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has joined #bitcoin-core-dev 09:33 -!- timothy [~tredaelli@redhat/timothy] has quit [Quit: Konversation terminated!] 09:38 -!- ScurrilousG-__ [~lepton@173.245.202.168] has joined #bitcoin-core-dev 09:40 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 276 seconds] 09:44 -!- ClockCat [~CattieCat@75-109-228-219.stjocmtk01.res.dyn.suddenlink.net] has joined #bitcoin-core-dev 09:44 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev 09:59 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 10:03 -!- Cheeseo [~Cheeseo@gateway/vpn/privateinternetaccess/cheeseo] has joined #bitcoin-core-dev 10:05 -!- chjj [~chjj@2603:3024:1c04:eae0:813b:f4a4:d908:58df] has quit [Changing host] 10:05 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev 10:06 -!- Dizzle [~dizzle@108.171.182.16] has joined #bitcoin-core-dev 10:09 -!- goatpig [5a5c653a@gateway/web/freenode/ip.90.92.101.58] has quit [Quit: Page closed] 10:18 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Quit: WeeChat 1.4] 10:30 < bitcoin-git> [bitcoin] morcos opened pull request #10712: Add change output if necessary to reduce excess fee (master...addchangehwenoverpaying) https://github.com/bitcoin/bitcoin/pull/10712 10:30 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 10:38 -!- vicenteH [~user@195.235.96.150] has quit [Ping timeout: 240 seconds] 10:49 -!- cryptapus is now known as cryptapus_afk 10:49 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 10:50 -!- Aaronvan_ [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 246 seconds] 11:03 -!- cheese_ [~Cheeseo@gateway/vpn/privateinternetaccess/cheeseo] has joined #bitcoin-core-dev 11:06 -!- Cheeseo [~Cheeseo@gateway/vpn/privateinternetaccess/cheeseo] has quit [Ping timeout: 240 seconds] 11:08 -!- ClockCat [~CattieCat@75-109-228-219.stjocmtk01.res.dyn.suddenlink.net] has quit [Quit: Leaving] 11:24 < bitcoin-git> [bitcoin] jnewbery closed pull request #10015: Wallet should reject long chains by default (master...walletrejectlongchains) https://github.com/bitcoin/bitcoin/pull/10015 11:26 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 11:27 < luke-jr> wumpus: is your current key on bitcoin.org? https://bitcoin.org/laanwj-releases.asc 11:27 < bitcoin-git> [bitcoin] jnewbery closed pull request #9943: Make script.py wildcard importable. (master...rpctestsprimitives) https://github.com/bitcoin/bitcoin/pull/9943 11:31 < wumpus> luke-jr: that's the release signing key, not my personal key, that's laanwj.asc 11:31 < wumpus> but both should be up to date 11:31 -!- tmddzk [~tom@c-73-189-35-88.hsd1.ca.comcast.net] has joined #bitcoin-core-dev 11:32 < luke-jr> wumpus: hmm, someone's saying it doesn't match the UASF sigs 11:32 < wumpus> the release key certainly won't 11:33 < wumpus> I only use that for sha256sums.asc, which wasn't provided for uasf sigs 11:33 < luke-jr> ah 11:33 < luke-jr> so he should use the laanwj.asc? 11:33 < wumpus> yes 11:34 < wumpus> 0x1E4AED62986CD25D is the only subkey I use for signing 11:45 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 240 seconds] 11:50 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev 12:00 -!- cheese_ [~Cheeseo@gateway/vpn/privateinternetaccess/cheeseo] has quit [Ping timeout: 260 seconds] 12:04 -!- dermoth [~dermoth@gateway/tor-sasl/dermoth] has quit [Remote host closed the connection] 12:06 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 12:07 -!- cheese_ [~Cheeseo@unaffiliated/cheeseo] has joined #bitcoin-core-dev 12:07 < wumpus> strange, I can't get one node to sync from another with master, which works with 0.14 12:08 < wumpus> (the node I'm syncing from is not up-to-date, but does that matter? it's sending the 'headers', just seems to get ignored) 12:11 < sipa> i believe nodes don't respond to headers question until they're in sync 12:11 < wumpus> just trying to do a benchmark, why does this have to be so difficut every time :( 12:12 < wumpus> yes, I already patched the client node for this, it responds to all getheaders 12:12 < wumpus> I'm sure the headers is being sent, the 0.14 branch node synced fine from it 12:13 < sipa> so what is the setup exactly? 12:13 < sipa> you have a not-fully synced node A, and a new node B, connected to A? 12:14 < wumpus> https://github.com/bitcoin/bitcoin/issues/9923 is the problem you're talking about, but I know about that one :) 12:14 < wumpus> yes 12:15 < wumpus> A only listens, has a part of the block chain, B only connects, and should sync from A so that they end up with the same blocks 12:15 < wumpus> (and most important, same utxo database) 12:19 -!- elkalamar_ [~elkalamar@84.126.69.179.dyn.user.ono.com] has joined #bitcoin-core-dev 12:21 < wumpus> A has blocks up to 430241 12:22 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 255 seconds] 12:22 -!- schmidty [~schmidty@unaffiliated/schmidty] has quit [Read error: Connection reset by peer] 12:22 -!- elkalamar [~elkalamar@84.126.69.179.dyn.user.ono.com] has quit [Remote host closed the connection] 12:22 -!- MarcoFalke [~none@198.12.116.246] has quit [Ping timeout: 255 seconds] 12:22 -!- nOgAnOo [sid146237@gateway/web/irccloud.com/x-dqlplazdqqnmocor] has quit [Ping timeout: 255 seconds] 12:22 -!- MarcoFalke [~none@198.12.116.246] has joined #bitcoin-core-dev 12:22 < sipa> A sends getheaders, B responds with headers? 12:23 < sipa> eh, other way around 12:23 < gmaxwell> if you're too far back you'll stay stuck in IBD. In particular you have to at least meet the minimum chain work to get out of IBD. 12:23 < wumpus> 2017-06-30 19:22:11 received: getheaders (997 bytes) peer=5 12:23 < wumpus> 2017-06-30 19:22:11 getheaders 430241 to end from peer=5 12:23 -!- nOgAnOo [sid146237@gateway/web/irccloud.com/x-kxxjjzdrccfymxkv] has joined #bitcoin-core-dev 12:23 < gmaxwell> and I believe 430241 will not. 12:23 < wumpus> 2017-06-30 19:22:11 sending headers (82 bytes) peer=5 12:23 < wumpus> (that's on A) 12:23 < wumpus> B gets a headers message with one entry: 2017-06-30 19:22:11 ProcessNewBlockHeaders: 000000000000000003f1410b194facc9092a2b76e99db8653ec1a32edfd2ab29 12:23 < sipa> that's a remarkably small headers packet 12:23 -!- PRab [~chatzilla@c-68-56-234-28.hsd1.mi.comcast.net] has quit [Remote host closed the connection] 12:23 < gmaxwell> if you make IsInitialBlockDownload return true, you'll be good to go; my bet. 12:24 < wumpus> (that's the blockhash for block 430241 ) 12:24 < gmaxwell> er return false. 12:24 < wumpus> on A or B? 12:24 < sipa> so it looks like B already has all the headers? 12:24 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 260 seconds] 12:24 < sipa> (up to 430241) 12:24 < gmaxwell> on the thing with 430241 blocks (I believe that is A?) 12:25 < wumpus> sipa: seems so! I can delete B's state and retry if that helps 12:25 < sipa> wumpus: what does getblockchaininfo on B say? 12:25 < gmaxwell> (though if B has the headers this sounds like it might be something else!) 12:25 < sipa> or even getchaintips 12:25 < wumpus> "headers": 430241, 12:26 < wumpus> so it has all the headers, but only blocks up to the genesis block 12:27 < wumpus> so the problem is in B which is not requesting any blocks 12:27 < sipa> so while B is in IBD, there are some restrinction on where it downloads from 12:28 < wumpus> gmaxwell: yes I already patched out the code on A that would make it refuse to send headers when in IBD, I struggled with that one too many times to forget 12:28 < gmaxwell> or it did but A didn't reply? (I assume you have debug=net and so you can tell?) 12:28 < sipa> B only has one connection? 12:28 < sipa> wumpus: getpeerinfo on B does not list any blocks in flight? 12:28 < wumpus> sipa: yes, B can get only one connection, and nothing inflight 12:29 < wumpus> gmaxwell: no getblocks 12:29 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev 12:30 < sipa> does getchaintips say anything about invalid chains? 12:30 < gmaxwell> sweet, sounds like a bug. 12:32 < wumpus> getchaintips for A and B (guess B is the relevant one) https://0bin.net/paste/6Kqmm+1VMAhkOTJy#T9erOX4pcLnHu0uW++cugcWEkwDSFdKPlrefkP1nL86 12:32 < sipa> to debug (if the behaviour remains after a restart of B, perhaps add LogPrintf's to all the return statements in FindNextBlocksToDownload on B 12:32 < sipa> there are many cases in which it decides not to return anything 12:32 < sipa> i'd be helpful to know if a) it is being invoked and b) if yes, why it does not return anything 12:33 < sipa> getchaintips looks totally normal 12:33 < gmaxwell> or just attach gdb breakpoint at FindNextBlocksToDownload and restart node A and step through it? 12:33 < wumpus> trying with a C (fresh B) first, to see if it getting stuck after the headers is reproducible 12:34 < wumpus> yep, same problem second time 12:34 < wumpus> received all the headers but making no block requests 12:35 -!- chjj [~chjj@2603:3024:1c04:eae0:813b:f4a4:d908:58df] has joined #bitcoin-core-dev 12:36 < sipa> wumpus: i'll give you a patch to debug 12:37 < sipa> ah, i think i found it 12:37 < sipa> if (state->pindexBestKnownBlock->nChainWork < UintToArith256(consensusParams.nMinimumChainWork)) { // This peer has nothing interesting 12:38 < gmaxwell> ah! 12:38 < wumpus> "This peer has nothing interesting." 12:38 < wumpus> yes, just narrowed it down to there 12:38 < wumpus> 0.14.2 thinks the peer does have something interesting 12:38 < gmaxwell> yea, thats it. e2652002b6011f793185d473f87f1730c625593b added that. 12:39 < gmaxwell> -- 12:39 < gmaxwell> Delay parallel block download until chain has sufficient work 12:39 < gmaxwell> 12:39 < gmaxwell> nMinimumChainWork is an anti-DoS threshold; wait until we have a proposed 12:39 < gmaxwell> tip with more work than that before downloading blocks towards that tip. 12:39 < gmaxwell> -- 12:39 < wumpus> just going to delete that code for now, see if it works then 12:39 < gmaxwell> basically it impedes a dos attack where I make a long fork of low difficulty blocks with an asic miner and run you out of diskspace fetching those blocks. 12:39 -!- chjj [~chjj@2603:3024:1c04:eae0:813b:f4a4:d908:58df] has quit [Changing host] 12:39 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev 12:40 < wumpus> can we please have a mode for benchmarking that disables all these annoying checks :) 12:42 < gmaxwell> wumpus: IIRC sdaftuar wanted a hidden commandline option to let you set the nMinimumChainWork to another value. 12:43 < wumpus> now there's this one, as well as #9923, every time it becomes more difficult to sync nodes to each other in synthethic circumstances 12:43 < gribble> https://github.com/bitcoin/bitcoin/issues/9923 | Whitelisting outgoing connections · Issue #9923 · bitcoin/bitcoin · GitHub 12:43 < wumpus> ok 12:43 < sipa> #10357 12:43 < gribble> https://github.com/bitcoin/bitcoin/issues/10357 | Allow setting nMinimumChainWork on command line by sdaftuar · Pull Request #10357 · bitcoin/bitcoin · GitHub 12:43 < gmaxwell> hurray I remembered who was doing what for once. 12:44 < wumpus> lol apparently that was a bad idea, now it segfaults 12:44 < gmaxwell> you can't remove the null check. :P 12:44 < gmaxwell> git revert e2652002b6011f793185d473f87f1730c625593b 12:45 < wumpus> 478 state->pindexLastCommonBlock = chainActive[std::min(state->pindexBestKnownBlock->nHeight, chainActive.Height())]; 12:46 < wumpus> right 12:47 < wumpus> that did it, it's syncing 12:48 < wumpus> thanks! 12:52 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 276 seconds] 12:53 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 12:58 -!- vicenteH [~user@135.234.15.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 12:58 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 13:06 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has quit [Remote host closed the connection] 13:07 -!- flerpaderp is now known as florpadorp 13:08 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 13:26 -!- cheese_ [~Cheeseo@unaffiliated/cheeseo] has quit [Ping timeout: 240 seconds] 13:26 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 255 seconds] 13:34 -!- technoid [~technoid@198.199.115.42] has joined #bitcoin-core-dev 13:40 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev 13:45 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 13:51 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 14:01 -!- ScurrilousG-__ [~lepton@173.245.202.168] has quit [Quit: thanks] 14:02 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 14:24 -!- Guyver2 [~Guyver@guyver2.xs4all.nl] has quit [Quit: Going offline, see ya! (www.adiirc.com)] 14:32 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 276 seconds] 14:33 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 260 seconds] 14:36 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev 14:44 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev 14:45 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 14:53 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 255 seconds] 15:03 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 15:06 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev 15:14 -!- Cheeseo [~Cheeseo@gateway/vpn/privateinternetaccess/cheeseo] has joined #bitcoin-core-dev 15:31 -!- Dizzle [~dizzle@108.171.182.16] has quit [Quit: Leaving...] 15:32 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 15:34 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Client Quit] 15:36 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 255 seconds] 15:42 -!- Cheeseo [~Cheeseo@gateway/vpn/privateinternetaccess/cheeseo] has quit [Ping timeout: 240 seconds] 15:44 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Ping timeout: 276 seconds] 15:50 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev 15:51 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 15:58 -!- technoid [~technoid@198.199.115.42] has quit [Quit: Leaving] 16:00 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev 16:05 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 16:08 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 16:14 -!- Cheeseo [~Cheeseo@unaffiliated/cheeseo] has joined #bitcoin-core-dev 16:22 -!- Cheeseo [~Cheeseo@unaffiliated/cheeseo] has quit [Read error: Connection reset by peer] 16:50 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 16:52 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 17:08 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 17:13 -!- Netsplit *.net <-> *.split quits: arubi 17:15 -!- Yogaqueef [~textual@dsl-hkibng42-5673c3-32.dhcp.inet.fi] has quit [Quit: My iMac has gone to sleep. ZZZzzz…] 17:19 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 17:20 -!- arubi [~ese168@gateway/tor-sasl/ese168] has joined #bitcoin-core-dev 17:49 -!- arubi [~ese168@gateway/tor-sasl/ese168] has quit [Ping timeout: 248 seconds] 17:53 < AdrianG> whats the alt stack for in bitcoin script? 17:54 < luke-jr> AdrianG: an alternate stack 17:55 < AdrianG> so its a two-stack pushdown automata then? 17:55 < AdrianG> are the stacks fully equivalent? 17:56 < sipa> bitcoin script is not a pushdown automata 17:56 < sipa> because it has OP_PICK 17:57 < sipa> code has access to all elements, always, not just the top 17:58 < luke-jr> AdrianG: no, the alt stack is wiped between scripts, and has fewer opcodes to use it 17:58 * luke-jr OP_PICKs sipa 17:58 < AdrianG> sipa: that sounds like it would be a superset of a PDA then? 18:03 -!- arubi [~ese168@gateway/tor-sasl/ese168] has joined #bitcoin-core-dev 18:07 < sipa> it also follows intructions, not a state transition table 18:18 -!- jannes [~jannes@178.132.211.90] has quit [Quit: Leaving] 18:26 -!- ovovo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev 18:28 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 240 seconds] 18:30 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 260 seconds] 18:32 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.] 18:45 -!- goofie [~goofie@104-7-174-119.lightspeed.hstntx.sbcglobal.net] has joined #bitcoin-core-dev 18:50 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-snztfnuyjppnjgen] has quit [Quit: Connection closed for inactivity] 19:13 -!- ovovo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 260 seconds] 19:18 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev 19:19 -!- dermoth [~dermoth@gateway/tor-sasl/dermoth] has joined #bitcoin-core-dev 19:24 -!- MarcoFalke [~none@198.12.116.246] has quit [Ping timeout: 255 seconds] 19:24 -!- MarcoFalke [~none@198.12.116.246] has joined #bitcoin-core-dev 19:26 -!- tmddzk [~tom@c-73-189-35-88.hsd1.ca.comcast.net] has quit [Ping timeout: 255 seconds] 19:31 < gmaxwell> AdrianG: you can rewrite any script using the altstack without it (just using pick). The altstack is there for the same reason that it's there in any forth implementation-- its a convenience and can result in fewer operations for some things. 19:31 < gmaxwell> apparently you've heard this craig wright stuff, it's pure gibberish. 19:33 < gmaxwell> (he was sending it to me on reddit several weeks ago too: https://people.xiph.org/~greg/temp/DSScamamoto.png ) 19:34 < gmaxwell> AdrianG: his gibbering is basically trying to argue that script is universal but you don't need any of this technobable to show that, all you need to show is that it has a controlled-not. 19:34 < gmaxwell> That doesn't mean much of anything interesting though because there are very tight limits on how much computation you can do (which is why it isn't turing complete) 19:36 < gmaxwell> Wright is doing some pattern matching with finite state machines, where there is a well known result that one equipt with two 'stacks' (which are not like the forth-like stacks in bitcoin, but are simple things where you can only access the top elements) can be universal. 20:13 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Ping timeout: 248 seconds] 20:17 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has quit [Remote host closed the connection] 20:18 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has joined #bitcoin-core-dev 20:28 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev 20:43 < BlueMatt> gmaxwell: no wonder wright hates you....you dont trivially fall for stupid technobabble like so many others 20:55 -!- goatpig [5a5c653a@gateway/web/freenode/ip.90.92.101.58] has joined #bitcoin-core-dev 21:18 -!- ProfMac_ [43c671dc@gateway/web/freenode/ip.67.198.113.220] has joined #bitcoin-core-dev 21:26 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 248 seconds] 21:27 -!- ivan [~ivan@unaffiliated/ivan/x-000001] has joined #bitcoin-core-dev 21:31 -!- owowo [~ovovo@s1349015191.blix.com] has joined #bitcoin-core-dev 21:31 -!- owowo [~ovovo@s1349015191.blix.com] has quit [Changing host] 21:31 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev 22:32 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Remote host closed the connection] 22:58 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 23:00 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has quit [Client Quit] 23:01 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 23:01 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 268 seconds] 23:50 < wumpus> craig wright hates us all 23:50 < wumpus> but yes that's a good thing 23:52 < wumpus> results from yesterday's benchmark sync up to block 430241: 0.14.2 took 08:12:29 (29549s), master took 06:20:57 (22857s), so ~23% faster 23:53 < wumpus> really neat 23:55 < wumpus> both with default dbcache setting