--- Day changed Thu Aug 03 2017
00:07 -!- JackH [~laptop@46.231.18.66] has joined #bitcoin-core-dev     
00:10 -!- marcoagner [~user@177.41.203.37] has joined #bitcoin-core-dev     
00:11 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
00:12 -!- BashCo_ [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
00:16 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 260 seconds]     
00:20 -!- Orie [~Orie@ns334669.ip-5-196-64.eu] has quit [Remote host closed the connection]     
00:41 -!- clarkmoody [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has joined #bitcoin-core-dev     
00:41 -!- timothy [~tredaelli@redhat/timothy] has joined #bitcoin-core-dev     
00:46 -!- timothy [~tredaelli@redhat/timothy] has quit [Quit: Konversation terminated!]     
00:47 -!- timothy [~tredaelli@redhat/timothy] has joined #bitcoin-core-dev     
00:47 -!- Cheeseo [~Cheeseo@unaffiliated/cheeseo] has quit [Ping timeout: 246 seconds]     
00:47 -!- timothy [~tredaelli@redhat/timothy] has quit [Remote host closed the connection]     
00:50 -!- timothy [~tredaelli@redhat/timothy] has joined #bitcoin-core-dev     
00:52 -!- jtimon [~quassel@6.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev     
01:01 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev     
01:02 -!- yRDIUTgn [~YRxcrYTVB@2a02:2f0a:b0a0:857:ff57:a224:17bd:a712] has joined #bitcoin-core-dev     
01:03 < gmaxwell> I wonder if we should adopt a patch to instantly disconnect bcash nodes based on their service flags.  Sucks to burn a service bit forever to their recklessness though. :(
01:03 -!- Aaronvan_ [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev     
01:04 < gmaxwell> and presumably the constant inadvertant dos attack of connecting to nodes on another network will move them onto another port eventually.
01:06 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 240 seconds]     
01:07 -!- Deacyde [~Deacyde@unaffiliated/deacyde] has joined #bitcoin-core-dev     
01:09 < praxeology> potentially in the future the burned service code could be recovered in the future after the problem goes away
01:09 < praxeology> are they refusing to move to a different port?
01:10 < gmaxwell> they refused people people asked them previously.
01:10 < gmaxwell> it's not all that easy to recover a service bit where its use results in instantly being disconnected!
01:11 < praxeology> Maybe in a month bcash won't have anyone mining it anymore
01:11 < gmaxwell> perhaps we shouldn't worry much about burning one because we know we're due for some other pretty substantial p2p revisions, and other updates can add new capabilities flags.
01:12 < gmaxwell> I think we only use service flags now for things we absolutely need to have in addr messages, and if we create a addr message replacement (to support NG hidden services and I2P, we'd probably give it a different capabilities signaling tool)
01:14 < praxeology> service flags... "flags" implies a 32 bit or 64 bit number?
01:14 < praxeology> Maybe switch to a set of service tags instead?
01:15 < gmaxwell> well, they need to be small because they're rumored everwhere in addr messages.
01:15 -!- d_t [~d_t@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 276 seconds]     
01:15 < gmaxwell> making them fat sidechannels would likely have clowns using them for file trading. :)
01:17 < praxeology> ok, well you could put a size constraint on the tag set... but whatever I'm just sleep dep and over engineering something I don't know enough about
01:25 -!- tucenaber [~tucenaber@unaffiliated/tucenaber] has quit [Read error: Connection reset by peer]     
01:25 -!- tucenaber [~tucenaber@o144.231.lokis.net.pl] has joined #bitcoin-core-dev     
01:25 -!- tucenaber [~tucenaber@o144.231.lokis.net.pl] has quit [Changing host]     
01:25 -!- tucenaber [~tucenaber@unaffiliated/tucenaber] has joined #bitcoin-core-dev     
01:55 -!- goatpig [56f75436@gateway/web/freenode/ip.86.247.84.54] has quit [Quit: Page closed]     
02:02 -!- ivan [~ivan@unaffiliated/ivan/x-000001] has joined #bitcoin-core-dev     
02:10 -!- bincap [~vd@85-222-72-154.dynamic.chello.pl] has quit [Ping timeout: 255 seconds]     
02:10 -!- rafalcpp [~racalcppp@84-10-11-234.static.chello.pl] has quit [Ping timeout: 246 seconds]     
02:11 -!- miknotauro [~miknotaur@S0106a84e3fc27d33.vc.shawcable.net] has quit [Ping timeout: 276 seconds]     
02:27 < Eliel> jonasschnelli: is bitcoind's code that does that too difficulty to understand?
02:27 < Eliel> (never mind, was looking at the past)
02:27 < jonasschnelli> Eliel: depends on you experience
02:27 < Eliel> ah true
02:34 -!- jimpo [~jimpo@89.197.87.82] has joined #bitcoin-core-dev     
02:34 -!- LeMiner [~LeMiner@unaffiliated/leminer] has joined #bitcoin-core-dev     
02:59 -!- Austindoggie [~austindog@2601:647:ca80:1f82:fd06:4cbc:9cdf:75c0] has joined #bitcoin-core-dev     
03:01 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has quit [Remote host closed the connection]     
03:03 -!- karelb [~karelb@li1380-211.members.linode.com] has joined #bitcoin-core-dev     
03:03 < gmaxwell> Hi all, karelb is working on the trezor wallet, and they've been trying to use these patches to bitcoin that implement the bitcore address indexing stuff, but they're finding it really slow to the point where performance is problematic.
03:04 < karelb> we are using it for a while now :)
03:04 < karelb> but right now we are reindexing bitcoin blockchain and it takes foreved
03:04 < gmaxwell> Maybe someone would be interested in giving them a bit of a hand at looking at it? (I need to get to bed); I've already provided the standard disclaimers about the inherent non-scalability of address-indexes-of-all-history. :)
03:04 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev     
03:05 < karelb> (It is actually bitcoin-abc, I hope I won't be banned :D, but this issue popped up in bitcoin core too)
03:05 < gmaxwell> karelb: one question would be what kind of system is this running on? e.g. is it on some VPS with remote storage that may have poor IO speed?
03:05 < gmaxwell> karelb: we'll forgive you for your sins. though obviously can't help with any abc specific issues.
03:05 < karelb> it is our local server that has SSD, lots of RAM and processors
03:06 < gmaxwell> darn.
03:06 < karelb> actually one issue was sort-of ABC related, but it was because of a commit ported from master from bitcoin core, so it will be relevant anyway
03:07 < karelb> https://github.com/Bitcoin-ABC/bitcoin-abc/issues/43 , https://github.com/satoshilabs/bitcoin-abc/commit/5337f8f210eaa34d1212103f700698dd4989f479
03:07 < gmaxwell> so since I doubt that addrindex has a useful caching layer, you could look for the leveldb::Options object for the database it creates and try increasing the options.block_cache and options.write_buffer_size to really large levells.
03:07 < karelb> it's because the bitcore patches do address index in ConnectBlock/DisconnectBlock, even on the start during the testing thing
03:07 -!- Aaronvan_ [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 268 seconds]     
03:08 < karelb> (jpochyla is my colleague)
03:08 < gmaxwell> karelb: probably code based on Bitcoin Core master (including ABC) will not be reliably compatible with that address indexing stuff until it is changed. 
03:08 < karelb> And the introduction of ApplyBlockUndo somehow caused that
03:09 < karelb> yeah. We want to write our own indexing thing that you won't need to put inside the C++ code, since that is a little insane and we need to keep on rebasing that
03:09 -!- erts [bb655492@gateway/web/freenode/ip.187.101.84.146] has joined #bitcoin-core-dev     
03:10 -!- erts is now known as Guest16355     
03:10 < karelb> gmaxwell: just fyi, ABC is not based on master, but on 0.14.1, but they took that thing from master
03:10 < gmaxwell> karelb: bitcoin core master changed the atomiticity requirements for the backend database, but a side effect of this is that it needs special replay logic to handle crash recovery.  ABC has partially ported some of these changes. I am not sure, but I wouldn't be surprised if the address indexing would get corrupted until updated to have the right synchronization behavior.
03:10 < gmaxwell> I know it's based on 0.14.1, but they copied some of these database changes.
03:10 < karelb> ok     
03:11 < gmaxwell> in any case, beyond the cache options I mentioned above, I am out of ideas for making it faster without substantial design changes.
03:11 < gmaxwell> I think you should probably also make an extra effort to always cleanly start and stop that node, because I wouldn't be confident that it is durable across crashes without corruption.
03:13 < Austindoggie> Did it take a long time to reindex because you went back a version of bitcoin core?
03:14 < Austindoggie> Sorry if im not allowed to talk here...
03:14 < karelb> gmaxwell: hah, that is not a good news. We also noticed the node randomly crashes once in a while and we are not sure why
03:15 < gmaxwell> ::sigh::
03:15 < karelb> yeah
03:16 < karelb> block_cache and write_buffer_size can be set via conf, or only in code?
03:16 < karelb> wait I will have a look
03:16 < gmaxwell> karelb: run more nodes, reindex anytime one crashes?  At least until you can test if the address index is accurate across crashes?  I'm not completely confident that it won't be, but it would take some careful review and testing to be sure (especially in the context of ABC that has really scrambled things up a lot)
03:17 < gmaxwell> karelb: I think only in the code, I don't think there is an external way to override.
03:17 < gmaxwell> in any case, I have to go to bed. Hopefully someone else will wake up and have some other suggestions.
03:17 < karelb> (I really hate how ABC reformatted everything for no good reason, but that is another issue from this)
03:17 -!- jannes [~jannes@095-097-246-234.static.chello.nl] has joined #bitcoin-core-dev     
03:18 -!- Guest16355 [bb655492@gateway/web/freenode/ip.187.101.84.146] has left #bitcoin-core-dev []     
03:18 < gmaxwell> yes, it makes it really hard to see what exactly has changed because the reformats were heavily intermixed with real changes. :(
03:19 < karelb> @austindoggie nope we started to download blockchain from 0. And it is always stuck on IO for insanely long times
03:20 < karelb> and normal bitcoin without bitcore patches doesn't do it, on the same HW
03:20 < gmaxwell> karelb: gross final suggestion before I really go:  if you really have a lot of ram, create a tmpfs mount big enough for the entire datadir, and sync in there, when it finishes copy it to disk.
03:21 < gmaxwell> karelb: the challenge there is that we have insanely optimized bitcoin core's sync process.. we avoid writing to leveldb at all costs, basically, and a significant fraction of UTXO never hit the database at all during normal sync because they're spent before the dbcache fills)
03:24 -!- dobak [~dobak@c237-242.icpnet.pl] has joined #bitcoin-core-dev     
03:25 < karelb> thanks a lot for your help
03:25 < karelb> going to IRC was a wild shot but it seems it might help :)
03:25 -!- dobak [~dobak@c237-242.icpnet.pl] has left #bitcoin-core-dev []     
03:26 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/659c09613408...2e857bb619f5
03:26 < bitcoin-git> bitcoin/master 49d903e Alex Morcos: Eliminate fee overpaying edge case when subtracting fee from recipients
03:26 < bitcoin-git> bitcoin/master 2e857bb Wladimir J. van der Laan: Merge #10942: Eliminate fee overpaying edge case when subtracting fee from recipients...
03:27 -!- LordCow [~lordcow@lordcow.org] has joined #bitcoin-core-dev     
03:27 < bitcoin-git> [bitcoin] laanwj closed pull request #10942: Eliminate fee overpaying edge case when subtracting fee from recipients (master...subtractfee) https://github.com/bitcoin/bitcoin/pull/10942
03:34 < karelb> Hm. options.block_cache and options.write_buffer_size are derived from dbcache
03:35 < karelb> dbcache option
03:35 < jonasschnelli> Updated to Debian 9 (stretch) and suddenly get gitian build errors: init.lxc: failed to mount /dev/shm : No such file or directory
03:35 < jonasschnelli> Anyone else experiences this?
03:39 -!- Austindoggie [~austindog@2601:647:ca80:1f82:fd06:4cbc:9cdf:75c0] has quit [Quit: AndroIRC - Android IRC Client ( http://www.androirc.com )]     
03:39 -!- Austindoggie [~austindog@2601:647:ca80:1f82:fd06:4cbc:9cdf:75c0] has joined #bitcoin-core-dev     
03:39 -!- Deacydal [~Deacyde@unaffiliated/deacyde] has joined #bitcoin-core-dev     
03:40 < karelb> maximum dbcache is 16384 MB hard-capped in code; is there a reason for that?
03:40 < jonasschnelli> karelb: why would you need more?
03:41 -!- Deacyde [~Deacyde@unaffiliated/deacyde] has quit [Ping timeout: 240 seconds]     
03:42 -!- sam_c [~sam_c@gateway/tor-sasl/stanley] has joined #bitcoin-core-dev     
03:47 < karelb> see the past discussion... we are running bitcore address index patches and they are very slow and stuck on disk operations. And it happens probably because it gets stuck on adding things to address index and commiting it to hard disk on every block
03:47 < karelb> probably
03:48 < karelb> it must be because of the address index somehow, because on the same PC, bitcoin core without bitcore (sigh) patches runs fine
03:48 -!- sam_c [~sam_c@gateway/tor-sasl/stanley] has quit [Remote host closed the connection]     
03:49 < karelb> I am actually talking about ABC, but the same issue crops up in bitcoin, plus it might be because ABC added some db logic from master
03:49 -!- sam_c [~sam_c@gateway/tor-sasl/stanley] has joined #bitcoin-core-dev     
03:50 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev     
03:51 -!- Austindoggie [~austindog@2601:647:ca80:1f82:fd06:4cbc:9cdf:75c0] has quit [Quit: AndroIRC - Android IRC Client ( http://www.androirc.com )]     
03:53 < karelb> hm, we stopped the bitcoind and it got into some crashed state which is inconsistent and nothing happens
03:53 < karelb> :/     
03:53 < karelb> this is hell
03:54 < gmaxwell> karelb: the leveldb caching isn't very useful for the leveldb databases in core because we've already cached the heck out of those things at a higher level...  but for the options set for your custom address index it may be very helpful.
03:55 -!- jr2014 [~dev@120.6.97.168] has quit [Ping timeout: 260 seconds]     
04:03 -!- sam_c [~sam_c@gateway/tor-sasl/stanley] has quit [Remote host closed the connection]     
04:05 < karelb> well it started chugging again, but is still gets stuck. With dbcache set at 16384. Memory has 1GB full and 63GB free
04:06 < gmaxwell> karelb: what does stuck mean exactly
04:07 -!- felco [~felco@unaffiliated/felco] has joined #bitcoin-core-dev     
04:08 < karelb> Log says nothing, and iotop shows 100% and doing a lot of reading/writing in the leveldb
04:09 < karelb> and after about 20 minutes, log messages start to appear again
04:09 < gmaxwell> dear lord. :(
04:13 < karelb> and the binary doesn't reply even to kill signals
04:19 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Read error: No route to host]     
04:19 -!- paveljanik [~paveljani@79.98.72.176] has joined #bitcoin-core-dev     
04:19 -!- paveljanik [~paveljani@79.98.72.176] has quit [Changing host]     
04:19 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev     
04:24 -!- brg444 [sid207215@gateway/web/irccloud.com/x-tgmixjhlzalpzrpf] has quit [Ping timeout: 255 seconds]     
04:26 -!- jr2014 [~dev@120.6.97.168] has joined #bitcoin-core-dev     
04:27 -!- brg444 [sid207215@gateway/web/irccloud.com/x-pbrwlibkjykanuzw] has joined #bitcoin-core-dev     
04:29 < karelb> data/blocks/index is compacting like crazy when it is stuck
04:30 < karelb> https://pastebin.com/SvsyQyiL
04:33 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has quit [Remote host closed the connection]     
04:34 < karelb> The bitcoind is stuck and at the same time when this is hapenning
04:34 < karelb> and it keeps writing compacting
04:34 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has joined #bitcoin-core-dev     
04:34 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev     
04:40 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt]     
04:55 -!- snq [~random@93.190.141.159] has joined #bitcoin-core-dev     
05:02 -!- drizztbsd [~tredaelli@redhat/timothy] has joined #bitcoin-core-dev     
05:03 -!- timothy [~tredaelli@redhat/timothy] has quit [Ping timeout: 260 seconds]     
05:11 -!- blznblzn2 [~blzn@2605:6001:f28d:e600:cf36:4af4:2baf:900a] has quit [Remote host closed the connection]     
05:23 -!- jimpo [~jimpo@89.197.87.82] has quit [Ping timeout: 268 seconds]     
05:31 -!- dobak [~dobak@c237-242.icpnet.pl] has joined #bitcoin-core-dev     
06:03 -!- Cheeseo [~Cheeseo@unaffiliated/cheeseo] has joined #bitcoin-core-dev     
06:04 -!- jeep-ss [~chatzilla@78.31.64.4] has joined #bitcoin-core-dev     
06:07 -!- rafalcpp [~racalcppp@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev     
06:07 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/2e857bb619f5...e222618a32a1
06:07 < bitcoin-git> bitcoin/master 3498a8d Cory Fields: depends: fix fontconfig with newer glibc...
06:07 < bitcoin-git> bitcoin/master e222618 Wladimir J. van der Laan: Merge #10851: depends: fix fontconfig with newer glibc...
06:08 < bitcoin-git> [bitcoin] laanwj closed pull request #10851: depends: fix fontconfig with newer glibc (master...fontconfig-bump) https://github.com/bitcoin/bitcoin/pull/10851
06:14 -!- dobak [~dobak@c237-242.icpnet.pl] has quit [Ping timeout: 260 seconds]     
06:16 -!- dobak [~dobak@c237-242.icpnet.pl] has joined #bitcoin-core-dev     
06:17 -!- cheese_ [~Cheeseo@unaffiliated/cheeseo] has joined #bitcoin-core-dev     
06:18 -!- dobak [~dobak@c237-242.icpnet.pl] has left #bitcoin-core-dev []     
06:20 -!- spudowiar [~spudowiar@unaffiliated/saleemrashid] has joined #bitcoin-core-dev     
06:20 -!- Cheeseo [~Cheeseo@unaffiliated/cheeseo] has quit [Ping timeout: 255 seconds]     
06:21 -!- Guyver2 [~Guyver@guyver2.xs4all.nl] has joined #bitcoin-core-dev     
06:24 -!- dobak [~dobak@c237-242.icpnet.pl] has joined #bitcoin-core-dev     
06:27 -!- Chris_Stewart_5 [~chris@gateway/vpn/privateinternetaccess/chrisstewart5/x-62865615] has joined #bitcoin-core-dev     
06:56 -!- dobak [~dobak@c237-242.icpnet.pl] has quit []     
07:30 -!- jtimon [~quassel@6.31.134.37.dynamic.jazztel.es] has quit [Ping timeout: 260 seconds]     
07:30 -!- PaulCapestany [~PaulCapes@ip72-209-228-52.dc.dc.cox.net] has quit [Quit: .]     
07:34 -!- PaulCapestany [~PaulCapes@ip72-209-228-52.dc.dc.cox.net] has joined #bitcoin-core-dev     
07:36 -!- timothy [~tredaelli@redhat/timothy] has joined #bitcoin-core-dev     
07:37 -!- drizztbsd [~tredaelli@redhat/timothy] has quit [Ping timeout: 260 seconds]     
07:45 -!- yRDIUTgn [~YRxcrYTVB@2a02:2f0a:b0a0:857:ff57:a224:17bd:a712] has quit [Remote host closed the connection]     
07:45 -!- yRDIUTgn [~YRxcrYTVB@2a02:2f0a:b0a0:857:ff57:a224:17bd:a712] has joined #bitcoin-core-dev     
07:46 < bitcoin-git> [bitcoin] NicolasDorier opened pull request #10980: [WIP] Decouple CKeyStore from CWatchOnlyStore (master...decouplewatchonly) https://github.com/bitcoin/bitcoin/pull/10980
07:49 -!- clarkmoody [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has quit [Quit: Leaving]     
07:50 -!- clarkmoody [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has joined #bitcoin-core-dev     
07:54 -!- BashCo_ [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection]     
07:55 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
07:56 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev     
07:58 -!- Mirobit [25c9f1fd@gateway/web/freenode/ip.37.201.241.253] has joined #bitcoin-core-dev     
07:59 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 240 seconds]     
08:00 < Mirobit> Doesn't anyone know BCash nodes aren't sending BCash transactions to Core peers? My node has only recieved 2 BCash tx and banned both peers. But the other ~5-10 peers don't seem to send any txs. Weird?
08:16 -!- jr2014 [~dev@120.6.97.168] has quit [Read error: Connection reset by peer]     
08:20 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
08:30 -!- snkey [~random@185.93.182.151] has joined #bitcoin-core-dev     
08:33 -!- snq [~random@93.190.141.159] has quit [Ping timeout: 276 seconds]     
08:41 -!- jimpo [~jimpo@89.197.87.82] has joined #bitcoin-core-dev     
08:42 -!- Dizzle [~dizzle@108.171.182.16] has joined #bitcoin-core-dev     
08:52 -!- Drabiv [3e50a908@gateway/web/cgi-irc/kiwiirc.com/ip.62.80.169.8] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]     
08:52 -!- Deacydal [~Deacyde@unaffiliated/deacyde] has quit [Ping timeout: 260 seconds]     
08:52 -!- Drabiv [3e50a908@gateway/web/cgi-irc/kiwiirc.com/ip.62.80.169.8] has joined #bitcoin-core-dev     
08:59 < BlueMatt> achow101: re: #10952: Do you have any idea *how* these folks' wallets got corrupted like that?
08:59 < gribble> https://github.com/bitcoin/bitcoin/issues/10952 | [wallet] Remove vchDefaultKey and have better first run detection by achow101 · Pull Request #10952 · bitcoin/bitcoin · GitHub
08:59 < BlueMatt> I'm highly skeptical that "just write a new default key" is the right solution here
08:59 < BlueMatt> their wallet got confused somehow, auto-fixing without telling them something may be wrong is probably not a good idea
09:05 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev     
09:12 -!- sam_c [~sam_c@gateway/tor-sasl/stanley] has joined #bitcoin-core-dev     
09:16 -!- snq [~random@185.32.222.12] has joined #bitcoin-core-dev     
09:19 -!- snkey [~random@185.93.182.151] has quit [Ping timeout: 240 seconds]     
09:19 -!- JackH [~laptop@46.231.18.66] has quit [Remote host closed the connection]     
09:24 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev     
09:27 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Remote host closed the connection]     
09:31 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt]     
09:38 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-core-dev     
09:40 -!- sam_c [~sam_c@gateway/tor-sasl/stanley] has quit [Remote host closed the connection]     
09:42 -!- sam_c [~sam_c@gateway/tor-sasl/stanley] has joined #bitcoin-core-dev     
09:44 -!- fizzwont [~irc3@unaffiliated/fizzwont] has quit [Remote host closed the connection]     
09:45 -!- fizzwont [~irc3@192.169.7.153] has joined #bitcoin-core-dev     
09:45 -!- fizzwont is now known as Guest95816     
09:46 -!- Yogaqueef [~textual@dsl-hkibng42-5673c3-32.dhcp.inet.fi] has joined #bitcoin-core-dev     
09:48 -!- timothy [~tredaelli@redhat/timothy] has quit [Quit: Konversation terminated!]     
09:55 < achow101> BlueMatt: absolutely no idea how they got corrupted
09:55 < achow101> I was not able to get a copy of their wallet files either so I couldn't examine them
09:56 < achow101> BlueMatt: the solution isn't to "just write a new default key". The solution is to change the first run checker from "there's a valid default key" to "there are keys in the wallet"
09:57 < achow101> so with 10952, those wallets would not be considered to be new first run wallets as they have keys but not a valid default key
10:00 -!- jtimon [~quassel@6.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev     
10:06 < wumpus> default key should completely go (for post-0.15 though)
10:08 < BlueMatt> achow101: well my point is more broadly that their wallets clearly got corrupted
10:08 < wumpus> the check for a new allet should be replaced by a proper "is this an empty database" check
10:08 < BlueMatt> achow101: so silently continuing isnt really the right solution
10:09 < wumpus> a wallet without any keys yet could in principle be valid
10:09 < wumpus> though a bit strange as we initially generate a mempool
10:09 < wumpus> (and we don't allow mempoolsize=0)
10:09 < karelb> ok we got abc node working, but it was some crazy solution of switching between various binaries and invalidating nodes; it did not crash (yet). But it seems that bitcore patches inside connectblock/disconnectblock are really not the way for the future; it will probably break again (both in abc and later in bitcoin core once you release the db changes)
10:10 < achow101> BlueMatt: that would require a specific check for the case that a default key is invalid
10:10 -!- Chris_Stewart_5 [~chris@gateway/vpn/privateinternetaccess/chrisstewart5/x-62865615] has quit [Ping timeout: 255 seconds]     
10:10 < achow101> wumpus: a database is initially created when the wallet file is created. I'm not sure how to check it is empty except that when the database is then read in, there are no keys
10:11 < achow101> that's that 10952 does; it checks if there are any keys in the database
10:11 < wumpus> yes, that sounds sane
10:12 < wumpus> for a new format I'd suggest adding a "database-type" "wallet" k/v pair to distinguish it from other bdb databases, and check for that, but for backwards compatibility that works better
10:12 < wumpus> (and no one would be so stupid to put a random other berkeleydb database as wallet.dat? right? :-)
10:15 < Lightsword> anyone working on switching bdb out for something else like sqlite?
10:15 < achow101> Lightsword: there's a pr somewhere for that I think
10:15 < sipa> meh, overkill
10:15 < sipa> all we need is a key-value store that's effectively read into memory entirely anyway
10:16 < sipa> i had a patch years ago to switch it to an append-only flat file
10:16 < Lightsword> the main advantage to sqlite though is that it has good data integrity protection
10:16 < achow101> we could upgrade to bdb whatever latest and change how records are stored there
10:16 < wumpus> Lightsword: it's quite easy to swap the database for any k/v store, I have a local branch that uses leveldb
10:17 < BlueMatt> achow101: if you have a wallet with keys, and a default key that is invalid or missing, your wallet was clearly either corrupted or generated by something other than bitcoin core
10:17 < wumpus> achow101: bdb latest (6.x) has serious license issues
10:17 < BlueMatt> achow101: continuing silently in that case is not what we want
10:18 < Lightsword> sipa, would there not be some potential future use cases where having sql support be helpful?
10:18 < achow101> BlueMatt: if there is corruption in the keys themselves or other data, that will be caught elsewhere
10:18 < BlueMatt> achow101: we can also change the "first run" check to ignore vchDefaultKey, which we clearly should, but you're pointing to wallets in the wild that have been corrupted and suggesting we should silently continue
10:18 < achow101> BlueMatt: I don't think we should care that much about corruption in the default key as it has no use
10:19 < sipa> Lightsword: i don't wee how
10:19 < sipa> *see
10:19 < BlueMatt> achow101: yes we should!
10:19 < achow101> BlueMatt: only one of those wallets would silently continue, and the guy had no problems with older versions of core
10:19 < BlueMatt> the user should stop using that computer for a wallet!
10:19 < wumpus> sql could be useful for metadata kind of things, but meh, I don't think there's really an advantage to it for us we don't index anything
10:19 < achow101> BlueMatt: the other wallet ran into other corruption problems
10:19 < BlueMatt> that user should be told to throw out their hard drive and get a better one
10:19 < BlueMatt> or a new computer
10:19 < wumpus> although the current 'keep everything in memory' is kind of dumb
10:19 < BlueMatt> not continue
10:20 < Lightsword> yeah, I was thinking maybe metadata/multiple accounts or something along those lines could possibly make use of sql
10:20 < wumpus> if the wallet would keep things in the database instead and query them when needed, indexes could be useful
10:20 < wumpus> accounts are deprecated, if anything we're simplifying the wallet
10:21 < achow101> BlueMatt: sure they should be warned, but a corrupted default key should not be a reason to halt the software entirely as default key is useless
10:21 < Lightsword> wumpus, what about multiwallet?
10:22 < sipa> Lightsword: that uses multiple wallet files
10:22 < achow101> with older versions of core, IIRC the default key would have either been overwritten or ignored and allowed the user to continue
10:22 < BlueMatt> achow101: if someone's wallet is corrupted, we should exit with an error....if they wish to then restart with -salvagewallet or equivalent, that is also ok
10:22 < BlueMatt> achow101: and that is a bug!
10:22 < BlueMatt> if we read something and see that someone's hardware is silently corrupting their wallet, we should exit the same way we do with any other wallet corruption errors
10:23 < BlueMatt> silently fixing wallets is not ok
10:23 < sipa> wumpus: well if we expect to ever need indexes on the future because we won't keep everything in memory, i'd say that sqlite is a good choice
10:23 < wumpus> with older versions of core the lack of the default key record would make it assume it's a new wallet, which could do all kinds of bad things? I don't think that's abetter
10:23 < BlueMatt> there are clearly bugs here that achow101 identified (that i think need fixing for 15)
10:23 < sipa> wumpus: not sure if it's worthwhile though
10:24 < BlueMatt> I'm just not sure that silently correcting /anything/ is every good in wallets
10:24 < wumpus> for 0.15 it's too lte imo
10:24 < wumpus> we should do rc1 asap
10:24 < BlueMatt> wumpus: we've seen it in the wild, and it can be a simple fix
10:24 < BlueMatt> :(     
10:24 < wumpus> not add new stuff
10:24 < sipa> have we even identified the bug?
10:24 < wumpus> but that's just my opinion (and being harried at all sides to do 0.15 asap)
10:24 < BlueMatt> sipa: as far as anyone knows its hardware/bdb silently corrupting things
10:24 < sipa> as opposed to "we have ween a wallet with no default key, somehow"
10:24 < achow101> sipa: I can replicate the problem, not necessarily the cause
10:25 < wumpus> but sure if it's a clearly defined problem, with a clearly defined fix, and it can be reviewed in the next days, we can include it
10:25 < sipa> achow101: elaborate
10:25 < achow101> sipa: encrypt a wallet, use db_dump to dump it, remove the default key, load with db_load, start core, runtime exception
10:26 < sipa> well, ok
10:26 < wumpus> that's what I expect if you just remove a record
10:26 < wumpus> don't do that.
10:26 < sipa> but if you permit random file changes, you can make anything happen
10:26 < BlueMatt> wumpus: yes, sorry, my understanding is its that kind of "simple fix" that would be easy to get in in a day or two
10:26 < achow101> if you do it unencrypted, it works fine
10:26 < wumpus> if you delete a random record from the utxo database you'll also be in for a world of pain
10:26 -!- jimpo [~jimpo@89.197.87.82] has quit [Ping timeout: 246 seconds]     
10:26 < BlueMatt> the issue is that this encourages people to downgrade wallets
10:27 < wumpus> we're not resistant to that kind of corruption, and one special case is not going to change that
10:27 < achow101> I suppose then a check for a valid default key can be added and that would just be a corruption warning?
10:27 < BlueMatt> just making it a nice error message and making sure -salvagewallet works correctly is the Correct (tm) fix, imo
10:27 < wumpus> could just as well have been a key record that disappears
10:27 < BlueMatt> achow101: yes
10:27 < achow101> fine
10:27 < wumpus> salvagewallet should work as long as there is any private key left
10:27 < achow101> btw the current corruption warnings are kinda bad
10:27 < wumpus> that's a known issue...
10:28 < wumpus> there's even a chance of running into asserts on db corruption
10:29 < wumpus> would be nice if it could detect corruption at run time and offer the user a chance to repair
10:29 < wumpus> though if a wallet gets corrupted that's a good indication you should stop using that computer for bitcoin, now
10:30 < wumpus> so I agree with BlueMatt in that regard
10:31 < achow101> the main problem I have with that is it encourages people who don't know what they are doing (i.e. don't ask, can't use -salvagewallet, etc.) to downgrade
10:31 < sipa> ?     
10:31 < wumpus> why would they think downgrading would work?
10:32 < achow101> wumpus: because it does.
10:32 < wumpus> say, your wallet is suddenly corrupted one day
10:32 < wumpus> why would you consider downgrading?
10:32 < wumpus> only if it happens on upgrade I suppose
10:32 < achow101> with this specific corruption problem, downgrading to non-hd wallet software (0.12.0- or -usehd=0) will ignore this problem
10:32 < sipa> if the corruption ransomly happens right after uograding, i can see why someone would think that
10:32 < sipa> *randomly
10:32 < sipa> not ransomly
10:32 < wumpus> yes, if it happens right after upgrading, but then it's probably a version issue and not a random corruption in any case, would be extremely unlikely otherwise
10:33 < sipa> achow101: update wallet version number?
10:33 < BlueMatt> anyway, this is why i think we should give an error message
10:33 < achow101> sipa: bleh
10:33 < sipa> oh, we can't because the optional stuff uses it
10:33 < sipa> we really need wallet features grr
10:33 < BlueMatt> the crash is better than continuing silently, but if it makes people downgrade, we should instead tell them whats up
10:34 < achow101> IIRC salvagewallet didn't fix their problems either, so that will need to be updated
10:34 < BlueMatt> sipa: arent you the one who added wallet versioning?
10:34 < BlueMatt> achow101: yes, lets do that :)
10:35 < wumpus> wallet versioning is used for *optional* features?
10:35 < BlueMatt> oh, no
10:35 < achow101> I think it would be fine to write some random default key right? it isn't used for anything
10:35 < achow101> wumpus: yes, hd and hd chain split or optional, but they have version numbers
10:35 < BlueMatt> achow101: its used for pay-to-IP :p
10:36 < BlueMatt> (I think)
10:36 < sipa> BlueMatt: yes, and now we can't use it
10:36 < sipa> because the version number is used for optional features
10:36 < sipa> like HD and split HD
10:37 < sipa> those should have been a separate record, rather the version number
10:37 < wumpus> would have been better to use a new key to indicate those (as well as bump the version number, but not in itself)
10:37 < wumpus> well teh version should be bumped too to make it incompatible, but yeah
10:38 < sipa> right
10:39 -!- tyngdekraften [~tyngdekra@2a01:e34:eeef:f5f0:1dc2:526e:f459:8b8b] has joined #bitcoin-core-dev     
10:39 < BlueMatt> Make Bitcoin Great Again: Bring back Pay-2-IP
10:40 < sipa> BlueMatt: i tried :(
10:40 < wumpus> with .onion addresses or some other way to authenticate an IP at the transport layer (such as ipsec) it even could work securely!
10:41 < BlueMatt> wumpus: I'm currently working on tcpcrypt for linux patchset :p
10:41 < wumpus> BlueMatt: I have no idea what that is
10:41 < BlueMatt> wumpus: tcp-crypt :p
10:42 < sipa> en..crypt...TCP?
10:42 < Lightsword> http://www.tcpcrypt.org/
10:42 < BlueMatt> yes, that
10:43 < sipa> ah, nice
10:44 -!- tyngdekraften [~tyngdekra@2a01:e34:eeef:f5f0:1dc2:526e:f459:8b8b] has quit [Quit: tyngdekraften]     
10:45 < wumpus> "and your network connections will continue to work even if the remote end does not support Tcpcrypt, in which case connections will gracefully fall back to standard clear-text TCP" - that screams downgrade attack :)
10:46 < achow101> oh damn, now default key can't be removed :(
10:46 < sipa> wumpus: read on
10:46 < sipa> achow101: ?
10:46 < sipa> wumpus: it only protects against passive attacks
10:46 < achow101> sipa: to check if default key is valid, it needs to be in CWallet
10:46 < sipa> achow101: ?
10:46 -!- Chris_Stewart_5 [~chris@gateway/vpn/privateinternetaccess/chrisstewart5/x-62865615] has joined #bitcoin-core-dev     
10:46 < BlueMatt> wumpus: yes, if the application layer does nto support it it only protects against passive mitm attacks, if the application layer supports it you can disconnect if tcpcrypt failed (but dont do that, cause middleware sucks)
10:47 < achow101> I need some way to get default key out of database into a checker, which also checks if default key exists
10:47 < sipa> achow101: you can just check when loading the wallet
10:47 < achow101> the only vehicle for that right now is cwallet I think
10:48 < sipa> i'm sure there is or can be a LoadDefaultKey function on CWallet, called from walletdb
10:48 < sipa> switch that to do a check, but not store
10:48 < achow101> I guess I could also just add more parameters too
10:49 -!- Mirobit [25c9f1fd@gateway/web/freenode/ip.37.201.241.253] has quit [Quit: Page closed]     
11:10 -!- miknotauro [~miknotaur@S0106a84e3fc27d33.vc.shawcable.net] has joined #bitcoin-core-dev     
11:14 < earlz> I'm trying to get gitian working with 0.14.0. Has anyone seen an error like this? lxcbr0: ERROR while getting interface flags: No such device
11:14 < earlz> SIOCSIFADDR: No such device
11:14 < earlz> just following the gitian-building.md document exactly and getting that error
11:18 < earlz> ifconfig
11:18 < earlz> er, woops
11:19 < earlz> do I need a kernel flag or something? It's failing at the sudo ifconfig lxcbr0 up 10.0.2.2 bit
11:20 < sipa> do you have the lxc kernel module loaded?
11:21 < earlz> I pretty much just followed the gitian build instructions and I don't see anywhere in them that the kernel module was explicitly installed unless the lxc package does that
11:24 < wumpus> looks like you don't have a lxcbr0 interface for bridging between the lxc and host
11:25 < wumpus> not sure how and where that gets created though
11:25 < earlz> yea, I'm not understanding where that gets created in the guide. Last time I setup gitian it was for Bitcoin 0.9 so not sure what all has changed since then
11:25 -!- Mattie161 [~Matt@host86-143-110-130.range86-143.btcentralplus.com] has joined #bitcoin-core-dev     
11:27 < earlz> maybe related: https://github.com/lxc/lxc/issues/1218
11:28 < earlz> LXC seems to be broken in some subtle way every time I try to do a new setup of gitian 
11:28 < sipa> wumpus, BlueMatt: i think we really need to fix the versioning issues
11:28 < BlueMatt> wallet versioning?
11:28 < sipa> that would make it so much easier (you could say, past wallet version X, no default key is needed anymore)
11:29 < sipa> i want to move hd to a separate record
11:29 < sipa> rather than store it in the version number
11:29  * BlueMatt is actually confused why
11:29 < sipa> because we can't upgrade the wallet format, ever, now
11:29 < sipa> in a backward incompatible way
11:29 < BlueMatt> version number = backwards-incompatible change (ie hd), new-key is backwards-compatible extra features
11:29 < sipa> without turning the wallet into an hd wallet
11:30 < sipa> yes, that's what it's supposed to be
11:30 < sipa> but HD is a version number
11:30 < BlueMatt> so?     
11:30 < sipa> we can't increment the version number to indicate something incompatible now
11:30 < BlueMatt> what do you want to add that needs to upgrade without being hd?
11:30 < sipa> anything
11:30 < BlueMatt> we should implement hd upgrade
11:30 < BlueMatt> thats something we very much need to do anyway
11:30 < sipa> what does that mean?
11:31 < BlueMatt> taking a non-hd wallet and adding an hd key
11:31 < sipa> yes, ok, that too
11:31 < sipa> but do you want to force everyone to have an hd wallet?
11:31 < BlueMatt> (probably also need an hd-key-rotate option, but thats separate and I think not related to hd)
11:31 < BlueMatt> ^     
11:32 < BlueMatt> I'm happy to force everyone into an hd wallet if we have an hd-master-key-rotate option
11:32 < sipa> yeah, ok...
11:32 < sipa> but those are all bigger features
11:32 < BlueMatt> well i think all of those are relatively limited code changes
11:32 < BlueMatt> and at least hd-master-key-rotate can happen with no format change, I think
11:32 < sipa> i am talking about 0.15
11:32 < BlueMatt> even if "big" features
11:33 < BlueMatt> do we need to add anything else? why rush the no vchdefaultkey thing?
11:33 < sipa> otherwise we'll complicate things further for ourselves
11:33 < sipa> to add a compatibility layer for split hd
11:33 < BlueMatt> ok, I'm missing something...need context
11:33 < sipa> no, ignore the vchdefaultkey thing
11:34 < sipa> hd and hdsplit, i think, are optional features - things that people can choose not to use
11:34 < BlueMatt> now, yes, but i have no problem with them not being optional in the future
11:34 < sipa> as they break existing wallet's expectation
11:35 < sipa> hmm, ok
11:36 < BlueMatt> not being optional in our traditional -walletupgrade sense, that is
11:36 < BlueMatt> i mean I'm happy for someone to disagree, I just dont see much downside to it (as long as we have an hd-master-key-rotate option and good documentation on it)
11:36 < sipa> i think i agree
11:37 < sipa> except i'm not sure we'll have that feature implemented by the time we need it
11:37 < BlueMatt> well hd-master-key-rotate is ~trivial with today's format
11:37 < sipa> okay
11:37 < BlueMatt> hd-upgrade may be slightly less so, I havent thought about it
11:37 < BlueMatt> but I expect it to be
11:37 < sipa> in that case, i guess it makes sense that those things are in fact in the version number
11:39 < sipa> i just wasn't seeing hd as a 'next version', and more as an optional but recommended feature
11:40 < BlueMatt> i mean its possible i did /because/ of our versioning scheme, but it is simpler to see it as such and there seem to be relatively limited downsides for it, given some code that doesnt exist yet :p
11:53 < wumpus> I'd say using hdsplit isn't really optional, given that you're using hd
11:54 < wumpus> hdsplit is a pure improvement on hd
11:57 < sipa> agree
11:57 < sipa> but do we support upgrading from hd to hdsplit?
11:57 < sipa> (right now)
11:58 < gmaxwell> ugh. how could we except via invalidating backups which people wouldn't expect...
11:58 < wumpus> no, we don't support that right now
11:59 < wumpus> hdsplit was sort-of rushed to make 0.15, so that at least new wallets would use it
11:59 < sipa> so, until we find a way to (forcefully) upgrade non-hd and hd to hdsplit, we should consider hdsplit an optional feature
11:59 < wumpus> but it's strictly superior to hd without hdsplit, so there should be no way to choose that for new wallets
11:59 < gmaxwell> I don't think forcefully upgrading is at all possible, because it will invalidate backups.
12:00 < wumpus> it wouldn't need to be 'forceful' upgrading, just a *way*
12:00 < wumpus> and in any case we don't ever automatically upgrade wallets 
12:00 < wumpus> (because of backwards compatibility)
12:00 < gmaxwell> switch to using segwit, and that will upgrade you. :)
12:00 < wumpus> #startmeeting
12:00 < lightningbot> Meeting started Thu Aug  3 19:00:54 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:00 < lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
12:01 < achow101> hi     
12:01 < jonasschnelli> hi     
12:01 < sipa> hi     
12:01 < wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101
12:01 < kanzure> hi.     
12:01 < jtimon> hi     
12:01 < cfields> hi     
12:01 < wumpus> 0.15.0rc1 is planned for the 6th (next sunday), so let's go over the open issues again
12:02 < wumpus> there's not much anymore
12:02 < wumpus> https://github.com/bitcoin/bitcoin/milestones/0.15.0
12:03 < paveljanik> Hi     
12:03 < wumpus> (and the scripted-diffs are option, and should be done at the last minute to not conflict with anything else)
12:03 < wumpus>  Keypool topup #10882  is the most complicated PR open still, but should be almost ready
12:03 < gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
12:04 < BlueMatt> https://github.com/bitcoin/bitcoin/issues/10977
12:04 < BlueMatt> could go
12:04 < BlueMatt> (makes test_bitcoin valgrind-better)
12:04 < BlueMatt> and is trivial
12:04 < jnewbery> yeah, I've been working on 10882 today. Should be able to push my commits in the next hour or two
12:04 < wumpus> I'm not really fishing for new things to add to 0.15
12:04 < gmaxwell> jnewbery made a suggestion to fix my outstanding concern in review.
12:05 < wumpus> but if there are things that could be merged without affecting anything else that's ok
12:05 < jtimon> after #10758, #10919 seems simple to review, it's only +14-13
12:05 < gribble> https://github.com/bitcoin/bitcoin/issues/10758 | Fix some chainstate-init-order bugs. by TheBlueMatt · Pull Request #10758 · bitcoin/bitcoin · GitHub
12:05 < gribble> https://github.com/bitcoin/bitcoin/issues/10919 | Fix more init bugs. by TheBlueMatt · Pull Request #10919 · bitcoin/bitcoin · GitHub
12:05 < sipa> it's also already marked 0.15
12:05 < cfields> i think there's a one-liner that could be used to fix the issue in 10977, if it's deemed too much of a change
12:05 < gmaxwell> BlueMatt: I'd like to see 10977 fixed! but darn I wish that patch was smaller and easier to review.
12:05 < wumpus> yes, just needs some ACKs
12:05 < BlueMatt> could be smaller, but is easy to review imo
12:05 < sdaftuar> there's one commit in #10919 that i'm hoping others can review/ack
12:05 < gribble> https://github.com/bitcoin/bitcoin/issues/10919 | Fix more init bugs. by TheBlueMatt · Pull Request #10919 · bitcoin/bitcoin · GitHub
12:07 < wumpus> which one?
12:07 < BlueMatt> the first, i believe
12:07 < sdaftuar> yep     
12:08 < wumpus> the threadgroup one? seems obviously sane to me, though it does mean things need to be interrupted too
12:08 -!- goatpig [56f75436@gateway/web/freenode/ip.86.247.84.54] has joined #bitcoin-core-dev     
12:08 < BlueMatt> well i think the point is that there is a comment there that notes we dont do it "because dragons"
12:09 < BlueMatt> i believe strongly that it is safe, and qt does it, but "dragons"
12:09 < wumpus> it is very bad practice not to wait for threads before exiting
12:09 < wumpus> yes, qt does it already, it's somewhat less scared of dragons it seems :)
12:09 < BlueMatt> isnt qt's logo a dragon or something?
12:10 < cfields> heh, think you're thinking of kde
12:10 < BlueMatt> oh, yea
12:10 < wumpus> (e.g. due to qt's handling of shutdown we also needed #10832)
12:10 < gribble> https://github.com/bitcoin/bitcoin/issues/10832 | init: Factor out AppInitLockDataDirectory and fix startup core dump issue by laanwj · Pull Request #10832 · bitcoin/bitcoin · GitHub
12:10 < wumpus> anyhow that commit looks good to me, I don't think there's any dragons left
12:11 < sdaftuar> sounds-good-to-me-ack
12:12 < wumpus> ok, wow, that seems to be all that is left between here and 0.15.0rc1
12:12 < BlueMatt> !     
12:12 < cfields> :)     
12:13 < morcos> wumpus: i had an assert crash this morning, i imagine it'll be a simple bug.. hopefully i'll have a PR this afternoon, just haven't had time to look at it yet
12:13 < wumpus> (there's another PR #10971 by cfields for fixing depends builds, but I don't think that needs disussion, it's a one-liner in the build system)
12:13 < gribble> https://github.com/bitcoin/bitcoin/issues/10971 | build: fix missing warnings and sse42 in depends builds by theuni · Pull Request #10971 · bitcoin/bitcoin · GitHub
12:13 < wumpus> morcos: ouch, can you open an issue to track it?
12:13 < cfields> yea, nothing major
12:14 < morcos> yes will open one or the other shortly
12:14 < wumpus> ok, thanks
12:15 < wumpus> do we need any updates to bips.md for 0.15?
12:15 < sipa> hmm, good question
12:15 < wumpus> (that's the item in the release process that still has a ? here)
12:15 < BlueMatt> is there a bip that recommends hd split?
12:15 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 276 seconds]     
12:15 < sipa> BlueMatt: bip32? :p
12:16 < wumpus> there's also "Update `src/chainparams.cpp` chainTxData with statistics about the transaction count and rate." left
12:16 < sipa> want me to do that?
12:16 < wumpus> and the BLOCK_CHAIN_SIZE, but that's straightforward
12:16 < wumpus> yes, if you know what's exactly to be done there that'd help :)
12:17 < sipa> sure
12:18 < wumpus> thanks
12:18 < sipa> short topic: bip173 unit tests issue
12:19 < wumpus> #topic bip173 unit tests issue
12:19 < jnewbery> There are a few more things for release notes
12:19 -!- clarkmoody_ [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has joined #bitcoin-core-dev     
12:19 < sipa> so, bip173 specifies how to translate address strings to witness version/program, and defers to bip141 for encoding that to scriptPubKeys
12:19 < sipa> however, the unit tests actually test the whole step from address to scriptPubKey
12:20 < sipa> turns out, incorrectly
12:20 < sipa> the tests and reference implementation (of the tests) was wrong, and every reimplementation copied it
12:21 < gmaxwell> The the error was that it confused hex and dec values.
12:21 < sipa> i've made a PR to update the BIP, and all reference implementations i could find, but this is kind of scary
12:21 < cfields> corner-cases wrong? or in all cases?
12:21 < wumpus> jnewbery: agreed, but release notes need to be finished before -final, not -rc1, so it's not a blocker
12:21 < sipa> cfields: it assumed OP_n was encoded as 0x80 + n, rather than 80 + n
12:21 < BlueMatt> sipa: so they generate garbage scripts?
12:22 < jnewbery> got it. Thanks wumpus
12:22 < sipa> BlueMatt: the tests, yes
12:22 < sipa> the code itself doesn't contain a conversion to scriptPubKey at all, only a conversion to witness version/program
12:22 < gmaxwell> cfields: It was wrong for witness program versions other than 0
12:22 < cfields> yikes
12:22 < wumpus> oops
12:22 < gmaxwell> so this could happily have been deployed and started causing problems when v1 was used.
12:22 < sipa> but the tests contain a version/program -> scriptPubKey converter in order to be able the test
12:22 -!- marcoagner [~user@177.41.203.37] has quit [Ping timeout: 276 seconds]     
12:23 < BlueMatt> well if it generated garbage scripts, not much that can be done but fix it
12:23 < BlueMatt> are you asking if we should like change the prefix now?
12:23 < sipa> no     
12:23 < sipa> just raising awareness
12:23 < BlueMatt> ok, cool
12:23 < sdaftuar> perhaps an email to the -dev list would also be good?
12:23 < gmaxwell> Also, it highlighes an implementation footgun, I suggested some warning text in the BIP itself.   One protection here is that the particular error in sipas' code would result in non-standard outputs.
12:23 < sipa> sdaftuar: yes
12:24 < gmaxwell> BlueMatt: I did make a suggestion that we consider changing it to break the checksum, but there doesn't appear to be reason to.
12:24 -!- clarkmoody [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has quit [Ping timeout: 276 seconds]     
12:24 < BlueMatt> ok     
12:24 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has quit [Quit: ZNC - http://znc.in]     
12:24 < morcos> just to be clear what we are talking about, we're not talking about anything merged into Core, but code referenced from the BIP
12:24 < BlueMatt> awareness raised!
12:24 < gmaxwell> Especially since if someone had slavishly reimplemented the error in the reference, they'd produce non-standard outputs.
12:24 < sipa> morcos: indeed.
12:24 < morcos> still, a bit scary
12:25 < sipa> (though i'd like to PR it to core soon - apparently last week it was suggested to do that in 0.15.1?)
12:25 < gmaxwell> Don't start a debate about the name of the version. :P
12:25 < sipa> haha
12:25 < sipa> (though i'd like to PR it to core soon - apparently last week it was suggested to do that in some soon next version)
12:25 -!- jonasschnelli [~jonasschn@bitcoinsrv.jonasschnelli.ch] has joined #bitcoin-core-dev     
12:25 < gmaxwell> It also suggests that BIP173 support's test plan should include testing it with other witness version numbers. :)
12:25 < sdaftuar> prs welcome :)
12:26 < gmaxwell> sipa: well fix the testing shortfalls I found in the C++ version please. :)
12:26 < wumpus> PRs to improve the tests are always welcome anyhow
12:26 < sipa> gmaxwell: of course
12:26 < sipa> anyway, end topic
12:26 < wumpus> ok, other topics?
12:26 -!- jonasschnelli [~jonasschn@bitcoinsrv.jonasschnelli.ch] has quit [Changing host]     
12:26 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has joined #bitcoin-core-dev     
12:28 < gmaxwell> uh      
12:28 < gmaxwell> yes.
12:28 < gmaxwell> So service bits and altcoins.
12:28 < wumpus> #topic service bits and altcoins
12:28 < BlueMatt> wait are altcoins using serice bits now?
12:28 < BlueMatt> oh, right 2x did
12:28 < gmaxwell> Bcash is using our port, p2pmagic, etc. They distinguish themselve with a blinking service bit.
12:28 < BlueMatt> what is wrong with people
12:28 < gmaxwell> (also 2x will do this too it seems)
12:29 < BlueMatt> gmaxwell: can someone open a pr to change this? or do they refuse to work properly?
12:29 < cfields> gmaxwell: i was under the impression that they were planning to change the magic soon
12:29 < gmaxwell> We mostly ban them when they send us transactions or headers.
12:29 < gmaxwell> cfields: not when I looked three days ago, maybe now.
12:29 < karelb> OK, maybe I will ask here. What format are the bitcoin .dat files in data/blocks/*.dat? is that leveldb? what is it exactly?
12:29 < jonasschnelli> karelb: meeting, not now
12:29 < gmaxwell> If so then the issue becomes moot, otherwise I was going to suggest we ban these bits on connect. The downside is we lose the bits basically forever.
12:29 < sipa> they are p2p network format
12:29 < karelb> ok sorry
12:29 < sipa> oops, yes, layer
12:30 < karelb> sorry going out
12:30  * karelb apologizes
12:30 < BlueMatt> gmaxwell: yes, first should be someone bludgening them to work properly
12:30 < BlueMatt> gmaxwell: before we start burning service bits
12:30 < gmaxwell> BlueMatt: people have been since before they started. Obviously I'll go monitor but I'm not super confident.
12:30 < sdaftuar> gmaxwell: why not just ban for eg the next 3 months or something?
12:30 < achow101> BlueMatt: gmaxwell IIRC they will be changing their magic and port
12:30 < sdaftuar> if we need to do anything at all
12:31 < achow101> dunno about 2x
12:31 < gmaxwell> One reason burning service bits may not be so bad is because we are due to replace the addr message for i2p and NG-HS support.
12:31 < BlueMatt> achow101: what about 2x?
12:31 < gmaxwell> So we could at that point just define a new service flagging mechenism.
12:31 < BlueMatt> gmaxwell: yea, does anyone have a spec for that?
12:31 < gmaxwell> Not yet.
12:31 < BlueMatt> k     
12:31 < gmaxwell> Well if they're finally going to change it, it becomes moot.. but the same issue arises with 2x.
12:32 < wumpus> how would a service bit help here?
12:32 < BlueMatt> well someone needs to bludgen the 2x folks to change it, otherwise we start banning for a few months
12:32 < wumpus> just ban everyone without NODE_SEGWIT? :p
12:32 < gmaxwell> wumpus: we still want to support non-upgraded nodes.
12:32 < wumpus> but they won't have any new version bit either
12:32 < wumpus> that was my point, not to suggest that seriously :)
12:33 < gmaxwell> wumpus: oh no, you misunderstand: ABC and 2x both set randomly generated service bits.
12:33 < cfields> gmaxwell: eh?
12:33 < BlueMatt> I think sdaftuar's suggestion is reasonable, assuming they refuse to do something sane
12:33 < gmaxwell> (which they've helpfully ignored the gigantic comment in the code that tells you to at least inform the list.)
12:33 < cfields> oh     
12:33 < gmaxwell> sdaftuar: I hadn't considered a time limited ban. Good point.
12:33 < wumpus> oh you mean banning everything that sets their version bit?
12:33 < wumpus> yes, that'd be doable
12:33 < BlueMatt> wumpus: yes, with a time limit
12:33 < gmaxwell> wumpus: well disconnecting, not banning. 
12:34 < BlueMatt> nah, ban for 3 months
12:34 < gmaxwell> Okay thanks, Time limit makes sense. duh. 
12:34 < wumpus> temporarily, yes
12:34 < morcos> wumpus: opened issue #10981, easy fix, but i'll let someone else do the PR as i'm not in the office for next week.  please tag with 0.15.
12:34 < gribble> https://github.com/bitcoin/bitcoin/issues/10981 | resendwallettransactions asserts if walletbroadcast=0 · Issue #10981 · bitcoin/bitcoin · GitHub
12:34 < gmaxwell> BlueMatt: banning creates problems when you run multiple things on one machine.
12:34 < BlueMatt> gmaxwell: meh
12:34 < wumpus> morcos: thanks, will tag later (not logged in now)
12:35 < BlueMatt> gmaxwell: they refused to do something that wasnt astoundingly broken, if it means their users get fucked, its not really my problem
12:35 < wumpus> (or if someone else can do it)
12:35 < morcos> BlueMatt: so chaincode ip will be banned.  nice.
12:35 < BlueMatt> morcos: -connect=altcoin.dns.seed
12:35 < BlueMatt> :)     
12:36 < wumpus> agree that banning goes too far, just not allow connections
12:36 < sipa> maye just disconnect?
12:36 < wumpus> there's no point in banning everything after that
12:36 < gmaxwell> what? no. matt, doing that will ban Bitcoin Core users when someone on the same IP ran crapware.
12:36 < jtimon> perhaps better for after the meeting, but I'm still not sure why #8498 wasn't suitable for 0.15 ...
12:36 < gribble> https://github.com/bitcoin/bitcoin/issues/8498 | Near-Bugfix: Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
12:36 < gmaxwell> To be clear this is important because these useless altcoin nodes waste connection slots, and are potentially at risk of gobbling up our initial headers fetch.
12:36 < BlueMatt> gmaxwell: ugh
12:37 < BlueMatt> fine, disconnect
12:37 < BlueMatt> at some point someone is gonna create some altcoin crapware that reconnects agressively, though
12:37 < wumpus> disconnect up until a certain date
12:37 < BlueMatt> some spv forks probably will
12:37 < wumpus> what would be that point?
12:37 < BlueMatt> because crapware
12:37 < wumpus> it would disconnect immediately after the version message
12:37 < gmaxwell> Also, looks like ABC has some kind of deadlocking bug, because I see a few of them just going unresponsive to anything but pings, which delays them getting banned for being on the wrong network.
12:37 < morcos> +1 disconnect up to certain date, but i think it should be 12 mos and not 3
12:37 < BlueMatt> do not make assumptions about crapware working in a sane way
12:37 < wumpus> banning would ban 1 message sooner
12:37 < morcos> no reason we'll need that next service bit right away
12:37 < sdaftuar> morcos: sure, that sounds fine
12:38 < wumpus> s/ban/disconnect
12:38 < BlueMatt> morcos: seems reasonable
12:38 < gmaxwell> ack on disconnect based on service bits for 12months or similar.
12:38 < wumpus> unless we start adding banned nodes to the local firewall, there's no serious difference between disconnecting on connect or after the version message
12:38 < gmaxwell> though in general, one of these clowns is going to squat service buts we're in the process of trying to use. :( I have no suggestion on dealing with that.
12:39 -!- CubicEarth [~cubiceart@c-73-181-185-197.hsd1.wa.comcast.net] has joined #bitcoin-core-dev     
12:39 < morcos> but i think it'd be worth a quick email/github message to jgarzik to check that they aren't imminently changing their plan
12:39 < BlueMatt> lets deal with that when we get there
12:39 < gmaxwell> s/buts/bits/
12:39 < wumpus> well if they change the magic and port we can stop worrying about the service bits they claim
12:39 < BlueMatt> morcos: yes, as I stated previously we should tell these guys to change network magic *first*
12:39 < wumpus> also we could at that point check for NODE_SEGWIT + our service bit
12:39 < morcos> yes but what if they change to a different service bit
12:39 < gmaxwell> morcos: there is some PR where people have been arguing for ages, about this, so I'm doubtful but sure.
12:39 < morcos> might as well ask first and tell him what we're planning on doing
12:39 < wumpus> change to a different version bit? what would that accomplish?
12:40 < morcos> whooo knows?!!
12:40 < gmaxwell> At the end we're doing them a favor, there are a lot more bitcoin nodes than random altcoin nodes, so these incorrect connections tend to cause them a lot more problems than us.
12:40 < BlueMatt> yup     
12:40 < BlueMatt> ok, probalem solved
12:40 < BlueMatt> who wants to go tell them that we're gonna disconnect them?
12:40 < wumpus> if avoiding detection is the point, they could better stop setting their version bti at that point is better than randomly cycling it according to moon phases
12:41 < gmaxwell> BlueMatt: perhaps we should just open the disconnect PR and ping them to comment on it?
12:41 < wumpus> bleh
12:41 < BlueMatt> gmaxwell: wfm, but seems like someone should open an issue
12:41 < BlueMatt> I vote morcos does it
12:41 < gmaxwell> throw him to the wolves... enh? what did he do so wrong?
12:41 < BlueMatt> actually, it was sdaftuar's idea, he can do it
12:41 < gmaxwell> throw him to the wolves... enh? what did he do so wrong?
12:42 < wumpus> seems we'd rather not invite certain discussions to our github but eh
12:42 < BlueMatt> gmaxwell: fine, I'll deal with it
12:42 < gmaxwell> BlueMatt: good, I know what you did so wrong. :P
12:43 < jtimon> but if the bits are selected randomly, how does burning them help?
12:43 < sipa> jtimon: s/randomly/arbitrarily/
12:43 < wumpus> they aren't selected randomly, they're not doing service bit hopping or something like that
12:43 < sipa> they're not different every time
12:43 < sipa> they just arbitrarily picked one
12:43 < jtimon> sipa: I see, thanks
12:43 < gmaxwell> jtimon: I don't follow your question. The altcoin efforts have selected randomly but hardcoded the result or their fair dice roll. :)
12:44 < morcos> +1 sdaftuar doing it...  i'm trying to pack
12:44 < jtimon> gmaxwell: yeah, got it
12:44 < gmaxwell> and just failed to follow the giant comment in the code to make a public announcement about it even.
12:44 < wumpus> e.g. they have monkeys throw darts to select one when they need it, not every connection
12:44 < morcos> oh nm, or bluematt
12:45 < BlueMatt> I think morcos is clearly just trying to throw anyone else under the bus, sounds like he should do it, then :p
12:45 < BlueMatt> anyway, next topic?
12:45 < gmaxwell> ;;action bluematt goes under the bus
12:45  * gribble bluematt goes under the bus
12:46 < gmaxwell> see, the robot overlords agree
12:46 < BlueMatt> ;;action goes under the bus
12:46  * gribble goes under the bus
12:46 < achow101> we can't/shouldn't ban 2x peers until they fork
12:46 < BlueMatt> achow101: yes we should
12:46 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has quit [Read error: Connection reset by peer]     
12:46 < wumpus> I think this was mainly about BCC which already forked
12:46 < achow101> BlueMatt: why? we won't be giving them invalid stuff until they fork, and vice versa
12:47 < gmaxwell> hash that out outside of the meeting plz.
12:47 < wumpus> achow101: on the other hand, adding logic to the code to check whether they've forked would complicate things more than just disconnecting on a service bit
12:47 < wumpus> yeah
12:47 < BlueMatt> next topic?
12:47 < gmaxwell> But in general if someone is going to make broken software, we can only go so far to accomidate it.
12:47 < wumpus> we've run out of topics
12:48 < achow101> wumpus: we know when they activate. at block X start banning them
12:48 < gmaxwell> I doubt we do.
12:48 < jtimon> perhaps better for after the meeting, but I'm still not sure why #8498 wasn't suitable for 0.15 ...
12:48 < BlueMatt> achow101: I'm not jumping through hoops to make sure altcoins stay in consensus until *right before* they fork...
12:48 < gribble> https://github.com/bitcoin/bitcoin/issues/8498 | Near-Bugfix: Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
12:48 < wumpus> #endmeeting
12:48 < lightningbot> Meeting ended Thu Aug  3 19:48:33 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
12:48 < lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-08-03-19.00.html
12:48 < lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-08-03-19.00.txt
12:48 < lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-08-03-19.00.log.html
12:48 < gmaxwell> since they still don't have a correct published specification and keep changing things.
12:48 < achow101> gmaxwell: they have stayed on using 12960 blocks after segwit activation
12:50 < gmaxwell> achow101: and what happens if they pull it up to 0 blocks, with a weeks notice? They're clearly being adversarial and trying to harm things, otherwise they'd change port/magic. 
12:50 < BlueMatt> achow101: and what if they change it next week? No point, if they want to make sure they stay in consensus, they can run bridge nodes, its not hard
12:51 < achow101> we risk partitioning the network since there are miners that use btc1
12:52 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has joined #bitcoin-core-dev     
12:52 < BlueMatt> achow101: that can be remedied otherwise
12:52 < BlueMatt> (and will be)
12:52 < BlueMatt> and, mostly, hopefully, they just change network magic
12:52 < BlueMatt> and then they can do whatever they want
12:53 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has quit [Quit: Leaving]     
12:54 < bitcoin-git> [bitcoin] TheBlueMatt opened pull request #10982: Disconnect network service bits 6 and 8 until Aug 1, 2018 (master...2017-08-bad-service-bits) https://github.com/bitcoin/bitcoin/pull/10982
12:57 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev     
12:59 < jnewbery> gmaxwell BlueMatt new commits in #10882 ready for review
12:59 < gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
13:02 < BlueMatt> thanks
13:08 -!- Cobra-Bitcoin [8a44f8f5@gateway/web/freenode/ip.138.68.248.245] has joined #bitcoin-core-dev     
13:09 < Cobra-Bitcoin> Any core dev around?
13:09 -!- Dizzle [~dizzle@108.171.182.16] has quit [Ping timeout: 255 seconds]     
13:09 < sipa> many
13:12 < Cobra-Bitcoin> Anyone know what btcdrak means when he says "future releases will be linked" here https://github.com/bitcoin-core/bitcoincore.org/issues/33?
13:12 < Cobra-Bitcoin> So the release notes will point users to bitcoincore.org binaries?
13:15 < sipa> BlueMatt: ^
13:17 < BlueMatt> Cobra-Bitcoin: that is the intention, yes, I mailed you about this a while back :). I think the goal is to start pointing people to bitcoincore.org, but also sicne y'all wanted to keep mirroring on bitcoin.org (and cause no point telling people to go somewhere else if they're already used to bitcoin.org), they'd be in both places
13:18 < BlueMatt> ie cause security isnt helped by making people make hops
13:18 < BlueMatt> Cobra-Bitcoin: see-also #10651
13:18 < gribble> https://github.com/bitcoin/bitcoin/issues/10651 | Verify binaries from bitcoincore.org and bitcoin.org by TheBlueMatt · Pull Request #10651 · bitcoin/bitcoin · GitHub
13:23 < Cobra-Bitcoin> But is bitcoin.org not doing enough to already distribute the binaries effectively?
13:23 < Cobra-Bitcoin> We have no choice but to mirror, since we have a lot of pages structured around the binaries and Core
13:25 < achow101> wumpus: BlueMatt so what do we do about salvagewallet and this default key thing?
13:25 < achow101> write a new one?
13:27 -!- snq [~random@185.32.222.12] has quit [Remote host closed the connection]     
13:27 -!- snq [~random@185.32.222.12] has joined #bitcoin-core-dev     
13:27 < BlueMatt> Cobra-Bitcoin: see pm
13:27 < BlueMatt> achow101: I havent looked at salvage, I mean certainly making it return an error instead of throwing like it does is not too hard
13:28 < wumpus> Cobra-Bitcoin: hosting on both bitcoin.org and bitconcore.org is perfectly acceptable, preferable even IMO
13:28 < achow101> BlueMatt: I have it return a db corrupt error now (local change, not yet pushed)
13:28 < wumpus> Cobra-Bitcoin: more (trusted) places to get the binaries from gives some redundancy
13:28 < BlueMatt> plus what wumpus said
13:29 < achow101> but my concern is that people will downgrade because downgrading works. there's no way to prevent downgrade is to use a new wallet version, but that's a mess that I don't want to deal with right now
13:30 < BlueMatt> achow101: well a reasonable error message should help, then
13:30 < BlueMatt> I dont care about preventing users from doing X or Y, but giving them an error message informing them that their hardware appears to have silently corrupted their wallet is appropriate
13:30 < BlueMatt> ideally salvagewallet would handle this case (does it not, my understanding was it should?)
13:31 < achow101> currently it says "Error loading wallet.dat: Wallet corrupted" with a more specific "invalid default key" message in the debug.log
13:31 < BlueMatt> thats fine?
13:31 < achow101> salvagewallet doesn't handle uncorrupting keys, it only pulls data that it can find being valid and passes it through
13:31 < achow101> so any corrupted keys will get passed through to a salvaged wallet
13:32 < achow101> (not like you could uncorrupt a key anyways)
13:32 < wumpus> is there even such a thing as 'uncorrupting keys'?
13:32 -!- Cobra-Bitcoin [8a44f8f5@gateway/web/freenode/ip.138.68.248.245] has quit [Quit: Page closed]     
13:32 < wumpus> it's not like we encode in some redundant way that allows recovery
13:32 < wumpus> we could do that, but we don't
13:32 < achow101> wumpus: well you could easily uncorrupt a default key by just writing a generic valid key like the generator
13:32 < wumpus> you could erase/ignore corrupted keys, though that's somewhat scary...
13:33 < BlueMatt> wumpus: isnt salvagewallet always scary?
13:33 < wumpus> (but not much different from what bdb salvage already does)
13:33 < wumpus> yes, sure, but this isn't black and white
13:33 < BlueMatt> true
13:33 < wumpus> there are certainly ways to make it even scarier :)
13:33 < sipa> so there is an added point here: any key that got corrupted ever probably resulted in the wallet just failing, and the user starting over or finding other solution
13:33 < sipa> but if it was the default key that was corrupted, it likely just meant that the wallet replenished the keypool at every startup, ever
13:34 < sipa> and everything kept working
13:34 < wumpus> yes     
13:34 < sipa> so there may be a survivalship bias, resulting in currently a higher than expected number of people with a corrupted default key
13:34 < wumpus> there probably should be a way to salvage and ignore corrupt keys...
13:34 < sipa> yes.
13:35 < wumpus> (another salvage level!)
13:35 < sipa> unfortunately, not possible for encrypted keys
13:35 < sipa> at least not without passphrase
13:35 < wumpus> yeah...
13:35 < wumpus> repair for an encrypted wallet should ideally ask for the passphrase
13:35 < wumpus> it's the only way to recover most effectively
13:38 < achow101> it also seems like the two reported cases of this involve wallets that have corrupted keys, which salvagewallet doesn't help with
13:39 < achow101> it's just that the user doesn't notice those keys were corrupted until they try to spend
13:39 -!- davec [~davec@cpe-24-243-249-218.hot.res.rr.com] has quit [Ping timeout: 260 seconds]     
13:39 < achow101> then the wallet fails to decrypt
13:40 -!- davec [~davec@cpe-24-243-249-218.hot.res.rr.com] has joined #bitcoin-core-dev     
13:42 < wumpus> yes that's not good
13:44 < achow101> so i guess we could just leave this as a warning about wallet corruption and not do any salvaging which would exacerbate the problem?
13:45 -!- Guest95816 is now known as fizzwont     
13:45 -!- fizzwont [~irc3@192.169.7.153] has quit [Changing host]     
13:45 -!- fizzwont [~irc3@unaffiliated/fizzwont] has joined #bitcoin-core-dev     
13:47 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has quit [Read error: Connection reset by peer]     
13:57 -!- Dizzle [~dizzle@108.171.182.16] has joined #bitcoin-core-dev     
14:00 < wumpus> agreed, any 'salvaging' should be at user initiative
14:17 -!- jeep-ss [~chatzilla@78.31.64.4] has quit [Ping timeout: 276 seconds]     
14:26 -!- Giszmo1 [~leo@ppp-88-217-110-125.dynamic.mnet-online.de] has quit [Quit: Leaving.]     
14:26 -!- Giszmo [~leo@ppp-88-217-110-125.dynamic.mnet-online.de] has joined #bitcoin-core-dev     
14:32 -!- elias19r [~elias19r@179.234.188.139] has joined #bitcoin-core-dev     
14:33 -!- elias19r [~elias19r@179.234.188.139] has left #bitcoin-core-dev []     
14:38 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 276 seconds]     
14:42 -!- Guyver2 [~Guyver@guyver2.xs4all.nl] has quit [Quit: Going offline, see ya! (www.adiirc.com)]     
14:50 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev     
15:00 -!- spudowiar [~spudowiar@unaffiliated/saleemrashid] has quit [Quit: WeeChat 1.9]     
15:10 -!- Chris_Stewart_5 [~chris@gateway/vpn/privateinternetaccess/chrisstewart5/x-62865615] has quit [Ping timeout: 260 seconds]     
15:14 < earlz> Is there any easy way to make gitian skip building dependencies? I'm having some trouble with a modification I made breaking the gitian build for bitcoind itself, but the 2 hours spent compiling dependencies is killing me
15:16 -!- ekerstein [~ekerstein@rrcs-74-218-188-194.midsouth.biz.rr.com] has joined #bitcoin-core-dev     
15:19 -!- fizzwont [~irc3@unaffiliated/fizzwont] has quit [Ping timeout: 255 seconds]     
15:20 -!- fizzwont [~irc3@192.169.7.153] has joined #bitcoin-core-dev     
15:21 -!- fizzwont is now known as Guest17799     
15:21 < cfields> earlz: the dependencies are cached, they'll only be built once
15:21 -!- Guest17799 is now known as fizzwont     
15:21 -!- fizzwont [~irc3@192.169.7.153] has quit [Changing host]     
15:21 -!- fizzwont [~irc3@unaffiliated/fizzwont] has joined #bitcoin-core-dev     
15:25 -!- LampTreadStone07 [~androirc@2602:306:cf70:cb0:ec9b:4551:5240:ccc5] has joined #bitcoin-core-dev     
15:26 -!- Drabiv [3e50a908@gateway/web/cgi-irc/kiwiirc.com/ip.62.80.169.8] has quit [Ping timeout: 255 seconds]     
15:27 < earlz> cfields: that does not seem to be the case if gbuild fails
15:29 < earlz> is there some command line argument or something I'm missing to prevent it from saving the cache, cause right now I basiclaly do gbuild, it compiles all deps, then encounters failure, I'll apply a fix and commit/push it, then update the gbuild command for the new commit hash and I observe it compiles all deps again
15:30 < cfields> earlz: do a vanilla build, make sure it finishes, get the deps cached
15:30 < cfields> then rebuild with your changes
15:30 < earlz> oh I see, so it only caches if the build succeeds? 
15:30 < cfields> yes     
15:31 -!- Mattie161 [~Matt@host86-143-110-130.range86-143.btcentralplus.com] has quit []     
15:35 -!- Dizzle [~dizzle@108.171.182.16] has quit [Quit: Leaving...]     
15:48 < bitcoin-git> [bitcoin] TheBlueMatt opened pull request #10984: Allow 2 simultaneous (compact-)block downloads (master...2017-08-paralell-block-downloads) https://github.com/bitcoin/bitcoin/pull/10984
16:05 < BlueMatt> cfields: when you do the next round of cleanups on #10756, can you not move InitializeNode/FinalizeNode? I'm starting work on building on top and have patches that conflict :/
16:05 < gribble> https://github.com/bitcoin/bitcoin/issues/10756 | net processing: swap out signals for an interface class by theuni · Pull Request #10756 · bitcoin/bitcoin · GitHub
16:06 < cfields> BlueMatt: heh, i thought we agreed that was the next step?
16:06 < BlueMatt> cfields: nono, i mean dont move the code down in the file
16:06 < BlueMatt> not dont move
16:07 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 240 seconds]     
16:08 < cfields> BlueMatt: oh, I see what you mean
16:08 < cfields> BlueMatt: they have to move out of the anon namespace though :(
16:09 -!- d_t [~d_t@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has joined #bitcoin-core-dev     
16:09 -!- ekerstein [~ekerstein@rrcs-74-218-188-194.midsouth.biz.rr.com] has quit [Quit: ZZZzzz…]     
16:09 < BlueMatt> ohh, didnt realize they were in a namespace
16:09 < BlueMatt> damn, ok, fine
16:09 < BlueMatt> I'll just have to do some dirty rebase
16:10 < cfields> yea, that's the only reason i moved them
16:10 < cfields> could do something ugly like closing/reopening the namespace, but i'd rather not :)
16:19 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev     
16:23 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev     
16:23 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit]     
16:26 < bitcoin-git> [bitcoin] sipa opened pull request #10985: Add undocumented -forcecompactdb to force LevelDB compactions (master...20170803_forcecompactdb) https://github.com/bitcoin/bitcoin/pull/10985
16:39 -!- Yogaqueef [~textual@dsl-hkibng42-5673c3-32.dhcp.inet.fi] has quit [Quit: My iMac has gone to sleep. ZZZzzz…]     
16:40 < bitcoin-git> [bitcoin] sipa opened pull request #10986: Update chain transaction statistics (master...20170803_txstats) https://github.com/bitcoin/bitcoin/pull/10986
16:47 < gmaxwell> #10985: \0/
16:47 < gribble> https://github.com/bitcoin/bitcoin/issues/10985 | Add undocumented -forcecompactdb to force LevelDB compactions by sipa · Pull Request #10985 · bitcoin/bitcoin · GitHub
16:47 < sipa> gmaxwell: ^ untested
16:56 -!- marcoagner [~user@177.41.193.82] has joined #bitcoin-core-dev     
16:59 -!- praxeology [~praxeolog@unaffiliated/praxeology] has quit [Quit: Leaving.]     
16:59 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Textual IRC Client: www.textualapp.com]     
17:04 -!- treebeardd [~treebeard@50-248-210-131-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev     
17:04 -!- Chris_Stewart_5 [~chris@gateway/vpn/privateinternetaccess/chrisstewart5/x-62865615] has joined #bitcoin-core-dev     
17:07 -!- elkalamar [~elkalamar@84.126.69.179.dyn.user.ono.com] has quit [Ping timeout: 246 seconds]     
17:08 -!- vicenteH [~user@13.232.15.37.dynamic.jazztel.es] has quit [Ping timeout: 258 seconds]     
17:10 -!- CubicEarth [~cubiceart@c-73-181-185-197.hsd1.wa.comcast.net] has quit [Remote host closed the connection]     
17:13 -!- Chicago [shikaakwa@unaffiliated/chicago] has joined #bitcoin-core-dev     
17:14 -!- LampTreadStone07 [~androirc@2602:306:cf70:cb0:ec9b:4551:5240:ccc5] has quit [Ping timeout: 255 seconds]     
17:41 -!- cheese_ [~Cheeseo@unaffiliated/cheeseo] has quit [Ping timeout: 240 seconds]     
17:41 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-svwftktvnzfuevxw] has quit [Quit: Connection closed for inactivity]     
17:45 -!- treebeardd [~treebeard@50-248-210-131-static.hfc.comcastbusiness.net] has quit [Quit: Leaving...]     
17:46 -!- praxeology [~praxeolog@unaffiliated/praxeology] has joined #bitcoin-core-dev     
17:46 -!- treebeardd [~treebeard@50-248-210-131-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev     
18:00 -!- CubicEarth [~cubiceart@c-73-181-185-197.hsd1.wa.comcast.net] has joined #bitcoin-core-dev     
18:04 -!- jannes [~jannes@095-097-246-234.static.chello.nl] has quit [Quit: Leaving]     
18:04 -!- Murch [~murch@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.]     
18:07 -!- miknotauro [~miknotaur@S0106a84e3fc27d33.vc.shawcable.net] has quit [Ping timeout: 248 seconds]     
18:39 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has joined #bitcoin-core-dev     
18:39 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev     
18:39 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has quit [Client Quit]     
18:40 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has quit [Remote host closed the connection]     
18:40 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev     
18:40 -!- shangzhou [uid156782@gateway/web/irccloud.com/x-kujkqirarmpjxbhb] has joined #bitcoin-core-dev     
18:42 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 260 seconds]     
18:44 -!- Chris_Stewart_5 [~chris@gateway/vpn/privateinternetaccess/chrisstewart5/x-62865615] has quit [Ping timeout: 240 seconds]     
18:47 -!- LampTreadStone07 [~androirc@2602:306:cf70:cb0:ec9b:4551:5240:ccc5] has joined #bitcoin-core-dev     
18:58 -!- Chris_Stewart_5 [~chris@gateway/vpn/privateinternetaccess/chrisstewart5/x-62865615] has joined #bitcoin-core-dev     
19:00 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 255 seconds]     
19:01 -!- vyger [924a5e4e@gateway/web/freenode/ip.146.74.94.78] has joined #bitcoin-core-dev     
19:02 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev     
19:14 -!- http_GK1wmSU [~deep-book@129.232.221.173] has joined #bitcoin-core-dev     
19:14 -!- http_GK1wmSU [~deep-book@129.232.221.173] has left #bitcoin-core-dev []     
19:24 -!- vyger [924a5e4e@gateway/web/freenode/ip.146.74.94.78] has quit [Ping timeout: 260 seconds]     
19:30 -!- tunafizz [~tuna@c-71-207-56-72.hsd1.pa.comcast.net] has joined #bitcoin-core-dev     
19:38 -!- Chris_Stewart_5 [~chris@gateway/vpn/privateinternetaccess/chrisstewart5/x-62865615] has quit [Ping timeout: 240 seconds]     
19:39 -!- treebeardd [~treebeard@50-248-210-131-static.hfc.comcastbusiness.net] has quit [Quit: Leaving...]     
19:44 -!- somenick_29345 [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has joined #bitcoin-core-dev     
19:47 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has quit [Quit: WeeChat 1.7.1]     
19:47 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev     
19:48 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Client Quit]     
19:48 -!- clarkmoody_ [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has quit [Ping timeout: 255 seconds]     
19:48 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev     
20:00 -!- clarkmoody [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has joined #bitcoin-core-dev     
20:04 -!- somenick_29345 [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has quit [Ping timeout: 260 seconds]     
20:09 -!- kallewoof [~karl@67.205.138.199] has quit [Quit: leaving]     
20:17 -!- kallewoof [~karl@67.205.138.199] has joined #bitcoin-core-dev     
20:19 -!- LampTreadStone07 [~androirc@2602:306:cf70:cb0:ec9b:4551:5240:ccc5] has quit [Ping timeout: 255 seconds]     
20:26 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 276 seconds]     
20:30 -!- Austindoggie [~austindog@2601:647:ca80:1f82:fd06:4cbc:9cdf:75c0] has joined #bitcoin-core-dev     
20:32 -!- ekerstein [~ekerstein@rrcs-74-218-188-194.midsouth.biz.rr.com] has joined #bitcoin-core-dev     
20:32 -!- ekerstein [~ekerstein@rrcs-74-218-188-194.midsouth.biz.rr.com] has quit [Client Quit]     
20:39 -!- Giszmo1 [~leo@ppp-88-217-108-210.dynamic.mnet-online.de] has joined #bitcoin-core-dev     
20:41 -!- Giszmo [~leo@ppp-88-217-110-125.dynamic.mnet-online.de] has quit [Ping timeout: 255 seconds]     
20:43 -!- jtimon [~quassel@6.31.134.37.dynamic.jazztel.es] has quit [Remote host closed the connection]     
20:50 -!- shangzhou [uid156782@gateway/web/irccloud.com/x-kujkqirarmpjxbhb] has quit [Quit: Connection closed for inactivity]     
21:20 -!- clarkmoody [~clarkmood@47-218-249-135.bcstcmta04.res.dyn.suddenlink.net] has quit [Quit: Leaving]     
22:11 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 268 seconds]     
22:26 -!- cluelessperson [~cluelessp@unaffiliated/cluelessperson] has quit [Quit: Laters]     
22:50 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev     
23:12 < venzen> wumpus: https://www.reddit.com/r/Bitcoin/comments/6rijhe/bitcoin_core_binary_signing_keys_not_matching/
23:13 < venzen> wumpus: i assume those are different keys for different uses
23:18 -!- CubicEarth [~cubiceart@c-73-181-185-197.hsd1.wa.comcast.net] has quit []     
23:28 -!- dobak [~dobak@c237-242.icpnet.pl] has joined #bitcoin-core-dev     
23:41 -!- juscamarena_ [~justin@47.148.176.74] has quit [Ping timeout: 240 seconds]     
23:42 -!- juscamarena_ [~justin@47.148.176.74] has joined #bitcoin-core-dev     
23:45 -!- elkalamar [~elkalamar@84.126.69.179.dyn.user.ono.com] has joined #bitcoin-core-dev     
23:50 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection]