--- Day changed Thu Dec 01 2016 00:00 -!- aalex_ [~aalex@64.187.177.58] has quit [Ping timeout: 260 seconds] 00:00 < sipa> rebroad: again, please don't see a "go discuss this elsewhere" as "don't discuss this". people _are_ very willing to discuss these things and the reasons behind it. but not here. 00:01 < rebroad> sipa, thank you for that clarification. EOT 00:04 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has quit [Quit: Leaving.] 00:13 -!- jl2012 [uid133844@gateway/web/irccloud.com/x-arwzoxxktyootzux] has quit [Ping timeout: 244 seconds] 00:14 -!- jl2012 [uid133844@gateway/web/irccloud.com/x-sczzggonijjticjv] has joined #bitcoin-core-dev 00:22 -!- rebroad [~rebroad@119.42.105.208] has quit [Ping timeout: 258 seconds] 00:24 -!- LeMiner2 [~LeMiner@5ED1AFBF.cm-7-2c.dynamic.ziggo.nl] has joined #bitcoin-core-dev 00:25 -!- LeMiner [~LeMiner@unaffiliated/leminer] has quit [Ping timeout: 260 seconds] 00:25 -!- LeMiner2 is now known as LeMiner 00:27 -!- moli [~molly@unaffiliated/molly] has quit [Read error: Connection reset by peer] 00:28 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 00:28 -!- LeMiner2 [~LeMiner@5ED1AFBF.cm-7-2c.dynamic.ziggo.nl] has joined #bitcoin-core-dev 00:28 -!- Cory [~Cory@unaffiliated/cory] has quit [Ping timeout: 260 seconds] 00:30 -!- LeMiner [~LeMiner@5ED1AFBF.cm-7-2c.dynamic.ziggo.nl] has quit [Ping timeout: 260 seconds] 00:30 -!- LeMiner2 is now known as LeMiner 00:31 -!- Cory [~Cory@unaffiliated/cory] has joined #bitcoin-core-dev 00:36 < jonasschnelli> sipa: why would it break backwards compatibility? 00:36 < jonasschnelli> (remove of the default key) 00:36 < jonasschnelli> Because of fFirstRunRet? 00:36 < sipa> jonasschnelli: yup 00:36 < jonasschnelli> hmm.. 00:37 < sipa> specifically, not having a default key in a wallet file would cause it to write the current chainstate 00:37 < sipa> and thus potentially miss a rescan 00:37 < sipa> writing a dummy default key... could result in the old wallet giving it out as address 00:38 < luke-jr> what's the harm in having a dummy default key that's actually in the wallet? O.o 00:38 < jonasschnelli> I don't like the default key for two reasons... 00:38 < sipa> luke-jr: that's basically what we have now :D 00:38 < jonasschnelli> 1) seems to be unused/miused 00:38 < luke-jr> exactly; I miss context as to why it should change 00:38 < jonasschnelli> 2) I want to have a private-key free wallet mode 00:38 < luke-jr> jonasschnelli: just pretend it isn't there? 00:38 < sipa> luke-jr: technical debt 00:39 < jonasschnelli> We probably should have removed it with 0.13 hd wallets (not backward comp.) 00:47 -!- LeMiner [~LeMiner@5ED1AFBF.cm-7-2c.dynamic.ziggo.nl] has quit [Read error: Connection reset by peer] 00:48 -!- LeMiner [~LeMiner@unaffiliated/leminer] has joined #bitcoin-core-dev 00:50 -!- wasi [~wasi@gateway/tor-sasl/wasi] has quit [Ping timeout: 245 seconds] 00:51 -!- wasi [~wasi@gateway/tor-sasl/wasi] has joined #bitcoin-core-dev 01:02 -!- shesek [~shesek@bzq-84-110-178-187.cablep.bezeqint.net] has quit [Ping timeout: 252 seconds] 01:25 -!- Atomicat_ [~Atomicat@unaffiliated/atomicat] has joined #bitcoin-core-dev 01:25 -!- cysm [cysm@unaffiliated/paracyst] has quit [Ping timeout: 260 seconds] 01:26 -!- Atomicat [~Atomicat@unaffiliated/atomicat] has quit [Ping timeout: 260 seconds] 01:26 -!- Atomicat_ is now known as Atomicat 01:30 -!- laurentmt [~Thunderbi@80.215.138.96] has joined #bitcoin-core-dev 01:40 -!- laurentmt [~Thunderbi@80.215.138.96] has quit [Quit: laurentmt] 01:51 -!- Atomicat [~Atomicat@unaffiliated/atomicat] has quit [Ping timeout: 250 seconds] 01:52 -!- ratoder [~ratoder@static.111.19.201.138.clients.your-server.de] has quit [Ping timeout: 248 seconds] 01:57 -!- LeMiner [~LeMiner@unaffiliated/leminer] has quit [Read error: Connection reset by peer] 01:57 -!- LeMiner [~LeMiner@5ED1AFBF.cm-7-2c.dynamic.ziggo.nl] has joined #bitcoin-core-dev 01:57 -!- LeMiner [~LeMiner@5ED1AFBF.cm-7-2c.dynamic.ziggo.nl] has quit [Changing host] 01:57 -!- LeMiner [~LeMiner@unaffiliated/leminer] has joined #bitcoin-core-dev 02:18 -!- Atomicat [~Atomicat@unaffiliated/atomicat] has joined #bitcoin-core-dev 02:26 -!- jannes [~jannes@178.132.211.90] has joined #bitcoin-core-dev 02:31 -!- sdaftuar_ [~sdaftuar@static-100-38-11-146.nycmny.fios.verizon.net] has joined #bitcoin-core-dev 02:32 -!- phantomcircuit [~phantomci@192.241.205.97] has quit [Ping timeout: 244 seconds] 02:32 -!- ghtdak [~ghtdak@unaffiliated/ghtdak] has quit [Ping timeout: 252 seconds] 02:36 -!- phantomcircuit [~phantomci@192.241.205.97] has joined #bitcoin-core-dev 02:39 -!- Netsplit *.net <-> *.split quits: xiangfu, sdaftuar, juscamarena, Evel-Knievel, LeMiner, warren, TD-Linux 02:42 -!- Evel-Knievel [~Evel-Knie@d5152f744.static.telenet.be] has joined #bitcoin-core-dev 02:42 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 250 seconds] 02:42 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev 02:43 -!- LeMiner [~LeMiner@5ED1AFBF.cm-7-2c.dynamic.ziggo.nl] has joined #bitcoin-core-dev 02:43 -!- 7GHAAUHIH [~Evel-Knie@d5152f744.static.telenet.be] has joined #bitcoin-core-dev 02:43 -!- TD-Linux [~Thomas@about/essy/indecisive/TD-Linux] has joined #bitcoin-core-dev 02:43 -!- juscamarena [~justin@47.148.176.74] has joined #bitcoin-core-dev 02:43 -!- xiangfu [~xiangfu@223.223.187.142] has joined #bitcoin-core-dev 02:43 -!- warren [~warren@fedora/wombat/warren] has joined #bitcoin-core-dev 02:44 -!- 7GHAAUHIH [~Evel-Knie@d5152f744.static.telenet.be] has quit [Ping timeout: 260 seconds] 02:46 < bitcoin-git> [bitcoin] laanwj opened pull request #9255: qt: layoutAboutToChange signal is called layoutAboutToBeChanged (master...2016_12_qt_signals_warnings) https://github.com/bitcoin/bitcoin/pull/9255 02:47 -!- ghtdak [~ghtdak@unaffiliated/ghtdak] has joined #bitcoin-core-dev 02:48 < bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/3bf06e9bac57...c79e52ad304a 02:48 < bitcoin-git> bitcoin/master 507145d Matt Corallo: Fix race when accessing std::locale::classic()... 02:48 < bitcoin-git> bitcoin/master 8b22efb Matt Corallo: Make fStartedNewLine an std::atomic_bool... 02:48 < bitcoin-git> bitcoin/master c79e52a Wladimir J. van der Laan: Merge #9230: Fix some benign races in timestamp logging... 02:48 < bitcoin-git> [bitcoin] laanwj closed pull request #9230: Fix some benign races in timestamp logging (master...2016-11-loglocks) https://github.com/bitcoin/bitcoin/pull/9230 02:56 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 03:02 -!- ghtdak [~ghtdak@unaffiliated/ghtdak] has quit [Ping timeout: 250 seconds] 03:04 -!- ghtdak [~ghtdak@unaffiliated/ghtdak] has joined #bitcoin-core-dev 03:20 -!- protomar [~protomar@91.214.169.69] has quit [Quit: Leaving] 03:20 -!- ThomasV [~ThomasV@unaffiliated/thomasv] has joined #bitcoin-core-dev 03:38 -!- aalex_ [~aalex@64.187.177.58] has joined #bitcoin-core-dev 03:45 -!- aalex_ [~aalex@64.187.177.58] has quit [Ping timeout: 265 seconds] 03:45 -!- ThomasV [~ThomasV@unaffiliated/thomasv] has quit [Ping timeout: 256 seconds] 03:50 -!- waxwing [~waxwing@62.205.214.125] has quit [Ping timeout: 244 seconds] 03:58 -!- waxwing [~waxwing@62.205.214.125] has joined #bitcoin-core-dev 04:16 < bitcoin-git> [bitcoin] jonasschnelli opened pull request #9256: Fix more CWallet/CWalletDB layer violations (master...2016/12/ref_walletdb) https://github.com/bitcoin/bitcoin/pull/9256 04:18 < jonasschnelli> https://github.com/bitcoin/bitcoin/pull/9143 04:32 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 258 seconds] 04:33 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev 04:56 -!- CubicEarth [~cubiceart@c-50-159-126-21.hsd1.wa.comcast.net] has quit [Remote host closed the connection] 05:22 -!- dermoth_ [~thomas@dsl-66-36-132-180.mtl.aei.ca] has joined #bitcoin-core-dev 05:23 -!- dermoth [~thomas@dsl-66-36-131-123.mtl.aei.ca] has quit [Disconnected by services] 05:23 -!- dermoth_ is now known as dermoth 05:28 -!- laurentmt [~Thunderbi@80.215.178.228] has joined #bitcoin-core-dev 05:30 -!- rafalcpp_ [~racalcppp@84-10-11-234.static.chello.pl] has quit [] 05:31 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 05:31 -!- rafalcpp [~racalcppp@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev 05:44 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Quit: WeeChat 0.4.2] 05:49 -!- crudel [crudel@gateway/shell/fnordserver.eu/x-xoaaqdfpbqesuyzd] has joined #bitcoin-core-dev 05:49 -!- laurentmt [~Thunderbi@80.215.178.228] has quit [Quit: laurentmt] 05:55 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 05:56 -!- CubicEarth [~cubiceart@c-50-159-126-21.hsd1.wa.comcast.net] has joined #bitcoin-core-dev 06:01 -!- CubicEarth [~cubiceart@c-50-159-126-21.hsd1.wa.comcast.net] has quit [Ping timeout: 260 seconds] 06:18 -!- Guyver2__ [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 06:21 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Ping timeout: 264 seconds] 06:21 -!- Guyver2__ is now known as Guyver2 06:26 -!- aalex_ [~aalex@64.187.177.58] has joined #bitcoin-core-dev 06:29 -!- ThomasV [~ThomasV@unaffiliated/thomasv] has joined #bitcoin-core-dev 06:32 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-hfkljsoftdntytqx] has quit [Ping timeout: 260 seconds] 06:34 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-viadtpqugiemcugz] has joined #bitcoin-core-dev 07:12 < bitcoin-git> [bitcoin] sdaftuar opened pull request #9257: [qa] Dump debug logs on travis failures. (master...travis-debug-logs) https://github.com/bitcoin/bitcoin/pull/9257 07:12 -!- molz [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 07:14 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 248 seconds] 07:19 -!- To7 [~theo@cpe-158-222-222-232.nyc.res.rr.com] has quit [Quit: Whatever] 07:32 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has joined #bitcoin-core-dev 08:12 -!- laurentmt [~Thunderbi@80.215.178.228] has joined #bitcoin-core-dev 08:13 -!- arubi [~ese168@gateway/tor-sasl/ese168] has quit [Remote host closed the connection] 08:32 -!- molz [~molly@unaffiliated/molly] has quit [Ping timeout: 240 seconds] 08:33 -!- sdaftuar_ is now known as sdaftuar 08:41 -!- aalex__ [~aalex@64.187.177.58] has joined #bitcoin-core-dev 08:44 -!- To7 [~theo@cpe-158-222-222-232.nyc.res.rr.com] has joined #bitcoin-core-dev 08:44 -!- aalex_ [~aalex@64.187.177.58] has quit [Ping timeout: 244 seconds] 09:08 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 09:35 -!- abpa [~abpa@96-82-80-25-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 09:45 -!- jtimon [~quassel@186.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 09:48 -!- ThomasV [~ThomasV@unaffiliated/thomasv] has quit [Ping timeout: 244 seconds] 10:10 -!- aalex_ [~aalex@64.187.177.58] has joined #bitcoin-core-dev 10:13 -!- arubi [~ese168@gateway/tor-sasl/ese168] has joined #bitcoin-core-dev 10:14 -!- aalex__ [~aalex@64.187.177.58] has quit [Ping timeout: 260 seconds] 10:34 -!- laurentmt [~Thunderbi@80.215.178.228] has quit [Quit: laurentmt] 10:37 -!- Anduck [~Anduck@unaffiliated/anduck] has quit [Ping timeout: 256 seconds] 10:39 -!- bitcoin070 [369924af@gateway/web/freenode/ip.54.153.36.175] has joined #bitcoin-core-dev 10:39 < bitcoin070> anyone else having nuclear problems with bitcoin core 0.13.1? 10:39 < sipa> elaborate? 10:40 < wumpus> nuclear problems, wow 10:40 < bitcoin070> unfortunately wumpus 10:40 < bitcoin070> yes 10:40 < bitcoin070> I will elaborate in a minute here 10:40 < sipa> what version were you using that worked, what system specifications, what used to work, what does not? 10:40 < bitcoin070> sipa/wumpus 10:40 < bitcoin070> some very unfortunate behavior on m3.mediums on Amazon EC2 10:41 < wumpus> it's not recommended to use it on nuclear reactors 10:41 < bitcoin070> I have three separate instances and four different occasions now .... 10:41 < bitcoin070> extremely bad 10:41 < bitcoin070> on phone call but will elaborate fully in a moment 10:41 < sipa> can you please explain what is going on? 10:41 < wumpus> what is so nuclear about that? 10:42 < bitcoin070> essentially some form of mempool corruption where the bitcoin-cli getbalance / getinfo reads 0 10:42 < bitcoin070> but bitcoin-cli getbalance "" reads the true balance 10:42 < bitcoin070> RPC calls to sendtoaddress return insufficient balance 10:42 < bitcoin070> also .. the mega bad fuck up is 10:42 < sipa> what version were you using before? 10:42 < bitcoin070> as the behavior starts to unravel 10:43 < bitcoin070> bitcoind will return TXID that don't reflect on the network 10:43 < bitcoin070> and then when rebooting the node 10:43 < bitcoin070> it can result in some very unexpected behavior 10:43 < bitcoin070> 0.12 before 10:43 < bitcoin070> the only thing that is off on these boxes was the time -- 7 minutes or so 10:43 < bitcoin070> as the VPS aren't running ntpdate 10:43 < sipa> that's unlikely to be a problem 10:44 < bitcoin070> agreed 10:44 < bitcoin070> bitcoin-cli getinfo { "version": 130100, "protocolversion": 70014, "walletversion": 130000, "balance": 0.00107642, "blocks": 441461, "timeoffset": -432, "connections": 29, "proxy": "", "difficulty": 281800917193.1958, "testnet": false, "keypoololdest": 1480563469, "keypoolsize": 100, "paytxfee": 0.00000000, "relayfee": 0.00001000, "errors": "" } 10:44 < bitcoin070> bitcoin-cli getbalance "" 34.71912966 10:44 < bitcoin070> this is the 4th time this has happened 10:44 < bitcoin070> when it happened previously, I modified bitcoin.conf on these two boxes to limit mempool, outbound connections, etc 10:44 -!- droark [~droark@c-24-22-123-27.hsd1.or.comcast.net] has quit [Quit: Later.] 10:45 < bitcoin070> mempool=150, dbcache=70 10:45 -!- lightningbot [~supybot@2400:8901::f03c:91ff:febb:bbc1] has quit [Remote host closed the connection] 10:45 < bitcoin070> fwiw I never once had a problem with bitcoind before 10:45 -!- lightningbot [~supybot@2400:8901::f03c:91ff:febb:bbc1] has joined #bitcoin-core-dev 10:45 < sipa> my guess is that your funds are tied up in unconfirmed change 10:45 < sipa> which is being kicked out of the mempool 10:45 < bitcoin070> but these transactions are all recent 10:46 < bitcoin070> why would they be getting kicked out of the mempool, should mempool not prioritize my own TX? 10:46 < wumpus> do you have spendzeroconfchange set to 0 perhaps? 10:46 < bitcoin070> nope 10:46 < bitcoin070> sorry to sound brash guys but i am a power user 10:46 < bitcoin070> been running nodes for 3+ years 10:46 < bitcoin070> well-versed in bitcoin.conf 10:46 < sipa> maybe too long chains of unconfirmed transactions? 10:46 < sipa> how many unconfirmed outgoing transactions do you have? 10:47 < bitcoin070> I think that's it 10:47 < bitcoin070> sipa there's no good way to tell as the node becomes completely nonsensical 10:47 < bitcoin070> listunspent 0 doesn't even show the outputs 10:47 < bitcoin070> the only thing I can do is track it back in a block explorer 10:48 < sipa> listtransactions should list them 10:48 < wumpus> if it's an unconfirmed change problem then it should resolve itself after the transactions are confirmed 10:48 -!- droark [~droark@c-24-22-123-27.hsd1.or.comcast.net] has joined #bitcoin-core-dev 10:48 < morcos> what does getunconfirmedbalance say 10:49 < bitcoin070> so 10:49 < bitcoin070> there is a large large amount of unconfirmed spending unconfirmed 10:49 < bitcoin070> however 10:49 < bitcoin070> annoyingly, for instance 10:49 < bitcoin070> bitcoin-cli getmempooldescendants 10d6ab99beb58c22197c3ba0f2072ea2275660fbc03c8f1cff444247faa86d0e 10:49 < bitcoin070> returns an empty array 10:50 < bitcoin070> sorry 10:50 < bitcoin070> maybe not a good example let me find one 10:51 < bitcoin070> bitcoin-cli getmempoolancestors e2f2ff83b69a1b11e735cc4fac0a2b00d9eb3641913a3dbdd52043ca8f7b0ad7 10:53 < bitcoin070> so, I found a TX with 19 ancestors that are unconfirmed 10:54 < bitcoin070> bitcoin-cli getmempoolancestors 4c4dd9034d215a95dd951901271f38aaa02f14cf4be0150e2a1d4c7996aa710b 10:54 < bitcoin070> they are in the mempool though 10:54 < bitcoin070> it returns all 19 true txid 10:54 -!- jcorgan [~jcorgan@unaffiliated/jcorgan] has joined #bitcoin-core-dev 10:54 < bitcoin070> so, whatever is causing this, it's dangerous behavior and never happened before because 10:55 < bitcoin070> a lot of scripts and daemons are monitoring bitcoin node balance 10:55 < gmaxwell> bitcoin070: I suggest downgrading to 0.12.1, which will behave the same but will confirm that there is nothing new going on. 10:55 < bitcoin070> and when it goes from XXX to 0 without any transactions it starts a lot of commotion unfortunately 10:56 < bitcoin070> we want to support segwit, want HD wallet, etc 10:56 < wumpus> sounds like the whole chain of transactions was evicted 10:56 < gmaxwell> In prior attempts to assist you, going back a month ago, I was unable to extract the information I needed to make sense of your reports. If you have the patience now to work through things thats great, though we're about to begin a meeting in here. 10:56 < sipa> a larger max mempool size may help to avoid that 10:57 < wumpus> are you overriding fee or using the smart fee estimate? 10:57 < bitcoin070> not doing anything with fees 10:57 < bitcoin070> just letting the node handle it 10:57 < bitcoin070> I have all the time in the world, obviously I don't want to interrupt your meeting but 10:57 < bitcoin070> FWIW, BitGo is having issues right now too 10:57 < sipa> though i'm not sure what the correct behaviour in this case is... you effectively don't have spendable balance until those transactions confirm 10:58 < sipa> or you abandon them 10:58 < bitcoin070> their node is returning a TXID which is also not propagating the network 10:58 < wumpus> if you can queue up sends then use a sendmany you'd prevent nesting transactions so deeply 10:58 < bitcoin070> wumpus/sipa: interestingly though this only started becoming problematic in bitcoin core 10:58 < bitcoin070> sorry I mean 0.13 10:58 < sipa> bitcoin070: network conditions change all the time 10:59 < gmaxwell> bitcoin070: nothing about this was changed in 0.13, AFAIR. 10:59 < sipa> my guess is that this is just due to interaction with the limited mempool, not the new bitcoin core version 10:59 < sdaftuar> i think we do have an issue to fix, in that we should make it harder for the wallet to generate a transaction that violates the ancestor/descendant chain limits, right? 10:59 < sdaftuar> wasn't there a PR relating to that? 10:59 < sipa> sdaftuar: agree 10:59 < gmaxwell> We do, I believe I created that in response to this person. 10:59 < bitcoin070> damn, it's just strange because 10:59 < bitcoin070> I didn't see this ever happen in .12 10:59 < sipa> i believe that 10:59 < bitcoin070> I know network conditions aren't static, etc 11:00 < bitcoin070> so you're saying the root cause is too long of unconfirmed chain 11:00 < instagibbs> meeting taimu 11:00 < wumpus> if there is a PR for that, it'd make sense for bitcoin070 to test that 11:00 < sipa> or an evicted chain 11:00 < BlueMatt> meeting? 11:00 < wumpus> #startmeeting 11:00 < lightningbot> Meeting started Thu Dec 1 19:00:33 2016 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot. 11:00 < lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic. 11:00 < sipa> present 11:00 < bitcoin070> sipa -- evicted meaning what exactly? 11:00 < bitcoin070> sorry guys I don't want to interrupt 11:00 < gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier 11:00 < jonasschnelli> here 11:00 < kanzure> hi. 11:01 < btcdrak> here 11:01 < cfields> here 11:01 < paveljanik> Hi 11:01 < achow101> hi 11:01 < wumpus> bitcoin070: evicted because they pay the least fee/kb of the transactions in the mempool and the maximum size is reached, you can increase the maximum mempool size though 11:01 < michagogo> Oh, right, now that I'm in the U.S. these are in the early afternoon 11:01 < morcos> bitcoin070: please if you file an issue about this include the output of getbalance() and getunconfirmedbalance() without giving any accounts 11:01 < bitcoin070> okay 11:01 < bitcoin070> fwiw, bitcoin-cli estimatefee 1 returns -1 11:01 < morcos> both "" and "*" when used as the account, give different answers than putting nothing in for the account 11:01 < wumpus> anyhow, meeting started. Any proposed topics? 11:01 < morcos> bitcoin070: expected 11:02 < wumpus> bitcoin070: yes, that is known behavior 11:02 < bitcoin070> okay 11:02 < bitcoin070> 2 and 3 are okay 11:02 < bitcoin070> should these unconfirmed chains eventually confirm and the balance will correct? 11:02 < gmaxwell> wumpus: What issues do people feel aren't getting attention right now? 11:03 < wumpus> gmaxwell: in what regard? 11:03 < paveljanik> review etc. 11:03 < gmaxwell> like PRs and what not, proposed topic: does anyone need some love. 11:03 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Ping timeout: 252 seconds] 11:04 < wumpus> bitcoin070: re: estimatefee returning -1 there's an issue about that: https://github.com/bitcoin/bitcoin/issues/9106 11:04 < bitcoin070> thanks guys 11:04 < bitcoin070> I'll wait and see what happens here 11:04 < bitcoin070> thanks again 11:04 < BlueMatt> so #9183 is proabbly ready for merge after travis passes, means we need to discuss when to do The(tm) split 11:04 < gribble> https://github.com/bitcoin/bitcoin/issues/9183 | Final Preparation for main.cpp Split by TheBlueMatt · Pull Request #9183 · bitcoin/bitcoin · GitHub 11:04 < sipa> i have a short topic for later, about vchDefaultKey (walking to office right now) 11:04 < wumpus> wallet PRs need review at least 11:04 < wumpus> espeically the multiwallet ones 11:04 < jonasschnelli> Yes. An HD. 11:04 < gmaxwell> The test before evict PR seems to be being forgotten a bit. 11:05 < jonasschnelli> We need a restore feature. 11:05 < jonasschnelli> We shouldn't keep users to long in the dark about how they can restore a HD wallet 11:05 < jonasschnelli> Would be nice to have something in 0.14 11:05 < wumpus> vchDefaultKey should die 11:05 < jonasschnelli> heh. Lets discuss that when sipa is back 11:06 < morcos> Now is probably a good time to do a thorough evaluation of which PR's need 0.14.0 milestone... who should we ping if we want someone to mark a PR? 11:06 < wumpus> sipa: re: vchDefaultkey I once created this issue: https://github.com/bitcoin/bitcoin/issues/8416 11:06 * jonasschnelli marks pr 11:06 < BlueMatt> morcos: usually fanquake is pretty on top of it if you mention it in the pr or leave a note on here 11:07 < wumpus> usually if you just ask in teh channel someone will do it 11:08 < jonasschnelli> Tag #9239 for 0.14? IMO we should 11:08 < gribble> https://github.com/bitcoin/bitcoin/issues/9239 | Disable fee estimates for 1 block target by morcos · Pull Request #9239 · bitcoin/bitcoin · GitHub 11:08 < gmaxwell> BlueMatt: do you have any more big wads of data race fixes. 11:08 < bitcoin070> guys I agree 100% we need an HD wallet restore feature 11:08 < gmaxwell> jonasschnelli: we should. 11:08 < morcos> jonasschnelli: definitely 11:08 < bitcoin070> I would make that top priority 11:08 < morcos> bitcoin070: ha ha, it'll just make it always give -1 11:08 < bitcoin070> :) 11:08 < BlueMatt> gmaxwell: I think not...maybe one or two more that I should PR and then net things, but since net is in the middle of so mouch touching I dont want to step all over cfields' toes trying to fix things he already has fixes for 11:09 < gmaxwell> BlueMatt: have you advised cfields about the racy things you found there? 11:09 < BlueMatt> to we have topics? 11:09 < BlueMatt> yes, we've been discussing them 11:09 < morcos> answering your own question? 11:09 < BlueMatt> topics? or are we meeting-chat-time-ing? 11:09 < gmaxwell> I'd like to discuss #9194 11:10 < gribble> https://github.com/bitcoin/bitcoin/issues/9194 | Add option to return non-segwit serialization via rpc by instagibbs · Pull Request #9194 · bitcoin/bitcoin · GitHub 11:10 < cfields> gmaxwell: yes, i'm nuking things in parallel. Simple atomic changes aren't interrupting anything of mine 11:10 < kanzure> morcos: he means he is answering the 'racy' question 11:10 < BlueMatt> I'd like to discuss when to do The Main Split 11:10 < wumpus> #topic Non-segwit serialization vai rpc (#9194) 11:10 < gribble> https://github.com/bitcoin/bitcoin/issues/9194 | Add option to return non-segwit serialization via rpc by instagibbs · Pull Request #9194 · bitcoin/bitcoin · GitHub 11:11 < gmaxwell> Thanks, where are we on this? (the change to let the rawtxn returning rpcs return witness stripped results) I think it's moderately important but there seemed to be some disagreement on the general direction. 11:11 -!- protomar [~protomar@91.214.169.69] has joined #bitcoin-core-dev 11:11 < cfields> wumpus: as part of your named-arg PR, did you consider allowing any global named args? 11:12 < sdaftuar> gmaxwell: initially i wasn't a huge fan of it, but it ended up being less work than i thought, so i don't really care if we do it 11:12 < gmaxwell> sdaftuar: okay, it was mostly you I was concerned about. 11:12 < cfields> where for ^^, any command could accept a "serialversion" named arg 11:12 < wumpus> cfields: we're in a meeting 11:12 < sdaftuar> "sdaftuar was here" i did mean to review the test changes though 11:12 < instagibbs> sdaftuar, yes i basically failed to remember how the commitment worked :) thanks for the review 11:12 < cfields> wumpus: eh? I was referencing 9194 :) 11:13 < gmaxwell> wumpus: that is directly relevant here. :) 11:13 < gmaxwell> okay, if sdaftuar is not concerned about it, -- instagibbs are you waiting for anything there or just more review? 11:13 < wumpus> cfields: well that's not implemented in my pull, could be added later on I guess if you need "context" arguments 11:14 < instagibbs> gmaxwell, more review I think. Want to make sure it doesn't screw up anything I don't touch often 11:14 < wumpus> cfields: I don't have any problem with the idea at least. 11:14 < instagibbs> (ie zmq/rest suggestion) 11:14 < instagibbs> I think the tests actually work now, so confident on rpc side 11:14 -!- CubicEarth [~cubiceart@c-50-159-126-21.hsd1.wa.comcast.net] has joined #bitcoin-core-dev 11:14 < gmaxwell> instagibbs: okay, sounds good to me then. I'll be happy to test and review. 11:14 < instagibbs> great 11:16 -!- jannes [~jannes@178.132.211.90] has quit [Quit: Leaving] 11:16 < petertodd> re: luke-jr's point on "stripped sigs", lots of code gets written that calculates its own txids and isn't using the RPC for that, e.g. python-bitcoinlib RPC code would likely do that, so stripped sigs mode isn't useful there; 100% backwards compatibility is 11:16 < gmaxwell> I'd also like to ask about some other PRs? ##8895 and the checkqueue work for example-- but I don't think JeremyRubin is around. 11:16 < gribble> https://github.com/bitcoin/bitcoin/issues/8895 | Better SigCache Implementation by JeremyRubin · Pull Request #8895 · bitcoin/bitcoin · GitHub 11:16 < morcos> i think 8895 is ready to go in (maybe a little bit more review) 11:17 < morcos> in my opinion checkqueue is still a work in progress 11:17 < wumpus> is that a next topic? 11:17 < morcos> i would really like to get 8895 in for 0.14 11:17 < wumpus> if so, BlueMatt proposed one first 11:17 < BlueMatt> what morcos said, jeremyrubin is just waiting on review for #8895, I think 11:17 < gribble> https://github.com/bitcoin/bitcoin/issues/8895 | Better SigCache Implementation by JeremyRubin · Pull Request #8895 · bitcoin/bitcoin · GitHub 11:17 < sipa> i'll review 8895 again after the last round of changed to it 11:17 < gmaxwell> sorry, tangent. 11:17 < gmaxwell> sounds like the question is answered then. 11:17 < BlueMatt> wumpus: either way we finished that topic :p 11:18 < wumpus> #topic Main split 11:18 < sipa> let's do it 11:18 < morcos> to summarize last 2 topics... concept ack 9194 and 8895 for 0.14.0 11:18 < instagibbs> #9194 11:18 < gribble> https://github.com/bitcoin/bitcoin/issues/9194 | Add option to return non-segwit serialization via rpc by instagibbs · Pull Request #9194 · bitcoin/bitcoin · GitHub 11:18 < instagibbs> oh right 11:19 < BlueMatt> well I think #9183 is +/- ready for merge, jtimon just posted another comment I should look at, but probably ready in an hour or two, it has lots of acks, so question is when to do The Split 11:19 < gribble> https://github.com/bitcoin/bitcoin/issues/9183 | Final Preparation for main.cpp Split by TheBlueMatt · Pull Request #9183 · bitcoin/bitcoin · GitHub 11:19 < wumpus> if it has lots of acks it should be merged 11:19 < cfields> no objections to splitting asap here. It'll only collide trivially with my current work. 11:19 < jonasschnelli> BlueMatt: why when? Because of upcoming rebases? 11:19 < jtimon> I would say open the PR as soon as 9183 gets merged 11:19 < wumpus> no need to delay it if it is ready 11:19 < BlueMatt> jonasschnelli: question is if we do it now or wait until like right before 0.14 11:20 < jonasschnelli> I think all of us are happy to rebase for a such split 11:20 < BlueMatt> ok! 11:20 < jonasschnelli> IMO asap 11:20 < wumpus> let's just go on with it 11:20 < jtimon> BlueMatt: why? what's the advantage of waiting? 11:20 < BlueMatt> ok! sounds good, will open as soon as 9183 is merged 11:20 < morcos> Perhaps BlueMatt is asking if there are any more 0.13 backports to worry about first? 11:20 < cfields> jtimon: to ease backports in the meantime i guess? 11:20 < BlueMatt> morcos: oh, yea, that too 11:20 < jtimon> morcos: cfields oh, right 11:21 < jtimon> are there any backports on the horizon? 11:21 < jtimon> do they conflict with this? 11:21 < wumpus> speaking of 0.13.2, what still needs to go in? (that isn't merged into master yet) 11:21 < gmaxwell> I'm more comfortable with 9183 since matt has been doing a lot of data race and locking testing. usually the worry I have with these sorts of rearrangements is that they'll invalidate hidden locking assumptions. 11:21 < gmaxwell> wumpus: the estimatefee 1 thing. and uhhh 11:21 < sdaftuar> i have one that could go in, the cs_main locking thing with compact blocks 11:21 < sdaftuar> plus 9194 11:21 < cfields> BlueMatt: is it 100% move-only? 11:21 < BlueMatt> cfields: yes 11:22 < morcos> huh is what move only 11:22 < BlueMatt> splitting main.cpp 11:22 < gmaxwell> yea, the locking around compact blocks 11:22 < gmaxwell> #9253 perhaps (though its minor) 11:22 < wumpus> gmaxwell: that one isn't tagged https://github.com/bitcoin/bitcoin/milestone/24 11:22 < gribble> https://github.com/bitcoin/bitcoin/issues/9253 | Fix calculation of number of bound sockets to use by TheBlueMatt · Pull Request #9253 · bitcoin/bitcoin · GitHub 11:22 < sdaftuar> that doens't cleanly apply anyway though, so i'll open a separate PR for it for the backport 11:22 < gmaxwell> #9229 11:22 < gribble> https://github.com/bitcoin/bitcoin/issues/9229 | Remove calls to getaddrinfo_a by TheBlueMatt · Pull Request #9229 · bitcoin/bitcoin · GitHub 11:23 * jonasschnelli tagged 9239 (estimtefee -1) req. backport 11:23 < gmaxwell> #9194 I think (which is why I was asking about it) 11:23 < wumpus> so does the main split pose a difficulty for any of those? 11:23 < gribble> https://github.com/bitcoin/bitcoin/issues/9194 | Add option to return non-segwit serialization via rpc by instagibbs · Pull Request #9194 · bitcoin/bitcoin · GitHub 11:23 < morcos> jonasschnelli: i was wondering about that, i think that would be my preference 11:23 < bitcoin070> hey guys, how often does a wallet rebroadcast transactions that aren't on the network? 11:23 < gmaxwell> I just noticed #9188 isn't merged. that too 11:23 < gribble> https://github.com/bitcoin/bitcoin/issues/9188 | Make orphan parent fetching ask for witnesses. by gmaxwell · Pull Request #9188 · bitcoin/bitcoin · GitHub 11:23 * gmaxwell looks to see if he's the delay on that one 11:24 < wumpus> bitcoin070: after the meeting please 11:24 * gmaxwell is not the delay 11:24 < bitcoin070> sorry.. 11:24 < jonasschnelli> I think after the main split, backports can get a bit more complicated. 11:24 < wumpus> jonasschnelli: thanks 11:25 < gmaxwell> oh good point. 11:25 < wumpus> jonasschnelli: move-only isn't that much of a difficulty if the functions remain the same name and keep the same calling relationship 11:25 < jonasschnelli> #9229 does not touch main, #9239 also not. 11:25 < gmaxwell> well all our backports should be small at least. 11:25 < gribble> https://github.com/bitcoin/bitcoin/issues/9229 | Remove calls to getaddrinfo_a by TheBlueMatt · Pull Request #9229 · bitcoin/bitcoin · GitHub 11:25 < gribble> https://github.com/bitcoin/bitcoin/issues/9239 | Disable fee estimates for 1 block target by morcos · Pull Request #9239 · bitcoin/bitcoin · GitHub 11:25 < wumpus> the only thing that really messes up backports is if the logic changes 11:25 < morcos> can someone tag all the backports we just mentioned... 11:25 < jonasschnelli> Indeed. 11:25 < sdaftuar> #9252 does, but it's easy for me to backport, so i am not worried 11:25 < gribble> https://github.com/bitcoin/bitcoin/issues/9252 | Release cs_main before calling ProcessNewBlock (cmpctblock handling) by sdaftuar · Pull Request #9252 · bitcoin/bitcoin · GitHub 11:25 < sipa> general request for move-only commits: leave functions/methods in the same order in the new file as in the old file; makes diffing much easier to verify move-onlyness 11:25 < BlueMatt> I can rebase main split on top of all the "Backport needed" PRs 11:25 < BlueMatt> but then we need to move fast on the backport-needed PRs 11:25 < BlueMatt> sipa: kk 11:26 < jonasschnelli> backport taged: https://github.com/bitcoin/bitcoin/pulls?q=is%3Aopen+is%3Apr+label%3A%22Needs+backport%22 11:26 < gmaxwell> okay, I don't see anything else thats open which I strongly believe needs to be in 0.13.2 11:26 < wumpus> great 11:27 < wumpus> BlueMatt: makes sense to move fast on 0.13.2 in any case 11:27 < BlueMatt> true 11:27 < jtimon> wumpus: +1 11:27 < wumpus> there's plenty of stuff for a release already 11:27 < morcos> jonasschnelli: 9194, 9252 for backport i think is what we said 11:27 < BlueMatt> ok, so then everyone's review focus is "Needs backport" tag, and then we main-split after those are done? 11:28 < sipa> BlueMatt: sgtm 11:28 < sdaftuar> agreed 11:28 < jonasschnelli> tagged 9194 9252 11:28 < wumpus> sgtm too 11:28 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 11:28 < gmaxwell> instagibbs: could you give #9019 a look? we might want a simple fix for that in 0.13.2. It might be a two liner. 11:28 < gribble> https://github.com/bitcoin/bitcoin/issues/9019 | Avoid making chains of txn >25 deep. · Issue #9019 · bitcoin/bitcoin · GitHub 11:28 < jtimon> fine with me but will actually review 9183 first 11:28 < instagibbs> yeah sure, ive run into that a number of times :) 11:29 < wumpus> okay, that concludes the 0.13.2 and main split topic I guess 11:30 < wumpus> any other topics proposed? 11:30 < jonasschnelli> proposed topic from sipa: wallets default key, another topic could be: HD restore 11:30 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit] 11:30 < wumpus> ah yes 11:30 < sipa> ok 11:30 < wumpus> #topic vchdefault in wallet 11:30 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 11:30 < sipa> so we currently have a leftover remnant of ancient times, vchDefaultKey in the wallet 11:30 < jonasschnelli> The default key is misused for detecting the first run IMO 11:30 < wumpus> #link https://github.com/bitcoin/bitcoin/issues/8416 11:30 < sipa> which is absolutely unused, except for determining whether a new wallet was just created 11:30 < sipa> i'd like to get rid of it 11:30 < sipa> however 11:31 < wumpus> yes, we should get rid of it, but maybe not for 0.14, it's not really urgent 11:31 < sipa> if we do that, a downgrade to an older wallet version would result in failing to rescan 11:31 < jonasschnelli> Yes. We should combine it with other features. 11:31 < sipa> as it would trigger the new wallet logic, which writes the current chainstate to the wallet file 11:31 < jonasschnelli> We should have combined it with HD in 0.13. :/ 11:31 < wumpus> sipa: we could add fallback logic into 0.14 11:31 < wumpus> then remove it for 0.15 11:31 < sipa> writing a dummy has the disadvantage of actually risking giving out a dummy key as address (in very old versions) 11:32 < wumpus> then it'd be possible to go back to 0.14 when 0.15 is released 11:32 < wumpus> but o further back 11:32 < sipa> a third option is introducing a new min wallet version, so for example 0.15 wallets will never be openable with 0.13 11:32 < wumpus> (unless we backport the fallback logic ofc) 11:32 < wumpus> yes, that'd be my preference 11:32 < wumpus> no dummies and other hacks, just versioining 11:33 < jonasschnelli> Yes. This only affect new wallets, right? 11:33 < wumpus> it'd only apply to new wallets created with the new version anyway 11:33 < wumpus> right. 11:33 < gmaxwell> I'm okay with versioning, but we should keep it simple. 11:33 < wumpus> it's not like you can't go back with an old wallet 11:33 < jonasschnelli> If you have a wallet from 0.12, it won't upgrade unless you do an explicit upgrtadew 11:33 < sipa> so let's switch in 0.14 to stop relying on vchDefault key, but still write it (as an actually valid key) 11:33 < wumpus> in any case this isn't urgent 11:33 < sipa> and then in 0.15 delete the vchDefaultKey and increase the min version to 0.14? 11:33 < jonasschnelli> sipa: ack 11:33 < wumpus> we could do it over 2-3 major versions 11:34 < gmaxwell> ACK sipa. 11:34 < wumpus> sipa: yes that's what I meant too 11:34 < jonasschnelli> Downgrading new wallets is probably not required over more then 2 major versions. 11:34 < wumpus> we could even backport that to 0.13 11:35 < jonasschnelli> wumpus: Yes. We should. 11:35 < wumpus> (e.g. work if there is no vchdefault, but do make it) 11:35 < wumpus> that particular code hasn't been touched for years so backporting is trivial 11:36 < sipa> ok, end topic 11:36 < morcos> my 2 cents would be against backporting to 0.13.2 at least.. since i think 1 major version backport for downgrading the wallet seems sufficient 11:36 < jonasschnelli> HD restore? anyone interested? 11:36 < morcos> just to not hold up 0.13.2 11:36 < jonasschnelli> morcos: whats the downside of the (simple) backport? 11:36 < jonasschnelli> ok. 11:37 < jtimon> ack min wallet version 11:37 < sipa> jonasschnelli: certainly, but it requires some pretty big changes (like removing keypool entries with seen keys in received transactions) 11:37 < wumpus> morcos: I mean to 0.13 *branch*, so 0.13.2 or 0.13.3 or whatever happens to be then 11:37 < wumpus> morcos: I certainly wouldn't propose holding up 0.13.2 for it 11:37 < morcos> wumpus: +1 if it doesn't hold 13.2 11:37 < jonasschnelli> Okay. Yes. If the BP is non trivial, we can skip it. 11:37 < wumpus> morcos: no one is saying that! 11:37 < gmaxwell> yea, no holdup. but BP would be nice. 11:38 < jtimon> jonasschnelli: who requires to downgrade wallets? 11:38 < wumpus> "removing keypool entries with seen keys in received transactions"? 11:38 < wumpus> is that required for removing the default key? 11:39 < morcos> wumpus: is that HD restore he's talking about? 11:39 < jonasschnelli> jtimon: Is more about the option to run your wallet.dat on an older version 11:40 < jonasschnelli> morcos: not yet 11:40 < jonasschnelli> default key != HD 11:40 < morcos> jonasschnelli: yes understood, just trying to decipher sipas comment 11:40 < wumpus> morcos: I don't know? I'm confused 11:40 < jtimon> jonasschnelli: right, why do you want that? anyway, I was reading slow, we can discuss after the meeting 11:40 < gmaxwell> I thought sip was responding to "hd restore" 11:40 < wumpus> it'd make more sense, we're combinding topics is an awkward way now 11:40 < gmaxwell> s/sip/sipa/ 11:40 < jonasschnelli> Ah. Sorry 11:41 -!- Anduck [~Anduck@unaffiliated/anduck] has joined #bitcoin-core-dev 11:41 < wumpus> #topic HD restore 11:41 < jonasschnelli> Can we have the HD restore topic then? 11:41 < jonasschnelli> thx 11:41 < gmaxwell> TM: Prefix all messages related to topic 1 with T1: and for topic 2 with T2: 11:41 < jonasschnelli> Since 0.13 we have HD by default, we should have a feature to restore hd wallets 11:41 < jonasschnelli> Maybe to late for 0.14, but in 0.15. 11:41 < jonasschnelli> I think we need a concept first 11:41 < jonasschnelli> IMO it should go over a seperate tool (bitcoin-wallet) 11:42 < morcos> jonasschnelli: can you explain what that means... you lost the full wallet backup but have the master seed/key whatever it's called? 11:42 < jonasschnelli> Because you ideally don't want to handle master xpriv in RPC or -cmd 11:42 < gmaxwell> seperate tool sounds like something that would need a non-pruned blockchain .. which I don't think is desirable. 11:42 < jonasschnelli> Yes. You have an (olx) xpriv or a wallet.dump and you wish to restore the complete wallet 11:43 < jonasschnelli> gmaxwell: Yes. This is a good point. 11:43 < gmaxwell> how is this different from having a wallet.dat that you backed up right at the start? 11:43 < jonasschnelli> The tool should create wallet(s).dat and then you can run a rescan 11:43 < gmaxwell> okay thats how. 11:43 < jonasschnelli> Maybe the tool can interact with RPC and the UTXO set to detect the gap limits 11:44 < gmaxwell> I think a tool to create a wallet.dat from dump data would be good, but perhaps not essential for restore functionality. which could work from just a wallet.dat that the user already backed up. 11:44 < jonasschnelli> IMO it should result with a wallet with the corresponding keys up to the last detected UTXO (respecting a large gap) 11:44 < gmaxwell> I'm really concnered that we didn't manage to split the change keypool for the HD wallet support. This makes recovery a mess. 11:45 < jtimon> jonasschnelli: what about something like this, create first 100 addresses, rescan, while some of them got any funds, create the next 100 and loop 11:45 < jtimon> I assume is horribly inefficient, reading slow again 11:45 < gmaxwell> jtimon: sometime 100 years later your wallet is restored. 11:45 < jonasschnelli> gmaxwell: there was a PR with that. But nobody reviewd it (external and internal chain) 11:45 < gmaxwell> (rescans take hours, we should stop thinking of rescans as an option generally.) 11:45 < wumpus> jonasschnelli: yea :/ review is always a problem with wallet pulls 11:45 < jtimon> gmaxwell: thanks for confirming that is the flaw of my naive design 11:46 < wumpus> I'd recommend that we first review and get the current wallet pulls merged, before working on even more 11:46 < sipa> wumpus: +1 11:46 < jonasschnelli> I think it would help to review #9143 and #9256 11:46 < gribble> https://github.com/bitcoin/bitcoin/issues/9143 | Refactor ZapWalletTxes to avoid layer violations by jonasschnelli · Pull Request #9143 · bitcoin/bitcoin · GitHub 11:46 < gribble> https://github.com/bitcoin/bitcoin/issues/9256 | Fix more CWallet/CWalletDB layer violations by jonasschnelli · Pull Request #9256 · bitcoin/bitcoin · GitHub 11:46 < wumpus> I don't think HD recovery will make 0.4 as it still has to be written 11:46 < gmaxwell> jonasschnelli: I thought it was merged in fact, at the time. oh well. 11:47 < jonasschnelli> This would slowly allow creating a such tool 11:47 < wumpus> but we can make e.g. multiwallet land 11:47 < wumpus> s/0.4/0.14 11:47 < jonasschnelli> #8723 would also nice for HD 11:47 < gribble> https://github.com/bitcoin/bitcoin/issues/8723 | [Wallet] Add support for flexible BIP32/HD keypath-scheme by jonasschnelli · Pull Request #8723 · bitcoin/bitcoin · GitHub 11:47 < wumpus> and yes the refactors 11:47 < jtimon> what about not restoring the whole wallet but only what's currently in the utxo? would that be acceptable? 11:48 < jonasschnelli> jtimon: Yes. That should be the default IMO 11:48 < jonasschnelli> Then you can optionally do a rescan if you like. 11:48 < gmaxwell> I think we should not add any more complexity to the HD support until we fix the path split issue. 11:48 < gmaxwell> We're already going to have to support one legacy setup, lets try to not proliferate little changes. 11:49 < jonasschnelli> Okay. I can focus on the path split 11:49 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt] 11:49 < jcorgan> gmaxwell: background on the "path split issue" i can go read? 11:49 < jonasschnelli> But users start to ask,... how can I recover a HD wallet. We need to give them a reasonable answer. 11:49 < jonasschnelli> jcorgan: bip32 internal and external chains 11:50 < instagibbs> jcorgan, change output is on same chain as spending 11:50 < jcorgan> ah, yes, i should ack 8723 11:50 < instagibbs> err receiving* 11:50 < gmaxwell> jcorgan: the consequence is that you can end up giving out change keys as addresses for people to pay (hiding their payments from you) or have change show up as payments if you have wallets recovered from hd data. 11:51 < jcorgan> yeah, i get it 11:51 < gmaxwell> jcorgan: e.g. I give you an address. then recover my wallet. Then create a transaction and use the same addr for change. Then you pay that address, and I don't see the payment. 11:51 < gmaxwell> yadda yadda. 11:51 < Chris_Stewart_5> gmaxwell: Thanks for the explanation 11:52 < gmaxwell> jonasschnelli: load your old wallet.dat. rescan. Where does that currently fall down? 11:52 < jonasschnelli> gmaxwell: missing keys 11:52 < sipa> ? 11:52 < jonasschnelli> you mean restore a old wallet.dat? 11:52 < jonasschnelli> you need to loop(1000, getnewaddress) first. :) 11:52 < gmaxwell> right we don't remove all keys up to the highest observed, only observed? that sounds like a simple fix. 11:53 < jonasschnelli> Well, if you have 1001 keys, you miss one. 11:53 < jonasschnelli> gmaxwell: this would result in multiple rescans. 11:53 < wumpus> right, it doesn't automatically wind forward when it sees known keys 11:53 < sipa> gmaxwell: what do you mean remove? 11:53 < gmaxwell> sipa: mark used in keypool. 11:53 < sipa> gmaxwell: that's not implemented at all 11:53 < luke-jr> :x 11:53 < gmaxwell> jonasschnelli: which is a workaround that you can answer. 11:53 < sipa> gmaxwell: you need the keypool 11:54 < jonasschnelli> Yes. The loop getnewaddress is the current workaround. 11:54 < jcorgan> it seems the bigger issue is there is no standard way of archiving derivation path usage + metadata 11:54 < gmaxwell> sipa: that needs to be implemented, at least that much would be small. 11:54 < jonasschnelli> But we don't even offer a rescan down to the HD feature birthdate. 11:54 < jonasschnelli> The UX is bad 11:54 < sipa> gmaxwell: it's not easy 11:54 < jonasschnelli> yes. It's pretty complex. 11:54 < sipa> gmaxwell: as we don't have a way to store keys without private key 11:54 < sipa> or rescan would require the passphrase 11:54 < gmaxwell> sipa: we can still mark the right things as used! 11:54 < jonasschnelli> We first need a Wallet record type hdkey 11:55 < sipa> gmaxwell: ah, yes! 11:55 < sipa> jonasschnelli: no we don't 11:55 < gmaxwell> sipa: e.g. rescan, unlock, rescan. not great, but failing to mark things as used is really goofy. 11:55 < gmaxwell> but should also be trivial to fix. 11:55 < sipa> jonasschnelli: just a way to store a key without known private key (with semantics that it will get computed at first unlock) 11:55 < sipa> or not at all, i guess 11:56 < sipa> and always derive it on the fly 11:56 < gmaxwell> sipa: No, we can't. 11:56 < gmaxwell> (keys are hardened because we support export) 11:56 < jonasschnelli> IMO we should just store pubkeys and the according keypath 11:56 < sipa> gmaxwell: oh, right 11:56 < jonasschnelli> maybe the relevant master key (if we support multiple= 11:56 < gmaxwell> and yes, storing the private keys is a bit silly. :P but an optimization. 11:56 < jcorgan> jonasschnelli: agree, if there were a standard way of documenting that 11:57 < sipa> 3 minutes 11:57 < gmaxwell> ( I meant that not storing the private keys is mearly an optimization and not important.) 11:58 < sipa> gmaxwell: we could also stop rescanning whenever the keypool goes below some value, and require unlocking first 11:58 < sipa> or something 11:58 < jtimon> gmaxwell: I still don't know how you solve the problem you described 11:58 < gmaxwell> in any case, IMO low hanging is to correctly do the use-marking, and also increase the default keypool for hdwallets.. 100 is somewhat absurdly small, 1000 would be better. And getting splitting in. 11:59 < gmaxwell> the last is a path incompatible change unfortunately, :( 11:59 < gmaxwell> but the rest does not need coordination with anything. 11:59 < sipa> we should first split up the keypools into change and non-change, iguess 11:59 < jonasschnelli> Okay. I can work on a patch. 11:59 < jtimon> sipa: oh, I see 11:59 < sipa> then do the use marking (as it will need to mark within each path) 11:59 < jcorgan> jonasschnelli: let's pm after the mtg on that 11:59 < gmaxwell> sipa: the split will make wallets that do that incompatible with older software. 12:00 < sipa> gmaxwell: yes 12:00 < jonasschnelli> gmaxwell: Yes. We just need to support both types 12:00 < jtimon> does the change keypool have its own seed or is it derived? 12:00 < wumpus> #endmeeting 12:00 < lightningbot> Meeting ended Thu Dec 1 20:00:40 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) 12:00 < lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-12-01-19.00.html 12:00 < lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-12-01-19.00.txt 12:00 < lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-12-01-19.00.log.html 12:00 < jonasschnelli> the one-chain-hd-wallet and the flexible-keypath-wallet with ext/int chain 12:00 < BlueMatt> wumpus, cfields re #9229 should I just switch it to be a full-revert of #4421 instead of fighting with autotools to keep the inet_pton working? (is anyone aware of any bugs from dropping those calls and just always using getaddrinfo?) 12:00 < gribble> https://github.com/bitcoin/bitcoin/issues/9229 | Remove calls to getaddrinfo_a by TheBlueMatt · Pull Request #9229 · bitcoin/bitcoin · GitHub 12:00 < gribble> https://github.com/bitcoin/bitcoin/issues/4421 | Use async name resolving to improve net thread responsiveness by 4tar · Pull Request #4421 · bitcoin/bitcoin · GitHub 12:00 < sipa> jtimon: the keypool is just a set of keys 12:00 < jonasschnelli> jcorgan: same seed 12:00 < gmaxwell> jtimon: see bip32.. internal vs external. 12:00 < instagibbs> jtimon, it's just a different path 12:01 < jonasschnelli> jcorgan: meant jtimon 12:01 < sipa> jtimon: the seed information is in how to replenish it, hwich happens elsewhere 12:01 < gmaxwell> jonasschnelli: whenever the answer is 'support both types' that strongly encourages batching changes. 12:01 < jtimon> sipa: thanks, I'm not so familiar with the bip32 wallet 12:01 < jonasschnelli> Combining the splitting with https://github.com/bitcoin/bitcoin/pull/8723/files would make sense IMO 12:01 < gmaxwell> I really do not want to support 30 different kinds of wallets 5 years from now. :) 12:02 < cfields> BlueMatt: looking 12:02 < jonasschnelli> Yes. We don't have the splitting because we wanted a minimal sane change. 12:02 < sipa> gmaxwell: we'll just do a softfork to require SNARKs in signatures that prove that keys were derived deterministically? 12:02 < jonasschnelli> I think this was resonable. But we shouln't wait to long now 12:02 < sipa> *ducks* 12:02 < jonasschnelli> heh 12:03 < jtimon> jonasschnelli: regarding donwgrading the wallet, no need for further discussion, I thought of an example case where I could want that myself already 12:03 < wumpus> cfields: I guess we will use libevent for resolving anyway, when that lands? 12:03 < BlueMatt> wumpus: yes, but need something to backport 12:03 < gmaxwell> If we're going to be incompatible, not storing private keys for hd keys would also be a good move... as it will make it less costly to make the lookahead larger. 12:03 < gmaxwell> so in any case, thats just why I was suggesting fixing the use-flagging first, no compatiblity concerns. 12:03 < wumpus> BlueMatt: sure in the meantime we can revert #4421 if it's buggy 12:03 < gribble> https://github.com/bitcoin/bitcoin/issues/4421 | Use async name resolving to improve net thread responsiveness by 4tar · Pull Request #4421 · bitcoin/bitcoin · GitHub 12:03 < cfields> wumpus: yes. I'm not really concerned about dropping this back to sync for now for that reason 12:04 < BlueMatt> ok, so I'll just do a full-revert of 4421 12:04 < BlueMatt> sgtm 12:04 < cfields> BlueMatt: i'm tracing through the ifdefs locally now to make sure it works like i assume 12:04 < gmaxwell> BlueMatt mentioned to me that he thinks he's actually seen a getaddrinfo_a crash in production. (backtrace was really short and without symbols because it was off in libc someplace) 12:04 < wumpus> BlueMatt: please do comment in the issue that you'regoing to revert it and why 12:05 < BlueMatt> kk 12:05 < jtimon> not sure what the conclusions about the wallet are 12:05 < BlueMatt> yes, if you've ever seen a crash in prod with a backtrace of length three with ???s for all of them...it might be gai_a 12:06 < bitcoin-git> [bitcoin] sdaftuar opened pull request #9259: [0.13 backport] Release cs_main before calling ProcessNewBlock or processing header (cmpctblock handling) (0.13...cb-lock-13) https://github.com/bitcoin/bitcoin/pull/9259 12:06 < wumpus> espeically if you have no debug symbols for libc I guess? 12:06 < BlueMatt> wumpus: yes 12:06 < BlueMatt> wait, gmaxwell why do we want backport of #9252? I dont think releasing cs_main gives us anything in backport, but does carry risk 12:06 < gribble> https://github.com/bitcoin/bitcoin/issues/9252 | Release cs_main before calling ProcessNewBlock, or processing headers (cmpctblock handling) by sdaftuar · Pull Request #9252 · bitcoin/bitcoin · GitHub 12:07 -!- d9b4bef9 [~d9b4bef9@web419.webfaction.com] has quit [Remote host closed the connection] 12:07 < wumpus> I still wonder if we're misusing it or whether libc is really buggy. It's not impossible that libc is buggy just sounds so unlikely 12:07 < wumpus> in any case reverting it for now makes sense 12:08 < BlueMatt> wumpus: https://sourceware.org/bugzilla/show_bug.cgi?id=20874#c4 12:08 -!- d9b4bef9 [~d9b4bef9@web419.webfaction.com] has joined #bitcoin-core-dev 12:08 < BlueMatt> "This does look like a bug." 12:09 < BlueMatt> its possible I'm thinking of another issue wrt having seen this in prod - I've seen other issues in -qt that I think are just X instability 12:09 < BlueMatt> and I may be thinking of those, but gai_a is definitely unsafe 12:09 < cfields> in any case, I'm working furiously now on event-ifying the net code, pr should be up within the next few days, and is looking pretty simple. After that, the libevent changes come in. So the end is near for that code anyway :) 12:10 < BlueMatt> good riddance 12:10 < luke-jr> sdaftuar: BIP 90; what's with the "foo" and "tmp.mediawiki" files? 12:10 < sdaftuar> luke-jr: gah, let check 12:10 < sdaftuar> let me check* 12:10 < wumpus> cfields: thanks for the update, looking forward to that :) 12:12 < cfields> BlueMatt: ah i see, the inet_pton is just used to skip the getaddrinfo in the event that an ip was passed as a string 12:12 < cfields> (sorry, was trying to figure out what that had to do with gai_a) 12:13 < cfields> and actually, still not sure why they're entangled 12:13 < cfields> but either way, full revert sounds good to me 12:23 < sdaftuar> BlueMatt: gmaxwell: regarding #9252, i appear to have misremembered and thought we backported #8968 -- looking back on it, though, it appears we didn't (though it was tagged for backport at one point). 12:23 < gribble> https://github.com/bitcoin/bitcoin/issues/9252 | Release cs_main before calling ProcessNewBlock, or processing headers (cmpctblock handling) by sdaftuar · Pull Request #9252 · bitcoin/bitcoin · GitHub 12:23 < gribble> https://github.com/bitcoin/bitcoin/issues/8968 | An error has occurred and has been logged. Please contact this bot's administrator for more information. 12:26 < BlueMatt> sdaftuar: yea, so I'd think dont backport 9252 12:28 -!- gabridome [~gabridome@87.18.61.184] has joined #bitcoin-core-dev 12:28 < bitcoin-git> [bitcoin] sdaftuar closed pull request #9259: [0.13 backport] Release cs_main before calling ProcessNewBlock or processing header (cmpctblock handling) (0.13...cb-lock-13) https://github.com/bitcoin/bitcoin/pull/9259 12:28 < sdaftuar> BlueMatt: done. er, undone. whateer. 12:31 < BlueMatt> cool, thanks 12:37 -!- LeMiner [~LeMiner@5ED1AFBF.cm-7-2c.dynamic.ziggo.nl] has quit [Read error: Connection reset by peer] 12:37 -!- LeMiner [~LeMiner@unaffiliated/leminer] has joined #bitcoin-core-dev 12:39 < morcos> jonasschnelli: Can I bother you to milestone for 0.14.0 #9107 and #9138 12:39 < gribble> https://github.com/bitcoin/bitcoin/issues/9107 | Safer modify new coins by morcos · Pull Request #9107 · bitcoin/bitcoin · GitHub 12:39 < gribble> https://github.com/bitcoin/bitcoin/issues/9138 | Improve fee estimation by morcos · Pull Request #9138 · bitcoin/bitcoin · GitHub 12:42 < BlueMatt> ACK on those for 14 12:52 -!- randy-waterhouse [~kiwigb@43.228.156.103] has joined #bitcoin-core-dev 12:53 -!- randy-waterhouse [~kiwigb@43.228.156.103] has quit [Changing host] 12:53 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has joined #bitcoin-core-dev 12:53 -!- bsm117532 [~mcelrath@38.121.165.30] has quit [Quit: Leaving.] 12:53 -!- bsm117532 [~mcelrath@38.121.165.30] has joined #bitcoin-core-dev 12:58 < cfields> wumpus: are you aiming for 0.14 for #8811 ? 12:58 < gribble> https://github.com/bitcoin/bitcoin/issues/8811 | rpc: Add support for JSON-RPC named arguments by laanwj · Pull Request #8811 · bitcoin/bitcoin · GitHub 12:58 < luke-jr> sigh, MemorySanitizer is broken in Travis (not affecting Core) 13:00 -!- bsm117532 [~mcelrath@38.121.165.30] has quit [Quit: Leaving.] 13:00 < cfields> luke-jr: doesn't it require an instrumented libc++ ? 13:01 < luke-jr> cfields: it worked before; seems some Linux kernel bugfix broke it 13:07 < bitcoin-git> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/c79e52ad304a...c1a52276848d 13:07 < bitcoin-git> bitcoin/master 9e1f468 Matt Corallo: Fix calculation of number of bound sockets to use 13:07 < bitcoin-git> bitcoin/master c1a5227 Pieter Wuille: Merge #9253: Fix calculation of number of bound sockets to use... 13:07 < bitcoin-git> [bitcoin] sipa closed pull request #9253: Fix calculation of number of bound sockets to use (master...2016-11-nbind) https://github.com/bitcoin/bitcoin/pull/9253 13:11 -!- mrkent [~textual@unaffiliated/mrkent] has joined #bitcoin-core-dev 13:35 < bitcoin-git> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/c1a52276848d...ad826b3df9f7 13:35 < bitcoin-git> bitcoin/master 5b0150a Gregory Maxwell: Make orphan parent fetching ask for witnesses.... 13:35 < bitcoin-git> bitcoin/master ad826b3 Pieter Wuille: Merge #9188: Make orphan parent fetching ask for witnesses.... 13:35 < bitcoin-git> [bitcoin] sipa closed pull request #9188: Make orphan parent fetching ask for witnesses. (master...witness_the_orphans) https://github.com/bitcoin/bitcoin/pull/9188 13:46 -!- mrkent [~textual@unaffiliated/mrkent] has quit [] 13:49 -!- mrkent [~textual@unaffiliated/mrkent] has joined #bitcoin-core-dev 13:51 -!- wvr [~wvr@189.red-83-33-12.dynamicip.rima-tde.net] has quit [] 13:52 -!- aalex__ [~aalex@64.187.177.58] has joined #bitcoin-core-dev 13:54 -!- aalex [~aalex@64.187.177.58] has joined #bitcoin-core-dev 13:56 -!- aalex_ [~aalex@64.187.177.58] has quit [Ping timeout: 244 seconds] 13:57 -!- aalex__ [~aalex@64.187.177.58] has quit [Ping timeout: 245 seconds] 14:06 -!- AaronvanW [~ewout@207pc74.sshunet.nl] has joined #bitcoin-core-dev 14:06 -!- AaronvanW [~ewout@207pc74.sshunet.nl] has quit [Changing host] 14:06 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 14:07 -!- bitcoin070 [369924af@gateway/web/freenode/ip.54.153.36.175] has quit [Ping timeout: 260 seconds] 14:08 -!- droark [~droark@c-24-22-123-27.hsd1.or.comcast.net] has quit [Read error: Connection reset by peer] 14:10 < BlueMatt> jtimon: ok with #9183 as-is now (with print cleanups maybe happening later?) 14:11 < gribble> https://github.com/bitcoin/bitcoin/issues/9183 | Final Preparation for main.cpp Split by TheBlueMatt · Pull Request #9183 · bitcoin/bitcoin · GitHub 14:11 < jtimon> sure 14:13 < jtimon> BlueMatt: reacted with emojis to your answers, thanks 14:13 -!- gabridome [~gabridome@87.18.61.184] has quit [Quit: gabridome] 14:14 < BlueMatt> cool, so i should bug sipa to press merge then... 14:15 < jtimon> right duh, it could work without the {} 14:15 < jtimon> fine with me 14:15 < BlueMatt> it could? 14:15 < BlueMatt> i dont think it does that level of magic, does it? 14:15 < jtimon> s/couldn't 14:16 < BlueMatt> ahh 14:16 < jtimon> I mean I should have realized that myself on review, but thanks for the clarification 14:26 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has quit [Ping timeout: 260 seconds] 14:30 -!- randy-waterhouse [~kiwigb@43.228.156.103] has joined #bitcoin-core-dev 14:30 -!- gabridome [~gabridome@87.18.61.184] has joined #bitcoin-core-dev 14:31 -!- randy-waterhouse [~kiwigb@43.228.156.103] has quit [Changing host] 14:31 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has joined #bitcoin-core-dev 14:44 -!- gabridome [~gabridome@87.18.61.184] has quit [Quit: gabridome] 14:47 < BlueMatt> oh, hey, the backport-needed PRs no longer touch main at all 14:47 < BlueMatt> hmm...sipa you wanna split main today? 14:47 < BlueMatt> or wumpus, if you havent gone to bed yet 14:48 < BlueMatt> also, anyone need some review-love? 14:56 -!- protomar [~protomar@91.214.169.69] has quit [Quit: Leaving] 14:58 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 244 seconds] 15:00 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev 15:06 < sipa> BlueMatt: does 9183 conflict with #9229, #9239, or #9194 ? 15:06 < gribble> https://github.com/bitcoin/bitcoin/issues/9229 | Remove calls to getaddrinfo_a by TheBlueMatt · Pull Request #9229 · bitcoin/bitcoin · GitHub 15:06 < gribble> https://github.com/bitcoin/bitcoin/issues/9239 | Disable fee estimates for 1 block target by morcos · Pull Request #9239 · bitcoin/bitcoin · GitHub 15:06 < gribble> https://github.com/bitcoin/bitcoin/issues/9194 | Add option to return non-segwit serialization via rpc by instagibbs · Pull Request #9194 · bitcoin/bitcoin · GitHub 15:06 < BlueMatt> sipa: none of those touch main.{h,cpp}, which is the only thing 9183 touches 15:07 < BlueMatt> so...I doubt it? 15:07 < sipa> ok! 15:16 -!- bitcoin272 [369924af@gateway/web/freenode/ip.54.153.36.175] has joined #bitcoin-core-dev 15:17 < bitcoin272> does sendtoaddress have any way to prioritize not making transactions that will have issues because of too many unconfirmed ancestors? 15:17 < bitcoin272> running into a ton of issues here and i don't want to disable spending 0 conf change tb 15:17 < bitcoin272> tbh8 15:17 < bitcoin272> tbh* 15:18 < bitcoin272> but sendtoaddress is making huge chains when it would be more sensible to build multiple chains based on the output set as opposed to only building 1 so that it goes over 25 unconfirmed ancestors and makes a huge mess 15:19 < instagibbs> bitcoin272, I started take a look at that today. I can probably at least have the send fail if it's going to run into mempool limits 15:19 < sipa> BlueMatt: it actually tries to avoid using very recently created change 15:19 < sipa> eh 15:19 < bitcoin272> instagibbs -- it's a huge huge issue unfortunately 15:19 < sipa> bitcoin272: ^ 15:20 < bitcoin272> I don't know if this was ever a problem pre 0.13 15:20 < sipa> it should in 0.12 as well 15:20 -!- nickler [~nickler@185.12.46.130] has quit [Ping timeout: 268 seconds] 15:20 < instagibbs> coin selection didn't change since then 15:20 < instagibbs> afaik 15:20 < sipa> indeed 15:20 < bitcoin272> what about mempool limits? 15:20 < instagibbs> nope 15:20 < sipa> mempool limits were added in 0.12 15:20 < instagibbs> right, since 0.12 i mean 15:20 < bitcoin272> i see 15:21 < bitcoin272> this behavior guys 15:21 < bitcoin272> is resulting in some very unfortunate things :/ 15:21 < bitcoin272> where is the mempool constraint defined 15:21 < bitcoin272> is that something we could modify / recompile 15:21 < sipa> it's a setting 15:21 -!- nickler [~nickler@185.12.46.130] has joined #bitcoin-core-dev 15:22 < sipa> limitancestorcount and limitdescendantcount 15:22 < bitcoin272> it defaults to 25? 15:22 < sipa> yes 15:22 < instagibbs> you can crank those up, but the wallet simply shouldn't violate those limits(I'm working on that) 15:22 < bitcoin272> and that's specifically meaning, dump things after 25 from the mempool, even if they are our own TX? 15:22 < sipa> yes, indeed 15:22 < sipa> bitcoin272: they're not dumped; they're just not accepted 15:22 < bitcoin272> i see... but 15:22 < bitcoin272> they are still broadcast to the network? 15:23 < sipa> no 15:23 < sipa> not until the rest is confirmed 15:23 < bitcoin272> ok right but 15:23 < bitcoin272> so to be clear 15:23 < bitcoin272> sendtoaddress will fail 15:23 < bitcoin272> the TX will sit in the wallet though 15:23 < sipa> yes, the wallet behaviour is bad currently 15:23 < bitcoin272> and when the parents confirm, it will auto rebroad cast 15:23 < bitcoin272> right? 15:23 < sipa> it should at least give an error instead of silently not doing anything 15:23 < sipa> yes, i believe it will 15:23 < bitcoin272> oh man so 15:24 < bitcoin272> this is the problem then 15:24 < bitcoin272> okay.. at least I know 15:24 < bitcoin272> it shouldn't queue that transaction because what happens is 15:24 < sipa> well it would be useful to verify this, by increasing your descendant and ancestor counts 15:24 < bitcoin272> it's easy for business logic to believe a transaction send has failed 15:24 < bitcoin272> and resend it 15:24 < bitcoin272> only for the wallet to send the same coins again when the parents confirm 15:24 < bitcoin272> resulting in multiple payments being facilitated 15:25 < sipa> yup 15:25 < bitcoin272> most unfortunate indeed 15:25 < bitcoin272> okay well this is a start 15:25 < bitcoin272> and this would've never been apparent pre 0.12 right 15:25 < BlueMatt> aaccctuuallllyyyyy....anyone have any objections if main.cpp gets renamed to "blocktxprocessing.cpp"? 15:25 < sipa> indeed 15:25 < bitcoin272> okay 15:25 < bitcoin272> sipa, I really appreciate this 15:25 < bitcoin272> this clarifies things 15:26 < bitcoin272> what is a reasonable limitancestorcount if resource consumption is low 15:26 < sipa> bitcoin272: thanks; if you have further results, can you comment here? https://github.com/bitcoin/bitcoin/issues/9019 15:27 < sipa> bitcoin272: you're not mining, right? 15:27 < bitcoin272> nope 15:27 < bitcoin272> I am going to bump ancestor/descendant count to 200 15:27 < sipa> that sounds reasonable to me 15:27 < sipa> just to be clear: don't expect these chains to confirm quickly 15:28 < sipa> because other nodes in the network won't accept these long chains, at least not immediately 15:28 < sipa> on rebroadcasts they should go out piecewise, though 15:28 < bitcoin272> well 15:28 < bitcoin272> it's more important that the sendtoaddress command doesn't fail 15:28 < bitcoin272> but still enqueue an automatic payment 15:28 < bitcoin272> that is 100% the most important 15:28 < bitcoin272> we just need sendtoaddress to return a txid 15:29 < bitcoin272> whether or not it hits the network immediately isn't so bad, we just can't have sendtoaddress failing but also having the wallet sending the coins again of its own accord 15:33 < bitcoin272> is there a way to purge a transaction 15:33 < bitcoin272> so it doesn't send? 15:33 < bitcoin272> (on this automatic queue) 15:34 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Read error: Connection reset by peer] 15:34 < bitcoin272> will abandontransaction do it? 15:38 < sipa> yes 15:38 < sipa> that's the intended behaviour... if your transaction doesn't confirm and gets evicted, you can manually abandon it 15:39 < sipa> though it's not very well integrated or good guidelines for it 15:39 < bitcoin272> i hear you 15:39 < bitcoin272> well 15:39 < sipa> as that obviously doesn't guarantee that the old one won't confirm still if it went out to the network 15:39 < bitcoin272> this has been a fun day, lol 15:39 < bitcoin272> right 15:39 < sipa> there is work on a bumpfee command which will use bip125 to increase a txn's fee, and replace it with another one 15:40 < bitcoin272> so, the biggest issue with this ancestor/descendant stuff is that the wallet doesn't keep count of how many are chained when doing input selection 15:40 < sipa> indeed 15:40 < bitcoin272> and sendtoaddress fails but it enqueues the transaction for broadcast automatically 15:42 < bitcoin272> is there any way to verify what the ancestor limit is 15:42 < bitcoin272> I bumped both limits to 250 15:43 < bitcoin272> just want to know if it's reflected in current config (I restarted daemon) 15:44 < sipa> if you used the correct option name, it should 15:44 < sipa> limitancestorcount=250 15:44 < sipa> in bitcoin.conf 15:48 < sipa> BlueMatt: so there will be the blockprocessing file and the rest? where does ProcessMessage go? 15:49 < BlueMatt> sipa: net_processing 15:49 < BlueMatt> net_processing.cpp and blocktx_processing.cpp 15:49 < BlueMatt> or blockandtxprocessing.cpp 15:49 < sipa> and will there still be main.cpp? 15:49 < BlueMatt> no 15:49 < BlueMatt> (!) 15:49 < sipa> really, is there nothing that doesn't fit either of those 15:50 < BlueMatt> theres some variables declared that dont 15:50 < BlueMatt> but aside from that, not really 15:52 < BlueMatt> FormatStateMessage, GetTransaction, AbortNode are the only things you can argue in that file (outside of declarations) dont fall into mempool/block processing/block disk state management 15:52 < BlueMatt> and all of those arguably do 15:53 < sipa> GetTransaction, LastCommonAncestor, FormatStateMessage, AlertNotify, GetWarnings, 15:53 < gmaxwell> bitcoin272: aside and unrelated to your issues, you should _really_ adjust your workflow to use sendmany. And if you cannot for some reason and must continue to depend on unconfirmed chains you should start paying higher fees than default to assure that they get confirmed relatively quickly. 15:54 < BlueMatt> AlertNotify is used exclusively for fork notification 15:54 < BlueMatt> and GetWarnings mostly is as well 15:54 < sipa> BlueMatt: how about calling it validation.cpp? 15:54 < bitcoin272> gmaxwell: understood, thank you 15:54 < BlueMatt> ok! 15:54 < sipa> BlueMatt: in that it's a bit more generic than just processing... something like TestBlockValidity would go there as well, i guess 15:55 < gmaxwell> bitcoin272: you can also split up coins you deposit into the wallet to reduce the need for chaining 15:55 < bitcoin272> gmaxwell: yup 15:55 < gmaxwell> e.g. if you were going to put 10 BTC in that will likely be paid out in .9 BTC chunks, making your payment of ten is as a sendmany with ten 1 BTC outputs would be prudent. 16:00 -!- aalex [~aalex@64.187.177.58] has quit [Ping timeout: 265 seconds] 16:03 -!- cysm [cysm@unaffiliated/paracyst] has joined #bitcoin-core-dev 16:03 -!- bsm117532 [~mcelrath@38.121.165.30] has joined #bitcoin-core-dev 16:04 -!- Evel-Knievel [~Evel-Knie@d5152f744.static.telenet.be] has quit [Ping timeout: 248 seconds] 16:07 < bitcoin-git> [bitcoin] sipa pushed 6 new commits to master: https://github.com/bitcoin/bitcoin/compare/ad826b3df9f7...dc6dee41f7cf 16:07 < bitcoin-git> bitcoin/master 4a6b1f3 Matt Corallo: Expose AcceptBlockHeader through main.h 16:07 < bitcoin-git> bitcoin/master 63fd101 Matt Corallo: Split ::HEADERS processing into two separate cs_main locks... 16:07 < bitcoin-git> bitcoin/master a8b936d Matt Corallo: Use exposed ProcessNewBlockHeaders from ProcessMessages 16:07 < bitcoin-git> [bitcoin] sipa closed pull request #9183: Final Preparation for main.cpp Split (master...net_processing_5) https://github.com/bitcoin/bitcoin/pull/9183 16:08 -!- jtimon [~quassel@186.31.134.37.dynamic.jazztel.es] has quit [Remote host closed the connection] 16:10 < bitcoin-git> [bitcoin] TheBlueMatt opened pull request #9260: Mrs Peacock in The Library with The Candlestick (killed main.{h,cpp}) (master...net_processing_file) https://github.com/bitcoin/bitcoin/pull/9260 16:10 -!- bsm117532 [~mcelrath@38.121.165.30] has quit [Ping timeout: 265 seconds] 16:10 -!- Evel-Knievel [~Evel-Knie@d5152f744.static.telenet.be] has joined #bitcoin-core-dev 16:11 < sipa> BlueMatt: haha 16:11 < BlueMatt> did you see the pr text? 16:12 < sipa> yes 16:12 -!- alpalp [~allen@unaffiliated/alpalp] has joined #bitcoin-core-dev 16:16 < bitcoin272> hey guys I'm curious, why was 25 chosen for the ancestor count? 16:18 < BlueMatt> ugh, git isnt smart enough to realize when you rename a file and then create a dumb #include "newfile.h" is a move :( 16:19 < Eliel> BlueMatt: it should understand it if you do the rename with git mv 16:19 < sipa> BlueMatt: where does it not realize this? 16:20 < BlueMatt> main.h -> validation.h; main.h == #include "validation.h" 16:20 < BlueMatt> because i dont want to touch literally every file 16:20 < sipa> i don't understand what the problem is 16:21 < BlueMatt> will make rebase of ~everything impossible 16:21 < BlueMatt> git is smart when rebasing/merging across moves otherwise 16:22 < BlueMatt> Eliel: no, it doesnt cache that info 16:22 < Eliel> if it won't understand it in a single commit, try first renaming and then creating the new file. 16:22 < BlueMatt> it regenerates it itself, so there is no place for it to figure it out 16:22 < Eliel> in two commits 16:22 < BlueMatt> yea 16:26 -!- bsm117532 [~mcelrath@38.121.165.30] has joined #bitcoin-core-dev 16:36 -!- justanotheruser is now known as Hismione 16:40 -!- bitcoin272 [369924af@gateway/web/freenode/ip.54.153.36.175] has quit [Ping timeout: 260 seconds] 16:41 < sipa> BlueMatt: does it still do this tracking after a merge commit is introduced around the two commits? 16:42 < sipa> maybe it treats it as one then 16:42 < BlueMatt> sipa: I have no idea... 16:42 < gmaxwell> bitcoin272: measurements on the actual network, combined with the compute time created for longer chains (They're more expensive to process). 16:42 < sipa> BlueMatt: can you check? if not, it's probably not worth deviating from the "every commit needs to compile and run tests" policy 16:43 < BlueMatt> willdo 16:43 < sipa> actually, i'll check myself - i want to know 16:43 < BlueMatt> heh, ok 16:43 * BlueMatt is currently checking which files still dont compile with 1GB of memory in kvm.... 16:44 < sipa> good 16:45 < BlueMatt> lots of them, it looks like :( 16:45 < sipa> BlueMatt: doesn't seem to work 16:45 < BlueMatt> across a merge? 16:45 < BlueMatt> damn git 16:46 < BlueMatt> does rebase work at all across a rename? merge does, but rebase might now 16:46 < BlueMatt> not, actually 16:46 < Eliel> maybe you can make it a symlink instead? 16:46 < BlueMatt> eww 16:46 < BlueMatt> I think i might rather sed all the files 16:46 < sipa> i tried rebasing #8580 on top of a merged 9260 (with --no-ff) 16:46 < gribble> https://github.com/bitcoin/bitcoin/issues/8580 | Make CTransaction actually immutable by sipa · Pull Request #8580 · bitcoin/bitcoin · GitHub 16:46 < sipa> and it conflicts in main.h 16:47 < BlueMatt> damn 16:47 < sipa> like... the whole file is a conflict block 16:47 < BlueMatt> ok, well I guess its gonna be rebase-hell either way.... 16:47 < sipa> we could choose to leave one of the two named main.h/.cpp 16:47 < BlueMatt> yea... 16:47 < sipa> which at least would make rebases that touch the not-moved-out part work 16:48 < BlueMatt> i mean could leave it as main.h and change the #define in main.h to VALIDATION and add a TODO thats like "move this shits" 16:48 < BlueMatt> then do it like when we close merge window so that its easier 16:48 < Eliel> would symlink cause more trouble than it'd solve? 16:48 < BlueMatt> alternatively, I could actually go sed the entire codebase 16:48 < BlueMatt> Eliel: I dont think it'd cause trouble since we have the #define guards 16:49 < BlueMatt> but its super ugly 16:49 < BlueMatt> (and, actually, git might not track that, either?) 16:49 < Eliel> right, true 16:49 -!- alpalp [~allen@unaffiliated/alpalp] has quit [Ping timeout: 256 seconds] 16:49 < Eliel> but yes, I think sedding the whole codebase is best 16:49 < sipa> validation.h is much larger than net_processing.h, so i'd suggest having main.h and net_processing.h, rather than validation.h and main.h 16:49 < BlueMatt> yea 16:52 < sipa> so just remove the last two commits? 16:53 < BlueMatt> sipa: I went ahead and did as you suggested and left main.cpp moved to validation.cpp, and just added a TODO to main.h to move it 16:54 < sipa> ah, i'd just have left validation.cpp to be main.cpp 16:54 < sipa> that move can easily be do at the same time as the .h rename 16:55 < BlueMatt> welllll....i mean sed wont cuase (m)any rebase issues........ 16:56 < sipa> yes, but it's a weird state to have main.h but validation.cpp 16:56 < sipa> and there is no need to that move early 16:56 < BlueMatt> there is also no need to wait on the wholesale main/validation move/sed 16:56 < BlueMatt> :p 16:57 < sipa> well, so either do the whole thing now, or not at all 16:57 < BlueMatt> I'm doing it now :) 17:24 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has quit [Read error: Connection reset by peer] 17:29 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 258 seconds] 17:42 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 17:50 < phantomcircuit> sipa, after looking at 8831 again i can see why you wanted to not have the ReadKeyValue logic in CWallet 17:50 < phantomcircuit> im not sure a class with virtual functions is going to be enough either though 17:52 -!- mrkent [~textual@unaffiliated/mrkent] has quit [] 17:54 -!- abpa [~abpa@96-82-80-25-static.hfc.comcastbusiness.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 18:11 < phantomcircuit> sipa, i could just add a bunch of methods to CWallet like LoadName LoadPurpose etc 18:17 -!- rafalcpp [~racalcppp@84-10-11-234.static.chello.pl] has quit [Read error: Connection reset by peer] 18:17 -!- rafalcpp [~racalcppp@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev 18:25 < morcos> sipa: for bitcoin272's problem, the wallet doesn't rebroadcast things that aren't in its own mempool correct? so is his node restarting (which would then maybe accept some of the later chain txs) or something else? 18:27 < sipa> morcos: uh, right 18:28 < morcos> gmaxwell: in the code that doesn't store an orphan if its parent was rejected... are we sure that's a good idea? 18:29 < morcos> if a parent had too low fee 18:29 < morcos> it is entirely possible that the parent + orphan would then stay in the mempool 18:29 < gmaxwell> morcos: well we don't have the parent now and won't request it so the orphan will not get resolved. We might want to keep it around for BIP152 but right now BIP152 makes NO use of the orphan pool.. 18:30 < morcos> what do you mean we won't request it? 18:30 < gmaxwell> while it's in the reject filter we won't request it. 18:31 < gmaxwell> ('cause thats what the reject filter does) 18:31 < morcos> argh, thats just kind of an accident about the way alreadyhave works though right? 18:31 < gmaxwell> Am I confused? Highly likely... I have a cold. 18:32 < gmaxwell> Well I thought that was the intent-- we don't want to request things we 'know' we will just reject. 18:32 < morcos> i mean in the orphan processing code we're specifically requesting the parents, but you're right we "AlreadyHave" things that are recentrejects 18:32 < gmaxwell> Yea, we could request it again if the orphan's fee was high for example and maybe then they both could be accepted.. would make ancestor feerate mining work better. 18:33 < morcos> yeah, but i guess i'm drawing a distinction between requesting txs in response to inv's (in which case you dont' want them if you recentlyrejected) 18:33 < morcos> and requesting them as a parent, in which case, maybe you want to try again 18:34 < morcos> in fact, we could almost bypass the mempool minfee check altogether since most of our recent peers won't send us stuff that violates it anyway and then it would basically just work 18:34 < morcos> but i guess we can't quite do that 18:35 < gmaxwell> I'd like rejection to work differently, putting rejected things in a datastructure that works like the new sigcache open-hashtable where they're taged for eager eviction but kept around until actually evicted... and then BIP152 can use that pool, and orphan resolution can use it and so on. 18:35 < morcos> does that work for non-POD 18:36 < gmaxwell> 'works like' I don't mean the lockless aspects of it either, but just the fact that there is a generation/deleted flag. 18:37 < morcos> ok.. i think i'll still PR that takes non-stored orphans with rejected parents and adds them to rejects, but just comment that we might want to remove that if we fix up the rest. i think its strictly better than existing 18:37 < gmaxwell> the principle is that we already took the cost of transfering the data, we should keep as of the most useful 'useless' stuff as we can afford... in case it turns up in a block or as a parent. 18:38 < gmaxwell> But I dunno if it's better to spend time working on that or mempool sync. 18:39 < morcos> gmaxwell: arghh.. you guys and your mempool sync.. i tend to like the other methods better.. but BlueMatt was trying to convince me nothing can match the privacy of mempool sync 18:39 < gmaxwell> hah 18:39 < BlueMatt> I'm still a fan 18:40 < gmaxwell> well perhaps I'm also chasing it because in theory it's possible to get pretty close to optimal bandwidth efficiency and today we waste a lot of bandwidth on relay. (though it's gotten incrementally better in the last several releases) 18:41 < gmaxwell> but it's easy to venture into overdesign. 18:41 < gmaxwell> OTOH, Fibre exists and is pretty awesome. 18:42 < gmaxwell> and I could have also said that it was a crazy excercise in chasing optimality. 18:43 < morcos> we should set up some data-collecting nodes that see how much better our block reconstruction would be if we'd kept everything we were told about 18:43 < BlueMatt> lol, by adding #include to fix windows build error, net_processing.cpp ticked over the 1GB-memory-usage mark :( 18:43 < BlueMatt> fucking boost 18:44 < BlueMatt> gmaxwell: I'm more excited by better relay privacy 18:44 < gmaxwell> morcos: I've been monitoring a bit of that on and off. 18:44 < sipa> BlueMatt: i think more recent gccs have lower memory usage? :0 18:44 < gmaxwell> actually I find a lot of the blocks that are almost perfectly reconstucted miss due to replacements / doublespends. 18:45 < morcos> gmaxwell: how much of the gap can you close? 18:45 < BlueMatt> sipa: I'm sure, this is what was in default-debian on digitalocean 18:45 < morcos> replacements/doublespends that you heard about? 18:45 < gmaxwell> Yes. 18:45 < morcos> but weren't RBF? 18:45 < BlueMatt> huh, can anyone reproduce travis' hangs on #9260 18:45 < morcos> why didnt you have them? 18:45 < BlueMatt> i cant... 18:45 < gribble> https://github.com/bitcoin/bitcoin/issues/9260 | Mrs Peacock in The Library with The Candlestick (killed main.{h,cpp}) by TheBlueMatt · Pull Request #9260 · bitcoin/bitcoin · GitHub 18:45 < gmaxwell> As in I replaced the transaction, and the block confirmed the other (presumably the miner didn't support replacement or it didn't make itt othem yet) 18:45 < gmaxwell> morcos: some were RBF. 18:46 < gmaxwell> Though I did see a doublespend that wasn't at least once. 18:46 < BlueMatt> oh, maybe I can 18:46 < morcos> yeah, so thats a good use of your eager eviction 18:46 < gmaxwell> morcos: if you have debug=1 on, and the BIP152 missed by only a few txids it will log the missing txids and you can check your logs to see if you'd previously heard them. 18:47 < morcos> gmaxwell: ha, even easier than i thought 18:47 < BlueMatt> oops lol 18:47 < gmaxwell> during the period I last looked this was the overwhelming majority of blocks that missed only a couple. But there is a lot of variance since it depends on miner policy... 18:47 < gmaxwell> morcos: I think 'few' might only be <5 so you might want to turn that up. 18:48 < gmaxwell> The purpose of that logging was to explore the impacts of prefill policies, and we wouldn't ever prefill more than a couple. 18:48 < sipa> yup, 5 18:48 < sipa> <=5 18:48 < gmaxwell> (I don't think prefill is worth working on until we eliminate all the preventable misses) 18:49 < gmaxwell> (e.g. by using the orpan pool and by keeping at least replacements around in some kind of pool...) 18:49 -!- abpa [~abpa@2602:306:b837:dbf0:10d2:337:7ea0:6306] has joined #bitcoin-core-dev 18:49 -!- rafalcpp [~racalcppp@84-10-11-234.static.chello.pl] has quit [Read error: Connection reset by peer] 18:49 -!- rafalcpp [~racalcppp@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev 18:50 < morcos> sdaftuar and i were saying that we might also want to have a super-HB peer among our 3 HB peers, as you maybe don't want 3 peers prefilling for you 18:50 < morcos> for instance one kind of obvious prefill policy is if the tx is non-standard (such as >100k) prefill, but then thats big 18:50 < gmaxwell> yes, though it's little enough data that it probably doesn't matter. The spec recommends you don't prefill more than 10kb in total. 18:53 < gmaxwell> morcos: of course, there is also the Fibre technique where you send FEC data... and then all the peers could usefully contribute. :P 18:54 < gmaxwell> We also don't have a way to NAK a HB request-- I reasoned that this was okay because even if every peer requests HB from you... you still end up not really any worse than relaying a full block to a single peer. Same kind of thinking applies for excessive prefill. 18:54 < morcos> yeah we should probably right a new mining api first. :) 18:54 < morcos> s/right/write/ 18:54 < gmaxwell> Yea, no kidding. 18:56 < gmaxwell> I've also thought that we should probably not be using HB mode at all unless we have inbound connections or we're mining (or we've been asked by the user). but that kind of complexity also got answered with 'the overhead is irrelevant'. 19:11 -!- jtimon [~quassel@186.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 19:12 * jtimon rebased #8855 again 19:12 < gribble> https://github.com/bitcoin/bitcoin/issues/8855 | Use a proper factory for creating chainparams by jtimon · Pull Request #8855 · bitcoin/bitcoin · GitHub 19:12 < BlueMatt> sipa: even with gcc 6.2 net_processing ticks over 1GB (incl host) 19:15 < gmaxwell> :( 19:15 < gmaxwell> BlueMatt: you have failed at main splitting! :) 19:15 < jtimon> intirestingly enough, with +15-22 in main.cpp, #8498 doesn't need rebase since sep1 19:15 < gribble> https://github.com/bitcoin/bitcoin/issues/8498 | Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub 19:15 < BlueMatt> clearly 19:16 < BlueMatt> to be fair, so does init 19:16 < jtimon> what did I missed? 19:17 * jtimon went to the logs 19:18 < bitcoin-git> [bitcoin] morcos opened pull request #9261: Add unstored orphans with rejected parents to recentRejects (master...orphanRejects) https://github.com/bitcoin/bitcoin/pull/9261 19:26 < jtimon> BlueMatt: not renaming main to validation in the same PR would make it simpler to review 19:27 < BlueMatt> jtimon: that pr should be easy to review 19:27 < BlueMatt> the main.cpp rename is a clean rename, so that especially 19:29 < jtimon> I guess I'm not enthusiastic about renaming main in general, I believe it still does a lot of things beyond consensus validation, plus it still includes most globals 19:29 < jtimon> but will keep looking 19:31 < jtimon> but yeah, review it's not a good argument 19:34 < BlueMatt> jtimon: see scrollback, there are ~no functions which are not block/tx processing left in validation.cpp 19:34 < BlueMatt> jtimon: except, yes, globals 19:36 < jtimon> https://github.com/bitcoin/bitcoin/pull/9260#issuecomment-264365400 19:36 < BlueMatt> jtimon: what /did/ you review? 19:43 < jtimon> commit by commit, if the moveonlys are moveonlys (ie https://github.com/bitcoin/bitcoin/pull/9260/commits/84922e4bf4c38227fbbbede50e09c87fe2a5c7f0 ) and what you say in https://github.com/bitcoin/bitcoin/pull/9260/commits/87c35f584397e2309970afdcca8e03731a86639e is true, everything seems fine or it shouldn't compile, to give an utACK I would need to grep mapOrphanTransactions and mapOrphanTransactionsByPrev and verify the 19:43 < jtimon> moveonlies 19:46 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has quit [Quit: Leaving.] 19:54 < jtimon> re rename right, git knows the file is renamed but you eat the include changes which I agree is not hard to review 19:55 -!- dermoth [~thomas@dsl-66-36-132-180.mtl.aei.ca] has quit [Ping timeout: 258 seconds] 19:57 < wumpus> cfields: I thought about this global version/context parameters thing in RPC a bit, and there's several other potentially controversial solutions that are used for JSON API versioning we could consider as well: an alternative (say "v2") entry point, or using a HTTP header. An advantage would be that these are conceptually separate from normal arguments 19:58 < wumpus> cfields: I'm not specifically aiming 8811 for any release but I'd sure hope it can be merged before 0.14 19:58 < wumpus> cfields: as it changes all the RPC dispatch tables it's pretty annoying to keep up to date 19:59 < wumpus> cfields: I just wish people would test it a bit 20:02 < wumpus> cfields: regarding testing it: maybe it would make sense to port at least one of the RPC tests to fully using named arguments? currently it adds a test but that one is specifically for the named-arguments-parsing functionality 20:05 < jtimon> sipa: BlueMatt: if we're doing file renames, maybe we can move ahead with some in https://github.com/bitcoin/bitcoin/pull/8328 (those people can agree on) 20:05 -!- dermoth [~thomas@dsl-66-36-132-180.mtl.aei.ca] has joined #bitcoin-core-dev 20:09 < bitcoin-git> [bitcoin] instagibbs opened pull request #9262: Don't consider coins available if too many ancestors in mempool (master...toolong) https://github.com/bitcoin/bitcoin/pull/9262 20:37 -!- fanquake [~fanquake@unaffiliated/fanquake] has joined #bitcoin-core-dev 20:47 -!- juscamarena [~justin@47.148.176.74] has quit [Quit: leaving] 20:48 -!- CubicEarth [~cubiceart@c-50-159-126-21.hsd1.wa.comcast.net] has quit [Remote host closed the connection] 20:49 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/dc6dee41f7cf...c4d22f6eb216 20:49 < bitcoin-git> bitcoin/master 10ae7a7 Matt Corallo: Revert "Use async name resolving to improve net thread responsiveness"... 20:49 < bitcoin-git> bitcoin/master c4d22f6 Wladimir J. van der Laan: Merge #9229: Remove calls to getaddrinfo_a... 20:49 < bitcoin-git> [bitcoin] laanwj closed pull request #9229: Remove calls to getaddrinfo_a (master...2016-11-gai) https://github.com/bitcoin/bitcoin/pull/9229 20:49 -!- CubicEarth [~cubiceart@2002:329f:7e15:0:d6d:f992:60a4:689a] has joined #bitcoin-core-dev 20:49 < wumpus> btw flagging things as "needs backport" with "0.14" doesn't make much sense :-) 20:51 < bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.13: https://github.com/bitcoin/bitcoin/commit/b172377857f9b5a0b2f43e0e57be9acf82a6c50a 20:51 < bitcoin-git> bitcoin/0.13 b172377 Matt Corallo: Revert "Use async name resolving to improve net thread responsiveness"... 20:52 < sipa> wumpus: i merged a few things that still need backporting 20:53 < wumpus> sipa: that's fine, we should probably do that in a pull grouping them together 20:53 < wumpus> sipa: I just have a really bad feeling about the async resolving thing so backported that immediately 20:54 -!- CubicEarth [~cubiceart@2002:329f:7e15:0:d6d:f992:60a4:689a] has quit [Ping timeout: 258 seconds] 20:55 < wumpus> sipa: I suppose they're all in this list? https://github.com/bitcoin/bitcoin/pulls?q=is%3Apr+label%3A%22Needs+backport%22+is%3Aclosed+milestone%3A0.13.2 20:55 < wumpus> if not they should be labeled "Needs backport" with milestone 0.13.2 20:56 < sipa> wumpus: yes 21:00 < wumpus> heh this is the code in libc where it crashes: https://0bin.net/paste/V1n0GkHdlDatZrnO#N5htO2+DbXw1EtNNKz4oB-3ykuixE4KGJTLiZ56/V9L to be specific: the if (--waitlist) line 21:00 < wumpus> mind the "This is tricky. See getaddrinfo_a.c for the reason why this works." comment :) 21:02 < sipa> that does not look thread safe at all 21:02 < Lightsword> wumpus, that’s not going to be externally exploitable is it for things that uses getaddrinfo? 21:03 < wumpus> Lightsword: I don't *think* so, but it depends on what kind of bug this turns out to be 21:03 < Lightsword> has upstream confirmed? 21:05 < wumpus> it's not triggered by any specific data the remote is sending, if that's what you're worried about 21:06 < wumpus> also it's not possible for P2P clients to make bitcoind do DNS lookups 21:06 < wumpus> so no, I don't think this is a security issue for us, but you can't be too careful right... 21:08 < wumpus> it's a confirmed use after free, not a NULL pointer dereference 21:09 < fanquake> When are we planning on releasing 0.13.2 ? Looks like 5 backports left. 21:09 < wumpus> it's doing "0x7ffff79b77f0 <__gai_notify+48> subl $0x1,(%rdx)" where rdx is, say, 0x441f0fc3c08944, then if you try to inspect that "0x441f0fc3c08944: Cannot access memory at address 0x441f0fc3c08944" -- too bad, already unmapped 21:09 < fanquake> *the first rc of 0.13.2 21:09 < wumpus> Lightsword: sort of https://sourceware.org/bugzilla/show_bug.cgi?id=20874 21:11 < wumpus> fanquake: well yesterday in the meeting there was agreement that all the 0.13.2 backports are labeled - so after these are backported rc1 can be released any time 21:13 < fanquake> Ok. I need to start attending the meetings, but difficult with time-diff though. 21:13 < wumpus> yes it's not a good time for australia/most of asia 21:14 < fanquake> Also, not sure why I said 5 PRs left, there is only 9239, 9194 and 9191 (which includes multiple, mostly test-related fixes). 21:15 < fanquake> It'll be right. Always have the logs, just need to make time for reading them. I think someone also posts a meeting summary to reddit or something. 21:21 < wumpus> not just to reddit :) https://bitcoincore.org/en/meetings/2016/10/20/ 21:34 < wumpus> anyhow we could, say, alternate between two meeting times if there is a lot of demand for people from asia/australia to attend the meetings. But I've never really got that impression. 21:36 < sipa> the meeting is on SHA256(days_since_1970) % 24 UTC. 21:38 < gmaxwell> I ws thinking of suggesting alternating but then though that there probably isn't a good time for both asia and europe. 21:38 < wumpus> sipa: theoretically that would be fair, in practice if you don't take the distribution of people over timezones into account, you may end up with a time that's only good for people who live in the middle of the atlantic ocean :-) 21:39 < wumpus> gmaxwell: right I don't think there is 21:39 < sipa> gmaxwell: 8 am UTC would be relatively good for both asia and europe 21:40 < wumpus> 8 am UTC would be quite a good time for me 21:41 < sipa> (9am in western europe, 4pm in hong kong, midnight on west coast) 21:41 < sipa> 3am on east coast, however 21:41 < gmaxwell> thats too late for east coast US though I don't know if we currently have any eastcoasters. 21:41 < gmaxwell> oh instagibbs is duh. 21:41 < sipa> morcos, suhas, instagibbs 21:41 < gmaxwell> oh double duh. 21:41 < wumpus> in any case every time would be bad for someone - so to motivate a (even bi-weekly) time change there has to be enough demand 21:41 < gmaxwell> I just think of NY as existing in a parallel universe. 21:42 < wumpus> e.g. just rebroad asking for it is not enough :p 21:42 < gmaxwell> well we lose jl2012 due to this. 21:42 < wumpus> yes and fanquake 21:42 -!- ThomasV [~ThomasV@unaffiliated/thomasv] has joined #bitcoin-core-dev 21:43 < BlueMatt> lol, google has a thing where they will hook up your project to run in fuzzing on some machiens 24/7, except to do it you have to agree that they will announce the bug to the world 7 days after you release a fix 21:44 < BlueMatt> who in their right mind would agree to that? thats like stupid high-risk for users 21:45 < wumpus> it could make sense for software that auto-updates quickly and automatically 21:45 < wumpus> but no, certainly not for bitcoin 21:46 < BlueMatt> I mean they're talking about it for "critical infrastructure" (ie common libraries) 21:47 < BlueMatt> like, sure, maybe google updates their libcs quickly, but the vast majority of folks do not at all 21:47 < sipa> 4pm UTC: 8am westcoast, 11am eastcoast, 5pm europe, midnight hong kong 21:48 < wumpus> BlueMatt: for libraries it's much harder to say, agreed, there will be tons of especially embedded platforms that never update them at all 21:49 < wumpus> then again that's not google's fault - finding the vulnerabilities before the 'bad people' do is still important 21:49 < BlueMatt> wumpus: sure, but that doesnt mean you announce them publicly 7 days after the first release with the fix 21:50 < wumpus> (or alternatively, after the "bad people" have already used them for years to get access anwyay...) 21:50 < BlueMatt> yes, you fix quickly, and then announce it much later 21:50 < gmaxwell> BlueMatt: it's fine for projects that are alread well fuzzed, I suppose. 21:51 < gmaxwell> NSA already found those vulns last week anyways. 21:51 < BlueMatt> it might be better than /not/ doing the fuzzing, but its still a super shitty policy 21:51 < wumpus> it's the old responsible disclosure discussion 21:51 -!- Hismione is now known as justanotheruser 21:52 < BlueMatt> wumpus: responsible disclosure (usually) is a discussion about what to do when upstream tells you to fuck off, not what to do when upstream is responsive 21:53 < wumpus> BlueMatt: sure, though part of it is how long to wait with public announcement 21:53 < BlueMatt> suresure, but who the fuck ever suggested 7 days? 22:01 < midnightmagic> responsible disclosure is using a timeline which is fair both to the public who is affected by the bug and the vendor, without unfairly prioritizing one over the other. 22:02 < midnightmagic> The timelines *were once* measured in weeks, since the public good of disclosure was more important than protecting the financial interests of a corporation who wasn't remunerating the researcher. 22:05 < BlueMatt> midnightmagic: yes, but one week? fuck people who just blindly update and would be protected wouldnt even get protected if you only wait 7 days 22:06 < BlueMatt> midnightmagic: I mean I'm with you....fuck the vendor and their financial interests, users must be protected, but 7 days is short enough that you're harming users, too 22:07 < midnightmagic> 7 days is pretty sure. At least Gobbles and/or that italian ultra-prolific guy isn't just dropping 0-days and letting the vendors scramble. 22:07 < midnightmagic> *pretty short 22:10 * midnightmagic tries to remember that italian guy again.. 22:12 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-viadtpqugiemcugz] has quit [Quit: Connection closed for inactivity] 22:13 < midnightmagic> ah, here he is. I've come to appreciate the way he does things: http://aluigi.altervista.org/ 22:13 < midnightmagic> (not even vendor notification.) 22:23 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-iwdwktcinbcydgue] has joined #bitcoin-core-dev 22:44 -!- cryptapus [~cryptapus@unaffiliated/cryptapus] has joined #bitcoin-core-dev 22:45 -!- cryptapus_afk [~cryptapus@unaffiliated/cryptapus] has quit [Ping timeout: 250 seconds] 23:00 -!- dermoth [~thomas@dsl-66-36-132-180.mtl.aei.ca] has quit [Read error: Connection reset by peer] 23:00 -!- dermoth [~thomas@dsl-66-36-132-180.mtl.aei.ca] has joined #bitcoin-core-dev 23:02 -!- ThomasV [~ThomasV@unaffiliated/thomasv] has quit [Ping timeout: 250 seconds] 23:03 < bitcoin-git> [bitcoin] luke-jr opened pull request #9263: release notes: Only free transactions are being removed, not priority. (master...relnotes_freetxn) https://github.com/bitcoin/bitcoin/pull/9263 23:05 -!- RoyceX [~x@208.167.254.25] has joined #bitcoin-core-dev 23:05 -!- RoyceX [~x@208.167.254.25] has quit [Changing host] 23:05 -!- RoyceX [~x@unaffiliated/cheeseo] has joined #bitcoin-core-dev 23:08 -!- Cheeseo [~x@unaffiliated/cheeseo] has quit [Ping timeout: 268 seconds] 23:09 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Ping timeout: 250 seconds] 23:10 -!- aalex [~aalex@64.187.177.58] has joined #bitcoin-core-dev 23:12 < bitcoin-git> [bitcoin] laanwj opened pull request #9264: 0.13.2 backports (0.13...2016_12_backports_0_13) https://github.com/bitcoin/bitcoin/pull/9264 23:16 < jonasschnelli> What is preferable: two keypools one for change, one for external OR a keypool with keys flagged for internal or external use? 23:17 < bitcoin-git> [bitcoin] laanwj closed pull request #9191: [qa] 0.13.2 Backports (0.13...Mf1611-q01302) https://github.com/bitcoin/bitcoin/pull/9191 23:17 < wumpus> possibly the flagging approach is easier to do in a backwards compatible way - old versions can ignore the flags 23:18 -!- aalex [~aalex@64.187.177.58] has quit [Ping timeout: 244 seconds] 23:19 < jonasschnelli> wumpus: good point.. 23:21 < jonasschnelli> for the deserialization (SerializationOp(Stream& s, ...)), if the stream is longer then the acctual READWRITE, it will be ignored? right? (for backward comp.) 23:21 < jtimon> python3 ./qa/pull-tester/rpc-tests.py mempool_packages takes 31s in my computer, would it be crazy to move it from the extended test to the regular ones? 23:21 < bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/c4d22f6eb216...3fbf07926240 23:21 < bitcoin-git> bitcoin/master d824ad0 Alex Morcos: Disable fee estimates for a confirm target of 1 block 23:21 < bitcoin-git> bitcoin/master e878689 Alex Morcos: Make GUI incapable of setting tx confirm target of 1 23:21 < bitcoin-git> bitcoin/master 3fbf079 Wladimir J. van der Laan: Merge #9239: Disable fee estimates for 1 block target... 23:21 < bitcoin-git> [bitcoin] laanwj closed pull request #9239: Disable fee estimates for 1 block target (master...blockstreamtil2blocks) https://github.com/bitcoin/bitcoin/pull/9239 23:25 < luke-jr> wumpus: doh, I was about to do that (more backports) 23:26 < dcousens> BlueMatt: don't miners use priority for their own transactions? 23:26 < fanquake> jtimon takes 51s here 23:26 < dcousens> (not the coinbase, just, "other" transactions) 23:29 < jonasschnelli> wumpus: I could do a BP of #8989 23:29 < gribble> https://github.com/bitcoin/bitcoin/issues/8989 | [Qt] overhaul smart-fee slider, adjust default confirmation target by jonasschnelli · Pull Request #8989 · bitcoin/bitcoin · GitHub 23:29 < wumpus> jonasschnelli: not sure that's what we want though? 23:29 < luke-jr> dcousens: fee delta works fine for that use case 23:29 < wumpus> jonasschnelli: I mean, this is a minor release, how much do we want the GUI to change? 23:29 < jonasschnelli> Yes. It would be a notable change. 23:30 < wumpus> maybe it's ok though in this case I'm not sure 23:30 < jonasschnelli> Can we BP #9239 without the GUI changes? 23:30 < gribble> https://github.com/bitcoin/bitcoin/issues/9239 | Disable fee estimates for 1 block target by morcos · Pull Request #9239 · bitcoin/bitcoin · GitHub 23:30 < jtimon> fanquake: thanks, still, I don't see a consistency in times between extended and non-extended tests, I have a little commit in a long branch that I can cherry pick based only on what makes sense for my computer based only on time, introducing -skipslow that works with or without extended, but it looks a little bit too complicated 23:30 < wumpus> you could reply that in the #9239 topic, probably morcos has an opinoin on it too :) 23:30 < gribble> https://github.com/bitcoin/bitcoin/issues/9239 | Disable fee estimates for 1 block target by morcos · Pull Request #9239 · bitcoin/bitcoin · GitHub 23:31 < jonasschnelli> wumpus: or just disabled (shorten the slider range) for target 1 in the backport 23:31 * wumpus wishes gribble had rate-limiting 23:31 < wumpus> jonasschnelli: yep, exactly 23:31 < wumpus> jonasschnelli: although that may turn out to be a riskier/less-tested solution than #8989 which at least has been tested in master. DIfficult. 23:31 < gribble> https://github.com/bitcoin/bitcoin/issues/8989 | [Qt] overhaul smart-fee slider, adjust default confirmation target by jonasschnelli · Pull Request #8989 · bitcoin/bitcoin · GitHub 23:32 < jonasschnelli> Yes. IMO 8989 and 9239 is sort of one BP "package" 23:32 < wumpus> jonasschnelli: in that case we should just backport both I guess 23:32 < jonasschnelli> Agree 23:33 < jonasschnelli> I can do that next week... 23:33 < jonasschnelli> (if nobody did it in the meantime) 23:34 < wumpus> luke-jr: good that you hadn't started yet, then, would have been a waste of work as some had manual conflicts to resolve (though you could check if you resolved them in the same way :) 23:34 < luke-jr> wumpus: well, I had started.. but I can rebase :x 23:34 < luke-jr> I went through the PRs manually, not using tags, so I probably got a few you missed 23:34 < luke-jr> (well, *almost* manually.. XD) 23:35 < wumpus> well I strictly backport what is tagged, not more, if not it should have been tagged 23:37 < luke-jr> wumpus: erg, your backports aren't on top of Marco's? :x 23:37 < wumpus> luke-jr: no, I forgot about that one, will rebase 23:38 < wumpus> now they are (luckily no new conflicts) 23:41 < luke-jr> woo no conflicts here either it seems 23:42 -!- jannes [~jannes@178.132.211.90] has joined #bitcoin-core-dev 23:57 < bitcoin-git> [bitcoin] laanwj opened pull request #9265: bitcoin-cli: Make error message less confusing (master...2016_12_rpccli_message) https://github.com/bitcoin/bitcoin/pull/9265 23:59 < luke-jr> wumpus: pushed backports-0.13 to my github; want to just pull it into yours?