--- Day changed Thu Dec 22 2016
00:05 -!- jtimon [~quassel@231.110.132.37.dynamic.jazztel.es] has joined #bitcoin-core-dev     
00:22 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
00:57 -!- kallewoof [~kallewoof@zz2014411797B6F89E81.userreverse.dion.ne.jp] has quit [Remote host closed the connection]     
00:57 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has quit [Remote host closed the connection]     
00:58 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has joined #bitcoin-core-dev     
01:03 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has quit [Ping timeout: 248 seconds]     
01:13 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has joined #bitcoin-core-dev     
01:13 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has quit [Read error: Connection reset by peer]     
01:27 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has joined #bitcoin-core-dev     
01:33 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has quit [Ping timeout: 258 seconds]     
01:35 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has joined #bitcoin-core-dev     
01:40 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has quit [Ping timeout: 268 seconds]     
01:45 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer]     
01:46 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
01:48 < btcdrak> wumpus: what's the plan for 0.13.2 - I assume we dont need another RC for the version number thing.
01:50 -!- JackH [~laptop@79-73-186-204.dynamic.dsl.as9105.com] has joined #bitcoin-core-dev     
02:01 < gmaxwell> btcdrak: still three days to christmas, plenty of time for more RCs.
02:02 < rabidus_> :D     
02:03 < btcdrak> "on the day of Christmas, my true love gave to me, a RC in a merkle tree"
02:04 < gmaxwell> man I ran 0.13.2(rc) in valgrind overnight last night on my laptop (which was a day or so behind) and it only synced 111 blocks in about 8 hours.
02:07 < midnightmagic> time to write a custom memory manager
02:08 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has joined #bitcoin-core-dev     
02:12 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has quit [Ping timeout: 246 seconds]     
02:12 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Read error: Connection reset by peer]     
02:13 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has joined #bitcoin-core-dev     
02:16 < bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e8cfe1ee2d01...041331e1da23
02:16 < bitcoin-git> bitcoin/master afe5b3f Anditto Heristyo: Added missing colons in when running help command
02:16 < bitcoin-git> bitcoin/master 041331e MarcoFalke: Merge #9407: [Trivial] Added missing colons in when running help command...
02:16 < bitcoin-git> [bitcoin] MarcoFalke closed pull request #9407: [Trivial] Added missing colons in when running help command (master...minor-style-fixes) https://github.com/bitcoin/bitcoin/pull/9407
02:30 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has joined #bitcoin-core-dev     
02:45 < jonasschnelli> gmaxwell: "synced 111 blocks in about 8 hours" ... hah!
02:45 < jonasschnelli> Whats the result? Any major leaks?
02:47 < gmaxwell> no, course not.. bitcoind is leak free (ignoring the one shot openssl stupidity), I run every release in valgrind and have for a long time.  (QT I wasn't doing that.)
02:48 < gmaxwell> valgrind's leak detection is really not the interesting feature.
02:48 < gmaxwell> the inderesting feature is uninitilized memory access detection.
02:54 < gmaxwell> it's just become increasingly slow, I guess due to transaction traffic growth. :(
02:58 < luke-jr> LLVM's thing is somewhat faster, but.. a hassle to use and broken with some LLVM/kernel pairs :/
03:20 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has joined #bitcoin-core-dev     
03:24 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has quit [Ping timeout: 258 seconds]     
03:29 < bitcoin-git> [bitcoin] MarcoFalke closed pull request #9391: wallet: Remove sendfree (master...Mf1612-015walletSendFreeNONO) https://github.com/bitcoin/bitcoin/pull/9391
03:34 -!- afk11 [~afk11@unaffiliated/afk11] has quit [Read error: Connection reset by peer]     
03:37 -!- afk11 [~afk11@176.61.67.182] has joined #bitcoin-core-dev     
03:37 -!- afk11 [~afk11@176.61.67.182] has quit [Changing host]     
03:37 -!- afk11 [~afk11@unaffiliated/afk11] has joined #bitcoin-core-dev     
03:39 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has left #bitcoin-core-dev []     
03:50 -!- To7 [~theo@cpe-158-222-222-232.nyc.res.rr.com] has quit [Quit: Whatever]     
04:12 -!- sanada [sanada@36-2-119-80.chiba.ap.gmo-isp.jp] has quit []     
04:14 < wumpus> btcdrak: that's mostly an academic concern, how large is the chance that no issues come up with rc1 :) but if none happen, yeah I tend to agree, version bump can be part of final
04:16 < wumpus> btw what about the meetings around xmas? I won't be able to attend this week's and next one - should I cancel them or just find another chair?
04:18 < gmaxwell> You mean today's? for this weeks'?  well we can always chat with whomever shows up, meeting or not.
04:18 < wumpus> yes     
04:19 < gmaxwell> Hotel Ircfornia. You can check-out any time you like but you can never leave.
04:19 < wumpus> but some people may be making time or getting up early for it or I dunno, if almost no one turns up that may be wasted effort
04:20 < gmaxwell> I'll be around.
04:21 < wumpus> ok, I guess you can be chair then :)
04:21 < gmaxwell> k     
04:26 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has joined #bitcoin-core-dev     
04:30 -!- ratoder [~ratoder@static.111.19.201.138.clients.your-server.de] has quit [Remote host closed the connection]     
04:31 -!- anditto [~anditto@zz2014411797B6F89E81.userreverse.dion.ne.jp] has quit [Remote host closed the connection]     
04:44 -!- dermoth_ [~thomas@dsl-66-36-136-48.mtl.aei.ca] has joined #bitcoin-core-dev     
04:44 -!- dermoth [~thomas@dsl-216-221-50-195.mtl.aei.ca] has quit [Disconnected by services]     
04:44 -!- dermoth_ is now known as dermoth     
04:55 -!- jtimon [~quassel@231.110.132.37.dynamic.jazztel.es] has quit [Ping timeout: 264 seconds]     
04:58 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has joined #bitcoin-core-dev     
05:03 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has quit [Ping timeout: 258 seconds]     
05:05 -!- Sosumi [~Leon@bl10-113-190.dsl.telepac.pt] has joined #bitcoin-core-dev     
05:06 -!- alpalp [~alpalp@unaffiliated/alpalp] has joined #bitcoin-core-dev     
05:38 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev     
05:38 -!- alpalp [~alpalp@unaffiliated/alpalp] has quit [Ping timeout: 250 seconds]     
05:39 -!- jtimon [~quassel@231.110.132.37.dynamic.jazztel.es] has joined #bitcoin-core-dev     
05:40 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit]     
05:44 < morcos> instagibbs: heard you were thinking about allowing multiple outstanding block requests... 
05:45 < morcos> this is an idea of how to do it:  https://gist.github.com/morcos/c70ca4f4f3cedf26f1d37b324933afad
05:46 < morcos> havent thought through yet how it might be implemented
05:47 < morcos> i'm not sure that's 100% clear but there are a couple of different changes there
05:48 < morcos> i think its kind of silly that we ignore cmpctblocks we receive from HB peers (if they don't immediately reconstruct) if we have an outstanding request for a cmpctblock from a LB peer.
05:50 < instagibbs> trying to parse it now, it seems related to what I'm trying
05:51 < instagibbs> Right now all I've attempted is to reply to a CMPCTBLOCK while the same block is in flight from another peer, responding up to 1 non-marked-as-hb peer, and responding to any marked-as-hb peer.
05:52 < instagibbs> meaning if marked-as-hb is first, i wont respond to not-marked-as-hb peer
05:52 < morcos> but you will respond to multiple hb peers?
05:53 < morcos> i do think that if the first peer that sends you something is an HB peer, you should still request a cmpct block from a LB peer if they are the next one to inform you (via header)
05:54 < instagibbs> Well, as-is yes, but I can make this "no" just as easily
05:55 < morcos> basically i think if you consider a getdata BLOCKTXN or getdata BLOCK a second stage request, and a getdata CMPCTBLOCK a first stage request
05:56 < morcos> you should be willing to always make sure you have at least two requests at as a high stage as you have opportunity for (except not 2 full BLOCK)
05:57 < instagibbs> high stage? first? second? sorry this terminology is putting me in loops
05:57 < morcos> eh, everytime i try to describe it even i don't think its clear
05:57 < instagibbs> yeah...
05:57 < morcos> i'll try another approach
05:58 < instagibbs> Might be easier to talk about motivation, what we're gaining and avoiding
06:01 < morcos> sorry i'm trying to build a spreadsheet, ha
06:02 < instagibbs> oh jeez :)
06:02 < morcos> my goal is to make it so that no one slow peer can stop one fast peer from getting the block to you as quickly as possible
06:02 < morcos> as long as that fast peer is at least a compact block type (but he might be LB)
06:03 < instagibbs> Agreed, my patch supposedly does this
06:24 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer]     
06:24 -!- BashCo_ [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
06:30 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev     
06:44 -!- echonaut [~echonaut@46.101.192.134] has joined #bitcoin-core-dev     
06:55 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has joined #bitcoin-core-dev     
07:05 -!- molz [~molly@unaffiliated/molly] has joined #bitcoin-core-dev     
07:07 -!- instagibbs [~instagibb@pool-100-15-114-3.washdc.fios.verizon.net] has quit [Ping timeout: 250 seconds]     
07:08 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 258 seconds]     
07:10 -!- instagibbs [~instagibb@pool-100-15-114-3.washdc.fios.verizon.net] has joined #bitcoin-core-dev     
07:22 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
07:24 -!- BashCo_ [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 258 seconds]     
07:27 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection]     
07:28 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
07:32 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 264 seconds]     
08:07 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev     
08:13 -!- jtimon [~quassel@231.110.132.37.dynamic.jazztel.es] has quit [Remote host closed the connection]     
08:29 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev     
09:00 -!- abpa [~abpa@96-82-80-25-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev     
09:03 -!- grim210 [~grim210@137.59.122.2] has joined #bitcoin-core-dev     
09:21 -!- Eliel [~jojkaart@104-250-47-212.rev.cloud.scaleway.com] has quit [Ping timeout: 265 seconds]     
09:21 -!- wasi [~wasi@gateway/tor-sasl/wasi] has quit [Ping timeout: 245 seconds]     
09:32 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has joined #bitcoin-core-dev     
09:34 -!- wasi [~wasi@gateway/tor-sasl/wasi] has joined #bitcoin-core-dev     
09:37 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has quit [Ping timeout: 258 seconds]     
09:38 -!- Eliel [~jojkaart@104-250-47-212.rev.cloud.scaleway.com] has joined #bitcoin-core-dev     
09:40 -!- Atomicat_ [~Atomicat@unaffiliated/atomicat] has joined #bitcoin-core-dev     
09:41 -!- Atomicat [~Atomicat@unaffiliated/atomicat] has quit [Ping timeout: 250 seconds]     
09:41 -!- Atomicat_ is now known as Atomicat     
09:56 -!- grim210 [~grim210@137.59.122.2] has quit [Quit: Leaving]     
10:02 -!- _mn3monic [~guido@176.9.68.68] has quit [Ping timeout: 252 seconds]     
10:03 -!- _mn3monic [~guido@176.9.68.68] has joined #bitcoin-core-dev     
10:03 -!- To7 [~theo@cpe-158-222-222-232.nyc.res.rr.com] has joined #bitcoin-core-dev     
10:08 -!- adiabat [~adiabat@67.205.158.84] has quit [Remote host closed the connection]     
10:10 -!- adiabat [~adiabat@67.205.158.84] has joined #bitcoin-core-dev     
10:33 < dgenr8> morcos: does core try to ensure somehow that incoming txes paying the wallet are not dumped by mempool limiting?
10:36 < gmaxwell> dgenr8: no doing so would be bad for privacy and reliablity (making the wallet think transactions were likely in other peoples mempools when they are not), and isn't needed.
10:43 < dgenr8> gmaxwell: thanks
10:50 -!- MarcoFalke_web [8af602fa@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.250] has joined #bitcoin-core-dev     
10:58 -!- TomMc [~tom@unaffiliated/tommc] has quit [Ping timeout: 250 seconds]     
10:59 < gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
10:59 < jonasschnelli> Hi     
10:59 < btcdrak> hi     
10:59 < CodeShark> hello
11:00 < gmaxwell> #startmeeting
11:00 < btcdrak> ping jl2012
11:00 < kanzure> hi.     
11:00 < phantomcircuit> ping
11:00 < jonasschnelli> gmaxwell: it's meetingstart I guess
11:00 < jl2012> hi     
11:00 < gmaxwell> #meetingstart
11:00 < jonasschnelli> well...
11:00 < CodeShark> I just have one item on my agenda for today's meeting, but we don't have to go over it first
11:00 < gmaxwell> hah the bot isn't here.
11:01 < instagibbs> here
11:01 < gmaxwell> well we don't need it. We can pretend its here.
11:01 < jonasschnelli> Lightningbot has already xmas break,
11:01 < jonasschnelli> sure
11:01 < gmaxwell> In any case wumpus is out today, I'm sure many people are.
11:01 < gmaxwell> Suggested topics?
11:01 < CodeShark> WitnessMerkleBlock
11:01 < jonasschnelli> If priorities allows, hd chain split and pub-key-der
11:02 < btcdrak> lightningbot seems to be offline
11:02 < CodeShark> we'll have to do it manually
11:02 < CodeShark> unless there's a quick fix
11:02 < cfields> sorry, here
11:02 < gmaxwell> I presume everyone knows 0.13.2rc is out, and is busy testing it? :)
11:02 < instagibbs> oh, no i didnt
11:02 < CodeShark> +1 :)
11:03 < instagibbs> #action test 0.13.2rc2
11:03 < instagibbs> oh rc1 misread
11:03 < gmaxwell> we tagged rc1, 0.13 branch is one PR ahead ... rc1 failed to bump the version. 0.13 branch does.
11:03 < gmaxwell> Indeed:
11:04 < gmaxwell> #action test 0.13.1rc1 or 0.13 branch
11:04 < gmaxwell> (we're pretending the bot is here, remember.)
11:04 < jonasschnelli> Yes. Testing here. Can't sign the gitian assets file right now,... key expired.
11:04 < gmaxwell> jonasschnelli: you can always edit the expiration date on your pgp keys. the idea is to force others to get an updated copy of your key so they would have an oppturnity to learn of a revocation.
11:05 < jonasschnelli> I really should create a new one.
11:05 < MarcoFalke_web> ?why
11:05 < jonasschnelli> I try to use a HWW
11:05 < instagibbs> gmaxwell, huh, TIL.
11:05 < gmaxwell> (So it's a good practice to set a short expiration and then just keep periodically pushing it forward so long as your key isn't compromised)
11:06 < gmaxwell> (e.g. short like a year and push it forward every six months)
11:06 < BlueMatt> next topic? I know everyone has their pet feature for 0.14, given that we're getting close to feature-freeze already - but I wanted to nominate my pet for review-help :)
11:06 < CodeShark> WitnessMerkleBlock :)
11:06 < instagibbs> shoot blueman
11:06 < CodeShark> but go for it, matt
11:06 < gmaxwell> CodeShark: we'rell get to it!
11:06 < CodeShark> :)     
11:06 < jl2012> topic suggestion: excessive sighashing protection policy #8755 and #8654. Maybe too late for 0.14?
11:06 < gribble> https://github.com/bitcoin/bitcoin/issues/8755 | Implement excessive sighashing protection policy with tight sighash estimation by jl2012 · Pull Request #8755 · bitcoin/bitcoin · GitHub
11:06 < gribble> https://github.com/bitcoin/bitcoin/issues/8654 | Reuse sighash computations across evaluation (rebase of #4562) by jl2012 · Pull Request #8654 · bitcoin/bitcoin · GitHub
11:06 < gmaxwell> #topic 0.14 pet features
11:07 < instagibbs> When are we trying to freeze
11:07  * BlueMatt wants to see parallel ProcessMessages - it seems like a lot to chew, but I think y'all might be surprised by the relatively simple patches to do it
11:07  * CodeShark bites
11:07  * BlueMatt did some work on running bitcoin core in helgrind, and finds it to be realatively simple
11:07 < BlueMatt> but has opportunity to be a massive speedup in block-relay times
11:07 < jonasschnelli> gmaxwell did
11:08 < gmaxwell> jl2012: I don't know what wumpus is going to call for 0.14 though due to the net refactors I feel like we're going to have a relatively long finalization cycle for this release.
11:08 < BlueMatt> instagibbs: mid-january feature freeze, actual freeze later
11:08 < gmaxwell> I guess it would be good for people to propose 0.14 tags on whatever PRs are already in flight that you want in 0.14.
11:09 < cfields> gmaxwell: i don't think it'll be too bad. Sadly it looks like the real async changes aren't going to make it for 0.14. I think the only real gotcha right now is the assert that keeps finding new issues, but we'll need to remove that soon
11:09 < btcdrak> we should make a point to review 8755 and 8654.
11:09 < jonasschnelli> 10 0.14 tags right now: https://github.com/bitcoin/bitcoin/pulls?q=is%3Aopen+is%3Apr+milestone%3A0.14.0
11:09 < instagibbs> #855
11:09 < gribble> https://github.com/bitcoin/bitcoin/issues/855 | Toggle hide by sje397 · Pull Request #855 · bitcoin/bitcoin · GitHub
11:09 < instagibbs> #8755
11:09 < BlueMatt> I believe cfields is gonna push a new version to #9289, which is pretty important, but also #9375
11:10 < gribble> https://github.com/bitcoin/bitcoin/issues/8755 | Implement excessive sighashing protection policy with tight sighash estimation by jl2012 · Pull Request #8755 · bitcoin/bitcoin · GitHub
11:10 < luke-jr> #7533 is annoying to rebase; would be nice to get multiwallet in, but not sure we have time (#8776 looks reasonably ready to merge?)
11:10 < gribble> https://github.com/bitcoin/bitcoin/issues/9289 | net: drop boost::thread_group by theuni · Pull Request #9289 · bitcoin/bitcoin · GitHub
11:10 < gribble> https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
11:10 < gribble> https://github.com/bitcoin/bitcoin/issues/7533 | RPC: sendrawtransaction: Allow the user to ignore/override specific rejections by luke-jr · Pull Request #7533 · bitcoin/bitcoin · GitHub
11:10 < gribble> https://github.com/bitcoin/bitcoin/issues/8776 | Wallet refactoring leading up to multiwallet by luke-jr · Pull Request #8776 · bitcoin/bitcoin · GitHub
11:10 < gmaxwell> cfields: I'm disappointed with how ineffective testing is at finding issues behind that assert.
11:10 < cfields> heh. let's take turns :)
11:10 < gmaxwell> luke-jr: What do you think remains for multiwallet just more review? I admit I haven't been looking at it. I'd certantly like to see it in.
11:11 < gmaxwell> luke-jr: I'm also reminded that you PRed a feature to sweep funds based on scanning the utxo set. Where is that right now?
11:11 < BlueMatt> cfields: I'm done with my pet, just noting that to make it happen its gonna be a lot of prs that build on each other in quick succession so gonna need help with review there
11:11 < luke-jr> gmaxwell: yes, pretty much; the other pre-multiwallet PR needs rebasing yet again, but that shouldn't be hard
11:11 < instagibbs> I'm working on a couple CB PRs that would be nice to be in. But both are quite raw, maybe shoot for at least the first.
11:11 < jonasschnelli> Multiwallet project page: https://github.com/bitcoin/bitcoin/projects/2
11:12 -!- TomMc [~tom@unaffiliated/tommc] has joined #bitcoin-core-dev     
11:12 < luke-jr> sweepprivkeys is #9152 but hasn't had any review yet (besides concept discussion of rawtx interfaces and such)
11:12 < gribble> https://github.com/bitcoin/bitcoin/issues/9152 | Wallet/RPC: sweepprivkeys method to scan UTXO set and send to local wallet by luke-jr · Pull Request #9152 · bitcoin/bitcoin · GitHub
11:12 < jonasschnelli> IMO most important wallet features is the hd chain split
11:13 < jonasschnelli> Better fix sooner then later
11:13 < luke-jr> no objection to targetting 0.14 for it, but I don't consider it a huge priority since it is fairly isolated and easy to rebase
11:14 < jonasschnelli> sweepprivkey should fit into a the rawtx line. I think this is too late for 0.14
11:14 < jonasschnelli> Multiwallet probably also. This is an impactfull change.
11:14 < jonasschnelli> I think it would be great if someone could review #9294
11:14 < gribble> https://github.com/bitcoin/bitcoin/issues/9294 | Use internal HD chain for change outputs (hd split) by jonasschnelli · Pull Request #9294 · bitcoin/bitcoin · GitHub
11:15 < gmaxwell> jonasschnelli: Because of the lack of backwards compatiblity there can be an argument for batching it with other wallet changes. Though I agree it is really important.
11:15 < jonasschnelli> I'd like to batch it with the HD-pubkey pr. But not sure if this is realistic.
11:15 < gmaxwell> jonasschnelli: HD-pubkey pr?
11:15 < jonasschnelli> IMO we should slowly think about a feature to use the wallet without privatekeys but with HD metadata.
11:16 < jonasschnelli> derive private-keys on the fly.
11:16 < jonasschnelli> #9298
11:17 < gribble> https://github.com/bitcoin/bitcoin/issues/9298 | An error has occurred and has been logged. Please contact this bot's administrator for more information.
11:17 < jcorgan>      
11:17 < jonasschnelli> Not sure if this is the right approach
11:17 < gmaxwell> jonasschnelli: ah the change to not save private keys for the HD children but only derrive them when needed? yes, thats a wallet format change too.
11:17 < jonasschnelli> But if we once like to support a process handling the private keys (like GPG agent) or compatibility with hardware wallets, we need something like this
11:17 < CodeShark> it's the way to do it, ultimately
11:17 < gmaxwell> I'll review at least.
11:18 < jonasschnelli> Because both PRs are not downwareds comp., we should bundle them.
11:18 < jonasschnelli> But IMO the chain split is important.
11:18 < jonasschnelli> We shouldn't create to many wallet without the hd change chain
11:18 < jonasschnelli> *wallets
11:18 < gmaxwell> I'm a little concerned that we're thin on user visible features in 0.14, which will make uptake slower, ultimately resulting in slower testing and feedback-- it is what it is. 
11:18 < jonasschnelli> Heh. Yes.
11:19 < CodeShark> add some animated gifs :p
11:19 < jonasschnelli> We could change the splash-screen...
11:19 < jonasschnelli> ;-)     
11:19 < jonasschnelli> Multiwallet would be realtitic if everyone helps reviewing
11:19 < gmaxwell> Replace the logo with a B engraved moon in celebration of the recent price activity. :P
11:19 < jonasschnelli> realistic for 0.14
11:19 < jonasschnelli> bah     
11:20 < gmaxwell> I'll see what efforts I can drum up for multiwallet. I agree with that view.
11:20 < CodeShark> thanks for reviving this effort, luke-jr :)
11:20 < jcorgan> i'll throw my vote in for the hd chain split and only deriving keys on the fly 
11:20 < CodeShark> I had almost given up hope
11:20 < jonasschnelli> What concerns me with multiwallet is the possible performance downsides.. needs more testing I guess.
11:20 < gmaxwell> jonasschnelli: have you done any work on they keypool management? e.g. removing all the keys up to a wallet observed key to that recovery works better? 
11:20 < BlueMatt> gmaxwell: with #9375 alone we could argue "massive improvements in network-propagation speed
11:20 < gribble> https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
11:20 < luke-jr> I'm going to review the HD split stuff when the meeting is over.
11:20 < jonasschnelli> gmaxwell: I did,... but IMO it's non trivial.
11:21 < jonasschnelli> Problem: encrypted wallets
11:21 < gmaxwell> BlueMatt: yes sure, 0.14 is guarenteed to be very attractive to miners already. :) but not much else I think.
11:21 < jonasschnelli> You can't top up the keypool during some internal operations (like SyncTx)
11:21 < instagibbs> alright gotta bounce, tap me for needed review please
11:21 < luke-jr> CodeShark: ☺
11:21 < gmaxwell> jonasschnelli: why is that a problem?  You can still remove the keys, even if rescanning isn't magically solved in the first step. (and if the user unlocks before rescan it should be magically solved)
11:22 < luke-jr> jonasschnelli: people can always not-use multiwallet if performance is critical
11:22 < gmaxwell> luke-jr: I think he meant it might have performance regressions on the single wallet case.
11:22 < jonasschnelli> For rescan, yes. But for detecting running with an old wallet (without rescan), its harder
11:22 < jonasschnelli> old wallet = backup
11:22 < gmaxwell> jonasschnelli: oh can't top up during some internal operations... that would be a challenge.
11:23 < jonasschnelli> But agree, we should at least fix the rescan-with-an-old-hd-wallet feature
11:23 -!- TomMc [~tom@unaffiliated/tommc] has quit [Quit: WeeChat 1.4]     
11:23 < jonasschnelli> I'll work on that.
11:24 < jonasschnelli> But IMO, multiwallet is the lowest hanging fruit for some nice wallet features in 0.14
11:24 < gmaxwell> jonasschnelli: even without fully improving rescan, we should be removing keys from the pool that we know from transactions that they are used.  It will help when people have started multiple copies of a wallet concurrently.
11:24 < gmaxwell> okay any other PRs we should discuss briefly for more attention for 0.14 before we move onto proposed topics that don't have PRs yet?
11:24 < luke-jr> gmaxwell: only thing I can think of that could impact it is the lookup of the specific wallet for RPC calls, but I would think that'd be fairly fast
11:25 < jonasschnelli> luke-jr: I think simplest thing would be URL endpoints.
11:26 < jonasschnelli>  /<walletA> affects walletA.dat
11:26 < gmaxwell> #action review meeting log and review things people are working on for potentially 0.14.
11:26 -!- droark [~droark@c-24-22-123-27.hsd1.or.comcast.net] has joined #bitcoin-core-dev     
11:26 < CodeShark> may I talk a little about WitnessMerkleBlocks?
11:26 < gmaxwell> #topic WitnessMerkleBlocks
11:27 < gmaxwell> CodeShark: You have the floor.
11:27 < CodeShark> thank you :)
11:27 < CodeShark> so I started working on a new message type that for filtered blocks that includes the path to coinbase and a partial merkle tree for the witnesses
11:27 < CodeShark> while I don't really like BIP37 at all, we currently lack another query mechanism that doesn't require full block downloads
11:28 < CodeShark> I started working on this code: I started working on a new message type that for filtered blocks that includes the path to coinbase
11:28 < CodeShark> argh
11:28 < CodeShark> clipboard error
11:28 < CodeShark> https://github.com/bitcoin/bitcoin/compare/master...CodeShark:WitnessMerkleBlock2
11:28 < CodeShark> in addition, rather than sending all the transactions as separate messages, it includes them in the same structure
11:29 < CodeShark> while this does not allow for some minor network optimizations, I believe these optimizations are misplaced
11:29 < gmaxwell> I think we like the proof style returns. Does your use case benefit from the bloom style queries? or, for example, would a getblocktxn like query be more useful to you?
11:29 < CodeShark> hmm - good question
11:29 < CodeShark> so getblocktxn would have you specify a block hash and a transaction index?
11:30 < gmaxwell> (I'm not sure you're familar with the BIP152 getblocktxn message, -- it is a "request transactions by their index in a block")
11:30 < CodeShark> oh...I haven't looked at that
11:30 < jonasschnelli> Is this related to the filter commitments?
11:30 < gmaxwell> Yes, you request transactions for a specific block hash by index, in a very efficient way, and get back a single message with the txn packed in it.
11:30 < jonasschnelli> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-May/012636.html
11:31 < CodeShark> hmmm - that requires more roundtrips...but it's a more fundamentally important query
11:31 < gmaxwell> So I was thinking before that your particular need for witnesses could be answered by a version of getblocktxn whos reply included the membership proofs. But I wasn't sure if it fully covered your needs.
11:32 < BlueMatt> committed bloom filters could even be deployed w/o soft-forking them in - just have nodes be able to serve them for blocks
11:32 < CodeShark> yes, BlueMatt - I was thinking the same thing
11:32 < BlueMatt> might be expensive to generate, but would be an interesting test
11:32 < CodeShark> I'll have to look at getblocktxn more
11:33 < gmaxwell> In any case, if CodeShark thinks many of the interesting usecases could be answered by a query by index... I would rather work on that first.  I am realllly not eager to continue to invest in the pretty much known broken BIP37 approach.
11:33 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has joined #bitcoin-core-dev     
11:33 < CodeShark> I'll definitely take a look
11:33 < BlueMatt> gmaxwell: agreed
11:33 < gmaxwell> (in particular I was thinking you could use BIP37 on the non-witness data, and when you need witnesses, query by index for them)
11:34 < CodeShark> extending BIP37 would be a stopgap measure at best - if we can deploy a better solution nearterm I'm all for it
11:35 < gmaxwell> Sounds good. In any case, I'm glad to help talk it through. I wouldn't want to force you to wait on the BIP37 replacement to do something, but if we can be less stopgap that would be better.
11:35 < CodeShark> excellent :)
11:35 < CodeShark> I'll look into BIP152 a bit more after the meeting
11:36 < gmaxwell> You should still have the option of requesting full blocks... and for privacy reasons I'd really suggest all SPV wallets expose that as at least an option generally.  But sure, I agree that we should figure how how to accomidate some kind of sparse fetching for you.
11:36 < CodeShark> I don't really have much more to say on this topic until I have a chance to tinker a bit with that
11:36 < gmaxwell> OKAY, what other topics have I forgotten.  We talked some about jonasschnelli's chain split in the prior topic.
11:36 < CodeShark> thanks, gmaxwell :)
11:37 < jonasschnelli> IMO the chain split needs review, thats all. :)
11:37 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has quit [Ping timeout: 248 seconds]     
11:38 < cfields> <jl2012> topic suggestion: excessive sighashing protection policy #8755 and #8654. Maybe too late for 0.14?
11:38 < gribble> https://github.com/bitcoin/bitcoin/issues/8755 | Implement excessive sighashing protection policy with tight sighash estimation by jl2012 · Pull Request #8755 · bitcoin/bitcoin · GitHub
11:38 < gribble> https://github.com/bitcoin/bitcoin/issues/8654 | Reuse sighash computations across evaluation (rebase of #4562) by jl2012 · Pull Request #8654 · bitcoin/bitcoin · GitHub
11:38 < cfields> jl2012: did you have more to discuss there?
11:39 < gmaxwell> Another thing that needs review is #9138  really we don't have enough testing infrastructure for the wallet and fee estimation, so we really do count on eyeballs.
11:39 < gribble> https://github.com/bitcoin/bitcoin/issues/9138 | Improve fee estimation by morcos · Pull Request #9138 · bitcoin/bitcoin · GitHub
11:39 < jl2012> cfields: I hope people could read this: https://github.com/jl2012/bips/blob/sighash/bip-sighash.mediawiki . Didn't receive any reply on ML
11:40 < gmaxwell> #action read (and maybe reply) to https://github.com/jl2012/bips/blob/sighash/bip-sighash.mediawiki
11:40 < jl2012> That's a long post, but it explains the rationale of 8755 and 8654
11:41 < luke-jr> not quite Core-related, but it would be nice to get some BIP Comments posted now that BIP 2 is Active
11:41 < jonasschnelli> I have tagged 8755 and 8564 for 0.14.
11:41 < gmaxwell> jonasschnelli: thanks.
11:41 < gmaxwell> luke-jr: Can you explain briefly what BIP comments are for us?
11:42 < jl2012> 8654 is also somewhat a risky refactor
11:42 < jl2012> a bug was found after it got several ACKs
11:42 < luke-jr> BIP Comments are simply put a GitHub wiki page where people can post a brief opinion on the BIP
11:42 < gmaxwell> jl2012: have we at least added a test that would have found that bug?
11:43 < gmaxwell> luke-jr: is there any procedure for providing one?
11:43 < luke-jr> the idea being to provide a central location users can use to differentiate between inadvisable and recommended BIPs
11:43 < jl2012> yes, it now includes the tests to detect that bug
11:44 < luke-jr> gmaxwell: just editing the wiki page, but should ideally be done after the BIP is proposed, and before that point comments ought to be posted to bitcoin-dev instead to allow the drafter to possibly revise the BIP to address concerns
11:44 < luke-jr> https://github.com/bitcoin/bips/blob/master/bip-0002.mediawiki#BIP_comments may be useful to read over
11:45 < gmaxwell> luke-jr: thanks!
11:45 < gmaxwell> Anything else?
11:45 < sipa> hi!     
11:45 < sipa> i forgot
11:46 < sipa> good morning
11:46 < CodeShark> welcome!
11:46 < gmaxwell> sipa: we've assigned you all the work, so no worries.
11:46 < luke-jr> lol     
11:46 < sipa> great
11:46 < gmaxwell> sipa: anything you'd like to discuss?
11:47 < jonasschnelli> morning sipa!
11:47 < sipa> gmaxwell: did you report that the per-txout cache is now a clear performance win?
11:48 < gmaxwell> I thought about that but did not! 
11:48 < sipa> so: it turned out that in our earlier benchmark, some debug code was left that resulted in a database read for every txin, which was killing performance
11:49 < sipa> it's now around 30% faster to IBD, with sigchrck disabled, both for small and large fbcache
11:49 < sipa> so... work continues
11:49 < sipa> </report>
11:49 < luke-jr> wow, nice
11:50 < gmaxwell> Further speedups are expected.  In one sense this is bad news, since now we'll have to figure out how to make the migration. :)
11:50 < luke-jr> we've required reindexing before
11:51 < gmaxwell> luke-jr: the bigger the chain the larger the pill that is to swallow.
11:51 < sipa> we're investigating better cache eviction policies (instead of the wipe-cache-on-flush method...), no good results yet, but a few ideas remain
11:51 < gmaxwell> esp now that there are pruned nodes.
11:51 < luke-jr> but it's mostly the same problem as IBD for new users
11:51 < sipa> luke-jr: also, pruning is now stable... ideally we eon't require people to redownload to uograde a pruned node
11:51 < luke-jr> true re pruning
11:52 < sipa> anyway, i believe upgrading is theoretically possible, but it's not trivial (chainstate and undo files need processing)
11:52 < gmaxwell> I guess thats an interesting point to bring up: we've found the current wipe behavior is hard to beat, many things we tried that preserved entries were slightly _slower_. It appears that reading is fast and the caches improvement comes largely from preventing writes.
11:52 < luke-jr> upgrading is also consensus-critical, so it will be important to test it well
11:52 < sipa> luke-jr: agree, which is the annoying part
11:54 < gmaxwell> Okay. nothing else?
11:54 < luke-jr> 5 minutes left.
11:55 < gmaxwell> okay. I think we can end early. Happy holidays everyone! and if you're travling, travel safely.
11:55 < luke-jr> ^ +1
11:56 < gmaxwell> #endmeetingItDoesntMatterHowWeSpellItBecauseTheBotIsntHere
11:56 < luke-jr> it will matter when the log is replayed to the bot? :P
11:56 < gmaxwell> #endmeeting
11:56 < roasbeef> as is now, one wouldn't be able to use getblocktxn to fetch the transactions from blocks further than 10 blocks deep afaict
11:56 < roasbeef> will that policy be lifted? 
11:56 < jonasschnelli> thanks gmaxwell for hosting.
11:57 < roasbeef> in lightning we'd have another use for it as the outpoints for chanenls are currently communicated using 8bytes the details the blockheight+txindex+txout
11:57 < phantomcircuit> roasbeef, why?
11:57 < gmaxwell> roasbeef: we're not even talking about getblocktxn itself there-- codeshark needs something that returns membership proofs and not just entries.
11:57 < gmaxwell> If there is a utility to making getblocktxn go further back, I think that could be discussed.
11:57 < luke-jr> oh, are we meeting next week?
11:58 < luke-jr> it's not a particularly relevant day in of itself, but it's during Christmas, and many people are probably still doing stuff?
11:58 < gmaxwell> In _general_ we've tried to be conservative about p2p methods that give remote hosts random access to the blockchain; they tend to be DOS vectors (random IO on a 100GB working set) and encourage abuse of the system ("blockchain FUSE module").
11:59 < gmaxwell> Wladimir won't be here for a meeting next week, but I believe I'm willing and able.  A lot of people will be out.
12:00 < CodeShark> I've done my traveling for the year already - I'm enjoying some time at home now, so I'll be around :)
12:00 < luke-jr> I can probably be here if we have a meeting, but if a lot of people will be out it may make more sense to just cancel
12:01 < gmaxwell> Okay, lets cancel it then. It isn't like were aren't talking here all the time in any case.
12:01 < sipa> i'll be at 33c3 for next meeting
12:01 < gmaxwell> roasbeef: so you should consider the current limitation on getblocktxn mostly a product of "existing uses had no need for more, and in principle we want to be conservative with this kind of query." rather than any specific need to not permit it.
12:01 < waxwing> sipa: are you giving a talk?
12:01 < sipa> waxwing: no
12:02 < waxwing> sipa: i think they have a Lightning round or something .. insert pun here :)
12:02 < bitcoin-git> [bitcoin] jonasschnelli opened pull request #9408: Allow shutdown during LoadMempool, dump only when necessary (master...2016/12/memp_dump) https://github.com/bitcoin/bitcoin/pull/9408
12:04 < gmaxwell> jonasschnelli: so why can't we repopulate the keypool during SyncTx ?
12:04 < jonasschnelli> it would require an unlock
12:04 < jonasschnelli> (in case of encrypted wallets)
12:04 < jonasschnelli> non-encrypted wallets, fine.
12:04 < gmaxwell> ... but what if it is already unlocked?
12:04 < jonasschnelli> Then we could.
12:05 < jonasschnelli> We should do this,... but we can't leave the locked wallets in the dark.
12:05 < gmaxwell> we should do that, then making the rescan work is simply a matter of "unlock first, then rescan"-- not ideal but at least there is a procedure.
12:06 < gmaxwell> Once that works the way to handle rescan is to check if an unlock would be needed and pause the rescan until unlocked. (and in the GUI pop up the password dialog at that point)
12:06 < jonasschnelli> The problem is that there is no interruption point during SyncTx and currently no way to give user a feedback on that level
12:06 < jonasschnelli> Yes. Agree, something like that would be great
12:06 < jonasschnelli> Also, we need to define the gap limit.
12:06 < gmaxwell> Yes, but for rescan we can prompt to unlock before the keypool is empty.
12:06 < jonasschnelli> Which I rather select very high.
12:07 < gmaxwell> well thats just the keypool size. Which I think we should increase to at least 1000, I just haven't pressed for that because we haven't done the chainsplit and privkey on demand changes.
12:07 < jonasschnelli> Yes. 1000 seems to be sane.
12:08 < jonasschnelli> This would be very handy for HD rescans https://github.com/bitcoin/bitcoin/pull/7061
12:08 < jonasschnelli> You don't want to rescan down to genesis
12:08 < jonasschnelli> HD was introduces in 0.13
12:08 < jonasschnelli> Well,.. at least you should have an option to not go down to genesis
12:08 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has joined #bitcoin-core-dev     
12:09 < jonasschnelli> Also, -rescan has no user feedback options, RPC rescan would allow that
12:09 -!- MarcoFalke_web [8af602fa@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.250] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]     
12:10 < gmaxwell> I somewhat think that -rescan should be removed in favor of rpc / UI rescan.
12:10 < jonasschnelli> Yes. But most of the others where against that. Well,... pre HD though.
12:10 < gmaxwell> oh? okay. I missed or blissfully forgot that discussion.
12:11 < gmaxwell> We have too many parameters IMO and should constantly look for ones to remove.
12:12 < jonasschnelli> Yes.
12:12 < gmaxwell> rescan rpc could check if the wallet would need to be unlocked (is a locked HD wallet) and return an error, perhaps even have an override if you want to rescan without unlocking (e.g. if your keypool is already big enough)
12:12 < jonasschnelli> Yes.
12:13 < jonasschnelli> Also, we need to be carefull about the lock timeout. :)
12:13 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has quit [Ping timeout: 246 seconds]     
12:13 < gmaxwell> I was thinking that perhaps the rescan should just block the timeout.
12:13 < gmaxwell> Though all this talk of rescans makes me feel dirty. Rescans are already a hack, (largely) incompatible with pruning, and unacceptably slow.
12:13 < jonasschnelli> Yes. Just careful to not leave the wallet unlocked in case of en error/execption
12:14 < jonasschnelli> gmaxwell: yes. UTXO rescan should be an alternative
12:14 < jonasschnelli> If you have pruned, you maybe just want to kick out your funds to a new addr.
12:14 < jonasschnelli> (and don't care about tx hist)
12:14 < gmaxwell> yes, though it isn't a good alternative either, because it leaves the wallet in a weird state where it doesn't know the history and doesn't know it doesn't know the history.
12:15 < jonasschnelli> Thats true. But IMO the only option if you prune.
12:15 < jonasschnelli> Execpt your willing to re-download
12:15 < gmaxwell> meaning you might do that with a wallet, then six months later forget that you did that, and then make errors that get you in trouble as a result.
12:15 < gmaxwell> jonasschnelli: sure but we could do better with the UI/RPC to make it clear that the wallet doesn't know the history before a particular point.
12:16 < jonasschnelli> Yes. That's possible.
12:16 < gmaxwell> We can define a "pruned wallet"  this is a wallet that only knows transactions up to height X,  and knows all its coins.
12:16 < jonasschnelli> Add something in getwalletinfo, ... add something to the UI
12:16 < jonasschnelli> Yes. We should do that.
12:17 < gmaxwell> And then when you want to use the utxo scanning you have to convert your wallet to a pruned wallet.  And the UI would show at the earliest position in the tx list  ---- history before block X not shown due to wallet pruning ----
12:17 < jonasschnelli> Not sure if we get this into 0.14. Xmas, then feature freeze is close.
12:17 < gmaxwell> and likewise rpc could show a dummy transaction in position 0 to indicate the same thing.
12:17 < jonasschnelli> Yes. I like that.
12:17 < jonasschnelli> Yes. The dummy transaction might work. But careful to not break API consumers.
12:18 < jonasschnelli> Otherwise just getwalletinfo
12:18 < gmaxwell> And anyone could prune an existing wallet... which would probably make a lot of commercial users happy when they've ended up with 500 MB transactions.
12:18 < jonasschnelli> heh. Yes. Ask jouke_ . :)
12:18 < gmaxwell> yea, you're right. better to not do the dummy... just an info field.
12:18 < gmaxwell> er 500 MB wallets.
12:19 < gmaxwell> in any case, this would give a rational way to handle these partial imports that wouldn't result in any surprises.
12:19 < gmaxwell> and "unpruning" a wallet would require a rescan.
12:19 -!- echonaut [~echonaut@46.101.192.134] has quit [Remote host closed the connection]     
12:20 < jonasschnelli> Right. You could switch back anytime.
12:20 < gmaxwell> I dunno if we could do this for 0.14-- some things might be somewhat hard to handle completely, e.g. conflict tracking.
12:20 < jonasschnelli> We could use the auxiliary block requests here
12:20 < jonasschnelli> In case we don't want the re-validate the historical stuff before pruning basepoint. Though, meh.
12:20 < gmaxwell> I don't think we care about revalidating, we already validated it once.
12:21 < jonasschnelli> https://github.com/bitcoin/bitcoin/pull/9171 works pretty neat.
12:21 < jonasschnelli> Includes RPC tests, etc.
12:21 -!- echonaut [~echonaut@46.101.192.134] has joined #bitcoin-core-dev     
12:22 < jonasschnelli> I tried to slice something out that could have reasons to be independent from the rest of the "SPV" work.
12:22 < jonasschnelli> There is also an short explanation https://github.com/bitcoin/bitcoin/pull/9171#issuecomment-267616678 how it work
12:22 < jonasschnelli> Ideal for pruned-wallet to full-wallet
12:24 < jonasschnelli> I'd also like gmaxwell, and sipa comment on luke-jr comment (https://github.com/bitcoin/bitcoin/pull/9294#discussion_r93689481)
12:24 < jonasschnelli> This is a fine-adjustment thing.
12:25 < jonasschnelli> I kinda agree with luke-jr 
12:25 < jonasschnelli> though, if you set a keypool to 100, you probably don't 100 external keys + some internal keys.
12:25 < jonasschnelli> *don't expect
12:27 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-jdqeohxmhlafoqte] has quit [Quit: Connection closed for inactivity]     
12:38 < luke-jr> I would expect 100 external + 100 internal
12:40 < sipa> internal can be 5 or so
12:40 < sipa> as change addresses are always used immediately
12:41 < sipa> and don't depend on other people
12:41 < sipa> though i guess it does not matter much
12:41 < sipa> agree that keypool=X should mean external keypool
12:42 < jonasschnelli> sipa: thanks for the feedback. Yes. Let me then change that.
12:42 < jonasschnelli> It might surprise heave getrawchangeaddress users (in conjunction with enctypted wallets).
12:42 < jonasschnelli> But I guess these are adge cases.
12:43 < jonasschnelli> Clear release notes may help there
12:43 < jonasschnelli> *heavy getrawchangeaddress users
12:43 < luke-jr> oh right, forgot it was HD >_<
12:43 < luke-jr> sipa: however, your change might not get mined in order?
12:45 < jonasschnelli> dropping out, influenza sick.
12:45 -!- arubi [~ese168@gateway/tor-sasl/ese168] has quit [Remote host closed the connection]     
12:45 -!- arubi [~ese168@gateway/tor-sasl/ese168] has joined #bitcoin-core-dev     
12:45 < luke-jr> jonasschnelli: ☹ get well soon
12:46 < morcos> Hey..  sorry I missed the meeting, and yes please review #9138, but I do have something else that I think is of high priority for 0.14
12:46 < gribble> https://github.com/bitcoin/bitcoin/issues/9138 | Improve fee estimation by morcos · Pull Request #9138 · bitcoin/bitcoin · GitHub
12:46 < morcos> BlueMatt: you mentioned #9375 will help propagation speed, but thats not necessarily true
12:46 < gribble> https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
12:47 < morcos> as long as we have these nodes that send headers very fast and then delay sending us the block
12:47 < morcos> 9375 is insufficient.  i've been trying to test with it and find myself stymied by that problem.
12:47 < morcos> i think even more important than parallel ProcessMessages is the ability to intelligently have more than one block request (of some form) out at the same time
12:48 < morcos> I've been talkign about it with instagibbs, and have in my mind an outline of an idea.. but not 100% clear how to cleanly implement
12:48 < morcos> i think we'll want to be able to have more than 1 (probably 2 is sufficient) PartiallyDownloaded blocks in progress at the same time
12:49 < phantomcircuit> morcos, i dont think you want to have requests out for full blocks in parallel
12:49 < phantomcircuit> just sketches
12:50 < morcos> phantomcircuit: agreed, in short my idea is only if you have no outstanding request will you request a full block
12:52 < morcos> regardless of that, until you have 2 blocktxn requests outstanding, you'll request cmpctblocks from headers or blocktxns from cmpctblocks
12:52 < morcos> or roughly something like that...
12:53 -!- Atomicat [~Atomicat@unaffiliated/atomicat] has quit [Ping timeout: 248 seconds]     
12:56 < morcos> sdaftuar also ping on above when you get a chance ^
13:07 -!- Atomicat [~Atomicat@unaffiliated/atomicat] has joined #bitcoin-core-dev     
13:34 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]     
13:38 -!- Sosumi [~Leon@bl10-113-190.dsl.telepac.pt] has quit [Quit: Bye]     
14:02 -!- paveljanik [~paveljani@79.98.72.176] has joined #bitcoin-core-dev     
14:02 -!- paveljanik [~paveljani@79.98.72.176] has quit [Changing host]     
14:02 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev     
14:20 < BlueMatt> morcos: yes, initially I had thought that I should avoid PR'ing that until we had parallel process messages for that reason
14:20 < BlueMatt> morcos: however, the caching that that PR does will, by itself, fix a ton of the issues with being slow to send the block
14:20 < BlueMatt> indeed, not entirely, but somewhat
14:20 < BlueMatt> having parallel processmessages should fix much of the test
14:21 < BlueMatt> s/test/rest/
14:25 < gmaxwell> why is reading a block from disk so slow?
14:25 < gmaxwell> no doubt thats part of the reason that rescan is insanely slow.
14:25 < BlueMatt> IIRC we check the merkle tree on it
14:25 < gmaxwell> can we like.. not do that sometimes?
14:26 < BlueMatt> we may no longer
14:26 < BlueMatt> we used to...like 2 years ago
14:26 < BlueMatt> no, nvm, just CheckProofOfWork now
14:27 < BlueMatt> anyway, its slow because deserialize, I'd bet, but havent benchmarked deserialize + build compact block/blocktxn response
14:30 < gmaxwell> well I wouldn't be surprised if it was the deseralize, we know that rescan is quite slow.. but I don't know why.
14:31 < BlueMatt> there's a bench_bitcoin thing for deserialize and deserialize + CheckBlock
14:31 < BlueMatt> its like 10ms
14:31 < BlueMatt> or so
14:31 < BlueMatt> so not incredibly slow, but also definitely not fast
14:32 < sipa> i assume the deserialization is the slowest part
14:32 < BlueMatt> I believe that was the case when i checked
14:34 < gmaxwell> well 10ms * 100 peers is much of the slow results you were reporting.  I think you were saying lightsword was seeing 2 second service times?
14:45 -!- MykelSIlver [~MykelSIlv@192-228-145-85.ftth.glasoperator.nl] has joined #bitcoin-core-dev     
15:00 -!- harrymm [~wayne@188.42.255.195] has quit [Ping timeout: 252 seconds]     
15:11 -!- tadasv [ttttt@gateway/shell/panicbnc/x-wpbnavqaqjbwhsio] has quit [Ping timeout: 250 seconds]     
15:19 -!- tadasv [ttttt@gateway/shell/panicbnc/x-knoqdwxmxcvogfyy] has joined #bitcoin-core-dev     
15:21 -!- harrymm [~wayne@188.42.255.212] has joined #bitcoin-core-dev     
15:25 -!- harrymm1 [~wayne@223.204.198.168] has joined #bitcoin-core-dev     
15:27 -!- harrymm [~wayne@188.42.255.212] has quit [Ping timeout: 250 seconds]     
15:28 -!- harrymm1 [~wayne@223.204.198.168] has quit [Client Quit]     
15:29 -!- harrymm [~wayne@223.204.198.168] has joined #bitcoin-core-dev     
15:29 -!- harrymm [~wayne@223.204.198.168] has quit [Client Quit]     
15:29 -!- harrymm [~wayne@223.204.198.168] has joined #bitcoin-core-dev     
15:29 -!- harrymm [~wayne@223.204.198.168] has quit [Client Quit]     
15:30 -!- harrymm [~wayne@223.204.198.168] has joined #bitcoin-core-dev     
15:35 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has joined #bitcoin-core-dev     
15:35 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has quit [Remote host closed the connection]     
15:35 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has joined #bitcoin-core-dev     
15:37 -!- To7 [~theo@cpe-158-222-222-232.nyc.res.rr.com] has quit [Quit: Whatever]     
15:44 -!- wvr [~wvr@215.red-83-59-62.dynamicip.rima-tde.net] has quit [Quit: Leaving]     
15:45 -!- anditto [~anditto@KD106177210036.ppp-bb.dion.ne.jp] has quit []     
16:01 -!- harrymm [~wayne@223.204.198.168] has quit [Ping timeout: 264 seconds]     
16:06 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-qpzkivuusdyzjwld] has joined #bitcoin-core-dev     
16:09 -!- alpalp [~alpalp@cpe-24-27-58-209.austin.res.rr.com] has joined #bitcoin-core-dev     
16:09 -!- alpalp [~alpalp@cpe-24-27-58-209.austin.res.rr.com] has quit [Changing host]     
16:09 -!- alpalp [~alpalp@unaffiliated/alpalp] has joined #bitcoin-core-dev     
16:10 -!- officialjustan [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 258 seconds]     
16:15 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev     
16:17 -!- harrymm [~wayne@188.42.252.243] has joined #bitcoin-core-dev     
16:38 -!- alpalp [~alpalp@unaffiliated/alpalp] has quit [Ping timeout: 250 seconds]     
16:47 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev     
16:48 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit]     
17:02 -!- abpa [~abpa@96-82-80-25-static.hfc.comcastbusiness.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]     
17:07 -!- shesek [~shesek@bzq-84-110-233-45.cablep.bezeqint.net] has quit [Ping timeout: 268 seconds]     
17:12 -!- alpalp [~alpalp@unaffiliated/alpalp] has joined #bitcoin-core-dev     
17:13 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev     
17:13 < dcousens> OOI, is there PR following https://gist.github.com/sipa/c65665fc360ca7a176a6?
17:20 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Ping timeout: 252 seconds]     
17:21 -!- shesek [~shesek@bzq-84-110-60-18.cablep.bezeqint.net] has joined #bitcoin-core-dev     
17:27 -!- alpalp [~alpalp@unaffiliated/alpalp] has quit [Ping timeout: 250 seconds]     
17:30 -!- echonaut [~echonaut@46.101.192.134] has quit [Remote host closed the connection]     
17:30 -!- echonaut [~echonaut@46.101.192.134] has joined #bitcoin-core-dev     
17:34 -!- grubles_ [~grubles@unaffiliated/grubles] has joined #bitcoin-core-dev     
17:38 -!- alpalp [~alpalp@cpe-24-27-58-209.austin.res.rr.com] has joined #bitcoin-core-dev     
17:38 -!- alpalp [~alpalp@cpe-24-27-58-209.austin.res.rr.com] has quit [Changing host]     
17:38 -!- alpalp [~alpalp@unaffiliated/alpalp] has joined #bitcoin-core-dev     
17:41 -!- abpa [~abpa@2602:306:b837:dbf0:61e4:c65b:eb0c:7ff3] has joined #bitcoin-core-dev     
17:42 -!- abpa [~abpa@2602:306:b837:dbf0:61e4:c65b:eb0c:7ff3] has quit [Client Quit]     
18:47 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-qpzkivuusdyzjwld] has quit [Quit: Connection closed for inactivity]     
18:48 -!- tunafizz [~tuna@c-71-207-55-31.hsd1.pa.comcast.net] has quit [Ping timeout: 258 seconds]     
18:57 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 250 seconds]     
18:58 -!- tunafizz [tunafizz@c-71-207-55-31.hsd1.pa.comcast.net] has joined #bitcoin-core-dev     
19:06 -!- tunafizz [tunafizz@c-71-207-55-31.hsd1.pa.comcast.net] has quit [Read error: No route to host]     
19:06 -!- tunafizz [~tuna@c-71-207-55-31.hsd1.pa.comcast.net] has joined #bitcoin-core-dev     
19:13 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev     
19:19 < bitcoin-git> [bitcoin] jl2012 closed pull request #8756: Implement excessive sighashing protection policy with loose sighash estimation (master...sighashpolicylite) https://github.com/bitcoin/bitcoin/pull/8756
19:35 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 245 seconds]     
19:57 -!- alpalp [~alpalp@unaffiliated/alpalp] has quit [Ping timeout: 246 seconds]     
20:20 -!- chris200_ [~chris2000@p5DCB47D8.dip0.t-ipconnect.de] has joined #bitcoin-core-dev     
20:23 -!- chris2000 [~chris2000@p5DCB45FA.dip0.t-ipconnect.de] has quit [Ping timeout: 246 seconds]     
20:31 -!- arubi [~ese168@gateway/tor-sasl/ese168] has quit [Remote host closed the connection]     
20:32 -!- kadoban [~mud@unaffiliated/kadoban] has quit [Quit: bye]     
20:33 -!- arubi [~ese168@gateway/tor-sasl/ese168] has joined #bitcoin-core-dev     
21:28 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has quit [Quit: Leaving.]     
21:43 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 250 seconds]     
21:45 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev     
22:11 -!- blkdb [~blkdb@2a01:4f8:140:1407::2] has quit [Remote host closed the connection]     
22:14 -!- cannon-c [ccc23f04@gateway/web/freenode/ip.204.194.63.4] has joined #bitcoin-core-dev     
22:31 -!- windsok [~windsok@45.63.59.8] has quit [Ping timeout: 264 seconds]     
23:00 -!- dermoth [~thomas@dsl-66-36-136-48.mtl.aei.ca] has quit [Read error: Connection reset by peer]     
23:00 -!- dermoth [~thomas@dsl-66-36-136-48.mtl.aei.ca] has joined #bitcoin-core-dev     
23:14 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 260 seconds]     
23:20 -!- windsok [~windsok@45.63.59.8] has joined #bitcoin-core-dev