--- Day changed Tue Oct 27 2015
00:02 < wumpus> gmaxwell: I guess if the workload is very predictable
00:03 < wumpus> it sounds like the way to go for e.g. embedded devices
00:06 < wumpus> but certainly not if you run desktop applications like browsers :-)
00:07 < Luke-Jr> dcousens: note that Linux likes to OOM-kill processes long before it runs out of RAM, if you don't have swap :P
00:07 < dcousens> gmaxwell: I know.  Eh, I've always kept that at the default, though, I'd be half tempted to turn it off if it meant I'd actually get std:error in my logs haha
00:08 < dcousens> Luke-Jr: hasn't been my experience haha, I don't run swap on my most of my work machines and if I OOM, it just grinds to a complete halt usually
00:15 < wumpus> I used to have no swap enabled when 8GB still seemed like a lot of memory, but I can confirm what Luke-Jr says, even compile with -j3 would invoke the killer penguin
00:18 < wumpus> the problem was that any amount of swapping somehow instantly brought the entire system to a halt (even mouse cursor would no longer move) until it was done.  Managed to improve this somewhat by setting high swappiness.
00:18 < wumpus> (which makes it swap little used pages in advance instead of as last desperate measure)
00:19 < Luke-Jr> nice thing about a desktop PC is I have like 4-6 hard drives I can RAID0 the swap across..
00:23 < wumpus> good idea
00:27 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-jqukbmqkowrwqugb] has joined #bitcoin-core-dev
00:29 < Luke-Jr> (fwiw, that's builtin to Linux by using swapon -o pri=N where N is identical for everything)
00:37 < wumpus> oh!
00:48 < GitHub175> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/2b625510d374...38369dda325d
00:48 < GitHub175> bitcoin/master 0d699fc Jonas Schnelli: fix locking issue with new mempool limiting...
00:48 < GitHub175> bitcoin/master 38369dd Wladimir J. van der Laan: Merge pull request #6889...
00:48 < GitHub109> [bitcoin] laanwj closed pull request #6889: fix locking issue with new mempool limiting (master...2015/10/fix_mempool_lock) https://github.com/bitcoin/bitcoin/pull/6889
01:04 < wumpus> sipa: what is the idea with #6865? I think we should either remove or change the meaningless test
01:59 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection]
02:21 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev
02:47 -!- deepcore [~deepcore@2a01:79d:469e:ed94:8e70:5aff:fe5c:ae78] has quit [Ping timeout: 246 seconds]
02:50 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
02:53 -!- molly [~molly@unaffiliated/molly] has quit [Ping timeout: 240 seconds]
03:09 -!- isis [~isis@abulafia.patternsinthevoid.net] has quit [Ping timeout: 268 seconds]
03:09 < GitHub182> [bitcoin] paveljanik closed pull request #5525: Add removeaddress RPC call (remove watch-only address) (master...rpc_removeaddress) https://github.com/bitcoin/bitcoin/pull/5525
03:19 -!- Arnavion [arnavion@unaffiliated/arnavion] has quit [Remote host closed the connection]
03:20 -!- AtashiCon [arnavion@unaffiliated/arnavion] has quit [Remote host closed the connection]
03:24 -!- AtashiCon [arnavion@unaffiliated/arnavion] has joined #bitcoin-core-dev
03:24 -!- Arnavion [arnavion@unaffiliated/arnavion] has joined #bitcoin-core-dev
03:59 < GitHub78> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/38369dda325d...8f3b3cdee497
03:59 < GitHub78> bitcoin/master 2d8c49d Casey Rodarmor: Clean up tx prioritization when conflict mined
03:59 < GitHub78> bitcoin/master 8f3b3cd Wladimir J. van der Laan: Merge pull request #6464...
03:59 < GitHub144> [bitcoin] laanwj closed pull request #6464: Always clean up manual transaction prioritization (master...manual-tx-prioritization-cleanup) https://github.com/bitcoin/bitcoin/pull/6464
05:04 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 244 seconds]
05:35 -!- droark [~droark@c-24-22-36-12.hsd1.or.comcast.net] has quit [Ping timeout: 240 seconds]
05:48 -!- ParadoxSpiral [~ParadoxSp@p508B98E7.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
06:02 -!- isis [~isis@abulafia.patternsinthevoid.net] has joined #bitcoin-core-dev
07:05 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
07:07 < morcos> I have some questions about changes I'd like to make to manual transaction prioritization and CreateNewBlock and block priority space.
07:09 < morcos> I added a fee rate sort to the mempool.  Using this and examining 600 calls to CreateNewBlock over the last 12 hours, the time to create a new block has been reduced from 2.337 seconds + 222 ms to validate to 11 ms + 345 ms to validate
07:09 < morcos> I think the increased validation time is because the originaly version made the cache warm first
07:10 < morcos> However, this does not create a priority space and does not apply manual transaction prioritization
07:11 < morcos> Fixing the manual transaction prioritzation shouldn't be a problem as long as we're willing to store that data in the CTxMemPoolEntry.  I know it was previously discussed not to do that because it might only apply to a few txs
07:11 < morcos> However I don't think a sort based on the modified fee will work very well otherwise.  So question 1, can I store the fee delta in the CTxMemPoolEntry.
07:12 < morcos> Question 2:  I assume I'm not allowed to get rid of block priority space?   
07:13 < morcos> I was thinking I could just basically repeat the priority sort of the old alogrithm first, but I believe we might lose some advantages of the speedup.  However I think we could just skip that if blockprioritysize=0.  The code will be a bit uglier that way, but sort of optimized for the case where blockprioritysize=0?  
07:13 < morcos> Any thoughts?
07:19 < morcos> Question 3: BlueMatt: In your issue about tie-breaking you seem to suggest its random.  But isn't the current tiebreaker (for the fee sort) priority?  What would you like the tiebreaker to be?
07:20 < morcos> BTW, since the current tie-breaker is priority, and its expensive to calculate priority, I'm not going to try to make my code generate the exact same blocks as the old code.
07:21 < morcos> sipa: gmaxwell: wumpus: see above questions please
07:28 < sipa> morcos: BlueMatt's problem occurs when there is an abundance of transactions with the exact same priority, i think
07:29 < sipa> morcos: i too believe we should get rid of priority, and replace it with a custom function that alters size or fee of a transaction
07:40 < gavinandresen> sipa: I agree, but a simple function with a miner-tweakable multiplier (maybe zero by default to mean "no priority adjustment") would be fine, I think.
07:41 -!- bsm1175321 [~bsm117532@38.121.165.30] has quit [Remote host closed the connection]
07:42 < sipa> gavinandresen: right, i was taking code-wise... the policy code can implement an overridable function, and we know that everything keeps.workimg whatever that function is
07:42 < sipa> the default implementation can be a simple multiplier
07:43 < sipa> or a constant that get added to both fee and size
07:43 < gavinandresen> spiffy, I was worried somebody would go and implement a javascript interpreter so miners could tweak the function at runtime... :-)
07:43 < sipa> ha
07:44 < wumpus> should definitely use lua for that *ducks*
07:44 < sipa> we should use bitcoin script with the op_x86 extension
07:45 < wumpus> morcos: what is the main drawback of storing the fee delta in CTxMemPoolEntry? increasing the size of the structure?
07:47 < wumpus> bitcoin script, hah, even better
07:47 -!- danielsocials [~quassel@45.32.248.113] has quit [Ping timeout: 265 seconds]
07:47 -!- bsm1175321 [~bsm117532@38.121.165.30] has joined #bitcoin-core-dev
07:50 < wumpus> morcos: optimizing for blockprioritysize=0 sounds good to me, removing it is somewhat controversial - but as it seems to be where things are going anyway, and simplifies the algorithm a lot it makes sense to optimize for the case
07:52 < morcos> ok thanks everyone.  yes only downside is more bloat to the mempool, but its relatively small.
07:53 < morcos> sipa: so perhaps i'll make a CTxMemPoolEntry function GetMiningPriority (ugh I hate overuse of the word priority) which for now = GetFee() + feeDelta
07:56 < morcos> or should I store a miningPriority variable, and use an outside of CTxMempool function to calculate it and set it? I guess it could set it at creation and further calls to prioritizeTransaction would adjust it?
07:59 -!- danielsocials [~quassel@45.32.248.113] has joined #bitcoin-core-dev
08:13 -!- treehug88 [~textual@cpe-74-66-7-27.nyc.res.rr.com] has joined #bitcoin-core-dev
08:36 < jonasschnelli> cfields: any idea why i get "clang: error: unknown argument: '-framework QtCore'" on my osx machine with qt5.5 [homebrew]?
08:39 < cfields> jonasschnelli: mm, that's an odd error. is anything quoted funny?
08:39 < jonasschnelli> cfields: i just switched back to Qt 5.4.1 .. there i don't have a such problem..
08:40 < jonasschnelli> cfields: no additional details... just the clang error output during linking
08:40 < cfields> jonasschnelli: can you paste the exact link-line? (make V=1)
08:40 < cfields> if there's a problem with the framework, i'd expect "framework not found" or so
08:41 < jonasschnelli> cfields: will do soon... currently switched back to qt5.4.1
08:42 < bsm1175321> Can discuss c++11 here if you prefer.  But then what's #bitcoin-dev for?
08:44 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
08:44 < wumpus> for more abstract things, nitty gritty implementation details for bitcoin core belong here
08:46 -!- danielsocials [~quassel@45.32.248.113] has quit [Remote host closed the connection]
08:46 < bsm1175321> wumpus: indeed an explicit shared_ptr gets rid of the compiler error.  Let's see if it runs...
08:47 < wumpus> welcome zooko
08:47 < wumpus> bsm1175321: great!
08:49 < cfields> jonasschnelli: https://github.com/Homebrew/homebrew/commit/fdbb338f2274e093b96209e7ca3ff9bf5460714e
08:50 < cfields> patch is here: https://gist.githubusercontent.com/UniqMartin/a54542d666be1983dc83/raw/f235dfb418c3d0d086c3baae520d538bae0b1c70/qtbug-47162.patch
08:50 < cfields> it seems to leave out the closing quote
08:51 < cfields> er, nm, misread
08:51 < jonasschnelli> cfields : you mean a syntax error for the two ends?
08:54 < cfields> jonasschnelli: oh, according to the bug report, that patchshould actually fix your problem
08:55 < cfields> possible you hadn't updated brew lately before building 5.5?
08:55 -!- malte [~malte@2a00:d0c0:200:0:b9:1a:9c2c:1] has left #bitcoin-core-dev ["Textual IRC Client: www.textualapp.com"]
08:57 < cfields> (remember that we use static libs for depends builds, so we wouldn't hit this there for our 5.5.0 build)
08:58 -!- yammq [d4044622@gateway/web/freenode/ip.212.4.70.34] has joined #bitcoin-core-dev
09:14 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
09:17 -!- yammq [d4044622@gateway/web/freenode/ip.212.4.70.34] has quit [Ping timeout: 246 seconds]
09:30 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection]
09:32 -!- rubensayshi [~ruben@91.206.81.13] has joined #bitcoin-core-dev
09:40 < jonasschnelli> cfields: thanks for looking at it! I think i can solve it now... but need to finish stuff with qt 5.4.1 now until i go back to 5.5.
09:41 < GitHub34> [bitcoin] Diapolo opened pull request #6891: constify missing catch cases (master...const-ex) https://github.com/bitcoin/bitcoin/pull/6891
09:46 < GitHub81> [bitcoin] Diapolo opened pull request #6892: [Trivial] ensure minimal header conventions (master...headers-new) https://github.com/bitcoin/bitcoin/pull/6892
09:54 < GitHub176> [bitcoin] Diapolo opened pull request #6893: forward-declare univalue in rpcclient + remove include in rpcserver.cpp (master...univalue) https://github.com/bitcoin/bitcoin/pull/6893
09:56 -!- treehug88 [~textual@cpe-74-66-7-27.nyc.res.rr.com] has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
09:59 -!- lclc [~lclc@unaffiliated/lclc] has joined #bitcoin-core-dev
10:04 < wumpus> sigh...
10:09 < jonasschnelli> ;-)
10:19 -!- treehug88 [~textual@cpe-74-66-7-27.nyc.res.rr.com] has joined #bitcoin-core-dev
10:27 -!- MarcoFalke [3e4bef14@gateway/web/cgi-irc/kiwiirc.com/ip.62.75.239.20] has joined #bitcoin-core-dev
10:28 < btcdrak> Trolling for review: https://github.com/bitcoin/bitcoin/pull/6312
10:33 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev
10:38 -!- treehug88 [~textual@cpe-74-66-7-27.nyc.res.rr.com] has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
10:40 -!- treehug88 [~textual@cpe-74-66-7-27.nyc.res.rr.com] has joined #bitcoin-core-dev
10:42 -!- MarcoFalke [3e4bef14@gateway/web/cgi-irc/kiwiirc.com/ip.62.75.239.20] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]
10:55 -!- treehug88 [~textual@cpe-74-66-7-27.nyc.res.rr.com] has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
11:00 < gmaxwell> btcdrak: is no one backporting mediantimepast to 0.11?
11:00 < btcdrak> gmaxwell: I backported it...
11:00 < btcdrak> gmaxwell: https://github.com/bitcoin/bitcoin/issues/6884
11:01 < btcdrak> gmaxwell: Not sure if we need to backport to 0.10, didnt have time yet anyway.
11:01 < gmaxwell> ah okay I missed it.
11:04 < btcdrak> gmaxwell: I also made a branch with BIP68+OP_CSV in one as some people asked for it. I'm debating if I should make a PR or just leave the two separate PRs as is.
11:06 -!- rubensayshi [~ruben@91.206.81.13] has quit [Remote host closed the connection]
11:10 < gmaxwell> btcdrak: this patch does not appear to enforce (as standardness) the new behavior.
11:13 < morcos> there are some unit tests in miner_tests.cpp which verify that CreateNewBlock won't mine non-final txs that are in the mempool
11:13 < morcos> they were added by petertodd 
11:14 < morcos> i was under the impression that we don't want to allow non-final txs into the mempool
11:15 < morcos> i suppose i should have checked that my code passed the unit tests earlier on, but the whole idea is to not recheck everything while building the block
11:16 < gmaxwell> morcos: the finality test is against the mined block itself, while the mempool is running one block behind.
11:16 < morcos> i think that was fixed so the mempool checks against +1, but that would have been a problem in the opposite direction
11:16 < gmaxwell> So the mepool need a one block lookahead or there will be an extra delay in mining transactions. (which would presumably mean a competative advantage for miners that change that code)
11:17 < morcos> the unit test specifically calls addUnchecked with a non-final tx, and then sees if CreateNewBlock would mine it
11:17 < morcos> non-final for 1 block ahead (and a time based version as well)
11:18 < morcos> what i'm saying is I'd like to be able ot rely on there being no non-final txs in the mempool, i think this was the idea sipa had in mind too?
11:19 < morcos> the one thing I can think of is that its possible we need a function analogous to removeCoinbaseSpends for locktime based txs in the event of a reorg if thats not happening already
11:19 < gmaxwell> The hight based can be precise, e.g. so it's not possible to add something that won't be final; and I think that would be good enough for what you're doing. But time based can't be precise; but MPL is fixing that.  I'm fine with there beinging nothing that won't be final. 
11:20 < gmaxwell> I'm not sure if we're on the same page, so I'm saying that I think there should be currently non-final tx in the mempool which will be final from the perspective of createnewblock,  but none that won't be final from the perspective of createnewblock. 
11:21 < gmaxwell> Before MTP this was hard to achieve without adding additional delays, but MTP will effectively make that additional delay universal.
11:21 -!- treehug88 [~textual@cpe-74-66-7-27.nyc.res.rr.com] has joined #bitcoin-core-dev
11:21 < morcos> I guess thats the same page.  Mempool adds 1 to chainActive.height before checking finality, so i would consider it to be final from the perspective of both the mempool and CNB
11:22 < morcos> and yes i agree that the current state is no non-final txs will end up in the mempool
11:22 < morcos> what i'm saying is the unit tests require that even if they do, CNB will not put them in the template.  (that breaks for me, and i think is unnecessary computation)
11:22 < gmaxwell> okay, fair enough, I was defining current as chainActive.Tip().
11:22 < morcos> also what i'm saying is that a reorg could probably break that assumption
11:23 < gmaxwell> I think we can drop that test, but we need to make sure that reorgs are removing things correctly at the same time. (I don't know if we have a test for that).
11:23 < sdaftuar> morcos: bluematt had a pr recently to fix that
11:24 < sdaftuar> #6595
11:24 < sdaftuar> we need to pick that up again...
11:24 < sdaftuar> oh ha. i remember now, we were going to revamp the way reorgs are handled
11:24 < morcos> ah, ok good.  yes we need that.
11:25 < morcos> and yeah thats what i was thinking, that you only want to run this after the reorg
11:25 < morcos> but why is that difficult
11:25 < sdaftuar> i'll dust off that code.  after packages went in, bluematt's open pulls needed some rebasing
11:25 < sdaftuar> see also #6656
11:25 < morcos> you odn't have to redo the whole way reorgs are handled, you just have to call the removeForReorg after ActivateBestChainStep returns
11:26 < sdaftuar> yeah i think that's what 6656 does
11:26 < sdaftuar> er, did
11:26 < morcos> perhaps you need to flag that the height or mediantimepast might have changed
11:26 < morcos> down
11:26 < gmaxwell> Perhaps that unit test could be changed to addunchecked but then trigger the logic that handles a reorg.
11:27 < morcos> heh, i think thats in a different unit test
11:28 < morcos> sdaftuar: so 6656 really supercedes 6595 then.  jsut for some reason 6595 was left open
11:29 < morcos> gmaxwell: this actually does lead me to a concern.  i was leaving in the final TestBlockValidity test, but what should the code do in the event it fails?
11:29 < morcos> I'm thinking it should switch into mining empty block mode, because otherwise there is no way to figure out what part of your block assembly broke
11:30 < morcos> the down side of this, is if there is some error in mempool code that lets non valid txs in, all miners will just start switching to mining empty blocks
11:30 < morcos> ugh that sounds terrible
11:30 < gmaxwell> We assert now, no? .. yea, but I was just about to suggest it could start doing that. OTOH, that doesn't exactly encourage people to fix things, and empty blocks may mean that you don't failover to another daemon.
11:31 < morcos> its much less likely to happen now, because we CheckInputs on every tx as we add it anyway, and if that fails we skip that txs
11:31 < gmaxwell> morcos: well empty blocks are safe(er). Assuming that _prior_ chain is not invalid, they're no worse than not mining and likely better.
11:31 < morcos> perhaps i should put that code back in
11:31 < morcos> and just leave it off, unless you hit an invalid block, and then switch it on
11:31 < morcos> because that way you'll just skip over the invalid txs
11:31 < morcos> at least for that failure mode
11:32 < gmaxwell> Running checkinputs again sounds expensive.
11:32 < morcos> but then it won't be a very tested code path, unless we have a mode to test it
11:32 < morcos> yeah well thats why the existing code is slow
11:33 < morcos> its a nice property of the existing code, that if somebody injects an invalid tx into mempools, the mining code will just keep skipping it and assembling valid blocks from the rest of the txs
11:34 < gmaxwell> In any case, thinking in terms of what the user of this code wants;  they either want empty blocks (if they have no fail over) or they want the node to stop responding (if they have fail-over).
11:34 < gmaxwell> morcos: right, it keeps the mempool behavior seperate from consensus.
11:39 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Ping timeout: 250 seconds]
11:44 < sipa> morcos: i think the mempool should avoid but not outlaw non-final transactions; CNB should skip the nonfinal ones though
11:48 < gmaxwell> what about checkinputs redundancy?  at least doing another IsFinal check is cheap.
11:52 < sipa> i think all mempool entries should be valid regarding 1) not spending nonexisting inputs 2) script validity
11:53 < sipa> if we have that, i think CNB could just not validate tye result
11:53 < sipa> but that seems controversial
11:56 < morcos> sipa: so you'd like me to put IsFinal check back in?, you might be right, it might be cheap...
11:57 < morcos> i suppose it would be nice to not have to worry about removing txs in the event of a reorg, you know they'll be valid again very soon 
11:57 < morcos> what do you think about BIP68 and sequence numbers though
11:58 < morcos> thats not cheap to check b/c you have to look up the inputs
11:58 < gmaxwell> sipa: really if we make CNB no longer latency critical (via mechenisms discussed before) then I think validate after remains a cheap protection.
11:59 < gmaxwell> but I'd rather remove some of the internal double checking where it is expensive and really shouldn't be needed but continue to validate after.
12:00 < sipa> morcos: i think enforcing nonfinalness in the mempool.is expensive, and the worst a reorg can cheaply do is put a bunch of things in that will be check anyway
12:01 < sipa> morcos: fair enough, bip68 does make this significantly more expensive
12:01 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Ping timeout: 240 seconds]
12:02 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
12:03 < morcos> sipa: i feel like i'm really overloading CTxMempoolEntry, but seems like it could store a block height and time at which the tx is final
12:04 < morcos> that way the locktime and bip68 checks that you do at ATMP don't have to be repeated (ugh... i guess unless there is a reorg that affects inputs)
12:04 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has joined #bitcoin-core-dev
12:07 -!- zooko [~user@50.141.117.36] has joined #bitcoin-core-dev
12:14 < sipa> morcos: sgtm, if that can guarantee not needing to verify after creation
12:16 -!- zooko [~user@50.141.117.36] has quit [Read error: Connection reset by peer]
12:18 -!- zooko [~user@50.141.117.36] has joined #bitcoin-core-dev
12:19 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
12:23 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
12:27 < morcos> sipa: well then we're back to the reorg problem right?  but maybe that's solvable more quickly by only having to do work on smaller subset of the txs.  for instance if you know the lowest height and mediantPast time you disconnected down to, you only need to look at txs with CTxMemPoolEntry valid time and heights > that
12:32 < GitHub163> [bitcoin] giacecco closed pull request #6885: Instructions on how to make the Homebrew OpenSSL headers visible... (master...master) https://github.com/bitcoin/bitcoin/pull/6885
12:33 -!- belcher [~user@unaffiliated/belcher] has joined #bitcoin-core-dev
12:36 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Quit: .]
12:37 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
12:39 -!- zooko` [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
12:40 -!- CodeShark [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has joined #bitcoin-core-dev
12:41 -!- zooko [~user@50.141.117.36] has quit [Ping timeout: 264 seconds]
12:41 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
12:43 -!- zooko` [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Client Quit]
12:43 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Client Quit]
12:43 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
12:45 -!- CodeShark [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has quit [Client Quit]
12:46 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has quit [Quit: Leaving.]
12:58 < GitHub51> [bitcoin] sdaftuar opened pull request #6894: [Tests] Fix BIP65 p2p test (master...fix-bip65-p2p-test) https://github.com/bitcoin/bitcoin/pull/6894
12:59 < morcos> sipa: BlueMatt: sdaftuar and i were just talking.  the ccoinsviewcache growing out of control.  the biggest problem is a reorg!  a reorg will cause removeCoinbaseSpends to run, which will pull every single txin into the cache
12:59 < morcos> i assume it'll flush right after that, but at that point it'll be too late if you OOM'ed
12:59 < sipa> hmmm
13:00 < morcos> i think storing this valid height/time will be very useful for coinbase spends, and moderately useful for checklocktime and csv
13:01 < morcos> arguably we should actually store 3 numbers, the height/time its valid at, and the height which if you reorg below you need to recalculate validity
13:01 < morcos> then on a reorg we just scan the whole mempool checking that last number, and recalculate validity
13:01 < morcos> unfortunatley anti-fee sniping will cause any txs which have arrived in the last couple blocks to be triggered for recalc
13:02 < morcos> and wihtout storing more info, you'd have to recalc for coinbase, checklocktime and bip68 sequence, bc you don't know which was invalidated
13:02 < morcos> so its a bit of an expensive recalc
13:06 -!- Thireus1 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
13:10 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Ping timeout: 260 seconds]
13:31 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Ping timeout: 272 seconds]
13:35 < BlueMatt> morcos: no, there is no current tie breaker in CreateNewBlock
13:35 < morcos> BlueMatt: sure why not, it uses TxPriorityCompare right?  which sorts by fee then priority
13:36 < BlueMatt> morcos: the way I read it, it first creates a heap according to priority, then resorts to create a heap using fee
13:36 < BlueMatt> it never uses fee then priority
13:37 < morcos> yes correct, but if you look at that compartor, in either case it uses the other one as the tie break
13:37 < morcos> comparatator
13:38 < BlueMatt> hmmmm, indeed, I hadnt seen that before....how is it that ordering seems randomish, then?
13:39 < BlueMatt> lemme go look at mempool
13:40 < morcos> yeah i was wondering that, you would think that would be fairly consistent across nodes
13:41 < morcos> i wonder if in this recent attack there are many things with also the same priority...  seems a bit unlikely..
13:41 < sipa> that was my assumption
13:42 < sipa> if all outputs are created in the same block, and the spends are the same size
13:42 < sipa> that's what you would get
13:57 -!- zooko [~user@2601:281:8001:26aa:a052:7c51:a1e8:65a8] has joined #bitcoin-core-dev
14:03 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
14:04 -!- ParadoxSpiral [~ParadoxSp@p508B98E7.dip0.t-ipconnect.de] has quit [Remote host closed the connection]
14:07 -!- Thireus1 [~Thireus@icy.thireus.fr] has quit [Ping timeout: 240 seconds]
14:27 -!- Thireus1 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
14:27 -!- Thireus2 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
14:28 -!- Thireus1 [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
14:28 -!- Thireus1 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
14:29 -!- Thireus3 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
14:30 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Ping timeout: 272 seconds]
14:30 -!- Thireus2 [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
14:33 -!- Thireus1 [~Thireus@icy.thireus.fr] has quit [Ping timeout: 255 seconds]
14:51 -!- deepcore [~deepcore@2a01:79d:469e:ed94:8e70:5aff:fe5c:ae78] has joined #bitcoin-core-dev
14:55 < BlueMatt> morcos: hmmmm, now I'm very confused
14:55 < BlueMatt> morcos: there are almost no txn in my mempool which have the same fee and prio
14:56 < morcos> BlueMatt: I didn't look into the root issue, its just that you are not seeing the blocks you expect to see?  It's not a result of malleated txs?  Where some nodes have one version and others have another
14:59 < BlueMatt> it could be related to malleated txn, but ive definitely seen it every time mempools get large, not necessarily when malleation is happening
15:01 < sipa> might it just be different miners using different policies?
15:05 < BlueMatt> afaiu there are effectively no different policies in use by miners today
15:05 < BlueMatt> (aside from eligius)
15:06 -!- droark [~droark@c-24-22-36-12.hsd1.or.comcast.net] has joined #bitcoin-core-dev
15:06 -!- deepcore [~deepcore@2a01:79d:469e:ed94:8e70:5aff:fe5c:ae78] has quit [Ping timeout: 240 seconds]
15:12 -!- deepcore [~deepcore@2a01:79d:469e:ed94:8e70:5aff:fe5c:ae78] has joined #bitcoin-core-dev
15:14 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev
15:50 -!- deepcore [~deepcore@2a01:79d:469e:ed94:8e70:5aff:fe5c:ae78] has quit [Ping timeout: 256 seconds]
16:11 < dcousens> when are the IRC meetings planned?
16:11 < dcousens> ooi
16:15 < sipa> thursday 7pm UTC
16:15 < dcousens> ta
16:27 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
16:29 -!- Thireus3 [~Thireus@icy.thireus.fr] has quit [Ping timeout: 255 seconds]
16:33 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Quit: Leaving.]
16:34 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
16:39 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
16:40 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
16:51 -!- Thireus1 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
16:54 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Ping timeout: 240 seconds]
17:01 -!- zooko [~user@2601:281:8001:26aa:a052:7c51:a1e8:65a8] has quit [Ping timeout: 240 seconds]
17:07 -!- Thireus1 [~Thireus@icy.thireus.fr] has quit [Quit: Leaving.]
17:07 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
17:09 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
17:12 -!- Thireus1 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
17:13 -!- Thireus2 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
17:15 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Ping timeout: 240 seconds]
17:17 -!- Thireus1 [~Thireus@icy.thireus.fr] has quit [Ping timeout: 265 seconds]
17:21 < GitHub30> [bitcoin] sipa opened pull request #6895: Update to my new key (master...newkey) https://github.com/bitcoin/bitcoin/pull/6895
17:36 -!- Apocalyptic [~Apocalypt@unaffiliated/apocalyptic] has quit [Ping timeout: 250 seconds]
17:42 -!- Apocalyptic [~Apocalypt@unaffiliated/apocalyptic] has joined #bitcoin-core-dev
17:55 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Ping timeout: 264 seconds]
18:07 < GitHub19> [bitcoin] gmaxwell pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/8f3b3cdee497...d0badb916e51
18:07 < GitHub19> bitcoin/master 298e040 Pieter Wuille: Fix chainstate serialized_size computation
18:07 < GitHub19> bitcoin/master d0badb9 Gregory Maxwell: Merge pull request #6865...
18:07 < GitHub22> [bitcoin] gmaxwell closed pull request #6865: Fix chainstate serialized_size computation (master...fixchainsize) https://github.com/bitcoin/bitcoin/pull/6865
18:20 -!- treehug88 [~textual@cpe-74-66-7-27.nyc.res.rr.com] has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
18:22 < GitHub43> [bitcoin] sipa pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/d0badb916e51...93521a4f56ce
18:22 < GitHub43> bitcoin/master 27252b7 Matt Corallo: Fix pre-push-hook regexes
18:22 < GitHub43> bitcoin/master 1d94b72 Matt Corallo: Whitelist commits signed with Pieter's now-revoked key
18:22 < GitHub43> bitcoin/master 6e800c2 Matt Corallo: Add Pieter's new PGP key to verify-commits/trusted-keys
18:22 < GitHub113> [bitcoin] sipa closed pull request #6875: Fix pre-push-hook regexes (master...verify-commits-fixes) https://github.com/bitcoin/bitcoin/pull/6875
18:24 < GitHub157> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/93521a4f56ce...8756c986420c
18:24 < GitHub157> bitcoin/master 4252cd0 Pieter Wuille: Update to my new key
18:24 < GitHub157> bitcoin/master 8756c98 Pieter Wuille: Merge pull request #6895...
18:24 < GitHub179> [bitcoin] sipa closed pull request #6895: Update to my new key (master...newkey) https://github.com/bitcoin/bitcoin/pull/6895
18:25 < GitHub93> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/8756c986420c...e06c14fb59ee
18:25 < GitHub93> bitcoin/master ab1f560 Pieter Wuille: Support -checkmempool=N, which runs checks on average once every N transactions
18:25 < GitHub93> bitcoin/master e06c14f Pieter Wuille: Merge pull request #6776...
18:25 < GitHub73> [bitcoin] sipa closed pull request #6776: Support -checkmempool=N, which runs checks once every N transactions (master...fraccheck) https://github.com/bitcoin/bitcoin/pull/6776
18:31 < GitHub67> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e06c14fb59ee...4764f5db9d2c
18:31 < GitHub67> bitcoin/master 214de7e Philip Kaufmann: [Trivial] ensure minimal header conventions...
18:31 < GitHub67> bitcoin/master 4764f5d Pieter Wuille: Merge pull request #6892...
18:31 < GitHub110> [bitcoin] sipa closed pull request #6892: [Trivial] ensure minimal header conventions (master...headers-new) https://github.com/bitcoin/bitcoin/pull/6892
18:34 < GitHub1> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/4764f5db9d2c...8daffe227bc6
18:34 < GitHub1> bitcoin/master ad5aae1 Philip Kaufmann: constify missing catch cases...
18:34 < GitHub1> bitcoin/master 8daffe2 Pieter Wuille: Merge pull request #6891...
18:34 < GitHub130> [bitcoin] sipa closed pull request #6891: constify missing catch cases (master...const-ex) https://github.com/bitcoin/bitcoin/pull/6891
18:46 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
18:53 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has quit [Ping timeout: 260 seconds]
19:01 < gmaxwell> Oh I think I know one reason we see corruption reports on windows.
19:02 < gmaxwell> Stopnode often takes an awful long time, for example on my laptop running defaults it just took 12 seconds.  In windows, on shutdown, tasks that don't stop pretty much immediately get killed.
19:02 < gmaxwell> thats probably a maximally bad case for leveldb as it'll be in the middle of flushing out a bunch of cached updates.
19:10 < GitHub4> [bitcoin] sipa opened pull request #6896: Make -checkmempool=1 not fail through int32 overflow (master...fixchainsize) https://github.com/bitcoin/bitcoin/pull/6896
19:32 -!- belcher [~user@unaffiliated/belcher] has quit [Quit: Leaving]
19:35 < tripleslash> gmaxwell: its specifically HungAppTimeout and is defaulted to 5 seconds.
19:38 < morcos> test_bitcoin hangs for me a decent fraction of the time.  It looks like it gets stuck at line 109 in scheduler_tests.cpp
19:56 < morcos> ah, i see it is an issue, #6540
20:12 -!- fkhan [weechat@gateway/vpn/mullvad/x-slnmaxmiuwdufown] has quit [K-Lined]
20:44 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-jqukbmqkowrwqugb] has quit [Quit: Connection closed for inactivity]
21:02 -!- baldur [~baldur@pool-173-52-43-219.nycmny.fios.verizon.net] has quit [Read error: Connection reset by peer]
21:02 -!- baldur [~baldur@pool-173-52-43-219.nycmny.fios.verizon.net] has joined #bitcoin-core-dev
21:03 -!- blur3d [~blur3d@pa49-197-130-167.pa.qld.optusnet.com.au] has joined #bitcoin-core-dev
22:12 -!- blur3d [~blur3d@pa49-197-130-167.pa.qld.optusnet.com.au] has quit [Ping timeout: 240 seconds]
22:24 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Remote host closed the connection]
22:27 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-core-dev
22:45 -!- zooko [~user@2601:281:8001:26aa:a052:7c51:a1e8:65a8] has joined #bitcoin-core-dev
23:27 -!- ParadoxSpiral [~ParadoxSp@p508B98E7.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
23:36 -!- gavinandresen [~gavin@unaffiliated/gavinandresen] has quit [Remote host closed the connection]
23:47 -!- ParadoxSpiral [~ParadoxSp@p508B98E7.dip0.t-ipconnect.de] has quit [Remote host closed the connection]