--- Day changed Wed Feb 10 2016
00:06 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev
00:29 -!- molz [~molly@unaffiliated/molly] has quit [Read error: Connection reset by peer]
00:35 < GitHub101> [bitcoin] Flowdalic opened pull request #7495: Make genbuild.sh check for '.git'  (master...5902) https://github.com/bitcoin/bitcoin/pull/7495
00:36 -!- cheese_ [~x@unaffiliated/cheeseo] has quit [Read error: Connection reset by peer]
00:42 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
00:46 -!- adnn [~adnn@cpe-158-222-198-108.nyc.res.rr.com] has quit [Remote host closed the connection]
00:47 -!- adnn [~adnn@cpe-158-222-198-108.nyc.res.rr.com] has joined #bitcoin-core-dev
01:00 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev
01:00 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Client Quit]
01:14 -!- AtashiCon [arnavion@unaffiliated/arnavion] has quit [Remote host closed the connection]
01:15 -!- AtashiCon [~Arnavion@unaffiliated/arnavion] has joined #bitcoin-core-dev
01:26 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
01:53 -!- adnn [~adnn@cpe-158-222-198-108.nyc.res.rr.com] has quit [Remote host closed the connection]
02:25 -!- adnn [~adnn@cpe-158-222-198-108.nyc.res.rr.com] has joined #bitcoin-core-dev
02:45 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
02:56 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has joined #bitcoin-core-dev
02:57 -!- drnet [~drnett@193-81-57-254.adsl.highway.telekom.at] has joined #bitcoin-core-dev
03:03 -!- drnet [~drnett@193-81-57-254.adsl.highway.telekom.at] has quit [Quit: Leaving]
03:08 < Luke-Jr> wumpus: is there a good reason for -qt keeping a second copy of every setting somewhere else?
03:32 < GitHub71> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/b49a62379900...d007511ebdfa
03:32 < GitHub71> bitcoin/master acf5983 Wladimir J. van der Laan: tests: Remove May15 test...
03:32 < GitHub71> bitcoin/master d007511 Wladimir J. van der Laan: Merge #7490: tests: Remove May15 test...
03:32 -!- adnn [~adnn@cpe-158-222-198-108.nyc.res.rr.com] has quit [Remote host closed the connection]
03:32 < GitHub36> [bitcoin] laanwj closed pull request #7490: tests: Remove May15 test (master...2016_02_may12forkdat) https://github.com/bitcoin/bitcoin/pull/7490
03:33 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has quit [Quit: laurentmt]
03:43 < wumpus> Luke-Jr: yes, it makes it possible to change the settings through the GUI without having to do something ugly like write a bitcoin.conf
04:01 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has quit [Max SendQ exceeded]
04:17 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
04:27 -!- gevs [~greg@unaffiliated/gevs] has joined #bitcoin-core-dev
04:53 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev
05:00 < GitHub161> [bitcoin] promag opened pull request #7498: [WIP][RPC] Add createtransaction (master...feature/rpc-createtransaction) https://github.com/bitcoin/bitcoin/pull/7498
05:02 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has quit [Max SendQ exceeded]
05:07 -!- jtimon [~quassel@126.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev
05:18 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
05:22 < GitHub113> [bitcoin] sipa opened pull request #7500: Correctly report high-S violations (master...reporthighs) https://github.com/bitcoin/bitcoin/pull/7500
05:37 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has joined #bitcoin-core-dev
05:41 -!- wasi [~wasi@25.22.3.213.static.wline.lns.sme.cust.swisscom.ch] has joined #bitcoin-core-dev
05:45 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Quit: Leaving]
06:11 -!- p15x [~p15x@13.91.145.64.client.static.strong-tk2.bringover.net] has quit [Ping timeout: 240 seconds]
06:23 -!- Chris_Stewart_5 [~Chris_Ste@173-31-39-168.client.mchsi.com] has joined #bitcoin-core-dev
06:24 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has quit [Max SendQ exceeded]
06:28 -!- Chris_Stewart_5 [~Chris_Ste@173-31-39-168.client.mchsi.com] has quit [Ping timeout: 240 seconds]
06:33 -!- frankenmint [~frankenmi@174-25-38-39.ptld.qwest.net] has quit [Remote host closed the connection]
06:34 -!- cjcj [82ebca3a@gateway/web/freenode/ip.130.235.202.58] has quit [K-Lined]
06:45 -!- xabbix [~xabbix@unaffiliated/xabbix] has quit [Ping timeout: 256 seconds]
06:46 -!- xabbix [~xabbix@unaffiliated/xabbix] has joined #bitcoin-core-dev
06:48 < GitHub58> [bitcoin] sipa opened pull request #7502: Update the wallet best block marker before pruning (master...betterflush) https://github.com/bitcoin/bitcoin/pull/7502
06:49 -!- sipa [~pw@2a02:348:86:3011::1] has joined #bitcoin-core-dev
06:56 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Ping timeout: 250 seconds]
07:00 -!- cjcj [82ebca3a@gateway/web/freenode/ip.130.235.202.58] has joined #bitcoin-core-dev
07:04 -!- Chris_Stewart_5 [~Chris_Ste@173-31-39-168.client.mchsi.com] has joined #bitcoin-core-dev
07:06 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has quit [Quit: laurentmt]
07:07 < morcos> sipa: you there to discuss 7502?
07:08 < morcos> i'm not sure how we made that mistake in the first place, we explicitly discussed this problem in IRC on 5/5, but in any case, i'm not loving your solution.   
07:10 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
07:11 < sipa> morcos: oh, what's the problem with it? (i can't remember the discussion about it)
07:11 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has joined #bitcoin-core-dev
07:12 -!- Cheeseo [~x@c-71-58-178-138.hsd1.pa.comcast.net] has joined #bitcoin-core-dev
07:12 -!- Cheeseo [~x@c-71-58-178-138.hsd1.pa.comcast.net] has quit [Changing host]
07:12 -!- Cheeseo [~x@unaffiliated/cheeseo] has joined #bitcoin-core-dev
07:12 < morcos> i think there is too much complication around settingbestchain that caused this to happen in the first place.  setbestchain is not expensive, lets just call it whenever we flush the utxo set, then we're only worried about keepign one thing in sync
07:13 < morcos> but also i don't like moving the unlinking of files down to where you moved it
07:13 < morcos> once you modify the blockindex database, they are useless anyway
07:13 < sipa> hmm?
07:13 < morcos> and its slightly beneficial to delete them before the CheckDiskSpace call
07:14 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has quit [Client Quit]
07:14 -!- bityogi [~textual@208-104-143-200.brvd.dsl.dyn.comporium.net] has joined #bitcoin-core-dev
07:14 < sipa> right, i started writing this under the assumption that UnlinkPrunedFiles modified the block index
07:15 < sipa> i feel there is a cyclic dependency here
07:15 < sipa> the wallet shouldn't be updated until the chainstate is updated
07:15 < sipa> the chainstate needs to have the block index flushed first
07:16 < sipa> pruning deleted files, so should be done after updating any reference to them
07:16 < morcos> we already "solved" the problem of keeping the utxo state in sync with pruning files
07:16 < morcos> lets reuse that solution
07:17 < morcos> just move setbestchain inside the block above that's if fDoFullFlush
07:17 < morcos> the utxoset and the wallet will then be permanently in sync
07:18 < sipa> well it means that the wallet can be ahead of the chainstate
07:18 < sipa> but that shouldn't be a problem, i guess?
07:19 < morcos> wait i'm confused by the terminology one of us is using
07:19 < sipa> you're suggesting to put the "GetMainSignals().SetBestChain(chainActive.GetLocator());" above the "if (!pcoinsTip->Flush())" line?
07:20 < morcos> or right after, but in that code block
07:21 < sipa> if you put it after, the wallet can be behind the chainActive, and you get the pruned beyond error
07:22 < sipa> but the wallet is allowed to be ahead of the chainstate
07:23 < morcos> sipa: no, i don't think so
07:23 < morcos> if what you were saying was true, then it would be possible for your chainstate to be behind your pruned blocks as well
07:23 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has joined #bitcoin-core-dev
07:23 < morcos> which hopeuflly isn't
07:23 < morcos> possible
07:24 < sipa> if you first write X, and then write Y, you only have a guarantee that the state on disk for X is >= that of Y
07:25 < morcos> ok so lets back up one second, what stops the problem of pruning and then not yet having written the chainstate , and crashing before you do
07:25 < sipa> nothing
07:26 < sipa> i think the correct thing to do is to first do the blockchain AND chainstate write without considering pruning, then doing findfiles to prune, then potentially writing the blockindex again, and then deleting files
07:26 < morcos> yeah, perhaps, i think we didn't do it that way because of wanting to free up space on disk with the pruning to make room for writing the chainstate
07:27 < sipa> or we could remember the last written chainstate sync, and never prune beyond that
07:27 < sipa> and then loop twice
07:28 < sipa> also, the wallet beyond chainstate check will fail in case of a reorg...
07:28 < sipa> it checks whether the wallet state is a descendant of the chainstate tip; that's too strong
07:30 < sdaftuar> sipa: where is that wallet check?  i thought we call FindForkInGlobalIndex on the wallet locator versus chainActive.Tip
07:30 < sdaftuar> and rescan from there
07:31 < morcos> OK, well to get back to a short term fix, I'd recommend not moving the UnlinkPrunedFiles call and changing the additional condition in the existing if block to be fDoFullFlush instead of fFlushForPrune
07:31 < sipa> sdaftuar: oh, yes!
07:32 < morcos> Then we can fix it more completely later, but we'll at least have reduced the problem to catastrophic crashes in the middle of FlushStateToDisk
07:34 < sipa> morcos: ack
07:34 -!- frankenmint [~frankenmi@174-25-38-39.ptld.qwest.net] has joined #bitcoin-core-dev
07:35 < sipa> morcos: actually, i think that flushing the wallet may be more expensive than chainstate flushes in some cases
07:35 < morcos> yeah thats what i was just getting confused about
07:35 < morcos> setbestchain only writes the locator
07:35 < sdaftuar> we're just writing one locator to the db, right?
07:35 < morcos> how does the rest of the wallet keep in sync with that
07:37 < sipa> the wallet is regularly checkpointed
07:38 < sipa> and any change to the wallet is written immediately
07:38 < sipa> but that's not a problem, because the wallet is allowed to be ahead of the chainstate
07:38 < sipa> and it used to also be allowed to be behind... until pruning
07:39 < morcos> yeah sorry, i'm not familiar with the wallet database, but it seems to me that there are lots of writes to the wallet that aren't immediately written, what causes those to be flushed?
07:39 -!- frankenmint [~frankenmi@174-25-38-39.ptld.qwest.net] has quit [Ping timeout: 272 seconds]
07:39 < sipa> there is a thread that occasionally flushes
07:40 < morcos> so what stops setbestchain from being ahead of that?
07:40 < sipa> nothing, i guess...
07:41 < wumpus> actually there are a lot of flushes to the wallet
07:41 < wumpus> what the 'flush thread' does is not actually flush, but make wallet.dat self-contained
07:41 < sipa> ah
07:41 -!- jannes [~jannes@178.132.211.90] has quit [Ping timeout: 245 seconds]
07:41 < sdaftuar> wumpus: can you explain?  i am trying to read/understand that now
07:41 -!- molz [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
07:41 < morcos> sdaftuar: its in the documentation
07:42 < sipa> sdaftuar: writes to the wallet are synchronous i think
07:42 < morcos> :)
07:42 < sipa> but only to the db logfile; not to wallet.dat
07:42 < sdaftuar> ah ok
07:42 < wumpus> yes it's best to check the berkeleydb documentation, but AFAIK we do a flush after almost every operation
07:42 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 276 seconds]
07:42 < wumpus> this is what made some things so slow
07:42 < sdaftuar> there are some operations that are commented as not doing a flush, for performance reasons
07:42 < sipa> morcos: what about moving the wallet update to be above the chainstate write (and always triggering it in case of a pruning)?
07:42 < wumpus> this has been improved a bit for some commands in 0.12, but it's still erring on the safe side not the unsafe one
07:43 < sipa> morcos: that way the wallet will always be ahead of the chainstate when pruning
07:43 < wumpus> sdaftuar: as I understand it, every creation of a CWalletDB(strWalletFile) makes an operation, that will be flushed when the object is destroyed
07:44 < morcos> wumpus: // Do not flush the wallet here for performance reasons // this is safe, as in case of a crash, we rescan the necessary blocks on startup through our SetBestChain-mechanism
07:44 < morcos> is a common comment
07:44 < sdaftuar> wumpus: there's an optional 3rd argument to the CWalletDB constructor, to avoid flushing on close
07:44 < wumpus> sure, there are some places where flushing is skipped, just trying to dispel the myth that the only flushing happens in the flush thread :)
07:44 < sdaftuar> ah ok
07:44 < wumpus> sdaftuar: yes
07:45 < wumpus> it was added relatively recently
07:45 < sipa> wumpus: no :)
07:45 < sipa> i think that trick existed in the 0.3.x codebase
07:45 < wumpus> ok
07:45 < sipa> we should do that during keypool topup, btw
07:45 < sipa> which is ridiculously slow
07:46 < morcos> wumpus: but what i'm trying to understand is how in those places where it isn't "flushed" is it supposed to be depending on SetBestChain to flush it?  
07:47 < wumpus> keypool topup has already been sped up in 0.12 afaik
07:47 < sipa> morcos: the flush isn't prevented; just delayed
07:47 < sipa> morcos: it flushes when that CWalletDB object goes out of scope
07:47 < wumpus> morcos:  on the next operation, or shutdown, whatever happens first
07:47 < wumpus> sipa: he means when the no-flush argument is set
07:48 < sipa> wumpus: there is no no-flush argument
07:48 < sipa> just a second CWalletDB object that lives longer; the flush only happens when there are no CWalletDB objects
07:48 < wumpus> CWalletDB(const std::string& strFilename, const char* pszMode = "r+", bool fFlushOnClose = true) : CDB(strFilename, pszMode, fFlushOnClose)
07:49 < sipa> oh!
07:49 < sipa> TIL: nobody knows how the wallet works
07:50 < phantomcircuit> what now
07:50 < sipa> wumpus: ok, i confused things
07:50 < morcos> heh, i was about to offer to shut up if it was just me beign confused
07:50 < wumpus> sipa: that's not news :)
07:50 < phantomcircuit> <morcos> if what you were saying was true, then it would be possible for your chainstate to be behind your pruned blocks as well
07:50 < phantomcircuit> morcos, that actually happens and causes a crash on restart
07:50 < sipa> wumpus: the long-living CWalletDb object exists to prevent the _checkpointing_ thread (caled ThreadFlushWalletDb)
07:51 < wumpus> that no one really understands the wallet code is a common argument why people don't like it
07:51 < sipa> wumpus: the fFlushOnClose argument is to prevent flushing to the logfile
07:51 < wumpus> sipa: right!
07:51 < wumpus> ThreadFlushWalletDb has a stupid name
07:51 < wumpus> ThreadCheckpointWalletDb would be much better
07:52 < wumpus> everyone gets confused on that
07:52 < phantomcircuit> wumpus, in practice virtually all operations actually compact the wallet since that background thread triggers within a few seconds of any operation
07:52 < wumpus> I probably only learned it about a year ago too
07:52 < morcos> you are using "checkpoint" to mean moving from logfiles to wallet.dat?
07:52 < sipa> morcos: yes, BDB terminology :)
07:52 < sipa> actually, i'm not sure
07:52 < morcos> but you are saying that all wallet writes are synchronously flushed to log files?
07:52 < phantomcircuit> sipa, same term is used in all sane sql databases
07:52 < wumpus> morcos: yes that is the assumption
07:52 < phantomcircuit> morcos, that they are, it's why the thing is so slow
07:52 < sipa> morcos: flushing to logfiles is prevented with the CWalletDb fFlushOnClose bool
07:53 < wumpus> morcos: if you can disprove it, we found a huge issue
07:53 < sipa> morcos: flushing to wallet.dat is prevented by having a concurrent long-lived CWalletDb object
07:53 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
07:53 < sipa> so SetBestChain synchronously writes to the logfile
07:54 < phantomcircuit> wumpus, trust me virtually all wallet functions cause *multiple* fsync calls
07:54 < sipa> phantomcircuit: yes, writing to the log file causes an fsync
07:55 -!- Chris_Stewart_5 [~Chris_Ste@173-31-39-168.client.mchsi.com] has quit [Ping timeout: 240 seconds]
07:55 < morcos> so can i walk through an example
07:55 < morcos> new block comes in
07:55 < morcos> you call SyncTransaction for every tx in block
07:56 < morcos> this walletdb is opened with fFlushOnClose = false
07:56 < morcos> then you call SetBestChain because its time to
07:56 < phantomcircuit> sipa, hmm actually the only places we dont flush might cause a problem
07:56 < morcos> does the call to SetBestChain, which just contains a write
07:56 < phantomcircuit> AddToWalletIfInvolvingMe doesn't cause a flush
07:56 < sipa> morcos: that will flush all writes, including the new txn
07:56 < morcos> somehow cause the AddToWalletInvolvingMe calls (inside SyncTransaction)
07:56 < morcos> ok they get flushed to
07:57 -!- jujumax [~jujumax@host154.200-45-34.telecom.net.ar] has joined #bitcoin-core-dev
07:57 < sdaftuar> so that could be a slow operation
07:57 < morcos> so thats why it can be expensive is what you're syaing
07:57 < phantomcircuit>  // this is safe, as in case of a crash, we rescan the necessary blocks on startup through our SetBestChain-mechanism
07:57 < sipa> morcos: yeah, there is a single Db* object which is cached, and every CWalletDb updates gets redirected to that
07:57 < phantomcircuit> is that correct with pruning?
07:58 < sipa> phantomcircuit: all pruning requires is that the wallet is not behind the chainstate
07:58 < sipa> so i think we should first write the wallet.setchainstate, and then flush the chainstate
07:58 < phantomcircuit> well i think in practice there's little risk of that because SetBestChain causes a wallet flush
07:59 < morcos> sipa: that sounds good to me, i doubt the order really matters b/c it seems like we have problems anyway if we crash inside FSTD but can't hurt.  But i'd make the additional condition fDoFullFlush and not fFlushForPrune if it were me...
07:59 < morcos> if nothing else, will help the wallet stay more close to caught up during a reindex
07:59 < sipa> phantomcircuit: in the current code, if you crash in between the chainstate update and the wallet update, you can see a wallet that refers to a pruned block
08:00 < phantomcircuit> sipa, and actually CDB::Flush does a checkpoint also
08:00 < phantomcircuit> txn_checkpoint
08:00 < sipa> phantomcircuit: that's a checkpoint to the logfile
08:00 < phantomcircuit> sipa, it also flushes the log file
08:00 < morcos> sipa: in i think a related question, did you see what i asked on #7491.   why do we even need to be calling MarkConflicted on loading the wallet?
08:01 < phantomcircuit> the bsb logic around this is all insanely confusing
08:01 < sipa> morcos: backward compatibility
08:01 < morcos> ?
08:01 < sipa> i think (i haven't looked)
08:01 < sipa> when you load a wallet that was written by a pre-0.12, it won't have conflict information
08:01 < phantomcircuit> for example DB->sync doesn't always cause an fsync call
08:01 < phantomcircuit> >.>
08:02 < sipa> phantomcircuit: bsb, that's BDB on LSD?
08:02 < phantomcircuit> er
08:02 < morcos> hmm...   but aren't you only creating a weird subset of the conflict information
08:02 < phantomcircuit> they're like, right next to each other mannn
08:02 < sipa> morcos: yeah, it can't be complete unless you rescan iirc
08:02 < morcos> ok
08:02 < sipa> which is in the release notes, i think!
08:03 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection]
08:03 < sdaftuar> so one question about the wallet being ahead of the chainstate: should we be checking that conflicted block information is for blocks that are on chainActive at startup?
08:04 < sdaftuar> not sure i have a sepcific broken scenario in mind
08:04 < sipa> sdaftuar: that's checked at runtime
08:04 < sdaftuar> ah okay
08:04 < sipa> if the conflicted block hash refers to a non-active block, it's considered to be non-conflicted
08:06 < phantomcircuit> i've got a question
08:06 < phantomcircuit> does anybody actually use the conflict logic?
08:07 < morcos> phantomcircuit: it was originally created to distinguish between txs that just aren't in your mempool and txs that are actually conflicted
08:08 < sipa> phantomcircuit: https://github.com/bitcoin/bitcoin/pull/7105#issuecomment-160609039
08:08 < morcos> so that you wouldn't be able to auto doublespend the first
08:09 < sipa> morcos: so i think moving the wallet flush doesn't help, unless we also move the chainstate write to be ahead of pruning
08:09 -!- Chris_Stewart_5 [~Chris_Ste@179.43.155.2] has joined #bitcoin-core-dev
08:10 < sipa> just going to change the wallet write condition
08:11 < morcos> sipa: agreed, and unmove the Unlink?  
08:11 < sipa> morcos: yes
08:11 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
08:24 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev
08:27 < morcos> sipa: Should I change CheckSequenceLocks to take a CoinsViewCache instead of create its own so that it doesn't need to be inside the mempool lock in ATMP?
08:27 < morcos> or not bother
08:31 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has quit [Max SendQ exceeded]
08:35 -!- frankenmint [~frankenmi@174-25-38-39.ptld.qwest.net] has joined #bitcoin-core-dev
08:40 -!- frankenmint [~frankenmi@174-25-38-39.ptld.qwest.net] has quit [Ping timeout: 250 seconds]
08:53 -!- wasi [~wasi@25.22.3.213.static.wline.lns.sme.cust.swisscom.ch] has quit [Quit: Leaving]
08:59 < sdaftuar> looks like travis isn't running #7502 for some reason?
09:00 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has quit [Remote host closed the connection]
09:00 < sdaftuar> never mind - it did
09:00 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has joined #bitcoin-core-dev
09:00 < phantomcircuit> sdaftuar, there's a limited number of concurrent runs available
09:00 < phantomcircuit> with heavy activity it can take a long time
09:01 < sdaftuar> oh does it take a while for it to even get queued?
09:01 < sdaftuar> looks like sipa's first commit in #7502 was run, but when he force pushed a new commit, that one hasn't been run, and doesn't seem to be queued that i can see 
09:02 < sipa> sometimes it seems to throttle
09:09 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has quit [Remote host closed the connection]
09:09 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has joined #bitcoin-core-dev
09:17 -!- zooko` [~user@50.141.117.255] has joined #bitcoin-core-dev
09:19 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Ping timeout: 250 seconds]
09:42 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
09:56 < GitHub10> [bitcoin] Kammi6187 opened pull request #7503: Master (master...master) https://github.com/bitcoin/bitcoin/pull/7503
09:57 < GitHub186> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/d007511ebdfa...c9da9c4bd83c
09:57 < GitHub186> bitcoin/master 40e7b61 Wladimir J. van der Laan: wallet: Ignore MarkConflict if block hash is not known...
09:57 < GitHub186> bitcoin/master c9da9c4 Wladimir J. van der Laan: Merge #7491: wallet: Ignore MarkConflict if block hash is not known...
09:57 < GitHub29> [bitcoin] laanwj closed pull request #7491: wallet: Ignore MarkConflict if block hash is not known (master...2016_02_wallet_markconflict_assert) https://github.com/bitcoin/bitcoin/pull/7491
09:57 < GitHub145> [bitcoin] Kammi6187 closed pull request #7503: Master (master...master) https://github.com/bitcoin/bitcoin/pull/7503
09:57 -!- molz [~molly@unaffiliated/molly] has quit [Read error: Connection reset by peer]
09:58 -!- molz [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
10:08 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev
10:12 -!- zooko` [~user@50.141.117.255] has quit [Ping timeout: 248 seconds]
10:32 -!- zooko [~user@2601:281:8001:26aa:137:7c3a:23c8:6aa5] has joined #bitcoin-core-dev
10:40 < GitHub2> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/c9da9c4bd83c...b93f07849648
10:40 < GitHub2> bitcoin/master e4eebb6 Pieter Wuille: Update the wallet best block marker when pruning
10:40 < GitHub2> bitcoin/master b93f078 Wladimir J. van der Laan: Merge #7502: Update the wallet best block marker when pruning...
10:40 < GitHub75> [bitcoin] laanwj closed pull request #7502: Update the wallet best block marker when pruning (master...betterflush) https://github.com/bitcoin/bitcoin/pull/7502
10:48 -!- zooko [~user@2601:281:8001:26aa:137:7c3a:23c8:6aa5] has quit [Ping timeout: 250 seconds]
10:54 < morcos> sipa: weird. the unit test passes for me but i don't understand how. in any case, should i just LOCK(mempool.cs) in CheckSequenceLocks or leave it as AssertLockHeld and lock it in miner_tests
10:54 < morcos> the miner test do all kinds of messing with the mempool anyway, so wouldn't be weird to lock it for the whole test
10:54 < morcos> but just want to think about moving these locks in the right direction
10:54 < morcos> its such a mess now without how much cs_main is used to protect mempool stuff already
10:55 < morcos> s/without/with/
10:55 < sipa> morcos: i generally consider functions that optionally run in a locked context bad design
10:55 < sipa> so i'd say leave the assert in CheckSequenceLocks, and add a LOCK in the tests
10:57 < morcos> any idea why the unit test passes for sdaftuar and i?
10:57 < sipa> do you run with -DDEBUG_LOCKORDER ?
10:58 -!- zooko [~user@c-73-229-199-227.hsd1.co.comcast.net] has joined #bitcoin-core-dev
10:58 < sipa> (no idea if that's done by default or only specifically from travis)
10:58 < morcos> no..  so AssertLockHeld only applies with DEBUG_LOCKORDER?
10:58 < morcos> i never knew that, i thought it was just an assert
10:59 < GitHub22> [bitcoin] laanwj pushed 2 new commits to 0.12: https://github.com/bitcoin/bitcoin/compare/827a2b6736d0...132996300134
10:59 < GitHub22> bitcoin/0.12 00ec73e Wladimir J. van der Laan: wallet: Ignore MarkConflict if block hash is not known...
10:59 < GitHub22> bitcoin/0.12 1329963 Pieter Wuille: Update the wallet best block marker when pruning...
11:00 < morcos> sipa: i'm not sure i follow exactly what you're saying about optionally running in a locked context.  you don't like it when the locks take advantage of being reentrant?  
11:01 < sipa> morcos: indeed
11:01 < sipa> morcos: locks should be internal to some level of encapsulation, and not escape it
11:01 < morcos> the downside to locking in the tests, is it'll work fine now (i think) but what happens latter if CreateNewBlock spins off new threads to do things and can't lock the mempool if we've got it locked for the whole test
11:02 < morcos> that would mean we'd need to separately lock for each call to CheckSequenceLocks in the test
11:02 < sipa> morcos: if necessary, have two versions of a function; an internal one that asserts the lock and is not public, and an external one that just locks and grabs the internal one. non-reentrant locks are also signifciantly more efficient
11:02 < sipa> morcos: that's theory, of course...
11:03 < sipa> morcos: meh, just lock once; fixing tests can be done later
11:03 < sipa> morcos: see addrman.h :)
11:04 < morcos> ha!  thats how we got into such a messy locking situation in the first place.
11:06 < sipa> morcos: the messy situation is a result of: satoshi, who wrote unintelligable but working code with relatively smart but ugly locking design that was not written down anywhere; other people took over who didn't understand that design, and race conditions appears; we got scared and added locking everywhere to be sure; now we're slowly trying to push locks down to improve performance
11:06 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has quit [Ping timeout: 240 seconds]
11:06 < morcos> so if we're pushing locks down, shouldn't the lock be in CheckSequenceLocks?
11:07 < sipa> ah, are there no callers for CheckSequenceLocks that already have the lock?
11:07 < morcos> well, there is, removeForReorg.  let me see if thats even necessary though
11:08 < morcos> yeah i guess that is necessary b/c it iterates the mempool
11:09 < sipa> you can add a tiny wrapper around CheckSequenceLocks inside the test code that grabs the lock
11:09 < morcos> ok i don't want to waste your time, i just wanted to help move things in the right direction..  
11:09 < sipa> i don't really care :)
11:10 < morcos> i'll just throw it at the top for now.. the tests are already broken with addUnchecked sitting there anyway, we can fix it when its a problem
11:10 < sipa> yeah
11:10 < morcos> oh that grabs it...  hmm ok i'll do the wrapper..
11:15 < GitHub0> [bitcoin] paveljanik opened pull request #7504: Crystal clean make clean (master...20160210_crystal_clean_make_clean) https://github.com/bitcoin/bitcoin/pull/7504
11:17 -!- afk11 [~afk11@unaffiliated/afk11] has quit [Remote host closed the connection]
11:19 -!- afk11 [~afk11@unaffiliated/afk11] has joined #bitcoin-core-dev
11:31 -!- morcos [~morcos@static-100-38-11-146.nycmny.fios.verizon.net] has quit [Remote host closed the connection]
11:32 -!- zooko [~user@c-73-229-199-227.hsd1.co.comcast.net] has quit [Ping timeout: 240 seconds]
11:32 < GitHub139> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/b93f07849648...2f3f4af4cc2b
11:32 < GitHub139> bitcoin/master 9d95187 Pieter Wuille: Correctly report high-S violations
11:32 < GitHub139> bitcoin/master 2f3f4af Wladimir J. van der Laan: Merge #7500: Correctly report high-S violations...
11:32 < GitHub165> [bitcoin] laanwj closed pull request #7500: Correctly report high-S violations (master...reporthighs) https://github.com/bitcoin/bitcoin/pull/7500
11:34 < GitHub1> [bitcoin] laanwj pushed 1 new commit to 0.12: https://github.com/bitcoin/bitcoin/commit/889e5b3050e78614acb45ea0845dc8fd33b157bf
11:34 < GitHub1> bitcoin/0.12 889e5b3 Pieter Wuille: Correctly report high-S violations...
11:40 < GitHub179> [bitcoin] laanwj pushed 2 new commits to 0.12: https://github.com/bitcoin/bitcoin/compare/889e5b3050e7...947c4ff72495
11:40 < GitHub179> bitcoin/0.12 9cb31e6 Matt: Fix spelling: misbeha{b,v}ing...
11:40 < GitHub179> bitcoin/0.12 947c4ff mrbandrews: [rpc-tests] Change solve() to use rehash...
11:45 -!- morcos [~morcos@static-100-38-11-146.nycmny.fios.verizon.net] has joined #bitcoin-core-dev
11:46 < GitHub165> [bitcoin] laanwj pushed 1 new commit to 0.12: https://github.com/bitcoin/bitcoin/commit/c3faf78c0e96a8c64a5ff8c37ef6fc4cfb35d125
11:46 < GitHub165> bitcoin/0.12 c3faf78 instagibbs: Changed getnetworkhps value to double to avoid overflow....
11:48 < wumpus> there we go, time for rc5...
11:48 < Luke-Jr> wumpus: having two sets of settings is far uglier than writing bitcoin.conf IMO
11:49 < wumpus> I don't agree, but we've already had this argument before
11:52 < Luke-Jr> wumpus: any chance that would change if I'm looking at putting like 20 more options in there?
11:53 < wumpus> well I don't want the daemon nor GUI writing to bitcoin.conf. There should be no assumption that the configuration file is writing at all
11:53 < wumpus> writable*
11:53 < Luke-Jr> how about a separate .conf writable loaded alongside bitcoin.conf?
11:53 < wumpus> the number of options doesn't change that
11:53 < wumpus> what's wrong with the qsettings?
11:54 < wumpus> it is like a separate .conf writable alongside bitcoin.conf, except that it is the OS-sanctified way of storing application settings
11:54 -!- wallet42 [~wallet42@172.56.19.6] has joined #bitcoin-core-dev
11:54 < Luke-Jr> and the var names are all different
11:54 < Luke-Jr> and it won't get picked up on bitcoind
11:55 < wumpus> yes, the difference in naming is ugly, that could be solved with some kind of mapping for old settings + use the same name for new settings
11:55 < wumpus> that naming is inherited from satoshi-ism, the old place for the settings used to be the wallet(!)
11:55 < Luke-Jr> heh
11:58 < Luke-Jr> wumpus: and no value in bitcoind using the same settings? ;)
11:59 < wumpus> and remember that whatever you do has to be backwards compatible, so what you imply is to add *another* source of settings. The initialization code (making sure that the option settings take precedence in the right order) is already complex enough. I'd really prefer not to tweak with it
11:59 < wumpus> we have enough actual issues to fix...
12:00 < Luke-Jr> well, not really another. I'd just load two conf files. :p
12:01 < Luke-Jr> and start with just the new settings (which are mostly policy-related) in the new one
12:01 -!- Guest87 [~textual@s529c4fa8.adsl.online.nl] has joined #bitcoin-core-dev
12:03 -!- Guest87 [~textual@s529c4fa8.adsl.online.nl] has quit [Client Quit]
12:05 -!- murch [~murch@p4FE38275.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
12:14 -!- skyraider_ [uid41097@gateway/web/irccloud.com/x-tbrswmrkmlmqlxbo] has joined #bitcoin-core-dev
12:19 < GitHub3> [bitcoin] laanwj pushed 2 new commits to 0.12: https://github.com/bitcoin/bitcoin/compare/c3faf78c0e96...68134263e2bf
12:19 < GitHub3> bitcoin/0.12 10be44a Wladimir J. van der Laan: doc: Release notes update pre-rc5
12:19 < GitHub3> bitcoin/0.12 6813426 Wladimir J. van der Laan: qt: Translation update pre-rc5
12:19 < gmaxwell> Hurray RC5.
12:21 < wumpus>  * [new tag]         v0.12.0rc5 -> v0.12.0rc5
12:21 < wumpus> the really-really-really last rc :p
12:22 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
12:28 < michagogo> Did rc4 detached sigs happen?
12:29 < wumpus> michagogo: yes
12:30 < sipa> wumpus: instagibbs == Gregory Sanders
12:30 < wumpus> no binaries, though
12:30 < wumpus> sipa: ok!
12:31 < wumpus> sipa: looks like he is using different git author names
12:34 < Luke-Jr> need a rc6 for .git problem
12:35 < Luke-Jr> half-/s
12:35 < sipa> half-/s ?
12:38 < GitHub120> [bitcoin] laanwj pushed 1 new commit to 0.12: https://github.com/bitcoin/bitcoin/commit/772863583c35e4344c0020445ede04d7a2e5837b
12:38 < GitHub120> bitcoin/0.12 7728635 Wladimir J. van der Laan: doc: fix author list in release notes
12:43 < Luke-Jr> sipa: half-sarcasm.
12:43 < sipa> ha
12:44 < Luke-Jr> it's a bug I absolutely need fixed to do Gentoo packaging, but I can throw it in with the system library stuff
12:44 < wumpus> build system changes that don't influence the executable don't really warrant a new rc
12:46 < Luke-Jr> it might. I noticed there are weird circumstances over when the 'g' prefixes the commit hash or not
12:46 < wumpus> then again, you still have to fight it out with cfields what the genbuild.sh check should do at all, no way that will make 0.12.0 in teh first place :)
12:46 < Luke-Jr> I didn't think it was worth looking into further at the time
12:47 < wumpus> I don't think so either, it works ok, any well-meant change will probably break it in some way again
12:47 < wumpus> are we really at the point that we need a test suite for the build system :p
12:47 < Luke-Jr> I mean the build difference
12:47 < Luke-Jr> I was wondering the other day..
12:48 < Luke-Jr> how can we add a test suite for the test suite? :P
12:48 < wumpus> lol
12:48 < sipa> seems we need a recursive test system
12:48 < sipa> which is able to test itself
12:48 < OxADADA> we should also have 100% coverage of the test of the tests
12:48 < OxADADA> :P
12:49 < wumpus> quine test
12:53 < OxADADA> my favorite test runner is the one that when detecting errors, deletes the offending function from the source code; recursively until no error is thrown.
12:54 < sipa> what if the error is in main() ? 
12:54 < OxADADA> sipa: it will delete main()
12:54 < paveljanik> it generates new one.
12:55 < sipa> which will result in a linker error :p
12:55 < sipa> complaining about the lack of main()
12:55 < OxADADA> the most bug-free line of code is the line that was never written.
12:57 < wumpus> I certainly prefer pulls that delete code to those that add code :)
13:06 < morcos> wumpus: on that note and Luke-Jr this is mostly directed at you.   are you guys ok with deleting the free transaction code from AcceptToMemoryPool then?  I agree deleting from wallet is a first step, and maybe in an ideal world would have happened in an earlier release.  But it seems to me by the time 0.13 is released its a lost cause anwyay.
13:07 < morcos> Luke-Jr: what do you think if I create a PR to remove sending free transactions first (i suspect it'll probably be too big to back port to 0.12.1 though) and then follow on with a mempool cleanup.
13:07 < Luke-Jr> morcos: IMO one of the conditions must first be met 1) removal from wallet in prior release; 2) de facto never working for the forseeable future
13:07 < morcos> yeah i think any reasonable adoption of 0.12 is going to cause it to just not work...   
13:07 < Luke-Jr> although I would prefer maintaining relay of non-free txns as long as possible
13:07 < wumpus> I tend to agree with Luke-Jr, sending them should be gone (or at least heavily discouraged) for a while then
13:08 < morcos> wumpus: its been defaulted off for some time
13:08 < wumpus> right
13:08 < Luke-Jr> I still never use a fee myself FWIW
13:08 < morcos> Ok, so lets break this into 2 parts
13:08 < Luke-Jr> as long as I can successfully do so, I plan to re-add it to my own wallet ;)
13:08 < morcos> Part 1) you're ok with removing the ability to send them?
13:09 < wumpus> yeah, free transactions still seem to be working at this point
13:09 < Luke-Jr> morcos: part 1, I prefer not to unless there is a good reason
13:09 < wumpus> may take a long time to be included though, but not everyone is in a hurry
13:09 < Luke-Jr> (expecting it to stop working before the next release would be a good reason)
13:09 < morcos> well the reason i'm asking is i feel like cleaning up stuff like this towards the beginning of a release cycle makes other work easier
13:10 < Luke-Jr> removing features is not really cleaning up. :p
13:10 < wumpus> does it really make other work that much easier?
13:10 < morcos> i find it hard to imagine its going to work.  i think the size of your 0.12 mempool would need to be multiple GB's in order to not get full.  
13:11 < morcos> wumpus: yes, both in the wallet and in ATMP, you have to be careful around the special casing for free/priority stuff.  Maybe thats only a limited area, but I seem to end up in that area a lot.
13:11 < Luke-Jr> I think we'll be in a much better place to discuss this once 0.12 is released for a while
13:11 < Luke-Jr> right now there's just too much to speculate on
13:11 < morcos> okey dokey (is that what I'm supposed to say)
13:12 < sipa> haha
13:12 < morcos> Luke-Jr: alright related question
13:12 < morcos> do you care about priority estimation
13:13 < Luke-Jr> as things stand right now, I prefer gratis transactions to continue working as long as possible, but I won't fight removal of it like I am with priority-mining ;)
13:13 < Luke-Jr> morcos: no
13:13 < morcos> i think you are right that until and unless miners stop doing priority mining, fee estimation has the same problem with priority txs
13:13 < Luke-Jr> priority is a fallback and anti-spam measure, it shouldn't be relied on for more than that IMO
13:14 < morcos> but i want to decide if maintaining the priority estimation code is worth the benefit it has to fee estimation.  its unclear at this point.
13:14 < morcos> ok, so if improving fee estimation results in the removal of priority estimation, no big complaints from anyone?
13:14 < Luke-Jr> I wasn't aware we had priority estimation code XD
13:15 < Luke-Jr> unless you mean the priority display in coin control, which I find handy, but could live without if there's a benefit to removing it
13:15 < sipa> Luke-Jr: so what do you think the Bitcoin Core wallet should do in the near future? always use fee estimation, and pay the estimated/chosen fee?
13:15 < morcos> Luke-Jr: hmm.. good point, that used to return some hardcoded value, i think now it returns something based off priority estimation
13:15 < Luke-Jr> actually, I'd probably hack it back in..
13:16 < Luke-Jr> sipa: by default at least.
13:16 < Luke-Jr> sipa: (this is already the case)
13:16 < morcos> sipa: i'm confused by the question.  isn't that what it currently does?
13:16 < Luke-Jr> morcos: I did much prefer the pre-estimation priority estimates in coin control, actually
13:16 -!- adnn [~adnn@cpe-158-222-198-108.nyc.res.rr.com] has joined #bitcoin-core-dev
13:17 < Luke-Jr> morcos: found it strange that priority changed with the slider..
13:18 < morcos> Luke-Jr: i think it was a small change, might be easy to go back...   but i'm not going to remove priority estimation unless i significantly rework the estimation code which is unknown right now.
13:19 < sipa> morcos: i mean: should paying free transactions still be supported?
13:19 < sipa> i never use Qt; i don't actually know what the sending dialog looks like currently
13:20 < morcos> Well thats what I was starting by asking him about.  Right now you can checkbox to sendfree if possible
13:20 < Luke-Jr> brb
13:20 < morcos> Luke-Jr and wumpus would like to not remove that yet, but are open to doing it for 0.13 if they become useless anyway due to limited mempools (which seems likely IMO)
13:21 < sipa> that seems reasonable
13:21 < sipa> but at least removing the priority estimation code could be done noe
13:21 < Luke-Jr> note: also much less likely to be controversial if we wait until it breaks
13:21 < morcos> well, its a bit circular
13:21 < wumpus> right
13:22 < morcos> its actually the priority estimation code that stops you from doing something stupid and trying to send a free tx when your own mempool is limited
13:22 < morcos> it also won't let you send a free tx with priority less than that returned from the estimates
13:22 < wumpus> I don't think supporting sending free transactions, for people that really want to and understand the risks, is so much of a maintenance burden
13:22 < morcos> which haven't quite disappeared entirely yet
13:22 < wumpus> and it is already not the default, you could hide the checkbox further, etc
13:23 < morcos> wumpus: I think the UI in the QT makes it a little too easy to try to do that
13:23 < morcos> yep
13:23 < wumpus> but people seem to understand it
13:23 < jtimon> why remove free transaction? not being default? obviously. a warning message discouraging it? why not?, but why prohibit the functionality forcing people like luke to remove the unwanted restriction?
13:23 < sipa> you could use the coin control dialog is you really want to create a gratis transaction
13:23 < morcos> well except they are about to become way way more unreliable
13:23 < wumpus> everyone seems to intuitively know that they need to attach a fee to get transactions confirm
13:23 < wumpus> jtimon: yeah, exactly
13:23 < wumpus> jtimon: seems too much user babysitting
13:24 < morcos> i guess i'm confused as to what the purpose of them is since the current design of mempools doesn't relay them 99% of the time
13:24 < sipa> i care much more about the complexity of priority sorting, priority-based relay acceptance, and priority estimation  than about the mere functionality of free transactions
13:24 < Luke-Jr> sipa: coin control does not at this time allow overriding fees
13:25 < Luke-Jr> adding it there makes sense though
13:25 < wumpus> it could be moved there
13:25 < morcos> sipa: aren't those all intertwined.  if your own mempool won't accept it, its bad to let you send it
13:25 < morcos> it just ties up your inputs
13:26 < Luke-Jr> there used to be a "no forced fees" fork. if we just allow fee overriding in coin control, we can satisfy those people too :p
13:26 < jtimon> are free transactions currently the default for the wallet?
13:26 < wumpus> jtimon: no
13:27 < jtimon> then at most I would just discourage them
13:27 < wumpus> yes
13:27 < morcos> wumpus: perhaps a valuable UI fix would be to make the checkbox clear after each tx or something.  the fact that the setting gets saved is a bit disturbing
13:27 < wumpus> agreed. on the other hand that's what people expect now, changing it is also going to disturb some people :(
13:28 < jtimon> like sipa says, the hard part is the priority/separated space (not having a uniform metric to order all transactions) 
13:28 < morcos> so just so we're on the same page here
13:28 < Luke-Jr> replacing it with a fee override in coin control seems ideal to me now IMO
13:28 -!- wallet42 [~wallet42@172.56.19.6] has quit [Quit: Leaving.]
13:28 < wumpus> yes if you move the checkbox too you can change its behavior
13:28 < morcos> you want to preserve the ability to send free txs, so that if you start your node and quickly send a tx before your mempool fills up you can send one (even though it likely won't propagate far)
13:28 < Luke-Jr> fee override can ignore whether it's "safe" or not, so you don't need to calculate that anymore
13:29 < morcos> it just seems a bit crazy to me that you can only do it in this race condition
13:29 < Luke-Jr> morcos: it's possible the spam stops after >1 MB blocks
13:29 < Luke-Jr> some of it anyway
13:29 < wumpus> it's more of a philosophical issue, people don't want to be forced by their software to do something, even if it doesn't really make sense
13:29 < Luke-Jr> also, the new spam filtering based on descendents can help
13:30 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has quit [Quit: laurentmt]
13:30 < morcos> ok, well i just realized it might stop working entirely anyway
13:30 < jtimon> morcos: who cares? don't use the functionality if you think it's stupid, just like I don't "echo asdgsdfgh > /dev/null" anymore 
13:31 < morcos> it used to be the case that if you couldn't get priority estimates you defaulted to allowminfree priority
13:31 < Luke-Jr> it will probably be a long time before non-spam can fill mempools
13:31 < morcos> this caused people who hadn't gotten estimates yet to send free txs with way too low priorities
13:31 < morcos> so i changed it so that if you have no priority estimates, it requires infinite priority (so you can't send free txs)
13:32 < morcos> i believe, although i can't be sure, that once 0.12 is rolled out in enough places, priority estimates will not have enough data and will always return -1 
13:32 < morcos> since that state is saved, it won't matter what you do with restarting your node or your mempool, your node won't let you send a free tx
13:32 < Luke-Jr> morcos: I'm curious why you seem so sure that mempools will be full
13:33 < morcos> although arguably thats exactly what we want?   if you can configure your node in such a way that it receives enough data points to see free txs still being mined, it'll let you send one
13:34 < morcos> so maybe that removes the concern of eliminating the sending ability a cycle before , and now we just have to wait and see what happens, and if it turns out no one can ever send them (bc limited mempools), then we can consider removing relay of them
13:34 < morcos> Luke-Jr: it depends on your minrelayfee i suppose.  certainly with 1000 sat/kB they fill up.
13:34 < Luke-Jr> morcos: and your spam filtering capabilities
13:35 < morcos> I have a 0.12 node up and running for about a week now with a 2GB mempool, its filled up to 1.8G so far
13:35 < Luke-Jr> weird
13:36 < morcos> its all txs at the lowest fee rate.
13:36 < Luke-Jr> I wonder if we have regressions; my 0.10 node never overflowed my 32-bit process space
13:36 < Luke-Jr> and I didn't set a minrelayfee on it
13:36 < sipa> we keep much larger utxo caches now
13:37 < sipa> in 0.10 nearly every block would flush it (past ibd)
13:37 < morcos> a related problem i discussed with gmaxwell a couple days ago is we need to stop inving all these txs that will fall below most nodes effective min
13:37 < morcos> i was thinking of implementing a p2p feefilter message
13:37 < paveljanik> morcos, can you please post btc getrawmempool true | grep '"fee"' | sort | uniq -c | sort -rn | head from such node?
13:38 < morcos> paveljanik: sure but i'm outputting more detailed stats, what would you like to know
13:38 < Luke-Jr> 0.10's minrelaytxfee is 5000
13:38 < paveljanik> I'd like to see some oldest transactions coming from the most frequently used fee...
13:39 < paveljanik> is it the remnant from the spam in July?
13:39 < paveljanik> hmm, I can maybe try it myself on some node.
13:40 < morcos> http://0bin.net/paste/R0vBCFDb3poeSK5O#i5IWrBXUvMBB6-zgfS4WoizXaGBBpL4pvIoAiRqcIXe
13:40 < morcos> that doesn't seem like what you wanted?
13:40 -!- goregrin1 [~goregrind@unaffiliated/goregrind] has joined #bitcoin-core-dev
13:40 < paveljanik> but this is the same I investigated in Dec.
13:40 < paveljanik> 15000 and 192 satoshis
13:40 < paveljanik> Thank you
13:42 < morcos> anyway, idea of a feefilter is you tell other nodes not even to bother inv'ing you stuff below the feerate your accepting now
13:42 < morcos> would save a lot of inv bandwidth
13:42 < morcos> but only works in the situation that you're not ALSO accepting free txs below your min fee rate (which is the case if your fee rate is caused by mempool limiting)
13:43 -!- pigeons_ [~pigeons@94.242.209.214] has joined #bitcoin-core-dev
13:43 -!- PRab_ [~chatzilla@c-68-34-102-231.hsd1.mi.comcast.net] has joined #bitcoin-core-dev
13:43 -!- nullpt_ [~pi@gateway/vpn/privateinternetaccess/nullpt] has joined #bitcoin-core-dev
13:44 -!- adam3us [~Adium@unaffiliated/adam3us] has joined #bitcoin-core-dev
13:47 -!- lesderid_ [~lesderid@anna.lesderid.net] has joined #bitcoin-core-dev
13:48 -!- Netsplit *.net <-> *.split quits: lesderid, teward, PRab, guruvan, goregrind, nullpt, BashCo, pigeons
13:48 -!- BashCo_ [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev
13:48 -!- PRab_ is now known as PRab
13:50 -!- lesderid_ is now known as lesderid
13:50 -!- Netsplit over, joins: guruvan
13:52 -!- Netsplit over, joins: teward
14:03 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Quit: Leaving]
14:08 -!- wallet42 [~wallet42@2604:2000:ef4b:c500:3903:a921:73ac:a77] has joined #bitcoin-core-dev
14:10 < Luke-Jr> weird, when the same setting is set multiple times in bitcoin.conf, it's the first that has effect
14:13 < sipa> Luke-Jr: ha, i never knew that!
14:17 -!- skyraider__ [uid41097@gateway/web/irccloud.com/x-teazntwyuswwkxyu] has joined #bitcoin-core-dev
14:22 -!- btcdrak_ [uid115429@gateway/web/irccloud.com/x-wxmabpmrqhklrimy] has joined #bitcoin-core-dev
14:22 -!- skyraider_ [uid41097@gateway/web/irccloud.com/x-tbrswmrkmlmqlxbo] has quit [Ping timeout: 240 seconds]
14:22 -!- teward [teward@ubuntu/member/teward] has quit [Ping timeout: 240 seconds]
14:22 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has quit [Ping timeout: 240 seconds]
14:22 -!- btcdrak [uid115429@gateway/web/irccloud.com/x-phuigjwogyhiesjs] has quit [Ping timeout: 240 seconds]
14:23 -!- skyraider__ is now known as skyraider_
14:23 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has joined #bitcoin-core-dev
14:23 -!- teward [teward@ubuntu/member/teward] has joined #bitcoin-core-dev
14:24 -!- murch [~murch@p4FE38275.dip0.t-ipconnect.de] has quit [Quit: Leaving.]
14:24 -!- btcdrak_ is now known as btcdrak
14:25 -!- frankenmint [~frankenmi@174-25-38-39.ptld.qwest.net] has joined #bitcoin-core-dev
14:29 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Ping timeout: 260 seconds]
14:35 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Ping timeout: 245 seconds]
14:44 -!- wasi [~wasi@25.22.3.213.static.wline.lns.sme.cust.swisscom.ch] has joined #bitcoin-core-dev
14:46 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
14:55 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has quit [Remote host closed the connection]
14:55 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has joined #bitcoin-core-dev
15:01 -!- wallet42 [~wallet42@2604:2000:ef4b:c500:3903:a921:73ac:a77] has quit [Quit: Leaving.]
15:05 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Ping timeout: 250 seconds]
15:07 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]
15:12 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
15:12 -!- molz [~molly@unaffiliated/molly] has quit [Ping timeout: 276 seconds]
15:34 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
15:41 -!- molz [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
15:42 -!- pigeons_ is now known as pigeons
15:45 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 276 seconds]
15:48 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has quit [Read error: Connection reset by peer]
15:49 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has joined #bitcoin-core-dev
15:49 -!- frankenmint [~frankenmi@174-25-38-39.ptld.qwest.net] has quit [Remote host closed the connection]
15:52 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has joined #bitcoin-core-dev
16:08 -!- frankenmint [~frankenmi@174-25-38-39.ptld.qwest.net] has joined #bitcoin-core-dev
16:20 -!- skyraider_ [uid41097@gateway/web/irccloud.com/x-teazntwyuswwkxyu] has quit [Quit: Connection closed for inactivity]
16:24 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has joined #bitcoin-core-dev
16:24 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has quit [Client Quit]
16:28 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-gttnrltrvobxdnbk] has quit [Quit: Connection closed for inactivity]
16:50 -!- goregrind [~goregrind@unaffiliated/goregrind] has joined #bitcoin-core-dev
16:53 -!- goregrin1 [~goregrind@unaffiliated/goregrind] has quit [Ping timeout: 245 seconds]
17:09 < GitHub59> [bitcoin] promag opened pull request #7506: Use CCoinControl selection in CWallet::FundTransaction (master...enhancement/use-coin-control-selection) https://github.com/bitcoin/bitcoin/pull/7506
17:11 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has quit [Remote host closed the connection]
17:16 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has joined #bitcoin-core-dev
17:28 -!- wallet42 [~wallet42@2604:2000:ef4b:c500:30f8:822e:cc44:61b3] has joined #bitcoin-core-dev
17:34 -!- mkarrer_ [~mkarrer@75.Red-83-43-123.dynamicIP.rima-tde.net] has joined #bitcoin-core-dev
17:36 -!- mkarrer [~mkarrer@75.Red-83-43-123.dynamicIP.rima-tde.net] has quit [Ping timeout: 245 seconds]
18:10 -!- lecusemb1e [~lecusembl@f9beb4d9.violates.me] has joined #bitcoin-core-dev
18:12 -!- justanot1eruser [~Justan@unaffiliated/justanotheruser] has joined #bitcoin-core-dev
18:13 -!- justanotheruser [~Justan@unaffiliated/justanotheruser] has quit [Read error: Connection reset by peer]
18:14 -!- afk11_ [~afk11@109.255.154.81] has joined #bitcoin-core-dev
18:15 -!- Netsplit *.net <-> *.split quits: afk11, lecusemble
18:26 -!- bityogi [~textual@208-104-143-200.brvd.dsl.dyn.comporium.net] has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
18:29 < GitHub87> [bitcoin] Leviathn opened pull request #7507: Remove internal miner (master...master) https://github.com/bitcoin/bitcoin/pull/7507
18:58 -!- jtimon [~quassel@126.31.134.37.dynamic.jazztel.es] has quit [Ping timeout: 245 seconds]
19:16 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has quit [Remote host closed the connection]
19:19 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has joined #bitcoin-core-dev
19:20 -!- Evel-Knievel [~Evel-Knie@d5152f744.static.telenet.be] has quit [Read error: Connection reset by peer]
19:21 -!- Evel-Knievel [~Evel-Knie@d5152f744.static.telenet.be] has joined #bitcoin-core-dev
19:21 -!- afk11_ [~afk11@109.255.154.81] has quit [Ping timeout: 240 seconds]
19:22 -!- afk11 [~afk11@unaffiliated/afk11] has joined #bitcoin-core-dev
19:33 -!- Chris_Stewart_5 [~Chris_Ste@179.43.155.2] has quit [Ping timeout: 276 seconds]
19:38 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has quit [Ping timeout: 256 seconds]
19:48 -!- justanot1eruser is now known as justanotheruser
20:06 -!- wasi [~wasi@25.22.3.213.static.wline.lns.sme.cust.swisscom.ch] has quit [Ping timeout: 240 seconds]
20:28 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
20:49 -!- p15 [~p15@71.91.145.64.client.static.strong-tk2.bringover.net] has joined #bitcoin-core-dev
21:12 -!- Cory [~C@unaffiliated/cory] has quit [Ping timeout: 256 seconds]
21:15 -!- Pasha [~C@unaffiliated/cory] has joined #bitcoin-core-dev
21:18 -!- wasi [~wasi@25.22.3.213.static.wline.lns.sme.cust.swisscom.ch] has joined #bitcoin-core-dev
21:22 -!- Pasha is now known as Cory
21:26 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has quit [Ping timeout: 245 seconds]
21:27 -!- wallet42 [~wallet42@2604:2000:ef4b:c500:30f8:822e:cc44:61b3] has quit [Quit: Leaving.]
21:42 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
22:25 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has quit [Ping timeout: 252 seconds]
22:32 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
22:38 < GitHub131> [bitcoin] luke-jr opened pull request #7509: Common argument defaults for NODE_BLOOM stuff and -wallet (master...common_defaults_0.12) https://github.com/bitcoin/bitcoin/pull/7509
22:46 -!- dermoth [~thomas@189-79.162.dsl.aei.ca] has joined #bitcoin-core-dev
22:58 < GitHub178> [bitcoin] luke-jr opened pull request #7510: Read/write bitcoin_rw.conf for exposing shared Daemon/GUI options in the GUI (master...rwconf) https://github.com/bitcoin/bitcoin/pull/7510
23:04 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has quit [Ping timeout: 276 seconds]
23:06 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev
23:11 -!- Sparyx [~Sparyx@c-73-224-87-206.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
23:18 < GitHub1> [bitcoin] paveljanik opened pull request #7511: [WIP] New ax_pthread.m4 from upstream - draft 3 (not final), for testing on all platforms (master...20160211_WIP_test_new_ax_pthread) https://github.com/bitcoin/bitcoin/pull/7511
23:22 -!- BashCo_ [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection]
23:29 -!- Alopex [~bitcoin@cyber.dealing.ninja] has quit [Remote host closed the connection]
23:30 -!- Alopex [~bitcoin@cyber.dealing.ninja] has joined #bitcoin-core-dev
23:41 < GitHub17> [bitcoin] laanwj closed pull request #7493: rpc: add null assert (master...patch-1) https://github.com/bitcoin/bitcoin/pull/7493
23:52 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-rrmmiqojrakeklsd] has joined #bitcoin-core-dev