--- Day changed Thu Apr 20 2017 00:03 -!- goksinen [~goksinen@2604:2000:c591:8400:380c:3163:2c88:cc60] has quit [Ping timeout: 260 seconds] 00:04 -!- tw2006 [~tw2006@2601:187:8480:2770:fd35:b686:77de:d7f1] has joined #bitcoin-core-dev 00:08 -!- tw2006 [~tw2006@2601:187:8480:2770:fd35:b686:77de:d7f1] has quit [Ping timeout: 260 seconds] 00:15 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 00:17 -!- d_t [~textual@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 00:20 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 240 seconds] 00:22 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 00:30 < sipa> NicolasDorier: we already have a maximum size on the cache 00:30 < sipa> it's what -dbcache controls 00:30 < NicolasDorier> I know 00:31 < sipa> then i don't understand your suggestion 00:32 < NicolasDorier> sipa: basically following the same thing as done here 00:32 < NicolasDorier> https://github.com/bitcoin/bitcoin/blob/2584925077f9658b3953ad931b74779006e59807/src/validation.cpp#L1982 00:32 < NicolasDorier> with a fCacheCriticalBatchSize 00:33 < NicolasDorier> the problem that I found out by trying that 00:33 < NicolasDorier> is that I was not expecting a flush to wipeout the cache clean 00:33 < NicolasDorier> I thought only dirty entries in the cache would be commited into a batch 00:33 < sipa> i think you misunderstand what this does 00:34 < sipa> it's not about limiting the number of dirty entries 00:34 < NicolasDorier> This flush the CoinViewCache when certain condition arise 00:34 < NicolasDorier> right ? 00:34 < sipa> it's splitting them up over different batches 00:34 < sipa> we want to limit the size of the batches, as they cause a memory blowup 00:34 < NicolasDorier> I know 00:35 < NicolasDorier> My point is that we can prevent a batch to be too big at the CoinViewCache level 00:35 < sipa> no 00:36 < NicolasDorier> I was assuming CoinViewCache was not deleting all cached coin during a flush 00:36 < NicolasDorier> I was wrong on that 00:36 < sipa> yeah, that's weird 00:36 < sipa> but we've tried mny things to change that, and they all make things slower 00:36 < NicolasDorier> if it was not deleting all cached coin during a flush, then we could flush when the CoinViewCache know that a batch size would be too big 00:37 < NicolasDorier> ha I see 00:37 < sipa> that's an eventual goal here 00:37 < sipa> but this is doing something much more basic: not requiring that flushes are consistent with blocks 00:38 < sipa> let me take a step back 00:38 < NicolasDorier> I understood your PR and the goal of it 00:39 < NicolasDorier> I thought there was a simpler way of doing, but I relied on the assumption that a flush did not throw away all the cachedCoins 00:39 < NicolasDorier> if my assumption was right, fixing the problem you attempt to solve would have been way more easy. 00:40 < sipa> the reason why we have the cache, and go through all this effort (as opposed to just using leveldb's caching), is that we can do an awesome optimization: if a utxo is created in the cache, and spent before it is ever flushed, we can just delete it from the cache, preventing it from ever being serialized or written to disk 00:40 < NicolasDorier> yes I am aware of it 00:40 < sipa> that means that we must maximize the time a utxo is in the cache before being flushed 00:41 < sipa> what you're suggesting is limiting the amount of dirty entries in the cache 00:41 < NicolasDorier> right 00:41 < sipa> that would radically reduce our ability to use that optimization above 00:41 < NicolasDorier> yes because at every flush 00:41 < NicolasDorier> you throw away all the cachedCoins 00:41 < sipa> no 00:41 < sipa> that has nothing to do with it 00:42 < sipa> it's more fundamental 00:42 < NicolasDorier> ah yes let me think 00:42 < sipa> you're suggesting more frequent flushes (which don't wipe the cache, just mark the written entries as non-dirty, roght?) 00:42 < NicolasDorier> yes 00:43 < sipa> more frequent flushes would reduce the time between a utxo is created and the time they hit disk 00:43 < NicolasDorier> aaaaah 00:43 < NicolasDorier> got it 00:43 < sipa> at that point, it's too late 00:43 < NicolasDorier> yes, I understand now 00:43 < NicolasDorier> I remove my comment 00:44 < NicolasDorier> removed 00:44 < sipa> so my eventual goal is to have continuous flushing that indeed only writes small amounts of entries 00:44 < sipa> but not by limiting the amount of dirty entries 00:44 -!- JackH [~laptop@79-73-191-98.dynamic.dsl.as9105.com] has joined #bitcoin-core-dev 00:44 < NicolasDorier> mmh so how ? 00:45 < sipa> just have sort of a rolling window... once an entry has been in the cache without being written, flush it 00:45 < sipa> however, that requires that the disk cache is allowed to be in an inconsistent state 00:45 < sipa> and needs replay to correct at startup 00:46 < sipa> which this PR is the first step towards (and a great memory usage improvement on its own) 00:46 < NicolasDorier> I see 00:47 < sipa> it's a very usual cache structure, which only works because we mostly create and delete recently created entries 00:47 < sipa> and only write them once, read them once, delete them once 00:47 < sipa> most caches are optimized for many reads 00:48 < NicolasDorier> ok this is clear thanks. So I am fine with 10148, would just like to see a simple python test though 00:49 < sipa> agree, if you have ideas for what it should do, let me know 00:51 < NicolasDorier> sipa: A dbbatchsize of 1 bytes, dbcrashratio of 10. Check if the node can eventually sync 200 blocks and have the same utxoset hash than the other node. 00:51 < NicolasDorier> does not test the forking logic though 00:51 < sipa> ah, yes 00:52 < sipa> well we could construct forks as well, i guess 00:52 < NicolasDorier> yes, I don't think this is very difficult to test. Just the dbbatchsize being in MB make it inconvenient 00:52 < NicolasDorier> maybe have a magic number just for the tests be enough 00:52 < sipa> agree 00:53 -!- goksinen [~goksinen@2604:2000:c591:8400:380c:3163:2c88:cc60] has joined #bitcoin-core-dev 00:53 < sipa> it can be in bytes 00:53 < sipa> it's a test-only option really 00:53 < NicolasDorier> yes indeed 00:54 -!- cysm_ [cysm@gateway/shell/elitebnc/x-byaivjzytfwheiqo] has quit [Excess Flood] 00:54 -!- xinxi [~xinxi@116.87.187.139] has joined #bitcoin-core-dev 00:56 -!- cysm_ [cysm@gateway/shell/elitebnc/x-kpoozktxvtftwzgd] has joined #bitcoin-core-dev 00:57 -!- goksinen [~goksinen@2604:2000:c591:8400:380c:3163:2c88:cc60] has quit [Ping timeout: 260 seconds] 01:06 < xinxi> gmaxwell: Please check my PM. 01:20 -!- RubenSomsen [~RubenSoms@5ED2CA1D.cm-7-3d.dynamic.ziggo.nl] has quit [Ping timeout: 255 seconds] 01:21 -!- timothy [tredaelli@redhat/timothy] has joined #bitcoin-core-dev 01:24 -!- xinxi [~xinxi@116.87.187.139] has quit [Remote host closed the connection] 01:24 -!- xinxi [~xinxi@116.87.187.139] has joined #bitcoin-core-dev 01:29 -!- xinxi [~xinxi@116.87.187.139] has quit [Ping timeout: 240 seconds] 01:35 -!- jtimon [~quassel@9.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 01:46 -!- xinxi [~xinxi@101.100.176.174] has joined #bitcoin-core-dev 01:51 -!- CubicEarth [~cubiceart@c-67-168-4-85.hsd1.wa.comcast.net] has quit [] 01:53 -!- tw2006 [~tw2006@2601:187:8480:2770:51e2:2151:ffbd:732d] has joined #bitcoin-core-dev 01:57 -!- tw2006 [~tw2006@2601:187:8480:2770:51e2:2151:ffbd:732d] has quit [Ping timeout: 260 seconds] 02:03 <@wumpus> indeed, no specific reason that abortrescan should not allowed in safe mode, though should anything that triggers rescan be allowed? 02:04 <@wumpus> safe mode = the block chain is in uncertain state 02:05 <@wumpus> I guess it cannot really hurt 02:06 -!- xinxi_ [~xinxi@101.100.176.174] has joined #bitcoin-core-dev 02:06 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Quit: Leaving] 02:07 -!- paveljanik [~paveljani@79.98.72.176] has joined #bitcoin-core-dev 02:07 -!- paveljanik [~paveljani@79.98.72.176] has quit [Changing host] 02:07 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev 02:08 -!- xinxi_ [~xinxi@101.100.176.174] has quit [Remote host closed the connection] 02:09 -!- xinxi_ [~xinxi@101.100.176.174] has joined #bitcoin-core-dev 02:09 -!- xinxi [~xinxi@101.100.176.174] has quit [Ping timeout: 260 seconds] 02:13 -!- xinxi_ [~xinxi@101.100.176.174] has quit [Ping timeout: 245 seconds] 02:15 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 02:25 -!- harrymm [~wayne@104.237.91.140] has quit [Ping timeout: 240 seconds] 02:28 < bitcoin-git> [bitcoin] laanwj closed pull request #10232: [0.14] release-notes: Accurately explain getblocktemplate improvements (0.14...0.14_relnotes_mining) https://github.com/bitcoin/bitcoin/pull/10232 02:41 < gmaxwell> sipa: I think for testing that atomic flushing what we really need is a few bugs to insert in the code, and see that the tests catch them... at least that would give some feel for how adequate the tests are. 02:44 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 02:46 < jonasschnelli> gmaxwell: You once mentioned that there are better filters for "block filters" then bloom. I think what mostly matters is the compactness. What filter type would you recommend? 02:47 < bitcoin-git> [bitcoin] laanwj pushed 9 new commits to master: https://github.com/bitcoin/bitcoin/compare/c91ca0ace9bd...a987def4f629 02:47 < bitcoin-git> bitcoin/master 23e6e64 John Newbery: Allow disconnectnode() to be called with node id... 02:47 < bitcoin-git> bitcoin/master d6564a2 John Newbery: [tests] fix nodehandling.py flake8 warnings 02:47 < bitcoin-git> bitcoin/master e367ad5 John Newbery: [tests] rename nodehandling to disconnectban 02:48 < bitcoin-git> [bitcoin] laanwj closed pull request #10143: [net] Allow disconnectnode RPC to be called with node id (master...disconnect_node_by_id) https://github.com/bitcoin/bitcoin/pull/10143 02:48 < sipa> jonasschnelli: the optimal datastructure is something like a bloom filter, but with only 1 hash function, and then instead of storing the bits directly, store the distances between the 1s 02:48 < jonasschnelli> sipa: okay.. I try to parse your answer.. give me some days. :) 02:50 < sipa> it's 44% smaller than bloom filters, afaik 02:50 < sipa> but much much slower to look things up 02:50 < sipa> as you need to decompress the data 02:52 < jonasschnelli> So just use a single MurmurHash? I need to understand what you mean with "store the distances between the 1s". Let me think a bit about it. 02:53 < sipa> if you only have 1 hash function, there will be a few 1s and many 0s in your filter 02:54 < sipa> which can be compressed using run length encoding 02:54 < jonasschnelli> Okay. Got that part. 02:54 < gmaxwell> jonasschnelli: I think this stuff is a waste of time. even with commited filters the users privacy is significantly harmed. Saving 14kb/s hardly seems worth it. 02:55 < jonasschnelli> gmaxwell: You propose to just scan all blocks? 02:55 < gmaxwell> sipa: as far as slower, I doubt it, even a pretty huge filter the decompression time would be insignificant compared to transfer time. 02:55 -!- elkalamar [~pepe@84.126.69.179.dyn.user.ono.com] has joined #bitcoin-core-dev 02:55 < gmaxwell> jonasschnelli: scan blocks since the creation of the keys. 02:56 < sipa> gmaxwell: i mean cpu time 02:56 < jonasschnelli> gmaxwell: Yes. But catching up two weeks on a cellphone would result in 144*14MB = 2GB of data... 02:56 < jonasschnelli> Resulting in users stick to the current BF SPV model 02:57 < gmaxwell> users don't use that in any large number now. 02:57 < gmaxwell> jonasschnelli: already almost no one uses multibit/android wallet due to poor performance, even though it completely destroys the user's privacy. You cannot compete with server based lookup which is what almost everyone uses. 02:58 < gmaxwell> also, why would the cellphone be two weeks out of date? should be catching up in the background... 02:58 < jonasschnelli> Hmm... well, .. SPV BF is pretty fast and I think it's used by a large usergroup... though, I don't have reliable numbers. 02:59 < sipa> what is BF? 02:59 < jonasschnelli> gmaxwell: Catching up data in the background puts your app in a different app-group 02:59 < jonasschnelli> BF = bloom filter 02:59 < jonasschnelli> gmaxwell: the group that has the significant warning about battery consumption. :) 02:59 < sipa> what is an app group? 03:00 < gmaxwell> jonasschnelli: http://luke.dashjr.org/programs/bitcoin/files/charts/software.html BF using wallets are relatively rare I seldom see more than one connected. 03:00 < jonasschnelli> sipa: It may be different on Android. But on apples iOS, if you use backgroup activity, you'll end up in a different app-group resulting to different reviews,... I think you need to add warnings to your app description, etc. 03:01 < gmaxwell> sipa: I fail to see how cpu time alone is interesting for what is being discussed. :) 03:01 < sipa> gmaxwell: that chart is just reachable nodes 03:01 < sipa> no? 03:01 < gmaxwell> sipa: no. 03:01 < gmaxwell> sipa: the opposite of that, in fact. 03:01 < sipa> gmaxwell: i was discussing the data structure, not the use case 03:02 < gmaxwell> ah well, you wouldn't use it for any of the things you normally use a bloomfilter for. 03:02 < gmaxwell> Though FWIW, the cuckoo-like filter also is close to capacity achieving (at least if you high enough N to allow a high fill rate) and works incrementally. 03:29 -!- jannes [~jannes@095-097-246-234.static.chello.nl] has joined #bitcoin-core-dev 03:36 -!- NewLiberty [~NewLibert@12.246.8.14] has joined #bitcoin-core-dev 03:37 -!- NewLiberty_ [~NewLibert@12.246.8.14] has joined #bitcoin-core-dev 03:38 -!- NewLiberty_ [~NewLibert@12.246.8.14] has quit [Read error: Connection reset by peer] 03:38 -!- Joseph__ [~NewLibert@12.246.8.14] has joined #bitcoin-core-dev 03:41 -!- NewLiberty [~NewLibert@12.246.8.14] has quit [Ping timeout: 240 seconds] 03:42 -!- tw2006 [~tw2006@2601:187:8480:2770:40c8:98f5:9507:15b9] has joined #bitcoin-core-dev 03:46 -!- tw2006 [~tw2006@2601:187:8480:2770:40c8:98f5:9507:15b9] has quit [Ping timeout: 245 seconds] 04:08 < jonasschnelli> Hmm... is it entirely stupid to only extend and check the kepool-keyrange (HD restore) if the wallet's bestblock lacks some blocks behind the chaintip? I think a check during init could give an indication if the wallet is in restore mode or not. 04:08 < jonasschnelli> Alternatively we could add an option hdalwayscheckkeypool=1 04:11 <@wumpus> what would be the rationale behind doing that, then only? 04:21 < jonasschnelli> wumpus: a) performance for unencrypted wallets, b) [more important] encrypted wallets would require a unlock during the time of the scan 04:22 <@wumpus> "encrypted wallets would require a unlock during the time of the scan" but only if the scan notices that new keys are needed, right? 04:22 < jonasschnelli> for bitcoind, there is the problem how to want the user if the gap limit is reached, because, at this point, he would need to unlock the wallet in order to continue scanning 04:22 < jonasschnelli> wumpus: Right. 04:23 <@wumpus> yes I understand there's a notificatino problem there. The GUI could just pop up a dialog, not so much for bitcoind 04:23 < jonasschnelli> And... to do a precaution scan, you probably should use a gap limit of 100 (configurable). 04:24 < jonasschnelli> Yes. The GUI way is much simpler.. but even there. Do we always want to extend up to a default gap limit (even in normal operations)? 04:24 <@wumpus> so yes it may make sense to have a seprate 'wallet is reconstructing' mode 04:24 < jonasschnelli> Because someone may have handed out 100+ addresses and want to make sure he catches all of them in a HD rescan 04:24 < jonasschnelli> Though, most people probably dont want to auto-extend their keypool over 100+ 04:24 <@wumpus> this flag could also be stored in the wallet (like the reindex flag in the utxo db) instead of determining this based on the wallet's bestblock 04:25 < jonasschnelli> but if you load an initial HD wallet backup, you probably can only identify the possible backup by comparing the bestblock against the chaintip 04:25 <@wumpus> one argument against treating reconstruction specially would be simultaneous use of the wallet on different machines. I know, we don't support this, but then it could be detected. 04:26 <@wumpus> jonasschnelli: that's true 04:26 < jonasschnelli> Yes. That. 04:26 < jonasschnelli> Asking the user make sense... (GUI). 04:26 <@wumpus> yes 04:26 < jonasschnelli> "Wallets is out of sync, do you want to restore a backup?" 04:27 < jonasschnelli> Then extend the keypool +1000 or ask about the previous usage 04:27 <@wumpus> #10231 gives me a compile error : bitcoin/src/qt/clientmodel.h:85:30: error: implicit instantiation of undefined template 04:27 <@wumpus> 'std::atomic' 04:27 < gribble> https://github.com/bitcoin/bitcoin/issues/10231 | [Qt] Reduce a significant cs_main lock freeze by jonasschnelli · Pull Request #10231 · bitcoin/bitcoin · GitHub 04:27 <@wumpus> needs a header probably 04:27 < jonasschnelli> Oh... missed include... 04:28 < jonasschnelli> Thanks. Let me fix this 04:29 < jonasschnelli> Added a commit. But why did travis not complain?! 04:32 <@wumpus> my report is on ubuntu 16.04 at least, maybe it's different for 14.04 04:33 <@wumpus> most likely cause: different boost versions 04:33 <@wumpus> or qt 04:38 <@wumpus> anyhow with your change it all compiles 04:41 -!- xinxi [~xinxi@116.87.187.139] has joined #bitcoin-core-dev 04:41 < bitcoin-git> [bitcoin] laanwj pushed 6 new commits to master: https://github.com/bitcoin/bitcoin/compare/a987def4f629...987a6c09562e 04:41 < bitcoin-git> bitcoin/master 7148f5e Jonas Schnelli: Reduce cs_main locks during modal overlay by adding an atomic cache 04:41 < bitcoin-git> bitcoin/master cf92bce Jonas Schnelli: Update the remaining blocks left in modaloverlay at init. 04:41 < bitcoin-git> bitcoin/master 610a917 Jonas Schnelli: Declare headers height/time cache mutable, re-set the methods const 04:42 < bitcoin-git> [bitcoin] laanwj closed pull request #10231: [Qt] Reduce a significant cs_main lock freeze (master...2017/04/qt_freeze) https://github.com/bitcoin/bitcoin/pull/10231 04:44 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Ping timeout: 260 seconds] 04:46 -!- xinxi [~xinxi@116.87.187.139] has quit [Ping timeout: 255 seconds] 04:48 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 04:48 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit] 05:14 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 05:23 -!- xinxi [~xinxi@116.87.187.139] has joined #bitcoin-core-dev 05:25 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has joined #bitcoin-core-dev 05:30 -!- tw2006 [~tw2006@2601:187:8480:2770:985e:f883:22de:28f5] has joined #bitcoin-core-dev 05:31 -!- mol [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 05:34 -!- moli_ [~molly@unaffiliated/molly] has quit [Ping timeout: 252 seconds] 05:36 -!- tw2006 [~tw2006@2601:187:8480:2770:985e:f883:22de:28f5] has quit [Ping timeout: 260 seconds] 05:38 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 05:41 -!- RubenSomsen [~RubenSoms@5ED2CA1D.cm-7-3d.dynamic.ziggo.nl] has joined #bitcoin-core-dev 05:59 -!- n1ce [~n1ce@unaffiliated/n1ce] has joined #bitcoin-core-dev 06:03 -!- gielbier [~michiel@unaffiliated/gielbier] has quit [Ping timeout: 260 seconds] 06:03 -!- tw2006 [~tw2006@2601:187:8480:2770:81fe:647:d7e9:666b] has joined #bitcoin-core-dev 06:03 -!- talmai [~T@c-24-147-97-55.hsd1.ma.comcast.net] has joined #bitcoin-core-dev 06:14 -!- gielbier [~michiel@2001:981:9573:1:3815:9c0f:650:f7bc] has joined #bitcoin-core-dev 06:22 -!- talmai [~T@c-24-147-97-55.hsd1.ma.comcast.net] has quit [Quit: mining] 06:24 -!- talmai [~T@c-24-147-97-55.hsd1.ma.comcast.net] has joined #bitcoin-core-dev 06:28 -!- gielbier [~michiel@2001:981:9573:1:3815:9c0f:650:f7bc] has quit [Changing host] 06:28 -!- gielbier [~michiel@unaffiliated/gielbier] has joined #bitcoin-core-dev 06:29 -!- talmai [~T@c-24-147-97-55.hsd1.ma.comcast.net] has quit [Quit: mining] 06:29 -!- RubenSomsen [~RubenSoms@5ED2CA1D.cm-7-3d.dynamic.ziggo.nl] has quit [Ping timeout: 252 seconds] 06:34 -!- Guest12838 [~justin@47.148.176.74] has quit [Remote host closed the connection] 06:35 -!- Guest12838 [~justin@47.148.176.74] has joined #bitcoin-core-dev 06:36 < morcos> gmaxwell: fee estimation currently does not use mempool queue (nor in the improvements for 0.15) it's an idea that i've been contemplating since the beginning, but i never settled on a design that i thought met all the criteria 06:36 < morcos> balancing performance, usefulness, and security is hard. 06:41 < bitcoin-git> [bitcoin] jonasschnelli opened pull request #10238: Change setKeyPool to hold flexible entries (master...2017/04/keypool_fix_a) https://github.com/bitcoin/bitcoin/pull/10238 06:45 -!- JackH [~laptop@79-73-191-98.dynamic.dsl.as9105.com] has quit [Quit: Leaving] 06:47 < bitcoin-git> [bitcoin] sipa opened pull request #10239: Make Boost use std::atomic internally (master...boost_std_atomic) https://github.com/bitcoin/bitcoin/pull/10239 06:48 < BlueMatt> sipa: hmm...why does boost prefer its own impls? just for compat? 06:49 < sipa> BlueMatt: only bug report i found about it was "gcc's atomics were not so good in the past... it's probably better now... discussion dies" 06:49 < BlueMatt> lol, yay boost 06:49 < sipa> these macros were only added in 1.54 though... i don't know what versions of boost we're using everywhere 06:50 -!- d_t [~textual@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has joined #bitcoin-core-dev 06:51 < sipa> http://boost.2283326.n4.nabble.com/compiling-boost-using-C-11-atomics-td4687878.html 06:51 < BlueMatt> well i suppose thats good...if you have a new boost you'll probably have a new gcc which has good atomics as well :) 06:52 < sipa> well we already rely on std::atomics anywhere 06:53 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt] 06:53 < BlueMatt> indeed 06:55 -!- d_t [~textual@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 258 seconds] 07:04 -!- felco [~felco@unaffiliated/felco] has quit [Quit: All your IRC are belong to ZNC] 07:07 -!- xinxi [~xinxi@116.87.187.139] has quit [Quit: Leaving...] 07:08 -!- felco [~felco@unaffiliated/felco] has joined #bitcoin-core-dev 07:11 -!- goksinen [~goksinen@cpe-74-71-4-175.nyc.res.rr.com] has joined #bitcoin-core-dev 07:16 -!- goksinen [~goksinen@cpe-74-71-4-175.nyc.res.rr.com] has quit [Ping timeout: 268 seconds] 07:19 -!- vicenteH` is now known as vicenteH 07:32 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev 07:44 -!- jtimon [~quassel@9.31.134.37.dynamic.jazztel.es] has quit [Ping timeout: 240 seconds] 08:02 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev 08:05 -!- goksinen [~goksinen@cpe-74-71-4-175.nyc.res.rr.com] has joined #bitcoin-core-dev 08:10 -!- goksinen [~goksinen@cpe-74-71-4-175.nyc.res.rr.com] has quit [Ping timeout: 260 seconds] 08:19 < luke-jr> wumpus: FYI I get a different result of update-translations than rc2 has: specifically, bitcoin_af is not created 08:20 < bitcoin-git> [bitcoin] jnewbery reopened pull request #10198: [tests] Remove is_network_split from functional test framework (master...remove_is_network_split) https://github.com/bitcoin/bitcoin/pull/10198 08:22 -!- harrymm [~wayne@104.237.91.122] has joined #bitcoin-core-dev 08:22 -!- harrymm [~wayne@104.237.91.122] has quit [Max SendQ exceeded] 08:23 -!- harrymm [~wayne@104.237.91.122] has joined #bitcoin-core-dev 08:26 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Quit: mining] 08:29 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev 08:30 -!- gm2051 [~gm2051@2a02:c7d:12e:100:7cc9:a92d:462:d811] has joined #bitcoin-core-dev 08:30 < bitcoin-git> [bitcoin] jonasschnelli opened pull request #10240: [WIP] Add basic HD wallet restore functionality (master...2017/04/hd_rescan) https://github.com/bitcoin/bitcoin/pull/10240 08:34 -!- mol [~molly@unaffiliated/molly] has quit [Read error: Connection reset by peer] 08:34 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Quit: Leaving.] 08:34 -!- moli_ [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 08:47 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Ping timeout: 240 seconds] 08:52 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 09:11 -!- gm2052 [~gm2051@2a02:c7d:12e:100:413f:4916:4ac0:d671] has joined #bitcoin-core-dev 09:15 -!- gm2051 [~gm2051@2a02:c7d:12e:100:7cc9:a92d:462:d811] has quit [Ping timeout: 245 seconds] 09:16 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev 09:17 -!- jtimon [~quassel@9.31.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 09:23 -!- n1ce [~n1ce@unaffiliated/n1ce] has quit [Read error: Connection reset by peer] 09:25 -!- n1ce [~n1ce@unaffiliated/n1ce] has joined #bitcoin-core-dev 09:28 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev 09:42 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 255 seconds] 09:50 -!- Sosumi [~Leon@bl10-113-190.dsl.telepac.pt] has joined #bitcoin-core-dev 10:02 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 10:12 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Quit: mining] 10:16 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev 10:25 <@wumpus> luke-jr: did I add bitcoin_af for rc2? 10:26 <@wumpus> adding languages is something I do manually, when I think there's enough in a .ts file to warrant it 10:26 -!- gm2053 [~gm2051@2a02:c7d:12e:100:49d:92bc:3989:ab0d] has joined #bitcoin-core-dev 10:26 < luke-jr> wumpus: I don't know about added, but it's not created by the translation scripts anymore 10:26 <@wumpus> but I think _af was a while ago 10:26 <@wumpus> interesting, maybe it was deleted on transifex 10:26 < luke-jr> when updating translations, I delete *.ts (except en) first 10:27 <@wumpus> I don't track lanugage deletions, only adding 10:27 <@wumpus> unless someone notified me that it was removed for a good reason (e.g. the fake austrian(?) translation that was there at some point) 10:28 <@wumpus> but I'll check that for next language update, thanks 10:29 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Ping timeout: 255 seconds] 10:29 < luke-jr> np 10:29 -!- gm2052 [~gm2051@2a02:c7d:12e:100:413f:4916:4ac0:d671] has quit [Ping timeout: 252 seconds] 10:31 -!- mol [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 10:34 -!- moli_ [~molly@unaffiliated/molly] has quit [Ping timeout: 240 seconds] 10:36 -!- d_t [~textual@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has joined #bitcoin-core-dev 10:44 -!- tw2006 [~tw2006@2601:187:8480:2770:81fe:647:d7e9:666b] has quit [Read error: Connection reset by peer] 10:45 -!- tw2006 [~tw2006@2601:187:8480:2770:81fe:647:d7e9:666b] has joined #bitcoin-core-dev 10:45 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Remote host closed the connection] 10:48 -!- goksinen [~goksinen@2604:2000:c591:8400:b56e:e81:f344:41c5] has joined #bitcoin-core-dev 10:53 -!- goksinen [~goksinen@2604:2000:c591:8400:b56e:e81:f344:41c5] has quit [Ping timeout: 258 seconds] 10:55 -!- timothy [tredaelli@redhat/timothy] has quit [Quit: Konversation terminated!] 11:35 -!- Joseph__ [~NewLibert@12.246.8.14] has quit [Ping timeout: 240 seconds] 11:45 < jonasschnelli> Why do test fail when they are successful: :/ https://travis-ci.org/bitcoin/bitcoin/jobs/224004681#L5249 11:46 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has quit [Remote host closed the connection] 11:46 < jonasschnelli> A stderr warning leads always to a test failure 11:48 <@wumpus> stderr output in itself results in test failure? 11:48 <@wumpus> I don't think so, it's based o nthe return code 11:48 < jonasschnelli> wumpus: Yes. It looks like. 11:48 < jonasschnelli> They pass locally... 11:48 < jonasschnelli> and they pass on travis... but test runner markes them as failed 11:49 < jonasschnelli> wumpus: https://travis-ci.org/bitcoin/bitcoin/jobs/224004681#L5249 11:49 < jonasschnelli> IMO at least we should flag them as passed if stderr contains only warnings... 11:49 < jonasschnelli> otherwise we can't test warnings 11:50 <@wumpus> test fail/pass shouldn't be based on stderr output at all 11:50 <@wumpus> if it is, that's kind of weird 11:50 < luke-jr> jonasschnelli: if an early step fails, travis keeps running the rest and still marks it as failed 11:51 < jonasschnelli> luke-jr: I think its not that: check the signmessage test: https://travis-ci.org/bitcoin/bitcoin/jobs/224004681#L5243 11:51 <@wumpus> where does it check stderr output? 11:51 < luke-jr> looks like a test_runner.py thing 11:51 < jonasschnelli> (INFO): Tests successful... but signmessages.py failed, Duration: 3 s 11:52 < jonasschnelli> if proc.returncode == TEST_EXIT_PASSED and stderr == "": 11:53 < jonasschnelli> the later if statement 11:53 < jonasschnelli> or at least split by newline and pass if all lines start with /Warning/ 11:53 < jonasschnelli> (or a clever regex) 11:54 <@wumpus> stderr == "" should go 11:54 <@wumpus> return code should be what determines whether a test passed 11:54 <@wumpus> anything else is insane 11:54 < jonasschnelli> I think so. Tests may by successful is there is something in stderr 11:55 < jonasschnelli> Okay. I'll PR 11:58 < bitcoin-git> [bitcoin] jonasschnelli opened pull request #10241: Allow tests to pass even when stderr got populated (master...2017/04/test_stderr) https://github.com/bitcoin/bitcoin/pull/10241 11:59 < gmaxwell> I think the sanitizer stuff is only useful in our current test harnesses because we fail on stderr output. 11:59 < luke-jr> ah 12:00 <@wumpus> sanitizer stuff? 12:00 < gmaxwell> TSAN/ASAN/UBSAN. 12:00 <@wumpus> do we use that in travis? 12:01 < jonasschnelli> Well, we could add a test_runner argument (fail_on_stderr) if someone wants to use that with sanitizer 12:02 < sdaftuar> meeting time? 12:02 <@wumpus> but ok, at least I understand why the stderr check is there now, it's for private test runs with sanitizer? 12:02 < jonasschnelli> however.. meeting 12:02 <@wumpus> #startmeeting 12:02 < lightningbot> Meeting started Thu Apr 20 19:02:12 2017 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:02 < lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic. 12:02 < luke-jr> IIRC one of the sanitisers used to require a special env var to cause an exit 12:02 < gmaxwell> Not yet, only with some not yet merged PRs are we finally TSAN clean, but many of us run it locally and it has found real bugs. I'm not protesting, but just bringing up the one thing I remember that interacts with that assumption. 12:02 < luke-jr> but I can't find that now (some do need an extra build option tho) 12:02 < gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier 12:02 < instagibbs> present 12:02 <@wumpus> gmaxwell: yes I think it's a good point, not trying to disparage it, but we should document things like this 12:02 < kanzure> hi. 12:02 < cfields> hi 12:02 < jtimon> hola 12:03 <@wumpus> topics? 12:03 < gmaxwell> wumpus: my thinking on seeing the comments above was "oh oh ... that interacts with something ... what was it? what was it?" 12:03 < gmaxwell> wumpus: 0.14.x release? 12:03 <@wumpus> #topic 0.14.1 release 12:03 <@wumpus> let's push the button? 12:03 < luke-jr> k 12:04 < jonasschnelli> Okay for me... to bad #10231 missed 0.14.1 12:04 < gribble> https://github.com/bitcoin/bitcoin/issues/10231 | [Qt] Reduce a significant cs_main lock freeze by jonasschnelli · Pull Request #10231 · bitcoin/bitcoin · GitHub 12:04 < gmaxwell> https://www.youtube.com/watch?v=rLMCjuge6oE "Turn your key SIR" 12:04 < cfields> hooray! 12:04 < luke-jr> jonasschnelli: I can put it in Knots 0.14.1 12:04 < jonasschnelli> luke-jr: Yes. Do that. 12:04 <@wumpus> jonasschnelli: is that even tagged for backport? anyhow, tag it for 0.14.2 I'd say 12:05 < jonasschnelli> wumpus: Yeah. I tagged (not the project though).. 0.14.2 is good IMO. 12:05 <@wumpus> jonasschnelli: ok! 12:06 <@wumpus> next topic? 12:06 * jonasschnelli damns cs_main 12:07 <@wumpus> jonasschnelli: if it's any consolation, many projects had a similar issue with a central lock 12:07 * luke-jr coughs at Python 12:07 <@wumpus> I was thinkinkg about the Big Kernel Lock, but yes, python is guilty too 12:07 < jonasschnelli> wumpus: Yes. I guess there is much room for optimisation. 12:08 < gmaxwell> There has been some interesting discussion in github related to the wallets handling of address reuse and dust and what not. anyone interested in that subject might want to check out the discussion on #10233 and PRs linked from there. 12:08 < gribble> https://github.com/bitcoin/bitcoin/issues/10233 | Wallet: Support not reusing addresses by jet0 · Pull Request #10233 · bitcoin/bitcoin · GitHub 12:08 <@wumpus> #topic blocker PRs for review 12:09 < gmaxwell> jonasschnelli: on locking we need some better lock profiling. If we have some instrumention that yelled anytime lock contention caused >100ms delays, we'd probably find a number of things to fix. 12:09 < jonasschnelli> gmaxwell: Yes. That! 12:09 < gmaxwell> I don't think cs_main is itself really the issue there... just not carefully avoiding it via things like caches. 12:09 < jonasschnelli> gmaxwell: I was printf profiling yesterday 12:09 < luke-jr> I looked into disabling address reuse and it looks harder than I'd like :/ 12:10 < morcos> i would like to briefly discuss fee estimation (maybe as separate topic) 12:10 < gmaxwell> on the blocker PRs-- I'm kinda lost where we are with non-atomic writes. 12:10 < jonasschnelli> blocker PR: https://github.com/bitcoin/bitcoin/projects/8 12:10 * BlueMatt #10179 12:10 < gribble> https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub 12:10 < BlueMatt> gm2053: i think its ready for review now? 12:10 < morcos> gmaxwell: #10148 in its current form without multi head just needs more review i think 12:10 < gribble> https://github.com/bitcoin/bitcoin/issues/10148 | [WIP] Use non-atomic flushing with block replay by sipa · Pull Request #10148 · bitcoin/bitcoin · GitHub 12:11 < morcos> and maybe more tests? 12:11 < jonasschnelli> sipa still has the chacha20 rnd as blocker #9792 ... 12:11 * BlueMatt utacked this morning 12:11 < gribble> https://github.com/bitcoin/bitcoin/issues/9792 | FastRandomContext improvements and switch to ChaCha20 by sipa · Pull Request #9792 · bitcoin/bitcoin · GitHub 12:11 <@wumpus> #10179 is already on the list BlueMatt 12:11 < gribble> https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub 12:12 < BlueMatt> wumpus: oh, wasnt sure if it got switched after the last merge, sorry 12:12 <@wumpus> adding 10148 12:12 < gmaxwell> 9792 isn't hard to review, FWIW, in my expirence. 12:12 < sdaftuar> gmaxwell: i've become more comfortable conceptually with the non-atomic writes (it did take me a while to come around to it being worth the effort). i'd like to review and test more. 12:13 < morcos> ditto 12:13 <@wumpus> 9792 is also already on the list 12:13 < BlueMatt> #9942 can get merged, I think... 12:13 < gribble> https://github.com/bitcoin/bitcoin/issues/9942 | Refactor CBlockPolicyEstimator by morcos · Pull Request #9942 · bitcoin/bitcoin · GitHub 12:13 < jtimon> #8855 isn't hard to review either... 12:13 < gribble> https://github.com/bitcoin/bitcoin/issues/8855 | Use a proper factory for creating chainparams by jtimon · Pull Request #8855 · bitcoin/bitcoin · GitHub 12:14 <@wumpus> if there's something that an be merged you should tell me, preferably outside the meeting :) 12:14 < morcos> yeah wumpus i think that is now just wasting review cycles, it has more than enouch ACK's (we have told you a couple times.. :) ) 12:14 < luke-jr> multiwallet is rebased and nits fixed btw 12:14 < gmaxwell> sdaftuar: will let us effectively double the dbcache size being a first benefit... plus it should allow some really nince improvements later post per-txo. Sorry if it wasn't communicated well, the value was more obvious to pieter and I perhaps because we've been hammering on caching policy changes based on per-txo for a while. 12:14 < luke-jr> CWalletDB still needs some serious refactoring, but IMO that's something to do outside multiwallet's PR 12:15 < jonasschnelli> luke-jr: agree 12:15 <@wumpus> morcos: I don't remember 12:15 < morcos> wumpus: no problem, i just wasnt telling you because i didn't want to tell you too many times.. in any case i think its ready (9942 that is) 12:16 < bitcoin-git> [bitcoin] luke-jr closed pull request #7289: [WIP] Make arguments reconfigurable at runtime via RPC (master...rpc_setarg) https://github.com/bitcoin/bitcoin/pull/7289 12:16 < sdaftuar> gmaxwell: yeah, makes sense to me now -- there are a lot of steps of "why don't we do X simpler thing instead" that i know you guys have tried/thought through already, that i needed to think through myself 12:17 < morcos> fee estimation? 12:17 < jonasschnelli> ack 12:18 < bitcoin-git> [bitcoin] laanwj pushed 8 new commits to master: https://github.com/bitcoin/bitcoin/compare/987a6c09562e...14c948987f0b 12:18 < bitcoin-git> bitcoin/master ae7327b Alex Morcos: Make feeEstimator its own global instance of CBlockPolicyEstimator 12:18 < bitcoin-git> bitcoin/master f6187d6 Alex Morcos: Make processBlockTx private. 12:18 < bitcoin-git> bitcoin/master dbb9e36 Alex Morcos: Give CBlockPolicyEstimator it's own lock 12:18 < morcos> Thanks! 12:18 < bitcoin-git> [bitcoin] laanwj closed pull request #9942: Refactor CBlockPolicyEstimator (master...moveTxConfirmStats) https://github.com/bitcoin/bitcoin/pull/9942 12:18 < BlueMatt> morcos: yes, fee estimation 12:18 < morcos> I wrote this to describe the existing algorithm https://gist.github.com/morcos/d3637f015bc4e607e1fd10d8351e9f41 , which I'm happy to discuss if anyone has any questions on it. 12:18 < gmaxwell> morcos: thanks for that fee estimation writeup, I guess I understood it better than I thought I did, I think I thought more of the discussed things were actually implemented. 12:18 < bitcoin-git> [bitcoin] ryanofsky opened pull request #10242: [qt] Don't call method on null WalletModel object (master...pr/rbfnull) https://github.com/bitcoin/bitcoin/pull/10242 12:19 < gmaxwell> morcos: I think that writeup is good and should go into the codebase. 12:19 < morcos> And then I wrote #10199 with a bunch of improvements. I suppose it makes sense to add another section the gist that provides a high level overview of the improvements? 12:19 < gribble> https://github.com/bitcoin/bitcoin/issues/10199 | Better fee estimates by morcos · Pull Request #10199 · bitcoin/bitcoin · GitHub 12:19 < gmaxwell> morcos: that would be good. 12:20 < gmaxwell> I think the estimatior is a complex enough machine that we should maintain a seperate description of it, if not an actual spec. Just like we do for many major protocol features. 12:20 < morcos> But what I would like to do is err on the side of merging 10199 early and then if there are small bugs or fixes, we can fix them in master 12:20 * jtimon remembers that he also wants to decouple the estimator from the mempool 12:20 < sipa> oops, forgot about meeting 12:21 < gmaxwell> morcos: the writeup could use some more details about the reliablity estimates and how it merges bins. 12:21 < morcos> it takes 2 weeks of continuous up time to even explore all the code paths 12:21 < gmaxwell> sipa: the meeting did not forget about you. 12:21 < morcos> jtimon: yes, i have a plan to do that that builds off BlueMatt's CValidationInterface. The groundwork is laid in 9942 that was just merged 12:21 < gmaxwell> morcos: are we not saving enough data between restarts that we really do need two weeks continious uptime to hit it all? 12:21 < morcos> reliability estimates? reliability OF estimates? 12:22 < gmaxwell> morcos: I know that if there aren't many samples in a bin it doesn't use the bin. 12:22 < morcos> gmaxwell: well if you want to know how much fee it'll take to be confirmed in a week, you sure as hell better wait at least a week (but yes once you've done that once, you may not need to do it again on a restart) 12:23 < morcos> gmaxwell: some of that stuff is changed in 10199 (for the better, obviously i guess) 12:23 < luke-jr> if anyone else acks #10242, maybe mention the meeting going on in a P.S. :p 12:23 < gribble> https://github.com/bitcoin/bitcoin/issues/10242 | An error has occurred and has been logged. Please contact this bot's administrator for more information. 12:23 < jtimon> morcos: I know, I reveiwed it yesterday and linked to similar PRs of my own, at the time you only wanted to decouple the mempool from the estimator (9942 just did it), but not the estimator from the mempool, happy if you changed your mind and want both like me now 12:23 < gmaxwell> I guess in general a thing to keep in mind for this sort of description is that we should try to make it detailed enough that if an academic showed up and wrote a paper on it based on only the description (which they will) would their results be likely useful to us or not. :) 12:23 < morcos> but there are some open questions in 10199 that would be helpful to get feedback on 12:24 < morcos> such as starting with not being able to upgrade from the old estimates 12:24 <@wumpus> imo the most important about the description is that we understand it 12:24 < gmaxwell> morcos: we should think about saving more of its state in the future. I have nodes that don't spend more than a few minutes down per month, but don't often make it to two weeks up. 12:25 < luke-jr> morcos: how complicated is the upgrading? we only would need it for one version at most IMO 12:25 < morcos> gmaxwell: i think thats problematic for being able to predict really long time horizons... 12:25 <@wumpus> upgrading isn't much of an issue, if the estimation algorithm changes, feel free to throw away the data from the previous one 12:25 <@wumpus> just make sure it doesn't crash on upgrading/downgrading 12:26 < gmaxwell> well, to really advance I think what we would probably want is a simulator (perhaps based on historical data) and a metric of success. 12:26 <@wumpus> yes 12:26 < morcos> luke-jr: the complicated part is deciding what we want to do, implementing it probably isn't that bad... but for instance the new estimates are smart about whether your estimates file is stale... but should it just dumbly use your old estimates until it has new estimates... what if the new estimates for 5 blocks which you do have is lower than the old estimate for 25 (which you dont' have a new estimate for) 12:26 < morcos> etc. 12:26 < gmaxwell> I think it's more or less fine to toss out data on algo changes. we could worry about doing better when the algo is stable for a long time. 12:26 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Ping timeout: 260 seconds] 12:27 <@wumpus> the difficult part, as with coin selection, is evaluationg algorithms 12:27 < morcos> the historical data is useless... the question is whether you use the old estimates until your new estimates are warmed up (by calculating them before you throw away the data) 12:27 < luke-jr> huh, someone's playing malleability games on testnet. 12:27 -!- belcher [~belcher@unaffiliated/belcher] has joined #bitcoin-core-dev 12:28 < gmaxwell> Electrum does some things with using static estimates when it doesn't have data to estimate on its own. I think there are a lot of interesting tradeoffs we could make to hotstart. But I don't think starting speed is at all our biggest concern. 12:28 < gmaxwell> luke-jr: they have been for months. 12:28 < gmaxwell> The purely retrospective algorithim is really slow to update to changing network conditions, in particular, it doesn't track the weekly load cycle well. 12:28 < morcos> gmaxwell: right so that is the question.. my preference would be to merge as is.. and then if we get around to it before 0.15 we add a smarter hotstart 12:29 < luke-jr> morcos: well, if it's not already implemented, I'd say it's not important enough to spend time implementing 12:29 -!- arubi_ [~ese168@gateway/tor-sasl/ese168] has joined #bitcoin-core-dev 12:29 < luke-jr> (upgrading, that is) 12:29 < morcos> gmaxwell: yes... one of the main problems the new design was meant to address... still using only a purely retrospective algorithm, so the problem fundamentally remains, but in practice its much more responsive (b/c it looks at different time horizons simultaneously) 12:30 < jcorgan> clearly this calls for Deep Fee Estimation 12:31 < gmaxwell> die 12:31 < jcorgan> tell me what you *really* think 12:31 <@wumpus> hehe, deep fee estimation 12:31 < luke-jr> no no, Xtreme Deep Fee Estimation! 12:32 < morcos> anyway, ok for now i'll update the gist with a high level description of the algorithm 12:32 < gmaxwell> I have a lovely algorithim for an efficient limited memory 2D exponentially weighed moving average somewhere.... 12:32 < gmaxwell> morcos: great. 12:32 < sipa> Xthin fees 12:32 -!- arubi [~ese168@gateway/tor-sasl/ese168] has quit [Ping timeout: 248 seconds] 12:32 < luke-jr> XD 12:32 < morcos> but my basic point here is that ideally we need weeks/months of testing in master to uncover possible edge cases 12:33 < morcos> i'm relatively confident that overall this is better, but thats not the same thing as saying it doesn' thave problems that need fixing... 12:33 <@wumpus> yes, it shouldn't be merged last minute 12:33 < gmaxwell> morcos: well get your description up soon, and I'll review shortly after. I think fee estimation is self contained enough we could merge something and back it out if we don't like it... but we need to have more than you understanding what we're doing at least. :) 12:33 < morcos> gmaxwell: yes basically my point.. ok sounds good 12:34 < gmaxwell> (if for no other reason than we need to understand it better to spot failures with it.) 12:34 < jonasschnelli> morcos: maybe it was asked already, how fast are the estimations available after startup? Does it work with prune=550? 12:34 < morcos> prune is irrelevant 12:34 < morcos> it can give you an estimate for a target of N once it has been caught up for 2*N blocks... 12:35 < gmaxwell> jonasschnelli: Estimations for depth X need to at least see some multiple of X blocks. 12:35 < morcos> but then it saves that 12:35 < gmaxwell> Becuase you have a moving window of analysis, and no data for longer windows. 12:35 < jonasschnelli> So for a conf target of 2 you need to wait ~40min after startup? 12:35 < morcos> so if you stop and restart you're starting over again for increasing your max possible target, but you still have access for up to that max possible target 12:36 < morcos> jonasschnelli: correct, but again, only the first time (or if you have been down for more than 6 weeks i think) 12:36 < gmaxwell> jonasschnelli: well not really, because you save the results. so the first time, yes. But that goes back to the hot start question.. and there are lots of ways we could hot start these things, if we really had something that was working well otherwise. 12:36 < jtimon> jcorgan: after alphago took go away from me I was looking for other problem to solve with https://github.com/jtimon/preann as an excuse to run it again 12:37 < jonasschnelli> Okay. Thanks... I'll test 10199 with the mainnet GUI then a bit (before of after merging) 12:37 < bitcoin-git> [bitcoin] ryanofsky opened pull request #10244: [qt] Add abstraction layer for accessing node and wallet functionality from gui (master...pr/ipc-local) https://github.com/bitcoin/bitcoin/pull/10244 12:37 -!- Dyaheon [~Dya@a91-156-192-24.elisa-laajakaista.fi] has quit [Ping timeout: 255 seconds] 12:37 < jonasschnelli> *or 12:38 < morcos> gmaxwell: in the meantime the PR description in 10199 covers the changes pretty close to what i will write up on the gist 12:39 -!- Dyaheon [~Dya@a91-156-192-24.elisa-laajakaista.fi] has joined #bitcoin-core-dev 12:40 < morcos> i guess i need to do a quick rebase now that 9942 is done 12:44 * luke-jr crickets 12:44 <@wumpus> any other topics? 12:44 < gmaxwell> I want to talk to luke some about the address reuse thing, but it can be post meeting. 12:45 <@wumpus> time to tag 0.14.1 final and start gitian building 12:45 * luke-jr spins out a Knots branch :p 12:45 <@wumpus> gmaxwell: well there's time 12:45 <@wumpus> #topic address reuse thing 12:46 < gmaxwell> so a serious privacy problem which has been actively exploited for a long time is that people make near-dust payments to addresses once they've been spent from, so that you spend from them again in new txn, creating snowballing that links all your transactions togeather. 12:46 <@wumpus> * [new tag] v0.14.1 -> v0.14.1 12:47 < instagibbs> https://www.reddit.com/r/BitcoinBeginners/comments/66az3b/why_do_i_keep_getting_000000001_deposits/ for example 12:47 < gmaxwell> Latest discussions seem to be driven by a user that runs a gambling site and whom cares about this because his customers get running into issues transactions that link back to him. 12:47 <@wumpus> do we need a 'block transacton' functionality against such transaction abuse? 12:47 < gmaxwell> but it's a general concern for everyone. 12:48 < gmaxwell> So there have been discussion about some very heavy handed manual methods, but I think I have a suggestion that could potentially be a default behavior: 12:48 < gmaxwell> but I'm interested in hearing if other people think I'm crazy. 12:48 <@wumpus> wouldn't be the first time I have an UTXO I just want to ignore 12:48 < morcos> stop the suspense! 12:48 < BlueMatt> morcos: ack 12:48 < gmaxwell> First create a seperate quarenteened balance. Any address or specfic txo could be manually quarenteened or unquarenteed at any itme. 12:49 < gmaxwell> Then adjust coin selection to always spend all payments to a particular address at once (+/- some filtering with dust that might be needed to prevent dust attacks). 12:49 < gmaxwell> Then once an address has been spent from, it's automatically added to tue quarenteen list (with any outputs that weren't spent, e.g. whatever failed the dust filtering). 12:50 <@wumpus> I think the quarantaine is a good idea, not so sure about adding things automatically though 12:50 < gmaxwell> If users want to intentionally reuse an address, I suppose they'd need a way to prevent them from being reblocked. 12:50 < morcos> i like the general idea 12:50 < luke-jr> gmaxwell: and quarantined funds are excluded from balance or tx list somehow? 12:51 < gmaxwell> Well I think the attacks will continue unless we could come up with something that was close to automatic... Could be something that gives a GUI user a choice the first time it happens or if the Q balance becomes non-negligble. 12:51 < BlueMatt> morcos: I might prefer if we were Quarantine ing things and not quarantaine or quarenteened :p 12:51 < morcos> but what if you have 10 10btc utxos at the same address and you need to pay someone 1 btc 12:51 < morcos> you spend them all? 12:51 < gmaxwell> luke-jr: they'd be shown in the tx list, but skipped for spending, and shown as a seperate balances. Like confirmed vs unconfirmed balance. 12:51 < BlueMatt> gmaxwell: I'm somewhat unsold as default policy 12:52 < gmaxwell> morcos: yep. and create a big change. Which I think is fine. I think a seperate issue is that we should auto-split very large change. But thats 90% independant. 12:52 < BlueMatt> it seems to be a somewhat-surprising break 12:52 < jcorgan> and the name 'quarantined' might be a bit heavy handed 12:52 < gmaxwell> BlueMatt: the current privacy trashing is itself a very surprising break. 12:52 < BlueMatt> fair 12:52 < luke-jr> it might be less confusing if only the first receive ever was displayed/accepted, and all subsequent ones got quarantined 12:52 < gmaxwell> jcorgan: well I came up with that on the spot, on github they're calling it frozen, which I think is super misleading (bank froze my funds!). :P 12:52 < jcorgan> reserved? 12:53 < luke-jr> suspicious? :P 12:53 <@wumpus> trash can 12:53 < Chris_Stewart_5> extraneous? 12:53 < gmaxwell> luke-jr: first recieved leads to an immediate attack: dust spammer races the payment then you get the dust and not the payment. :) 12:53 < morcos> gmaxwell: hmm... i do like the idea of auto-quarantining spent address or dust left over in mostly spent addresses, but not sure i like default spending all the inputs and possibly giving you large change 12:53 < luke-jr> gmaxwell: first confirmed with a larger value? 12:53 < gmaxwell> morcos: really I'm surprised at that. That change alone is something I've wanted to do for a while (and was carrying patches for it for a bit) 12:53 < morcos> quarantine is a good name, but lets not bikeshed that 12:54 < luke-jr> gmaxwell: maybe auto-quarantine dust too 12:54 < instagibbs> morcos, assuming spending the dust is worthwhile, what's the concern? 12:54 < BlueMatt> gmaxwell: I'm not sold on non-end-user wallets here. it seems like it woul dbreak many merchant workflows that use bitcoind 12:54 < sdaftuar> auto-spending all the funds with a given address makes sense to me as well 12:54 <@wumpus> morcos: let's quarantine the bikeshed 12:54 < luke-jr> BlueMatt: merchant workflows that reuse addresses are broken anyway 12:54 < BlueMatt> (eg you receive half a payment, your coin selection spends from that addr, then you receive the other half, and now you dont realize you got paid?) 12:54 < luke-jr> wumpus: lol 12:54 < gmaxwell> BlueMatt: why? (thats why I'm asking.) -- obviously it would be configurable. 12:55 < jtimon> re bikeshedding: the class managing this obviously needs to be called quarantiner 12:55 < gmaxwell> BlueMatt: how often are merchants doing that? I mean you can get into advanced things like biasing selection to chose SPK that have least recently recieved funds to avoid that. 12:55 < BlueMatt> gmaxwell: I assume most merchants at least support multiple txn to complete your payment? 12:56 < BlueMatt> most guis ive read seem to imply that 12:56 < morcos> yeah i suppose if its configurable, an option that autospends everythign from any address that gets spent makes sense 12:56 < gmaxwell> BlueMatt: kinda. but they also require them to be recieved at effectively the same time. I think it's managable. 12:56 < instagibbs> I'm not sure they accept multiple txn as policy 12:56 < instagibbs> err automatically 12:57 < gmaxwell> Just the autospend alone would radically improve privacy, and would almost be enough except for the malicious dust creation. 12:57 < BlueMatt> gmaxwell: to make it compatible you'd have to never spend outputs newer than X, where X is merchant time frame 12:57 < gmaxwell> BlueMatt: ya, which would be a trivial 'first try without x' in the current framework. 12:57 < BlueMatt> i agree in principal, but it sounds like you'd break some folks' workflow in subtle ways. adding the option and defaulting off for non-gui users, maybe? 12:58 < sipa> i don't see how autospending would break anything? 12:58 < luke-jr> or just tweak how RPC shows quarantine 12:58 < gmaxwell> well it could evolve over time, too-- I do think it's not worth our time to do things here that we don't think could be on for a majority of users eventually in some form. 12:58 < instagibbs> At a minimum you could make near-dust be quarantined 12:58 < morcos> i would think there could be some threshold for auto-un-quarantining too right? like if your quarantine address receives 1 BTC? or maybe not, maybe it just becomes common sense to check that 12:58 <@wumpus> in the first version this is introduced it should be disabled by default in any case, I think, let's present it as a security feature first. Could always be enabled by default later but that should not be the initial goal. 12:58 < gmaxwell> Because already people who are super aware of privacy can and will already manually do coin selection to achieve ends like this. 12:58 < BlueMatt> gmaxwell: agreed in principal, but there are also easier fixes we can do initially. eg bias coin selection towards this with fallbacks? 12:58 < luke-jr> wumpus: true 12:58 < gmaxwell> wumpus: absolutely. 12:59 -!- d9b4bef9 [~d9b4bef9@207.38.86.239] has quit [Remote host closed the connection] 12:59 <@wumpus> anything that potentially 'disappears' funds shouldn't be enabled lightly 12:59 < luke-jr> it's harmless to add if it's disabled by default initially 12:59 <@wumpus> luke-jr: exactly 12:59 < morcos> ok, so this sounds like general agreement that this is a good idea and has degenerated into arguing about defaults. all development discussion in a nutshell! 12:59 < BlueMatt> "disabled by default" can also mean "if something fails, fall back to the current behavior" 12:59 < gmaxwell> Just in principle I don't think the resource investment is worth if it we don't think that the end goal couldn't be default-ish (e.g. GUI) use. 12:59 < BlueMatt> morcos: yup 12:59 <@wumpus> I'd enable it personally 12:59 <@wumpus> it's worth the resource investment if we think it's useful to have 13:00 -!- d9b4bef9 [~d9b4bef9@207.38.86.239] has joined #bitcoin-core-dev 13:00 < sdaftuar> so perhaps a first simple step would be to enable auto-spending of all funds from a given address in the coin selection logic? 13:00 < gmaxwell> I think users should be okay with 'multiple balances' we already have confirmed vs unconfirmed, and normal bank accounts have multiple balances. 13:00 < luke-jr> IMO the end goal should be to treat address reuse as something that just doesn't work, and have a quarantine people can dig out lost funds if necessary 13:00 < BlueMatt> sdaftuar: ack 13:00 < BlueMatt> DONG 13:00 < jtimon> I would start with the quarantine thing as only usable manually, which we all seem to like, and then propose automatic things 13:00 <@wumpus> #endmeeting 13:00 < gmaxwell> sdaftuar: yea, that would be a good first step with minimal impact, we might have to add some extra features with it, like automatic change splitting. 13:00 < lightningbot> Meeting ended Thu Apr 20 20:00:36 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) 13:00 < lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-04-20-19.02.html 13:00 < lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-04-20-19.02.txt 13:00 < lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-04-20-19.02.log.html 13:00 < morcos> i do think it could be default on.. i just think its a matter of avoiding surprised users who are wondering why money appears to be lost, but that can be solved with informing them 13:00 <@wumpus> luke-jr: yeah, for the long long term I agree on that 13:00 < BlueMatt> morcos: or fallbacks 13:00 < luke-jr> jtimon: +1 13:00 -!- Sosumi [~Leon@bl10-113-190.dsl.telepac.pt] has quit [Quit: Bye] 13:01 < BlueMatt> "wait, i hit spend and it says not enough money" can be prevented while still using this 13:01 < gmaxwell> sdaftuar: I used to have a patch that basically post processes coin selection to add all other inputs that were in the same "address group" (listaddressgroupings) as anything selected. 13:02 < gmaxwell> sdaftuar: but it was kind of suboptimal, and with the existance of coinjoin it's less important to use a whole group rather than a whole address. 13:02 <@wumpus> no one would send 1 BTC to an address to breach someones privacy 13:02 <@wumpus> it's always small amounts 13:02 < luke-jr> wumpus: if they did, many people would be happy to give up their privacy XD 13:02 < gmaxwell> wumpus: well never say never, but "champaign problems" 13:02 <@wumpus> luke-jr: xD 13:03 < sdaftuar> gmaxwell: one thing just occurred to me, if you receive lots of payments to the same address, and issue lots of payments yourself, then this will tie up lots of your utxo's, which could be operationally annoying? 13:03 < gmaxwell> unfortunately it can be realistic to send several dollars to do so, no problem, but there is only so much we can do. 13:03 < sdaftuar> ie you'll be generating lots of big unconfirmed chains, and run out of utxos 13:03 < sdaftuar> could* 13:03 < gmaxwell> sdaftuar: thus the comment about change splitting. 13:03 < sdaftuar> that doesn't help? 13:03 < sdaftuar> oh, some 13:04 < gmaxwell> yes it does, they'll go to different addresses. 13:04 < jcorgan> jtimon: i'll take a look at it 13:04 < sdaftuar> well i was thinking that you're still spending an unconfirmed output, which will be under the descendant limit for the parent... eh, not sure how it would work out. 13:04 < morcos> also the threshold as to what is too small to include in the spend and leave quarantined is tricky... 13:04 <@wumpus> in any case I agree on the long run, address reuse should be seen as something suspicious unless the user opted in to it (e.g. a publically published address) 13:04 < gmaxwell> morcos: well I think there is a simple objective measure: at the current target feerate, what is the break even level? 13:05 < gmaxwell> monero seems to get along okay with protocol prohibited address reuse, we're maybe too conservative on some of these things. :) 13:05 < morcos> lets say you have 1mBTC that would cost 0.5mBTC to add as an input at the fee rate you are proposing for this tx... maybe you prefer to leave that quarantined and send it separately at a lower fee rate .. certainly if you have multiple ones liek that that could be combined 13:05 < jtimon> jcorgan: unfortunately the documentation for the university had to be in spanish and I never bothered translating it https://github.com/jtimon/preann/blob/master/doc/pfc-jorge-timon.org (there's a latex generated from that and can give you a pdf as well) 13:06 <@wumpus> so isn't such a bad idea to 'forget' old addresses after they've been used too long ago, or auto-quarantaine at least 13:06 < gmaxwell> morcos: if only we had a fee estimator that could give us a reasonable floor feerate! :) 13:06 < morcos> we do! 13:06 < morcos> 2 sat/byte will get you confirmed within 500 blocks right now 13:06 <@wumpus> but shouldn't be enabled by default at this point 13:07 < morcos> ahh, it went up to 3.3 since i last checked.. :) 13:07 <@wumpus> I'd like my transactions to be confirmed within 100 years at least :) 13:08 < gmaxwell> morcos: right so you could use a floor feerate. But also I think it would be reasonable for us to have behavior that cleans up the UTXO set some at the users (small) expense, I think most users would support that, especially when it has privacy benefits. 13:08 <@wumpus> otherwise I'd have to restart my node first to prevent 64 bit node ids from overflowing 13:08 < instagibbs> wumpus, no patience eh? 13:09 < gmaxwell> There is a whole layer of extra features we could think about what to do with the quarantined funds... but I think that should be future work. 13:09 < jcorgan> jtimon: eso no es problema 13:09 < jtimon> hahaha, estupendo! 13:10 < gmaxwell> e.g. if later we have some kind of coinjoin intergration, they could be preferentially sent into that. 13:11 < gmaxwell> too many things going on, we've responded to this too slowly. :( oh well, in any case, I think that this will prevent a lot of dust from getting created. 13:12 < gmaxwell> so it's kind of counter intutive, you might worry that the quarantine would result in UTXO bloat, but I suspect the opposite, at least if we're able to make this default-ish: with the incentive to make the tracing payments gone, they'll stop being created. 13:12 <@wumpus> add an 'empty trash can' that sends the quarantained funds into devnull 13:13 < morcos> participate in network spam attack using quarantined funds button 13:13 < gmaxwell> I've mused about the idea about having some shred wallet feature that creates some long timelocked spend of any remaining coins and gives them over to fees... then sends them off somewhere. 13:14 <@wumpus> 'send wallet to wumpus' 13:14 < sipa> ACK 13:14 < gmaxwell> because I am somewhat pained by all the utxo bloat created by people who end up with 0.00001 BTC in a wallet, in 100 inputs, and then they just delete the file because its effectively worthless. 13:14 < gmaxwell> yea, wumpus is fine too. tricky part is timelocking it so that they have some time to reconsider their decision. :P 13:15 < gmaxwell> personally I wouldn't want it, you'll get clowns using it as a backup service and demanding their funds back. :P 13:15 <@wumpus> gmaxwell: yes, there are certainly some practical issues :p 13:15 < morcos> that reminds me, we should revisit before 0.15 the dust level the wallet will create 13:16 <@wumpus> sending to fees would be a better idea 13:16 <@wumpus> especially those small amounts... 13:16 < gmaxwell> morcos: if we had a really good lower bound fee estimat it would be sensible to use that. e.g. don't create any change output where more than half its value would be lost in fees. 13:17 < morcos> but it depends on what you mean by that 13:17 <@wumpus> morcos: what would you like to revisit it to? 13:18 < morcos> wumpus: not change the network definition, but make the wallet smarter about not creating (ever) outputs just about the network standard limit 13:18 < morcos> for instance like #9343 does 13:18 < gribble> https://github.com/bitcoin/bitcoin/issues/9343 | An error has occurred and has been logged. Please contact this bot's administrator for more information. 13:18 < morcos> gmaxwell: the problem is historically the lower bound fee estimate is 1 sat/byte 13:19 < morcos> i think any transaction ever created which paid more than that could have been mined by now, some probably weren't because they were collectively forgotten about 13:19 < morcos> but the lowest feerate mined on the weekend often drops that low 13:19 < morcos> people are just in a hurry to be confirmed quickly 13:20 <@wumpus> morcos: tagging #9343 for 0.15 13:20 < gribble> https://github.com/bitcoin/bitcoin/issues/9343 | An error has occurred and has been logged. Please contact this bot's administrator for more information. 13:20 < morcos> tag 10199 too please 13:20 <@wumpus> done 13:29 -!- Squidicuz [~squid@pool-173-48-116-49.bstnma.fios.verizon.net] has quit [Quit: Oh no, not again] 13:32 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev 13:45 -!- tw2006 [~tw2006@2601:187:8480:2770:81fe:647:d7e9:666b] has quit [Remote host closed the connection] 13:47 -!- molz_ [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 13:49 -!- mol [~molly@unaffiliated/molly] has quit [Ping timeout: 240 seconds] 14:01 < sipa> nanotube: can we have the bot not produce output instead of "An error has occurred" ? 14:08 < BlueMatt> why does it fial so much in the first place? 14:24 < luke-jr> is 0.14.1 supposed to be building a new Qt? 14:24 -!- mol [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 14:24 <@wumpus> relative to 0.14.1rcX? no 14:24 < bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/14c948987f0b...86ea3c2ff247 14:24 < bitcoin-git> bitcoin/master a1fd450 Jorge Timón: Trivial: Remove unneeded includes from .h:... 14:24 < bitcoin-git> bitcoin/master 1c897fc Jorge Timón: Missing includes 14:24 < bitcoin-git> bitcoin/master 86ea3c2 Wladimir J. van der Laan: Merge #10181: Include cleanup... 14:25 < bitcoin-git> [bitcoin] laanwj closed pull request #10181: Include cleanup (master...2017-04-10-includes) https://github.com/bitcoin/bitcoin/pull/10181 14:25 <@wumpus> doesn't seem to be doing that here, win already finished buildling 14:26 < luke-jr> wumpus: relative to 0.14.0 14:26 < luke-jr> I missed the RCs 14:27 <@wumpus> I'm not sure of that 14:27 < cfields> yes 14:27 -!- molz_ [~molly@unaffiliated/molly] has quit [Ping timeout: 258 seconds] 14:27 < cfields> due to zlib bump 14:28 < luke-jr> ah, since zlib is a dep? 14:28 < cfields> yep 14:28 * luke-jr goes to figure out food then 14:36 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has joined #bitcoin-core-dev 14:43 < jtimon> regarding https://github.com/bitcoin/bitcoin/pull/10193 I'm still failing at replacing BOOST_REVERSE_FOREACH and I don't understand why, maybe I should reduce the scope of the PR (I moved from working stuff to trying to fully remove boost/foreach.hpp by popular demand) 14:44 < jtimon> ? 14:46 <@wumpus> zlib is at the bottom of the food chain, dependency-wise, not surprised it triggers rebuild of everything 14:46 -!- tw2006 [~tw2006@2601:187:8480:2770:a13b:3261:42b0:45cc] has joined #bitcoin-core-dev 14:46 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has quit [Ping timeout: 240 seconds] 14:46 <@wumpus> jtimon: what's the problem with BOOST_REVERSE_FOREACH? 14:47 -!- molz_ [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 14:49 < jtimon> wumpus: I'm trying to replace it with https://github.com/bitcoin/bitcoin/pull/10193/commits/1e59de57eecc1a61b036d7f89ff6bf918c2c7f67, which I found in the interwebs but although I thought I fully understood, it seems I don't 14:49 < nanotube> sipa: yes, though it's easier to just fix the problem :) which is that github decided to add fancy middle-dots to its title, which make the existing code barf with unicode errors >_< 14:50 < luke-jr> lol 14:50 < luke-jr> nanotube: you're alive! 14:51 -!- mol [~molly@unaffiliated/molly] has quit [Ping timeout: 260 seconds] 14:51 -!- tw2006 [~tw2006@2601:187:8480:2770:a13b:3261:42b0:45cc] has quit [Ping timeout: 260 seconds] 14:51 < nanotube> luke-jr: o/ :) 14:52 < jtimon> wumpus: it seems to interfere with prevector templates in prevector_tests.cpp, see https://github.com/bitcoin/bitcoin/pull/10193/commits/cfef34884684e71c6f43ef3e4f2e87590fc87c9e for the trick to make the PR pass travis. probably I should remove that commit already, but I wanted to make sure the commits after removing BOOST_REVERSE_FOREACH weren't breaking something else, plus that commit is kind of the link that maybe answers your 14:52 < jtimon> question 14:53 < jtimon> I would really prefer to just solve the problem but I'm kind of lost 14:54 <@wumpus> jtimon: it's sad that c++11 doesn't provide an as-is replacement 14:54 < jtimon> wumpus: yep, it seems things get slightly better on c 14:54 < cfields> wumpus: does reverse_iterate not end up being a dangling reference? I'm not sure what lifetime is expected for the container in a range-based-for 14:55 < jtimon> c++0.14, I mean...c++14 14:55 < cfields> (rather, I don't know if the loop extends the lifetime of the container) 14:55 < cfields> er, that was for jtimon, sorry 14:56 <@wumpus> jtimon: ok, so it's just a matter of waiting a few years (hey at least not 100 years :p) 14:56 <@wumpus> c++17 is pretty nice too, esp std::optional 14:57 < nanotube> testing gribble title fix #9343 14:57 < gribble> https://github.com/bitcoin/bitcoin/issues/9343 | Dont create change at dust limit by morcos · Pull Request #9343 · bitcoin/bitcoin · GitHub 14:57 < nanotube> \o/ 14:57 <@wumpus> woohoo 14:57 < jtimon> no, this can be certainly solved in c++11, but maybe not in a very clean way, perhaps we want our own macro to replace it 14:57 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has joined #bitcoin-core-dev 14:58 <@wumpus> I'd say replacing it with another macro would not be worth it 14:59 <@wumpus> there are still significant impediments to ditching boost wholesale, and until we have replacements for those, there's no use in rolling our own for the simpler things. Certainly not if they're not clean and simple. 14:59 < jtimon> or just get rid of BOOST_FOREACH, ¿Q_FOREACH? and PAIRTYPE for now, but not BOOST_REVERSE_FOREACH or #include for now (although the include would only be used for BOOST_REVERSE_FOREACH now) 15:00 <@wumpus> but maybe we can get your reverse_iterator to work 15:00 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Quit: Leaving.] 15:00 <@wumpus> I would be surprised if it's not possible 15:02 < jtimon> I am already surprised that this is not working, it's working for all the other cases using BOOST_REVERSE_FOREACH, just not compiling prevector_tests for reasons beyond me. it seems the templating is colliding somehow 15:03 < jtimon> but the option of reducing the scope and leave fully removing boost/foreach.hpp for a later PR is also there, that's why I ask 15:03 < jtimon> btw, sorry for asking again, but any blockers for #10189 ? 15:03 < gribble> https://github.com/bitcoin/bitcoin/issues/10189 | devtools/net: add a verifier for scriptable changes. Use it to make CNode::id private. by theuni · Pull Request #10189 · bitcoin/bitcoin · GitHub 15:05 -!- mol [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 15:05 < TD-Linux> gmaxwell, you could have a checkbox when generating an address that indicates it's supposed to be public 15:05 < jtimon> nevermind, just remembered cfields needs to enforce a commit tittle prefix 15:05 < cfields> jtimon: that's done 15:05 < TD-Linux> it seems more useful than the "reuse an existing address" checkbox there now 15:06 < cfields> I suppose I should comment 15:06 < gmaxwell> TD-Linux: there is a reuse checkbox? I don't recall that. 15:07 < TD-Linux> gmaxwell, maybe it's gone in the latest version, let me check. regardless I don't understand the use case 15:07 < luke-jr> gmaxwell: I have a PR open to remove it.. 15:08 -!- molz_ [~molly@unaffiliated/molly] has quit [Ping timeout: 240 seconds] 15:08 < jtimon> cfields: oops, great! thanks 15:08 < gmaxwell> is that just the thing that brings up the dislog that shows you existing addresses. 15:12 < jtimon> let me rebase and remove the travis-cheating commit for everyone to see the error without having to build locally, maybe it's obvious to someone else 15:15 < TD-Linux> https://github.com/bitcoin/bitcoin/pull/3716 15:17 < gmaxwell> oh well generating a QR code for an older address still seems useful. 15:17 < gmaxwell> should probably be elsewhere though. 15:18 < TD-Linux> I'd rather have it accessible via a right click in the list below but I guess that list is only so long 15:28 -!- Giszmo [~leo@ip-112-233.219.201.nextelmovil.cl] has joined #bitcoin-core-dev 15:38 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 15:39 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit] 15:41 < bitcoin-git> [bitcoin] shigeya opened pull request #10245: Minor fix in build documentation for FreeBSD 11 (master...freebsd-11-build-doc-fix) https://github.com/bitcoin/bitcoin/pull/10245 15:44 -!- d_t [~textual@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 15:54 -!- d_t [~textual@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has joined #bitcoin-core-dev 15:58 -!- Squidicuz [~squid@pool-173-48-116-49.bstnma.fios.verizon.net] has joined #bitcoin-core-dev 16:02 < bitcoin-git> [bitcoin] practicalswift opened pull request #10246: Silence "warning: "MSG_DONTWAIT" redefined" when compiling under Linux (master...silence-msg_dontwait-warning) https://github.com/bitcoin/bitcoin/pull/10246 16:06 < jtimon> also I insist if you don't find #8855 interesting to review, maybe you do find #8994 interesting 16:06 < gribble> https://github.com/bitcoin/bitcoin/issues/8855 | Use a proper factory for creating chainparams by jtimon · Pull Request #8855 · bitcoin/bitcoin · GitHub 16:06 < gribble> https://github.com/bitcoin/bitcoin/issues/8994 | Testchains: Introduce custom chain whose constructor... by jtimon · Pull Request #8994 · bitcoin/bitcoin · GitHub 16:12 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Remote host closed the connection] 16:12 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 16:18 -!- jouke [~worst@unaffiliated/komkommer] has quit [Ping timeout: 260 seconds] 16:20 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 260 seconds] 16:21 < bitcoin-git> [bitcoin] practicalswift closed pull request #10246: Silence "warning: "MSG_DONTWAIT" redefined" when compiling under Linux (master...silence-msg_dontwait-warning) https://github.com/bitcoin/bitcoin/pull/10246 16:22 -!- jouke [~worst@2001:1c02:1600:5a00:51c5:9806:36ec:8ea9] has joined #bitcoin-core-dev 16:22 -!- jouke [~worst@2001:1c02:1600:5a00:51c5:9806:36ec:8ea9] has quit [Changing host] 16:22 -!- jouke [~worst@unaffiliated/komkommer] has joined #bitcoin-core-dev 16:32 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 16:33 -!- Giszmo [~leo@ip-112-233.219.201.nextelmovil.cl] has quit [Ping timeout: 260 seconds] 16:35 -!- tw2006 [~tw2006@2601:187:8480:2770:a43d:da3:2f00:847] has joined #bitcoin-core-dev 16:36 < luke-jr> y'all slow today; I had to build the new deps and still beat everyone but achow101 :P 16:37 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 255 seconds] 16:39 -!- tw2006 [~tw2006@2601:187:8480:2770:a43d:da3:2f00:847] has quit [Ping timeout: 240 seconds] 16:46 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 16:49 -!- Giszmo [~leo@ip-103-233.219.201.nextelmovil.cl] has joined #bitcoin-core-dev 16:52 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has quit [Read error: Connection reset by peer] 16:54 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has joined #bitcoin-core-dev 17:13 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 240 seconds] 17:16 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 17:18 -!- d_t [~textual@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 17:19 -!- tw2006 [~tw2006@2601:187:8480:2770:1cb8:8217:620f:871b] has joined #bitcoin-core-dev 17:29 -!- jannes [~jannes@095-097-246-234.static.chello.nl] has quit [Quit: Leaving] 17:43 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 240 seconds] 17:51 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Textual IRC Client: www.textualapp.com] 18:00 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-mvaeeeagbzxtqqpd] has quit [Quit: Connection closed for inactivity] 18:14 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 18:27 < michagogo> 23:13:32 I've mused about the idea about having some shred wallet feature that creates some long timelocked spend of any remaining coins and gives them over to fees... then sends them off somewhere. 18:27 < michagogo> 23:14:25 because I am somewhat pained by all the utxo bloat created by people who end up with 0.00001 BTC in a wallet, in 100 inputs, and then they just delete the file because its effectively worthless. 18:27 < michagogo> Isn't that already a thing? 18:28 -!- d9b4bef9 [~d9b4bef9@207.38.86.239] has quit [Remote host closed the connection] 18:28 < michagogo> I mean, not built in to Core, but I could have sworn there was something that collects dust and bundles it into an ANYONECANPAY to a 0-value OP_RETURN 18:29 -!- d9b4bef9 [~d9b4bef9@207.38.86.239] has joined #bitcoin-core-dev 18:30 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 260 seconds] 18:33 < gmaxwell> yea, peter todd's dustbgone 18:34 < michagogo> Yep, that's the one. 18:34 < michagogo> BTW, something occurred to me earlier. 18:34 < michagogo> In the latest update (Creators', I think it was called?) to Windows 10, they upgraded WSL 18:35 < michagogo> And I think one of the big things, besides the fact that it now uses Xenial, is that you can now call Windows binaries from within bash. 18:36 < michagogo> (Up until now, if you tried putting an exe into there it would try to execute it as a Linux binary and fail) 18:36 < achow101> oh, they updated to xenial? then you can't cross compile from wsl 18:36 < michagogo> achow101: yeah, for new installations 18:36 < michagogo> Wait, really? 18:36 < achow101> mingw on xenial does work IIRC 18:37 < michagogo> So why can't you cross-compile? 18:37 < achow101> well, not that it doesn't work, but rather that something (unknown) is different about it that makes it fail. there's an issue about it 18:38 < michagogo> Weird. 18:38 < achow101> michagogo: https://github.com/bitcoin/bitcoin/projects/1 18:39 < michagogo> I was just confused because you said mingw on xenial does work -- do you mean that there's something different about one of the other packages that does it? 18:39 < michagogo> Anyway, what occurred to me was this: Gitian supports virtualbox, right? 18:39 < achow101> yes 18:39 < achow101> I don't think it does it well though 18:39 < michagogo> And I assume vboxmanage works the same (in terms of parameters etc) on Windows and Linux? 18:40 < achow101> idk. I don't use virtualbox 18:40 < michagogo> If that is the case, it should (I think?) now be possible to run Gitian from within WSL 18:41 < achow101> couldn't you already do it with lxc? 18:41 < michagogo> Can you? 18:42 < michagogo> I would not expect LXC to work in WSL 18:42 < achow101> i thought someone did it a while ago and it worked, but I don't quite remember. I haven't tried it myself 18:44 -!- jtimon [~quassel@9.31.134.37.dynamic.jazztel.es] has quit [Ping timeout: 252 seconds] 18:56 -!- goksinen [~goksinen@cpe-74-71-4-175.nyc.res.rr.com] has joined #bitcoin-core-dev 18:58 < cfields> Any other gitian builders? Waiting for 1 more sig. 19:01 -!- goksinen [~goksinen@cpe-74-71-4-175.nyc.res.rr.com] has quit [Ping timeout: 240 seconds] 19:04 < achow101> how many sigs do you need before doing the signed binaries? 19:10 < luke-jr> https://github.com/bitcoin/bitcoin/pull/7107 should probably be reopened 19:17 < cfields> achow101: usually 3. But usually wumpus is one of them. Since he uploads the tarballs, it's reassuring to know he's gotten the same result. So without his, I'd prefer to wait for another one/two before signing. 19:40 < michagogo> cfields: mine's building right now 19:41 < michagogo> dammit, accidentally hit ctrl-c 19:42 < michagogo> Anyway, managed to get this for Linux: https://www.irccloud.com/pastebin/NPg3fSPm/ 19:43 < michagogo> Restarting the build for Windows and macOS now 19:49 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 240 seconds] 19:50 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 19:56 < cfields> michagogo: thanks! 19:58 -!- Giszmo [~leo@ip-103-233.219.201.nextelmovil.cl] has quit [Read error: Connection reset by peer] 20:00 -!- goksinen [~goksinen@2604:2000:c591:8400:9daa:eab8:a221:1989] has joined #bitcoin-core-dev 20:03 -!- gm2052 [~gm2051@2a02:c7d:12e:100:49d:92bc:3989:ab0d] has joined #bitcoin-core-dev 20:03 -!- PaulCape_ [~PaulCapes@136.24.141.225] has joined #bitcoin-core-dev 20:03 -!- gm2053 [~gm2051@2a02:c7d:12e:100:49d:92bc:3989:ab0d] has quit [Read error: Connection reset by peer] 20:03 -!- PaulCapestany [~PaulCapes@2604:5500:17:2ea:c83a:20bc:e733:117f] has quit [Read error: Connection reset by peer] 20:05 -!- goksinen [~goksinen@2604:2000:c591:8400:9daa:eab8:a221:1989] has quit [Ping timeout: 255 seconds] 20:06 -!- n1ce [~n1ce@unaffiliated/n1ce] has quit [Remote host closed the connection] 20:07 -!- n1ce [~n1ce@unaffiliated/n1ce] has joined #bitcoin-core-dev 20:09 -!- n1ce [~n1ce@unaffiliated/n1ce] has quit [Remote host closed the connection] 20:10 -!- n1ce [~n1ce@unaffiliated/n1ce] has joined #bitcoin-core-dev 20:10 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 20:15 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 252 seconds] 20:18 < michagogo> cfields: https://www.irccloud.com/pastebin/XrNoFzY5/ 20:20 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev 20:25 -!- justan0theruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 20:25 -!- n1ce [~n1ce@unaffiliated/n1ce] has quit [Quit: Leaving] 20:27 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 260 seconds] 20:35 -!- waxwing [~waxwing@38.132.120.4] has quit [Ping timeout: 260 seconds] 20:47 -!- waxwing [waxwing@gateway/vpn/mullvad/x-rqwwovvmbeaqslbd] has joined #bitcoin-core-dev 20:49 -!- tw2006 [~tw2006@2601:187:8480:2770:1cb8:8217:620f:871b] has quit [Remote host closed the connection] 21:20 < michagogo> cfields: PR up now 21:26 < cfields> michagogo: thanks! 21:27 -!- harrymm [~wayne@104.237.91.122] has quit [Remote host closed the connection] 21:31 < cfields> gitian builders: detached sigs pushed for 0.14.1 21:32 -!- harrymm [~wayne@104.237.91.32] has joined #bitcoin-core-dev 21:41 < michagogo> Oops. 21:42 < michagogo> I was running a while ! loop on the script to do the signed build 21:43 < michagogo> But the script was returning failure, because the last step is opening a PR, which fails when there's already a PR open for the same branch 21:49 -!- tw2006 [~tw2006@2601:187:8480:2770:78c8:de71:4e9:c146] has joined #bitcoin-core-dev 21:54 -!- tw2006 [~tw2006@2601:187:8480:2770:78c8:de71:4e9:c146] has quit [Ping timeout: 260 seconds] 21:57 -!- d_t [~textual@108-65-78-188.lightspeed.sntcca.sbcglobal.net] has joined #bitcoin-core-dev 22:11 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 22:17 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 240 seconds] 22:29 -!- d9b4bef9 [~d9b4bef9@207.38.86.239] has quit [Remote host closed the connection] 22:32 -!- d9b4bef9 [~d9b4bef9@207.38.86.239] has joined #bitcoin-core-dev 22:39 -!- arubi_ is now known as arubi 22:56 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-wretphkizwdpbdhl] has joined #bitcoin-core-dev 23:01 -!- str4d [~str4d@27.110.123.91] has joined #bitcoin-core-dev 23:32 -!- RubenSomsen [~RubenSoms@5ED2CA1D.cm-7-3d.dynamic.ziggo.nl] has joined #bitcoin-core-dev 23:32 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has joined #bitcoin-core-dev 23:38 -!- tw2006 [~tw2006@2601:187:8480:2770:a0cd:afb0:b4e3:e83b] has joined #bitcoin-core-dev 23:43 -!- tw2006 [~tw2006@2601:187:8480:2770:a0cd:afb0:b4e3:e83b] has quit [Ping timeout: 255 seconds] 23:48 -!- gm2053 [~gm2051@2a02:c7d:12e:100:dc6d:7a8:2e69:4416] has joined #bitcoin-core-dev 23:49 -!- gm2051 [~gm2051@2a02:c7d:12e:100:7d6e:9411:9766:c3eb] has joined #bitcoin-core-dev 23:52 -!- gm2052 [~gm2051@2a02:c7d:12e:100:49d:92bc:3989:ab0d] has quit [Ping timeout: 260 seconds] 23:53 -!- gm2053 [~gm2051@2a02:c7d:12e:100:dc6d:7a8:2e69:4416] has quit [Ping timeout: 240 seconds] 23:58 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 258 seconds]