--- Day changed Sat Nov 14 2015
00:01 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-qihldvqzjefcxiyv] has joined #bitcoin-core-dev
00:05 -!- paveljanik [~paveljani@79-98-72-216.sys-data.com] has joined #bitcoin-core-dev
00:05 -!- paveljanik [~paveljani@79-98-72-216.sys-data.com] has quit [Changing host]
00:05 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev
00:07 < gmaxwell> wumpus: do you have any advice for phantomcircuit on my DEFAULT_BLOCKSONLY complaint?  My response to him was not terribly constructive... in part because I don't have any good advice. 
00:08 < gmaxwell> I'd have merged it already but for that issue.  
00:10 < wumpus> what is the issue exactly? can't find it in the comments
00:11 < wumpus> I do see DEFAULT_BLOCKSONLY is not actually used (besides documentation)
00:11 < wumpus> but what prevents him from doing that?
00:11 < gmaxwell> oh he adds a DEFAULT_BLOCKSONLY, but then doesn't use it in the getargs because some of them are in net.cpp.
00:11 < wumpus> move the constant to net.h?
00:11 < gmaxwell> And he would pass it as an argument but apparently pushversion is called in a blinking constructor.
00:12  * gmaxwell checks if he can do that easily
00:12 < wumpus> in general the constants should be defined in the header of the cpp they are used in
00:12 < wumpus> if they are used in multiple places that could be annoying, but at least init.cpp already inclused net.h so that's not a problem
00:12 < gmaxwell> :) I'd assumed there was a reason he couldn't do that, but looking now.
00:13 < gmaxwell> yup he can. one is in main.cpp but it also pulls in net.h.
00:13 < wumpus> otherwise define a global bool and assign that in init.cpp, instead of calling GetBoolArg every time, we do that for more settings (especially those used in inner loops where the overhead of argument-parsing-every-time hurts)
00:13 < wumpus> ok!
00:14 < gmaxwell> okay thanks for the cluestick. I'd assumed that he used them someplace that didn't have net.h but I didn't look.
00:25 < gmaxwell> sipa: someone was seeking feedback from you here: https://github.com/bitcoin/bitcoin/pull/6844
00:35 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev
00:46 -!- PaulCape_ [~PaulCapes@204.28.124.82] has quit [Quit: .]
00:46 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
01:22 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
01:26 -!- molly [~molly@unaffiliated/molly] has quit [Ping timeout: 276 seconds]
01:39 -!- molly [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
01:44 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 276 seconds]
01:49 < wumpus> interesting, so this seccomp filter is a little program (in a restricted instruction set) that is uploaded to the kernel that checks system calls and arguments and returns whether allowed or not
01:50 < wumpus> never expected it to be so dynamic
01:51 < wumpus> and it should be possible to set it as non-privileged user, given that PR_SET_NO_NEW_PRIVS is set first to avoid tricking set-uid executables with it
01:54 < wumpus> sounds ActuallyUseful, should do some experiments with it some time...
01:55 < gmaxwell> Yes. It is. I tried to do it for bitcoin core eons ago, but the utility library was GPLed... they since changed the licensing.
01:55 < wumpus> (somehow always avoided use of seccomp because I assumed you'd need root to set them, like chroot etc)
01:56 < wumpus> cjdns uses it without utility library
01:56 < gmaxwell> nope. The thing that makes me most sad right now is walletbackup and the import thing, since they require arbritary file access. But I'd like to replace them with little utility programs, which we'd let ourselves exec and talk to over pipes.
01:57 < wumpus> (but cjdns does recommend starting it as root, which was one of the things that put the idea in my head, but they need it for tun setup)
01:57 < wumpus> (as well as other jail functionality, which, unfortunately, does need root)
01:58 < wumpus> well my idea at least at first would be to have a restricted, secure mode, that disabled calls like that (and also walletnotify/blocknotify and everything that calls external processes)
01:58 < gmaxwell> ACK
02:00 < wumpus> but you're right about walletbackup/import, I would prefer if they just dumped/retrieved their data over HTTP instead of making files
02:01 < wumpus> within the JSON RPC framework this is difficult to solve, but outside that w/ using the http server directly it'd be pretty easy to stream instead
02:05 -!- Madars [~null@unaffiliated/madars] has joined #bitcoin-core-dev
02:06 < gmaxwell> I think there is some utility for triggering local backups, at least. But instead of arbritary file access it could simply be making it write out dated files; without giving the rpc caller huge freedom.
02:17 -!- btcdrak_ [uid115429@gateway/web/irccloud.com/x-feqmqxzmdxuavzvt] has joined #bitcoin-core-dev
02:17 -!- Arnavion [arnavion@unaffiliated/arnavion] has quit [Disconnected by services]
02:17 -!- Arnavion3 [arnavion@unaffiliated/arnavion] has joined #bitcoin-core-dev
02:17 -!- Arnavion3 is now known as Arnavion
02:17 -!- jgarzik_ [~jgarzik@104-178-201-106.lightspeed.tukrga.sbcglobal.net] has joined #bitcoin-core-dev
02:19 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has joined #bitcoin-core-dev
02:20 -!- Thireus1 [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
02:20 -!- midnightmagic_ [~midnightm@unaffiliated/midnightmagic] has joined #bitcoin-core-dev
02:20 -!- sipa_ [~pw@2a02:348:86:3011::1] has joined #bitcoin-core-dev
02:20 -!- tripleslash_l [~triplesla@unaffiliated/imsaguy] has quit [Ping timeout: 252 seconds]
02:20 -!- btcdrak [uid115429@gateway/web/irccloud.com/x-imyuodkxuzcjbsnp] has quit [Ping timeout: 252 seconds]
02:20 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has quit [Ping timeout: 252 seconds]
02:20 -!- zarathustra [zanoni@unaffiliated/tolstoi] has quit [Ping timeout: 252 seconds]
02:20 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Ping timeout: 252 seconds]
02:20 -!- jgarzik [~jgarzik@unaffiliated/jgarzik] has quit [Ping timeout: 252 seconds]
02:20 -!- sipa [~pw@unaffiliated/sipa1024] has quit [Ping timeout: 252 seconds]
02:20 -!- btcdrak_ is now known as btcdrak
02:30 < wumpus> one thing that would make sense is to write to the datadir only
02:30 < wumpus> it also proves that the user using the calls has access to it
02:31 < wumpus> on the other hand, of course people want to backup to mounted filesystems etc
02:31 -!- ParadoxSpiral [~ParadoxSp@p508B9C4B.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
02:32 < wumpus> hence I'd really prefer it to stream from/to http so that the client can decide where to put it or take it from, anywhere *they* have access to
02:33 < wumpus> this avoids using bitcoind in a confused deputy attack (e.g have it write /etc/passwd :-) )
02:34 < wumpus> (or read, for that matter, although you'd be hard pressed to find a file that it likes to read and interpret)
02:34 < wumpus> (except maybe *other* backups it happens to have access to)
02:43 -!- midnightmagic_ is now known as midnightmagic
02:50 -!- tulip [~tulip@128.199.135.43] has quit [Remote host closed the connection]
02:53 -!- zarathustra [zanoni@znc.openshells.net] has joined #bitcoin-core-dev
02:58 < gmaxwell> IMO if you want to backup to a mounted FS you can either symlink it into the datadir, or have something external copy there. but ::shrugs::
03:08 -!- tulip [~tulip@128.199.135.43] has joined #bitcoin-core-dev
03:26 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev
03:32 < morcos> phantomcircuit: I wrote a prototype separate thread for running CreateNewBlock, but the naive approach turned out much worse than the existing implementation.
03:33 < morcos> The issue was that the template creation code kept needing to hold cs_main, so you ended up holding that lock all the time and holding up things like block relay.
03:34 < morcos> Actually it makes a lot of sense to be sure you are not holding that lock at all when a new block comes in so you can validate it as quickly as possilbe and decide whether you should now be building off that
03:34 < morcos> Of course redesigning the template generation code to not need all the locking is probably what we need to do, but it turned out to be a much easier win just to speed up how long it takes
03:35 < morcos> It's really a mempool lock it needs but thats unfortunately cs_main partly.
03:36 < morcos> sipa and I discussed some ideas on having a transaction storage object, which the mempool and the mining code could separately refer to.
03:36 < gmaxwell> a while back matt was also just suggesting that CNB could terminate early if there was contention for the lock from a new block coming in.
03:37 < gmaxwell> which sounds a little hackish, but probably the kind of prudent hack that would work pretty well.
03:37 < gmaxwell> "Look, we've invented cooperative multitasking!"
03:37 < morcos> unfortunately everything uses cs_main though
03:37 < morcos> the hold createnewblock would have constantly been holding up ATMP by 3 secs or more
03:37 < morcos> old
03:37 < gmaxwell> yea, not at all a replacement for making it not embarassingly slow.
03:38 < morcos> i'm not saying its not fixable, but its a bit of work
03:38 -!- CodeShark_ [CodeShark@cpe-76-167-237-202.san.res.rr.com] has quit [Ping timeout: 260 seconds]
03:38 < gmaxwell> ::nods::
03:38 < morcos> also the other issue is all the package stucture that sdaftuar put into the mempool is potentially really important for mining
03:38 < morcos> and so we have to decide whether it makes sense to duplicate all that code
03:38 < morcos> should their be two mempools?
03:39 < morcos> that are each dynamically updated
03:39 < morcos> should the mining code just copy all it cares about from the real mempool and then do its work
03:39 < morcos> or perhaps my favorite approach is to fine grain the locks a little better, and then have a mode where txs can queue up to enter the mempool
03:40 < morcos> so you can manipulate the actual mempool in the mining code (i think it'll be relatively fast)
03:40 < morcos> but anyway, properly sorting out the locks is a precursor to any of it
03:40 < morcos> btw, i'm a bit confused by your 30 secs of the same work for mining hardware
03:41 < morcos> i didn't analyze too closely, but the antminer seemed to be sending new work much more often than that, or at least considering doing so
03:43 < gmaxwell> All modern devices have internal queing and pipelines so they don't stall, some are unfortunately pretty deep.
03:45 < morcos> will the pool software that phantomcircuit suggested use the longpool feature of gbt, or is that not what most people do
03:46 < morcos> constantly polling getblockcount and getblockhash seems pretty silly (what cgminer does)
03:46 < gmaxwell> the 30 second rule of thumb is an amalgmation of total delays, in pratice;-- including most pool server software being fairly slow. as far as hardware goes I'm not sure what the antminer s7 is like; S1 was very good (I beat on them pre-release); S2 was pretty terrible (and completely unusable with p2pool which has 30 second blocks); I think I heard S5 was better than S2, I haven't heard about S7
03:46 < gmaxwell> .  KNC devices have quite high latency (seconds to respond to new work in my expirence),  SP10/SP20 have quite low latency. 
03:47 < gmaxwell> ckpool uses blocknotify, many things (including p2pool) talk to bitcoind on the p2p port.  Eloipool will use GBT longpool I think, as well as p2p port.
03:47 < morcos> shocking that there are so many different mining software implementations, when its essentially consensus critical too
03:47 < gmaxwell> Even when things long poll they can take a really long time to get it out to the devices due to 'reasons'. (such as almost no one ever actually measuring)
03:48 < morcos> and do the big miners use the existing implementations, or have custom code.   and who maintains all those things
03:48 < morcos> sorry for the barrage of ?'s
03:48 < gmaxwell> morcos: well it's armored by bitcoin nodes in and out; not that there haven't been problems.
03:49 < gmaxwell> morcos: so historically mining pools have mostly been custom code; sometimes hacked on top of parts openly available.  Historically, public pools spend a lot of their development brainpower on dealing with DOS attacks; then data management for payout computation..
03:50 < morcos> are "pools" still what dominate mining or is it mostly single entities using the pool infrastructure b/c thats what exists
03:51 < gmaxwell> So there are still big public pools but many of them have their own hashpower (either directly or in commonly owned partner companies). The exact breakdown of things is unclear.
03:53 < gmaxwell> There are, of course, big entities now and they have simpler mining challenges (e.g. not so much having to worry about DOS).  They sometimes use existing public pools. (I mean some large entities are always on public pools; some only use them sometimes for various reasons)
03:54 < gmaxwell> Mining ecosystem is weird; as you observe there are a bazillion seperate implementations of this or that... and then no one goes and makes createnewblock fast.
03:54 < morcos> yeah, thats what i was wondering about
03:55 < morcos> but i suppose for our purposes (not that it would lead to a different design goal) what we should envision is i guess a p2pool implementation
03:55 < gmaxwell> Only 'mining pool' people who've really been active in core are luke-jr and phantomcircuit (who used to run the gear for a big-mining entitiy before working at BS).  And mostly, even for these guys the ideal strategy is to work around limitations; because doing so eas easier/safer.
03:55 < morcos> i mean the whole point is to make the small miner able to compete?
03:56 < gmaxwell> E.g. eloipool (the mining pool software luke-jr wrote that runs eligius) compensates for CNB latency by constructing empty blocks on its own, until CNB responds. And most mining operations just run multiple bitcoinds: one for GBT and several others that they submit through.
03:56 < tulip> morcos: many of the larger pools appear to run completely custom software, for the most part their quirks are completely unique. many of the smaller ones now use ckpool, which seems to be for the most part significantly faster than everybody else. 
03:57 < morcos> tulip: thanks.  i'm trying to narrow the universe of pool/mining software to explore
03:59 < tulip> morcos: ckpool, eloipool, p2pool, NOMP 
03:59 < gmaxwell> P2pool is nice idea with cute features, but has always struggled with the high startup cost of having to run a bitcoin node with it; and then high latency asics showed up (in particular, some of the early BFL products had 10%+ hashrate loss from 10 second retasking)
04:00 < gmaxwell> and that pretty much killed it there, and when it wasn't yet dead enough; somewhat later antpool announced that they'd be converting to p2pool based (but with a friendly front end where they run it for you) and lots of p2pool users moved over, then they didn't actually do it. 
04:00 < gmaxwell> so now p2pool is too low of a hashrate for anyone who is variance twitchy to use. :(
04:01 < morcos> oh no, thats sad
04:01 < gmaxwell> Also through this time the developer burned out on Bitcoin, and only does life support maintaince on p2pool now.
04:02 < gmaxwell> at the moment P2Pool's hashrate is 918 TH, says my local p2pool node.
04:02 < gmaxwell> (it's pretty neat, has a built in webserver and graphs and such)
04:03 < gmaxwell> (Forrestv seems to have flamed out due to a mixture of bitcoin drama, and then donations to support p2pool being relatively low (compared to life changing amounts of income centeralized pool operators were getting), and then he got ripped off ... twice, I think, by mining hardware makers.)
04:07 < tulip> morcos: there's also stratum proxies like bfgminer and stratum-mining-proxy which present work to downstream miners, based on either an upstream work server or a Bitcoin node. the latter was meant for use with early hardware miners which supported getwork but not stratum. 
04:10 < gmaxwell> then of course there is all these mining devices with embedded mips and arm linux systems that run usually very old versions of cgminer (or sometimes bfgminer)
04:24 < gmaxwell> morcos: while mentioning p2pool I was looking at my stats, and the 1 year view on GBT latency is amusing: https://people.xiph.org/~greg/p2pool-gbt.png
04:43 -!- tulip [~tulip@128.199.135.43] has quit []
05:06 < midnightmagic> sigh
05:23 < GitHub86> [bitcoin] gmaxwell pushed 9 new commits to master: https://github.com/bitcoin/bitcoin/compare/9ffc687288dd...b632145edeb3
05:23 < GitHub86> bitcoin/master 4044f07 Patick Strateman: Add blocksonly mode
05:23 < GitHub86> bitcoin/master 420fa81 Patick Strateman: Do not process tx inv's in blocksonly mode
05:23 < GitHub86> bitcoin/master 3a96497 Patick Strateman: Add whitelistalwaysrelay option
05:23 < GitHub168> [bitcoin] gmaxwell closed pull request #6993: Add -blocksonly option (master...blocksonly) https://github.com/bitcoin/bitcoin/pull/6993
05:28 < phantomcircuit> morcos, generally speaking.. yes i dont care that cnb is slow because it's mostly irrelevant
05:28 < phantomcircuit> and i really dont care that it holds a lock for ages since blocks are submitted to nodes dedicated for that
05:29 < gmaxwell> it's much less irrelevant if you're not assuming a big miner farm.
05:29 < phantomcircuit> gmaxwell, true
05:30 < morcos> phantomcircuit: i thinking having getblocktemplate and createnewblock quickly return a valid block after a new tip is important to everyone
05:30 < morcos> even better if it is with txs, but not critical
05:31 < phantomcircuit> morcos, well making it return *something* immediately afterwards would certainly be nice
05:31 < phantomcircuit> that would reduce the need for some of the comical hacks
05:31 < phantomcircuit> but as gmaxwell said most of the hardware has queues that are very long and dont flush
05:31 < phantomcircuit> (there's good reason for this but i wont get into that)
05:32 < morcos> right so the way i look at it is assuming we don't want to code up validationless mining (which i am not yet convinced of)  then we need 100ish ms to validate the new tip and about 100ish ms to generate a new block with txs (with the new code)
05:32 < morcos> so at most we could carve out the second 100ms, but at this point we've gotten 95% of the improvement, so maybe not worth doing that unless we are going all the way
05:32 < phantomcircuit> morcos, 100ms to return an empty but validated block template would be better :P
05:33 < morcos> and before gmaxwell reaches through the tubes to strangle me, my primary argument for considering doing validationless mining is that if people are going to implement it anyway, we might as well make it as safe as possible
05:34 < morcos> but i agree that if there are other bottlenecks to switching work, then its not something we can do...   (validationless block header then 100-200ms later the validated version, if you can't switch after 200ms, then its not safe to start on the unvalidated version)
05:35 < morcos> but that same argument applies much less importantly to empty blocks.  if you're not going to switch after 100ms to the block with txs...  then you probably dont' want to start with the empty block...   depends on the amount of fees and the minimum switching delay i guess
05:35 < gmaxwell> If we want to implement validationless mining, then we need to do it right; which would mean doing something like signaling in the block header what we haven't verified prev, and so SPV clients should ignore the block for conf count purposes.
05:36 < gmaxwell> But I'd rather get validation lower first. :)
05:36 < phantomcircuit> morcos, there's basically no bottleneck in the validation
05:37 < morcos> gmaxwell: sure agreed.   phantomcircuit: no bottleneck? i'm talking about latency in sending new work to hardware outside of bitcoind
05:37 < morcos> i'd guess that a significant chunk of the remaining 100ms is allocation (in the non degenerate block case)
05:38 < phantomcircuit> morcos, yes... the issue is almost entirely not in validation but in selecting transactions
05:38 < gmaxwell> it's an interesting point that the extra 100ms doesn't matter. But rather, I think getting the 100ms upfront matters, and then it doesn't really matter if CNB is slow (except for lock holding reasons)
05:38 < phantomcircuit> the mempool indexing + limiting mostly fixes that
05:38 < morcos> phantomcircuit: huh? no thats backwards.  selecting txs is blazingly fast.  (well until cpfp)
05:39 < morcos> oh you're talking about in master
05:39 < morcos> yeah sure.. i'm talking about the remaining 100ms...  which is 10ms tx selection and 90 ms validation.   tx selection is still only 20ms if you scan an entire 300mb mempool to also sort by priority (with a fast priority calc)
05:40 < phantomcircuit> morcos, im not (and i dont think anybody paying attention) is worried about validation times of 100ms
05:41 < morcos> i see 100 ms, and i think the units still need to change.  :)
05:41 < phantomcircuit> that's like 0.01% orphan rate
05:41 < phantomcircuit> yes well faster is always better
05:42 < gmaxwell> hah. I think 100ms is really slow, but people clearly don't mind that much. But they don't mind in part because they perform hacks; some of which are harmful and we'd like them to stop. :)
05:42 < morcos> actually we are forgetting to measure some things there
05:42 < morcos> i'm measuring from receive block to new tip = 100ms  and new tip to new template = 100ms
05:42 < gmaxwell> when I say they don't mind that much; a year ago there were two places in the network handling code that just interted 100ms _sleeps_. 
05:42 < morcos> but we also need to worry about receive most work header to receive block
05:43 < morcos> that's probably what we need to optimize now
05:43 < phantomcircuit> yes the push head instead of inv patch is a huge win for that
05:43 < morcos> or even someone else genarates new tip to receiving most work header.  direct headers will help with that
05:44 < gmaxwell> yes, well we can remove a one way delay by nominating your favorite peer to relay uninvited; beyond that needs something like matt's relay protocol. (or better)
05:44 < morcos> yeah i need to look into how the relay client works...  if you generate a new tip... how do you communicate that to the relay network, same as p2p
05:44  * phantomcircuit looks at brain clock *wow so late* goes to sleep
05:45 < gmaxwell> morcos: the relay network has its own protocol, and a local proxy you run that speaks bitcoin p2p.
05:45 < morcos> oh i suppose if you submitted to a node thats not mining, then it doesn't have the problem of that immediate cs_main lock from gbt after you generate a new tim
05:45 < morcos> tip
05:45 < gmaxwell> yea, thats why the earlier mentioned architecture of multiple nodes.
05:46 < gmaxwell> The relay protocol is super simple. It can send lose transactions, and keeps a history of the X sent. When you want to send a block it sends the header and a list of two byte indexes into that transaction history, as well as any history-miss transactions.
05:47 < gmaxwell> other end reconstructs the reblock and passes it on to bitcond unsolicited.
05:48 < gmaxwell> his server side, passes all transactions through a local bitcoind, and only relays what it spits out.. but for blocks it checks work and relays them on without more then minimal verification.
05:50 < gmaxwell> This really simple approach frequently managed to turn 1MB blocks into 3.6KB. http://bitcoinrelaynetwork.org/stats.html  and leaves matt either fighting sha256 speed ... or mempool spam that breaks his hitrates.
05:57 < GitHub101> [bitcoin] gmaxwell opened pull request #7016: Avoid a compile error on hosts with libevent too old for EVENT_LOG_WARN. (master...without_EVENT_LOG_WARN) https://github.com/bitcoin/bitcoin/pull/7016
06:19 -!- zarathustra [zanoni@znc.openshells.net] has quit [Ping timeout: 252 seconds]
07:13 -!- zarathustra [zanoni@znc.openshells.net] has joined #bitcoin-core-dev
07:25 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
07:26 -!- You're now known as kanzure
08:23 -!- bsm117532 [~mcelrath@static-108-21-236-13.nycmny.fios.verizon.net] has quit [Remote host closed the connection]
08:27 -!- jgarzik_ [~jgarzik@104-178-201-106.lightspeed.tukrga.sbcglobal.net] has quit [Changing host]
08:27 -!- jgarzik_ [~jgarzik@unaffiliated/jgarzik] has joined #bitcoin-core-dev
08:27 -!- jgarzik_ is now known as jgarzik
08:30 -!- sipa_ is now known as sipa
08:34 -!- vev__ [~vev@67.95.148.77.rev.sfr.net] has joined #bitcoin-core-dev
08:34 < vev__> we need people to support us #libreidea
08:39 -!- vev__ [~vev@67.95.148.77.rev.sfr.net] has left #bitcoin-core-dev ["Leaving, see you on libreidea"]
08:52 -!- jl2012 [~jl2012@unaffiliated/jl2012] has quit [Ping timeout: 240 seconds]
08:52 -!- jl2012 [~jl2012@unaffiliated/jl2012] has joined #bitcoin-core-dev
09:31 -!- zooko [~user@2601:281:8001:26aa:9116:88c2:9cf:8889] has joined #bitcoin-core-dev
10:23 -!- lightningbot [supybot@2400:8900::f03c:91ff:fedf:3a06] has quit [Remote host closed the connection]
10:25 -!- lightningbot [supybot@2400:8900::f03c:91ff:fedf:3a06] has joined #bitcoin-core-dev
10:28 -!- Thireus1 [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
10:31 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
10:34 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
10:40 -!- jl2012 [~jl2012@unaffiliated/jl2012] has quit [Read error: Connection reset by peer]
10:58 -!- zooko [~user@2601:281:8001:26aa:9116:88c2:9cf:8889] has quit [Ping timeout: 240 seconds]
11:03 < GitHub50> [bitcoin] morcos closed pull request #6292: Rename and comment priority calculation in TxMemPoolEntry (master...PriorityComment) https://github.com/bitcoin/bitcoin/pull/6292
11:06 < morcos> Can someone tag #6134 with 0.12 milestone.
11:13 < morcos> I also think #6915 should go in for 0.12.  
11:17 -!- jl2012 [~jl2012@unaffiliated/jl2012] has joined #bitcoin-core-dev
11:21 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
11:42 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has joined #bitcoin-core-dev
11:44 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Ping timeout: 250 seconds]
11:50 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev
12:26 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
12:27 -!- zarathustra [zanoni@znc.openshells.net] has quit [Changing host]
12:27 -!- zarathustra [zanoni@unaffiliated/tolstoi] has joined #bitcoin-core-dev
13:39 -!- PRab [~chatzilla@2601:40a:8000:8f9b:c49b:a2dc:3cab:6419] has quit [Quit: ChatZilla 0.9.92 [Firefox 42.0/20151029151421]]
13:42 -!- andytoshi [~andytoshi@wpsoftware.net] has quit [Changing host]
13:42 -!- andytoshi [~andytoshi@unaffiliated/andytoshi] has joined #bitcoin-core-dev
13:43 < GitHub136> [bitcoin] MarcoFalke opened pull request #7019: [trivial] travis: cover *receivedby* rpcs (master...MarcoFalke-2015-receivedby) https://github.com/bitcoin/bitcoin/pull/7019
13:46 -!- PaulCape_ [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
13:49 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Ping timeout: 240 seconds]
13:57 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Ping timeout: 240 seconds]
14:03 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]
14:05 -!- petertod1 is now known as petertodd
14:05 -!- petertodd is now known as Guest19775
14:14 -!- MarcoFalke [8af6026a@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.106] has joined #bitcoin-core-dev
14:32 -!- PaulCape_ [~PaulCapes@204.28.124.82] has quit [Quit: .]
14:32 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
14:35 -!- zooko [~user@2601:281:8001:26aa:280c:9c50:9fff:58b7] has joined #bitcoin-core-dev
15:10 -!- CodeShark_ [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has joined #bitcoin-core-dev
15:11 -!- MarcoFalke [8af6026a@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.106] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]
15:17 -!- Guest19775 is now known as petertodd
15:25 -!- alpalp [~alp@104-54-235-28.lightspeed.austtx.sbcglobal.net] has joined #bitcoin-core-dev
15:25 -!- alpalp [~alp@104-54-235-28.lightspeed.austtx.sbcglobal.net] has quit [Remote host closed the connection]
15:35 -!- MarcoFalke [d9739264@gateway/web/cgi-irc/kiwiirc.com/ip.217.115.146.100] has joined #bitcoin-core-dev
15:44 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
15:48 -!- molly [~molly@unaffiliated/molly] has quit [Ping timeout: 276 seconds]
15:55 -!- molly [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
15:59 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 276 seconds]
16:07 -!- MarcoFalke [d9739264@gateway/web/cgi-irc/kiwiirc.com/ip.217.115.146.100] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]
16:08 -!- MarcoFalke [d9739264@gateway/web/cgi-irc/kiwiirc.com/ip.217.115.146.100] has joined #bitcoin-core-dev
16:08 < gmaxwell> so at devcore one of the things I talked about was some analysis taken from monitoring miners and mining pools. Someone collected data from all accessible stratum endpoints over several months
16:10 < gmaxwell> and from it I can extract data like how much time from the first pool working on extending block X was it until the 2nd, 3rd, ... nth pool.  Taking the time after the first pool to reach half the pools, fit a linear model reasonably well.
16:10 < jgarzik> cool
16:11 < gmaxwell> oh yea you weren't there when I presented on this.
16:12 < gmaxwell> I tried all sorts of different analysis approaches, including including factors for in china or not, pool origin, etc. but there really aren't enough blocks (esp from smaller miners) to say much of statistical significance.  But size, did.
16:12 < gmaxwell> Model that R comes up with for that is:
16:12 < gmaxwell> 20:40 <gmaxwell> (Intercept) 2.491e+00  2.674e-01   9.318   <2e-16 ***
16:12 < gmaxwell> 20:40 <gmaxwell> size        1.106e-05  4.025e-07  27.474   <2e-16 ***
16:13 < gmaxwell> which means a 2.491 second constant delay, plus 732kbit/sec. (1/1.106*10^-5*8)
16:15 < gmaxwell> interestingly, plugging this into an orphaning calculation vs the subsidy---- suggests that the final byte of the block should have a feerate of ((((e^(-1/600*(2.491+((1000000)/90415.91)))))*25)-(((e^(-1/600*(2.491+((1000001)/90415.91)))))*25))*1000 = .00045054 BTC per 1000 bytes, or otherwise you're losing money just considering the subsidy.
16:15 < gmaxwell> Though reality is not that simple, because of hashpower distribution dynamics, large miners don't really care if it takes small miners a long time to get their blocks.
16:17 < gmaxwell> so, take with a metric ton of salt, but I thought it was interesting that these figures are in roughly this magnitude.
16:18 -!- alpalp [~alp@104-54-235-28.lightspeed.austtx.sbcglobal.net] has joined #bitcoin-core-dev
16:23 < jgarzik> definitely interesting. mostly aligns with my guesstimation of miner+network behavior.
16:26 < gmaxwell> This way of looking it has some surprising results, but I think correct-- e.g. if you decrease the fixed delay then you actually want a _higher_ feerate for the last byte of the block for it to break even.  because of the exponentially decreasing slope of the orphaning rate as you move away from 0 latency.  1 byte of extra delay makes more of a difference if your base delay is lower.
16:27 -!- pmienk_ [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has quit [Ping timeout: 240 seconds]
16:29 < sipa> to reach half the pools... weighted by hashrate?
16:29 < gmaxwell> yea, not weighed; thats what I was trying to talk about with my "not that simple"
16:29 < sipa> right
16:29 < sipa> hard to do with data that spans several months
16:30 < gmaxwell> because of that the information is a bit more informative about equality/fairness than it is about the economics of fees.
16:30 < gmaxwell> Though it's an interesting question if the network should be relaying transactions with fees so low that only very large hashpower consolidations could mine them except at a loss.
16:31 < gmaxwell> also it suggests a framework for setting minimum feerates which are independant of bitcoin's price-- though dependant on communications efficiency, which is perhaps no better. :)
16:38 < gmaxwell> The reason I went to go crunch the numbers into a feerate is that I was thinking about what the minimum really should be.
16:39 -!- pmienk_ [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has joined #bitcoin-core-dev
16:42 < zooko> gmaxwell: I really like what you wrote on -wizards after I parted last time about why people don't treat solo mining as gambling.
16:43 < zooko> I really think you are right that it is a user-experience issue, not an economic issue.
16:43 < zooko> If some state lottery offered a scheme where you subscribed and then it would run in the background and eventually someday maybe it would pop up and give you money, 
16:43 < zooko> I think that would be a stinkier.
16:43 < zooko> a stinker.
16:43 < zooko>  I mean, nobody would play.
16:44 < zooko> Instead, you get the build-up-and-anticipation-and-reveal cycle, like scratching off the silver coating to reveal the numbers beneath and find out if you won.
16:44 < jgarzik> ...maybe this is -wizards material ;p
16:44 < zooko> If that's right, you could add lottery UX on top of mining, by giving people a button that they can push and then it let ....
16:44 < zooko> WRONG CHAMN
16:44 < zooko> Thanks, jg.
16:45 < sipa> that's a contraction of "wrong chan" and "damn" ?
16:45 < gmaxwell> Well it's also a bit #bitcoin-core-dev too, in that I think it would be useful if work were done in the GUI to make mining fun, ... but probably more than that is just speculation :)
16:46 -!- ParadoxSpiral [~ParadoxSp@p508B9C4B.dip0.t-ipconnect.de] has quit [Remote host closed the connection]
16:46 < sipa> hmm, if only we had some sort of hash-within-range unlockable scripts, where a block's coinbase is assigned to, so you can postfacto determine who gets it
16:47 < sipa> then you could buy a range of hashes from a miner
17:16 -!- MarcoFalke [d9739264@gateway/web/cgi-irc/kiwiirc.com/ip.217.115.146.100] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]
17:23 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
17:35 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Ping timeout: 260 seconds]
17:44 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
17:52 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
18:24 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-qihldvqzjefcxiyv] has quit [Quit: Connection closed for inactivity]
18:26 < GitHub184> [bitcoin] morcos opened pull request #7020: Implement helper class for CTxMemPoolEntry constructor (master...EntryHelper) https://github.com/bitcoin/bitcoin/pull/7020
18:35 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has quit [Ping timeout: 240 seconds]
18:50 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
18:53 -!- molly [~molly@unaffiliated/molly] has quit [Ping timeout: 276 seconds]
19:01 -!- jl2012_ [~jl2012@119246245241.ctinets.com] has joined #bitcoin-core-dev
19:02 -!- jl2012 [~jl2012@unaffiliated/jl2012] has quit [Ping timeout: 276 seconds]
19:10 < Luke-Jr> sipa: is there a purpose for such a construct?
19:23 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
20:07 < gmaxwell> ::sigh:: libpng security firedrill.
20:09 < Luke-Jr> was there a real risk for us?
20:10 < Luke-Jr> (only PNGs we ever use are our own, right?)
20:11 < CodeShark_> are you referring to https://www.cvedetails.com/cve/CVE-2015-8126/ ?
20:11 < gmaxwell> yes, not a risk for us but we'll probably get asked to update when people notice bitcoin-qt links it.
20:12 < CodeShark_> we don't use libpng with any user or network supplied data directly, do we?
20:13 < gmaxwell> FWIW, I've unsubscribed from bitcoin-dev mailing list.
20:14 < Luke-Jr> sigh
20:15 < gmaxwell> Luke-Jr: no biggie, it's just still very low SNR.
20:18 < CodeShark_> oh well...I guess the value of the list tanks even further
20:28 -!- PRab [~chatzilla@2601:40a:8000:8f9b:d904:3340:2389:a85e] has joined #bitcoin-core-dev
20:42 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
20:45 -!- lecusemble [~lecusembl@104.233.76.133] has quit [Ping timeout: 264 seconds]
20:45 -!- guruvan [~guruvan@unaffiliated/guruvan] has quit [Ping timeout: 264 seconds]
20:45 -!- baldur [~baldur@pool-173-52-43-219.nycmny.fios.verizon.net] has quit [Ping timeout: 264 seconds]
20:45 -!- gavinandresen [~gavin@unaffiliated/gavinandresen] has quit [Ping timeout: 264 seconds]
20:46 -!- gavinandresen [~gavin@unaffiliated/gavinandresen] has joined #bitcoin-core-dev
20:46 -!- lecusemble [~lecusembl@f9beb4d9.violates.me] has joined #bitcoin-core-dev
20:48 -!- PaulCape_ [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
20:50 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Ping timeout: 246 seconds]
20:52 -!- guruvan [~guruvan@unaffiliated/guruvan] has joined #bitcoin-core-dev
20:59 -!- baldur [~baldur@pool-173-52-43-219.nycmny.fios.verizon.net] has joined #bitcoin-core-dev
21:13 < Luke-Jr> FWIW, I figured out why the CLTV tests failed on 0.11.2: https://github.com/bitcoin/bitcoin/pull/6523#issuecomment-156782660
21:14 < Luke-Jr> (Nothing to worry about)
21:26 < gmaxwell> Luke-Jr: I think your request translates to "please don't write software in python" :)
21:27 < Luke-Jr> gmaxwell: well, C++ has most of the same problems in this regard, and the same solutions work in Python
21:27 < Luke-Jr> (eg, renaming the function)
21:27 < kanzure> not sure python is the cause of the problem here. taking same value type but treating differently seems like sort of thing you would catch while grepping for the callers, or checking whether your function previously had different meaning..
21:27 < Luke-Jr> actually, Python has an additional option: it could be made to reject unnamed parameters, so the caller must explicitly specify height=N
21:28 < Luke-Jr> kanzure: in this case, it was breaking a backport
21:28 < Luke-Jr> kanzure: CLTV tests were written *after* this change, and when I went to use them with 0.11.2, it silently behaved differently
21:29 < kanzure> was there a test failure that caught this?
21:30 < Luke-Jr> the test failed as a result
21:32  * Luke-Jr wonders if he's the only one who tried to run the CLTV tests against 0.11  :x
22:06 -!- Guest25458 [~pigeons@94.242.209.214] has quit [Ping timeout: 240 seconds]
22:07 -!- pigeons [~pigeons@94.242.209.214] has joined #bitcoin-core-dev
22:08 -!- pigeons is now known as Guest95455
22:10 -!- zooko [~user@2601:281:8001:26aa:280c:9c50:9fff:58b7] has quit [Ping timeout: 240 seconds]
23:41 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev
23:48 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Remote host closed the connection]
23:48 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev