--- Day changed Sat Nov 21 2015
00:24 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 276 seconds]
00:35 -!- guest234234 [~guest2342@223.207.239.241] has joined #bitcoin-core-dev
01:11 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-ubvfbctavfqlbwtn] has joined #bitcoin-core-dev
02:03 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has quit [Read error: Connection reset by peer]
02:04 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has joined #bitcoin-core-dev
02:07 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has quit [Quit: Leaving.]
02:08 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev
02:39 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 272 seconds]
03:02 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev
03:35 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Quit: .]
03:37 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
04:03 -!- guest234234 [~guest2342@223.207.239.241] has quit [Ping timeout: 250 seconds]
04:35 -!- belcher [~user@unaffiliated/belcher] has joined #bitcoin-core-dev
04:57 -!- guest234234 [~guest2342@223.207.239.241] has joined #bitcoin-core-dev
05:05 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
05:35 -!- tulip [~tulip@unaffiliated/tulip] has quit []
05:44 < jtimon> morcos the linked commit doesn't touch TrimToSize, that's the next one
05:46 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has quit [Ping timeout: 246 seconds]
05:46 < jtimon> ACK on policy estimator called directly instead of through the mempool (but I've been asked not to worked not to work on policy refactors for now) and that one is certainly disruptive
05:47 < jtimon> setting the attribute by calling TrimToSize seems weird, what's wrong with doing it in the constructor (or in a setter if necessary)?
05:48 < jtimon> morcos_: the estimator is currently not asking questions to the mempool and it doesn't need to, please don't couple that again
05:55 < jtimon> you want the mempool decoupled from policy/fees and I agree, and have done it already in a "private" outdated branch
05:55 < jtimon> but to do so, it is not necessary to couple policy/fees to the mempool instead, they can be completely independent
05:56 < jtimon> whenever we think is the moment, I can completely decouple them
06:05 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has quit [Ping timeout: 265 seconds]
06:29 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
06:41 -!- Anduck_ is now known as Anduck
07:02 -!- morcos_ is now known as morcos
07:04 < morcos> jtimon: the question is where shoould the logic live in smartly estimate fees.  in the policyestimator.  that logic requires asking question of the mempool. otherwise you're going to have to put that logic repeated in several different places in wallet and gui.
07:04 < morcos> s/live in/live to/
07:06 < morcos> jtimon: the reason I like setting the mempool size in TrimToSize is you can easily imagine logic where the size is non-constant.  So after you have trimmed it the mempool knows what size it currently is  (the logic as to what size to trim it lives outside mempool)
07:06 < morcos> for instance we discussed using less size for the mempool during IBD so you could use more of your allocated memory for the dbcache
07:06 < morcos> or you can imagine other scenarios
07:24 -!- zookolaptop [~user@2601:281:8001:26aa:3d71:f893:f860:7f61] has joined #bitcoin-core-dev
07:26 -!- guest234234 [~guest2342@223.207.239.241] has quit [Ping timeout: 240 seconds]
07:35 -!- trippysalmon [rob@2001:984:6466:0:a0d3:d9b8:fab7:bab3] has joined #bitcoin-core-dev
07:44 -!- trippysalmon [rob@2001:984:6466:0:a0d3:d9b8:fab7:bab3] has quit [Ping timeout: 250 seconds]
07:57 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has joined #bitcoin-core-dev
08:09 -!- JackH [~Jack@host-80-43-142-236.as13285.net] has joined #bitcoin-core-dev
08:25 -!- zmanian_ [sid113594@gateway/web/irccloud.com/x-trszzgfujeczpftq] has quit [Ping timeout: 272 seconds]
08:32 -!- zookolaptop [~user@2601:281:8001:26aa:3d71:f893:f860:7f61] has quit [Ping timeout: 240 seconds]
08:33 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has quit [Ping timeout: 240 seconds]
08:35 -!- MarcoFalke [8af602bc@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.188] has joined #bitcoin-core-dev
08:49 < gmaxwell> 1h 11m to rescan a wallet with 11.7m transactions now.
08:51 < sipa> how long for a wallet with 0?
08:53 < gmaxwell> sipa: less than 10 minutes.
08:53 < gmaxwell> listunspent on that wallet is taking 2m10s right now (to return 15870 coins)
08:54 -!- zmanian_ [sid113594@gateway/web/irccloud.com/x-sskeiezucgcweaqb] has joined #bitcoin-core-dev
08:55 < gmaxwell> Wallet.dat is 11G which isn't bad.
08:58 < gmaxwell> bleh, and getinfo takes 1m8s.
08:59 < gmaxwell> Luke-Jr: It might be useful to add an index of which transactions have unspent coins to make listunspent fast.  But I think with the way balance calculations work getinfo is going to remain slow. :(
09:01 < Luke-Jr> I have never once had a reason to use listunspent..
09:01 < gmaxwell> Luke-Jr: it also means selectcoins will be superslow.
09:04 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-ubvfbctavfqlbwtn] has quit [Quit: Connection closed for inactivity]
09:07 < gmaxwell> 4m 15s for a getbalance "*" 0 true, I dunno why getinfo is faster.
09:14 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-wlatfwewhaqzrxjb] has joined #bitcoin-core-dev
09:34 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has joined #bitcoin-core-dev
09:45 -!- teward [teward@ubuntu/member/teward] has quit [Ping timeout: 240 seconds]
09:52 -!- teward [teward@ubuntu/member/teward] has joined #bitcoin-core-dev
10:14 < gmaxwell> ::sigh:: we really need a remove feature for the wallet, but the remove needs to keep track of what was removed so rescan doesn't read... and we can't remove things with spendable txouts because they're not seperate.
10:18 < Luke-Jr> jonas is still rewriting it, right?
10:22 < gmaxwell> We've fucked over the project for years with that kind of thinking; we can't stop improving the wallet because of a grand rewrite.
10:24 < Luke-Jr> sure, it's just one of those things that if I were to personally try to do it, I would end up probably rewriting the wallet myself :p
10:24 < sipa> gmaxwell: what do you mean by remove?
10:25 < sipa> remove keys?
10:25 < gmaxwell> sipa: no, no reason to remove keys. Remove transactions.
10:26 < gmaxwell> sipa: right now large parties using bitcoin core have to periodically rotate out wallets to keep things managable. Things are much better now because of varrious fat trimming. (Including the addtowallet fix we just merged from luke)
10:27 < sipa> so more like mark transactions as archived
10:27 < sipa> ?
10:27 < sipa> so they are no longer considered for balance computations etc
10:27 < gmaxwell> Yea, in particular, get them out of the linear scans used to getbalance/listunspent/etc.
10:28 < gmaxwell> But not do so in a way that causes coins to fall out of balance and listunspent, because that will cause funds loss when you think a wallet is empty and it's not.
10:29 < gmaxwell> so that could be refusing to archive when there are unspent outputs, but thats kind of obnoxious, since it would make visible wallet internal behavior that the caller shouldn't really care about.
10:29 < gmaxwell> (Though, otoh, it might encourage sweeping the utxo set. :) )
10:30 < gmaxwell> Also, 'the must be spent completely' rule wouldn't be reorg robust.
10:30 < gmaxwell> And, of course, an archived transaction shouldn't be added back by rescanning.
10:31 < gmaxwell> Some parties (e.g. bitstamp about a year ago) also complained about the size of the wallet files when they had long histories because it complicated backups; but I think the key exports patch that somewhat.
10:32 < sipa> seems easy enough to build a second map inside that does not contain transactions whose outputs have been spent for ages
10:32 < sipa> andnuse that for balance calculations etc
10:34 < gmaxwell> I think accounts mess this up somewhat; or at least make it not a transparent implementation detail.
10:38 < sipa> uh, right
10:38 < gmaxwell> thats why I was talking about remove/archive. :-/
10:43 -!- CodeShark [uid126576@gateway/web/irccloud.com/x-tquvgkenoajvipxr] has quit [Quit: Connection closed for inactivity]
11:24 -!- zarathustra [zanoni@unaffiliated/tolstoi] has quit [Quit: do what thou wilt shall be the whole of the law]
12:19 -!- belcher [~user@unaffiliated/belcher] has quit [Quit: Leaving]
12:23 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
12:34 -!- ParadoxSpiral [~ParadoxSp@p508B9D5F.dip0.t-ipconnect.de] has quit [Remote host closed the connection]
12:37 -!- ParadoxSpiral [~ParadoxSp@p508B9D5F.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
12:46 -!- AtashiCon [arnavion@unaffiliated/arnavion] has quit [Quit: AtashiCon]
12:50 -!- AtashiCon [arnavion@unaffiliated/arnavion] has joined #bitcoin-core-dev
13:31 -!- helo [~helo@unaffiliated/helo] has quit [Ping timeout: 240 seconds]
13:32 -!- helo [~helo@unaffiliated/helo] has joined #bitcoin-core-dev
14:01 < GitHub9> [bitcoin] gmaxwell pushed 5 new commits to master: https://github.com/bitcoin/bitcoin/compare/616d61b20d56...31de2414c65d
14:01 < GitHub9> bitcoin/master 748321e Peter Todd: Add mediantime field to getblockchaininfo RPC call...
14:01 < GitHub9> bitcoin/master c277a63 Peter Todd: Clarify nLockTime-by-time comment in CheckFinalTx()
14:01 < GitHub9> bitcoin/master 7259769 Peter Todd: Document new mediantime field in getblockchaininfo
14:01 < GitHub66> [bitcoin] gmaxwell closed pull request #7011: Add mediantime to getblockchaininfo (master...2015-11-add-mediantime-to-getblockchaininfo) https://github.com/bitcoin/bitcoin/pull/7011
14:23 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]
14:56 -!- CodeShark [uid126576@gateway/web/irccloud.com/x-xvbbvxshszhsbbaa] has joined #bitcoin-core-dev
15:34 < gmaxwell> phantomcircuit: were you going to fix the trickle logic? it really is broken.
15:35 < sipa> yes, please!
15:36 < gmaxwell> I'm testing the blocks only mode with relaynetwork client as a whitelisted peer (and I cut out the forced broadcasting for whitelisted peers)... and I'm watching it instantly inv any tx that comes in to all it's neighbors.
15:44 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev
15:52 -!- MarcoFalke [8af602bc@gateway/web/cgi-irc/kiwiirc.com/ip.138.246.2.188] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]
15:55 < gmaxwell> of course, there is some node connected to me which is sending me every transaction without an INV.
16:08 -!- guest234234 [~guest2342@223.207.239.241] has joined #bitcoin-core-dev
16:14 -!- guest234234 [~guest2342@223.207.239.241] has quit [Read error: Connection reset by peer]
16:26 -!- ParadoxSpiral [~ParadoxSp@p508B9D5F.dip0.t-ipconnect.de] has quit [Remote host closed the connection]
16:30 < davec> gmaxwell: I have noticed that happening to me a while ago as well.  Unfortunately with there being impls like BitcoinJ (and probably others) that rely on that behavior can't really easily stop it
16:31 < davec> I haven't done any testing, but at one point I was considering doing some type of logic to only allow X unsolicited transactions in Y timeframe
16:31 < gmaxwell> davec: Part of the motivation for DOS in bitcoin core is that we could ban peers that do a lot of something while leaving others alone.
16:32 < gmaxwell> though I'd prefer to get bitcoinj fixed first; and have that just clean up whatever remains.
16:32 < davec> That is certainly preferred if the new maintainers are willing to change it
16:34 < CodeShark> I think they would given sufficient pressure ;)
16:35 < gmaxwell> We haven't asked the new maintainer yet, I think.
16:35 < gmaxwell> I was thinking of seeing if I could sweet talk bluematt into just sending a patch. :)
16:38 < BlueMatt> i know i know
16:38 < BlueMatt> i told you i would....
16:38 < BlueMatt> oh, this is unrelated to other things
16:39 < BlueMatt> yea, i mean you should be able to ping andreas
16:39 < gmaxwell> yea, actually I'm still waiting on the other thing. :)
16:39 < CodeShark> too late - now you've committed yourself to this as well!
16:39 < CodeShark> slick move, gmaxwell :)
16:39 < BlueMatt> gmaxwell: i think i saw someone else starting to implement that
16:40 < gmaxwell> BlueMatt: This is just the "bitcoinj sends unsolicitied full transactions, with no inv" ... which by itself isn't that awful, but as a result of tolerating it there are clowns now relaying every transaction they get this way.
16:40 < BlueMatt> but i havent looked closely
16:40 < BlueMatt> yea, ok
16:40 < BlueMatt> i mean we should really just disconnect people who relay something we've already seen
16:40 < BlueMatt> dont ban them, but disconnect them
16:40 < BlueMatt> if someone implements this in a broken way, they'll notice a ton of disconnects and stop
16:41 < gmaxwell> Hm, that we already have and didn't request from them? hm. that should actually be okay.
16:41 < BlueMatt> if its a wallet like bitcoinj doing it, the disconnects will almost never happen
16:41 < gmaxwell> (won't actually rescue blocksonly nodes from the noise, but thats fine)
16:42 < BlueMatt> blockonly nodes can also disconnect
16:42 < gmaxwell> I mean they'll never already have.
16:42 -!- tulip [~tulip@unaffiliated/tulip] has joined #bitcoin-core-dev
16:43 < BlueMatt> yes, but they can also disconnect
16:44 < davec> Well depending on the relay speed, that could cause BitcoinJ to get disconnected when the node they are relaying to saw it from elsewhere first (though given the time to process and re-relay, that should be exceedingly rare)
16:45 < davec> assuming the new tx is relayed to all connected nodes immediately that is
16:46 < CodeShark> race conditions are bad ;)
16:46 < gmaxwell> well getting disconnected is something that happens which you must be able to handle.
16:46 < gmaxwell> still best to get bitcoinj fixed.
16:46 < davec> I'm definitely in favor of getting BitcoinJ fixed and then tightening the protocol
16:47 < davec> I think it should be allowed for whitelisted nodes however
16:47 < davec> if you have a relay node, proxy, etc setup locally, requiring the extra round trip is probably not desirable
16:48 < gmaxwell> Well, it's probably irrelevant -- if your transactions are latency critical you're doing something wrong. But sure, I don't see a reason whitelisted hosts couldn't be excempted from that.
16:48 < gmaxwell> though since a peer doesn't know its whitelisted they shouldn't rely on that behavior. 
16:48 < gmaxwell> main argument for it I see is enabling very simplistic scripts.
16:49 < CodeShark> yes, exempt whitelisted nodes...but in general, it would be best to send invs first if sending to multiple nodes
16:49 < davec> Right, I was thinking more for specialized nodes (like relay nodes or proxies, which know they're only talking to a local node)
16:50 < davec> of course, being local on what's probably a 1 or 10Gb link I doubt it would really make any difference anyways
16:51 < CodeShark> latency isn't really the issue with that, as gmaxwell said - but it simplifies the propagation logic
16:52 < CodeShark> disconnecting whitelisted nodes is probably a bad idea anyhow, though, since such incidents are more likely to be due to bugs or errors rather than DoS
16:52 < davec> agreed
16:52 < CodeShark> and you're not spamming anyone else
16:53 < CodeShark> although perhaps we could have a "strict mode" for testing clients
16:53 < CodeShark> use it for testing, not for production
16:54 < gmaxwell> I'd like that but am not sure that it's worth the code overhead unless it was just a milepost on the way to enforcing those things for everyone.
16:55 < gmaxwell> At some point just starting on a new, parallel, P2P protocol would be a better investment of time-- and of course that could be maximally strict from the start.
16:55 < CodeShark> yeah, you're probably right
16:56 < gmaxwell> doing so would allow replacing the inv process with something that actually scales, and so on.
16:57 < davec> while dreaming it would be nice to design it so things like I2P can work too
17:00 < gmaxwell> not just I2P, tor's next HS protocol has larger addresses.
17:01 < gmaxwell> Though those would be not so hard to shoehorn into ye'ole p2p compatbily.
17:01 < davec> yeah I think there was a patch running around for it at some point
17:06 < CodeShark> is it feasible (or even desirable) to move entirely to a tx propagation protocol where txs can be directly routed to miners and recipients rather than flooded?
17:07 < gmaxwell> I don't think it's particularly desirable.
17:07 < CodeShark> reasons?
17:08 < gmaxwell> makes mining more vulnerable to policy risk, dos attack etc. Also forwarding ahead reduces probagation times tremendously (with no caching at all it takes seconds to verify blocks now)
17:09 < gmaxwell> What I'd like to try doing is to do periodic set reconcillation of the top N mbytes of the mempool, and have that be the only relay mechenism in a p2p protocol.
17:09 < gmaxwell> This can get efficiency beyond the peers*transactions bound, and has nice anti-DOS properties.
17:11 < CodeShark> assuming blocks are flooded, all transactions that confirm must get flooded sooner or later...but things like RBF mean that many transactions might get flooded that never confirm
17:11 < CodeShark> whereas routing directly to miners (assuming it is practical) might allow direct fee bidding
17:11 < gmaxwell> with that only things likely to get mined soon relay, naturally. ... and it means that replacements that happen between the batching intervals get nicely suppressed. You can scale your bandwidth by scaling how deep and often you reconcile.
17:12 < tulip> it's effectively required that nodes in the network see transactions before they see them in blocks, 'blocksonly' with 'dbcache=100' frequently sees 20 second or higher times for block verification, I don't think there's any way around that. 
17:12 < gmaxwell> CodeShark: in any case, I think you can take what you're thinking and relax a bit to public relay hubs and then we'd agree.
17:13 < gmaxwell> but as tulip notes, you don't want any mempoolless nodes in your block propagation critical path.
17:14 < gmaxwell> I want to get initial announcement of the p2p network in any case; because of privacy problems and the network attacks they're encouraging.
17:14 < gmaxwell> though I consider that largely orthorgonal to general relay changes.
17:15 < gmaxwell> tulip: I'm surprised it's that bad. :( might this be on a VPS with poor IO?
17:17 < tulip> gmaxwell: it's a lot better with a very large dbcache set (more like a normal node, though obviously still slower). that's on a 4+4HT core i7, 7200RPM 2.5" HDD. 
17:19 < CodeShark> and disabling signature validation doesn't significantly reduce the time,  right?
17:19 < gmaxwell> well default dbcache is 100.  but okay, yea so signature checks won't have that big an effect (esp not since libsecp256k1; which I assume you're using if you're using blocksonly)
17:19 < tulip> CodeShark: is there a patch around for doing that? 
17:20 < CodeShark> it's a one line change in the code, I believe
17:20 < CodeShark> trying to remember which line :)
17:20 < tulip> assuming I can just short circuit the checkpoints line. 
17:21 < CodeShark> yeah, I think that's it
17:21 < gmaxwell> yes.
17:23 < tulip> (log snippet from blocksonly dbcache=100 http://pastebin.com/raw.php?i=JGSHbjwW ) 
17:26 < gmaxwell> if you've got bench on (I guess you do) you can see where the time is going.
17:26 < gmaxwell> The numbers in brackets are cumulative for the whole run.
17:29 < tulip> yep. http://pastebin.com/raw.php?i=QCkcLBNm
17:30 < davec> I'm with gmaxwell here on the announcement bit.  One of the things I've measured as a part of the multi-peer work is how much data transmission is avoided by keeping track of inflight data and avoiding re-requesting it and such.  It's roughly been on the order of 30-100MB/hr (depending on number of txns, connected nodes, etc)
17:30 < gmaxwell> wow, all the time in verify. interesting!
17:34 < gmaxwell> hm. I thought we had more resolution in verify. I can't distinguish signature checks from txin lookups. darn.
17:36 < tulip> guess more granularity wouldn't hurt. 
17:37 < davec> that said, the idea of set reconciliation is pretty attractive and would save a lot more
18:04 < midnightmagic> ooookay, primary node validates itself just fine.
18:27 < GitHub188> [bitcoin] CodeShark opened pull request #7074: Added a command line option -scriptchecks (master...disable_script_checks) https://github.com/bitcoin/bitcoin/pull/7074
18:35 < GitHub106> [bitcoin] CodeShark closed pull request #7074: Added a command line option -checkscripts (master...disable_script_checks) https://github.com/bitcoin/bitcoin/pull/7074
18:35 < gmaxwell> CodeShark: hah sorry!
18:36 < CodeShark> ok, what about if disabling script checks also disables relay?
18:36 < CodeShark> :)
18:37 < CodeShark> or disables something else that makes it unusable as an actual node but can still be used for benchmarking
18:38 < tulip> anecdotally there's some configurations out there people have copied onto their nodes with no real idea of what they do beyond being "suggested", one of them has gen=1 in it, another has par=1 (which is non-obviously named). things like that seem to get entrenched pretty easily. 
18:38 < gmaxwell> CodeShark: I think people will just bypass those other things. It's a bad route in general. 
18:40 < CodeShark> but a strictly benchmarking mode would mean people would need to recompile to bypass it
18:40 < CodeShark> and that's already the case anyhow :)
18:40 < gmaxwell> ya, but thats effectively what we have: you just add a comment in one line.
18:41 < CodeShark> yes, but a runtime switch that allows benchmarking while making the node useless as a node seems...useful :)
18:41 < gmaxwell> Though we don't promise that it won't dangerously break things, and in fact it does-- if someone wanted to do that to their mining op to 'make it faster' they'd end up mining soft-fork ops. We already had an issue with that this year, with 1% of the hashrate running of an rpi configured thusly. :(
18:42 < CodeShark> disable GBT, disable the wallet, disable all relay
18:42 < gmaxwell> CodeShark: then we'll get another "forced fee disabled fork" promoted by some idiot.
18:42 < gmaxwell> I really don't think this is that useful. Also if all that stuff is off then it's more you can't benchmark!
18:43 < tulip> I'd like to break out the script portions of debug=bench. 
18:43 < gmaxwell> if we just want to know the time contribution; then thats addressable via making the timing more granular.
18:43 < gmaxwell> which we really should do.
18:44 < tulip> is there any reason debug=bench also dumps the peer timestamp offsets? I thought that would be in debug=net. 
18:44 < gmaxwell> tulip: so go try? 
18:44 < gmaxwell> tulip: no clue, sounds like an error, go look for the commit that did it?
18:44 < tulip> am. 
18:44 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-wlatfwewhaqzrxjb] has quit [Quit: Connection closed for inactivity]
18:48 < tulip> looks like the timezone offset thing gets printed for everybody (not specific to bench), my mistake. 
18:50 < gmaxwell> I /thought/ it did. But trusted the guy with the code open. :)
18:51 < tulip> I've not run in non-bench mode for so long I've forgotten what log prints are normal. 
18:52 < gmaxwell> I ran without debugging options recently and I was super pleasntly surprised how much we've cut down the chattyness.
18:53 < tulip> yes, it's a lot easier to glance at a log and get a good idea of what's happening. 
19:01 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
19:03 -!- JackH [~Jack@host-80-43-142-236.as13285.net] has quit [Ping timeout: 265 seconds]
19:04 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Quit: .]
19:06 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
19:08 -!- d_t [~textual@c-50-136-139-144.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
--- Log closed Sat Nov 21 19:39:48 2015
--- Log opened Sat Nov 21 19:45:26 2015
19:45 -!- kanzure_ [~kanzure@unaffiliated/kanzure] has joined #bitcoin-core-dev
19:45 -!- Irssi: #bitcoin-core-dev: Total of 73 nicks [0 ops, 0 halfops, 0 voices, 73 normal]
19:45 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has joined #bitcoin-core-dev
19:46 -!- Netsplit *.net <-> *.split quits: jgarzik, crescendo, Guest1235, moli, nkuttler, nanotube, petertodd, isis, Anduck, sdaftuar,  (+3 more, use /NETSPLIT to show all of them)
19:46 -!- Apocalyptic [~Apocalypt@unaffiliated/apocalyptic] has joined #bitcoin-core-dev
19:47 -!- wumpus [~quassel@pdpc/supporter/professional/wumpus] has joined #bitcoin-core-dev
19:50 -!- go1111111 [~go1111111@104.200.154.92] has joined #bitcoin-core-dev
19:51 -!- challisto [~challisto@unaffiliated/challisto] has joined #bitcoin-core-dev
19:52 -!- wangchun [~wangchun@li414-193.members.linode.com] has quit [Ping timeout: 250 seconds]
19:52 -!- zxzzt [~prod@static-100-38-11-146.nycmny.fios.verizon.net] has quit [Ping timeout: 250 seconds]
19:52 -!- warren [~warren@fedora/wombat/warren] has quit [Ping timeout: 250 seconds]
19:53 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has quit [Ping timeout: 250 seconds]
19:54 -!- Irssi: Join to #bitcoin-core-dev was synced in 559 secs
19:56 -!- bsm117532 [~mcelrath@static-108-21-236-13.nycmny.fios.verizon.net] has joined #bitcoin-core-dev
19:56 -!- Guest1234 [~ubuntu@ec2-52-0-91-57.compute-1.amazonaws.com] has joined #bitcoin-core-dev
19:56 -!- Netsplit over, joins: crescendo, nanotube, petertodd, Anduck, isis
19:56 -!- zxzzt [~prod@static-100-38-11-146.nycmny.fios.verizon.net] has joined #bitcoin-core-dev
19:56 -!- Netsplit over, joins: moli, jgarzik, murr4y, pigeons, nkuttler
19:56 -!- andytoshi [~andytoshi@wpsoftware.net] has joined #bitcoin-core-dev
19:58 -!- warren [~warren@fedora/wombat/warren] has joined #bitcoin-core-dev
20:07 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has joined #bitcoin-core-dev
20:08 -!- wangchun [~wangchun@li414-193.members.linode.com] has joined #bitcoin-core-dev
20:38 -!- jonasschnelli [~jonasschn@2a01:4f8:200:7025::2] has joined #bitcoin-core-dev
21:19 < phantomcircuit> gmaxwell, every time i've gone to fix the trickle logic i end up going down the rabbit hole
21:21 < gmaxwell> phantomcircuit: when that happens it can be useful to look to see what the smallest possible change that makes it clearly better is.. rather than trying to redesign it.
21:24 < gmaxwell> like ... non-trickle to a fixed number of peers rather than a fixed proportion.
21:30 < gmaxwell> (and perhaps favor outbound peers)
21:33 < phantomcircuit> gmaxwell, there's trickle logic for the local peers address as well
21:36 < phantomcircuit> gmaxwell, it's really not about the minimal change, im not sure that the original logic really worked either tbh
21:41 < phantomcircuit> oh and there's a race in the inv stuff 
21:42 < phantomcircuit> you cant just decide to disconnect from peers who give you a transaction w/o inv that you've seen before
21:42 < gmaxwell> phantomcircuit: whats the race? if you get a transaction and you didn't getdata. They're naughty.
21:43 < phantomcircuit> gmaxwell, yes but if we dont want to disconnect broken spv clients doing the same thing
21:43 < phantomcircuit> then there's a race
21:44 < gmaxwell> sure we can't do it yet.
21:45 < gmaxwell> I suggest step (1) we ask bitcoinj to change, step (2) we do what matt suggests which is disconnect and don't ban on unsolicited tx when we've already have the tx (usually not the case for SPV)
21:45 < gmaxwell> (2) will usually not punt spv, but if it does they can just reconnect. And if will reliably hang up on the really dumb nodes.
21:46 < phantomcircuit> gmaxwell, ok that's reasonable
21:47 < gmaxwell> and (2) is enough to stop people from making more software that behaves like this.
21:48 < phantomcircuit> gmaxwell, i actually suggest as the first step to fixing the trickle logic... to delete the trickle logic
21:48 < gmaxwell> oh come on now, it's not that broken.
21:49 < phantomcircuit> then mark specific peers as "instant relay" peers
21:50 < phantomcircuit> gmaxwell, there's two goals with the trickle logic
21:50 < gmaxwell> phantomcircuit: well if you're going that route, rather-- get rid of the idea of instant relay. And just trigger relay more often on some peers.
21:51 < phantomcircuit> first is to prevent fingerprinting attacks that allow mapping the network
21:51 < gmaxwell> and to try to stop duplicate data crossing in flight.
21:51 < phantomcircuit> second is to prevent fingerprinting attacks on the wallet
21:51 < gmaxwell> then three goals.
21:51 < gmaxwell> oh you mean addr trivle vs tx.
21:52 < phantomcircuit> sort of but not exactly
21:52 < phantomcircuit> those two should definitely be strongly separated in the code as they have similar but different goals
21:53 < gmaxwell> the mapping should be fixed by peer rotation resulting in little to no static topology.
21:53 < phantomcircuit> today if you connect to all the nodes on the network you can get a pretty good idea about who is connected to who based on the timing of transaction inv's
21:54 < phantomcircuit> none of the peer rotation suggestions have been completely dynamic
21:54 < gmaxwell> well, little, because we want the robustness to short lived eclipse attacks that comes from long lived connections.
21:54 < gmaxwell> right
21:54 < phantomcircuit> they all include some fixed set of nodes to make partitioning harder
21:54 < phantomcircuit> those links would still be vulnerable to fingerprinting attacks
21:56 < tulip> I don't think you're going to be able to get rid of the timing aspect unless you remove listening nodes altogether. 
21:57 < gmaxwell> well if you effectively firewallet the results of communication between your long lived links and your short lived ones. I dunno if it can be done without overhead.
21:58 < gmaxwell> e.g. if you effectively run two p2p stacks, one for the long lived connections one for everyone else.. and don't leak data between them except in hugely delayed batches. yadda yadda.
22:01 < phantomcircuit> gmaxwell, the last time i did this i ended up basically writing a very shitty mixmaster thing
22:01 < phantomcircuit> it would hold onto all the transactions it received for 1 minute and then send them all out at once to everybody
22:02 < phantomcircuit> but wlel all invs cross in flight now
22:57 < GitHub191> [bitcoin] tulip0 opened pull request #7075: Move time data log print to 'net' category to reduce noise (master...no-time-offset-logging) https://github.com/bitcoin/bitcoin/pull/7075
23:03 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has quit [Ping timeout: 272 seconds]
23:04 < gmaxwell> phantomcircuit: they don't all cross in flight.. but a lot do.
23:06 < phantomcircuit> gmaxwell, assuming most clocks are on ntp and that they're +-1 second
23:06 < phantomcircuit> then they do mostly cross with that kind of simple mixing
23:07 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has joined #bitcoin-core-dev
23:08 < tulip> phantomcircuit: most bitcoin nodes don't seem to have very accurate time.  
23:08 < phantomcircuit> tulip, well... i'd rather not write something that assumes that as it's starting point :P
23:15 < tulip> -95  -84  -22  -14  -5  -4  -3  -3  -2  -2  -1  -1  -1  +0  +0  +0  +15
23:15 < tulip> -153  -83  -7  -5  -4  -3  -3  -3  -2  -2  -1  -1  -1  -1  -1  -1  -1  -1  +0  +0  +0  +0  +0  +0  +0  +0  +0  +0  +0  +1  +1  +2  +4  +4  +7  +9  +13  +13  +31
23:20 < tulip> there's something beautiful about a decentralised consensus running on hardware with less time accuracy than a sundial. 
23:41 -!- tulip [~tulip@unaffiliated/tulip] has quit []