--- Day changed Tue Nov 10 2015 00:11 < jonasschnelli> PRab: would be nice if you could test the v0.11.2rc1 on real-windows (non VM). 00:12 < jonasschnelli> I did some VMWare power-off simulations... it did mess up the db even with the fix. But this particular fix is better tested in a non-vm environment. 00:17 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 00:18 -!- Thireus [~Thireus@icy.thireus.fr] has joined #bitcoin-core-dev 00:28 -!- fanquake [~Adium@unaffiliated/fanquake] has joined #bitcoin-core-dev 00:31 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection] 00:44 -!- fanquake [~Adium@unaffiliated/fanquake] has quit [Quit: Leaving.] 00:48 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has quit [Quit: Leaving.] 00:53 -!- trippysalmon [~Rob@ip4da81ded.direct-adsl.nl] has joined #bitcoin-core-dev 00:57 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 01:17 -!- CodeShark [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has joined #bitcoin-core-dev 01:22 < wumpus> I tested it on a real windows laptop (with NotMyFault to inject kernel faults) and wasn't able to get any corruption with the new syncing behavior, while the old behavior was to corrupt every single time 01:22 < wumpus> so even if not perfect it's a lot better 01:58 -!- rubensayshi [~ruben@91.206.81.13] has joined #bitcoin-core-dev 02:03 < wumpus> gitian is broken for me :( "Could not download some packages, please run gbuild --upgrade" when trying to sign the mac package 02:03 < dcousens> jonasschnelli: I've had similar nodes get 'stuck' before, so unless its repeatable, I'm not sure that is related to that PR 02:03 < wumpus> nothing wrong in install.log 02:04 < jonasschnelli> dcousens: right. I wrote that in a comment. Probably unrelated to the secp256k1 switch PR 02:04 < jonasschnelli> But a serious bug,... 02:04 < wumpus> faketime is already the newest version. libc6:i386 is already the newest version. 0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded. 02:04 < dcousens> jonasschnelli: I know, just providing an anecdote that might further that 02:04 < dcousens> further support* 02:05 < wumpus> jonasschnelli: I had a similar issue a while back of a node that just stopped catching up with the chain - never been able to repeat it 02:05 < dcousens> also, you wrote F.I.Y, for interested yours? :P 02:05 < jonasschnelli> yeah... it seams to be hard to create clear steps to reproduce... 02:05 < wumpus> (and I didn't build with debug symbols at the time, so was unable to do anything useful to troubleshoot internal state) 02:05 < jonasschnelli> s/F.I.Y/F.Y.I... :) 02:06 < wumpus> if it happens again I'm prepared. But it never happened again 02:06 < dcousens> wumpus: I had it a week ago 02:06 < jonasschnelli> wumpus: while(1) { IBD() } ... 02:06 < wumpus> here: https://github.com/bitcoin/bitcoin/issues/6188 02:07 < wumpus> well it didn't happen during IBD in my case, just with a node that was synced but suddenly stopped without rejecting the chain or anything like that 02:07 < wumpus> (and after four days it magically started again) 02:07 < dcousens> That said, in restarting the node, I found the most prominent factor was the peers it was connected to 02:08 < jonasschnelli> true. 02:08 -!- CodeShark [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has quit [Ping timeout: 260 seconds] 02:09 < jonasschnelli> the strange thing is, as you can see here (https://bitcoin.jonasschnelli.ch/secp/stuck_debug.log), no block was rejected. 02:09 < wumpus> that wasn't the problem in my case. it was a node with many incoming connections, and I did try dropping the network to get new connections 02:09 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 02:09 < wumpus> it is likely some race condition, where it forgets about being insistent about requesting the next block it needs 02:10 < wumpus> likely caused by one or more uncooprorative peers 02:15 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has joined #bitcoin-core-dev 02:20 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Read error: Connection reset by peer] 02:21 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 02:23 < wumpus> re: the gitian issue, adding --upgrade to my gbuild line seems to have solved it. I don't understand why this is suddenly needed, but ok, great :) 03:07 -!- ParadoxSpiral [~ParadoxSp@p508B9112.dip0.t-ipconnect.de] has joined #bitcoin-core-dev 03:48 -!- MarcoFalke [c3523fc6@gateway/web/cgi-irc/kiwiirc.com/ip.195.82.63.198] has joined #bitcoin-core-dev 04:43 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has quit [Ping timeout: 246 seconds] 04:44 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has joined #bitcoin-core-dev 04:50 -!- Naphex [~naphex@xotika.tv] has joined #bitcoin-core-dev 04:50 < GitHub175> [bitcoin] jonasschnelli opened pull request #6979: [Qt] simple mempool info in debug window (master...2015/11/qt_mempool_easyinfo) https://github.com/bitcoin/bitcoin/pull/6979 04:58 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has quit [Excess Flood] 05:00 -!- jonasschnelli [~jonasschn@unaffiliated/jonasschnelli] has joined #bitcoin-core-dev 05:04 -!- MarcoFalke [c3523fc6@gateway/web/cgi-irc/kiwiirc.com/ip.195.82.63.198] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client] 05:14 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Read error: Connection reset by peer] 05:14 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 05:26 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Read error: Connection reset by peer] 05:26 -!- GAit [~GAit@2.230.161.158] has joined #bitcoin-core-dev 05:44 < dcousens> jonasschnelli: what block did you pause on OOI? 05:45 < jonasschnelli> dcousens: OOI? 05:45 < dcousens> out of interest 05:45 < jonasschnelli> dcousens: 00000000000000001043ada919c3851e17876deca67acc19f365fab4a79bd59d, 382748 05:45 < jonasschnelli> dcousens: check: https://bitcoin.jonasschnelli.ch/secp/stuck_debug.log, search after last UpdateTip 05:46 < GitHub50> [bitcoin] laanwj pushed 2 new commits to 0.11: https://github.com/bitcoin/bitcoin/compare/3dcb390fe9e2...7e278929df53 05:46 < GitHub50> bitcoin/0.11 ab6ff12 David A. Harding: [doc] 0.11.2 release notes: use original pull numbers... 05:46 < GitHub175> [bitcoin] laanwj closed pull request #6975: [doc] 0.11.2 release notes: use original pull numbers (0.11...note-0.11.2-orig-prs) https://github.com/bitcoin/bitcoin/pull/6975 05:46 < GitHub50> bitcoin/0.11 7e27892 Wladimir J. van der Laan: Merge pull request #6975... 05:47 < dcousens> jonasschnelli: interesting 05:47 < dcousens> I stopped around 362250 05:48 < dcousens> But wumpus' post was around 370k 05:48 < jonasschnelli> dcousens: do you see a relation between 362250 and 382748? 05:49 < dcousens> jonasschnelli: i haven't checked, the only relation I have so far is that my block parser has exploded around those blocks recently 05:49 < dcousens> and I haven't yet debugged it, could be completely unrelated 05:49 < jonasschnelli> hmm... yes. Would be nice if we could track down that problem. 05:50 < dcousens> anyway, late here, will have a closer look in the morrow :) 05:50 < dcousens> night 05:52 < wumpus> 0.10.4rc1 executables live https://bitcoin.org/bin/bitcoin-core-0.10.4/test/ 05:55 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 250 seconds] 06:05 -!- zooko [~user@2601:281:8001:26aa:3c89:60d9:cbe:edd1] has joined #bitcoin-core-dev 06:12 < jonasschnelli> wumpus : thanks for the reminder on #6900, just started a gitian build with the new descriptor... 06:12 < jonasschnelli> hmm... i need the base images first i assume 06:25 -!- fanquake [~Adium@unaffiliated/fanquake] has joined #bitcoin-core-dev 06:25 < fanquake> wumpus building the gitian pull now 06:32 < GitHub179> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/503ff6e1ae69...77beab70deae 06:32 < GitHub179> bitcoin/master 87cbdb8 Jorge Timón: Globals: Explicit Consensus::Params arg for main:... 06:32 < GitHub179> bitcoin/master 77beab7 Wladimir J. van der Laan: Merge pull request #6163... 06:32 < GitHub104> [bitcoin] laanwj closed pull request #6163: Chainparams: Explicit Consensus::Params arg for almost all remaining functions (master...chainparams-remaining-consensus) https://github.com/bitcoin/bitcoin/pull/6163 06:32 < GitHub170> [bitcoin] laanwj closed pull request #6248: Fix Qt build on arch by setting -fPIC (master...archbuild) https://github.com/bitcoin/bitcoin/pull/6248 06:35 -!- ParadoxSpiral_ [~ParadoxSp@p508B9D94.dip0.t-ipconnect.de] has joined #bitcoin-core-dev 06:38 -!- ParadoxSpiral [~ParadoxSp@p508B9112.dip0.t-ipconnect.de] has quit [Ping timeout: 240 seconds] 06:38 -!- zooko [~user@2601:281:8001:26aa:3c89:60d9:cbe:edd1] has quit [Ping timeout: 240 seconds] 06:48 < GitHub35> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/77beab70deae...755b4ba848bc 06:48 < GitHub35> bitcoin/master fd55571 Luke Dashjr: wallet: Expose GUI labels in RPC 06:48 < GitHub35> bitcoin/master 755b4ba Wladimir J. van der Laan: Merge pull request #5574... 06:48 < GitHub43> [bitcoin] laanwj closed pull request #5574: Expose GUI labels in RPC as comments (master...rpc_labels) https://github.com/bitcoin/bitcoin/pull/5574 06:51 < GitHub120> [bitcoin] laanwj closed pull request #6693: Set Windows TCP buffers to 64KB to match OSX and Unix (master...issue_6554) https://github.com/bitcoin/bitcoin/pull/6693 06:58 < GitHub78> [bitcoin] laanwj pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/755b4ba848bc...9fa54a1b0c1a 06:58 < GitHub78> bitcoin/master 5f46a7d MarcoFalke: transaction_tests: Be more strict checking dust... 06:58 < GitHub78> bitcoin/master 536766c MarcoFalke: [trivial] New DEFAULT_MIN_RELAY_TX_FEE = 1000 06:58 < GitHub78> bitcoin/master e20d924 MarcoFalke: [trivial] init: Use defaults MIN_RELAY_TX_FEE & TRANSACTION_MAXFEE 06:58 < GitHub65> [bitcoin] laanwj closed pull request #6822: [tests] Be more strict checking dust (master...MarcoFalke-2015-minRelayTxFeeCleanup) https://github.com/bitcoin/bitcoin/pull/6822 06:59 < wumpus> jonasschnelli: you need one new base image, trusty amd64 07:01 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has joined #bitcoin-core-dev 07:08 < fanquake> I'll put up a pull to bump boost, as well as ccache & miniupnpc 07:09 < fanquake> ccache because of an osx compile regression & miniupnpc so we get wumpus's string handling/overflow fixes. 07:10 -!- jgarzik [~jgarzik@unaffiliated/jgarzik] has joined #bitcoin-core-dev 07:11 < wumpus> awesome 07:13 < jonasschnelli> wumpus: did you face similar issues when building the trusy base image:... 07:13 < jonasschnelli> Process (['mkfs.ext4', '-F', '/dev/mapper/loop0p1']) returned 1. stdout: , stderr: mke2fs 07:13 < jonasschnelli> The file /dev/mapper/loop0p1 does not exist and no size was specified. 07:13 < fanquake> jonasschnelli yes 07:13 < jonasschnelli> i have updates gitian to the master git tip 07:13 < jonasschnelli> fanquake: ah. How did you solve that? 07:13 < wumpus> don't remember that, I did succeed in making the image 07:14 < fanquake> jonasschnelli I haven't yet. Got sidetracked looking at depends. 07:15 < jonasschnelli> fanquake: okay. Thanks.. then i go down that rabbit hole... 07:16 < wumpus> just did `bin/make-base-vm --arch amd64 --suite trusty` 07:18 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Read error: Connection reset by peer] 07:26 < jonasschnelli> wumpus: i guess your on ubuntu? On debian i always had issues with gitian... 07:26 < wumpus> right,ubuntu 07:31 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-core-dev 07:32 < GitHub111> [bitcoin] fanquake opened pull request #6980: [Depends] Bump Boost, miniupnpc & ccache (master...depends-bump-boost) https://github.com/bitcoin/bitcoin/pull/6980 07:35 < GitHub99> [bitcoin] laanwj closed pull request #6937: Fix Boost 1.58.0 build for mips arch (master...mips-options-fix) https://github.com/bitcoin/bitcoin/pull/6937 07:37 -!- jtimon [~quassel@74.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev 07:39 < GitHub85> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/9fa54a1b0c1a...32d8b1570cb0 07:39 < GitHub85> bitcoin/master 7ca73dc Jonathan Cross: Improving labels for Sent / Received "Bytes"... 07:39 < GitHub85> bitcoin/master 32d8b15 Wladimir J. van der Laan: Merge pull request #6940... 07:39 < GitHub102> [bitcoin] laanwj closed pull request #6940: Improving labels for Sent / Received "Bytes" (master...patch-1) https://github.com/bitcoin/bitcoin/pull/6940 07:45 < GitHub4> [bitcoin] laanwj pushed 1 new commit to master: https://github.com/bitcoin/bitcoin/commit/b56953e9bb5a32bc35365d1f0c5de5528c0650dd 07:45 < GitHub4> bitcoin/master b56953e Wladimir J. van der Laan: qt: Periodic translations update 07:53 -!- bsm1175321 [~mcelrath@38.121.165.30] has quit [Remote host closed the connection] 08:00 -!- zooko [~user@c-73-217-16-2.hsd1.co.comcast.net] has quit [Ping timeout: 240 seconds] 08:03 -!- fanquake [~Adium@unaffiliated/fanquake] has left #bitcoin-core-dev [] 08:14 -!- Thireus [~Thireus@icy.thireus.fr] has quit [Quit: Leaving.] 08:26 < wumpus> jonasschnelli: any luck building the image? maybe try updating vmbuilder, if you're building it from source 08:30 -!- GAit [~GAit@2.230.161.158] has quit [Quit: Leaving.] 08:34 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Ping timeout: 260 seconds] 08:34 -!- PaulCape_ [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev 08:35 -!- trippysalmon [~Rob@ip4da81ded.direct-adsl.nl] has quit [Read error: Connection reset by peer] 08:43 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has joined #bitcoin-core-dev 08:43 -!- Thireus [~Thireus@37.235.49.227] has joined #bitcoin-core-dev 08:45 < sipa> wumpus, jonasschnelli: where in the codebase are we using powers-of-1024 based units? 08:46 < sipa> bitcoin traditionally uses 1000-based units (feerate is per 1000 bytes, block limit is 1000000 bytes), but I can't say I've payed much attention to it myself 08:46 < jgarzik> sipa, it's all over Qt 08:47 < sipa> if anything, be consistent (don't use MB for 1048576 bytes, and certainly don't use KB - that would mean kelvin bytes) 08:47 < jgarzik> several core uses in default values, usually related to file size 08:52 < wumpus> sipa: I'm not sure. But in this case jonasschnelli tried to introduce a use of 1024*1024 08:53 < sipa> I know 08:53 < wumpus> which was sensibly commented on 08:54 < sipa> I think using MB = 1024^2 is a no-go. My question is whether or not we should aim for only MB = 1000000 or only MiB = 1024^2 08:54 < wumpus> there's probably a few other cases but in general the idea is to use 1000000/MB 08:55 < wumpus> only 1000 *unless* there is a convincing reason to use 1048576/MiB, which I'm not sure exists 08:55 -!- rubensayshi [~ruben@91.206.81.13] has quit [Ping timeout: 240 seconds] 08:55 < sipa> We clearly need a --si commandline flag 08:55 < sipa> *ducks* 08:55 < wumpus> e.g. I can understand using MiB when you're selling memory chips which for hardware reasons, only in powers of two 08:56 < wumpus> but for bitcoin core which is a high-level application there shoulod be no reason to not just use SI units 08:56 < sipa> agree 08:56 < wumpus> and it's always been that way, there's not really a reason to discuss or revise this :) 08:57 * wumpus likes Kelvin*Bytes though 08:57 < sipa> I was just asking whether or not we currently have cases where 1024-based units are exposed to users. Consistency with already existing uses would be a reason. 08:58 < sipa> "Our cache is very hot. Many KB." 08:58 < wumpus> hehe 08:59 * wumpus going to add a note about using SI units to the developer notes 08:59 < jgarzik> as noted, we have many cases exposed to users 09:00 < sipa> i think -dbcache may be 1024-based 09:01 < wumpus> should be changed 09:01 < jgarzik> -dblogsize, several Qt UI attributes 09:01 < jgarzik> several defaults are pow2 09:02 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 09:02 < sipa> i'm in favor of making user-exposed amounts consistently 1000-based, if there aren't too many 09:03 < wumpus> yes 09:07 < jgarzik> +1 09:08 < wangchun> Dear core devs, how do you plan to response to the latest actions regarding BIP101? 09:09 < wumpus> not at all 09:10 < sipa> Bitcoin Core will implement any uncontroversial hard fork. 09:11 < jgarzik> wangchun, Scaling Bitcoin Pt 2 is the next step in figuring out uncontroversial hard fork 09:11 -!- skyraider [uid41097@gateway/web/irccloud.com/x-izzrcrsvkufmyinx] has joined #bitcoin-core-dev 09:14 -!- bsm1175321 [~mcelrath@38.121.165.30] has joined #bitcoin-core-dev 09:14 -!- paveljanik [~paveljani@79-98-72-216.sys-data.com] has joined #bitcoin-core-dev 09:14 -!- paveljanik [~paveljani@79-98-72-216.sys-data.com] has quit [Changing host] 09:14 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has joined #bitcoin-core-dev 09:15 -!- PaulCape_ [~PaulCapes@204.28.124.82] has quit [Quit: .] 09:16 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev 09:21 < btcdrak> sipa: I just added some commits which I think address your concerns https://github.com/bitcoin/bitcoin/pull/6312#issuecomment-154652895 09:26 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Read error: Connection reset by peer] 09:26 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 09:34 < GitHub159> [bitcoin] laanwj opened pull request #6981: doc: Add note about SI units to developer notes (master...2015_11_si_units) https://github.com/bitcoin/bitcoin/pull/6981 09:41 -!- molly [~molly@unaffiliated/molly] has quit [Read error: Connection reset by peer] 09:53 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection] 09:57 -!- molly [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 10:19 < GitHub108> [bitcoin] laanwj closed pull request #6965: Benchmark sanity checks and fork checks in ConnectBlock (master...bench) https://github.com/bitcoin/bitcoin/pull/6965 10:19 < GitHub111> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/b56953e9bb5a...de7d4591a7ce 10:19 < GitHub111> bitcoin/master 77f1f59 Matt Corallo: Benchmark sanity checks and fork checks in ConnectBlock 10:19 < GitHub111> bitcoin/master de7d459 Wladimir J. van der Laan: Merge pull request #6965... 10:21 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Remote host closed the connection] 10:31 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-core-dev 10:47 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Quit: Leaving.] 11:01 < GitHub176> [bitcoin] jtimon opened pull request #6982: Fix #6163 (AcceptBlockHeader) (master...fix-6163) https://github.com/bitcoin/bitcoin/pull/6982 11:06 < jonasschnelli> consensus on MB and /1000^2? 11:06 < jgarzik> IMO the consensus was already "move to MiB" before today... 11:06 * jonasschnelli is confused... :/ 11:09 * Luke-Jr dislikes "_iB" notation and would avoid other software enforcing it. 11:09 < Luke-Jr> I don't suppose Qt has a nice localisation method for these? 11:09 < sipa> jgarzik: now i am confused too 11:09 < jgarzik> ok, I meant SI units 11:10 < jgarzik> whatever the abbrevation is 11:10 < Luke-Jr> SI units are 1000-based kB/MB/GB 11:10 < jgarzik> Luke-Jr, agree! 11:10 < jgarzik> AFAIK the consensus is SI units 11:10 < Luke-Jr> I'd prefer configurable at some level (DE level would be nice), but not worth the time if Qt doesn't make it easy. 11:11 < sipa> jgarzik: there are 2 questions? "should we use 1000 or 1024-based units" and "should we use Mi-style prefixes for 1024-based units?" 11:11 < Luke-Jr> Right now, the traffic thing uses 1024-based KB/MB/GB 11:11 < Luke-Jr> (which is what I prefer) 11:12 < sipa> that's the only unacceptable thing for me :) 11:12 < sipa> either 1000000/MB, or 1049576/MiB 11:12 < sipa> with a preference for the first 11:12 < sipa> 1048576 i mean 11:12 < gmaxwell> sipa: really you have a prefered for the first? 11:13 < sipa> yes, though i have a tendency for the aecond 11:13 < jgarzik> 1) 1000-based units, 2) I hate Mi style prefixes -- grew up thinking 1024==KB, 1024*1024==MB, etc. However I think that's the direction of the world... 11:14 < jgarzik> easy enough to ensure we eliminate as many 1024-based units as possible, to minimize Mi prefixes 11:14 < Luke-Jr> jgarzik: the direction of the world is defined by people in the world, such as us to a small degree ;) 11:14 < gmaxwell> Normally _bandwidth_ is measured in megabits which is a 1000 based unit. Transfer regretably is often in 1024 based units. "MB" should never be used for 1024 based because it's unnecessarily confusing; MiB is unambigious. The purpose of the messages is to communicate, not to be pretty. 11:15 < sipa> gmaxwell: i think the world would be a (marginally) better place if everything used 1000-based prefixes, it would make reasoning so much simpler 11:15 < jgarzik> problem - so few know WTF MiB is, out in the world 11:15 * Luke-Jr did not realise megabits were typically 1000-based 11:16 < gmaxwell> Luke-Jr: not just typically but always except when handled by drooling idiots. :) 11:17 < gmaxwell> jgarzik: far more common than you think, I think. But even still, if so then their ignorance is apparent to them and a 10 second search will resolve it. 11:17 < sipa> Luke-Jr: a 100 Mb/s link can do 11.9 MiB/s :) 11:17 -!- zooko [~user@174-16-148-51.hlrn.qwest.net] has joined #bitcoin-core-dev 11:18 < Luke-Jr> Well, in that case, /me suggests 1000-based with an unchecked-by-default checkbox that uses 1024 in GUI without the "i" silliness. <.< 11:19 < gmaxwell> Luke-Jr: please, no freeing options. 11:19 < gmaxwell> er freeking. 11:19 < gmaxwell> It'll pepper the code with conditional logic that will never get tested. 11:19 < jgarzik> nod 11:19 < sipa> agree 11:19 < Luke-Jr> gmaxwell: this is an option that different people want to use. so obviously it would get tested. 11:20 < sipa> nobody will bother 11:20 < sipa> no actual user will bother, rather 11:20 < Luke-Jr> at least I will 11:20 < gmaxwell> Do you even actually use the GUI except for testing? :) 11:20 < Luke-Jr> yes 11:21 < Luke-Jr> the only time I use the RPC is for testing. 11:21 < jgarzik> Similar to -logtimemicros option -- it's basically an option 1-2 people will use, and the rest of the world will be unaware. 11:21 < jgarzik> Should just pick a useful behavior and not make it an option. 11:22 < Luke-Jr> let's just drop all non-English languages too :P 11:22 < sipa> it doesn't hurt to add more digits 11:22 < sipa> and the cases when they're useful is not when you know it in advance 11:22 < jgarzik> I would rather remove -logtimemicros and make it unconditionally default-on. 11:22 < sipa> agree 11:23 < sipa> me too 11:23 < wumpus> I disagree 11:23 < jgarzik> From OS perspective there is no added cost 11:23 < wumpus> seconds are precise enough for logging 11:23 < jgarzik> 1) other userland servers log microseconds, 2) it's a pointless option that will be used by no one - yet we must maintain 11:23 < sipa> i would love to be able to get bug reports "when did X take 10ms?! it should just be changing a variable!" 11:23 < Luke-Jr> let's go with systemd binary logs! /s 11:23 < sipa> nobody will even notice now 11:24 < Luke-Jr> sipa: lol 11:24 < jgarzik> default-off options should be terminated with extreme prejudice. 11:24 < zooko> jgarzik: +1 11:24 < kanzure> replaced with what? 11:24 < jgarzik> (translation: there should be a good argument for keeping and maintaining them, above "it's nice") 11:24 < wumpus> bla, this is useless 11:25 < sipa> wumpus: what is your reason against microsecond logs? 11:25 < kanzure> ah okay. "it's nice" as insufficient. ok. 11:25 < kanzure> you could send all logs through zeromq inside the process, then people can subscribe to logs in different ways using zeromq. 11:25 < wumpus> sipa: too precise timestamps are a possible security issue 11:25 < wumpus> e.g. timing attacks 11:25 < sipa> hmm 11:26 < wumpus> also it makes correlation easier, to breach privacy 11:26 < sipa> i thought we dropped that concern when switching to -logtimestamps default on 11:26 < jgarzik> sipa, yep 11:26 < wumpus> yes, for seconds 11:26 < gmaxwell> I would rather stop logging things that breach privacy. 11:26 < Luke-Jr> as long as sipa hates 1024 KB, and I prefer 1024 KB, there's no way to satisfy both without an option. If you all decide to make it 1000-based only, I can just add an option to Bitcoin LJR if I ever care enough (unlikely) 11:26 < wumpus> microsconds just goes too far, I see no point in that kind of precision 11:26 < sipa> Luke-Jr: kB 11:26 < sipa> :p 11:26 < Luke-Jr> sipa: kB is 1000-based always 11:26 < gmaxwell> We generate a LOT of excess IO via logging. Lets reduce the chattyness, that is a much more robust privacy protection than decreasing timestamp precision. 11:27 < jgarzik> There is no useful privacy argument for seconds, which does not also apply to microseconds. The matter of degrees is tiny. 11:27 < jgarzik> +1 gmaxwell 11:27 < wumpus> god damnit 11:27 < kanzure> timing attacks tho 11:27 < kanzure> oh 11:27 < wumpus> I agree with reducing chattyness as well, but that's a different concern 11:28 < kanzure> right, i think you mean to say "microseconds does not have any useful privacy advantage over seconds". 11:28 < Luke-Jr> it does have a screen-space advantage, but I can't believe we're spending time discussing this. 11:29 < jgarzik> kanzure, no. the argument being made is "microseconds goes too far" and "it makes correlation easier, to breach privacy" 11:29 < wumpus> Luke-Jr: and it's already possible to enable microseconds if you want to use them 11:29 < sipa> Luke-Jr: KB is kelvin byte. I don't understand where your capital K even comes from; IEC uses Ki for 1024 because ki would make it look like a unit rather than a prefix 11:30 < jgarzik> kanzure, no privacy advantage in microseconds is being claimed - just the opposite - though the matter of degree is tiny 11:30 < wumpus> I don't understand this insistence on enabling it by default 11:30 < jgarzik> versus the diagnostic utility and common practice elsewhere 11:30 < kanzure> jgarzik: understood 11:30 < Luke-Jr> KB is the traditional standard notation for 1024 bytes, defined in at least JEDEC standards 11:31 < Luke-Jr> sipa: ^ 11:31 < kanzure> i believe the BFOH approach was "KiBi" instead of KB or kB or KiB. (not serious here) (but i did see this proposed once by a BOFH). 11:31 < sipa> Luke-Jr: interesting :) 11:32 < jgarzik> wumpus, Because merging options that nearly-nobody will use is dumb 11:32 < sipa> wumpus: being able to benchmark things after the fact 11:32 < wumpus> jgarzik: people that want to benchmark things with precision will enable it 11:32 < wumpus> if no one wanted it it wouldn't have been merged at all 11:32 < kanzure> was this cli option or was this build option? 11:32 < sipa> cli 11:32 < jgarzik> c.f. "1-2 users" Most people will not even know about it. 11:32 < jgarzik> cli 11:33 < kanzure> was build option considered? 11:33 < wumpus> why? 11:33 < kanzure> because cli options are disagreeable for good reasons 11:33 < wumpus> what? 11:33 < wumpus> disagreeable for what reason? 11:34 < kanzure> 11:24 < jgarzik> default-off options should be terminated with extreme prejudice. 11:34 < wumpus> and that doesn't apply to build options? 11:34 < jgarzik> build option is even worse - conditional compilation creates code not even built always, but still must be maintained. 11:34 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 11:34 < gmaxwell> wumpus: it's more conditional code which will be inadequately tested in one form or another, especially all the combinations will not be tested. 11:34 < kanzure> hmm okay. 11:34 < wumpus> yeah this is absolutely going the wrong way 11:35 < wumpus> gmaxwell: I agree if it's a complex conditional, but come on, a logging option 11:35 < gmaxwell> E.g. all of us will turn microseconds on; and then no-microseconds + disable wallet will end up crashing; and we won't notice until after a release. ... but yes; in this particular case this argument is not strong. I agree that it's mostly safe. 11:35 < wumpus> but anyhow feel free to remove the option, I'm done discussing this 11:36 < kanzure> what about zeromq as logging transport, then just use whatever temporal resolution at receive time of each message? 11:36 < kanzure> ok nevermind then 11:36 < Luke-Jr> kanzure: ZeroMQ is stupidly unreliable. 11:36 < wumpus> debug logging is *not* meant as application interface 11:36 < kanzure> Luke-Jr: fedpeg.py is just misconfigured :P 11:36 < wumpus> it's just for debugging and troubleshooting 11:36 < wumpus> not for processing by other applications 11:36 < Luke-Jr> kanzure: fedpeg.py is not my only experience with ZeroMQ> 11:37 < wumpus> if you insist you can already use -logtoconsole and pipe it to something 11:38 < gmaxwell> kanzure: I have other expirence with ZeroMQ too. It does not correctly handle non-lossless networks. 11:40 < jgarzik> Makes sense. zmq was built for LANs, with similar use cases to RDMA. 11:40 < Luke-Jr> LANs aren't always lossless (hi wifi) 11:40 -!- berndj [~berndj@azna.co.za] has quit [Quit: ZNC - http://znc.in] 11:40 < morcos> just to be clear, i love and will always use logtimemicros, but i have another annoying question 11:40 < kanzure> so complaint is lack of application-level retries? or tcp delivery guarantees too strongly stated in zeromq docs? 11:40 < sipa> zmq is explicitly lossy 11:40 < sipa> like udp 11:40 < Luke-Jr> also, ZeroMQ 4.0 is incompatible with 4.1 11:41 < sipa> i thought? 11:41 < morcos> wumpus: i have created two new functions EstimateApproxFee and EstimateApproxPriority in 6134 11:41 < morcos> my plan was not to expose them via RPC 11:41 < morcos> because i think over time the fee estimator still has some more significant evolution 11:41 < morcos> and then in the end it might be nice to expose what we think the final interface should be 11:42 < morcos> however sdaftuar suggested i might be able to expose them now and comment they might change or keep them hidden or something 11:42 < morcos> they would primarily be useful in developing tests for the new functionality i think? 11:43 < morcos> in any case, would you like me to keep them unexposed for now? 11:43 -!- Guest8443 [~berndj@azna.co.za] has joined #bitcoin-core-dev 11:44 < jgarzik> morcos, Respectfully submitted, wumpus is the release manager not The DecisionMaker - ask the crowd not the one 11:45 < jgarzik> There's a reason why I pushed Gavin hard for multiple committers when Satoshi disappeared - Gavin was release manager, not Bitcoin Leader 11:45 < morcos> jgarzik: sure, s/wumpus/everyone/ . although i thought i recalled him expressing an opinion about rpc api churn before 11:50 < jgarzik> "release early, release often" :) IMO Expose them. Maybe name the methods "beta.XXXX" to emphasize they are not final. The goal is the communicate to users that churn will be forthcoming, but also expose them for testing and further field development. 11:52 < wumpus> morcos: I don't mind, if it's useful to have them then add them 11:52 < wumpus> morcos: if it is an unstable interface mention it in the help 11:53 < morcos> ok thanks 11:53 < wumpus> morcos: there's less an issue with adding new commands than changing old ones, especially adding arguments to existing calls is messy 11:53 < jgarzik> +1 11:53 -!- zooko [~user@174-16-148-51.hlrn.qwest.net] has quit [Ping timeout: 240 seconds] 11:54 < morcos> wumpus: yes that was my concern with the fact that this one might change or disappear later, but i'll clearly mention it 11:58 -!- d4de [~d4de@41.234.235.169] has joined #bitcoin-core-dev 11:59 < wumpus> morcos: you could decide to not list them in the overview, like invalidateblock/reconsiderblock 12:00 < wumpus> (the "hidden" category) 12:01 -!- skyraider [uid41097@gateway/web/irccloud.com/x-izzrcrsvkufmyinx] has quit [] 12:01 -!- skyraider [uid41097@gateway/web/irccloud.com/x-tkcaiftqhxkeqsdn] has joined #bitcoin-core-dev 12:08 < GitHub60> [bitcoin] laanwj closed pull request #6981: doc: Add note about SI units to developer notes (master...2015_11_si_units) https://github.com/bitcoin/bitcoin/pull/6981 12:11 < morcos> sipa: question re: in memory sizes.. i was thinking of making a quick pull that defaults the dbcache to 2 * maxmempool and the maxsigcachesize to 1/5 * maxmempool. or do you think there is a better way to control total mem usage? 12:11 < morcos> i'd like to test out a couple of different configurations just to make sure there aren't any regressions with a particularly large or particularly small mempool, and i was hoping to have an idea of what a typical large configuration and typical small configuration might look like 12:13 < morcos> if those are the default ratios, then i'd just try out 50M mempool , 300M mempool and 2G mempool (hmm maybe 4G is too much for dbcache in that case?) seems like it woudl be nice to just control 1 number 12:14 < jgarzik> nod - in general users should not need to fiddle N settings just to get a usable configuration 12:14 -!- fkhan [weechat@gateway/vpn/mullvad/x-aopqxjyqxklngfzi] has quit [Ping timeout: 240 seconds] 12:14 < sipa> morcos: where do you 2G and 4G numbers from from? 12:16 < morcos> i didn't get it from anywhere, i was just goign to try out somethign thats pretty big. i think 2G is about as big as mempools got in the recent spam attack, maybe between 2-3... also easily enough for a weeks worth of backlog in txs which we had previously discussed aiming for 12:17 < morcos> mostly i want to see if any of the code that traverses the whole mempool gets a bit slow then or if the resorts from multindex changing get slow. 12:18 < sipa> I think 2G is too much.. 12:18 < morcos> maybe i explained wrong 12:18 < morcos> i was going to leave maxmempool default at 300 12:19 < morcos> and change dbache default to 2x maxmempool and maxsigcachesize to 1/5 12:19 < morcos> and then i was going to test what happens if someone runs a particularly small or big node. 12:20 < morcos> but i think it makes sense as jgarzik says for their other defaults to scale with setting one value (unless they explicitly set them otherwise) 12:20 < morcos> so yeah 2G is huge, but thats sure what i'd do if i was a miner 12:22 < jgarzik> We're well away from miners caring about maximizing fees to an Nth degree... Reliability, orphaning and other factors drive miner conservatism in settings like this... 12:23 < gmaxwell> The benchmarks w/ the libsecp256k1 pull really highlight the impact of increasing the size of the cache on IBD time. 12:24 < morcos> gmaxwell, if you're in IBD, we should steal maxmempool size for dbacache . :) 12:24 < sdaftuar> +1 12:25 < morcos> so the question is what should the defaults for each of these 3 things be. and if you want to turn 1 knob to change to big mem footprint or small, how should that work? 12:25 < jgarzik> Related tech note - if one option sets another option, it becomes order-dependent 12:26 < jgarzik> i.e. set dbcache before mempoolsize, and dbcache gets stomped. 12:26 < jgarzik> Still, I think the base argument holds - users do not want to set N settings to achieve a useful config. 12:26 < morcos> jgarzik, sure if you change the order of lines of code things get messed up 12:26 < jgarzik> morcos, no, order of configuration file defines 12:27 < morcos> no i dont' think so 12:27 < sipa> ideally there is just a single setting for "amount of cache memory" which is sum of dbcache and mempool, and the flush criteron becomes "too large percentage of the memory is dirty cache entries" 12:27 < sipa> jgarzik: no, the logic for options settings other options is earlier in init.cpp 12:27 < jgarzik> ok, good 12:27 < wumpus> jgarzik: in case of bitcoin core that's noto the case, only still-unset settings get overridden 12:28 < morcos> sipa: so you still need a limit for the mempool right, which makes sense to be a fraction of your total 12:29 < morcos> you cant just go and trim your mempool a bunch when a whole new set of txins get cached 12:29 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 12:29 < wumpus> I don't think utxo cache and mempool are very related 12:30 < morcos> yeah i mostly agree with wumpus there 12:30 < sipa> wumpus: well they're both essentially caches of information that we'll except to need for validating the next block 12:30 < wumpus> enough reasons to increase one and not the other. I'm fine with setting some sane defaults, but they shouldn't be linked in general 12:30 < sipa> *expect 12:30 < sipa> except the utxo cache is *also* a buffer of unwritten changes 12:30 < wumpus> well a smaller mempool doens't make your node slower, a smaller dbcache does 12:30 < morcos> sipa: the mempool doesn't serve as a cache does it 12:30 < morcos> right 12:31 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Client Quit] 12:31 < wumpus> so even though they're 'caches' they have a widely different reason 12:31 -!- fkhan [weechat@gateway/vpn/mullvad/x-nvqlplnrfnptmhrh] has joined #bitcoin-core-dev 12:31 < wumpus> mempool is necessary for correct functioning, dbcache is just for performance 12:31 < morcos> wumpus, the idea of linking though is that why make the user figure out how to divide up his available N gigs of memory 12:32 < jgarzik> User experience. What does the user need to do to achieve a useful config on both large & small boxes? 12:32 < morcos> we should just do soemthing vaguely smart by default 12:32 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 12:32 < jgarzik> nod 12:32 < wumpus> morcos: yes as said by default you could do that, make a -memoryquota parameter or such that automatically allocates them, like automatic partitioning in an OS instlal 12:32 < wumpus> morcos: but it should certainly be possible to set them separately 12:32 < dhill> getrlimit and base initial memory sizes on that? 12:33 < morcos> ok agreed 12:33 < wumpus> dhill: that assumes everyone wants to give all their memory to bitcoind 12:33 < dhill> naw 12:34 < wumpus> and no, just because I have 32GB of memory now doesn't mean I want to allocate it all to my node, and have compiles crash again... 12:34 -!- jgarzik [~jgarzik@unaffiliated/jgarzik] has quit [Quit: This computer has gone to sleep] 12:34 < dhill> i didn't suggest that 12:34 < morcos> so -maxmempool default is some fraction of memoryquota which also has a default. , but then if you individually set your maxmempool,sigcahe,or dbcache, you are not necessarily going to respect your memoryquota 12:34 < sipa> i think it makes sense for bitcoin-qt to do something smartish to guess how much memory to use for caches 12:35 -!- jgarzik [~jgarzik@unaffiliated/jgarzik] has joined #bitcoin-core-dev 12:35 < morcos> well if we do this memory quota idea, then we can always have smart code that sets that as a later addition if we want 12:35 < wumpus> agreed, doing something smart is good, but basing it (trivially) on the available memory does't make too much sense imo 12:36 < wumpus> people generally want to run their node inthe background 12:36 < morcos> so back to my original quesiton if i set memoryquota=N. what should the 3 components each be 12:36 < wumpus> having applications use more memory just because it is available, and no othe reason is...weird 12:36 -!- jgarzik [~jgarzik@unaffiliated/jgarzik] has quit [Client Quit] 12:37 < morcos> .4 N for maxmempol, .5N for dbacache , .1N for maxsigcachesize ? 12:37 < wumpus> I would be angry if firefox used more memory per tab just because I have more, that's so self defeating 12:37 < morcos> sdaftuar was trying to convince me 2 * is too much for dbcache 12:38 < morcos> and default memory quota to 800M which should be around the 300M mempool we had previously been planning for, and keep total usage to under a gig (ugh, i hope, what else uses memory) 12:38 < wumpus> dbcache is nice but apart from during IBD ,not too important 12:39 < morcos> wumpus: i like the idea that block relay over p2p is very fast. i think that depending on the relay network for efficient block relay is a bad idea, even if p2p will never be quite as fast 12:39 < wumpus> I sort of like the idea to use the mempool quota for UTXO cache during IBD, after all the mempool will be almost empty at that time 12:39 < wumpus> though making them linked is less nice for seperation of concerns 12:40 < morcos> for block relay to be fast, you need decent dbcache size 12:43 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Quit: Leaving] 12:43 < wumpus> a better eviction policy would help too there :) 12:44 -!- jgarzik [~jgarzik@rrcs-208-105-49-99.nyc.biz.rr.com] has joined #bitcoin-core-dev 12:44 -!- jgarzik [~jgarzik@rrcs-208-105-49-99.nyc.biz.rr.com] has quit [Changing host] 12:44 -!- jgarzik [~jgarzik@unaffiliated/jgarzik] has joined #bitcoin-core-dev 12:44 < morcos> i have a PR for that. :) but it requires still a decent size dbcache. a 1MB block might need over 50MB of dbcache just for itself, if you knew exactly what was goign to be confirmed in that block 12:46 < wumpus> ok cool 12:47 -!- zooko [~user@174-16-148-51.hlrn.qwest.net] has joined #bitcoin-core-dev 12:48 < jgarzik> "having applications use more memory just because it is available, and no othe reason is...weird" <-- That's pretty much exactly how OS page cache works 12:48 < jgarzik> And I'm looking at how memory usage changes once mempool is on disk, and nearly everything goes through OS page cache 12:49 < wumpus> yes at the OS level you can get away wit hit 12:50 < wumpus> e.g. page cache isn't considered 'used' memory in most computations, it can be evicted when an application really needs it 12:52 < jgarzik> quite true if you s/application/OS/ -- though "working set" is part of the app. If working set size gets below a threshold, performance falls due to increased I/O. As such, it is part of the app, and one that scales when more memory is available. 12:52 < wumpus> you really can't do that in a well-behaving application - yes I know it's possible to use some madvise trick to map application pages as volatile, but supporting that cross platform will be a nightmare, and it will probably still look to the user as if it uses a lot of memory unconditionally 12:55 -!- Guest8443 is now known as berndj 12:57 -!- zooko [~user@174-16-148-51.hlrn.qwest.net] has quit [Read error: Connection reset by peer] 12:58 -!- CodeShark [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has joined #bitcoin-core-dev 12:58 < wumpus> yes I wasn't talking about the page cache, but a hypothetical application that would ask the OS how much memory is available and claim a fixed part of that. It would make the egotistical assumption that a user buys more memory just to give your application more space :) 12:59 < jgarzik> hehe 13:00 -!- zooko [~user@174.16.148.51] has joined #bitcoin-core-dev 13:05 < wumpus> it would be very nice if we could rely on the page cache though for caching the db, this whole utxo cache is mostly a workaround because there is a lot of retrieve overhead even if the data is cached in RAM (do we even know why? deserialization/allocation overhead?) 13:06 < gmaxwell> has nothing to do with storage vs ram, you get basically the same speedups from big utxo cache even if the datadir is in tmpfs. 13:07 < morcos> gmaxwell: oh really?? why is that 13:08 < wumpus> that's what I mean with overhead even if the data is already in RAM 13:09 < wumpus> storing the datadir in tmpfs is an extreme example of that, but normally the page cache would make sure at least a part of the files are still cached 13:10 < wumpus> morcos: that's what I wonder too, probably deserialization/allocation overhead, going from storage representation to internal data structure 13:11 -!- belcher [~user@unaffiliated/belcher] has joined #bitcoin-core-dev 13:12 < gmaxwell> I had a prior belief but pieter tells me my understanding of leveldb's behavior was not correct. (I thought leveldb did a sequential scan of the log to find any records there) 13:12 < wumpus> another possibility is that leveldb queries are really slow even if there is no seek/io overhead - maybe because of the all-pervasive checksum verification 13:13 < sipa> gmaxwell: the log is sequential, but is never read 13:13 < wumpus> anecdotally when I broke off block verification a few times in gdb it always ended up in leveldb checksum verification. That's no serious way to do profiling though :-) 13:13 < sipa> gmaxwell: it's compacted into the database at startup (which is why startup after shutdown with high dbcache is slow, it has large log to process) 13:14 < sipa> the database however has several levels, and several (non-overlapping) files for every level 13:14 < sipa> which means it may need to scan through multiple levels to find data 13:14 < sipa> every file has a bloom filter, so it will quickly skip the ones that don't have the requested amount 13:14 < wumpus> right 13:15 < sipa> within every file, it uses a bisection search to find the data, i think 13:15 < wumpus> yes IIRC too 13:15 < sipa> or a bisection search in its index 13:15 < sipa> which points to the data 13:18 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Quit: Leaving.] 13:18 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 13:26 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Quit: Leaving.] 13:28 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 13:32 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #bitcoin-core-dev 13:34 -!- molly [~molly@unaffiliated/molly] has quit [Quit: //..//..] 13:40 -!- ParadoxSpiral_ [~ParadoxSp@p508B9D94.dip0.t-ipconnect.de] has quit [Remote host closed the connection] 13:56 -!- CodeShark_ [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has joined #bitcoin-core-dev 13:56 -!- CodeShark_ [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has quit [Client Quit] 13:57 -!- CodeShark_ [CodeShark@cpe-76-167-237-202.san.res.rr.com] has joined #bitcoin-core-dev 13:58 -!- CodeShark [~CodeShark@cpe-76-167-237-202.san.res.rr.com] has quit [Ping timeout: 240 seconds] 14:04 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)] 14:32 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Quit: Leaving.] 14:33 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 14:44 -!- CodeShark_ [CodeShark@cpe-76-167-237-202.san.res.rr.com] has quit [] 14:45 -!- CodeShark [CodeShark@cpe-76-167-237-202.san.res.rr.com] has joined #bitcoin-core-dev 14:52 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Quit: Leaving.] 14:53 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #bitcoin-core-dev 14:57 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Client Quit] 15:15 -!- molly [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 15:58 < gmaxwell> Would having a protocol message that says "please don't INV loose transactions to me, I'm mostly only interested in transactions in blocks" be a bad idea? I want to have bandwidth limited nodes that don't care about unconfirmed txn not recieve anything but confirmed transactions. 16:00 < sipa> you could start by not downloading them :) 16:00 < sipa> s/downloading/getdata'ing/ 16:00 < gmaxwell> yea, not getdata is an obvious first step and need no protocol handling. 16:01 < gmaxwell> but INVs actually use a lot of bandwidth... on the order of 5% of the size of all transactions per peer. 16:04 < CodeShark> an extra protocol message rather than negotiating it during the handshake? 16:05 < CodeShark> presumably miners would not want to connect to peers that won't relay unconfirmed transactions 16:05 < gmaxwell> CodeShark: shouldn't be a handshake thing, since you may with to adaptive cork and uncork. 16:06 < gmaxwell> er adaptively* 16:06 < CodeShark> if a node does not intend to accept nor relay unconfirmed transactions, it can't really hurt for others to know this 16:11 < CodeShark> one would hope that of 8 randomly chosen nodes at least one is relaying transactions 16:13 < CodeShark> furthermore, given a more efficient block propagation protocol (i.e. one that does not require resending txs to nodes that already have them in their mempool) the main benefit would be only for transactions that take a very long time to confirm or never get confirmed 16:15 < phantomcircuit> gmaxwell, filterload an empty filter 16:15 < phantomcircuit> tada 16:15 < CodeShark> heh 16:15 < wumpus> but then you won't receive any transactions in blocks either? 16:15 < CodeShark> right 16:15 < phantomcircuit> wumpus, getdata MSG_BLOCK 16:15 < phantomcircuit> isn't filtered at all 16:15 < CodeShark> true 16:16 < phantomcircuit> (i run with empty filters on mining full nodes) 16:16 < wumpus> nice 16:18 < gmaxwell> CodeShark: you wouldn't be listening (or at least not node-network and advertising yourself) while also doing this. 16:19 < gmaxwell> CodeShark: thats assuming spherical cow efficiency, and again; it leaves the user saddled with the inefficienct gossiping of transactions. If you don't care about unconfirmed transactions all that is a waste of time. 16:20 < gmaxwell> beyond the bandwidth efficiency gains, not taking lose transactions removes an attack vector. 16:21 < wumpus> empty filterload should indeed work, the filter is checked before relaying transactions to a node 16:21 < gmaxwell> wumpus: won't that screw up relaying blocks? 16:22 < phantomcircuit> gmaxwell, nope 16:22 < wumpus> that's what I thought too, but phantomcircuit corrected me 16:23 < gmaxwell> phantomcircuit: care to clean up a patch that adds a blocksonly=1 that turns off node network and does the empty filterloads? 16:25 < wumpus> only getdata w/ MSG_FILTERED_BLOCK will have the filter applied 16:25 < gmaxwell> yup. duh. 16:26 < gmaxwell> phantomcircuit: should also check that the neighbor supports bloomfilters and not do it otherwise. And not getdata transactions even if INVed except from whitebind peers, I guess. 16:27 < wumpus> MSG_FILTERED_BLOCK with an empty filter could be handled very quickly, but it doesn't seem that we special case it :-) 16:28 < gmaxwell> hm. I thought I added some special case handling of empty filters before. 16:29 < wumpus> yes you probably want to disconnect from peers that don't support bloom filters 16:29 < wumpus> maybe I missed it 16:30 < wumpus> looks like there is no way to query isEmpty from outside CBloomFilter 16:31 < gmaxwell> inside it there is an isEmpty test that shortcuts contains. 16:31 < gmaxwell> (it was added as the 'cover change' for that divide by zero crash... but it's an actual optimization too) 16:31 -!- belcher [~user@unaffiliated/belcher] has quit [Quit: Leaving] 16:32 < wumpus> what I meant was more something that bypasses *everything* and just returns empty if you query with an empty filter 16:32 -!- zooko [~user@174.16.148.51] has quit [Ping timeout: 276 seconds] 16:32 < wumpus> now it still happily reads the blocks from disk, just to discard every transaction 16:33 < gmaxwell> Would be reasonable to bubble up the full/empty state. 16:33 < phantomcircuit> wumpus, it is special cased 16:33 < gmaxwell> phantomcircuit: in the wrong place. 16:33 < wumpus> phantomcircuit: oh? where? 16:33 < phantomcircuit> it's special cased in the bloom filtering code 16:34 < gmaxwell> Existing special casing inside the bloom object itself was really just done to make that divide by zero bug unreachable. But as wumpus notes it means you'll read the block and test every transaction against the filter, the shortcutting is a bit late. 16:35 < phantomcircuit> yes agreed 16:37 < wumpus> it's interesting how with three people in a low-traffic IRC channel we're stlll managing to talk past each other half the time :-) 16:44 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [] 16:45 < gmaxwell> Following this thinking, might be interesting to support fee-rate filtering for inv. 17:15 -!- zooko [~user@2601:281:8001:26aa:90a:ed6a:1101:a1c0] has joined #bitcoin-core-dev 17:34 < gmaxwell> Bleh, three days of AFLing univalue on 48 cores and it's still finding new paths. 17:35 < gmaxwell> I hate parsers. 17:35 < sipa> haha 17:40 -!- MetaTrading5 [~MetaTradi@198.46.233.75] has joined #bitcoin-core-dev 17:58 -!- lecusemble [~lecusembl@f9beb4d9.violates.me] has quit [Ping timeout: 255 seconds] 18:07 < dcousens> gmaxwell AFLing? 18:07 < dcousens> (guessing some kind of fuzzing) 18:10 -!- lecusemble [~lecusembl@f9beb4d9.violates.me] has joined #bitcoin-core-dev 18:10 < sipa> https://en.wikipedia.org/wiki/American_fuzzy_lop_%28fuzzer%29 18:10 < dcousens> ta 18:14 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-wtmiirdfedouefgi] has quit [Quit: Connection closed for inactivity] 18:14 -!- CodeShark [CodeShark@cpe-76-167-237-202.san.res.rr.com] has quit [Ping timeout: 240 seconds] 18:29 -!- sipa [~pw@2a02:348:86:3011::1] has quit [Changing host] 18:29 -!- sipa [~pw@unaffiliated/sipa1024] has joined #bitcoin-core-dev 18:30 < gmaxwell> can someone kick metatrading5 ? he's PM spamming. 18:30 -!- mode/#bitcoin-core-dev [+o sipa] by ChanServ 18:30 -!- mode/#bitcoin-core-dev [+b *!*MetaTradi@198.46.233.*] by sipa 18:30 -!- MetaTrading5 was kicked from #bitcoin-core-dev by sipa [MetaTrading5] 18:31 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Ping timeout: 250 seconds] 19:07 -!- tulip [~tulip@46.101.245.204] has joined #bitcoin-core-dev 19:17 -!- tulip [~tulip@46.101.245.204] has quit [] 19:19 -!- tulip [~tulip@46.101.245.204] has joined #bitcoin-core-dev 19:43 < phantomcircuit> jtimon, there wouldn't happen to be a library somewhere of all the pure functions in bitcoin core would there? 19:51 -!- tulip [~tulip@46.101.245.204] has quit [] 19:52 -!- tulip [~tulip@46.101.245.204] has joined #bitcoin-core-dev 20:50 -!- tulip [~tulip@46.101.245.204] has quit [] 21:00 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Remote host closed the connection] 21:08 -!- NLNico [~NLNico@unaffiliated/nlnico] has joined #bitcoin-core-dev 21:09 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-core-dev 22:00 < GitHub119> [bitcoin] sipa opened pull request #6983: Update libsecp256k1 (master...secp256k1new) https://github.com/bitcoin/bitcoin/pull/6983 22:29 -!- ParadoxSpiral [~ParadoxSp@p508B9D94.dip0.t-ipconnect.de] has joined #bitcoin-core-dev 22:36 -!- ParadoxSpiral [~ParadoxSp@p508B9D94.dip0.t-ipconnect.de] has quit [Remote host closed the connection] 22:52 -!- skyraider [uid41097@gateway/web/irccloud.com/x-tkcaiftqhxkeqsdn] has quit [Quit: Connection closed for inactivity] 23:03 < warren> Running Bitcoin Core v0.11.2rc1 with blank data dir on Windows 10 ... it connected to 9 peers including a local peer. Stuck at 0 blocks for the past 10 minutes. what should I try? 23:04 <@sipa> warren: does it have any headers? 23:04 <@sipa> warren: what does getchaintips RPC report? 23:05 < warren> ah, it's working now 23:38 < gmaxwell> wumpus: I guess you did more security review on miniupnpc? ... I went to review that update and I wanted to jump off a snprintf bridge. :) 23:42 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Disconnected by services] 23:43 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has quit [Ping timeout: 244 seconds] 23:44 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has quit [Ping timeout: 260 seconds] 23:44 -!- dgenr8 [~dgenr8@unaffiliated/dgenr8] has joined #bitcoin-core-dev 23:44 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has joined #bitcoin-core-dev 23:45 -!- Thireus [~Thireus@37.235.49.227] has quit [Quit: Leaving.] 23:45 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-core-dev 23:48 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-nekjdpoeudabktmv] has joined #bitcoin-core-dev 23:53 -!- NLNico [~NLNico@unaffiliated/nlnico] has quit [Ping timeout: 264 seconds]