--- Day changed Fri Mar 03 2017 00:05 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 00:05 -!- Joe___ [dcf025f8@gateway/web/freenode/ip.220.240.37.248] has joined #bitcoin-core-dev 00:07 -!- Joe___ [dcf025f8@gateway/web/freenode/ip.220.240.37.248] has quit [Client Quit] 00:14 -!- frabrunelle [frabrunell@safenetwork/frabrunelle] has quit [Remote host closed the connection] 00:14 -!- kewde[m] [kewdematri@gateway/shell/matrix.org/x-sutqxisefckbwwfy] has quit [Write error: Connection reset by peer] 00:30 -!- kewde[m] [kewdematri@gateway/shell/matrix.org/x-qaoczcvfvpcxdgqz] has joined #bitcoin-core-dev 00:42 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has joined #bitcoin-core-dev 00:47 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has quit [Ping timeout: 246 seconds] 00:53 -!- frabrunelle [frabrunell@safenetwork/frabrunelle] has joined #bitcoin-core-dev 01:05 -!- juscamarena [~justin@47.148.176.74] has quit [Remote host closed the connection] 01:05 -!- juscamarena [~justin@47.148.176.74] has joined #bitcoin-core-dev 01:29 < wumpus> jeremyrubin: well cloudabi executables are marked with a special "OS/ABI" type, so it's easy to identify them as such. Also it's recommended to use cloudabi-run to run them, which should not launch native executables 01:30 < gmaxwell> wumpus: nice blog post btw. 01:30 < wumpus> but yes it'd make sense to add a check to whatever processes submitted executables that it's really the right kind of executable 01:30 < wumpus> gmaxwell: thanks :) 01:31 < gmaxwell> wumpus: how is the kernel security for it implemented? assuming you're using its special libc and whatnot, does it just use seccomp mode 1 on Linux? 01:33 < gmaxwell> ah, it's the Capsicum stuff. 01:33 < wumpus> gmaxwell: the preferred approach is to have it run on top of capsicum, this is what is used on freebsd at least. There's also an emulator, which may use seccomp, not sure. 01:36 < gmaxwell> it's a little sad that dns requests have to be external, since libc vulnerablities in dns handling is one of the things that bothers me more about our common attack surface. E.g. DNSseeds start returning a glibc dns 0day. 01:37 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has joined #bitcoin-core-dev 01:37 < wumpus> well it could be implemented differently 01:37 < wumpus> you could have something that passes in a ready-to-go UDP file descriptor for communicating to your preferred DNS server 01:38 < gmaxwell> oh. hm yea, it's true. 01:38 < wumpus> then use an internal DNS implementation 01:38 < gmaxwell> or even a mini dns resolver process that just uses a non-sandboxed udp shim. 01:38 < wumpus> (e.g. libevent's) 01:39 < wumpus> exactly 01:39 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-zwbsvuiemfdwaiju] has joined #bitcoin-core-dev 01:40 < wumpus> my initial usecase will be to have all outgoing network traffic go through tor, so it won't be doing any DNS lookups locally at all, only at the exit nodes I guess 01:41 < gmaxwell> wumpus: for mlock, would it be possible for an external process to create an anonymous mmap, mlock it, then pass the descriptor over a unix domain socket to the sandboxed process? 01:41 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has quit [Ping timeout: 246 seconds] 01:41 < wumpus> gmaxwell: yes, that's possible 01:42 < jonasschnelli> What I once started is to use sipas crawler (bitcoin-seeder) and fill everything into a db, then use that db from djbdns (which is still one of the best dnsd IMO). 01:42 < wumpus> gmaxwell: it's a bit scary though as the external process will be able to keep a reference, and have all your key data :) 01:42 < gmaxwell> yes, though it could presumably also ptrace your process. and hopefully it wouldn't keep a reference. :) 01:43 < wumpus> ideally we'd not want to store any key data in the process at all; e.g. interface to yet another module for that, which could be hw 01:44 < gmaxwell> yea, indeed, should just be a signer process that has an interface like a hardware wallet. Sometimes it's an actual hardware wallet, other times it's a sandboxed process, SGX module, or whatnot. 01:44 < wumpus> right 01:45 < wumpus> jonasschnelli: to make a DNS server that can only look up bitcoind hosts? 01:45 -!- nemgun [~nemgun@105.101.209.63] has joined #bitcoin-core-dev 01:45 < wumpus> jonasschnelli: oh I get it, of course, you want to run an alternative DNS server as DNS seeder 01:45 < jonasschnelli> wumpus: yes. 01:45 < jonasschnelli> The crawler is great,... but not the dnsd part 01:46 < gmaxwell> well we do use dns for things other than introduction. 01:46 < wumpus> do we? 01:46 < gmaxwell> sure, things like addnode can use it. 01:47 < gmaxwell> and it's not a bad idea there simply because ... IPs change. 01:47 < jonasschnelli> isn't that pure IPv4/v6? 01:47 < jonasschnelli> we could or we can? 01:47 < nemgun> Hello guys 01:48 < wumpus> it's possible to pass some of the connect-to command line options (proxy, tor, etc) as hostnames.... but meh, there's no use for that in cloudabi because it won't be able to connect to those anyway. The right wya to do that would be to pass in a fd to a pipe or such... 01:49 < wumpus> as for RPC you can do the lookups at the client side 01:49 < gmaxwell> wumpus: ah, well I thought it would eventually be good to be able to support the complete feature set (except for some antiquated features like the notify interfaces which might be better done another way). 01:50 < wumpus> well my goal with this experiment would be to make something better, sometimes that means making decisions to drop some functionality or do it some other way 01:51 < wumpus> it'd be possible to emulate the complete old feature set on top of the sandbox, but that's not very interesting 01:55 < wumpus> anyhow if addnode stores a name and does periodic DNS lookups, to track IP changes over time, there's something to be said that bitcoind needs to be able to initiate DNS lookups 01:57 < gmaxwell> it does (well it will relookup again on a reconnect, e.g. after a disconnect under the old name...) 01:57 < wumpus> simplest would be to use the "RESOLVE" [F0] SOCKS5 extension for that. This means the actual lookup can be done from another process (which could be sandboxed too) 01:58 < wumpus> this also means that if the DNS resolver process is compromised, bitcoind itself isn't 02:00 -!- JackH [~laptop@217.149.140.177] has quit [Ping timeout: 264 seconds] 02:02 < wumpus> there's a bit of a bootstrapping problem here, ideally there would be a command 'instantiate a graph of cloudabi processes and connect them such and such' 02:07 < wumpus> btw verify-commits.sh is broken again 02:07 -!- azuchi [~azuchi@shuttle.haw.jp] has quit [Ping timeout: 240 seconds] 02:08 < wumpus> looks like it croaks on MarcoFalke's key 02:09 < jonasschnelli> hmm... 02:19 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 02:23 -!- JackH [~laptop@2a02:a210:681:980:953e:d391:fecc:4d2f] has joined #bitcoin-core-dev 02:31 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has joined #bitcoin-core-dev 02:33 < bitcoin-git> [bitcoin] ryanofsky opened pull request #9908: Define 7200 second timestamp window constant (master...pr/timewin) https://github.com/bitcoin/bitcoin/pull/9908 02:35 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has quit [Ping timeout: 246 seconds] 02:44 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev 02:52 -!- whphhg [~whphhg@unaffiliated/whphhg] has joined #bitcoin-core-dev 03:13 -!- whphhg [~whphhg@unaffiliated/whphhg] has quit [Quit: Leaving] 03:25 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has joined #bitcoin-core-dev 03:26 < bitcoin-git> [bitcoin] ryanofsky opened pull request #9909: Add FindEarliestAtLeast test for edge cases (master...pr/faltest) https://github.com/bitcoin/bitcoin/pull/9909 03:30 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has quit [Ping timeout: 246 seconds] 04:15 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 04:16 < paveljanik> when running with -blocksonly, do we want to load and dump the memory pool? 04:18 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has joined #bitcoin-core-dev 04:19 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has joined #bitcoin-core-dev 04:20 < bitcoin-git> [bitcoin] ian-kelling opened pull request #9910: Docs: correct and elaborate -rpcbind doc (master...docs-rpcbind) https://github.com/bitcoin/bitcoin/pull/9910 04:23 -!- jannes [~jannes@095-097-246-234.static.chello.nl] has joined #bitcoin-core-dev 04:24 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has quit [Ping timeout: 246 seconds] 04:37 < wumpus> paveljanik: does it hurt to do so? if not, better not to couple those options 04:39 < bitcoin-git> [bitcoin] laanwj pushed 10 new commits to master: https://github.com/bitcoin/bitcoin/compare/58861ad91b49...eb281842b7df 04:39 < bitcoin-git> bitcoin/master eca550f Luke Dashjr: RPC/Wallet: Pass CWallet as pointer to helper functions 04:39 < bitcoin-git> bitcoin/master d77ad6d Luke Dashjr: RPC: Do all wallet access through new GetWalletForJSONRPCRequest 04:39 < bitcoin-git> bitcoin/master 2e518e3 Luke Dashjr: Move nWalletUnlockTime to CWallet::nRelockTime, and name timed task unique per CWallet 04:39 < bitcoin-git> [bitcoin] laanwj closed pull request #8775: RPC refactoring: Access wallet using new GetWalletForJSONRPCRequest (master...multiwallet_prefactor_rpc) https://github.com/bitcoin/bitcoin/pull/8775 04:39 -!- CubicEarth [~cubiceart@201.191.198.112] has joined #bitcoin-core-dev 04:42 -!- CubicEarth [~cubiceart@201.191.198.112] has quit [Client Quit] 05:01 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [] 05:03 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 05:04 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has joined #bitcoin-core-dev 05:05 < paveljanik> wumpus, dumping is fast, but loading is very slow and CPU consuming. And it consumes memory if there was a large mempool.dat. But I do not care enough ;-) 05:06 < wumpus> I think there should be an option to bypass mempool loading/writing. And possibly it could parameter-interact with disable-blocks mode, as long as it's clearly documented. 05:06 < wumpus> or maybe an option to disable the mempool completely? I don't know. 05:07 < wumpus> I don't care enough either. It seems optimizing for something that is an incedigly rare edge case 05:07 -!- wasi [~wasi@gateway/tor-sasl/wasi] has quit [Remote host closed the connection] 05:07 < wumpus> if you want to optimize actual bottlenecks, please pick my sha256 or crc32 optimization experiment :-) 05:08 -!- wasi [~wasi@gateway/tor-sasl/wasi] has joined #bitcoin-core-dev 05:08 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has quit [Ping timeout: 246 seconds] 05:12 < paveljanik> Well, I'll first optimise my blocksonly gateway to the world ;-) 05:13 < wumpus> both optimizations that I mention are very relevant to validating blocks, too 05:13 < paveljanik> do you have the PR numbers? ;-) 05:13 -!- nemgun32 [~nemgun@105.101.0.91] has joined #bitcoin-core-dev 05:14 < wumpus> no PRs yet,but you can find the branches linked in the README here: https://github.com/laanwj/bitcoin 05:16 -!- d9b4bef9 [~d9b4bef9@web419.webfaction.com] has quit [Remote host closed the connection] 05:16 -!- nemgun [~nemgun@105.101.209.63] has quit [Ping timeout: 246 seconds] 05:17 -!- d9b4bef9 [~d9b4bef9@web419.webfaction.com] has joined #bitcoin-core-dev 05:29 < paveljanik> how do you satisfy multiarchitecture/multiOS with yasm? 05:37 < wumpus> dunno? 05:38 < wumpus> is yasm platform specific? 05:39 < wumpus> as for architectures, use a general interface, and compile on the assembly appropriate for the architecture. For specific instruction sets it makes sense to do run-time detection. 05:42 < wumpus> on x86 you can use the CPUID instruction for that (see for example the SDL2 implementation: https://github.com/spurious/SDL-mirror/blob/master/src/cpuinfo/SDL_cpuinfo.c), on ARM it's somewhat more difficult as the instruction to detect what instruction sets are available is privileged. So applications resort to OS-specific parsing of /dev/cpuinfo, or just trying out the instructions and catching SIGILL 05:43 < wumpus> openssl, most notably, uses the latter approach, which is why you'lll always have to skip a SIGILL when running bitcoind in gdb on ARM 05:52 -!- ennui [~user@unaffiliated/ennui] has joined #bitcoin-core-dev 05:53 -!- ennui [~user@unaffiliated/ennui] has left #bitcoin-core-dev ["ERC (IRC client for Emacs 24.5.1)"] 05:56 -!- bsm1175322 [~mcelrath@135.84.167.210] has quit [Ping timeout: 256 seconds] 05:57 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/eb281842b7df...ba80a684cfef 05:57 < bitcoin-git> bitcoin/master 90a1d9a Ian Kelling: Docs: add details to -rpcclienttimeout doc 05:57 < bitcoin-git> bitcoin/master ba80a68 Wladimir J. van der Laan: Merge #9903: Docs: add details to -rpcclienttimeout doc... 05:57 < bitcoin-git> [bitcoin] laanwj closed pull request #9903: Docs: add details to -rpcclienttimeout doc (master...docs-client-timeout) https://github.com/bitcoin/bitcoin/pull/9903 05:58 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has joined #bitcoin-core-dev 06:03 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has quit [Ping timeout: 246 seconds] 06:06 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has joined #bitcoin-core-dev 06:11 -!- wudayoda [~goksinen@2604:2000:c591:8400:a461:484c:704:7cc6] has quit [Ping timeout: 246 seconds] 06:11 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt] 06:12 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev 06:22 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has quit [Quit: MarcoFalke] 06:22 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has joined #bitcoin-core-dev 06:27 < jonasschnelli> What's the plan in how we solve the current travis problem in master? 06:27 < jonasschnelli> Anyone working on that? 06:28 < wumpus> I suppose you mean https://github.com/bitcoin/bitcoin/pull/9880#issuecomment-283959063 ? 06:28 < wumpus> I don't really know how to solve it 06:29 < wumpus> do we need to add a new key somewhere? 06:29 < wumpus> would prefer to leave it up to BlueMatt 06:31 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev 06:32 -!- bityogi [~textual@208-104-132-26.brvd.dsl.dyn.comporium.net] has joined #bitcoin-core-dev 06:41 < jonasschnelli> wumpus: https://github.com/bitcoin/bitcoin/issues/9898 isn't solved yet. right? 06:42 < jonasschnelli> Ah. But right, the issue on master is verify-commits. 06:42 < jonasschnelli> I though it was the travis OSX issue 06:43 < wumpus> oh I had forgot about that one 06:43 < wumpus> seems it went away out of itself though? pulls are passing 06:43 < wumpus> it's just master, where verify-commits is run, where things fail 06:44 < jonasschnelli> Okay. I'm happy 9898 is gone... 06:49 < bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/ba80a684cfef...75d012e8c7d0 06:49 < bitcoin-git> bitcoin/master 9de90bb Pavel Janík: Do not shadow variables (gcc set) 06:49 < bitcoin-git> bitcoin/master ad1ae7a Pavel Janík: Check and enable -Wshadow by default. 06:49 < bitcoin-git> bitcoin/master 75d012e Wladimir J. van der Laan: Merge #8808: Do not shadow variables (gcc set)... 06:54 -!- To7 [~theo@cpe-158-222-192-214.nyc.res.rr.com] has quit [Quit: Whatever] 06:57 < jonasschnelli> I think it would be great if #9294 could get a review from on of the bip32 authors (ping sipa, gmaxwell) 06:58 < gribble> https://github.com/bitcoin/bitcoin/issues/9294 | Use internal HD chain for change outputs (hd split) by jonasschnelli · Pull Request #9294 · bitcoin/bitcoin · GitHub 07:09 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/75d012e8c7d0...67c5cc19178d 07:09 < bitcoin-git> bitcoin/master 09fe346 Russell Yanofsky: Avoid -Wshadow warnings in wallet_tests... 07:09 < bitcoin-git> bitcoin/master 67c5cc1 Wladimir J. van der Laan: Merge #9828: Avoid -Wshadow warnings in wallet_tests... 07:09 < bitcoin-git> [bitcoin] laanwj closed pull request #9828: Avoid -Wshadow warnings in wallet_tests (master...pr/multishadow) https://github.com/bitcoin/bitcoin/pull/9828 07:22 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/67c5cc19178d...90cb2a218e58 07:22 < bitcoin-git> bitcoin/master f36bdf0 Johnathan Corgan: Enable host lookups for -proxy and -onion parameters... 07:22 < bitcoin-git> bitcoin/master 90cb2a2 Wladimir J. van der Laan: Merge #9774: Enable host lookups for -proxy and -onion parameters... 07:23 < bitcoin-git> [bitcoin] laanwj closed pull request #9774: Enable host lookups for -proxy and -onion parameters (master...hostname-lookups) https://github.com/bitcoin/bitcoin/pull/9774 07:39 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:2d70:c5e8:9879:e2e2] has joined #bitcoin-core-dev 07:43 -!- BashCo_ [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 07:44 -!- BashCo_ [~BashCo@unaffiliated/bashco] has quit [Remote host closed the connection] 07:45 -!- bsm1175322 [~mcelrath@135.84.167.210] has joined #bitcoin-core-dev 07:46 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 246 seconds] 07:48 < bitcoin-git> [bitcoin] jonasschnelli closed pull request #8764: [Wallet] get rid of pwalletMain, add simple CWallets infrastructure (master...2016/09/wallet_pointer) https://github.com/bitcoin/bitcoin/pull/8764 07:52 -!- dodomojo [~goksinen@2604:2000:c591:8400:c597:103:82a1:f791] has joined #bitcoin-core-dev 07:55 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:2d70:c5e8:9879:e2e2] has quit [Ping timeout: 258 seconds] 08:00 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has quit [Quit: Leaving] 08:01 -!- harrymm [~wayne@104.222.140.129] has quit [Ping timeout: 256 seconds] 08:01 -!- nemgun [~nemgun@105.101.248.132] has joined #bitcoin-core-dev 08:03 -!- nemgun32 [~nemgun@105.101.0.91] has quit [Ping timeout: 246 seconds] 08:16 -!- harrymm [~wayne@104.222.140.65] has joined #bitcoin-core-dev 08:21 -!- arowser [~quassel@106.120.101.38] has quit [Quit: No Ping reply in 180 seconds.] 08:22 -!- arowser [~quassel@106.120.101.38] has joined #bitcoin-core-dev 08:23 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/90cb2a218e58...53c300fb525a 08:23 < bitcoin-git> bitcoin/master e5b449c Ian Kelling: Docs: correct and elaborate -rpcbind doc... 08:23 < bitcoin-git> bitcoin/master 53c300f Wladimir J. van der Laan: Merge #9910: Docs: correct and elaborate -rpcbind doc... 08:23 < bitcoin-git> [bitcoin] laanwj closed pull request #9910: Docs: correct and elaborate -rpcbind doc (master...docs-rpcbind) https://github.com/bitcoin/bitcoin/pull/9910 08:24 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 08:39 -!- wudayoda [~goksinen@2604:2000:c591:8400:45f6:3838:740c:e363] has joined #bitcoin-core-dev 08:47 -!- abpa [~abpa@96-82-80-25-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 09:19 -!- bsm1175322 [~mcelrath@135.84.167.210] has quit [Remote host closed the connection] 09:26 -!- bsm1175322 [~mcelrath@135.84.167.210] has joined #bitcoin-core-dev 09:37 < bitcoin-git> [bitcoin] paveljanik opened pull request #9911: WIP: Wshadow: gcc 4.8.5 fixes (master...20170303_Wshadow_streams) https://github.com/bitcoin/bitcoin/pull/9911 09:38 -!- lightningbot [~supybot@2400:8901::f03c:91ff:febb:bbc1] has quit [Disconnected by services] 09:38 -!- Taek42 [~quassel@2001:41d0:1:472e::] has joined #bitcoin-core-dev 09:39 -!- lightningbot [~supybot@2400:8901::f03c:91ff:febb:bbc1] has joined #bitcoin-core-dev 09:41 -!- jrayhawk_ [~jrayhawk@unaffiliated/jrayhawk] has joined #bitcoin-core-dev 09:41 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt] 09:42 -!- ryanofsky_ [russ@jumpy.yanofsky.org] has joined #bitcoin-core-dev 09:42 -!- BlueMatt_ [~BlueMatt@mail.bluematt.me] has joined #bitcoin-core-dev 09:43 -!- murchandamus1 [~murchghos@81.169.211.205] has joined #bitcoin-core-dev 09:43 -!- Netsplit *.net <-> *.split quits: ensign_, murchandamus, da2ce7, BlueMatt, nsh, Taek, ryanofsky, jrayhawk 09:44 -!- Netsplit over, joins: da2ce7 09:47 -!- nsh [~lol@wikipedia/nsh] has joined #bitcoin-core-dev 09:47 -!- cryptapus_afk is now known as cryptapus 09:49 -!- ensign- [~ensign@2001:41d0:8:d711::1] has joined #bitcoin-core-dev 09:51 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has quit [Remote host closed the connection] 09:52 -!- ryanofsky_ is now known as ryanofsky 09:55 -!- whphhg [~whphhg@unaffiliated/whphhg] has joined #bitcoin-core-dev 10:12 -!- JackH [~laptop@2a02:a210:681:980:953e:d391:fecc:4d2f] has quit [Quit: Leaving] 10:38 -!- BlueMatt_ is now known as BlueMatt 10:38 -!- BlueMatt [~BlueMatt@mail.bluematt.me] has quit [Changing host] 10:38 -!- BlueMatt [~BlueMatt@unaffiliated/bluematt] has joined #bitcoin-core-dev 10:49 < bitcoin-git> [bitcoin] sdaftuar opened pull request #9912: Optimize GetWitnessHash() for non-segwit transactions (master...2017-03-optimize-witness-hash) https://github.com/bitcoin/bitcoin/pull/9912 10:57 -!- dodomojo [~goksinen@2604:2000:c591:8400:c597:103:82a1:f791] has quit [Remote host closed the connection] 11:03 -!- To7 [~theo@cpe-158-222-192-214.nyc.res.rr.com] has joined #bitcoin-core-dev 11:04 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 246 seconds] 11:10 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Remote host closed the connection] 11:10 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 11:15 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 240 seconds] 11:38 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 12:15 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Remote host closed the connection] 12:39 -!- james [ad475dd2@gateway/web/freenode/ip.173.71.93.210] has joined #bitcoin-core-dev 12:39 -!- james is now known as Guest86728 12:40 -!- Guest86728 [ad475dd2@gateway/web/freenode/ip.173.71.93.210] has left #bitcoin-core-dev [] 12:44 -!- fanquake [~fanquake@unaffiliated/fanquake] has joined #bitcoin-core-dev 12:45 -!- waxwing [~waxwing@185.65.135.88] has quit [Ping timeout: 240 seconds] 12:46 -!- fanquake [~fanquake@unaffiliated/fanquake] has quit [Client Quit] 12:55 -!- waxwing [~waxwing@31.7.59.226] has joined #bitcoin-core-dev 13:09 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev 13:12 -!- paveljanik [~paveljani@unaffiliated/paveljanik] has quit [Quit: Leaving] 13:26 -!- AaronvanW [~AaronvanW@182.red-83-34-57.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 13:26 -!- AaronvanW [~AaronvanW@182.red-83-34-57.dynamicip.rima-tde.net] has quit [Changing host] 13:26 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev 14:18 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 258 seconds] 15:01 -!- wasi [~wasi@gateway/tor-sasl/wasi] has quit [Remote host closed the connection] 15:01 -!- wasi [~wasi@gateway/tor-sasl/wasi] has joined #bitcoin-core-dev 15:09 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)] 15:42 < gmaxwell> What are the barriers on getting SSE2 SHA2 in use? The speedup for initial sync should be far from trivial now. 15:42 < gmaxwell> (plus lowering latency of everything else we do) 15:43 < sipa> what is the speedup factor? 15:49 < luke-jr> why not SSE4 or AVX? 15:51 < gmaxwell> If I'm not misremembering SSE2 was 95% of the speedup of SSE4 and AVX was not faster on the hardware we tested (and in fact AVX was slower on AMD). But just replace SSE2 in my comment with whatever made sense. 15:51 < gmaxwell> sipa: before jonasschnelli benchmarked IBD with the change and it was 5% faster. IIRC... but now with assume valid a MUCH larger percentage of the time is in hashing. 15:53 < gmaxwell> So "more than 5%", though I don't know how much more. It will only grow after your UTXO changes. 15:54 < sipa> does it need parallel hashing? 16:02 < gmaxwell> no those figures are from using the scalar sha2 code. 16:03 < gmaxwell> AFAIK the only real place we can make good use of the parallel sha2 SSE code would be in hash tree computation, but that is complicated as you know. 16:03 < gmaxwell> IIRC the SIMD scalar sha2 is ~2x faster than ours, and the SIMD parallel sha2 is 3x faster than ours. 16:03 < luke-jr> merkle trees probably aren't a significant amount of hashing I think? 16:05 < gmaxwell> They're actually tons. because every node in it is three compression function runs. and there are txn*2 nodes in total. 16:05 < gmaxwell> so a block can have something like 24000 compression function runs. 16:06 < luke-jr> hmm 16:08 < sipa> so 8ms? 16:08 < sipa> that's significant 16:08 < sipa> assuming 3 GHz and 15 cpb for sha256 16:09 < TD-Linux> gmaxwell: it's a bit scary though as the external process will be able to keep a reference, and have all your key data :) <- you can seal the fd and verify the seal in the sandboxed process to eliminate this vulnerability 16:10 < gmaxwell> yes, it's non-trivial in terms of validation latency. 16:11 < TD-Linux> er actually disregard me, there is no read seal 16:15 -!- afk11_ [~afk11@176.61.69.103] has joined #bitcoin-core-dev 16:16 -!- mobilemo [ac381c38@gateway/web/freenode/ip.172.56.28.56] has joined #bitcoin-core-dev 16:18 -!- wudayoda [~goksinen@2604:2000:c591:8400:45f6:3838:740c:e363] has quit [Remote host closed the connection] 16:19 -!- mobilemo [ac381c38@gateway/web/freenode/ip.172.56.28.56] has left #bitcoin-core-dev [] 16:22 -!- bityogi [~textual@208-104-132-26.brvd.dsl.dyn.comporium.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 16:50 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 16:51 -!- dodomojo [~goksinen@2604:2000:c591:8400:dc0a:9dd0:74ee:7639] has joined #bitcoin-core-dev 16:52 < jeremyrubin> hm Bitcoin Unlimited just added ~parallel block validation~ but I'm pretty sure it has no perforance benefit 16:52 < jeremyrubin> (in case anyone is looking at what they implemented) 16:53 < TD-Linux> jeremyrubin, correct, it is not a performance enhancement but an attempt to "fix" quadratic hashing 16:53 < jeremyrubin> wait what 16:54 < jeremyrubin> The parallel block validation? 16:54 < TD-Linux> yes 16:54 < sipa> it means that the node doesn't stall if your block takes a day to validate 16:54 < TD-Linux> yup, it'll just burn a core for a day 16:54 < TD-Linux> of course under the assumption that the block also gets orphaned. 16:55 < jeremyrubin> Yeah isn't that block hopefully just going to orphan 16:55 < jeremyrubin> if it takes that long to validate 16:55 < jeremyrubin> hmm 16:55 < sipa> also, since our validation is already parallel anyway, it makes both blocks slower 16:56 < jeremyrubin> yeah that's why I thought it was useless 16:56 < jeremyrubin> core can only do so many IPC 16:56 < TD-Linux> jeremyrubin, thus the question "how long is too long" 16:56 < jeremyrubin> Well... that seems to be a hard fork then? 16:57 < jeremyrubin> Because now you'll partition old nodes trying to validate whatever monster block (assuming it crashes/kills your node on old hardware) 16:57 < TD-Linux> jeremyrubin, it is likely to cause forks, yes. 16:57 < TD-Linux> itself it is not a hard fork. 16:59 < jeremyrubin> yeah, sorry was slightly imprecise with terminology 17:00 -!- JackH [~laptop@79-73-188-131.dynamic.dsl.as9105.com] has joined #bitcoin-core-dev 17:00 < jeremyrubin> it seems they don't mention it being a quadratic hashing fix in the documentation 17:00 < jeremyrubin> Also it seems quadratic hashing isn't really a problem before this either, just wait until the block gets orphaned? 17:01 < gmaxwell> yea, it clearly has some unexplored interactions with selfish mining too. E.g. if you mine an empty block, now you hold it locally for a bit comfortable that if there is a block race, you'll win even though you announced later... you can be up to the typical validation time late in your announcement. 17:01 < gmaxwell> jeremyrubin: qudratic hashing is a _huge_ problem if you've ripped off the blocksize limit and done nothing about it. 17:02 < gmaxwell> a block could take days to hash and shut down the network. :P (except for collaborating miners that know to 'optimize out' checking that transaciton) :) 17:03 < jeremyrubin> Do we have a provision for abandoning a block mid validation if a longer header chain is seen? 17:03 < jeremyrubin> Probably a tighter way to address same concern. 17:04 < gmaxwell> it's not a concern for us. As it's not really easy to make excessively slow blocks in the concernsus rules and segwit completely fixes quadratic hashing. 17:05 < gmaxwell> could basically worry about it once it _ever_ would have made a difference, rather than adding complexity now. 17:05 < gmaxwell> (the complexity would be that that longer chain may be invalid, so you'll have to go back to validating the other thing, seems messy as heck) 17:06 < jeremyrubin> well... it can be a good "defensive" code in case a new compelxity attaack is ever found. 17:06 < gmaxwell> sure but has to be weighed against the complexity of the fix and the risk it implies. 17:06 < jeremyrubin> I think it's a simple rule; always be trying to validate the largest-POW chain 17:07 < TD-Linux> if such an attack was made, simply stalling is a pretty good failure option 17:07 < gmaxwell> (maybe if someone implemented it they'd find it was easy... though just testing it makes me feel uneasy, concurrency is really hard to test well.) 17:07 < jeremyrubin> fair! 17:08 < gmaxwell> jeremyrubin: we do that, subject to the fact that while validating a block we're effectively non-concurrent, so we won't learn about the longer chain until we're done. So really the complexity there is just in safely increasing the concurrency. Which might be an independant good--- e.g. a side effect of the changes for the block testing stuff we were talking about a day ago. 17:11 < TD-Linux> certainly I'd hope for more tests than parallel validation has :^) 17:12 < gmaxwell> we don't really have a good test harness for testing concurrency. data race freeness doesn't mean that a parallel algorihim will yield expected results in all ordering sequences. 17:15 < TD-Linux> some sort of framework that would cause all mutexes to block until the test explicitly lets them continue would be neat. 17:17 < gmaxwell> TD-Linux: well RR actually has neat stuff for making threaded execution determinstic, that I think could be listed into being a concurrency fuzzing tool. 17:18 < gmaxwell> e.g. replay from an to a given point... and then repeat the replay with many different values given to the RNG that schedules the threads, and see if you get different results. 17:18 < TD-Linux> gmaxwell, well if you want to fuzz rather than be explicit, doesn't rr's chaos mode already count? 17:18 < gmaxwell> oh does it already do this? lol 17:19 < luke-jr> [01:01:39] yea, it clearly has some unexplored interactions with selfish mining too. E.g. if you mine an empty block, now you hold it locally for a bit comfortable that if there is a block race, you'll win even though you announced later… you can be up to the typical validation time late in your announcement. <-- they may claim this is a good thing, since it incentivises smaller blocks 17:20 < luke-jr> although in fact it incentivises blocks which meet relay-network policy even better 17:20 < luke-jr> or rather, the most-limited relay policy 17:20 < gmaxwell> the key point about selfish mining is that it gives excess returns to larger miners. So "incenticizes smaller blocks at the expense of decenteralization" ... missing the point. :P Also, not smaller but empty the validation time difference between a block and a slightly smaller one is neglgible, you have to make the block empty to reliably cut in front of others. 17:21 < gmaxwell> the latest BIP152 stuff is much more policy durable than prior stuff, since it will retain transactions rejected for policy reasons and still use them to reconstruct blocks. 17:23 < luke-jr> that's not a good thing IMO 17:23 < luke-jr> the network policy putting pressure on miners is a desirable trait 17:23 < luke-jr> although most-restrictive isn't the ideal either, so meh 17:24 < gmaxwell> 'network policy' doesn't matter, other miners policy matters. And moreover: doublespend is 'policy' that is not in the miners control. 17:24 < gmaxwell> Without extra someone spamming doublespends can considerably slow propagation. 17:25 < luke-jr> true, but if that's the only concern, we would want to limit the extra pool to just double spends 17:25 < gmaxwell> luke-jr: consider, without it there is pressure to not increase the minimum feerate in your mempool. 17:26 < gmaxwell> because it will make you slower to accept blocks from others with a lower minimum feerate. 17:26 < gmaxwell> (though I think there should probably be seperate extra pools for different kinds of rejections, adding complexity would have delayed getting it in) 17:26 < gmaxwell> luke-jr: also keep in mind that any time miners expirence delays the easiest solution for them is to just centeralize their pooling more. 17:27 < gmaxwell> they're not going to sit and go "oh that sucks, I'll twiddle my policy." 17:27 * luke-jr notes he didn't oppose extra-pool :P 17:28 < gmaxwell> fair enough. 17:54 -!- abpa [~abpa@96-82-80-25-static.hfc.comcastbusiness.net] has quit [Quit: Textual IRC Client: www.textualapp.com] 17:55 -!- dodomojo [~goksinen@2604:2000:c591:8400:dc0a:9dd0:74ee:7639] has quit [Remote host closed the connection] 17:57 -!- dodomojo [~goksinen@2604:2000:c591:8400:addb:9543:ac1a:9240] has joined #bitcoin-core-dev 18:08 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:4119:8c2e:bd3c:1984] has joined #bitcoin-core-dev 18:11 -!- dodomojo [~goksinen@2604:2000:c591:8400:addb:9543:ac1a:9240] has quit [Ping timeout: 246 seconds] 18:12 -!- kadoban [~mud@unaffiliated/kadoban] has joined #bitcoin-core-dev 18:22 -!- dodomojo [~goksinen@2604:2000:c591:8400:4cd2:2dd3:d101:9ddb] has joined #bitcoin-core-dev 18:24 -!- dodomoj__ [~goksinen@2604:2000:c591:8400:8d09:5c1a:98d6:cde8] has joined #bitcoin-core-dev 18:25 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:4119:8c2e:bd3c:1984] has quit [Ping timeout: 246 seconds] 18:27 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:6043:dbbf:e139:2fc8] has joined #bitcoin-core-dev 18:27 -!- dodomojo [~goksinen@2604:2000:c591:8400:4cd2:2dd3:d101:9ddb] has quit [Ping timeout: 246 seconds] 18:30 -!- dodomoj__ [~goksinen@2604:2000:c591:8400:8d09:5c1a:98d6:cde8] has quit [Ping timeout: 246 seconds] 18:34 -!- dodomojo [~goksinen@2604:2000:c591:8400:ed43:95e9:4852:c00e] has joined #bitcoin-core-dev 18:34 -!- rafalcpp_ [~racalcppp@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev 18:35 -!- rafalcpp [~racalcppp@84-10-11-234.static.chello.pl] has quit [Read error: Connection reset by peer] 18:37 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:6043:dbbf:e139:2fc8] has quit [Ping timeout: 246 seconds] 18:40 -!- str4d [~str4d@50-255-21-53-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev 18:44 -!- rafalcpp_ [~racalcppp@84-10-11-234.static.chello.pl] has quit [Ping timeout: 240 seconds] 18:45 -!- rafalcpp_ [~racalcppp@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev 18:46 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:ad3b:2baa:1a7:8cf0] has joined #bitcoin-core-dev 18:47 -!- dodomoj__ [~goksinen@2604:2000:c591:8400:2c0c:89c0:231a:c66] has joined #bitcoin-core-dev 18:48 -!- dodomoj__ [~goksinen@2604:2000:c591:8400:2c0c:89c0:231a:c66] has quit [Remote host closed the connection] 18:48 -!- dodomojo [~goksinen@2604:2000:c591:8400:ed43:95e9:4852:c00e] has quit [Ping timeout: 246 seconds] 18:50 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:ad3b:2baa:1a7:8cf0] has quit [Ping timeout: 246 seconds] 18:54 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-zwbsvuiemfdwaiju] has quit [Quit: Connection closed for inactivity] 19:02 -!- MarcoFalke [~marco@host10-2.natpool.mwn.de] has quit [Quit: MarcoFalke] 19:35 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Remote host closed the connection] 19:42 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 19:44 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 240 seconds] 19:54 < luke-jr> #8694 is finally ready for final reviewing 19:54 < gribble> https://github.com/bitcoin/bitcoin/issues/8694 | Basic multiwallet support by luke-jr · Pull Request #8694 · bitcoin/bitcoin · GitHub 20:15 -!- cryptapus is now known as cryptapus_afk 20:15 < jeremyrubin> I'm debugging something; is there a threadsafe way to access pcoinstip during connectblock? I want to be able to access it from a scriptcheck 20:15 < jeremyrubin> I'm guessing I would have to add locks around the usage 20:16 < jeremyrubin> curious if anyone's done this before and if the performance decrease is bad 20:16 < jeremyrubin> (I think it might be doable to parallelize checking the inputs) 20:26 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 264 seconds] 20:29 -!- moli_ [~molly@unaffiliated/molly] has joined #bitcoin-core-dev 20:30 < jeremyrubin> hm I think i have something workable using shared_mutex 20:31 -!- kadoban [~mud@unaffiliated/kadoban] has quit [Quit: bye] 20:31 -!- molz_ [~molly@unaffiliated/molly] has quit [Ping timeout: 246 seconds] 20:33 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 21:04 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has quit [Read error: Connection reset by peer] 21:05 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 21:07 -!- str4d [~str4d@50-255-21-53-static.hfc.comcastbusiness.net] has quit [Ping timeout: 260 seconds] 21:42 -!- dodomojo [~goksinen@2604:2000:c591:8400:bae8:56ff:fe2c:6c02] has joined #bitcoin-core-dev 21:49 -!- Robinson [~user1@2a02:a03f:a66:9300:18a8:3d09:45ec:b133] has joined #bitcoin-core-dev 21:57 -!- jannes [~jannes@095-097-246-234.static.chello.nl] has quit [Quit: Leaving] 21:59 -!- dodomojo [~goksinen@2604:2000:c591:8400:bae8:56ff:fe2c:6c02] has quit [Read error: Connection reset by peer] 22:00 -!- dodomojo [~goksinen@2604:2000:c591:8400:c55e:f423:e06e:a1c5] has joined #bitcoin-core-dev 22:02 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:a0b2:85de:cfe2:4cfe] has joined #bitcoin-core-dev 22:05 -!- dodomojo [~goksinen@2604:2000:c591:8400:c55e:f423:e06e:a1c5] has quit [Ping timeout: 246 seconds] 22:20 -!- Robinson [~user1@2a02:a03f:a66:9300:18a8:3d09:45ec:b133] has quit [Ping timeout: 264 seconds] 22:21 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:a0b2:85de:cfe2:4cfe] has quit [Remote host closed the connection] 22:23 -!- dodomojo [~goksinen@2604:2000:c591:8400:25f5:7195:66f3:8e2d] has joined #bitcoin-core-dev 22:34 -!- str4d [~str4d@107-204-212-44.lightspeed.sntcca.sbcglobal.net] has joined #bitcoin-core-dev 22:36 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:113c:7218:f169:827] has joined #bitcoin-core-dev 22:37 -!- dodomoj__ [~goksinen@2604:2000:c591:8400:2593:cc88:1bdd:f385] has joined #bitcoin-core-dev 22:39 -!- dodomojo [~goksinen@2604:2000:c591:8400:25f5:7195:66f3:8e2d] has quit [Ping timeout: 258 seconds] 22:40 -!- dodomojo_ [~goksinen@2604:2000:c591:8400:113c:7218:f169:827] has quit [Ping timeout: 246 seconds] 22:44 -!- whphhg [~whphhg@unaffiliated/whphhg] has quit [Quit: Leaving] 22:45 -!- whphhg [~whphhg@unaffiliated/whphhg] has joined #bitcoin-core-dev 22:46 -!- whphhg [~whphhg@unaffiliated/whphhg] has quit [Client Quit] 22:47 -!- whphhg [~whphhg@unaffiliated/whphhg] has joined #bitcoin-core-dev 22:48 -!- whphhg [~whphhg@unaffiliated/whphhg] has quit [Client Quit] 22:48 -!- whphhg [~whphhg@unaffiliated/whphhg] has joined #bitcoin-core-dev 22:55 -!- str4d [~str4d@107-204-212-44.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 268 seconds] 22:59 -!- dodomoj__ [~goksinen@2604:2000:c591:8400:2593:cc88:1bdd:f385] has quit [Remote host closed the connection] 23:05 -!- To7 [~theo@cpe-158-222-192-214.nyc.res.rr.com] has quit [Quit: Whatever] 23:08 -!- whphhg [~whphhg@unaffiliated/whphhg] has quit [Quit: Leaving] 23:08 -!- whphhg [~whphhg@unaffiliated/whphhg] has joined #bitcoin-core-dev 23:33 -!- dodomojo [~goksinen@2604:2000:c591:8400:2593:cc88:1bdd:f385] has joined #bitcoin-core-dev 23:38 -!- dodomojo [~goksinen@2604:2000:c591:8400:2593:cc88:1bdd:f385] has quit [Ping timeout: 246 seconds] 23:51 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has quit [Read error: Connection reset by peer] 23:52 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 23:55 -!- whphhg [~whphhg@unaffiliated/whphhg] has quit [Quit: Leaving]