--- Log opened Sun Aug 26 00:00:50 2018 00:03 -!- vexbuy [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has quit [Remote host closed the connection] 00:05 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has quit [Remote host closed the connection] 00:05 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 00:06 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has joined #bitcoin-core-dev 00:06 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 00:09 -!- plankers [~plank@38.87.81.82] has quit [Ping timeout: 264 seconds] 00:10 -!- Taifta [~Taifta@31.210.189.17] has joined #bitcoin-core-dev 00:11 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 00:14 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 00:15 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #bitcoin-core-dev 00:15 < wumpus> 0.17.0rc2 binaries up https://bitcoincore.org/bin/bitcoin-core-0.17.0/test.rc2/ 00:19 -!- vexbuy [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 00:23 -!- d_t [~d_t@108-65-77-11.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 268 seconds] 00:37 -!- vexbuy [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has quit [Remote host closed the connection] 00:48 -!- vexbuy [~vexbuy@174.161.15.62.static.jazztel.es] has joined #bitcoin-core-dev 00:50 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 00:50 < wumpus> luke-jr: the only reason that the change to num files was acceptable is because leveldb, at least the version in our tree, doesn't actually keep more files open (on platforms with mmap) see also https://github.com/bitcoin/bitcoin/pull/12495#issuecomment-377228329 00:50 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 00:53 < luke-jr> gmaxwell: cannot reproduce with -checkblocks=10000 :/ 00:53 < luke-jr> wumpus: I understand that. I just don't understand why increase the mmap-specific limit, without increasing the num files limit that is required for any mmap 00:53 < luke-jr> we will never have more than 1001 mmaps because the files limit is 1000 01:04 < wumpus> --I don't think mappings into closed files count against the open files ulimit 01:06 -!- vexbuy [~vexbuy@174.161.15.62.static.jazztel.es] has quit [Remote host closed the connection] 01:07 -!- vexbuy [~vexbuy@89.39.107.205] has joined #bitcoin-core-dev 01:07 -!- Rootsudo [~textual@180.190.116.181] has quit [Quit: Textual IRC Client: www.textualapp.com] 01:09 < sipa> wumpus: luke's point is that internally in leveldb, the mmapped files count towards the open file limit 01:10 < wumpus> but we increased leveldb's limit 01:11 < wumpus> though to be honest I'm completely confused now with all the different limites, how they interact, and how they're supposed to interact 01:11 < sipa> wumpus: yeah... 01:12 < wumpus> given all the confusion about this, I'm starting to think it might be better to revert these changes, I like understanding things even if it means not getting every last possible % of performance 01:12 < sipa> so there is a configurable leveldb max open files limit (which is 1000), and a mmap limit (which is hardcoded to 1000, but we raised it to 4096) 01:12 < wumpus> and seemingly it now relies on all kinds of undocumented behavior 01:12 < sipa> luke says (i haven't verified) that mmapped files count towards that limit... which is inconsistent with our observations 01:13 < sipa> i'm just clarifying my understanding as there seems to be some confusion 01:13 -!- Krellan_ [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has quit [Read error: Connection reset by peer] 01:13 < wumpus> thanks 01:13 < gmaxwell> right if we revert though we should revert both changes. 01:13 < wumpus> yes 01:13 < gmaxwell> I wouldn't oppose doing that for 0.17. 01:13 < sipa> both? what other chanfe? 01:14 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has joined #bitcoin-core-dev 01:14 < sipa> i saw the mmap increaee 01:14 < gmaxwell> sipa: the fd count change and the mmap limit change. 01:15 < gmaxwell> first FD count was increased with an argument that basically it wouldn't use more FDs due to using mmap. But then we found out that actually when it hit the mmap limit, it started actually using more FDs. 01:15 < wumpus> #12495 and https://github.com/bitcoin-core/leveldb/pull/19 01:15 < gribble> https://github.com/bitcoin/bitcoin/issues/12495 | Increase LevelDB max_open_files by eklitzke · Pull Request #12495 · bitcoin/bitcoin · GitHub 01:16 < gmaxwell> Since the lack of mmaps is probably also bad for performance (though I don't think anyone benchmarked specifically for that), it seemed like it made sense to increase mmaps rather than revert the fd count change. 01:16 < wumpus> yes 01:16 < wumpus> "By default LevelDB will only mmap() up to 1000 ldb files for reading and then fall back 01:16 < wumpus> to using file desciptors." 01:17 < wumpus> that pull 19 is *supposed* to change that 01:17 < wumpus> (from 1000 to 4096, which is still an arbitrary value) -- maybe luke-jr is using a system leveldb? 01:17 < sipa> where did we increase the FDs? 01:18 < wumpus> didn't that happen in #12495? 01:18 < gribble> https://github.com/bitcoin/bitcoin/issues/12495 | Increase LevelDB max_open_files by eklitzke · Pull Request #12495 · bitcoin/bitcoin · GitHub 01:18 < wumpus> I don't know anymore, sorry, should probably get coffee 01:18 < gmaxwell> wumpus: 12495, as you say. 01:19 < sipa> ah yes 01:19 < sipa> hmm 01:19 < sipa> we were using 64 files only before 01:19 < sipa> but using 1000 there seems pretty standard practice; it's leveldb's own default 01:19 < luke-jr> wumpus: not for this testing 01:19 < wumpus> sipa: correct 01:20 < gmaxwell> sipa: after increasing it, we were using a lot more file descriptors. 01:20 < luke-jr> wumpus: but in any case, for testing this, I'm changing the 4096 back to 1000, and can't get LDB to use a fd 01:20 < gmaxwell> luke-jr: are you using some system leveldb? 01:20 < luke-jr> gmaxwell: like I said, not for this; but in any case, it would be the same 1000 mmaps 01:21 < gmaxwell> luke-jr: dunno why it didn't reproduce for you, I saw leveldb using FDs, after using 1000 maps... and not anymore after increasing the map limit. 01:21 < luke-jr> hmm 01:22 < gmaxwell> sipa also saw it using fd's on his node. 01:22 < luke-jr> testnet or mainnet? 01:22 < gmaxwell> mainnet 01:22 < sipa> mainnet 01:22 < luke-jr> I wonder if that's why. :x 01:22 < gmaxwell> testnet probably doesn't have enough ldb files. 01:22 < luke-jr> ah 01:22 < sipa> how large is testnet's utxo set? 01:24 -!- PatBoy [xyz@192.99.249.194] has quit [Ping timeout: 268 seconds] 01:26 -!- PatBoy [xyz@192.99.249.194] has joined #bitcoin-core-dev 01:35 < wumpus> "disk_size": 1025042445, as of height 1384188 (it is at 1392776 now, don't know how much it changed since then) 01:48 < luke-jr> hopefully I won't "damage" my mainnet .bitcoin trying this.. :x 01:49 < luke-jr> (I still run 0.13 wallets from time to time) 01:50 -!- plankers [~plank@c-98-238-141-78.hsd1.ca.comcast.net] has joined #bitcoin-core-dev 01:50 < luke-jr> looks like the UTXO db upgrade isn't triggering it, at least 01:55 -!- plankers [~plank@c-98-238-141-78.hsd1.ca.comcast.net] has quit [Ping timeout: 272 seconds] 02:04 * luke-jr facepalms. After UTXO upgrade, it says I need to reindex >_< 02:09 -!- plankers [~plank@200.7.98.14] has joined #bitcoin-core-dev 02:10 < wumpus> does it say why / give an error message? the whole point of the utxo upgrade would be to not reindex, unless it fails 02:11 < provoostenator> If you're traveling and stuck with a slow laptop, the following gist lets you compile remotely and then connect via SSH tunnel: https://gist.github.com/Sjors/fda9b601e9464f61332ac6490f487c0a (improvements welcome) 02:26 < wumpus> provoostenator: nit: you could pass NO_UPNP to the depends build instead of --with-miniupnpc=no 02:26 < wumpus> not that libminiupnp takes long to compile 02:27 < provoostenator> wumpus: thanks, in this case it's probably easier to keep the configure line flexible. 02:27 -!- murrayn [~dafuq@unaffiliated/murrayn] has quit [Quit: Adios mofos] 02:28 < provoostenator> I first tried copying bitcoind over and just running it locally but that resulted in a world of pain, probably because the two systems aren't perfectly identical. 02:28 -!- plankers [~plank@200.7.98.14] has quit [Ping timeout: 252 seconds] 02:28 < wumpus> yes copying executables between linux systems is fraught with problems 02:29 < provoostenator> It's works fine with c-lightning though. 02:29 < wumpus> though it should work with depends and the glibc compat enabled 02:29 < provoostenator> I tried with depends as well, haven't tried glibc compat, might try that next time. But this also prevents me from accidentally syncing the chain over 4G. 02:30 < wumpus> -static-libstdc++ helps as well 02:30 < wumpus> (but only if all c++ dependencies are static, which is the case with a depends build but not otherwise) 02:32 < wumpus> then you end up with an executable that is as-good-as-static and it should be portable to non-ancient linux versions with the same CPU architecture 02:33 < wumpus> of course, that's the theory, there are still some things that can interfere for example if glibc is built with different options (I vaguely remember redhat and some security/hardening flag), or the target platform has a different libc 02:35 < wumpus> this is one reason they came up with "snap"s which are executables that essentially carry around part of the system dependencies with them in a container 02:35 < wumpus> (along with some security/isolation features) 02:35 < provoostenator> Being able to run locally is especially useful for QT, and with the multiprocess stuff it would be nice to run bitcoind remote and QT locally. 02:36 < wumpus> aanyhow if your two systems have the same release of the same distro, and are at the same update level, it should work too, that's probably easiest for most... 02:50 < luke-jr> wumpus: I reflink'd the blocks/ and chainstate/ dirs to a new datadir. I'm guessing that simply wasn't sufficient somehow 02:50 -!- plankers [~plank@c-98-238-141-78.hsd1.ca.comcast.net] has joined #bitcoin-core-dev 02:52 -!- murrayn [~murrayn@unaffiliated/murrayn] has joined #bitcoin-core-dev 02:54 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 02:54 < luke-jr> oh well, I'll let it sync overnight and see if it hits the breakpoint 02:55 < luke-jr> env_posix.cc:379 if anyone else who can reproduce it wants to try 02:55 < luke-jr> (if you hit it, maybe see if you can work out why it got there..) 02:56 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 03:18 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has quit [Quit: Leaving] 03:20 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has quit [Read error: Connection reset by peer] 03:21 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has joined #bitcoin-core-dev 03:33 -!- as1nc [~as1nc@ax213-1-82-66-157-194.fbx.proxad.net] has quit [Quit: Leaving] 03:46 -!- JackH [~laptop@host86-182-8-23.range86-182.btcentralplus.com] has joined #bitcoin-core-dev 04:01 -!- vexbuy [~vexbuy@89.39.107.205] has quit [Remote host closed the connection] 04:14 -!- vexbuy [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 04:23 < provoostenator> Any volunteer for making a PR based on this commit? https://github.com/bitcoin/bitcoin/commit/a9db3dada0119c183d16627805e90c4dbca05c6a 04:23 < provoostenator> I left it out of my rebase #13937 because I'd rather not touch thread stuff. 04:23 < gribble> https://github.com/bitcoin/bitcoin/issues/13937 | Track best-possible-headers (TheBlueMatt) by Sjors · Pull Request #13937 · bitcoin/bitcoin · GitHub 04:32 -!- belcher_ [~belcher@unaffiliated/belcher] has joined #bitcoin-core-dev 04:32 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has joined #bitcoin-core-dev 04:37 -!- str4d [~str4d@252.196.6.51.dyn.plus.net] has joined #bitcoin-core-dev 04:41 -!- cdecker [~cdecker@mail.snyke.net] has joined #bitcoin-core-dev 04:51 -!- str4d [~str4d@252.196.6.51.dyn.plus.net] has quit [Quit: Leaving] 04:51 -!- str4d [~str4d@252.196.6.51.dyn.plus.net] has joined #bitcoin-core-dev 04:52 -!- str4d [~str4d@252.196.6.51.dyn.plus.net] has quit [Client Quit] 05:06 < provoostenator> Never mind, that's already been done in #13023 05:06 < gribble> https://github.com/bitcoin/bitcoin/issues/13023 | Fix some concurrency issues in ActivateBestChain() by skeees · Pull Request #13023 · bitcoin/bitcoin · GitHub 05:13 -!- Emcy [~Emcy@unaffiliated/emcy] has quit [Ping timeout: 260 seconds] 05:19 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has quit [Remote host closed the connection] 05:19 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has joined #bitcoin-core-dev 05:20 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has quit [Remote host closed the connection] 05:28 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has quit [Read error: Connection reset by peer] 05:29 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has joined #bitcoin-core-dev 05:31 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 05:33 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 05:34 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has joined #bitcoin-core-dev 05:36 -!- farmerwampum [~farmerwam@104.129.29.18] has joined #bitcoin-core-dev 05:36 -!- farmerwampum [~farmerwam@104.129.29.18] has quit [Client Quit] 05:37 -!- farmerwampum [~farmerwam@104.129.29.18] has joined #bitcoin-core-dev 05:39 -!- wxss [~user@5.187.21.78] has quit [Quit: leaving] 05:47 -!- csknk [~csknk@unaffiliated/csknk] has joined #bitcoin-core-dev 05:58 -!- Taifta [~Taifta@31.210.189.17] has quit [Ping timeout: 244 seconds] 06:02 -!- Guyver2 [AdiIRC@guyver2.xs4all.nl] has joined #bitcoin-core-dev 06:15 -!- Taifta [~Taifta@37.26.147.206] has joined #bitcoin-core-dev 06:15 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Read error: Connection reset by peer] 06:16 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 06:28 -!- Taifta [~Taifta@37.26.147.206] has quit [Ping timeout: 272 seconds] 06:28 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 06:28 -!- booyah [~bb@193.25.1.157] has quit [Read error: Connection reset by peer] 06:29 -!- booyah [~bb@193.25.1.157] has joined #bitcoin-core-dev 06:31 -!- booyah [~bb@193.25.1.157] has quit [Read error: Connection reset by peer] 06:31 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 06:32 -!- vexbuy [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has quit [Remote host closed the connection] 06:36 -!- vexbuy [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 06:44 -!- booyah [~bb@193.25.1.157] has joined #bitcoin-core-dev 06:50 -!- Emcy [~Emcy@unaffiliated/emcy] has joined #bitcoin-core-dev 06:55 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 06:56 -!- Taifta [~Taifta@5.102.218.15] has joined #bitcoin-core-dev 07:06 < luke-jr> wumpus: heh, I was tired last night; the reason it needed a reindex (or I guess re-IBD) seems to be that I named the blocks directory "newblocks" XD 07:06 < luke-jr> err, "tmpblocks" 07:25 -!- Taifta [~Taifta@5.102.218.15] has quit [Ping timeout: 244 seconds] 07:34 -!- Emcy [~Emcy@unaffiliated/emcy] has quit [Ping timeout: 268 seconds] 07:36 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has quit [Read error: Connection reset by peer] 07:37 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has joined #bitcoin-core-dev 07:39 -!- elichai2 [uid212594@gateway/web/irccloud.com/x-yyoqbckblwfhimpu] has joined #bitcoin-core-dev 07:51 -!- vexbuy [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has quit [Remote host closed the connection] 07:54 -!- vexbuy [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 08:02 -!- vexbuy [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has quit [Remote host closed the connection] 08:03 -!- vexbuy [~vexbuy@89.39.107.205] has joined #bitcoin-core-dev 08:06 -!- spinza [~spin@155.93.246.187] has quit [Excess Flood] 08:06 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 252 seconds] 08:20 -!- spinza [~spin@155.93.246.187] has joined #bitcoin-core-dev 08:29 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 08:30 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 08:34 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 08:46 -!- plankers [~plank@c-98-238-141-78.hsd1.ca.comcast.net] has quit [Quit: Leaving.] 08:48 -!- vexbuy_ [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 08:48 -!- vexbuy [~vexbuy@89.39.107.205] has quit [Read error: Connection reset by peer] 08:49 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has quit [Remote host closed the connection] 08:50 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has joined #bitcoin-core-dev 08:55 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has quit [Ping timeout: 276 seconds] 08:55 -!- Emcy [~Emcy@unaffiliated/emcy] has joined #bitcoin-core-dev 08:56 -!- vexbuy_ [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has quit [Remote host closed the connection] 08:56 -!- vexbuy [~vexbuy@46.166.142.222] has joined #bitcoin-core-dev 09:07 -!- csknk [~csknk@unaffiliated/csknk] has quit [Quit: leaving] 09:12 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has joined #bitcoin-core-dev 09:13 -!- sipa_ [~pw@gateway/tor-sasl/sipa1024] has joined #bitcoin-core-dev 09:14 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has quit [Remote host closed the connection] 09:14 < Chris_Stewart_5> BIP157 doesn't currently have an open PR does it? 09:15 -!- intcat [~zshlyk@gateway/tor-sasl/intcat] has quit [Ping timeout: 250 seconds] 09:17 -!- Krellan [~Krellan@2601:640:4000:9258:6159:ba39:f506:92bc] has quit [Ping timeout: 276 seconds] 09:18 < instagibbs> Chris_Stewart_5, https://github.com/bitcoin/bitcoin/pull/13144 09:18 -!- intcat [~zshlyk@gateway/tor-sasl/intcat] has joined #bitcoin-core-dev 09:26 < Chris_Stewart_5> instagibbs: So that is just a refactor PR to make it easier to implement BIP157 right? 09:29 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 09:30 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 09:36 < instagibbs> doh wrong one 09:36 < instagibbs> one sec 09:36 < instagibbs> Chris_Stewart_5, merged two hours ago, didnt see that https://github.com/bitcoin/bitcoin/pull/12254 09:37 < instagibbs> nothing open after that 09:37 < Chris_Stewart_5> Hmm, is that only BIP158 though? I don't think that BIP158 includes any p2p networking stuff 09:37 < Chris_Stewart_5> BIP157 seems to have the semantics for that 09:48 -!- Krellan [~Krellan@2601:640:4000:9258:6518:5acd:4792:2a87] has joined #bitcoin-core-dev 09:50 -!- itaseski [~itaseski@213.135.176.242] has joined #bitcoin-core-dev 09:55 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 09:57 -!- Taifta [~Taifta@5.102.218.15] has joined #bitcoin-core-dev 09:58 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 10:18 -!- reallll [~belcher@unaffiliated/belcher] has joined #bitcoin-core-dev 10:21 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Read error: Connection reset by peer] 10:21 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 10:21 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 10:22 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 10:22 -!- belcher_ [~belcher@unaffiliated/belcher] has quit [Ping timeout: 272 seconds] 10:34 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has joined #bitcoin-core-dev 10:36 -!- reallll [~belcher@unaffiliated/belcher] has quit [Quit: Leaving] 10:41 -!- belcher_ [~belcher@unaffiliated/belcher] has joined #bitcoin-core-dev 10:42 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has quit [Remote host closed the connection] 11:03 -!- vexbuy_ [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has joined #bitcoin-core-dev 11:05 -!- vexbuy [~vexbuy@46.166.142.222] has quit [Read error: Connection reset by peer] 11:10 -!- sipa_ is now known as sipa 11:15 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 11:17 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 11:24 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has joined #bitcoin-core-dev 11:26 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has quit [Read error: Connection reset by peer] 11:28 < jimpo> Chris_Stewart_5: No, I'm working on the next PR towards BIP 157 support 11:28 < jimpo> Which is building filter indexes 11:28 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has joined #bitcoin-core-dev 11:33 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has quit [Ping timeout: 244 seconds] 11:34 -!- Krellan [~Krellan@2601:640:4000:9258:6518:5acd:4792:2a87] has quit [Remote host closed the connection] 11:34 -!- Krellan [~Krellan@2601:640:4000:9258:6518:5acd:4792:2a87] has joined #bitcoin-core-dev 11:37 -!- jb55 [~jb55@S010660e327dca171.vc.shawcable.net] has joined #bitcoin-core-dev 11:39 < Chris_Stewart_5> gmaxwell: PIR? Without a PIR method of fetching blocks, use of this is still toxic to user privacy 11:43 < sipa> Chris_Stewart_5: anything that doesn't fetch all blocks will leak some information, inevitably 11:45 < Chris_Stewart_5> What is the acronym? PIR -- i'm guessing it is not Passive Infrared Sensor hah 11:47 < sipa> private information retrieval 11:53 -!- Taifta [~Taifta@5.102.218.15] has quit [Ping timeout: 252 seconds] 12:01 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 12:03 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 12:03 < Chris_Stewart_5> jimpo: These test vectors seem to be a dead link https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki#appendix-c-test-vectors 12:03 < Chris_Stewart_5> at least the first link 12:07 -!- Salve [~salve@139.60.181.221] has joined #bitcoin-core-dev 12:18 -!- elichai2 [uid212594@gateway/web/irccloud.com/x-yyoqbckblwfhimpu] has quit [Quit: Connection closed for inactivity] 12:25 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Read error: Connection reset by peer] 12:26 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev 12:33 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [] 12:41 -!- Salve [~salve@139.60.181.221] has quit [] 12:53 -!- rex4539 [~rex4539@ppp-2-87-178-187.home.otenet.gr] has quit [Quit: Textual IRC Client: www.textualapp.com] 12:54 -!- Scrat [~herp@unaffiliated/scrat] has joined #bitcoin-core-dev 12:56 -!- Scrat [~herp@unaffiliated/scrat] has quit [Client Quit] 13:04 -!- rex4539 [~rex4539@2a02:587:3516:600:5ddb:4e0f:be31:6141] has joined #bitcoin-core-dev 13:04 -!- rex4539 [~rex4539@2a02:587:3516:600:5ddb:4e0f:be31:6141] has quit [Client Quit] 13:05 -!- rex4539 [~rex4539@2a02:587:3516:600:45a3:d887:95a3:ea8d] has joined #bitcoin-core-dev 13:06 -!- JackH [~laptop@host86-182-8-23.range86-182.btcentralplus.com] has quit [Ping timeout: 244 seconds] 13:08 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has joined #bitcoin-core-dev 13:09 -!- hebasto [~hebasto@195.60.70.234] has quit [Remote host closed the connection] 13:10 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Read error: Connection reset by peer] 13:10 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has quit [Remote host closed the connection] 13:16 < jimpo> Oh, thanks, will open a PR to fix that. Need to add a test block anyway. 13:17 < Chris_Stewart_5> Thanks! 13:43 -!- Krellan [~Krellan@2601:640:4000:9258:6518:5acd:4792:2a87] has quit [Read error: Connection reset by peer] 13:44 -!- Krellan [~Krellan@2601:640:4000:9258:6518:5acd:4792:2a87] has joined #bitcoin-core-dev 14:05 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has quit [Ping timeout: 272 seconds] 14:14 -!- itaseski [~itaseski@213.135.176.242] has quit [Ping timeout: 252 seconds] 14:15 -!- chainhead [~chainhead@2001:19f0:5:e14:5400:ff:fe77:e3d4] has joined #bitcoin-core-dev 14:41 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has joined #bitcoin-core-dev 14:44 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has quit [Read error: Connection reset by peer] 14:44 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has joined #bitcoin-core-dev 14:51 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has quit [Remote host closed the connection] 14:52 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has joined #bitcoin-core-dev 14:53 -!- belcher_ [~belcher@unaffiliated/belcher] has quit [Quit: Leaving] 15:03 -!- plankers [~plank@c-98-238-141-78.hsd1.ca.comcast.net] has joined #bitcoin-core-dev 15:06 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has quit [Remote host closed the connection] 15:07 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has joined #bitcoin-core-dev 15:12 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has quit [Ping timeout: 244 seconds] 15:18 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has joined #bitcoin-core-dev 15:28 < luke-jr> gmaxwell: sipa: wumpus: sync'd up to 341411 and still no non-mmap opens from LDB.. 15:29 < sipa> luke-jr: ? 15:29 < sipa> what does that mean 15:29 < gmaxwell> 341411 is a long time ago. 15:30 < gmaxwell> sipa: he's trying to reproduce the behavior ossifrage, you, and I saw where leveldb would hit the mmap limit and start using more FDs. 15:30 < sipa> you'll only exceed 1000 files/mmaps once you're above ~2GB UTXO set size 15:30 < sipa> as each file is 2MB ish 15:31 < luke-jr> hmm 15:32 < ossifrage> pmap 24199 | grep \.ldb | wc -l == 1894 (for a node up ~26 days) 15:34 < sipa> luke-jr: so you'll need to get to somewhere in 2017 at least 15:35 * luke-jr retries the UTXO upgrade, this time with the blocks dir named correctly 15:35 < ossifrage> That includes the txindex and the chainstate 15:36 < sipa> ossifrage: per database; txindex and chainstate are separate 15:37 < gmaxwell> has anyone just tried making a dummy program that mmaps a couple thousand files to see what the RES shows up as? 15:38 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Read error: Connection reset by peer] 15:39 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 15:40 < ossifrage> is having a larger resident size really an issue, you only get those pages if memory is available, the os should evict them right quick under memory pressure 15:40 < sipa> presumably, indeed 15:41 < sipa> it may be just an methodology problem, what rss doesn't exactly corresponds to what we care about 15:41 < gmaxwell> ossifrage: if thats actually the case, thing things are fine. 15:43 < gmaxwell> and if so we probably need to figure out how to start tracking "total dirty pages" or something. 15:44 < ossifrage> leveldb only uses mmap for reading, so the dirty pages from all the maps should be 0 15:44 < gmaxwell> Right. 15:44 < luke-jr> personally, I only look at ps's "size" metric 15:45 < luke-jr> size SIZE approximate amount of swap space that would be required if the process were to dirty all 15:45 < luke-jr> writable pages and then be swapped out. This number is very rough! 15:45 < gmaxwell> ossifrage: so to catch you up. We have infrastructure that benchmarks the software regularly and charts performance... so we find out if some commit accidentally hurts performance. 15:46 < gmaxwell> ossifrage: we noticed around the time of the 0.17 feature freeze that the claimed memory usage suddenly went up 20%! 15:46 < gmaxwell> during the week where a lot of last minute stuff got merged. 15:46 < gmaxwell> Bisection turned up your change to increase the leveldb maps count. 15:46 < ossifrage> gmaxwell, where does the claimed memory measurement come from? 15:47 < gmaxwell> Thats what we're trying to determine. after finding out that mmap automatically pages in some of the file tried madvising DONTNEED all the maps, but that didn't seem to fix it. 15:48 < gmaxwell> It might well be that the increased usage is totally fictional... that it's just counting paged-in non-dirty pages.... and they're not real memory pressure and we don't need to worry about them. 15:48 < ossifrage> I'm writing a mmap() test case out of morbid curiosity 15:49 < gmaxwell> Or maybe, though we haven't spotted it yet, leveldb is using 200kb of god knows what per open map. :P 15:49 < gmaxwell> Meanwhile, luke-jr went and read more of the leveldb code and believes that it can't have more mmaps open than the FD limit, and so he thinks the mmap increase should be a noop. 15:50 < gmaxwell> (which it clearly isn't, but it would be good if everyone was on the same page about how leveldb works) 15:50 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 272 seconds] 15:53 < ossifrage> I currently have 992 chainstate ldb files mapped and 859 txindex and 43 blocks/indexes ldb files, but it took a fairly long uptime for the original problem to show up 15:53 -!- Krellan [~Krellan@2601:640:4000:9258:6518:5acd:4792:2a87] has quit [Read error: Connection reset by peer] 15:54 -!- Krellan [~Krellan@2601:640:4000:9258:6518:5acd:4792:2a87] has joined #bitcoin-core-dev 15:58 -!- Guyver2 [AdiIRC@guyver2.xs4all.nl] has quit [Quit: Going offline, see ya! (www.adiirc.com)] 16:00 < ossifrage> I don't see much of a change between the htop reported RES size between mapping 1 txindex ldb file and mmaping all 2041 of them 16:00 < luke-jr> gmaxwell: well, I can reproduce it now, but I haven't figure out why yet 16:00 < ossifrage> (but I never read from the map) 16:02 < gmaxwell> ossifrage: k? try reading like, the last byte of them or something? 16:02 < ossifrage> The VIRT line goes up as expected but RES stays fairly small 16:03 < luke-jr> hmm, is everyone reproducing this using txindex? 16:04 < ossifrage> gmaxwell, yeah that caused the RES size to approach the VIRT size (read the last byte) 16:05 < sipa> gmaxwell: "it can't have more mmaps open than the FD limit" -> s/FD limit/max_open_files setting/ 16:05 < gmaxwell> now, try MADV_RANDOM ? 16:05 < gmaxwell> ossifrage: ^ 16:05 < ossifrage> I read the last byte, so I'm not sure what it would prefetch :-) 16:06 -!- andytoshi [~apoelstra@unaffiliated/andytoshi] has joined #bitcoin-core-dev 16:06 < gmaxwell> I'm guessing that it's prefetching the whole thing. 16:06 < gmaxwell> andytoshi: HI! 16:07 < andytoshi> gmaxwell: hiya, sorry, my server shit out this morning .. i have no idea why, the logs show everything clear, but it was apparently completely locked up even to a local terminal user 16:07 < andytoshi> so i called my dad to hard-boot it and it seems ok now 16:07 < ossifrage> MADV_RANDOM didn't seem to change the bump in RES caused by reading last byte 16:08 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has quit [Remote host closed the connection] 16:09 < gmaxwell> ossifrage: saddness. ... so run pmap on the process, and see how many dirty pages it says you have, presumably like none. 16:11 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 16:12 < sipa> gmaxwell: mmap is only used in leveldb for read-only files; none should ever be dirty 16:12 < gmaxwell> Interesting, there are only 41.3 BTC stored in bare multisig outputs, but there are 410k of them in total. 16:13 < gmaxwell> sipa: I know but I also would have expected that increasing the mmap limit wouldn't have increased RES. :) 16:13 < ossifrage> One sec, I might have screwed up... 16:14 < gmaxwell> I'm going to suggest we start charting the total dirty pages from pmap... and not worry about RES... 16:14 < luke-jr> it looks like the mmap limit is global 16:18 < ossifrage> Okay, it looks like MADV_RANDOM isn't helping, but just reading the last byte does cause the RES size to jump quite a bit (but I'm not sure how much I expect it to go up) 16:18 < ossifrage> at least 1 page per file? 16:18 < sipa> yes, 1 page per file is expected 16:21 < gmaxwell> I expect that 1 page per file is what you should get, which is a modest amount of memory. 16:21 < gmaxwell> in bitcoin we're seeing an increase of hundreds of meg. 16:21 < gmaxwell> way more than one page per file. 16:23 < ossifrage> When reading the first byte, MAD_RANDOM does decrease the RES usage 16:28 < ossifrage> Even with MADV_RANDOM, it would depend entirely on how many pages are faulted in by ldb 16:34 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has joined #bitcoin-core-dev 16:37 < ossifrage> Err, it looks like for me it is prefetching ~64K 16:43 < gmaxwell> 64K with MADV_RANDOM? 16:44 < ossifrage> I think I might be suffering from previous state problems 16:45 < ossifrage> Okay, dropping the cache and madv_random does seem to only read 4K for most of the files mapped (some have more for some odd reason) 16:47 < gmaxwell> oh interesting, so, if you cause the files to be paged in, then start a program with madv_random it will reflect higher res becuase you mmaped files that were already in cache? 16:47 < ossifrage> Yeah, without MADV_RANDOM all the files have 64K RES when I read the 1st byte 16:47 < ossifrage> echo 3 > /proc/sys/vm/drop_caches between runs 16:47 < ossifrage> Yeah I was getting confusing results until I realized that 16:47 < gmaxwell> Cool so also we might not have seen the effects of MADV_RANDOM when testing with actual leveldb due to that. 16:48 < ossifrage> Depends heavily on the specific hardware yeah 16:48 < ossifrage> But drop_caches is a mighty big hammer 16:49 < gmaxwell> sure, just in terms of "huh why did MADV_RANDOM seem to do nothing"... now we know. 16:50 < ossifrage> I suspect the minimum amount read without MADV_RANDOM depends on things like the filesystem and the underlying block device 16:51 < ossifrage> Like reading 1 page with a SSD is no big deal vs 1 page with a RAID5 md array 16:55 < ossifrage> Any ideas how much leveldb actually reads at a time? 16:55 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Read error: Connection reset by peer] 16:56 < ossifrage> It would have to touch at least some of the index and then the actual data it wants to read 16:56 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 16:57 < gmaxwell> it does a bisection search in the index and then I think it only reads the record of interest. 17:00 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has quit [Remote host closed the connection] 17:50 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 268 seconds] 17:52 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has joined #bitcoin-core-dev 17:57 -!- promag [~promag@bl22-247-244.dsl.telepac.pt] has quit [Ping timeout: 268 seconds] 17:57 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev 18:15 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has quit [Remote host closed the connection] 18:16 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has joined #bitcoin-core-dev 18:17 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has quit [Remote host closed the connection] 18:20 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has joined #bitcoin-core-dev 18:40 -!- pkx1 [~pkx@unaffiliated/pkx] has joined #bitcoin-core-dev 18:43 -!- pkx1 [~pkx@unaffiliated/pkx] has quit [Remote host closed the connection] 18:53 -!- pkx1 [~pkx@unaffiliated/pkx] has joined #bitcoin-core-dev 19:08 -!- intcat [~zshlyk@gateway/tor-sasl/intcat] has quit [Remote host closed the connection] 19:10 -!- intcat [~zshlyk@gateway/tor-sasl/intcat] has joined #bitcoin-core-dev 19:14 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [] 19:20 -!- plankers [~plank@c-98-238-141-78.hsd1.ca.comcast.net] has quit [Ping timeout: 268 seconds] 19:34 -!- plankers [~plank@38.87.81.82] has joined #bitcoin-core-dev 19:37 -!- phantomcircuit [~phantomci@192.241.205.97] has joined #bitcoin-core-dev 19:41 -!- pkx1 [~pkx@unaffiliated/pkx] has quit [Remote host closed the connection] 19:42 -!- pkx1 [~pkx@unaffiliated/pkx] has joined #bitcoin-core-dev 19:48 -!- pkx1 [~pkx@unaffiliated/pkx] has quit [Remote host closed the connection] 19:48 -!- Jmabsd [~jmabsd@1.186.68.250] has joined #bitcoin-core-dev 19:48 < Jmabsd> wait, where in the code again are the logics for checking if the PoW is on-target - i think there are two locations that do this comparison. 19:53 < sipa> pow.cpp, CheckProofOfWork 19:56 < phantomcircuit> sipa, can pnode->GetRefCount() be == instead of <= 0 19:56 < phantomcircuit> you put in an assert that it's >= 0 in 2013 soo 19:56 < sipa> phantomcircuit: i'm probably the wrong person to ask 20:03 -!- StopAndDecrypt [~StopAndDe@unaffiliated/stopanddecrypt] has joined #bitcoin-core-dev 20:11 -!- ken2812221_ [~ken281222@118-160-144-34.dynamic-ip.hinet.net] has joined #bitcoin-core-dev 20:28 < ossifrage> gmaxwell, it seems adding MADV_RANDOM to the leveldb mmap would help reduce the memory footprint, but it isn't going to help performance if there is any locality or larger reads 20:29 < sipa> there is no locality 20:29 < sipa> they're sorted by txid 20:30 < gmaxwell> ossifrage: I think it's somewhat likely to help performance specifically because there is no locality in reads. 20:31 < gmaxwell> ossifrage: I think the thing we really need to do is start measuring dirty pages, rather than RES. And I expect MADV_RANDOM will make it faster. 20:31 < ossifrage> gmaxwell, it all depends on the timeframe, if over enough time does it hit contiguous blocks and is that timeframe shorter then the the life of the page 20:31 < ossifrage> pay now vs pay later 20:36 -!- vexbuy_ [~vexbuy@147.red-83-47-212.dynamicip.rima-tde.net] has quit [Remote host closed the connection] 20:39 < ossifrage> looking at: pmap -x $(pidof bitcoin-qt) | grep .ldb | awk '{if($3 == 0){print}}' shows that many of the ldb files aren't using any RSS memory (after being up for almost a month) 20:39 < ossifrage> But I tend to OOM the POS box far too often 20:39 < Jmabsd> crypto/sha256.* is *single-iteration* SHA256? (would be great if this was marked out in the file) 20:40 < sipa> Jmabsd: of course; SHA256 is SHA256, not double SHA256 20:40 < Jmabsd> (y) 20:41 < sipa> double SHA256 is called "Hash" in the sourcecode 20:41 < sipa> and SHA256+RIPEMD160 is called Hash160 20:41 < Jmabsd> (y) 21:03 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 252 seconds] 21:11 < Jmabsd> wait, in the Bitcoin PoW 32bit floating point structure "deserializer" at https://github.com/bitcoin/bitcoin/blob/eb7daf4d600eeb631427c018a984a77a34aca66e/src/arith_uint256.cpp#L209 21:11 < Jmabsd> , you first shift-right a 32bit uint by 24 positions - this means the highest 8 bits become the lowest 8 bits, and the remainder of bits will be zero 21:11 < Jmabsd> then you bitwise-and the whole thing with 0x007fffff - but that's useless, isn't it? 21:12 < Jmabsd> here that's the same as bitwise-anding with 0xff, which was already implied by the shift-right ?? 21:13 -!- thaumavorio [~thaumavor@thaumavor.io] has joined #bitcoin-core-dev 21:14 < sipa> Jmabsd: read again 21:15 < Jmabsd> >> is right-shift, line 208 does "int nSize = nCompact >> 24;". 21:16 < Jmabsd> nCompact is uint32_t which is afaik an unsigned 32bit integer. 21:16 < Jmabsd> right-shifting 24 bits means what i suggested above. "int" here doesn't matter because it's at least 32bits and we're decimating the size here. 21:16 < Jmabsd> OH 21:16 < Jmabsd> sipa: ah you're right, the AND is on the original compact value. thx. 21:22 -!- kallewoof [~quassel@240d:1a:759:6000:a7b1:451a:8874:e1ac] has joined #bitcoin-core-dev 21:29 -!- instagibbs [~instagibb@pool-100-15-122-172.washdc.fios.verizon.net] has quit [Ping timeout: 276 seconds] 21:30 -!- instagibbs [~instagibb@pool-100-15-122-172.washdc.fios.verizon.net] has joined #bitcoin-core-dev 22:03 < luke-jr> FWIW, the most non-mmap files open I've seen is 39 22:03 < luke-jr> seems like it's txindex-related, as that's using 47 mmaps itself 22:07 < luke-jr> err, not txindex, I have that disabled apparently 22:07 < luke-jr> blocks/index :p 22:08 < luke-jr> I imagine it'd be worse with txindex 22:10 < Jmabsd> Wait, just a funny detail on the "showing block hash in reverse hex byte order in order for the PoW zeroes to be represented in the string's leading position": 22:10 < Jmabsd> *actually*, the hex string printer does NOT reverse the hex positions (https://github.com/bitcoin/bitcoin/blob/eb7daf4d600eeb631427c018a984a77a34aca66e/src/utilstrencodings.h#L121 - see here there's no reversing of the order of the bits in the byte) - and the reversing logic applied on the whole per-structure level is reversing byte order only, never bit order (ref. https://github.com/bitcoin/bitcoin/blob/eb7daf4d600eeb631427c018a984a77a34aca66e/sr 22:10 < Jmabsd> c/uint256.cpp#L23) 22:10 < Jmabsd> for this reason, actually, within each 8-multiple of PoW zeroes, actually the serialization will NOT be made in reverse order, and, we can see this already at block #5 here: https://www.blockchain.com/btc/block/000000003031a0e73735690c5a1ff2a4be82553b2a12b776fbd3a215dc8f778d 22:10 < gribble> https://github.com/bitcoin/bitcoin/issues/5 | Make the version number the protocol version and not the client version · Issue #5 · bitcoin/bitcoin · GitHub 22:10 < Jmabsd> *#6 22:10 < Jmabsd> Its hash is "000000003031a0e73735690c5a1ff2a4be82553b2a12b776fbd3a215dc8f778d". 22:10 < gribble> https://github.com/bitcoin/bitcoin/issues/6 | Treat wallet as a generic keystore · Issue #6 · bitcoin/bitcoin · GitHub 22:10 < Jmabsd> Do you see there, that, zeroes at the end of the block hash, which do count as PoW zeroes, are actually represented in the hex string *BEHIND* the digit 3, 22:10 < Jmabsd> yeah so this is not a big deal but the reversing (done for illustrative purposes) is only partial. 22:11 < luke-jr> Jmabsd: pretty sure you are wrong 22:11 < luke-jr> endianness deals with the bytes, not the bits 22:13 < sipa> Jmabsd: you're getting far too worked up about endianness issues 22:13 < sipa> they're conventions, and usually every convention sucks for some reason, but you still need to pick one 22:13 < sipa> deal with it 22:14 < sipa> if you get it wrong, try the other way 22:14 < luke-jr> only downside to big endian AFAIK is that you can't truncate by a simple cast :P 22:15 -!- Krellan [~Krellan@2601:640:4000:9258:6518:5acd:4792:2a87] has quit [Read error: Connection reset by peer] 22:16 -!- Krellan [~Krellan@2601:640:4000:9258:6518:5acd:4792:2a87] has joined #bitcoin-core-dev 22:17 < sipa> but yes, endianness is about the bytes (or symbols), not the indivudual bits 22:19 -!- plankers [~plank@38.87.81.82] has quit [Ping timeout: 244 seconds] 22:20 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has quit [Remote host closed the connection] 22:25 -!- d9b4bef9 [~d9b4bef9@web501.webfaction.com] has quit [Remote host closed the connection] 22:25 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has joined #bitcoin-core-dev 22:27 < Jmabsd> sipa: right, and an example of where PoW-significant zeroes show up in a non-leading position is block #6, https://www.blockchain.com/btc/block/000000003031a0e73735690c5a1ff2a4be82553b2a12b776fbd3a215dc8f778d 22:27 < gribble> https://github.com/bitcoin/bitcoin/issues/6 | Treat wallet as a generic keystore · Issue #6 · bitcoin/bitcoin · GitHub 22:28 < Jmabsd> the 5:th byte in the reverse order there, is 0x30, the 0 there is zeroes however representation is byte-reversed only not bit reversed, and so the last bits in that byte get on the second hex position behind the 3. 22:28 < Jmabsd> this is trivial but anyhow. 22:28 < sipa> Jmabsd: i think it's getting offtopic here 22:29 < sipa> we can discuss the general principle, but at some point you just to try and see 22:30 < Jmabsd> yep. 22:33 -!- savil [savilmatri@gateway/shell/matrix.org/x-etxndewahlljdhqo] has quit [Ping timeout: 240 seconds] 22:43 -!- plankers [~plank@c-98-238-141-78.hsd1.ca.comcast.net] has joined #bitcoin-core-dev 22:45 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has joined #bitcoin-core-dev 22:54 -!- savil [savilmatri@gateway/shell/matrix.org/x-xhybwtwfzlarnrip] has joined #bitcoin-core-dev 23:01 -!- Jmabsd [~jmabsd@1.186.68.250] has quit [Quit: Leaving] 23:02 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has quit [Ping timeout: 272 seconds] 23:03 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has joined #bitcoin-core-dev 23:04 < ossifrage> luke-jr, the original reason for the leveldb mmap change was that it was using a large number of file descriptors, enough to push incoming connections out of the <1024 fd select() limit. Bumping up the mmap count was sorta a workaround for that problem 23:05 -!- karelb [uid316741@gateway/web/irccloud.com/x-mkfofbuiiybjpdcw] has quit [Ping timeout: 264 seconds] 23:05 -!- jl2012 [sid133844@gateway/web/irccloud.com/x-mspisgraidtswzjf] has quit [Ping timeout: 264 seconds] 23:06 -!- jl2012 [sid133844@gateway/web/irccloud.com/x-dsllevwlihqjtfrl] has joined #bitcoin-core-dev 23:07 -!- karelb [uid316741@gateway/web/irccloud.com/x-ixtseyxbnagrlyfv] has joined #bitcoin-core-dev 23:10 -!- karelb [uid316741@gateway/web/irccloud.com/x-ixtseyxbnagrlyfv] has quit [Read error: Connection reset by peer] 23:11 -!- atroxes [~atroxes@unaffiliated/atroxes] has quit [Quit: bye] 23:11 -!- karelb [uid316741@gateway/web/irccloud.com/x-dmutzicesyqazbok] has joined #bitcoin-core-dev 23:11 -!- atroxes [~atroxes@unaffiliated/atroxes] has joined #bitcoin-core-dev 23:12 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has quit [Ping timeout: 272 seconds] 23:13 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has joined #bitcoin-core-dev 23:22 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has quit [Ping timeout: 252 seconds] 23:23 -!- rex4539 [~rex4539@2a02:587:3516:600:45a3:d887:95a3:ea8d] has quit [Quit: Textual IRC Client: www.textualapp.com] 23:23 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has joined #bitcoin-core-dev 23:27 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Read error: Connection reset by peer] 23:29 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 23:32 -!- kallisteiros [scientist@gateway/vpn/privateinternetaccess/kallisteiros] has quit [Ping timeout: 244 seconds] 23:35 < luke-jr> ossifrage: yes, I understand why now 23:46 -!- takinbo [sid19838@gateway/web/irccloud.com/x-qfssmoijxnzmlvbo] has joined #bitcoin-core-dev --- Log closed Mon Aug 27 00:00:51 2018