From: Jonathan Toomim <j@toom•im>
To: Bitcoin Dev <bitcoin-dev@lists•linuxfoundation.org>
Cc: bitcoin-xt <bitcoin-xt@googlegroups•com>
Subject: Re: [bitcoin-dev] Memory leaks?
Date: Sun, 18 Oct 2015 08:59:11 -0700 [thread overview]
Message-ID: <F938BD02-3D80-4E99-BD1C-490543187895@toom.im> (raw)
In-Reply-To: <20151014093913.GB19607@amethyst.visucore.com>
[-- Attachment #1.1: Type: text/plain, Size: 3537 bytes --]
On Oct 14, 2015, at 2:39 AM, Wladimir J. van der Laan <laanwj@gmail•com> wrote:
> This is *most likely* the mempool, but is just not reported correctly.
I did some testing with PR #6410's better mempool reporting. The improved reporting suggests that actual in-memory usage ("usage":) by CTxMemPool is about 2.5x to 3x higher than the serialized transaction sizes ("bytes":). The excess memory usage that I'm seeing is on the order of 100x higher than the mempool "bytes": value. As such, I think it's unlikely that this is the mempool, or at least not normal/correct mempool behavior.
Another user (admin@multipool•us) reported 35 GB of RSS usage. I'm guessing his bitcoind has been running longer than any of mine. His server definitely has more RAM. I don't know which email list he is subscribed to (probably XT), so I'm sharing it with both lists to make sure you're all aware of how big an issue this can be.
> In the meantime you can mitigate the mempool growth by setting `-mintxfee`, see
> https://github.com/bitcoin/bitcoin/blob/v0.11.0/doc/release-notes.md#transaction-flooding
I have mintxfee and minrelaytxfee set to about 0.00003, which is high enough to exclude essentially all of the of the 14700-14800 byte flood transactions. My nodes' mempools only contain about one or two blocks' worth of transactions. So I don't think this is correct either.
Some additional notes on this issue:
1. I think it's related to CreateNewBlock() and getblocktemplate. I ran a Core bitcoind process (commit d78a880) overnight with no mining connected to it, and (IIRC -- my memory is fuzzy) when I woke up it was using around 400 MB of RSS and the mempool was at around "bytes":10MB, "usage": 25MB. I ran ./bitcoin-cli getblocktemplate once, and IIRC the RSS shot up to around 800 MB. I then ran getblocktemplate every 5 seconds for about 30 minutes, and RSS climbed to 1180 MB. An hour after that with more getblocktemplates, and now RSS is at 1350 MB. [Edit: 1490 MB about 30 minutes later.] getmempoolinfo is still showing "usage" around 25MB or less.
I'll do some more testing with this and see if I can make it repeatable, and record the results more carefully. Expect a follow-up from me in a day or two.
2. valgrind did not show anything super promising. It did report this:
==6880== LEAK SUMMARY:
==6880== definitely lost: 0 bytes in 0 blocks
==6880== indirectly lost: 0 bytes in 0 blocks
==6880== possibly lost: 288 bytes in 1 blocks
==6880== still reachable: 10,552 bytes in 39 blocks
==6880== suppressed: 0 bytes in 0 blocks
(Bitcoin Core commit d78a880)
and this:
==6778== LEAK SUMMARY:
==6778== definitely lost: 0 bytes in 0 blocks
==6778== indirectly lost: 0 bytes in 0 blocks
==6778== possibly lost: 320 bytes in 1 blocks
==6778== still reachable: 10,080 bytes in 32 blocks
==6778== suppressed: 0 bytes in 0 blocks
(Bitcoin XT commit fe446d)
I haven't found anything in there yet that I think would produce the multi-GB memory usage after running for a few days, but I could be missing it. Email me if you want the full log.
I did not try running getblocktemplate while valgrind was running. I'll have to try that. I also have not let valgrind run for more than an hour.
P.S.: Sorry for all the cross-post confusion and consequent flamewar fallout. While it's probably too late for this thread, I'll make sure to post in a manner that keeps the threads clearly separate in the future (e.g. different subject lines).
[-- Attachment #1.2: Type: text/html, Size: 5893 bytes --]
[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]
next prev parent reply other threads:[~2015-10-18 15:59 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-13 21:56 Jonathan Toomim (Toomim Bros)
2015-10-13 22:49 ` odinn
2015-10-13 23:14 ` Jonathan Toomim (Toomim Bros)
2015-10-13 23:52 ` Dave Scotese
2015-10-14 0:08 ` Jonathan Toomim (Toomim Bros)
2015-10-14 0:25 ` Dave Scotese
2015-10-13 23:59 ` odinn
2015-10-14 7:22 ` Pavel Janík
2015-10-14 7:58 ` Tom Zander
2015-10-14 9:09 ` Mike Hearn
2015-10-14 15:19 ` odinn
2015-10-14 9:39 ` Wladimir J. van der Laan
2015-10-18 15:59 ` Jonathan Toomim [this message]
2015-10-19 19:17 ` Multipool Admin
2015-10-19 22:24 ` Multipool Admin
2015-10-20 10:12 ` Mike Hearn
2015-10-20 12:39 ` Jonathan Toomim
2015-10-21 3:01 ` Jonathan Toomim
2015-10-21 7:29 ` Tom Zander
2015-10-21 17:58 ` Jonathan Toomim
2015-10-22 16:06 ` Multipool Admin
2015-10-22 16:27 ` Jonathan Toomim
2015-10-22 17:01 ` Btc Drak
2015-10-23 6:41 ` Rusty Russell
2015-10-21 7:59 ` Ross Bennetts
2015-10-17 4:37 ` admin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=F938BD02-3D80-4E99-BD1C-490543187895@toom.im \
--to=j@toom$(echo .)im \
--cc=bitcoin-dev@lists$(echo .)linuxfoundation.org \
--cc=bitcoin-xt@googlegroups$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox