I should also mention that this is definitely not an attack coming from connected nodes. My node experiencing the issue is only connected to 3 other nodes, all of which I control (via connect=). --Adam On Mon, Oct 19, 2015 at 12:17 PM, Multipool Admin wrote: > My nodes are continuously running getblocktemplate and getinfo, and I also > suspected the issue is in either gbt or the rpc server. > > The instance only takes a few hours to get up to that memory usage. > On Oct 18, 2015 8:59 AM, "Jonathan Toomim via bitcoin-dev" < > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> On Oct 14, 2015, at 2:39 AM, Wladimir J. van der Laan >> wrote: >> >> This is *most likely* the mempool, but is just not reported correctly. >> >> >> I did some testing with PR #6410's better mempool reporting. The improved >> reporting suggests that actual in-memory usage ("usage":) by CTxMemPool is >> about 2.5x to 3x higher than the serialized transaction sizes ("bytes":). >> The excess memory usage that I'm seeing is on the order of 100x higher than >> the mempool "bytes": value. As such, I think it's unlikely that this is the >> mempool, or at least not normal/correct mempool behavior. >> >> Another user (admin@multipool.us) reported 35 GB of RSS usage. I'm >> guessing his bitcoind has been running longer than any of mine. His server >> definitely has more RAM. I don't know which email list he is subscribed to >> (probably XT), so I'm sharing it with both lists to make sure you're all >> aware of how big an issue this can be. >> >> In the meantime you can mitigate the mempool growth by setting >> `-mintxfee`, see >> >> https://github.com/bitcoin/bitcoin/blob/v0.11.0/doc/release-notes.md#transaction-flooding >> >> >> I have mintxfee and minrelaytxfee set to about 0.00003, which is high >> enough to exclude essentially all of the of the 14700-14800 byte flood >> transactions. My nodes' mempools only contain about one or two blocks' >> worth of transactions. So I don't think this is correct either. >> >> >> >> Some additional notes on this issue: >> >> 1. I think it's related to CreateNewBlock() and getblocktemplate. I ran a >> Core bitcoind process (commit d78a880) overnight with no mining connected >> to it, and (IIRC -- my memory is fuzzy) when I woke up it was using around >> 400 MB of RSS and the mempool was at around "bytes":10MB, "usage": 25MB. I >> ran ./bitcoin-cli getblocktemplate once, and IIRC the RSS shot up to around >> 800 MB. I then ran getblocktemplate every 5 seconds for about 30 minutes, >> and RSS climbed to 1180 MB. An hour after that with more getblocktemplates, >> and now RSS is at 1350 MB. [Edit: 1490 MB about 30 minutes later.] >> getmempoolinfo is still showing "usage" around 25MB or less. >> >> I'll do some more testing with this and see if I can make it repeatable, >> and record the results more carefully. Expect a follow-up from me in a day >> or two. >> >> 2. valgrind did not show anything super promising. It did report this: >> >> ==6880== LEAK SUMMARY: >> ==6880== definitely lost: 0 bytes in 0 blocks >> ==6880== indirectly lost: 0 bytes in 0 blocks >> ==6880== possibly lost: 288 bytes in 1 blocks >> ==6880== still reachable: 10,552 bytes in 39 blocks >> ==6880== suppressed: 0 bytes in 0 blocks >> (Bitcoin Core commit d78a880) >> >> and this: >> ==6778== LEAK SUMMARY: >> ==6778== definitely lost: 0 bytes in 0 blocks >> ==6778== indirectly lost: 0 bytes in 0 blocks >> ==6778== possibly lost: 320 bytes in 1 blocks >> ==6778== still reachable: 10,080 bytes in 32 blocks >> ==6778== suppressed: 0 bytes in 0 blocks >> (Bitcoin XT commit fe446d) >> >> I haven't found anything in there yet that I think would produce the >> multi-GB memory usage after running for a few days, but I could be missing >> it. Email me if you want the full log. >> >> I did not try running getblocktemplate while valgrind was running. I'll >> have to try that. I also have not let valgrind run for more than an hour. >> >> >> >> P.S.: Sorry for all the cross-post confusion and consequent flamewar >> fallout. While it's probably too late for this thread, I'll make sure to >> post in a manner that keeps the threads clearly separate in the future >> (e.g. different subject lines). >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >>