public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
* [Bitcoin-development] Implementing batch processing for -blocknotify
@ 2013-05-31 11:56 Rune Kjær Svendsen
  2013-05-31 12:10 ` Michael Hendricks
                   ` (3 more replies)
  0 siblings, 4 replies; 10+ messages in thread
From: Rune Kjær Svendsen @ 2013-05-31 11:56 UTC (permalink / raw)
  To: bitcoin-development

[-- Attachment #1: Type: text/plain, Size: 2991 bytes --]

Hello dear list

I have an application that wants to keep up with new blocks as they come
in. For that I can use the -blocknotify option with bitcoind, which will
execute my application for each new block.

The problem is that my app isn't necessarily quick enough to finish its
work before a new block comes in and the app is executed again. This means
the that bitcoind might keep executing my application even though the
previous instance hasn't finished, and that's fairly inefficient
resource-wise, as many instances of the application will be running
simultaneously.

I've discussed this with wumpus on bitcoin-dev, and we figured out a
solution that might be better. It could replace -blocknotify or we could
put it in a new function called -batchblocknotify

The idea is that when bitcoind is executed with the -batchblocknotify
option, and it's missing a lot of blocks, upon the first incoming block,
the command specified by -batchblocknotify is executed, and if additional
blocks come in while this command is still running, we add the block hashes
to a list instead of executing the command again. When the previous command
finishes, we execute it again and pass two parameters to it: 1. the first
block hash in the list of queued blocks, and 2. the number of blocks that
have come in while the last command was executing.

This prevents bitcoind from "fork bombing" the system, and allows the
command to handle incoming blocks in batches.

Would this make sense as an approach?

I've been looking at the code and I'm not sure how to implement it.

As far as I can see, I need to pass an object - whose state is retained
between calls - to the thread function (runCommand) that runs the command,
which contains a variable that keeps track of whether a previously executed
command is still running, and that contains a list of block hashes that
haven't been processed. And I'm not sure how to do this.

The runCommand thread is started in SetBestChain() in
main.cpp. SetBestChain() is executed by ConnectBestBlock() in main.cpp.
ConnectBestBlock() is executed by CBlock::AddToBlockIndex() in main.cpp.
CBlock::AddToBlockIndex() is executed by CBlock::AcceptBlock() in main.cpp.
CBlock::AcceptBlock() is executed by ProcessBlock() in main.cpp.
ProcessBlock() is executed by ProcessMessage() in main.cpp. And so on, and
so forth.

What's the right way to create an object that can be passed to the
runCommand thread, whose state is retained, so it can hold information
about whether the -batchblocknotify command is still executing, and contain
a list of blocks that are waiting to be passed to the -batchblocknotify
command?

I assume I shouldn't add a new parameter to the ProcessMessage() function,
which passes it to ProcessBlock(), which passes it to AcceptBlock() which
passes it to AddToBlockIndex()... and so on.

Would it be appropriate to store this object inside the CValidationState
class that is passed to SetBestChain()?

I'm not quite so how to go about this.

/Rune

[-- Attachment #2: Type: text/html, Size: 3449 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-06-01 13:12 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-05-31 11:56 [Bitcoin-development] Implementing batch processing for -blocknotify Rune Kjær Svendsen
2013-05-31 12:10 ` Michael Hendricks
2013-05-31 12:37   ` Rune Kjær Svendsen
2013-05-31 19:25     ` Jeff Garzik
2013-05-31 12:54 ` Andy Parkins
2013-05-31 13:05 ` Jeff Garzik
2013-05-31 22:20 ` Chris Double
2013-05-31 23:29   ` Wladimir
2013-05-31 23:47     ` Chris Double
2013-06-01 13:12       ` Rune Kjær Svendsen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox