From: Matthew Mitchell <matthewmitchell@godofgod•co.uk>
To: Pieter Wuille <pieter.wuille@gmail•com>,
"bitcoin-development@lists•sourceforge.net"
<bitcoin-development@lists•sourceforge.net>
Subject: Re: [Bitcoin-development] Segmented Block Relaying BIP draft.
Date: Thu, 13 Sep 2012 21:24:35 +0100 [thread overview]
Message-ID: <9F7EFD92-B922-45E9-97A8-03FFC811502D@godofgod.co.uk> (raw)
In-Reply-To: <20120913185900.GA393@vps7135.xlshosting.net>
On 13 Sep 2012, at 19:59, Pieter Wuille <pieter.wuille@gmail•com> wrote:
> You want to parallellize block downloads, while at the same time preventing re-download of transactions that are already known.
> To do so, a requesting node would first request (for example) the 8 level-3 hashes, then start 8 parallel threads to download the
> transactions from presumably 8 different peers. Each thread then fetches the transaction id's that correspond to its own 1/8th of
> the block, and requests the transactions whose txid is not yet known.
> Comparing this with Gregory's own suggestion (just fetch the entire txid list at first, and then (again as parallellized as needed)
> fetch the unknown transactions from several peers), this does indeed have an advantage: you need to download (relatively) far less
> data before the threaded part can start. If this is what you propose, it is certainly meaningful, but the gains aren't very large,
> in my opinion.
This is not fully correct. I propose downloading the roots of the segments when you are not looking to remove redundant transaction downloads. This would be the case when the node is not up-to-date and is unlikely to have transactions in the required blocks. When a node is up-to-date and can benefit from removing redundant transaction downloads it can switch to downloading all the transactions hashes by specifying a high level number. Also I wouldn't recommend using one thread per connection, I'd recommend using one thread for all connections.
> However, my impression while reading your proposal was at times that you intend to process the different layers of the
> Merkle tree iteratively after starting the initial parallel segments. I don't think that is useful, as you'll need the actual
> txids anyway before deciding whether they need to be downloaded at all, it adds several round-trips, and once you have downloaded
> the intermediate merkle hashes for about 8 levels, you've already transferred more data than the transactions themselves. I think
> Gregory also assumed something like this, making him question why it's useful. Anyway, this whole paragraph is one assumption, so
> if it's not the case, ignore me.
This isn't what I was suggesting. I was suggesting you only need to download one level. Once you have done that you verify everything for the hashes on that level.
>
> Can you clarify what you mean? Preferably by giving a concrete sequence of exchanged messages, with a given purpose?
Well you will need to ask for the headers first. Once you do that you can start downloading the full blocks for the headers. The node should use "get blocks" to find nodes with segments of the blocks they need. Now for each block:
1. Send getsiginv to a number of peers to know the segments of the blocks they have.
2. Send gettreelevel requesting a level of the merkle tree from a peer that can provide it. When up-to-date use a high level to get the transaction hashes to find redundant data.
3. Validate the treelevel response
4. Send getsegment for each segment wanted (at the same time where possible) to the peers that have these segments. Skip transactions already known.
5. Validate the transactions in each segment received. Stop if the block is invalid and disconnect peers that give transactions which do not fit the merkle tree.
6. Revert to getdata if peers using the protocol cannot satisfy the block download.
When a valid block segment is received, include the block in inv and headers messages for other peers using the protocol. Thus relaying can begin before the entire block is downloaded.
I'm thinking about improvements to this proposal. I'll get to that tomorrow perhaps…
Thank you everyone for the replies.
next prev parent reply other threads:[~2012-09-13 20:24 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-09-11 19:07 Matthew Mitchell
2012-09-11 19:42 ` Gregory Maxwell
2012-09-11 21:48 ` Matthew Mitchell
2012-09-11 23:22 ` Gregory Maxwell
2012-09-13 8:42 ` Mike Hearn
2012-09-13 14:05 ` Matthew Mitchell
2012-09-13 15:16 ` Gregory Maxwell
[not found] ` <2B95CF41-4304-4D2A-9ABF-198D97B7449B@godofgod.co.uk>
2012-09-13 15:46 ` Matthew Mitchell
[not found] ` <CAAS2fgQi8QFwU2M=wLiDodt3SmO48vUV5Sp3YCb1OmGJ5m=E7Q@mail.gmail.com>
2012-09-13 17:49 ` Matthew Mitchell
2012-09-13 18:59 ` Pieter Wuille
2012-09-13 20:24 ` Matthew Mitchell [this message]
-- strict thread matches above, loose matches on Subject: below --
2012-09-10 15:07 Matthew Mitchell
2012-09-10 15:14 ` Gregory Maxwell
2012-09-10 16:29 ` Matt Corallo
2012-09-10 18:59 ` Luke-Jr
2012-09-10 19:34 ` Matthew Mitchell
2012-09-10 19:53 ` Matt Corallo
2012-09-10 20:00 ` Gregory Maxwell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9F7EFD92-B922-45E9-97A8-03FFC811502D@godofgod.co.uk \
--to=matthewmitchell@godofgod$(echo .)co.uk \
--cc=bitcoin-development@lists$(echo .)sourceforge.net \
--cc=pieter.wuille@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox