From: Rusty Russell <rusty@rustcorp•com.au>
To: Pieter Wuille <pieter.wuille@gmail•com>,
Bitcoin Protocol Discussion
<bitcoin-dev@lists•linuxfoundation.org>,
bitcoin-dev@lists•linuxfoundation.org
Subject: Re: [bitcoin-dev] Compact Block Relay BIP
Date: Tue, 10 May 2016 14:58:19 +0930 [thread overview]
Message-ID: <87twi6zdl8.fsf@rustcorp.com.au> (raw)
In-Reply-To: <5730C37E.2000004@gmail.com>
Pieter Wuille via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> writes:
> On 05/03/2016 12:13 AM, lf-lists at mattcorallo.com (Matt Corallo) wrote:
>> Hi all,
>>
>> The following is a BIP-formatted design spec for compact block relay
>> designed to limit on wire bytes during block relay. You can find the
>> latest version of this document at
>> https://github.com/TheBlueMatt/bips/blob/master/bip-TODO.mediawiki.
>
> Hi Matt,
>
> thank you for working on this!
Indeed! Sorry for the delayed feedback.
>> |shortids||List of uint64_ts||8*shortids_length bytes||Little
>> Endian||The short transaction IDs calculated from the transactions which
>> were not provided explicitly in prefilledtxn
>
> I tried to derive what length of short ids is actually necessary (some
> write-up is on
> https://gist.github.com/sipa/b2eb2e486156b5509ac711edd16153ed but it's
> incomplete).
I did this for IBLT testing.
I used variable-length bit encodings, and used the shortest encoding
which is unique to you (including mempool). It's a little more work,
but for an average node transmitting a block with 1300 txs and another
~3000 in the mempool, you expect about 12 bits per transaction. IOW,
about 1/5 of your current size. Critically, we might be able to fit in
two or three TCP packets.
The wire encoding of all those bit arrays was:
[varint-min-numbits] - Shortest bit array length
[varint-array-size] - Number of bit arrays.
[varint-num].... - Number of entries in array N (x varint-array-size)
[packed-bit-arrays...]
Last byte was padded with zeros.
See: https://github.com/rustyrussell/bitcoin-iblt/blob/master/wire_encode.cpp#L12
I would also avoid the nonce to save recalculating for each node, and
instead define an id as:
[<64-bit-short-id>][txid]
Since you only ever send as many bits as needed to distinguish, this only
makes a difference if there actually are collisions.
As Peter R points out, we could later enhance receiver to brute force
collisions (you could speed that by sending a XOR of all the txids, but
really if there are more than a few collisions, give up).
And a prototype could just always send 64-bit ids to start.
Cheers,
Rusty.
next prev parent reply other threads:[~2016-05-10 5:39 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-02 22:13 Matt Corallo
2016-05-03 5:02 ` Gregory Maxwell
2016-05-06 3:09 ` Matt Corallo
2016-05-08 0:40 ` Johnathan Corgan
2016-05-08 3:24 ` Matt Corallo
2016-05-09 9:35 ` Tom Zander
2016-05-09 10:43 ` Gregory Maxwell
2016-05-09 11:32 ` Tom
[not found] ` <CAAS2fgR01=SfpAdHhFd_DFa9VNiL=e1g4FiguVRywVVSqFe9rA@mail.gmail.com>
2016-05-09 12:12 ` [bitcoin-dev] Fwd: " Gregory Maxwell
2016-05-09 23:37 ` [bitcoin-dev] " Peter R
2016-05-10 1:42 ` Peter R
2016-05-10 2:12 ` Gregory Maxwell
2016-05-09 13:40 ` Peter Todd
2016-05-09 13:57 ` Tom
2016-05-09 14:04 ` Bryan Bishop
2016-05-09 17:06 ` Pieter Wuille
2016-05-09 18:34 ` Peter R
2016-05-10 5:28 ` Rusty Russell [this message]
2016-05-10 10:07 ` Gregory Maxwell
2016-05-10 21:23 ` Rusty Russell
2016-05-11 1:12 ` Matt Corallo
2016-05-18 1:49 ` Matt Corallo
2016-05-08 10:25 Nicolas Dorier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87twi6zdl8.fsf@rustcorp.com.au \
--to=rusty@rustcorp$(echo .)com.au \
--cc=bitcoin-dev@lists$(echo .)linuxfoundation.org \
--cc=pieter.wuille@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox