Subject: Re: [Bitcoin-development] BIP70/71 issue, RFD

From: Gavin <[email-masked]>
Date: Sun, 26 Jan 2014 21:05:31 -0500

Message encoding and length (or terminator or checksum or error correction or...) should be part of the transport protocol, in my humble opinion.

--
Gavin Andresen

> On Jan 26, 2014, at 6:01 PM, Mike Hearn <[email-masked]> wrote:
> 
> To be more accurate, the embedded messages already have length prefixes :-)
> 
> 
>> On Sun, Jan 26, 2014 at 11:14 PM, Andreas Schildbach <[email-masked]> wrote:
>> Just keep in mind that if it turns out we need to retrofit messages with
>> length prefixes I fear we will have to restart with new mime-types. The
>> nature of prefixes being always at the start means this rules out simply
>> incrementing a protocol version number.
>> 
>> Correcting myself:
>> 
>> > it might be a good idea to apply this to all messages if any.
>> 
>> Of course I only mean the "root" messages, namely PaymentRequest,
>> Payment, and PaymentACK. The embedded messages don't need length prefixes.
>> 
>> 
>> On 01/26/2014 11:00 PM, Mike Hearn wrote:
>> > I think for "binding" the payment protocol to those transports we should
>> > indeed use protobuf varint length prefixes. But it's unnecessary for all
>> > cases. Unless Gavin feels it'd be better to be consistent everywhere and
>> > is willing to change the spec and code - as far as I know though we're
>> > trying to ship 0.9rc1 soonish .....
>> >
>> >
>> > On Sun, Jan 26, 2014 at 10:32 PM, Andreas Schildbach
>> > <[email-masked] <mailto:[email-masked]>> wrote:
>> >
>> >     Bluetooth, Wifi Direct, HTTP request/responses via broken proxies, smoke
>> >     signals... basically anything that is a stream rather than a file.
>> >
>> >     Right, NFC NDEF and QR codes are not affected, so we can skip the
>> >     delimiter for these mediums.
>> >
>> >
>> >     On 01/26/2014 10:24 PM, Mike Hearn wrote:
>> >     > Which medium is this an issue for? As you note, for files and HTTP
>> >     > responses it's not a problem in practice. i'd guess nor for NFC
>> >     tags nor
>> >     > QR codes.
>> >     >
>> >     >
>> >     > On Sun, Jan 26, 2014 at 10:11 PM, Andreas Schildbach
>> >     > <[email-masked] <mailto:[email-masked]>
>> >     <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>> >     >
>> >     >     I'm experimenting with BIP70/71 (payment protocol) usage in
>> >     face to face
>> >     >     payments (more on that soon).
>> >     >
>> >     >     I've excountered an issue with the protobuf format. Protobufs
>> >     are not
>> >     >     self-delimiting. That means if you're reading from an undelimited
>> >     >     stream, you will read endlessly because you don't know how
>> >     much to read.
>> >     >
>> >     >     The current BIP70 implementations probably work because
>> >     they're reading
>> >     >     either from a file or from an HTTP resource which sets the
>> >     >     Content-Length header. Trouble is the Content-Length header is
>> >     optional,
>> >     >     and also there are many kinds of streams that don't have this
>> >     built-in
>> >     >     delimiting mechanism.
>> >     >
>> >     >     The Java protobuf API solves this by offering delimited I/O, like
>> >     >
>> >     >     payment.writeDelimitedTo(os);
>> >     >
>> >     >     This writes the size of the message as a varint before writing
>> >     the data.
>> >     >     I don't know about protobuf implementations for other
>> >     languages but I'd
>> >     >     expect them to offer something compatible.
>> >     >
>> >     >     However, this leading varint is an incompatible change and
>> >     would need to
>> >     >     be added to the spec.
>> >     >
>> >     >     I specifically encountered this with PaymentMessage and
>> >     PaymentACK, but
>> >     >     it might be a good idea to apply this to all messages if any.
>> >     Open for
>> >     >     discussion.
>> >     >
>> >     >
>> >     >
>> >     ------------------------------------------------------------------------------
>> >     >     CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> >     >     Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> >     >     Critical Workloads, Development Environments & Everything In
>> >     Between.
>> >     >     Get a Quote or Start a Free Trial Today.
>> >     >
>> >     http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>> >     >     _______________________________________________
>> >     >     Bitcoin-development mailing list
>> >     >     [email-masked]
>> >     <mailto:[email-masked]>
>> >     >     <mailto:[email-masked]
>> >     <mailto:[email-masked]>>
>> >     >     https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>> >     >
>> >     >
>> >     >
>> >     >
>> >     >
>> >     ------------------------------------------------------------------------------
>> >     > CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> >     > Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> >     > Critical Workloads, Development Environments & Everything In Between.
>> >     > Get a Quote or Start a Free Trial Today.
>> >     >
>> >     http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>> >     >
>> >     >
>> >     >
>> >     > _______________________________________________
>> >     > Bitcoin-development mailing list
>> >     > [email-masked]
>> >     <mailto:[email-masked]>
>> >     > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>> >     >
>> >
>> >
>> >
>> >     ------------------------------------------------------------------------------
>> >     CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> >     Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> >     Critical Workloads, Development Environments & Everything In Between.
>> >     Get a Quote or Start a Free Trial Today.
>> >     http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>> >     _______________________________________________
>> >     Bitcoin-development mailing list
>> >     [email-masked]
>> >     <mailto:[email-masked]>
>> >     https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>> >
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> > Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> > Critical Workloads, Development Environments & Everything In Between.
>> > Get a Quote or Start a Free Trial Today.
>> > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>> >
>> >
>> >
>> > _______________________________________________
>> > Bitcoin-development mailing list
>> > [email-masked]
>> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>> >
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>> Learn Why More Businesses Are Choosing CenturyLink Cloud For
>> Critical Workloads, Development Environments & Everything In Between.
>> Get a Quote or Start a Free Trial Today.
>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Bitcoin-development mailing list
>> [email-masked]
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 
> ------------------------------------------------------------------------------
> CenturyLink Cloud: The Leader in Enterprise Cloud Services.
> Learn Why More Businesses Are Choosing CenturyLink Cloud For
> Critical Workloads, Development Environments & Everything In Between.
> Get a Quote or Start a Free Trial Today.
> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> [email-masked]
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development