From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpZKI-0000Zp-DT for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 07:30:38 +0000 X-ACL-Warn: Received: from wp059.webpack.hosteurope.de ([80.237.132.66]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1UpZKG-0001f1-Iv for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 07:30:38 +0000 Received: from [37.143.74.116] (helo=[192.168.0.102]); authenticated by wp059.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) id 1UpZKA-0005PW-1Q; Thu, 20 Jun 2013 09:30:30 +0200 From: Tamas Blummer Content-Type: multipart/alternative; boundary="Apple-Mail=_82274919-95DF-485D-9943-ED393C3F9E3E" Message-Id: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> Date: Thu, 20 Jun 2013 09:30:29 +0200 To: bitcoin-development@lists.sourceforge.net Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-bounce-key: webpack.hosteurope.de; tamas@bitsofproof.com; 1371713436; df345564; X-Spam-Score: 1.0 (+) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 1.0 HTML_MESSAGE BODY: HTML included in message X-Headers-End: 1UpZKG-0001f1-Iv Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 07:30:38 -0000 --Apple-Mail=_82274919-95DF-485D-9943-ED393C3F9E3E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi Mike, The issue with the current parser is that those fields are conditionally = optional on that there will be no subsequent fields added. If there will be further fields they will become manadory.=20 =20 Why not bump the version and parse the fields as mandatory from then on? = This would be backward compatible and cleaner going forward. Tamas Blummer http://bitsofproof.com --Apple-Mail=_82274919-95DF-485D-9943-ED393C3F9E3E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
Hi Mike,

The issue with the current parser is that those = fields are conditionally optional on that there will be no subsequent = fields added.
If there will be = further fields they will become manadory. 
 
Why not bump the version and parse the fields as = mandatory from then on? This would be backward compatible and = cleaner
going = forward.

Tamas Blummer

= --Apple-Mail=_82274919-95DF-485D-9943-ED393C3F9E3E-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpZQF-0000xO-Ci for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 07:36:47 +0000 Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.214.171 as permitted sender) client-ip=209.85.214.171; envelope-from=mh.in.england@gmail.com; helo=mail-ob0-f171.google.com; Received: from mail-ob0-f171.google.com ([209.85.214.171]) by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1UpZQE-0000oT-BC for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 07:36:47 +0000 Received: by mail-ob0-f171.google.com with SMTP id dn14so6891725obc.30 for ; Thu, 20 Jun 2013 00:36:41 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.182.215.133 with SMTP id oi5mr978701obc.83.1371713800875; Thu, 20 Jun 2013 00:36:40 -0700 (PDT) Sender: mh.in.england@gmail.com Received: by 10.76.23.36 with HTTP; Thu, 20 Jun 2013 00:36:40 -0700 (PDT) In-Reply-To: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> Date: Thu, 20 Jun 2013 09:36:40 +0200 X-Google-Sender-Auth: 1Y-hWUGdWNhuEKvPpMcGlNS1TjE Message-ID: From: Mike Hearn To: Tamas Blummer Content-Type: multipart/alternative; boundary=001a11c25434d5bbee04df9101d8 X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1UpZQE-0000oT-BC Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 07:36:47 -0000 --001a11c25434d5bbee04df9101d8 Content-Type: text/plain; charset=UTF-8 Sure but why not do that when there's an actual new field to add? Does anyone have a proposal for a feature that needs a new version field at the moment? There's no point changing the protocol now unless there's actually a new field to add. Anyway I still don't see why anyone cares about this issue. The Bitcoin protocol does not and never has required that all messages have a fixed number of fields per version. Any parser written on the assumption it did was just buggy. Look at how tx messages are relayed for the most obvious example of that pattern in action - it's actually the raw byte stream that's stored and relayed to ensure that fields added in new versions aren't dropped during round-tripping. Old versions are supposed to preserve fields from the future. On Thu, Jun 20, 2013 at 9:30 AM, Tamas Blummer wrote: > Hi Mike, > > The issue with the current parser is that those fields are conditionally > optional on that there will be no subsequent fields added. > If there will be further fields they will become manadory. > > Why not bump the version and parse the fields as mandatory from then on? > This would be backward compatible and cleaner > going forward. > > Tamas Blummer > http://bitsofproof.com > > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > --001a11c25434d5bbee04df9101d8 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Sure but why not do that when there's an actual new fi= eld to add? Does anyone have a proposal for a feature that needs a new vers= ion field at the moment? There's no point changing the protocol now unl= ess there's actually a new field to add.

Anyway I still don't see why anyone cares about th= is issue. The Bitcoin protocol does not and never has required that all mes= sages have a fixed number of fields per version. Any parser written on the = assumption it did was just buggy. Look at how tx messages are relayed for t= he most obvious example of that pattern in action - it's actually the r= aw byte stream that's stored and relayed to ensure that fields added in= new versions aren't dropped during round-tripping. Old versions are su= pposed to preserve fields from the future.



On Thu, Jun 20, 2013 at 9:30 AM, Tamas Blummer <tamas@bi= tsofproof.com> wrote:
Hi Mike,

The issue with the current parser is that those fields are conditionally op= tional on that there will be no subsequent fields added.
=C2=A0
Why not bump the version and parse the fields as mandatory=C2=A0from then o= n? This would be backward compatible and cleaner
going forward.

Tamas Blummer


---------------------------------------------------= ---------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.= sf.net/sfu/windows-dev2dev
_________________________________________= ______
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment


--001a11c25434d5bbee04df9101d8-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Upa3N-0002yS-Ho for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 08:17:13 +0000 X-ACL-Warn: Received: from wp059.webpack.hosteurope.de ([80.237.132.66]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1Upa3L-0003bl-LC for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 08:17:13 +0000 Received: from [37.143.74.116] (helo=[192.168.1.5]); authenticated by wp059.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) id 1Upa3E-0002Z8-V1; Thu, 20 Jun 2013 10:17:05 +0200 Content-Type: multipart/alternative; boundary="Apple-Mail=_89C91DFB-D4A3-4955-A2B1-97BF686F7F4F" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) From: Tamas Blummer In-Reply-To: Date: Thu, 20 Jun 2013 10:17:04 +0200 Message-Id: References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> To: Mike Hearn X-Mailer: Apple Mail (2.1503) X-bounce-key: webpack.hosteurope.de; tamas@bitsofproof.com; 1371716231; c85edd7f; X-Spam-Score: 1.0 (+) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 1.0 HTML_MESSAGE BODY: HTML included in message X-Headers-End: 1Upa3L-0003bl-LC Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 08:17:14 -0000 --Apple-Mail=_89C91DFB-D4A3-4955-A2B1-97BF686F7F4F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 I agree that this can be deferred until there is an actual new field = without any harm. But then remember to update the BIP37 too saying that = it is optional only if flag added in BIPXX is not present. Your argument is that this complexity is already there so why not = preserve it. I think eliminating complexity (that has no benefit) = strengthens the system. Tam=E1s Blummer http://bitsofproof.com On 20.06.2013, at 09:36, Mike Hearn wrote: > Sure but why not do that when there's an actual new field to add? Does = anyone have a proposal for a feature that needs a new version field at = the moment? There's no point changing the protocol now unless there's = actually a new field to add. >=20 > Anyway I still don't see why anyone cares about this issue. The = Bitcoin protocol does not and never has required that all messages have = a fixed number of fields per version. Any parser written on the = assumption it did was just buggy. Look at how tx messages are relayed = for the most obvious example of that pattern in action - it's actually = the raw byte stream that's stored and relayed to ensure that fields = added in new versions aren't dropped during round-tripping. Old versions = are supposed to preserve fields from the future. >=20 >=20 >=20 > On Thu, Jun 20, 2013 at 9:30 AM, Tamas Blummer = wrote: > Hi Mike, >=20 > The issue with the current parser is that those fields are = conditionally optional on that there will be no subsequent fields added. > If there will be further fields they will become manadory.=20 > =20 > Why not bump the version and parse the fields as mandatory from then = on? This would be backward compatible and cleaner > going forward. >=20 > Tamas Blummer > http://bitsofproof.com >=20 >=20 > = --------------------------------------------------------------------------= ---- > This SF.net email is sponsored by Windows: >=20 > Build for Windows Store. >=20 > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >=20 >=20 --Apple-Mail=_89C91DFB-D4A3-4955-A2B1-97BF686F7F4F Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1
I agree that this can be deferred until there is an = actual new field without any harm. But then remember to update the BIP37 = too saying that it is optional only if flag added in BIPXX is not = present.

Your argument is that this complexity is already = there so why not preserve it. I think eliminating complexity (that = has no benefit) strengthens the system.

http://bitsofproof.com

On 20.06.2013, at 09:36, Mike Hearn <mike@plan99.net> wrote:

Sure but why not do that when there's an actual new field to = add? Does anyone have a proposal for a feature that needs a new version = field at the moment? There's no point changing the protocol now unless = there's actually a new field to add.

Anyway I still don't see why anyone cares = about this issue. The Bitcoin protocol does not and never has required = that all messages have a fixed number of fields per version. Any parser = written on the assumption it did was just buggy. Look at how tx messages = are relayed for the most obvious example of that pattern in action - = it's actually the raw byte stream that's stored and relayed to ensure = that fields added in new versions aren't dropped during round-tripping. = Old versions are supposed to preserve fields from the future.



On Thu, Jun 20, 2013 at 9:30 AM, Tamas Blummer = <tamas@bitsofproof.com> wrote:
Hi Mike,

The issue with the current parser is that those fields are conditionally = optional on that there will be no subsequent fields added.
If there will be further fields they will become = manadory. 
 
Why not bump the version and parse the fields as mandatory from = then on? This would be backward compatible and cleaner
going forward.

Tamas Blummer
=


--------------------------------------------------= ----------------------------
This SF.net email is sponsored by = Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_____________= __________________________________
Bitcoin-development mailing list
Bitcoin-developm= ent@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-dev= elopment



= --Apple-Mail=_89C91DFB-D4A3-4955-A2B1-97BF686F7F4F-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpaHN-0007EB-LK for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 08:31:41 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.219.46 as permitted sender) client-ip=209.85.219.46; envelope-from=mh.in.england@gmail.com; helo=mail-oa0-f46.google.com; Received: from mail-oa0-f46.google.com ([209.85.219.46]) by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1UpaHL-0005oO-Um for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 08:31:41 +0000 Received: by mail-oa0-f46.google.com with SMTP id h1so7623432oag.5 for ; Thu, 20 Jun 2013 01:31:34 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.145.167 with SMTP id sv7mr4080490oeb.56.1371717094452; Thu, 20 Jun 2013 01:31:34 -0700 (PDT) Sender: mh.in.england@gmail.com Received: by 10.76.23.36 with HTTP; Thu, 20 Jun 2013 01:31:34 -0700 (PDT) In-Reply-To: References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> Date: Thu, 20 Jun 2013 10:31:34 +0200 X-Google-Sender-Auth: I-Ue0Un7hYQ3PbIYVct5rBN6xeo Message-ID: From: Mike Hearn To: Tamas Blummer Content-Type: multipart/alternative; boundary=047d7b5d94fb25b90504df91c6f4 X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1UpaHL-0005oO-Um Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 08:31:41 -0000 --047d7b5d94fb25b90504df91c6f4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable You can't eliminate the complexity (yet), otherwise you wouldn't be able to talk to old nodes. You'll have to wait until versions prior to a particular version are hard-forked off and can be safely dropped at connect time. That said the reason I'm being so grumpy about this is that compared to the complexity in the rest of the system, this is such a trivial and minor detail. It's hardly even worth thinking about. I mean, we have a scripting language full of opcodes nobody ever figured out how to use and the protocol uses a mixture of byte orders, so an optional field in the version message is really not such a big deal :) On Thu, Jun 20, 2013 at 10:17 AM, Tamas Blummer wrot= e: > I agree that this can be deferred until there is an actual new field > without any harm. But then remember to update the BIP37 too saying that i= t > is optional only if flag added in BIPXX is not present. > > Your argument is that this complexity is already there so why not preserv= e > it. I think eliminating complexity (that has no benefit) strengthens the > system. > > *Tam=C3=A1s Blummer* > http://bitsofproof.com > > > On 20.06.2013, at 09:36, Mike Hearn wrote: > > Sure but why not do that when there's an actual new field to add? Does > anyone have a proposal for a feature that needs a new version field at th= e > moment? There's no point changing the protocol now unless there's actuall= y > a new field to add. > > Anyway I still don't see why anyone cares about this issue. The Bitcoin > protocol does not and never has required that all messages have a fixed > number of fields per version. Any parser written on the assumption it did > was just buggy. Look at how tx messages are relayed for the most obvious > example of that pattern in action - it's actually the raw byte stream > that's stored and relayed to ensure that fields added in new versions > aren't dropped during round-tripping. Old versions are supposed to preser= ve > fields from the future. > > > > On Thu, Jun 20, 2013 at 9:30 AM, Tamas Blummer wro= te: > >> Hi Mike, >> >> The issue with the current parser is that those fields are conditionally >> optional on that there will be no subsequent fields added. >> If there will be further fields they will become manadory. >> >> Why not bump the version and parse the fields as mandatory from then on? >> This would be backward compatible and cleaner >> going forward. >> >> Tamas Blummer >> http://bitsofproof.com >> >> >> >> >> ------------------------------------------------------------------------= ------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> >> > > --047d7b5d94fb25b90504df91c6f4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
You can't eliminate the complexity (yet), otherwise yo= u wouldn't be able to talk to old nodes. You'll have to wait until = versions prior to a particular version are hard-forked off and can be safel= y dropped at connect time.

That said the reason I'm being so grumpy about this is t= hat compared to the complexity in the rest of the system, this is such a tr= ivial and minor detail. It's hardly even worth thinking about. I mean, = we have a scripting language full of opcodes nobody ever figured out how to= use and the protocol uses a mixture of byte orders, so an optional field i= n the version message is really not such a big deal :)


On Thu,= Jun 20, 2013 at 10:17 AM, Tamas Blummer <tamas@bitsofproof.com>= ; wrote:
I agree that this can be deferred until there is an actual new field withou= t any harm. But then remember to update the BIP37 too saying that it is opt= ional only if flag added in BIPXX is not present.

Your argument is that this complexity is already there so why not preserve = it. I think eliminating=C2=A0complexity (that has no benefit) strengthens t= he system.

Tam=C3=A1s Blummer

On 20.06.2013, at 09:36, Mik= e Hearn <mike@plan9= 9.net> wrote:

Su= re but why not do that when there's an actual new field to add? Does an= yone have a proposal for a feature that needs a new version field at the mo= ment? There's no point changing the protocol now unless there's act= ually a new field to add.

Anyway I still don't see why anyone cares about this iss= ue. The Bitcoin protocol does not and never has required that all messages = have a fixed number of fields per version. Any parser written on the assump= tion it did was just buggy. Look at how tx messages are relayed for the mos= t obvious example of that pattern in action - it's actually the raw byt= e stream that's stored and relayed to ensure that fields added in new v= ersions aren't dropped during round-tripping. Old versions are supposed= to preserve fields from the future.



On Thu, Jun 20, 2013 at 9:30 AM, Tamas Blummer &l= t;tamas@bitsofpr= oof.com> wrote:
Hi Mike,

The issue with the current parser is that those fields are conditionally op= tional on that there will be no subsequent fields added.
=C2=A0
Why not bump the version and parse the fields as mandatory=C2=A0from then o= n? This would be backward compatible and cleaner
going forward.

Tamas Blummer


---------------------------------------------------= ---------------------------
This SF.net email is sponso= red by Windows:

Build for Windows Store.

http://p.= sf.net/sfu/windows-dev2dev
_________________________________________= ______
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment




--047d7b5d94fb25b90504df91c6f4-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpaOl-000603-BT for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 08:39:19 +0000 X-ACL-Warn: Received: from wp059.webpack.hosteurope.de ([80.237.132.66]) by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1UpaOj-0006D1-Ek for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 08:39:19 +0000 Received: from [37.143.74.116] (helo=[192.168.1.5]); authenticated by wp059.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) id 1UpaOc-0000nF-L3; Thu, 20 Jun 2013 10:39:10 +0200 Content-Type: multipart/alternative; boundary="Apple-Mail=_4A2B3B2A-66B7-49FC-947F-47A201BD0F89" Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) From: Tamas Blummer In-Reply-To: Date: Thu, 20 Jun 2013 10:39:10 +0200 Message-Id: <84938F7F-B75C-434B-9B7A-EC406CE42385@bitsofproof.com> References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> To: Mike Hearn X-Mailer: Apple Mail (2.1503) X-bounce-key: webpack.hosteurope.de; tamas@bitsofproof.com; 1371717557; 08621cee; X-Spam-Score: 1.0 (+) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 1.0 HTML_MESSAGE BODY: HTML included in message X-Headers-End: 1UpaOj-0006D1-Ek Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 08:39:19 -0000 --Apple-Mail=_4A2B3B2A-66B7-49FC-947F-47A201BD0F89 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Yes it is trivial. I do not think greater complexity in the system = should keep us from addressing low complexity issues. You can't blame me or others not trying to simplify scripts, if there is = such a headwind simplifying a version message. You are right there is too much fuss about this. Tam=E1s Blummer Founder, CEO http://bitsofproof.com On 20.06.2013, at 10:31, Mike Hearn wrote: > You can't eliminate the complexity (yet), otherwise you wouldn't be = able to talk to old nodes. You'll have to wait until versions prior to a = particular version are hard-forked off and can be safely dropped at = connect time. >=20 > That said the reason I'm being so grumpy about this is that compared = to the complexity in the rest of the system, this is such a trivial and = minor detail. It's hardly even worth thinking about. I mean, we have a = scripting language full of opcodes nobody ever figured out how to use = and the protocol uses a mixture of byte orders, so an optional field in = the version message is really not such a big deal :) >=20 >=20 > On Thu, Jun 20, 2013 at 10:17 AM, Tamas Blummer = wrote: > I agree that this can be deferred until there is an actual new field = without any harm. But then remember to update the BIP37 too saying that = it is optional only if flag added in BIPXX is not present. >=20 > Your argument is that this complexity is already there so why not = preserve it. I think eliminating complexity (that has no benefit) = strengthens the system. >=20 > Tam=E1s Blummer > http://bitsofproof.com >=20 > On 20.06.2013, at 09:36, Mike Hearn wrote: >=20 >> Sure but why not do that when there's an actual new field to add? = Does anyone have a proposal for a feature that needs a new version field = at the moment? There's no point changing the protocol now unless there's = actually a new field to add. >>=20 >> Anyway I still don't see why anyone cares about this issue. The = Bitcoin protocol does not and never has required that all messages have = a fixed number of fields per version. Any parser written on the = assumption it did was just buggy. Look at how tx messages are relayed = for the most obvious example of that pattern in action - it's actually = the raw byte stream that's stored and relayed to ensure that fields = added in new versions aren't dropped during round-tripping. Old versions = are supposed to preserve fields from the future. >>=20 >>=20 >>=20 >> On Thu, Jun 20, 2013 at 9:30 AM, Tamas Blummer = wrote: >> Hi Mike, >>=20 >> The issue with the current parser is that those fields are = conditionally optional on that there will be no subsequent fields added. >> If there will be further fields they will become manadory.=20 >> =20 >> Why not bump the version and parse the fields as mandatory from then = on? This would be backward compatible and cleaner >> going forward. >>=20 >> Tamas Blummer >> http://bitsofproof.com >>=20 >>=20 >> = --------------------------------------------------------------------------= ---- >> This SF.net email is sponsored by Windows: >>=20 >> Build for Windows Store. >>=20 >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >>=20 >>=20 >=20 >=20 --Apple-Mail=_4A2B3B2A-66B7-49FC-947F-47A201BD0F89 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1
Yes it is trivial. I do not think greater complexity = in the system should keep us from addressing low complexity = issues.
You can't blame me = or others not trying to simplify scripts, if there is such a headwind = simplifying a version message.
You are right there is too much fuss about = this.

Tam=E1s Blummer
Founder, CEO
http://bitsofproof.com

On 20.06.2013, at 10:31, Mike Hearn <mike@plan99.net> wrote:

You can't eliminate the complexity (yet), otherwise you = wouldn't be able to talk to old nodes. You'll have to wait until = versions prior to a particular version are hard-forked off and can be = safely dropped at connect time.

That said the reason I'm being so grumpy about this is = that compared to the complexity in the rest of the system, this is such = a trivial and minor detail. It's hardly even worth thinking about. I = mean, we have a scripting language full of opcodes nobody ever figured = out how to use and the protocol uses a mixture of byte orders, so an = optional field in the version message is really not such a big deal = :)


On = Thu, Jun 20, 2013 at 10:17 AM, Tamas Blummer <tamas@bitsofproof.com> wrote:
I agree that this can be deferred until there is an actual new field = without any harm. But then remember to update the BIP37 too saying that = it is optional only if flag added in BIPXX is not present.

Your argument is that this complexity is already there so why not = preserve it. I think eliminating complexity (that has no benefit) = strengthens the system.

Tam=E1s Blummer

On 20.06.2013, at 09:36, = Mike Hearn <mike@plan99.net> wrote:

Sure but why not do that when there's an = actual new field to add? Does anyone have a proposal for a feature that = needs a new version field at the moment? There's no point changing the = protocol now unless there's actually a new field to add.

Anyway I still don't see why anyone cares about this = issue. The Bitcoin protocol does not and never has required that all = messages have a fixed number of fields per version. Any parser written = on the assumption it did was just buggy. Look at how tx messages are = relayed for the most obvious example of that pattern in action - it's = actually the raw byte stream that's stored and relayed to ensure that = fields added in new versions aren't dropped during round-tripping. Old = versions are supposed to preserve fields from the future.



On Thu, Jun 20, 2013 at 9:30 AM, Tamas Blummer = <tamas@bitsofproof.com> wrote:
Hi Mike,

The issue with the current parser is that those fields are conditionally = optional on that there will be no subsequent fields added.
If there will be further fields they will become = manadory. 
 
Why not bump the version and parse the fields as mandatory from = then on? This would be backward compatible and cleaner
going forward.

Tamas Blummer
=


--------------------------------------------------= ----------------------------
This SF.net email is = sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_____________= __________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-dev= elopment





= --Apple-Mail=_4A2B3B2A-66B7-49FC-947F-47A201BD0F89-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpapW-0000MT-7z for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 09:06:58 +0000 X-ACL-Warn: Received: from vps7135.xlshosting.net ([178.18.90.41]) by sog-mx-3.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1UpapU-0000LW-G1 for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 09:06:58 +0000 Received: by vps7135.xlshosting.net (Postfix, from userid 1000) id 615F7BCD02; Thu, 20 Jun 2013 11:06:50 +0200 (CEST) Date: Thu, 20 Jun 2013 11:06:50 +0200 From: Pieter Wuille To: Mike Hearn Message-ID: <20130620090649.GA17765@vps7135.xlshosting.net> References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-PGP-Key: http://sipa.ulyssis.org/pubkey.asc User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Score: -0.1 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (pieter.wuille[at]gmail.com) 0.0 DKIM_ADSP_CUSTOM_MED No valid author signature, adsp_override is CUSTOM_MED -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain 1.2 NML_ADSP_CUSTOM_MED ADSP custom_med hit, and not from a mailing list X-Headers-End: 1UpapU-0000LW-G1 Cc: Bitcoin Dev , Tamas Blummer Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 09:06:58 -0000 On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote: > Sure but why not do that when there's an actual new field to add? Does > anyone have a proposal for a feature that needs a new version field at the > moment? There's no point changing the protocol now unless there's actually > a new field to add. > > Anyway I still don't see why anyone cares about this issue. The Bitcoin > protocol does not and never has required that all messages have a fixed > number of fields per version. Any parser written on the assumption it did > was just buggy. Look at how tx messages are relayed for the most obvious > example of that pattern in action - it's actually the raw byte stream > that's stored and relayed to ensure that fields added in new versions > aren't dropped during round-tripping. Old versions are supposed to preserve > fields from the future. Actually, that is not the same issue. What is being argued for here is that the version in the version message itself should indicate which fields are present, so a parser doesn't need to look at the length of the message. That seems like a minor but very reasonable request to me, and it's trivial to do. That doesn't mean that you may receive versions higher than what you know of, and thus messages with fields you don't know about. That doesn't matter, you can just ignore them. I see no problem with raising the protocol version number to indicate "all fields up to fRelayTxes are required, if the announced nVersion is above N". In fact, I believe (though haven't checked) all previous additions to the version message were accompanied with a protocol version (then: client version) increase as well. -- Pieter From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Upaza-0007eK-2q for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 09:17:22 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.214.174 as permitted sender) client-ip=209.85.214.174; envelope-from=mh.in.england@gmail.com; helo=mail-ob0-f174.google.com; Received: from mail-ob0-f174.google.com ([209.85.214.174]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1UpazV-0006sw-5K for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 09:17:22 +0000 Received: by mail-ob0-f174.google.com with SMTP id wd20so7011710obb.33 for ; Thu, 20 Jun 2013 02:17:11 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.133.243 with SMTP id pf19mr89718oeb.118.1371719831741; Thu, 20 Jun 2013 02:17:11 -0700 (PDT) Sender: mh.in.england@gmail.com Received: by 10.76.23.36 with HTTP; Thu, 20 Jun 2013 02:17:11 -0700 (PDT) In-Reply-To: <20130620090649.GA17765@vps7135.xlshosting.net> References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> <20130620090649.GA17765@vps7135.xlshosting.net> Date: Thu, 20 Jun 2013 11:17:11 +0200 X-Google-Sender-Auth: m1YpdqqI24OSmS9JEZd_-PGGQHQ Message-ID: From: Mike Hearn To: Pieter Wuille Content-Type: multipart/alternative; boundary=047d7b47245a4d6d8d04df9269a7 X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1UpazV-0006sw-5K Cc: Bitcoin Dev , Tamas Blummer Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 09:17:22 -0000 --047d7b47245a4d6d8d04df9269a7 Content-Type: text/plain; charset=UTF-8 There's no problem, but there's no benefit either. It also locks us in to a potentially problematic guarantee - what if in future we want to have, say, two optional new pieces of data in two different messages. We don't want to require that if version > X then you have to implement all features up to and including that point. Essentially the number of fields in a message is like a little version number, just for that message. It adds flexibility to keep it that way, and there's no downside, seeing as that bridge was already crossed and people with parsers that can't handle it need to fix their code anyway. So I have a slight preference for keeping things the way they are, it keeps things flexible for future and costs nothing. On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille wrote: > On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote: > > Sure but why not do that when there's an actual new field to add? Does > > anyone have a proposal for a feature that needs a new version field at > the > > moment? There's no point changing the protocol now unless there's > actually > > a new field to add. > > > > Anyway I still don't see why anyone cares about this issue. The Bitcoin > > protocol does not and never has required that all messages have a fixed > > number of fields per version. Any parser written on the assumption it did > > was just buggy. Look at how tx messages are relayed for the most obvious > > example of that pattern in action - it's actually the raw byte stream > > that's stored and relayed to ensure that fields added in new versions > > aren't dropped during round-tripping. Old versions are supposed to > preserve > > fields from the future. > > Actually, that is not the same issue. What is being argued for here is that > the version in the version message itself should indicate which fields are > present, so a parser doesn't need to look at the length of the message. > That > seems like a minor but very reasonable request to me, and it's trivial to > do. > That doesn't mean that you may receive versions higher than what you know > of, > and thus messages with fields you don't know about. That doesn't matter, > you > can just ignore them. > > I see no problem with raising the protocol version number to indicate > "all fields up to fRelayTxes are required, if the announced nVersion is > above N". > In fact, I believe (though haven't checked) all previous additions to the > version > message were accompanied with a protocol version (then: client version) > increase > as well. > > -- > Pieter > > --047d7b47245a4d6d8d04df9269a7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
There's no problem, but there's no benefit either.= It also locks us in to a potentially problematic guarantee - what if in fu= ture we want to have, say, two optional new pieces of data in two different= messages. We don't want to require that if version > X then you hav= e to implement all features up to and including that point.

Essentially the number of fields in a message is like a litt= le version number, just for that message. It adds flexibility to keep it th= at way, and there's no downside, seeing as that bridge was already cros= sed and people with parsers that can't handle it need to fix their code= anyway.

So I have a slight preference for keeping things the wa= y they are, it keeps things flexible for future and costs nothing.



On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille <pieter.wuille@gmail= .com> wrote:
On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrot= e:
> Sure but why not do that when there's an actual new field to add? = Does
> anyone have a proposal for a feature that needs a new version field at= the
> moment? There's no point changing the protocol now unless there= 9;s actually
> a new field to add.
>
> Anyway I still don't see why anyone cares about this issue. The Bi= tcoin
> protocol does not and never has required that all messages have a fixe= d
> number of fields per version. Any parser written on the assumption it = did
> was just buggy. Look at how tx messages are relayed for the most obvio= us
> example of that pattern in action - it's actually the raw byte str= eam
> that's stored and relayed to ensure that fields added in new versi= ons
> aren't dropped during round-tripping. Old versions are supposed to= preserve
> fields from the future.

Actually, that is not the same issue. What is being argued for here i= s that
the version in the version message itself should indicate which fields are<= br> present, so a parser doesn't need to look at the length of the message.= That
seems like a minor but very reasonable request to me, and it's trivial = to do.
That doesn't mean that you may receive versions higher than what you kn= ow of,
and thus messages with fields you don't know about. That doesn't ma= tter, you
can just ignore them.

I see no problem with raising the protocol version number to indicate
"all fields up to fRelayTxes are required, if the announced nVersion i= s above N".
In fact, I believe (though haven't checked) all previous additions to t= he version
message were accompanied with a protocol version (then: client version) inc= rease
as well.

--
Pieter


--047d7b47245a4d6d8d04df9269a7-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpcEv-000355-IU for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 10:37:17 +0000 X-ACL-Warn: Received: from nm18-vm0.bullet.mail.bf1.yahoo.com ([98.139.213.138]) by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1UpcEq-0005Yp-CZ for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 10:37:17 +0000 Received: from [98.139.212.147] by nm18.bullet.mail.bf1.yahoo.com with NNFMP; 20 Jun 2013 10:37:05 -0000 Received: from [98.139.212.202] by tm4.bullet.mail.bf1.yahoo.com with NNFMP; 20 Jun 2013 10:37:05 -0000 Received: from [127.0.0.1] by omp1011.mail.bf1.yahoo.com with NNFMP; 20 Jun 2013 10:37:05 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 942963.70156.bm@omp1011.mail.bf1.yahoo.com Received: (qmail 51059 invoked by uid 60001); 20 Jun 2013 10:37:05 -0000 X-YMail-OSG: PMWlfS0VM1kaz7noV9t7416HlBfI74NLAJfeqseGVdwoN_4 .cicUw7tQxkBTbJhWVEJUEcqGCtMn4H5naOr7Y5f0bcE.YnqVsGzziKxpDX3 JUz.ftPsNVH7ECEzrTY_iR5RKqa.wRR9mhe0TMVUyK09GrQBN8gF7VzgPBnO 6hafbVQhqaWM22UzVtADRAezC.PsC13seOVRw0uBErzykPlzwxqvL3u90dCQ dE5AkI7fzDz_.VIyK5HmE_YeN5v5.vvs1_kmRq5GMlopPA5HLz_juErlsxqZ yQRnfaQ_pRoYTFi.VrvFQ_0vCEc5IlxqicbB2S.Cj5r8CLGebXdeosfRsBwN geDQnJxIj8oHsNVuKPscvXXeWiQrH0.ogownHUZ7SP7JwVuwEpYOSXFdMFYo zpc76RKS1PJd_vg6zGjc2CRUBLNbn944x._BUvce7IWI6QBoLAGgtoc2aQxP 6oTitq4vcI..lP3VSabmiAwF.C.KB25pcaooOcSfNOv._CfEEGSB.lYkBT7X jno5Yuhku3mPb3HxtvpCEW3N1w3c8T83g23oL7eC6d6UQAq282Yc- Received: from [87.160.177.196] by web162706.mail.bf1.yahoo.com via HTTP; Thu, 20 Jun 2013 03:37:05 PDT X-Rocket-MIMEInfo: 002.001, SSBkb24ndCBnZXQgd2h5IHRoaXMgaXMgc3VjaCBhIGNvbnRlbnRpb3VzIGNoYW5nZT8KCkJlZm9yZSBJIHdhcyBhYmxlIHRvIHVzZSBhc3NlcnRzIHRvIGNoZWNrIHRoZSBleHBlY3RlZCBsZW5ndGggb2YgbGVuZ3RoIG9mIG1lc3NhZ2VzIHBlciBwcm90b2NvbCB2ZXJzaW9uLCBJIGNvdWxkIHBhc3MgaW4gZHVtYiBpdGVyYXRvcnMgdGhhdCBqdXN0IHBhcnNlIHRoZSBieXRlIHN0cmVhbSBhbmQgSSBjb3VsZCBzZXJpYWxpemUgYW5kIGRlc2VyaWFsaXplIGEgbWVzc2FnZSB0byBjaGVjayB0aGUgcGFyc2VyIGkBMAEBAQE- X-Mailer: YahooMailWebService/0.8.147.553 References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> <20130620090649.GA17765@vps7135.xlshosting.net> Message-ID: <1371724625.50978.YahooMailNeo@web162706.mail.bf1.yahoo.com> Date: Thu, 20 Jun 2013 03:37:05 -0700 (PDT) From: Turkey Breast To: "bitcoin-development@lists.sourceforge.net" In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-1963244382-266082894-1371724625=:50978" X-Spam-Score: -0.4 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [98.139.213.138 listed in list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (turkeybreast[at]yahoo.com) -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1UpcEq-0005Yp-CZ Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Turkey Breast List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 10:37:17 -0000 ---1963244382-266082894-1371724625=:50978 Content-Type: text/plain; charset=us-ascii I don't get why this is such a contentious change? Before I was able to use asserts to check the expected length of length of messages per protocol version, I could pass in dumb iterators that just parse the byte stream and I could serialize and deserialize a message to check the parser is correct (in debug mode). This 'simple' change causes all that behaviour to be lost. You can no longer just use iterators but must know the remaining length (or if you use std::distance, you can only use specific std containers - not just anything with an iterator and an operator++). You cannot check the deserialization process by serializing the deserialized message and comparing it to the original data (because the bool is always present in the serializer). It's a bit stupid you call it buggy code when this behaviour has never been present in Bitcoin. The BIP doesn't introduce any unwanted side-effects and is a trivial reasonable change. If you want optional fields then the proper way to do it, is to either set a flag in the Services field of the "version" message to indicate different formats for messages (i.e use this template structure for a message, not that one), introduce a new message (if the changes are big), to approve/improve Stefan's BIP 32 for custom services or to have a value in the byte stream indicating which fields are present (maybe a bitfield or so). Using a quirk of an implementation is just bad form and sloppy coding. Optional fields should have their own mechanism that allows them to remain as optional fields between protocol version upgrades. The bitcoind software can probably be improved too, by checking that the length of the version message is consistent for the protocol version given by the connected node. Right now it makes no assumptions based on that which is a mistake (new clients can broadcast older version messages that don't have all the fields required). Probably the software should penalise hosts which do that. What's the big deal to update the protocol version number from 70001 to 70002? It's not like we'll run out of integers. The field has now gone from optional to required now anyway - that's a behaviour change. It'd be good to enforce that. I see this as a bug. ________________________________ From: Mike Hearn To: Pieter Wuille Cc: Bitcoin Dev ; Tamas Blummer Sent: Thursday, June 20, 2013 11:17 AM Subject: Re: [Bitcoin-development] Missing fRelayTxes in version There's no problem, but there's no benefit either. It also locks us in to a potentially problematic guarantee - what if in future we want to have, say, two optional new pieces of data in two different messages. We don't want to require that if version > X then you have to implement all features up to and including that point. Essentially the number of fields in a message is like a little version number, just for that message. It adds flexibility to keep it that way, and there's no downside, seeing as that bridge was already crossed and people with parsers that can't handle it need to fix their code anyway. So I have a slight preference for keeping things the way they are, it keeps things flexible for future and costs nothing. On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille wrote: On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote: >> Sure but why not do that when there's an actual new field to add? Does >> anyone have a proposal for a feature that needs a new version field at the >> moment? There's no point changing the protocol now unless there's actually >> a new field to add. >> >> Anyway I still don't see why anyone cares about this issue. The Bitcoin >> protocol does not and never has required that all messages have a fixed >> number of fields per version. Any parser written on the assumption it did >> was just buggy. Look at how tx messages are relayed for the most obvious >> example of that pattern in action - it's actually the raw byte stream >> that's stored and relayed to ensure that fields added in new versions >> aren't dropped during round-tripping. Old versions are supposed to preserve >> fields from the future. > >Actually, that is not the same issue. What is being argued for here is that >the version in the version message itself should indicate which fields are >present, so a parser doesn't need to look at the length of the message. That >seems like a minor but very reasonable request to me, and it's trivial to do. >That doesn't mean that you may receive versions higher than what you know of, >and thus messages with fields you don't know about. That doesn't matter, you >can just ignore them. > >I see no problem with raising the protocol version number to indicate >"all fields up to fRelayTxes are required, if the announced nVersion is above N". >In fact, I believe (though haven't checked) all previous additions to the version >message were accompanied with a protocol version (then: client version) increase >as well. > >-- >Pieter > > ------------------------------------------------------------------------------ This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev _______________________________________________ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development ---1963244382-266082894-1371724625=:50978 Content-Type: text/html; charset=us-ascii
I don't get why this is such a contentious change?

Before I was able to use asserts to check the expected length of length of messages per protocol version, I could pass in dumb iterators that just parse the byte stream and I could serialize and deserialize a message to check the parser is correct (in debug mode).

This 'simple' change causes all that behaviour to be lost. You can no longer just use iterators but must know the remaining length (or if you use std::distance, you can only use specific std containers - not just anything with an iterator and an operator++). You cannot check the deserialization process by serializing the deserialized message and comparing it to the original data (because the bool is always present in the serializer).

It's a bit stupid you call it buggy code when this behaviour has never been present in Bitcoin. The BIP doesn't introduce any unwanted side-effects and is a trivial reasonable change.

If you want optional fields then the proper way to do it, is to either set a flag in the Services field of the "version" message to indicate different formats for messages (i.e use this template structure for a message, not that one), introduce a new message (if the changes are big), to approve/improve Stefan's BIP 32 for custom services or to have a value in the byte stream indicating which fields are present (maybe a bitfield or so).

Using a quirk of an implementation is just bad form and sloppy coding. Optional fields should have their own mechanism that allows them to remain as optional fields between protocol version upgrades.

The bitcoind software can probably be improved too, by checking that the length of the version message is consistent for the protocol version given by the connected node. Right now it makes no assumptions based on that which is a mistake (new clients can broadcast older version messages that don't have all the fields required). Probably the software should penalise hosts which do that.

What's the big deal to update the protocol version number from 70001 to 70002? It's not like we'll run out of integers. The field has now gone from optional to required now anyway - that's a behaviour change. It'd be good to enforce that. I see this as a bug.

From: Mike Hearn <mike@plan99.net>
To: Pieter Wuille <pieter.wuille@gmail.com>
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>; Tamas Blummer <tamas@bitsofproof.com>
Sent: Thursday, June 20, 2013 11:17 AM
Subject: Re: [Bitcoin-development] Missing fRelayTxes in version

There's no problem, but there's no benefit either. It also locks us in to a potentially problematic guarantee - what if in future we want to have, say, two optional new pieces of data in two different messages. We don't want to require that if version > X then you have to implement all features up to and including that point.

Essentially the number of fields in a message is like a little version number, just for that message. It adds flexibility to keep it that way, and there's no downside, seeing as that bridge was already crossed and people with parsers that can't handle it need to fix their code anyway.

So I have a slight preference for keeping things the way they are, it keeps things flexible for future and costs nothing.



On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille <pieter.wuille@gmail.com> wrote:
On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote:
> Sure but why not do that when there's an actual new field to add? Does
> anyone have a proposal for a feature that needs a new version field at the
> moment? There's no point changing the protocol now unless there's actually
> a new field to add.
>
> Anyway I still don't see why anyone cares about this issue. The Bitcoin
> protocol does not and never has required that all messages have a fixed
> number of fields per version. Any parser written on the assumption it did
> was just buggy. Look at how tx messages are relayed for the most obvious
> example of that pattern in action - it's actually the raw byte stream
> that's stored and relayed to ensure that fields added in new versions
> aren't dropped during round-tripping. Old versions are supposed to preserve
> fields from the future.

Actually, that is not the same issue. What is being argued for here is that
the version in the version message itself should indicate which fields are
present, so a parser doesn't need to look at the length of the message. That
seems like a minor but very reasonable request to me, and it's trivial to do.
That doesn't mean that you may receive versions higher than what you know of,
and thus messages with fields you don't know about. That doesn't matter, you
can just ignore them.

I see no problem with raising the protocol version number to indicate
"all fields up to fRelayTxes are required, if the announced nVersion is above N".
In fact, I believe (though haven't checked) all previous additions to the version
message were accompanied with a protocol version (then: client version) increase
as well.

--
Pieter



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development


---1963244382-266082894-1371724625=:50978-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpcRo-00066r-8N for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 10:50:36 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.219.49 as permitted sender) client-ip=209.85.219.49; envelope-from=mh.in.england@gmail.com; helo=mail-oa0-f49.google.com; Received: from mail-oa0-f49.google.com ([209.85.219.49]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1UpcRm-0003sv-AU for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 10:50:36 +0000 Received: by mail-oa0-f49.google.com with SMTP id n9so7660724oag.22 for ; Thu, 20 Jun 2013 03:50:29 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.47.130 with SMTP id d2mr4288453oen.67.1371725428886; Thu, 20 Jun 2013 03:50:28 -0700 (PDT) Sender: mh.in.england@gmail.com Received: by 10.76.23.36 with HTTP; Thu, 20 Jun 2013 03:50:28 -0700 (PDT) In-Reply-To: <1371724625.50978.YahooMailNeo@web162706.mail.bf1.yahoo.com> References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> <20130620090649.GA17765@vps7135.xlshosting.net> <1371724625.50978.YahooMailNeo@web162706.mail.bf1.yahoo.com> Date: Thu, 20 Jun 2013 12:50:28 +0200 X-Google-Sender-Auth: Lp5UE29Y7rtkiYtSimaQjvk9dNY Message-ID: From: Mike Hearn To: Turkey Breast Content-Type: multipart/alternative; boundary=001a11c30992eb171e04df93b649 X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1UpcRm-0003sv-AU Cc: "bitcoin-development@lists.sourceforge.net" Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 10:50:36 -0000 --001a11c30992eb171e04df93b649 Content-Type: text/plain; charset=UTF-8 Sure, the issue isn't running out of integers, it's that you have to handle the case of truncated messages whether you like it or not so it doesn't add any simplicity. Even if Bitcoin-Qt starts only sending the new field with a new version number, there are tens of thousands of bitcoinj based wallets out there now that send the current version number and the fRelayTx field as well, so you cannot assume anything about whether the field will exist or not based on the version number regardless of what is changed on the C++ side. Assuming you care about your code being able to serve Bloom-filtering clients of course. With regards to relying on quirks, etc, this is the old "is the protocol defined by Satoshi's code" debate again ... as I said, version messages have always had a variable number of fields. You didn't notice before because it was a long time since any fields were added. Perhaps it's indeed not ideal, perhaps if Bitcoin was designed in 2013 it'd be using protobufs or some other pre-packaged serialization system. But it is what it is. On Thu, Jun 20, 2013 at 12:37 PM, Turkey Breast wrote: > I don't get why this is such a contentious change? > > Before I was able to use asserts to check the expected length of length of > messages per protocol version, I could pass in dumb iterators that just > parse the byte stream and I could serialize and deserialize a message to > check the parser is correct (in debug mode). > > This 'simple' change causes all that behaviour to be lost. You can no > longer just use iterators but must know the remaining length (or if you use > std::distance, you can only use specific std containers - not just anything > with an iterator and an operator++). You cannot check the deserialization > process by serializing the deserialized message and comparing it to the > original data (because the bool is always present in the serializer). > > It's a bit stupid you call it buggy code when this behaviour has never > been present in Bitcoin. The BIP doesn't introduce any unwanted > side-effects and is a trivial reasonable change. > > If you want optional fields then the proper way to do it, is to either set > a flag in the Services field of the "version" message to indicate different > formats for messages (i.e use this template structure for a message, not > that one), introduce a new message (if the changes are big), to > approve/improve Stefan's BIP 32 for custom services or to have a value in > the byte stream indicating which fields are present (maybe a bitfield or > so). > > Using a quirk of an implementation is just bad form and sloppy coding. > Optional fields should have their own mechanism that allows them to remain > as optional fields between protocol version upgrades. > > The bitcoind software can probably be improved too, by checking that the > length of the version message is consistent for the protocol version given > by the connected node. Right now it makes no assumptions based on that > which is a mistake (new clients can broadcast older version messages that > don't have all the fields required). Probably the software should penalise > hosts which do that. > > What's the big deal to update the protocol version number from 70001 to > 70002? It's not like we'll run out of integers. The field has now gone from > optional to required now anyway - that's a behaviour change. It'd be good > to enforce that. I see this as a bug. > > ------------------------------ > *From:* Mike Hearn > *To:* Pieter Wuille > *Cc:* Bitcoin Dev ; Tamas > Blummer > *Sent:* Thursday, June 20, 2013 11:17 AM > *Subject:* Re: [Bitcoin-development] Missing fRelayTxes in version > > There's no problem, but there's no benefit either. It also locks us in to > a potentially problematic guarantee - what if in future we want to have, > say, two optional new pieces of data in two different messages. We don't > want to require that if version > X then you have to implement all features > up to and including that point. > > Essentially the number of fields in a message is like a little version > number, just for that message. It adds flexibility to keep it that way, and > there's no downside, seeing as that bridge was already crossed and people > with parsers that can't handle it need to fix their code anyway. > > So I have a slight preference for keeping things the way they are, it > keeps things flexible for future and costs nothing. > > > > On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille wrote: > > On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote: > > Sure but why not do that when there's an actual new field to add? Does > > anyone have a proposal for a feature that needs a new version field at > the > > moment? There's no point changing the protocol now unless there's > actually > > a new field to add. > > > > Anyway I still don't see why anyone cares about this issue. The Bitcoin > > protocol does not and never has required that all messages have a fixed > > number of fields per version. Any parser written on the assumption it did > > was just buggy. Look at how tx messages are relayed for the most obvious > > example of that pattern in action - it's actually the raw byte stream > > that's stored and relayed to ensure that fields added in new versions > > aren't dropped during round-tripping. Old versions are supposed to > preserve > > fields from the future. > > Actually, that is not the same issue. What is being argued for here is that > the version in the version message itself should indicate which fields are > present, so a parser doesn't need to look at the length of the message. > That > seems like a minor but very reasonable request to me, and it's trivial to > do. > That doesn't mean that you may receive versions higher than what you know > of, > and thus messages with fields you don't know about. That doesn't matter, > you > can just ignore them. > > I see no problem with raising the protocol version number to indicate > "all fields up to fRelayTxes are required, if the announced nVersion is > above N". > In fact, I believe (though haven't checked) all previous additions to the > version > message were accompanied with a protocol version (then: client version) > increase > as well. > > -- > Pieter > > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > --001a11c30992eb171e04df93b649 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Sure, the issue isn't running out of integers, it'= s that you have to handle the case of truncated messages whether you like i= t or not so it doesn't add any simplicity. Even if Bitcoin-Qt starts on= ly sending the new field with a new version number, there are tens of thous= ands of bitcoinj based wallets out there now that send the current version = number and the fRelayTx field as well, so you cannot assume anything about = whether the field will exist or not based on the version number regardless = of what is changed on the C++ side. Assuming you care about your code being= able to serve Bloom-filtering clients of course.

With regards to relying on quirks, etc, this is the old &quo= t;is the protocol defined by Satoshi's code" debate again ... as I= said, version messages have always had a variable number of fields. You di= dn't notice before because it was a long time since any fields were add= ed. Perhaps it's indeed not ideal, perhaps if Bitcoin was designed in 2= 013 it'd be using protobufs or some other pre-packaged serialization sy= stem. But it is what it is.



On Thu, Jun 20, 2013 at 12:37 PM, Turkey Breast &= lt;turkeybreast= @yahoo.com> wrote:
I don't get why this= is such a contentious change?

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> Before I was able to use asserts to check the expected length of leng= th of messages per protocol version, I could pass in dumb iterators that ju= st parse the byte stream and I could serialize and deserialize a message to= check the parser is correct (in debug mode).

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> This 'simple' change causes all that behaviour to be lost. Yo= u can no longer just use iterators but must know the remaining length (or i= f you use std::distance, you can only use specific std containers - not jus= t anything with an iterator and an operator++). You cannot check the deseri= alization process by serializing the deserialized message and comparing it = to the original data (because the bool is always present in the serializer)= .

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> It's a bit stupid you call it buggy code when this behaviour has = never been present in Bitcoin. The BIP doesn't introduce any unwanted s= ide-effects and is a trivial reasonable change.

<= span>If you want optional fields then the proper way to do it, is to either= set a flag in the Services field of the "version" message to ind= icate different formats for messages (i.e use this template structure for a= message, not that one), introduce a new message (if the changes are big), = to approve/improve Stefan's BIP 32 for custom services or to have a val= ue in the byte stream indicating which fields are present (maybe a bitfield or so).

Using a quirk of an implementation is just bad form and sloppy coding= . Optional fields should have their own mechanism that allows them to remai= n as optional fields between protocol version upgrades.

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> The bitcoind software can probably be improved too, by checking that the length of the version message is consistent for = the protocol version given by the connected node. Right now it makes no ass= umptions based on that which is a mistake (new clients can broadcast older = version messages that don't have all the fields required). Probably the= software should penalise hosts which do that.

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> What's the big deal to update the protocol version number from 70= 001 to 70002? It's not like we'll run out of integers. The field ha= s now gone from optional to required now anyway - that's a behaviour ch= ange. It'd be good to enforce that. I see this as a bug.


There's no problem, but there's no benefit ei= ther. It also locks us in to a potentially problematic guarantee - what if = in future we want to have, say, two optional new pieces of data in two diff= erent messages. We don't want to require that if version > X then yo= u have to implement all features up to and including that point.

Essentially the number of fields in a message is like a litt= le version number, just for that message. It adds flexibility to keep it th= at way, and there's no downside, seeing as that bridge was already cros= sed and people with parsers that can't handle it need to fix their code= anyway.

So I have a slight preference for keeping things the wa= y they are, it keeps things flexible for future and costs nothing.



On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille <pi= eter.wuille@gmail.com> wrote:
On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote:
> Sure but why not do that when there's an actual new field to add? = Does
> anyone have a proposal for a feature that needs a new version field at= the
> moment? There's no point changing the protocol now unless there= 9;s actually
> a new field to add.
>
> Anyway I still don't see why anyone cares about this issue. The Bi= tcoin
> protocol does not and never has required that all messages have a fixe= d
> number of fields per version. Any parser written on the assumption it = did
> was just buggy. Look at how tx messages are relayed for the most obvio= us
> example of that pattern in action - it's actually the raw byte str= eam
> that's stored and relayed to ensure that fields added in new versi= ons
> aren't dropped during round-tripping. Old versions are supposed to= preserve
> fields from the future.

Actually, that is not the same issue. What is being argued for here i= s that
the version in the version message itself should indicate which fields are<= br> present, so a parser doesn't need to look at the length of the message.= That
seems like a minor but very reasonable request to me, and it's trivial = to do.
That doesn't mean that you may receive versions higher than what you kn= ow of,
and thus messages with fields you don't know about. That doesn't ma= tter, you
can just ignore them.

I see no problem with raising the protocol version number to indicate
"all fields up to fRelayTxes are required, if the announced nVersion i= s above N".
In fact, I believe (though haven't checked) all previous additions to t= he version
message were accompanied with a protocol version (then: client version) inc= rease
as well.

--
Pieter



-----------------------------------= -------------------------------------------
This SF.net email is sponsor= ed by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-de= v2dev
_______________________________________________
Bitcoin-development mail= ing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-developmen= t



-----------------------= -------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.= sf.net/sfu/windows-dev2dev
_________________________________________= ______
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment


--001a11c30992eb171e04df93b649-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpcU6-0000WH-1t for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 10:52:58 +0000 Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.223.173 as permitted sender) client-ip=209.85.223.173; envelope-from=pieter.wuille@gmail.com; helo=mail-ie0-f173.google.com; Received: from mail-ie0-f173.google.com ([209.85.223.173]) by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1UpcU4-0002uv-KH for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 10:52:58 +0000 Received: by mail-ie0-f173.google.com with SMTP id k13so16094116iea.32 for ; Thu, 20 Jun 2013 03:52:51 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.42.47.69 with SMTP id n5mr2975623icf.24.1371725571261; Thu, 20 Jun 2013 03:52:51 -0700 (PDT) Received: by 10.50.149.233 with HTTP; Thu, 20 Jun 2013 03:52:51 -0700 (PDT) Received: by 10.50.149.233 with HTTP; Thu, 20 Jun 2013 03:52:51 -0700 (PDT) In-Reply-To: <1371724625.50978.YahooMailNeo@web162706.mail.bf1.yahoo.com> References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> <20130620090649.GA17765@vps7135.xlshosting.net> <1371724625.50978.YahooMailNeo@web162706.mail.bf1.yahoo.com> Date: Thu, 20 Jun 2013 12:52:51 +0200 Message-ID: From: Pieter Wuille To: Turkey Breast Content-Type: multipart/alternative; boundary=90e6ba6149c667c07804df93bf35 X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (pieter.wuille[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1UpcU4-0002uv-KH Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 10:52:58 -0000 --90e6ba6149c667c07804df93bf35 Content-Type: text/plain; charset=ISO-8859-1 Let's just increase the version number and be done with this discussion. It's a small benefit, but it simplifies things and it's trivial to do. I don't understand how a policy of requiring version increases could limit future extensions: after the version/verack exchange, the protocol version is negotiated between peers, and there is no need for anything optional anymore. Note thay this is just about parsing, not about relaying - you should still relay parts of a message you haven't parsed. But that doesn't apply to the version message anyway, which is the only place where this matters. -- Pieter On 20 Jun 2013 12:38, "Turkey Breast" wrote: > I don't get why this is such a contentious change? > > Before I was able to use asserts to check the expected length of length of > messages per protocol version, I could pass in dumb iterators that just > parse the byte stream and I could serialize and deserialize a message to > check the parser is correct (in debug mode). > > This 'simple' change causes all that behaviour to be lost. You can no > longer just use iterators but must know the remaining length (or if you use > std::distance, you can only use specific std containers - not just anything > with an iterator and an operator++). You cannot check the deserialization > process by serializing the deserialized message and comparing it to the > original data (because the bool is always present in the serializer). > > It's a bit stupid you call it buggy code when this behaviour has never > been present in Bitcoin. The BIP doesn't introduce any unwanted > side-effects and is a trivial reasonable change. > > If you want optional fields then the proper way to do it, is to either set > a flag in the Services field of the "version" message to indicate different > formats for messages (i.e use this template structure for a message, not > that one), introduce a new message (if the changes are big), to > approve/improve Stefan's BIP 32 for custom services or to have a value in > the byte stream indicating which fields are present (maybe a bitfield or > so). > > Using a quirk of an implementation is just bad form and sloppy coding. > Optional fields should have their own mechanism that allows them to remain > as optional fields between protocol version upgrades. > > The bitcoind software can probably be improved too, by checking that the > length of the version message is consistent for the protocol version given > by the connected node. Right now it makes no assumptions based on that > which is a mistake (new clients can broadcast older version messages that > don't have all the fields required). Probably the software should penalise > hosts which do that. > > What's the big deal to update the protocol version number from 70001 to > 70002? It's not like we'll run out of integers. The field has now gone from > optional to required now anyway - that's a behaviour change. It'd be good > to enforce that. I see this as a bug. > > ------------------------------ > *From:* Mike Hearn > *To:* Pieter Wuille > *Cc:* Bitcoin Dev ; Tamas > Blummer > *Sent:* Thursday, June 20, 2013 11:17 AM > *Subject:* Re: [Bitcoin-development] Missing fRelayTxes in version > > There's no problem, but there's no benefit either. It also locks us in to > a potentially problematic guarantee - what if in future we want to have, > say, two optional new pieces of data in two different messages. We don't > want to require that if version > X then you have to implement all features > up to and including that point. > > Essentially the number of fields in a message is like a little version > number, just for that message. It adds flexibility to keep it that way, and > there's no downside, seeing as that bridge was already crossed and people > with parsers that can't handle it need to fix their code anyway. > > So I have a slight preference for keeping things the way they are, it > keeps things flexible for future and costs nothing. > > > > On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille wrote: > > On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote: > > Sure but why not do that when there's an actual new field to add? Does > > anyone have a proposal for a feature that needs a new version field at > the > > moment? There's no point changing the protocol now unless there's > actually > > a new field to add. > > > > Anyway I still don't see why anyone cares about this issue. The Bitcoin > > protocol does not and never has required that all messages have a fixed > > number of fields per version. Any parser written on the assumption it did > > was just buggy. Look at how tx messages are relayed for the most obvious > > example of that pattern in action - it's actually the raw byte stream > > that's stored and relayed to ensure that fields added in new versions > > aren't dropped during round-tripping. Old versions are supposed to > preserve > > fields from the future. > > Actually, that is not the same issue. What is being argued for here is that > the version in the version message itself should indicate which fields are > present, so a parser doesn't need to look at the length of the message. > That > seems like a minor but very reasonable request to me, and it's trivial to > do. > That doesn't mean that you may receive versions higher than what you know > of, > and thus messages with fields you don't know about. That doesn't matter, > you > can just ignore them. > > I see no problem with raising the protocol version number to indicate > "all fields up to fRelayTxes are required, if the announced nVersion is > above N". > In fact, I believe (though haven't checked) all previous additions to the > version > message were accompanied with a protocol version (then: client version) > increase > as well. > > -- > Pieter > > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > --90e6ba6149c667c07804df93bf35 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Let's just increase the version number and be done with = this discussion. It's a small benefit, but it simplifies things and it&= #39;s trivial to do.

I don't understand how a policy of requiring version inc= reases could limit future extensions: after the version/verack exchange, th= e protocol version is negotiated between peers, and there is no need for an= ything optional anymore.

Note thay this is just about parsing, not about relaying - y= ou should still relay parts of a message you haven't parsed. But that d= oesn't apply to the version message anyway, which is the only place whe= re this matters.

--
Pieter

On 20 Jun 2013 12:38, "Turkey Breast" = <turkeybreast@yahoo.com>= ; wrote:
I don't get why this is such a contentious change?<= /span>

<= span>Before I was able to use asserts to check the expected length of lengt= h of messages per protocol version, I could pass in dumb iterators that jus= t parse the byte stream and I could serialize and deserialize a message to = check the parser is correct (in debug mode).

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> This 'simple' change causes all that behaviour to be lost. Yo= u can no longer just use iterators but must know the remaining length (or i= f you use std::distance, you can only use specific std containers - not jus= t anything with an iterator and an operator++). You cannot check the deseri= alization process by serializing the deserialized message and comparing it = to the original data (because the bool is always present in the serializer)= .

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> It's a bit stupid you call it buggy code when this behaviour has = never been present in Bitcoin. The BIP doesn't introduce any unwanted s= ide-effects and is a trivial reasonable change.

<= span>If you want optional fields then the proper way to do it, is to either= set a flag in the Services field of the "version" message to ind= icate different formats for messages (i.e use this template structure for a= message, not that one), introduce a new message (if the changes are big), = to approve/improve Stefan's BIP 32 for custom services or to have a val= ue in the byte stream indicating which fields are present (maybe a bitfield or so).

Using a quirk of an implementation is just bad form and sloppy coding= . Optional fields should have their own mechanism that allows them to remai= n as optional fields between protocol version upgrades.

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> The bitcoind software can probably be improved too, by checking that the length of the version message is consistent for = the protocol version given by the connected node. Right now it makes no ass= umptions based on that which is a mistake (new clients can broadcast older = version messages that don't have all the fields required). Probably the= software should penalise hosts which do that.

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> What's the big deal to update the protocol version number from 70= 001 to 70002? It's not like we'll run out of integers. The field ha= s now gone from optional to required now anyway - that's a behaviour ch= ange. It'd be good to enforce that. I see this as a bug.


There's no problem, but there's no benefit ei= ther. It also locks us in to a potentially problematic guarantee - what if = in future we want to have, say, two optional new pieces of data in two diff= erent messages. We don't want to require that if version > X then yo= u have to implement all features up to and including that point.

Essentially the number of fields in a message is like a litt= le version number, just for that message. It adds flexibility to keep it th= at way, and there's no downside, seeing as that bridge was already cros= sed and people with parsers that can't handle it need to fix their code= anyway.

So I have a slight preference for keeping things the wa= y they are, it keeps things flexible for future and costs nothing.



On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille <pi= eter.wuille@gmail.com> wrote:
On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote:
> Sure but why not do that when there's an actual new field to add? = Does
> anyone have a proposal for a feature that needs a new version field at= the
> moment? There's no point changing the protocol now unless there= 9;s actually
> a new field to add.
>
> Anyway I still don't see why anyone cares about this issue. The Bi= tcoin
> protocol does not and never has required that all messages have a fixe= d
> number of fields per version. Any parser written on the assumption it = did
> was just buggy. Look at how tx messages are relayed for the most obvio= us
> example of that pattern in action - it's actually the raw byte str= eam
> that's stored and relayed to ensure that fields added in new versi= ons
> aren't dropped during round-tripping. Old versions are supposed to= preserve
> fields from the future.

Actually, that is not the same issue. What is being argued for here i= s that
the version in the version message itself should indicate which fields are<= br> present, so a parser doesn't need to look at the length of the message.= That
seems like a minor but very reasonable request to me, and it's trivial = to do.
That doesn't mean that you may receive versions higher than what you kn= ow of,
and thus messages with fields you don't know about. That doesn't ma= tter, you
can just ignore them.

I see no problem with raising the protocol version number to indicate
"all fields up to fRelayTxes are required, if the announced nVersion i= s above N".
In fact, I believe (though haven't checked) all previous additions to t= he version
message were accompanied with a protocol version (then: client version) inc= rease
as well.

--
Pieter



-----------------------------------------------------------------= -------------
This SF.net email is sponsored by Windows:

Build fo= r Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Bitcoin-development mail= ing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-developmen= t



-----------------------------= -------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.= sf.net/sfu/windows-dev2dev
_________________________________________= ______
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment

--90e6ba6149c667c07804df93bf35-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpcZb-0006SV-9Q for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 10:58:39 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.219.46 as permitted sender) client-ip=209.85.219.46; envelope-from=mh.in.england@gmail.com; helo=mail-oa0-f46.google.com; Received: from mail-oa0-f46.google.com ([209.85.219.46]) by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1UpcZZ-0005uZ-Cm for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 10:58:39 +0000 Received: by mail-oa0-f46.google.com with SMTP id h1so7776971oag.5 for ; Thu, 20 Jun 2013 03:58:32 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.145.167 with SMTP id sv7mr4308265oeb.56.1371725911978; Thu, 20 Jun 2013 03:58:31 -0700 (PDT) Sender: mh.in.england@gmail.com Received: by 10.76.23.36 with HTTP; Thu, 20 Jun 2013 03:58:31 -0700 (PDT) In-Reply-To: References: <4DE0E45E-BB48-4DFF-9C86-ACBE312B3049@bitsofproof.com> <20130620090649.GA17765@vps7135.xlshosting.net> <1371724625.50978.YahooMailNeo@web162706.mail.bf1.yahoo.com> Date: Thu, 20 Jun 2013 12:58:31 +0200 X-Google-Sender-Auth: k-c3UkJJkdvFkuBbiudFm4bgetE Message-ID: From: Mike Hearn To: Pieter Wuille Content-Type: multipart/alternative; boundary=047d7b5d94fbb67a9b04df93d3e4 X-Spam-Score: -0.5 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (mh.in.england[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1UpcZZ-0005uZ-Cm Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Missing fRelayTxes in version X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 10:58:39 -0000 --047d7b5d94fbb67a9b04df93d3e4 Content-Type: text/plain; charset=UTF-8 As I said, there's no benefit. Even if we do that on the C++ side, you still have to handle connections from bitcoinj clients which will send the field with the old version number. You can't assume they'll all be updated simultaneously, even though both the Android app and MultiBit do have update notifications these days and eventually old versions will presumably disappear. Re: flexibility. Let's say version V+1 adds a complicated new set of data to some messages. Not every client wants or needs the feature enabled by them. Now version V+2 adds a simple extension to a basic message that everyone wants/needs. To get the latter feature, all clients now have to support the first feature as well because the version number is monotonic. OK, we can use a service bit to handle these cases, if we anticipate that not all clients will want the first feature. But then again, we can also use the presence of the additional data as the ground truth instead of duplicating that fact. I don't really mind either way. It just seems that parsing always requires you to be able to handle truncated messages anyway (without asserting or crashing), because a bogus client can always send you partial data. So I don't see what effort is saved. On Thu, Jun 20, 2013 at 12:52 PM, Pieter Wuille wrote: > Let's just increase the version number and be done with this discussion. > It's a small benefit, but it simplifies things and it's trivial to do. > > I don't understand how a policy of requiring version increases could limit > future extensions: after the version/verack exchange, the protocol version > is negotiated between peers, and there is no need for anything optional > anymore. > > Note thay this is just about parsing, not about relaying - you should > still relay parts of a message you haven't parsed. But that doesn't apply > to the version message anyway, which is the only place where this matters. > > -- > Pieter > On 20 Jun 2013 12:38, "Turkey Breast" wrote: > >> I don't get why this is such a contentious change? >> >> Before I was able to use asserts to check the expected length of length >> of messages per protocol version, I could pass in dumb iterators that just >> parse the byte stream and I could serialize and deserialize a message to >> check the parser is correct (in debug mode). >> >> This 'simple' change causes all that behaviour to be lost. You can no >> longer just use iterators but must know the remaining length (or if you use >> std::distance, you can only use specific std containers - not just anything >> with an iterator and an operator++). You cannot check the deserialization >> process by serializing the deserialized message and comparing it to the >> original data (because the bool is always present in the serializer). >> >> It's a bit stupid you call it buggy code when this behaviour has never >> been present in Bitcoin. The BIP doesn't introduce any unwanted >> side-effects and is a trivial reasonable change. >> >> If you want optional fields then the proper way to do it, is to either >> set a flag in the Services field of the "version" message to indicate >> different formats for messages (i.e use this template structure for a >> message, not that one), introduce a new message (if the changes are big), >> to approve/improve Stefan's BIP 32 for custom services or to have a value >> in the byte stream indicating which fields are present (maybe a bitfield or >> so). >> >> Using a quirk of an implementation is just bad form and sloppy coding. >> Optional fields should have their own mechanism that allows them to remain >> as optional fields between protocol version upgrades. >> >> The bitcoind software can probably be improved too, by checking that the >> length of the version message is consistent for the protocol version given >> by the connected node. Right now it makes no assumptions based on that >> which is a mistake (new clients can broadcast older version messages that >> don't have all the fields required). Probably the software should penalise >> hosts which do that. >> >> What's the big deal to update the protocol version number from 70001 to >> 70002? It's not like we'll run out of integers. The field has now gone from >> optional to required now anyway - that's a behaviour change. It'd be good >> to enforce that. I see this as a bug. >> >> ------------------------------ >> *From:* Mike Hearn >> *To:* Pieter Wuille >> *Cc:* Bitcoin Dev ; Tamas >> Blummer >> *Sent:* Thursday, June 20, 2013 11:17 AM >> *Subject:* Re: [Bitcoin-development] Missing fRelayTxes in version >> >> There's no problem, but there's no benefit either. It also locks us in to >> a potentially problematic guarantee - what if in future we want to have, >> say, two optional new pieces of data in two different messages. We don't >> want to require that if version > X then you have to implement all features >> up to and including that point. >> >> Essentially the number of fields in a message is like a little version >> number, just for that message. It adds flexibility to keep it that way, and >> there's no downside, seeing as that bridge was already crossed and people >> with parsers that can't handle it need to fix their code anyway. >> >> So I have a slight preference for keeping things the way they are, it >> keeps things flexible for future and costs nothing. >> >> >> >> On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille wrote: >> >> On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote: >> > Sure but why not do that when there's an actual new field to add? Does >> > anyone have a proposal for a feature that needs a new version field at >> the >> > moment? There's no point changing the protocol now unless there's >> actually >> > a new field to add. >> > >> > Anyway I still don't see why anyone cares about this issue. The Bitcoin >> > protocol does not and never has required that all messages have a fixed >> > number of fields per version. Any parser written on the assumption it >> did >> > was just buggy. Look at how tx messages are relayed for the most obvious >> > example of that pattern in action - it's actually the raw byte stream >> > that's stored and relayed to ensure that fields added in new versions >> > aren't dropped during round-tripping. Old versions are supposed to >> preserve >> > fields from the future. >> >> Actually, that is not the same issue. What is being argued for here is >> that >> the version in the version message itself should indicate which fields are >> present, so a parser doesn't need to look at the length of the message. >> That >> seems like a minor but very reasonable request to me, and it's trivial to >> do. >> That doesn't mean that you may receive versions higher than what you know >> of, >> and thus messages with fields you don't know about. That doesn't matter, >> you >> can just ignore them. >> >> I see no problem with raising the protocol version number to indicate >> "all fields up to fRelayTxes are required, if the announced nVersion is >> above N". >> In fact, I believe (though haven't checked) all previous additions to the >> version >> message were accompanied with a protocol version (then: client version) >> increase >> as well. >> >> -- >> Pieter >> >> >> >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> >> >> >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> >> > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > --047d7b5d94fbb67a9b04df93d3e4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
As I said, there's no benefit. Even if we do that on t= he C++ side, you still have to handle connections from bitcoinj clients whi= ch will send the field with the old version number. You can't assume th= ey'll all be updated simultaneously, even though both the Android app a= nd MultiBit do have update notifications these days and eventually old vers= ions will presumably disappear.

Re: flexibility. Let's say version V+1 adds a complicate= d new set of data to some messages. Not every client wants or needs the fea= ture enabled by them.

Now version V+2 adds a simple exte= nsion to a basic message that everyone wants/needs.

To get the latter feature, all clients now have to supp= ort the first feature as well because the version number is monotonic.

OK, we can use a service bit to handle these cases, if= we anticipate that not all clients will want the first feature. But then a= gain, we can also use the presence of the additional data as the ground tru= th instead of duplicating that fact. I don't really mind either way. It= just seems that parsing always requires you to be able to handle truncated= messages anyway (without asserting or crashing), because a bogus client ca= n always send you partial data. So I don't see what effort is saved.
=C2=A0


On Thu, Jun 20, 2013 at 12:52 PM, Pieter Wuille <p= ieter.wuille@gmail.com> wrote:

Let's just increase the v= ersion number and be done with this discussion. It's a small benefit, b= ut it simplifies things and it's trivial to do.

I don't understand how a policy of requiring version inc= reases could limit future extensions: after the version/verack exchange, th= e protocol version is negotiated between peers, and there is no need for an= ything optional anymore.

Note thay this is just about parsing, not about relaying - y= ou should still relay parts of a message you haven't parsed. But that d= oesn't apply to the version message anyway, which is the only place whe= re this matters.

--
Pieter

On 20 Jun 2013 12:38, "Turkey Breast" = <turkeybreas= t@yahoo.com> wrote:
I don't get why this is such a contentious change?<= /span>

<= span>Before I was able to use asserts to check the expected length of lengt= h of messages per protocol version, I could pass in dumb iterators that jus= t parse the byte stream and I could serialize and deserialize a message to = check the parser is correct (in debug mode).

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> This 'simple' change causes all that behaviour to be lost. Yo= u can no longer just use iterators but must know the remaining length (or i= f you use std::distance, you can only use specific std containers - not jus= t anything with an iterator and an operator++). You cannot check the deseri= alization process by serializing the deserialized message and comparing it = to the original data (because the bool is always present in the serializer)= .

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> It's a bit stupid you call it buggy code when this behaviour has = never been present in Bitcoin. The BIP doesn't introduce any unwanted s= ide-effects and is a trivial reasonable change.

<= span>If you want optional fields then the proper way to do it, is to either= set a flag in the Services field of the "version" message to ind= icate different formats for messages (i.e use this template structure for a= message, not that one), introduce a new message (if the changes are big), = to approve/improve Stefan's BIP 32 for custom services or to have a val= ue in the byte stream indicating which fields are present (maybe a bitfield or so).

Using a quirk of an implementation is just bad form and sloppy coding= . Optional fields should have their own mechanism that allows them to remai= n as optional fields between protocol version upgrades.

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> The bitcoind software can probably be improved too, by checking that the length of the version message is consistent for = the protocol version given by the connected node. Right now it makes no ass= umptions based on that which is a mistake (new clients can broadcast older = version messages that don't have all the fields required). Probably the= software should penalise hosts which do that.

<= div style=3D"font-style:normal;font-size:16px;background-color:transparent;= font-family:times new roman,new york,times,serif"> What's the big deal to update the protocol version number from 70= 001 to 70002? It's not like we'll run out of integers. The field ha= s now gone from optional to required now anyway - that's a behaviour ch= ange. It'd be good to enforce that. I see this as a bug.


There's no problem, but there's no benefit ei= ther. It also locks us in to a potentially problematic guarantee - what if = in future we want to have, say, two optional new pieces of data in two diff= erent messages. We don't want to require that if version > X then yo= u have to implement all features up to and including that point.

Essentially the number of fields in a message is like a litt= le version number, just for that message. It adds flexibility to keep it th= at way, and there's no downside, seeing as that bridge was already cros= sed and people with parsers that can't handle it need to fix their code= anyway.

So I have a slight preference for keeping things the wa= y they are, it keeps things flexible for future and costs nothing.



On Thu, Jun 20, 2013 at 11:06 AM, Pieter Wuille <pi= eter.wuille@gmail.com> wrote:
On Thu, Jun 20, 2013 at 09:36:40AM +0200, Mike Hearn wrote:
> Sure but why not do that when there's an actual new field to add? = Does
> anyone have a proposal for a feature that needs a new version field at= the
> moment? There's no point changing the protocol now unless there= 9;s actually
> a new field to add.
>
> Anyway I still don't see why anyone cares about this issue. The Bi= tcoin
> protocol does not and never has required that all messages have a fixe= d
> number of fields per version. Any parser written on the assumption it = did
> was just buggy. Look at how tx messages are relayed for the most obvio= us
> example of that pattern in action - it's actually the raw byte str= eam
> that's stored and relayed to ensure that fields added in new versi= ons
> aren't dropped during round-tripping. Old versions are supposed to= preserve
> fields from the future.

Actually, that is not the same issue. What is being argued for here i= s that
the version in the version message itself should indicate which fields are<= br> present, so a parser doesn't need to look at the length of the message.= That
seems like a minor but very reasonable request to me, and it's trivial = to do.
That doesn't mean that you may receive versions higher than what you kn= ow of,
and thus messages with fields you don't know about. That doesn't ma= tter, you
can just ignore them.

I see no problem with raising the protocol version number to indicate
"all fields up to fRelayTxes are required, if the announced nVersion i= s above N".
In fact, I believe (though haven't checked) all previous additions to t= he version
message were accompanied with a protocol version (then: client version) inc= rease
as well.

--
Pieter



-----------------------------------------------------------------= -------------
This SF.net email is sponsored by Windows:

Build fo= r Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Bitcoin-development mail= ing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-developmen= t



-----------------------------= -------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.= sf.net/sfu/windows-dev2dev
_________________________________________= ______
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment


-----------------------------------------------------------= -------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.= sf.net/sfu/windows-dev2dev
_________________________________________= ______
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment


--047d7b5d94fbb67a9b04df93d3e4--