* [Bitcoin-development] Revisiting the BIPS process, a proposal
@ 2013-10-21 14:30 Jeff Garzik
2013-10-21 14:34 ` Jeff Garzik
` (2 more replies)
0 siblings, 3 replies; 36+ messages in thread
From: Jeff Garzik @ 2013-10-21 14:30 UTC (permalink / raw)
To: Bitcoin Dev
This summarizes some rambling on IRC about revising the BIPS process.
Right now, the BIPS process is a bit haphazard. Previously, BIPS were
in a git repo, and the BIPS on the wiki were locked against editing.
The BIPS editor at the time started off well, but was eventually
M.I.A. So the BIPS "home" moved de facto to where everyone was
reading them anyway, the wiki. They were made editable, and it became
easier to Just Pick A Number And Write One. However, this inevitably
became a bit disorganized. Further, there was a recent incident --
easily reverted -- where someone hopped on the wiki and started
arbitrarily editing an existing standard.
BIPs need to move back to git, in my opinion. Standards should be
hash-sealed against corruption. Anything less would be uncivilized,
and un-bitcoin. However, many on IRC pointed out requiring a git pull
request might be a burdensome process, and discourage some
contributors. The following is a sketch of an improved process.
1) BIP Draft.
Modelled after IETF drafts. Anybody may submit a BIP draft, as long
as it meets two very loose requirements:
* At least somewhat related to bitcoin. Note, I did not say "crypto-currency".
* Formatted similarly to existing BIPs (i.e. markdown, or whatever the
community prefers)
BIP drafts may be submitted via git pull request, or by emailing an
attachment to bips.editor@bitcoin•org. This mirrors the Linux kernel
change submission process: git is preferred, but there is always a
non-git method for folks who cannot or do not wish to use git or
github.
BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and
are not automatically assigned a BIPS number.
2) Time passes. Software for BIP drafts is developed, tested,
published, and publicly discussed in a typical open source manner.
3) If interest and use cases remain strong, a BIP number may be
requested, and the BIP draft is moved to
git://github.com/bitcoin/bips.git main directory.
4) If there is general consensus that the BIP should be adopted, the
BIP status is changed to "accepted."
There are no specified time limits. Sometimes consensus about a BIP
is reached in days, sometimes 12+ months or more. It varies widely
depending on the feature's complexity and impact.
As with the IETF, it will be q
--
Jeff Garzik
Senior Software Engineer and open source evangelist
BitPay, Inc. https://bitpay.com/
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 14:30 [Bitcoin-development] Revisiting the BIPS process, a proposal Jeff Garzik
@ 2013-10-21 14:34 ` Jeff Garzik
2013-10-21 15:46 ` Andreas Schildbach
2013-10-21 17:17 ` Jeff Garzik
2013-11-19 16:32 ` Wladimir
2 siblings, 1 reply; 36+ messages in thread
From: Jeff Garzik @ 2013-10-21 14:34 UTC (permalink / raw)
To: Bitcoin Dev
Continuing. (grumble gmail grumble)
As with the IETF, there will be a great many drafts that do not make
it to BIPS status. That is normal, and a sign of a healthy process.
I'll volunteer as the BIPS editor.
There needs to be some backups with commit access to bips.git, in case
the BIPS editor is hit by a bus or goes crazy or on vacation. This
can be some core devs, but I would like at least one or two folks who
are not Satoshi-client devs on the list. Maybe Andreas, Michael G,
Alan R, and others working on non-Satoshi clients.
--
Jeff Garzik
Senior Software Engineer and open source evangelist
BitPay, Inc. https://bitpay.com/
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 14:34 ` Jeff Garzik
@ 2013-10-21 15:46 ` Andreas Schildbach
2013-10-21 16:14 ` Jeff Garzik
0 siblings, 1 reply; 36+ messages in thread
From: Andreas Schildbach @ 2013-10-21 15:46 UTC (permalink / raw)
To: bitcoin-development
On 10/21/2013 04:34 PM, Jeff Garzik wrote:
> I'll volunteer as the BIPS editor.
>
> There needs to be some backups with commit access to bips.git, in case
> the BIPS editor is hit by a bus or goes crazy or on vacation. This
> can be some core devs, but I would like at least one or two folks who
> are not Satoshi-client devs on the list. Maybe Andreas, Michael G,
> Alan R, and others working on non-Satoshi clients.
I accept the nomination as a backup (-:
So the duty of the editor is merging pull requests and/or proxying
between email and git for those who do not use git?
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 15:46 ` Andreas Schildbach
@ 2013-10-21 16:14 ` Jeff Garzik
0 siblings, 0 replies; 36+ messages in thread
From: Jeff Garzik @ 2013-10-21 16:14 UTC (permalink / raw)
To: Andreas Schildbach; +Cc: Bitcoin Dev
On Mon, Oct 21, 2013 at 11:46 AM, Andreas Schildbach
<andreas@schildbach•de> wrote:
> I accept the nomination as a backup (-:
Cool.
> So the duty of the editor is merging pull requests and/or proxying
> between email and git for those who do not use git?
Correct. And assigning BIP numbers. Ideally a boring administrative
position. :)
The main tensions will be in gauging whether there is sufficient
consensus and review to boost a draft into BIP/proposed status, and
then promoting a numbered BIP to the final/accepted status.
--
Jeff Garzik
Senior Software Engineer and open source evangelist
BitPay, Inc. https://bitpay.com/
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 14:30 [Bitcoin-development] Revisiting the BIPS process, a proposal Jeff Garzik
2013-10-21 14:34 ` Jeff Garzik
@ 2013-10-21 17:17 ` Jeff Garzik
2013-10-21 19:38 ` Jean-Paul Kogelman
2013-11-19 16:32 ` Wladimir
2 siblings, 1 reply; 36+ messages in thread
From: Jeff Garzik @ 2013-10-21 17:17 UTC (permalink / raw)
To: Bitcoin Dev
Added: I'm happy with gmaxwell as BIP editor as well, as he is
apparently the current BIP-number-assigner-in-chief. :)
The goal is to improve the process, hash-seal our specs, and create an
easy way for anyone with at least an email address to participate.
On Mon, Oct 21, 2013 at 10:30 AM, Jeff Garzik <jgarzik@bitpay•com> wrote:
> This summarizes some rambling on IRC about revising the BIPS process.
>
> Right now, the BIPS process is a bit haphazard. Previously, BIPS were
> in a git repo, and the BIPS on the wiki were locked against editing.
> The BIPS editor at the time started off well, but was eventually
> M.I.A. So the BIPS "home" moved de facto to where everyone was
> reading them anyway, the wiki. They were made editable, and it became
> easier to Just Pick A Number And Write One. However, this inevitably
> became a bit disorganized. Further, there was a recent incident --
> easily reverted -- where someone hopped on the wiki and started
> arbitrarily editing an existing standard.
>
> BIPs need to move back to git, in my opinion. Standards should be
> hash-sealed against corruption. Anything less would be uncivilized,
> and un-bitcoin. However, many on IRC pointed out requiring a git pull
> request might be a burdensome process, and discourage some
> contributors. The following is a sketch of an improved process.
>
> 1) BIP Draft.
>
> Modelled after IETF drafts. Anybody may submit a BIP draft, as long
> as it meets two very loose requirements:
> * At least somewhat related to bitcoin. Note, I did not say "crypto-currency".
> * Formatted similarly to existing BIPs (i.e. markdown, or whatever the
> community prefers)
>
> BIP drafts may be submitted via git pull request, or by emailing an
> attachment to bips.editor@bitcoin•org. This mirrors the Linux kernel
> change submission process: git is preferred, but there is always a
> non-git method for folks who cannot or do not wish to use git or
> github.
>
> BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and
> are not automatically assigned a BIPS number.
>
> 2) Time passes. Software for BIP drafts is developed, tested,
> published, and publicly discussed in a typical open source manner.
>
> 3) If interest and use cases remain strong, a BIP number may be
> requested, and the BIP draft is moved to
> git://github.com/bitcoin/bips.git main directory.
>
> 4) If there is general consensus that the BIP should be adopted, the
> BIP status is changed to "accepted."
>
> There are no specified time limits. Sometimes consensus about a BIP
> is reached in days, sometimes 12+ months or more. It varies widely
> depending on the feature's complexity and impact.
>
> As with the IETF, it will be q
>
> --
> Jeff Garzik
> Senior Software Engineer and open source evangelist
> BitPay, Inc. https://bitpay.com/
--
Jeff Garzik
Senior Software Engineer and open source evangelist
BitPay, Inc. https://bitpay.com/
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 17:17 ` Jeff Garzik
@ 2013-10-21 19:38 ` Jean-Paul Kogelman
2013-10-21 19:47 ` Luke-Jr
0 siblings, 1 reply; 36+ messages in thread
From: Jean-Paul Kogelman @ 2013-10-21 19:38 UTC (permalink / raw)
To: Jeff Garzik; +Cc: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 519 bytes --]
I have some more questions.
1) Should the protocol specification page also be codified into BIP(s)?
2) Should the current wiki pages be taken down / forwarded to the git repo or be auto updated from the git repo?
3) Even though the information in BIP 50 is valuable, should it really be considered a BIP?
On Oct 21, 2013, at 10:17 AM, Jeff Garzik <jgarzik@bitpay•com> wrote:
The goal is to improve the process, hash-seal our specs, and create an
easy way for anyone with at least an email address to participate.
[-- Attachment #2.1: Type: text/html, Size: 784 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 19:38 ` Jean-Paul Kogelman
@ 2013-10-21 19:47 ` Luke-Jr
2013-10-21 20:57 ` Benjamin Cordes
` (2 more replies)
0 siblings, 3 replies; 36+ messages in thread
From: Luke-Jr @ 2013-10-21 19:47 UTC (permalink / raw)
To: bitcoin-development
On Monday, October 21, 2013 7:38:37 PM Jean-Paul Kogelman wrote:
> 1) Should the protocol specification page also be codified into BIP(s)?
Probably wouldn't hurt, but it'd likely need a rewrite in a more modular and
formal form.
> 2) Should the current wiki pages be taken down / forwarded to the git repo
> or be auto updated from the git repo?
Since it's the same format, I'd keep it up there, maybe with a link to the git
repo on the main BIP index wiki page.
> 3) Even though the information in BIP 50 is valuable, should it really be
> considered a BIP?
It's a hardforking protocol change, so IMO yes.
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 19:47 ` Luke-Jr
@ 2013-10-21 20:57 ` Benjamin Cordes
2013-10-21 20:59 ` Benjamin Cordes
2013-10-22 6:39 ` Martin Sustrik
2 siblings, 0 replies; 36+ messages in thread
From: Benjamin Cordes @ 2013-10-21 20:57 UTC (permalink / raw)
To: Luke-Jr; +Cc: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 1970 bytes --]
I believe a better solution would to use a gitlab clone such as gitlab,
which sits on top of the git repo, and allows for custom code around the
BIP process. Potentially one could even build Bitcoin into such a BIP
system. If somebody wants to support a BIP he donates Bitcoins to that
proposal. Somebody who actually implements the BIP can receive some percent
of the bounty (while some percent goes to the Bitcoin foundation). Via such
a platform one could create assurance contracts to kickstart BIP
developments or Bitcoin extensions (public infrastructure which is not part
of the core, such as opensourced exchanges).
On Mon, Oct 21, 2013 at 9:47 PM, Luke-Jr <luke@dashjr•org> wrote:
> On Monday, October 21, 2013 7:38:37 PM Jean-Paul Kogelman wrote:
> > 1) Should the protocol specification page also be codified into BIP(s)?
>
> Probably wouldn't hurt, but it'd likely need a rewrite in a more modular
> and
> formal form.
>
> > 2) Should the current wiki pages be taken down / forwarded to the git
> repo
> > or be auto updated from the git repo?
>
> Since it's the same format, I'd keep it up there, maybe with a link to the
> git
> repo on the main BIP index wiki page.
>
> > 3) Even though the information in BIP 50 is valuable, should it really be
> > considered a BIP?
>
> It's a hardforking protocol change, so IMO yes.
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
[-- Attachment #2: Type: text/html, Size: 2788 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 19:47 ` Luke-Jr
2013-10-21 20:57 ` Benjamin Cordes
@ 2013-10-21 20:59 ` Benjamin Cordes
2013-10-22 6:39 ` Martin Sustrik
2 siblings, 0 replies; 36+ messages in thread
From: Benjamin Cordes @ 2013-10-21 20:59 UTC (permalink / raw)
To: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 1972 bytes --]
I believe a better solution would to use a github clone such as gitlab,
which sits on top of the git repo, and allows for custom code around the
BIP process. Potentially one could even build Bitcoin into such a BIP
system. If somebody wants to support a BIP he donates Bitcoins to that
proposal. Somebody who actually implements the BIP can receive some percent
of the bounty, while some percentage goes to the Bitcoin foundation. Via
such a platform one could create assurance contracts to kickstart BIP
developments or Bitcoin extensions (public infrastructure which is not part
of the core, such as opensourced exchanges).
On Mon, Oct 21, 2013 at 9:47 PM, Luke-Jr <luke@dashjr•org> wrote:
> On Monday, October 21, 2013 7:38:37 PM Jean-Paul Kogelman wrote:
> > 1) Should the protocol specification page also be codified into BIP(s)?
>
> Probably wouldn't hurt, but it'd likely need a rewrite in a more modular
> and
> formal form.
>
> > 2) Should the current wiki pages be taken down / forwarded to the git
> repo
> > or be auto updated from the git repo?
>
> Since it's the same format, I'd keep it up there, maybe with a link to the
> git
> repo on the main BIP index wiki page.
>
> > 3) Even though the information in BIP 50 is valuable, should it really be
> > considered a BIP?
>
> It's a hardforking protocol change, so IMO yes.
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
[-- Attachment #2: Type: text/html, Size: 2859 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 19:47 ` Luke-Jr
2013-10-21 20:57 ` Benjamin Cordes
2013-10-21 20:59 ` Benjamin Cordes
@ 2013-10-22 6:39 ` Martin Sustrik
2013-10-22 6:59 ` Jean-Paul Kogelman
2 siblings, 1 reply; 36+ messages in thread
From: Martin Sustrik @ 2013-10-22 6:39 UTC (permalink / raw)
To: Luke-Jr, bitcoin-development
On 21/10/13 21:47, Luke-Jr wrote:
> On Monday, October 21, 2013 7:38:37 PM Jean-Paul Kogelman wrote:
>> 1) Should the protocol specification page also be codified into BIP(s)?
>
> Probably wouldn't hurt, but it'd likely need a rewrite in a more modular and
> formal form.
I wanted to have a look at how the whole Bitcoin thing works recently.
Being a distributed application, I've searched for the protocol spec.
What I found were two wiki pages (Protocol & ProtocolRules) that looked
more like notes someone wrote down while implementing the application.
Have I missed something? Is there any effort underway trying to produce
a decent spec? If not so, I am willing to help with that.
Martin
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-22 6:39 ` Martin Sustrik
@ 2013-10-22 6:59 ` Jean-Paul Kogelman
2013-10-22 7:03 ` Gregory Maxwell
0 siblings, 1 reply; 36+ messages in thread
From: Jean-Paul Kogelman @ 2013-10-22 6:59 UTC (permalink / raw)
To: Martin Sustrik; +Cc: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 503 bytes --]
> I wanted to have a look at how the whole Bitcoin thing works recently.
> Being a distributed application, I've searched for the protocol spec.
> What I found were two wiki pages (Protocol & ProtocolRules) that looked
> more like notes someone wrote down while implementing the application.
>
> Have I missed something? Is there any effort underway trying to produce
> a decent spec? If not so, I am willing to help with that.
Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ?
[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-22 6:59 ` Jean-Paul Kogelman
@ 2013-10-22 7:03 ` Gregory Maxwell
2013-10-22 7:34 ` Martin Sustrik
0 siblings, 1 reply; 36+ messages in thread
From: Gregory Maxwell @ 2013-10-22 7:03 UTC (permalink / raw)
To: Jean-Paul Kogelman; +Cc: Bitcoin Development
On Mon, Oct 21, 2013 at 11:59 PM, Jean-Paul Kogelman
<jeanpaulkogelman@me•com> wrote:
> Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ?
Take care, the information in the wiki is woefully incomplete.
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-22 7:03 ` Gregory Maxwell
@ 2013-10-22 7:34 ` Martin Sustrik
2013-10-22 7:49 ` Peter Todd
` (2 more replies)
0 siblings, 3 replies; 36+ messages in thread
From: Martin Sustrik @ 2013-10-22 7:34 UTC (permalink / raw)
To: Gregory Maxwell, Jean-Paul Kogelman; +Cc: Bitcoin Development
On 22/10/13 09:03, Gregory Maxwell wrote:
> On Mon, Oct 21, 2013 at 11:59 PM, Jean-Paul Kogelman
> <jeanpaulkogelman@me•com> wrote:
>> Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ?
>
> Take care, the information in the wiki is woefully incomplete.
Imagine myself, with no prior knowledge of Bitcoin looking at the
document. It starts with "Hashes". What hashes? No idea what's going on.
Etc.
Now compare that to a well written RFC. It starts with introduction,
description of the problem, explains the conceptual model of the
solution, then dives into the details. There's also Security
Considerations part in every RFC that is pretty relevant for Bitcoin.
As I said, I am willing to help with writing such document, it would be
a nice way of learning the stuff, however, help from core devs, such as
answering question that may arise in the process, or reviewing the
document would be needed.
Martin
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-22 7:34 ` Martin Sustrik
@ 2013-10-22 7:49 ` Peter Todd
2013-10-22 7:56 ` Gregory Maxwell
2013-10-22 14:08 ` Jeff Garzik
2 siblings, 0 replies; 36+ messages in thread
From: Peter Todd @ 2013-10-22 7:49 UTC (permalink / raw)
To: Martin Sustrik; +Cc: Bitcoin Development
[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]
On Tue, Oct 22, 2013 at 09:34:57AM +0200, Martin Sustrik wrote:
> On 22/10/13 09:03, Gregory Maxwell wrote:
> > On Mon, Oct 21, 2013 at 11:59 PM, Jean-Paul Kogelman
> > <jeanpaulkogelman@me•com> wrote:
> >> Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ?
> >
> > Take care, the information in the wiki is woefully incomplete.
>
> Imagine myself, with no prior knowledge of Bitcoin looking at the
> document. It starts with "Hashes". What hashes? No idea what's going on.
> Etc.
>
> Now compare that to a well written RFC. It starts with introduction,
> description of the problem, explains the conceptual model of the
> solution, then dives into the details. There's also Security
> Considerations part in every RFC that is pretty relevant for Bitcoin.
>
> As I said, I am willing to help with writing such document, it would be
> a nice way of learning the stuff, however, help from core devs, such as
> answering question that may arise in the process, or reviewing the
> document would be needed.
Writing such RFCs is dangerous due to the consensus nature of Bitcoin -
it makes people think the standard is the RFC, rather than the code.
I hear one of the better intros to Bitcoin is the Khan academy videos,
but I've never watched them myself. Once you understand how it works,
start reading source code - the Bitcoin codebase is actually really
simple and readable. However remember that the implications of that
codebase are anything but simple; there's lots of reasons to think
Satoshi himself didn't understand Bitcoin all that well, even by the
time he left the project.
--
'peter'[:-1]@petertodd.org
000000000000000f155e7a648e84a83589048ae1cacb0c60bfce2437553b6af4
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 685 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-22 7:34 ` Martin Sustrik
2013-10-22 7:49 ` Peter Todd
@ 2013-10-22 7:56 ` Gregory Maxwell
2013-10-22 8:20 ` Martin Sustrik
2013-10-22 14:08 ` Jeff Garzik
2 siblings, 1 reply; 36+ messages in thread
From: Gregory Maxwell @ 2013-10-22 7:56 UTC (permalink / raw)
To: Martin Sustrik; +Cc: Bitcoin Development
On Tue, Oct 22, 2013 at 12:34 AM, Martin Sustrik <sustrik@250bpm•com> wrote:
> There's also Security Considerations part in
> every RFC that is pretty relevant for Bitcoin.
Which would say something interesting like "If the bitcoin network
implements inconsistent behavior in the consensus critical parts of
the protocol the world ends. As such, conformance or _non_-conformance
with this specification (in particular, sections 4. 5. and 6.) may be
required for security."
A Bitcoin protocol RFC would be a great place to exercise RFC 6919
keywords. ( http://tools.ietf.org/html/rfc6919 )
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-22 7:56 ` Gregory Maxwell
@ 2013-10-22 8:20 ` Martin Sustrik
0 siblings, 0 replies; 36+ messages in thread
From: Martin Sustrik @ 2013-10-22 8:20 UTC (permalink / raw)
To: Gregory Maxwell; +Cc: Bitcoin Development
On 22/10/13 09:56, Gregory Maxwell wrote:
> On Tue, Oct 22, 2013 at 12:34 AM, Martin Sustrik <sustrik@250bpm•com> wrote:
>> There's also Security Considerations part in
>> every RFC that is pretty relevant for Bitcoin.
>
> Which would say something interesting like "If the bitcoin network
> implements inconsistent behavior in the consensus critical parts of
> the protocol the world ends. As such, conformance or _non_-conformance
> with this specification (in particular, sections 4. 5. and 6.) may be
> required for security."
In fact, yes.
In the end it boils down to saying something like: "Bitcoin is a unique
global distributed application and thus all implementations MUST support
the version of the protocol currently in use, irrespective of whether it
have been documented and/or published. This RFC is meant only for
informational purposes and is a snapshot of the protocol as to Oct 22nd
2013."
That being said, I understand the idea of not publishing the spec so
that everyone is forced to work with live data.
> A Bitcoin protocol RFC would be a great place to exercise RFC 6919
> keywords. ( http://tools.ietf.org/html/rfc6919 )
Heh. Haven't seen that one.
Martin
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-22 7:34 ` Martin Sustrik
2013-10-22 7:49 ` Peter Todd
2013-10-22 7:56 ` Gregory Maxwell
@ 2013-10-22 14:08 ` Jeff Garzik
2013-10-23 7:38 ` Martin Sustrik
2 siblings, 1 reply; 36+ messages in thread
From: Jeff Garzik @ 2013-10-22 14:08 UTC (permalink / raw)
To: Martin Sustrik; +Cc: Bitcoin Development
All that is good practice, but we should avoid adding burdensome
process that might discourage BIP writing.
Consider a distributed approach: if you feel a draft needs more
sections or better language, submit a pull request yourself and help
community-edit the document.
On Tue, Oct 22, 2013 at 3:34 AM, Martin Sustrik <sustrik@250bpm•com> wrote:
> On 22/10/13 09:03, Gregory Maxwell wrote:
>> On Mon, Oct 21, 2013 at 11:59 PM, Jean-Paul Kogelman
>> <jeanpaulkogelman@me•com> wrote:
>>> Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ?
>>
>> Take care, the information in the wiki is woefully incomplete.
>
> Imagine myself, with no prior knowledge of Bitcoin looking at the
> document. It starts with "Hashes". What hashes? No idea what's going on.
> Etc.
>
> Now compare that to a well written RFC. It starts with introduction,
> description of the problem, explains the conceptual model of the
> solution, then dives into the details. There's also Security
> Considerations part in every RFC that is pretty relevant for Bitcoin.
>
> As I said, I am willing to help with writing such document, it would be
> a nice way of learning the stuff, however, help from core devs, such as
> answering question that may arise in the process, or reviewing the
> document would be needed.
>
> Martin
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
--
Jeff Garzik
Senior Software Engineer and open source evangelist
BitPay, Inc. https://bitpay.com/
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-22 14:08 ` Jeff Garzik
@ 2013-10-23 7:38 ` Martin Sustrik
2013-10-23 19:40 ` Peter Todd
0 siblings, 1 reply; 36+ messages in thread
From: Martin Sustrik @ 2013-10-23 7:38 UTC (permalink / raw)
To: Jeff Garzik; +Cc: Bitcoin Development
On 22/10/13 16:08, Jeff Garzik wrote:
> All that is good practice, but we should avoid adding burdensome
> process that might discourage BIP writing.
>
> Consider a distributed approach: if you feel a draft needs more
> sections or better language, submit a pull request yourself and help
> community-edit the document.
I would love to do so.
However, from what Peter Todd said above, my feeling was that spec is
deliberately vague to force compatibility with the reference
implementation rather than with a document.
While that kind of compatibility-via-obscurity won't probably work in a
long run, in short run it can prevent proliferation of implementations
and thus give protocol more space and flexibility to evolve (I've done
the same trick with ZeroMQ myself once).
Anyway, if my impression was wrong I am happy to give it a try.
Martin
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-23 7:38 ` Martin Sustrik
@ 2013-10-23 19:40 ` Peter Todd
2013-10-23 20:05 ` Martin Sustrik
0 siblings, 1 reply; 36+ messages in thread
From: Peter Todd @ 2013-10-23 19:40 UTC (permalink / raw)
To: Martin Sustrik; +Cc: Bitcoin Development
[-- Attachment #1: Type: text/plain, Size: 1475 bytes --]
On Wed, Oct 23, 2013 at 09:38:31AM +0200, Martin Sustrik wrote:
> On 22/10/13 16:08, Jeff Garzik wrote:
> > All that is good practice, but we should avoid adding burdensome
> > process that might discourage BIP writing.
> >
> > Consider a distributed approach: if you feel a draft needs more
> > sections or better language, submit a pull request yourself and help
> > community-edit the document.
>
> I would love to do so.
>
> However, from what Peter Todd said above, my feeling was that spec is
> deliberately vague to force compatibility with the reference
> implementation rather than with a document.
>
> While that kind of compatibility-via-obscurity won't probably work in a
> long run, in short run it can prevent proliferation of implementations
> and thus give protocol more space and flexibility to evolve (I've done
> the same trick with ZeroMQ myself once).
The reference implementation is the specification - the "specification"
on the wiki is best thought of as a set of Coles Notes on the real
specification. If you don't already understand that and the nuance of
that statement you should assume the protocol is fixed in stone and
doesn't evolve at all; that statement is not quite true, but it's very
close to the truth.
I gotta get around to writing a "Developers" section for the FAQ
explaining this stuff....
--
'peter'[:-1]@petertodd.org
0000000000000007362b283ac07839aba795dbfb3c5c4e831d80df9cf3bea2d5
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-23 19:40 ` Peter Todd
@ 2013-10-23 20:05 ` Martin Sustrik
2013-10-23 20:27 ` Peter Todd
0 siblings, 1 reply; 36+ messages in thread
From: Martin Sustrik @ 2013-10-23 20:05 UTC (permalink / raw)
To: Peter Todd; +Cc: Bitcoin Development
On 23/10/13 21:40, Peter Todd wrote:
> The reference implementation is the specification - the "specification"
> on the wiki is best thought of as a set of Coles Notes on the real
> specification. If you don't already understand that and the nuance of
> that statement you should assume the protocol is fixed in stone and
> doesn't evolve at all; that statement is not quite true, but it's very
> close to the truth.
Does that imply that the notes are deliberately obscured to force
everyone to check the source code?
Martin
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-23 20:05 ` Martin Sustrik
@ 2013-10-23 20:27 ` Peter Todd
2013-10-23 21:07 ` Pieter Wuille
0 siblings, 1 reply; 36+ messages in thread
From: Peter Todd @ 2013-10-23 20:27 UTC (permalink / raw)
To: Martin Sustrik; +Cc: Bitcoin Development
[-- Attachment #1: Type: text/plain, Size: 844 bytes --]
On Wed, Oct 23, 2013 at 10:05:56PM +0200, Martin Sustrik wrote:
> On 23/10/13 21:40, Peter Todd wrote:
>
> >The reference implementation is the specification - the "specification"
> >on the wiki is best thought of as a set of Coles Notes on the real
> >specification. If you don't already understand that and the nuance of
> >that statement you should assume the protocol is fixed in stone and
> >doesn't evolve at all; that statement is not quite true, but it's very
> >close to the truth.
>
> Does that imply that the notes are deliberately obscured to force
> everyone to check the source code?
What's on the wiki is mostly the work of people who aren't working on
the reference implementation, so no, you can't say that.
--
'peter'[:-1]@petertodd.org
0000000000000003c1d48b638b9857cb56b6fe9188a60c481fbc9b738ccb4663
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-23 20:27 ` Peter Todd
@ 2013-10-23 21:07 ` Pieter Wuille
2013-10-23 21:42 ` Allen Piscitello
2013-10-24 7:03 ` Martin Sustrik
0 siblings, 2 replies; 36+ messages in thread
From: Pieter Wuille @ 2013-10-23 21:07 UTC (permalink / raw)
To: Peter Todd; +Cc: Bitcoin Development
On Wed, Oct 23, 2013 at 10:27 PM, Peter Todd <pete@petertodd•org> wrote:
> On Wed, Oct 23, 2013 at 10:05:56PM +0200, Martin Sustrik wrote:
>> On 23/10/13 21:40, Peter Todd wrote:
>>
>> >The reference implementation is the specification - the "specification"
>> >on the wiki is best thought of as a set of Coles Notes on the real
>> >specification. If you don't already understand that and the nuance of
>> >that statement you should assume the protocol is fixed in stone and
>> >doesn't evolve at all; that statement is not quite true, but it's very
>> >close to the truth.
>>
>> Does that imply that the notes are deliberately obscured to force
>> everyone to check the source code?
>
> What's on the wiki is mostly the work of people who aren't working on
> the reference implementation, so no, you can't say that.
Indeed, I know of few people who are familiar with the source code
that use the wiki.
I do think that is a pity. The openness and transparency of the
protocol is essential to trusting the system (and shouldn't be limited
to those digging through the source code), and for that reason alone I
think it needs to be well-documented.
I also do agree with earlier comments, that due to the nature of the
consensus problem Bitcoin solves, it will always be the network that
dictates what the actual rules are - anything else can result in
inresolvable forks. If a "formal" specification were written, and we
would find out that the majority of nodes on the network deviate from
it in a subtle way, those nodes would be buggy in the sense that they
aren't doing what was expected, but it would be the specification that
is incorrect for not following the rules of the network. In short,
consistency is more important than correctness, and for that reason,
writing alternate implementation will always be hard and dangerous.
However, I do not think that making it hard to find information about
the details of the system is the way to go. Alternate implementations
are likely inevitable, and in the long run probably a win for the
ecosystem. If effort is put into accurately describing the rules, it
should indeed carry a strong notice about it being descriptive rather
than normative.
If someone is willing to work on that, I am (and likely many people in
#bitcoin-dev are) available for any questions about the protocol and
its semantics.
--
Pieter
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-23 21:07 ` Pieter Wuille
@ 2013-10-23 21:42 ` Allen Piscitello
2013-10-23 21:49 ` Luke-Jr
2013-10-24 7:03 ` Martin Sustrik
1 sibling, 1 reply; 36+ messages in thread
From: Allen Piscitello @ 2013-10-23 21:42 UTC (permalink / raw)
To: Bitcoin Development
[-- Attachment #1: Type: text/plain, Size: 3930 bytes --]
I think formalizing the specification could go a long way and encouraging
alternate implementations is going to be the best way to reduce unexpected
small bugs having a bad effect except on the "buggy" node.
That being said, it's a huge chicken and egg problem. No one wants to go
off the reference client since it could lead to working on a forked chain
as a miner or having bad data as a client.
I don't know if there is a good way to try to take small pieces, get
consensus, possibly have some type of universal test cases and running on
testnet that would solve the problem.
I wouldn't mind taking on parts of this when I have time, specifically
transactions/scripting. Obviously if there are better qualified people who
are interested, have at it!
On Wed, Oct 23, 2013 at 4:07 PM, Pieter Wuille <pieter.wuille@gmail•com>wrote:
> On Wed, Oct 23, 2013 at 10:27 PM, Peter Todd <pete@petertodd•org> wrote:
> > On Wed, Oct 23, 2013 at 10:05:56PM +0200, Martin Sustrik wrote:
> >> On 23/10/13 21:40, Peter Todd wrote:
> >>
> >> >The reference implementation is the specification - the "specification"
> >> >on the wiki is best thought of as a set of Coles Notes on the real
> >> >specification. If you don't already understand that and the nuance of
> >> >that statement you should assume the protocol is fixed in stone and
> >> >doesn't evolve at all; that statement is not quite true, but it's very
> >> >close to the truth.
> >>
> >> Does that imply that the notes are deliberately obscured to force
> >> everyone to check the source code?
> >
> > What's on the wiki is mostly the work of people who aren't working on
> > the reference implementation, so no, you can't say that.
>
> Indeed, I know of few people who are familiar with the source code
> that use the wiki.
>
> I do think that is a pity. The openness and transparency of the
> protocol is essential to trusting the system (and shouldn't be limited
> to those digging through the source code), and for that reason alone I
> think it needs to be well-documented.
>
> I also do agree with earlier comments, that due to the nature of the
> consensus problem Bitcoin solves, it will always be the network that
> dictates what the actual rules are - anything else can result in
> inresolvable forks. If a "formal" specification were written, and we
> would find out that the majority of nodes on the network deviate from
> it in a subtle way, those nodes would be buggy in the sense that they
> aren't doing what was expected, but it would be the specification that
> is incorrect for not following the rules of the network. In short,
> consistency is more important than correctness, and for that reason,
> writing alternate implementation will always be hard and dangerous.
>
> However, I do not think that making it hard to find information about
> the details of the system is the way to go. Alternate implementations
> are likely inevitable, and in the long run probably a win for the
> ecosystem. If effort is put into accurately describing the rules, it
> should indeed carry a strong notice about it being descriptive rather
> than normative.
>
> If someone is willing to work on that, I am (and likely many people in
> #bitcoin-dev are) available for any questions about the protocol and
> its semantics.
>
> --
> Pieter
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
[-- Attachment #2: Type: text/html, Size: 5113 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-23 21:42 ` Allen Piscitello
@ 2013-10-23 21:49 ` Luke-Jr
0 siblings, 0 replies; 36+ messages in thread
From: Luke-Jr @ 2013-10-23 21:49 UTC (permalink / raw)
To: bitcoin-development
On Wednesday, October 23, 2013 9:42:14 PM Allen Piscitello wrote:
> That being said, it's a huge chicken and egg problem. No one wants to go
> off the reference client since it could lead to working on a forked chain
> as a miner or having bad data as a client.
Thankfully, miners are incentivised to run one of every widespread node to
ensure their blocks are accepted by the network. Eloipool already supports
cross-referencing block templates between multiple clients and using the one
that is accepted by most/all (and logging any discrepancies with coredump-like
details).
Luke
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-23 21:07 ` Pieter Wuille
2013-10-23 21:42 ` Allen Piscitello
@ 2013-10-24 7:03 ` Martin Sustrik
2013-10-24 10:39 ` Jeff Garzik
2013-10-24 11:11 ` Christian Decker
1 sibling, 2 replies; 36+ messages in thread
From: Martin Sustrik @ 2013-10-24 7:03 UTC (permalink / raw)
To: Pieter Wuille, Peter Todd; +Cc: Bitcoin Development
On 23/10/13 23:07, Pieter Wuille wrote:
> In short,
> consistency is more important than correctness.
That's a nice and concise way to put it and any potential protocol
documentation should start with a statement like that.
> However, I do not think that making it hard to find information about
> the details of the system is the way to go. Alternate implementations
> are likely inevitable, and in the long run probably a win for the
> ecosystem. If effort is put into accurately describing the rules, it
> should indeed carry a strong notice about it being descriptive rather
> than normative.
One interesting question is whather alternative implementations are more
likely to get it wrong because the protocol description is wrong or
because the authors misunderstood the reference implementation source code.
Extensive documentation of the source code, a la Knuth's literate
programming, may be some kind of a middle ground.
> If someone is willing to work on that, I am (and likely many people in
> #bitcoin-dev are) available for any questions about the protocol and
> its semantics.
Ok. Several people expressed an interest in the topic, so I'll give it a
try and see how it fares.
Martin
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-24 7:03 ` Martin Sustrik
@ 2013-10-24 10:39 ` Jeff Garzik
2013-10-24 11:11 ` Christian Decker
1 sibling, 0 replies; 36+ messages in thread
From: Jeff Garzik @ 2013-10-24 10:39 UTC (permalink / raw)
To: Martin Sustrik; +Cc: Bitcoin Development
Yes. I had pointed people in IRC to Knuth's literate programming, as
an example of how we might document bitcoin.
On Thu, Oct 24, 2013 at 3:03 AM, Martin Sustrik <sustrik@250bpm•com> wrote:
> On 23/10/13 23:07, Pieter Wuille wrote:
>
>> In short,
>> consistency is more important than correctness.
>
> That's a nice and concise way to put it and any potential protocol
> documentation should start with a statement like that.
>
>> However, I do not think that making it hard to find information about
>> the details of the system is the way to go. Alternate implementations
>> are likely inevitable, and in the long run probably a win for the
>> ecosystem. If effort is put into accurately describing the rules, it
>> should indeed carry a strong notice about it being descriptive rather
>> than normative.
>
> One interesting question is whather alternative implementations are more
> likely to get it wrong because the protocol description is wrong or
> because the authors misunderstood the reference implementation source code.
>
> Extensive documentation of the source code, a la Knuth's literate
> programming, may be some kind of a middle ground.
>
>> If someone is willing to work on that, I am (and likely many people in
>> #bitcoin-dev are) available for any questions about the protocol and
>> its semantics.
>
> Ok. Several people expressed an interest in the topic, so I'll give it a
> try and see how it fares.
>
> Martin
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
--
Jeff Garzik
Senior Software Engineer and open source evangelist
BitPay, Inc. https://bitpay.com/
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-24 7:03 ` Martin Sustrik
2013-10-24 10:39 ` Jeff Garzik
@ 2013-10-24 11:11 ` Christian Decker
2013-10-24 19:43 ` Jeremy Spilman
1 sibling, 1 reply; 36+ messages in thread
From: Christian Decker @ 2013-10-24 11:11 UTC (permalink / raw)
To: Martin Sustrik; +Cc: Bitcoin Development
I'd like to add some historical background about how the "protocol
specification" came to be in the first place.
A bit over three years [1] ago I started an attempt to document the
network protocol, by reverse engineering it from the satoshi
client. My goal, back then, was to enable like-minded engineers to
create alternative clients and move away from the client-monoculture
that is still predominant today. It was clear from the beginning that
it would merely be a reverse engineering effort, and that it would
likely lag a bit behind the changes in the main client. It was meant
as a help for engineers that are not well versed in C/C++ to enable
them to contribute by creating new clients, but the satoshi client
would always be the de-facto standard.
With the move from Google Code to the Bitcoin.it wiki somehow this
notion of it being a reverse engineering effort was lost and people
started assuming that if the behavior of the satoshi client did not
match the protocol description it was a bug on the client
side. Instead it is because the reverse engineering of the protocol is
incorrect or simply missing some details. Although the protocol
description is far more complete than it was back when we started, I
still don't feel comfortable giving it the name specification.
I still believe that a client monoculture is bad for the system as a
whole, because a single bug might bring down the whole network. Giving
people the necessary tools to implement new clients brings
stability. I do understand the criticism that writing a specification
might hinder future development as it restricts the possible changes
to the protocol, but isn't this already the case as long as we have
legacy versions of the client participating in the network? I would
also argue that having a specification allows an application
independent review of the protocol to identify possible improvements
and bugs.
I think the protocol description has an important place in the
development of Bitcoin, so much so that we pushed a long time ago to
separate protocol version from the client version. I would love to see
the protocol specification becoming official part of the bitcoin
github repository, which would ideally be maintained alongside the
satoshi client to keep it up to date.
Regards,
Christian Decker
[1] https://bitcointalk.org/index.php?topic=231
--
Christian Decker
On Thu, Oct 24, 2013 at 9:03 AM, Martin Sustrik <sustrik@250bpm•com> wrote:
> On 23/10/13 23:07, Pieter Wuille wrote:
>
>> In short,
>> consistency is more important than correctness.
>
> That's a nice and concise way to put it and any potential protocol
> documentation should start with a statement like that.
>
>> However, I do not think that making it hard to find information about
>> the details of the system is the way to go. Alternate implementations
>> are likely inevitable, and in the long run probably a win for the
>> ecosystem. If effort is put into accurately describing the rules, it
>> should indeed carry a strong notice about it being descriptive rather
>> than normative.
>
> One interesting question is whather alternative implementations are more
> likely to get it wrong because the protocol description is wrong or
> because the authors misunderstood the reference implementation source code.
>
> Extensive documentation of the source code, a la Knuth's literate
> programming, may be some kind of a middle ground.
>
>> If someone is willing to work on that, I am (and likely many people in
>> #bitcoin-dev are) available for any questions about the protocol and
>> its semantics.
>
> Ok. Several people expressed an interest in the topic, so I'll give it a
> try and see how it fares.
>
> Martin
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-24 11:11 ` Christian Decker
@ 2013-10-24 19:43 ` Jeremy Spilman
0 siblings, 0 replies; 36+ messages in thread
From: Jeremy Spilman @ 2013-10-24 19:43 UTC (permalink / raw)
To: Bitcoin Development
Thanks Christian, this is a really interesting bit of history. My own
personal experience from when I wrote my own client and BCCAPI-ish server
was that the protocol specification on the Wiki was hugely valuable, and
rarely sent me astray. Supplement that with the occasional questions on
#bitcoin-dev, and then just coding, coding, coding and getting unit tests
to pass.
Nothing compares (IMO) to stepping through your own code watching the unit
tests run, scripts evaluate, calculating transaction hashes for the
different SIGHASH modes, and finally getting your first transaction into
the block chain. I really appreciated the .json files holding the unit
test data, which were easy to load into my own test harness, the tables on
the Wiki showing what the stack should look like at each point in a script
execution, and the diagrams showing transaction signing.
Bitcoin takes some time to "grok" when you first approach; more than a
day, less than a month, and really no amount of reading documentation or
specs will get you to that "ah ha" moment. When the fog lifts and the
blockchain, scripting, signing, and wallet handling really click, suddenly
the bitcoind code (and many other great public sources in just about any
language you could want) actually does starts to feel fairly simple and
obvious. But it certainly doesn't start out that way on day one.
I think the majority of client code development is actually people writing
'agents' not end-user P2P wallets, and they tend to be written to connect
to a single bitcoind acting as a proxy to the network. Even some end-user
wallets work this way! As such, I spent very little time in my own client
writing P2P protocol code, no peer discovery code, no anti-DoS, etc.
Clients like this also don't pose much systemic risk, because they don't
mine, they don't connect directly to external nodes, etc. They can
certainly be used to "cause trouble" though, but so can
'sendrawtransaction'.
I chose to speak the P2P protocol to bitcoind versus using some of the
other options like ZeroMQ, but it still didn't take long to get headers,
blocks, and transactions downloading. I remember getting stuck on the very
first version message, because of missing the checksum and user-agent or
something caused the latest bitcoind to just ignore me. A little wireshark
capture of the exchange between two working bitcoind instances cleared it
right up. I didn't mind the leg work, I don't think everything needs to be
spoon fed, and it's certainly not purposefully obfuscated. Maybe one
exception is the mix-matched endianness will throw you off, especially if
you are developing on LE! Anyway, I have huge respect for how much effort
it takes to keep even small bits of documentation up-to-date. For as
"slow" as bitcoin moves, it's actually moving incredibly fast.
Finally, the bitcoind console and debug logs, as well sites like
blockchain.info and blockexplorer.com are hugely helpful for debugging raw
and live transactions for when you get stuck. There's a surprisingly large
tooling and support ecosystem out there.
Moral of the story, I think, is everything you need is there. No, it's not
all in one place. Yes, it takes time to find it and assimilate all that
knowledge. It also really helps that the community is extremely willing to
help and answer technical questions, and point you in the right direction,
even when you're working on your own private client code. The IRC channel
can certainly be intimidating because it seems like every time I hit enter
to send a question, gmaxwell's respond 300ms later would invoke an
immediate forehead slap and a groan of "shit, I knew/should have known
that, now I feel dumb" ;-) but if you're working on bitcoin, you better
get used to not being the smartest person in the room! The responses I got
were never arrogant or disparaging, but they were straight to-the-point
and surprisingly high quality. Ain't no slouches in that channel, yes you
will have to bring your A-game and you are expected to have "tried first"
before just asking. I have fairly limited experience working on open
source projects, but I'm extremely happy with my experience with the
Bitcoin community and found writing Bitcoin code hugely enjoyable.
The flip side to helping people implement their own clients, agents, or
even miners, is helping people to contribute pulls requests, or at the
very highest echelon, a BIP. If you haven't written any significant
Bitcoin code, you might want to consider investing in that first before
submitting a BIP. :-)
For a BIP to be valuable, often it requires widespread or even consensus
adoption. BIPs are probably not the place to toss just any old 'good idea'
because BIPs impose a cost on all active developers. I want to read and
understand 'all the BIPs' because for the most part they are actually
essential, like, how to handle duplicate transactions in BIP30 - if you
don't read BIP30 you very likely totally miss that, until your code throws
exceptions while processing block 91842.
And perhaps the hardest kind of BIP of all is the "lets get wallets to add
this user-facing feature" where it has no bearing on the blockchain or
transaction processing, it doesn't make the network more resilient or add
crucial functionality for increasing scalability. Kind of like JPK's HD
wallet encryption proposal, which I love, and I tried to contribute to in
the forums, but I can totally understand the headwinds for making progress
on BIPs like that one and BIP39. No one is against it per-say, it's just
much harder to articulate and justify the NEED for everyone to implement,
test, and support this new not-yet-standard, nice-to-have feature. For
those kinds of BIPs you probably have to go out and get some wallets to
implement it, or implement it yourself, to prove the value and kick start
critical mass before you will even get enough support for getting a BIP
number assigned. IMO, it's not a Bad Thing.
TL;DR; The current support systems worked very well for me. I was able to
accomplish all my goals, and I would even say it was a pleasure. Keep a
high bar for assigning BIP numbers. And I hope to be able to jump back in
and do more with Bitcoin soon.
Thanks all, sorry if I'm rambling,
Jeremy Spilman
On Thu, 24 Oct 2013 04:11:05 -0700, Christian Decker
<decker.christian@gmail•com> wrote:
> I'd like to add some historical background about how the "protocol
> specification" came to be in the first place.
>
> A bit over three years [1] ago I started an attempt to document the
> network protocol, by reverse engineering it from the satoshi
> client. My goal, back then, was to enable like-minded engineers to
> create alternative clients and move away from the client-monoculture
> that is still predominant today. It was clear from the beginning that
> it would merely be a reverse engineering effort, and that it would
> likely lag a bit behind the changes in the main client. It was meant
> as a help for engineers that are not well versed in C/C++ to enable
> them to contribute by creating new clients, but the satoshi client
> would always be the de-facto standard.
>
> With the move from Google Code to the Bitcoin.it wiki somehow this
> notion of it being a reverse engineering effort was lost and people
> started assuming that if the behavior of the satoshi client did not
> match the protocol description it was a bug on the client
> side. Instead it is because the reverse engineering of the protocol is
> incorrect or simply missing some details. Although the protocol
> description is far more complete than it was back when we started, I
> still don't feel comfortable giving it the name specification.
>
> I still believe that a client monoculture is bad for the system as a
> whole, because a single bug might bring down the whole network. Giving
> people the necessary tools to implement new clients brings
> stability. I do understand the criticism that writing a specification
> might hinder future development as it restricts the possible changes
> to the protocol, but isn't this already the case as long as we have
> legacy versions of the client participating in the network? I would
> also argue that having a specification allows an application
> independent review of the protocol to identify possible improvements
> and bugs.
>
> I think the protocol description has an important place in the
> development of Bitcoin, so much so that we pushed a long time ago to
> separate protocol version from the client version. I would love to see
> the protocol specification becoming official part of the bitcoin
> github repository, which would ideally be maintained alongside the
> satoshi client to keep it up to date.
>
> Regards,
> Christian Decker
>
> [1] https://bitcointalk.org/index.php?topic=231
> --
> Christian Decker
>
>
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-10-21 14:30 [Bitcoin-development] Revisiting the BIPS process, a proposal Jeff Garzik
2013-10-21 14:34 ` Jeff Garzik
2013-10-21 17:17 ` Jeff Garzik
@ 2013-11-19 16:32 ` Wladimir
2013-11-19 16:53 ` Drak
2013-11-19 17:06 ` Peter Todd
2 siblings, 2 replies; 36+ messages in thread
From: Wladimir @ 2013-11-19 16:32 UTC (permalink / raw)
To: Jeff Garzik; +Cc: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 879 bytes --]
On Mon, Oct 21, 2013 at 4:30 PM, Jeff Garzik <jgarzik@bitpay•com> wrote:
> BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and
> are not automatically assigned a BIPS number.
>
Are we going to move ahead with this?
If so, I'm volunteering to create the repository and import the current
BIPs from the wiki there (and convert from wiki markup to markdown where
necessary).
2) Time passes. Software for BIP drafts is developed, tested,
> published, and publicly discussed in a typical open source manner.
>
Personally I think it is useful to have a number as soon as a BIP can be
implemented, even if still in draft status; it gives something to refer to
when mentioning a certain improvement proposal (in commit messages and such
it could be called BIP xxx Draft).
I don't think we are at risk of running out of numbers to assign any time
soon.
Wladimir
[-- Attachment #2: Type: text/html, Size: 1512 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-11-19 16:32 ` Wladimir
@ 2013-11-19 16:53 ` Drak
2013-11-19 17:01 ` Gregory Maxwell
2013-11-19 17:06 ` Peter Todd
1 sibling, 1 reply; 36+ messages in thread
From: Drak @ 2013-11-19 16:53 UTC (permalink / raw)
To: Wladimir; +Cc: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 1206 bytes --]
On 19 November 2013 16:32, Wladimir <laanwj@gmail•com> wrote:
>
> On Mon, Oct 21, 2013 at 4:30 PM, Jeff Garzik <jgarzik@bitpay•com> wrote:
>
>> BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and
>> are not automatically assigned a BIPS number.
>>
>
> Are we going to move ahead with this?
>
> If so, I'm volunteering to create the repository and import the current
> BIPs from the wiki there (and convert from wiki markup to markdown where
> necessary).
>
> 2) Time passes. Software for BIP drafts is developed, tested,
>> published, and publicly discussed in a typical open source manner.
>>
>
> Personally I think it is useful to have a number as soon as a BIP can be
> implemented, even if still in draft status; it gives something to refer to
> when mentioning a certain improvement proposal (in commit messages and such
> it could be called BIP xxx Draft).
> I don't think we are at risk of running out of numbers to assign any time
> soon.
>
It's quite normal for standards bodies to allocate numbers when in draft
status. If they don't pass, they don't pass - they are clearly labelled
DRAFTs.
+1 on having things in a github repository. Much better for collaboration,
Drak
[-- Attachment #2: Type: text/html, Size: 2184 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-11-19 16:53 ` Drak
@ 2013-11-19 17:01 ` Gregory Maxwell
2013-11-19 17:07 ` Drak
2013-11-19 17:45 ` Wladimir
0 siblings, 2 replies; 36+ messages in thread
From: Gregory Maxwell @ 2013-11-19 17:01 UTC (permalink / raw)
To: Drak; +Cc: Bitcoin Dev
On Tue, Nov 19, 2013 at 8:53 AM, Drak <drak@zikula•org> wrote:
> It's quite normal for standards bodies to allocate numbers when in draft
> status. If they don't pass, they don't pass - they are clearly labelled
> DRAFTs.
>
> +1 on having things in a github repository. Much better for collaboration,
The IETF makes a clear distinction between individual proposals and
documents which have been accepted by a working group. The former are
named after their authors. Work is not assigned a number until it is
complete.
I believe it is important to distinguish complete work that people
should be implementing from things which are incomplete, and even
more important to distinguish the work of single parties.
Otherwise you're going to get crap like BIP90: "Increase the supply of
Bitcoins to 210 million" being confused as an earnest proposal
supported by many that has traction.
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-11-19 16:32 ` Wladimir
2013-11-19 16:53 ` Drak
@ 2013-11-19 17:06 ` Peter Todd
[not found] ` <CA+s+GJA=p+yvoJqUAMQQRcfYK1B8eMVSJDWaXW8o+X5dzCXkdA@mail.gmail.com>
1 sibling, 1 reply; 36+ messages in thread
From: Peter Todd @ 2013-11-19 17:06 UTC (permalink / raw)
To: Wladimir; +Cc: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 867 bytes --]
On Tue, Nov 19, 2013 at 05:32:55PM +0100, Wladimir wrote:
> On Mon, Oct 21, 2013 at 4:30 PM, Jeff Garzik <jgarzik@bitpay•com> wrote:
>
> > BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and
> > are not automatically assigned a BIPS number.
> >
>
> Are we going to move ahead with this?
>
> If so, I'm volunteering to create the repository and import the current
> BIPs from the wiki there (and convert from wiki markup to markdown where
> necessary).
I already did that:
https://github.com/petertodd/bips
GitHub can render MediaWiki just fine, so I think leaving the BIPs as
MediaWiki is the way to go. New BIPs may want to use either markdown or
MediaWiki - the latter has advantages in terms of formatting
capabilities over the former, particularly when math needs to be
displayed.
--
'peter'[:-1]@petertodd.org
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-11-19 17:01 ` Gregory Maxwell
@ 2013-11-19 17:07 ` Drak
2013-11-19 17:45 ` Wladimir
1 sibling, 0 replies; 36+ messages in thread
From: Drak @ 2013-11-19 17:07 UTC (permalink / raw)
To: Gregory Maxwell; +Cc: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 1679 bytes --]
On 19 November 2013 17:01, Gregory Maxwell <gmaxwell@gmail•com> wrote:
> On Tue, Nov 19, 2013 at 8:53 AM, Drak <drak@zikula•org> wrote:
> > It's quite normal for standards bodies to allocate numbers when in draft
> > status. If they don't pass, they don't pass - they are clearly labelled
> > DRAFTs.
> >
> > +1 on having things in a github repository. Much better for
> collaboration,
>
> The IETF makes a clear distinction between individual proposals and
> documents which have been accepted by a working group. The former are
> named after their authors. Work is not assigned a number until it is
> complete.
>
> I believe it is important to distinguish complete work that people
> should be implementing from things which are incomplete, and even
> more important to distinguish the work of single parties.
>
> Otherwise you're going to get crap like BIP90: "Increase the supply of
> Bitcoins to 210 million" being confused as an earnest proposal
> supported by many that has traction.
>
I wasnt suggesting people add drafts willy nilly to the repository.
When working on a proposal you can work on it in your own fork and create a
PR. When it's ready to be accepted as a working draft by the WG, then it
can be merged into the draft folder. At which point, PRs are made to that
draft copy until it gets into a ready state to become final. If passed,
it's moved to the accepted/ folder.
This way random BIPS cannot be added to the drafts/ folder in the official
repo. They are only added once they are accepted as a working draft
proposal by Gavin or whatever. Now you get all the niceties of github
workflow for collaboration and tweaking of the draft proposal.
Drak
[-- Attachment #2: Type: text/html, Size: 2384 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* [Bitcoin-development] Fwd: Revisiting the BIPS process, a proposal
[not found] ` <CA+s+GJA=p+yvoJqUAMQQRcfYK1B8eMVSJDWaXW8o+X5dzCXkdA@mail.gmail.com>
@ 2013-11-19 17:21 ` Wladimir
0 siblings, 0 replies; 36+ messages in thread
From: Wladimir @ 2013-11-19 17:21 UTC (permalink / raw)
To: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 1043 bytes --]
On Tue, Nov 19, 2013 at 6:06 PM, Peter Todd <pete@petertodd•org> wrote:
> On Tue, Nov 19, 2013 at 05:32:55PM +0100, Wladimir wrote:
> > On Mon, Oct 21, 2013 at 4:30 PM, Jeff Garzik <jgarzik@bitpay•com> wrote:
> >
> > > BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and
> > > are not automatically assigned a BIPS number.
> > >
> >
> > Are we going to move ahead with this?
> >
> > If so, I'm volunteering to create the repository and import the current
> > BIPs from the wiki there (and convert from wiki markup to markdown where
> > necessary).
>
> I already did that:
>
> https://github.com/petertodd/bips
>
Ok cool, I forked it into https://github.com/bitcoin/bips
> GitHub can render MediaWiki just fine, so I think leaving the BIPs as
> MediaWiki is the way to go. New BIPs may want to use either markdown or
> MediaWiki - the latter has advantages in terms of formatting
> capabilities over the former, particularly when math needs to be
> displayed.
>
Agreed, I had no idea github could do that too.
Wladimir
[-- Attachment #2: Type: text/html, Size: 2128 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-11-19 17:01 ` Gregory Maxwell
2013-11-19 17:07 ` Drak
@ 2013-11-19 17:45 ` Wladimir
2013-11-19 17:54 ` Gregory Maxwell
1 sibling, 1 reply; 36+ messages in thread
From: Wladimir @ 2013-11-19 17:45 UTC (permalink / raw)
To: Gregory Maxwell; +Cc: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 790 bytes --]
On Tue, Nov 19, 2013 at 6:01 PM, Gregory Maxwell <gmaxwell@gmail•com> wrote:
> On Tue, Nov 19, 2013 at 8:53 AM, Drak <drak@zikula•org> wrote:
> > It's quite normal for standards bodies to allocate numbers when in draft
> > status. If they don't pass, they don't pass - they are clearly labelled
> > DRAFTs.
> >
> > +1 on having things in a github repository. Much better for
> collaboration,
>
> The IETF makes a clear distinction between individual proposals and
> documents which have been accepted by a working group. The former are
> named after their authors. Work is not assigned a number until it is
> complete.
>
Talking about complete, BIP 40 and 41 don't even have an associated
document:
https://github.com/bitcoin/bips
I agree that was over-eager number assigning.
Wladimir
[-- Attachment #2: Type: text/html, Size: 1329 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
2013-11-19 17:45 ` Wladimir
@ 2013-11-19 17:54 ` Gregory Maxwell
0 siblings, 0 replies; 36+ messages in thread
From: Gregory Maxwell @ 2013-11-19 17:54 UTC (permalink / raw)
To: Wladimir; +Cc: Bitcoin Dev
On Tue, Nov 19, 2013 at 9:45 AM, Wladimir <laanwj@gmail•com> wrote:
> Talking about complete, BIP 40 and 41 don't even have an associated
> document:
> https://github.com/bitcoin/bips
> I agree that was over-eager number assigning.
Maybe! The subject matter its assigned for is already _widely_
deployed, for better or worse.
(by comparison in the IETF, informational RFCs for already widely
deployed things are issued pretty liberally)
I'm not sure how we should be distinguish BIPs which are documenting
things which are already defacto standards vs ones which are proposing
that people do something new.
Mostly I think we don't want the BIP itself being a lever to force
something down people's throats, but rather the process should help
build consensus and review about how to do something— and then
document that consensus.
^ permalink raw reply [flat|nested] 36+ messages in thread
end of thread, other threads:[~2013-11-19 17:54 UTC | newest]
Thread overview: 36+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-10-21 14:30 [Bitcoin-development] Revisiting the BIPS process, a proposal Jeff Garzik
2013-10-21 14:34 ` Jeff Garzik
2013-10-21 15:46 ` Andreas Schildbach
2013-10-21 16:14 ` Jeff Garzik
2013-10-21 17:17 ` Jeff Garzik
2013-10-21 19:38 ` Jean-Paul Kogelman
2013-10-21 19:47 ` Luke-Jr
2013-10-21 20:57 ` Benjamin Cordes
2013-10-21 20:59 ` Benjamin Cordes
2013-10-22 6:39 ` Martin Sustrik
2013-10-22 6:59 ` Jean-Paul Kogelman
2013-10-22 7:03 ` Gregory Maxwell
2013-10-22 7:34 ` Martin Sustrik
2013-10-22 7:49 ` Peter Todd
2013-10-22 7:56 ` Gregory Maxwell
2013-10-22 8:20 ` Martin Sustrik
2013-10-22 14:08 ` Jeff Garzik
2013-10-23 7:38 ` Martin Sustrik
2013-10-23 19:40 ` Peter Todd
2013-10-23 20:05 ` Martin Sustrik
2013-10-23 20:27 ` Peter Todd
2013-10-23 21:07 ` Pieter Wuille
2013-10-23 21:42 ` Allen Piscitello
2013-10-23 21:49 ` Luke-Jr
2013-10-24 7:03 ` Martin Sustrik
2013-10-24 10:39 ` Jeff Garzik
2013-10-24 11:11 ` Christian Decker
2013-10-24 19:43 ` Jeremy Spilman
2013-11-19 16:32 ` Wladimir
2013-11-19 16:53 ` Drak
2013-11-19 17:01 ` Gregory Maxwell
2013-11-19 17:07 ` Drak
2013-11-19 17:45 ` Wladimir
2013-11-19 17:54 ` Gregory Maxwell
2013-11-19 17:06 ` Peter Todd
[not found] ` <CA+s+GJA=p+yvoJqUAMQQRcfYK1B8eMVSJDWaXW8o+X5dzCXkdA@mail.gmail.com>
2013-11-19 17:21 ` [Bitcoin-development] Fwd: " Wladimir
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox