* [Bitcoin-development] Revisiting the BIPS process, a proposal
@ 2013-10-21 14:30 Jeff Garzik
2013-10-21 14:34 ` Jeff Garzik
` (2 more replies)
0 siblings, 3 replies; 36+ messages in thread
From: Jeff Garzik @ 2013-10-21 14:30 UTC (permalink / raw)
To: Bitcoin Dev
This summarizes some rambling on IRC about revising the BIPS process.
Right now, the BIPS process is a bit haphazard. Previously, BIPS were
in a git repo, and the BIPS on the wiki were locked against editing.
The BIPS editor at the time started off well, but was eventually
M.I.A. So the BIPS "home" moved de facto to where everyone was
reading them anyway, the wiki. They were made editable, and it became
easier to Just Pick A Number And Write One. However, this inevitably
became a bit disorganized. Further, there was a recent incident --
easily reverted -- where someone hopped on the wiki and started
arbitrarily editing an existing standard.
BIPs need to move back to git, in my opinion. Standards should be
hash-sealed against corruption. Anything less would be uncivilized,
and un-bitcoin. However, many on IRC pointed out requiring a git pull
request might be a burdensome process, and discourage some
contributors. The following is a sketch of an improved process.
1) BIP Draft.
Modelled after IETF drafts. Anybody may submit a BIP draft, as long
as it meets two very loose requirements:
* At least somewhat related to bitcoin. Note, I did not say "crypto-currency".
* Formatted similarly to existing BIPs (i.e. markdown, or whatever the
community prefers)
BIP drafts may be submitted via git pull request, or by emailing an
attachment to bips.editor@bitcoin•org. This mirrors the Linux kernel
change submission process: git is preferred, but there is always a
non-git method for folks who cannot or do not wish to use git or
github.
BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and
are not automatically assigned a BIPS number.
2) Time passes. Software for BIP drafts is developed, tested,
published, and publicly discussed in a typical open source manner.
3) If interest and use cases remain strong, a BIP number may be
requested, and the BIP draft is moved to
git://github.com/bitcoin/bips.git main directory.
4) If there is general consensus that the BIP should be adopted, the
BIP status is changed to "accepted."
There are no specified time limits. Sometimes consensus about a BIP
is reached in days, sometimes 12+ months or more. It varies widely
depending on the feature's complexity and impact.
As with the IETF, it will be q
--
Jeff Garzik
Senior Software Engineer and open source evangelist
BitPay, Inc. https://bitpay.com/
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 14:30 [Bitcoin-development] Revisiting the BIPS process, a proposal Jeff Garzik @ 2013-10-21 14:34 ` Jeff Garzik 2013-10-21 15:46 ` Andreas Schildbach 2013-10-21 17:17 ` Jeff Garzik 2013-11-19 16:32 ` Wladimir 2 siblings, 1 reply; 36+ messages in thread From: Jeff Garzik @ 2013-10-21 14:34 UTC (permalink / raw) To: Bitcoin Dev Continuing. (grumble gmail grumble) As with the IETF, there will be a great many drafts that do not make it to BIPS status. That is normal, and a sign of a healthy process. I'll volunteer as the BIPS editor. There needs to be some backups with commit access to bips.git, in case the BIPS editor is hit by a bus or goes crazy or on vacation. This can be some core devs, but I would like at least one or two folks who are not Satoshi-client devs on the list. Maybe Andreas, Michael G, Alan R, and others working on non-Satoshi clients. -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 14:34 ` Jeff Garzik @ 2013-10-21 15:46 ` Andreas Schildbach 2013-10-21 16:14 ` Jeff Garzik 0 siblings, 1 reply; 36+ messages in thread From: Andreas Schildbach @ 2013-10-21 15:46 UTC (permalink / raw) To: bitcoin-development On 10/21/2013 04:34 PM, Jeff Garzik wrote: > I'll volunteer as the BIPS editor. > > There needs to be some backups with commit access to bips.git, in case > the BIPS editor is hit by a bus or goes crazy or on vacation. This > can be some core devs, but I would like at least one or two folks who > are not Satoshi-client devs on the list. Maybe Andreas, Michael G, > Alan R, and others working on non-Satoshi clients. I accept the nomination as a backup (-: So the duty of the editor is merging pull requests and/or proxying between email and git for those who do not use git? ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 15:46 ` Andreas Schildbach @ 2013-10-21 16:14 ` Jeff Garzik 0 siblings, 0 replies; 36+ messages in thread From: Jeff Garzik @ 2013-10-21 16:14 UTC (permalink / raw) To: Andreas Schildbach; +Cc: Bitcoin Dev On Mon, Oct 21, 2013 at 11:46 AM, Andreas Schildbach <andreas@schildbach•de> wrote: > I accept the nomination as a backup (-: Cool. > So the duty of the editor is merging pull requests and/or proxying > between email and git for those who do not use git? Correct. And assigning BIP numbers. Ideally a boring administrative position. :) The main tensions will be in gauging whether there is sufficient consensus and review to boost a draft into BIP/proposed status, and then promoting a numbered BIP to the final/accepted status. -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 14:30 [Bitcoin-development] Revisiting the BIPS process, a proposal Jeff Garzik 2013-10-21 14:34 ` Jeff Garzik @ 2013-10-21 17:17 ` Jeff Garzik 2013-10-21 19:38 ` Jean-Paul Kogelman 2013-11-19 16:32 ` Wladimir 2 siblings, 1 reply; 36+ messages in thread From: Jeff Garzik @ 2013-10-21 17:17 UTC (permalink / raw) To: Bitcoin Dev Added: I'm happy with gmaxwell as BIP editor as well, as he is apparently the current BIP-number-assigner-in-chief. :) The goal is to improve the process, hash-seal our specs, and create an easy way for anyone with at least an email address to participate. On Mon, Oct 21, 2013 at 10:30 AM, Jeff Garzik <jgarzik@bitpay•com> wrote: > This summarizes some rambling on IRC about revising the BIPS process. > > Right now, the BIPS process is a bit haphazard. Previously, BIPS were > in a git repo, and the BIPS on the wiki were locked against editing. > The BIPS editor at the time started off well, but was eventually > M.I.A. So the BIPS "home" moved de facto to where everyone was > reading them anyway, the wiki. They were made editable, and it became > easier to Just Pick A Number And Write One. However, this inevitably > became a bit disorganized. Further, there was a recent incident -- > easily reverted -- where someone hopped on the wiki and started > arbitrarily editing an existing standard. > > BIPs need to move back to git, in my opinion. Standards should be > hash-sealed against corruption. Anything less would be uncivilized, > and un-bitcoin. However, many on IRC pointed out requiring a git pull > request might be a burdensome process, and discourage some > contributors. The following is a sketch of an improved process. > > 1) BIP Draft. > > Modelled after IETF drafts. Anybody may submit a BIP draft, as long > as it meets two very loose requirements: > * At least somewhat related to bitcoin. Note, I did not say "crypto-currency". > * Formatted similarly to existing BIPs (i.e. markdown, or whatever the > community prefers) > > BIP drafts may be submitted via git pull request, or by emailing an > attachment to bips.editor@bitcoin•org. This mirrors the Linux kernel > change submission process: git is preferred, but there is always a > non-git method for folks who cannot or do not wish to use git or > github. > > BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and > are not automatically assigned a BIPS number. > > 2) Time passes. Software for BIP drafts is developed, tested, > published, and publicly discussed in a typical open source manner. > > 3) If interest and use cases remain strong, a BIP number may be > requested, and the BIP draft is moved to > git://github.com/bitcoin/bips.git main directory. > > 4) If there is general consensus that the BIP should be adopted, the > BIP status is changed to "accepted." > > There are no specified time limits. Sometimes consensus about a BIP > is reached in days, sometimes 12+ months or more. It varies widely > depending on the feature's complexity and impact. > > As with the IETF, it will be q > > -- > Jeff Garzik > Senior Software Engineer and open source evangelist > BitPay, Inc. https://bitpay.com/ -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 17:17 ` Jeff Garzik @ 2013-10-21 19:38 ` Jean-Paul Kogelman 2013-10-21 19:47 ` Luke-Jr 0 siblings, 1 reply; 36+ messages in thread From: Jean-Paul Kogelman @ 2013-10-21 19:38 UTC (permalink / raw) To: Jeff Garzik; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 519 bytes --] I have some more questions. 1) Should the protocol specification page also be codified into BIP(s)? 2) Should the current wiki pages be taken down / forwarded to the git repo or be auto updated from the git repo? 3) Even though the information in BIP 50 is valuable, should it really be considered a BIP? On Oct 21, 2013, at 10:17 AM, Jeff Garzik <jgarzik@bitpay•com> wrote: The goal is to improve the process, hash-seal our specs, and create an easy way for anyone with at least an email address to participate. [-- Attachment #2.1: Type: text/html, Size: 784 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 19:38 ` Jean-Paul Kogelman @ 2013-10-21 19:47 ` Luke-Jr 2013-10-21 20:57 ` Benjamin Cordes ` (2 more replies) 0 siblings, 3 replies; 36+ messages in thread From: Luke-Jr @ 2013-10-21 19:47 UTC (permalink / raw) To: bitcoin-development On Monday, October 21, 2013 7:38:37 PM Jean-Paul Kogelman wrote: > 1) Should the protocol specification page also be codified into BIP(s)? Probably wouldn't hurt, but it'd likely need a rewrite in a more modular and formal form. > 2) Should the current wiki pages be taken down / forwarded to the git repo > or be auto updated from the git repo? Since it's the same format, I'd keep it up there, maybe with a link to the git repo on the main BIP index wiki page. > 3) Even though the information in BIP 50 is valuable, should it really be > considered a BIP? It's a hardforking protocol change, so IMO yes. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 19:47 ` Luke-Jr @ 2013-10-21 20:57 ` Benjamin Cordes 2013-10-21 20:59 ` Benjamin Cordes 2013-10-22 6:39 ` Martin Sustrik 2 siblings, 0 replies; 36+ messages in thread From: Benjamin Cordes @ 2013-10-21 20:57 UTC (permalink / raw) To: Luke-Jr; +Cc: bitcoin-development [-- Attachment #1: Type: text/plain, Size: 1970 bytes --] I believe a better solution would to use a gitlab clone such as gitlab, which sits on top of the git repo, and allows for custom code around the BIP process. Potentially one could even build Bitcoin into such a BIP system. If somebody wants to support a BIP he donates Bitcoins to that proposal. Somebody who actually implements the BIP can receive some percent of the bounty (while some percent goes to the Bitcoin foundation). Via such a platform one could create assurance contracts to kickstart BIP developments or Bitcoin extensions (public infrastructure which is not part of the core, such as opensourced exchanges). On Mon, Oct 21, 2013 at 9:47 PM, Luke-Jr <luke@dashjr•org> wrote: > On Monday, October 21, 2013 7:38:37 PM Jean-Paul Kogelman wrote: > > 1) Should the protocol specification page also be codified into BIP(s)? > > Probably wouldn't hurt, but it'd likely need a rewrite in a more modular > and > formal form. > > > 2) Should the current wiki pages be taken down / forwarded to the git > repo > > or be auto updated from the git repo? > > Since it's the same format, I'd keep it up there, maybe with a link to the > git > repo on the main BIP index wiki page. > > > 3) Even though the information in BIP 50 is valuable, should it really be > > considered a BIP? > > It's a hardforking protocol change, so IMO yes. > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > [-- Attachment #2: Type: text/html, Size: 2788 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 19:47 ` Luke-Jr 2013-10-21 20:57 ` Benjamin Cordes @ 2013-10-21 20:59 ` Benjamin Cordes 2013-10-22 6:39 ` Martin Sustrik 2 siblings, 0 replies; 36+ messages in thread From: Benjamin Cordes @ 2013-10-21 20:59 UTC (permalink / raw) To: bitcoin-development [-- Attachment #1: Type: text/plain, Size: 1972 bytes --] I believe a better solution would to use a github clone such as gitlab, which sits on top of the git repo, and allows for custom code around the BIP process. Potentially one could even build Bitcoin into such a BIP system. If somebody wants to support a BIP he donates Bitcoins to that proposal. Somebody who actually implements the BIP can receive some percent of the bounty, while some percentage goes to the Bitcoin foundation. Via such a platform one could create assurance contracts to kickstart BIP developments or Bitcoin extensions (public infrastructure which is not part of the core, such as opensourced exchanges). On Mon, Oct 21, 2013 at 9:47 PM, Luke-Jr <luke@dashjr•org> wrote: > On Monday, October 21, 2013 7:38:37 PM Jean-Paul Kogelman wrote: > > 1) Should the protocol specification page also be codified into BIP(s)? > > Probably wouldn't hurt, but it'd likely need a rewrite in a more modular > and > formal form. > > > 2) Should the current wiki pages be taken down / forwarded to the git > repo > > or be auto updated from the git repo? > > Since it's the same format, I'd keep it up there, maybe with a link to the > git > repo on the main BIP index wiki page. > > > 3) Even though the information in BIP 50 is valuable, should it really be > > considered a BIP? > > It's a hardforking protocol change, so IMO yes. > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > [-- Attachment #2: Type: text/html, Size: 2859 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 19:47 ` Luke-Jr 2013-10-21 20:57 ` Benjamin Cordes 2013-10-21 20:59 ` Benjamin Cordes @ 2013-10-22 6:39 ` Martin Sustrik 2013-10-22 6:59 ` Jean-Paul Kogelman 2 siblings, 1 reply; 36+ messages in thread From: Martin Sustrik @ 2013-10-22 6:39 UTC (permalink / raw) To: Luke-Jr, bitcoin-development On 21/10/13 21:47, Luke-Jr wrote: > On Monday, October 21, 2013 7:38:37 PM Jean-Paul Kogelman wrote: >> 1) Should the protocol specification page also be codified into BIP(s)? > > Probably wouldn't hurt, but it'd likely need a rewrite in a more modular and > formal form. I wanted to have a look at how the whole Bitcoin thing works recently. Being a distributed application, I've searched for the protocol spec. What I found were two wiki pages (Protocol & ProtocolRules) that looked more like notes someone wrote down while implementing the application. Have I missed something? Is there any effort underway trying to produce a decent spec? If not so, I am willing to help with that. Martin ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-22 6:39 ` Martin Sustrik @ 2013-10-22 6:59 ` Jean-Paul Kogelman 2013-10-22 7:03 ` Gregory Maxwell 0 siblings, 1 reply; 36+ messages in thread From: Jean-Paul Kogelman @ 2013-10-22 6:59 UTC (permalink / raw) To: Martin Sustrik; +Cc: bitcoin-development [-- Attachment #1: Type: text/plain, Size: 503 bytes --] > I wanted to have a look at how the whole Bitcoin thing works recently. > Being a distributed application, I've searched for the protocol spec. > What I found were two wiki pages (Protocol & ProtocolRules) that looked > more like notes someone wrote down while implementing the application. > > Have I missed something? Is there any effort underway trying to produce > a decent spec? If not so, I am willing to help with that. Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ? [-- Attachment #2: Message signed with OpenPGP using GPGMail --] [-- Type: application/pgp-signature, Size: 842 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-22 6:59 ` Jean-Paul Kogelman @ 2013-10-22 7:03 ` Gregory Maxwell 2013-10-22 7:34 ` Martin Sustrik 0 siblings, 1 reply; 36+ messages in thread From: Gregory Maxwell @ 2013-10-22 7:03 UTC (permalink / raw) To: Jean-Paul Kogelman; +Cc: Bitcoin Development On Mon, Oct 21, 2013 at 11:59 PM, Jean-Paul Kogelman <jeanpaulkogelman@me•com> wrote: > Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ? Take care, the information in the wiki is woefully incomplete. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-22 7:03 ` Gregory Maxwell @ 2013-10-22 7:34 ` Martin Sustrik 2013-10-22 7:49 ` Peter Todd ` (2 more replies) 0 siblings, 3 replies; 36+ messages in thread From: Martin Sustrik @ 2013-10-22 7:34 UTC (permalink / raw) To: Gregory Maxwell, Jean-Paul Kogelman; +Cc: Bitcoin Development On 22/10/13 09:03, Gregory Maxwell wrote: > On Mon, Oct 21, 2013 at 11:59 PM, Jean-Paul Kogelman > <jeanpaulkogelman@me•com> wrote: >> Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ? > > Take care, the information in the wiki is woefully incomplete. Imagine myself, with no prior knowledge of Bitcoin looking at the document. It starts with "Hashes". What hashes? No idea what's going on. Etc. Now compare that to a well written RFC. It starts with introduction, description of the problem, explains the conceptual model of the solution, then dives into the details. There's also Security Considerations part in every RFC that is pretty relevant for Bitcoin. As I said, I am willing to help with writing such document, it would be a nice way of learning the stuff, however, help from core devs, such as answering question that may arise in the process, or reviewing the document would be needed. Martin ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-22 7:34 ` Martin Sustrik @ 2013-10-22 7:49 ` Peter Todd 2013-10-22 7:56 ` Gregory Maxwell 2013-10-22 14:08 ` Jeff Garzik 2 siblings, 0 replies; 36+ messages in thread From: Peter Todd @ 2013-10-22 7:49 UTC (permalink / raw) To: Martin Sustrik; +Cc: Bitcoin Development [-- Attachment #1: Type: text/plain, Size: 1745 bytes --] On Tue, Oct 22, 2013 at 09:34:57AM +0200, Martin Sustrik wrote: > On 22/10/13 09:03, Gregory Maxwell wrote: > > On Mon, Oct 21, 2013 at 11:59 PM, Jean-Paul Kogelman > > <jeanpaulkogelman@me•com> wrote: > >> Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ? > > > > Take care, the information in the wiki is woefully incomplete. > > Imagine myself, with no prior knowledge of Bitcoin looking at the > document. It starts with "Hashes". What hashes? No idea what's going on. > Etc. > > Now compare that to a well written RFC. It starts with introduction, > description of the problem, explains the conceptual model of the > solution, then dives into the details. There's also Security > Considerations part in every RFC that is pretty relevant for Bitcoin. > > As I said, I am willing to help with writing such document, it would be > a nice way of learning the stuff, however, help from core devs, such as > answering question that may arise in the process, or reviewing the > document would be needed. Writing such RFCs is dangerous due to the consensus nature of Bitcoin - it makes people think the standard is the RFC, rather than the code. I hear one of the better intros to Bitcoin is the Khan academy videos, but I've never watched them myself. Once you understand how it works, start reading source code - the Bitcoin codebase is actually really simple and readable. However remember that the implications of that codebase are anything but simple; there's lots of reasons to think Satoshi himself didn't understand Bitcoin all that well, even by the time he left the project. -- 'peter'[:-1]@petertodd.org 000000000000000f155e7a648e84a83589048ae1cacb0c60bfce2437553b6af4 [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 685 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-22 7:34 ` Martin Sustrik 2013-10-22 7:49 ` Peter Todd @ 2013-10-22 7:56 ` Gregory Maxwell 2013-10-22 8:20 ` Martin Sustrik 2013-10-22 14:08 ` Jeff Garzik 2 siblings, 1 reply; 36+ messages in thread From: Gregory Maxwell @ 2013-10-22 7:56 UTC (permalink / raw) To: Martin Sustrik; +Cc: Bitcoin Development On Tue, Oct 22, 2013 at 12:34 AM, Martin Sustrik <sustrik@250bpm•com> wrote: > There's also Security Considerations part in > every RFC that is pretty relevant for Bitcoin. Which would say something interesting like "If the bitcoin network implements inconsistent behavior in the consensus critical parts of the protocol the world ends. As such, conformance or _non_-conformance with this specification (in particular, sections 4. 5. and 6.) may be required for security." A Bitcoin protocol RFC would be a great place to exercise RFC 6919 keywords. ( http://tools.ietf.org/html/rfc6919 ) ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-22 7:56 ` Gregory Maxwell @ 2013-10-22 8:20 ` Martin Sustrik 0 siblings, 0 replies; 36+ messages in thread From: Martin Sustrik @ 2013-10-22 8:20 UTC (permalink / raw) To: Gregory Maxwell; +Cc: Bitcoin Development On 22/10/13 09:56, Gregory Maxwell wrote: > On Tue, Oct 22, 2013 at 12:34 AM, Martin Sustrik <sustrik@250bpm•com> wrote: >> There's also Security Considerations part in >> every RFC that is pretty relevant for Bitcoin. > > Which would say something interesting like "If the bitcoin network > implements inconsistent behavior in the consensus critical parts of > the protocol the world ends. As such, conformance or _non_-conformance > with this specification (in particular, sections 4. 5. and 6.) may be > required for security." In fact, yes. In the end it boils down to saying something like: "Bitcoin is a unique global distributed application and thus all implementations MUST support the version of the protocol currently in use, irrespective of whether it have been documented and/or published. This RFC is meant only for informational purposes and is a snapshot of the protocol as to Oct 22nd 2013." That being said, I understand the idea of not publishing the spec so that everyone is forced to work with live data. > A Bitcoin protocol RFC would be a great place to exercise RFC 6919 > keywords. ( http://tools.ietf.org/html/rfc6919 ) Heh. Haven't seen that one. Martin ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-22 7:34 ` Martin Sustrik 2013-10-22 7:49 ` Peter Todd 2013-10-22 7:56 ` Gregory Maxwell @ 2013-10-22 14:08 ` Jeff Garzik 2013-10-23 7:38 ` Martin Sustrik 2 siblings, 1 reply; 36+ messages in thread From: Jeff Garzik @ 2013-10-22 14:08 UTC (permalink / raw) To: Martin Sustrik; +Cc: Bitcoin Development All that is good practice, but we should avoid adding burdensome process that might discourage BIP writing. Consider a distributed approach: if you feel a draft needs more sections or better language, submit a pull request yourself and help community-edit the document. On Tue, Oct 22, 2013 at 3:34 AM, Martin Sustrik <sustrik@250bpm•com> wrote: > On 22/10/13 09:03, Gregory Maxwell wrote: >> On Mon, Oct 21, 2013 at 11:59 PM, Jean-Paul Kogelman >> <jeanpaulkogelman@me•com> wrote: >>> Have you seen: https://en.bitcoin.it/wiki/Protocol_specification ? >> >> Take care, the information in the wiki is woefully incomplete. > > Imagine myself, with no prior knowledge of Bitcoin looking at the > document. It starts with "Hashes". What hashes? No idea what's going on. > Etc. > > Now compare that to a well written RFC. It starts with introduction, > description of the problem, explains the conceptual model of the > solution, then dives into the details. There's also Security > Considerations part in every RFC that is pretty relevant for Bitcoin. > > As I said, I am willing to help with writing such document, it would be > a nice way of learning the stuff, however, help from core devs, such as > answering question that may arise in the process, or reviewing the > document would be needed. > > Martin > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-22 14:08 ` Jeff Garzik @ 2013-10-23 7:38 ` Martin Sustrik 2013-10-23 19:40 ` Peter Todd 0 siblings, 1 reply; 36+ messages in thread From: Martin Sustrik @ 2013-10-23 7:38 UTC (permalink / raw) To: Jeff Garzik; +Cc: Bitcoin Development On 22/10/13 16:08, Jeff Garzik wrote: > All that is good practice, but we should avoid adding burdensome > process that might discourage BIP writing. > > Consider a distributed approach: if you feel a draft needs more > sections or better language, submit a pull request yourself and help > community-edit the document. I would love to do so. However, from what Peter Todd said above, my feeling was that spec is deliberately vague to force compatibility with the reference implementation rather than with a document. While that kind of compatibility-via-obscurity won't probably work in a long run, in short run it can prevent proliferation of implementations and thus give protocol more space and flexibility to evolve (I've done the same trick with ZeroMQ myself once). Anyway, if my impression was wrong I am happy to give it a try. Martin ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-23 7:38 ` Martin Sustrik @ 2013-10-23 19:40 ` Peter Todd 2013-10-23 20:05 ` Martin Sustrik 0 siblings, 1 reply; 36+ messages in thread From: Peter Todd @ 2013-10-23 19:40 UTC (permalink / raw) To: Martin Sustrik; +Cc: Bitcoin Development [-- Attachment #1: Type: text/plain, Size: 1475 bytes --] On Wed, Oct 23, 2013 at 09:38:31AM +0200, Martin Sustrik wrote: > On 22/10/13 16:08, Jeff Garzik wrote: > > All that is good practice, but we should avoid adding burdensome > > process that might discourage BIP writing. > > > > Consider a distributed approach: if you feel a draft needs more > > sections or better language, submit a pull request yourself and help > > community-edit the document. > > I would love to do so. > > However, from what Peter Todd said above, my feeling was that spec is > deliberately vague to force compatibility with the reference > implementation rather than with a document. > > While that kind of compatibility-via-obscurity won't probably work in a > long run, in short run it can prevent proliferation of implementations > and thus give protocol more space and flexibility to evolve (I've done > the same trick with ZeroMQ myself once). The reference implementation is the specification - the "specification" on the wiki is best thought of as a set of Coles Notes on the real specification. If you don't already understand that and the nuance of that statement you should assume the protocol is fixed in stone and doesn't evolve at all; that statement is not quite true, but it's very close to the truth. I gotta get around to writing a "Developers" section for the FAQ explaining this stuff.... -- 'peter'[:-1]@petertodd.org 0000000000000007362b283ac07839aba795dbfb3c5c4e831d80df9cf3bea2d5 [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-23 19:40 ` Peter Todd @ 2013-10-23 20:05 ` Martin Sustrik 2013-10-23 20:27 ` Peter Todd 0 siblings, 1 reply; 36+ messages in thread From: Martin Sustrik @ 2013-10-23 20:05 UTC (permalink / raw) To: Peter Todd; +Cc: Bitcoin Development On 23/10/13 21:40, Peter Todd wrote: > The reference implementation is the specification - the "specification" > on the wiki is best thought of as a set of Coles Notes on the real > specification. If you don't already understand that and the nuance of > that statement you should assume the protocol is fixed in stone and > doesn't evolve at all; that statement is not quite true, but it's very > close to the truth. Does that imply that the notes are deliberately obscured to force everyone to check the source code? Martin ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-23 20:05 ` Martin Sustrik @ 2013-10-23 20:27 ` Peter Todd 2013-10-23 21:07 ` Pieter Wuille 0 siblings, 1 reply; 36+ messages in thread From: Peter Todd @ 2013-10-23 20:27 UTC (permalink / raw) To: Martin Sustrik; +Cc: Bitcoin Development [-- Attachment #1: Type: text/plain, Size: 844 bytes --] On Wed, Oct 23, 2013 at 10:05:56PM +0200, Martin Sustrik wrote: > On 23/10/13 21:40, Peter Todd wrote: > > >The reference implementation is the specification - the "specification" > >on the wiki is best thought of as a set of Coles Notes on the real > >specification. If you don't already understand that and the nuance of > >that statement you should assume the protocol is fixed in stone and > >doesn't evolve at all; that statement is not quite true, but it's very > >close to the truth. > > Does that imply that the notes are deliberately obscured to force > everyone to check the source code? What's on the wiki is mostly the work of people who aren't working on the reference implementation, so no, you can't say that. -- 'peter'[:-1]@petertodd.org 0000000000000003c1d48b638b9857cb56b6fe9188a60c481fbc9b738ccb4663 [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-23 20:27 ` Peter Todd @ 2013-10-23 21:07 ` Pieter Wuille 2013-10-23 21:42 ` Allen Piscitello 2013-10-24 7:03 ` Martin Sustrik 0 siblings, 2 replies; 36+ messages in thread From: Pieter Wuille @ 2013-10-23 21:07 UTC (permalink / raw) To: Peter Todd; +Cc: Bitcoin Development On Wed, Oct 23, 2013 at 10:27 PM, Peter Todd <pete@petertodd•org> wrote: > On Wed, Oct 23, 2013 at 10:05:56PM +0200, Martin Sustrik wrote: >> On 23/10/13 21:40, Peter Todd wrote: >> >> >The reference implementation is the specification - the "specification" >> >on the wiki is best thought of as a set of Coles Notes on the real >> >specification. If you don't already understand that and the nuance of >> >that statement you should assume the protocol is fixed in stone and >> >doesn't evolve at all; that statement is not quite true, but it's very >> >close to the truth. >> >> Does that imply that the notes are deliberately obscured to force >> everyone to check the source code? > > What's on the wiki is mostly the work of people who aren't working on > the reference implementation, so no, you can't say that. Indeed, I know of few people who are familiar with the source code that use the wiki. I do think that is a pity. The openness and transparency of the protocol is essential to trusting the system (and shouldn't be limited to those digging through the source code), and for that reason alone I think it needs to be well-documented. I also do agree with earlier comments, that due to the nature of the consensus problem Bitcoin solves, it will always be the network that dictates what the actual rules are - anything else can result in inresolvable forks. If a "formal" specification were written, and we would find out that the majority of nodes on the network deviate from it in a subtle way, those nodes would be buggy in the sense that they aren't doing what was expected, but it would be the specification that is incorrect for not following the rules of the network. In short, consistency is more important than correctness, and for that reason, writing alternate implementation will always be hard and dangerous. However, I do not think that making it hard to find information about the details of the system is the way to go. Alternate implementations are likely inevitable, and in the long run probably a win for the ecosystem. If effort is put into accurately describing the rules, it should indeed carry a strong notice about it being descriptive rather than normative. If someone is willing to work on that, I am (and likely many people in #bitcoin-dev are) available for any questions about the protocol and its semantics. -- Pieter ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-23 21:07 ` Pieter Wuille @ 2013-10-23 21:42 ` Allen Piscitello 2013-10-23 21:49 ` Luke-Jr 2013-10-24 7:03 ` Martin Sustrik 1 sibling, 1 reply; 36+ messages in thread From: Allen Piscitello @ 2013-10-23 21:42 UTC (permalink / raw) To: Bitcoin Development [-- Attachment #1: Type: text/plain, Size: 3930 bytes --] I think formalizing the specification could go a long way and encouraging alternate implementations is going to be the best way to reduce unexpected small bugs having a bad effect except on the "buggy" node. That being said, it's a huge chicken and egg problem. No one wants to go off the reference client since it could lead to working on a forked chain as a miner or having bad data as a client. I don't know if there is a good way to try to take small pieces, get consensus, possibly have some type of universal test cases and running on testnet that would solve the problem. I wouldn't mind taking on parts of this when I have time, specifically transactions/scripting. Obviously if there are better qualified people who are interested, have at it! On Wed, Oct 23, 2013 at 4:07 PM, Pieter Wuille <pieter.wuille@gmail•com>wrote: > On Wed, Oct 23, 2013 at 10:27 PM, Peter Todd <pete@petertodd•org> wrote: > > On Wed, Oct 23, 2013 at 10:05:56PM +0200, Martin Sustrik wrote: > >> On 23/10/13 21:40, Peter Todd wrote: > >> > >> >The reference implementation is the specification - the "specification" > >> >on the wiki is best thought of as a set of Coles Notes on the real > >> >specification. If you don't already understand that and the nuance of > >> >that statement you should assume the protocol is fixed in stone and > >> >doesn't evolve at all; that statement is not quite true, but it's very > >> >close to the truth. > >> > >> Does that imply that the notes are deliberately obscured to force > >> everyone to check the source code? > > > > What's on the wiki is mostly the work of people who aren't working on > > the reference implementation, so no, you can't say that. > > Indeed, I know of few people who are familiar with the source code > that use the wiki. > > I do think that is a pity. The openness and transparency of the > protocol is essential to trusting the system (and shouldn't be limited > to those digging through the source code), and for that reason alone I > think it needs to be well-documented. > > I also do agree with earlier comments, that due to the nature of the > consensus problem Bitcoin solves, it will always be the network that > dictates what the actual rules are - anything else can result in > inresolvable forks. If a "formal" specification were written, and we > would find out that the majority of nodes on the network deviate from > it in a subtle way, those nodes would be buggy in the sense that they > aren't doing what was expected, but it would be the specification that > is incorrect for not following the rules of the network. In short, > consistency is more important than correctness, and for that reason, > writing alternate implementation will always be hard and dangerous. > > However, I do not think that making it hard to find information about > the details of the system is the way to go. Alternate implementations > are likely inevitable, and in the long run probably a win for the > ecosystem. If effort is put into accurately describing the rules, it > should indeed carry a strong notice about it being descriptive rather > than normative. > > If someone is willing to work on that, I am (and likely many people in > #bitcoin-dev are) available for any questions about the protocol and > its semantics. > > -- > Pieter > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > [-- Attachment #2: Type: text/html, Size: 5113 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-23 21:42 ` Allen Piscitello @ 2013-10-23 21:49 ` Luke-Jr 0 siblings, 0 replies; 36+ messages in thread From: Luke-Jr @ 2013-10-23 21:49 UTC (permalink / raw) To: bitcoin-development On Wednesday, October 23, 2013 9:42:14 PM Allen Piscitello wrote: > That being said, it's a huge chicken and egg problem. No one wants to go > off the reference client since it could lead to working on a forked chain > as a miner or having bad data as a client. Thankfully, miners are incentivised to run one of every widespread node to ensure their blocks are accepted by the network. Eloipool already supports cross-referencing block templates between multiple clients and using the one that is accepted by most/all (and logging any discrepancies with coredump-like details). Luke ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-23 21:07 ` Pieter Wuille 2013-10-23 21:42 ` Allen Piscitello @ 2013-10-24 7:03 ` Martin Sustrik 2013-10-24 10:39 ` Jeff Garzik 2013-10-24 11:11 ` Christian Decker 1 sibling, 2 replies; 36+ messages in thread From: Martin Sustrik @ 2013-10-24 7:03 UTC (permalink / raw) To: Pieter Wuille, Peter Todd; +Cc: Bitcoin Development On 23/10/13 23:07, Pieter Wuille wrote: > In short, > consistency is more important than correctness. That's a nice and concise way to put it and any potential protocol documentation should start with a statement like that. > However, I do not think that making it hard to find information about > the details of the system is the way to go. Alternate implementations > are likely inevitable, and in the long run probably a win for the > ecosystem. If effort is put into accurately describing the rules, it > should indeed carry a strong notice about it being descriptive rather > than normative. One interesting question is whather alternative implementations are more likely to get it wrong because the protocol description is wrong or because the authors misunderstood the reference implementation source code. Extensive documentation of the source code, a la Knuth's literate programming, may be some kind of a middle ground. > If someone is willing to work on that, I am (and likely many people in > #bitcoin-dev are) available for any questions about the protocol and > its semantics. Ok. Several people expressed an interest in the topic, so I'll give it a try and see how it fares. Martin ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-24 7:03 ` Martin Sustrik @ 2013-10-24 10:39 ` Jeff Garzik 2013-10-24 11:11 ` Christian Decker 1 sibling, 0 replies; 36+ messages in thread From: Jeff Garzik @ 2013-10-24 10:39 UTC (permalink / raw) To: Martin Sustrik; +Cc: Bitcoin Development Yes. I had pointed people in IRC to Knuth's literate programming, as an example of how we might document bitcoin. On Thu, Oct 24, 2013 at 3:03 AM, Martin Sustrik <sustrik@250bpm•com> wrote: > On 23/10/13 23:07, Pieter Wuille wrote: > >> In short, >> consistency is more important than correctness. > > That's a nice and concise way to put it and any potential protocol > documentation should start with a statement like that. > >> However, I do not think that making it hard to find information about >> the details of the system is the way to go. Alternate implementations >> are likely inevitable, and in the long run probably a win for the >> ecosystem. If effort is put into accurately describing the rules, it >> should indeed carry a strong notice about it being descriptive rather >> than normative. > > One interesting question is whather alternative implementations are more > likely to get it wrong because the protocol description is wrong or > because the authors misunderstood the reference implementation source code. > > Extensive documentation of the source code, a la Knuth's literate > programming, may be some kind of a middle ground. > >> If someone is willing to work on that, I am (and likely many people in >> #bitcoin-dev are) available for any questions about the protocol and >> its semantics. > > Ok. Several people expressed an interest in the topic, so I'll give it a > try and see how it fares. > > Martin > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/ ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-24 7:03 ` Martin Sustrik 2013-10-24 10:39 ` Jeff Garzik @ 2013-10-24 11:11 ` Christian Decker 2013-10-24 19:43 ` Jeremy Spilman 1 sibling, 1 reply; 36+ messages in thread From: Christian Decker @ 2013-10-24 11:11 UTC (permalink / raw) To: Martin Sustrik; +Cc: Bitcoin Development I'd like to add some historical background about how the "protocol specification" came to be in the first place. A bit over three years [1] ago I started an attempt to document the network protocol, by reverse engineering it from the satoshi client. My goal, back then, was to enable like-minded engineers to create alternative clients and move away from the client-monoculture that is still predominant today. It was clear from the beginning that it would merely be a reverse engineering effort, and that it would likely lag a bit behind the changes in the main client. It was meant as a help for engineers that are not well versed in C/C++ to enable them to contribute by creating new clients, but the satoshi client would always be the de-facto standard. With the move from Google Code to the Bitcoin.it wiki somehow this notion of it being a reverse engineering effort was lost and people started assuming that if the behavior of the satoshi client did not match the protocol description it was a bug on the client side. Instead it is because the reverse engineering of the protocol is incorrect or simply missing some details. Although the protocol description is far more complete than it was back when we started, I still don't feel comfortable giving it the name specification. I still believe that a client monoculture is bad for the system as a whole, because a single bug might bring down the whole network. Giving people the necessary tools to implement new clients brings stability. I do understand the criticism that writing a specification might hinder future development as it restricts the possible changes to the protocol, but isn't this already the case as long as we have legacy versions of the client participating in the network? I would also argue that having a specification allows an application independent review of the protocol to identify possible improvements and bugs. I think the protocol description has an important place in the development of Bitcoin, so much so that we pushed a long time ago to separate protocol version from the client version. I would love to see the protocol specification becoming official part of the bitcoin github repository, which would ideally be maintained alongside the satoshi client to keep it up to date. Regards, Christian Decker [1] https://bitcointalk.org/index.php?topic=231 -- Christian Decker On Thu, Oct 24, 2013 at 9:03 AM, Martin Sustrik <sustrik@250bpm•com> wrote: > On 23/10/13 23:07, Pieter Wuille wrote: > >> In short, >> consistency is more important than correctness. > > That's a nice and concise way to put it and any potential protocol > documentation should start with a statement like that. > >> However, I do not think that making it hard to find information about >> the details of the system is the way to go. Alternate implementations >> are likely inevitable, and in the long run probably a win for the >> ecosystem. If effort is put into accurately describing the rules, it >> should indeed carry a strong notice about it being descriptive rather >> than normative. > > One interesting question is whather alternative implementations are more > likely to get it wrong because the protocol description is wrong or > because the authors misunderstood the reference implementation source code. > > Extensive documentation of the source code, a la Knuth's literate > programming, may be some kind of a middle ground. > >> If someone is willing to work on that, I am (and likely many people in >> #bitcoin-dev are) available for any questions about the protocol and >> its semantics. > > Ok. Several people expressed an interest in the topic, so I'll give it a > try and see how it fares. > > Martin > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-24 11:11 ` Christian Decker @ 2013-10-24 19:43 ` Jeremy Spilman 0 siblings, 0 replies; 36+ messages in thread From: Jeremy Spilman @ 2013-10-24 19:43 UTC (permalink / raw) To: Bitcoin Development Thanks Christian, this is a really interesting bit of history. My own personal experience from when I wrote my own client and BCCAPI-ish server was that the protocol specification on the Wiki was hugely valuable, and rarely sent me astray. Supplement that with the occasional questions on #bitcoin-dev, and then just coding, coding, coding and getting unit tests to pass. Nothing compares (IMO) to stepping through your own code watching the unit tests run, scripts evaluate, calculating transaction hashes for the different SIGHASH modes, and finally getting your first transaction into the block chain. I really appreciated the .json files holding the unit test data, which were easy to load into my own test harness, the tables on the Wiki showing what the stack should look like at each point in a script execution, and the diagrams showing transaction signing. Bitcoin takes some time to "grok" when you first approach; more than a day, less than a month, and really no amount of reading documentation or specs will get you to that "ah ha" moment. When the fog lifts and the blockchain, scripting, signing, and wallet handling really click, suddenly the bitcoind code (and many other great public sources in just about any language you could want) actually does starts to feel fairly simple and obvious. But it certainly doesn't start out that way on day one. I think the majority of client code development is actually people writing 'agents' not end-user P2P wallets, and they tend to be written to connect to a single bitcoind acting as a proxy to the network. Even some end-user wallets work this way! As such, I spent very little time in my own client writing P2P protocol code, no peer discovery code, no anti-DoS, etc. Clients like this also don't pose much systemic risk, because they don't mine, they don't connect directly to external nodes, etc. They can certainly be used to "cause trouble" though, but so can 'sendrawtransaction'. I chose to speak the P2P protocol to bitcoind versus using some of the other options like ZeroMQ, but it still didn't take long to get headers, blocks, and transactions downloading. I remember getting stuck on the very first version message, because of missing the checksum and user-agent or something caused the latest bitcoind to just ignore me. A little wireshark capture of the exchange between two working bitcoind instances cleared it right up. I didn't mind the leg work, I don't think everything needs to be spoon fed, and it's certainly not purposefully obfuscated. Maybe one exception is the mix-matched endianness will throw you off, especially if you are developing on LE! Anyway, I have huge respect for how much effort it takes to keep even small bits of documentation up-to-date. For as "slow" as bitcoin moves, it's actually moving incredibly fast. Finally, the bitcoind console and debug logs, as well sites like blockchain.info and blockexplorer.com are hugely helpful for debugging raw and live transactions for when you get stuck. There's a surprisingly large tooling and support ecosystem out there. Moral of the story, I think, is everything you need is there. No, it's not all in one place. Yes, it takes time to find it and assimilate all that knowledge. It also really helps that the community is extremely willing to help and answer technical questions, and point you in the right direction, even when you're working on your own private client code. The IRC channel can certainly be intimidating because it seems like every time I hit enter to send a question, gmaxwell's respond 300ms later would invoke an immediate forehead slap and a groan of "shit, I knew/should have known that, now I feel dumb" ;-) but if you're working on bitcoin, you better get used to not being the smartest person in the room! The responses I got were never arrogant or disparaging, but they were straight to-the-point and surprisingly high quality. Ain't no slouches in that channel, yes you will have to bring your A-game and you are expected to have "tried first" before just asking. I have fairly limited experience working on open source projects, but I'm extremely happy with my experience with the Bitcoin community and found writing Bitcoin code hugely enjoyable. The flip side to helping people implement their own clients, agents, or even miners, is helping people to contribute pulls requests, or at the very highest echelon, a BIP. If you haven't written any significant Bitcoin code, you might want to consider investing in that first before submitting a BIP. :-) For a BIP to be valuable, often it requires widespread or even consensus adoption. BIPs are probably not the place to toss just any old 'good idea' because BIPs impose a cost on all active developers. I want to read and understand 'all the BIPs' because for the most part they are actually essential, like, how to handle duplicate transactions in BIP30 - if you don't read BIP30 you very likely totally miss that, until your code throws exceptions while processing block 91842. And perhaps the hardest kind of BIP of all is the "lets get wallets to add this user-facing feature" where it has no bearing on the blockchain or transaction processing, it doesn't make the network more resilient or add crucial functionality for increasing scalability. Kind of like JPK's HD wallet encryption proposal, which I love, and I tried to contribute to in the forums, but I can totally understand the headwinds for making progress on BIPs like that one and BIP39. No one is against it per-say, it's just much harder to articulate and justify the NEED for everyone to implement, test, and support this new not-yet-standard, nice-to-have feature. For those kinds of BIPs you probably have to go out and get some wallets to implement it, or implement it yourself, to prove the value and kick start critical mass before you will even get enough support for getting a BIP number assigned. IMO, it's not a Bad Thing. TL;DR; The current support systems worked very well for me. I was able to accomplish all my goals, and I would even say it was a pleasure. Keep a high bar for assigning BIP numbers. And I hope to be able to jump back in and do more with Bitcoin soon. Thanks all, sorry if I'm rambling, Jeremy Spilman On Thu, 24 Oct 2013 04:11:05 -0700, Christian Decker <decker.christian@gmail•com> wrote: > I'd like to add some historical background about how the "protocol > specification" came to be in the first place. > > A bit over three years [1] ago I started an attempt to document the > network protocol, by reverse engineering it from the satoshi > client. My goal, back then, was to enable like-minded engineers to > create alternative clients and move away from the client-monoculture > that is still predominant today. It was clear from the beginning that > it would merely be a reverse engineering effort, and that it would > likely lag a bit behind the changes in the main client. It was meant > as a help for engineers that are not well versed in C/C++ to enable > them to contribute by creating new clients, but the satoshi client > would always be the de-facto standard. > > With the move from Google Code to the Bitcoin.it wiki somehow this > notion of it being a reverse engineering effort was lost and people > started assuming that if the behavior of the satoshi client did not > match the protocol description it was a bug on the client > side. Instead it is because the reverse engineering of the protocol is > incorrect or simply missing some details. Although the protocol > description is far more complete than it was back when we started, I > still don't feel comfortable giving it the name specification. > > I still believe that a client monoculture is bad for the system as a > whole, because a single bug might bring down the whole network. Giving > people the necessary tools to implement new clients brings > stability. I do understand the criticism that writing a specification > might hinder future development as it restricts the possible changes > to the protocol, but isn't this already the case as long as we have > legacy versions of the client participating in the network? I would > also argue that having a specification allows an application > independent review of the protocol to identify possible improvements > and bugs. > > I think the protocol description has an important place in the > development of Bitcoin, so much so that we pushed a long time ago to > separate protocol version from the client version. I would love to see > the protocol specification becoming official part of the bitcoin > github repository, which would ideally be maintained alongside the > satoshi client to keep it up to date. > > Regards, > Christian Decker > > [1] https://bitcointalk.org/index.php?topic=231 > -- > Christian Decker > > ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-10-21 14:30 [Bitcoin-development] Revisiting the BIPS process, a proposal Jeff Garzik 2013-10-21 14:34 ` Jeff Garzik 2013-10-21 17:17 ` Jeff Garzik @ 2013-11-19 16:32 ` Wladimir 2013-11-19 16:53 ` Drak 2013-11-19 17:06 ` Peter Todd 2 siblings, 2 replies; 36+ messages in thread From: Wladimir @ 2013-11-19 16:32 UTC (permalink / raw) To: Jeff Garzik; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 879 bytes --] On Mon, Oct 21, 2013 at 4:30 PM, Jeff Garzik <jgarzik@bitpay•com> wrote: > BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and > are not automatically assigned a BIPS number. > Are we going to move ahead with this? If so, I'm volunteering to create the repository and import the current BIPs from the wiki there (and convert from wiki markup to markdown where necessary). 2) Time passes. Software for BIP drafts is developed, tested, > published, and publicly discussed in a typical open source manner. > Personally I think it is useful to have a number as soon as a BIP can be implemented, even if still in draft status; it gives something to refer to when mentioning a certain improvement proposal (in commit messages and such it could be called BIP xxx Draft). I don't think we are at risk of running out of numbers to assign any time soon. Wladimir [-- Attachment #2: Type: text/html, Size: 1512 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-11-19 16:32 ` Wladimir @ 2013-11-19 16:53 ` Drak 2013-11-19 17:01 ` Gregory Maxwell 2013-11-19 17:06 ` Peter Todd 1 sibling, 1 reply; 36+ messages in thread From: Drak @ 2013-11-19 16:53 UTC (permalink / raw) To: Wladimir; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 1206 bytes --] On 19 November 2013 16:32, Wladimir <laanwj@gmail•com> wrote: > > On Mon, Oct 21, 2013 at 4:30 PM, Jeff Garzik <jgarzik@bitpay•com> wrote: > >> BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and >> are not automatically assigned a BIPS number. >> > > Are we going to move ahead with this? > > If so, I'm volunteering to create the repository and import the current > BIPs from the wiki there (and convert from wiki markup to markdown where > necessary). > > 2) Time passes. Software for BIP drafts is developed, tested, >> published, and publicly discussed in a typical open source manner. >> > > Personally I think it is useful to have a number as soon as a BIP can be > implemented, even if still in draft status; it gives something to refer to > when mentioning a certain improvement proposal (in commit messages and such > it could be called BIP xxx Draft). > I don't think we are at risk of running out of numbers to assign any time > soon. > It's quite normal for standards bodies to allocate numbers when in draft status. If they don't pass, they don't pass - they are clearly labelled DRAFTs. +1 on having things in a github repository. Much better for collaboration, Drak [-- Attachment #2: Type: text/html, Size: 2184 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-11-19 16:53 ` Drak @ 2013-11-19 17:01 ` Gregory Maxwell 2013-11-19 17:07 ` Drak 2013-11-19 17:45 ` Wladimir 0 siblings, 2 replies; 36+ messages in thread From: Gregory Maxwell @ 2013-11-19 17:01 UTC (permalink / raw) To: Drak; +Cc: Bitcoin Dev On Tue, Nov 19, 2013 at 8:53 AM, Drak <drak@zikula•org> wrote: > It's quite normal for standards bodies to allocate numbers when in draft > status. If they don't pass, they don't pass - they are clearly labelled > DRAFTs. > > +1 on having things in a github repository. Much better for collaboration, The IETF makes a clear distinction between individual proposals and documents which have been accepted by a working group. The former are named after their authors. Work is not assigned a number until it is complete. I believe it is important to distinguish complete work that people should be implementing from things which are incomplete, and even more important to distinguish the work of single parties. Otherwise you're going to get crap like BIP90: "Increase the supply of Bitcoins to 210 million" being confused as an earnest proposal supported by many that has traction. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-11-19 17:01 ` Gregory Maxwell @ 2013-11-19 17:07 ` Drak 2013-11-19 17:45 ` Wladimir 1 sibling, 0 replies; 36+ messages in thread From: Drak @ 2013-11-19 17:07 UTC (permalink / raw) To: Gregory Maxwell; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 1679 bytes --] On 19 November 2013 17:01, Gregory Maxwell <gmaxwell@gmail•com> wrote: > On Tue, Nov 19, 2013 at 8:53 AM, Drak <drak@zikula•org> wrote: > > It's quite normal for standards bodies to allocate numbers when in draft > > status. If they don't pass, they don't pass - they are clearly labelled > > DRAFTs. > > > > +1 on having things in a github repository. Much better for > collaboration, > > The IETF makes a clear distinction between individual proposals and > documents which have been accepted by a working group. The former are > named after their authors. Work is not assigned a number until it is > complete. > > I believe it is important to distinguish complete work that people > should be implementing from things which are incomplete, and even > more important to distinguish the work of single parties. > > Otherwise you're going to get crap like BIP90: "Increase the supply of > Bitcoins to 210 million" being confused as an earnest proposal > supported by many that has traction. > I wasnt suggesting people add drafts willy nilly to the repository. When working on a proposal you can work on it in your own fork and create a PR. When it's ready to be accepted as a working draft by the WG, then it can be merged into the draft folder. At which point, PRs are made to that draft copy until it gets into a ready state to become final. If passed, it's moved to the accepted/ folder. This way random BIPS cannot be added to the drafts/ folder in the official repo. They are only added once they are accepted as a working draft proposal by Gavin or whatever. Now you get all the niceties of github workflow for collaboration and tweaking of the draft proposal. Drak [-- Attachment #2: Type: text/html, Size: 2384 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-11-19 17:01 ` Gregory Maxwell 2013-11-19 17:07 ` Drak @ 2013-11-19 17:45 ` Wladimir 2013-11-19 17:54 ` Gregory Maxwell 1 sibling, 1 reply; 36+ messages in thread From: Wladimir @ 2013-11-19 17:45 UTC (permalink / raw) To: Gregory Maxwell; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 790 bytes --] On Tue, Nov 19, 2013 at 6:01 PM, Gregory Maxwell <gmaxwell@gmail•com> wrote: > On Tue, Nov 19, 2013 at 8:53 AM, Drak <drak@zikula•org> wrote: > > It's quite normal for standards bodies to allocate numbers when in draft > > status. If they don't pass, they don't pass - they are clearly labelled > > DRAFTs. > > > > +1 on having things in a github repository. Much better for > collaboration, > > The IETF makes a clear distinction between individual proposals and > documents which have been accepted by a working group. The former are > named after their authors. Work is not assigned a number until it is > complete. > Talking about complete, BIP 40 and 41 don't even have an associated document: https://github.com/bitcoin/bips I agree that was over-eager number assigning. Wladimir [-- Attachment #2: Type: text/html, Size: 1329 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-11-19 17:45 ` Wladimir @ 2013-11-19 17:54 ` Gregory Maxwell 0 siblings, 0 replies; 36+ messages in thread From: Gregory Maxwell @ 2013-11-19 17:54 UTC (permalink / raw) To: Wladimir; +Cc: Bitcoin Dev On Tue, Nov 19, 2013 at 9:45 AM, Wladimir <laanwj@gmail•com> wrote: > Talking about complete, BIP 40 and 41 don't even have an associated > document: > https://github.com/bitcoin/bips > I agree that was over-eager number assigning. Maybe! The subject matter its assigned for is already _widely_ deployed, for better or worse. (by comparison in the IETF, informational RFCs for already widely deployed things are issued pretty liberally) I'm not sure how we should be distinguish BIPs which are documenting things which are already defacto standards vs ones which are proposing that people do something new. Mostly I think we don't want the BIP itself being a lever to force something down people's throats, but rather the process should help build consensus and review about how to do something— and then document that consensus. ^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [Bitcoin-development] Revisiting the BIPS process, a proposal 2013-11-19 16:32 ` Wladimir 2013-11-19 16:53 ` Drak @ 2013-11-19 17:06 ` Peter Todd [not found] ` <CA+s+GJA=p+yvoJqUAMQQRcfYK1B8eMVSJDWaXW8o+X5dzCXkdA@mail.gmail.com> 1 sibling, 1 reply; 36+ messages in thread From: Peter Todd @ 2013-11-19 17:06 UTC (permalink / raw) To: Wladimir; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 867 bytes --] On Tue, Nov 19, 2013 at 05:32:55PM +0100, Wladimir wrote: > On Mon, Oct 21, 2013 at 4:30 PM, Jeff Garzik <jgarzik@bitpay•com> wrote: > > > BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and > > are not automatically assigned a BIPS number. > > > > Are we going to move ahead with this? > > If so, I'm volunteering to create the repository and import the current > BIPs from the wiki there (and convert from wiki markup to markdown where > necessary). I already did that: https://github.com/petertodd/bips GitHub can render MediaWiki just fine, so I think leaving the BIPs as MediaWiki is the way to go. New BIPs may want to use either markdown or MediaWiki - the latter has advantages in terms of formatting capabilities over the former, particularly when math needs to be displayed. -- 'peter'[:-1]@petertodd.org [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 490 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
[parent not found: <CA+s+GJA=p+yvoJqUAMQQRcfYK1B8eMVSJDWaXW8o+X5dzCXkdA@mail.gmail.com>]
* [Bitcoin-development] Fwd: Revisiting the BIPS process, a proposal [not found] ` <CA+s+GJA=p+yvoJqUAMQQRcfYK1B8eMVSJDWaXW8o+X5dzCXkdA@mail.gmail.com> @ 2013-11-19 17:21 ` Wladimir 0 siblings, 0 replies; 36+ messages in thread From: Wladimir @ 2013-11-19 17:21 UTC (permalink / raw) To: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 1043 bytes --] On Tue, Nov 19, 2013 at 6:06 PM, Peter Todd <pete@petertodd•org> wrote: > On Tue, Nov 19, 2013 at 05:32:55PM +0100, Wladimir wrote: > > On Mon, Oct 21, 2013 at 4:30 PM, Jeff Garzik <jgarzik@bitpay•com> wrote: > > > > > BIP drafts are stored in git://github.com/bitcoin/bips.git/drafts/ and > > > are not automatically assigned a BIPS number. > > > > > > > Are we going to move ahead with this? > > > > If so, I'm volunteering to create the repository and import the current > > BIPs from the wiki there (and convert from wiki markup to markdown where > > necessary). > > I already did that: > > https://github.com/petertodd/bips > Ok cool, I forked it into https://github.com/bitcoin/bips > GitHub can render MediaWiki just fine, so I think leaving the BIPs as > MediaWiki is the way to go. New BIPs may want to use either markdown or > MediaWiki - the latter has advantages in terms of formatting > capabilities over the former, particularly when math needs to be > displayed. > Agreed, I had no idea github could do that too. Wladimir [-- Attachment #2: Type: text/html, Size: 2128 bytes --] ^ permalink raw reply [flat|nested] 36+ messages in thread
end of thread, other threads:[~2013-11-19 17:54 UTC | newest]
Thread overview: 36+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-10-21 14:30 [Bitcoin-development] Revisiting the BIPS process, a proposal Jeff Garzik
2013-10-21 14:34 ` Jeff Garzik
2013-10-21 15:46 ` Andreas Schildbach
2013-10-21 16:14 ` Jeff Garzik
2013-10-21 17:17 ` Jeff Garzik
2013-10-21 19:38 ` Jean-Paul Kogelman
2013-10-21 19:47 ` Luke-Jr
2013-10-21 20:57 ` Benjamin Cordes
2013-10-21 20:59 ` Benjamin Cordes
2013-10-22 6:39 ` Martin Sustrik
2013-10-22 6:59 ` Jean-Paul Kogelman
2013-10-22 7:03 ` Gregory Maxwell
2013-10-22 7:34 ` Martin Sustrik
2013-10-22 7:49 ` Peter Todd
2013-10-22 7:56 ` Gregory Maxwell
2013-10-22 8:20 ` Martin Sustrik
2013-10-22 14:08 ` Jeff Garzik
2013-10-23 7:38 ` Martin Sustrik
2013-10-23 19:40 ` Peter Todd
2013-10-23 20:05 ` Martin Sustrik
2013-10-23 20:27 ` Peter Todd
2013-10-23 21:07 ` Pieter Wuille
2013-10-23 21:42 ` Allen Piscitello
2013-10-23 21:49 ` Luke-Jr
2013-10-24 7:03 ` Martin Sustrik
2013-10-24 10:39 ` Jeff Garzik
2013-10-24 11:11 ` Christian Decker
2013-10-24 19:43 ` Jeremy Spilman
2013-11-19 16:32 ` Wladimir
2013-11-19 16:53 ` Drak
2013-11-19 17:01 ` Gregory Maxwell
2013-11-19 17:07 ` Drak
2013-11-19 17:45 ` Wladimir
2013-11-19 17:54 ` Gregory Maxwell
2013-11-19 17:06 ` Peter Todd
[not found] ` <CA+s+GJA=p+yvoJqUAMQQRcfYK1B8eMVSJDWaXW8o+X5dzCXkdA@mail.gmail.com>
2013-11-19 17:21 ` [Bitcoin-development] Fwd: " Wladimir
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox