* [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
@ 2014-11-06 21:32 Peter Todd
2014-11-06 21:58 ` Tamas Blummer
0 siblings, 1 reply; 18+ messages in thread
From: Peter Todd @ 2014-11-06 21:32 UTC (permalink / raw)
To: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 7791 bytes --]
Recently wrote the following for a friend and thought others might learn
from it.
> Nope, never heard that term. By "bug-for-bug" compatibility, do you mean
> that, for each version which has a bug, each bug must behave in the *same*
> buggy way?
Exactly. tl;dr: if you accept a block as valid due to a bug that others reject,
you're forked and the world ends.
Long answer... well you reminded me I've never actually written up a good
example for others, and a few people have asked me for one. A great example of
this is the SIGHASH_SINGLE bug in the SignatureHash() function:
uint256 SignatureHash(CScript scriptCode, const CTransaction& txTo, unsigned int nIn, int nHashType)
{
<snip>
else if ((nHashType & 0x1f) == SIGHASH_SINGLE)
{
// Only lock-in the txout payee at same index as txin
unsigned int nOut = nIn;
if (nOut >= txTmp.vout.size())
{
printf("ERROR: SignatureHash() : nOut=%d out of range\n", nOut);
return 1;
}
<snip>
}
<snip>
// Serialize and hash
CHashWriter ss(SER_GETHASH, 0);
ss << txTmp << nHashType;
return ss.GetHash();
}
So that error condition results in SignatureHash() returning 1 rather than the
actual hash. But the consensus-critical code that implements the CHECKSIG
operators doesn't check for that condition! Thus as long as you use the
SIGHASH_SINGLE hashtype and the txin index is >= the number of txouts any valid
signature for the hash of the number 1 is considered valid!
When I found this bug¹ I used it to fork bitcoin-ruby, among others.
(I'm not the first; I found it independently after Matt Corallo) Those
alt-implementations handled this edge-case as an exception, which in
turn caused the script to fail. Thus they'd reject blocks containing
transactions using such scripts, and be forked off the network.
You can also use this bug for something even more subtle. So the
CHECKSIG* opcode evaluation does this:
// Drop the signature, since there's no way for a signature to sign itself
scriptCode.FindAndDelete(CScript(vchSig));
and CHECKMULTISIG* opcode:
// Drop the signatures, since there's no way for a signature to sign itself
for (int k = 0; k < nSigsCount; k++)
{
valtype& vchSig = stacktop(-isig-k);
scriptCode.FindAndDelete(CScript(vchSig));
}
We used to think that code could never be triggered by a scriptPubKey or
redeemScript, basically because there was no way to get a signature into a
transaction in the right place without the signature depending on the txid of
the transaction it was to be included in. (long story) But SIGHASH_SINGLE makes
that a non-issue, as you can now calculate the signature that signs '1' ahead
of time! In a CHECKMULTISIG that signature is valid, so is included in the list
of signatures being dropped, and thus the other signatures must take that
removal into account or they're invalid. Again, you've got a fork.
However this isn't the end of it! So the way FindAndDelete() works is as
follows:
int CScript::FindAndDelete(const CScript& b)
{
int nFound = 0;
if (b.empty())
return nFound;
iterator pc = begin();
opcodetype opcode;
do
{
while (end() - pc >= (long)b.size() && memcmp(&pc[0], &b[0], b.size()) == 0)
{
pc = erase(pc, pc + b.size());
++nFound;
}
}
while (GetOp(pc, opcode));
return nFound;
}
So that's pretty ugly, but basically what's happening is the loop iterates
though all the opcodes in the script. Every opcode is compared at the *byte*
level to the bytes in the argument. If they match those bytes are removed from
the script and iteration continues. The resulting script, with chunks sliced
out of it at the byte level, is what gets hashed as part of the signature
checking algorithm.
As FindAndDelete() is always called with CScript(vchSig) the signature
being found and deleted is reserialized. Serialization of bytes isn't
unique; there are multiple valid encodings for PUSHDATA operations. The
way CScript() is called the most compact encoding is used, however this
means that if the script being hashed used a different encoding those
bytes are *not* removed and thus the signature is different.
Again, if you don't get every last one of those details exactly right, you'll
get forked.
...and I'm still not done! So when you call CScript(vchSig) the relevant code
is the following:
class CScript : public std::vector<unsigned char>
{
<snip>
explicit CScript(const CScriptNum& b) { operator<<(b); }
<snip>
CScript& operator<<(const std::vector<unsigned char>& b)
{
if (b.size() < OP_PUSHDATA1)
{
insert(end(), (unsigned char)b.size());
}
else if (b.size() <= 0xff)
{
insert(end(), OP_PUSHDATA1);
insert(end(), (unsigned char)b.size());
}
else if (b.size() <= 0xffff)
{
insert(end(), OP_PUSHDATA2);
unsigned short nSize = b.size();
insert(end(), (unsigned char*)&nSize, (unsigned char*)&nSize + sizeof(nSize));
}
else
{
insert(end(), OP_PUSHDATA4);
unsigned int nSize = b.size();
insert(end(), (unsigned char*)&nSize, (unsigned char*)&nSize + sizeof(nSize));
}
insert(end(), b.begin(), b.end());
return *this;
}
<snip, rest of class definition>
}
Recently as part of BIP62 we added the concept of a 'minimal' PUSHDATA
operation. Using the minimum-sized PUSHDATA opcode is obvious; not so obvious
is that there are few "push number to stack" opcodes that push the numbers 0
through 16 and -1 to the stack, bignum encoded. If you are pushing data that
happens to match the latter, you're supposed to use those OP_1...OP_16 and
OP_1NEGATE opcodes rather than a PUSHDATA.
This means that calling CScript(b'\x81') will result in a non-standard
script. I know an unmerged pull-req² related to sipa's BIP62 work has
code in the CScript() class to automatically do that conversion; had
that code shipped we'd have a potential forking bug between new and old
versions of Bitcoin as the exact encoding of CScript() is consensus
critical by virtue of being called by the FindAndDelete() code!
Even had we made that mistake, I'm not sure how to actually exploit it...
FindAndDelete() is only ever called in a consensus-critical way with valid
signatures; the byte arrays 01, 02, ..., 81 are all totally invalid signatures.
The best I could think of would be to exploit the script verification
flag SCRIPT_VERIFY_STRICTENC by using the little-known hybrid-pubkey
encoding³, which I spent the past two hours looking at. However it isn't
even soft-fork safe in the current implementation! All I could find was
a new DoS attack⁴, and it's not exploitable in an actual release due to
the pre-v0.10 IsStandard() rules. :(
[¹]: https://bitcointalk.org/index.php?topic=260595.0
[²]: https://github.com/bitcoin/bitcoin/pull/5091
[³]: https://github.com/bitcoin/bitcoin/blob/cd9114e5136ecc1f60baa43fffeeb632782f2353/src/test/data/script_valid.json#L739
[⁴]: http://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg06458.html
--
'peter'[:-1]@petertodd.org
000000000000000019121d8632bcba14de98125e8a9affc7d07c760706ba3879
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 21:32 [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug Peter Todd
@ 2014-11-06 21:58 ` Tamas Blummer
2014-11-06 22:05 ` Matt Corallo
0 siblings, 1 reply; 18+ messages in thread
From: Tamas Blummer @ 2014-11-06 21:58 UTC (permalink / raw)
To: Peter Todd; +Cc: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 483 bytes --]
Thanks Peter,
Having tried to write a bug-for-bug compatible code with Satoshi, I can only second that it is rather close to impossible.
The aim of BIP62 is noble, still it does not feel right for me to increase the complexity of the code with e.g. soft-fork-ready versioning.
Freezing the consensus code, studying its bugs appears more appropriate to me. What we learn could define a hard fork or a better
chain we migrate to as discussed by blockstream.
Tamas Blummer
[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 21:58 ` Tamas Blummer
@ 2014-11-06 22:05 ` Matt Corallo
2014-11-06 22:11 ` Jeff Garzik
2014-11-06 23:12 ` Peter Todd
0 siblings, 2 replies; 18+ messages in thread
From: Matt Corallo @ 2014-11-06 22:05 UTC (permalink / raw)
To: bitcoin-development
Depends, without BIP62 a /lot/ of the even basic contracts that people
want to use today (or wanted to use 18 months ago) are unusable, in
fact, without BIP62, the atomic swaps suggested as important for
sidechains are not secure. While redoing Bitcoin in a hardfork is nice,
its a very long-term thing, so I'm not sure about making people wait for
a large hardfork just to use payment channels.
Also, I echo the difficulty of writing consensus-compatible code and
highly suggest anyone with money behind an implementation that is doing
script verification in code that isnt Bitcoin Core rethink that decision.
Matt
On 11/06/14 21:58, Tamas Blummer wrote:
> Thanks Peter,
>
> Having tried to write a bug-for-bug compatible code with Satoshi, I can only second that it is rather close to impossible.
>
> The aim of BIP62 is noble, still it does not feel right for me to increase the complexity of the code with e.g. soft-fork-ready versioning.
> Freezing the consensus code, studying its bugs appears more appropriate to me. What we learn could define a hard fork or a better
> chain we migrate to as discussed by blockstream.
>
> Tamas Blummer
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 22:05 ` Matt Corallo
@ 2014-11-06 22:11 ` Jeff Garzik
2014-11-06 22:48 ` Justus Ranvier
2014-11-06 23:19 ` Peter Todd
2014-11-06 23:12 ` Peter Todd
1 sibling, 2 replies; 18+ messages in thread
From: Jeff Garzik @ 2014-11-06 22:11 UTC (permalink / raw)
To: Matt Corallo; +Cc: Bitcoin Dev
IMO, CHECKLOCKTIMEVERIFY should be included in that list, too.
RE soft fork vs. hard fork: It's about this time at Mike Hearn will
chime in, on the side of hard forks. Hard forks are in a sense much
cleaner, and permit solving problems not otherwise solvable with a
hard fork. However, hard forks clearly have risks, notably the Big
Risk akin to a US Constitutional Convention: once you open the door,
anything can happen, any rule no matter how "sacred" can be changed.
Soft forks are not without their own risks, e.g. reducing some things
to SPV levels of security.
Leaning towards soft fork, but it is a good discussion to have. A
poorly implemented soft fork may potentially require a hard fork to
fix rollout bugs.
On Thu, Nov 6, 2014 at 11:05 PM, Matt Corallo <bitcoin-list@bluematt•me> wrote:
> Depends, without BIP62 a /lot/ of the even basic contracts that people
> want to use today (or wanted to use 18 months ago) are unusable, in
> fact, without BIP62, the atomic swaps suggested as important for
> sidechains are not secure. While redoing Bitcoin in a hardfork is nice,
> its a very long-term thing, so I'm not sure about making people wait for
> a large hardfork just to use payment channels.
>
> Also, I echo the difficulty of writing consensus-compatible code and
> highly suggest anyone with money behind an implementation that is doing
> script verification in code that isnt Bitcoin Core rethink that decision.
>
> Matt
>
> On 11/06/14 21:58, Tamas Blummer wrote:
>> Thanks Peter,
>>
>> Having tried to write a bug-for-bug compatible code with Satoshi, I can only second that it is rather close to impossible.
>>
>> The aim of BIP62 is noble, still it does not feel right for me to increase the complexity of the code with e.g. soft-fork-ready versioning.
>> Freezing the consensus code, studying its bugs appears more appropriate to me. What we learn could define a hard fork or a better
>> chain we migrate to as discussed by blockstream.
>>
>> Tamas Blummer
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
--
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc. https://bitpay.com/
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 22:11 ` Jeff Garzik
@ 2014-11-06 22:48 ` Justus Ranvier
2014-11-06 23:26 ` Peter Todd
2014-11-06 23:19 ` Peter Todd
1 sibling, 1 reply; 18+ messages in thread
From: Justus Ranvier @ 2014-11-06 22:48 UTC (permalink / raw)
To: bitcoin-development
[-- Attachment #1.1: Type: text/plain, Size: 1678 bytes --]
On 11/06/2014 04:11 PM, Jeff Garzik wrote:
> RE soft fork vs. hard fork: It's about this time at Mike Hearn will
> chime in, on the side of hard forks. Hard forks are in a sense much
> cleaner, and permit solving problems not otherwise solvable with a
> hard fork. However, hard forks clearly have risks, notably the Big
> Risk akin to a US Constitutional Convention: once you open the door,
> anything can happen, any rule no matter how "sacred" can be changed.
Yes, there are risks, but those risks could be managed with appropriate
effort. Major players could publicly commit to a set of ground rules vis
a vis what categories of changes are and are not acceptable.
Maybe at some point there could even be something that resembles project
management for the Bitcoin protocol.
Why not schedule protocol upgrades every two years for the foreseeable
future?
Spend one year achieving broad consensus regarding what changes to make
in the next upgrade, then spend one year in feature freeze (all future
proposals postponed for the next cycle) then execute the upgrade.
The top priority should be fixing bugs that make specifying and
re-implementing the protocol nearly impossible. Those kinds of changes
should have little difficulty achieving near-unanimous consensus.
There shouldn't be any problems separating obviously-needed changes from
the ones that let third parties blacklist coins, or a majority of miners
vote to confiscate block rewards from minority, tamper with the issuance
schedule, etc.
--
Support online privacy by using email encryption whenever possible.
Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k
[-- Attachment #1.2: 0x38450DB5.asc --]
[-- Type: application/pgp-keys, Size: 14265 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 22:05 ` Matt Corallo
2014-11-06 22:11 ` Jeff Garzik
@ 2014-11-06 23:12 ` Peter Todd
2014-11-07 2:04 ` Gregory Maxwell
1 sibling, 1 reply; 18+ messages in thread
From: Peter Todd @ 2014-11-06 23:12 UTC (permalink / raw)
To: Matt Corallo; +Cc: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 2724 bytes --]
On Thu, Nov 06, 2014 at 10:05:54PM +0000, Matt Corallo wrote:
> Depends, without BIP62 a /lot/ of the even basic contracts that people
> want to use today (or wanted to use 18 months ago) are unusable, in
> fact, without BIP62, the atomic swaps suggested as important for
> sidechains are not secure. While redoing Bitcoin in a hardfork is nice,
> its a very long-term thing, so I'm not sure about making people wait for
> a large hardfork just to use payment channels.
BIP62 is a less-than-ideal way of making contracts secure against
malleability as it relies on a "whack-a-mole" approach to security that
is insecure if any flaw is missed. If you only wanted to make contracts
secure, you'd either implement a new SignatureHash() that could leave
out the prevout field in favor of hashing the previous input's CTxOut()
structure, and/or implement the significantly more limited
CHECKLOCKTIMEVERIFY.
Equally BIP62 fails at making more complex types of contracts secure.
For instance suppose I had a multi-step protocol that required more than
two transactions:
tx1: Alice -> (Alice, Bob)
tx1_refund: (Alice, Bob) -> Alice
tx2: (Alice, Bob) -> Charlie
tx2_refund: (Alice, Bob) -> Bob
tx1 can only be modified by Alice, so tx1_refund is secure. However the
second stage, where the output of tx1 is spent by tx2, with a refund
transaction giving the funds back to Bob, can't be made secure as BIP62
can't prevent Alice from changing her signature, getting tx2' mined
instead, and making tx2_refund invalid.
OTOH a new form of signature hash that was a signature on tx2.vout
structure rather than it's txid would be secure, as tx2_refund would be
valid regardless of tx2's actual txid.
Obviously there are good reasons to not use such signature hashes in the
general case, as they imply you can't reuse scriptPubKeys securely, but
that's a minor problem for purpose-built contract protocols. It's
certainly a much more minor problem then the huge number of holes
possible with BIP62.
BIP62 does make life easier for wallet authors as they don't have to
deal with malleability - maybe! - but for contracts it's a bad design.
> Also, I echo the difficulty of writing consensus-compatible code and
> highly suggest anyone with money behind an implementation that is doing
> script verification in code that isnt Bitcoin Core rethink that decision.
FWIW I've done due-dilligence reviews for investors on projects and
companies that have re-implemented Bitcoin Core consensus-critical code,
and every time my review lists doing so as a major red flag.
--
'peter'[:-1]@petertodd.org
0000000000000000166801ed3959dde6b7d979735c290e7c4271ae3cf75ced63
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 22:11 ` Jeff Garzik
2014-11-06 22:48 ` Justus Ranvier
@ 2014-11-06 23:19 ` Peter Todd
1 sibling, 0 replies; 18+ messages in thread
From: Peter Todd @ 2014-11-06 23:19 UTC (permalink / raw)
To: Jeff Garzik; +Cc: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 1820 bytes --]
On Thu, Nov 06, 2014 at 11:11:52PM +0100, Jeff Garzik wrote:
> IMO, CHECKLOCKTIMEVERIFY should be included in that list, too.
>
> RE soft fork vs. hard fork: It's about this time at Mike Hearn will
> chime in, on the side of hard forks. Hard forks are in a sense much
> cleaner, and permit solving problems not otherwise solvable with a
> hard fork. However, hard forks clearly have risks, notably the Big
> Risk akin to a US Constitutional Convention: once you open the door,
> anything can happen, any rule no matter how "sacred" can be changed.
I think people in this community often miss the serious political and
legal ramifications of hard-forks. Being in the social position of being
able to succesfully pull off hard-forks, particularly for new features,
is clear evidence that you have de-facto control over the system.
Regulators around the world appear to be going in directions that would
make that control subject to regulation and licensing, e.g. the European
Banking Association proposals, and initial Bitlicense proposals.
Equally, look how hard-forks - known as flag days elsewhere - are
generally considered to be dangerous and worth avoiding in other
contexts due to simple engineering reasons. It's just easier to upgrade
systems in backward compatible ways, especially when they incorporate
features specifically to make that possible. (as does bitcoin!)
> Soft forks are not without their own risks, e.g. reducing some things
> to SPV levels of security.
This is a misconception; you can't prevent soft-forks from happening, so
you always have an SPV level of security by that standard.
People put *way* too much trust in small numbers of confirmations...
--
'peter'[:-1]@petertodd.org
00000000000000000094d543907eaf0f94f4ff5f4c760b3552d84ff811cd9053
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 22:48 ` Justus Ranvier
@ 2014-11-06 23:26 ` Peter Todd
2014-11-06 23:36 ` Justus Ranvier
0 siblings, 1 reply; 18+ messages in thread
From: Peter Todd @ 2014-11-06 23:26 UTC (permalink / raw)
To: Justus Ranvier; +Cc: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 497 bytes --]
On Thu, Nov 06, 2014 at 04:48:54PM -0600, Justus Ranvier wrote:
> Why not schedule protocol upgrades every two years for the foreseeable
> future?
For the same reason we don't do hard-forking upgrades of basically every
protocol on the planet on a regular basis, even when we don't have
consensus problems to worry about.
Flag days are really rare in engineering, and for good reason.
--
'peter'[:-1]@petertodd.org
000000000000000008f2290924a6882928d4566f487f33cc57203a6535795201
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 23:26 ` Peter Todd
@ 2014-11-06 23:36 ` Justus Ranvier
2014-11-07 0:03 ` Peter Todd
2014-11-15 4:43 ` Jean-Pierre Rupp
0 siblings, 2 replies; 18+ messages in thread
From: Justus Ranvier @ 2014-11-06 23:36 UTC (permalink / raw)
To: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 1654 bytes --]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 11/06/2014 05:26 PM, Peter Todd wrote:> For the same reason we
don't do hard-forking upgrades of basically every
> protocol on the planet on a regular basis, even when we don't have
> consensus problems to worry about.
>
> Flag days are really rare in engineering, and for good reason.
This explanation is completely incoherent.
Because Bitcoin has a extra consensus requirements, requirements which
are really rare in engineering, the necessity of fixing bugs is even
greater.
There are two general ways to fix bugs: either as part of a
controlled, planned, and managed process, or as a response to an
immediate disaster.
The alternative to scheduling and planning the upgrades which are
necessary to fix the bugs in the protocol, where such fixes can be
written, tested, and documented at leisure, is to wait for some crisis
and slap on another bandaid when the network breaks again (like it did
March of last year).
Who benefits from not fixing bugs in Bitcoin?
- --
Support online privacy by using email encryption whenever possible.
Learn how here: http://www.youtube.com/watch?v=bakOKJFtB-k
-----BEGIN PGP SIGNATURE-----
iQEcBAEBAgAGBQJUXAYVAAoJEMP3uyY4RQ21YxMH/3O+vFK2jDXV5V8IIsJnU/u1
D4gYyVm89E0zmGTyLAUYCJGj0eg5tMyUUzu62hOECOeQuKdVi+mbkLi4TiF0sHIb
8k25MgqJgzH/021eoI2g2w1FrDlZut02LNX/V09+owd1yhp+SEXZ3/+HlqsZXhsv
/u9u5OayzhGlzS6apQtrosl5P+KIquHqIbtwBtPOb2rvlL0miJ6sRcAH2JCXCBDT
HMcswMtIGZbgqL/K7e/6vH7dUWdp0866RZXvt7aWGNUgvxHbGMs+zsnxp4nNslxM
wqL71gTmtMnLcw0GtqmXPjcjo+adrPnqp45xc9lSt23PGjxxfR0FKYIPb2uZdq8=
=9GOY
-----END PGP SIGNATURE-----
[-- Attachment #2: 0x38450DB5.asc --]
[-- Type: application/pgp-keys, Size: 14265 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 23:36 ` Justus Ranvier
@ 2014-11-07 0:03 ` Peter Todd
2014-11-07 8:07 ` Tamas Blummer
2014-11-07 16:52 ` Mike Hearn
2014-11-15 4:43 ` Jean-Pierre Rupp
1 sibling, 2 replies; 18+ messages in thread
From: Peter Todd @ 2014-11-07 0:03 UTC (permalink / raw)
To: Justus Ranvier; +Cc: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 3759 bytes --]
On Thu, Nov 06, 2014 at 05:36:55PM -0600, Justus Ranvier wrote:
> This explanation is completely incoherent.
>
> Because Bitcoin has a extra consensus requirements, requirements which
> are really rare in engineering, the necessity of fixing bugs is even
> greater.
>
> There are two general ways to fix bugs: either as part of a
> controlled, planned, and managed process, or as a response to an
> immediate disaster.
>
> The alternative to scheduling and planning the upgrades which are
> necessary to fix the bugs in the protocol, where such fixes can be
> written, tested, and documented at leisure, is to wait for some crisis
> and slap on another bandaid when the network breaks again (like it did
> March of last year).
The protocol is what the protocol is; the bugs are when you don't match
the protocol.
> Who benefits from not fixing bugs in Bitcoin?
We can bring up politics if you want.
In the current model, the specification *is* the protocol, and the
Bitcoin Core team is scared to death of changing anything; they've got
very little real power. Soft-forks are the minimum-viable way of making
changes to the protocol, and it's very clear how they get adopted:
minerr consensus. They're also a fundemental way of changing the
protocol that is impossible to prevent, so you might as well use it.
Hard-forks require political consensus to achieve, and the way you
create that political consensus is by creating committes, groups,
associations... Foundations. Every last one of those things requires
centralization and political power.
You know, the smartest thing the Bitcoin Foundation could do if they
wanted to cement their place in the Bitcoin ecosystem as a power broker
would be to setup a program of periodic hard-forks, say every year or
two, and then manage the committees that decide what goes into those
hard-forks. That they haven't suggested that yet is a sign that they're
either not evil, or they don't understand Bitcoin very well.
I think programmers find this reality hard to accept, because they're
mostly interested in writing code that'll get widely used. To them it's
hard to accept that the Bitcoin protocol *is* a few thousand lines of
C++ code, and they're not good enough to write their own implementation
and make it match; if we replaced programmers with writers we might get
the equally bizzare and pointless situation of people taking perfectly
good RFCs and rewriting them in their own words.
If you do care about keeping the politics of Bitcoin development free
from centralized control you should do what I advised the Dark Wallet
team to do a year ago: fork Bitcoin Core and change the
non-consensus-critical code that implements policy. I've done this
myself in a minor way with my replace-by-fee(1) version. Luke-Jr has
also done this with his Eligius branch, a fork that something like 30%
of the Bitcoin hashing power appear to run. (Discus Fish has been mining
non-standard transactions(2) lately)
Multiple *forks* of the Bitcoin Core reference client that are actually
getting used by miners and other users ensures that no one group
maintaining such a fork has the ability to change anything without
strong consensus. Forking the codebase, rather than rewriting it, best
ensures that your code actually implements the protocol properly, is
safe to use for mining, and actually gets used.
Rewriting Bitcoin Core is a fun project, but it's terrible politics.
1) https://github.com/petertodd/bitcoin/tree/replace-by-fee-v0.9.3
2) https://blockchain.info/tx/e24a4085c54a6362e615f8eab758c12d80e488b73757e6d2b8ab6bfc8be7007e
--
'peter'[:-1]@petertodd.org
000000000000000008f2290924a6882928d4566f487f33cc57203a6535795201
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 23:12 ` Peter Todd
@ 2014-11-07 2:04 ` Gregory Maxwell
0 siblings, 0 replies; 18+ messages in thread
From: Gregory Maxwell @ 2014-11-07 2:04 UTC (permalink / raw)
To: Peter Todd; +Cc: Bitcoin Development
On Thu, Nov 6, 2014 at 11:12 PM, Peter Todd <pete@petertodd•org> wrote:
> BIP62 does make life easier for wallet authors as they don't have to
> deal with malleability - maybe! -
Yes, I agree for most contract purposes CTLV is what you want to be
using, instead of refund transactions beyond being more clearly
correct, it shrinks the protocol state machine by one step.
Though BIP62 also achieves the secondary goal of making required
implementation behaviour more explicit (e.g. the parts enforced in all
transactions), and that shouldn't be discounted.
They're somewhat orthogonal, somwhat complementary things.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-07 0:03 ` Peter Todd
@ 2014-11-07 8:07 ` Tamas Blummer
2014-11-07 8:48 ` Peter Todd
2014-11-07 16:52 ` Mike Hearn
1 sibling, 1 reply; 18+ messages in thread
From: Tamas Blummer @ 2014-11-07 8:07 UTC (permalink / raw)
To: Peter Todd; +Cc: bitcoin-development, Justus Ranvier
[-- Attachment #1: Type: text/plain, Size: 450 bytes --]
Peter,
forking would work best with a freeze of the consensus code. Do you see any chance for that?
Tamas Blummer
On Nov 7, 2014, at 1:03 AM, Peter Todd <pete@petertodd•org> wrote:
> Forking the codebase, rather than rewriting it, best
> ensures that your code actually implements the protocol properly, is
> safe to use for mining, and actually gets used.
>
> Rewriting Bitcoin Core is a fun project, but it's terrible politics.
[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-07 8:07 ` Tamas Blummer
@ 2014-11-07 8:48 ` Peter Todd
2014-11-07 11:30 ` Clément Elbaz
0 siblings, 1 reply; 18+ messages in thread
From: Peter Todd @ 2014-11-07 8:48 UTC (permalink / raw)
To: Tamas Blummer; +Cc: bitcoin-development, Justus Ranvier
[-- Attachment #1: Type: text/plain, Size: 851 bytes --]
On Fri, Nov 07, 2014 at 09:07:47AM +0100, Tamas Blummer wrote:
> Peter,
>
> forking would work best with a freeze of the consensus code. Do you see any chance for that?
To a first approximation the consensus code *is* frozen; if we introduce
any consensus changes into it at this point it's due to a mistake, not
intentionally.
Of course, that's not including the two serious soft-fork proposals in
the air right now, Pieter Wuille's BIP62 and my CHECKLOCKTIMEVERIFY.
However dealing with proposed changes like those in an environment where
the competing implementations all use essentially the same
consensus-critical code is much easier than in an environment where they
don't; I say this on both a technical and political level.
--
'peter'[:-1]@petertodd.org
00000000000000000c901eb1b6b765519b99c3afd7a9062ff4cfa29666ce140d
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-07 8:48 ` Peter Todd
@ 2014-11-07 11:30 ` Clément Elbaz
2014-11-07 11:47 ` Peter Todd
2014-11-07 12:01 ` Wladimir
0 siblings, 2 replies; 18+ messages in thread
From: Clément Elbaz @ 2014-11-07 11:30 UTC (permalink / raw)
To: Peter Todd, Tamas Blummer; +Cc: bitcoin-development, Justus Ranvier
[-- Attachment #1: Type: text/plain, Size: 1950 bytes --]
Thinking out loud here : would it make sense to separate the consensus code
into some kind of "Bitcoin Kernel" (similar to the Linux Kernel) project
that could be used by anyone ?
Bitcoin Core (and any other application wishing to do so) could be based on
it.
The kernel would just contain the absolute minimum code for reaching
consensus, leaving every other aspects of the implementation to the
applications built with it.
It would be stateless : it would provide an interface to submit a
block/transaction to be validated, including the context needed to validate
it (the previously validated blocks referenced by this block/transaction).
What do you think ?
Clément
Le Fri Nov 07 2014 at 9:49:05 AM, Peter Todd <pete@petertodd•org> a écrit :
On Fri, Nov 07, 2014 at 09:07:47AM +0100, Tamas Blummer wrote:
> > Peter,
> >
> > forking would work best with a freeze of the consensus code. Do you see
> any chance for that?
>
> To a first approximation the consensus code *is* frozen; if we introduce
> any consensus changes into it at this point it's due to a mistake, not
> intentionally.
>
> Of course, that's not including the two serious soft-fork proposals in
> the air right now, Pieter Wuille's BIP62 and my CHECKLOCKTIMEVERIFY.
> However dealing with proposed changes like those in an environment where
> the competing implementations all use essentially the same
> consensus-critical code is much easier than in an environment where they
> don't; I say this on both a technical and political level.
>
> --
> 'peter'[:-1]@petertodd.org
> 00000000000000000c901eb1b6b765519b99c3afd7a9062ff4cfa29666ce140d
> ------------------------------------------------------------
> ------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
[-- Attachment #2: Type: text/html, Size: 2732 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-07 11:30 ` Clément Elbaz
@ 2014-11-07 11:47 ` Peter Todd
2014-11-07 12:01 ` Wladimir
1 sibling, 0 replies; 18+ messages in thread
From: Peter Todd @ 2014-11-07 11:47 UTC (permalink / raw)
To: Clément Elbaz; +Cc: bitcoin-development, Justus Ranvier
[-- Attachment #1: Type: text/plain, Size: 2252 bytes --]
On Fri, Nov 07, 2014 at 11:30:22AM +0000, Clément Elbaz wrote:
> Thinking out loud here : would it make sense to separate the consensus code
> into some kind of "Bitcoin Kernel" (similar to the Linux Kernel) project
> that could be used by anyone ?
That's a pretty old idea, and we're working on it. First step is a
stand-alone script verification library:
https://github.com/theuni/bitcoin/blob/da18a0266c4de76c2a461cc2984aa2fa066c42f5/src/script/bitcoinconsensus.h
#ifndef H_BITCOIN_BITCOINCONSENSUS
#define H_BITCOIN_BITCOINCONSENSUS
#if defined(BUILD_BITCOIN_INTERNAL) && defined(HAVE_CONFIG_H)
#include "config/bitcoin-config.h"
#if defined(_WIN32)
#if defined(DLL_EXPORT)
#if defined(HAVE_FUNC_ATTRIBUTE_DLLEXPORT)
#define EXPORT_SYMBOL __declspec(dllexport)
#else
#define EXPORT_SYMBOL
#endif
#endif
#elif defined(HAVE_FUNC_ATTRIBUTE_VISIBILITY)
#define EXPORT_SYMBOL __attribute__ ((visibility ("default")))
#endif
#elif defined(MSC_VER) && !defined(STATIC_LIBBITCOINCONSENSUS)
#define EXPORT_SYMBOL __declspec(dllimport)
#endif
#ifndef EXPORT_SYMBOL
#define EXPORT_SYMBOL
#endif
#ifdef __cplusplus
extern "C" {
#else
#include <stdbool.h>
#endif
/** Script verification flags */
enum
{
bitcoinconsensus_SCRIPT_FLAGS_VERIFY_NONE = 0,
bitcoinconsensus_SCRIPT_FLAGS_VERIFY_P2SH = (1U << 0), // evaluate P2SH (BIP16) subscripts
};
/// Returns true if the input nIn of the serialized transaction pointed to by
/// txTo correctly spends the scriptPubKey pointed to by scriptPubKey under
/// the additional constraints specified by flags.
EXPORT_SYMBOL bool bitcoinconsensus_verify_script(const unsigned char *scriptPubKey, const unsigned int scriptPubKeyLen,
const unsigned char *txTo , const unsigned int txToLen,
const unsigned int nIn, const unsigned int flags);
EXPORT_SYMBOL unsigned int bitcoinconsensus_version();
#ifdef __cplusplus
} // extern "C"
#endif
#undef EXPORT_SYMBOL
#endif // H_BITCOIN_BITCOINCONSENSUS
--
'peter'[:-1]@petertodd.org
000000000000000001208038fd7130083ff118147890dbb37913ffa83c1f48cd
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-07 11:30 ` Clément Elbaz
2014-11-07 11:47 ` Peter Todd
@ 2014-11-07 12:01 ` Wladimir
1 sibling, 0 replies; 18+ messages in thread
From: Wladimir @ 2014-11-07 12:01 UTC (permalink / raw)
To: Clément Elbaz; +Cc: Bitcoin Dev, Justus Ranvier
On Fri, Nov 7, 2014 at 12:30 PM, Clément Elbaz <clem.ds@gmail•com> wrote:
> Thinking out loud here : would it make sense to separate the consensus code
> into some kind of "Bitcoin Kernel" (similar to the Linux Kernel) project
> that could be used by anyone ?
Yes, we're moving in that direction. First with a script verification
library in 0.10, which will be extended to other parts of the
consensus by 0.11 and after that.
Wladimir
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-07 0:03 ` Peter Todd
2014-11-07 8:07 ` Tamas Blummer
@ 2014-11-07 16:52 ` Mike Hearn
1 sibling, 0 replies; 18+ messages in thread
From: Mike Hearn @ 2014-11-07 16:52 UTC (permalink / raw)
To: Peter Todd; +Cc: Bitcoin Dev, Justus Ranvier
[-- Attachment #1: Type: text/plain, Size: 897 bytes --]
>
> > Who benefits from not fixing bugs in Bitcoin?
>
> We can bring up politics if you want.
No, please don't. That question was rhetorical, not an invitation for you
to try and convince bystanders that anyone who disagrees with you is a
shadowy Agent Of Centralisation or an idiot. You use that tactic way too
much: it's obnoxious and you need to stop it.
Hard forks vs soft forks are *purely* about whether you drag along old
nodes in a quasi-broken state. They do not reduce total work needed by the
community one iota. Non-miners who wish to reject a soft fork can easily
run a node that does so, if they wanted to - the voting mechanism still
boils down to "which side of the fork do I accept in my economic activity".
It's certainly garbage to claim that the reason to want to avoid soft forks
is being an Evil Centralised Foundation: this is about a set of
engineering tradeoffs only.
[-- Attachment #2: Type: text/html, Size: 1322 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug
2014-11-06 23:36 ` Justus Ranvier
2014-11-07 0:03 ` Peter Todd
@ 2014-11-15 4:43 ` Jean-Pierre Rupp
1 sibling, 0 replies; 18+ messages in thread
From: Jean-Pierre Rupp @ 2014-11-15 4:43 UTC (permalink / raw)
To: bitcoin-development
[-- Attachment #1.1: Type: text/plain, Size: 2364 bytes --]
Jean-Pierre Rupp from Haskoin here.
I support a hard fork to fix consensus bugs. The Bitcoin protocol should eventually get to a state where it is documented in a clear and understandable fashion. Bugs are bugs, and are the enemy. We should not attempt to live with them. We should be opening a process of thoroughly documenting and reparing consensus bugs on a separate branch, and eventually schedule a hard fork.
There are two good things that will come out of that:
1. Known bugs will be gone, and
2. We will have a process in place to get rid of future bugs in eventual future hard forks.
We do not need to become paranoid about the ramifications of a hard fork, or how it will open the door for unwanted changes in the protocol. We are discussing about removing bugs, and bugs that could be used to exploit the network in ways that may not be immediately obvious.
There are 144 blocks generated per day by groups of miners that are mostly identified. It is not going to be a titanic task to get consensus from the main mining pools on fixing this at the mining level. We must address how the fixes for some of these bugs affect other types of software such as wallets. I can think that fixing the bug where OP_CHECKMULTISIG pops an extra value from the stash could be more traumatic, since it requires anything that creates and validates multi-signature transactions to change the way it works. Hardware wallets could be impacted. But most of the consensus bugs would not affect the way the vast majority of bitcoin transactions that are currently created. Therefore it should not be traumatic at all for users, but only really affect mining pools, who would only need to be convinced to upgrade their bitcoind well in advance, which seems to me that it is not an issue at all.
We should not compare doing a Bitcoin hard-fork with doing something like deploying IPv6 world-wide or enforcing TLS and SPF on every SMTP connection. We should not conflate Bitcoin with other network protocols. The Bitcoin protocol is actually relatively easy to upgrade at this point. Let's take advantage of this fact.
On 06/11/14 15:36, Justus Ranvier wrote:
> Because Bitcoin has a extra consensus requirements, requirements which
> are really rare in engineering, the necessity of fixing bugs is even
> greater.
--
Be Happy :)
[-- Attachment #1.2: 0x310A8A5B.asc --]
[-- Type: application/pgp-keys, Size: 4154 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2014-11-15 5:38 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-11-06 21:32 [Bitcoin-development] The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug Peter Todd
2014-11-06 21:58 ` Tamas Blummer
2014-11-06 22:05 ` Matt Corallo
2014-11-06 22:11 ` Jeff Garzik
2014-11-06 22:48 ` Justus Ranvier
2014-11-06 23:26 ` Peter Todd
2014-11-06 23:36 ` Justus Ranvier
2014-11-07 0:03 ` Peter Todd
2014-11-07 8:07 ` Tamas Blummer
2014-11-07 8:48 ` Peter Todd
2014-11-07 11:30 ` Clément Elbaz
2014-11-07 11:47 ` Peter Todd
2014-11-07 12:01 ` Wladimir
2014-11-07 16:52 ` Mike Hearn
2014-11-15 4:43 ` Jean-Pierre Rupp
2014-11-06 23:19 ` Peter Todd
2014-11-06 23:12 ` Peter Todd
2014-11-07 2:04 ` Gregory Maxwell
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox