Re: [bitcoin-dev] proposal: extend WIF format for segwit - Karl-Johan Alm

From: Karl-Johan Alm <karljohan-alm@garage•co.jp>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] proposal: extend WIF format for segwit
Date: Tue, 10 Apr 2018 11:54:08 +0900	[thread overview]
Message-ID: <CALJw2w58QTHKUKjZBKAbkLexrEHG+OEqjtVB4=FBmth32H31CQ@mail.gmail.com> (raw)
In-Reply-To: <CALJw2w5qkyFNLCGsiObQTRb=FNac=DRt_i4B2S_99WTdr4v+xQ@mail.gmail.com>

Hello,

I made slight modification to the BIP, dropping the 0x80 jump to 0x10:
https://github.com/kallewoof/bips/blob/bip-typed-wif/bip-extended-privkey.mediawiki

I will make the corresponding changes to the reference implementation shortly.

If there are no objections I would also like to request a BIP number.

On Wed, Apr 4, 2018 at 3:06 PM, Karl Johan Alm
<karljohan-alm@garage•co.jp> wrote:
> I took the liberty of turning this into a BIP proposal -- the
> formatted version can be seen here:
> https://github.com/kallewoof/bips/blob/bip-typed-wif/bip-extended-privkey.mediawiki
>
> <pre>
>   BIP: XXX
>   Layer: Applications
>   Title: Typed Private Keys
>   Author: Karl-Johan Alm <karljohan-alm@garage•co.jp>
>   Comments-Summary: No comments yet.
>   Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-XXX
>   Status: Draft
>   Type: Standards Track
>   Created: 2018-04-04
>   License: CC0-1.0
> </pre>
>
> == Abstract ==
>
> An extension to the private key (WIF) format to specify what kind of
> public key the private key corresponds to.
>
> == Motivation ==
>
> There are several types of public keys which can all be associated
> with a given private key: P2PKH (legacy <code>1...</code> format),
> P2SH-P2WPKH (SegWit public key inside P2SH), P2WPKH (bech32), etc.
>
> While private keys have a 1-byte suffix indicating whether the
> corresponding public key is compressed (<code>0x01</code>) or not
> (<code>0x00</code>), there is no way of knowing what kind of public
> keys were associated with the private key. As a result, when importing
> a private key, the wallet has to assume all kinds, and keep track of
> each possible alternative.
>
> By extending the suffix, we can specify what kind of public key was
> associated with the given private key.
>
> == Specification ==
>
> Currently, private keys are stored as a uint256 (private key data)
> followed by a uint8 (compressed flag). The latter is extended to
> specify the public key types:
>
> {|class="wikitable" style="text-align: center;"
> |-
> !Value
> !Type
> !Compr
> !Clarification
> |-
> |<code>0x00</code>||P2PKH_UNCOMPRESSED||No||Uncompressed legacy public
> key. Unknown public key format
> |-
> |<code>0x01</code>||P2PKH_COMPRESSED||Yes||Compressed legacy public
> key. Unknown public key format
> |-
> |<code>0x80</code>||P2PKH||Yes||Compressed legacy public key. Legacy
> public key format (<code>1...</code>)
> |-
> |<code>0x81</code>||P2WPKH||Yes||Bech32 format (native Segwit)
> |-
> |<code>0x82</code>||P2WPKH_P2SH||Yes||Segwit nested in BIP16 P2SH
> (<code>3...</code>)
> |-
> |<code>0x85</code>||P2SH|| &mdash; ||Non-Segwit BIP16 P2SH (<code>3...</code>)
> |-
> |<code>0x86</code>||P2WSH|| &mdash; ||Native Segwit P2SH
> |-
> |<code>0x87</code>||P2WSH_P2SH|| &mdash; ||Native Segwit P2SH nested
> in BIP16 P2SH
> |}
>
> When a wallet imports a private key, it will have two outcomes:
>
> * the key is using one of the legacy types, in which case all types
> must be accounted for
> * the key is using one of the extended types, in which case the wallet
> need only track the specific corresponding public key
>
> == Rationale ==
>
> TODO
>
> == Compatibility ==
>
> This proposal is not backwards compatible, in that software that does
> not recognize the new types will not understand the compressed flag.
> It would be trivial to change this, by keeping the 'uncompressed'
> state as it is (0) and changing 'compressed' to be 'anything not 0',
> as opposed to 'the value 1'.
>
> The proposal *is* backwards compatible in that new wallet software
> will always understand the old WIF format, however. It will, as it
> does today, assume that any kind of public key is possible, and will
> have to track all of them, as it has to today.
>
> == Acknowledgements ==
>
> This BIP is based on the initial proposal by Thomas Voegtlin
> <thomasv@electrum•org> on the Bitcoin Dev mailing
> list<ref>https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015007.html</ref>
> and the Electrum 3.0
> implementation<ref>https://github.com/spesmilo/electrum/blob/82e88cb89df35288b80dfdbe071da74247351251/RELEASE-NOTES#L95-L108</ref>
>
> == Reference implementation ==
>
> There is a partial implementation which adds, but does not use, the
> types described in this BIP here:
> https://github.com/bitcoin/bitcoin/pull/12869
>
> == References ==
>
> <references/>
>
> == Copyright ==
>
> This document is licensed under the Creative Commons CC0 1.0 Universal license.
>
> On Mon, Sep 18, 2017 at 12:36 AM, Mark Friedenbach via bitcoin-dev
> <bitcoin-dev@lists•linuxfoundation.org> wrote:
>> Bech32 and WIF payload format are mostly orthogonal issues. You can design a
>> new wallet import format now and later switch it to Bech32.
>>
>> On Sep 17, 2017, at 7:42 AM, AJ West via bitcoin-dev
>> <bitcoin-dev@lists•linuxfoundation.org> wrote:
>>
>> Hi I have a small interjection about the point on error correction (excuse
>> me if it seems elementary). Isn't there an argument to be made where a
>> wallet software should never attempt to figure out the 'correct' address, or
>> in this case private key? I don't think it's crazy to suggest somebody could
>> import a slightly erroneous WIF, the software gracefully error-corrects any
>> problem, but then the user copies that error onward such as in their backup
>> processes like a paper wallet. I always hate to advocate against a feature,
>> I'm just worried too much error correcting removes the burden of exactitude
>> and attention of the user (eg. "I know I can have up to 4 errors").
>>
>> I'm pretty sure I read those arguments somewhere in a documentation or issue
>> tracker/forum post. Maybe I'm misunderstanding the bigger picture in this
>> particular case, but I was just reminded of that concept (even if it only
>> applies generally).
>>
>> Thanks,
>> AJ West
>>
>> On Sun, Sep 17, 2017 at 4:10 AM, Thomas Voegtlin via bitcoin-dev
>> <bitcoin-dev@lists•linuxfoundation.org> wrote:
>>>
>>> On 17.09.2017 04:29, Pieter Wuille wrote:
>>> >
>>> > This has been a low-priority thing for me, though, and the computation
>>> > work
>>> > to find a good checksum is significant.
>>> >
>>>
>>> Thanks for the info. I guess this means that a bech32 format for private
>>> keys is not going to happen soon. Even if such a format was available,
>>> the issue would remain for segwit-in-p2sh addresses, which use base58.
>>>
>>> The ambiguity of the WIF format is currently holding me from releasing a
>>> segwit-capable version of Electrum. I believe it is not acceptable to
>>> use the current WIF format with segwit scripts; that would just create
>>> technological debt, forcing wallets to try all possible scripts. There
>>> is a good reason why WIF adds a 0x01 byte for compressed pubkeys; it
>>> makes it unambiguous.
>>>
>>> I see only two options:
>>>  1. Disable private keys export in Electrum Segwit wallets, until a
>>> common WIF extension has been agreed on.
>>>  2. Define my own WIF extension for Electrum, and go ahead with it.
>>>
>>> Defining my own format does make sense for the xpub/xprv format, because
>>> Electrum users need to share master public keys across Electrum wallets.
>>> It makes much less sense for WIF, though, because WIF is mostly used to
>>> import/sweep keys from other wallets.
>>>
>>> I would love to know what other wallet developers are going to do,
>>> especially Core. Are you going to export private keys used in segwit
>>> scripts in the current WIF format?
>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists•linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists•linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists•linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>