Re: [bitcoin-dev] proposal: extend WIF format for segwit

[Karl Johan Alm <karljohan-alm@garage.co.jp>]

I took the liberty of turning this into a BIP proposal -- the
formatted version can be seen here:
https://github.com/kallewoof/bips/blob/bip-typed-wif/bip-extended-privkey.mediawiki

<pre>
  BIP: XXX
  Layer: Applications
  Title: Typed Private Keys
  Author: Karl-Johan Alm <karljohan-alm@garage.co.jp>
  Comments-Summary: No comments yet.
  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-XXX
  Status: Draft
  Type: Standards Track
  Created: 2018-04-04
  License: CC0-1.0
</pre>

== Abstract ==

An extension to the private key (WIF) format to specify what kind of
public key the private key corresponds to.

== Motivation ==

There are several types of public keys which can all be associated
with a given private key: P2PKH (legacy <code>1...</code> format),
P2SH-P2WPKH (SegWit public key inside P2SH), P2WPKH (bech32), etc.

While private keys have a 1-byte suffix indicating whether the
corresponding public key is compressed (<code>0x01</code>) or not
(<code>0x00</code>), there is no way of knowing what kind of public
keys were associated with the private key. As a result, when importing
a private key, the wallet has to assume all kinds, and keep track of
each possible alternative.

By extending the suffix, we can specify what kind of public key was
associated with the given private key.

== Specification ==

Currently, private keys are stored as a uint256 (private key data)
followed by a uint8 (compressed flag). The latter is extended to
specify the public key types:

{|class="wikitable" style="text-align: center;"
|-
!Value
!Type
!Compr
!Clarification
|-
|<code>0x00</code>||P2PKH_UNCOMPRESSED||No||Uncompressed legacy public
key. Unknown public key format
|-
|<code>0x01</code>||P2PKH_COMPRESSED||Yes||Compressed legacy public
key. Unknown public key format
|-
|<code>0x80</code>||P2PKH||Yes||Compressed legacy public key. Legacy
public key format (<code>1...</code>)
|-
|<code>0x81</code>||P2WPKH||Yes||Bech32 format (native Segwit)
|-
|<code>0x82</code>||P2WPKH_P2SH||Yes||Segwit nested in BIP16 P2SH
(<code>3...</code>)
|-
|<code>0x85</code>||P2SH|| &mdash; ||Non-Segwit BIP16 P2SH (<code>3...</code>)
|-
|<code>0x86</code>||P2WSH|| &mdash; ||Native Segwit P2SH
|-
|<code>0x87</code>||P2WSH_P2SH|| &mdash; ||Native Segwit P2SH nested
in BIP16 P2SH
|}

When a wallet imports a private key, it will have two outcomes:

* the key is using one of the legacy types, in which case all types
must be accounted for
* the key is using one of the extended types, in which case the wallet
need only track the specific corresponding public key

== Rationale ==

TODO

== Compatibility ==

This proposal is not backwards compatible, in that software that does
not recognize the new types will not understand the compressed flag.
It would be trivial to change this, by keeping the 'uncompressed'
state as it is (0) and changing 'compressed' to be 'anything not 0',
as opposed to 'the value 1'.

The proposal *is* backwards compatible in that new wallet software
will always understand the old WIF format, however. It will, as it
does today, assume that any kind of public key is possible, and will
have to track all of them, as it has to today.

== Acknowledgements ==

This BIP is based on the initial proposal by Thomas Voegtlin
<thomasv@electrum.org> on the Bitcoin Dev mailing
list<ref>https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-September/015007.html</ref>
and the Electrum 3.0
implementation<ref>https://github.com/spesmilo/electrum/blob/82e88cb89df35288b80dfdbe071da74247351251/RELEASE-NOTES#L95-L108</ref>

== Reference implementation ==

There is a partial implementation which adds, but does not use, the
types described in this BIP here:
https://github.com/bitcoin/bitcoin/pull/12869

== References ==

<references/>

== Copyright ==

This document is licensed under the Creative Commons CC0 1.0 Universal license.

On Mon, Sep 18, 2017 at 12:36 AM, Mark Friedenbach via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> Bech32 and WIF payload format are mostly orthogonal issues. You can design a
> new wallet import format now and later switch it to Bech32.
>
> On Sep 17, 2017, at 7:42 AM, AJ West via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> Hi I have a small interjection about the point on error correction (excuse
> me if it seems elementary). Isn't there an argument to be made where a
> wallet software should never attempt to figure out the 'correct' address, or
> in this case private key? I don't think it's crazy to suggest somebody could
> import a slightly erroneous WIF, the software gracefully error-corrects any
> problem, but then the user copies that error onward such as in their backup
> processes like a paper wallet. I always hate to advocate against a feature,
> I'm just worried too much error correcting removes the burden of exactitude
> and attention of the user (eg. "I know I can have up to 4 errors").
>
> I'm pretty sure I read those arguments somewhere in a documentation or issue
> tracker/forum post. Maybe I'm misunderstanding the bigger picture in this
> particular case, but I was just reminded of that concept (even if it only
> applies generally).
>
> Thanks,
> AJ West
>
> On Sun, Sep 17, 2017 at 4:10 AM, Thomas Voegtlin via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> On 17.09.2017 04:29, Pieter Wuille wrote:
>> >
>> > This has been a low-priority thing for me, though, and the computation
>> > work
>> > to find a good checksum is significant.
>> >
>>
>> Thanks for the info. I guess this means that a bech32 format for private
>> keys is not going to happen soon. Even if such a format was available,
>> the issue would remain for segwit-in-p2sh addresses, which use base58.
>>
>> The ambiguity of the WIF format is currently holding me from releasing a
>> segwit-capable version of Electrum. I believe it is not acceptable to
>> use the current WIF format with segwit scripts; that would just create
>> technological debt, forcing wallets to try all possible scripts. There
>> is a good reason why WIF adds a 0x01 byte for compressed pubkeys; it
>> makes it unambiguous.
>>
>> I see only two options:
>>  1. Disable private keys export in Electrum Segwit wallets, until a
>> common WIF extension has been agreed on.
>>  2. Define my own WIF extension for Electrum, and go ahead with it.
>>
>> Defining my own format does make sense for the xpub/xprv format, because
>> Electrum users need to share master public keys across Electrum wallets.
>> It makes much less sense for WIF, though, because WIF is mostly used to
>> import/sweep keys from other wallets.
>>
>> I would love to know what other wallet developers are going to do,
>> especially Core. Are you going to export private keys used in segwit
>> scripts in the current WIF format?
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>