* [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat?
@ 2015-06-19 20:42 Matt Smith
2015-06-19 21:25 ` Matt @ Envrin Group
2015-06-20 10:11 ` Jonas Schnelli
0 siblings, 2 replies; 7+ messages in thread
From: Matt Smith @ 2015-06-19 20:42 UTC (permalink / raw)
To: bitcoin-development
[-- Attachment #1.1: Type: text/plain, Size: 1952 bytes --]
Hey guys,
The crew at Gem is considering a new HD wallet path structure for our
wallets, which are coin-agnostic, that separates the coin_type field
into two fields as such:
m / purpose' / network' / asset_type' / account' / change / index
where network refers to the blockchain (0 - bitcoin, 1 - testnet3, 2 -
litecoin, etc) and the new asset_type refers to the kind of asset to be
held in accounts below that path (Open Assets, Omni, Counterparty).
The intent is to allow us to validate the address format, select the
appropriate daemon to scan for tokenized assets, and choose multiple
blockchain data sources (that may not know anything about token systems
running on the blockchain they expose) relevant to an HDNode in the
wallet using only information in the HDNode's path -- without having to
maintain an explicit mapping of coin_type -> network.
For example, we already have the issue of mapping network identifiers
because of the lack of standardization across cryptocurrency libraries
which ends up being ugly and obnoxious to maintain, i.e.
netcode_map = {
testnet: testnet3,
bitcoin_testnet: testnet3,
testnet3: testnet3,
XTN: testnet3, ...
}
netcode_i_want = netcode_map[netcode_returned_by_libwhatever]
We want to avoid maintaining a similar asset_type_to_blockchain mapping.
Additionally, it would be helpful for utxo selection to exclude utxos
tied to assets based on path.
BIP43 seems to suggest that we request a BIP number and publish an
informational BIP specifying the new purpose. If that's not appropriate,
then maybe we just need to publish the information in a blog post to
allow any wallet developers who want to to implement
import_from_gem_structure.
I was also wondering if anyone had previously suggested something
similar that I missed when cruising the mailing list archives on the
subject.
Thanks,
–
Matt Smith | Gem
https://gem.co | GH: @thedoctor
[-- Attachment #1.2: 0x63331857.asc --]
[-- Type: application/pgp-keys, Size: 2201 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat?
2015-06-19 20:42 [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat? Matt Smith
@ 2015-06-19 21:25 ` Matt @ Envrin Group
2015-06-19 23:31 ` Matt Smith
2015-06-20 10:11 ` Jonas Schnelli
1 sibling, 1 reply; 7+ messages in thread
From: Matt @ Envrin Group @ 2015-06-19 21:25 UTC (permalink / raw)
To: Matt Smith, bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 2958 bytes --]
Hi Matt,
I think your best bet is probably just push it out privately via blog
post / Github, and see if it gains any traction with other developers.
I'm a little uncertain as to the relevance though. All those variables
(purpose, network, asset_type, account, change, index) need to be stored
internally within the wallet database, as there's no way to retrieve the
path used from just the address, correct? In that case, what's the
meaning of that exact path structure when a) it can't be retrieved from
just the address, and b) the values will be stored internally within the
wallet when you lookup the address.
Matt
On 06/20/2015 03:42 AM, Matt Smith wrote:
> Hey guys,
>
> The crew at Gem is considering a new HD wallet path structure for our
> wallets, which are coin-agnostic, that separates the coin_type field
> into two fields as such:
>
> m / purpose' / network' / asset_type' / account' / change / index
>
> where network refers to the blockchain (0 - bitcoin, 1 - testnet3, 2 -
> litecoin, etc) and the new asset_type refers to the kind of asset to be
> held in accounts below that path (Open Assets, Omni, Counterparty).
>
> The intent is to allow us to validate the address format, select the
> appropriate daemon to scan for tokenized assets, and choose multiple
> blockchain data sources (that may not know anything about token systems
> running on the blockchain they expose) relevant to an HDNode in the
> wallet using only information in the HDNode's path -- without having to
> maintain an explicit mapping of coin_type -> network.
>
> For example, we already have the issue of mapping network identifiers
> because of the lack of standardization across cryptocurrency libraries
> which ends up being ugly and obnoxious to maintain, i.e.
>
> netcode_map = {
> testnet: testnet3,
> bitcoin_testnet: testnet3,
> testnet3: testnet3,
> XTN: testnet3, ...
> }
> netcode_i_want = netcode_map[netcode_returned_by_libwhatever]
>
> We want to avoid maintaining a similar asset_type_to_blockchain mapping.
> Additionally, it would be helpful for utxo selection to exclude utxos
> tied to assets based on path.
>
> BIP43 seems to suggest that we request a BIP number and publish an
> informational BIP specifying the new purpose. If that's not appropriate,
> then maybe we just need to publish the information in a blog post to
> allow any wallet developers who want to to implement
> import_from_gem_structure.
>
> I was also wondering if anyone had previously suggested something
> similar that I missed when cruising the mailing list archives on the
> subject.
>
> Thanks,
> --
> Matt Smith | Gem
> https://gem.co | GH: @thedoctor
>
>
> ------------------------------------------------------------------------------
>
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[-- Attachment #2: Type: text/html, Size: 3792 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat?
2015-06-19 21:25 ` Matt @ Envrin Group
@ 2015-06-19 23:31 ` Matt Smith
2015-06-20 0:57 ` Andreas Petersson
2015-06-20 1:58 ` Matt @ Envrin Group
0 siblings, 2 replies; 7+ messages in thread
From: Matt Smith @ 2015-06-19 23:31 UTC (permalink / raw)
To: Matt @ Envrin Group, bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 2380 bytes --]
I'm not sure I understand your question about the need to store paths in
the wallet database -- there's no way to infer the path of an address
inside an HD wallet from the address alone (short of an exhaustive
search), and HD wallets need to store either the paths, addresses, or
both that have been previously derived/used to monitor the blockchain
usefully, but those facts aren't new or specific to this path format.
The motivation for this path structure over standard bip44 is that it
separates the concept of network (or which blockchain I'm using) and
coin_type (or what kind of thing I'm sending over that blockchain).
This is useful, for example, if I want to import a wallet into my
application and I know that an account was in use at
m/##'/0'/99'/0'
where 99 is the identifier for, say, counterparty - I only need to check
the addresses derived below that path for balances against
counterpartyd. It may be worth pointing out that I expect multisig HD
wallet imports to require master keys and a list of account paths – not
a list of addresses, as it's very possible that a new address could be
derived between the time when the wallet data was exported and when it
will be imported.
This use case might be very specific to our model, but the reason I
figured we should request a BIP # for this is that to start using it, we
need to pick a number for the purpose field and don't want to do it
arbitrarily (and risk having to change it later) or overload 44 (which
would be misleading).
Did I either a) answer or b) misunderstand your questions?
--
Matt Smith | Gem
https://gem.co | GH: @thedoctor
On 6/19/15 2:25 PM, Matt @ Envrin Group wrote:
>
> Hi Matt,
>
> I think your best bet is probably just push it out privately via blog
> post / Github, and see if it gains any traction with other developers.
>
> I'm a little uncertain as to the relevance though. All those variables
> (purpose, network, asset_type, account, change, index) need to be stored
> internally within the wallet database, as there's no way to retrieve the
> path used from just the address, correct? In that case, what's the
> meaning of that exact path structure when a) it can't be retrieved from
> just the address, and b) the values will be stored internally within the
> wallet when you lookup the address.
>
> Matt
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat?
2015-06-19 23:31 ` Matt Smith
@ 2015-06-20 0:57 ` Andreas Petersson
2015-06-20 2:40 ` Matt Smith
2015-06-20 1:58 ` Matt @ Envrin Group
1 sibling, 1 reply; 7+ messages in thread
From: Andreas Petersson @ 2015-06-20 0:57 UTC (permalink / raw)
To: bitcoin-development
>m/##'/0'/99'/0'
>
>where 99 is the identifier for, say, counterparty
What is stopping you from using m/44'/9'/a'/c/i as descibed here:
http://doc.satoshilabs.com/slips/slip-0044.html
to avoid having an internal mapping from 9'-> 0' to find out what
blockchain to query, this sounds like it should be trivial for any wallet.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat?
2015-06-19 23:31 ` Matt Smith
2015-06-20 0:57 ` Andreas Petersson
@ 2015-06-20 1:58 ` Matt @ Envrin Group
1 sibling, 0 replies; 7+ messages in thread
From: Matt @ Envrin Group @ 2015-06-20 1:58 UTC (permalink / raw)
To: Matt Smith, bitcoin-development
Say you generate a child key using the path m/6'/4'/7'/99'/0/196, which
is what your proposed path structure would be, and it results in the
address 1DpY7PtPVURvjrGsdAjbZAZ7cL9GD8tc5w.
When the wallet notices a transaction in the blockchain that has
1DpY7PtPVURvjrGsdAjbZAZ7cL9GD8tc5w as an output, it's going to have to
lookup the address within its database to get the values 6/4/7/99/0/196,
as there's no way to retrieve them from just the address. So
technically, you might as well just use m/account'/change/index if using
hardened child keys, or m/change/index if not, as recommended, because
the wallet will still function the exact same way.
Matt
On 06/20/2015 06:31 AM, Matt Smith wrote:
> I'm not sure I understand your question about the need to store paths in
> the wallet database -- there's no way to infer the path of an address
> inside an HD wallet from the address alone (short of an exhaustive
> search), and HD wallets need to store either the paths, addresses, or
> both that have been previously derived/used to monitor the blockchain
> usefully, but those facts aren't new or specific to this path format.
>
> The motivation for this path structure over standard bip44 is that it
> separates the concept of network (or which blockchain I'm using) and
> coin_type (or what kind of thing I'm sending over that blockchain).
>
> This is useful, for example, if I want to import a wallet into my
> application and I know that an account was in use at
>
> m/##'/0'/99'/0'
>
> where 99 is the identifier for, say, counterparty - I only need to check
> the addresses derived below that path for balances against
> counterpartyd. It may be worth pointing out that I expect multisig HD
> wallet imports to require master keys and a list of account paths – not
> a list of addresses, as it's very possible that a new address could be
> derived between the time when the wallet data was exported and when it
> will be imported.
>
> This use case might be very specific to our model, but the reason I
> figured we should request a BIP # for this is that to start using it, we
> need to pick a number for the purpose field and don't want to do it
> arbitrarily (and risk having to change it later) or overload 44 (which
> would be misleading).
>
> Did I either a) answer or b) misunderstand your questions?
> --
> Matt Smith | Gem
> https://gem.co | GH: @thedoctor
>
>
>
> On 6/19/15 2:25 PM, Matt @ Envrin Group wrote:
>> Hi Matt,
>>
>> I think your best bet is probably just push it out privately via blog
>> post / Github, and see if it gains any traction with other developers.
>>
>> I'm a little uncertain as to the relevance though. All those variables
>> (purpose, network, asset_type, account, change, index) need to be stored
>> internally within the wallet database, as there's no way to retrieve the
>> path used from just the address, correct? In that case, what's the
>> meaning of that exact path structure when a) it can't be retrieved from
>> just the address, and b) the values will be stored internally within the
>> wallet when you lookup the address.
>>
>> Matt
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat?
2015-06-20 0:57 ` Andreas Petersson
@ 2015-06-20 2:40 ` Matt Smith
0 siblings, 0 replies; 7+ messages in thread
From: Matt Smith @ 2015-06-20 2:40 UTC (permalink / raw)
To: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 1760 bytes --]
> to avoid having an internal mapping from 9'-> 0' to find out what
> blockchain to query, this sounds like it should be trivial for any wallet.
Trivial to implement, a headache to *maintain*
But if a new platform is released on an existing blockchain, my wallet
doesn't need to know about the new magic number it claims in order to
handle it correctly.
Say I make a new token layer, BobCoin, which runs on bitcoin and say I
use an HD wallet and always generate new BobCoin token addresses as
m/##'/0'/808'/*'/*/*. If I import that wallet into older HD wallet
software that doesn't know anything about BobCoin, it will still:
- understand what blockchain to query for utxos on the addresses below
that path
- be able to generate valid BobCoin addresses without any updates
I think this is particularly valuable if you're developing against a
platform where updates can't be forced on clients.
To be clear: I am not suggesting this as a general-purpose successor to
BIP44.
–
Matt Smith | Gem
https://gem.co | GH: @thedoctor
On 6/19/15 5:57 PM, Andreas Petersson wrote:
>> m/##'/0'/99'/0'
>>
>> where 99 is the identifier for, say, counterparty
>
>
> What is stopping you from using m/44'/9'/a'/c/i as descibed here:
> http://doc.satoshilabs.com/slips/slip-0044.html
>
> to avoid having an internal mapping from 9'-> 0' to find out what
> blockchain to query, this sounds like it should be trivial for any wallet.
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat?
2015-06-19 20:42 [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat? Matt Smith
2015-06-19 21:25 ` Matt @ Envrin Group
@ 2015-06-20 10:11 ` Jonas Schnelli
1 sibling, 0 replies; 7+ messages in thread
From: Jonas Schnelli @ 2015-06-20 10:11 UTC (permalink / raw)
To: bitcoin-development
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
> m / purpose' / network' / asset_type' / account' / change / index
>
> ...
>
> I was also wondering if anyone had previously suggested something
> similar that I missed when cruising the mailing list archives on
> the subject.
Hi Matt
a)
In my opinion, HD chain standards like Bip43/44 are there to allow
users to use different wallets with the same seed/wordlist (switch
over or use in parallel).
In practice, wallet re-importing / re-creating is not trivial and has
risks of missing out spent or spendable coins.
Take the current landscape. Electrum doesn’t follow the Bip39 standard
[1]. Breadwallet does not support Bip44 [2].
If users start to think, as long as they keep the Bip39 wordlist save,
nothing can happen, is a material fallacy (and i see a trend into this
direction).
A hd seed will not recover your non-blockchain-metadata. HD seed
recovery is a disaster recovery method.
Because of that I am unsure if standards of hd chainstructurs does lead
users and developers into the right directions.
In my opinion, reimporting a hd seed is a experts feature that requires
manual configuration.
b)
Creating deep level chains like m/a/b/c/d/e/f/g/h requires more cpu
power to derive private key with a given seed or wordlist.
If you are using hardware wallets like Trezor or Digitalbitbox, you
probably feel the required time when signing/deriving.
[1] https://bitcointalk.org/index.php?topic=1082586.0
[2] https://github.com/voisine/breadwallet/issues/131
</jonas>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBAgAGBQJVhTxdAAoJECnUvLZBb1Ps7HIP/00wqG9uEDGE77S6QsSmWWWm
OHYSlWzSeFURnkWyH4B1pcyGL9JxkHJMqmiHr5aBGIt0WGM8S1xwFImg7aFeWe8B
s5tjT80QwH6/RrGnrIFwdJp4QdAriVq7G0lfexika8O8LxW45LzP7BTQ1SoFVZ5o
mQuEzLG3A+NaKREqUMXBiSWAAuM9m08GBcRGsqd9tpvUUxqiJFHAbKqn2yVtuR3h
f+wJeQyjUf6ZqkIDor73lyjay6YpDyYveHaIxP9WfKJGvEVCVisuOXx0C3C7u5ps
SF1p2xCQUEC1xRVHySBgr0VdTYQlMpSVg7XOWSPRPXjfyBBwGW9aLZyGuDDTe0ca
Wa+u3CNRg3wIwsp3gG15wWK/M2eFPMKNwlE0DqSP9Siin0QlLRcuRe39D5BSjcsQ
xfQXNEf2gA0C7rMGA47K5fmxnB4qukkM8UqrOSqtczUO2AP1vdhhwJRi+/Cq9l1r
i1isL4cncwtf3/z965eN9vvou+a1dgU9L2gONmHp5eHwOtPh83u8BXEUof+VMUrN
B+vLHJMRRMucyASlehmzGe8EJiJ3Hlo4NjVpTNg/V8bnHY59lpm6GylvCa7YjR/d
xWB6jmG/7nqyRrJ0byIdZRGLtr/LuleIoFCwPUAO1jR4x626h0iVo82wPBPQXAJ1
IhkX/FW90s9i8vErMjwf
=To1w
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-06-20 10:11 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-06-19 20:42 [Bitcoin-development] Alternate HD path structure: BIP, blog, or wat? Matt Smith
2015-06-19 21:25 ` Matt @ Envrin Group
2015-06-19 23:31 ` Matt Smith
2015-06-20 0:57 ` Andreas Petersson
2015-06-20 2:40 ` Matt Smith
2015-06-20 1:58 ` Matt @ Envrin Group
2015-06-20 10:11 ` Jonas Schnelli
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox