--- Log opened Sat Nov 17 07:35:01 2018
07:35 -!- gnusha [~gnusha@unaffiliated/kanzure/bot/gnusha] has joined #rust-bitcoin
07:35 -!- Topic for #rust-bitcoin: Rust-Bitcoin Libraries Discussion. Artisinal Bitcoin for the rest of us. Do NOT have private conversations on the FreeNode network! See https://github.com/rust-bitcoin If you created a repo, and I forgot to make you admin on it (stupid github defaults), ping BlueMatt
07:35 -!- Topic set by BlueMatt [~BlueMatt@unaffiliated/bluematt] [Sun Aug  5 02:06:05 2018]
07:35 [Users #rust-bitcoin]
07:35 [+andytoshi  ] [ dpc             ] [ grubles      ] [ mikethetike  ] [ sdaftuar    ] [ treyzania] 
07:35 [ ariard     ] [ e4xit           ] [ gwillen      ] [ molz         ] [ sgeisler    ] [ warren   ] 
07:35 [ belcher    ] [ elichai2        ] [ harding      ] [ neonknight   ] [ simian_za   ] [ windsok  ] 
07:35 [ Blackwolfsa] [ esotericnonsense] [ havik        ] [ nickler      ] [ sipa        ] [ wraithm  ] 
07:35 [ BlueMatt   ] [ gmaxwell        ] [ instagibbs   ] [ savil        ] [ stevenroose ] [ wumpus   ] 
07:35 [ cjs77      ] [ gnusha          ] [ jamesob      ] [ schmidty     ] [ TamasBlummer] 
07:35 [ dongcarl   ] [ gribble         ] [ midnightmagic] [ scoobybejesus] [ Terr        ] 
07:35 -!- Irssi: #rust-bitcoin: Total of 40 nicks [0 ops, 0 halfops, 1 voices, 39 normal]
07:35 -!- Channel #rust-bitcoin created Fri Mar  9 09:46:56 2018
07:35 -!- Irssi: Join to #rust-bitcoin was synced in 1 secs
07:36 < stevenroose> an example reasoning for that is that most software projects will either always use compressed or uncompressed keys
07:37 < gnusha> irc logs http://gnusha.org/rust-bitcoin/
07:37 < stevenroose> so they really don't need to keep that extra metadata around in all their types. they can just have a global variable that they pass the moment it's needed, like for address creation
07:37 < stevenroose> gnusha: awesome!
07:37 -!- kanzure [~kanzure@unaffiliated/kanzure] has joined #rust-bitcoin
07:38 < kanzure> if you have old irssi-formatted irc logs then i can backpopulate that archive
07:43 <+andytoshi> stevenroose: the compressed/uncompressed distinction happens whenever you want to encode an EC key on the blockchain
07:43 <+andytoshi> as for "why should a private key correspond to an address", what else would it correspond to?
07:45 < stevenroose> nothing in particular. you can use it to crate a p2pk scriptPubKey, an p2pkh address, a p2sh address, a p2wpkh address, make it part of a p2sh multisig scriptPubKey, a p2wsh multisig scriptPubKey
07:47 <+andytoshi> heh, you need compressedness for literally everything you just listed
07:51 < stevenroose> Well, that was a response to what should a private key correspond to.  No strong opinions, though, just thinking.  Wrote it up here: https://github.com/rust-bitcoin/rust-bitcoin/pull/194
07:52 < stevenroose> I don't think the address example makes any less sense: https://github.com/rust-bitcoin/rust-bitcoin/pull/194/files#diff-7769ce7612a45cca417dce18a5aa6a50R32
07:53 <+andytoshi> oh, goodness, IIRC we had an API like this long in the past and it was so freeking annoying passing network flags around everywhere
07:54 <+andytoshi> and users would have to provide them in some sort of config file even though they were implied by multiple other things, because otherwise the ordering got complicated
07:55 <+andytoshi> also why would we even have PrivateKey and PublicKey types in rust-bitcoin if they're just wrapping the types from rust-secp
07:55 <+andytoshi> i think this API is extremely unergonomic. users are forced to carry around this aux data literally every time they want to use an EC key
07:56 < stevenroose> Hmm, I don't really see why. I mean isn't almost any application doing only one network at a time and can just default to either compressed or uncompressed?
07:57 < stevenroose> You need to have that aux available either at key construction or at address construction fwiw.
07:57 <+andytoshi> so, the liquid functionaries are never working with less than two networks
07:57 <+andytoshi> and i have zero interest in hardcoding `true` in 6 billion places because some API author didn't want to store a boolean
07:58 < stevenroose> well, if compressed is very much the default, an Address::p2pkh_uncompressed is easy
07:58 <+andytoshi> if you have an uncompressed secret key you cannot get an uncompressed public key
07:58 <+andytoshi> or vice versa
07:58 <+andytoshi> we are not implementing an API where that is permitted and produces nonsense
07:58 <+andytoshi> please
07:59 < stevenroose> fwiw, don't the liquid functionaries use rust-elements and rust-bitcoin for both network? so they do one network per library basically
07:59 <+andytoshi> yes, ok, so they'd have to hardcode twice as many constants
08:00 < stevenroose> that's not true, you have to pass that data also when you create the privkey/pubkey struct
08:00 < stevenroose> so it's basically just doing it in another place
08:00 <+andytoshi> you mean instead of getting it from users' addresses i have to force them to redundantly put it in some config file, and then deal with inconsistencies?
08:00 < stevenroose> your argument is that you have to drag that context around from the one place to the other, but I'm saying in the first place it's irrelevant, so it's just moving to the other place
08:00 <+andytoshi> or i hardcode it somewhere and recompile every time the network changes?
08:01 <+andytoshi> this context is not irrelevant, it is literally part of the key and is needed for basically every use of the key
08:01 <+andytoshi> and if you remove it then it can become mangled or become inconsistent, users need to figure out a story for where they're going to store/infer the extra context, etc
08:04 <+andytoshi> i would sooner delete the PrivateKey type entirely than i would remove the compressedness
08:04 <+andytoshi> and i'm not really sold on removing the network flag either
08:05 < stevenroose> in all fairness, like this, I'd also just remove the both types, use the secp structs and have util::wif::encode_wif and util::wif::decode_wif
08:06 < stevenroose> I'm not saying this design is better, I don't have that much experience building large applications using this either.  Just seems weird that a private key, in essense just any random 32-byte string, is having a network and compressedness attached
08:06 <+andytoshi> ok, i can almost get behind that
08:06 <+andytoshi> my next question is, what key type would go into descriptors then
08:06 <+andytoshi> a (secp::PublicKey, bool) tuple?
08:06 < stevenroose> for which the compressedness if only used to pass into the pubkey struct later
08:07 < stevenroose> well, one big question is: how frequently used are uncompressed keys?
08:07 <+andytoshi> infrequently enough that i'm willing to make people type _uncompressed in a bunch of places to use them
08:07 < stevenroose> if it's really niche, having _uncompressed everywhere just for the awkward user would be fine
08:07 <+andytoshi> yeah
08:08 <+andytoshi> but i _don't_ want to have a type structure exported where people can accidentally lose track of compressedness
08:08 < stevenroose> would it be possible to have descriptors also explicitly indicate compressedness?
08:08 < stevenroose> or would that become very verbose?
08:08 <+andytoshi> i think we're going to need to have a PrivateKey type and a PublicKey type that include compressedness, but we can probably get away with just using them to instantiate a Descriptor<T> type
08:09 <+andytoshi> stevenroose: they already do
08:09 <+andytoshi> you cannot de/serialize a descriptor without the compressedness of its keys (using the 'standard' WIF/hex encoding of private/public keys that wallets use)
08:09 <+andytoshi> this is literally why we're having this discussion .. because we needed types for PSBT that wouldn't lose this info
08:11 <+andytoshi> so, i'd be willing to have a `pub struct PublicKey { key: secp256k1::PublicKey; compressed: bool; }` with nothing implemented on them except the de/serialization support that descriptors need, and make users manually construct these types when they need them
08:11 <+andytoshi> and also be willing to require users provide a Network when serializing private keys and descriptors
08:11 <+andytoshi> and when converting to Addresses
08:13 <+andytoshi> it'd be cool to have no Privatekey/PublicKey and just work with (secp256k1::SecretKey, bool) tuples, but rust doesn't let us add methods to those
08:13 < stevenroose> I was thinking of a CompressedPublicKey type but that also doesn't work
08:14 < stevenroose> need to have a unified type for a method signature, right?
08:14 <+andytoshi> yeah, annoyingly you sometimes need both at once
08:14 <+andytoshi> yeah
08:14 <+andytoshi> you can parameterize over different kinds of keys
08:14 <+andytoshi> so we could have both
08:14 <+andytoshi> but if you had a multisig output that had both compressed and uncompressed keys in it, you'd need some sort of Descriptor type that could support that
08:15 < stevenroose> who came up with this idea of optional compression in the first place :D
08:15 <+andytoshi> lol
08:15 <+andytoshi> openssl
08:15 <+andytoshi> actually no.
08:15 <+andytoshi> it was in the secp256k1 standard
08:15 <+andytoshi> in 2001
08:16 < stevenroose> did they cite any reason for which one would use uncompressed keys? other than tx malleability attacks of some sort?
08:16 < stevenroose> performance?
08:16 <+andytoshi> yes, there is a dramatic time/memory tradeoff
08:17 <+andytoshi> err, time/storage
08:17 <+andytoshi> you need to decompress a key to validate a signature, which requires a field sqrt
08:18 <+andytoshi> ah, it was actually 2000, not 2001 .. i couldn't find the doc but djb rehosted it https://safecurves.cr.yp.to/www.secg.org/SEC2-Ver-1.0.pdf
08:19 <+andytoshi> lol some of these curves are hilarious, sect113r1 etc ... those didn't make it to v2 i'm pretty sure
08:20 < stevenroose> those kind of "papers"/documents are so strange to me :D
08:21 < stevenroose> but RSA primes are even stranger I guess; "hey guys we found some super nice numbers, check them out!"
08:21 <+andytoshi> they don't give a justification for having compressed public keys .. which is kinda interesting, in those days people cared about signing perf but basically nothing else
08:21 <+andytoshi> so you'd think they'd have just stuck with a single compact public key format
08:22 <+andytoshi> as that has no effect on signing at all
08:24 < stevenroose> academics will never miss an opportunity to casually mention that you could easily half any of their metric, even if no industry use :D
08:26 <+andytoshi> heh, well, secg was mostly certicom, which had smartcards deployed all over the place
08:31 < stevenroose> hmm, so I came to my coworking to install Qubes OS on an Intel MUC I had lying around but I can't find a USB stick :/
08:32 <+andytoshi> perhaps if you yelled "what a shitty coworking space, nobody has a usb stick" ;)
08:36 < stevenroose> it's saturday, I'm alone here. Plus it is actually a shitty coworking space, I'm usually alone with some (non-software) architects here :/
08:36 < stevenroose> The perks of living in a small town
08:38 <+andytoshi> heh. i'm in a random coffee shop and there are several other software people here
08:41 < stevenroose> places are different I guess
08:55  * esotericnonsense is sitting in london hackspace which is currently in the process of refurbishment so quite empty
08:56 < esotericnonsense> if i'm outside i generally go to the pub to work because the food is better and it's cheaper, not many software people there. :P
08:56 < esotericnonsense> also usually more comfortable than coffee shops.
08:57 <+andytoshi> esotericnonsense: i also do that, though i went out this morning at 8AM, which is a bit early for the pub :)
08:57 < esotericnonsense> i enjoy watching the clientele change during the day
08:58 < esotericnonsense> like, during working hours, lunchtime ramping up is quite fun
08:58 < esotericnonsense> 10am - oh it's me, hardcore alcoholics, and pensioners
08:58 < esotericnonsense> 12pm - city workers
08:58 < esotericnonsense> 2pm is 10am again. :p
08:58 <+andytoshi> lolol
09:02 <+andytoshi> i could maybe do that at a pub that i never go to
09:03 <+andytoshi> but if i recognized regulars, i wouldn't be able to resist talking to people (maybe this is less acceptable in the UK) and i'd never get anything done
09:03 < esotericnonsense> london is pretty anonymous i think just cause it's so massive? i dunno
09:03 < esotericnonsense> it's pretty uncommon to see the same person twice on a train commute
09:04 < esotericnonsense> i recognise staff in places I go to often, that's about it, perhaps I'm just buried though
09:04 <+andytoshi> oh wow, i constantly see the same people around town, people say hi to me in the grocery store and stuff
09:05 < esotericnonsense> it's weird how it works because if I go to my hometown it's pretty likely I'll bump in to someone
09:06 <+andytoshi> i live in austin, which supposedly has a couple million people, but the downtown core area feels kinda like a small town (though i guess it's much more crowded, there -are- a lot of anonymous people surrounding the folks that i know)
09:07 < stevenroose> yeah living in a town that doesn't speak your language makes those things a bit harder too :D though I do recognize people and say hi etc all the time, but more than hi gets challenging
09:08 < stevenroose> unless it's related to the task at hand.. ordering food, selecting vegetables, ..
09:11 <+andytoshi> i'd have a super hard time of that
09:12 < stevenroose> Ok so I took out wif to util::wif and now it's really just secp wrappers.  I could drop them now.  For descriptors a simple PublicKeyWithCompressedness(pub secp256k1::PublicKey, pub bool) would be sufficient in that case, no?
09:12 < stevenroose> I mean I'm still not convinced it makes sense per se, though :D
09:12 < stevenroose> Just trying to see how it feels.
09:13 < stevenroose> being able to drop the secp context for serialization would certainly make that a lot more ergonomic.
09:17 <+andytoshi> yeah, we will be able to in the next rust-secp version. probably by end of next week
09:17 <+andytoshi> so, i would prefer `PublicKeyWithCompressedness` just be called bitcoin::PublicKey :P
09:18 < stevenroose> yeah that's fair enough I think, would otherwise also get very messy with deserializing
09:28 < stevenroose> hehe, I guess I did quite a circular change by now
09:28 < stevenroose> I actually like having bitcoin::PrivateKey even though it's an empty type
09:29 < stevenroose> makes it easier if there would be upstream api changes f.e.
09:29 <+andytoshi> i continue to think that bitcoin::PrivateKey should have the compressedness flag
09:30 < stevenroose> I currently did it with a `.to_public_key()` and `to_public_key_uncompressed()`, which seemed fair.
09:30 <+andytoshi> no it doesn't
09:30 <+andytoshi> a compressed private key can't become an uncompressed public key
09:31 <+andytoshi> or vice-versa
09:31 <+andytoshi> no wallet software allows that to happen
09:31 < stevenroose> a compressed private key doesn't exist afaik..
09:31 < stevenroose> you mean "a private key that is supposed to be used to create a compressed public key when generating an address"?
09:31 < stevenroose> :p
09:32 <+andytoshi> i mean a private key with the "compressed" flag not set
09:32 <+andytoshi> one that corresponds to a compressed public key, yes
09:32 <+andytoshi> uncompressed*
09:32 <+andytoshi> core has a notion of compressed/uncompressed private keys
09:33 <+andytoshi> so does electrum
09:34 <+andytoshi> icboc does not support uncompressed keys, its keys are all compressed
09:35 <+andytoshi> ledger wallet app appears to have some global option to switch between uncompressed and compressed keys
09:35 <+andytoshi> perhaps you could name a wallet that has ever existed which did not make this distinction?
09:39 < stevenroose> don't think I can, don't know much about wallet internals. how do wallets that are fully bip32 count?
09:40 <+andytoshi> typically they only support compressed keys
09:40 <+andytoshi> but e.g. ledger can be run in an "uncompressed key" mode (maybe this requires recompiling from source, i'm not sure)
09:40 <+andytoshi> bip32 doesn't make any reference to compressedness IIRC
09:40 < stevenroose> I guess they do xpriv -> path -> key -> address
09:41 <+andytoshi> but in bitcoin,  EC keypairs have a compressedness bit associated with them, you cannot reference either half of the pair without knowing its compressedness
09:41 < stevenroose> fwiw Core has CKey which is basically everything together
09:41 < stevenroose>     static const unsigned int PRIVATE_KEY_SIZE            = 279;
09:41 < stevenroose> hey
09:41 < stevenroose> why don't we have pubkey and keypair?
09:42 < stevenroose> :D
09:42 <+andytoshi> hmmm
09:42 <+andytoshi> it's useful sometimes to refer to the private key by itself
09:43 <+andytoshi> and if you wanted to deserialize a private key to a keypair (which would be the only thing you could do), this would involve a key derivation which is horribly expensive
09:43 < stevenroose> yeah was thinking the same
09:43 < stevenroose> even though you don't necessarily have to store the pubkey non-lazily to call it a keypair
09:43 < stevenroose> but yeah I agree it would make less sense to not have the pubkey there always
09:45 <+andytoshi> well, rust makes lazy storage a bit unergonomic
09:45 <+andytoshi> because you'd have to mutate an internal field to cache it
09:54 < stevenroose> Don't give the PR too much value, will overthink things for a while.
09:54 <+andytoshi> hehe, sounds good
10:10 < dongcarl> andytoshi: could you put this up as a log? http://gnusha.org/rust-bitcoin/
10:10 < dongcarl> kanzure helped set it up
10:13 <+andytoshi> sure
10:13 -!- mode/#rust-bitcoin [+o andytoshi] by ChanServ
10:14 -!- andytoshi changed the topic of #rust-bitcoin to: Rust-Bitcoin Libraries Discussion. Artisinal Bitcoin for the rest of us. Do NOT have private conversations on the FreeNode network! | https://github.com/rust-bitcoin | Logs at http://gnusha.org/rust-bitcoin/
14:45 < dongcarl> stevenroose: I don't mind, but what's missing right now?
14:46 < dongcarl> ariard: if you have old irssi-formatted irc logs kanzure can backpopulate the gnusha archive
15:04 < ariard> kanzure: hey thanks for setting this up, I've old irssi-formatted logs from july, how do I transfer them to you ?
15:36 < sipa> stevenroose: CKey is just a 32-byte secret + compressedness flag
15:37 < sipa> stevenroose: there is also CPrivKey which is a private key in OpenSSL DER serialization (which contains pubkey, private, but also the curve parameters; that's the 279 byte thing)
15:37 < sipa> but CPrivKey is only used for backward compatibility with old wallet file formats
15:47 < kanzure> ariard: send me a link. use dropbox. upload to a file server. whatever.
16:44 < ariard> kanzure : https://github.com/ariard/irc-rust-bitcoin-log/invitations
16:48 < kanzure> ariard: alright i have a copy. you can delete on github if you'd like.
18:03 -!- _cryptosignal_me [~John@91.245.76.90] has joined #rust-bitcoin
18:37 -!- _cryptosignal_me [~John@91.245.76.90] has quit [Ping timeout: 240 seconds]
--- Log closed Sun Nov 18 00:00:25 2018