--- Day changed Mon Aug 06 2018 00:39 < andytoshi> cool, thanks 00:40 < andytoshi> btw dongcarl i have a parser and serializer for a subset of bitcoin script, and the fuzzer can't find any more scriptpubkeys that fail to round trip 00:41 < andytoshi> i need to write code to extract lists of required sigs, and to make scriptsigs/witnesses 00:41 < andytoshi> but i'm close to something general that should let us support psbt in a useful way without a bunch of narrow template 00:55 < dongcarl> andytoshi: perfect... BTW I've switched to using a generic SimpleDecoder for consensus_decode... But I need to specify SimpleDecoder as I'd like to use my own Error enum for Errors instead of a generic string... Is that okay? 00:56 < dongcarl> It seems like someone else ran into this limitation in RawNetworkMessage 00:56 < dongcarl> (which I assume is deprecated) 00:58 < dongcarl> As in, I added a PSBT variant to util::Error, added From for util::Error 00:58 < andytoshi> Yeah, I see 00:59 < andytoshi> I think SimpleDecoder should just fix the error type tbh 00:59 < andytoshi> but fixing the hasher story is a higher priority (and there's still a fair bit to do there), so what you've done is fine for new 01:00 < dongcarl> cool cool, I tried to be fancy by constraining the error type with From but it just makes things hard to introspect 01:00 < dongcarl> andytoshi: Have you seen: https://github.com/rust-lang-nursery/failure 01:01 < andytoshi> yeah, don't bother tryingto be too generic with errors 01:01 < dongcarl> I've watched the talk, it's a pretty good abstraction tbh 01:02 < andytoshi> nice 01:02 < andytoshi> one day :P 01:02 < dongcarl> andytoshi: It's one of the main reasons I wanted to hurry the debian devs... It requires 1.18+ 01:02 * dongcarl longs for that day 01:02 < andytoshi> well, it looks like it's still in flux anyway 01:02 < andytoshi> the failure crate 01:03 < dongcarl> andytoshi: yeah it did go thru several iterations 01:03 < dongcarl> andytoshi: I'm gunna tidy things up, rebase, and document now... Lmk if anything else needs doing 01:04 < andytoshi> cool, thanks. i think once i get the script AST stuff working i'd like you to take a look at that and figure out how to implement it 01:04 < andytoshi> unfortunately both PSBT and the script-AST stuff are pretty big chunks of code 01:05 < dongcarl> Yeah, I'll do a fine-grain review of AST once you're done if that's cool 01:05 < andytoshi> and the design of script-AST is still somewhat in flux while sipa works on optimal compilation from script descriptors ... probably we should try to make both of those into BIPs 01:05 < andytoshi> that'd be awesome 01:06 < dongcarl> andytoshi: did you guys make any more progress with script descriptors? 01:06 < dongcarl> andytoshi: I was working out the 48 different cases ytd... Good excercise for the brain lol 01:07 < andytoshi> lol 01:07 < andytoshi> dongcarl: yes, we've pretty much nailed down the descriptor language as well as language that it compiles to 01:07 < andytoshi> but we can prune the language down because there are a lot of things that will never be optimal to compile to 01:07 < dongcarl> andytoshi: Hmm... when it's not optimal... does the compiler throw a warning? 01:08 < andytoshi> no, the compiler starts from a script descriptor which is purely declarative 01:09 < andytoshi> like, the liquid policy would look like `asymmetric_or(multi([functionary keys]), and(csv(100000), multi([emergency keys])))` 01:09 < dongcarl> Sexy 01:09 < andytoshi> and then the compiler's job is to turn that into an AST representation corresponding to the smallest possible script. (asymmetric_or tells it not to try to optimize the emergency case, because it's not expected to be used) 01:10 < andytoshi> i actually have code that handles this specifically, and the produced scriptpubkey is smaller than the one that liquid currently uses 01:10 < andytoshi> at the expense of having more expensive emergency spends 01:11 < dongcarl> Well... less expected cost so... good overall 01:11 < dongcarl> So happy this is happening 01:11 < andytoshi> dongcarl: https://0bin.net/paste/mjcwkHJ0loGXAISl#3dHBwOlwBPtHHFbhNgCpuIZJH15OOQyOGFDJzUpmGCH here is a snipped of the unit test 01:12 < dongcarl> Love it 01:12 < andytoshi> notice that the compiler decides when to use CHECKMULTISIG and when to use CHECKMULTISIGVERIFY; it moves the CSV to a different location than where i wrote it in the policy because then it can use the locktime as the final script result, saving a DROP, etc 01:13 < andytoshi> so, i need to (a) write code to produce pubkey lists and witnesses, so that the AST type is useful for PSBT; (b) write a parser for the descriptor language which will be an extension of the language that Core wallet currently uses; (c) replace the contracthash module with one that uses descriptors instead of the current hacky templates 01:13 < andytoshi> once i've done (a) i'll post a PR because that's alreday usable and i don't want overwhelm the review process 01:14 < dongcarl> andytoshi: if you've got something that works, I'd be happy to do (a), seems like you've got a lot on your plate lol 01:14 < andytoshi> i'm almost done (a), thanks 01:15 < andytoshi> if you want to give (b) a shot that'd be cool 01:15 < dongcarl> DOWN 01:15 < dongcarl> (b) in rust-bitcoin right? 01:15 < andytoshi> heh, k, lemme publish a branch somewhere one sec 01:15 < andytoshi> yeah 01:15 < dongcarl> Yayayayayayay 01:16 < andytoshi> dongcarl: https://github.com/rust-bitcoin/rust-bitcoin/blob/script-descriptor/src/blockdata/script/descriptor.rs#L47-L63 is the type that internally represents the descriptor language 01:16 < andytoshi> it needs to be extended to handle the p2pkh/p2sh/segwit distinctions in https://gist.github.com/sipa/e3d23d498c430bb601c5bca83523fa82 01:17 < andytoshi> then it would be cool if it were possible to parse text strings to such an object 01:18 < dongcarl> Cool... I'll do both? 01:18 < andytoshi> in https://github.com/rust-bitcoin/rust-bitcoin/blob/script-descriptor/src/blockdata/script/parse.rs#L200 you can see how descriptors are compiled to script-ASTs (and that the compiler is super unfinished, in fact the AST language itself needs a bit more expressivity i think) 01:18 < andytoshi> yeah, that'd be great 01:18 < andytoshi> meanwhile i'm just hacking on the AST, which is independent 01:19 < dongcarl> Ah... okay so we just need String -> Descriptor 01:19 < andytoshi> yeah 01:19 < dongcarl> roger roger 01:20 < andytoshi> so i guess, as a start it would be good to implement just the stuff defined in sipa's gist .. then talk to him about what extensions would look like 01:21 < andytoshi> also, the parsing of pubkeys should somehow go through the `Pubkey` trait, which probably needs some more methods for that purpose 01:22 < andytoshi> that way it's easy to support explicit pubkeys and bip32 paths, like Core, but also pay-to-contract keys 01:23 < dongcarl> Right 01:23 < dongcarl> Implement the p2pkh/p2sh/segwit extensions? 01:23 < dongcarl> Anything else missing? 01:24 < andytoshi> well, the xpub stuff 01:24 < andytoshi> i think that's it 01:24 < dongcarl> On it! 02:32 -!- lae [~lae@unaffiliated/musee] has quit [Quit: see you on the other side] 07:52 < dongcarl> andytoshi: Any reservations against renaming Descriptor::Key as Descriptor::Pk? It's more consistent with other things I have like Descriptor::Pkh(P) 12:29 < andytoshi> dongcarl: sure go for it 12:29 < andytoshi> you can even add `Multi` if you want, even though it's kinda redundant with `Threshold` 14:26 -!- savil[m]2 [savilmatri@gateway/shell/matrix.org/x-rhajeqmfmgutpydc] has quit [Ping timeout: 240 seconds] 14:29 -!- savil[m]2 [savilmatri@gateway/shell/matrix.org/x-apauksymihwplyga] has joined #rust-bitcoin 17:52 < dongcarl> andytoshi: BTW: https://github.com/rust-bitcoin/rust-bitcoin/pull/115, not sure if this breaks things for you so opened a PR just in case 17:54 < andytoshi> heh, lol at that being a PR on the main project .. i guess i should start using my own fork for working stuff 17:54 < dongcarl> Lol true 17:54 < dongcarl> It's to your branch tho 17:55 < andytoshi> yep, i saw that 17:55 < andytoshi> yeah i'm fine with this 17:55 < andytoshi> in future, can you just link me commits on your branch instead of doing github PRs? it's easy for me to use git (not github) to cherry-pick stuff from you 17:55 < dongcarl> Hmmm... nit... Shall we specify Sha256dHash? I know there's a single SHA-256 Opcode somewhere but I don't think anyone uses it 17:56 < andytoshi> no it should be sha256 17:56 < andytoshi> it is only sha256d right now because we don't have a sha256 type in rust-bitcoin 17:56 < andytoshi> but we will when we move to bitcoin_hashes crate 17:56 < andytoshi> sha256d will require an extra byte on every single hash preimage check 17:57 < dongcarl> Cool, if you got primitives to implement in bitcoin_hashes lmk as well... I wanna practice my chops 18:04 < dongcarl> andytoshi: what's the second element of the tuple that the compile function outputs supposed to be? The size of the *Expr in bytes? 18:08 < andytoshi> dongcarl: roughly 18:09 < andytoshi> i think in some contexts in needs to also include the size of the input 18:09 < andytoshi> i think it probably needs to output a larger tuple actually .. size in scriptpubkey, size of satisfying input, size of non-satisfying input 18:12 < dongcarl> andytoshi: yeah... make it a struct? 18:12 < andytoshi> yeah 18:13 < andytoshi> the top-level output of compile also should be a struct containing a scriptpubkey, redeemscript, witnessscript (if applicable) and list of pubkeys 18:14 < dongcarl> andytoshi: did you guys eliminate Wexprs? 18:14 < andytoshi> i subsumed it into expr in my code, which i think is fine, but sipa has not ACKed that (and his current writeup still has the compiler trying to compile random shit as wexprs even outside of a boolor or threshold) 18:15 < andytoshi> are you doing anything that needs the compiler to be mature? 18:17 < dongcarl> andytoshi: no... just trying to implement P2PKH right now... not sure how to express DUP Vexpr::HASH160EqualVerify OP_CHECKSIG 18:17 < andytoshi> oh right 18:17 < andytoshi> you can't 18:17 < andytoshi> we need to add that to the language 18:17 < andytoshi> i will do that for you 18:17 < andytoshi> for now just make it compile to something stupid, if rustc needs you to do something 18:19 < dongcarl> andytoshi: Cool cool, I can do this as well if you want... Do you just add Vexpr::P2pkh? Wanna keep the Vexpr variants clean and generic tho... 18:22 < andytoshi> no, please don't modify the language more than you need, i'm still hacking on all that stuff 18:22 < andytoshi> the AST language* 18:23 < andytoshi> you can go crazy on the descriptor type and the descriptor language..though see https://gist.github.com/sipa/b7eec358de29d8e54c74e811820ed662 for the current spec 18:26 < dongcarl> Cool 18:32 < andytoshi> sipa asked me in PM to put wexpr back btw 18:32 < andytoshi> maybe after more instrumentation and analysis we can get rid of it, but for now we should have it cuz we're unsure 18:32 < dongcarl> andytoshi: Cool cool, yeah I'm gunna leave parse.rs alone except for adding stubs to it'll compile 18:33 < andytoshi> awesome thanks 18:36 < dongcarl> andytoshi: Question, shall I implement Xpub(ExtendedPubKey, Vec) as a variant of Descriptor or do I `impl PublicKey for (ExtendedPubKey, Vec)`? 18:37 < andytoshi> the latter 18:38 < dongcarl> Right... Wouldn't this mean we can't mix normal and ExtendedPubKeys, ChildNumber tuples in the same descriptor then? 18:39 < andytoshi> yeah, you also can't impl PublicKey ... i did this because i thought i wanted to (or could) go from a script AST back to a descriptor 18:39 < andytoshi> but sipa convinced me this is impossible and undesirable 18:39 < andytoshi> so you can channge Descriptor to use Box everywhere that it uses T 18:39 < andytoshi> and remove `from_pubkey` from the `PublicKey` trait 18:40 < dongcarl> How about Box> ? 18:41 < andytoshi> no, that will make your life harder.. 18:41 < andytoshi> you'll want to add a `parse` method to the PublicKey trait 18:41 < andytoshi> it's more than a Into, it needs to be de/serializable somehow in the descriptor language 18:43 < dongcarl> Into + FromStr + Display seems to encapsulate what we want? 18:44 < andytoshi> lol, well, ToString is better than Display (which implies ToString) 18:45 < andytoshi> maybe do a `type PublicKey = Box + FromStr + ToString>` 18:45 < dongcarl> I think it says in the docs not to implement ToString but to instead implement Display... 18:45 < andytoshi> yes 18:46 < dongcarl> `type PublicKey = Box + FromStr + Display>` then? 18:46 < andytoshi> sure 18:46 < dongcarl> (this is so cleeeean) 18:46 < andytoshi> lol 18:46 < andytoshi> I'd prefer ToString but if you're using a formatter i guess Display is theoretically more efficient.. 18:47 < andytoshi> more things implement ToString than implement Display, even if the docs say people shouldn't do that 18:47 < dongcarl> andytoshi: Also makes it nicely printable! 18:47 < andytoshi> ah, yeah 18:47 < andytoshi> i guess you can implmeent Display for the whole descriptor this way 18:48 < andytoshi> without .to_string() on every public key 18:48 < dongcarl> andytoshi: that's my plan, stan 18:48 < andytoshi> kool 20:03 -!- schmidty [~schmidty@unaffiliated/schmidty] has joined #rust-bitcoin