--- Day changed Fri Nov 27 2020
03:40 -!- shesek [~shesek@unaffiliated/shesek] has quit [Remote host closed the connection]     
03:51 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has quit [Remote host closed the connection]     
03:52 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has joined ##miniscript     
05:43 -!- shesek [~shesek@164.90.217.137] has joined ##miniscript     
05:43 -!- shesek [~shesek@164.90.217.137] has quit [Changing host]     
05:43 -!- shesek [~shesek@unaffiliated/shesek] has joined ##miniscript     
08:27 < andytoshi> nice!!
08:46 < andytoshi> lol justinmoon i think we're gonna wind up with different sorts in rust-miniscript, rust-bitcoin and rust-secp
08:46 < andytoshi> because there's opposition to using this weird sort-by-serialization in rust-bitcoin, but we need it in rust-miniscript for sortedmulti compat
10:09 -!- shesek [~shesek@unaffiliated/shesek] has quit [Ping timeout: 264 seconds]     
11:43 < jeremyrubin> yo yo yo
11:44 < jeremyrubin> hope everyone has had a nice thanksgiving :)
11:45 < jeremyrubin> I was hacking on better support for CTV in Miniscript over the holiday and had some conceptual design questions I wanted to put out before I get too far down the rabbit hole
11:46 < jeremyrubin> So now that Miniscript/Policy does more thorough timing analysis, and OP_CTV has a fixed-but-unknown locktime/sequence requirement, I wanted to see what people think about some of the diff ways of making it compatible/analyzable
11:48 < jeremyrubin> To describe the issue more exactly:
11:49 < jeremyrubin> Previously, I had introduced (in my own fork/branch) CTV patches which add a Policy::TxTemplate(Hash) which compiles to Miniscript::TxTemplate(Hash) which compiles to <H> CTV DROP which extracts to Miniscript::TxTemplate(Hash) and to Semantic::TxTemplate(Hash) with appropriate type signatures throughout
11:50 < jeremyrubin> However, the script representation <H> CTV DROP is incompatible with analyzing H for locktime/sequence constraints
11:51 < jeremyrubin> This gives a few options:
11:52 < jeremyrubin> 1) Disallow CTV + Any Older/After clause. This is "simple", but not a neccessary restriction as if the Older/After clauses are compatible with the CTV Hash it is still spendable.
11:53 < jeremyrubin> 2) Embed Metadata in the script. E.g., the simplest form would be <template> DROP <h(template)> CTV DROP
11:54 < jeremyrubin> then the extractor can verify the mapping from template to hash and all is good. The downside is that template might be e.g. the entire txn. There are some optimized ways of doing this that would still permit analysis for *just seuqence/locktime* without unreasonable overhead.
11:55 < jeremyrubin> 3) Make timing analysis *require* a mapping of known Template hashes for spendability. At face, this seems unreasonable, unless you consider that if you don't know the full template anyways you can't spend that way anyways. So it's not as bad as it seems.
11:56 < jeremyrubin> This sort of "added context to validate" seems important anyways when you're analyzing a script for e.g. IsMine properties, you need to provide your own list of keys and stuff.
11:57 < sipa> i mean... descriptors (and miniscript) are effectively a way of storing all context information in a string
11:58 < sipa> the split-out format where the context is more nebulous is pretty much PSBT
11:58 < andytoshi> right, typically you can't have a descriptor with just a hash in it and nothing else
11:58 < andytoshi> core has `address` and `raw` "descriptors" but these are more like special inputs to scantxoutset than they are descriptors
11:58 < sipa> the point is having a way of conveniently packing all that information together
11:58 < jeremyrubin> Yeah, the issue is more the reverse direction script to Miniscript
11:59 < sipa> that too requires context
11:59 < sipa> e.g. BIP32 paths and redeemscripts
11:59 < jeremyrubin> So it's reasonable to e.g. provide a translation table of H(tmpl) -> tmpl?
11:59 < andytoshi> yeah
12:00 < jeremyrubin> One of the interesting things that occurs to me with taproot btw
12:00 < sipa> that seems like a conceptual match with what descriptors are already doing
12:00 < jeremyrubin> is if wallets store all the script paths then it's easier to make these "fully self describing scripts"
12:00 < sipa> but it implies you'll have the full spending descendant transaction inside the descriptors...
12:01 < jeremyrubin> sipa: yep, that's where I was a bit skeptical, although that seems something you might already want, e.g., if a key will only sign with a known business logic policy you'd want that in the descriptor anyways
12:02 < jeremyrubin> CTV just emulates a trusted business logic executor
12:02 < andytoshi> jeremyrubin: so, raw miniscript (like what you'd get from converting from script) is really only useful for (a) blockchain analysis, (b) giving a friendlier human-readable script output than decodescript rpc
12:02 < andytoshi> wallets can't do muchwith them
12:03 < andytoshi> for wallets you really need descriptors, and there is not really any way to map from script (or even a full tx to a descriptor
12:03 < andytoshi> you will always be missing key preimages, and keyorigin info...with sortedmulti you lose key ordering...with taproot you'll lose whole chunks of your script tree
12:03 < andytoshi> and CTV to me sounds like a descriptor
12:04 < jeremyrubin> Do we have a concept of a key descriptor v.s. a script descriptor?
12:04 < andytoshi> maybe the template can be a miniscript
12:04 < andytoshi> no, they're all just descriptors ... i guess pk/pkh are "key descriptors" and sh/wsh would be "script descriptors" ?
12:05 < sipa> they're output descriptors
12:05 < sipa> they describe a scriptPubKey (or set thereof), ultimately
12:05 < sipa> together with information about how to spend it
12:06 < jeremyrubin> got it. I mean it might just be convenient to make a CTV descriptor type which just stores a template descriptor. and then scripts just store <H> CTV and you lookup <H> in your descriptor set when you see it. This concept would extend to regular hashed keys too
12:06 < sipa> there are "key expressions" inside descriptors (pubkey, xpubs, derivation paths, origin info, ...) and "script expressions" (pkh, wpkh, sh, wsh, ...)
12:06 < andytoshi> jeremyrubin: i started out trying to work with miniscript directly, and increasingly have had to admit that it's not that useful by itself.....and you can see this in the evolution of rust-miniscript
12:06 < sipa> but the top level is always defining a script (which matches the expected scriptPubKey)
12:06 < andytoshi> where the _Miniscript type is increasingly unergonomic and requires extra context
12:07 < andytoshi> and new stuff gets added to Descriptor
12:07 < jeremyrubin> Yeah, I did make an effort for the whole self describing thing so a semantic policy can be extracted, but it ends up seeming not super useful overall v.s. having the original concrete 
12:07 < andytoshi> and right - you can sorta look at how pkh (or wsh) works with the API. that's probably what you want to model ctv on
12:08 < jeremyrubin> haha I wish rust had some sort of GADT Map where you could have a dependently typed lookup
12:09 < jeremyrubin> but i can add a separate map for CTV context
12:09 < andytoshi> yeah me too. you can sorta emulate it with Box<Any>
12:09 < jeremyrubin> or just enum wrap and use smart constructors
12:09 < andytoshi> right yeah
12:10 < andytoshi> i've done that, but its hard to get it to work ergonomically the way you want
12:10 < sipa> i'm sure lots of useful haskell features will eventually make its way to rust
12:10 < andytoshi> and there are dumb limitations on your ability to upcast/downcast, and to box multiple traits at once ... those should eventually go away
12:10 < jeremyrubin> min_const_generic is supposed to be in 4 weeks
12:11 < andytoshi> yeah, rust is definitely getting better quickly
12:11 < andytoshi> sadly we're stuck on this old version (1.29) that we can build from C
12:11 < andytoshi> for all the rust-bitcoin stuff
12:13 < jeremyrubin> Ah is bootstrapping past 1.29 broken or just complicated?
12:14 < andytoshi> just complicated
12:14 < jeremyrubin> (anyways -- to put a cap on the prior questions, thanks for your input, I'll abandon my self-describing from script for Semantic Policy extraction work and just add context for reversing script)
12:14 < andytoshi> each compiler version is only guaranteed to be buildable by the previous version
12:14 < andytoshi> so you have to go one by one
12:15 < andytoshi> starting from 1.29, which can be built by mrustc
12:15 < andytoshi> 1.30 onward use new stuff that mrustc doesn't supportyet
12:15 < andytoshi> jeremyrubin: good call
12:15 < jeremyrubin> or figure out the right skip-list of buildable -- is it really that each version breaks the next :scream:
12:16 < sipa> andytoshi: hmm, but you build using rustc, not mrustc, so you already have two steps?
12:16 < sipa> or are you actually building stuff with mrustc itself?
12:16 < andytoshi> sipa: yeah, though building rustc you only have to do once
12:16 < andytoshi> but yes, 2 steps
12:16 < andytoshi> so in principle going from 2 steps to 30 is not that much of a leap
12:17 < andytoshi> given that the first N-1 you only have to do once
12:17 < sipa> andytoshi: right, but so there isn't anything fundamentally harder about using 1.30 if you wanted, with 3 steps instead of 2
12:17 < andytoshi> yep exactly
12:17 < sipa> right
13:23 -!- jonatack [~jon@82.102.27.171] has quit [Ping timeout: 256 seconds]     
13:24 -!- jonatack [~jon@82.102.27.171] has joined ##miniscript     
13:25 -!- justinmoon [~quassel@157.245.122.126] has quit [Ping timeout: 256 seconds]     
13:26 -!- justinmoon [~quassel@157.245.122.126] has joined ##miniscript     
15:22 < andytoshi> sanket1729_: it would be neat to have a miniscript "diff" tool
15:22 < andytoshi> like imagine i change one branch of a miniscript, would it be possible to efficiently identify that
15:24 < andytoshi> yeah i think this is well-defined and straightforward. you just recursively check equality starting from the root
15:25 < andytoshi> and whenever some node is different (i.e. is a different terminal/fragment) is different you say "this whole tree is different" without trying to reason about the children at all
15:35 < andytoshi> i wonder what a good API is for this. similarly if there's a good API for giving a descriptor to the interpreter and highlighting which branches were taken/not taken
15:35 < andytoshi> this would be easier with a weakly typed language i guess where we could tack arbirtary data onto every node
15:36 < andytoshi> i guess we can make such a thing with some Box<Any> magic. will think about tihs
15:53 < darosior> sanket1729_: what do the new check_[global | local]_[consensus | policy]_validity methods refer to as being "global" or "local" ?
15:53 < darosior> I at first thought it was regarding if the limitations are satisfaction-related but it does not seem to be this
16:00 < darosior> Ok, self-answered :) It was that but reversed. global == satisfaction independent, local == satisfaction dependent
16:04 < andytoshi> local means it's about a particular branch
16:04 < andytoshi> global means it's about the whole script
16:15 < darosior> Thanks
18:21 < jeremyrubin> andytoshi: it might be simplest to just literally diff the string serializations if it's for a human readable output
20:59 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection]     
20:59 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined ##miniscript