--- Day changed Thu Sep 24 2020 01:23 -!- jonatack [~jon@82.102.27.171] has quit [Ping timeout: 256 seconds] 02:02 -!- jonatack [~jon@213.152.161.211] has joined ##miniscript 02:03 -!- jonatack [~jon@213.152.161.211] has quit [Read error: Connection reset by peer] 02:08 -!- jonatack [~jon@37.166.17.41] has joined ##miniscript 02:58 -!- justinmoon [~quassel@157.245.122.126] has quit [Quit: No Ping reply in 180 seconds.] 03:00 -!- justinmoon [~quassel@157.245.122.126] has joined ##miniscript 03:59 -!- jonatack [~jon@37.166.17.41] has quit [Read error: Connection reset by peer] 04:49 -!- jonatack [~jon@37.166.201.89] has joined ##miniscript 05:33 -!- jonatack [~jon@37.166.201.89] has quit [Read error: Connection reset by peer] 07:42 -!- harding [quassel@2600:3c03::f03c:91ff:fe7b:78d1] has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.] 07:43 -!- harding [quassel@2600:3c03::f03c:91ff:fe7b:78d1] has joined ##miniscript 10:32 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined ##miniscript 13:57 < sgeisler> sipa: you had a discussion with andytoshi some time ago about how wildcard descriptors work at the moment. While working on rust-miniscript descriptors I made a conscious decision to deviate from Core's current behavior re key derivation to support more use cases and I'd like to discuss if you think there might be a way to include the same features in core when adding miniscript descriptor support. 13:57 < sgeisler> https://github.com/rust-bitcoin/rust-miniscript/pull/131#discussion_r491191615 13:57 < sgeisler> The gist of the change is that I redefined * to mean 'any application-specified derivation (of unknown depth)' instead of 'all unhardened derivations in application-defined range (depth+1)'. In a wallet setting this allows to use the same descriptor for both the internal and the external wallet by deriving child descriptors with m/0/{0..x} and m/1/{0..x}. 13:57 < sgeisler> The only drawback to exposing a similar API in Core (e.g. in scantxoutset) is that one had to specify the range of descriptors to derive for every depth, but that should be possible (especially since adding miniscript descriptors would probably be a breaking change anyway). The range member of scanobject would become a list of ranges instead of a single range. 14:01 < sipa> sgeisler: i'm sympathetic the idea of being able to share (parts of) descriptors between different use cases, but i'm not sure that should be part of the descriptors themselves or something on top 14:02 < sipa> i think of descriptors as an abstract representation of a list of scriptPubKeys (+ associated metadata), not something that has more structure, as it becomes unclear how gap limits etc work if things aren't just a list 14:04 < sipa> i guess part of the question is just where do you draw the line between "this is part of descriptors' own semantics" and "this is part of what the application on top does with it" 14:06 < sipa> sgeisler: and i don't see why miniscript support (or anything) would be a breaking change to any RPC 14:06 < sipa> regarding descriptors... in fact descriptors are exactly why those things can be added without breaking things 14:07 < sgeisler> Oh, I didn't think that all the keywords were compatible enough to not be a breaking change, happy to stand corrected. 14:08 < sgeisler> I guess my proposal would remove semantic meaning from descriptors and make it a responsibility of the application. That could put a certain pressure on core to have a more general API when using descriptors for querying (not necessarily when using them for wallet stuff). 14:10 < sgeisler> The gap limit problem could certainly be solved by requiring one for each derivation depth. 14:10 < sipa> that seems like a serious complication from a usability standpoint 14:13 < sgeisler> I mean, normally you only want it for the final derivation anyway. All preceding ones should be well known. So e.g. `[{from: 0, to: 1}, {from: 0, gap_limit: 10}]` could be the derivation description to get all outputs from some wallet descriptor with a gap limit of 10. 14:13 < sgeisler> (afaik gap limit isn't supported in scantxoutset anyway at the moment) 14:14 < sipa> gaps don't make sense for anything but the final step 14:14 < sipa> as you can't automatically derive intermediary steps anyway 14:14 < sgeisler> I mean they kinda could if you want to find out which accounts were used 14:15 < sipa> but every leaf could have a different one 14:15 < sipa> and you'd need a way to enumerate all the exact intermediary paths 14:15 < sgeisler> yeah, you'd have to choose an upper bound 14:15 < sipa> this sounds horrible 14:15 < sipa> we shouldn't encourage people to do wildcarding on intermediary paths 14:15 < sipa> as each path should have exactly one meaning 14:18 < sipa> what i mean is that a wallet would right now be defined as e.g. {"change": , "payments": }, and you get the unfortunate situation that in many (but not all), descriptor1 and descriptor2 will share a significant part of their expressions 14:18 < sgeisler> My primary use case for this feature is describing wallets anyway, not querying. So this limitation seems sensible (I always want to work with exactly one wallet, but e.g. query both internal and external UTXOs). 14:18 < sipa> it seems useful to have instead something where you can have {"base_descriptor": , "change": , "payments": } 14:19 < sipa> but individually, change and payments are still "1-dimensional" expressions 14:20 < sipa> and i worry about creating an interface where it becomes possible/attractive to just define one descriptor and only support an automatic base+"/0/*" is change, base+"/1/*" is payments, for example 14:20 < sipa> as now you can't just take one descriptor and move it to some other context anymore 14:21 < sipa> you need to keep the semantics "change" and "payments" associated with specific subexpressions 14:23 < sgeisler> I see, so we possibly want to allow something like `wpkh(xpub…123/{i:0;o:1}/*)` so that the internal/external chain stays a property of the descriptor and no the wallet software. 14:24 < sgeisler> s/o:/e:/ 14:24 < sipa> but now you're requiring everything that deals with descriptors to understand the concept of internal/external 14:24 < sipa> which doesn't make sense for rescanning eg 14:25 < sipa> for scantxoutset, i mean 14:25 < sipa> the current interface means all of that can stay on the application side of things 14:25 < sipa> but the downside is duplication 14:25 < sgeisler> to some degree it makes sense for scantxoutset to know there are two chains to scan 14:25 < sipa> sure, give it both descriptors 14:25 < sipa> it supports that :) 14:27 < sgeisler> I really dislike the duplication for wallets. You really don't need different descriptors for internal and external scripts, so it's just a possible source of problems to have this redundancy. 14:27 < sipa> yes, agree 14:28 < sipa> but i really dislike making descriptors something so abstract that they don't correspond to actual lists of scriptPubKeys anymore 14:29 < sipa> or putting knowledge about how they're going to be used in them 14:30 < sipa> one thing that was suggested before, but i'm not sure what happened to it (or where the discussion happened...), was something that you could have a descriptor like {"desc1": "sh(wpkh(xpub/.../$CHANGE/*))"}, and then set your wallet to use {"payments": ["desc1", "CHANGE=0"], "change": ["desc1", "CHANGE=1"]} or something like that 14:31 < sgeisler> Like you said, an abstraction on top might work, let's call them wallet descriptors: `{"desc": "wpkh(xpub…123/*)", internal: "0", "external": 1, gap_limit: 10}` but this still requires to loosen the semantics of `*` 14:32 < sgeisler> That doesn't sound simple at all 14:32 < sipa> agree 14:32 < sipa> but it avoids the reuse 14:32 < sipa> *duplication 14:33 < sipa> i personally don't care about the duplication... it's something that's done automatically once when you define the wallet, and users generally won't interact with it 14:34 < sipa> or it could be something that's done by expansion from some higher-level concept, say something zpub-like (but sane) 14:34 < sipa> and it lets you export/import things wherever easily without breaking things, leaving checksums in place, ... 14:35 < sipa> and doesn't force every applications that supports descriptors to deal with arbitrarily-complex tweaks to it 14:40 < sgeisler> Even if you allowed arbitrary derivations whenever * occurs at the end of a derivation path that wouldn't require most software to handle such arbitrary derivations. Wallets know what to do, use 0 for external, 1 for internal. When querying: let the user define what to do. When doing something else: you better write a BIP for interoperability, but the descriptor doesn't limit you. 14:40 < sgeisler> Sure a wallet descriptor would only work in wallets, or with the right query arguments but is that a problem? 14:41 < sipa> GAH 14:41 < sipa> i couodn't more strongly disagree 14:42 < sipa> you don't need bips for interoperability the way descriptors work now :( 14:43 < sipa> they're just a thing that expands to a list of scriptPubKeys 14:43 < sipa> import and export it wherever you want 14:44 < sipa> there shouldn't "wallet descriptors" and other descriptors that each have additional semantics imbued into them 14:48 < sgeisler> I guess I'll have to build wallet specific things on top then. I hoped you wouldn't feel that strongly about it, but I can guess it comes from fighting incompatibility for a long time. I'll try to come up with a solution that leaves *-descriptor semantic untouched. 14:48 < sipa> what if you want a multisig that consists of a combination of xpub expressions, but one comes from an application that doesn't use the {0,1}/* convention for internal/external 14:49 < sipa> and another one does 14:50 < sipa> you could say "that's not allowed", but now we're making additional semantics associated with them, and i think those are just unnecessary 14:50 < sgeisler> I would have hoped that nobody does that as all BIPs I know about derivation paths clearly differentialte between internal and external :( 14:50 < sipa> right... but where is the line. 14:51 < sipa> what part of "established behavior" is actually becoming implicitly frozen by making it part of the language 14:51 < sipa> it just feels so much cleaner to have descriptors that are just lists of scriptPubKeys and nothing else 14:52 < sipa> i wishi could convince you that that is valuable on its own 14:55 < sgeisler> I can see that it's theoretically more beautiful and maybe practically better in the long run by not freezing behavior. I just wanted to save a layer of abstraction and redundancy on top in the now, but it probably is cleaner not to do so. 14:56 < sgeisler> So thank you for explaining your point of view so patiently :) 14:57 < sipa> i mean... i recognize the desire to avoid the redundancy 14:57 < sipa> but it seems every solution has downsides and risks of incompatibilities 14:57 < sipa> i also wish i had more time myself to work on this and help pushing descriptors and miniscript forward as something interoperable with some level of standardization 14:58 < sipa> and i'm frustrated that i can't find the time for being involved enough 15:23 -!- shesek [~shesek@164.90.217.137] has joined ##miniscript 15:23 -!- shesek [~shesek@164.90.217.137] has quit [Changing host] 15:23 -!- shesek [~shesek@unaffiliated/shesek] has joined ##miniscript 18:21 -!- instagibbs [~instagibb@pool-71-178-191-230.washdc.fios.verizon.net] has quit [Quit: ZNC 1.7.4+deb7 - https://znc.in] 18:22 -!- instagibbs [~instagibb@pool-71-178-191-230.washdc.fios.verizon.net] has joined ##miniscript 21:37 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Quit: jonatack]