--- Log opened Wed Sep 30 00:00:33 2020 02:18 -!- shesek [~shesek@unaffiliated/shesek] has quit [Remote host closed the connection] 03:18 -!- Ariel64Weber [~Ariel64We@static.57.1.216.95.clients.your-server.de] has joined #rust-bitcoin 03:38 -!- midnight [~midnight@unaffiliated/midnightmagic] has quit [Ping timeout: 272 seconds] 03:43 -!- midnight [~midnight@unaffiliated/midnightmagic] has joined #rust-bitcoin 05:53 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Quit: jonatack] 05:57 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #rust-bitcoin 07:34 < andytoshi> review beg for https://github.com/rust-bitcoin/rust-bitcoin/pull/478 07:34 < andytoshi> BlueMatt: lol yeah, i am aware of that 07:34 < andytoshi> it's pretty nuts 08:33 < BlueMatt> yeesh 10:15 -!- Ariel64Weber [~Ariel64We@static.57.1.216.95.clients.your-server.de] has quit [Remote host closed the connection] 10:19 -!- Ruthie27Bartell [~Ruthie27B@static.57.1.216.95.clients.your-server.de] has joined #rust-bitcoin 11:38 < BlueMatt> why does anyone use associated types instead of explicit trait bounds with <>? explicit bounds are much more flexible, and associated types just blow up in your face twenty layers up later :( 11:40 < BlueMatt> what a shame, too, cause they read nicer, but they're hobbled and blow up with nonsense error messages :( 13:03 < gwillen> there are certain specific cases where associated types are required, but frankly I don't remember when or why 13:04 < BlueMatt> huh, news to me. i mean i get why the rustc folks want to push them, they require only a single copy of the impl thing to be generated instead of generic-izing it and ending up with 20 like C++ templates, but that results in a ton of arbitrary-seeming restrictions and inscruitable error messages :( 13:18 < sgeisler> andytoshi: #478 looks okish, there's this one macro `impl_psbt_insert_hash_pair` which I don't fully understand: why is this a macro and not a generic fn? https://github.com/rust-bitcoin/rust-bitcoin/pull/478#discussion_r497772695 13:18 < sgeisler> It unneccesarily 13:18 < sgeisler> obscures things imo 13:19 < sgeisler> if that's really intentional I'd ACK, the remaining remarks are minor 13:41 < sanket1729> sgeisler: Thanks for the review. I agree that we should be using generic functions over macros. 13:42 < sgeisler> Can you ping me again then please? If I'm still awake I'll re-review then 13:42 < sanket1729> sure. Thanks 14:06 < andytoshi> nice, good catch 14:06 < andytoshi> BlueMatt: i don't see how associated types and trait bounds are equivalent at all 14:07 < andytoshi> the language was extremely hard to use before associated types. in particular iterators were shit 14:07 < BlueMatt> right, they arent equivalent, but at first pass they seem like it, i mean they do accomplish a similar goal. 14:07 < BlueMatt> but associated types are way more restrictive than trait bounds, afaict? 14:08 < andytoshi> i don't think you can order them by restrictiveness 14:08 < BlueMatt> maybe rustc is just much better at figuring out the implied type today than it was back then? 14:08 < andytoshi> and i'm having trouble conceptualizing either in terms of the other 14:09 < BlueMatt> huh, maybe there's some part of them I'm not groking, but basically istm that both basically define a concretization of a given trait by declaring what the concrete types are of some otherwise-floating type. 14:09 < andytoshi> like, what is the trait-bound-based replacement for Iterator::Item 14:10 < BlueMatt> oh you mean because its an unconstrained generic? 14:10 < andytoshi> ah, you have an Iterator and then require the next() method return an I 14:10 < andytoshi> ok then how do you implment that 14:10 < BlueMatt> right, I suppose basically the language would have to allow "unconstrained" (in today's world) generics, and then have functions be allowed to use it 14:10 < andytoshi> you need to restrict I in your implementation, usually to a concrete type 14:10 < andytoshi> but then the compiler has no way to infor which concrete type should be used by somebody who wants an iterator 14:10 < BlueMatt> right, I suppose the issue with generics is rustc forces you to constrain them. 14:11 < andytoshi> it's not rustc, it's logic 14:11 < BlueMatt> right, but it shouldnt need to - i mean it cant on associated types either 14:11 < BlueMatt> C++ does it just fine :) 14:11 < andytoshi> yeah by using text substitution and defering type checking until after instantiation 14:11 < andytoshi> which has insane consequences 14:11 < andytoshi> in particular you can instantiate templates in completely broken ways 14:11 < BlueMatt> right, i mean or you can do java-style and say X : Object 14:12 < BlueMatt> as a declaration of "I dont care what type this is, its just gotta be a concrete thing" 14:12 < andytoshi> what if i want a function which takes an iterator over anything that impls Into<[u8]> say 14:12 < BlueMatt> which is basically what an associated type is 14:12 < BlueMatt> right, you'd have to `dyn` it 14:12 < andytoshi> lol Object throws away almost all of the type system 14:12 < BlueMatt> right, well in this case you would impl with some concrete X 14:12 < andytoshi> ok but then how do i use that 14:12 < BlueMatt> resulting in the type system working again, just at impl-time, not decl-time 14:13 < BlueMatt> which, again, is what associated types do 14:13 < andytoshi> i don't know what impl-time and decl-time are supposed to mean here 14:13 < BlueMatt> i mean I guess I dont understand what the issue is with taking anywhere where associated types are used and using for it 14:13 < andytoshi> and again, if i have a function `use_iterator>()` this is impossible for me to describe without associated types 14:14 < andytoshi> that's one issue 14:14 < BlueMatt> > 14:14 < BlueMatt> or where X: Into<[u8]>> 14:14 < andytoshi> lol that's terrible, how do i define such an iterator 14:14 < BlueMatt> i mean both of those would be completely tractible for a compiler to monomorphize and then generate code/errors for? 14:15 < BlueMatt> after all, that is what rustc does with associated types 14:15 < BlueMatt> just different syntax 14:15 < andytoshi> in the first case you can't actually define iterators 14:15 < BlueMatt> huh? I dont see what you mean by "can't actually define" 14:15 < andytoshi> you'd have to `impl Iterator> for MyIterator { ... }` and then repeat that for every single trait you might imagine users would want the actual item for MyIterator to impl 14:16 < andytoshi> instead of just doing `impl Iterator> for MyIterator` and having that work for every trait that Vec implements 14:16 < BlueMatt> ah, right, sure, if you make it concrete, but the generic bound would also work. 14:16 < andytoshi> https://github.com/rust-lang/rfcs/blob/master/text/1598-generic_associated_types.md 14:16 < BlueMatt> which is more akin to what compiler does today 14:17 < andytoshi> oh this is the wrong rfc 14:17 < BlueMatt> right was about to say lol 14:18 < andytoshi> fucking duckduckgo 14:18 < BlueMatt> though I'll note thats basically moving associated types closer and closer to where X: Into<[u8]>> 14:18 < BlueMatt> cause, ultimately, the compiler for both does a very, very similar task, and both are hobbled somewhat needlessly to have different logic for each. 14:18 < andytoshi> these claims are untrue 14:19 < BlueMatt> maybe I failed to parse your earlier argument, then. 14:19 < andytoshi> well now i'm rabbit-holed reading old type theory stuff lol 14:19 < andytoshi> at the time associated types were added this was all fresh in my mind 14:19 < andytoshi> because the language was seriously unusable before them 14:19 < BlueMatt> lol. right, I'm not arguing in a type-theory level, I do agree they are somewhat different in that world 14:20 < BlueMatt> however, I'm arguing that you could define, practically, a superset of both associated and generics and have something that is very tractible to implement but doesn't have the restrictions that either have. 14:21 < andytoshi> i would be extremely suprised 14:21 < andytoshi> as would niko who spent several years defining the existing trait system 14:21 < andytoshi> cf 14:21 < andytoshi> http://smallcultfollowing.com/babysteps/blog/2017/05/25/query-structure-in-chalk/ 14:21 < andytoshi> oh http://smallcultfollowing.com/babysteps/blog/2017/01/26/lowering-rust-traits-to-logic/ is the better blog post 14:22 < BlueMatt> i dunno, C++ does it? 14:22 < andytoshi> it really doesn't 14:22 < BlueMatt> though its possible you end up hitting some diamond pattern inheritence nonsense which results in chasing edge cases somewhat 14:22 < BlueMatt> instead of something neat that doesnt, but ends up being hard to use in some edge cases. 14:22 < andytoshi> you don't need diamond inheritance for C++ templates to result in a giant mess 14:22 < andytoshi> and you can't do the kind of trait bounding you can in Rust in C++ 14:23 < BlueMatt> specifically, the "unconstrained parameter" restriction 14:23 < andytoshi> https://stackoverflow.com/questions/56045846/what-are-the-similarities-and-differences-between-cs-concepts-and-rusts-trai 14:24 < andytoshi> C++ lets you instantiate any template with any parameter, and then checks after instantiation whether the result makes sense 14:24 < andytoshi> so you can't do anything resembling trait bounds 14:24 < andytoshi> until concepts 14:25 < andytoshi> which also have serious limitations, cf that SE answer 14:25 < andytoshi> you can get this in Rust BTW by using macro_rules! instead of type parameters 14:26 < andytoshi> if you step outside macro_rules and define your own syntax extension you will even get the full power of C++ templates 14:27 < BlueMatt> C++ lets you instantiate any template with any parameter, and then checks after instantiation whether the result makes sense <-- right, thats basically what I'm suggesting, except you could practically limit it in the sense that you do associated types whereby you cant write code in the object itself to do anything with the type except return it. 14:28 < BlueMatt> I'm trying to dig it up, but the limitations here end up with some fun long blog posts that I stumbled on googling errors where people just moan and moan about how stupid the rust compiler is 14:28 < andytoshi> i'm very interested to see this 14:29 < andytoshi> if the answer is to stop using the type system and try to blindly substitute parameters ... this is possible in rust, but nobody ever does it :) 14:29 < andytoshi> without knowing the kinds of problems you're running into it's hard to say anything helpful tho 14:29 < BlueMatt> see-also https://github.com/rust-bitcoin/rust-lightning/pull/681#discussion_r494532289 14:30 < BlueMatt> where val set a trait with a associated type, resulting in an awkward dance and PhantomData in something in another crate that impl'd it 14:30 < BlueMatt> cause, like, its not really clear that the right answer was to make the trait a template instead of a trait with an associated type - they seem like similar things at first glance (and kinda are, at least from a resulting-llvm-code perspecitve, though not a type-theory perspective) 14:31 < andytoshi> this may be an issue where what you "really" want is higher-kinded types 14:31 < BlueMatt> I buy that there is a type-theory answer which I dont know anything about :) 14:32 < andytoshi> but lemme take a few minutes to understand what this type parameter is suposed to me 14:32 < andytoshi> mean 14:32 < BlueMatt> that is separate from my "but these two things that are both limited in different ways end up with the same llvm ir and you could write a compiler which merges the two to still end up with the same llvm ir without confusing people who dont know a lot of type theory" 14:33 < gwillen> BlueMatt: for that I think the best answer is the description I found when I was refreshing my memory on associated types earlier today, after reading your comments 14:33 < gwillen> which is "a template parameter is an input; an associated type is an output" 14:33 < gwillen> although of course the whole concept of input vs output is a little fuzzy here 14:33 < andytoshi> BlueMatt: the C++ strategy is to basically not use type theory and then your language has a much less expressive type system 14:33 < andytoshi> and again, you can do this in rust, with syntax extensions 14:34 < andytoshi> but you will quickly regret this :P 14:34 < BlueMatt> heh, right, "can do by writing your own language that compiles to rust" isnt a great answer :) 14:34 < gwillen> but like, when you instantiate a template you can make a type parameter be whatever you want, but for any given instantiation there is always exactly one possible type for each associated type 14:34 < andytoshi> C++ templates are "your own language that compiles to C++" 14:34 < BlueMatt> gwillen: thanks, that is actually helpful, though I'm now just bitching about rust and not being productive :) 14:34 < gwillen> heh, np :-) 14:35 < BlueMatt> andytoshi: i mean....yes, but at least only marginally. 14:35 < gwillen> C++ templates are a goddamn nightmare, and if you want evidence I have a compile-time lisp interpreter I can show you :-P 14:35 < andytoshi> BlueMatt: i really don't understand why this phantomdata is there or why that type parameter is on that type 14:36 < andytoshi> ah is it for the impl of channelmonitor::Persist 14:36 < gwillen> sometimes the answer to these things is "the type theory doesn't actually require it, but the current compiler implementation can't figure that out" 14:36 < andytoshi> which has an associated typpe that they are trying to be generic over 14:36 < BlueMatt> andytoshi: basically it was the result of a trait which used an associated type (with constraints) to declare the type of another trait which had a generic across that type. 14:37 < andytoshi> yeah ok, so what they did makes perfect sense, they parameterized the type with the associated type 14:37 < andytoshi> i'm not sure how else this ought to work? 14:37 < andytoshi> if i want to use their struct as a channelmonitor::Persist then how do i specify what the associated type is? 14:38 < andytoshi> the solution as implemented is to specify it as a parameter on FilesystemPersister, which makes sense 14:38 < BlueMatt> andytoshi: I guess my concrete complaint is, eg, imagine an extension to rust's generics system whereby you can have a variable which is a generic parameter in a trait but has no constraints - eg, I accept a variable of this type, and then maybe store it, and then maybe return it. the compiler doesn't need to check anything at compile time cause all it needs to know is the size of the variable (ok, so maybe the one constraint is Sized). 14:38 < andytoshi> o.O types are characterized by far far far more than their size 14:38 < BlueMatt> yes, but not once they are converted to llvm ir :) 14:39 < andytoshi> i don't think we're anywhere close to speaking the same language here 14:39 < BlueMatt> i think we're just speaking two different languages 14:39 < andytoshi> i would like rust's type system to enforce type safety 14:39 < BlueMatt> lol, yes. 14:39 < andytoshi> lol jinx 14:39 < andytoshi> in any case .. the reason associated types can't be generic like this is that the compiler couldn't determine a concrete type for the associated type 14:39 < BlueMatt> i mean what i described is trivially safe - sure the compiler will enforce other things at the call site, cause at that point it'll monomorphize and concretize the type into something with more bounds 14:39 < andytoshi> even given a concrete type for the base type 14:40 < andytoshi> but this is exactly the purpose of associated types 14:40 < BlueMatt> but in the trait itself, the only thing that matters for safety is that it has a fixed size 14:40 < andytoshi> "into something" 14:40 < andytoshi> if the only thing that matters for safety is the size then you should be using mem::transmute and accepting byte vectors 14:41 < BlueMatt> the call-site will have a ton more required for safety, of course 14:41 < BlueMatt> but, a trait that just has a single fn return_arg(x: X) -> X { x } is pretty trivially safe ;) 14:41 < gwillen> once you do that you are going to have the C++ template problem that your template expansion can be valid until you try to use it 14:41 < gwillen> like, you can make an object of this type, no sweat, then you try to call a method on it and it doesn't compile 14:41 < BlueMatt> not if you require no access to x 14:42 < BlueMatt> i mean the issue with C++ is you can do x.do_things() when the compiler doesnt have a type for x 14:42 < BlueMatt> but it would be totally reasonable for rust to not allow that (in fact, it doesnt, for a reason, it needs to at least know the trait for x to let you call a function on x) 14:42 < andytoshi> i think i see what you're saying, and i think you need higher kinded types 14:42 < gwillen> like, Iterator::Item is an associated type, I can call methods on the type of an item I get from an iterator 14:43 < andytoshi> if you're literally doing nothing with X then you should specify X to be a Box 14:43 < andytoshi> or in this case a Box<(ChannelKeys + Readable + Writeable + Send + Sync)> 14:43 < gwillen> if you allow creating an iterator with the type Item being arbitrary, I can get an iterator instance that is perfectly good until I try to call a method on an item it returned, and then that call doesn't compile, yes? 14:43 < andytoshi> +Any, so you can downcast :P 14:43 < BlueMatt> gwillen: well in both C++ and rust and my rust-with-higher-kinded-types? you'd still monomorphize the thing that impl's iterator and then check it at the callsite, which is what everyhing does :) 14:44 < BlueMatt> andytoshi: heh, without boxes now so that its concrete instead of 20 layers of runtime indirection :) 14:44 < gwillen> in C++, if iterator items are required to have a .foo() method, everything will compile fine if you instantiate Iterator with an Item that doesn't, as long as you never call .foo() 14:44 < andytoshi> it's one layer of indirection 14:45 < BlueMatt> gwillen: right, well no one is going for C++'s nonsense, either :p 14:45 < andytoshi> i don't see how "check it at the callsite" differs from the nightmare that gwillen is describing 14:47 < BlueMatt> because in C++ you realize that you cant instantiate the class required because templates are just string replacement. you could, instead, require that all access to things is through a trait that implements the things that are done, when you read the block where that thing is done. ie all I'm suggesting is make it possible/practical to define an Iterator instead of Iterator - it doesn't imply that an impl Iterator for Z 14:47 < BlueMatt> { Y::do_meth() } would compile without bounding the Y on that impl block. 14:48 < BlueMatt> because associated types randomly blow up in your face in a way, eg in that thing that val's doing, that is mega-reminiscient of C++ 14:48 < BlueMatt> cause you are three impl-s away from the original trait and suddenly realize you cant do something because the compiler wants a concrete associated type but you *really* need to template it 14:48 < andytoshi> this isn't randomly blowing up, it makes perfect sense to me, i don't get where the confusion is. and i'm very curious what the compiler error was that led to this, and in what way that was confusing 14:48 < andytoshi> because i wouldn't have even needed a compiler error 14:49 < andytoshi> yes, if you want to template it then you add a type parameter 14:49 < andytoshi> which val did 14:50 < sanket1729> sgeisler, andytoshi: https://github.com/rust-bitcoin/rust-bitcoin/pull/478#discussion_r497819147 14:50 -!- Ruthie27Bartell [~Ruthie27B@static.57.1.216.95.clients.your-server.de] has quit [Ping timeout: 246 seconds] 14:51 < andytoshi> sanket1729: can you push some code to describe why you had to kill the enum? 14:53 < sanket1729> andytoshi: The enum had concrete types for hash160::Hash, sha256::Hash and so on it in 14:53 < sanket1729> I had to construct a variant for that enum from the generic function. 14:53 < sanket1729> settled up with this instead 14:54 < sanket1729> this line here: https://github.com/rust-bitcoin/rust-bitcoin/blob/a7f6d1010fd1573384e7e316d6457ff66bac5b2b/src/util/psbt/map/input.rs#L259 14:55 < andytoshi> oh lol just for the error? 14:56 < andytoshi> yeah converting to hex was probably the righ tcall 14:56 < sanket1729> That was a error type :) 14:56 < andytoshi> oh lol 14:56 < andytoshi> lol so i think you could do this with associated constants 14:57 < andytoshi> but imo your solution is just as good, and less esoteric 14:59 < sanket1729> I feel something wrong if you have information specifically for inferring a concrete type inside a trait. 14:59 < BlueMatt> hmm? no one is inferring anything? 14:59 < BlueMatt> the issue was something templated awkwardly on an associated type which templated a concrete other thing 15:00 < BlueMatt> andytoshi: ah, here's the blog post i referenced https://fasterthanli.me/articles/frustrated-its-not-you-its-rust 15:01 < andytoshi> thanks 15:02 < sgeisler> sanket1729: why convert to hex there instead of using a `Vec`? Additionally you could also save the type of hash as another field (enum or just the "map key constant", idk how you call them), that would give the error some more context. When serializing you could write `HashType(hex_hash)`. 15:06 < BlueMatt> IIRC at the time I was fighting with auto-generated rust code which did an impl Deref for X { fn deref(&self) { self } } (which is an incredibly useful pattern in general - templating something by Deref is great, and then eg in our C bindings the generated concrete trait is just a jump table, so no need for the extra indirection, just embed it instead of a reference) 15:07 < BlueMatt> but, rustc...is not a fan. i mean we do it anyway, but it results in some incredibly awkward compiler errors that are inscruitable to someone who doesn't know rust's type theory (and just knows concrete compiler outputs instead) 15:07 < BlueMatt> anyway 15:07 * BlueMatt -> back to work 15:10 < andytoshi> ok BlueMatt `error[E0225]: only auto traits can be used as additional traits in a trait object 15:11 < andytoshi> is rust-specific and it's really annoying and shitty 15:11 < BlueMatt> i dunno that that was the thing i was htiting, I dont think so, IIRC the actual error was "cannot be made into an object" about something 15:11 < BlueMatt> where i was tryint to tell the rust compiler exactly what the concrete type was, but it wouldnt let me 15:11 < andytoshi> but .. the obvious solution in this case (in the blog post you linked) is to just make an enum with the different return types of this function, because boxed traits are very-much second-class citizens in rust 15:12 < BlueMatt> this was several weeks ago 15:12 < andytoshi> lemme see if i can find a blog post about "cannot be made into an object" 15:12 < BlueMatt> its referenced a few times in that post 15:12 < andytoshi> this one is also rust-specific but there are better reasons for it, related to crate coherency 15:12 < BlueMatt> which is how i stumbled upon it...google 15:12 < andytoshi> ah kk 15:16 < sanket1729> sgeisler: Thanks for suggestions. indeed both of them sound better 15:22 < sanket1729> pushed again 15:27 -!- valwal_ [sid334773@gateway/web/irccloud.com/x-quzwnfdrpkglflcc] has joined #rust-bitcoin 16:39 -!- Dean_Guss [~dean@gateway/tor-sasl/deanguss] has joined #rust-bitcoin 16:41 -!- DeanGuss [~dean@gateway/tor-sasl/deanguss] has quit [Ping timeout: 240 seconds] 17:00 -!- Dean_Guss [~dean@gateway/tor-sasl/deanguss] has quit [Remote host closed the connection] 17:05 -!- DeanGuss [~dean@gateway/tor-sasl/deanguss] has joined #rust-bitcoin 22:09 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Ping timeout: 240 seconds] 22:21 -!- DeanGuss [~dean@gateway/tor-sasl/deanguss] has quit [Remote host closed the connection] 22:21 -!- DeanGuss [~dean@gateway/tor-sasl/deanguss] has joined #rust-bitcoin 23:23 -!- tibo [~tibo@2400:4050:2a83:7000:4821:e7a:d939:17b8] has joined #rust-bitcoin 23:54 -!- Netsplit *.net <-> *.split quits: fjahr, Aleru, felixweis 23:57 < tibo> Hi all, I wanted to know if it'd be possible to get a high level review of my PR to add Schnorr to rust-secp256k1. I'd really like to clarify what kind of types and interfaces people would like to have rather soon so that the code that I am building on top doesn't need heavy refactoring in the future. https://github.com/rust-bitcoin/rust-secp256k1/pull/237 --- Log closed Thu Oct 01 00:00:34 2020