--- Log opened Thu May 23 00:00:22 2019 00:27 -!- schmidty [~schmidty@unaffiliated/schmidty] has quit [Ping timeout: 250 seconds] 00:28 -!- schmidty [~schmidty@unaffiliated/schmidty] has joined #rust-bitcoin 00:35 -!- TamasBlummer1 [~Thunderbi@p200300DD67196B1130B55871BB7F48A7.dip0.t-ipconnect.de] has joined #rust-bitcoin 00:37 -!- TamasBlummer [~Thunderbi@p200300DD67196B9644FFE6639ADFCA33.dip0.t-ipconnect.de] has quit [Ping timeout: 264 seconds] 00:37 -!- TamasBlummer1 is now known as TamasBlummer 01:04 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has joined #rust-bitcoin 02:33 < stevenroose> RFC: I'm thinking of adding a "to_string_with_prefix" and "from_str_with_prefixes" to bip32 extended priv and pubkeys so that we can reuse the code with Liquid prefixes 02:34 < stevenroose> ah that won't work before we have bitcoin_constants, darn 03:15 < stevenroose> With all the Elements stuff (PSBT, addresses, now this), the bitcoin_constants stuff seems really useful.. 03:27 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has quit [Remote host closed the connection] 04:03 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has joined #rust-bitcoin 04:08 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has quit [Ping timeout: 248 seconds] 04:35 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 04:39 -!- willpiers [~willpiers@38.75.231.30] has quit [Ping timeout: 246 seconds] 04:47 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has joined #rust-bitcoin 04:53 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has quit [Remote host closed the connection] 05:07 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has joined #rust-bitcoin 05:45 -!- DeanWeen [~dean@gateway/tor-sasl/deanguss] has quit [Ping timeout: 256 seconds] 07:04 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 07:24 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has quit [Remote host closed the connection] 07:31 -!- willpiers [~willpiers@38.75.231.30] has quit [Remote host closed the connection] 07:34 -!- michaelsdunn1 [~michaelsd@38.126.31.226] has joined #rust-bitcoin 07:34 -!- michaelsdunn1 [~michaelsd@38.126.31.226] has quit [Changing host] 07:34 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has joined #rust-bitcoin 07:37 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 07:40 -!- willpiers [~willpiers@38.75.231.30] has quit [Remote host closed the connection] 07:54 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 08:00 -!- willpiers [~willpiers@38.75.231.30] has quit [Remote host closed the connection] 08:01 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 08:05 -!- willpiers [~willpiers@38.75.231.30] has quit [Ping timeout: 250 seconds] 08:17 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 08:23 -!- willpiers [~willpiers@38.75.231.30] has quit [Remote host closed the connection] 08:24 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 08:28 -!- willpiers [~willpiers@38.75.231.30] has quit [Ping timeout: 244 seconds] 08:31 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 09:07 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has quit [Remote host closed the connection] 09:12 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has joined #rust-bitcoin 09:30 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has quit [Remote host closed the connection] 09:32 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has joined #rust-bitcoin 09:51 -!- willpiers [~willpiers@38.75.231.30] has quit [Remote host closed the connection] 09:52 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 09:56 -!- willpiers [~willpiers@38.75.231.30] has quit [Ping timeout: 246 seconds] 10:02 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 10:20 < BlueMatt> andytoshi: you probably have interest in https://github.com/rust-bitcoin/rust-bitcoin/pull/264, if only because it seems to cut strongly against your vision for consensus/encode/decode, but I strongly disagree with it :p 10:22 < andytoshi> BlueMatt: cool, no, it's not "against my vision", but i was trusting the compiler to basically do this for us 10:22 < andytoshi> i think it's perfectly reasonable to privilege bytestrings in the encoding traits 10:22 < andytoshi> ah, killing the generic impl for [T; $len] might upset me though.. 10:22 < andytoshi> (and might be why i didn't do this in the first place) 10:23 < BlueMatt> andytoshi: indeed, and also removing a bunch of generic impls 10:23 < BlueMatt> andytoshi: my point being we shouldnt be exposing consensu::encode for user types 10:23 < andytoshi> yeah, brutal 10:23 < BlueMatt> personally, I'd vote very, very strongly in favor of making Encodable crate-final 10:23 < andytoshi> welll, we do this in liquid all the time for liquid-specific network messages 10:23 < BlueMatt> ie make it not possible to use for user types 10:23 < andytoshi> because serde is full of crazy shit, variable-sized types, unicode strings, etc 10:23 < BlueMatt> yes, and those shouldnt be bitcoin::consensus::encadable 10:23 < BlueMatt> if you want that you can have liquid::consensus::encodable 10:24 < andytoshi> hmmmmm 10:24 < andytoshi> in principle i agree with you 10:24 < BlueMatt> this is why things are versioned :p 10:24 < andytoshi> what about rust-elements? sholud it have its own encodable trait too? 10:24 < BlueMatt> if it has a reason not to use Writeable, sure? 10:25 < BlueMatt> I mean either way its not up to rust-bitcoin to support that 10:25 < andytoshi> what is Writeable 10:25 < BlueMatt> std::io:: 10:25 < andytoshi> i don't think there's been such a thing as std::io::Writeable for like 5 years :P 10:25 < andytoshi> Write is completely different 10:27 < BlueMatt> ah, right, well, then yes. 10:27 < BlueMatt> unless we want a rust-bitcoin-serializer 10:27 < BlueMatt> crate 10:27 < BlueMatt> I dont think it makes sense for rust-bitcoin to provide a flexible serialization api.... 10:29 < BlueMatt> not to mention the fact that that serialization API does something different for [u8; 32] than it does for [u8; 33] 10:29 < BlueMatt> which makes it...not usable outside of bitcoin consensus :p 10:29 < andytoshi> it does? 10:30 < BlueMatt> [u8; 32] => write the 32 bytes, [u8] -> VarInt + u8s 10:30 < andytoshi> yes, [u8; fixed] and [u8] are different 10:30 < BlueMatt> cause it gets munged into &[u8] 10:30 < andytoshi> bluh 10:30 < andytoshi> ok 10:31 < andytoshi> let me think about this. i'm really not looking forward to duplicating a shitload of code from rust-bitcoin in rust-elements and liquid 10:31 < BlueMatt> mmmm the Vec specific impl seems to have made a further huge improvement 10:32 < BlueMatt> now chugging 50Mbps in bitcoin data doesnt make my odroid-c2 sweat (too much) 10:32 < andytoshi> OTOH we really need these perf improvements for liquid, scanning blocks takes fscking forever 10:32 < andytoshi> multiple hours and we're just skimming them looking for fixed scriptpubkeys 10:32 < BlueMatt> ~20*4 cores% cpu usage instead of 100% pegged on all 4 10:33 < BlueMatt> yea, my dnsseed scanner was very very sad crawling the network and downloading a block from each peer 10:33 < BlueMatt> and 50Mbps isnt that much traffic 10:40 < andytoshi> BlueMatt: without exposing the traits how can users encode stuff into a Write? 10:41 < BlueMatt> hmm? oh I dont mean dont expose the traits, I mean make it impossible for a user to implement Encodable on their own trait 10:41 < BlueMatt> I have not done that in that pr 10:41 < BlueMatt> but it would be cool 10:41 < andytoshi> oh i see, just seal the trait 10:41 < BlueMatt> yes 10:41 < BlueMatt> sorry, forgot the term for it 10:41 < andytoshi> yeah, it's a weird term (and there's a weird mechanism to do it) 10:41 < andytoshi> making it depend on a non-exported trait iirc 10:41 < BlueMatt> yes, I recall a mechanism existing, not what it is :p 10:42 < BlueMatt> ah, right 10:45 < andytoshi> i wonder how many of read_{i,u}{8,16,32,64} we can get rid of.. 10:45 < BlueMatt> dont think it matters much? 10:45 < BlueMatt> you dont have to impl those, you can just impl Write/Read 10:45 < andytoshi> it matters to how much copy/pasting needs to be done by everybody using these traits 10:45 < BlueMatt> in fact, we probably shouldnt let people impl those, cause you can fuck up the endianness 10:46 < BlueMatt> and they have to be LE 10:46 < BlueMatt> so we should probably seal that trait 10:46 < andytoshi> we already have impls on all the primitive types 10:46 < andytoshi> so that people can use the trait safely 10:46 < BlueMatt> only for Write/Read 10:47 < BlueMatt> if I go impl Encoder/Decoder, I can break things 10:47 < BlueMatt> cause LE is only specified in the Encoder for W: Write/Decoder for R: Read 10:47 < andytoshi> btw we should rename those to WriteConsensusExt/ReadConsensusExt or something, it's more idiomaic for something that just extends Write/Read 10:48 < BlueMatt> sure 10:48 < andytoshi> right, so, currently i don't have to think about this ever, and after sealing the trait i'm going to have to copy these traits into rust-elements 10:48 < andytoshi> so i don't buy that it's reducing room for mistakes 10:48 < BlueMatt> no? 10:48 < BlueMatt> you *do* have to think about this 10:48 < BlueMatt> I've almost made this mistake, fwiw 10:49 < BlueMatt> at some point I went and looked at the docs and started writing an Encoder/Decoder impl before realizing there is a default wrapper for Write/Read that I should use instead 10:49 < andytoshi> i don't, i know that elements is a fork of core so i just pass through to the rust-bitcoin encoding functions for all the rust-elements types 10:49 < andytoshi> ok, we can update the docs to say "don't impl this", renaming it would help 10:49 < andytoshi> to WriteExt 10:50 < BlueMatt> I dont see what elements types have to do with this? 10:50 < BlueMatt> you mean you need to be able to impl Encodable, yes, if you want to use this stuff 10:50 < BlueMatt> but I think we should seal *both* traits 10:51 < andytoshi> i know, and i'm saying that this will increase room for mistakes 10:51 < BlueMatt> (sealing either one individually would break your stuff) 10:51 < BlueMatt> only for forks of bitcoin.... 10:51 < BlueMatt> for rust-bitcoin users it would reduce room for mistakes 10:51 < andytoshi> yes 10:51 < BlueMatt> I almost made this mistake in rust-lightning 10:51 < BlueMatt> and you'd think I would know better 10:51 < andytoshi> your mistake was caused by the traits having shitty names 10:52 < andytoshi> if it was called WriteExt you wouldn't have tried to implement it on your own type 10:52 < BlueMatt> well, we could rename read_32 to read_i32_le 10:52 < BlueMatt> that would probably fix this specific case 10:52 < BlueMatt> meh, I dont know what WriteExt means "dont use this" 10:52 < andytoshi> WriteExt is used by everything that extends the Write trait, e.g. byteorder 10:52 < BlueMatt> if anything I'd assume WriteExt means "Externally-exposed Write, you can use this as you see fit!" 10:53 < andytoshi> it means 'Write extensions' and it's an idiom 10:53 < BlueMatt> and I'll bet its usually sealed :p 10:53 < andytoshi> we're using it in the consensus encoding https://github.com/rust-bitcoin/rust-bitcoin/blob/master/src/consensus/encode.rs#L40 10:53 < andytoshi> heh good q, lemme check 10:54 < andytoshi> nope 10:54 < andytoshi> although, in byteorder the methods are more explicitly named https://docs.rs/byteorder/1.3.1/src/byteorder/io.rs.html#1047-1553 10:56 < andytoshi> ok. on further reflection i'm ok sealing both traits. 10:57 < andytoshi> because i realize that if rust-bitcoin makes its own WriteConsensusExt i can just do WriteConsensusExtExt in rust-elements which passes through to it 10:57 < andytoshi> and that at least requires i be very explicit about "i forked bitcoin and i'm copying exactly these things from it" 10:58 < BlueMatt> cool, yea, I think thats the "right" way 10:59 -!- willpiers [~willpiers@38.75.231.30] has quit [Remote host closed the connection] 11:02 < andytoshi> commented on the PR 11:04 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 11:05 < BlueMatt> andytoshi: can we do the renames/seal'ds in another pr 11:05 < BlueMatt> and just take the current speedups/simplifications? 11:05 < BlueMatt> I guess I need to fix travis cause now the fuzzer cant do as interesting stuff so it makes travis sad 11:05 < andytoshi> yeah .. i restarted the job at least 11:06 < andytoshi> but if you think it's gonna timeout again i guess we need to fix it 11:06 < BlueMatt> I dont think that'll fix it 11:06 < BlueMatt> the fix is a while [ true ]; do echo still running; sleep 1 & 11:06 < BlueMatt> in the fuzz script 11:07 < andytoshi> i think you need something more exotic than `true`, or the echoes will keep the test running 11:07 < BlueMatt> hmm, probably 11:46 < andytoshi> BlueMatt: so, i think we can't seal the Encoder and Decoder traits 11:46 < andytoshi> we'd need to impl Sealed on all W: Write and R: Read 11:46 < BlueMatt> oh? 11:46 < BlueMatt> ah 11:47 < andytoshi> you can't do blanket impls on two different traits like that 11:47 < andytoshi> and also it'd be of very limited usefulness 11:47 < andytoshi> as anybody could implement one of those traits to evade it 11:47 < BlueMatt> well the point is to allow you to only impl Write/Read 11:47 < BlueMatt> so that you cant screw up the read_* functions 11:47 < andytoshi> yeah .. I suspect there's no way to do this unfortunately 11:47 < BlueMatt> but, if we cant, lets just rename read_{int} to read_{int}_le 11:47 < andytoshi> we can rename the read_* functions to emphasize that they're le 11:47 < BlueMatt> and move on 11:47 < andytoshi> sure 11:51 -!- gmaxwell [gmaxwell@wikimedia/KatWalsh/x-0001] has quit [Ping timeout: 245 seconds] 11:51 -!- gmaxwell [gmaxwell@mf4-xiph.osuosl.org] has joined #rust-bitcoin 11:52 -!- gmaxwell is now known as Guest58856 12:11 < andytoshi> lol fuck me 12:11 < andytoshi> error[E0445]: private trait `Sealed` in public interface 12:12 < andytoshi> maybe that's not how to seal things after all ... thanks rustc for telling me that after i corrected 118 types to implement Sealed.. 12:16 < andytoshi> lol, ok, you're supposed to exploit bugs in their public/private checking https://rust-lang-nursery.github.io/api-guidelines/future-proofing.html 12:20 < gwillen> how does that work? 12:20 < andytoshi> gwillen: if you have a `pub Thing` inside a private module, rustc will give you no indication whatsoever that the `Thing` is not actually exported 12:21 < gwillen> ahhhh, heh 12:21 < gwillen> so instead of a private trait you make a pub trait in a private module? 12:21 < gwillen> and then rustc doesn't yell at you even though it probably should? 12:21 < andytoshi> yep 12:21 < andytoshi> and yes, i've had bugs caused by this 12:22 < andytoshi> and in typical #rust fashion, was told notabug 12:22 < gwillen> heh 12:24 -!- Guest58856 [gmaxwell@mf4-xiph.osuosl.org] has quit [Changing host] 12:24 -!- Guest58856 [gmaxwell@wikimedia/KatWalsh/x-0001] has joined #rust-bitcoin 12:28 -!- Guest58856 is now known as gmaxwell 12:35 < BlueMatt> a bug, but people are now relying on it, so we cant change it...... 12:39 < andytoshi> anyway https://github.com/rust-bitcoin/rust-bitcoin/pull/265 is my extension to 264 13:11 -!- jtimon [~quassel@181.61.134.37.dynamic.jazztel.es] has joined #rust-bitcoin 14:03 < stevenroose> sgeisler: I might have gone a step too far with this PR... https://github.com/rust-bitcoin/rust-bech32/pull/34 14:03 < stevenroose> It's basically an entire rewrite.. I'd understand if it doesn't make you happy. To be honest, I'm OK with having just the other two PRs merged (#30 and #31) 14:04 < stevenroose> Just in response to your "normal users using that method", I don't think a lib like bech32 has "normal users". Will always be some other lib using it internally. 14:04 < stevenroose> Like base58. 14:05 < sgeisler> stevenroose: will have a look at it 14:13 < sgeisler> The new API looks nice, but I'd like to test integrating it into bech32-bitcoin and lightning-invoice 14:14 < sgeisler> One thing I thought about repeatedly was having a `Base32Writer` 14:19 < sgeisler> with the new API you'd probably have a function `fn bech32_writer<'a>(fmt: &'a mut fmt::Write, hrp: &str) -> Base32Writer<'a>` that gives you an object that can write the data part without allocating and accumulates the necessary data to build the checksum at the end (it could print the checksum on `Drop`) 14:20 < sgeisler> That would be super useful for lightning-invoice since I'm building the data part recursively which currently means lots of allocations. 14:24 < sgeisler> I think your new design actually makes this approach nicer to implement, previously I still had to accumulate a `Vec`on the top most level. That would make a bunch of WIP PRs obsolete. 14:24 < sgeisler> I'll try to implement this tomorrow and you can look if you want to include it in your PR 14:25 < stevenroose> sgeisler: An allocationless writer would be awesome! 14:26 < stevenroose> About bech32-bitcoin, I talked with Andrew about dropping it in favor of https://github.com/rust-bitcoin/rust-bitcoin/pull/255 14:26 < stevenroose> I did the same in rust-elements because there we used blech32 and I didn't want to create a blech32-elements or something else as an indirection. 14:28 < sgeisler> makes sense as long as clarkmoody has no too big problems with that, having so many crates makes updates more complicated at times 14:30 < sgeisler> it seems to work quite well in #255 and I can't imagine any problems in lightning-invoice 14:31 < sgeisler> so just let me figure the writer thing out, I think it fits well into this PR 14:34 < stevenroose> sgeisler: Don't pay too much attention to the changes in my bech32 PR if you're gonna implement a writer. As long as I have something that is more efficient then allocating the struct, I'm satisfied :) 14:35 < sgeisler> I think I'll just use the writer stuff inside the encode function and put most of the code from encode into the writer 14:54 -!- willpiers [~willpiers@38.75.231.30] has quit [Remote host closed the connection] 14:55 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 14:59 -!- willpiers [~willpiers@38.75.231.30] has quit [Ping timeout: 246 seconds] 15:09 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has quit [Remote host closed the connection] 16:04 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has joined #rust-bitcoin 16:15 -!- jtimon [~quassel@181.61.134.37.dynamic.jazztel.es] has quit [Quit: gone] 17:02 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 18:12 -!- willpiers [~willpiers@38.75.231.30] has quit [Remote host closed the connection] 18:13 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 18:17 -!- willpiers [~willpiers@38.75.231.30] has quit [Ping timeout: 244 seconds] 18:26 -!- BlueMatt [~BlueMatt@unaffiliated/bluematt] has quit [Ping timeout: 252 seconds] 18:26 -!- BlueMatt [~BlueMatt@unaffiliated/bluematt] has joined #rust-bitcoin 18:30 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has joined #rust-bitcoin 19:29 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has quit [Remote host closed the connection] 19:29 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has joined #rust-bitcoin 19:35 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has quit [Remote host closed the connection] 19:36 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has joined #rust-bitcoin 20:05 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has quit [Remote host closed the connection] 20:05 -!- michaelsdunn1 [~michaelsd@208.59.170.5] has joined #rust-bitcoin 20:05 -!- michaelsdunn1 [~michaelsd@208.59.170.5] has quit [Changing host] 20:05 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has joined #rust-bitcoin 20:08 -!- DeanWeen [~dean@gateway/tor-sasl/deanguss] has joined #rust-bitcoin 20:12 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 20:18 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has quit [Remote host closed the connection] 20:25 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has quit [Remote host closed the connection] 20:28 -!- instagibbs_ [~instagibb@pool-100-15-135-248.washdc.fios.verizon.net] has quit [Ping timeout: 258 seconds] 21:04 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has joined #rust-bitcoin 21:08 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has quit [Ping timeout: 246 seconds] 21:19 -!- willpiers [~willpiers@38.75.231.30] has quit [Remote host closed the connection] 21:38 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has joined #rust-bitcoin 21:43 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has quit [Remote host closed the connection] 22:25 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has joined #rust-bitcoin 22:30 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has quit [Ping timeout: 268 seconds] 23:05 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has joined #rust-bitcoin 23:10 -!- ccdle12 [~ccdle12@n218103251046.netvigator.com] has quit [Ping timeout: 268 seconds] 23:19 -!- willpiers [~willpiers@38.75.231.30] has joined #rust-bitcoin 23:24 -!- willpiers [~willpiers@38.75.231.30] has quit [Ping timeout: 244 seconds] --- Log closed Fri May 24 00:00:22 2019