--- Day changed Wed Aug 22 2018
00:58 < andytoshi> afaict there is no way i can output data from build.rs
00:58 < andytoshi> it seems like it just eats it
01:00 < andytoshi> https://github.com/rust-lang/cargo/issues/985 you have to pass -vv parrantly
05:11 -!- windsok [~windsok@unaffiliated/windsok] has quit [Remote host closed the connection]
05:14 -!- windsok [~windsok@rarepepe.cash] has joined #rust-bitcoin
05:14 -!- windsok [~windsok@rarepepe.cash] has quit [Changing host]
05:14 -!- windsok [~windsok@unaffiliated/windsok] has joined #rust-bitcoin
08:05 -!- itaseski [~itaseski@213.135.176.241] has joined #rust-bitcoin
08:16 -!- itaseski_ [~itaseski@213.135.176.241] has joined #rust-bitcoin
08:16 -!- itaseski [~itaseski@213.135.176.241] has quit [Ping timeout: 264 seconds]
09:35 -!- elichai2 [uid212594@gateway/web/irccloud.com/x-hccdbyzcballmczo] has joined #rust-bitcoin
10:45 -!- itaseski_ [~itaseski@213.135.176.241] has quit [Quit: Leaving]
13:32 -!- TamasBlummer [~Thunderbi@2003:e3:8f0d:6d13:d442:ffd7:9bbc:f8e9] has joined #rust-bitcoin
15:16 -!- itaseski [~itaseski@213.135.176.241] has joined #rust-bitcoin
16:11 < andytoshi> BlueMatt: https://github.com/rust-bitcoin/rust-secp256k1/pull/54 locks the `cc` version to 1.0.18 (meanwhile it looks like there's some work upstream to fix the issue)
16:11 < andytoshi> if you can ACK that, i'll rebase the 64-bit compilation stuff and release 0.11
16:12 < andytoshi> then i'll PR to rust-bitcoin to support 0.11, and we can maybe get rust-bitcoin 0.14 out today (if jeandudey can fix #142 and you can ack #136)
16:12 < BlueMatt> done
16:12 < andytoshi> thanks!
16:12 < andytoshi> fingers crossed about travis
16:13 < BlueMatt> ok, will review 136 now,t hen
16:13 < BlueMatt> err, needs rebase, though
16:15 < andytoshi> yep will do, one sec
16:17 < andytoshi> done
16:18 < andytoshi> heh cc was fixed upstream (for now) https://github.com/alexcrichton/cc-rs/pull/337
16:19 < BlueMatt> heh, ok
16:23 < andytoshi> ok, restarting travis on rust-secp #252 (64-bit compilation) .. if that passes i'll merge it and PR to increase version to 0.11
16:24  * BlueMatt -> lunch, will finish 136 in a bit
16:25 < andytoshi> cool me too
17:43 < andytoshi> https://github.com/rust-bitcoin/rust-secp256k1/pull/55 0.11 bump
18:33 < BlueMatt> andytoshi: do you not care about Op_PUSHBYTES_1 allowing non-minimal pushes?
18:33 < BlueMatt> ie OP_PUSHBYTES_1 00 instead of OP_0
18:34 < BlueMatt> as described in bip62 which is linked to in the docs
18:37 < andytoshi> BlueMatt: oh oops
18:37 < andytoshi> no, i care, i just missed that
18:39 < BlueMatt> heh somethingsomething write a fuzzer that checks that no two different scripts have the same collect()ed minimal-required iters?
18:40 < andytoshi> hmmm
18:40 < BlueMatt> heh, but I've *definitely* been going overboard on fuzz testing, soooo
18:41 < andytoshi> it's hard to fuzz for "no two things have the same collect() result"
18:41 < andytoshi> i mean, i can just generate pairs and compare them
18:41 < andytoshi> but that'd take a long time to find bad pairs
18:41 < BlueMatt> no? I mean you just do a fuzzer that deserializes two scripts
18:42 < andytoshi> right .. so say 2 in a million scripts collide
18:42 < BlueMatt> like two-byte-len||script1||remainder_is_script2 and then as long as neither decodes with error
18:42 < andytoshi> the fuzzer has to find those exact two
18:42 < andytoshi> i know it's not naive, but the naive search space is a trillion large
18:43 < BlueMatt> yea, I mean its definitely kinda bogus
18:43 < andytoshi> well, i'll try it and see if it finds the 0/OP_PUSHBYTES_1 thing
18:43 < andytoshi> fuzzing is kinda magic
18:43 < BlueMatt> heh, fair
18:44 < BlueMatt> I mean it may be that it'd catch the current stuff but only after like a day of runtime
18:44 < andytoshi> yeah
18:44 < BlueMatt> I mean afl was able to take a testcase that routes a payment through the being-fuzzed node to another node and turn it into a testcase that *pays* the being-fuzzed node
18:44 < BlueMatt> which I found to be rather magical
18:45 < BlueMatt> despite me still not having written an afl grammar
18:46 < andytoshi> hah holy shit
18:48 < BlueMatt> and *then* find a crash in the fuzz harness in handling received-payments correctly
18:51 < andytoshi> lol, right now it's beating the shit out of my "read the first 2 bytes of the input as length" code because apparently i'm incompetent :P
18:51 < BlueMatt> lol, yea, its good at punishing you
18:51 < BlueMatt> also, in part, I'm writing stupid absurd fuzz harnesses cause I want to throw angora at it if they ever get around to open sourcing it
18:52 < andytoshi> there we go, i got it after like 10 tries (it'd give me scripts with less than 2 bytes, then it'd give me an encoded length that went past the end of the vector, then apparently i had an order of operations wrong and it managed to overflow a left shift..)
18:52 < BlueMatt> yea, afl is super good at punishing you for overflows in rust
18:52 < andytoshi> i should figure out how to use afl
18:52 < BlueMatt> well honggfuzz too
18:52 < andytoshi> i'm just running hongfuzz because i'm starting from the travis-fuzz.sh script..
18:53 < BlueMatt> I havent noticed either being particularly more or less effecient
18:54 < BlueMatt> need more people to whack this guy to release angora: https://twitter.com/spinpx
18:55 < andytoshi> is he keeping it for business reasons?
18:55 < andytoshi> i suspect it is very lucrative if he has a closed source fuzzer
18:55 < BlueMatt> no idea, website says ms student
18:55 < BlueMatt> my presumption is he's just an academic and got it far enough to write the paper
18:55 < BlueMatt> and then moved on
18:55 < andytoshi> oh, that's dumb, it's probably just shitty code then, which people would fix quickly
18:55 < andytoshi> yeah
18:55 < BlueMatt> and diesnt feel like cleaning it up
18:56 < BlueMatt> I mean I'd personally clean it up if I got my hands on it
18:57 < andytoshi> lol https://github.com/rust-bitcoin/rust-bitcoin/pull/142 finally got his udecimal parser/serializer to pass fuzzing
18:57 < andytoshi> (actually it's largely my untested 2014 code, which apparently could be crashed in multiple ways..)
18:58 < BlueMatt> lol oops
18:58 < BlueMatt> ariard: yo
19:01 < andytoshi> oh i'm being dumb about minimality .. i should be reserializing scripts and making sure they roundtrip
19:01 < BlueMatt> oh from the iter? yea, I guess, as long as the serializer does all the fancy things like OP_0
19:02 < andytoshi> i'll try to parse pushes as numbers and then reencode them as numbers
19:02 < andytoshi> which would catch this
19:07 < andytoshi> ok now it caught things immediately
19:07 < andytoshi> 0109 (pushbytes_1 9)
19:07 < BlueMatt> cool!
19:08 < andytoshi> ok, i'll have this fixed in the next half hour .. then i'll PR to update rust-secp to 0.11 in rust-bitcoin (which i think will be basically a no-op) and then we're set for 0.14
19:08 < BlueMatt> yay!
19:25 < andytoshi> ok, is pushing a long string of zeroes considered a nonminimal push?
19:25 < andytoshi> ccccccfcjbdbfcbhdcbjnhhtbcthnnnjrvkcnelthnvu
19:25 < andytoshi> sorry
19:25 < andytoshi> i'm actually unsure about any of these .. other than the OP_n numbers being pushed exactly
19:26 < BlueMatt> it shouldnt be, no
19:26 < BlueMatt> a long string of 0s cant be pushed any other way
19:26 < andytoshi> right .. and neithrer can numbers with too much padding
19:26 < BlueMatt> yep
19:34 < andytoshi> ok i think i've got it
19:35 -!- TamasBlummer [~Thunderbi@2003:e3:8f0d:6d13:d442:ffd7:9bbc:f8e9] has quit [Remote host closed the connection]
19:39 < andytoshi> ok, updated minimal-iter PR https://github.com/rust-bitcoin/rust-bitcoin/pull/136 ... also added secp 0.11 PR https://github.com/rust-bitcoin/rust-bitcoin/pull/146 ... after these we should be good to go for 0.14
19:40 < andytoshi> then for 0.15 i'd like bitcoin_hashes and PSBT but i'm not in a huge rush for either of those (sorry dongcarl), will spend some time working on updating my downstream projects
19:40 < andytoshi> and also i should fixed compilation on windows, i'm pretty sure i should just detect windows and disable whatever flag i'm adding there, it's nonessential
19:45 < BlueMatt> hmm, dont think its right for 0x00
19:45 < BlueMatt> OP_0 pushes [], not [0]
19:45 < BlueMatt> also, you should check that PushBytes isnt pushing []
19:45 < BlueMatt> andytoshi: ^
19:45 < andytoshi> hmm, i think the fuzzer should've caught this .. strange
19:46 < andytoshi> yeah, will check that it's not pushing [] (fuzz wouldn't have caught that)
19:46 < BlueMatt> is the re-encoder wrong?
19:46 < BlueMatt> the encoder should do that too
19:47 < andytoshi> the re-encoder encodes pushes as pushes .. and special cases the numbers -1 through 16
19:47 -!- elichai2 [uid212594@gateway/web/irccloud.com/x-hccdbyzcballmczo] has quit [Quit: Connection closed for inactivity]
19:47 < BlueMatt> yea, so the re-encoder is wrong :p
19:47 < andytoshi> if you do pushdata_1 0, it should reencode that as pushdata_0
19:47 < BlueMatt> there is no pushdata_0?
19:47 < BlueMatt> ohoh, yes, OP_0
19:48 < andytoshi> yeah, sorry, it has many names in rust-bitcoin
19:48 < andytoshi> op_0, op_false, op_pushbytes_0
19:50 < BlueMatt> why does add_exp_assign return a Result?
19:50 < BlueMatt> can that fail?
19:50 < andytoshi> because you can get the point at infinity
19:50 < BlueMatt> ah, ugh
19:50 < andytoshi> by adding the negative of the secret key corresponding to the public key
19:50 < andytoshi> yep
19:50 < BlueMatt> but combine didnt check that? so was broken
19:50 < andytoshi> heh good question
19:51 < andytoshi> no, it did
19:51 < andytoshi> and it also returned a Result
19:51 < andytoshi> BTW the iterator interprets OP_0 as a 0-length push, which i think is fine
19:52 < BlueMatt> oh, lol, I missed my function's return type
19:52 < andytoshi> hmm, i guess i could make the iterator also parse numbers
19:52 < BlueMatt> that is the correct (tm) interpretation, imo
19:52 < BlueMatt> meh
19:52 < BlueMatt> dont bother
19:52 < BlueMatt> thats weird
19:52 < andytoshi> but i'd rather not, the point is to make script lexing easier
19:52 < andytoshi> agreed
19:55 < BlueMatt> hmmmm...how do I impl this without combine: https://github.com/rust-bitcoin/rust-lightning/blob/master/src/ln/chan_utils.rs#L114
19:55 < andytoshi> fixed the 00 thing
19:56 < andytoshi> hmm, lemme look into this
19:59 < andytoshi> we may need to move this question to upstream #secp256k1
20:14 < BlueMatt> hmm, well I guess I'll stick with secp 0.10 for now :/
20:16 < andytoshi> i'll give you `combine` back and release 0.11.1
20:17 < BlueMatt> I mean its ok for now
20:18 < BlueMatt> I can just stay on git master-as-of-now of rust-bitcoin
20:18 < BlueMatt> until we figure out what to do upstream
20:19 < andytoshi> well, upstream is going to take a while .. we'll make our musig API more general in some way
20:19 < andytoshi> but then musig has a lot of review cycles to go before it gets merged
20:19 < BlueMatt> ugh
20:19 < andytoshi> https://github.com/rust-bitcoin/rust-secp256k1/pull/56 for now
20:20 < BlueMatt> k
20:21 < andytoshi> oh actually lemme put the version bump into that PR too
20:22 < andytoshi> done
20:24 < BlueMatt> hmm, why doesnt the fuzz thinggy push OP_0 for len 0?
20:24 < BlueMatt> why isnt that broken
20:25 < andytoshi> pushbytes_0 is the same as op_0
20:26 < BlueMatt> not the way I read push_slice?
20:26 < andytoshi> push_slice(&[]) should add 0x00 to the script
20:26 < BlueMatt> the code says PushBytes if bytes.len() ==1 else push_slice
20:26 < BlueMatt> ohoh, duh
20:26 < BlueMatt> I cant read
20:26 < BlueMatt> yea, k
20:26 < andytoshi> heh kk
20:26 < andytoshi> this is why normal languages don't make "zero" and "nothing" the same..
20:27 < BlueMatt> wait, damn it why did travis fail
20:27 < BlueMatt> fuzzer broke your shit
20:27 < andytoshi> lol dammit
20:27 < andytoshi> i ran it for 10x the travis iterations locally
20:27 < BlueMatt> heh, randomness sucks :p
20:36 < andytoshi> something very strange is going on with this script
20:36 < andytoshi> i've shrunk the error to "ff0300000000000000010000"
20:36 < andytoshi> which it reads as OP_PUSHBYTES_1 00 OP_0
20:38 < BlueMatt> your parse_int is converting 00 to OP_0
20:38 < BlueMatt> I'd assume?
20:38 < BlueMatt> which is correct, but will break the test
20:38 < andytoshi> no
20:38 < andytoshi> oh, yeah, it will
20:38 < andytoshi> but why is the script being parsed like that
20:39 < BlueMatt> I mean ff should immediately return err, no?
20:39 < andytoshi> i think that's a length
20:39 < andytoshi> i think it's a nonminimal 3-byte length
20:39 < andytoshi> oh lol
20:40 < andytoshi> ff0300000000000000 is a varint 3
20:40 < andytoshi> fucking fuzzer
20:40 < BlueMatt> ahah
20:40 < BlueMatt> heh, why is it doing the varint hop?
20:40 < BlueMatt> I mean can leave it, but seems kinda dumb
20:41 < andytoshi> ok, fixed the fuzz test. travis should pass now
20:41 < andytoshi> serialize::deserialize reads a length prefix
20:41 < BlueMatt> yea, looks good
20:41 < andytoshi> we could convert the byte slice directly into a Script, but the point of this fuzz test was originally to check that that deserialization worked
20:41 < BlueMatt> yea, ok, fair enough i guess
20:43 < andytoshi> ok, so if travis passes we'll have 0.14 done :)
20:44 < andytoshi> meanwhile on rust-secp i forgot to re-add a `use std::mem` so travis failed .. i fixed #56, it's redoing travis
20:44 < BlueMatt> lol
20:44 < BlueMatt> k
21:10 < BlueMatt> andytoshi: you can merge the rust-bitcoin one :p
21:13 < BlueMatt> andytoshi: and secp :p
21:16 < andytoshi> dope
21:17 < BlueMatt> gotta bump version for rust-bitcoin too
21:17 < andytoshi> yep
21:17 < andytoshi> writing changelog .. almost done
21:17 < andytoshi> there was a shitload of stuff this time
21:18 < BlueMatt> heh, yay?
21:18 < andytoshi> heh yep
21:25  * andytoshi anxiously watches travis
21:26 < BlueMatt> heh, yea, I just cancelled and restarted a build so that the ordering would result in it finishing earlier lol
21:55 < andytoshi> wtf
21:55 < andytoshi> error: crates cannot be published to crates.io with dependencies sourced from a repository
21:55 < andytoshi> either publish `hex` as its own crate on crates.io and specify a crates.io version as a dependency or pull it into this repository and specify it with a path and version
21:55 < andytoshi> (crate `hex` has repository path `https://github.com/KokaKiwi/rust-hex?rev=19fd37137686c30058bd9d11d21590e726ffdf31`)
22:04 < andytoshi> i think we have to pin to version rather than rev
22:13 < andytoshi> BlueMatt: https://github.com/rust-bitcoin/rust-bitcoin/pull/148 .. 0.3.2 matches the git rev that we were pinned to
22:13 < andytoshi> crates.io will not let you bypass their index
22:28 < BlueMatt> thats fucking dumb, but ok
22:28 < BlueMatt> guess we just need to make it a dev-dep
22:28 < BlueMatt> or optional
22:30 < andytoshi> well, it's replacing rustc-serialize
22:30 < BlueMatt> yea, i know
22:30 < andytoshi> lemme see if we can make it optional
22:30 < andytoshi> there was discussion on the PR to the effect that we were willing to make a net-zero change in the number of dependencies so we wouldn't have to delay 0.14 bending over backwards for hex deserialization
22:30 < andytoshi> and that it'd go away in 0.15
22:31 < BlueMatt> nah, its fine, I think
22:31 < BlueMatt> for now
22:31 < andytoshi> i've got hex stuff in the `bitcoin_hashes` crate, so i'd rather minimize the work we spend on it for new
22:31 < BlueMatt> agreed
22:55 < andytoshi> publishing 0.14
22:56 -!- itaseski [~itaseski@213.135.176.241] has quit [Ping timeout: 264 seconds]