--- Log opened Fri Sep 25 00:00:31 2020 01:48 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection] 01:48 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined #rust-bitcoin 01:50 -!- vindard [~vindard@190.83.165.233] has quit [Quit: No Ping reply in 180 seconds.] 01:51 -!- vindard [~vindard@190.83.165.233] has joined #rust-bitcoin 02:54 -!- jonatack [~jon@37.164.166.21] has joined #rust-bitcoin 03:18 -!- jonatack [~jon@37.164.166.21] has quit [Ping timeout: 240 seconds] 03:19 -!- jonatack [~jon@109.202.107.10] has joined #rust-bitcoin 03:21 -!- Jeremy19Weimann [~Jeremy19W@static.57.1.216.95.clients.your-server.de] has joined #rust-bitcoin 07:06 -!- jonatack [~jon@109.202.107.10] has quit [Ping timeout: 272 seconds] 07:51 < elichai2> BlueMatt: idk if you've seen this but some interesting info on how debian packages rust https://www.reddit.com/r/rust/comments/c8bgwf/ripgrep_dependency_has_been_marked_for/ 08:09 < BlueMatt> elichai2: lol damn. 08:10 < BlueMatt> elichai2: still, that seems....good? rejecting nonsense garbage that uses RUSTC_BOOTSTRAP=1 to build *should* be kicked out of the repo 08:10 < BlueMatt> even if its simd... 08:13 < elichai2> yeah, it's interesting to read the maintainer post there about how they try to enable features in crates 08:13 < BlueMatt> yea, i mean makes sense? people bitching about it I dont get? I assume its manual per crate, so, like, they're not going to accidentally enable _LOL_test_only_features 08:14 < BlueMatt> I've generally been more impressed with the debian-rust packagers than most of rust upstream 08:14 < elichai2> I'm not quite sure what he meant there 08:14 < BlueMatt> (though a lot because they do the heavy lifting to integrate with system llvm, so you can do llvm plugins that work with rustc and cross-language-lto) 08:15 < BlueMatt> unlike rustup 08:15 < elichai2> it's upsetting that people use RUSTC_BOOTSTRAP in production 08:15 < BlueMatt> yea, wtf...there is some discussion of enforcing it in crates.io, which makes sense 08:15 < elichai2> BlueMatt: I just compile clang myself to make sure the LLVM versions match lol 08:15 < BlueMatt> though I'm glad they didnt remove it...i use that in our bindings test script to turn on sanitizers! 08:15 < elichai2> firefox does that IIRC 08:15 < BlueMatt> not on debian, its all system llvm :) 08:16 < BlueMatt> $ ldd `which rustc` | grep LLVM 08:16 < BlueMatt> libLLVM-10.so.1 => /usr/lib/x86_64-linux-gnu/libLLVM-10.so.1 (0x00007f3c0b659000) 08:16 < BlueMatt> and I've even played with llvm plugins which are as easy as apt-get install llvm-dev, build, and LD_PRELOAD and suddenly you can llvm plugin in rustc 08:16 < BlueMatt> I dont get why anyone uses rustup, debian-packaged rustc is soooo much better 08:17 < BlueMatt> (though not ubuntu, they do their own packages which dont have the work put in) 08:17 < elichai2> well I don't use debian hehe 08:18 < BlueMatt> ah...i assume other distros also do good packaging? 08:18 < BlueMatt> i mean not ubuntu, but, like, ubuntu is trash 08:19 < elichai2> idk I like compiling my own compilers for some reason 08:19 < BlueMatt> heh, i cant fault you for that. 08:19 < elichai2> although I had some problems with llvm-libs because of that 08:20 < elichai2> so for me both clang and rustc are statically linked with libLLVM, for better and for worse 08:20 < elichai2> any interesting llvm plugins though? I might change that if I see something cool hehe 08:20 < BlueMatt> I was trying to play with a fancy taint-tracing fuzz thing 08:20 < BlueMatt> but never *quite* got it to work. 08:21 < elichai2> (FYI idk if you've seen the new gcc fiasco) 08:21 < BlueMatt> yea, i heard 08:21 < BlueMatt> holy fuck 08:21 < elichai2> exactly 08:21 < BlueMatt> I was trying to get angora to work - https://github.com/AngoraFuzzer/Angora 08:21 < BlueMatt> i made progress, then they needed to update llvm support, which they did, but i never circled back around to it 08:21 < BlueMatt> its probably within reach to get that to work. 08:21 < BlueMatt> oof, though it now says "LLVM 4 - 7", so it may be out of date again :( 08:22 < BlueMatt> see https://github.com/AngoraFuzzer/Angora/issues/10 if you feel like picking it up 08:24 < BlueMatt> (I was interested cause we have a lot of super complicated fuzzers in rust-lightning, and just throwing CPU at the wall isn't the most effecient option there...getting something like angora or driller to do some more careful analysis would probably improve coverage by a ton) 08:24 < BlueMatt> though i dont think it would make a difference in eg rust-bitcoin cause thats mostly serialization fuzzers 08:49 -!- vindard [~vindard@190.83.165.233] has quit [Ping timeout: 260 seconds] 08:53 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection] 08:53 -!- ghost43_ [~daer@gateway/tor-sasl/daer] has joined #rust-bitcoin 09:50 < ariard> BlueMatt: re https://github.com/rust-bitcoin/rust-lightning/pull/716#discussion_r495103385 secp256k1_ecdsa_sign will always create a signature with a low-s 09:50 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #rust-bitcoin 09:50 < ariard> so I think our test framework will never generate such 72 bytes signature 09:51 < ariard> but our serializer (secp256k1_ecdsa_signature_parse_compact) doesn't seem to enforce s normalization 09:54 -!- Jeremy19Weimann [~Jeremy19W@static.57.1.216.95.clients.your-server.de] has quit [Ping timeout: 264 seconds] 11:22 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Ping timeout: 260 seconds] 11:44 < BlueMatt> ariard: ah, ok, that makes sense... 11:45 < BlueMatt> ariard: wait, no, cause high-s is nonstandard 11:45 < BlueMatt> I think secp256k1::Signature::from_compact normalizes 11:45 < BlueMatt> or, like, it should. 11:58 < BlueMatt> andytoshi or elichai2 do y'all recall? 11:59 < BlueMatt> i could have sworn it did but now I dont see it anywhere, and it would be a somewhat critical vuln for a number of things if it didnt 11:59 < BlueMatt> errr, right, I think its required by verify, ariard 11:59 < elichai2> it doesn't 11:59 < elichai2> iirc you need to explicitly normalize 12:00 < BlueMatt> that seems like it woul dbe a vuln for ~every user of libsecp that verifies third-party sigs, no? 12:00 < elichai2> because the library isn't bitcoin specific 12:01 < elichai2> ecdsa allow both s's, and it isn't considered really a vulnerability 12:01 < BlueMatt> sure, but thats not a good reason to have an api that is very unsafe. 12:01 < BlueMatt> sure, but the api is unsafe for use in bitcoin contexts with third-party signatures 12:01 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #rust-bitcoin 12:02 < elichai2> if you verify by just calling `from_der(..).verify(..)` then yeah it might be non standard bitcoin 12:02 -!- ghost43_ [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection] 12:02 < BlueMatt> right, so, that at least needs to be changed to verify_bitcoin_safe and verify_non_bitcoin_users :p 12:03 < elichai2> hmm let me remember a sec 12:03 < BlueMatt> i could have sworn I've looked at this before. 12:03 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined #rust-bitcoin 12:03 < elichai2> ok, right, so libsecp only verifies lower S, my mistake. But because bitcoin's consensus allows also high s then if you want to verify correctly you need to first normalize all signatures 12:04 < BlueMatt> righttt, ok, yes, thats what i thought 12:04 < elichai2> so libsecp verify *doesn't* adhere to ECDSA + bitcoin consensus, unless you first call `normalize` 12:04 < BlueMatt> sure, you cant use it for consensus, but for all practical users thats not the goal :) 12:04 < elichai2> yeah it took me a second to remember :) 12:04 < BlueMatt> sure, right, but no non-consensus users care about consensus, they care about standardness rules :) 12:04 < BlueMatt> and low-S *is* standard-required 12:05 < BlueMatt> hmm, why cant i find that code anywhere anymore? 12:05 < elichai2> do they even care about verify or just signing? 12:05 < elichai2> BlueMatt: here 12:05 < elichai2> https://docs.rs/secp256k1/0.19.0/secp256k1/struct.Signature.html#method.normalize_s 12:06 < BlueMatt> if you're dealing with third-party sigs (eg lightning, and I assume psbt-type stuff) then you need to be able to verify the sig, and presumably care that the sig you verified will broadcast on the bitcoin network 12:06 < BlueMatt> elichai2: no, I meant the code in verify() that checks its low-S 12:06 < elichai2> here: https://github.com/bitcoin-core/secp256k1/blob/master/src/secp256k1.c#L432 12:06 < elichai2> `!secp256k1_scalar_is_high(&s)` 12:07 < BlueMatt> ah, i was just looking at ecdsa_sig_verify, not ecdsa_verify 12:08 < BlueMatt> but, indeed, rust side calls the right one. 12:08 < BlueMatt> ok, phew 12:09 < elichai2> well valid signatures doesn't mean it will pass standardness rules :) so you'll anyway have to call libconsensus if you wanna be sure 12:09 < BlueMatt> sure, but at least in the lightning case you build the transaction locally and just add their signature that they provide 12:09 < BlueMatt> so in this case it mostly does :) 12:09 < BlueMatt> and i presume psbt does something vaguely similar? or maybe it doesnt concern itself with that? I dont actually know 12:11 < elichai2> good question 12:24 < andytoshi> heh, sorry for missing this convo when it happened 12:24 < andytoshi> but sounds like y'all figured it out -- ecdsa_verify indeed only accepts low-s 12:54 -!- BlueMatt [~BlueMatt@unaffiliated/bluematt] has quit [Ping timeout: 246 seconds] 12:55 -!- BlueMatt [~BlueMatt@unaffiliated/bluematt] has joined #rust-bitcoin 13:01 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Quit: jonatack] 13:10 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #rust-bitcoin 15:32 < ariard> BlueMatt, elichai: fyi Core explicitly normalize any signature provided to verify here https://github.com/bitcoin/bitcoin/blob/78f912c9010f686e2d1bbdc1c51f381b496c2a1b/src/pubkey.cpp#L183 15:33 < ariard> BlueMatt: we will reject any third-party high low-S as we used libsecp256k1 verify ofc 15:34 < ariard> BlueMatt: though I'm quite sure that's untested for now 15:35 < BlueMatt> ariard: it may be untested, but if dont reject high-S then we are hella vulnerable to counterparties sending us nonstandard things that we can't broadcast ;) 15:36 < BlueMatt> so, I'm ok relying on it for a tx-size-assertsion 15:37 < ariard> BlueMatt: you know you know ;) 15:37 < ariard> #630 is here for a reason 15:37 < BlueMatt> sure, we should def add tests for this :) 15:38 < ariard> lol have you look on the last p2p meeting logs ? WIP 15:38 < ariard> for #715 I verify assert are after sigs verification 15:38 < BlueMatt> hmm? bitcoin core meeting stuff? I dont even think I'm in core-dev anymore 15:38 < ariard> so it should be good for 71, as if we don't fail that means we have non-standard shit 15:38 < ariard> BlueMatt: we do have a new meeting around tea time tuesday afternoon biweekly 15:39 < BlueMatt> right, i heard there was a p2p meeting now, glad its getting more love :) 15:40 < ariard> yeah those p2p core issues are consuming a bit of my time, but we know that's worth it :) 15:40 < ariard> BlueMatt: how close we are for landing #649 ? 15:41 < BlueMatt> ariard: it got a rebase an hour or five ago, so hopefully close. i need to do a careful review this weekend or so 15:42 < BlueMatt> yea, glad core p2p is finally getting a bit of love again...dont think I'll manage to find time to work on it, but, still. 15:44 < ariard> okay reviewing #649 test changes again to see nothing is wrongly changed 15:45 < ariard> wrt to interfaces/doc it sounds good to me, further reviewing will be bikeshedding 15:50 < BlueMatt> ok, cool. feel free to ack, though I'm sure I'll be pretty nitpicky about docs, I tend to be :) 17:00 -!- DeanGuss [~dean@gateway/tor-sasl/deanguss] has quit [Remote host closed the connection] 17:05 -!- DeanGuss [~dean@gateway/tor-sasl/deanguss] has joined #rust-bitcoin --- Log closed Sat Sep 26 00:00:29 2020