--- Day changed Mon Nov 23 2015 00:32 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has quit [Ping timeout: 250 seconds] 01:29 -!- Luke-Jr [~luke-jr@unaffiliated/luke-jr] has joined #secp256k1 05:18 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has joined #secp256k1 09:04 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #secp256k1 09:51 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Quit: Leaving.] 10:09 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #secp256k1 10:47 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Quit: Leaving.] 10:48 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #secp256k1 10:50 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Client Quit] 10:50 -!- GAit [~GAit@2.230.161.158] has joined #secp256k1 10:52 -!- GAit [~GAit@2.230.161.158] has quit [Client Quit] 11:21 -!- andytoshi [~andytoshi@wpsoftware.net] has quit [Changing host] 11:21 -!- andytoshi [~andytoshi@unaffiliated/andytoshi] has joined #secp256k1 11:40 -!- nickler [~nickler@185.12.46.130] has joined #secp256k1 11:41 < gmaxwell> If anyone cares, my last coverage report is at https://people.xiph.org/~greg/secp256k1_coverage_2015_10_31/tt2/secp256k1/src/index.html it hasn't changed since then. 11:43 < gmaxwell> nickler was asking me 'did someone already think about/try symbolic execution for libsecp to generate testcases?' 11:43 < gmaxwell> I responded " I've used klee in the past to do that on other projects but it's really hard to setup and I lost my install. So instead I've used whilebox directed fuzzing (AFL and the some custom code I wrote) to generate test vectors which I've then minimized. I've been slowly pipelining them into the code base. We have ~95% condition/decision coverage currently. I'd really like to get that to 100% min 11:43 < gmaxwell> us the couple branches that are explicitly noted as cryptographically unreachable." 11:44 < gmaxwell> Right now there are a wad of comparisons in the scalar code that both sides aren't hit on, even with really insane specially generated random inputs. I strongly suspect at least some of them are just actually unreachable; might be a good candidate for symbolic execution to try to hit them. 11:44 < nickler> did you have success using symbolic execution in theseother projects? I have seen it mentioned quite often in security conferences but didn't really found any success stories apart from the authors' claims. 11:46 < gmaxwell> nickler: yes/no. So I used klee on opus and managed to hit some execptional cases and found bugs. But I found more bugs later with non-uniformly distributed pseudorandom inputs. I'm not sure if the symbolic execution was a big win or not. The problem was that it was much slower than strategies more like AFL. 11:48 < gmaxwell> I think it is worth trying for libsecp specifically to try to reach the additional carry conditions in the scalar arithemetic. The only other option I have there is to work out by hand what inputs would trigger them, if any. 11:48 < gmaxwell> (as I've already expended many CPU users of time with random inputs and specially distributed random inputs, and was not able to get a hit) 11:48 < gmaxwell> er. s/users/years/ 11:51 < gmaxwell> I feel like if we had a symbolic execution rig setup it could have saved some time for sipa in writing his BER parser fuzzer. 11:52 < gmaxwell> (it very quickly gets 100% condition/decision coverage in our code; but a simplistic random generator or even AFL does not.) 11:54 < gmaxwell> Hm. it seems that klee has a docker image now. 11:55 < nickler> not sure if that alone simplifies things, but we could try to set up a secp256k1-testing docker image 11:55 < gmaxwell> nickler: well when I first ran klee several years ago I lost over a week building obscure piece of ocaml software. :) 11:56 < gmaxwell> I upgraded the machine it was on, which broke it and I was too tramatized by the first expirence to try again. :) 11:57 < nickler> I see :D 12:00 < nickler> btw, I just saw some presentation slides today from bhusa15 where someone found sipa's BN_sqr bug with afl in one hour 12:01 < nickler> https://comsecuris.com/research/slides-bignum-bhus2015.pdf 12:02 < nickler> slide 26 12:04 < gmaxwell> nickler: Thats not _that_ shocking to me, many AFL's inputs aren't random and one of the non-random things it does is sets mostly 1s which is what you need to do to triggr that... but it says less about AFL than it does just about the nature of that particular bug. 12:05 < gmaxwell> At least my philosophy on these things is that every approach we take to quality assurance has blind spots, and so we should use many approaches and hope that they cover for each other. 12:05 < sipa> nickler: interesting! 12:05 < gmaxwell> For things like BN arithemetic the gold standard should be sound formal verification. It's sad that the tools for this are not yet mature enough to really deploy them. 12:12 < gmaxwell> nickler: oh also, once andytoshi implements the new modular inversion he was working on, I expect it to introduce a big pile of branchy and hard to verify code that we'll want to check. 13:35 < andytoshi> gmaxwell: the existing crappy modular inverse code was surprisingly easy to hit all branches for 13:35 < sipa> andytoshi: because it only has one branch? 13:36 < andytoshi> sipa: hehe, i mean the new crappy modular inverse code 13:36 < sipa> oh 13:36 < andytoshi> the gcd+lehmer one that's not merged 13:36 < sipa> btw, can you fix the bug in it? 13:37 < sipa> (or work around the toolchain bug being hit in travis) 13:37 < andytoshi> oh, the timeout thing? 13:37 < andytoshi> i don't know what's causing it 13:37 < andytoshi> i'll find some time this week to work on it again 13:37 < sipa> it was due to a conflict on the 'div' function, no 13:37 < andytoshi> oh, that thing 13:37 < andytoshi> ok, fine, i'll rename div 14:10 -!- belcher [~user@unaffiliated/belcher] has joined #secp256k1 19:08 -!- belcher [~user@unaffiliated/belcher] has quit [Quit: Leaving]