--- Day changed Wed Sep 02 2015
01:35 < midnightmagic> I often wonder how accurately the from-the-manufacturer emulators emulate chip undocumented features.
01:36 < gmaxwell> probably depends on why its undocumented!
01:37 < gmaxwell> "no no we totally intended for it to work that way.."
01:44 < midnightmagic> :-)
01:45 < midnightmagic> My ahem prior employer has a *pile* of undocumented features, a great deal of them there specifically to support internal software projects, and the rest are mostly the beginnings of infrastructural support for new features. "Use of these features is entirely unsupported."
02:05 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #secp256k1
04:48 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Read error: Connection reset by peer]
04:49 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has joined #secp256k1
08:24 -!- nullbyte [NSA@gateway/vpn/mullvad/x-aprqdfjpgdvjhibh] has quit [Ping timeout: 246 seconds]
08:26 -!- nullbyte [~NSA@193.138.219.233] has joined #secp256k1
13:22 -!- GAit [~GAit@2-230-161-158.ip202.fastwebnet.it] has quit [Quit: Leaving.]
15:02 -!- jtimon [~quassel@159.30.134.37.dynamic.jazztel.es] has joined #secp256k1
19:08 < sipa> Formula secp256k1_gej_add_ge_old [should fail]:
19:08 < sipa>   branch 1:
19:08 < sipa>     Add: requires nonzero [by + ay]
19:08 < sipa>     Double: OK (assuming (Bz - 1) = 0)
19:08 < sipa>   FAIL
19:09 < sipa> if a.y == b.y, is a.x == b.x?
19:15 < Luke-Jr> sipa: how do you get strict-DER parsing ?
19:15 < sipa> Luke-Jr: not implemented
19:15 < Luke-Jr> sipa: but we need it for Bitcoin? O.o
19:15 < sipa> no
19:15 < sipa> BIP66 specifies what is acceptable
19:16 < Luke-Jr> oh, so we're going to just keep that in Core? :/
19:16 < sipa> BIP66 does not say "whatever libsecp256k1 treats as strict DER"
19:16 < sipa> inevitably yes
19:16 < Luke-Jr> might still be nice to add a flag for the lib to do it?
19:16 < sipa> perhaps, but it won't help bitcoin
19:17 < Luke-Jr> is libsecp256k1 not guaranteeing consensus-compatibility?
19:18 < sipa> it guarantees correct ECDSA
19:18 < sipa> and it guarantees parsing DER
19:18 < Luke-Jr> so if there's a bug, libsecp256k1 will fix it and Bitcoin will have no real solution? :/
19:18 < gmaxwell> sipa: we could easily add an openssl style round trip check to the der parser?
19:18 < sipa> gmaxwell: yes, but it won't be usable for bitcoin
19:19 < gmaxwell> Luke-Jr: bitcoin would use a fork, in that unlikely case.
19:19 < gmaxwell> sipa: yea, due to history.
19:19 < Luke-Jr> maybe most (all?) functions should have a expected-version argument that allows emulating older versions
19:19 < gmaxwell> Well we now have an exposed parse function so we could have a _permissive version?
19:19 < sipa> gmaxwell: even ignoring that, BIP66 does not specify that signatures be valid
19:19 < gmaxwell> yes so? it's not a replacement for BIP66. I do not agree with luke please don't misunderstand.
19:20 < sipa> i'm fine with a stricter parser function
19:20 < sipa> but it's very hard to specify
19:20 < gmaxwell> I'm suggesting from a purely libsecp256k1 perspective, perhaps the default parser we should offer should be strict-- to avoid helping others make bitcoin's mistakes. :)
19:20 < sipa> what if one of the integers overflows or is negative?
19:21 < gmaxwell> it fails. Strict parser, I think should fail all edge cases where you think it might?
19:21 < sipa> basically, computing whether a signature is valid DER without making any assumptions about what contents might be a valid is nearly impossible
19:21 < Luke-Jr> it doesn't help with BIP66, but it could be useful for <future merkletree-based script that doesn't support inversion>
19:21 < sipa> Luke-Jr: such future script would not use DER
19:22 < sipa> gmaxwell: you can make a "is this (a) valid DER and (b) potentially a valid signature", but that is pretty complicated to replicate as consensus code
19:22 < sipa> if someone would not want to use libsecp256k1 for that
19:23 < sipa> (as it needs bigint math)
19:24 < gmaxwell> oh I see what you mean, round trip isn't enough to catch range errors.
19:24 < gmaxwell> (or at least not necessarily enough)
19:24 < sipa> well you can define that parsing function as "accepts valid DER signatures with integers in the range 0, 2^256-1]"
19:25 < sipa> which is what BIP66 does implicitly
19:25 < sipa> though BIP66 also accepts empty sigs
19:25 < Luke-Jr> any reason not to just have a compatibility-version flag on the context object, which guaranteed consensus-compatible behaviour?
19:26 < gmaxwell> "consensus-compatible" behavior is under-defined unless it means we change nothing at all.
19:26 < sipa> exactly
19:26 < gmaxwell> You can imagine a system that falls out of consensus if the library is made 2x faster but nothing else changes. Bitcoin wouldn't do that, but some imaginary system might.
19:26 < sipa> i would feel very unconfortable with moving consensus logic which we now _have_ specified at the bitcoin layer to libsecp256k1
19:27 < Luke-Jr> sipa: you want to stick to OpenSSL in Core forever? or am I misunderstanding you?
19:30 < sipa> Luke-Jr: since BIP66, our requirement from OpenSSL changed from "parses & verifies exactly the same set of signatures" to "verifies exactly the same set of DER-valid signatures"
19:30 < sipa> libsecp256k1 can replace that
19:30 < sipa> (assuming the ECDSA implementation in OpenSSL and libsecp256k1 are identical)
19:31 < Luke-Jr> sipa: ok, so it's just the BIP66 logic you don't want to move to libsecp256k1
19:31 < sipa> indeed
19:31 < Luke-Jr> ok, that makes sense if we plan to abandon DER in new work
19:32 < Luke-Jr> I do think at least trying to provide the consensus-compatibility Bitcoin needs would be a good idea though; even if we have to give up on it later.
19:33 < Luke-Jr> if we want to be super-conservative about it, we could export a second identical API that worst-case-scenario executes the original functions ?
19:34 < Luke-Jr> (I suppose adding this later is always a possibility too)
19:34 < sipa> one way to do that is to not touch the subtree :)
19:36 < Luke-Jr> well, as usual, I strongly disagree with subtree/bundled dependencies. :p
19:36 < Luke-Jr> even while understanding why it is semi-necessary in this case
19:37 < Luke-Jr> I'll have to do the work to maintain a consensus-compatible fork inevitably either way, so might as well try to do it upstream if you'll take it.
19:38 < sipa> since BIP66, it is consensus-compatible as long as you only pass DER signatures to the library
19:38 < sipa> that's the same as saying that libc is consensus-compatible as long as the malloc() function actually allocates memory
19:39 < Luke-Jr> assuming no bugs
19:39 < sipa> of course
19:40 < gmaxwell> and assuming by consensus compatible you only mean what bitcoin requires from it!
19:40 < Luke-Jr> I guess we can cross that bridge when/if we find a bug
19:40 < Luke-Jr> gmaxwell: yes, that's hard enough to guarantee; no need to worry about stricter until there's a need IMO
19:50 < sipa> gmaxwell: go have a look at my new sage branch
22:57 < gmaxwell> sipa: Think you can add a comment that describes the general strategy and its limitations? Also your improvements over Lange?  (I presume you'd want to write that down for the purpose of emailing her in any case)