--- Log opened Tue Jun 29 00:00:51 2021
00:11 -!- johnzweng [~johnzweng@80-108-14-34.cable.dynamic.surfer.at] has joined #secp256k1
00:44 -!- johnzweng [~johnzweng@80-108-14-34.cable.dynamic.surfer.at] has quit [Quit: Leaving...]
01:09 -!- robertspigler [~robertspi@2001:470:69fc:105::2d53] has quit [Quit: Bridge terminating on SIGTERM]
01:11 < real_or_random> gmaxwell: indeed, but one step at a time :D 
01:13 < real_or_random> roconnor gmaxwell sipa: for the build system stuff in https://github.com/bitcoin-core/secp256k1/pull/956 :
01:15 < real_or_random> I think if we ship with the generated files, then ideally `make` would recognize via dependency tracking that it should essentially never rebuild them. but I bet that will be cumbersome with timestamps etc.
01:15 < real_or_random> I think  we should entirely remove the generation of files from the normal build process. and maybe add a separate make target for them, so you can rebuild on demand.
01:16 < gmaxwell> I never looked into it but that behavior is probably tunable. but .. right I was just about to say "just add another target"
01:16 < gmaxwell> make bobbytables
01:16 < gmaxwell> part of the ci process could also run that and make sure the resulting file doesn't change.
01:17 < real_or_random> then it's more like a bash/whatever script for the developers, just written in autotools for compiler detection etc.
01:17 < real_or_random> lol @ bobbytables
01:18 < real_or_random> strictly speaking, you could run into issues when you forget to rebuild. but I assume that's a really rare case. and 99% of the cases it will be syntax error and not just a silent bug 
01:19 < gmaxwell> Forget to rebuild?  like change some thing that would change the generation output?  meh. That doesn't concern me.
01:20 < real_or_random> e.g., imagine we switch the limb order in the constant macros. This is a very obscure case
01:20 < gmaxwell> Besides, the tests could be comprehensive enough to actually check the table. We could even make the tests actually generate the table and compare it to the one in memory.
01:23 < real_or_random> argh but what I have in mind doesn't work
01:24 < real_or_random> or it does but it's not optimal. the tables are still only valid for a specific config: `#if ECMULT_GEN_PREC_N != 64 || ECMULT_GEN_PREC_G != 16`
01:31 < real_or_random> hm, well, say have 2 or 3 options per table ("small", "big"), and it's okay to have one pregenerated table in the repo, why not have 2 or 3? 
01:34 < gmaxwell> real_or_random: well for the ecmult_gen table the largest table contains all smaller tables, and roconnor's patch already handles that.
01:34 < real_or_random> roconnor: selftest.h : either C implementation-defined behavior we rely on, or we think there's a footgun, e.g,. we plan to support user overrides for the sha functions. so it makes sense to run a simple test. Do you think we should include a simple test for the tables? 
01:34 < gmaxwell> We might not want to expose all the choices in autoconf just for testing reasons.
01:35 < gmaxwell> Well I think there sould be much more comprehensive selftests, I believe I had a PR (now closed) to do so.
01:35 < real_or_random> gmaxwell: I don't see how the patch handles this. it literally has `#if ECMULT_GEN_PREC_N != 64 || ECMULT_GEN_PREC_G != 1
01:35 < real_or_random> #error`
01:36 < real_or_random> ah shiot
01:36 < real_or_random> I'm looking at the wrong table
01:39 < gmaxwell> For the signing tables sadly we don't get the same properties, so I think they'll ultimately need to have a couple options and switch.  But I think even for the vartime table, we should ultimately only support a couple options-- the others should work if you want to override the defines to get them-- but no promises.  Otherwise too much cpu time is wasted on combinitoric testing.
01:39 < real_or_random> nevermind. we should also rename the old table. -.- ecmult_static_context.h should be ecmult_gen_static_context.h ...
01:40 < gmaxwell> and I really doubt anyone needs 15 levels of table size... the speed differences between the larger sizes are just not that great.
01:40 < real_or_random> fully agree
01:40 < gmaxwell> I guess we're already violating file names best practices so I don't need to worry if ecmult_gen_static_context.h is too long. :P
01:41 < real_or_random> gmaxwell: what do you mean we don't get the same properties? we can still have a large file with all the tables for all supported options?
01:41 < gmaxwell> (the big violation we have is that there are files with the same name in different directories, and this is traditionally a source of nightmares for older VC versions)
01:41 < real_or_random> I guess "best" depends on everyone's personal experience :P
01:41 < gmaxwell> real_or_random: we can have one file with multiple tables sure, but we can't have one table with multiple options in it.
01:42 < gmaxwell> real_or_random: the vartime table is just the biggest table and the different options truncate it.
01:42 < real_or_random> yeah ok but I don't think this this should stop us from just having three tables
01:42 < gmaxwell> which is pretty elegant, all sizes supported with no overhead but a couple ifdefs.
01:42 < gmaxwell> Right sure!
01:43 < elichai2> The reason we don't want ecmult_gen_context to also be static to allow safe randomization, right?
01:43 < gmaxwell> elichai2: it's been static for a long time...
01:44 < gmaxwell> or do you mean the actual context and not just the table?
01:44 < gmaxwell> the actual context is just a few bytes, it fits on the stack, and the whole purpose of it (except for the table) is randomization.
01:45 < elichai2> oh right I forgot that the table itself is static with the PRECOMPUTATION define
01:46 < elichai2> hmm I guess I wonder if we can make the context somehow not opaque, so that it can easily be made on the stack, instead of guessing a size, asserting its correct, and then using the prealloc api
01:47 < elichai2> on the other hand that will make future changes to the context a breaking change
01:48 < gmaxwell> yeah, I've had bad expirence in other libraries w/ insufficient opacity making upgrading extremely hard (libogg is an example of this).
01:49 < real_or_random> ok tl;dr build system: we should a make target 
01:49 < real_or_random> roconnor: I'll see if you can contribute a patch to your PR
01:56 < elichai2> I wonder if after this we can change rust-secp's signing API to not require a context, create a context with unique randomization on the fly and use that for signing. and also for verification(but without randomization), because in rust-secp we override the callbacks in compile time, so we don't really use that feature. and now creating a context should be extremely cheap
01:57 < elichai2> there's still the sha256 self test, so we'll need to benchmark it, anyway sounds pretty great, Thanks roconnor :)
02:01 < gmaxwell> well, there could be (are?) flags to diable the selftest. And you could have some seperate call do do that selftest and do some rust magic to guarentee it gets called least once per execution.
02:01 < gmaxwell> but the randomization rolls data between uses, if you're freshly rerandomizing from scratch each time that is likely less secure.
02:01 < gmaxwell> The issue is that the attacker can get a trace of the randomization too.
02:02 < gmaxwell> If they have to restart the device for every attempt, then in some cases that will block their attacks or make them uselessly slow.
02:03 < gmaxwell> This may be even more true in the future because ideally every signing should do some cheap update of the randomization (e.g. add one of two constants at random chosen by some side effect random data during signing)
02:03 < gmaxwell> We really haven't done enough to protect against EMI sidechannels, it's just hard to justify doing more without a measurement rig to get an indcation that the countermeasures work.
02:05 < gmaxwell> but against totally reasonable bitcoiny threat models they do matter...  E.g. I rent the space next to IdiotExchange's value and point my bigantennas at the wall to get trances every time they use their trezor, then correlate that against signature on the blockchain.
02:05 < gmaxwell> Keep in mind, if I can get a few hundred signatures where I know the MSB of the nonce your key is compromised. (and by key I all keys sharing public derrivation).
02:07 < gmaxwell> and sure, idiot exchange should actually use their hardware wallet in an emi shielded enfiviroment, but emi shielding is so hard to get right that I'm confident that no one is successful.  (e.g. coinbase has some blog post showing using an emi tent, which is a rather low level of shielding, and they have AC powered devices in it, which likely means the shielding is compromised)
02:26 < elichai2> using fresh randomness each time might be less secure? that's interesting
02:34 < gmaxwell> yeah, because the random initilication is itself just an ecmult... from a totally unrandomized state.
02:35 < gmaxwell> so if ecmult leaks information (which we assume, otherwise why randomize) then the ecmult used to make the random state leaks information.
02:37 < elichai2> so why reusing the same randomization is more secure? if it was leaked the first time then it should be the same as fresh randomness, no?
02:44 < gmaxwell> not reusing the same randomization but adding more randomness.
02:44 < gmaxwell> when you call the randomize call it doesn't wipte out the state it combines the new randomness with the existing randomness.
02:45 < gmaxwell> so even if individual steps leak some information the state becomes more unpredictable.
02:45 < gmaxwell> Vs if you wipe it each time, it's never any better than the single shot result.
02:57 < roconnor> My understanding was that randomness goes out the window for the gen_context once multicomb gets implemented. ... That said I have little idea what gen_context does or what multicomb is and how it relates to randomness.
02:57 < roconnor> I pay very little attention to the signing side of the library unfortunately.
03:00 < real_or_random> no surprise that your focus is on verification :P
03:06 < gmaxwell> roconnor: no, not at all.
03:06 < gmaxwell> the only thing multicomb changes is that the unknown dl blinding can't be used but that isn't even random.
03:08 < roconnor> oh I didn't realize the blinding wasn't random.  It is NUMS?
03:08 < gmaxwell> roconnor: the randomness changes  xG into  rG+(x-r)G  and rG gets a random z.
03:08 < gmaxwell> yeah, there is a NUMS blinding. just so that basically the table rows all have a random point added to them and the last row has the negation of their sum added to it.
03:09 < gmaxwell> It's unclear if it does anything useful but it was totally free.
03:09 < gmaxwell> and that can't be preserved with multicomb because the usage is less structured.
03:09 < roconnor> I see.
03:10 < gmaxwell> but the randomZ(rG)+(x-r)G blinding can still be used, and thats the only part that is actually randomized.
03:11 < roconnor> what is r?
03:11 < gmaxwell> a random value stored in the context.
03:11 < gmaxwell> which can be updated with an api call.
03:12 < gmaxwell> When you call the API, IIRC, it hashes your input with the current r value.  Then uses the current r value to compute randomZ(rG)+(x-r)G where x is the new hash, and then that result replaces R. 
03:14 < gmaxwell> I think eventually there needs to be a cheaper way to update randomness, that can happen e.g. at every signature.
03:15 < gmaxwell> E.g. use the R.y sign to throw off an extra secret bit every time you sign.  And then use that bit to update the randomness by doubling and adding one init-time-random constant or another init-time random constant.
03:16 < gmaxwell> it's just really hard to justify doing a lot of work here without a measurement setup that tells us if we're makiking it _worse_.
03:17 < real_or_random> plus there are API questions, e.g., should _sign write to the context 
03:23 -!- roconnor_ [~roconnor@host-45-58-226-40.dyn.295.ca] has joined #secp256k1
03:24 < gmaxwell> I think it should, yes.
03:24 -!- roconnor [~roconnor@host-23-91-184-5.dyn.295.ca] has quit [Ping timeout: 268 seconds]
03:25 < gmaxwell> -- there isn't any efficient alternative, certantly not ones callers will get right.
03:26 < gmaxwell> Like telling callers to call randomize after every signature, when it has no visible effect just means it's it will never get done.
04:13 < real_or_random> yep, also it's a drawback the current context creation function doesn't have an argument for initial randomness
04:19 -!- johnzweng [~johnzweng@zweng.at] has joined #secp256k1
10:09 -!- roconnor_ [~roconnor@host-45-58-226-40.dyn.295.ca] has quit [Ping timeout: 240 seconds]
10:46 -!- roconnor [~roconnor@host-45-78-199-87.dyn.295.ca] has joined #secp256k1
11:49 -!- jesseposner [~jesse@2601:647:0:89:1597:3bb6:d9c9:249b] has joined #secp256k1
12:12 < roconnor> Peek memory use of gcc when compiling with ECMULT_WINDOW_SIZE 24 (in my PR) is somewhere north of 10 GB.  I had to kill it on my laptop so I could get back to work.  I might try again later.
12:13 < roconnor> The header file is 1.2G in size IIRC.
12:15 < sipa> Those are rookie numbers.
12:16 < roconnor> I know, but I'd like to eventally know that someone can compile WINDOW_SIZE 24.
12:16 < roconnor> I strongly suspect that it is compilable.
12:16 < sipa> happy to try
12:17 < roconnor> you can use my PR; just configure the window size upto 24.
12:17 < roconnor> ideally you'd record peek memory use, but I don't know how to do that.
12:17 < roconnor> and maybe isn't that important.
12:18 < sipa> that's easy
12:18 < sipa> run it in /bin/time
13:13 < gmaxwell> I think I posted benchmarks with up to whatever the largest actually working size is and thats why there is some comment in the code that says something about what actually works.
14:27 -!- roconnor_ [~roconnor@host-192.252-165-177.dyn.295.ca] has joined #secp256k1
14:28 -!- roconnor [~roconnor@host-45-78-199-87.dyn.295.ca] has quit [Ping timeout: 256 seconds]
15:03 -!- robertspigler [~robertspi@2001:470:69fc:105::2d53] has joined #secp256k1
15:12 -!- robertspigler [~robertspi@2001:470:69fc:105::2d53] has quit [Quit: node-irc says goodbye]
15:21 -!- belcher_ [~belcher@user/belcher] has quit [Read error: Connection reset by peer]
15:22 -!- belcher [~belcher@user/belcher] has joined #secp256k1
15:28 -!- robertspigler [~robertspi@2001:470:69fc:105::2d53] has joined #secp256k1
15:33 -!- roconnor_ is now known as roconnor
17:21 -!- belcher_ [~belcher@user/belcher] has joined #secp256k1
17:24 -!- belcher [~belcher@user/belcher] has quit [Ping timeout: 256 seconds]
--- Log closed Wed Jun 30 00:00:52 2021