--- Log opened Mon Jul 17 00:00:02 2023
02:29 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
02:35 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has joined #secp256k1
03:27 -!- jonatack1 [~jonatack@user/jonatack] has quit [Ping timeout: 245 seconds]
04:25 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has quit [Read error: Connection reset by peer]
04:44 -!- jonatack1 [~jonatack@user/jonatack] has joined #secp256k1
06:00 -!- jonatack2 [~jonatack@user/jonatack] has joined #secp256k1
06:00 -!- jonatack1 [~jonatack@user/jonatack] has quit [Read error: Connection reset by peer]
06:43 -!- preimage [~halosghos@user/halosghost] has joined #secp256k1
08:02 < real_or_random> sipa nickler hebasto: meeting :)
08:02 < sipa> hi
08:03 < nickler> hi
08:04 < nickler> topics?
08:04 < real_or_random> hi
08:04 < sipa> Using magic field for upgrading APIs? https://github.com/bitcoin-core/secp256k1/pull/1140#discussion_r1259467057
08:05 < real_or_random> can you summarize?
08:06 < nickler> "can we (ab)use the magic for this [avoiding incompatible API break]?" I hope so, that was the idea iirc.
08:06 < sipa> That PR adds a new field to the secp256k1_schnorrsig_extraparams struct, which is an API break, as old user code linked against a new library will cause the library to access out of bounds.
08:06 < sipa> Oh, that was the intent even!
08:06 < sipa> I was unaware.
08:07 < nickler> https://github.com/bitcoin-core/secp256k1/pull/844#issuecomment-765732776 
08:07 < nickler> "Instead it's a simple struct now with a magic value which can be used to achieve backward compatibility. If we decide to change the struct, we also change the magic which then acts as a version number and allows libsecp to distinguish between different structs."
08:08 < sipa> Great.
08:08 < nickler> sorry, I saw benma's comment, then forgot about it. Should've answered.
08:09 < real_or_random> so that means we'd need to upgrade the major version?
08:09 < real_or_random> or does it mean we don't need to do this? 
08:09 < sipa> If we use this change-magic approach, It means we don't, as I understand it.
08:10 < nickler> Yeah I don't think we need to do this.
08:10 < sipa> Because code, compiled against the old library headers, can be linked against a new library then.
08:10 < real_or_random> ah so the new lib code would handle both old and new callers? (by branching)
08:11 < sipa> Is it allowed to cast a pointer to struct1 to a pointer of type struct2, if struct1 is a prefix of struct2, and you never access the fields added by struct2?
08:12 < real_or_random> I think the standard says no
08:12 < real_or_random> in practice? idk
08:13 < real_or_random> I think the issue is that the standard doesn't guaranteed the sizes of padding between members
08:14 < real_or_random> it just guarantees that there's no padding before the first member. but AFAIK in practice, compilers are consistent about padding
08:14 < real_or_random> hm
08:14 < sipa> Using the old struct as a nested struct may work.
08:14 < nickler> I wonder if we've detailed how this is supposed to work, but it's not mentioned in the PR.
08:16 < real_or_random> I can't remember exactly what we thought back then, but I think the conclusion was that other programs do this (linux kernel?) 
08:16 < real_or_random> *do this somehow
08:17 < real_or_random> and yep, a nested struct could be a clean approach
08:18 < nickler> found this branch: https://github.com/jonasnick/secp256k1/commit/e0ec76d637c94fd46783586771895f4259ae0a3f
08:19 < real_or_random> a related question is we want have magic bytes for structs where we don't expect backward-compat issues
08:19 < real_or_random> I think secp256k1-zkp has magic bytes in all new datatypes  
08:19 < real_or_random> just for safety
08:20 < nickler> For new modules (musig) I followed the rule that all structs that are potentially uninitialized get magic bytes
08:21 < real_or_random> yep, I think it won't hurt. 
08:22 < sipa> https://stackoverflow.com/a/47710813/8342274
08:22 < sipa> Union of structs with common initial sequence, seems to be a thing.
08:23 < real_or_random> oh, unions also work
08:23 < real_or_random> good to know
08:26 < real_or_random> ah yes, and that's even a C89 thing
08:27 < real_or_random> "One special guarantee is made in order to simplify the use of unions: If a union contains several structures that share a common initial sequence, and if the union object currently contains one of these structures, it is permitted to inspect the common initial part of any of them. Two structures share a common initial sequence if corresponding members have compatible types for a sequence of one or more initial members. "
08:28 < real_or_random> more to discuss about this topic?
08:28 < nickler> the branch I posted above doesn't really apply to our sitauation anymore because it would've required the user to pass sizeof(extraparams) as an arg to sign_extraparams
08:30 < real_or_random> hm ok
08:31 < real_or_random> sipa: want to reply in the PR?
08:31 < nickler> I'll see if I can find more info on how it was supposed to work
08:32 < sipa> real_or_random: I can, but if you feel like digging into exactly what the spec allows, feel free too.
08:34 < real_or_random> I'm pretty confident that the above says all we need to know: unions work. nested structs work.
08:34 < nickler> Oh, I just found this "versioned struct playground" https://github.com/jonasnick/versioned-param
08:34 < real_or_random> sipa: maybe go ahead and reply and I can try to elaborate if necessary
08:35 < sipa> Sure, these mechanisms work, but I haven't thought through how that'd be used for the API.
08:35 < real_or_random> ok, I see
08:36 < sipa> Because there are two concerns; one is link-time compatibility with the library (old header new library, new header old library).
08:37 < sipa> The other is source code compatibility with user code.
08:37 < sipa> Changing a struct to a union (if it's not anonymous?) or a nested struct will break user code.
08:39 < real_or_random> ha yeah, hm, maybe we should have started with a union... hm
08:45 < real_or_random> ok, whatever, I can try to have a closer look ^^
08:46 < sipa> ok
08:48 < sipa> no other topics?
08:48 < fanquake> Just wanted to mention that we'll likely do a subtree pull, after #1378, to avoid any further CI failures.
08:49 < real_or_random> makes sense
08:50 < real_or_random> just as a reminder, next regular release should be around ~ september 1st
08:50 < real_or_random> so we should think about this in august
08:51 < real_or_random> https://github.com/bitcoin-core/secp256k1/milestone/4
08:54 < real_or_random> but no other topics from my side
08:58 < nickler> no other topics from me either
09:00 < sipa> same
09:14 -!- jonatack2 [~jonatack@user/jonatack] has quit [Read error: Connection reset by peer]
09:15 -!- jonatack3 [~jonatack@user/jonatack] has joined #secp256k1
14:07 -!- BlueMatt [~BlueMatt@ircb.bluematt.me] has quit [Ping timeout: 260 seconds]
14:10 -!- BlueMatt [~BlueMatt@ircb.bluematt.me] has joined #secp256k1
14:39 -!- preimage [~halosghos@user/halosghost] has quit [Quit: WeeChat 4.0.2]
15:57 -!- sanket1729_ [~sanket172@ec2-100-24-255-95.compute-1.amazonaws.com] has quit [Read error: Connection reset by peer]
16:00 -!- sanket1729 [~sanket172@ec2-100-24-255-95.compute-1.amazonaws.com] has joined #secp256k1
20:28 -!- BlueMatt [~BlueMatt@ircb.bluematt.me] has quit [Quit: Quit]
20:29 -!- BlueMatt [~BlueMatt@ircb.bluematt.me] has joined #secp256k1
--- Log closed Tue Jul 18 00:00:02 2023