--- Log opened Mon Mar 11 00:00:23 2024
00:49 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has joined #secp256k1
05:38 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
07:39 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has joined #secp256k1
07:57 < sipa> i won't be available for meeting today (an hour later works, but don't have anything to bring up)
08:01 < nickler> hi
08:01 < real_or_random> ok. I think at some point we should have a conversation about the silent payment PR and the API questions I raised there
08:01 < nickler> ^
08:01 < real_or_random> but this should involve at least all the maintainers ^^
08:01 < real_or_random> nickler: ?
08:02 < nickler> I set the date of the next release milestone to May 1st, 6 months after the last release (cc fanquake)
08:02 < fanquake> 👀
08:02 < real_or_random> nickler: okay with postponing an hour, or prefer to do the meeting now?
08:03 < nickler> real_or_random: agree that we should have the conversation
08:03 < nickler> at some point
08:03 < real_or_random> :D
08:04 < nickler> I can do it in an hour too. Not sure if I have much to contribute though.
08:04 < nickler> Probably best to give the silent payments folks feedback sooner than later
08:06 < real_or_random> well yeah, I don't think we can finish the full discussion here in a single meeting on IRC, but at least we can start the conversation. then I'd say let's try again in an hour?
08:07 < nickler> it seems like the latest state is that josie and theStack are working on a) benchmarks and b) evaluating josie's API proposal (which looks much more like a secp module)
08:07 < nickler> real_or_random: ok
08:13 < josie> happy to join for the silent payments topic, if needed
08:33 -!- Guest65 [~Guest65@node-1w7jr9upc4rt7n3exzmcvvdim.ipv6.telus.net] has joined #secp256k1
08:34 < Guest65> How do I use this ?..
08:39 -!- Guest65 [~Guest65@node-1w7jr9upc4rt7n3exzmcvvdim.ipv6.telus.net] has quit [Ping timeout: 250 seconds]
08:49 -!- preimage [~halosghos@user/halosghost] has joined #secp256k1
08:51 < sipa> i'm available now
09:00 < real_or_random> hi
09:00 < josie> hi
09:00 < sipa> hi
09:01 < nickler> re-hi
09:04 < real_or_random> sipa: did you have a look at the silent payments PR in the past? 
09:05 < sipa> i have not
09:05 < sipa> i saw some discussion about how high-level the API should be
09:05 < sipa> but i haven't looked at the code
09:06 < real_or_random> ok
09:07 < real_or_random> I know jonas had a glimpse, too. I think that suffices to start a conversation
09:07 < real_or_random> I haven't had a detailed look at the code either
09:08 < nickler> I only looked at the API and skimmed some code
09:08 < real_or_random> I think there are two main topics here
09:09 < real_or_random> 1) to what degree do we expect a module to be high-level? and how does that match SP?
09:10 < real_or_random> 2) independently of SP, should we tidy our tweak / pubkey_combine pseudo-high-level functions and move them into a new separate module?
09:11 < sipa> well
09:11 < real_or_random> these are different questions, but they're not entirely unrelated. if we implement 2), that would make it possible  (or easier) to build SP on top of a low-level module. this may or may be better.
09:12 < real_or_random> may  or may NOT :D
09:13 < sipa> i like the idea of moving to safer APIs
09:14 < sipa> including bip32 etc APIs
09:14 < real_or_random> not sure where to start .... I think for 1), josie and theStack are currently exploring options to encapsulate the cryptographic core  in more well defined borders than the current PR, and that sounds like a good idea anyway, and I guess we simply need to see how this evolves? 
09:15 < real_or_random> sipa: I mean, the idea to build a bip32 module has been there for a while. but noone worked on it... it will certainly be useful 
09:16 < real_or_random> but what I was contemplating about in the PR wrt to 2) was mostly orthogonal
09:16 < nickler> What's really nice in the current SP module is that it looks very easy to use by wallet devs wanting to implement SP thanks to the API and docs.
09:16 < real_or_random> yep indeed
09:18 < nickler> ..but on the other hand it may introduce unnecessary maintenance burden to add 9 functions most of which are very simple (such as just scalar_add)
09:18 < real_or_random> sipa: it turns out that we anyway have some low-level functions such was the aforementioned pubkey_combine, and I bet some people rely on them. my thinking is: we don't like them too much. if we have them anyway, why not provide cleaner low-level functions, with a proper "use this only if you know what you're doing" warning
09:19 < sipa> real_or_random: that's a good question
09:19 < real_or_random> I recently saw this in some python crypto library where they call it a "hazmat" API / module.
09:20 < nickler> according to my notes every time we discuss exposing the "internals" we land on "well this could generally be useful, but we should benchmark it"
09:20 < real_or_random> I believe, at a minimum, we could add warnings to the existing functions (and I don't think this will be controversial)
09:20 < real_or_random> nickler: right, and it seems that theStack is doing some work in this direction wrt to sp?
09:21 < real_or_random> performance matter a lot for sp, but my intuition is that access to deep secp256k1 internals wouldn't give you much of a benefit. 
09:22 < sipa> ok
09:22 < nickler> pubkey_combine/*_tweak is probably relied on pretty commonly... I recently learned that this is how the "cashu" ecash project implements David Wagner's DH-based blind sigs.
09:23 < real_or_random> a simple exposure of scalar/privkey, and ge/pubkey with add, sub, etc. could be enough, and also help other protocol builders 
09:23 < sipa> maybe we should move those to a hazmat API, then
09:24 < sipa> a question is what level such an API should have
09:24 < sipa> it could deal with pubkeys and seckeys etc
09:24 < real_or_random> that would be chance to rethink them a bit. the functions are a bit strange when it comes to 0/infinity. for example pubkey_combine is not associative, and tweaking is not commutative nor associative...
09:24 < sipa> or it could have abstract field and point structures, which don't force normalization/affinemaking
09:25 < real_or_random> sipa: right. that's very much related to what I just said. so far we had pubkey/privkey functions, that why we yell at the user if we reach infinity
09:25 < sipa> right
09:26 < real_or_random> I tend to think that moving to the field/ge is a bit cleaner, with conversion functions to pubkey/privkey afterwards. because then the algebra is clean at least
09:26 < sipa> could do ge, without exposing fe
09:27 < real_or_random> yep indeed
09:27 < sipa> because there is a cost to normalizing an FE, but it is perhaps so small that it's not significant compared to a function call
09:27 < real_or_random> right... also FE is very low-level. you usually don't need this for protocols
09:28 < real_or_random> (in the sense that most protocols work on a generic group) 
09:28 < sipa> it matters for things like ellswift, if you want to do it with similar efficincy outside the library
09:28 < nickler> what about gej? I guess anytime you want to use that it'd be better to write a high level function
09:28 < sipa> could even only expose gej
09:28 < sipa> well, abstracted through a "point" typr
09:29 < nickler> true
09:29 < real_or_random> true, yeah
09:29 < sipa> it could even remember internally whether a z is 1, and then use ge logic
09:29 < sipa> though the branching may be undesirable if timing matters
09:29 < sipa> another question: is hazmat guaranteed to be const time?
09:30 < sipa> or not?
09:30 < sipa> or will it have a mix?
09:30 < real_or_random> I believe this is worth looking into. but as all these conversations (remember context API etc), it's only meaningful if someone wants to do the work ^^ 
09:30 < sipa> yeah
09:30 < real_or_random> yeah, that's another good technical question.
09:32 < real_or_random> the thing is ... if it turns out that this is is a good way to implement sp (and other stuff in the future), it may save us (the project here) time in the future.  
09:32 < real_or_random> (by pushing the work to others ^^ but that may be a reasonable thing ^^)
09:34 < real_or_random> nickler created this PoC for -zkp. it just exposes scalar. https://github.com/BlockstreamResearch/secp256k1-zkp/pull/153 it's very simple. of course, scalar is simpler than ge ^^ 
09:34 < sipa> yeah
09:34 < real_or_random> but still, I expect the thing to be a rather thin layer. we have all the code somewhere, it's more about exposing it in the right way
09:37 < real_or_random> if we agree that this is worth looking into, perhaps theStack wants to have a closer look? he said in the PR: " After thinking about it for a while, I sympathize with the "libsilentpayments" idea more and more." 
09:38 < nickler> When I wrote the code I thought that this would end up being an "unstable" part of the lib. But I think we actually won't need to change the API often (besides adding stuff perhaps).
09:39 < real_or_random> nickler: right, the same occurred to me. when we had this discussion in the past, we considered guaranteeing no stability at all/making the API version the  commit hash
09:39 < sipa> we can still do that for hazmat, i think
09:40 < real_or_random> and that would make sense if we literally exposed the current gej_, but I don't think that will be required to that extent
09:41 < nickler> real_or_random: yes I still think that this is worth looking into
09:41 < sipa> right
09:42 < real_or_random> if we start with a few basic functions, I don't expect API stability to be so much of an issue. I mean we won't expose secp256k1_gej_add_zinv_var ^^
09:42 < sipa> right
09:43 < real_or_random> okay, I think then the next step may be to have theStack's opinion on this. not sure if he made any progress on the benchmarks
09:44 < nickler> josie: for prioritization, to what extent is adding SP functionality to libsecp (in the form of the current PR or hazmat) blocking SP progress in Core? 
09:44 < real_or_random> (and how he wants to run the benchmarks... I mean that involves creating some form of a hazmat API, if even just as a very simple hack) 
09:45 < real_or_random> yep, that's a good question
09:46 < josie> nickler: i think there was general agreement that having the low level crypto in core was not ideal (i.e. adding methods to CPubKey and CKey for adding pubkeys/privkeys etc), so id say the core work is blocked for now by the libsecp work
09:48 < josie> one thing thats not clear to me: if the hazmat api is the preferred approach, it seems we'd be back to square one with bitcoin core consuming he hazmat api and adding pubkeys/privkeys etc. unless the assumption is something like bitcoin core -> libsilentpayments -> hazmat api? 
09:49 < sipa> oh my impression here was about hazmat basically being unrelated to sp
09:49 < sipa> if you're already willing to work on a real high-level sp module in libsecp256k1, that's far better
09:49 < real_or_random> oh I mean my point of view was that both options are possible:
09:50 < sipa> the reasoning for having hazmat is because it appears inevitable that some - other - users want to do stuff like that anyway, even abusing pubkey tweaking APIs etc for it, which is definitely worse
09:51 < real_or_random> first, hazmat makes sense (or doesn't) independently of sp. but if we decide that it makes sense, there are two options for sp: a libsecp module or a separate lib  
09:51 < sipa> right
09:51 < real_or_random> sipa: do you think a sp module is clearly superior?
09:52 < sipa> well the trade-off is that an sp module would be blocked by libsecp review
09:52 < sipa> but if that's acceptable (assuming reasonable interest from maintainers etc), i think an sp module is preferable to a separate library
09:52 < real_or_random> I mean one question then is who will maintain it. IF it's going to be used in bitcoin core in the future, then it would live somewhere in the bitcoin core project  
09:53 < real_or_random> and I see that it doesn't make that much of a difference in that case
09:54 < real_or_random> a separate lib may be preferable for things for which there's no clear maintainer buyin
09:55 < real_or_random> (which I think we have here in the sense that noone is opposed to a module. it's more of a question which approach is the best from a technical point of view) 
09:56 < real_or_random> anyway, I'd be interested to hear if theStack was able to run some benchmarks, no matter if a separate lib is the way or not. ^^ 
09:57 < real_or_random> and  there's not much more to discuss from my side (at this point in time) 
09:58 < real_or_random> more points?
09:58 < sipa> it all sounds pretty speculative here :)
09:59 < nickler> the only reason why there wouldn't be a clear secp maintainer buyin is because as designed right now it's not a "crypto primitive"?
09:59 < nickler> (which may change with josie's API redesign)
10:00 < nickler> in any case sounds like we should put SP on the release milestone :)
10:00  * nickler needs to leave now
10:00 < real_or_random> nickler: for me SP ticks all the boxes in https://github.com/bitcoin-core/secp256k1/blob/master/CONTRIBUTING.md#adding-new-functionality-or-modules
10:00 < real_or_random> even in its current form
10:01 < real_or_random> the question of maintainer buy-in then (for me) is more whether we want to spend the time reviewing and maintaining it. and this seems to be the case
10:02 < real_or_random> I also need to leave :)
10:04 < sipa> end of meeting, i guess!
10:53 < theStack> real_or_random: i haven't started with the sp benchmarks yet
10:53 < theStack> the rough idea there would be to focus first on sp functionality that is most performance-critical in practice (that is the receiver side / scanning, i.e. the functions _create_public_tweak_data, _create_shared_secret and _create_output_pubkey)
10:54 < theStack> for those functions i'd implement alternative variants which only use the public secp256k1 API (e.g. using pubkey_combine for the pubkey additions)
10:55 < theStack> then see how those perform, compared to the functions as they are implemented in #1471; then, if there's significant difference, investigate deeper what causes that etc.
10:55 < theStack> does that approach make sense?
10:55 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
10:56 < theStack> i guess the critical part could the pubkey adding here (at least for txs that have a high number of silent payments elligible inputs)
10:56 < theStack> oh, and probably the tagged hashing is also noticably slower with a public API, as there is no way to pass a midstate to `secp256k1_sha256_tagged`... maybe also something we should talk about next time
10:57 < theStack> anyway, i'm positive to have some benchmark numbers ready within the next few days (hopefully useful ones :))
10:58 < theStack> fwiw, i still like the "libsilentpayments" idea, but having a secp module in the spirit of what josie proposed sounds also like a very charming idea to me
10:58 < theStack> bitcoin core -> secp256k1-silentpayments seems overall easier to maintain than bitcoin core -> libsilentpayments -> secp256k1-hazmat?
11:01 < real_or_random> ok yes, using the existing functions makes sense if you want to obtain benchmarks
11:03 < real_or_random> I mean, if we're converging to a module anyway, then we wouldn't need the benchmarks. but I think they're still interesting. if this ends up as just as a small case study on whether a hazmat module would be meaningful at all (even if we choose another route for sp), that's also good. 
11:10 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has joined #secp256k1
11:17 < josie> real_or_random, theStack: based on this discussion, seems like the benchmarks are a nice to have but not a prerequisite for pursuing a high-level module
11:19 < josie> fwiw, unless the hazmat api approach is significantly faster, i think id prefer a slightly slower but safer to use high-level module to a slightly faster hazmat api for sp
11:24 < sipa> how could hazmat be faster? it'd need strictly more function calls and marshalling than an sp module would
11:40 -!- ajonas [sid385278@helmsley.irccloud.com] has quit [Read error: Connection reset by peer]
11:40 -!- ajonas [sid385278@id-385278.helmsley.irccloud.com] has joined #secp256k1
11:40 -!- stickies-v [sid544753@uxbridge.irccloud.com] has quit [Read error: Connection reset by peer]
11:40 < josie> sipa: for sending and receiving without labels, a high level mod should always be faster for the reasons you mentioned. for scanning with labels, my intuition is it may be slighter faster providing a few low level functions vs a safe to use high level function since scanning with labels requires calculating `potential_label = pubkeyA - pubkeyB`, and then checking if `potential_label` matches the
11:40 < josie> receivers label set before proceeding
11:40 -!- stickies-v [sid544753@id-544753.uxbridge.irccloud.com] has joined #secp256k1
11:41 < sipa> josie: but an sp module could have a high-level API for doing the label scanning operations too, and it'd be faster than using hazmat as it can avoid the marshalling?
11:42 < sipa> (only marginally faster, probably, but i don't how it could be slower)
11:48 < josie> sipa: yes! altho, its not immediately clear to me on how to make a high-level label scanning api without exposing a function that could be abused for adding arbitrary pubkeys
11:48 < josie> so in making the high level functions hard to abuse, you might miss out on some "early stopping" opportunities
11:49 < josie> but i havent spent that much time thinking about it (and could also be mising something obvious / lacking imagination)
11:51 < sipa> josie: right, i think this is a benefit of conceding the addition of a hazmat API, even if unused for the purpose of SP: it means we don't need to restrain ourselves from adding functions that might be abused - if people want to abuse the API, they can use hazmat instead
11:55 < josie> sipa: thats a great point i hadnt considered. if the hazmat api exists, your functions for module X only need to be slightly more difficult to use than the hazmat api to prevent people from using them for operations unrelated to protocol X 
12:02 < theStack> real_or_random: josie: yeah, agree, if we follow the sp module approach the benchmarks are not that relevant. my initial motivation for the benchmarks was to see if we could get away with neither having a hazmat API nor a new secp256k1 module, by having a libsilentpayments only using the current public secp256k1 API. just curious how much slower that really would be and where the most performance is lost
12:37 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
13:11 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has joined #secp256k1
13:54 -!- preimage [~halosghos@user/halosghost] has quit [Quit: WeeChat 4.2.1]
15:09 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
17:45 -!- jonatack [~jonatack@user/jonatack] has quit [Ping timeout: 272 seconds]
17:47 -!- jonatack [~jonatack@user/jonatack] has joined #secp256k1
20:21 -!- Guest65 [~Guest65@node-1w7jr9upc4rt7n3exzmcvvdim.ipv6.telus.net] has joined #secp256k1
20:21 -!- Guest65 [~Guest65@node-1w7jr9upc4rt7n3exzmcvvdim.ipv6.telus.net] has quit [Client Quit]
23:23 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has joined #secp256k1
23:33 -!- tromp [~textual@92-110-219-57.cable.dynamic.v4.ziggo.nl] has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
--- Log closed Tue Mar 12 00:00:24 2024