--- Log opened Mon Jul 04 00:00:11 2022
03:20 -!- gnusha [~gnusha@user/gnusha] has joined #secp256k1
03:20 -!- Topic for #secp256k1: libsecp256k1 development discussion | https://github.com/bitcoin-core/secp256k1 | Channel logs: http://gnusha.org/secp256k1
03:20 -!- Topic set by real_or_random [] [Tue Jun  1 01:19:08 2021]
03:20 [Users #secp256k1]
03:20 [ ajonas      ] [ cfields_ ] [ gnusha       ] [ larryruane    ] [ real_or_random__] [ shiza       ] 
03:20 [ andytosh1   ] [ darosior ] [ harding      ] [ lightningbot  ] [ realtbast[m]    ] [ sigurdson[m]] 
03:20 [ Apocalyptic_] [ DeanGuss ] [ hebasto      ] [ luke-jr       ] [ robertspigler   ] [ sipa        ] 
03:20 [ ariard      ] [ elichai2 ] [ jesseposner  ] [ meshcollider  ] [ robot-dreams    ] [ siv2r[m]    ] 
03:20 [ b10c        ] [ fanquake ] [ jnewbery_    ] [ michaelfolkson] [ rockhouse       ] [ stickies-v  ] 
03:20 [ berryyoghurt] [ FelixWeis] [ johnzweng    ] [ midnight      ] [ RubenSomsen     ] [ uasf        ] 
03:20 [ BlueMatt    ] [ fjahr    ] [ jonasschnelli] [ Nebraskka     ] [ sanket1729      ] [ waxwing_    ] 
03:20 [ BlueMatt[m] ] [ ghost43  ] [ kanzure      ] [ nickler       ] [ sebx2a          ] [ windsok     ] 
03:20 [ calvinalvin ] [ glozow   ] [ laanwj       ] [ real_or_random] [ sgiath          ] [ yakshaver123] 
03:20 -!- Irssi: #secp256k1: Total of 54 nicks [0 ops, 0 halfops, 0 voices, 54 normal]
03:20 -!- Channel #secp256k1 created Wed May 19 12:44:13 2021
03:22 -!- Irssi: Join to #secp256k1 was synced in 148 secs
07:47 < real_or_random> hi, we're trying to establishing bi-weekly dev meetings on IRC, on Mondays at 17:00 CET / 11:00 ET  (that's 15:00 UTC currently)
07:48 < real_or_random> so that's in < 15 min... :D I know it's on short notice for today, but we just managed to agree on this slot among maintainers. so it's on short notice for today for everyone else who wants to be involved  but then you have two weeks to prepare for the next occurence
07:59 < sipa> excited
07:59 < real_or_random> lol
08:01 < real_or_random> okay let's do this :) randomly pinging some recent contributors elichai2 hebasto berryyoghurt in case they're around and interested 
08:01 < real_or_random> siv2r[m]
08:01 < hebasto> hi
08:01 < nickler> hi
08:01 < fanquake> hi
08:01 < sipa> maybe robot-dreams, fanquake, roconnor 
08:02 < siv2r[m]> Hi
08:02 < real_or_random> yep!
08:02 < real_or_random> so there are multiple reasons I'm interested to start these meetings 
08:02 < sipa> nickler
08:02 < sipa> ;)
08:02 < nickler> :o
08:02 < real_or_random> 1) some things are just easier to discuss "in person" in a chat then on github
08:03 < real_or_random> 2) we could try to coordinate our contributions/reviews better, e.g., we could select PRs with a high prio for review
08:04 < real_or_random> (maybe high prio is the wrong word here but if we agree to review a single PR, then it may get enough ACKs. instead of getting one ACK, and then falling behind again)
08:04 < real_or_random> any other things you'd expect from a meeting?
08:04 < sipa> Maybe also (3) getting an idea of what people are working on... that may help garner interest from other contributors to build momentum.
08:04 < real_or_random> (in general)
08:04 < real_or_random> that's good, yes!
08:06 < real_or_random> on logistics, we may want to add a meeting bot etc later but I don't think it's crucial. and we may want to find another slot if there's need. and we'll see if doing this every two weeks is good. if not, it's easy to adjust
08:06 < sipa> And perhaps a topic to bring up the first few meetings: is this a good meeting timeslot?
08:07 < real_or_random> yep, and obviously if there are people who prefer another slot, they probably won't be able to complain right now. so if you read this later, feel free to tell us after the meeting
08:07 < sipa> Right.
08:09 < sipa> So should we start by gathering topics?
08:09 < real_or_random> for concrete topics, I'd have a bunch of vague suggestions on my list. Release is always an interesting topic, scope of the library is interesting (but probably a long debate), a discussion on cmake would be helpful for hebasto (to avoid wasting time on this, if we're actually not interested)  
08:09 < real_or_random> (so I'm not proposing "scope" today as a topic ^^)
08:10 < nickler> https://github.com/bitcoin-core/secp256k1/pull/1055 is the most popular PR as measured by heart emoji
08:10 < sipa> Asking around what people are working on, which PRs seem perhaps close to merge or deserve prioritization/review/...
08:11 < sipa> Maybe a bit more specific than a general philosophical question about what the scope of the library should be, we could make it a habit to discuss specific new proposed features/modules as whether they belong in the library.
08:12 < real_or_random> if you ask me, https://github.com/bitcoin-core/secp256k1/pull/1000 (Synthetic int128 type) is probably in a very good shape and I'd like to avoid that roconnor needs to rebase this every two months 
08:12 < real_or_random> sipa: right, that's a good point
08:13 < sipa> Yeah, we should get that int128 in. I'll try to prioritize reviewing it.
08:13 < real_or_random> yes, me too...
08:13 < nickler> me2
08:14 < sipa> It's also a step towards removing the performance penalty due to lack of 64-bit mul on MSVC.
08:14 < real_or_random> right, that's probably the main motivation on our side
08:14 < real_or_random> for https://github.com/bitcoin-core/secp256k1/pull/1055 (release) , the question is really if there's anything that stops us from releasing... in the past we had lists of things we wanna do before the release, but that's probably the wrong approach as it will never be ready
08:15 < real_or_random> if you ask me right now, there's just one thing where I'm not sure if we should have this in before a release:
08:15 < sipa> For myself, I'd like to see some progress on 1066, but happy to wait until after 1000, if we just want a single focus point.
08:16 < real_or_random> we had a bunch of API changes regarding the contexts and precomputation (context flags are deprecated/ignored, there's basically no special context anymore except for the static contexts)
08:17 < real_or_random> but those API changes haven't been documented yet. at the moment the API docs aren't "wrong" (you can still pass flags and they will do the job) but probably misleading.
08:17 < nickler> real_or_random: the only lists I'm aware of is this https://github.com/bitcoin-core/secp256k1/milestone/1 and this is for a 1.0 release, not 0.X release
08:18 < real_or_random> that's a point where the code is currently somewhere in half-done state in the "middle", so this is the only thing where I'm not sure if we want to have these doc changes in first
08:18 < real_or_random> nickler: indeed, yeah, those are for a 1.0 release
08:19 < real_or_random> you could also argue the other way around: if we tag a release now, it'll be easier to document the changes in a futher release (changelog), even though these are then actually only doc changes
08:20 < sipa> I'm happy with a 0.x release first with lower requirements, but something that's been holding back my enthousiasm for doing so is the fact that it feels like we're very close to 1.0 already, so it feels nice if the first release could be something like a 0.9 or 1.0rc immediately.
08:20 < sipa> Perhaps that's the wrong way to go about it, and we should do it.
08:20 < sipa> *just do it
08:20 < nickler> I don't think we're close to releasing 1.0 to be honest.
08:21 < sipa> I'm mostly talking in terms of API.
08:21 < sipa> And public interface in general
08:21 < nickler> Not that there's a lot missing right now that I could point to, but in the process of getting closer we'll find more and more blockers. At least that's the historical experience.
08:22 < sipa> That's fair.
08:22 < real_or_random> hm, I think when it comes to the public API, there are some larger changes we'd like to make (new context API) but that's a non-trivial thing
08:22 < sipa> If so, I think it would be good to make those changes, or at least have a plan about them, before a releasem
08:23 < sipa> The mutable context/state for signing is interesting, and with herzbleed perhaps even more relevant now
08:23 < sipa> *hertzbleed
08:23 < real_or_random> https://github.com/bitcoin-core/secp256k1/issues/780 is the issue on this
08:24 < real_or_random> hm yes, but I'd also fear that this pushes a release far into the future 
08:25 < sipa> That depends on prioritization :)
08:25 < nickler> I don't see why we need to have planned out the context api before doing a 0.1 release. Lots of users seem to have problems right now without a proper release. No one is helped by pushing it out further.
08:25 < real_or_random> sipa: why do you think it'll be nice to keep the span between a "current API" release and "nextgenAPI" release short?
08:26 < real_or_random> I agree with both of this... it depends on prioritization but even if we get this right, I agree with nickler that I don't see the advantage of having the plan ready
08:26 < sipa> By "close" I mean in terms of API changes, not time.
08:27 < sipa> But I'm a bit wary about doing a release now, if we already intend to change the API.
08:27 < real_or_random> oh ok but can you elaborate?
08:27 < real_or_random> wouldn't it be better to release now then? because the release process gives us a proper way to deal with API breakages?
08:28 < sipa> That's a fair point. Yeah, maybe this intuition of wanting to not have planned API changes is wrong.
08:28 < real_or_random> I think at the moment we really avoid breaking the API because we don't have a process of documenting API changes
08:29 < sipa> My fear is that once we release, even if it is a 0.1, people will start using that release as stable, because in practice it already has been for so long... and then immediately changing it again, we'd have been better off making the change firstm
08:30 < real_or_random> ok yeah, I see that point.
08:30 < sipa> But if we don't expect that an API change, even if it is envisioned, won't happen quickly, maybe that's not a big concern.
08:30 < real_or_random> on the other hand, based on historical evidence, I'm pretty sure we won't have that API change too quickly ^^
08:30 < elichai2> Hi
08:30 < real_or_random> hi elichai2 
08:31 < sipa> So maybe my point is rather that if we plan an API change, we should do it sooner rather than later - regardless of whether that's before or after release.
08:31 < nickler> I don't see that point, because if you view people as behaving arbitrarily stupid (like treating 0.1 as stable??) then everything is lost anyway
08:31 < nickler> these people probably treat it as stable already, regardless of whether we tack a version number onto it
08:32 < sipa> But a reason to have a discussion is maybe that we see a way to do these changes in a backward-compatible way, in which case it all doesn't matter much.
08:33 < sipa> In any case, is your thinking then that we can basically do 0.1 right now, @nickler?
08:34 < sipa> Or are there some things you'd like to see beforehand still?
08:35 < real_or_random> hm I think at least some of things will require breakage, e.g., trying to simply the callbacks (though that's not that important)
08:35 < nickler> sipa: I'm not aware of a super important blocker for the release (nor has anything been mentioned in the PR)
08:36 < sipa> I guess let's do it then
08:36 < sipa> I'll review the PR again.
08:36 < real_or_random> the most important thing we want is "automatic" randomization. this is in theory doable with the current context...  
08:37 < hebasto> is package/library versioning documented somewhere?
08:37 < real_or_random> what about the point I brought up? (documenting the API changes) is this a blocker?
08:37 < nickler> sounds good. real_or_random feel free to mention the API inconsistencies you brought up earlier in the PR
08:38 < sipa> Some way of marking "i promise this context is only accessed from a single thread at a time, even when using a const pointer" could be sufficient for automatic randomization.
08:38 < nickler> real_or_random: would need to think about what exactly this entails but it seems to me that this would be more like a nice to have (of which we have plenty)
08:38 < real_or_random> I tend to say I'd like to see it in but it's not a blocker. I "I need to do this until the end of the week" or we
08:38 < real_or_random> * I'd be happy with ""I need to do this until the end of the week or we do it after the release"
08:39 < real_or_random> this gives me good a reason to work on this finally
08:39 < sipa> How about until the next meeting?
08:39 < nickler> hebasto: yes, see https://github.com/bitcoin-core/secp256k1/pull/1055
08:40 < real_or_random> hebasto: https://github.com/bitcoin-core/secp256k1/pull/964/files added a release procress
08:40 < real_or_random> sipa: until the next meeting is also good
08:40 < real_or_random> (for me)
08:40 < hebasto> ty
08:40 < nickler> to be clear I don't think we need to ultra-rush through this initial release, so if we do have simple things that we want to do before then we should do it
08:41 < nickler> so next meeting sounds good to me as well
08:41 < real_or_random> (I'd  still document the changes in the changelog then, it's then just a change from "pre-released" to 0.1. but it'll be at the right point in time in the changelog then)
08:41 < real_or_random> ok this all sounds good
08:42 < sipa> That's also a good reason to have a release sooner, to get a clear changelogm
08:43 < real_or_random> ok :D nice! next topic? cmake maybe? :)
08:44 < hebasto> it would be nice :)
08:44 < sipa> I haven't followed in detail, but it looks like there is good progress.
08:45 < real_or_random> yeah I think the main reason why I bring this up is more to get a few more Concept (N)ACKs
08:45 < real_or_random> at least I'm the only one who commented in the PR so far
08:45 < nickler> where do I get started with cmake? or should I just read the PR and the doc side-by-side?
08:46 < sipa> So the plan is maintaining both in parallel?
08:46 < hebasto> nickler: https://github.com/hebasto/secp256k1/blob/220628-cmake/README.md#building-with-cmake
08:46 < real_or_random> my reason for concept ACK is still that 1) this has been requested a few times from users, so there's demand. 2) if core will help us, maintaining is less of a burden, and 3) it doesn't need to replace autotools with all of its advanced stuff
08:47 < nickler> hebasto: thanks
08:47 < real_or_random> sipa: that's the thing I'd imagine at least. I'd say dropping autotools would piss off a few people. and so far the cmake PR doesn't add really that many lines of code
08:47 < sipa> Right, it can target a smaller (but common) subset of configurations, if autoconf remains maintained.
08:47 < sipa> And I guess we want to maintain autotools anyway for compatibility requirementsm
08:47 < real_or_random> for example, we have functionality in autotools that help the maintainers rebuild the precomputed files. no need to replicate this in autoconf
08:48 < sipa> Right.
08:48 < real_or_random> or my recent PR that adds "wine msvc" cross builds. this is mostly for CI, and CI can do this with autotools  
08:48 < nickler> hebasto: but I meant more "getting started with reading CMakeLists.txt"
08:49 < real_or_random> yeah I'd think that dropping autotools would make a few people unhappy. and it would be a bad move, I mean it's working fine so far
08:49 < hebasto> nickler: maybe https://cmake.org/cmake/help/book/mastering-cmake ?
08:50 < real_or_random> hebasto: would there be any benefit from us having cmake if core switches? or doesn't this matter? 
08:50 < hebasto> it doesn't matter
08:50 < hebasto> (mostly)
08:50 < real_or_random> also for cross builds?
08:50 < sipa> I'd hope it would mean that Bitcoin Core needs no more manual build config for libsecp256k1?
08:51 < fanquake> it matters in the sense that we'd have to maintain our own cmake build system for libsecp256 downstream, otherwise our switch to cmake isn't really going to be one, if we still have to call out to autotools for libsecp
08:51 < fanquake> Although that is predicated on the secp256k1 cmake build system being equivalent enough to the autotools systems that it covers all our uses
08:52 < sipa> Right now it's sort of 50/50, with part of the build config in Bitcoin Core, but reusing the secp256k1 Makefile.include, right?
08:52 < hebasto> for now, it is implemented as https://github.com/hebasto/bitcoin/blob/c928933896c0731f9d38985fec7ee273a940a6f0/cmake/subtree-secp256k1.cmake
08:53 < real_or_random> ok well if you say that you'd anyway maintain a cmake config, then there's no reason not to have it in our repo
08:53 < hebasto> riight
08:53 < hebasto> currently, it is core-specific
08:53 < fanquake> I say that base on it actually working. Which I think is still unclear to me. i.e in the cmake file posted above, does cross-compiling "just work"?
08:54 < fanquake> If I'm building on aarch64, and targetting riscv-linux, does that work? Or do we end up needing to maintain endless toolchain files?
08:54 < hebasto> not endless, but for supported platforms
08:55 < fanquake> currently we build releases for 9 (iirc) different hosts, on x86_64, arm64 and riscv, and that all works nicely with autotools
08:55 < sipa> We need a separate file for each supported platform?
08:55 < hebasto> yes
08:55 < sipa> yuck
08:56 < hebasto> or provide options manually, as we do with autotools `--host`
08:56 < hebasto> android ndk provides its own toolchain file which we are going to use
08:57 < fanquake> right, so the toolchain files would also be somewhat duplicated between libsecp256k1 and core then, as libsecp256k1 would need it's own toolchain files, just as we would
08:57 < fanquake> although that is somewhat of a separate issue
08:58 < fanquake> or maybe we can provide the toolchain input, and leave libsecp to just maintain the minimum required cmake stuff? Need to do some more reading
08:58 < sipa> i don't really understand the point/purpose of these files, but I'll read up more.
08:59 < real_or_random> well I see. I think it's okay for us to provide some "example" tool chain files and those can happen to be the ones used by core
08:59 < real_or_random> but replicating them would be not so nice, yes...
08:59 < hebasto> https://github.com/hebasto/secp256k1/blob/220628-cmake/cmake/arm-linux-gnueabihf.toolchain.cmake
09:00 < real_or_random> sipa: IIUC in autotools, just everything has a prefix with the triplet
09:00 < real_or_random> so it won't need a config, it's clear where the tools are
09:00 < real_or_random> cmake is more flexible, it supports other ways of naming tools but then it needs to config to tell it where to find the right tool for the right target
09:01 < hebasto> true
09:01 < sipa> but are these files then libsecp256k1-specific? host/platform configuration makes sense, but I hope you don't need a configuration for every (host, project) combination?
09:01 < real_or_random> that's a valid question yes... then shouldn't be libsecp specific
09:02 < real_or_random> rather be specific to host and build system
09:02 < sipa> so why do they need to be part of our build system?
09:02 < sipa> or is that just for CI?
09:02 < sipa> s/build system/repo/
09:03 < real_or_random> yep, that's indeed a valid question... 
09:03 < fanquake> I guess it depends how supported you want cmake to be in the standalone libsecp repo
09:03 < hebasto> to make them easy accessible to users?
09:04 < real_or_random> maybe because we have a standard "build host" in core, e.g., guix?
09:04 < hebasto> most projects in github do not bother about toolchain files
09:04 < fanquake> that's because most projects dont care about cross-compiling
09:04 < sipa> ok maybe my question is: what precisely do i need this file for?
09:05 < sipa> nothing in https://github.com/bitcoin-core/secp256k1/pull/1113/files#diff-e525f91162b9f327a05d7231d98f9db9f1a27bc3dded91c5330594369e3ddcb6 seems related to libsecp
09:05 < real_or_random> ok but we're over the hour, so maybe we can try to close the meeting officially :) then I'd stop this with a call to look at the PR and give concept (N)ACK or ask question.
09:06 < real_or_random> feel free to continue here of course ^^
09:06 < sipa> Sure!
09:06 < real_or_random>  I think that was very useful for a first meeting :D
09:06 < fanquake> My understanding is that if you want to do a build that is for anything other than your own system, you'll need to a toolchain file
09:06 < fanquake> Yes
09:06 < sipa> But can't I construct a generic toolchain file for "arm64 cross-compiling on my system", which would be usable by any project?
09:07 < real_or_random> sipa: the file tells cmake where to find tools on your system. of course, it's sepcific to your system but it's probably also useful to have some examples for users because the files will look very similar on recent linux distros for example
09:07 < real_or_random> sipa: yeah I'm wondering about the same... 
09:08 < fanquake> I assume that could be possible, but haven't done it
09:10 < hebasto> I guess, they just use `cmake .. -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=arm -DCMAKE_C_COMPILER=arm-linux-gnueabihf-gcc` instead of a toolchain file
09:13 < sipa> well at least it seems such files probably need very little maintenance
09:13 < hebasto> true :)
09:15 < hebasto> also they can a bit complex and include, for example, platform specific compiler flags
09:16 < hebasto> *can be a bit more complex
10:28 < real_or_random> sipa: so you see a higher priority for #1066 (abstract out magnitude) than #1058 (multicomb)?
10:31 < sipa> I think 1066 may be simpler as it's just a refactor, and 1058 may need more domain-level review (the math is nontrivial).
10:38 < real_or_random> I see
10:50 < sipa> But 1066 is also just a precursor to things like the 5x64 fields, while 1058 is an immediate performance improvement.
12:17 < nickler> Thanks real_or_random for initiating the meeting btw. Was a good start! Monday UTC 1500 every 2 weeks works for me.
12:20 < sipa> Indeed! Likewise.
--- Log closed Tue Jul 05 00:00:11 2022