--- Log opened Tue Jan 07 00:01:02 2020
00:42 -!- Kiminuo [~mix@141.98.103.220] has joined #rust-bitcoin
01:33 -!- belcher [~belcher@unaffiliated/belcher] has joined #rust-bitcoin
02:18 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Ping timeout: 248 seconds]
03:27 -!- michaelfolkson [~textual@host-92-3-210-254.as43234.net] has joined #rust-bitcoin
03:34 -!- michaelfolkson [~textual@host-92-3-210-254.as43234.net] has quit [Quit: Sleep mode]
03:36 -!- Dean_Guss [~dean@gateway/tor-sasl/deanguss] has joined #rust-bitcoin
03:36 -!- DeanWeen [~dean@gateway/tor-sasl/deanguss] has quit [Remote host closed the connection]
03:54 -!- michaelfolkson [~textual@host-92-3-210-254.as43234.net] has joined #rust-bitcoin
04:02 -!- michaelfolkson [~textual@host-92-3-210-254.as43234.net] has quit [Quit: Sleep mode]
04:21 -!- michaelfolkson [~textual@46.233.77.205] has joined #rust-bitcoin
04:39 -!- jonatack [~jon@213.152.161.170] has joined #rust-bitcoin
05:24 < stevenroose> So I'm struggling with fixing rust-bitcoinconsensus
05:24 < stevenroose> Anyone any experience? This is the problem:
05:24 < stevenroose> Their build was implicitly using the secp symbols that rust-secp exposed.
05:25 < stevenroose> Since the new rust-secp doesn't expose the default secp symbols anymore, the linking broke.
05:25 < stevenroose> So what I'm trying to achive is for the rust-bitcoinconsensus build process to somehow include libsecp256k1.
05:25 < stevenroose> I have very little experience with build.rs and cc, though. So I'm a bit lost and tried all the things I could find from docs/googling.
05:28 < stevenroose> Did anyone PM me? My client flashes when that happens, but I can't find new PMs usually :) So please ping me here first if you do.
05:28 < stevenroose> Also still asking for an ACK on the new release PR: https://github.com/rust-bitcoin/rust-bitcoin/pull/366
05:38 < elichai2> stevenroose: I can give it a try if you want
05:39 < elichai2> and ACKed
05:41 < elichai2> hmm so it means that you want to compile secp256k1 through rust-bitcoinconsensus and not use rust-secp for the bindings. should be easy enough
05:46 < stevenroose> elichai2: I'd share you my branch, but all I did was move some parts of rust-secp's build.rs into rust-bitcoinconsensus's build.rs and try move different parts around.
05:46 < elichai2> k. will try myself
05:47 < stevenroose> elichai2: yeah I think we will have to accept the results of that: (1) secp will be compiled twice when you use rust-bitcoinconsensus and (2) rust-bitcoinconsensus will cause symbol collision issues with other crates that try to compile the original secp symbols
05:47 < stevenroose> elichai2: thanks, awesome!
05:48 < stevenroose> elichai2: perhaps a feature "external-symbols" like rust-secp has has that *doesn't* compile secp would be awesome.
05:49 < elichai2> I think compiling twice is fine. but after i'll make it work we'll see about that feature :)
05:49 < elichai2> the only problem is that this basically means copying rust-secp256k1-sys/build.rs
05:50 < elichai2> meaning that any "upgrade" for that requires copuing here
05:50 < elichai2> (we need to define the right precompiled macros)
05:51 < stevenroose> elichai2: yeah well, rust-secp's build.rs has a lot of optimizations we don't need here
05:52 < stevenroose> like optinal stuff like recovery and a lowmemory option (I odn't think lowmemory makes sense for this crate), ...
05:52 < stevenroose> I just added the 64-bit checker thing.
05:52 < elichai2> well recovery is required
05:52 < elichai2> the rest i'll leave like bitcoin core configurations
06:13 < elichai2> done :)
06:13 < elichai2> there are some other things that can be better here, should I do in a seperate PR or just seperate commits same PR?
06:14 < elichai2> stevenroose:
06:18 < elichai2> I'm also trying to decide if I should split it into 2 separate compilation units
06:30 < elichai2> https://github.com/rust-bitcoin/rust-bitcoinconsensus/pull/13
06:30 -!- Kiminuo [~mix@141.98.103.220] has quit [Ping timeout: 268 seconds]
06:37 -!- jonatack [~jon@213.152.161.170] has quit [Read error: Connection reset by peer]
06:37 < stevenroose> ok now I'm super curious. I spent almost an entire day fiddling with it. Did cargo test work? Somehow it seems like the linking issue only shows with cargo test not with cargo build.
06:41 -!- jonatack [~jon@54.76.13.109.rev.sfr.net] has joined #rust-bitcoin
06:45 < elichai2> yeah cargo test works. travis failed because of `cc` version in 1.22
06:46 < elichai2> now lets see if I can bump it to 0.19.0.1 :)
06:50 < elichai2> BlueMatt: are you aware of the fact that this compiles even though it has half the files in Makefile.am `libbitcoinconsensus_la_SOURCES`?
07:06 < elichai2> done https://github.com/rust-bitcoin/rust-bitcoinconsensus/pull/14
07:14 < elichai2> stevenroose: it makes sens that build will pass but test won't, it's a library so it doesn't do any linking when building. but testing builds a binary so it does link
07:17 -!- michaelfolkson [~textual@46.233.77.205] has quit [Quit: Sleep mode]
07:17 -!- jonatack [~jon@54.76.13.109.rev.sfr.net] has quit [Ping timeout: 265 seconds]
07:20 -!- michaelfolkson [~textual@46.233.77.205] has joined #rust-bitcoin
07:20 -!- michaelfolkson [~textual@46.233.77.205] has quit [Client Quit]
08:00 < stevenroose> elichai2: yeah I figured that. But I'm still confused as to how you got it to work because on first glance I can't see what you did differently from what I did. Let me take a closer look. Thanks anyway!
08:04 < stevenroose> elichai2: yeah I'm super confused now. I somehow also added the lax_der C file and other than that I am confused of what is the difference. Thanks anyway, I'll test and ACK.
08:11 < stevenroose> andytoshi: could you take a look: https://github.com/rust-bitcoin/rust-bitcoinconsensus/pull/14 ?
08:12 < stevenroose> Tamas is not online right now, I'm hoping he gets the notification.
09:00 < elichai2> stevenroose: idk just added the necessary files, includes and macro definitions ?\_(?)_/?
09:35 < andytoshi> stevenroose: i'll try to take a look today
09:35 < andytoshi> but in general i'm uncomfortable with this approach. i wish we could use the rust-secp symbol-s
09:35 < andytoshi> it sucks that we'll have symbol conflicts related to libsecp, when we have multiple libbitcoinconsensus versions in play
09:36 < BlueMatt> elichai2: wait, why do you want to do that, though?
09:37 < BlueMatt> elichai2: just use libsecp with standard-exposed-functions...it avoids having two copies of libsecp in your binary
09:38 < elichai2> andytoshi: that's already the case but for bitcoinconsensus symbols
09:38 < elichai2> BlueMatt: because we renamed the symbols in rust-secp256k1
09:38 < BlueMatt> right, i thought there was an option in rust-secp to turn that off
09:39 < BlueMatt> in any case, you're duplicating the problem if you dont use it - the point of renaming the symbols is to allow two different versions in the same binary, but we have the same issue in bitcoinconsensus
09:43 < BlueMatt> so either a) we dont care, and we should just use rust-secp and maybe re-export the original symbols in their original names, or b) we do care, and we should rename the symbols in bitcoinconsensus too
09:43 < BlueMatt> personally, I think a is fine.
09:44 < BlueMatt> I mean you could even do it without changing how rust-secp is built - just re-export the symbols in their original form lol
09:46 < stevenroose> BlueMatt: yeah a good tool would be to optionally turn off the symbols renaming in rust-secp. That way it could be compiled with bitcoinconsensus if the latter uses the exernal-secp feature.
09:47 < BlueMatt> right, I thought that was already an option, or is it only an option to have rust-secp *use* the externally-provided symbols
09:47 < stevenroose> BlueMatt: yeah that
09:47 < stevenroose> So any complex system could have both rust-secp and rust-bitcoinconsensus use the same symbols but they'd have to link them themselves
09:48 < stevenroose> How about an `export-symbols` feature on rust-secp that makes it export the prefixed symbols unprefixed. I think we can even do that from Rust.
09:48 < BlueMatt> right. yea I mean I guess, at least personally, I'm a bit less concerned about a user importing two versions of bitcoinconsensus, so the only thing we'd need is an option to export the regular symbols from the one version that rust-bitcoinconsensus uses
09:49 < BlueMatt> yea, I think that would work, though thats hilariously awkward
09:49 < andytoshi> we cannot set a feature in rust-secp to disable symbol renaming
09:49 < andytoshi> because features are additive
09:49 < BlueMatt> so?
09:49 < andytoshi> it would cause problems for projects that depend on both rust-secp and rust-bitcoinconsensus
09:50 < andytoshi> or rather, projects which depend on multiple versions of rust-secp would be unable to also depend on rust-bitcoinconsensus
09:50 < BlueMatt> or, maybe to consider it a different way, I think steven's point is to do "re-export-symbols-under-original-name"
09:50 < BlueMatt> hmm? wouldnt it be that projects that depend on multiple versions of rust-secp that *also* set the feature on multiple versions, wont be able to?
09:50 < BlueMatt> like, are features additive across versions, too?
09:50 < andytoshi> ah good question
09:50 < andytoshi> i don't know
09:51 < andytoshi> my -guess- would be no
09:51 < BlueMatt> i suppose i kinda assumed not, but, yea, we should check.
09:51 < stevenroose> hmm, is there a way to make a definition pub or non-pub based on a feature?
09:51 < stevenroose> without repeating the entire definition (it's an `extern "C"` block
09:52 < andytoshi> not that i can think of
09:52 < stevenroose> I think if we would just make the `extern "C"` block in secp256k1-sys public (optionally), it would export the original symbols
09:52 < andytoshi> oh
09:52 < andytoshi> well, you can re-export symbols
09:52 < andytoshi> and duplicate the re-export
09:52 < andytoshi> one pub, one non-pub
09:52 < stevenroose> that would mean repeating all the symbols, but fair enough
09:52 < andytoshi> oh yeah, right
09:55 < stevenroose> We don't care about the prefixed symbols at all, right?
09:56 < stevenroose> I mean if we re-export the original ones with a feature ,both the original ones and the reexported ones will exist.
09:56 < stevenroose> Anyhow, I'm not even sure the re-exported symbols are actually linkable. I'm trying.
10:01 < stevenroose> aaarghhh
10:04 < stevenroose> Yeah so reexporting doesn't work because extern "C" symbols are already locally-scoped so they are double defined then. Let me try create a new module
10:04 < stevenroose> Ah no it doesn't make a difference.
10:05 < stevenroose> They are already pub inside the extern "C", I don't think you can export them. elichai2 ?
10:06 < stevenroose> One approach I'm seeing is that we can perform the symbol prefixing at-compile-time based on the feature. Then the export-symbols feature would just assume the external-symbols feature.
10:07 < stevenroose> Pff, I was glad we finally reached a point where rust-secp was a hassle-free way of *using* the secp symbols. Not we need to go back to *providing* them as well.
10:08 < elichai2> Hmmmmm
10:08 < elichai2> Theoretically we can re-export the question is how much effort and code would that be
10:09 < elichai2> (ie wrapping all of the function in pub extern c functions that just call them will for)
10:09 < elichai2> I'm on my phone right now but I can look into re-exporting later
10:26 < stevenroose> elichai2: or a macro that creates one extern "C" symbol and a reexport. the macro could even have the prefix defined so we also only have to update one line when we update upstream
10:27 < stevenroose> elichai2: what's your timezone? I'm almost getting off here (have visitors for dinner) so perhaps we can take a look tomorrow. You're clearly a better C linking wizard than I am :)
10:27 < elichai2> GMT+2 I think
10:44 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #rust-bitcoin
10:46 < stevenroose> elichai2: ah a macro of course doesn't work, because you also need the actual method declaration yourself that is different for each method
10:56 -!- Kiminuo [~mix@141.98.103.220] has joined #rust-bitcoin
11:09 -!- Dean_Guss [~dean@gateway/tor-sasl/deanguss] has quit [Quit: Leaving]
11:16 < andytoshi> can we just link libsecp statically into libbitcoinconsensus?
11:16 < andytoshi> i'm a little surprised this isn't done by default
11:16 < stevenroose> yeah that's what elichai does
11:17 < stevenroose> did* in his PR
11:17 < andytoshi> ah ok
11:17 < stevenroose> But it will again cause trouble with some other crate (like an older bitcoinconsensus) also providing those symbols
11:17 < stevenroose> Basically rust-secp all over again :D
11:17 < stevenroose> "(
11:18 < stevenroose> That's why we started talking about rust-secp exporting the original symbols. I realize now that that will also cause problems if multiple versions of rust-secp are used and somehow both are asked to export symbols.
11:19 < andytoshi> ok i see
11:19 < andytoshi> yeah i think we should just statically compile it
11:19 < stevenroose> *sigh*
11:19 < stevenroose> Me too. Is bitcoinconsensus default in rust-bitcoin?
11:19 < andytoshi> there's no reason to expect that rust-secp will export the same functionality and ABI that libbitcoinconsensus expects
11:19 < andytoshi> no it's not
11:19 < stevenroose> It's not. So that's fine.
11:20 < stevenroose> So this PR should fix it then: https://github.com/rust-bitcoin/rust-bitcoinconsensus/pull/14
11:20 < andytoshi> ok i acked tat
11:20 < stevenroose> With that one published, we can update all relevant deps of rust-bitcoin. I'll quickly PR that.
11:20 < stevenroose> andytoshi: can you merge and publish there or just Tamas?
11:24 < elichai2> The "symbols problem" in bitcoinconsensus different versions is a thing no matter if it also compile secp or not
11:25 < andytoshi> definitely i can't publish
11:25 < andytoshi> though i can merge
11:25 < andytoshi> elichai2: it should not be
11:25 < stevenroose> https://github.com/rust-bitcoin/rust-bitcoin/pull/380
11:25 < andytoshi> if you are linking statically
11:25 < stevenroose> That's the release PR with bumps. It's blocked by rust-bitcoinconsensus being published, though.
11:25 < andytoshi> nice stevenroose
11:25 < elichai2> andytoshi: even if you are statically linking the symbols are still exposed, right?
11:26 < andytoshi> no
11:26 < andytoshi> well
11:26 < andytoshi> maybe
11:27 < elichai2> I mean 2 static libs with the same symbols the linker will yell
11:28 < stevenroose> I'm off for today. If we didn't hear from Tamas by tomorrow, I'll kindly send him an e-mail. And perhaps request him to add someone of us as publishers. Cheers.
12:24 -!- mryandao [~mryandao@gateway/tor-sasl/mryandao] has quit [Remote host closed the connection]
12:25 -!- mryandao [~mryandao@gateway/tor-sasl/mryandao] has joined #rust-bitcoin
13:26 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection]
13:27 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined #rust-bitcoin
15:11 -!- DeanGuss [~dean@gateway/tor-sasl/deanguss] has joined #rust-bitcoin
15:17 -!- Kiminuo [~mix@141.98.103.220] has quit [Ping timeout: 268 seconds]
19:46 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 260 seconds]
20:27 -!- mryandao [~mryandao@gateway/tor-sasl/mryandao] has quit [Ping timeout: 240 seconds]
20:40 -!- mryandao [~mryandao@gateway/tor-sasl/mryandao] has joined #rust-bitcoin
23:06 -!- Kiminuo [~mix@141.98.103.220] has joined #rust-bitcoin
--- Log closed Wed Jan 08 00:00:01 2020