--- Log opened Tue Jan 07 00:01:02 2020 00:42 -!- Kiminuo [~mix@141.98.103.220] has joined #rust-bitcoin 01:33 -!- belcher [~belcher@unaffiliated/belcher] has joined #rust-bitcoin 02:18 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Ping timeout: 248 seconds] 03:27 -!- michaelfolkson [~textual@host-92-3-210-254.as43234.net] has joined #rust-bitcoin 03:34 -!- michaelfolkson [~textual@host-92-3-210-254.as43234.net] has quit [Quit: Sleep mode] 03:36 -!- Dean_Guss [~dean@gateway/tor-sasl/deanguss] has joined #rust-bitcoin 03:36 -!- DeanWeen [~dean@gateway/tor-sasl/deanguss] has quit [Remote host closed the connection] 03:54 -!- michaelfolkson [~textual@host-92-3-210-254.as43234.net] has joined #rust-bitcoin 04:02 -!- michaelfolkson [~textual@host-92-3-210-254.as43234.net] has quit [Quit: Sleep mode] 04:21 -!- michaelfolkson [~textual@46.233.77.205] has joined #rust-bitcoin 04:39 -!- jonatack [~jon@213.152.161.170] has joined #rust-bitcoin 05:24 < stevenroose> So I'm struggling with fixing rust-bitcoinconsensus 05:24 < stevenroose> Anyone any experience? This is the problem: 05:24 < stevenroose> Their build was implicitly using the secp symbols that rust-secp exposed. 05:25 < stevenroose> Since the new rust-secp doesn't expose the default secp symbols anymore, the linking broke. 05:25 < stevenroose> So what I'm trying to achive is for the rust-bitcoinconsensus build process to somehow include libsecp256k1. 05:25 < stevenroose> I have very little experience with build.rs and cc, though. So I'm a bit lost and tried all the things I could find from docs/googling. 05:28 < stevenroose> Did anyone PM me? My client flashes when that happens, but I can't find new PMs usually :) So please ping me here first if you do. 05:28 < stevenroose> Also still asking for an ACK on the new release PR: https://github.com/rust-bitcoin/rust-bitcoin/pull/366 05:38 < elichai2> stevenroose: I can give it a try if you want 05:39 < elichai2> and ACKed 05:41 < elichai2> hmm so it means that you want to compile secp256k1 through rust-bitcoinconsensus and not use rust-secp for the bindings. should be easy enough 05:46 < stevenroose> elichai2: I'd share you my branch, but all I did was move some parts of rust-secp's build.rs into rust-bitcoinconsensus's build.rs and try move different parts around. 05:46 < elichai2> k. will try myself 05:47 < stevenroose> elichai2: yeah I think we will have to accept the results of that: (1) secp will be compiled twice when you use rust-bitcoinconsensus and (2) rust-bitcoinconsensus will cause symbol collision issues with other crates that try to compile the original secp symbols 05:47 < stevenroose> elichai2: thanks, awesome! 05:48 < stevenroose> elichai2: perhaps a feature "external-symbols" like rust-secp has has that *doesn't* compile secp would be awesome. 05:49 < elichai2> I think compiling twice is fine. but after i'll make it work we'll see about that feature :) 05:49 < elichai2> the only problem is that this basically means copying rust-secp256k1-sys/build.rs 05:50 < elichai2> meaning that any "upgrade" for that requires copuing here 05:50 < elichai2> (we need to define the right precompiled macros) 05:51 < stevenroose> elichai2: yeah well, rust-secp's build.rs has a lot of optimizations we don't need here 05:52 < stevenroose> like optinal stuff like recovery and a lowmemory option (I odn't think lowmemory makes sense for this crate), ... 05:52 < stevenroose> I just added the 64-bit checker thing. 05:52 < elichai2> well recovery is required 05:52 < elichai2> the rest i'll leave like bitcoin core configurations 06:13 < elichai2> done :) 06:13 < elichai2> there are some other things that can be better here, should I do in a seperate PR or just seperate commits same PR? 06:14 < elichai2> stevenroose: 06:18 < elichai2> I'm also trying to decide if I should split it into 2 separate compilation units 06:30 < elichai2> https://github.com/rust-bitcoin/rust-bitcoinconsensus/pull/13 06:30 -!- Kiminuo [~mix@141.98.103.220] has quit [Ping timeout: 268 seconds] 06:37 -!- jonatack [~jon@213.152.161.170] has quit [Read error: Connection reset by peer] 06:37 < stevenroose> ok now I'm super curious. I spent almost an entire day fiddling with it. Did cargo test work? Somehow it seems like the linking issue only shows with cargo test not with cargo build. 06:41 -!- jonatack [~jon@54.76.13.109.rev.sfr.net] has joined #rust-bitcoin 06:45 < elichai2> yeah cargo test works. travis failed because of `cc` version in 1.22 06:46 < elichai2> now lets see if I can bump it to 0.19.0.1 :) 06:50 < elichai2> BlueMatt: are you aware of the fact that this compiles even though it has half the files in Makefile.am `libbitcoinconsensus_la_SOURCES`? 07:06 < elichai2> done https://github.com/rust-bitcoin/rust-bitcoinconsensus/pull/14 07:14 < elichai2> stevenroose: it makes sens that build will pass but test won't, it's a library so it doesn't do any linking when building. but testing builds a binary so it does link 07:17 -!- michaelfolkson [~textual@46.233.77.205] has quit [Quit: Sleep mode] 07:17 -!- jonatack [~jon@54.76.13.109.rev.sfr.net] has quit [Ping timeout: 265 seconds] 07:20 -!- michaelfolkson [~textual@46.233.77.205] has joined #rust-bitcoin 07:20 -!- michaelfolkson [~textual@46.233.77.205] has quit [Client Quit] 08:00 < stevenroose> elichai2: yeah I figured that. But I'm still confused as to how you got it to work because on first glance I can't see what you did differently from what I did. Let me take a closer look. Thanks anyway! 08:04 < stevenroose> elichai2: yeah I'm super confused now. I somehow also added the lax_der C file and other than that I am confused of what is the difference. Thanks anyway, I'll test and ACK. 08:11 < stevenroose> andytoshi: could you take a look: https://github.com/rust-bitcoin/rust-bitcoinconsensus/pull/14 ? 08:12 < stevenroose> Tamas is not online right now, I'm hoping he gets the notification. 09:00 < elichai2> stevenroose: idk just added the necessary files, includes and macro definitions ?\_(?)_/? 09:35 < andytoshi> stevenroose: i'll try to take a look today 09:35 < andytoshi> but in general i'm uncomfortable with this approach. i wish we could use the rust-secp symbol-s 09:35 < andytoshi> it sucks that we'll have symbol conflicts related to libsecp, when we have multiple libbitcoinconsensus versions in play 09:36 < BlueMatt> elichai2: wait, why do you want to do that, though? 09:37 < BlueMatt> elichai2: just use libsecp with standard-exposed-functions...it avoids having two copies of libsecp in your binary 09:38 < elichai2> andytoshi: that's already the case but for bitcoinconsensus symbols 09:38 < elichai2> BlueMatt: because we renamed the symbols in rust-secp256k1 09:38 < BlueMatt> right, i thought there was an option in rust-secp to turn that off 09:39 < BlueMatt> in any case, you're duplicating the problem if you dont use it - the point of renaming the symbols is to allow two different versions in the same binary, but we have the same issue in bitcoinconsensus 09:43 < BlueMatt> so either a) we dont care, and we should just use rust-secp and maybe re-export the original symbols in their original names, or b) we do care, and we should rename the symbols in bitcoinconsensus too 09:43 < BlueMatt> personally, I think a is fine. 09:44 < BlueMatt> I mean you could even do it without changing how rust-secp is built - just re-export the symbols in their original form lol 09:46 < stevenroose> BlueMatt: yeah a good tool would be to optionally turn off the symbols renaming in rust-secp. That way it could be compiled with bitcoinconsensus if the latter uses the exernal-secp feature. 09:47 < BlueMatt> right, I thought that was already an option, or is it only an option to have rust-secp *use* the externally-provided symbols 09:47 < stevenroose> BlueMatt: yeah that 09:47 < stevenroose> So any complex system could have both rust-secp and rust-bitcoinconsensus use the same symbols but they'd have to link them themselves 09:48 < stevenroose> How about an `export-symbols` feature on rust-secp that makes it export the prefixed symbols unprefixed. I think we can even do that from Rust. 09:48 < BlueMatt> right. yea I mean I guess, at least personally, I'm a bit less concerned about a user importing two versions of bitcoinconsensus, so the only thing we'd need is an option to export the regular symbols from the one version that rust-bitcoinconsensus uses 09:49 < BlueMatt> yea, I think that would work, though thats hilariously awkward 09:49 < andytoshi> we cannot set a feature in rust-secp to disable symbol renaming 09:49 < andytoshi> because features are additive 09:49 < BlueMatt> so? 09:49 < andytoshi> it would cause problems for projects that depend on both rust-secp and rust-bitcoinconsensus 09:50 < andytoshi> or rather, projects which depend on multiple versions of rust-secp would be unable to also depend on rust-bitcoinconsensus 09:50 < BlueMatt> or, maybe to consider it a different way, I think steven's point is to do "re-export-symbols-under-original-name" 09:50 < BlueMatt> hmm? wouldnt it be that projects that depend on multiple versions of rust-secp that *also* set the feature on multiple versions, wont be able to? 09:50 < BlueMatt> like, are features additive across versions, too? 09:50 < andytoshi> ah good question 09:50 < andytoshi> i don't know 09:51 < andytoshi> my -guess- would be no 09:51 < BlueMatt> i suppose i kinda assumed not, but, yea, we should check. 09:51 < stevenroose> hmm, is there a way to make a definition pub or non-pub based on a feature? 09:51 < stevenroose> without repeating the entire definition (it's an `extern "C"` block 09:52 < andytoshi> not that i can think of 09:52 < stevenroose> I think if we would just make the `extern "C"` block in secp256k1-sys public (optionally), it would export the original symbols 09:52 < andytoshi> oh 09:52 < andytoshi> well, you can re-export symbols 09:52 < andytoshi> and duplicate the re-export 09:52 < andytoshi> one pub, one non-pub 09:52 < stevenroose> that would mean repeating all the symbols, but fair enough 09:52 < andytoshi> oh yeah, right 09:55 < stevenroose> We don't care about the prefixed symbols at all, right? 09:56 < stevenroose> I mean if we re-export the original ones with a feature ,both the original ones and the reexported ones will exist. 09:56 < stevenroose> Anyhow, I'm not even sure the re-exported symbols are actually linkable. I'm trying. 10:01 < stevenroose> aaarghhh 10:04 < stevenroose> Yeah so reexporting doesn't work because extern "C" symbols are already locally-scoped so they are double defined then. Let me try create a new module 10:04 < stevenroose> Ah no it doesn't make a difference. 10:05 < stevenroose> They are already pub inside the extern "C", I don't think you can export them. elichai2 ? 10:06 < stevenroose> One approach I'm seeing is that we can perform the symbol prefixing at-compile-time based on the feature. Then the export-symbols feature would just assume the external-symbols feature. 10:07 < stevenroose> Pff, I was glad we finally reached a point where rust-secp was a hassle-free way of *using* the secp symbols. Not we need to go back to *providing* them as well. 10:08 < elichai2> Hmmmmm 10:08 < elichai2> Theoretically we can re-export the question is how much effort and code would that be 10:09 < elichai2> (ie wrapping all of the function in pub extern c functions that just call them will for) 10:09 < elichai2> I'm on my phone right now but I can look into re-exporting later 10:26 < stevenroose> elichai2: or a macro that creates one extern "C" symbol and a reexport. the macro could even have the prefix defined so we also only have to update one line when we update upstream 10:27 < stevenroose> elichai2: what's your timezone? I'm almost getting off here (have visitors for dinner) so perhaps we can take a look tomorrow. You're clearly a better C linking wizard than I am :) 10:27 < elichai2> GMT+2 I think 10:44 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #rust-bitcoin 10:46 < stevenroose> elichai2: ah a macro of course doesn't work, because you also need the actual method declaration yourself that is different for each method 10:56 -!- Kiminuo [~mix@141.98.103.220] has joined #rust-bitcoin 11:09 -!- Dean_Guss [~dean@gateway/tor-sasl/deanguss] has quit [Quit: Leaving] 11:16 < andytoshi> can we just link libsecp statically into libbitcoinconsensus? 11:16 < andytoshi> i'm a little surprised this isn't done by default 11:16 < stevenroose> yeah that's what elichai does 11:17 < stevenroose> did* in his PR 11:17 < andytoshi> ah ok 11:17 < stevenroose> But it will again cause trouble with some other crate (like an older bitcoinconsensus) also providing those symbols 11:17 < stevenroose> Basically rust-secp all over again :D 11:17 < stevenroose> "( 11:18 < stevenroose> That's why we started talking about rust-secp exporting the original symbols. I realize now that that will also cause problems if multiple versions of rust-secp are used and somehow both are asked to export symbols. 11:19 < andytoshi> ok i see 11:19 < andytoshi> yeah i think we should just statically compile it 11:19 < stevenroose> *sigh* 11:19 < stevenroose> Me too. Is bitcoinconsensus default in rust-bitcoin? 11:19 < andytoshi> there's no reason to expect that rust-secp will export the same functionality and ABI that libbitcoinconsensus expects 11:19 < andytoshi> no it's not 11:19 < stevenroose> It's not. So that's fine. 11:20 < stevenroose> So this PR should fix it then: https://github.com/rust-bitcoin/rust-bitcoinconsensus/pull/14 11:20 < andytoshi> ok i acked tat 11:20 < stevenroose> With that one published, we can update all relevant deps of rust-bitcoin. I'll quickly PR that. 11:20 < stevenroose> andytoshi: can you merge and publish there or just Tamas? 11:24 < elichai2> The "symbols problem" in bitcoinconsensus different versions is a thing no matter if it also compile secp or not 11:25 < andytoshi> definitely i can't publish 11:25 < andytoshi> though i can merge 11:25 < andytoshi> elichai2: it should not be 11:25 < stevenroose> https://github.com/rust-bitcoin/rust-bitcoin/pull/380 11:25 < andytoshi> if you are linking statically 11:25 < stevenroose> That's the release PR with bumps. It's blocked by rust-bitcoinconsensus being published, though. 11:25 < andytoshi> nice stevenroose 11:25 < elichai2> andytoshi: even if you are statically linking the symbols are still exposed, right? 11:26 < andytoshi> no 11:26 < andytoshi> well 11:26 < andytoshi> maybe 11:27 < elichai2> I mean 2 static libs with the same symbols the linker will yell 11:28 < stevenroose> I'm off for today. If we didn't hear from Tamas by tomorrow, I'll kindly send him an e-mail. And perhaps request him to add someone of us as publishers. Cheers. 12:24 -!- mryandao [~mryandao@gateway/tor-sasl/mryandao] has quit [Remote host closed the connection] 12:25 -!- mryandao [~mryandao@gateway/tor-sasl/mryandao] has joined #rust-bitcoin 13:26 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection] 13:27 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined #rust-bitcoin 15:11 -!- DeanGuss [~dean@gateway/tor-sasl/deanguss] has joined #rust-bitcoin 15:17 -!- Kiminuo [~mix@141.98.103.220] has quit [Ping timeout: 268 seconds] 19:46 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 260 seconds] 20:27 -!- mryandao [~mryandao@gateway/tor-sasl/mryandao] has quit [Ping timeout: 240 seconds] 20:40 -!- mryandao [~mryandao@gateway/tor-sasl/mryandao] has joined #rust-bitcoin 23:06 -!- Kiminuo [~mix@141.98.103.220] has joined #rust-bitcoin --- Log closed Wed Jan 08 00:00:01 2020