--- Log opened Fri Oct 25 00:00:45 2019 02:01 -!- jonatack [~jon@2a01:e35:8aba:8220:6627:dad:d967:649d] has quit [Ping timeout: 264 seconds] 03:03 -!- jonatack [~jon@213.152.161.234] has joined #bitcoin-builds 05:02 < fanquake> dongcarl did you still want to catch up today? 05:03 < fanquake> Also in regards to 16392, there is now a new cctools, ld64 etc out 05:05 < fanquake> Although not yet available in the cctools-port 05:18 < wumpus> didn't clang 9 have the stack checking issue? or am I confused now 05:19 < fanquake> wumpus are you talking in regards to the macOS issue from yesterday? 05:19 < wumpus> yes 05:20 < fanquake> Apple Clang doesn't map directly to actual Clang. 05:20 < wumpus> ohh! 05:20 < fanquake> Apple clang version 11.0.0 (clang-1100.0.33.8), which is likely based of Clang 5 or 6 I think, need to check again 05:21 < fanquake> That's Apple Clang on my machine at the moment. 05:27 -!- jonatack [~jon@213.152.161.234] has quit [Ping timeout: 268 seconds] 05:55 -!- jonatack [~jon@37.164.227.124] has joined #bitcoin-builds 06:11 -!- alko89 [~alko@cpe-85-10-28-138.static.amis.net] has joined #bitcoin-builds 06:40 -!- jonatack [~jon@37.164.227.124] has quit [Read error: Connection reset by peer] 07:02 < dongcarl> fanquake: Yeah let's do it 07:02 < dongcarl> Can we get Cory to join as well? 07:02 < dongcarl> I'm thinking 2:00 or so? 07:02 < fanquake> Ok. I'll hit him up 07:40 < fanquake> dongcarl theuni is in 07:43 < dongcarl> Great! 2PM EST it is 09:45 -!- jb55 [~jb55@gateway/tor-sasl/jb55] has quit [Remote host closed the connection] 09:49 -!- jonatack [~jon@2a01:e35:8aba:8220:6627:dad:d967:649d] has joined #bitcoin-builds 10:28 -!- ariard [~ariard@167.99.46.220] has joined #bitcoin-builds 10:52 -!- BlueMatt [~BlueMatt@unaffiliated/bluematt] has joined #bitcoin-builds 10:52 < fanquake> dongcarl did you have specific topics? I assume Clang bump is one of them? I'm getting coffee, will be back by 2 10:53 < dongcarl> Yeah clang bump is the most important one 11:00 < fanquake> ? 11:00 < dongcarl> Hi 11:00 < fanquake> Hi 11:01 < BlueMatt> FUCKYEA 11:01 < dongcarl> I need to get my head in the game for rust building 11:01 < dongcarl> What's missing right now for rust builds? Or is that all squared away? 11:01 < fanquake> Just pinged Cory as well 11:02 < fanquake> I know Cory was looking at it a couple days ago, but haven't heard the outcome of whatever he was doing 11:02 < fanquake> Apart from that, I think what's mostly missing is more buy-in heh 11:02 < cfields> Hi, here! 11:02 < fanquake> ! 11:03 < fanquake> dongcarl did you want to start with a Rust update or Clang? 11:03 < dongcarl> Yeah let's start with a rust update 11:03 < fanquake> cfields: anything new re Rust? 11:03 < cfields> dongcarl: first thing, sorry (#13430) for the painfully slow review lately. I've been pulled in a bunch of different directions. If it makes you feel any better, I'm not getting anyone else's stuff done either :p 11:04 < BlueMatt> the pr is Ready To Go (tm) from my side, cfields just said he wanted to redo someting 11:04 < cfields> That was meant to look like an apology counter but ended up looking like a PR#, heh. 11:04 < fanquake> https://github.com/bitcoin/bitcoin/pull/16834 11:04 < dongcarl> Lol no worries! 11:04 < fanquake> Yea I went looking for #13430 heh 11:05 < cfields> Yeah, I really want to once-over the Makefile, just haven't been able to get to it yet. 11:05 < cfields> I guess that can always happen post-merge, to keep from holding up forever. 11:05 < dongcarl> Okay, is there a "just build support" PR out? Or only feature PRs with build support in them? 11:05 < BlueMatt> right, yea, I mean it seems to work, it just might be a clusterfuck since I dont know make 11:06 < fanquake> cfields: could you comment to that effect in #16834? 11:06 < BlueMatt> dongcarl: the fetch-headers-over-dns code is about the smallest thing you could possible write 11:06 < BlueMatt> so its, essentially, a just-build-support 11:06 < cfields> fanquake: ack. 11:06 < BlueMatt> and, I'm really dubious of adding support without code 11:06 < fanquake> Yea I think if it's going to go in, it makes sense to have some sort of self-contained, off-by-default feature attached 11:07 < cfields> BlueMatt: +1. The code is what fleshed out the build problems. 11:07 < BlueMatt> its 65 lines of code :) 11:07 < dongcarl> BlueMatt: Oh for sure 11:07 < dongcarl> Oh... That's... 11:07 < dongcarl> Pretty good... 11:07 < dongcarl> (should put that in the title tbh) 11:07 < BlueMatt> oh, no, 129 11:07 < BlueMatt> whatever, same thing 11:08 < fanquake> Next steps might be to bring it up again in the meeting next week? Will the pointer that it's no-longer blocked on build concerns. 11:08 < BlueMatt> and like 300 LoC of makefile changes and rust framework additions 11:08 < fanquake> *With 11:08 < BlueMatt> I think next step is just acks 11:08 < BlueMatt> there isnt anything to discuss at the meeint 11:08 < BlueMatt> cause it just needs review and The Merge (tm) 11:08 < cfields> Commenting now before I forget. 11:09 < BlueMatt> meeting discussion was positive on the direction 11:09 < dongcarl> Urgh I'm sorry I'm so terrible with reviews... I'm hereby pledging to review #16834 next week 11:09 < fanquake> If you think we've go enough buy in then sure. Let pile up the ACKs 11:09 < BlueMatt> I dont recall anyone complaining last time it was discussed? 11:09 < fanquake> I've been telling you I'd review forever now, so I'll also get it done 11:09 < BlueMatt> sipa complained to me a tiny bit in person, but of the form "ehh, I'm tired of complaining about it" and once I pointed out that it is memory-limited his ears perked up :) 11:10 < fanquake> Yep, no complaints. I don't even think the original complaints came up again. 11:10 < fanquake> in regards to reproducibility etc 11:10 < dongcarl> BlueMatt: What's the rustc version? 11:10 < dongcarl> For 16834 11:11 < BlueMatt> dongcarl: oh, I guess I need to add to travis, but I havent actually tested.... 11:11 < BlueMatt> in theory pretty old lol 11:12 < fanquake> dongcarl some rustc versioning discussing here: https://github.com/bitcoin/bitcoin/issues/17090#issuecomment-540756099 11:12 < fanquake> I assume nothing has changed since targeting >1.34.2 11:12 < BlueMatt> no, I just havent bothered to test if most of my code compiles with it lol 11:13 < fanquake> sounds great, lets ship 11:13 < BlueMatt> travis currently tests 1.36 which is ubuntu xenial's default 11:13 < BlueMatt> (afaict) 11:13 < fanquake> Thanks for commenting cfields ? 11:13 < dongcarl> K, guix has those packaged, so I'll follow up after merge 11:14 < dongcarl> Sounds good, BlueMatt you can go back to the beach now 11:14 < fanquake> Seeing as Rust is unblocked. dongcarl want to switch to Clang? 11:14 < BlueMatt> lol nah, I've got work to do this morning :/ 11:14 < cfields> heh 11:14 < dongcarl> fanquake: Lol as a whole? 11:14 < cfields> dongcarl: I think he meant as a topic :) 11:14 < fanquake> BlueMatt: rewriting the tweet engine in Rust too? 11:15 < fanquake> Yea as a topic. 11:15 < dongcarl> Oh right, okay, so 11:15 < dongcarl> I want some form of #17099 11:15 < fanquake> https://github.com/bitcoin/bitcoin/pull/17099 11:15 < dongcarl> I see activity bumping to clang 9 11:16 < dongcarl> But it seems backwards compatible? 11:16 < fanquake> I somewhat mispoke in that PR. Basically that changes just allows us to bump that far if we want to. 11:16 < cfields> dongcarl: ok, for osx, we realy have to pin to a clang version. We can't just use whatever's on the system. 11:16 < dongcarl> cfields: Why can't we use whatever's on the system? 11:16 < BlueMatt> fanquake: you mean download headers over twitter? yea, lets do it :p 11:16 < cfields> dongcarl: because Apple just doesn't develop that way. 11:17 < fanquake> BlueMatt ACK. I'll point my bitcoind at the twitter bot as soon as it's deployed. 11:17 < cfields> Their sdks and tools aren't intended to be general-purpose, so it would be a step backwards (imo) to assume that any compiler will just work. 11:18 < dongcarl> cfields: Right, so let's pin it only in _our_ reproducible process, and let people try their own clang if they want to? 11:18 < BlueMatt> fanquake: you joke, but, actually, twitter is one of the least-censored social media sites....apparently its super popular in like uae cause they're not censored but they allow porn lolololol 11:18 < cfields> dongcarl: but that doesn't make any sense. 11:19 < cfields> dongcarl: I suspect there's a solution in search of a problem here? 11:19 < fanquake> Something GUIX related I assumed 11:19 < dongcarl> Well, for Guix I cannot just use the ubuntu-specific clang 11:19 < dongcarl> I also think that it'd be nice for people on Fedora to be able to cross-compile to OSX if they wanted to 11:20 < cfields> dongcarl: to be clear, it definitely doesn't need to be the official llvm one, or any specific llvm build. Only pegged to a version. 11:20 < dongcarl> cfields: Right... I'd be happy if there was just a way to indicate to depends that I want to use system clang and I promise it's the right version 11:22 < cfields> Hmm. 11:23 < cfields> dongcarl: guix will build the clang we want though, right. So the problem is just pointing depends at it? 11:23 < dongcarl> dongcarl: Yup 11:23 < dongcarl> cfields: Yup 11:23 < cfields> Ok, understood now. 11:24 < dongcarl> Guix has all clang versions up to 8 right now 11:24 < cfields> (Pretty sure we've had this discussion and caught me up once already :) 11:25 < dongcarl> cfields: Haha no worries :-) 11:25 < cfields> I wonder if we could teach depends about Guix somewhat. 11:25 < cfields> Ok, will come up with something for you. 11:25 < fanquake> lmn when it's safe to change topic slightly 11:25 < dongcarl> cfields: hmm 11:26 < dongcarl> I mean Ubuntu's been keeping old versions of clang too... 11:26 < dongcarl> So we could just have a CLANG_6_DIR/CLANG_9_DIR env var for depends or something like that 11:27 < dongcarl> And default to the llvm download? 11:27 < dongcarl> idk, might be stupid 11:27 < fanquake> In that case using 9 for releases or? 11:28 < cfields> dongcarl: something kinda like that, I think. 11:28 < dongcarl> No strong opinion, but would prefer 6 as that's what bionic's got 11:28 < dongcarl> cfields: Okay, let me know if you come up with something! 11:28 < cfields> One other thing I'd like to mention, that would change things significantly... 11:28 < dongcarl> yes? 11:29 < cfields> Have either of you looked into trying the llvm-* tools exclusively, rather than cctools? 11:29 < fanquake> No 11:29 < cfields> I think it's possible they might be far enough along for that by now. 11:29 < cfields> IIRC the hold-out last time I checked was lld, and the assumption that we could build with LTO. 11:29 < dongcarl> cfields: any links for llvm-* tools? 11:30 < fanquake> likely https://www.llvm.org/docs/CommandGuide/ 11:30 < cfields> dongcarl: they ship with clang, so the clang9 binary tarball would be the place to start. 11:31 < cfields> https://releases.llvm.org/9.0.0/clang+llvm-9.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz 11:31 < fanquake> What if he just wants to RTFM 11:31 < dongcarl> cfields: Huh... But if Apple is so finnicky with their toolchain... Shouldn't we stick to cctools for compat? 11:32 < cfields> dongcarl: it's mostly the compiler and the sdk that I worry about keeping in-sync. 11:32 < cfields> It'd be safe to assume that clang and llvm-foo are synced up. 11:33 < dongcarl> cfields: Okay sounds good... 11:33 < cfields> Also, apple's contributing to these tools, presumably with the end-goal of eliminating their custom tools. 11:34 < cfields> Just a thought. Eliminating cctools would make our lives much easier. 11:34 < dongcarl> cfields: Ah, that's good! 11:34 < dongcarl> Okay cool, fanquake did you have something? 11:34 < fanquake> So I was going to bring up some cctools related stuff, but maybe now it's redundant 11:35 < dongcarl> Eh what's up with cctools? 11:35 < cfields> I could throw together the changes to drop cctools and use the llvm-* stuff, and see how far it gets. 11:35 < dongcarl> cfields: YAAAAAAAAS 11:35 < cfields> I think the linker's still a problem, so I imagine we'd still have to use ld64 at minimum. 11:35 < fanquake> Basically just that there's now newer versions of cctools, LD64 etc available, but they haven't made it into the cctools-port yet, and was wondering if we care enough to wait for them 11:35 < fanquake> However if Corys going to look into dropping it entirely, that's much better 11:36 < cfields> fanquake: as long as we're using cctools, I'd say we should try to match to a clang release. 11:37 < cfields> dongcarl: ^^ is why I was trying to get you to ignore the clang9 PR. I wasn't suggesting we move to clang9. I wanted to point out that there's likely a "correct" version of clang that we should be using to match our cctools. 11:37 < fanquake> My other thought was in regards to the macOS SDK. I haven't looked at 10.15 at all yet. However should we look at just using 10.15 now that it's out.. ? Given I only picked 10.14 because it was the latest months ago when the PR was opened. 11:38 < dongcarl> cfields: Understood! We should figure it out for our SDK 11:38 < cfields> fanquake: +1 for that. 11:38 < cfields> fanquake: maybe bump the sdk, track down the other component versions as best as possible, and pin everything now? 11:38 < dongcarl> does 10.15 deprecate any OSXs? 11:38 < fanquake> cfields Ok. Can you also upload 10.15 to the depends-sources as well heh 11:38 < cfields> dongcarl: oh, right, good call. 11:38 < fanquake> dongcarl I'll have to check 11:38 < fanquake> Hopefully just < 10.12 11:39 < cfields> fanquake: will do. 11:39 < cfields> I'll have to ping BlueMatt for a reminder of htf to get into the server :) 11:39 < fanquake> I think dropping support for 32-bit apps was the big thing in 10.15 11:40 < cfields> fanquake: is there a convenient extracting script? :) 11:40 < dongcarl> cfields: I got one but it does the thing you don't like 11:40 < fanquake> cfields I have some hacked together steps in the docs in #16392 11:40 < cfields> dongcarl: heh 11:41 < cfields> dongcarl: I'm realizing now that my argument against that is completely hypocritical, given the discussion above. 11:41 < cfields> I just find it icky. Maybe I should just get over it. 11:41 < fanquake> do tell about this one thing cory doesn't like 11:42 < dongcarl> haha it's just packaging the libc++ headers in the OSX sdk 11:42 < cfields> fanquake: the c++ header splice into the sdk. 11:42 < fanquake> ah right 11:42 < dongcarl> It's what they do here: https://github.com/tpoechtrager/osxcross 11:42 < dongcarl> Same guy that does the cctools ports 11:43 < cfields> Conceptually, I just think they should be packaged differently. 11:43 < fanquake> Pretty sure he uses the same tool i was using to do the extraction as well https://github.com/NiklasRosenstein/pbzx 11:43 < cfields> But realistically, it's just over-engineering. I can't come up with a realistic reason to do it that way. 11:44 < fanquake> I had one other semi-build related topic 11:44 < cfields> And anyway, we already basically do the same thing, just from the llvm side. 11:44 < cfields> Ok, I've just talked myself out of caring about it. 11:44 < dongcarl> cfields: Haha nice 11:44 < dongcarl> fanquake: sup 11:44 < cfields> Still think it's not right, but it's not a reasonable objection. 11:44 < cfields> dongcarl: I'll be yelling "I TOLD YOU SO!!" if that changes, though :p 11:45 < dongcarl> cfields: Oh I'm ready for it 11:45 < dongcarl> ;-) 11:45 < fanquake> It seems like https://github.com/bitcoin/bitcoin/pull/17165 will be merged some, after some more build related review. 11:45 < cfields> dongcarl: actually... 11:45 < fanquake> Which means the path for removing OpenSSL is now open. 11:46 < fanquake> I'm going to spend some time looking at that over the next few weeks. Have already had some very initial discussions about it. 11:46 < cfields> before closing the book on that, I'd be curious to see a diff of the 3 header packages. 10.14 sdk, 10.15 sdk, llvm package. 11:46 < cfields> fanquake: awesome! 11:47 < dongcarl> cfields: Added to my todo list 11:47 < cfields> dongcarl: thanks! 11:47 < BlueMatt> cfields: the depends server folder thinggy you had sole access to 11:47 < BlueMatt> cfields: soooo...I have no fucking clue 11:47 < BlueMatt> lol 11:47 < cfields> oh, fun. 11:48 < cfields> fanquake: what build related review is necessary there? 11:48 < dongcarl> fanquake: It's a lot of commits... Will review after Matt's DNS thingy, not committing to more than I'll realistically do or else I'll do none of them :-) 11:49 < fanquake> cfields: basically just following up from your requests from last review 11:49 * cfields learns from dongcarl 11:49 < fanquake> dongcarl: Lots of lines and commits, but its all pretty straight forward 11:49 < fanquake> If you want more explanations added to the commit messages, just lmk 11:50 < dongcarl> fanquake: Sounds good! 11:50 < dongcarl> :-) 11:50 < cfields> fanquake: IIRC there was a good bit of cleanup stuff to do after removal. Lots of files/classes that are essentially unused after removal. Do you plan to address those in the PR, or maybe as a follow-up? 11:50 < fanquake> cfields: planning on doing all that in a follow up, otherwise I don't think it'll ever move forward 11:50 < cfields> Ok cool, agreed. 11:50 < dongcarl> Noice 11:50 < fanquake> Can look at dropping the network module from the qt build, and a bunch of other stuff 11:51 < fanquake> dropping openssl, refactoring the URI handling 11:51 < dongcarl> fanquake: don't expand the scope, we can do it in followup 11:51 < fanquake> dongcarl: yep, I'll leave that PR as is now 11:52 < fanquake> Should we conclude the first? unofficial build meeting heh 11:52 < dongcarl> hurray for unofficial meetings! 11:52 < dongcarl> #AC_DECLARE(endmeeting) 11:52 < fanquake> How topical 11:52 < cfields> Nerd. 11:53 < cfields> Lol 11:53 < cfields> I love it. 11:53 < dongcarl> cfields: I feel you'll appreciate this: https://twitter.com/carl_dong/status/1187791640506843136 11:53 < fanquake> cheers dongcarl cfields 11:53 < dongcarl> cheers! 11:53 < cfields> fanquake / dongcarl: Thanks for being so patient with me! 11:53 < dongcarl> and you me! 11:53 < BlueMatt> fanquake: onemoarack https://github.com/bitcoin/bitcoin/pull/17251 11:53 < BlueMatt> plz2merge 11:54 < fanquake> BlueMatt will take a look at it now 11:54 < cfields> BlueMatt: hahah, we should've done that ages ago. 11:54 < cfields> Pretty sure I've hacked that in locally a dozen times. 11:54 < BlueMatt> indeed 11:54 < BlueMatt> lolol, so go ack :) 11:55 < fanquake> cfields then go ACK it :p 12:00 < dongcarl> elichai2: I heard you were trying to remove boost? 12:00 < dongcarl> elichai2: Consider using this first, and chip away bit by bit: https://www.boost.org/doc/libs/1_71_0/tools/bcp/doc/html/index.html 12:01 < elichai2> dongcarl: ha. That's so we can ship only the needed parts of boost? 12:02 < elichai2> Yeah after I saw some notes in the code I thought i can slowly work my way through replacing boost bit by bit (at least except threading and fs for now) but apparently a lot of it have been tried and there are stupid little differences 12:02 < dongcarl> Yep! 12:02 < dongcarl> :-/ 12:03 < dongcarl> Urgh... 12:07 < cfields> Last I looked, boost::thread was really close to being ready for removal. 12:08 < cfields> Just needed to replace a few more interrupt points. 12:09 < cfields> fanquake: oh, before I forget, while you're looking at the cctools bumps... 12:09 < fanquake> cfields: hmm 12:10 < cfields> could you try turning off the "--disable-lto" option and seeing if that just magically works now? 12:10 < cfields> It'd be nice to be able to use lto, i assume the llvm-* tools will require that. 12:10 < fanquake> cfields: will do 12:11 < cfields> Thanks. It was in a funky state a few years ago because of all the transition pain, but I suspect that might all be overwith now. 12:13 < dongcarl> cfields: When depends is building native_* packages, it doesn't use clang when the target is osx does it? 12:13 < cfields> dongcarl: yes, it has to. 12:14 < cfields> dongcarl: that's the reason for a good bit of the weirdness in native_cctools. 12:14 < cfields> It can't be built with gcc, remember? 12:14 < cfields> We work around that by always building with our target clang, but using the system target. 12:15 < dongcarl> cfields: Oh I see... so just to make sure I understand: when the target is osx, even if we're building on ubuntu, all the native_* packages are still built using clang? 12:15 < cfields> (Er, did I understand what you were asking?) 12:15 < cfields> correct, the pegged clang 12:16 < dongcarl> okay, but the native_* packages will be --target=blahblah-linux and non-native ones are --target=blahdarwinblah ? 12:16 < cfields> Yup. Though in our case, we just leave out the target in the first case. 12:18 < dongcarl> understood! 12:18 < cfields> sorry, we leave out the target at runtime. It has nothing to do with the target used at configure-time, which is obviously set to native. 12:18 < dongcarl> so, there's no need for gcc at all then when cross-building for osx? 12:19 < cfields> correct, it should be unused. 12:19 < cfields> (which is why the llvm-* tools may be an option for us) 12:19 < dongcarl> cfields: for sure! 12:20 < cfields> Note that ideally that's unnecessary, and it should be perfectly possible to use system gcc for _native builds. But since cctools forces our hand, I figured it made sense to eliminate a variable and reuse the same clang. 12:21 < dongcarl> yeah it's good actually... makes my guix manifest simpler 12:21 < cfields> (I worry about cctools being brittle, since it's basically one person dealing with Apple's quirks) 12:21 < fanquake> dongcarl: how long until you get that one command you need into a new guix release? 12:22 < dongcarl> fanquake: until the next time they do a release! 12:22 < dongcarl> it's been in master for ages already lol 12:22 < fanquake> Oh right. So they haven't quite given you release powers yet :p 12:22 < fanquake> Is the next release going to be a 1.0.2, or 1.1 ? 12:22 < dongcarl> fanquake: Lol nope, I mean I could just produce my own release tarballs... But... eh it feels good to point to upstream 12:23 < dongcarl> fanquake: either 1.1 or 2.0 I'd say 12:23 < dongcarl> the gcc bump and mes reduced seed bootstrap were quite big changes 12:23 < fanquake> Yea fair enough 12:23 < cfields> fanquake: mind rebasing 16392 ? 12:24 < cfields> (no rush) 12:24 < fanquake> Will do it now. Should be trvial. 12:24 < fanquake> I assume this is so it's on top of your Boost change? 12:26 < cfields> Yeah, I just wanted to push up a clang9 branch so that we can poke at it and be on the same page. 12:26 < cfields> (again, not because I'm proposing that we bump to that. But so we can poke at the latest llvm-* tools and see what's working) 12:26 < fanquake> yea sounds good 12:27 < fanquake> cfields: pushed up the rebase 12:27 < cfields> Thanks! 12:42 < cfields> $ clang++ -B. -target x86_64-apple-darwin16 -mmacosx-version-min=10.12 --sysroot /home/cory/dev/bitcoin/depends/SDKs/MacOSX10.14.sdk -stdlib=libc++ -flto test.cpp test2.cpp -o testing 12:42 < cfields> cory@cory-desktop:15:41:57:~/dev/bitcoin/depends/x86_64-apple-darwin16/native/bin (bump-clang9) 12:42 < cfields> $ file testing 12:42 < cfields> testing: Mach-O 64-bit x86_64 executable, flags: 12:43 < cfields> fanquake: ^^ lto seems to just work already :) 12:43 < fanquake> ? 12:43 < cfields> whoops, sorry. 12:43 < cfields> lol, that was using my system clang, which worked. Depends clang doesn't. 12:43 < cfields> dongcarl should love that irony. 12:44 < dongcarl> MUAHAHAHAHAHAHA 12:44 < cfields> $ ./clang++ -B. -target x86_64-apple-darwin16 -mmacosx-version-min=10.12 --sysroot /home/cory/dev/bitcoin/depends/SDKs/MacOSX10.14.sdk -stdlib=libc++ -flto test.cpp test2.cpp -o testing 12:44 < cfields> '+cx8' is not a recognized feature for this target (ignoring feature) 12:44 < cfields> '+cx8' is not a recognized feature for this target (ignoring feature) 12:44 < dongcarl> cfields: where does that `+cx8` come from?! 12:44 < cfields> That's more like what I expected. Some busted IR generation I assume. 12:45 < dongcarl> Ah 12:45 < fanquake> Is it referring to this? https://www.felixcloutier.com/x86/cmpxchg8b:cmpxchg16b 12:47 < cfields> fanquake: that'd be my guess. 12:48 < fanquake> The compare-and-exchange 8 bytes (64 bits) instruction is supported (implicitly locked and atomic). 12:48 < fanquake> Some info in this Intel PDF: https://software.intel.com/file/36945 12:48 < cfields> It certainly should be a feature for that target. Sounds like a bug somewhere along the lines. 12:51 < cfields> Aha 12:51 < cfields> ld: could not parse object file /tmp/conftest-f84b31.o: 'Unknown attribute kind (62) (Producer: 'LLVM9.0.0' Reader: 'LLVM 6.0.0')', using libLTO version 'LLVM version 6.0.0' for architecture x86_64 12:51 < cfields> It's falling back to my sytem liblto. 12:57 < elichai2> cfields: may I ask what are you trying to do here? (just out of curiosity) 12:58 < cfields> elichai2: allow our linux -> osx toolchain to use lto. 12:58 < cfields> IIRC it's the only platform that can't be built that way atm. 12:58 < elichai2> Ohh OK. Oh mac os :/ 12:59 < cfields> elichai2: and by emitting llvm bitcode rather than native object files, I suspect that'll let us move to the llvm-tools rather than being constrained to apple's cctools. 12:59 < cfields> dongcarl: I probably should've explicitly stated ^^ above when mentioning the llvm-* tools. 13:00 < elichai2> I know almost nothing about Apple's stuff so don't mind me :) 13:00 < dongcarl> cfields: oh that part of the motivation to move to llvm-tools is to enable lto? 13:00 < cfields> dongcarl: other way around :) 13:00 < cfields> But yes. 13:01 < dongcarl> wait what other way? 13:01 * dongcarl confused 13:02 < elichai2> Cfields Btw if you're playing with low level Mac os stuff feel free to check this issue out :) https://github.com/bitcoin-core/secp256k1/issues/674 13:03 < cfields> Hmm, stack_not_16_byte_aligned_error 13:03 -!- jb55 [~jb55@gateway/tor-sasl/jb55] has joined #bitcoin-builds 13:04 < cfields> I assume that's due to asm stuff, but in that case I'd expect core's sha2 code to run into that as well? 13:04 < elichai2> Sounds like whatever tool chain it is meses up a vectorizing round by producing unaligned avx512 reading 13:05 < elichai2> lldb says it's movdql 13:05 < elichai2> Which is AVX. And Seco has no AVX 13:05 < elichai2> *libsecp 13:06 < cfields> Looks like sse2 to me? 13:10 < elichai2> Right I'm sorry. Idk why I remembered AVX 13:12 < cfields> fanquake: blah, cctools buildsystem needs some patches for liblto. It's not very graceful about finding the lib. 13:16 < cfields> elichai2: without looking deeper into it, and assuming it's not just an upstream compiler bug, I'd say clang might've tightened one of their requirements, and we may need to use unaligned moves. Should be simple enough to test, I should think. 13:17 < cfields> Should probably assume the stack is smashed there though, debugging those are tough :( 13:18 < elichai2> Well this still happens even when assembly and gmp are disabled. 13:18 < cfields> That's also assuming that it's something that we've done, and not it's not just being auto-vectorized. 13:19 < elichai2> And sjors said that O0 and O1 don't crash. Only O2 and above 13:19 < cfields> So no -msse* flags are being passed?? 13:20 < cfields> Oh wait, those might always be on by default for x86_64? Or if not, they could be turned on for the apple toolchain specifically, since their hardware is known. 13:21 < elichai2> Prettt sure libsecp autotools has no special instruction enabled on purpose 13:22 < fanquake> cfields: as in the cctools-port build system? Or cctools itself needs fixing? 13:22 < elichai2> Meaning you don't find any `-m*` or specifying a native cpu 13:22 < cfields> fanquake: cctools-port 13:24 < cfields> elichai2: yes, true. 13:26 < elichai2> If you have any ideas on how it's *not* the compiler I'd love to hear. Because I really don't want it to be the compiler (means that we can't know if/when fixed) 13:32 < cfields> elichai2: possible to build a minified testcase? 13:33 < elichai2> cfields: it's really hard for me without access to a mac 13:33 < elichai2> Is it possible to easily run osx in VirtualBox? 13:33 < fanquake> No 13:33 < cfields> Ah, gotcha. 13:33 < elichai2> :/ 13:35 < fanquake> It must be a macOS 10.15 thing. I've tested on macOS 10.14, and don't see any issues. I have no intention of upgrading to 10.15 anytime soon heh. 13:36 < cfields> elichai2: My guess otherwise would be that Catalina has some new stack canary trick that's either unfriendly or buggy. 13:37 < cfields> It'd be worth messing around with (and potentially try disabling) the stack protection options, and seeing if that changes anything. 13:37 < cfields> Sorry, by that I meant clang's stack protection options. 13:38 < cfields> This stuff: https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L749 13:47 -!- Brielle89Conn [~Brielle89@ns334669.ip-5-196-64.eu] has joined #bitcoin-builds 13:57 < fanquake> From what I can tell, the version of macOS your running here shouldn't actually make a difference, you'd still be seeing the issue if you've updated to the latest Xcode, which turned stack checking on by default? 13:57 < fanquake> However I cannot reproduce any of the issues with any of the test cases I've seen. 13:58 < fanquake> Hmm actually I see. 13:58 < fanquake> If you set MACOSX_DEPLOYMENT_TARGET=10.15 you see the issues. 13:58 < fanquake> frame #0: 0x00007fff7004d3a6 libdyld.dylib`stack_not_16_byte_aligned_error 14:00 < fanquake> cfields If you're looking for a minimal example, I've reproduced using the first post in this thread: https://forums.developer.apple.com/thread/121887 14:01 < fanquake> It doesn't happen without the -mavx flag 14:01 < cfields> fanquake: heh, getaddrinfo isn't exactly minimal. 14:01 < cfields> Hmm 14:01 < cfields> fanquake: what about with -msse4.2 ? 14:02 < fanquake> -msse4.2 works ok 14:02 < cfields> -mavx is a big hammer that implies other feature sets. I suspect it may be a red herring. 14:02 < cfields> Hmm. 14:02 < cfields> Does it compile to the same instructions? 14:02 < cfields> Blah, I'll just grab it and mess around. 14:02 < cfields> Thanks :) 14:02 < fanquake> heh ok 14:02 < fanquake> I'm just using MACOSX_DEPLOYMENT_TARGET=10.15 cc -mavx -O2 a.c && ./a.out 14:03 < cfields> And it crashes for you on 10.14 ? 14:03 < fanquake> yea 14:03 < cfields> Oh, interesting. 14:13 < fanquake> cfields: so I can make libsec crash as reported if I compile and make check while targetting 10.15 14:13 < cfields> fanquake: does it happen if you build on mac? 14:13 < cfields> Or cross only? 14:13 < fanquake> I've only been testing on a mac so far 14:14 < fanquake> using Xcode Clang 14:14 < cfields> Oh, even better. Ok. 14:19 < fanquake> I cannot reproduce with brew install LLVM Clang 9.0.0 when targetting 10.15. So definitely an Xcode only issue. 14:30 < dongcarl> cfields: Does clang just ignore LIBRARY_PATH entirely?! 14:31 < cfields> dongcarl: afaik that's a libc runtime thing, independent of the binary that's running. 14:31 < cfields> So if you built against musl, maybe :) 14:31 < dongcarl> cfields: I think you're talking about LD_LIBRARY_PATH? 14:32 < cfields> dongcarl: lol, sure was. I was just messing with it locally. 14:33 < dongcarl> Yeah gcc uses LIBRARY_PATH to give -L's to the linker I think... 14:33 < cfields> dongcarl: huh, I didn't even know that existed. 14:33 < dongcarl> And clang doesn't... So now I gotta write a wrapper... 14:33 < cfields> Ah, in collect2 or something? 14:33 < dongcarl> cfields: I think so? I keep seeing collect2 without knowing what that does... 14:34 < cfields> dongcarl: my understanding is that it's pretty much the wrapper you're about to write. 14:34 < cfields> But I don't have much of an understanding either. 14:34 < dongcarl> Ah gotcha 14:38 < cfields> dongcarl: actually, ignore that. I think I'm way off 14:39 < dongcarl> Way off about collect2? 14:41 < cfields> Yeah. Don't let me steer you wrong on that. 14:41 < dongcarl> roger 14:46 < cfields> Fixed lto! 14:47 < dongcarl> Haha everything's short work once cfields has his eyes on it 14:47 < cfields> https://github.com/theuni/bitcoin/commits/bump-clang9 14:48 < cfields> I suppose that branch is turning into: bump everything to the latest and greatest to play around 14:48 < cfields> dongcarl: If I can focus one something for more than 5min :\ 14:48 < cfields> *on 14:50 < cfields> Note that I haven't actually tried building depends/core with lto. Only a small stub test app. 18:17 -!- Brielle89Conn [~Brielle89@ns334669.ip-5-196-64.eu] has quit [Ping timeout: 276 seconds] --- Log closed Sat Oct 26 00:00:45 2019