--- Log opened Wed Mar 10 00:00:53 2021
00:03 < wumpus> hebasto: well the idea is that depends also works on other OSes but im sure there's always small stuff that causes incompatibilities, in my experience it tends to work on other linux variants but not BSDs
00:06 < hebasto> wumpus: thanks for clarification
00:08 < wumpus> shell script and make is unfortunately awful for supporting even slightly different UNIX variants
00:11 < wumpus> in any case if you run into specific issues please report them
00:15 < MarcoFalke> jarolrod_: Can you compile afl++ with clang on mac? (That'd be preferred)
00:15 < MarcoFalke> Ideally clang-11 or higher
00:16 < MarcoFalke> the afl-fuzz brew package is on the unmaintained old afl fork
02:27 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
02:33 -!- jonatack [~jon@37.173.131.107] has quit [Read error: Connection reset by peer]
02:34 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
02:36 -!- robert_spigler [robertspig@gateway/shell/matrix.org/x-fqzdcayiqboawwcs] has quit [Quit: Bridge terminating on SIGTERM]
02:44 -!- robert_spigler [robertspig@gateway/shell/matrix.org/x-xaqdenlnphxppscx] has joined #bitcoin-builds
02:52 -!- jonatack [~jon@37.173.131.107] has quit [Read error: Connection reset by peer]
02:53 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
03:22 < wumpus> doing guix build of 21376
06:44 -!- jonatack [~jon@37.173.131.107] has quit [Read error: Connection reset by peer]
06:45 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Remote host closed the connection]
06:46 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
06:51 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-builds
07:31 -!- mol_ [~mol@unaffiliated/molly] has quit [Ping timeout: 260 seconds]
07:37 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-builds
08:10 < jarolrod_> going to try my first guix build for 21376
08:11 < hebasto> jarolrod_: cool! prepare yourself for a long run :)
08:12 < jarolrod_> MarcoFalke: i was trying to compile I passed in brew LLVM (clang) but it also needs gcc to compile. The apple "GCC" fails on shmget() and the actual GNU one fails to find gmp.h, so I could not compile. But I will investigate further later. Could be an issue on my end.
08:14 -!- jonatack [~jon@37.173.131.107] has quit [Ping timeout: 246 seconds]
08:15 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
08:15 -!- mol [~mol@unaffiliated/molly] has quit [Ping timeout: 264 seconds]
08:16 < MarcoFalke> I was looking for compile instructions in afl++ for macos, but it doesn't seem they have any
08:17 < MarcoFalke> you could be the first person to try this
08:17 < jarolrod_> MarcoFalke: Haha, It can be done. I saw a blog post of someone compile it for the M1 Mac
08:18 < MarcoFalke> You could also just get a cloud box with ubuntu
08:21 < jarolrod_> it's definitely not recommendable for someone on macOS to fuzz with afl++ because of non uniform fork() semantics on macOS. But, I want to do it to be able to respond to an issue on this if it pops up :)
08:26 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-builds
08:32 -!- mol [~mol@unaffiliated/molly] has quit [Read error: Connection reset by peer]
08:33 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-builds
08:40 -!- mol [~mol@unaffiliated/molly] has quit [Ping timeout: 246 seconds]
08:53 < jarolrod_> hebasto: what am I in for in terms of time on a 16c/32t desktop with 32gb ram
08:54 < MarcoFalke> jarolrod_: With or without substitutes?
08:54 < jarolrod_> without
09:06 < jarolrod_> hebasto: where do I get/extract the macOS sdk
09:08 < hebasto> jarolrod_: https://github.com/bitcoin/bitcoin/tree/master/contrib/macdeploy
09:09 < hebasto> or https://bitcoincore.org/depends-sources/sdks/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers.tar.gz
09:10 < jarolrod_> hebasto: thanks!
09:43 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-builds
10:07 < jarolrod_> hebasto: `all build users are currently in use; consider creating additional users and adding them to the 'guixbuild' group`
10:08 < jarolrod_> is there a quick fix to this?
10:08 < sipa> MAXJOBS=10
10:08 < sipa> if you have 10 guixbuild users
10:10 -!- jonatack [~jon@37.173.131.107] has quit [Quit: jonatack]
10:10 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
10:10 -!- jonatack [~jon@37.173.131.107] has quit [Read error: Connection reset by peer]
10:10 < jarolrod_> sipa: thanks!
10:10 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
10:11 < MarcoFalke> or create 32 build users, if you want to use your 32threads
10:11 -!- jonatack [~jon@37.173.131.107] has quit [Client Quit]
10:11 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
10:12 < sipa> you can also increase max threads per build instead of having more builds in parallel
10:12 -!- jonatack [~jon@37.173.131.107] has quit [Client Quit]
10:12 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
10:13 < MarcoFalke> wait, there's a difference?
10:14 < sipa> yes
10:14 < sipa> one is just the make -j argument in individial builds
10:14 < sipa> the other is the number of builds
10:20 < MarcoFalke> MAX_JOBS is the -j argument?
10:21 < MarcoFalke> how to increase the number of builds?
10:22 < sipa> no, MAX_JOBS is the number of builds
10:22 < sipa> --cores is to change the -j
10:24 -!- copumpkin [~copumpkin@unaffiliated/copumpkin] has quit [Remote host closed the connection]
10:25 -!- copumpkin [~copumpkin@unaffiliated/copumpkin] has joined #bitcoin-builds
10:28 < jarolrod_> got guix building 👍
10:29 < jarolrod_> how long of a walk should I go for while its building?
10:29 < sipa> are you doing bootstrap?
10:29 < jarolrod_> im not using substitutes
10:29 < sipa> ok
10:30 < sipa> but guix still has two modes; one with the "normal" bootstrap, and one with the minimal bootstrap
10:30 < sipa> the latter can be selected with --bootstrap
10:30 < jarolrod_> oh I mean I have guix installed
10:30 -!- mol_ [mol@gateway/vpn/protonvpn/molly] has joined #bitcoin-builds
10:30 < jarolrod_> im running the deterministic build script
10:31 < jarolrod_> I didn't add bootstrap to the GUIX environment flags
10:31 < jarolrod_> so I assume minimal bootstrap
10:32 < MarcoFalke> by default is used substitutes, I think
10:32 < sipa> jarolrod_: --bootstrap == minimal bootstrap
10:32 < sipa> without == normal bootstrap
10:32 < jarolrod_> oh ok, when I Installed guix I chose no substitutes
10:32 < sipa> did you pass --no-substitutes to guix-daemon ?
10:33 < MarcoFalke> git grep cores ./contrib/guix/ | wc -l
10:33 < MarcoFalke> 0
10:34 < sipa> ADDITIONAL_GUIX_COMMON_FLAGS="--cores=8"
10:34 -!- mol [~mol@unaffiliated/molly] has quit [Ping timeout: 260 seconds]
10:34 < MarcoFalke> thx
10:35 < jarolrod_> i isntalled guix with the following install script: https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
10:35 < jarolrod_> and in the menu options i chose no substitutes
10:36 < sipa> ok, no idea what that does
10:37 < copumpkin> substitutes = go to binary cache instead of building from scratch, at least if the terminology was preserved from Nix
10:38 < MarcoFalke> maybe preserve in the config profile that you don't want substitutes?
10:38 < sipa> copumpkin: indeed
10:38 < sipa> also, hi!
10:38 < copumpkin> hi :)
10:39 < jarolrod_> The script automates the binary installation instructions: https://guix.gnu.org/manual/en/html_node/Binary-Installation.html
10:39 < jarolrod_> step 7: I didn't authorize using substitutes from guix
10:46 -!- jonatack [~jon@37.173.131.107] has quit [Read error: Connection reset by peer]
10:47 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
10:49 < jarolrod_> `guix time-machine: error: build of `/gnu/store/mvf88n2v90jjxg9n8b315p22r6jrkbyb-libx11-1.6.A.drv' failed`
10:49 < jarolrod_> I think I should just do this on ubuntu instead of arch
10:49 < sipa> just try again
10:49 < sipa> sometimes there are concurrency issues or test issues
10:49 < sipa> that are nondeterminsitic, despite the build product still being deterministic
10:50 < jarolrod_> yeah keeps failing at the same place as I retry
10:58 -!- jonatack [~jon@37.173.131.107] has quit [Quit: jonatack]
11:28 -!- jonatack [~jon@37.173.131.107] has joined #bitcoin-builds
11:41 < jonasschnelli> with --timeout-factor=8,... I'm still seeing test_runner timeouts (generatetoaddress)
11:41 < jonasschnelli> https://bitcoinbuilds.org/index.php?ansilog=28537952-2c92-46f2-9871-8918e5ba2738.log#l2398
11:41 < jonasschnelli> 'generatetoaddress' RPC took longer than 240.000000 seconds
11:42 -!- thrasher`_ [~thrasher@173.209.42.7] has joined #bitcoin-builds
11:42 -!- Netsplit *.net <-> *.split quits: thrasher`
12:04 -!- ctrlbreak_MAD [~ctrlbreak@159.2.165.130] has joined #bitcoin-builds
12:07 -!- ctrlbreak [~ctrlbreak@159.2.165.130] has quit [Ping timeout: 260 seconds]
12:11 < MarcoFalke> maybe increase it more?
12:12 < MarcoFalke> Alternative is to split the loop into separate calls
12:12 < MarcoFalke> generate(i) is identical to [generate(1) for _ in range(i)]
12:13 < jonatack> jonasschnelli: I'm seeing that error locally too, will propose a patch
12:14 < jonatack> istm it's in a recently merged test addition
12:15 < jonatack> "Test fundrawtx where BnB solution would result in a too large transaction, but Knapsack would not..."
12:29 < jonasschnelli> thanks...
12:40 -!- Jackielove4u [uid43977@gateway/web/irccloud.com/x-vulfyaligsvkvsgx] has quit []
12:41 -!- Jackielove4u [uid43977@gateway/web/irccloud.com/x-lbtetyyqbzzbklbc] has joined #bitcoin-builds
13:24 < jonatack> jonasschnelli: 21373, 21410 and 21411 hopefully help with timeouts in feature_nulldummy, rpc_fundrawtransaction, and wallet_groups
13:31 -!- jonatack_ [~jon@37.171.231.252] has joined #bitcoin-builds
13:34 -!- jonatack_ [~jon@37.171.231.252] has quit [Client Quit]
13:34 -!- jonatack_ [~jon@37.171.231.252] has joined #bitcoin-builds
13:35 -!- jonatack [~jon@37.173.131.107] has quit [Ping timeout: 264 seconds]
13:53 -!- jonatack_ [~jon@37.171.231.252] has quit [Quit: jonatack_]
13:54 -!- jonatack [~jon@37.171.231.252] has joined #bitcoin-builds
16:09 < jarolrod_> the guix process is surprisingly very simple
16:10 < jarolrod_> nice script 👍
16:15 < dongcarl> jarolrod_: Haha glad you like it :-)
16:17 < dongcarl> Argh I apologize for not seeing messages aboce
16:17 < dongcarl> above
16:17 < dongcarl> Feel free to ping me next time.
16:17 < dongcarl> A lot of instructions are in local branches that I'm cleaning up right now, but they answer all of these questions...
16:18 < dongcarl> If a derivation like /gnu/store/mvf88n2v90jjxg9n8b315p22r6jrkbyb-libx11-1.6.A.drv fails to build, I recommend doing `guix build --cores=1 /gnu/store/mvf88n2v90jjxg9n8b315p22r6jrkbyb-libx11-1.6.A.drv`
16:18 < dongcarl> Most of them are concurrency issues
16:18 < dongcarl> If you're stepping away for the night, you can also just ADDITIONAL_GUIX_COMMON_FLAGS="--cores=1"
16:18 < dongcarl> That way all packages will be built with 1 thread, which is slower, but avoids concurrency issues.
16:20 < dongcarl> jarolrod_: If you are using no substitutes, you may run into the gnutls certificate expiration problem, which will be solved in the next minor Guix release
16:20 < dongcarl> jarolrod_: Feel free to reach out to me directly for any help :-)
16:21 < jarolrod_> dongcarl: thanks I will next time. I ended up using substitutes because of the libx11 issue
16:21 < jarolrod_> Thanks for the tips!
16:22  * dongcarl makes note to self that it may be better to have a separate guide for --no-substitute builders
18:00 -!- Jackielove4u [uid43977@gateway/web/irccloud.com/x-lbtetyyqbzzbklbc] has quit [Quit: Connection closed for inactivity]
18:01 -!- molz_ [mol@gateway/vpn/protonvpn/molly] has joined #bitcoin-builds
18:05 -!- mol_ [mol@gateway/vpn/protonvpn/molly] has quit [Ping timeout: 265 seconds]
18:20 -!- jonatack [~jon@37.171.231.252] has quit [Ping timeout: 265 seconds]
18:45 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Ping timeout: 246 seconds]
19:56 -!- belcher_ [~belcher@unaffiliated/belcher] has joined #bitcoin-builds
19:59 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 246 seconds]
23:18 < wumpus> yes! the guix build is surprisingly simple (from a user perspective), i think it has the potential to just be a convenient build system, it has almost zero special setup, using gitian always had some mental overhead to me (even when automated with a stack of scripts)
23:31 -!- Jackielove4u [uid43977@gateway/web/irccloud.com/x-yxxvxjiwadouvumq] has joined #bitcoin-builds
--- Log closed Thu Mar 11 00:00:52 2021