--- Log opened Sat Dec 12 00:00:08 2020 --- Day changed Sat Dec 12 2020 00:00 < sipa> (i know we did that before, but perhaps that before we knew that history of sig sizes the binary has been through mattered) 00:00 < achow101> as before vmsize 0x000000000007c000 00:00 < sipa> ok 00:00 < sipa> well, if it's an alignment issue, 0x48000 is 0x4000 aligned, so unlikely to cause problem 00:01 < jonasschnelli> https://bitcoin.jonasschnelli.ch/bitcoin-0.21.0rc3-osx-unsigned-again-pagesize0.zip <--- signed with pagesize 0, check that is was not already codesigned 00:01 < sipa> so i expect this will be a solution 00:01 < jonasschnelli> (gitian output follows) 00:02 < achow101> sipa: it's aligning against 0x2000 00:03 < sipa> achow101: you found that by trying more sizes? 00:03 < achow101> if I use 4096 as the size, I get 46000. same with 8192 00:04 < jonasschnelli> and if anyone wants to test 0.20.2rc1, I just created the detached signature (no gitian build yet): https://bitcoin.jonasschnelli.ch/signature-osx_0.20.2rc1.tar.gz 00:04 < sipa> ok 00:04 < sipa> achow101: we could patch out cctools to do the same 00:04 < sipa> *our 00:04 < sipa> it's a tiny change 00:04 < achow101> I'm not optimistic that that's a lasting solution 00:05 < sipa> yeah, agree 00:06 < sipa> but it's simpler than trying to hack around the issue with multiple preallocate calls and passing the size through 00:07 < achow101> https://github.com/achow101/bitcoin/commit/26f8ebcca26eb74b6546b5d551f3ec468e2878be is my current proposal 00:07 < achow101> if we combine with pagesize 0, then it'll probably always work 00:08 < achow101> the largest signature since 0.12.1 is 407424 00:10 < sipa> achow101: if the gitian preallocate takes it to a large *odd* multiple of 0x1000, will the macos sign tool not increase it further to a multiple of 0x2000 ? 00:11 < sipa> if it's anything like the opensource version, it won't 00:11 < sipa> but who knows what it does 00:12 < jonasschnelli> oh! 00:12 < jonasschnelli> pagesize 0 result in a valid signatue (through gitian) 00:12 < jonasschnelli> I didn't before because I signed an already signed binary 00:13 < jonasschnelli> https://bitcoin.jonasschnelli.ch/bitcoin-osx-signed_pagesize0_clean.dmg 00:13 < jonasschnelli> (used detach signature from https://bitcoin.jonasschnelli.ch/bitcoin-0.21.0rc3-osx-unsigned-again-pagesize0.zip ) 00:13 < achow101> I think that is currently the expected result 00:14 < sipa> jonasschnelli: yeah, matches our understanding 00:14 < achow101> sipa: it does not appear to shrink the vmsize at all 00:15 < sipa> achow101: yes, but it may increase it further 00:15 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has quit [Ping timeout: 260 seconds] 00:15 < sipa> (i don't expect it to, but we should check i think) 00:15 < achow101> i'll try against the macports cctools 00:16 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has joined #bitcoin-core-dev 00:19 < achow101> it preserves it 00:20 < sipa> great 00:20 < sipa> 500000 is probably overkill if we're going to use pagesize 0 00:21 < sipa> but that does like a pretty simple fix 00:21 < achow101> probably, but I also have no idea what goes into the signature size 00:22 < achow101> i'll open a pr when i wake up 00:22 < jonasschnelli> thanks achow101! 00:22 < jonasschnelli> and sipa 00:22 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 00:22 < bitcoin-git> [bitcoin] MarcoFalke pushed 6 commits to master: https://github.com/bitcoin/bitcoin/compare/6a4806367177...ffc4d0499020 00:22 < bitcoin-git> bitcoin/master 5aaeb6c Russell Yanofsky: MOVEONLY: Move IsBDBFile, IsSQLiteFile, and ListWalletDir 00:22 < bitcoin-git> bitcoin/master 6ee9cbd Russell Yanofsky: refactor: Replace ListWalletDir() function with ListDatabases() 00:22 < bitcoin-git> bitcoin/master 6a7a636 Russell Yanofsky: refactor: Drop call to GetWalletEnv in wallet salvage code 00:22 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 00:23 < jonasschnelli> I knew that when I'm going to wake up that you guys will have a solution 00:23 < achow101> lol 00:23 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 00:23 < bitcoin-git> [bitcoin] MarcoFalke merged pull request #20275: wallet: List all wallets in non-SQLite and non-BDB builds (master...pr/exist) https://github.com/bitcoin/bitcoin/pull/20275 00:23 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 00:24 < sipa> achow101: number of pages, and "resources" are counted separately 00:24 < sipa> with pagesize=0 the executable code is always 1 page 00:27 < sipa> so i expect it to be mostly constant with pagesize=0 00:34 < jonasschnelli> I'm also getting an invalid signature for 0.20.2rc1 00:34 < jonasschnelli> shall I codesign with --pagesize 0? 00:35 < sipa> jonasschnelli: that may work (50% chance) 00:36 < sipa> jonasschnelli: basically any pagesize you try (has to be zero or a power of 2) has an independent 50% chance of being succesful with the current tooling 00:36 < sipa> with achow's fix it should always work 00:36 < sipa> better just wait; it can be backported too 00:38 < jonasschnelli> ok 01:40 -!- mj_node [~mj_node@122.0.25.130] has quit [Quit: Leaving] 01:49 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 264 seconds] 01:52 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:759a:a9e9:8a19:69d2] has quit [Remote host closed the connection] 02:04 -!- kexkey [~kexkey@static-198-54-132-158.cust.tzulo.com] has quit [Ping timeout: 260 seconds] 02:19 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 02:37 -!- instagibbs [~greg@061093103011.ctinets.com] has quit [Ping timeout: 246 seconds] 02:42 -!- Pavlenex [~Thunderbi@141.98.103.251] has joined #bitcoin-core-dev 03:07 -!- jonatack [~jon@213.152.161.244] has quit [Ping timeout: 256 seconds] 03:08 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 256 seconds] 03:18 -!- Kenny51Hand [~Kenny51Ha@static.57.1.216.95.clients.your-server.de] has joined #bitcoin-core-dev 03:23 -!- Kenny51Hand [~Kenny51Ha@static.57.1.216.95.clients.your-server.de] has quit [Ping timeout: 272 seconds] 03:24 -!- jonatack [~jon@88.124.242.136] has joined #bitcoin-core-dev 03:29 -!- jonatack [~jon@88.124.242.136] has quit [Ping timeout: 264 seconds] 03:30 -!- jonatack [~jon@213.152.161.40] has joined #bitcoin-core-dev 03:35 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 03:35 < bitcoin-git> [bitcoin] MarcoFalke pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/ffc4d0499020...b18978066d87 03:35 < bitcoin-git> bitcoin/master fad68af MarcoFalke: p2p: Ignore non-version msgs before version msg 03:35 < bitcoin-git> bitcoin/master faaad1b MarcoFalke: p2p: Ignore version msgs after initial version msg 03:35 < bitcoin-git> bitcoin/master b189780 MarcoFalke: Merge #20079: p2p: Treat handshake misbehavior like unknown message 03:35 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 03:36 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 03:36 < bitcoin-git> [bitcoin] MarcoFalke merged pull request #20079: p2p: Treat handshake misbehavior like unknown message (master...2010-p2pNoVersion) https://github.com/bitcoin/bitcoin/pull/20079 03:36 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 03:40 -!- sledges [~sledges@185.163.110.125] has quit [Remote host closed the connection] 03:58 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 04:11 -!- Eagle[TM] [~EagleTM@unaffiliated/eagletm] has joined #bitcoin-core-dev 04:18 -!- Talkless [~Talkless@mail.dargis.net] has joined #bitcoin-core-dev 04:21 -!- Talkless [~Talkless@mail.dargis.net] has quit [Client Quit] 04:21 -!- peterrizzo_ [~peterrizz@ool-44c18924.dyn.optonline.net] has joined #bitcoin-core-dev 04:35 -!- Talkless [~Talkless@mail.dargis.net] has joined #bitcoin-core-dev 04:35 -!- slidercrank1 [~slidercra@s91904426.blix.com] has joined #bitcoin-core-dev 04:40 -!- rex4539 [~rex4539@balticom-197-78.balticom.lv] has joined #bitcoin-core-dev 05:08 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 256 seconds] 05:23 -!- instagibbs [~greg@061093103011.ctinets.com] has joined #bitcoin-core-dev 05:24 -!- instagibbs [~greg@061093103011.ctinets.com] has quit [Client Quit] 05:27 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-core-dev 05:30 -!- mol_ [~mol@unaffiliated/molly] has quit [Ping timeout: 260 seconds] 05:35 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-dev 05:35 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Disconnected by services] 05:35 -!- vasild_ is now known as vasild 05:49 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:a9f4:e07:a3ac:95af] has joined #bitcoin-core-dev 05:53 -!- slidercrank1 [~slidercra@s91904426.blix.com] has quit [Remote host closed the connection] 05:55 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 05:56 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has joined #bitcoin-core-dev 05:57 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 272 seconds] 05:58 -!- rh0nj [~rh0nj@88.99.167.175] has quit [Remote host closed the connection] 05:59 -!- rh0nj [~rh0nj@88.99.167.175] has joined #bitcoin-core-dev 06:06 -!- twistedline [~twisted@unaffiliated/twistedline] has quit [Remote host closed the connection] 06:06 -!- twistedline [~twisted@unaffiliated/twistedline] has joined #bitcoin-core-dev 06:08 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has joined #bitcoin-core-dev 06:12 -!- veox [~veox@185.163.110.125] has joined #bitcoin-core-dev 06:18 -!- kristapsk_ [~KK@gateway/tor-sasl/kristapsk] has quit [Remote host closed the connection] 06:19 -!- kristapsk_ [~KK@gateway/tor-sasl/kristapsk] has joined #bitcoin-core-dev 06:20 -!- twistedline [~twisted@unaffiliated/twistedline] has quit [Ping timeout: 264 seconds] 06:27 -!- twistedline [~twisted@unaffiliated/twistedline] has joined #bitcoin-core-dev 06:36 -!- peterrizzo_ [~peterrizz@ool-44c18924.dyn.optonline.net] has quit [Quit: peterrizzo_] 06:38 -!- twistedline [~twisted@unaffiliated/twistedline] has quit [Ping timeout: 246 seconds] 06:44 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 06:45 -!- twistedline [~twisted@unaffiliated/twistedline] has joined #bitcoin-core-dev 06:48 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has joined #bitcoin-core-dev 06:55 -!- mol [~mol@unaffiliated/molly] has quit [Ping timeout: 240 seconds] 07:04 -!- Pavlenex [~Thunderbi@141.98.103.251] has quit [Quit: Pavlenex] 07:09 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 256 seconds] 07:22 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev 07:27 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has quit [Remote host closed the connection] 07:29 -!- darosior [~darosior@194.36.189.246] has quit [Remote host closed the connection] 07:30 -!- darosior [~darosior@194.36.189.246] has joined #bitcoin-core-dev 07:32 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev 07:36 -!- Guyver2 [Guyver@guyver2.xs4all.nl] has joined #bitcoin-core-dev 07:42 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has joined #bitcoin-core-dev 07:46 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 240 seconds] 07:48 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has quit [Remote host closed the connection] 07:48 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 07:49 -!- miketwenty1 [~miketwent@136.55.84.49] has joined #bitcoin-core-dev 07:52 -!- Pavlenex [~Thunderbi@141.98.103.251] has joined #bitcoin-core-dev 07:53 -!- miketwenty1 [~miketwent@136.55.84.49] has quit [Ping timeout: 258 seconds] 07:56 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev 07:57 -!- belcher_ [~belcher@unaffiliated/belcher] has joined #bitcoin-core-dev 07:59 -!- tryphe [~tryphe@unaffiliated/tryphe] has quit [Read error: Connection reset by peer] 07:59 -!- tryphe_ [~tryphe@unaffiliated/tryphe] has joined #bitcoin-core-dev 08:00 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 256 seconds] 08:04 -!- jonatack [~jon@213.152.161.40] has quit [Ping timeout: 240 seconds] 08:06 < prusnak> do you know if https://github.com/shesek is on IRC? 08:06 -!- belcher_ is now known as belcher 08:06 -!- jonatack [~jon@109.232.227.138] has joined #bitcoin-core-dev 08:06 < belcher> prusnak yes hes called shesek 08:06 < prusnak> shesek: no such nick 08:07 < prusnak> thanks, i was asking because he's offline most probably 08:25 < achow101> sipa: I found the codesign source. It's in a library called "Security": https://opensource.apple.com/source/Security/Security-59306.140.5/OSX/libsecurity_codesigning/lib/ 08:29 < achow101> Would it be worthwhile to write a tool that lets us code sign macOS from linux? Given that we can read the original code signing source, I don't think it would be too difficult to write one. 08:30 -!- wumpus [~ircclient@pdpc/supporter/professional/wumpus] has quit [Read error: Connection reset by peer] 08:32 -!- rex4539_ [~rex4539@8.40.29.10] has joined #bitcoin-core-dev 08:32 -!- wumpus [~ircclient@pdpc/supporter/professional/wumpus] has joined #bitcoin-core-dev 08:33 -!- rex4539 [~rex4539@balticom-197-78.balticom.lv] has quit [Read error: Connection reset by peer] 08:36 -!- rex4539_ [~rex4539@8.40.29.10] has quit [Ping timeout: 260 seconds] 08:40 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev 08:42 -!- jonatack [~jon@109.232.227.138] has quit [Quit: jonatack] 08:49 -!- dhruv [~dhruv@165.227.49.220] has joined #bitcoin-core-dev 08:50 -!- dhruv [~dhruv@165.227.49.220] has quit [Client Quit] 08:51 -!- dhruvm [~dhruv@165.227.49.220] has joined #bitcoin-core-dev 08:58 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 08:58 < bitcoin-git> [bitcoin] achow101 opened pull request #20638: Mac codesign fixed alloc (master...mac-codesign-fixed-alloc) https://github.com/bitcoin/bitcoin/pull/20638 08:58 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 09:01 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-core-dev 09:05 -!- jonatack [~jon@88.124.242.136] has joined #bitcoin-core-dev 09:08 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 256 seconds] 09:10 -!- jonatack [~jon@88.124.242.136] has quit [Ping timeout: 240 seconds] 09:10 -!- jonatack [~jon@213.152.162.84] has joined #bitcoin-core-dev 09:17 -!- dermoth_ [~dermoth@unaffiliated/dermoth] has joined #bitcoin-core-dev 09:18 -!- dermoth [~dermoth@unaffiliated/dermoth] has quit [Disconnected by services] 09:18 -!- dermoth_ is now known as dermoth 09:25 -!- davterra [~davterra@107.182.237.18] has joined #bitcoin-core-dev 09:30 -!- Pavlenex [~Thunderbi@141.98.103.251] has quit [Quit: Pavlenex] 09:38 < sipa> achow101: maybe, depending on how hard it is to compile that for linux 09:40 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 256 seconds] 09:40 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 256 seconds] 09:45 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev 09:58 < wumpus> achow101: nice! 10:01 < achow101> sipa: I don't think we can compile it ourselves. It would have to be a completely separate implementation. 10:02 < sipa> achow101: that sounds nontrivial 10:02 < wumpus> assuming, of course, it's the same as the binary they ship 10:03 < sipa> the benefit is that if someone actually makes that work, we may be able to replace it with multiparty RSA or s 10:03 < achow101> it appears to be nontrivial. the code is a bit hard to understand 10:03 < sipa> o 10:04 < achow101> I just wanted to make it use rfc6979 10:04 < wumpus> if it's RSA in the first place 10:04 < achow101> the cert is rsa 10:05 < achow101> it's amazing how hard it is to actually find the function that does the cryptographic signature 10:07 < sipa> i assume the signature contais a certnl chain too 10:07 < wumpus> you'd say if you're going to do something like per-page signing you'd use an algorithm with short signatures :) 10:07 < sipa> which may complicate thihgs 10:07 < sipa> and the code for deciding exactly is being signed may be nontrivial too 10:07 < wumpus> that explains why no one else made a portable signer i guess 10:08 < achow101> I'm surprised no one has put in the effort to make one given that it's actually open source 10:08 < sipa> i wonder how many people even care about doing codesigning macos binaries on a non-macos platform 10:09 < wumpus> true, not that many projects are doing deterministic builds 10:10 < wumpus> though even for non-deterministic builds it can often be useful to do all builds on a single build host platform 10:11 -!- mol_ [~mol@unaffiliated/molly] has joined #bitcoin-core-dev 10:12 < achow101> even with per-page signing, shouldn't the signature be way larger than 200k? 10:12 < achow101> given our binary size 10:13 < sipa> what is the actual binary size without resources? 10:14 < sipa> i think jonasschnelli's tgz has a file that lists all the resources 10:14 -!- mol [~mol@unaffiliated/molly] has quit [Ping timeout: 265 seconds] 10:15 < achow101> 26891600 Dec 12 11:40 Bitcoin-Qt 10:16 < sipa> do you know how many bits the RSA key is? 10:17 < achow101> hmm. the cert is not in contrib/macdeploy.. I thought it was 10:17 < sipa> achow101: perhaps you can find in that apple source code what the default page size is 10:19 < sipa> perhaps the default page size is 32k or 64k 10:19 < achow101> turns out codesign has a lot of --verbose options 10:19 < achow101> the page size is 4096 in the rc3 sig 10:19 < sipa> hmm 10:20 < sipa> that makes little sense 10:21 < achow101> here is the full verbose output of codesign -d: https://0bin.net/paste/2gi9BEF9#YOEGkrrhbpToojZbGKJhHvcYp5H1zV3MfO+skMBRotg 10:21 < sipa> well that"s 35 bytes per sig 10:21 < achow101> it looks like it's actually just a ton of hashes and the signature is over all of them? 10:21 < achow101> it says Signature size=8973 10:21 < sipa> ah! 10:21 < sipa> it's a merkle tree over the binary? 10:21 < sipa> that makes sens 10:22 < achow101> couldn't say. but 6463 * 32 is in the ballpark of the signature size 10:23 < achow101> and there's 6432 hashes. that also fits with the page size and binary size, 6432 * 4096 is near the binary size 10:23 < achow101> *6463 hashes 10:24 < sipa> yup 10:24 < sipa> i suspect that's it exactly 10:24 < achow101> making hashes of each page seems excessive though 10:25 < sipa> it allows for fast per-page validation 10:25 < sipa> without needing to hash all the leaves every time 10:28 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 10:28 < bitcoin-git> [bitcoin] jbampton opened pull request #20639: doc: fix case of GitHub (master...fix-case-of-github) https://github.com/bitcoin/bitcoin/pull/20639 10:28 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 10:32 < achow101> I don't think it's a merkle tree. I'm pretty sure it's just a flat list of hashes 10:33 < achow101> that's what gets hashes again and signed 10:35 < wumpus> that makes sense 10:36 < wumpus> a hash would be much more efficient to verify per page, after all, and the signature check can be done once at load time for the hashes section 10:37 < wumpus> per page (or bunch-of-pages) makes sense as it decreases the granularity of loading, a hash over the entire binary means the entire binary has to be read 10:38 < achow101> so our trick to do a single page may not be ideal 10:38 < wumpus> well if it works as a workaround 10:38 < wumpus> but not in the long term no i guess 10:39 < achow101> well we now know mostly how big the signature will be, so we could compute an overestimate instead 10:40 < sipa> achow101: why not? do we care about not loading all pages all at once? 10:40 < wumpus> if someone wants to reverse engineer this i'd prefer having a signature *verifier* first, so the gitian process can detect if it went wrong 10:40 < wumpus> sipa: i guess so? there's always a lot of data in a binary that's not 'hot', might never be accessed at all, especially as we static link qt 10:41 < achow101> i dunno 10:41 < achow101> it seems like there's a default for a reason 10:41 < achow101> and it's clearly architecture optimized 10:41 < wumpus> that said, given the memory requiremetns i'm not sure we care about paging much 10:42 < achow101> verification of the signature itself should only happen once 10:42 < sipa> oh you think it's re-verifying the pages every time they get loaded again from disk 10:42 < wumpus> yes, but the hash might be done more often, say if a page is paged out and in again? 10:42 < achow101> I would guess this is some kind of runtime integrity check 10:42 < sipa> that does make sense 10:42 < sipa> but just the hash, not the full sig 10:43 < achow101> yeah 10:43 < wumpus> yes, the sig verification for the hashes section is likely done only once 10:43 < sipa> but that means that the list of hashes must be loaded at all times 10:43 < wumpus> yes 10:44 < wumpus> so it'd be a compromise--too small page size and the list of hashes is huge, too big page size and it has very large paging granularity 10:46 < wumpus> and i'm not sure--if it wouldn't check every time a page is re-loaded from disk it might open up some kinds of exploits, espeically for network file systems if manipulation of the data can happen without the OS knowing 10:47 < wumpus> but that's not certain 10:48 < sipa> if it was a Merkle tree instead they wouldn't need to keep the hash lish loaded, but it'd be twice the size 10:52 < achow101> I think we can produce a good enough overestimate to allow for the default page size 10:53 < achow101> (((unsigned_size / 4096) + 1) * 32) + 50000 10:53 < sipa> yup 10:53 < wumpus> good! 10:53 < achow101> the sig is 9k, and there's an additional 10k of overhead. with + 50k, we get 30k of extra 10:54 < achow101> i'll update the pr 10:55 < phantomcircuit> it seems to be using https://en.wikipedia.org/wiki/Cryptographic_Message_Syntax 10:56 < achow101> oh! 10:56 < achow101> so openssl can be used? 10:56 -!- adiabat [~adiabat@63.209.32.102] has quit [Ping timeout: 272 seconds] 10:57 -!- adiabat [~adiabat@63.209.32.102] has joined #bitcoin-core-dev 10:59 < phantomcircuit> achow101, well to quote a comment 'Sign a Mach-O binary, using liberal dollops of that special Mach-O magic sauce.' 10:59 < phantomcircuit> so 10:59 < achow101> when I read that, I was not encouraged 10:59 < sipa> how to construct the Xcode extract for the 0.20 and 0.19 builds? 11:00 < achow101> sipa: see contrib/macdeploy/README in the 0.20 branch 11:00 < achow101> it explains how for both 11:00 < wumpus> if the download is still available on the apple site, hopefully 11:01 < sipa> https://github.com/bitcoin-core/docs/blob/master/gitian-building/gitian-building-mac-os-sdk.md seems to refer to very old stuff 11:01 < sipa> i guess it should be updated or deleted? 11:01 < sipa> or changed into a link to the macdeploy stuff? 11:01 < achow101> I think it should link to macdeploy 11:02 < wumpus> yea the information on a branch should probably only describe for that branch 11:02 < achow101> for some reason the 0.19 branch doesn't describe the extract process, but 0.20 does as "the old way" 11:03 < phantomcircuit> achow101, yeha i cant figure out where it's actually signing either 11:03 < phantomcircuit> im sure if i could download that entire directory from git or as a tar ball i could 11:04 < phantomcircuit> but no 11:04 -!- Zenton [~user@unaffiliated/vicenteh] has quit [Read error: Connection reset by peer] 11:04 < phantomcircuit> classic apple 11:04 < achow101> phantomcircuit: mirror of it here https://github.com/apple-opensource/Security/ 11:05 < achow101> slightly older version, but close enough 11:06 < achow101> I got to SecCodeSigner::Signer::signCodeDirectory in signer.cpp before I got super lost 11:06 < phantomcircuit> achow101, yeah that's definitely the right function 11:07 < phantomcircuit> i got to CMSEncoderAddSigners but can't tell what it's actually doing 11:07 < phantomcircuit> at least i think that's a decent place to look 11:08 < phantomcircuit> also, if(hashDict != NULL) {assert(hashDict != NULL); they must not trust their compiler very much 11:08 -!- Zenton [~user@unaffiliated/vicenteh] has joined #bitcoin-core-dev 11:08 < achow101> CMSEncoderCopyEncodedContent claims to do the actual signing, but I find no evidence that supports that. it just kinda goes into a while function stack that I can't understand 11:08 < achow101> *wild 11:09 < achow101> CMSEncoderAddSigners just adds some kind of signer object to a list 11:10 -!- Zenton [~user@unaffiliated/vicenteh] has quit [Read error: Connection reset by peer] 11:11 < phantomcircuit> achow101, CMSEncoderUpdateContent 11:13 < phantomcircuit> wait this looks like it's generating hashes and then encrypting them??? 11:13 < achow101> I think they use encrypt and sign interchangeably 11:14 -!- joelklabo [~textual@108-196-216-127.lightspeed.sntcca.sbcglobal.net] has quit [Quit: Textual IRC Client: www.textualapp.com] 11:14 < phantomcircuit> achow101, https://github.com/apple-opensource/Security/blob/3aaa9af09496be26b5c15ff011e3d65ca75821ac/libsecurity_smime/lib/cmsencode.c#L389 11:14 < phantomcircuit> iono kinda looks like it's actually encrypting things 11:15 < phantomcircuit> of course there's about 10 levels of indirection though so who can even tell 11:15 < achow101> I think it's just some more generic code that can also handle encryption? 11:15 < achow101> honstly it's hard to know 11:16 < sipa> for RSA encrypting and signing are kind-of interchangeable (but the devil is in the details) 11:18 -!- joelklabo [~textual@108-196-216-127.lightspeed.sntcca.sbcglobal.net] has joined #bitcoin-core-dev 11:20 < phantomcircuit> sipa, just gotta figure out where the ciphcx object is set.. which is uhhh 11:21 -!- Zenton [~user@unaffiliated/vicenteh] has joined #bitcoin-core-dev 11:29 < sipa> ok, downloading xocde 10.2.1 11:29 < sipa> *xcode 11:57 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:a9f4:e07:a3ac:95af] has quit [Remote host closed the connection] 12:04 < wumpus> can you see what hash algorithm they use? if so it might be most straightforward to check if the hashes are encrypted or otherwise obfuscating by hashing pages and looking if they appear in the signature section of the file 12:06 < wumpus> pages are bound to have an address aligned to the start of the file, otherwise it'd be wildly inefficient 12:08 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:a9f4:e07:a3ac:95af] has joined #bitcoin-core-dev 12:13 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:a9f4:e07:a3ac:95af] has quit [Ping timeout: 260 seconds] 12:15 < achow101> they use sha256, so it shouldn't be hard to check 12:21 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev 12:27 -!- sr_gi [~sr_gi@80.174.218.168.dyn.user.ono.com] has quit [Read error: Connection reset by peer] 12:27 -!- sr_gi [~sr_gi@80.174.218.168.dyn.user.ono.com] has joined #bitcoin-core-dev --- Log closed Sat Dec 12 12:37:25 2020 --- Log opened Sat Dec 12 12:37:25 2020 12:37 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 240 seconds] 12:42 < achow101> The hashes are definitely embedded 12:43 < achow101> From the 2nd page to the end of the file, the hashes in the signed binary match the unsigned one 12:43 < achow101> the 2nd page contains the alloc thing, so it's different, but if the alloc is done, it matches 12:43 < achow101> only the first page doesn't match. and then there are 3 "special" hashes that are of... something else 12:45 < wumpus> achow101: oh that's great! 12:46 < achow101> actually the first page matches, it just doesn't match the unsigned one 12:47 < wumpus> maybe it updates something in the header first 12:50 < sipa> well, we know it does :) 12:50 < sipa> if it didn't, this whole problem wouldn't exist 12:50 < sipa> the vmsize/filesize entries on the __LINKEDIT section are changed 12:50 < wumpus> yeah then it makes sense for the hash to match the new state, not the original one 12:51 < wumpus> right, true :) 12:58 < achow101> yep, first 2 pages differ from unsigned because of the addition of the code signing section 12:59 < achow101> so the hashes in the sig are 4096 byte pages from beginning of the signed binary to the beginning of the code signature 12:59 < achow101> this is something we can verify ourselves 13:07 -!- jonatack [~jon@213.152.162.84] has quit [Ping timeout: 256 seconds] 13:08 < sipa> ah, neat 13:09 < achow101> i figured out 2 of the 3 special hashes 13:10 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:25fd:4b36:9c2b:340] has joined #bitcoin-core-dev 13:10 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:25fd:4b36:9c2b:340] has quit [Client Quit] 13:17 < wumpus> that's fast :) 13:18 < sipa> last one may be some metadata with the signature/hashing parameters? 13:18 < achow101> looking at the source, I think it's "entitlements" 13:18 < achow101> but I don't know what that means 13:19 < sipa> the codesign tool has a way to provide something called entitlements to it 13:21 < achow101> one of the hashes is of the file Bitcoin-Qt.app/Contents/_CodeSignature/CodeResources, and the other is Bitcoin-Qt.app/Contents/Info.plist 13:22 < achow101> but this last hash is not any of the files in the bundle 13:23 < wumpus> if we don't pass "entitlements", maybe it's a hash of empty data? 13:23 < achow101> I don't think so 13:24 < achow101> The hash is Bitcoin-Qt.app/Contents 13:24 < achow101> 1761bc7a0f37e9809acd22c53f42ff21622ea02093c14058e7daa3a466c95d97 13:29 < sipa> is this hash the same for other releases? 13:29 < sipa> if it's just some constant we don't actually care what it's a hash of 13:34 -!- astrolince [astrolince@gateway/shell/matrix.org/x-pefoseecitekmcnv] has joined #bitcoin-core-dev 13:34 < wumpus> right a proto-verifyer could just compare it against the fixed hash and raise alarm if it doesn't match 13:34 < achow101> it's the same in 0.20.1, 0.21.0rc2, and 0.21.0rc3 13:35 < achow101> afaik we don't have any entitlements so it should just be the hash of an empty something 13:35 < sipa> google is of no help 13:39 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev 13:40 < wumpus> " 13:40 < wumpus> The specific entitlements will be embedded in the signature of an application. If you are having trouble, it can help to look at what the signature actually says about the entitlements: $ codesign -d --entitlements - Example.app will show an XML property list similar to the one above. 13:41 < achow101> it gives nothing so there are no entitlements 13:41 < wumpus> so it's a hash of something that is in the signature itself, that sounds strange but... 13:41 < sipa> maybe it's just or something 13:41 < wumpus> I'd expect so 13:42 < achow101> the source code has it as a datatype of CFDataRef which I think just means it's a void * 13:42 < sipa> wumpus: sounds similar to bitcoin's use of a sighash byte 13:42 < sipa> part of the signature itself, and also signed 13:43 < wumpus> sipa: ah yes 13:53 < achow101> is there such a thing as a generic DER decoder? 13:53 < sipa> yes 13:55 < sipa> https://github.com/pornin/DDer 13:55 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 256 seconds] 13:55 < sipa> ha, thomas pornin 13:55 < sipa> i've met him 13:57 < achow101> hmm, openssl should be able to decode this if it was properly CMS DER, but it doesn't like it 13:58 < wumpus> it apparently DiskRep::component(cdEntitlementSlot) gets the data that is hashed to verify the entitlement slot, there are different implementations of DispRep though, but in diskrep this is the file "CodeEntitlements" 13:58 < wumpus> filediskrep* 13:59 -!- abetusk [~abe@68.175.128.91] has joined #bitcoin-core-dev 14:00 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev 14:00 < wumpus> in CodeDirectory::slotIsPresent a non-zero digest in the slot means that is is present 14:02 < wumpus> i guess i should be looking at BundleDiskRep::component(cdEntitlementSlot) as it can fetch components from the executable itself 14:02 < achow101> I found a bunch of ASN.1 templates, this means it should be possible to figure the serialization, right? 14:03 < sipa> achow101: can you show me the DER data? 14:03 < achow101> sipa: in the bitcoin-detached-sigs repo osx/dist/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt.sign 14:03 < achow101> that's the code signature. in theory it's DER 14:03 < achow101> (checkout 0.21 branch) 14:03 < sipa> you don't actually need the schema to decode DER, it's self-descriptive 14:04 < sipa> the scheme just helps with naming stuff 14:04 < achow101> I'm not convinced it's actually DER 14:04 < wumpus> *i have no clue what i'm doing dog image* 14:05 < wumpus> this looks interesting: https://chromium.googlesource.com/experimental/chromium/src/+/refs/wip/bajones/webvr/chrome/browser/safe_browsing/signature_evaluator_mac.mm 14:06 < sipa> it's not 14:06 < wumpus> ok 14:07 < sipa> $ file Bitcoin-Qt.sign 14:07 < sipa> Bitcoin-Qt.sign: Mac OS X Detached Code Signature (non-executable) - 216249 bytes 14:07 < achow101> wait file recognizes this??? 14:09 < sipa> hmm where does file keeps its magic definitions these days? 14:09 < sipa> i only find a precompiled magic.mgc file 14:09 < achow101> I think that's it 14:09 < achow101> gotta find the source for file 14:09 < wumpus> from MachORep::compoenent to EmbeddedSignatureBlob::component now 14:10 < sipa> https://github.com/file/file/blob/master/magic/Magdir/apple 14:11 < wumpus> "An EmbeddedSignatureBlob is a SuperBlob indexed by component slot number." of coursee 14:11 < wumpus> class EmbeddedSignatureBlob : public SuperBlobCore 14:13 < sipa> _0x0ff: belong0xfade0cc1Mac OS X Detached Code Signature 14:13 < sipa> 0 belong 0xfade0cc1 Mac OS X Detached Code Signature 14:14 < wumpus> typedef SuperBlob<0xfade0cc1> DetachedSignatureBlob; // indexed by main architecture 14:14 < wumpus> superblobs, all of them 14:15 < sipa> turtles all the way down 14:15 < wumpus> hehe 14:15 < achow101> blobloblob 14:17 < wumpus> / as a genuine Blob (e.g. for insertion into a SuperBlob). 14:18 < wumpus> a superblob is a map of integers to blobs 14:18 < sipa> getting deeper 14:19 -!- yunier2002 [~yunier200@c-73-85-126-91.hsd1.fl.comcast.net] has joined #bitcoin-core-dev 14:19 < wumpus> "// echoes from parent BlobCore (the C++ type system is too restrictive here)" uh 14:21 < wumpus> tbh seems like it's actually all fairly straightforward C++ code 14:22 < wumpus> the comments are way scarier than the code itself 14:24 < sipa> wumpus: you're biased 14:24 < sipa> :) 14:25 < wumpus> i... suppose hehe 14:26 < achow101> i've somehow wandered into some code in this libsecurity that's Copyright Mozilla 14:44 -!- Guyver2 [Guyver@guyver2.xs4all.nl] has quit [Quit: Going offline, see ya! (www.adiirc.com)] 14:54 < sipa> so do we know how to extract the blobs from the EmbeddedSignatureBlob ? 15:00 < achow101> nope? 15:02 -!- Talkless [~Talkless@mail.dargis.net] has quit [Quit: Konversation terminated!] 15:04 < achow101> That unknown hash earlier is apparently "internal requirements", not entitlements 15:11 < wumpus> kind of, it's u32 magic (BE), u32 (BE) size, then u32 (LE) number of blobs, then per blob u32 (LE) blob type, u32 LE offset 15:11 < wumpus> then the data 15:12 < wumpus> blob type is the slot id, so 5 for the cdEntitlementSlot, but apprantly we're no longer looking for that 15:13 < sipa> ok so 3 blobs 15:14 < sipa> first blob is type 0, length 0x24 15:14 < wumpus> which is good as it doesn't have that one 15:14 < sipa> ah no, not length; offset 15:15 < sipa> the individual blobs don't have a length? 15:15 < wumpus> sipa: that seems correct? 15:15 < wumpus> yes, offset 15:15 < wumpus> the length is in the blob itself 15:15 < sipa> the first blob has magic fa de 0c 02 15:15 < wumpus> blobs are all 15:15 < sipa> got it 15:16 < achow101> list of magics in OSX/libsecurity_codesigning/lib/CSCommonPriv.h 15:16 < sipa> Mac OS X Code Directory is the first one 15:16 < wumpus> so yes we have 0 at offset 0x24, 2 at offset 0x2f7a4 and 0x100 at offset 0x2f864 15:17 < achow101> sipa: yes. this contains our list of hashes 15:17 < wumpus> (all integers are BE, it seemed from the code that they'd be LE but apparently they're swapped somewhere) 15:18 < achow101> it looks like the structure is the class members 15:18 < achow101> in order as listed 15:18 < sipa> second one has magic fa de 0c 01, Mac OS X Code Requirement Set 15:18 < wumpus> there's your requirement set to hash i guess 15:18 < sipa> last blob has magic fa de 0b 01 15:19 < wumpus> class BlobWrapper : public Blob 15:20 < sipa> the first blob has magic fa de 0c 02 15:20 < sipa> https://gist.github.com/nmoinvaz/cd8651ffb3659161423534b29824510a 15:20 < achow101> oh nice 15:21 < achow101> hmm, it can't be the structure as listed because I run into some strings before I'm supposed to 15:21 < achow101> but it's pretty close 15:21 < wumpus> euhh the third's slot id is 0x10000 which is cdSignatureSlot = 0x10000,| | | // CMS signature 15:22 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Remote host closed the connection] 15:22 < achow101> that might be DER encoded 15:22 < wumpus> which is wrapped in a BlobWrapper which is simply a blob header with arbitrary binary dat 15:22 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev 15:22 -!- miketwenty1 [~miketwent@ec2-34-202-224-110.compute-1.amazonaws.com] has joined #bitcoin-core-dev 15:22 < sipa> this has so many layers, i'd be surprised if it isn't somehow exploitable... 15:24 < achow101> In class CodeDirectory, everything in the class decodes in order up to spare3 15:25 < achow101> then we get to the offset for identifier string and the identifer string begins there, as expected 15:25 < wumpus> otoh the blob format is super simple, there could be a mistake of course, but yes e.g. DER parsers are notoriously exploitable 15:26 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Remote host closed the connection] 15:27 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev 15:30 < wumpus> the CMS signature is produced in SecCodeSigner::Signer::signCodeDirectory in signer.cpp 15:30 -!- veox [~veox@185.163.110.125] has quit [Remote host closed the connection] 15:30 < achow101> it seems like the message that is signed is the CodeDirectory blob 15:31 < wumpus> that's what the comment says too "Generate the CMS signature for a (finished) CodeDirectory.", in addition it takes a hashDict and hashList 15:32 -!- Barno7 [~bob@93.55.242.131] has quit [Remote host closed the connection] 15:38 < wumpus> where hashDict is apparently "v2" and hashList for "v1" 15:40 < wumpus> the encoding is indeed the Cryptographic Message Syntax what phantomcircuit linked to with apple specific tags 15:49 < phantomcircuit> this has so many layers, i'd be surprised if it isn't somehow exploitable... 15:49 < phantomcircuit> indeed 15:53 < achow101> wumpus: that Requirement Set blob is indeed the unknown hash. 15:55 -!- miketwenty1 [~miketwent@ec2-34-202-224-110.compute-1.amazonaws.com] has quit [Remote host closed the connection] 15:56 -!- jeremyrubin [~jr@c-73-15-215-148.hsd1.ca.comcast.net] has joined #bitcoin-core-dev 15:56 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 240 seconds] 15:58 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has joined #bitcoin-core-dev 16:00 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has quit [Remote host closed the connection] 16:00 -!- mol_ [~mol@unaffiliated/molly] has quit [Read error: Connection reset by peer] 16:01 < wumpus> achow101: great! 16:02 < wumpus> let's see if python asncrypto.cms can handle this 16:03 < achow101> I got something! 16:04 < achow101> the tool sipa mentioned earlier seems to work if I pass the CMS data through openssl cms first 16:06 -!- miketwenty1 [~miketwent@ec2-34-202-224-110.compute-1.amazonaws.com] has joined #bitcoin-core-dev 16:08 < wumpus> nice 16:09 < achow101> it's hard to tell where the signature is 16:09 -!- StopAndDecrypt [~StopAndDe@unaffiliated/stopanddecrypt] has joined #bitcoin-core-dev 16:10 < wumpus> what *kind* of CMS structure is it? 16:11 < wumpus> trying ContentInfo first as that's appearntly the normal wrapper type 16:11 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev 16:11 < achow101> ah openssl pkcs7 -in sig -inform der -print seems to come out with something usable 16:12 < wumpus> bingo content = ContentInfo.load(blob_data) content['content_type'] '1.2.840.113549.1.7.2': 'signed_data', 16:13 < achow101> The signed hash should be 137856b0cd53ec8e9053f3518b4edf864643138ed548ef61c400068756a2fe48 16:14 < wumpus> macho binaries aren't that intimidating to me anymore :p 16:17 -!- StopAndDecrypt [~StopAndDe@unaffiliated/stopanddecrypt] has left #bitcoin-core-dev [] 16:18 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-core-dev 16:21 -!- Tennis [~Tennis@unaffiliated/tennis] has joined #bitcoin-core-dev 16:23 < achow101> at least we can probably verify the signatures now 16:24 < wumpus> I get three different certificatechoices 16:24 -!- zpao [~zpao@139.28.218.148] has joined #bitcoin-core-dev 16:25 < achow101> what are they? 16:31 < wumpus> sha256_rsa sha1_rsa, and another sha256_rsa 16:31 < wumpus> this is for 0.20.1 fwiw 16:31 < achow101> should be the same for rc3 16:32 < achow101> I see there are 3 embedded certs: Apple, Devloper ID Certification Authority, and Bitcoin Core Code Signing Association 16:32 < achow101> presumably it's the cert chain 16:33 < wumpus> oh that makes sense i guess, have to agree the nesting on this is crazy 16:34 < achow101> there's also a timestamp token thing which seems to be another CMS sig that apple produces 16:35 < achow101> I wonder if codesign is phoning home for every sig 16:35 < wumpus> yes, it is 16:35 < achow101> not surprising 16:36 < sipa> that sounds annoying to replicate 16:36 < sipa> depending on the protocol 16:36 < wumpus> "Set up to call Timestamp server if requested" 16:36 < wumpus> line 820 in signer.cpp 16:37 < wumpus> it's rfc3161 iir 16:37 -!- sr_gi [~sr_gi@80.174.218.168.dyn.user.ono.com] has quit [Read error: Connection reset by peer] 16:37 < achow101> should we disable that? 16:37 -!- sr_gi [~sr_gi@80.174.218.168.dyn.user.ono.com] has joined #bitcoin-core-dev 16:37 < wumpus> the timestamp is used for notarization 16:37 < wumpus> so probably not 16:38 < achow101> ugh 16:38 < achow101> but we don't even notarize 16:40 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Read error: Connection reset by peer] 16:42 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 240 seconds] 16:43 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev 16:51 < wumpus> this looks similar https://blog.umangis.me/a-deep-dive-into-ios-code-signing/ 16:52 < achow101> seems so 16:52 < achow101> from what I can tell, iOS binaries are macho as well 16:53 < achow101> wumpus: are you currently writing a verification tool? 16:53 < wumpus> searching for oid 1.2.840.113635.100.9.1 and 1.2.840.113635.100.9.2 (as appear in the CMS OIDs) gives some matches 16:55 < wumpus> nah just trying to see if asn1crypto.cms in python can make sense of it, and it can, actually verifying anything is far away :): 16:55 < achow101> I'll put together something to extract the hashes so we can at least verify those 16:56 < wumpus> which oid has the hashlist/hashdict? 16:56 -!- bitdex [~bitdex@gateway/tor-sasl/bitdex] has quit [Quit: = ""] 16:56 < wumpus> or is that those 9.1/9.2 16:56 < achow101> 1.2.840.113635.100.9.2 probably 16:57 < wumpus> they're seperate so it makes sense to be those two 16:57 < achow101> 9.1 is the string for a plist file 16:57 < achow101> it does contain the hashes in there 16:57 -!- Pavlenex [~Thunderbi@185.245.85.251] has joined #bitcoin-core-dev 16:58 -!- Pavlenex [~Thunderbi@185.245.85.251] has quit [Client Quit] 16:58 < achow101> 9.2 is a sequence containing a single bye string that matches the code directory hash 16:59 < wumpus> yes that makes sense if it's the same data in some legacy format and the new format 16:59 < achow101> the hash in 9.1 is also trunctaed to 20 bytes 16:59 < wumpus> probably fine to ignore the .1 one 16:59 < achow101> yep 17:00 -!- Tennis [~Tennis@unaffiliated/tennis] has quit [Quit: Leaving] 17:02 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev 17:06 < sipa> by tomorrow morning i'm sure you guys have a tough compatible signer 17:06 -!- lontivero [~lontivero@186.183.48.121] has joined #bitcoin-core-dev 17:06 < sipa> the day after i expect one that doesn't need a private key 17:08 < wumpus> ok, after some more digging: 1.2.840.113635.100.9.1 is appleHashAgility/SEC_OID_APPLE_HASH_AGILITY/kCMSAttrAppleCodesigningHashAgility, .2 is appleHashAgilityV2/SEC_OID_APPLE_HASH_AGILITY_V2/kCMSAttrAppleCodesigningHashAgilityV2, .3 is appleExpirationTime/SEC_OID_APPLE_EXPIRATION_TIME/kCMSAttrAppleExpirationTime 17:08 < wumpus> sipa: haha yesss 17:09 * fanquake is just waiting for the TLDR 17:10 < achow101> i'm sure that once a compatible signer is written, apple will change codesigning 17:10 < sipa> fanquake: apple opensource codesign_allocate code behaves differently from the used binary; we found a workaround 17:11 < sipa> fanquake: now achow101 and wumpus are trying to reverse engineer the signature format 17:11 < achow101> so that we can implement an independent verifier, and maybe an independent signer 17:12 < fanquake> Sounds like everything is under control 17:12 < fanquake> Apple being a pain in the arse as per usual 17:13 < sipa> https://twitter.com/_jonasschnelli_/status/1337693216167120902 17:13 < sipa> https://twitter.com/pwuille/status/1337829501804265472 17:14 -!- miketwenty1 [~miketwent@ec2-34-202-224-110.compute-1.amazonaws.com] has quit [Remote host closed the connection] 17:15 -!- lontivero [~lontivero@186.183.48.121] has quit [Quit: WeeChat 2.8] 17:21 < fanquake> is the dmg signature being invalid only an issue when opening the .dmg on macOS Big Sur? 17:21 < achow101> it's invalid everywhere 17:21 < achow101> when the verifier tries to hash the binary, it's literally the wrong hash 17:21 < fanquake> what's meant to happen? The .dmg opens fine here 17:22 < fanquake> or does it just fail silently / log something 17:22 < sipa> the signature is on the binary, not on the dmg i think? 17:22 < achow101> try opening it 17:22 < achow101> the app itself 17:22 < fanquake> right, not the .dmg 17:22 < achow101> if you open the dmg and do the drag into Applications, it should give you the warning 17:22 < fanquake> I can run the app just fine from the cmd line 17:23 < sipa> fanquake: both when doing the actual signing and when attaching the sig to the binary, some changes to the binary need to be made first; the apple tool does this *slightly* differently than their published source code, so the resulting binary after attaching isn't exactly identical to what the codesigning app produced 17:23 < achow101> fanquake: do "codesign -v Bitcoin-Qt.app" 17:23 < achow101> with rc3 installed 17:25 < fanquake> Yea i see the invalid signature warnings. 17:25 < fanquake> It seems like macOS will only stop you from opening the .app though. You can still run Bitcoin-Qt with an invalid signature 17:25 < fanquake> Although that's not useful for us 17:26 < fanquake> By Bitcoin-Qt I mean: Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt 17:26 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 246 seconds] 17:26 < achow101> fanquake: #20638 for your enjoyment 17:26 < gribble> https://github.com/bitcoin/bitcoin/issues/20638 | Mac codesign fixed alloc by achow101 · Pull Request #20638 · bitcoin/bitcoin · GitHub 17:27 < fanquake> achow101: Nice. I do enjoy a good write-up 17:35 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-dev 17:35 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Disconnected by services] 17:35 -!- vasild_ is now known as vasild 18:19 -!- Asbestos_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev 18:22 -!- Chlorine_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev 18:22 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Ping timeout: 240 seconds] 18:24 -!- Asbestos_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Ping timeout: 258 seconds] 18:37 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 18:37 < bitcoin-git> [bitcoin] fanquake pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/b18978066d87...ade38b6ee8f9 18:37 < bitcoin-git> bitcoin/master faac315 MarcoFalke: Remove unused and confusing CTransaction constructor 18:37 < bitcoin-git> bitcoin/master fac39c1 MarcoFalke: wallet: document that tx in CreateTransaction is purely an out-param 18:37 < bitcoin-git> bitcoin/master ade38b6 fanquake: Merge #20588: Remove unused and confusing CTransaction constructor 18:37 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 18:37 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 18:37 < bitcoin-git> [bitcoin] fanquake merged pull request #20588: Remove unused and confusing CTransaction constructor (master...2012-txConstructor) https://github.com/bitcoin/bitcoin/pull/20588 18:37 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 18:41 -!- Asbestos_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev 18:44 -!- Chlorine_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Ping timeout: 256 seconds] 18:44 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev 18:47 -!- Asbestos_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Ping timeout: 260 seconds] 18:56 -!- EagleTM [~EagleTM@unaffiliated/eagletm] has joined #bitcoin-core-dev 18:56 -!- Eagle[TM] [~EagleTM@unaffiliated/eagletm] has quit [Ping timeout: 260 seconds] 19:20 -!- joelklabo [~textual@108-196-216-127.lightspeed.sntcca.sbcglobal.net] has quit [Read error: No route to host] 20:00 -!- bitdex [~bitdex@gateway/tor-sasl/bitdex] has joined #bitcoin-core-dev 20:03 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 20:03 < bitcoin-git> [bitcoin] theStack opened pull request #20640: wallet, refactor: return out-params of CreateTransaction() as optional struct (master...202012-refactor-wallet-createtransaction-return_out_params_in_optstruct) https://github.com/bitcoin/bitcoin/pull/20640 20:03 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 20:06 -!- kristapsk_ [~KK@gateway/tor-sasl/kristapsk] has quit [Remote host closed the connection] 20:17 -!- proofofkeags [~proofofke@174-16-212-53.hlrn.qwest.net] has quit [Remote host closed the connection] 20:37 -!- tryphe_ is now known as tryphe 21:19 -!- pinheadmz [~pinheadmz@71.190.30.138] has quit [Quit: pinheadmz] 21:36 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has joined #bitcoin-core-dev 22:08 -!- verybaddad [~thatdad@108-188-072-226.biz.spectrum.com] has joined #bitcoin-core-dev 22:08 -!- verybaddad [~thatdad@108-188-072-226.biz.spectrum.com] has left #bitcoin-core-dev [] 22:22 -!- Emcy [~Emcy@unaffiliated/emcy] has quit [Ping timeout: 272 seconds] 22:24 -!- Emcy [~Emcy@unaffiliated/emcy] has joined #bitcoin-core-dev 22:24 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev 22:24 < bitcoin-git> [bitcoin] MarcoFalke closed pull request #20639: doc: fix case of GitHub (master...fix-case-of-github) https://github.com/bitcoin/bitcoin/pull/20639 22:24 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev [] 22:50 < dhruvm> Networking question: Are legal CIDR netmasks always 1s followed by 0s? i.e. mask=255.96.0.0 is an illegal mask? 22:50 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 22:51 -!- da39a3ee5e6b4b0d [~da39a3ee5@mx-ll-171.5.29-209.dynamic.3bb.co.th] has joined #bitcoin-core-dev 22:56 < dhruvm> I am trying to reason whether given two CIDR subnet representations, it is possible to have partial address overlap between them. 22:57 < sipa> no, either they don't overlap at all, or one is a subset of the other 22:57 < sipa> or identical to it 22:58 < dhruvm> I see. So, the mask is always 1s followed by 0s then? 23:00 < dhruvm> They seem to be represented as /32 /24 in most places, but some places I've seen 255.255.0.0 etc which has me confused 23:00 < sipa> yeah, in CIDR the netmask has to be 1s and then 0s 23:00 < sipa> you can have netmask that are not CIDR though 23:01 < sipa> those aren't used in practocr afaik though 23:01 < sipa> *practice 23:03 < dhruvm> thanks sipa 23:10 < sipa> yw 23:16 -!- jeremyrubin [~jr@c-73-15-215-148.hsd1.ca.comcast.net] has quit [Ping timeout: 272 seconds] 23:39 -!- zpao [~zpao@139.28.218.148] has quit [Remote host closed the connection] 23:52 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Read error: Connection reset by peer] 23:52 -!- da39a3ee5e6b4b0d [~da39a3ee5@mx-ll-171.5.29-209.dynamic.3bb.co.th] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 23:54 -!- da39a3ee5e6b4b0d [~da39a3ee5@mx-ll-171.5.29-209.dynamic.3bb.co.th] has joined #bitcoin-core-dev 23:59 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-core-dev --- Log closed Sun Dec 13 00:00:45 2020