--- Log opened Sat Dec 12 00:00:08 2020
--- Day changed Sat Dec 12 2020
00:00 < sipa> (i know we did that before, but perhaps that before we knew that history of sig sizes the binary has been through mattered)
00:00 < achow101> as before vmsize 0x000000000007c000
00:00 < sipa> ok
00:00 < sipa> well, if it's an alignment issue, 0x48000 is 0x4000 aligned, so unlikely to cause problem
00:01 < jonasschnelli> https://bitcoin.jonasschnelli.ch/bitcoin-0.21.0rc3-osx-unsigned-again-pagesize0.zip <--- signed with pagesize 0, check that is was not already codesigned
00:01 < sipa> so i expect this will be a solution
00:01 < jonasschnelli> (gitian output follows)
00:02 < achow101> sipa: it's aligning against 0x2000
00:03 < sipa> achow101: you found that by trying more sizes?
00:03 < achow101> if I use 4096 as the size, I get 46000. same with 8192
00:04 < jonasschnelli> and if anyone wants to test 0.20.2rc1, I just created the detached signature (no gitian build yet): https://bitcoin.jonasschnelli.ch/signature-osx_0.20.2rc1.tar.gz
00:04 < sipa> ok
00:04 < sipa> achow101: we could patch out cctools to do the same
00:04 < sipa> *our
00:04 < sipa> it's a tiny change
00:04 < achow101> I'm not optimistic that that's a lasting solution
00:05 < sipa> yeah, agree
00:06 < sipa> but it's simpler than trying to hack around the issue with multiple preallocate calls and passing the size through
00:07 < achow101> https://github.com/achow101/bitcoin/commit/26f8ebcca26eb74b6546b5d551f3ec468e2878be is my current proposal
00:07 < achow101> if we combine with pagesize 0, then it'll probably always work
00:08 < achow101> the largest signature since 0.12.1 is 407424
00:10 < sipa> achow101: if the gitian preallocate takes it to a large *odd* multiple of 0x1000, will the macos sign tool not increase it further to a multiple of 0x2000 ?
00:11 < sipa> if it's anything like the opensource version, it won't
00:11 < sipa> but who knows what it does
00:12 < jonasschnelli> oh!
00:12 < jonasschnelli> pagesize 0 result in a valid signatue (through gitian)
00:12 < jonasschnelli> I didn't before because I signed an already signed binary
00:13 < jonasschnelli> https://bitcoin.jonasschnelli.ch/bitcoin-osx-signed_pagesize0_clean.dmg
00:13 < jonasschnelli> (used detach signature from https://bitcoin.jonasschnelli.ch/bitcoin-0.21.0rc3-osx-unsigned-again-pagesize0.zip )
00:13 < achow101> I think that is currently the expected result
00:14 < sipa> jonasschnelli: yeah, matches our understanding
00:14 < achow101> sipa: it does not appear to shrink the vmsize at all
00:15 < sipa> achow101: yes, but it may increase it further
00:15 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has quit [Ping timeout: 260 seconds]
00:15 < sipa> (i don't expect it to, but we should check i think)
00:15 < achow101> i'll try against the macports cctools
00:16 -!- tripleslash [~triplesla@unaffiliated/imsaguy] has joined #bitcoin-core-dev
00:19 < achow101> it preserves it
00:20 < sipa> great
00:20 < sipa> 500000 is probably overkill if we're going to use pagesize 0
00:21 < sipa> but that does like a pretty simple fix
00:21 < achow101> probably, but I also have no idea what goes into the signature size
00:22 < achow101> i'll open a pr when i wake up
00:22 < jonasschnelli> thanks achow101!
00:22 < jonasschnelli> and sipa
00:22 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
00:22 < bitcoin-git> [bitcoin] MarcoFalke pushed 6 commits to master: https://github.com/bitcoin/bitcoin/compare/6a4806367177...ffc4d0499020
00:22 < bitcoin-git> bitcoin/master 5aaeb6c Russell Yanofsky: MOVEONLY: Move IsBDBFile, IsSQLiteFile, and ListWalletDir
00:22 < bitcoin-git> bitcoin/master 6ee9cbd Russell Yanofsky: refactor: Replace ListWalletDir() function with ListDatabases()
00:22 < bitcoin-git> bitcoin/master 6a7a636 Russell Yanofsky: refactor: Drop call to GetWalletEnv in wallet salvage code
00:22 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
00:23 < jonasschnelli> I knew that when I'm going to wake up that you guys will have a solution
00:23 < achow101> lol
00:23 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
00:23 < bitcoin-git> [bitcoin] MarcoFalke merged pull request #20275: wallet: List all wallets in non-SQLite and non-BDB builds (master...pr/exist) https://github.com/bitcoin/bitcoin/pull/20275
00:23 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
00:24 < sipa> achow101: number of pages, and "resources" are counted separately
00:24 < sipa> with pagesize=0 the executable code is always 1 page
00:27 < sipa> so i expect it to be mostly constant with pagesize=0
00:34 < jonasschnelli> I'm also getting an invalid signature for 0.20.2rc1
00:34 < jonasschnelli> shall I codesign with --pagesize 0?
00:35 < sipa> jonasschnelli: that may work (50% chance)
00:36 < sipa> jonasschnelli: basically any pagesize you try (has to be zero or a power of 2) has an independent 50% chance of being succesful with the current tooling
00:36 < sipa> with achow's fix it should always work
00:36 < sipa> better just wait; it can be backported too
00:38 < jonasschnelli> ok
01:40 -!- mj_node [~mj_node@122.0.25.130] has quit [Quit: Leaving]
01:49 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 264 seconds]
01:52 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:759a:a9e9:8a19:69d2] has quit [Remote host closed the connection]
02:04 -!- kexkey [~kexkey@static-198-54-132-158.cust.tzulo.com] has quit [Ping timeout: 260 seconds]
02:19 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev
02:37 -!- instagibbs [~greg@061093103011.ctinets.com] has quit [Ping timeout: 246 seconds]
02:42 -!- Pavlenex [~Thunderbi@141.98.103.251] has joined #bitcoin-core-dev
03:07 -!- jonatack [~jon@213.152.161.244] has quit [Ping timeout: 256 seconds]
03:08 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 256 seconds]
03:18 -!- Kenny51Hand [~Kenny51Ha@static.57.1.216.95.clients.your-server.de] has joined #bitcoin-core-dev
03:23 -!- Kenny51Hand [~Kenny51Ha@static.57.1.216.95.clients.your-server.de] has quit [Ping timeout: 272 seconds]
03:24 -!- jonatack [~jon@88.124.242.136] has joined #bitcoin-core-dev
03:29 -!- jonatack [~jon@88.124.242.136] has quit [Ping timeout: 264 seconds]
03:30 -!- jonatack [~jon@213.152.161.40] has joined #bitcoin-core-dev
03:35 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
03:35 < bitcoin-git> [bitcoin] MarcoFalke pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/ffc4d0499020...b18978066d87
03:35 < bitcoin-git> bitcoin/master fad68af MarcoFalke: p2p: Ignore non-version msgs before version msg
03:35 < bitcoin-git> bitcoin/master faaad1b MarcoFalke: p2p: Ignore version msgs after initial version msg
03:35 < bitcoin-git> bitcoin/master b189780 MarcoFalke: Merge #20079: p2p: Treat handshake misbehavior like unknown message
03:35 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
03:36 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
03:36 < bitcoin-git> [bitcoin] MarcoFalke merged pull request #20079: p2p: Treat handshake misbehavior like unknown message (master...2010-p2pNoVersion) https://github.com/bitcoin/bitcoin/pull/20079
03:36 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
03:40 -!- sledges [~sledges@185.163.110.125] has quit [Remote host closed the connection]
03:58 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev
04:11 -!- Eagle[TM] [~EagleTM@unaffiliated/eagletm] has joined #bitcoin-core-dev
04:18 -!- Talkless [~Talkless@mail.dargis.net] has joined #bitcoin-core-dev
04:21 -!- Talkless [~Talkless@mail.dargis.net] has quit [Client Quit]
04:21 -!- peterrizzo_ [~peterrizz@ool-44c18924.dyn.optonline.net] has joined #bitcoin-core-dev
04:35 -!- Talkless [~Talkless@mail.dargis.net] has joined #bitcoin-core-dev
04:35 -!- slidercrank1 [~slidercra@s91904426.blix.com] has joined #bitcoin-core-dev
04:40 -!- rex4539 [~rex4539@balticom-197-78.balticom.lv] has joined #bitcoin-core-dev
05:08 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 256 seconds]
05:23 -!- instagibbs [~greg@061093103011.ctinets.com] has joined #bitcoin-core-dev
05:24 -!- instagibbs [~greg@061093103011.ctinets.com] has quit [Client Quit]
05:27 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-core-dev
05:30 -!- mol_ [~mol@unaffiliated/molly] has quit [Ping timeout: 260 seconds]
05:35 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-dev
05:35 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Disconnected by services]
05:35 -!- vasild_ is now known as vasild
05:49 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:a9f4:e07:a3ac:95af] has joined #bitcoin-core-dev
05:53 -!- slidercrank1 [~slidercra@s91904426.blix.com] has quit [Remote host closed the connection]
05:55 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev
05:56 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has joined #bitcoin-core-dev
05:57 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 272 seconds]
05:58 -!- rh0nj [~rh0nj@88.99.167.175] has quit [Remote host closed the connection]
05:59 -!- rh0nj [~rh0nj@88.99.167.175] has joined #bitcoin-core-dev
06:06 -!- twistedline [~twisted@unaffiliated/twistedline] has quit [Remote host closed the connection]
06:06 -!- twistedline [~twisted@unaffiliated/twistedline] has joined #bitcoin-core-dev
06:08 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has joined #bitcoin-core-dev
06:12 -!- veox [~veox@185.163.110.125] has joined #bitcoin-core-dev
06:18 -!- kristapsk_ [~KK@gateway/tor-sasl/kristapsk] has quit [Remote host closed the connection]
06:19 -!- kristapsk_ [~KK@gateway/tor-sasl/kristapsk] has joined #bitcoin-core-dev
06:20 -!- twistedline [~twisted@unaffiliated/twistedline] has quit [Ping timeout: 264 seconds]
06:27 -!- twistedline [~twisted@unaffiliated/twistedline] has joined #bitcoin-core-dev
06:36 -!- peterrizzo_ [~peterrizz@ool-44c18924.dyn.optonline.net] has quit [Quit: peterrizzo_]
06:38 -!- twistedline [~twisted@unaffiliated/twistedline] has quit [Ping timeout: 246 seconds]
06:44 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
06:45 -!- twistedline [~twisted@unaffiliated/twistedline] has joined #bitcoin-core-dev
06:48 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has joined #bitcoin-core-dev
06:55 -!- mol [~mol@unaffiliated/molly] has quit [Ping timeout: 240 seconds]
07:04 -!- Pavlenex [~Thunderbi@141.98.103.251] has quit [Quit: Pavlenex]
07:09 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 256 seconds]
07:22 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev
07:27 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has quit [Remote host closed the connection]
07:29 -!- darosior [~darosior@194.36.189.246] has quit [Remote host closed the connection]
07:30 -!- darosior [~darosior@194.36.189.246] has joined #bitcoin-core-dev
07:32 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #bitcoin-core-dev
07:36 -!- Guyver2 [Guyver@guyver2.xs4all.nl] has joined #bitcoin-core-dev
07:42 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has joined #bitcoin-core-dev
07:46 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 240 seconds]
07:48 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has quit [Remote host closed the connection]
07:48 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
07:49 -!- miketwenty1 [~miketwent@136.55.84.49] has joined #bitcoin-core-dev
07:52 -!- Pavlenex [~Thunderbi@141.98.103.251] has joined #bitcoin-core-dev
07:53 -!- miketwenty1 [~miketwent@136.55.84.49] has quit [Ping timeout: 258 seconds]
07:56 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #bitcoin-core-dev
07:57 -!- belcher_ [~belcher@unaffiliated/belcher] has joined #bitcoin-core-dev
07:59 -!- tryphe [~tryphe@unaffiliated/tryphe] has quit [Read error: Connection reset by peer]
07:59 -!- tryphe_ [~tryphe@unaffiliated/tryphe] has joined #bitcoin-core-dev
08:00 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 256 seconds]
08:04 -!- jonatack [~jon@213.152.161.40] has quit [Ping timeout: 240 seconds]
08:06 < prusnak> do you know if https://github.com/shesek is on IRC?
08:06 -!- belcher_ is now known as belcher
08:06 -!- jonatack [~jon@109.232.227.138] has joined #bitcoin-core-dev
08:06 < belcher> prusnak yes hes called shesek
08:06 < prusnak> shesek: no such nick
08:07 < prusnak> thanks, i was asking because he's offline most probably
08:25 < achow101> sipa: I found the codesign source. It's in a library called "Security": https://opensource.apple.com/source/Security/Security-59306.140.5/OSX/libsecurity_codesigning/lib/
08:29 < achow101> Would it be worthwhile to write a tool that lets us code sign macOS from linux? Given that we can read the original code signing source, I don't think it would be too difficult to write one.
08:30 -!- wumpus [~ircclient@pdpc/supporter/professional/wumpus] has quit [Read error: Connection reset by peer]
08:32 -!- rex4539_ [~rex4539@8.40.29.10] has joined #bitcoin-core-dev
08:32 -!- wumpus [~ircclient@pdpc/supporter/professional/wumpus] has joined #bitcoin-core-dev
08:33 -!- rex4539 [~rex4539@balticom-197-78.balticom.lv] has quit [Read error: Connection reset by peer]
08:36 -!- rex4539_ [~rex4539@8.40.29.10] has quit [Ping timeout: 260 seconds]
08:40 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev
08:42 -!- jonatack [~jon@109.232.227.138] has quit [Quit: jonatack]
08:49 -!- dhruv [~dhruv@165.227.49.220] has joined #bitcoin-core-dev
08:50 -!- dhruv [~dhruv@165.227.49.220] has quit [Client Quit]
08:51 -!- dhruvm [~dhruv@165.227.49.220] has joined #bitcoin-core-dev
08:58 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
08:58 < bitcoin-git> [bitcoin] achow101 opened pull request #20638: Mac codesign fixed alloc (master...mac-codesign-fixed-alloc) https://github.com/bitcoin/bitcoin/pull/20638
08:58 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
09:01 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-core-dev
09:05 -!- jonatack [~jon@88.124.242.136] has joined #bitcoin-core-dev
09:08 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 256 seconds]
09:10 -!- jonatack [~jon@88.124.242.136] has quit [Ping timeout: 240 seconds]
09:10 -!- jonatack [~jon@213.152.162.84] has joined #bitcoin-core-dev
09:17 -!- dermoth_ [~dermoth@unaffiliated/dermoth] has joined #bitcoin-core-dev
09:18 -!- dermoth [~dermoth@unaffiliated/dermoth] has quit [Disconnected by services]
09:18 -!- dermoth_ is now known as dermoth
09:25 -!- davterra [~davterra@107.182.237.18] has joined #bitcoin-core-dev
09:30 -!- Pavlenex [~Thunderbi@141.98.103.251] has quit [Quit: Pavlenex]
09:38 < sipa> achow101: maybe, depending on how hard it is to compile that for linux
09:40 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 256 seconds]
09:40 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 256 seconds]
09:45 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev
09:58 < wumpus> achow101: nice!
10:01 < achow101> sipa: I don't think we can compile it ourselves. It would have to be a completely separate implementation.
10:02 < sipa> achow101: that sounds nontrivial
10:02 < wumpus> assuming, of course, it's the same as the binary they ship
10:03 < sipa> the benefit is that if someone actually makes that work, we may be able to replace it with multiparty RSA or s
10:03 < achow101> it appears to be nontrivial. the code is a bit hard to understand
10:03 < sipa> o
10:04 < achow101> I just wanted to make it use rfc6979
10:04 < wumpus> if it's RSA in the first place
10:04 < achow101> the cert is rsa
10:05 < achow101> it's amazing how hard it is to actually find the function that does the cryptographic signature
10:07 < sipa> i assume the signature contais a certnl chain too
10:07 < wumpus> you'd say if you're going to do something like per-page signing you'd use an algorithm with short signatures :)
10:07 < sipa> which may complicate thihgs
10:07 < sipa> and the code for deciding exactly is being signed may be nontrivial too
10:07 < wumpus> that explains why no one else made a portable signer i guess
10:08 < achow101> I'm surprised no one has put in the effort to make one given that it's actually open source
10:08 < sipa> i wonder how many people even care about doing codesigning macos binaries on a non-macos platform
10:09 < wumpus> true, not that many projects are doing deterministic builds
10:10 < wumpus> though even for non-deterministic builds it can often be useful to do all builds on a single build host platform
10:11 -!- mol_ [~mol@unaffiliated/molly] has joined #bitcoin-core-dev
10:12 < achow101> even with per-page signing, shouldn't the signature be way larger than 200k?
10:12 < achow101> given our binary size
10:13 < sipa> what is the actual binary size without resources?
10:14 < sipa> i think jonasschnelli's tgz has a file that lists all the resources
10:14 -!- mol [~mol@unaffiliated/molly] has quit [Ping timeout: 265 seconds]
10:15 < achow101> 26891600 Dec 12 11:40 Bitcoin-Qt
10:16 < sipa> do you know how many bits the RSA key is?
10:17 < achow101> hmm. the cert is not in contrib/macdeploy.. I thought it was
10:17 < sipa> achow101: perhaps you can find in that apple source code what the default page size is
10:19 < sipa> perhaps the default page size is 32k or 64k
10:19 < achow101> turns out codesign has a lot of --verbose options
10:19 < achow101> the page size is 4096 in the rc3 sig
10:19 < sipa> hmm
10:20 < sipa> that makes little sense
10:21 < achow101> here is the full verbose output of codesign -d: https://0bin.net/paste/2gi9BEF9#YOEGkrrhbpToojZbGKJhHvcYp5H1zV3MfO+skMBRotg
10:21 < sipa> well that"s 35 bytes per sig
10:21 < achow101> it looks like it's actually just a ton of hashes and the signature is over all of them?
10:21 < achow101> it says Signature size=8973
10:21 < sipa> ah!
10:21 < sipa> it's a merkle tree over the binary?
10:21 < sipa> that makes sens
10:22 < achow101> couldn't say. but 6463 * 32 is in the ballpark of the signature size
10:23 < achow101> and there's 6432 hashes. that also fits with the page size and binary size, 6432 * 4096 is near the binary size
10:23 < achow101> *6463 hashes
10:24 < sipa> yup
10:24 < sipa> i suspect that's it exactly
10:24 < achow101> making hashes of each page seems excessive though
10:25 < sipa> it allows for fast per-page validation
10:25 < sipa> without needing to hash all the leaves every time
10:28 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
10:28 < bitcoin-git> [bitcoin] jbampton opened pull request #20639: doc: fix case of GitHub (master...fix-case-of-github) https://github.com/bitcoin/bitcoin/pull/20639
10:28 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
10:32 < achow101> I don't think it's a merkle tree. I'm pretty sure it's just a flat list of hashes
10:33 < achow101> that's what gets hashes again and signed
10:35 < wumpus> that makes sense
10:36 < wumpus> a hash would be much more efficient to verify per page, after all, and the signature check can be done once at load time for the hashes section
10:37 < wumpus> per page (or bunch-of-pages) makes sense as it decreases the granularity of loading, a hash over the entire binary means the entire binary has to be read
10:38 < achow101> so our trick to do a single page may not be ideal
10:38 < wumpus> well if it works as a workaround
10:38 < wumpus> but not in the long term no i guess
10:39 < achow101> well we now know mostly how big the signature will be, so we could compute an overestimate instead
10:40 < sipa> achow101: why not? do we care about not loading all pages all at once?
10:40 < wumpus> if someone wants to reverse engineer this i'd prefer having a signature *verifier* first, so the gitian process can detect if it went wrong
10:40 < wumpus> sipa: i guess so? there's always a lot of data in a binary that's not 'hot', might never be accessed at all, especially as we static link qt
10:41 < achow101> i dunno
10:41 < achow101> it seems like there's a default for a reason
10:41 < achow101> and it's clearly architecture optimized
10:41 < wumpus> that said, given the memory requiremetns i'm not sure we care about paging much
10:42 < achow101> verification of the signature itself should only happen once
10:42 < sipa> oh you think it's re-verifying the pages every time they get loaded again from disk
10:42 < wumpus> yes, but the hash might be done more often, say if a page is paged out and in again?
10:42 < achow101> I would guess this is some kind of runtime integrity check
10:42 < sipa> that does make sense
10:42 < sipa> but just the hash, not the full sig
10:43 < achow101> yeah
10:43 < wumpus> yes, the sig verification for the hashes section is likely done only once
10:43 < sipa> but that means that the list of hashes must be loaded at all times
10:43 < wumpus> yes
10:44 < wumpus> so it'd be a compromise--too small page size and the list of hashes is huge, too big page size and it has very large paging granularity
10:46 < wumpus> and i'm not sure--if it wouldn't check every time a page is re-loaded from disk it might open up some kinds of exploits, espeically for network file systems if manipulation of the data can happen without the OS knowing
10:47 < wumpus> but that's not certain
10:48 < sipa> if it was a Merkle tree instead they wouldn't need to keep the hash lish loaded, but it'd be twice the size
10:52 < achow101> I think we can produce a good enough overestimate to allow for the default page size
10:53 < achow101> (((unsigned_size / 4096) + 1) * 32) + 50000
10:53 < sipa> yup
10:53 < wumpus> good!
10:53 < achow101> the sig is 9k, and there's an additional 10k of overhead. with + 50k, we get 30k of extra
10:54 < achow101> i'll update the pr
10:55 < phantomcircuit> it seems to be using https://en.wikipedia.org/wiki/Cryptographic_Message_Syntax
10:56 < achow101> oh!
10:56 < achow101> so openssl can be used?
10:56 -!- adiabat [~adiabat@63.209.32.102] has quit [Ping timeout: 272 seconds]
10:57 -!- adiabat [~adiabat@63.209.32.102] has joined #bitcoin-core-dev
10:59 < phantomcircuit> achow101, well to quote a comment 'Sign a Mach-O binary, using liberal dollops of that special Mach-O magic sauce.'
10:59 < phantomcircuit> so
10:59 < achow101> when I read that, I was not encouraged
10:59 < sipa> how to construct the Xcode extract for the 0.20 and 0.19 builds?
11:00 < achow101> sipa: see contrib/macdeploy/README in the 0.20 branch
11:00 < achow101> it explains how for both
11:00 < wumpus> if the download is still available on the apple site, hopefully
11:01 < sipa> https://github.com/bitcoin-core/docs/blob/master/gitian-building/gitian-building-mac-os-sdk.md seems to refer to very old stuff
11:01 < sipa> i guess it should be updated or deleted?
11:01 < sipa> or changed into a link to the macdeploy stuff?
11:01 < achow101> I think it should link to macdeploy
11:02 < wumpus> yea the information on a branch should probably only describe for that branch
11:02 < achow101> for some reason the 0.19 branch doesn't describe the extract process, but 0.20 does as "the old way"
11:03 < phantomcircuit> achow101, yeha i cant figure out where it's actually signing either
11:03 < phantomcircuit> im sure if i could download that entire directory from git or as a tar ball i could
11:04 < phantomcircuit> but no
11:04 -!- Zenton [~user@unaffiliated/vicenteh] has quit [Read error: Connection reset by peer]
11:04 < phantomcircuit> classic apple
11:04 < achow101> phantomcircuit: mirror of it here https://github.com/apple-opensource/Security/
11:05 < achow101> slightly older version, but close enough
11:06 < achow101> I got to SecCodeSigner::Signer::signCodeDirectory in signer.cpp before I got super lost
11:06 < phantomcircuit> achow101, yeah that's definitely the right function
11:07 < phantomcircuit> i got to CMSEncoderAddSigners but can't tell what it's actually doing
11:07 < phantomcircuit> at least i think that's a decent place to look
11:08 < phantomcircuit> also, if(hashDict != NULL) {assert(hashDict != NULL); they must not trust their compiler very much
11:08 -!- Zenton [~user@unaffiliated/vicenteh] has joined #bitcoin-core-dev
11:08 < achow101> CMSEncoderCopyEncodedContent claims to do the actual signing, but I find no evidence that supports that. it just kinda goes into a while function stack that I can't understand
11:08 < achow101> *wild
11:09 < achow101> CMSEncoderAddSigners just adds some kind of signer object to a list
11:10 -!- Zenton [~user@unaffiliated/vicenteh] has quit [Read error: Connection reset by peer]
11:11 < phantomcircuit> achow101, CMSEncoderUpdateContent
11:13 < phantomcircuit> wait this looks like it's generating hashes and then encrypting them???
11:13 < achow101> I think they use encrypt and sign interchangeably
11:14 -!- joelklabo [~textual@108-196-216-127.lightspeed.sntcca.sbcglobal.net] has quit [Quit: Textual IRC Client: www.textualapp.com]
11:14 < phantomcircuit> achow101, https://github.com/apple-opensource/Security/blob/3aaa9af09496be26b5c15ff011e3d65ca75821ac/libsecurity_smime/lib/cmsencode.c#L389
11:14 < phantomcircuit> iono kinda looks like it's actually encrypting things
11:15 < phantomcircuit> of course there's about 10 levels of indirection though so who can even tell
11:15 < achow101> I think it's just some more generic code that can also handle encryption?
11:15 < achow101> honstly it's hard to know
11:16 < sipa> for RSA encrypting and signing are kind-of interchangeable (but the devil is in the details)
11:18 -!- joelklabo [~textual@108-196-216-127.lightspeed.sntcca.sbcglobal.net] has joined #bitcoin-core-dev
11:20 < phantomcircuit> sipa, just gotta figure out where the ciphcx object is set.. which is uhhh
11:21 -!- Zenton [~user@unaffiliated/vicenteh] has joined #bitcoin-core-dev
11:29 < sipa> ok, downloading xocde 10.2.1
11:29 < sipa> *xcode
11:57 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:a9f4:e07:a3ac:95af] has quit [Remote host closed the connection]
12:04 < wumpus> can you see what hash algorithm they use? if so it might be most straightforward to check if the hashes are encrypted or otherwise obfuscating by hashing pages and looking if they appear in the signature section of the file
12:06 < wumpus> pages are bound to have an address aligned to the start of the file, otherwise it'd be wildly inefficient
12:08 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:a9f4:e07:a3ac:95af] has joined #bitcoin-core-dev
12:13 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:a9f4:e07:a3ac:95af] has quit [Ping timeout: 260 seconds]
12:15 < achow101> they use sha256, so it shouldn't be hard to check
12:21 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev
12:27 -!- sr_gi [~sr_gi@80.174.218.168.dyn.user.ono.com] has quit [Read error: Connection reset by peer]
12:27 -!- sr_gi [~sr_gi@80.174.218.168.dyn.user.ono.com] has joined #bitcoin-core-dev
--- Log closed Sat Dec 12 12:37:25 2020
--- Log opened Sat Dec 12 12:37:25 2020
12:37 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 240 seconds]
12:42 < achow101> The hashes are definitely embedded
12:43 < achow101> From the 2nd page to the end of the file, the hashes in the signed binary match the unsigned one
12:43 < achow101> the 2nd page contains the alloc thing, so it's different, but if the alloc is done, it matches
12:43 < achow101> only the first page doesn't match. and then there are 3 "special" hashes that are of... something else
12:45 < wumpus> achow101: oh that's great!
12:46 < achow101> actually the first page matches, it just doesn't match the unsigned one
12:47 < wumpus> maybe it updates something in the header first
12:50 < sipa> well, we know it does :)
12:50 < sipa> if it didn't, this whole problem wouldn't exist
12:50 < sipa> the vmsize/filesize entries on the __LINKEDIT section are changed
12:50 < wumpus> yeah then it makes sense for the hash to match the new state, not the original one
12:51 < wumpus> right, true :)
12:58 < achow101> yep, first 2 pages differ from unsigned because of the addition of the code signing section
12:59 < achow101> so the hashes in the sig are 4096 byte pages from beginning of the signed binary to the beginning of the code signature
12:59 < achow101> this is something we can verify ourselves
13:07 -!- jonatack [~jon@213.152.162.84] has quit [Ping timeout: 256 seconds]
13:08 < sipa> ah, neat
13:09 < achow101> i figured out 2 of the 3 special hashes
13:10 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:25fd:4b36:9c2b:340] has joined #bitcoin-core-dev
13:10 -!- eugene-ff [~eugene_ff@2604:2000:1383:472b:25fd:4b36:9c2b:340] has quit [Client Quit]
13:17 < wumpus> that's fast :)
13:18 < sipa> last one may be some metadata with the signature/hashing parameters?
13:18 < achow101> looking at the source, I think it's "entitlements"
13:18 < achow101> but I don't know what that means
13:19 < sipa> the codesign tool has a way to provide something called entitlements to it
13:21 < achow101> one of the hashes is of the file Bitcoin-Qt.app/Contents/_CodeSignature/CodeResources, and the other is Bitcoin-Qt.app/Contents/Info.plist
13:22 < achow101> but this last hash is not any of the files in the bundle
13:23 < wumpus> if we don't pass "entitlements", maybe it's a hash of empty data?
13:23 < achow101> I don't think so
13:24 < achow101> The hash is Bitcoin-Qt.app/Contents
13:24 < achow101> 1761bc7a0f37e9809acd22c53f42ff21622ea02093c14058e7daa3a466c95d97
13:29 < sipa> is this hash the same for other releases?
13:29 < sipa> if it's just some constant we don't actually care what it's a hash of
13:34 -!- astrolince [astrolince@gateway/shell/matrix.org/x-pefoseecitekmcnv] has joined #bitcoin-core-dev
13:34 < wumpus> right a proto-verifyer could just compare it against the fixed hash and raise alarm if it doesn't match
13:34 < achow101> it's the same in 0.20.1, 0.21.0rc2, and 0.21.0rc3
13:35 < achow101> afaik we don't have any entitlements so it should just be the hash of an empty something
13:35 < sipa> google is of no help
13:39 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev
13:40 < wumpus> "
13:40 < wumpus> The specific entitlements will be embedded in the signature of an application. If you are having trouble, it can help to look at what the signature actually says about the entitlements: $ codesign -d --entitlements - Example.app will show an XML property list similar to the one above.
13:41 < achow101> it gives nothing so there are no entitlements
13:41 < wumpus> so it's a hash of something that is in the signature itself, that sounds strange but...
13:41 < sipa> maybe it's just or something
13:41 < wumpus> I'd expect so
13:42 < achow101> the source code has it as a datatype of CFDataRef which I think just means it's a void *
13:42 < sipa> wumpus: sounds similar to bitcoin's use of a sighash byte
13:42 < sipa> part of the signature itself, and also signed
13:43 < wumpus> sipa: ah yes
13:53 < achow101> is there such a thing as a generic DER decoder?
13:53 < sipa> yes
13:55 < sipa> https://github.com/pornin/DDer
13:55 -!- owowo [~ovovo@unaffiliated/ovovo] has quit [Ping timeout: 256 seconds]
13:55 < sipa> ha, thomas pornin
13:55 < sipa> i've met him
13:57 < achow101> hmm, openssl should be able to decode this if it was properly CMS DER, but it doesn't like it
13:58 < wumpus> it apparently DiskRep::component(cdEntitlementSlot) gets the data that is hashed to verify the entitlement slot, there are different implementations of DispRep though, but in diskrep this is the file "CodeEntitlements"
13:58 < wumpus> filediskrep*
13:59 -!- abetusk [~abe@68.175.128.91] has joined #bitcoin-core-dev
14:00 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev
14:00 < wumpus> in CodeDirectory::slotIsPresent a non-zero digest in the slot means that is is present
14:02 < wumpus> i guess i should be looking at BundleDiskRep::component(cdEntitlementSlot) as it can fetch components from the executable itself
14:02 < achow101> I found a bunch of ASN.1 templates, this means it should be possible to figure the serialization, right?
14:03 < sipa> achow101: can you show me the DER data?
14:03 < achow101> sipa: in the bitcoin-detached-sigs repo osx/dist/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt.sign
14:03 < achow101> that's the code signature. in theory it's DER
14:03 < achow101> (checkout 0.21 branch)
14:03 < sipa> you don't actually need the schema to decode DER, it's self-descriptive
14:04 < sipa> the scheme just helps with naming stuff
14:04 < achow101> I'm not convinced it's actually DER
14:04 < wumpus> *i have no clue what i'm doing dog image*
14:05 < wumpus> this looks interesting: https://chromium.googlesource.com/experimental/chromium/src/+/refs/wip/bajones/webvr/chrome/browser/safe_browsing/signature_evaluator_mac.mm
14:06 < sipa> it's not
14:06 < wumpus> ok
14:07 < sipa> $ file Bitcoin-Qt.sign
14:07 < sipa> Bitcoin-Qt.sign: Mac OS X Detached Code Signature (non-executable) - 216249 bytes
14:07 < achow101> wait file recognizes this???
14:09 < sipa> hmm where does file keeps its magic definitions these days?
14:09 < sipa> i only find a precompiled magic.mgc file
14:09 < achow101> I think that's it
14:09 < achow101> gotta find the source for file
14:09 < wumpus> from MachORep::compoenent to EmbeddedSignatureBlob::component now
14:10 < sipa> https://github.com/file/file/blob/master/magic/Magdir/apple
14:11 < wumpus> "An EmbeddedSignatureBlob is a SuperBlob indexed by component slot number." of coursee
14:11 < wumpus> class EmbeddedSignatureBlob : public SuperBlobCore
14:13 < sipa> _0x0ff: belong0xfade0cc1Mac OS X Detached Code Signature
14:13 < sipa> 0 belong 0xfade0cc1 Mac OS X Detached Code Signature
14:14 < wumpus> typedef SuperBlob<0xfade0cc1> DetachedSignatureBlob; // indexed by main architecture
14:14 < wumpus> superblobs, all of them
14:15 < sipa> turtles all the way down
14:15 < wumpus> hehe
14:15 < achow101> blobloblob
14:17 < wumpus> / as a genuine Blob (e.g. for insertion into a SuperBlob).
14:18 < wumpus> a superblob is a map of integers to blobs
14:18 < sipa> getting deeper
14:19 -!- yunier2002 [~yunier200@c-73-85-126-91.hsd1.fl.comcast.net] has joined #bitcoin-core-dev
14:19 < wumpus> "// echoes from parent BlobCore (the C++ type system is too restrictive here)" uh
14:21 < wumpus> tbh seems like it's actually all fairly straightforward C++ code
14:22 < wumpus> the comments are way scarier than the code itself
14:24 < sipa> wumpus: you're biased
14:24 < sipa> :)
14:25 < wumpus> i... suppose hehe
14:26 < achow101> i've somehow wandered into some code in this libsecurity that's Copyright Mozilla
14:44 -!- Guyver2 [Guyver@guyver2.xs4all.nl] has quit [Quit: Going offline, see ya! (www.adiirc.com)]
14:54 < sipa> so do we know how to extract the blobs from the EmbeddedSignatureBlob ?
15:00 < achow101> nope?
15:02 -!- Talkless [~Talkless@mail.dargis.net] has quit [Quit: Konversation terminated!]
15:04 < achow101> That unknown hash earlier is apparently "internal requirements", not entitlements
15:11 < wumpus> kind of, it's u32 magic (BE), u32 (BE) size, then u32 (LE) number of blobs, then per blob u32 (LE) blob type, u32 LE offset
15:11 < wumpus> then the data
15:12 < wumpus> blob type is the slot id, so 5 for the cdEntitlementSlot, but apprantly we're no longer looking for that
15:13 < sipa> ok so 3 blobs
15:14 < sipa> first blob is type 0, length 0x24
15:14 < wumpus> which is good as it doesn't have that one
15:14 < sipa> ah no, not length; offset
15:15 < sipa> the individual blobs don't have a length?
15:15 < wumpus> sipa: that seems correct?
15:15 < wumpus> yes, offset
15:15 < wumpus> the length is in the blob itself
15:15 < sipa> the first blob has magic fa de 0c 02
15:15 < wumpus> blobs are all
15:15 < sipa> got it
15:16 < achow101> list of magics in OSX/libsecurity_codesigning/lib/CSCommonPriv.h
15:16 < sipa> Mac OS X Code Directory is the first one
15:16 < wumpus> so yes we have 0 at offset 0x24, 2 at offset 0x2f7a4 and 0x100 at offset 0x2f864
15:17 < achow101> sipa: yes. this contains our list of hashes
15:17 < wumpus> (all integers are BE, it seemed from the code that they'd be LE but apparently they're swapped somewhere)
15:18 < achow101> it looks like the structure is the class members
15:18 < achow101> in order as listed
15:18 < sipa> second one has magic fa de 0c 01, Mac OS X Code Requirement Set
15:18 < wumpus> there's your requirement set to hash i guess
15:18 < sipa> last blob has magic fa de 0b 01
15:19 < wumpus> class BlobWrapper : public Blob
15:20 < sipa> the first blob has magic fa de 0c 02
15:20 < sipa> https://gist.github.com/nmoinvaz/cd8651ffb3659161423534b29824510a
15:20 < achow101> oh nice
15:21 < achow101> hmm, it can't be the structure as listed because I run into some strings before I'm supposed to
15:21 < achow101> but it's pretty close
15:21 < wumpus> euhh the third's slot id is 0x10000 which is cdSignatureSlot = 0x10000,| | | // CMS signature
15:22 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Remote host closed the connection]
15:22 < achow101> that might be DER encoded
15:22 < wumpus> which is wrapped in a BlobWrapper which is simply a blob header with arbitrary binary dat
15:22 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev
15:22 -!- miketwenty1 [~miketwent@ec2-34-202-224-110.compute-1.amazonaws.com] has joined #bitcoin-core-dev
15:22 < sipa> this has so many layers, i'd be surprised if it isn't somehow exploitable...
15:24 < achow101> In class CodeDirectory, everything in the class decodes in order up to spare3
15:25 < achow101> then we get to the offset for identifier string and the identifer string begins there, as expected
15:25 < wumpus> otoh the blob format is super simple, there could be a mistake of course, but yes e.g. DER parsers are notoriously exploitable
15:26 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Remote host closed the connection]
15:27 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev
15:30 < wumpus> the CMS signature is produced in SecCodeSigner::Signer::signCodeDirectory in signer.cpp
15:30 -!- veox [~veox@185.163.110.125] has quit [Remote host closed the connection]
15:30 < achow101> it seems like the message that is signed is the CodeDirectory blob
15:31 < wumpus> that's what the comment says too "Generate the CMS signature for a (finished) CodeDirectory.", in addition it takes a hashDict and hashList
15:32 -!- Barno7 [~bob@93.55.242.131] has quit [Remote host closed the connection]
15:38 < wumpus> where hashDict is apparently "v2" and hashList for "v1"
15:40 < wumpus> the encoding is indeed the Cryptographic Message Syntax what phantomcircuit linked to with apple specific tags
15:49 < phantomcircuit> this has so many layers, i'd be surprised if it isn't somehow exploitable...
15:49 < phantomcircuit> indeed
15:53 < achow101> wumpus: that Requirement Set blob is indeed the unknown hash.
15:55 -!- miketwenty1 [~miketwent@ec2-34-202-224-110.compute-1.amazonaws.com] has quit [Remote host closed the connection]
15:56 -!- jeremyrubin [~jr@c-73-15-215-148.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
15:56 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 240 seconds]
15:58 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has joined #bitcoin-core-dev
16:00 -!- miketwenty1 [~miketwent@ec2-18-205-136-236.compute-1.amazonaws.com] has quit [Remote host closed the connection]
16:00 -!- mol_ [~mol@unaffiliated/molly] has quit [Read error: Connection reset by peer]
16:01 < wumpus> achow101: great!
16:02 < wumpus> let's see if python asncrypto.cms can handle this
16:03 < achow101> I got something!
16:04 < achow101> the tool sipa mentioned earlier seems to work if I pass the CMS data through openssl cms first
16:06 -!- miketwenty1 [~miketwent@ec2-34-202-224-110.compute-1.amazonaws.com] has joined #bitcoin-core-dev
16:08 < wumpus> nice
16:09 < achow101> it's hard to tell where the signature is
16:09 -!- StopAndDecrypt [~StopAndDe@unaffiliated/stopanddecrypt] has joined #bitcoin-core-dev
16:10 < wumpus> what *kind* of CMS structure is it?
16:11 < wumpus> trying ContentInfo first as that's appearntly the normal wrapper type
16:11 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev
16:11 < achow101> ah openssl pkcs7 -in sig -inform der -print seems to come out with something usable
16:12 < wumpus> bingo content = ContentInfo.load(blob_data) content['content_type'] '1.2.840.113549.1.7.2': 'signed_data',
16:13 < achow101> The signed hash should be 137856b0cd53ec8e9053f3518b4edf864643138ed548ef61c400068756a2fe48
16:14 < wumpus> macho binaries aren't that intimidating to me anymore :p
16:17 -!- StopAndDecrypt [~StopAndDe@unaffiliated/stopanddecrypt] has left #bitcoin-core-dev []
16:18 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-core-dev
16:21 -!- Tennis [~Tennis@unaffiliated/tennis] has joined #bitcoin-core-dev
16:23 < achow101> at least we can probably verify the signatures now
16:24 < wumpus> I get three different certificatechoices
16:24 -!- zpao [~zpao@139.28.218.148] has joined #bitcoin-core-dev
16:25 < achow101> what are they?
16:31 < wumpus> sha256_rsa sha1_rsa, and another sha256_rsa
16:31 < wumpus> this is for 0.20.1 fwiw
16:31 < achow101> should be the same for rc3
16:32 < achow101> I see there are 3 embedded certs: Apple, Devloper ID Certification Authority, and Bitcoin Core Code Signing Association
16:32 < achow101> presumably it's the cert chain
16:33 < wumpus> oh that makes sense i guess, have to agree the nesting on this is crazy
16:34 < achow101> there's also a timestamp token thing which seems to be another CMS sig that apple produces
16:35 < achow101> I wonder if codesign is phoning home for every sig
16:35 < wumpus> yes, it is
16:35 < achow101> not surprising
16:36 < sipa> that sounds annoying to replicate
16:36 < sipa> depending on the protocol
16:36 < wumpus> "Set up to call Timestamp server if requested"
16:36 < wumpus> line 820 in signer.cpp
16:37 < wumpus> it's rfc3161 iir
16:37 -!- sr_gi [~sr_gi@80.174.218.168.dyn.user.ono.com] has quit [Read error: Connection reset by peer]
16:37 < achow101> should we disable that?
16:37 -!- sr_gi [~sr_gi@80.174.218.168.dyn.user.ono.com] has joined #bitcoin-core-dev
16:37 < wumpus> the timestamp is used for notarization
16:37 < wumpus> so probably not
16:38 < achow101> ugh
16:38 < achow101> but we don't even notarize
16:40 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Read error: Connection reset by peer]
16:42 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 240 seconds]
16:43 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev
16:51 < wumpus> this looks similar https://blog.umangis.me/a-deep-dive-into-ios-code-signing/
16:52 < achow101> seems so
16:52 < achow101> from what I can tell, iOS binaries are macho as well
16:53 < achow101> wumpus: are you currently writing a verification tool?
16:53 < wumpus> searching for oid 1.2.840.113635.100.9.1 and 1.2.840.113635.100.9.2 (as appear in the CMS OIDs) gives some matches
16:55 < wumpus> nah just trying to see if asn1crypto.cms in python can make sense of it, and it can, actually verifying anything is far away :):
16:55 < achow101> I'll put together something to extract the hashes so we can at least verify those
16:56 < wumpus> which oid has the hashlist/hashdict?
16:56 -!- bitdex [~bitdex@gateway/tor-sasl/bitdex] has quit [Quit: = ""]
16:56 < wumpus> or is that those 9.1/9.2
16:56 < achow101> 1.2.840.113635.100.9.2 probably
16:57 < wumpus> they're seperate so it makes sense to be those two
16:57 < achow101> 9.1 is the string for a plist file
16:57 < achow101> it does contain the hashes in there
16:57 -!- Pavlenex [~Thunderbi@185.245.85.251] has joined #bitcoin-core-dev
16:58 -!- Pavlenex [~Thunderbi@185.245.85.251] has quit [Client Quit]
16:58 < achow101> 9.2 is a sequence containing a single bye string that matches the code directory hash
16:59 < wumpus> yes that makes sense if it's the same data in some legacy format and the new format
16:59 < achow101> the hash in 9.1 is also trunctaed to 20 bytes
16:59 < wumpus> probably fine to ignore the .1 one
16:59 < achow101> yep
17:00 -!- Tennis [~Tennis@unaffiliated/tennis] has quit [Quit: Leaving]
17:02 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has joined #bitcoin-core-dev
17:06 < sipa> by tomorrow morning i'm sure you guys have a tough compatible signer
17:06 -!- lontivero [~lontivero@186.183.48.121] has joined #bitcoin-core-dev
17:06 < sipa> the day after i expect one that doesn't need a private key
17:08 < wumpus> ok, after some more digging: 1.2.840.113635.100.9.1 is appleHashAgility/SEC_OID_APPLE_HASH_AGILITY/kCMSAttrAppleCodesigningHashAgility, .2 is appleHashAgilityV2/SEC_OID_APPLE_HASH_AGILITY_V2/kCMSAttrAppleCodesigningHashAgilityV2, .3 is appleExpirationTime/SEC_OID_APPLE_EXPIRATION_TIME/kCMSAttrAppleExpirationTime
17:08 < wumpus> sipa: haha yesss
17:09 * fanquake is just waiting for the TLDR
17:10 < achow101> i'm sure that once a compatible signer is written, apple will change codesigning
17:10 < sipa> fanquake: apple opensource codesign_allocate code behaves differently from the used binary; we found a workaround
17:11 < sipa> fanquake: now achow101 and wumpus are trying to reverse engineer the signature format
17:11 < achow101> so that we can implement an independent verifier, and maybe an independent signer
17:12 < fanquake> Sounds like everything is under control
17:12 < fanquake> Apple being a pain in the arse as per usual
17:13 < sipa> https://twitter.com/_jonasschnelli_/status/1337693216167120902
17:13 < sipa> https://twitter.com/pwuille/status/1337829501804265472
17:14 -!- miketwenty1 [~miketwent@ec2-34-202-224-110.compute-1.amazonaws.com] has quit [Remote host closed the connection]
17:15 -!- lontivero [~lontivero@186.183.48.121] has quit [Quit: WeeChat 2.8]
17:21 < fanquake> is the dmg signature being invalid only an issue when opening the .dmg on macOS Big Sur?
17:21 < achow101> it's invalid everywhere
17:21 < achow101> when the verifier tries to hash the binary, it's literally the wrong hash
17:21 < fanquake> what's meant to happen? The .dmg opens fine here
17:22 < fanquake> or does it just fail silently / log something
17:22 < sipa> the signature is on the binary, not on the dmg i think?
17:22 < achow101> try opening it
17:22 < achow101> the app itself
17:22 < fanquake> right, not the .dmg
17:22 < achow101> if you open the dmg and do the drag into Applications, it should give you the warning
17:22 < fanquake> I can run the app just fine from the cmd line
17:23 < sipa> fanquake: both when doing the actual signing and when attaching the sig to the binary, some changes to the binary need to be made first; the apple tool does this *slightly* differently than their published source code, so the resulting binary after attaching isn't exactly identical to what the codesigning app produced
17:23 < achow101> fanquake: do "codesign -v Bitcoin-Qt.app"
17:23 < achow101> with rc3 installed
17:25 < fanquake> Yea i see the invalid signature warnings.
17:25 < fanquake> It seems like macOS will only stop you from opening the .app though. You can still run Bitcoin-Qt with an invalid signature
17:25 < fanquake> Although that's not useful for us
17:26 < fanquake> By Bitcoin-Qt I mean: Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt
17:26 -!- fearbeag [~sseanicid@bras-base-clwdon2201w-grc-18-216-209-44-58.dsl.bell.ca] has quit [Ping timeout: 246 seconds]
17:26 < achow101> fanquake: #20638 for your enjoyment
17:26 < gribble> https://github.com/bitcoin/bitcoin/issues/20638 | Mac codesign fixed alloc by achow101 · Pull Request #20638 · bitcoin/bitcoin · GitHub
17:27 < fanquake> achow101: Nice. I do enjoy a good write-up
17:35 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-dev
17:35 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Disconnected by services]
17:35 -!- vasild_ is now known as vasild
18:19 -!- Asbestos_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev
18:22 -!- Chlorine_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev
18:22 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Ping timeout: 240 seconds]
18:24 -!- Asbestos_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Ping timeout: 258 seconds]
18:37 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
18:37 < bitcoin-git> [bitcoin] fanquake pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/b18978066d87...ade38b6ee8f9
18:37 < bitcoin-git> bitcoin/master faac315 MarcoFalke: Remove unused and confusing CTransaction constructor
18:37 < bitcoin-git> bitcoin/master fac39c1 MarcoFalke: wallet: document that tx in CreateTransaction is purely an out-param
18:37 < bitcoin-git> bitcoin/master ade38b6 fanquake: Merge #20588: Remove unused and confusing CTransaction constructor
18:37 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
18:37 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
18:37 < bitcoin-git> [bitcoin] fanquake merged pull request #20588: Remove unused and confusing CTransaction constructor (master...2012-txConstructor) https://github.com/bitcoin/bitcoin/pull/20588
18:37 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
18:41 -!- Asbestos_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev
18:44 -!- Chlorine_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Ping timeout: 256 seconds]
18:44 -!- Mercury_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has joined #bitcoin-core-dev
18:47 -!- Asbestos_Vapor [~Mercury_V@174-082-166-092.res.spectrum.com] has quit [Ping timeout: 260 seconds]
18:56 -!- EagleTM [~EagleTM@unaffiliated/eagletm] has joined #bitcoin-core-dev
18:56 -!- Eagle[TM] [~EagleTM@unaffiliated/eagletm] has quit [Ping timeout: 260 seconds]
19:20 -!- joelklabo [~textual@108-196-216-127.lightspeed.sntcca.sbcglobal.net] has quit [Read error: No route to host]
20:00 -!- bitdex [~bitdex@gateway/tor-sasl/bitdex] has joined #bitcoin-core-dev
20:03 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
20:03 < bitcoin-git> [bitcoin] theStack opened pull request #20640: wallet, refactor: return out-params of CreateTransaction() as optional struct (master...202012-refactor-wallet-createtransaction-return_out_params_in_optstruct) https://github.com/bitcoin/bitcoin/pull/20640
20:03 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
20:06 -!- kristapsk_ [~KK@gateway/tor-sasl/kristapsk] has quit [Remote host closed the connection]
20:17 -!- proofofkeags [~proofofke@174-16-212-53.hlrn.qwest.net] has quit [Remote host closed the connection]
20:37 -!- tryphe_ is now known as tryphe
21:19 -!- pinheadmz [~pinheadmz@71.190.30.138] has quit [Quit: pinheadmz]
21:36 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has joined #bitcoin-core-dev
22:08 -!- verybaddad [~thatdad@108-188-072-226.biz.spectrum.com] has joined #bitcoin-core-dev
22:08 -!- verybaddad [~thatdad@108-188-072-226.biz.spectrum.com] has left #bitcoin-core-dev []
22:22 -!- Emcy [~Emcy@unaffiliated/emcy] has quit [Ping timeout: 272 seconds]
22:24 -!- Emcy [~Emcy@unaffiliated/emcy] has joined #bitcoin-core-dev
22:24 -!- bitcoin-git [~bitcoin-g@x0f.org] has joined #bitcoin-core-dev
22:24 < bitcoin-git> [bitcoin] MarcoFalke closed pull request #20639: doc: fix case of GitHub (master...fix-case-of-github) https://github.com/bitcoin/bitcoin/pull/20639
22:24 -!- bitcoin-git [~bitcoin-g@x0f.org] has left #bitcoin-core-dev []
22:50 < dhruvm> Networking question: Are legal CIDR netmasks always 1s followed by 0s? i.e. mask=255.96.0.0 is an illegal mask?
22:50 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:bbcd:2474:9519:9fb5:8678] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
22:51 -!- da39a3ee5e6b4b0d [~da39a3ee5@mx-ll-171.5.29-209.dynamic.3bb.co.th] has joined #bitcoin-core-dev
22:56 < dhruvm> I am trying to reason whether given two CIDR subnet representations, it is possible to have partial address overlap between them.
22:57 < sipa> no, either they don't overlap at all, or one is a subset of the other
22:57 < sipa> or identical to it
22:58 < dhruvm> I see. So, the mask is always 1s followed by 0s then?
23:00 < dhruvm> They seem to be represented as /32 /24 in most places, but some places I've seen 255.255.0.0 etc which has me confused
23:00 < sipa> yeah, in CIDR the netmask has to be 1s and then 0s
23:00 < sipa> you can have netmask that are not CIDR though
23:01 < sipa> those aren't used in practocr afaik though
23:01 < sipa> *practice
23:03 < dhruvm> thanks sipa
23:10 < sipa> yw
23:16 -!- jeremyrubin [~jr@c-73-15-215-148.hsd1.ca.comcast.net] has quit [Ping timeout: 272 seconds]
23:39 -!- zpao [~zpao@139.28.218.148] has quit [Remote host closed the connection]
23:52 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has quit [Read error: Connection reset by peer]
23:52 -!- da39a3ee5e6b4b0d [~da39a3ee5@mx-ll-171.5.29-209.dynamic.3bb.co.th] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
23:54 -!- da39a3ee5e6b4b0d [~da39a3ee5@mx-ll-171.5.29-209.dynamic.3bb.co.th] has joined #bitcoin-core-dev
23:59 -!- gribble [~gribble@unaffiliated/nanotube/bot/gribble] has joined #bitcoin-core-dev
--- Log closed Sun Dec 13 00:00:45 2020