--- Log opened Wed Nov 03 00:00:22 2021
01:47 -!- common [~common@096-033-221-075.res.spectrum.com] has quit [Remote host closed the connection]
01:47 -!- common [~common@096-033-221-075.res.spectrum.com] has joined #bitcoin-core-pr-reviews
02:59 -!- Guest25 [~Guest25@2401:4900:3b1e:5352:f972:bcb7:e55:ef96] has joined #bitcoin-core-pr-reviews
03:00 < Guest25> Hellooooo
03:00 < Guest25> ')
03:00 -!- Guest25 [~Guest25@2401:4900:3b1e:5352:f972:bcb7:e55:ef96] has quit [Client Quit]
03:03 -!- kexkey [~kexkey@static-198-54-132-134.cust.tzulo.com] has quit [Ping timeout: 260 seconds]
03:06 -!- kexkey [~kexkey@static-198-54-132-86.cust.tzulo.com] has joined #bitcoin-core-pr-reviews
07:26 -!- svav [~svav@82-69-86-143.dsl.in-addr.zen.co.uk] has joined #bitcoin-core-pr-reviews
07:59 -!- Clint2 [~Clint@68-255-237-145.lightspeed.hstntx.sbcglobal.net] has joined #bitcoin-core-pr-reviews
08:01 -!- nickbar86 [~nickbar86@91.206.0.63] has joined #bitcoin-core-pr-reviews
08:03 -!- nickbar86 [~nickbar86@91.206.0.63] has quit [Client Quit]
08:05 -!- svav [~svav@82-69-86-143.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 268 seconds]
08:10 -!- grettke [~grettke@cpe-65-29-228-30.wi.res.rr.com] has quit [Ping timeout: 268 seconds]
08:10 -!- grettke [~grettke@cpe-65-29-228-30.wi.res.rr.com] has joined #bitcoin-core-pr-reviews
08:18 < jnewbery> Hi folks. Just a quick reminder that our meeting is scheduled in UTC, so if you live somewhere where the clocks went back over the weekend, it'll be an hour earlier than last week for you.
08:19 < jnewbery> so we get started about 1hr 40 minutes from now
08:58 -!- mariona [~mariona@93.176.129.45] has joined #bitcoin-core-pr-reviews
08:59 -!- mariona is now known as seaona
09:01 -!- satoshi [~satoshi@gateway/tor-sasl/satoshi] has joined #bitcoin-core-pr-reviews
09:31 -!- satoshi [~satoshi@gateway/tor-sasl/satoshi] has quit [Ping timeout: 276 seconds]
09:45 < Kaizen_Kintsugi> thanks for the heads up
09:50 -!- tr3xx [~tr3xx@2a01:799:5f:ec00:dd3d:e841:2b0:279d] has joined #bitcoin-core-pr-reviews
09:52 -!- nickbar86 [~nickbar86@91.206.1.124] has joined #bitcoin-core-pr-reviews
09:59 -!- ccdle12 [~ccdle12@243.222.90.149.rev.vodafone.pt] has joined #bitcoin-core-pr-reviews
10:00 < jnewbery> #startmeeting
10:00 < jnewbery> Hi folks! Welcome to Bitcoin Core PR Review Club.
10:00 < jnewbery> Feel free to say hi to let everyone know you're here.
10:00 -!- stickies-v [~stickies-@89.38.69.191] has joined #bitcoin-core-pr-reviews
10:01 < stickies-v> hi everyone!
10:01 < tr3xx> Hi guys!
10:01 < michaelfolkson> hi
10:01 < seaona> hi :)
10:01 < ccdle12> hi everyone
10:01 < jnewbery> phew. I was a bit worried that it was just me this week!
10:01 < nickbar86> Hi!
10:01 < jnewbery> Thanks everyone for coming to learn more about the Bitcoin protocol, Bitcoin Core, and the review process.
10:02 < jnewbery> Is anyone here for the first time?
10:02 < nickbar86> First time for me
10:02 -!- Sachin [~Sachin@67.23.55.162] has joined #bitcoin-core-pr-reviews
10:02 < jnewbery> nickbar86: welcome! Feel free to jump in and ask questions at any point.
10:03 < jnewbery> We're all here to learn, so everyone should feel comfortable asking questions if anything is unclear
10:03 < Clint2> First timer here.
10:03 < jnewbery> Clint2: welcome :)
10:03 < Clint2> Thanks
10:03 < jnewbery> Notes and questions are in the normal place: https://bitcoincore.reviews/23173
10:04 < jnewbery> ok, let's get into it. Who had a chance to review the PR / notes&questions? (y/n)
10:04 < stickies-v> y
10:04 < tr3xx> y
10:04 < jnewbery> no worries if it's an 'n' this week.
10:04 < seaona> y
10:04 < ccdle12> y
10:04 < michaelfolkson> 0.5y
10:05 < jnewbery> oh great. That's a lot of review. What did you all think? Concept ACK? approach ACK? code review ACK? NACK?
10:05 < Kaizen_Kintsugi> hello
10:05 < Kaizen_Kintsugi> y
10:05 < stickies-v> utACK
10:06 < jnewbery> anyone want to have a go at explaining the motivation?
10:06 < Kaizen_Kintsugi> ACK to the best of my ability. I'm quite interested in this as it seems to push assume UTXO forward.
10:06 < ccdle12> concept ACK for the code redesign
10:06 < jnewbery> Kaizen_Kintsugi: can you explain why you think it pushes assume UTXO forward?
10:06 -!- jasan [~j@tunnel625336-pt.tunnel.tserv1.bud1.ipv6.he.net] has joined #bitcoin-core-pr-reviews
10:07 < Kaizen_Kintsugi> afaik this is a refactor for cleaner abstraction, the chain state manager handles relevant responsibilities
10:07 < Kaizen_Kintsugi> I read it in the description
10:07 < lsilva_> The PR adapts the ATMP code to Assume UTXO.
10:07 < michaelfolkson> Concept ACK, Approach ACK. Just skimmed commits so far (commit messages are great)
10:07 < Kaizen_Kintsugi> but I dont know how or the specifics, i remember reading that AssumeUTXO was a large undertaking and a large optimization
10:09 -!- pg2 is now known as pg156
10:09 < stickies-v> code outside of validation.cpp should not need to care about chainstate management, so through ChainstateManager::ProcessTransaction those callers can now add transactions without needing a reference to mempool or chaintip
10:09 < jnewbery> Kaizen_Kintsugi: AssumeUTXO is a huge project that touches lots of parts of validation and init. This PR wasn't motivated specifically by AssumeUTXO, but I'd argue that clarifying interfaces should help add new features if done right
10:09 < jnewbery> stickies-v: exactly right!
10:09 < jnewbery> let's get into more specifics
10:09 < Kaizen_Kintsugi> jnewbery ty, thx stickies
10:09 < jnewbery> What is cs_main?
10:10 < Kaizen_Kintsugi> that is a thread lock
10:10 < Kaizen_Kintsugi> and that part baffles me how the locks work in this
10:10 < lsilva_> A mutex to protect critical data ?
10:10 -!- Talkless [~Talkless@mail.dargis.net] has joined #bitcoin-core-pr-reviews
10:10 -!- SRT58 [~SRT@2800:e2:280:369:39e8:e758:d7e8:3f08] has joined #bitcoin-core-pr-reviews
10:10 < stickies-v> a mutex - a bit fuzzy on the details but i think it tries to ensure validation consistency by only allowing a single thread at the same time to modify crucial components like chainstate etc?
10:10 < seaona> it is a mutex class, and it is locked in several places of the codebase, specially around wallet sync and mempool processes ..
10:10 < ccdle12> the main mutex in bitcoin, I think it guards mainly chainstate and validation
10:11 < jnewbery> Kaizen_Kintsugi: being pedantic for a second, cs_main is a *mutex*, and that mutex is *locked* in code paths, which excludes other threads from reading/writing data at the same time
10:11 < jnewbery> lots of great answers!
10:11 < Kaizen_Kintsugi> Aye, I just refreshed my understanding of what a mutex is
10:12 < jnewbery> anyone know why it's called cs_main ?
10:12 < ccdle12> it used to live in the main.cpp file when it existed?
10:12 < lsilva_> Most of the protected code was in main.cpp ?
10:12 < stickies-v> it was used in satoshi's main.cpp which used to be much bigger than it is now
10:13 < jnewbery> yes, that's where the "main" part comes from. How about "cs_" ?
10:13 < sipa> stickies-v: not just bigger; it existed :)
10:13 < jnewbery> a bunch of the mutexes in bitcoin core are called "cs_". Why?
10:13 < ccdle12> chain state? :)
10:13 < seaona> chain state?
10:13 < lsilva_> Critical Section ?
10:13 < sipa> (main.cpp was split into validation.cpp and net_processing.cpp at some point, after many other things were already moved elsewhere)
10:13 < jnewbery> haha, chain state is a good guess, but it's actually Critical Section. Well done lsilva_.
10:14 -!- SRT58 [~SRT@2800:e2:280:369:39e8:e758:d7e8:3f08] has quit [Client Quit]
10:14 < stickies-v> sipa hah hadn't even noticed that it was gone entirely, ty!
10:14 < jnewbery> sipa: right, that was done in https://github.com/bitcoin/bitcoin/pull/9260
10:15 < tr3xx> lol @ TheBlueMatt's initial comment on that PR
10:15 < Kaizen_Kintsugi> is there a visual map of this stuff to help me wrap my head around this?
10:15 < michaelfolkson> Were all PRs introduced with song lyrics in 2016 lol
10:15 -!- SRT [~SRT@190.240.72.41] has joined #bitcoin-core-pr-reviews
10:15 < jnewbery> The mutexes from satoshi's time and for some time after were called "cs_", presumably because satoshi was developing on windows, and CriticalSection is a windows concurrency class: https://docs.microsoft.com/en-us/cpp/cppcx/wrl/criticalsection-class?view=msvc-160
10:16 < jnewbery> Kaizen_Kintsugi: not that I'm aware of, unfortunately. It's mostly in people's heads
10:16 < ccdle12> Kaizen_Kintsugi: I'm not sure, but this presetnation by jamesob helped me alot: https://jameso.be/dev++2018/#26
10:17 -!- gemini51 [~gemini@223.178.95.197] has joined #bitcoin-core-pr-reviews
10:17 < jnewbery> so to summarize, cs_main is the big lock in Bitcoin Core. It _should_ be predominantly to protect chainstate and validation, but there's also a lot of state in net_processing and other places that are guarded by cs_main
10:18 < jnewbery> sorry cs_main is the big *mutex. I'll try to be more precise in my language
10:18 -!- randymcmillan [~randymcmi@47.205.7.87] has joined #bitcoin-core-pr-reviews
10:18 < Kaizen_Kintsugi> ccdle12: this is amazing ty
10:18 < jnewbery> Breaking that mutex up so that not everything is guarded under the same mutex is a long-term goal of many contributors
10:19 < jnewbery> I'll ask the next question, but if anything so far is unclear, don't hesitate to go back and ask questions about it
10:19 < jnewbery> Which components currently call AcceptToMemoryPool()? List all the places that we’ll call ATMP.
10:19 < stickies-v> src/bench/block_assemble.cpp; src/net_processing.cpp; src/node/transaction.cpp; src/rpc/rawtransaction.cpp; src/test/fuzz/tx_pool.cpp; src/test/util/setup_common.cpp; src/validation.cpp
10:19 < lsilva_> https://www.irccloud.com/pastebin/fKR0mrp6/
10:20 < seaona> CChainState::MaybeUpdateMempoolForReorg()
10:20 < seaona> MempoolAcceptResult ChainstateManager::ProcessTransaction()
10:20 -!- Azorcode [~Azorcode@186.88.128.64] has joined #bitcoin-core-pr-reviews
10:21 < jnewbery> Ah, I see you know how to grep!
10:21 < jnewbery> ok, in words, which are the important call sites
10:21 < Kaizen_Kintsugi> stickies-v: how did you do that grep?
10:21 < glozow> (1) loading mempool.dat from disk, (2) receiving a new tx on p2p, (3) adding transactions from disconnected blocks (4) from clients like wallet and rpc via BroadcastTransaction
10:21 < jnewbery> glozow: great answer!
10:21 < stickies-v> Kaizen_Kintsugi I cheated and used VSCode's find all references...
10:22 < jnewbery> I'll add (4b) directly from rpc for `testmempoolaccept`
10:22 < jnewbery> but otherwise I think that's exhaustive, excludings tests/fuzzers/benches
10:23 < jnewbery> everyone happy with that? Or anyone have questions?
10:23 < stickies-v> I think it's also called when doing package validation? But that's not merged yet I think
10:23 < Kaizen_Kintsugi> 1 breaks my intuitive understanding, why is there a mempool dat? I thought that was just in memory
10:24 < Kaizen_Kintsugi> mempool is written to the db for a reason?
10:24 < glozow> package validation does not call ATMP, no
10:24 < lsilva_> To persist mempool when the node restarts 
10:24 < jnewbery> follow up question. Out of (1), (2), (3), (4) and (4b), which are calls from inside validation, and which are from external client code (where by client I mean other components within bitcoin core, not external programs)
10:24 < Kaizen_Kintsugi> oh cool cool that makes sense
10:25 < Kaizen_Kintsugi> 1 -validate, 2-validate- 3-external? 4=external
10:25 < Kaizen_Kintsugi> maybe 2 is external
10:25 < Kaizen_Kintsugi> shit
10:25 < lsilva_> 1 - validation 2 - validation 3 - validation 4 - external
10:26 < jnewbery> stickies-v: yes, the same logic is involved in package validation, although that doesn't actually use the ATMP function. Package validation would be from the same call sites as (2) and (4) when it gets merged
10:27 < jnewbery> yeah, mempool.dat is only written on shutdown and read on startup usually. I think there may be an rpc to dump it manually but I'm not 100% sure about that
10:27 < jnewbery> ok so 1 and 3 are definitely from within validation
10:27 < lsilva_> Yes, savemempool RPC dumps it manually
10:27 -!- svav [~svav@82-69-86-143.dsl.in-addr.zen.co.uk] has joined #bitcoin-core-pr-reviews
10:28 < jnewbery> here's 1: https://github.com/bitcoin/bitcoin/blob/23ae7931be50376fa6bda692c641a3d2538556ee/src/validation.cpp#L4489-L4490
10:28 < jnewbery> (it's actually calling AcceptToMemoryPoolWithTime() but whatever)
10:29 < jnewbery> and here's 3: https://github.com/bitcoin/bitcoin/blob/23ae7931be50376fa6bda692c641a3d2538556ee/src/validation.cpp#L352-L354
10:29 < lsilva_> The (2) is originally in net_processing
10:29 < jnewbery> (2) is being called from net_processing
10:29 < lsilva_> Before this PR, I mean
10:29 < jnewbery> lsilva_: yes, exactly right!
10:30 < jnewbery> I'd call net_processing a client of validation. It calls into various functions to either provide new data for validation to validate, or to to request what the current state is
10:30 < jnewbery> make sense?
10:30 < jnewbery> and (4) is here: https://github.com/bitcoin/bitcoin/blob/23ae7931be50376fa6bda692c641a3d2538556ee/src/node/transaction.cpp#L73-L83
10:30 < jnewbery> that's the interface through which the rpc and wallet clients submit transactions to the mempool
10:31 < lsilva_> Yes. net_processing calls validation layer to validate incoming data.
10:31 < jnewbery> lsilva_: yep!
10:31 < jnewbery> ok, onwards. What does CTxMemPool::check() do? Whose responsibility is it to call that function?
10:32 -!- gemini51 [~gemini@223.178.95.197] has quit [Quit: Connection closed]
10:32 < stickies-v> CTxMemPool::check() verifies that all the transactions in the mempool are consistent, e.g. they don't spend the same inputs twice. Since it's by default not used on mainnet, I think this is mostly a dev/debug feature?
10:33 < seaona> it asserts that total tx sizes, fees.. in the mempool are correct
10:33 < lsilva_> It validates the mempool. It was introduced in the PR #2876.
10:33 < jnewbery> very good!
10:34 < lsilva_> Net Processing layer (ProcessOrphanTx and ProcessMessage (NetMsgType::TX) calls this function.
10:34 < stickies-v> Previously, it was the responsibility of anyone interacting with the mempool. Since this PR, ChainstateManager::ProcessTransaction takes care of this automatically.
10:34 < jnewbery> exactly right. Lots of good answers today. Everyone's done their homework :)
10:35 < lsilva_> This PR changes the responsibility from net processing to Validation layer (ProcessTransaction and ActivateBestChainStep).
10:35 < jnewbery> stickies-v's point about it being off by default is important. If that wasn't the case, then this PR might be a performance pessimization. But since check() is generally only used for testing, it's not a problem.
10:36 < jnewbery> Next question. What does the bypass_limits argument in ATMP do? In which circumstances is ATMP call with bypass_limits set to true?
10:36 < pg156> If `bypass_limits` is true:
10:36 < pg156>     - the mempool size is no longer limited to the default 300MB. 
10:36 < pg156>     - the fee rate of a package doesn't need to be above a threshold
10:36 < seaona> bypass_limits means that we won't force the mempool fee and capacity limits. I don't know in which circumstances would be used
10:37 < pg156> I can see `bypass_limits` is set to true here:
10:37 < pg156> https://github.com/bitcoin/bitcoin/blob/f87e07c6fe321f0fb97703c82c0e4122f800589f/src/validation.cpp#L353
10:37 < pg156> but don't understand exactly why
10:37 < jnewbery> pg156: Almost! It's the feerate of the individual transaction that doesn't need to be above a the mempool's min feerate
10:37 < lsilva_> When true, bypass_limits  doesn't enforce mempool fee and capacity limits.
10:37 < jnewbery> rather than a package (the mempool doesn't currently accept packages atomically)
10:38 < jnewbery> (glozow is working on fixing that!)
10:38 < jnewbery> There's actually only one place where bypass_limits is set to true, and it's in MaybeUpdateMempoolForReorg()
10:38 -!- BlueMoon [~BlueMoon@dgb3.dgbiblio.unam.mx] has joined #bitcoin-core-pr-reviews
10:39 < jnewbery> What is that function doing, and why do we want to bypass those limits there? Any guesses?
10:39 < seaona> for testing?
10:39 < Kaizen_Kintsugi> is this when a new chain state is chosen
10:40 < jnewbery> nope and nope. Keep guessing :)
10:40 < Kaizen_Kintsugi> and transactions from orphaned blocks need to be put back into the mempool?
10:40 < pg156> When a node is disconnected?
10:40 < jnewbery> Kaizen_Kintsugi pg156: yes!
10:40 < lsilva_> To remove or re-add transactions from disconnected blocks ?
10:40 < jnewbery> right!
10:41 < pg156> But what exactly happens when a node is disconnected and how it rebuilds the mempool? I am confused.
10:42 < jnewbery> during a reorg, we'll disconnect one or more blocks and then connect one or more competing blocks. The blocks that are disconnected will probably have a bunch of transactions in them.
10:42 < Kaizen_Kintsugi> I imagine a lot of the transactions will be similar
10:42 < jnewbery> Now usually, we'd expect _most_ of those transactions to appear in the new blocks that are connected
10:42 < jnewbery> but there may be some that aren't, and it'd be a shame to lose those, so we try to put them back in our mempool
10:43 < stickies-v> so hypothetically, a very large reorg could crash low resource nodes because they run out of memory?
10:43 < jnewbery> stickies-v: that has been a potential problem in the past. We try to limit the worst case memory usage
10:44 < Kaizen_Kintsugi> yea it seems like potentially a giant blob of transactions could hit the mempool really quick
10:44 < jnewbery> here's the comment for the class that handles this: https://github.com/bitcoin/bitcoin/blob/23ae7931be50376fa6bda692c641a3d2538556ee/src/txmempool.h#L869-L882
10:44 < Kaizen_Kintsugi> i would assume older transactions would push out new transactions from the mempool if a node cant hold them?
10:44 < glozow> not just a shame, pretty sure that makes reorgs more dangerous if the disconnected blocks’ transactions are lost
10:45 < glozow> LimitMempoolSize expires transactions more than 2 weeks old first, then evicts by descendant feerate
10:46 < jnewbery> and here's where we limit the size of that disconnectpool to ensure that a large reorg doesn't blow up our memory usage: https://github.com/bitcoin/bitcoin/blob/23ae7931be50376fa6bda692c641a3d2538556ee/src/validation.cpp#L2197-L2202
10:47 < Kaizen_Kintsugi> oh neat, so there is a disconnect pool inside the mempool if I understand correctly, there is a 'partition' for this stuff to happen?
10:47 < glozow> pg156: not when a node is disconnected, when a block is disconnected.
10:48 < pg156> glozow: right, i can see that now
10:48 < jnewbery> Kaizen_Kintsugi: It's not inside the mempool. It's a separate object that is created during the disconnect/reconnect sequence
10:48 < jnewbery> short-lived
10:48 < Kaizen_Kintsugi> ah okay okay that makes sense
10:48 < lsilva_> What is exact the "disconnected block" concept ? Block that will be discarded in reorg ?
10:48 < Kaizen_Kintsugi> its only created when we need it
10:49 < jnewbery> lsilva_: "connecting" a block is applying its transactions to the UTXO set (removing outputs that are spent by txs in the block and adding new outputs that are created) and updating the chainstate
10:49 < stickies-v> hmm so during reorg we ignore the default mempool size of 300MB but instead use a (default) MAX_DISCONNECTED_TX_POOL_SIZE of 20MB of disconnected transactions? Then doesn't it make more sense to just keep the 300MB mempool limit?
10:50 < jnewbery> "disconnecting" a block is the reverse: removing the UTXOs that were created in the block and re-adding those that were spent in the block (which were stored in the 'undo data' for that block on disk)
10:50 < jnewbery> so during a re-org we disconnect one or more block from the tip of the block chain, and then connect the competing blocks
10:51 < Kaizen_Kintsugi> so disconnect -> undo utxo set -> add new block
10:51 < pg156> so for those transactions from disconnected blocks (but not in new connected blocks), are they added to mempool while bypassing the limits? If so, what if the fee rate of them is too low?
10:51 < lsilva_> jnewbery Thanks !!
10:51 < stickies-v> although I suppose pushing out 20MB of transactions from the mempool (assuming it's full) is much more costly than just temporarily allocating an extra 20MB
10:51 < jnewbery> stickies-v: that's a good question! I'm not entirely clear about why we should ignore the 300MB limit when we're putting transactions into the mempool during a re-org
10:51 < Kaizen_Kintsugi> is it incase the reorg is really big?
10:52 < Kaizen_Kintsugi> like 400-500 blocks
10:52 < Kaizen_Kintsugi> which kinda sounds insane
10:52 < Kaizen_Kintsugi> and near infeasable
10:52 < glozow> that’s 3 days worth of blocks
10:52 < Kaizen_Kintsugi> yea
10:53 < jnewbery> I think it may be due to ancestor/descendant chains. Since the transactions can only be added sequentially, we can't look at the ancestor/descendant feerates when trying to add the transactions back, so it's perhaps better to add everything, and then limit the mempool size if necessary
10:53 < Kaizen_Kintsugi> that sounds like a complication with child pays for parent or something?
10:54 < jnewbery> No, I don't think it's anything to do with a super-large re-org. In that case we'd lose most of the transactions, since the MAX_DISCONNECTED_TX_POOL_SIZE is limited to 20MB
10:54 < jnewbery> to be honest, I haven't figured out exactly what the rationale is, but this PR doesn't change the behaviour for bypass_limits at all so that's ok :)
10:55 < jnewbery> Question 6: One of the commit logs reads: [...] What was the change that removed that logic?
10:55 < SRT> Uploaded file: https://uploads.kiwiirc.com/files/f8e51cfaf20a4f92dc5a1a51f9804be9/pasted.txt
10:55 -!- danp [~danp@174-21-103-147.tukw.qwest.net] has joined #bitcoin-core-pr-reviews
10:56 < jnewbery> I'm not going to paste the entire code comment into the chat. You can read it here: https://bitcoincore.reviews/23173#questions
10:57 < Kaizen_Kintsugi> looks like it removes a redundancy
10:57 < lsilva_> Pruned blocks ?
10:57 < lsilva_> https://github.com/bitcoin/bitcoin/pull/1677 ?
10:57 < glozow> well if you were at 290mb and disconnectpool is 20mb, your only choices are (1) cut down to 280mb and then accept or (2) allow going to 310mb and then cut right? the former means you might end up with lower fee transactions
10:58 < glozow> i suppose you could statically analyze the disconnect pool, split them into packages, and then submit
10:58 < jnewbery> glozow: Yes, I agree!
10:58 < glozow> that’d be a use case for no-topology-restrictions package submission
10:58 -!- nickbar86 [~nickbar86@91.206.1.124] has quit [Quit: Connection closed]
10:59 < jnewbery> lsilva_: not pruned blocks, but pruning spent TXOs from our storage
10:59 < jnewbery> ultraprune was a really important change. If you're interested, you can hear Pieter talking about it here: https://podcast.chaincode.com/2020/01/27/pieter-wuille-1.html
10:59 < jnewbery> ok, that's time. I gotta run. Thanks everyone!
10:59 < jnewbery> #endmeeting
10:59 < Kaizen_Kintsugi> Thanks! Learning this is so awesome
10:59 < svav> Thanks
10:59 < lsilva_> Thanks 
10:59 < Kaizen_Kintsugi> I need to understand UTXO set better
11:00 < glozow> thanks jnewbery
11:00 < tr3xx> Thanks for a great session jnewberry!
11:00 < pg156> Thank you John and everyone!
11:00 < ccdle12> thanks!
11:00 < Kaizen_Kintsugi> from what I understand it is a set of all the unspent transactions in the entire blockchain
11:00 < stickies-v> thanks everyone!
11:00 < BlueMoon> Thanks!!
11:00 < Kaizen_Kintsugi> and is it a part of validation? chain state? where does that thing live?
11:01 < Kaizen_Kintsugi> *unpsent transaction outputs*
11:01 -!- tr3xx [~tr3xx@2a01:799:5f:ec00:dd3d:e841:2b0:279d] has quit [Quit: Leaving]
11:01 < seaona> thank you all!
11:01 < michaelfolkson> Kaizen_Kintsugi: You can think in terms of blocks or in terms of UTXOs and spent TXOs. Different data structures
11:02 < michaelfolkson> UTXO set is updated every time there is a new block
11:02 < lsilva_> Yes, it is part of validation. You can check class CChainState in validation.h
11:03 -!- randymcmillan [~randymcmi@47.205.7.87] has quit [Quit: Connection closed]
11:03 -!- seaona [~mariona@93.176.129.45] has quit [Quit: Client closed]
11:03 -!- Azorcode [~Azorcode@186.88.128.64] has quit [Ping timeout: 268 seconds]
11:03 -!- BlueMoon [~BlueMoon@dgb3.dgbiblio.unam.mx] has quit [Quit: Connection closed]
11:03 -!- ___nick___ [~quassel@cpc68286-cdif17-2-0-cust533.5-1.cable.virginm.net] has joined #bitcoin-core-pr-reviews
11:04 -!- svav [~svav@82-69-86-143.dsl.in-addr.zen.co.uk] has quit [Quit: Connection closed]
11:04 < SRT> Uploaded file: https://uploads.kiwiirc.com/files/f8e51cfaf20a4f92dc5a1a51f9804be9/pasted.txt
11:04 < SRT> Thanks you for this space 
11:04 -!- stickies-v [~stickies-@89.38.69.191] has quit [Quit: Connection closed]
11:11 -!- ccdle12 [~ccdle12@243.222.90.149.rev.vodafone.pt] has quit [Quit: Client closed]
11:14 -!- SRT [~SRT@190.240.72.41] has quit [Quit: Connection closed]
11:15 -!- stickies-v [~textual@193.117.145.70] has joined #bitcoin-core-pr-reviews
11:20 < michaelfolkson> Some of these Doxygen graphs are fun https://doxygen.bitcoincore.org/dir_68267d1309a1af8e8297ef4c3efbcdba.html
11:26 -!- stickies-v [~textual@193.117.145.70] has quit [Quit: stickies-v]
11:27 < Kaizen_Kintsugi> Gah
11:27 < Kaizen_Kintsugi> that's some spagetti but really helpful for me
11:28 < Kaizen_Kintsugi> from my reading so far, CCoinsView is the UTXO set?
11:28 < Kaizen_Kintsugi> or where it is kept?
11:29 -!- stickies-v [~textual@193.117.145.70] has joined #bitcoin-core-pr-reviews
11:38 < sipa> ccoinview is the abstract interface for things that represent *a* utxo set
11:39 < sipa> there are multiple of those; the one on disk, the cache in memory of the one one disk, the one implied by the mempool, ...
11:39 -!- stickies-v [~textual@193.117.145.70] has quit [Quit: stickies-v]
11:42 -!- stickies-v [~textual@193.117.145.70] has joined #bitcoin-core-pr-reviews
11:54 < Kaizen_Kintsugi> thanks,
12:14 -!- Sachin [~Sachin@67.23.55.162] has quit [Quit: Connection closed]
12:18 < michaelfolkson> sipa: What is the UTXO set implied by the mempool? There's no predictive element of what will be in the next mined block is there? So the mempool is treated by the codebase as irrelevant to any future changes to the UTXO set?
12:22 < sipa> michaelfolkson: for the mempool utxo set, the mempool is counted
12:22 < sipa> for the chain one, it isn not
12:23 < sipa> the mempool utxo set is effectively: take the tip of the active chain, and pretend every mempool transaction were mined
12:23 < michaelfolkson> A UTXO set assuming all the transactions in the mempool are applied to the current UTXO set?
12:23 < sipa> right
12:23 < sipa> tyat's what new mempool tx are validated against
12:24 < sipa> because those can spend outputs that only exist in the mempool
12:25 < michaelfolkson> Wow, did not know that. Presumably the UTXO set is updated every time a tx is booted from the mempool or a RBF transaction comes in?
12:26 < sipa> there is nothing "updated"
12:26 < sipa> the mempool just implies a utxo set
12:26 < sipa> it's an abstract concept; there is no materialized database for.it
12:26 < michaelfolkson> Oh gotcha. I thought you meant it was stored and updated with every change to the mempool
12:26  * michaelfolkson sweats
12:27 < michaelfolkson> Ok yeah you said "the one implied by the mempool". Sorry, my misunderstanding
13:21 -!- Talkless [~Talkless@mail.dargis.net] has quit [Quit: Konversation terminated!]
13:24 -!- Clint2 [~Clint@68-255-237-145.lightspeed.hstntx.sbcglobal.net] has quit [Ping timeout: 260 seconds]
13:41 -!- stickies-v [~textual@193.117.145.70] has quit [Quit: stickies-v]
14:04 -!- ___nick___ [~quassel@cpc68286-cdif17-2-0-cust533.5-1.cable.virginm.net] has quit [Ping timeout: 268 seconds]
14:14 -!- yonson [~yonson@2600:8801:d900:7bb::d7c] has quit [Remote host closed the connection]
14:14 -!- yonson [~yonson@2600:8801:d900:7bb::d7c] has joined #bitcoin-core-pr-reviews
15:12 -!- danp [~danp@174-21-103-147.tukw.qwest.net] has quit [Quit: Connection closed]
15:40 < Kaizen_Kintsugi> okay that makes sense now, the mempool is a utxo set implicitly.
15:59 -!- stickies-v [~textual@185.169.233.7] has joined #bitcoin-core-pr-reviews
16:38 -!- stickies-v [~textual@185.169.233.7] has quit [Quit: stickies-v]
17:05 -!- stickies-v [~textual@178.238.11.9] has joined #bitcoin-core-pr-reviews
18:01 -!- stickies-v [~textual@178.238.11.9] has quit [Quit: stickies-v]
20:33 -!- gnusha [~gnusha@user/gnusha] has quit [Ping timeout: 264 seconds]
--- Log closed Thu Nov 04 00:00:23 2021