--- Log opened Wed Jun 09 00:00:33 2021
00:18 -!- murch [~murch@user/murch] has quit [Ping timeout: 252 seconds]
08:47 -!- gnusha [~gnusha@user/gnusha] has joined #bitcoin-core-pr-reviews
08:47 -!- Topic for #bitcoin-core-pr-reviews: Meeting every Wednesday at 17:00 | See https://bitcoincore.reviews/ for details | This channel is logged
08:47 -!- Topic set by jnewbery [] [Mon May 24 09:24:20 2021]
08:47 [Users #bitcoin-core-pr-reviews]
08:47 [ _0x0ff     ] [ dhruv      ] [ gimballock] [ jb55_      ] [ mdrollette    ] [ qubenix   ] 
08:47 [ _aj_       ] [ djinni`    ] [ glix      ] [ jkczyz     ] [ meshcollider  ] [ ryanofsky ] 
08:47 [ achow101   ] [ dongcarl   ] [ glozow    ] [ jnewbery   ] [ michaelfolkson] [ S3RK      ] 
08:47 [ amiti      ] [ dr_orlovsky] [ gnusha    ] [ jrawsthorne] [ pinheadmz_    ] [ schmidty  ] 
08:47 [ avril      ] [ emzy       ] [ harding   ] [ kanzure    ] [ powerjungle   ] [ shaunapps ] 
08:47 [ belcher    ] [ fanquake   ] [ hebasto   ] [ koolazer   ] [ promag        ] [ sipa_     ] 
08:47 [ calvinalvin] [ FelixWeis  ] [ Hernan    ] [ laanwj     ] [ provoostenator] [ waxwing   ] 
08:47 [ davterra   ] [ fjahr      ] [ hugohn    ] [ luke-jr    ] [ prusnak       ] [ willcl_ark] 
08:47 [ dergoegge  ] [ ghost43    ] [ jarolrod  ] [ MarcoFalke ] [ prusnak[m]    ] 
08:47 -!- Irssi: #bitcoin-core-pr-reviews: Total of 53 nicks [0 ops, 0 halfops, 0 voices, 53 normal]
08:48 -!- Channel #bitcoin-core-pr-reviews created Wed May 19 12:43:59 2021
08:50 -!- Irssi: Join to #bitcoin-core-pr-reviews was synced in 175 secs
08:57 -!- gnusha [~gnusha@user/gnusha] has joined #bitcoin-core-pr-reviews
08:57 -!- Topic for #bitcoin-core-pr-reviews: Meeting every Wednesday at 17:00 | See https://bitcoincore.reviews/ for details | This channel is logged
08:57 -!- Topic set by jnewbery [] [Mon May 24 09:24:20 2021]
08:57 [Users #bitcoin-core-pr-reviews]
08:57 [ _0x0ff     ] [ dhruv      ] [ gimballock] [ jb55_      ] [ mdrollette    ] [ prusnak[m]] 
08:57 [ _aj_       ] [ djinni`    ] [ glix      ] [ jkczyz     ] [ meshcollider  ] [ qubenix   ] 
08:57 [ achow101   ] [ dongcarl   ] [ glozow    ] [ jnewbery   ] [ michaelfolkson] [ ryanofsky ] 
08:57 [ amiti      ] [ dr_orlovsky] [ gnusha    ] [ jrawsthorne] [ otto_a        ] [ S3RK      ] 
08:57 [ avril      ] [ emzy       ] [ harding   ] [ kanzure    ] [ pinheadmz_    ] [ schmidty  ] 
08:57 [ belcher    ] [ fanquake   ] [ hebasto   ] [ koolazer   ] [ powerjungle   ] [ shaunapps ] 
08:57 [ calvinalvin] [ FelixWeis  ] [ Hernan    ] [ laanwj     ] [ promag        ] [ sipa_     ] 
08:57 [ davterra   ] [ fjahr      ] [ hugohn    ] [ luke-jr    ] [ provoostenator] [ waxwing   ] 
08:57 [ dergoegge  ] [ ghost43    ] [ jarolrod  ] [ MarcoFalke ] [ prusnak       ] [ willcl_ark] 
08:57 -!- Irssi: #bitcoin-core-pr-reviews: Total of 54 nicks [0 ops, 0 halfops, 0 voices, 54 normal]
08:58 -!- Channel #bitcoin-core-pr-reviews created Wed May 19 12:43:59 2021
09:00 -!- Irssi: Join to #bitcoin-core-pr-reviews was synced in 175 secs
09:03 -!- murch [~murch@pool-162-83-132-16.nycmny.fios.verizon.net] has joined #bitcoin-core-pr-reviews
09:04 -!- murch [~murch@pool-162-83-132-16.nycmny.fios.verizon.net] has quit [Changing host]
09:04 -!- murch [~murch@user/murch] has joined #bitcoin-core-pr-reviews
09:17 -!- lightlike [~lightlike@user/lightlike] has joined #bitcoin-core-pr-reviews
09:25 -!- shaunapps [~shaunapps@146.70.38.134] has quit [Quit: Connection closed]
09:26 -!- Guest73 [~Guest73@2600:1702:210:21a0:b106:2010:4909:645f] has joined #bitcoin-core-pr-reviews
09:33 -!- shaunapps [~shaunapps@102.129.153.34] has joined #bitcoin-core-pr-reviews
09:39 -!- jonatack [jonatack@user/jonatack] has joined #bitcoin-core-pr-reviews
09:40 -!- RebelOfBabylon [~RebelOfBa@cpef81d0f878dc3-cmf81d0f878dc0.cpe.net.fido.ca] has joined #bitcoin-core-pr-reviews
09:46 -!- davterra [~davterra@178.128.106.205] has quit [Quit: Leaving]
09:48 -!- dunxen [dunxen@gateway/vpn/protonvpn/dunxen] has joined #bitcoin-core-pr-reviews
09:50 -!- dariusp [~dariusp@c-73-252-226-175.hsd1.ca.comcast.net] has joined #bitcoin-core-pr-reviews
09:50 -!- Guest73 [~Guest73@2600:1702:210:21a0:b106:2010:4909:645f] has quit [Quit: Client closed]
09:53 -!- shaunapps [~shaunapps@102.129.153.34] has quit [Quit: Connection closed]
09:57 -!- raj_ [~raj_@103.77.139.188] has joined #bitcoin-core-pr-reviews
09:58 < dunxen> i forgot the salsa
09:59 -!- shaunapps [~shaunapps@102.129.153.34] has joined #bitcoin-core-pr-reviews
09:59 -!- svav [~svav@82-69-86-143.dsl.in-addr.zen.co.uk] has joined #bitcoin-core-pr-reviews
09:59 < glozow> dunxen: issok we can make the review club 🔥 by ANSWERING QUESTIONS WITH ENTHUSIASM!
10:00 < glozow> #startmeeting
10:00 -!- davterra [~davterra@178.128.106.205] has joined #bitcoin-core-pr-reviews
10:00 < murch> Hi
10:00 < FelixWeis> hi
10:00 < dunxen> Hi!
10:00 < emzy> hi
10:00 < glozow> Welcome to PR Review Club!!
10:00 < dergoegge> hi
10:00 < schmidty> hi
10:00 < michaelfolkson> hi
10:00 < svav> Hi
10:00 < otto_a> hi
10:00 < glozow> Today we're looking at #21800 mempool/validation: mempool ancestor/descendant limits for packages
10:00 < lightlike> hi
10:00 < jnewbery> helloo!
10:00 < dariusp> hi
10:00 < glozow> Notes (and a pretty diagram) here: https://bitcoincore.reviews/21800
10:00 < shaunapps> hi!
10:01 < dunxen> glozow: love the diagram!
10:01 < raj_> hi.. 
10:01 < glozow> Did any of you get a chance to review the PR or check out the notes?
10:01 < glozow> dunxen: thank you!
10:01 < raj_> yes..
10:01 -!- Azorcode [~Azorcode@201.208.240.46] has joined #bitcoin-core-pr-reviews
10:01 < dunxen> yes
10:01 < sipa_> hi
10:02 -!- sipa_ is now known as sipa
10:02 < FelixWeis> very pretty diagrams!
10:02 < murch> y
10:02 < svav> y
10:02 < dariusp> checked out notes but not PR, and yes, great diagrams!
10:02 < dergoegge> review not yet, notes yes
10:02 < jnewbery> yes! Love the transaction family trees
10:02 < glozow> wonderful! :D
10:02 < michaelfolkson> ~y
10:03 < emzy> n
10:03 < glozow> Let's start with our first question then (about single transactions, we'll get to packages in abit). Why do we have ancestor/descendant limits in the mempool?
10:03 -!- cls [~cls@212.102.49.193] has joined #bitcoin-core-pr-reviews
10:03 < glozow>  Why do we use 25 transactions and 101K virtual bytes as our limit?
10:04 < RebelOfBabylon> Not to clog up the mempool?
10:04 < otto_a> 24 is too little and 26 is too much
10:04 < svav> So it can't become a sink of computational resources and an attack site
10:04 < glozow> <RebelOfBabylon> could you elaborate on what "clogged" means?
10:04 < raj_> to remove DOS attack vector by removing expensive computation if transactions later gets removed from mepool. 
10:04 < lightlike> protect against dos attacks
10:05 < glozow> svav raj_ lightlike: yeah! what specifically might get expensive?
10:05 < dunxen> we want to have a limit to the computation we'd need to perform for a chain of txs. things like adding and removing txs from our mempool can be expensive
10:05 < raj_> Not sure on the rationale on exact limits. Curious..
10:05 < michaelfolkson> 25 seems very high. What use cases need 25 ancestors/descendants?
10:05 < glozow> dunxen: yep! What do we need to do for tx chain when we're adding and removing?
10:06 < raj_> glozow, removing large number of descendant can be expensive..  
10:06 < FelixWeis> its not that high if you think of very simple 1 input, 2 output (1 being change)
10:06 < RebelOfBabylon> Well transactions in the mempool use RAM right? So too many large ones would fill up ram. Also you have to verify all parent transactions which can be computationally expensive?
10:06 < RebelOfBabylon> Or not use RAM but sit in RAM*
10:06 < svav> If you try to remove a conflicting transaction that has a large number of descendants in the mempool
10:06 < glozow> to be honest I don't know why 25 exactly. maybe sipa_ knows?
10:06 < FelixWeis> its obviously very inefficinet to do 25 transactions that way in a ~ 10 minute timeframe
10:06 -!- murch1 [~murch@user/murch] has joined #bitcoin-core-pr-reviews
10:07 < dunxen> i think we need to check all the descendants and probably remove them if they're not valid anymore
10:07 < glozow> RebelOfBabylon: in terms of space, we usually set a hard limit on how much space it can take up, and will trim if we get too many transactions
10:07 < glozow> here, we're worried about a high level of ancestor/descendant connectivity within the mempool
10:07 < jnewbery> Fun fact: the ancestor/descendant limits were originally 100/1000 txs: https://github.com/bitcoin/bitcoin/commit/5add7a74a672cb12b0a2a630d318d9bc64dd0f77 https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-August/010221.html
10:07 < glozow> jnewbery: waow!
10:08 < dunxen> wow
10:08 < michaelfolkson> You'd need to really screw up CPFP to need 25 attempts :)
10:08 < FelixWeis> if you know you have to create these 25 outputs, you can do it in 1 batched payment. electrum-wallet even supports "amend by fee" so if you have already an unconfirmed tx in the mempool, you can replace that one with the additional output
10:08 -!- jb55_ is now known as jb55
10:08 < glozow> svav: yeaah that's a good example! you'd need to go through and make sure you remove all of those descendants
10:08 < murch1> glozow: It's a DOS protection and the value is in the range of reasonable?
10:09 < glozow> btw, these limits are configurable using `-limitancestorsize`, etc.
10:09 < michaelfolkson> Or maybe you are desperate not to pay too high fees even a little bit. So you keep CPFP with a tiny fee increase 
10:09 -!- biteskola [~biteskola@170.76.76.188.dynamic.jazztel.es] has joined #bitcoin-core-pr-reviews
10:10 < glozow> well, you could be trying to CPFP 25 transactions at once?
10:10 < murch1> Also ancestor and descendant limits don't effectively limit clusters to be very broad, we found some components in old mempools that were several hundred connected transactions
10:10 < glozow> murch1: yep exactly, that's one of my followup questions.
10:10 < harding> For the vsize limit, there's also a fundamental limit at the size of a block's transaction space.  If the last descendant of a package is further away from its oldest unconfirmed ancestor, it can't contribute fee to that ancestor because they can't be included in the same block.
10:10 < glozow> Imagine a "family" to be any group of transactions connected by parent-child relationships, including descendants-of-ancestors and ancestors-of-descendants. What is the largest possible family that can exist in the mempool, assuming the default limits are applied?
10:10 < murch1> glozow: oops.
10:11 -!- Netsplit *.net <-> *.split quits: murch
10:12 < raj_> glozow, it seems there isn't a theoretical limit to that in mempool, other than max mempool size itself? Because a chain can be formed via multiple packages? 
10:12 < lightlike> harding: but is this a sufficient reason not to accept larger packages? After all, a miner could just mine the package in two batches, adding the newest descendants in a later block.
10:12 < murch1> hint: https://twitter.com/murchandamus/status/1352464020813524994
10:12 < svav> Largest possible family is 25?
10:12 < glozow> harding: mm that's interesting! so if we're building a block and we only have 100KvB of space left, we wouldn't be able to fit a 102KvB package that relies on the lowest descendant's feerate?
10:13 < glozow> svav: nope :)
10:13 < harding> lightlike: the miner of block n doesn't know that they'll mine block n+1, so by including less profitable ancestor transactions in block n, their competitior mining block n+1 profits more.  I don't think that's economically rational.
10:13 < jnewbery> lightlike: the miner is not incentivized to mine the first batch of transactions, since it doesn't expect that it'll mine the next block (unless it has a majority of hash rate on the network)
10:13 < murch1> glozow: It's not limited.
10:13 < glozow> any other guesses?
10:14 < harding> glozow: MAX_MEMPOOL_SIZE number of transactions.
10:14 < murch1> Because transaction graphs can extend sideways by descendants having new unrelated ancestors and vice versa
10:14 < glozow> murch1: I agree with you answer, and thanks for providing a diagram :P
10:14 < murch1> harding: good point ;)
10:14 < glozow> harding and raj_ too
10:14 < lightlike> jnewbery: ok, but only if the feerate of the partial package is not sufficient to be included otherwise
10:15 < glozow> ok, so we've established that families in mempools can get really hairy
10:15 < jnewbery> A related question is "what's the maximum number of ancestors + descendants a single transaction can have", and I think the answer there is 46
10:15 < svav> 25 to the 25?
10:15 -!- powerjungle [~powerjung@h081217087223.dyn.cm.kabsi.at] has quit [Quit: ZNC - https://znc.in]
10:15 < glozow> so we try to limit it, but it's hard to do so
10:15 < jnewbery> or 47 if you include the tx itself
10:16 < glozow> Let's move on to packages: In our package validation code, we already run `CalculateMemPoolAncestors()` on each transaction individually during PreChecks(). Why is this insufficient?
10:16 < jnewbery> lighlike: right - the only point is that the fees associated with txs at the end of that large package don't incentivize the miner to include the txs at the top of that package
10:16 -!- murch1 is now known as murch
10:17 < dergoegge> CalculateMemPoolAncestors() only takes txs into account that are already in the mempool, a tx in a package that has all its parents within the package might exceed the limits but CalculateMemPoolAncestors() would be fine with it because it has 0 ancestors in the mempool (example A)
10:17 < glozow> dergoegge: yes! it needs to know about both in-mempool and in-package ancestors
10:17 -!- Talkless [~Talkless@mail.dargis.net] has joined #bitcoin-core-pr-reviews
10:18 < glozow> What is the package ancestor/descendant policy proposed by this PR?
10:18 < harding> lightlike: beyond having a mempool whose size is greater by default than the size of a block, we don't included transactions in the mempool that can't be included in the next block (e.g. timelocked transactions).  This makes the transaction selection algorithm simple.  A descendent transaction that can't be included in the next block because it had >1e6 vbytes of ancestors would violate that rule.
10:19 < michaelfolkson> glozow: Treating every transaction in the package as each other's ancestor and descendant
10:19 < lightlike> harding: thanks, that makes sense.
10:20 < glozow> michaelfolkson: correct. <insert incest joke here> what does that look like in the code?
10:21 < raj_> glozow, considering the union of the sets of ancestors and descendant of all transactions in a package and then checking weather this giant set exceeds limit. 
10:21 < harding> It's same policy enforced by Bitcoin Core for standard relay and mempool inclusion, but applied using a simple one-line heuristic that each transaction in the package is counted as having as many more ancestors and descendants as there are transactions in the package being evaluated.
10:22 < jnewbery> A bit more historical context around the number 25 being chosen for the ancestor/descendant limit: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-October/011401.html https://github.com/bitcoin/bitcoin/pull/6771
10:22 < glozow> harding: well said!
10:23 < michaelfolkson> jnewbery: Nice, thanks
10:23 < glozow> raj_: yes! implementation-wise, we're putting everyone's ancestors in the same pot while calculating, and using package_count instead of 1 and total_package_size instead of 1 transaction's size
10:23 -!- dr_orlovsky [~dr-orlovs@31.14.40.19] has quit [Ping timeout: 252 seconds]
10:23 < glozow> I'm really curious to hear if anyone came up with alternative proposals? :)
10:24 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has joined #bitcoin-core-pr-reviews
10:24 -!- andrewtoth_ [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has joined #bitcoin-core-pr-reviews
10:24 -!- andrewtoth_ [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has quit [Client Quit]
10:25 < michaelfolkson> Well the obvious one is to ditch that assumption :)
10:25 < harding> One alternative would be to do the full computation on the package just like we would do if each tx was submitted individually, but for the sake of limiting the CPU/memory needed for an eventual P2P package relay, severely restrict the number of transactions allowed in a package from 25 currently to, say, 5.
10:25 < harding> s/needed for/allowed in the worst case/
10:26 < glozow> harding: right! that's an interesting idea
10:27 < murch> raj_: I think that might make it very difficult to assess whether a transaction you're building wuold be able to be put in the mempool
10:27 < raj_> murch, sorry didn't get that.. how so? 
10:28 < sipa> glozow: so it's really only considering whether the entire package can be accepted at once, and not subsets of it?
10:28 < lightlike> Maybe one could try to think of an algorithm to precisely find the set(s) of actually related transactions? Obviously this could easily become too complicated/time-consuming.
10:28 < glozow> sipa: yeah, atomic submission
10:28 < harding> Another alternative, also aimed at allowing P2P package relay, would be allowing the submitter to include a proof of the longest transaction chain in the package.  I *think* it would be possible to take the txids and input outpoints of all the transactions in the package, put them into a tree-like structure, and then to create compact proofs about the shape of that tree.  That's a ridiculously heavyweight solution if we expect most 
10:28 < harding> packages to be just a parent and child.
10:28 < murch> The union of all sets of ancestors and descendants is basically what Clara and I recently described as "Clusters" in our block template improvement proposal
10:29 < murch> They can be rather large. The tweet I linked shows a cluster of over 200 txs, but is followed by another that shows a cluster of 881 txs in the mempool together
10:30 < glozow> harding: are there use cases for packages with more than 2 or 3 transactions where it isn't all 1 chain?
10:30 < murch> I think it would be hard to set a meaningful limit that doesn't restrict naturally occuring clusters. In turn a relatively low limit might be easy to leverage to do things akin from transaction pinning or restricting other parties from bumping their txs
10:32 < murch> *akin to
10:32 < harding> glozow: zmnSCPxj has a proposal on the LN mailing list this week that would very long transaction chains with maybe two children at the top and definitely two children at the bottom.
10:33 < glozow> harding: ahh that's good to know!
10:34 < harding> (The children at the top would be of a confirmed ancestor, so scratch that.)
10:35 < glozow> I am looking forward to those meetings that ariard is hosting, would be good to be able to gather more info on what kinds of packages people are looking to make...
10:35 < harding> Oof, realized that I hadn't put that on my calendar yet.
10:36 < michaelfolkson> Why should diagram G not be a package?
10:36 < glozow> michaelfolkson: because they're not connected
10:36 < murch> harding, glozow: Is there any usecase for transaction packages that are not in the same cluster/family?
10:36 < glozow> there's no reason why you would submit any of those transactions as a package instead of individually
10:36 < michaelfolkson> Can't you combine two independent packages into one to try to save on fees?
10:36 < glozow> michaelfolkson: no?
10:36 < michaelfolkson> One package effectively subsidizes the other package?
10:37 < glozow> how would they do that without being dependent?
10:37 < murch> michaelfolkson: Since there is no dependency, there is no reasons for miners to include both at once.
10:37 < harding> michaelfolkson: I think that's the technique being called "transaction sponsorship".  It would require a consensus change.
10:37 < harding> (And it wouldn't save on fees, just allow more flexible CPFP-like behavior.)
10:38 < michaelfolkson> Hmm interesting, thanks
10:38 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has quit [Quit: Leaving]
10:39 < dariusp> similar question: why is [D] a package? Aren't A and B independent, spending different outputs of '1'?
10:39 < glozow> the only reason you would put transactions in a package instead of submitting them individually is if some package-specific policy will get you something that individual transactions won't. imo the only policy that you'd want is for a tx to be considered for its descendant feerate (or more generally, i guess it's feerate with sponsors in the package) rather than base feerate
10:39 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has joined #bitcoin-core-pr-reviews
10:40 < dariusp> ohh i guess they could both affect the average fee rate of '1'?
10:40 < murch> dariusp: Only if the miner considers candidate sets rather than ancestor sets. ^^
10:40 < glozow> dariusp: good question! yes, they both affect the descendant feerate of 1, although that's not a very good reason to submit them as a package
10:40 < glozow> i suppose they could be preventing 1 from getting evicted together haha
10:41 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has quit [Client Quit]
10:41 < dariusp> ah i see. So if they were submitted independently, does that mean one of them would be accepted and the other not?
10:41 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has joined #bitcoin-core-pr-reviews
10:41 < dariusp> got it, thanks!
10:41 < otto_a> dariusp: it might mean that none would be accepted
10:41 < murch> yeah, after the first got submitted, the other would be beyond the limit
10:42 < otto_a> actually I'm not sure that is true
10:42 < dariusp> but i guess the miner could still split the package and accept just one?
10:42 < murch> glozow: I don't see how they'd collaborate to keep 1 from being evicted in D
10:42 < murch> Unless https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2
10:42 < murch> :p
10:43 < glozow> murch: eviction is by descendant feerate, but this is after it's already accepted to mempool. them being in a package wouldn't really affect whether or not they're accepted
10:43 < glozow> anyway, I'll continue with the review club questions?
10:43 < michaelfolkson> Sorry :)
10:43 < murch> haha, yeah
10:43 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has quit [Client Quit]
10:43 < glozow> How might we test package ancestor/descendant limits? What are some edge cases we might want to consider?
10:43 -!- gimballock [~gimballoc@146.168.102.150] has quit [Quit: Connection closed]
10:43 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has joined #bitcoin-core-pr-reviews
10:43 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has quit [Remote host closed the connection]
10:43 < michaelfolkson> Another PR review club on candidate set based block building
10:44 < michaelfolkson> The edge cases are effectively the diagrams?
10:44 < harding> Testing is a bit tricky because it is more restrictive than the actual relay policy, so we can't be lazy and just spin up a synced node with no mempool and just feed it random packages from another node with a current mempool.
10:45 -!- gimballock [~gimballoc@146.168.102.150] has joined #bitcoin-core-pr-reviews
10:45 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has joined #bitcoin-core-pr-reviews
10:46 < michaelfolkson> A shapes and V shapes in the tests
10:46 < glozow> harding: right. I'm trying to write a fuzzer, but it's so hard to get a package accepted 😂
10:46 < murch> michaelfolkson: There is no PR to review yet
10:47 < harding> Maybe one edge case to watch out for is CPFP carve outs, where we can get a chain of transactions with length 26 (instead of 25) or combined size of 111,000 vbytes (instead of 101 vbytes).
10:47 < michaelfolkson> murch: We've had "conceptual" PR review clubs before. I think we had one on package relay when there was no code (before glozow got to work down this path)
10:48 < glozow> harding: yes, thank you! will work on writing a test for that
10:48 -!- andrewtoth [~andrewtot@bras-base-toroon0954w-grc-79-174-94-4-173.dsl.bell.ca] has quit [Client Quit]
10:48 < michaelfolkson> glozow: What do you mean hard to get a package accepted?
10:49 < otto_a> edge cases: anything that would incentivise miners to accept suboptimal packages
10:49 < michaelfolkson> It seems specific edge scenarios would be better approach than fuzzing for this
10:49 < dariusp> @otto_a what do you mean by suboptimal packages?
10:49 -!- raj_ [~raj_@103.77.139.188] has quit [Quit: Leaving]
10:50 < glozow> michaelfolkson: so the hope is to be able to create random packages, submit them to mempool, and make sure that we aren't exceeding limits (which would indicate that our algorithm underestimated). but i've had a hard time getting it to create valid random packages
10:51 < otto_a> dariusp: one with a lower feerate that a competing package. However I just realized that block acceptance is a different topic from mempool acceptance.
10:51 < michaelfolkson> glozow: So you are waiting a long time for the randomness to create what you need. Yeah deterministic scenarios like murch's crazy tweet picture seems more useful
10:52 < dariusp> otto_a ah right
10:53 < otto_a> are reorgs an edge case?
10:54 < glozow> otto_a: good thinking! no they aren't, because we'll only be using this for package validation. reorgs affect transactions that are already in the mempool, and we process those one at a time
10:55 < michaelfolkson> glozow otto_a: A re-org could cause your package to be dropped from a full mempool
10:55 < michaelfolkson> But that's not initial acceptance
10:55 < glozow> Ok next question: This PR turns the original `CalculateMemPoolAncestors()` operating on a `CTxMemPoolEntry` into a wrapper function, constructing a vector from the singular entry. What could happen if the wrapper function used a copy of the entry instead of the entry itself?
10:56 < glozow> (other than "we need to make a copy which takes extra space and time")
10:56 < otto_a> is there a lock on the mempool?
10:56 < dergoegge> iterator_to only accepts a ref and a ref to a copy would not find what we are looking for?
10:56 < glozow> otto_a: yep
10:58 -!- andrewtoth [~andrewtot@gateway/tor-sasl/andrewtoth] has joined #bitcoin-core-pr-reviews
10:58 < glozow> dergoegge: yes, bonus points to you for being well prepared ^_^ the container's `iterator_to` function requires it to be inside the container, so we'd get a bad value for the copy since it's not there
10:59 < glozow> Last question: What is a `CTxMemPoolEntryRef`?
11:00 < glozow> okie dokie that is left as an exercise to the reader
11:00 < murch> kthxbye
11:00 < glozow> thanks for coming everybody! I really appreciate the feedback and discussions today :) it helped me a lot
11:00 < glozow> #endmeeting
11:01 < dunxen> thank you glozow!
11:01 < dergoegge> thanks glozow! :) 
11:01 < Azorcode> Thanks Bye 
11:01 < lightlike> thanks glozow!
11:01 < dariusp> thank you glozow! super informative
11:01 < RebelOfBabylon> thank you glozow:
11:01 < otto_a> thank you!
11:01 -!- Azorcode [~Azorcode@201.208.240.46] has quit [Quit: Connection closed]
11:01 < dunxen> bye everybody! :)
11:01 -!- dunxen [dunxen@gateway/vpn/protonvpn/dunxen] has quit [Quit: Leaving...]
11:01 < svav> Thanks glozow
11:02 < michaelfolkson> A reference for mempool entry?
11:02 < emzy> thank you glozow!
11:02 < michaelfolkson> Don't know. Thanks glozow
11:02 < glozow> yes, reference to a `CTxMemPoolEntry`
11:02 < svav> I have a question. How many "core" or "serious" Bitcoin developers are there roughly?
11:03 < glozow> link: https://github.com/bitcoin/bitcoin/blob/7257e50dba36328be60f69c998632408802b9a29/src/txmempool.h#L84
11:03 < dergoegge> TIL std::reference_wrapper
11:03 < glozow> this lets you put references to a mempool entry in your containers
11:04 < jonatack> great job glozow! (meeting log up shortly)
11:04 < glozow> jonatack: thank you! :)
11:05 < michaelfolkson> svav: It depends how you define "serious". I'd say 20-50 
11:05 < FelixWeis> thanks glozow!
11:05 < svav> michaelfolkson: Developers making significant and regular contributions
11:06 < michaelfolkson> svav: Today or historically?
11:06 < svav> Today
11:07 < svav> Also, I see El Salvador has made Bitcoin legal tender
11:08 -!- shaunapps [~shaunapps@102.129.153.34] has quit [Quit: Connection closed]
11:10 -!- gimballock [~gimballoc@146.168.102.150] has quit [Quit: Connection closed]
11:11 < michaelfolkson> svav: In last month 40 people had PRs merged https://github.com/bitcoin/bitcoin/pulse/monthly
11:12 < michaelfolkson> Oh no that's wrong, sorry :)
11:12 < michaelfolkson> Excluding merges, 40 authors have pushed 279 commits to master and 287 commits to all branches.
11:13 < michaelfolkson> (in last month)
11:13 < svav> michaelfolkson: OK thank you
11:13 < michaelfolkson> Oh ok, yeah that is right
11:13 -!- otto_a [~ttll@HSI-KBW-46-223-163-155.hsi.kabel-badenwuerttemberg.de] has quit [Quit: Connection closed]
11:16 -!- svav [~svav@82-69-86-143.dsl.in-addr.zen.co.uk] has quit [Quit: Connection closed]
11:21 -!- dariusp [~dariusp@c-73-252-226-175.hsd1.ca.comcast.net] has quit [Quit: Ping timeout (120 seconds)]
11:24 < jonatack> Meeting log is up at https://bitcoincore.reviews/21800#meeting-log 🍰
11:25 < jonatack> glozow: dark-mode compatible drawings requested 😛
11:27 -!- cls [~cls@212.102.49.193] has quit [Quit: Ping timeout (120 seconds)]
11:43 -!- biteskola [~biteskola@170.76.76.188.dynamic.jazztel.es] has quit [Ping timeout: 244 seconds]
11:56 -!- biteskola [~biteskola@170.76.76.188.dynamic.jazztel.es] has joined #bitcoin-core-pr-reviews
11:58 -!- biteskola [~biteskola@170.76.76.188.dynamic.jazztel.es] has quit [Client Quit]
12:07 -!- RebelOfBabylon [~RebelOfBa@cpef81d0f878dc3-cmf81d0f878dc0.cpe.net.fido.ca] has quit [Quit: Connection closed]
12:28 -!- Talkless [~Talkless@mail.dargis.net] has quit [Quit: Konversation terminated!]
12:32 -!- davterra [~davterra@178.128.106.205] has quit [Ping timeout: 268 seconds]
12:33 -!- davterra [~davterra@178.128.106.205] has joined #bitcoin-core-pr-reviews
13:01 -!- belcher [~belcher@user/belcher] has quit [Quit: Leaving]
14:42 -!- lightlike [~lightlike@user/lightlike] has left #bitcoin-core-pr-reviews [Leaving]
14:47 -!- achow101 [~achow101@user/achow101] has quit [Quit: Bye]
14:49 -!- achow101 [~achow101@user/achow101] has joined #bitcoin-core-pr-reviews
14:59 -!- sanket1729 [~sanket172@ec2-100-24-255-95.compute-1.amazonaws.com] has joined #bitcoin-core-pr-reviews
15:01 < sanket1729> Hi, I am looking at https://bitcoincore.reviews/18261, can anyone elaborate on why minisketch is better than sending a bloom filter of unconfirmed txs to the peer?
15:02 < _aj_> you can reconstruct the missing txs' (short) ids from the minisketch, you can't from a bloom filter?
15:03 < sipa> first of all, erlay is not trying to synchronize mempools, it's making relay more efficient
15:03 < sipa> there is a distinction in that the cost is proportional to how much is being relayed, and not proportional to the size of the difference between mempools
15:04 < sipa> if two mempools are irreconcilable (different policies, double-spends, ...), you pay the cost of relay once - trying to synchronize mempools would incur a cost for differences every time
15:04 -!- reez [~textual@45-30-22-183.lightspeed.nsvltn.sbcglobal.net] has joined #bitcoin-core-pr-reviews
15:04 < sipa> so what erlay does is in certain cases instead of relaying, putting the to-be-relayed transactions into a buffer
15:04 < sipa> and then occasionally synchronizes that buffer with the equivalent buffer your peer has for you
15:05 < sipa> and then both buffers get wiped
15:05 < sipa> sanket1729: with me so far?
15:06 < sanket1729> still at the second line, what does "how much is being relayed mean"?
15:06 < sipa> it's like a speed and acceleration
15:07 < sipa> we're paying cost proportional to acceleration, not the resulting speed difference which would grow over time
15:07 < sipa> if we'd try to reconcile our mempools, and fail, because some transactions you have and i don't is one that i cannot accept
15:08 < sipa> then the next time we try to reconcile we'd pay the cost for trying to reconcile that tx again
15:08 < sanket1729> I think I get it
15:09 < sipa> that's not an answer to your question yet, but i think this is a first misconception you may have had
15:09 < _aj_> sipa: (more like velocity and position if you ask me)
15:09 < sanket1729> So, back to bloom filter comparison. 
15:09 < sipa> ok, so you and i both have a set of "would have relayed" transactions
15:09 < sipa> but we know those sets are very likely very similar
15:10 < sipa> a few seconds have passed, and likely both of us have heard about a bunch of transactions from various other sources
15:10 < sipa> many of those overlap
15:16 < sanket1729> I get the relaying part. My question is more about syncing mempools. minisketch vs bloom filter request model
15:17 < sipa> ok
15:17 < sanket1729> After the peers have the to-be-relayed buffer. 
15:17 < sipa> if you'd want to send a bloom filter, it needs to be proportional in size to the number of entries in that buffer
15:18 < sipa> with minisketch it only needs to have a size that's proportional to the _difference_ between the two buffers
15:18 < sanket1729> Ah, thanks :) 
15:18 < sanket1729> Makes more sense now
15:18 < sipa> there is a bloom-filter based construction that is invertible and thus allows the same kind of thing, called IBLT
15:19 < sipa> it's much more efficient CPU size than pinsketch, but its constant factors on bandwidth are much higher, especially for small sets
15:19 < sipa> the minisketch github page has a comparison
15:20 < sanket1729> Thanks, looking at it
15:38 -!- reez [~textual@45-30-22-183.lightspeed.nsvltn.sbcglobal.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
15:42 -!- gimballock [~gimballoc@146.168.102.150] has joined #bitcoin-core-pr-reviews
15:43 -!- gimballock [~gimballoc@146.168.102.150] has quit [Client Quit]
18:36 -!- belcher [~belcher@user/belcher] has joined #bitcoin-core-pr-reviews
19:01 -!- murch [~murch@user/murch] has quit [Quit: WeeChat 1.9.1]
20:40 -!- Hernan [~hernanmar@OL121-235.fibertel.com.ar] has quit [Ping timeout: 252 seconds]
21:33 -!- belcher_ [~belcher@user/belcher] has joined #bitcoin-core-pr-reviews
21:37 -!- belcher [~belcher@user/belcher] has quit [Ping timeout: 268 seconds]
21:47 -!- prusnak[m] [~stickmatr@2001:470:69fc:105::98c] has quit [Read error: Connection reset by peer]
21:48 -!- prusnak[m] [~stickmatr@2001:470:69fc:105::98c] has joined #bitcoin-core-pr-reviews
--- Log closed Thu Jun 10 00:00:33 2021