--- Day changed Wed Feb 10 2021
00:38 -!- jonatack [~jon@37.167.157.108] has joined #bitcoin-core-pr-reviews     
01:05 -!- jonatack_ [~jon@37.166.50.22] has joined #bitcoin-core-pr-reviews     
01:08 -!- jonatack_ [~jon@37.166.50.22] has quit [Client Quit]     
01:08 -!- jonatack_ [~jon@37.166.50.22] has joined #bitcoin-core-pr-reviews     
01:08 -!- jonatack_ [~jon@37.166.50.22] has quit [Client Quit]     
01:09 -!- jonatack [~jon@37.167.157.108] has quit [Ping timeout: 240 seconds]     
01:09 -!- jonatack [~jon@37.166.50.22] has joined #bitcoin-core-pr-reviews     
02:20 -!- mol_ [~mol@unaffiliated/molly] has joined #bitcoin-core-pr-reviews     
02:24 -!- mol [~mol@unaffiliated/molly] has quit [Ping timeout: 246 seconds]     
02:26 -!- jadi [~jadi@213.207.193.58] has quit [Read error: Connection reset by peer]     
02:27 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Disconnected by services]     
02:27 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-pr-reviews     
02:27 -!- vasild_ is now known as vasild     
02:38 -!- belcher_ is now known as belcher     
02:48 -!- jadi [~jadi@213.207.193.58] has joined #bitcoin-core-pr-reviews     
03:05 < michaelfolkson> For those who like videos, this a good summary on Erlay from gleb https://www.youtube.com/watch?v=1kpAiLO9Dv8
03:15 < jonatack> I get more than a dozen occurrences of the same -Wredundant-move warning when building #18261 (erlay) on gcc 10.2.1, is it just me?
03:17 < jonatack> and functional tests failing with "inconsistent lock order for 'peer->m_recon_state_mutex' in net_processing.cpp:4936"
03:19 < michaelfolkson> Haven't built yet but CI checks are failing
03:20 < michaelfolkson> From yesterday's P2P meeting I think gleb wants Approach (N)ACKs rather than ready for merge code reviews
03:20 < michaelfolkson> But he or someone else can correct me if wrong
03:22 -!- Melisa14Kuphal [~Melisa14K@static.57.1.216.95.clients.your-server.de] has joined #bitcoin-core-pr-reviews     
03:23 < michaelfolkson> "Anyway, would be great if some people give a high-level code overview, say by next meeting in 2 weeks, so that we decide how to proceed?"
03:23 < michaelfolkson> High level code review
03:24 < jonatack> sure, but if it doesn't build cleanly and tests fails everywhere it's a non-starter. it may be something on my end, however, thus my question
03:27 -!- Melisa14Kuphal [~Melisa14K@static.57.1.216.95.clients.your-server.de] has quit [Ping timeout: 240 seconds]     
03:27 < michaelfolkson> And "Obviously looking for review, let's try to start with a high-level concerns, and keep nits for later."
03:27 < michaelfolkson> I'll build it locally and see if I get same as you
03:27 < aj> michaelfolkson: "doesn't build / merge with HEAD" is a bit more than a nit :)
03:27 < michaelfolkson> I guess :)
03:33 < jonatack> trying with clang now
03:34 -!- shesek [~shesek@164.90.217.137] has joined #bitcoin-core-pr-reviews     
03:34 -!- shesek [~shesek@164.90.217.137] has quit [Changing host]     
03:34 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-pr-reviews     
03:41 < michaelfolkson> "The Erlay logic in this PR is spread across many functions in net_processing.cpp and reconcilliation.h, which means that it's impossible to test that logic in isolation."
03:42 < jonatack> yes... agree with jnewbery about isolating the logic like was done for the tx relay overhaul
03:42 < michaelfolkson> It sounds like there are going to be some significant changes to the code (probably should leave that discussion for the PR review club though)
03:43 < michaelfolkson> Do you have a PR number to hand for "txrequest was separated from the rest of net_processing"
03:43 < michaelfolkson> ?     
03:44 < michaelfolkson> https://github.com/bitcoin/bitcoin/pull/19184 ?
03:45 < jonatack> https://github.com/bitcoin/bitcoin/pull/19988
03:45 < michaelfolkson> Ah thanks
03:52 < jonatack> no warnings building with clang 9, but the functional tests are all still failing as with gcc
03:53 < michaelfolkson> So this PR merges in Minisketch library to Core. I was thinking it was going to be a libsecp style external library
03:54 < michaelfolkson> Another comment from John that that should probably be the first PR of many
03:54 < michaelfolkson> https://github.com/bitcoin/bitcoin/pull/18261#pullrequestreview-570100861
03:56 < michaelfolkson> Minisketch is currently a sipa repo, not a bitcoin-core repo
04:00 -!- jonatack [~jon@37.166.50.22] has quit [Quit: jonatack]     
04:06 < michaelfolkson> There is an applications section on Minisketch, I missed that yesterday https://github.com/sipa/minisketch#applications
04:06 -!- jonatack [~jon@37.166.50.22] has joined #bitcoin-core-pr-reviews     
04:07 < michaelfolkson> And improvements that are still to do on Minisketch https://github.com/sipa/minisketch#implementation-notes
04:07 < michaelfolkson> Don't know if any of those are needed for Erlay
04:08 -!- jonatack [~jon@37.166.50.22] has quit [Read error: Connection reset by peer]     
04:08 -!- jonatack [~jon@37.166.50.22] has joined #bitcoin-core-pr-reviews     
04:08 < michaelfolkson> I think those improvements are generally optimizations and additional flexibility for other applications
04:08 -!- b10c_ [~b10c@fttx-pool-185.76.96.175.bambit.de] has joined #bitcoin-core-pr-reviews     
04:09 < michaelfolkson> I'm not certain though
04:09 -!- b10c_ [~b10c@fttx-pool-185.76.96.175.bambit.de] has quit [Client Quit]     
04:10 < michaelfolkson> But if improvements are going to be worked on for other applications it surely makes sense to keep Minisketch as an external bitcoin-core library rather than merging it into Core repo
04:12 < jonatack> updated, rebooted, cleared build cache, make distclean, trying again
04:12 < michaelfolkson> Why? Expecting different result?
04:15 < jonatack> yes, doing that fixed build weirdness for me in december that i couldn't reproduce afterward...but this time I am seeing the same result.
04:19 < michaelfolkson> It has been almost a year with minimal review. Should probably be marked as draft though
04:20 < michaelfolkson> Actually that's not quite fair, there has been review over the year
04:25 < michaelfolkson> "So, for now, please review up to a30e861"
04:27 < michaelfolkson> Oh I think all commits are ready for high level code review after that
04:28 < michaelfolkson> https://github.com/bitcoin/bitcoin/pull/18261#issuecomment-730536171
04:29 < jonatack> the unit tests also hang for me at "Entering directory '/home/jon/projects/bitcoin/bitcoin/src/minisketch'"
04:30 < michaelfolkson> There are no unit tests are there other than Minisketch unit tests?
04:32 < michaelfolkson> Oh right, I didn't read the end of your message
04:32 < michaelfolkson> That is a Minisketch unit test. So that is a issue with the Minisketch library
04:32 < jonatack> sure, no worries, i was trying to run them all. anyway, enough time lost here, moving on
04:33 < jonatack> curious if it's just me having these issues, no one else seems to be
04:33 < michaelfolkson> I don't think anyone is building it. As I said it isn't ready for merge and probably should be in draft
04:34 < michaelfolkson> Though will happily be corrected if wrong
04:35 < michaelfolkson> It is a big, ambitious project. Personally I think it is fine to have a PR review club on something that isn't ready for merge
04:35 < michaelfolkson> (Can't speak for anyone else though obviously)
04:36 < michaelfolkson> At Approach (N)ACK stage. We've done PRs before that were at Concept (N)ACK stage
04:37 < michaelfolkson> That Minisketch unit test is an interesting one to follow up on though. Minisketch library seems more mature than this PR
04:38 < jonatack> If the issues I'm seeing aren't on my side, I don't think the PR is ready for review other than Concept ACK
04:38 < michaelfolkson> Maybe halfway between Concept and Approach ;)
04:45 -!- jonatack [~jon@37.166.50.22] has quit [Ping timeout: 272 seconds]     
04:46 -!- jonatack [~jon@37.166.50.22] has joined #bitcoin-core-pr-reviews     
04:50 < michaelfolkson> I definitely think the focus should be that minisketch library first though. For example this open PR would allow for some minisketch based functional tests in Core https://github.com/sipa/minisketch/pull/26
04:52 < michaelfolkson> To understand what that library is doing takes a lot of prep though so you can play around with it (I think)
04:53 < michaelfolkson> Maybe not... these is a short usage section https://github.com/sipa/minisketch#usage
04:55 -!- shafiunmiraz0 [~shafiunmi@103.220.205.190] has joined #bitcoin-core-pr-reviews     
04:56 < michaelfolkson> I don't think the Erlay PR will be ready for merge until Minisketch is ready to be used in production
04:57 < michaelfolkson> And the Minisketch build system is "very rudimentary for now"
04:58 < michaelfolkson> Maybe the PR review club should have been on that library instead ;)
05:01 < michaelfolkson> I guess it is too math-y
05:05 -!- shafiunmiraz0 [~shafiunmi@103.220.205.190] has left #bitcoin-core-pr-reviews ["Leaving"]     
05:06 -!- shafiunmiraz0 [~shafiunmi@103.220.205.190] has joined #bitcoin-core-pr-reviews     
05:11 -!- jonatack [~jon@37.166.50.22] has quit [Ping timeout: 272 seconds]     
05:18 -!- shafiunmiraz0 [~shafiunmi@103.220.205.190] has quit [Quit: Leaving]     
05:19 -!- shafiunmiraz0 [~shafiunmi@103.220.205.190] has joined #bitcoin-core-pr-reviews     
06:02 -!- rich [~rich@shindig.notmandatory.org] has left #bitcoin-core-pr-reviews ["Leaving..."]     
06:02 -!- rich [~rich@shindig.notmandatory.org] has joined #bitcoin-core-pr-reviews     
06:02 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:8dc5:1d88:1b78:e5ab:dd70] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]     
06:14 -!- jonatack [~jon@37.166.50.22] has joined #bitcoin-core-pr-reviews     
06:22 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:8dc5:1d88:1b78:e5ab:dd70] has joined #bitcoin-core-pr-reviews     
06:59 < sipa> jonatack, michaelfolkson: the minisketch tests just run forever
07:05 < jonatack> ah! (i suppose they shouldn't be in make check?)
07:07 < pinheadmz> I got compilation errors for the branch today, Undefined symbols for architecture x86_64: "ChaCha20::ChaCha20..."
07:07 < pinheadmz> am i missing a git submodule perhaps?
07:12 < jonatack> pinheadmz: idk, i had issues too (see above and my comments in the PR)
07:13 < pinheadmz> guess i dont need to run the tests for review club were only looking at a few commits
07:13 < michaelfolkson> Not this time, but usually makes sense to
07:14 < pinheadmz> i like to tweak the tests and log extra output to get a better understanding
07:17 < jonatack> yes, it would be nice to warn reviewers to not spend time trying to run the unit or functional tests and/or put the pull in Draft mode
07:22 -!- jadi [~jadi@213.207.193.58] has quit [Remote host closed the connection]     
07:23 < gleb> pinheadmz: did you do make clean?
07:24 < pinheadmz> gleb ill try now
07:24 < gleb> jonatack: on my macos machines it compiles just fine, so I'm not sure what to warn about. It also did work on linux, although that was a couple months ago.
07:25 -!- Oli54 [32f414b1@50-244-20-177-static.hfc.comcastbusiness.net] has joined #bitcoin-core-pr-reviews     
07:26 -!- Oli54 [32f414b1@50-244-20-177-static.hfc.comcastbusiness.net] has left #bitcoin-core-pr-reviews []     
07:27 < pinheadmz> gleb compiled! thanks
07:32 < gleb> pinheadmz: so im not sure how the general tests work here (might be a couple side-effects not reproduced on my setting), but functional/p2p_erlay.py should pass
07:33 < gleb> jonatack: "yes... agree with jnewbery about isolating the logic like was done for the tx relay overhaul". Could you show an example of a code block (lines in net_processing), which would benefit from moving to a separate module?
07:40 < jonatack> gleb: do the functional tests run for you? do the minisketch unit tests run for you in a bounded way?
07:40 < michaelfolkson> gleb: I think that is referring to the changes made for that tx relay overhaul ie before it was merged
07:41 < michaelfolkson> Not current code in net_processing
07:41 < michaelfolkson> Changes in this PR https://github.com/bitcoin/bitcoin/pull/19988
07:44 < gleb> jonatack: functional yes (although double-checking against the latest commit now), minisketch unit tests run forever I think, yeah.
07:45 < gleb> michaelfolkson: I understnd the overhaul part, I don't understand which Erlay code from net_processing should be moved.
07:45 < jonatack> gleb: yes, make check shouldn't run them then, i guess. i'll wait until others chime in on the functional tests before spending more time on it (too many PRs waiting for review)
07:46 < sipa> gleb: that is of course the hardest part :)
07:46 < gleb> jonatack: which functional test fails specifically? That would help a lot
07:46 < jonatack> gleb: most of them
07:47 < sipa> finding a good abstraction isn't just finding a set of things to move; doing that naively will just mean two very tightly interacting modules with cyclic dependencies between them
07:47 < jonatack> that is why i thought it was me. i tried gcc 10, clang 9, distclean, rebooting, rinse, repeat... no dice!
07:48 < jonatack> i'm building/running/testing other PRs fine today, so idk
07:49 < gleb> sipa: sure, I'm just trying to reconcile my understanding with john's suggestion. No luck yet. I know there's a bunch of specific code, but I think it's kind of natural and I'm not sure how to reorg.
07:49 < gleb> So yeah, trying to get a bit more specific advice.
07:50 < jonatack> gleb: maybe a pair programming session with someone like john or aj could be productive
07:50 < jonatack> (feel free to ignore, just an idea)
07:50 < gleb> jonatack: I'm sorry to hear you spent a lot of time. The only thing I can suggest is to hit me up earlier next time. I have no control of stuff failing outside of my setting, if I can't even reproduce it.
07:50 < ariard> I think it should be noted that tx relay overhaul was about encapsulating a global state, here with erlay do you have per-peer state, so it's not as straightforward
07:50 < gleb> Just replayed all p2p functional tests, they all pass.
07:51 < jonatack> gleb: yes, usually the CI gives more feedback but it this case it was pretty sparse
07:52 < sipa> ariard: i don't think that's necessarily a problem; you can easily add another global that has per-peer data (txrelay did that too, partially, btw)
07:52 < sipa> gleb: i think it's complicatec by the fact that there are two tx announcement strategies (flood & erlay), and you're not replacing the existing one
07:53 < ariard> sipa: obviously, but it's really likely you're coming up with a memory overhead, at least for the index
07:53 < sipa> gleb: so i think (if we decide this encapsulation is important enough), it may be best to first look for a way to encapsulate the existing announcement logic, without behaviour changes - and then add erlay to it
07:54 < sipa> ariard: a few bytes per node is absolutely negligible
07:56 < gleb> sipa: I see, that could be a way to go I guess. My approach was more to mimic what we have. Supposedly that makes review easier, if you just add stuff along here and there. That was my approach, it seems it's not that good.
07:56 < sipa> gleb: it works when making small changes, and it's often the path of least resistance
07:57 < sipa> but when trying to see how a big change interacts with the big picture it may not be ideal
07:58 < ariard> a good point would be to better specify the tx-announcement protocol selection, it doesn't seem to match the Erlay section of the bip
07:58 -!- jadi [~jadi@213.207.193.58] has joined #bitcoin-core-pr-reviews     
07:58 < ariard> and being spread across the change set in multiple comments
07:58 < gleb> ariard: pretty sure it should match the BIP at least, what kind of mismatch you're refererring to?
07:59 < ariard> gleb: at least the comment pointed by lightlike
08:00 < ariard> and the overlapping between flooding/reconciliation for strategic peers isn't that much clear
08:00 < ariard> (I do have pending comments, still reviewing)
08:02 -!- jadi [~jadi@213.207.193.58] has quit [Ping timeout: 240 seconds]     
08:04 < gleb> Okay, let's keep the rest of discussion for the meeting. Unless you can't build or something, in which case ask me.
08:05 < gleb> (i mean for the review club session)
08:07 < ariard> yeah sure :)
08:22 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:8dc5:1d88:1b78:e5ab:dd70] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]     
08:22 -!- DylanM [4461061f@ip68-97-6-31.ok.ok.cox.net] has joined #bitcoin-core-pr-reviews     
08:22 -!- lightlike [~lightlike@p200300c7ef180300394787c0e152616a.dip0.t-ipconnect.de] has joined #bitcoin-core-pr-reviews     
08:31 -!- DylanM [4461061f@ip68-97-6-31.ok.ok.cox.net] has quit [Ping timeout: 240 seconds]     
08:38 -!- kiltzman [~k1ltzman@195.189.99.96] has quit [Ping timeout: 256 seconds]     
08:41 -!- ecola [~3cola@95.175.17.147] has joined #bitcoin-core-pr-reviews     
08:41 -!- kiltzman [~k1ltzman@195.189.99.96] has joined #bitcoin-core-pr-reviews     
08:44 -!- DylanM [4461061f@ip68-97-6-31.ok.ok.cox.net] has joined #bitcoin-core-pr-reviews     
08:45 -!- tuxcanfly [~tuxcanfly@68.183.231.167] has quit [Ping timeout: 240 seconds]     
08:46 -!- tuxcanfly [~tuxcanfly@68.183.231.167] has joined #bitcoin-core-pr-reviews     
08:47 -!- ma33 [92732bd1@146.115.43.209] has joined #bitcoin-core-pr-reviews     
08:48 < jnewbery> gleb: I think the txrequest change in 19988 is quite instructive in this case. Prior to that PR, there was state that was available across net_processing (m_tx_download) and was updated/read in multiple places. It was therefore impossible to test in isolation, very difficult to reason about, and difficult to make changes without introducing bugs because of the subtle ways that different
08:49 < jnewbery> parts of the system interacted.
08:49 < jnewbery> after that PR, all of the tx request state and logic is isolated with a very narrow interface between net_processing and txrequest. Consequently, the txrequest logic can be tested with 100% coverage
08:52 -!- joelklabo [~textual@108-196-216-127.lightspeed.sntcca.sbcglobal.net] has joined #bitcoin-core-pr-reviews     
08:54 -!- effexzi [uid474242@gateway/web/irccloud.com/x-foessotidmamstta] has joined #bitcoin-core-pr-reviews     
08:55 -!- tuition [~tuition@195.181.160.175.adsl.inet-telecom.org] has joined #bitcoin-core-pr-reviews     
08:56 -!- NelsonGaldeman56 [5e0f2480@94.15.36.128] has joined #bitcoin-core-pr-reviews     
08:59 -!- maqusat [522fec0b@cpc73666-dals20-2-0-cust10.20-2.cable.virginm.net] has joined #bitcoin-core-pr-reviews     
08:59 -!- maqusat [522fec0b@cpc73666-dals20-2-0-cust10.20-2.cable.virginm.net] has quit [Client Quit]     
08:59 -!- julianmrodri [ba8bec2d@186.139.236.45] has joined #bitcoin-core-pr-reviews     
08:59 -!- AnthonyRonning [627f50cb@098-127-080-203.res.spectrum.com] has joined #bitcoin-core-pr-reviews     
08:59 -!- maqusat [522fec0b@cpc73666-dals20-2-0-cust10.20-2.cable.virginm.net] has joined #bitcoin-core-pr-reviews     
08:59 -!- OliP [32f414b1@50-244-20-177-static.hfc.comcastbusiness.net] has joined #bitcoin-core-pr-reviews     
09:00 < jnewbery> #startmeeting
09:00 < jnewbery> hi folks! Welcome to PR Review Club. Feel free to say hi to let everyone know you're here.
09:00 < glozow> supppp
09:00 < pinheadmz> hi !!
09:00 < dhruvm> hi     
09:00 < jnewbery> (or not. lurkers are also welcome at this club)
09:00 < dergoegge> hi!     
09:00 < joelklabo> Hey everyone 👋
09:00 < sipa> hi     
09:00 < OliP> Hi     
09:00 < lightlike> hi!     
09:00 < emzy> hi     
09:00 < maqusat> hey     
09:00 < michaelfolkson> hi     
09:00 < jnewbery> Today, we're going to talk about Erlay. Notes and questions are here: https://bitcoincore.reviews/18261
09:00 < ma33> hi     
09:00 < AnthonyRonning> hi     
09:00 < NelsonGaldeman56> Hi!     
09:00 < ajonas> Hi     
09:00 < gleb> hi! Thank you for coming.
09:00 < jnewbery> Thank you to lightlike for hosting and to gleb for writing the PR!
09:00 < ariard> hello
09:01 < jnewbery> is anyone here for the first time?
09:01 < effexzi> Hey...
09:01 < ecola> hi     
09:01 < felixweis> hi     
09:01 < rich> hi     
09:01 < dergoegge> first time not lurking
09:01 < willcl_ark> hi     
09:01 < OliP> First time here!
09:01 < jonatack> hi (lurking while reviewing 18017 ;)
09:01 < schmidty> hi     
09:01 < DylanM> Hi first time, lurking mostly
09:01 < gleb> A lot of new names since I attended last time, great :)
09:01 < jnewbery> dergoegge: lovely. Thanks for joining us!
09:01 < amiti> hi     
09:01 < jnewbery> OliP, DylanM: welcome, welcome
09:01 -!- fodediop [~fode@41.214.86.100] has joined #bitcoin-core-pr-reviews     
09:01 < fodediop> hi     
09:01 < OliP> jnewbery: Many thanks
09:02 < jnewbery> couple of reminders. We're all here to learn. Lightlike will guide the discussion but feel free to ask questions at any time
09:02 -!- svav [5245568f@82-69-86-143.dsl.in-addr.zen.co.uk] has joined #bitcoin-core-pr-reviews     
09:02 -!- larryruane_ [uid473749@gateway/web/irccloud.com/x-mvpbcqmcknxawhss] has joined #bitcoin-core-pr-reviews     
09:02 < jnewbery> Also, you don't have to ask if you can ask a question. Just ask!
09:02 < tuition> hi     
09:02 < svav> hi     
09:02 < larryruane_> hi     
09:02 < willcl_ark> great turnout!
09:02 < jnewbery> And one more reminder before I hand over: I'm always looking for hosts for review clubs. If you think that's something you'd like to have a go at, please message me
09:02 < jnewbery> ok, enough of me. Over to lightlike
09:03 -!- sergei-t [~sergei@static-94-103-211-82.internet.lu] has joined #bitcoin-core-pr-reviews     
09:03 < lightlike> Ok, thanks jnewbery - so today we'll talk about Erlay!
09:03 < lightlike> First, since this a really large PR and there is also a BIP and a paper with lots of information, we won't get deep into too much of the code today.
09:03 < lightlike> (that would be a possibility for a follow-up)
09:03 < lightlike> But let's start with first things first - Who had the time to review the PR this week? (y/n)
09:04 < willcl_ark> y     
09:04 < OliP> y     
09:04 < pinheadmz> y     
09:04 < ecola> n     
09:04 -!- tuition_ [~tuition@195.181.160.175.adsl.inet-telecom.org] has joined #bitcoin-core-pr-reviews     
09:04 < dergoegge> ~y     
09:04 < rich> n     
09:04 < sergei-t> n     
09:04 < svav> y     
09:04 < AnthonyRonning> n     
09:04 < DylanM> n     
09:04 < jnewbery> y (just the four commits)
09:04 < fodediop> n     
09:04 < ariard> y     
09:04 < emzy> y (without the code)
09:04 < michaelfolkson> y     
09:04 < dhruvm> n(ot yet)
09:04 < glozow> looked at bip, 0.2y for the PR
09:04 < joelklabo> n (read about erlay)
09:05 < lightlike> Great! So what is your initial impression? (Concept ACK or NACK)
09:05 < emzy> Concept ACK
09:05 < rich> Concept ACK
09:05 < dergoegge> Concept ACK
09:05 < pinheadmz> concept ACK
09:05 < joelklabo> concept ACK
09:05 < lightlike> seems still a bit early for code ACKs :-)
09:05 < OliP> Concept ACK
09:05 < larryruane_> n     
09:05 < fodediop> Concept ACK
09:05 < dhruvm> Concept ACK - the gains seem incredible!
09:05 < AnthonyRonning> concept ack
09:05 < sergei-t> concept ack (based on prior knowledge about erlay)
09:06 < svav> Concept ack
09:06 < lightlike> Ok - that's a lot of concept ACKS!
09:06 -!- tuition [~tuition@195.181.160.175.adsl.inet-telecom.org] has quit [Ping timeout: 240 seconds]     
09:06 < lightlike> So, let's dive into the questions:
09:07 -!- UdiP [4fb3dac7@bzq-79-179-218-199.red.bezeqint.net] has joined #bitcoin-core-pr-reviews     
09:07 < lightlike> What advantages does a reconciliation-based approach to transaction relay have over the current flooding method. What are the trade-offs?
09:07 < emzy> Less bandwidth usage at the cost of slower propagation of TX through the network.
09:07 < dhruvm> a reconciliation-based approach minimizes redundant INV messages sent to nodes which already have the transaction. 224/524 ~ 42.7% of all bytes exchanged for tx relay are redundant if flooding is used in contrast.
09:07 < ecola> savings in bandwith
09:07 < pinheadmz> less bandwidth required
09:07 < AnthonyRonning> Significantly reduces bandwidth and allows for better connectivity. It increases the time it takes to propagate a tx to all nodes.
09:07 < dergoegge> less bandwidth usage, increased connectivity and privacy benefits vs. slightly slower tx propagation.
09:07 -!- ccdle12 [955adef3@149.90.222.243] has joined #bitcoin-core-pr-reviews     
09:07 < pinheadmz> obfuscation of the network graph perhaps ?
09:07 < tuition_> lower transaction bandwidth use at the cost of marginal increases in txn propagation delays
09:08 < fodediop> It allows for susbtantial bandwidth savings on trasaction propagation
09:08 < sergei-t> Con: more complex protocol logic, the notion of "well-connected publicly reachable" nodes (centralization risks?)
09:08 < dhruvm> also the growth in bandwidth is sub-linear to number of peer connections per node.
09:08 < pinheadmz> trade offs might be the extra round trips if txs are missing
09:08 < willcl_ark> It will allow nodes to make more connections due to lowering bandwidth usage, and therefore make eclipse attacks more costly/difficult
09:08 < joelklabo> less bandwidth, more code complexity
09:09 < sergei-t> Privacy implications: do peers now know more about which txs I have and don't have?
09:09 < sipa> pinheadmz: that's already accounted for in the model
09:09 < tuition_> willcl_ark nice point about decreasing bandwidth constraints allowing more peering to mitigate eclipse attacks
09:09 < dhruvm> sergei-t: peers already knew that with flooding i think
09:09 < ajonas> +1 willcl_ark and that plays nicely with dhruvm's comment that things only gets better with more conns
09:10 < pinheadmz> tradeoff then might be computational resources
09:10 < gleb> sergei-t: Erlay attempts to leak no more than already leaks through flooding. In practice, reviewers should confirm the defences indeed work.
09:10 < pinheadmz> extra math per-peer
09:10 < lightlike> lots of great answers here! I think especially the scaling for more connections are important.
09:10 < sipa> tuition_: yeah, that's the big one; the savings are modest for current average connection counts, but with increased connections the benefits grow
09:11 < rich> seems to be some implication that tx origination could be obscured since it's not pushed out preemptively via inv (amiti?)
09:12 < lightlike> Not sure about this, but at the current connectivity (8 OB), probably a similar bandwidht saving could be gained by only using short TX Ids with Salting (as is part of Erlay) but no reconciliation
09:12 < lightlike> sipa, gleb: would you agree to this?
09:12 < amiti> rich: good question, I don't know the answer :)
09:12 -!- Dulce [bd97ad07@189.151.173.7] has joined #bitcoin-core-pr-reviews     
09:13 < sipa> lightlike: yeah, with salting
09:13 -!- Dulce is now known as Guest73056     
09:13 < gleb> lightlike sipa: but then it's hard to deduplicate.
09:13 < dhruvm> rich: origination seems to be done with reconciliation and not flooding, so originator gets privacy iiuc
09:13 < gleb> I would say just short ids are not so easy to get done right as well
09:13 -!- Guest73056 [bd97ad07@189.151.173.7] has quit [Client Quit]     
09:14 < sipa> gleb: right
09:14 -!- Dulce_Vird [bd97ad07@189.151.173.7] has joined #bitcoin-core-pr-reviews     
09:14 < lightlike> ok, moving on: Which of the existing message types participating in transaction relay will be sent less often with Erlay? Which ones will be unaffected?
09:14 -!- Dulce_Vird [bd97ad07@189.151.173.7] has quit [Client Quit]     
09:14 < pinheadmz> a lot less INV
09:14 < dhruvm> INV will be sent less often. TX messages will be unaffected.
09:14 < emzy> INVs will be send less often. The missing TX itself still needs to be send.
09:14 -!- Dulce_Vird [bd97ad07@189.151.173.7] has joined #bitcoin-core-pr-reviews     
09:14 < sipa> dhruvm: both recon-based announcement and inv announcement leak which txids the sender has; the point is that the recon-based ones are less frequent (on a per peer basis)
09:15 < pinheadmz> and a little less GETDATA as well i presume?
09:15 < dergoegge> unaffected: tx, getdata less: inv
09:15 < dhruvm> sipa: ah, that makes sense.
09:15 -!- Dulce_Vird [bd97ad07@189.151.173.7] has quit [Client Quit]     
09:15 < lightlike> pinheadmz: why would there be less GETDATA?
09:15 < gleb> pinheadmz: why do you think less getdata? I might be forgetting somethign at this point :)
09:15 < dhruvm> pinheadmz: there should be less GETDATA if the missing txes are known at set-difference time?
09:16 < pinheadmz> could be wrong but curent flood goes INV-> then GETDATA<- then TX-> right?
09:16 < gleb> number of getdata = number of txs, logically, right?
09:16 < felixweis> i wonder if sending a sketch of transactions that are dependent and required together to meet minrelayfee requirements of a nodes mempool can help with package relay?
09:17 < willcl_ark> sipa: Does it not make some timing analysis attacks to detect origination more difficult or, less accurate?
09:17 < gleb> felixweis: Hard to talk about optimizations while we have no basic design for package relay...
09:17 < sipa> pinheadmz: every node should GETDATA every transaction exactly once
09:17 < sipa> pinheadmz: both before and after erlay
09:17 < dhruvm> pinheadmz: I think that's right and am wondering why we'd need as many GETDATA as well. At set difference, the other node could just send the TX messages.
09:17 < lightlike> I think it's important thet the current flow with INV->GETDATA->TX will be unchanged - it's just that the initial INV is sent only for those transaction we are reasonably sure our peer actually needs.
09:17 < glozow> right now I think you'd only request 1 getdata at a time, unless you got it by txid and there's different wtxids you don't know about
09:17 < pinheadmz> sipa a ha right
09:18 < pinheadmz> but do we still use GETDATA to reconcile if a tx is missing?
09:18 < pinheadmz> i thought bip330 introduced other messgaes
09:18 < glozow> yeah, gettx
09:18 < gleb> pinheadmz: you're probably referring to an older version of the bip
09:18 < glozow> oh, i guess if you're sending gettxes then you're sending fewer getdatas
09:19 < pinheadmz> gleb https://github.com/naumenkogs/bips/blob/bip_0330_updates/bip-0330.mediawiki ?
09:19 < gleb> pinheadmz: yeah, no gettx there, right?
09:19 < gleb> The latest version is linked in the PR.
09:19 < rich> willcl_ark: that is also how I understood "the point is that the recon-based ones are less frequent (on a per peer basis)"
09:19 < pinheadmz> oh I see `reconcildiff` triggers the usual inv->getdata->tx
09:20 < gleb> pinheadmz: exactly!
09:20 < lightlike> pinheadmz: yes!
09:20 < pinheadmz> i guess i thought it was more like getblocktxn from compact blocks
09:20 < ariard> felixweis: thought about it either adding another snapshot for package ids only or eating the bullet of wasted bandwidth in case of redundancy
09:20 < gleb> dhruvm: in short, sketches operate over short ids, but we can't request by short ids for reasons.
09:20 < felixweis> whats the rationale for the last point in Short ID computation? 1 + s mod 0xFFFFFFFF. why not use s already?
09:21 < sipa> felixweis: the pinsketch algorithm needs item IDs that are nonzero
09:21 < lightlike> next question:
09:21 < dhruvm> gleb: i see. are the reasons documented somewhere?
09:21 < felixweis> ah! makes sense
09:21 < ma33> Is their an analysis of when Erlay becomes too computationally expensive? I remember seeing a comparison with another set-recon scheme in the paper with up to 50 differences. What if the number of differences increases to, say, 500?
09:22 < lightlike> Can you explain why a reconciliation-based approach scales better than flooding (after all, the reconciliation messages also consume regular traffic)?
09:22 < felixweis> re-reading the next sentence would have helped already lol
09:23 < pinheadmz> lots of redundancy sending 255 byte transactions to all nodes all the time
09:23 < rich> It doesn't if you only have one peer, but with more peers sending the same tx inv, there is deduplication savings.
09:23 < lightlike> ma33: I think in the paper there is a suggestion to make larger set differences computationally viable via bisection - which is not needed for bitcoin though becuse typical differences are smaller.
09:23 < ariard> dhruvm: 32-bit short ids means you can have tx-relay jamming based on malicious collisions
09:23 < sipa> pinheadmz: again, the transactions are only sent once already
09:23 < willcl_ark> instead of receiving every tx from every peer, you just receive it from one
09:23 < gleb> dhruvm: that's one of the design choices we made while working on the implementation, I think you get it naturally when you review the protocol closely. Not sure all of them should be documented. But we can discuss this as well
09:23 < sipa> willcl_ark: transactions are only sent once already
09:24 < pinheadmz> sipa we dont re-broadcast entire TX to every peer besides the one who sent it to us ?
09:24 < pinheadmz> (currently, before PR)
09:24 < dhruvm> ariard: gleb: thanks.
09:24 < gleb> ma33: yeah, check graphs in the minisketch repo. It has nice data on the time it takes to compute
09:24 < sipa> pinheadmz: we _announce_ it to every other peer, by txid; not the full tx, obviously
09:24 < pinheadmz> ahaahahaha yes
09:24 < sipa> the peer requests the actual transaction from one node that announced it
09:24 < willcl_ark> ah     
09:24 < pinheadmz> so what were saving is 32 byte txid > 32 bit short txid
09:25 < sipa> nope, far more than that
09:25 < sipa> because erlay doesn't send short ids; it sends sketches
09:25 < pinheadmz> oh right
09:25 < sipa> and the size of sketches scales with the bandwidth of actual differences
09:25 < rich> I wonder what the crossover point for number of peers where reconciliation starts to pay off?
09:25  * pinheadmz 💡 
09:26 < rich> I suppose it also has to due with how interconnected your peers are too.
09:26 < lightlike> yes, that is the core of it. If we just sent short txids over each link as part of the recon, the scaling wouldn't be better than now.
09:26 < sipa> rich: yeah, and on the exact parametrization; i believe gleb experimented with various configurations
09:26 < gleb> yeah, the data is in the paper.
09:26 < maqusat> only diff of tx ids sent instead of the whole set
09:27 < gleb> rich: I think 9 peers actually. Erlay would cost 8x, short ids would pay 9x, current flooding is 32x (hence full txid)
09:28 < gleb> or not.... sorry, disregard for now.
09:28 < rich> gleb: do we have some way to model how interconnected typical peers are?
09:28 < lightlike> sipa, gleb: how would Erlay deal with it if someone dumped thousands of transactions at the same time into the network (e.g. for utxo consolidation)? would sketches be able to deal with that?
09:29 -!- UdiP [4fb3dac7@bzq-79-179-218-199.red.bezeqint.net] has quit [Quit: Connection closed]     
09:29 < ma33> rich TxProbe paper had some stats on the Bitcoin graph, albeit only on testnet
09:29 < gleb> rich: my simulator generated a topology similar to bitcoin today: 10k reachable nodes + 50k non-reachable nodes. And then random distribution. That's it
09:29 < rich> gleb: +1 seems reasonable
09:30 < gleb> lightlike: i haven't tried this scenario. Worst case we waste a limited amount of bandwidth and fallback to flooding.
09:30 < dhruvm> lightlike: while that might make the sketch diffs larger it should still be atleast as good as flooding right (and then some due to short txids)?
09:30 < sipa> lightlike: if you dump 1000s of transactions on any given 0.21 node, the ones above 5000 will just be ignored; the rest would be requested with some delay... then that peer would start propagating them, but at a limited rate
09:31 < ariard> rich: https://github.com/bitcoin/bitcoin/pull/15759#issuecomment-480868516 see discussions about some interconnection models
09:31 < sipa> erlay doesn't really come into play much, i think
09:31 < rich> ariard: thx!
09:31 -!- brunog [bbb72beb@187.183.43.235] has joined #bitcoin-core-pr-reviews     
09:31 < lightlike> ok, thanks!
09:31 < dergoegge> why is the reconil. frequency fixed at 1 every 2 seconds? could it make sense to have the frequency change based on how many new txs the node has seen in the last x hours/minutes? (i.e. reconcil more frequently if there are more txs coming through)
09:32 < gleb> dergoegge: that's possibly, I kind of assumed a "normal" operation of the network in the window of 3..14 txs per second appearing randomly in the network.
09:33 < lightlike> moving on with the q (getting into the actual commits):
09:33 < lightlike> How do two peers reach agreement about whether to use reconciliation?
09:33 < dhruvm> When a node receives a WTXIDRELAY message from a peer that supports tx-relay, it announces a SENDRECON message and requests to act as a reconciliation-requestor for outbound connections and a reconciliation-responder for inbound connections.
09:33 < sipa> gleb: just because the initiating node hasn't received many transactions doesn't mean there aren't many; it may just not have heard about them>
09:34 < glozow> do i have this correct? :
09:34 < glozow> both send `SENDRECON`, both must not have txrelay off, both must use wtxid relay, peer who initiates outbound must have requestor=on/responder=off, peer who receives inbound connection must have requestor=off/responder=on
09:34 -!- julianmrodri [ba8bec2d@186.139.236.45] has quit [Quit: Connection closed]     
09:34 < sipa> dergoegge: more generally, i think it's a bit scary to make the frequency of propagation depend on seen transactions (e.g. i could imagine some scary fix point where all nodes end up deciding there are no transactions, and stop reconciling entirely...)
09:35 < lightlike> dhruvm, glozow: exactly! both nodes send a SENDRECON message.
09:35 < ariard> glozow: I think they should be mauve peers to be correct
09:35 < glozow> ariard: mauve?
09:35 < lightlike> so everything is happening during VERSION negotiation and fixed for the lifetime of the connection - no later changes.
09:35 < pinheadmz> as part of the version handshake, after WTXID is set
09:36 < ajonas> Well we would fall back to flooding on a conn that supports reconciliation if we don’t have enough outbound flood conns, right?
09:36 < jnewbery> Is there a reason that both parties add salt?
09:37 < glozow> so that nobody can get nodes to use duplicate salts?
09:37 < pinheadmz> jnewbery so attackers cant try to find coliding short ids
09:37 < pinheadmz> or if they do itll only affect one pair of nodes ;-)
09:37 < dergoegge> gleb, sipa: i see. although there could be a limit say "reconciliate at least every ~n seconds" to avoid stopping entirely
09:37 -!- andozw [617e21b5@97-126-33-181.tukw.qwest.net] has joined #bitcoin-core-pr-reviews     
09:38 < sipa> dergoegge: yeah, not saying that doesn't make sense, just that it does bring in some possibly scary considerations
09:38 < gleb> two salts may be redundant actually, what if the salt was always picked by the "connection initiator" side randomly per peer, and they use that?
09:39 < gleb> what do you think guys? sipa?
09:39 < sipa> gleb: i think that's probably ok, but i find it easier to reason about if both contribute
09:39 < gleb> i see.
09:39 < dergoegge> sipa thats true, thanks for the answer
09:39 < ma33> If I understand it correctly, a node can request a sketch from only a maximum of 8 of its (outbound) peers, correct? Or do peers in blocks-only-mode also participate in recon?
09:39 < gleb> yeah, it's indeed "safer" when both contribute
09:39 < jnewbery> I don't understand the attack. I could make more than one of my peers have the same salt?
09:40 < jnewbery> but I can't force the same salt transitively to other connections
09:40 < sipa> jnewbery: the biggest reason for the salt is that not all connections use the same salt; if there were, an attacker could easily grind transactions that propagate very badly over all connections
09:40 < jnewbery> certainly I understand the reason for a salt per connection, just not the need for both the requestor and responder to contribute
09:41 < sipa> so if both contribute entropy, you can just say "assume not both parties are the grinding attacker", which is obviously true - the attacker doesn't need to relay to themselves
09:41 < gleb> ma33: not blocks-only, no, but all tx-relaying peers can be requested for a sketch (if they indicated support). Yes, this is *currently* limited to 8.
09:41 < dhruvm> ma33: recon seems like a tx-relay thing, not sure blocks-only nodes would get the network characteristics they desire with recon. but erlay perhaps reduces the need to be blocks-only?
09:41 < lightlike> ma33: It would request sketches from all of its peers, its just that the current limit for outbounds is 8.
09:41 < sipa> blocks-only connections definitely should not do erlay; it reveals information about their mempool
09:42 < lightlike> ajonas: your question wrt fallback to flooding kind of touches my next question
09:42 < jnewbery> the party that sends the SENDRECON message second can grind the salt value
09:42 < lightlike> If two peers have agreed to use reconciliation, does that mean there will be no flooding on this connection?
09:42 < rich> if I understand correctly, blocks-only would still be lower bandwidth than erlay, but with the limitation of no mempool
09:42 < dhruvm> I am not certain but it seems flooding is between listening nodes. Reconciliation is between all pairs of peers?
09:43 < rich> (and blocks come too infrequently to worry about inv message deduplication for them)
09:43 < sipa> jnewbery: yes, but they can't grind it so that the same combined salt is used on all their connections
09:44 -!- NelsonGaldeman56 [5e0f2480@94.15.36.128] has quit [Quit: Ping timeout (120 seconds)]     
09:44 < sdaftuar> sipa: does it matter if a peer chooses to make relaying with themselves not work?
09:44 -!- AnthonyRonning [627f50cb@098-127-080-203.res.spectrum.com] has quit [Quit: Ping timeout (120 seconds)]     
09:44 -!- brunog [bbb72beb@187.183.43.235] has quit [Quit: Ping timeout (120 seconds)]     
09:44 < sdaftuar> ie, they could just not relay transactions anyway
09:44 < gleb> dhruvm: what you're referring to is an *intuition* behind how erlay achieves it's properties. In practice, the relay policy is a bit more specific. This is because we have no notion of reachable node in the codebase (a node can't for sure know if it's reachable or not )
09:44 < dergoegge> There will be at least 8 flooding connections even if they also support reconciliation. If the reconciliation set is full we also fall back to flooding i think.
09:44 -!- ccdle12 [955adef3@149.90.222.243] has quit [Quit: Ping timeout (120 seconds)]     
09:44 -!- AnthonyRonning [627f50cb@098-127-080-203.res.spectrum.com] has joined #bitcoin-core-pr-reviews     
09:44 < pinheadmz> and what is the worst case scenario for a succesful shortid collision? just one peer doesnt get a tx message? theyll see it in a block at least
09:44 -!- maqusat [522fec0b@cpc73666-dals20-2-0-cust10.20-2.cable.virginm.net] has quit [Quit: Ping timeout (120 seconds)]     
09:44 -!- svav [5245568f@82-69-86-143.dsl.in-addr.zen.co.uk] has quit [Quit: Ping timeout (120 seconds)]     
09:44 < lightlike> rich: yes, blocks-only is strictly lower bandwidth - but obviously tx relay is necessary for the network, so you dont contribute that by being blocks-only.
09:44 < ariard> dhruvm: beyond the bandwidth saving, blocks-only are also (hopefully!) hidden peers, making it harder to partition a victim node
09:44 < sipa> sdaftuar: generally in cryptographic analysis you always assume that at least one of the involved parties is honest; if everyone is an attacker, who cares?
09:44 -!- svav [5245568f@82-69-86-143.dsl.in-addr.zen.co.uk] has joined #bitcoin-core-pr-reviews     
09:44 < sipa> i think the same applies here
09:44 -!- OliP [32f414b1@50-244-20-177-static.hfc.comcastbusiness.net] has quit [Quit: Ping timeout (120 seconds)]     
09:44 < ma33> @gleb
09:45 < ma33> gleb got it, thanks
09:45 < ma33> dhruvm nice follow up question. any ideas?
09:45 -!- ma33 [92732bd1@146.115.43.209] has quit [Quit: Ping timeout (120 seconds)]     
09:45 -!- DylanM [4461061f@ip68-97-6-31.ok.ok.cox.net] has quit [Quit: Ping timeout (120 seconds)]     
09:45 -!- OliP [32f414b1@50-244-20-177-static.hfc.comcastbusiness.net] has joined #bitcoin-core-pr-reviews     
09:45 -!- andozw [617e21b5@97-126-33-181.tukw.qwest.net] has quit [Quit: Ping timeout (120 seconds)]     
09:45 -!- maqusat [522fec0b@cpc73666-dals20-2-0-cust10.20-2.cable.virginm.net] has joined #bitcoin-core-pr-reviews     
09:45 < sdaftuar> sipa: not sure i follow -- the idea is that one of the peers can break relay on their own link.  that doesn't generalize to any other connections on the network that don't involve them
09:45 < dhruvm> gleb: ah, i should have read more than the paper :)
09:45 -!- ma [92732bd1@146.115.43.209] has joined #bitcoin-core-pr-reviews     
09:45 < sdaftuar> i'm just saying they could do that already by not relaying to begin with
09:46 < sipa> sdaftuar: ah yes
09:46 < dhruvm> ariard: not sure i follow. how are block-only peers hidden?
09:46 -!- NelsonGaldeman [5e0f2480@94.15.36.128] has joined #bitcoin-core-pr-reviews     
09:46 < sipa> dhruvm: connectivity graph is much harder to infer for blocks-only connections (because of no addr and no tx relay)
09:46 < sdaftuar> i don't know that it matters much either way, no downside either to both sides contributing entropy
09:47 < lightlike> dergoegge: yes, but those connections won't be flooding only. We would flood TO up to 8 peers if we are a reachable note, but still get txes FROM these peers via reconciliation
09:47 -!- brunog [bbb72beb@187.183.43.235] has joined #bitcoin-core-pr-reviews     
09:47 < jnewbery> sdaftuar: yes, this is what I was trying to ask earlier. The worst I could do would be have the same salt for all my peers, which isn't attacking anyone else
09:47 -!- brunog [bbb72beb@187.183.43.235] has quit [Client Quit]     
09:47 < sdaftuar> jnewbery: yeah that makes sense to me
09:47 < gleb> jnewbery: this is my understanding as well.
09:48 < sipa> right, that's a fair point
09:48 < dhruvm> sipa: i see. so two incentives to be blocks-only: protection from graph inference and bandwidth reduction.
09:48 < dergoegge> lightlike: ah yes, thanks for clarifying
09:48 < gleb> In some older design, we needed 2 salts, but not really anymore
09:48 < sipa> dhruvm: indeed
09:48 < sdaftuar> one thing that occurred to me is if having a bad salt incurred cpu cost on your counterparty
09:48 < sdaftuar> in which case that might be a motivation to just avoid the influence of an attacker in that regard, but no idea if that might be true
09:48 -!- Talkless [~Talkless@mail.dargis.net] has joined #bitcoin-core-pr-reviews     
09:49 < lightlike> next q:
09:49 < lightlike> In the Limit transaction flooding commit, MAX_OUTBOUND_FLOOD_TO is set to 8. Considering that 8 is also the maximum number of outbound connections participating in transaction relay, why do you think this value was chosen?
09:49 < lightlike> (already partly answered I think)
09:49 < ariard> dhruvm: if you're picked up as a outbound block-relay-only the protection for graph inference is also benefiting your peer
09:49 -!- DylanM [4461061f@ip68-97-6-31.ok.ok.cox.net] has joined #bitcoin-core-pr-reviews     
09:50 < ariard> it's a shared incentive
09:50 < sipa> sdaftuar: i believe siphash does assume that keys are random; if the exact key can be chosen exactly, there may be attacks beyond the ability to grind colliding transactions (e.g. i could imagine that some pathological keys exist that result in strictly more collisions that average keys); the actual key is computed with SHA256, but with too little entropy, that might be grindable too
09:50 < dhruvm> ma33: as sipa and ariard mentioned, the other incentive for block-only is security related.
09:50 < dhruvm> ariard: yeah that makes sense because the inference ability is on the link.
09:51 < gleb> I think this q is a hard one. MAX_OUTBOUND_FLOOD_TO=8 is based on my simultions: it provides low enough latency while not flooding too much (assuming we have more max conns)
09:52 < gleb> (low latency achieved in a combination of "diffusion" interval we wait before flooding out a tx, which was another parameter)
09:53 < lightlike> gleb: Ok - I thought part of the motivation was to not flood more than right now when, as a next step, the number ob MAX outbound connections is adjusted upwards.
09:53 < dhruvm> gleb: so it was the empirical sweet-spot between all-flooding(bandwidth-intensive) and all-reconciliation(latency-intensive)?
09:53 < gleb> lightlike: yeah, that's a way to think about it.
09:53 < rich> gleb: so can we think about it as the default outbound relay connections are added to the flood connections?
09:53 < gleb> dhruvm: yeah, how often are reconciliations vs how often we flood
09:54 < lightlike> Final question: Can you think of possible new attack vectors that would specifically apply to Erlay?
09:54 < gleb> rich: sorry, i don't understand your question
09:54 < rich> gleb: nevermind
09:55 < sdaftuar> lightlike: i think that's a great question that i hope reviewers give a lot of thought to!
09:55 < rich> perhaps new attack vectors related to latency if enough nodes reduce/exchange flood connections and you can somehow synchronize when reconciliations happen
09:56 < glozow> fingerprinting-type attacks? (which is why there are delays in responses)?
09:56 < tuition_>  /me wonders about any sort of race conditions
09:56 < lightlike> gleb: in your simulations, did you simulate the mixed scenario of Erlay and legacy nodes? I am a bit concerned that, since flooding is faster, TX relay might be centralised towards the legacy nodes, and the new Erlay nodes only get the "leftovers" and don't really have anything to reconcile.
09:56 < gleb> glozow: yeah, I only accounted for a "first-spy estimator", and also my general understanding of spy heuristics. Def a lot of room for research
09:56 < ariard> targeting the limited size of the local reconcliation set?
09:57 < ariard> broadcasting hard to compute sketch?
09:57 < dhruvm> Reducing redundancy can potentially open up tx censorship, but I can't actually think of a specific attack.
09:57 < glozow> what are the upper bounds of how hard it could be to compute a sketch?
09:57 < gleb> lightlike: I think no, but that would mean erlay nodes just won't get the benefit in early days? That's possible.
09:57 < willcl_ark> what happens if the flood nodes censor certain transactions
09:58 < lightlike> gleb: it might also mean more bandwidth for legacy nodes.
09:58 < gleb> willcl_ark: every node gets every transaction anyway. Censorship can happen the same way it can happen today (if maaany reachable nodes censor a tx)
09:58 < ariard> willcl_ark: you damage their latency but they should propagate through reconciliation?
09:58 < tuition_> Do we expect miners will run with erlay and non erlay nodes?  it seems they'll want to be maximally aware of all txns (hence non erlay) but will also want to guard against eclipse attacks (and maybe run block only hence no erlay)
09:58 < gleb> lightlike: need to think about this one
09:58 < rich> I guess you are creating/storing a sketch and holding onto it for some response, you'd want to be careful someone couldn't blow up your memory.
09:58 < lightlike> becasue they send all the TXes now before Erlay nodes get a chance
09:59 < gleb> tuition: if they run reachable nodes, even erlay-enabled, their latency is gonna be lower than what we have today!
09:59 < gleb> because in erlay, flooding artificial delays are reduced.
09:59 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Remote host closed the connection]     
10:00 < ariard> lightlike: depend if you assume that reachable nodes are going to be deploy edfaster than non-listening ones?
10:00 -!- vasild [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-pr-reviews     
10:00 -!- ma [92732bd1@146.115.43.209] has quit [Quit: Connection closed]     
10:00 < lightlike> ok, great discussion, time is up already!
10:00 < lightlike> !endmeeting
10:00 < glozow> thank you lightlike!
10:00 < willcl_ark> thanks lightlike, very interesting!
10:00 < ajonas> thanks lightlike
10:00 < ecola> thank you lightlike, very interesting !
10:00 < dhruvm> Thanks lightlike, gleb, sipa! This was 🔥!
10:00 < rich> thanks lightlike!
10:00 < dergoegge> thank you, this was fun!
10:00 < maqusat> thank you!
10:01 < shafiunmiraz0> Thank you lightlike
10:01 < jonatack> thanks gleb!
10:01 < OliP> Thank you!
10:01 < tuition_> thanks lightlike gleb sipa
10:01 < jonatack> thanks lightlike!
10:01 < gleb> I hope you guys stick around for the PR until it's done :)
10:01 < felixweis> thanks lightlike!
10:01 < fodediop> Thank you lightlike 🙏🏿
10:01 < effexzi> Thanks.
10:01 -!- shafiunmiraz0 [~shafiunmi@103.220.205.190] has left #bitcoin-core-pr-reviews ["Leaving"]     
10:01 < jnewbery> thank you lightlike!
10:01 < pinheadmz> great job lightlike !
10:02 -!- fodediop [~fode@41.214.86.100] has quit [Quit: WeeChat 3.0]     
10:02 < sipa> so, a question: would people be interested in a (lib)minisketch review club, and if so, ideas about what that would look like?
10:02 < gleb> i would attend :)
10:02 < jonatack> yes     
10:03 < dergoegge> me too!
10:03 < sdaftuar> i think there are two things there, one is the math that explains how minisketch works, the other is how the code implements the math
10:03 < dhruvm> sipa: I would be!
10:03 < lightlike> sipa: I'd definitely be interested!
10:03 < sdaftuar> it's totally unclear to me how accessible either of those are to explain on IRC?
10:03 < felixweis> yeah i think so, i didn't know there's a python impl so thanks jonatack for pointing it out. going to play around with it.
10:03 < sdaftuar> but i would like to attend :)
10:04 < ariard> yes for sure, it could be either oriented a) is minisketch a good set reconciliation algo and b) is the code correct and what can be done on the test coverage
10:04 < sipa> yeah - i don't know what i'd want to focus on
10:04 -!- AnthonyRonning [627f50cb@098-127-080-203.res.spectrum.com] has quit [Quit: Connection closed]     
10:04 < ariard> I would attend both :)
10:05 < sipa> i can imagine some people care more about the math, and some only about the code
10:05 < dhruvm> sipa: Can we review the code without understanding the math though?
10:05 < glozow> reviewing the math would be a prerequisite for reviewing the code tho
10:05 < jnewbery> sipa: my suggestion would be for people to prepare by reading through the math in their own time, and then have a review club on reviewing the code
10:06 < amiti> two part series??
10:06 < jnewbery> I agree with sdaftuar that IRC is a challenging medium for discussing mathematical concepts
10:06 < gleb> jnewbery: I think that's a good way to do it if sipa scopes the depth of math preparation.
10:06 < sdaftuar> if you can do this in two parts that would be pretty amazing imo
10:06 < jonatack> Meeting log is up at https://bitcoincore.reviews/18261 🍰️
10:06 < sipa> dhruvm: parts of it, i think so... you could review for lack of undefined behavior or other errors, or completeness of the tests, without knowing how the algorithms work
10:07 < tuition_> ty jonatack
10:07 < sipa> a math focused one could perhaps take the form of a review of the python reimplementation
10:08 < dhruvm> sipa: that's true, could focus on the _how_ instead of the _what_
10:08 -!- OliP [32f414b1@50-244-20-177-static.hfc.comcastbusiness.net] has quit [Quit: Connection closed]     
10:09 -!- NelsonGaldeman [5e0f2480@94.15.36.128] has quit [Quit: Connection closed]     
10:11 < felixweis> reviewing python after having worked trough a small list of recommended reading sounds like a good idea.
10:11 -!- maqusat [522fec0b@cpc73666-dals20-2-0-cust10.20-2.cable.virginm.net] has quit [Quit: Connection closed]     
10:13 -!- svav [5245568f@82-69-86-143.dsl.in-addr.zen.co.uk] has quit [Quit: Connection closed]     
10:13 < tuition_> does presenting the math in an audio broadcast make sense rather than IRC?
10:14 < felixweis> chaincode pod 👀
10:17 -!- sergei-t [~sergei@static-94-103-211-82.internet.lu] has left #bitcoin-core-pr-reviews []     
10:21 -!- ecola [~3cola@95.175.17.147] has quit [Quit: Leaving]     
10:26 -!- tuition_ [~tuition@195.181.160.175.adsl.inet-telecom.org] has left #bitcoin-core-pr-reviews []     
10:53 -!- DylanM [4461061f@ip68-97-6-31.ok.ok.cox.net] has quit [Quit: Connection closed]     
11:03 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has quit [Remote host closed the connection]     
11:09 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has joined #bitcoin-core-pr-reviews     
11:12 -!- lightlike [~lightlike@p200300c7ef180300394787c0e152616a.dip0.t-ipconnect.de] has quit [Quit: Leaving]     
11:40 < jonatack> lightlike: the github for you on the review club website points to https://github.com/lightlike...is it correct?
12:13 -!- Talkless [~Talkless@mail.dargis.net] has quit [Quit: Konversation terminated!]     
12:13 -!- effexzi [uid474242@gateway/web/irccloud.com/x-foessotidmamstta] has quit [Quit: Connection closed for inactivity]     
13:05 -!- jonatack_ [~jon@37.173.248.254] has joined #bitcoin-core-pr-reviews     
13:06 -!- jonatack_ [~jon@37.173.248.254] has quit [Client Quit]     
13:06 -!- jonatack_ [~jon@37.173.248.254] has joined #bitcoin-core-pr-reviews     
13:06 -!- jonatack_ [~jon@37.173.248.254] has quit [Client Quit]     
13:08 -!- jonatack_ [~jon@37.173.248.254] has joined #bitcoin-core-pr-reviews     
13:08 -!- jonatack_ [~jon@37.173.248.254] has quit [Client Quit]     
13:09 -!- jonatack [~jon@37.166.50.22] has quit [Ping timeout: 256 seconds]     
13:10 -!- jonatack [~jon@37.173.248.254] has joined #bitcoin-core-pr-reviews     
13:28 -!- jonatack [~jon@37.173.248.254] has quit [Ping timeout: 264 seconds]     
13:40 -!- jonatack [~jon@37.173.248.254] has joined #bitcoin-core-pr-reviews     
14:03 -!- harrigan [~harrigan@ptr-93-89-242-202.ip.airwire.ie] has quit [Read error: Connection reset by peer]     
14:04 -!- harrigan [~harrigan@ptr-93-89-242-202.ip.airwire.ie] has joined #bitcoin-core-pr-reviews     
14:13 -!- harrigan [~harrigan@ptr-93-89-242-202.ip.airwire.ie] has quit [Read error: Connection reset by peer]     
14:17 -!- harrigan [~harrigan@ptr-93-89-242-202.ip.airwire.ie] has joined #bitcoin-core-pr-reviews     
14:19 -!- gwollon [~gwillen@unaffiliated/gwillen] has joined #bitcoin-core-pr-reviews     
14:19 -!- gwollon is now known as gwillen     
14:21 -!- ma [92732bd1@146.115.43.209] has joined #bitcoin-core-pr-reviews     
14:21 -!- ma [92732bd1@146.115.43.209] has quit [Client Quit]     
14:25 -!- jb55 [~jb55@gateway/tor-sasl/jb55] has quit [Remote host closed the connection]     
14:25 -!- davterra [~davterra@gateway/tor-sasl/tralfaz] has quit [Remote host closed the connection]     
14:25 -!- jb551 [~jb55@gateway/tor-sasl/jb55] has joined #bitcoin-core-pr-reviews     
14:25 -!- davterra [~davterra@gateway/tor-sasl/tralfaz] has joined #bitcoin-core-pr-reviews     
14:27 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Disconnected by services]     
14:27 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-pr-reviews     
14:27 -!- vasild_ is now known as vasild     
14:49 -!- jonatack [~jon@37.173.248.254] has quit [Ping timeout: 272 seconds]     
14:54 -!- jonatack [~jon@37.173.248.254] has joined #bitcoin-core-pr-reviews     
15:42 -!- rjected [~weechat-h@natp-128-119-202-60.wireless.umass.edu] has quit [Ping timeout: 240 seconds]     
15:44 -!- rjected [~weechat-h@natp-128-119-202-60.wireless.umass.edu] has joined #bitcoin-core-pr-reviews     
15:50 -!- jadi [~jadi@213.207.193.58] has joined #bitcoin-core-pr-reviews     
15:50 -!- jonatack [~jon@37.173.248.254] has quit [Ping timeout: 264 seconds]     
15:58 -!- jadi [~jadi@213.207.193.58] has quit [Ping timeout: 264 seconds]     
15:58 -!- jonatack [~jon@37.173.248.254] has joined #bitcoin-core-pr-reviews     
16:06 -!- Hernan_ [~Hernan@OL121-235.fibertel.com.ar] has joined #bitcoin-core-pr-reviews     
16:18 -!- Hernan_ [~Hernan@OL121-235.fibertel.com.ar] has quit [Quit: Leaving]     
16:22 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:8dc5:1d88:1b78:e5ab:dd70] has joined #bitcoin-core-pr-reviews     
16:23 -!- mol_ [~mol@unaffiliated/molly] has quit [Quit: Leaving]     
16:24 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-core-pr-reviews     
16:55 -!- jessepos_ [~jp@2601:645:200:162f:470:b123:9431:f230] has quit [Quit: Textual IRC Client: www.textualapp.com]     
17:19 -!- mdrollette [~mdrollett@cpe-70-123-125-237.tx.res.rr.com] has quit [Ping timeout: 246 seconds]     
17:22 -!- jesseposner [~jp@2601:645:200:162f:35da:3d92:c670:d60d] has joined #bitcoin-core-pr-reviews     
17:26 -!- mdrollette [~mdrollett@cpe-70-123-125-237.tx.res.rr.com] has joined #bitcoin-core-pr-reviews     
17:58 -!- belcher_ [~belcher@unaffiliated/belcher] has joined #bitcoin-core-pr-reviews     
18:01 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 240 seconds]     
18:02 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:8dc5:1d88:1b78:e5ab:dd70] has quit [Quit: Textual IRC Client: www.textualapp.com]     
19:12 -!- seven__ [~seven@90.157.197.248] has quit [Read error: Connection reset by peer]     
21:18 -!- jadi [~jadi@213.207.193.58] has joined #bitcoin-core-pr-reviews     
21:24 -!- jadi [~jadi@213.207.193.58] has quit [Ping timeout: 246 seconds]     
21:57 -!- jessepos_ [~jp@2601:645:200:162f:7169:2a92:e85f:5daa] has joined #bitcoin-core-pr-reviews     
22:00 -!- jesseposner [~jp@2601:645:200:162f:35da:3d92:c670:d60d] has quit [Ping timeout: 264 seconds]     
22:00 -!- jadi [~jadi@213.207.193.58] has joined #bitcoin-core-pr-reviews     
22:19 -!- jessepos_ [~jp@2601:645:200:162f:7169:2a92:e85f:5daa] has quit [Quit: Textual IRC Client: www.textualapp.com]     
22:19 -!- jesseposner [~jp@2601:645:200:162f:7169:2a92:e85f:5daa] has joined #bitcoin-core-pr-reviews     
22:23 -!- shesek [~shesek@unaffiliated/shesek] has quit [Remote host closed the connection]     
23:05 -!- jadi [~jadi@213.207.193.58] has quit [Ping timeout: 264 seconds]     
23:09 -!- jadi [~jadi@213.207.193.58] has joined #bitcoin-core-pr-reviews