--- Day changed Wed Nov 25 2020 00:03 -!- reallll is now known as belcher 01:15 -!- vasild_ is now known as vasild 03:08 -!- provoostenator [~quassel@provoostenator.sprovoost.nl] has quit [Remote host closed the connection] 03:11 -!- provoostenator [~quassel@provoostenator.sprovoost.nl] has joined #bitcoin-core-pr-reviews 03:20 -!- Austen16Jast [~Austen16J@static.57.1.216.95.clients.your-server.de] has joined #bitcoin-core-pr-reviews 03:21 -!- provoostenator [~quassel@provoostenator.sprovoost.nl] has quit [Ping timeout: 272 seconds] 03:22 -!- provoostenator_ [~quassel@provoostenator.sprovoost.nl] has joined #bitcoin-core-pr-reviews 03:25 -!- Austen16Jast [~Austen16J@static.57.1.216.95.clients.your-server.de] has quit [Ping timeout: 240 seconds] 03:42 -!- odv [~odv@unaffiliated/odv] has joined #bitcoin-core-pr-reviews 03:45 -!- odv [~odv@unaffiliated/odv] has quit [Quit: Lost terminal] 04:08 -!- da39a3ee5e6b4b0d [~da39a3ee5@67.23.55.162] has quit [Ping timeout: 256 seconds] 04:10 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Ping timeout: 240 seconds] 04:24 -!- vasild [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-pr-reviews 04:40 -!- mol [~mol@unaffiliated/molly] has quit [Quit: Leaving] 05:00 -!- da39a3ee5e6b4b0d [~da39a3ee5@171.5.161.165] has joined #bitcoin-core-pr-reviews 05:03 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 240 seconds] 05:15 -!- belcher [~belcher@unaffiliated/belcher] has joined #bitcoin-core-pr-reviews 05:24 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has quit [Remote host closed the connection] 05:31 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has joined #bitcoin-core-pr-reviews 05:35 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-pr-reviews 05:35 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has quit [Remote host closed the connection] 05:38 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Ping timeout: 240 seconds] 05:40 -!- vasild [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-pr-reviews 05:48 -!- pinheadm_ [~pinheadmz@91.207.175.28] has joined #bitcoin-core-pr-reviews 05:50 -!- pinheadmz [~pinheadmz@pool-100-33-69-78.nycmny.fios.verizon.net] has quit [Ping timeout: 240 seconds] 06:23 -!- da39a3ee5e6b4b0d [~da39a3ee5@171.5.161.165] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 06:40 -!- varioust [~varioust@gateway/tor-sasl/varioust] has joined #bitcoin-core-pr-reviews 06:49 -!- odv [6c3e3198@unaffiliated/odv] has joined #bitcoin-core-pr-reviews 06:51 -!- odv [6c3e3198@unaffiliated/odv] has quit [Remote host closed the connection] 07:12 -!- jeremyrubin [~jr@c-73-15-215-148.hsd1.ca.comcast.net] has joined #bitcoin-core-pr-reviews 08:35 -!- joelklabo [~textual@157-131-101-185.fiber.dynamic.sonic.net] has joined #bitcoin-core-pr-reviews 08:36 -!- thomasb06 [~user@eth-west-pareq2-46-193-0-224.wb.wifirst.net] has joined #bitcoin-core-pr-reviews 08:41 -!- dappdever [~dappdever@pool-100-8-184-107.nwrknj.fios.verizon.net] has joined #bitcoin-core-pr-reviews 08:41 -!- jesseposner [~textual@2601:643:8980:bfd2:f8c7:7086:b02c:8117] has joined #bitcoin-core-pr-reviews 08:42 -!- jesseposner [~textual@2601:643:8980:bfd2:f8c7:7086:b02c:8117] has quit [Client Quit] 08:43 -!- jesseposner [~jesseposn@2601:643:8980:bfd2:f8c7:7086:b02c:8117] has joined #bitcoin-core-pr-reviews 08:43 < jnewbery> Hi folks. We'll get started in 18 minutes. Notes and questions are at https://bitcoincore.reviews/19858. Looking forward to chatting p2p with you all! 08:53 -!- lightlike [~lightlike@p200300c7ef192500384a4babe11cbcaf.dip0.t-ipconnect.de] has joined #bitcoin-core-pr-reviews 08:53 < pinheadm_> I am going to periodically establish connections with other PR review groups to ensure I am in the best one ;-) 08:54 < jonatack> :D 08:54 -!- pinheadm_ [~pinheadmz@91.207.175.28] has quit [Quit: pinheadm_] 08:54 -!- andozw [496d3915@73.109.57.21] has joined #bitcoin-core-pr-reviews 08:54 -!- pinheadmz [~pinheadmz@91.207.175.28] has joined #bitcoin-core-pr-reviews 08:58 -!- elle [~ellemouto@155.93.252.70] has joined #bitcoin-core-pr-reviews 08:58 -!- blanching [~blanching@195.181.160.175.adsl.inet-telecom.org] has joined #bitcoin-core-pr-reviews 08:59 -!- glozow [uid453516@gateway/web/irccloud.com/x-wgwavzwbrixyodio] has joined #bitcoin-core-pr-reviews 09:00 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Quit: ZNC - http://znc.sourceforge.net] 09:00 < emzy> :) 09:01 < jnewbery> #startmeeting 09:01 < emzy> hi 09:01 < felixweis> hi all! 09:01 < glozow> hi 09:01 < dhruvm> hi 09:01 < willcl_ark> hi 09:01 < jesseposner> hi 09:01 < blanching> hi 09:01 < elle> hi 09:01 < pinheadmz> ¡hi! 09:01 < joelklabo> Hi! 09:01 < lightlike> hi! 09:01 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #bitcoin-core-pr-reviews 09:01 < sipa> hi 09:01 < michaelfolkson> hi 09:01 < amiti> hi 09:01 < jnewbery> welcome all! 09:01 < dappdever> hi 09:01 < jnewbery> great turnout today. Must be a popular PR! 09:01 < jnewbery> Notes and questions are here: https://bitcoincore.reviews/19858 09:02 < jnewbery> anyone here for the first time? 09:02 < joelklabo> Me 👋 09:02 < glozow> it's because jnewbery is a popular person 09:02 < jnewbery> Hi joelklabo. Thanks for joining us! 09:03 < jnewbery> Normal reminders: feel free to ask any questions at any time. We're all here to learn and all are welcome 09:03 < jnewbery> don't ask to ask, just ask 09:03 < ajonas> Hi 09:03 < blanching> welcome joelklabo 09:03 < jnewbery> amiti prepared some notes and questions for this meeting, which we'll use to guide the conversation, but feel free to jump in at any time 09:03 < jnewbery> who had a chance to review the PR (y/n)? 09:03 -!- sergei-t [~sergei@2001:a18:0:b23::6] has joined #bitcoin-core-pr-reviews 09:03 < jesseposner> y 09:04 < joelklabo> y 09:04 < pinheadmz> y just on github 09:04 < willcl_ark> a brief skim over the top... 09:04 < lightlike> y 09:04 < sipa> y 09:04 < dappdever> y 09:04 < felixweis> n 09:04 < jonatack> hi 09:04 < michaelfolkson> y 09:05 -!- larryruane_ [uid473749@gateway/web/irccloud.com/x-vuztyjlkacjmkbdi] has joined #bitcoin-core-pr-reviews 09:05 < emzy> n 09:05 < amiti> 0.5 y 09:05 < jnewbery> ok, any first thoughts about the PR? concept ACK/NACK? What's it trying to achieve? 09:05 < pinheadmz> more eclipse attack mitigation, more condfidence that the node is on the mnost work chain by explosring the p2p netowrk 09:06 < jonatack> y. i've been collecting data from a node running the branch with custom logging. need to analyze it further, but connectivity does seem improved. 09:06 < dappdever> Are block-relay-only connections less observable than address and transaction relays? 09:06 < jnewbery> pinheadmz: exactly right! 09:06 < pinheadmz> seems like overkill to me but i dunno if its a nack 09:06 < joelklabo> periodically checking for new block-relay-only peers to make sure your not only connected to dishonest peers 09:06 < blanching> pinheadmz: what's "explosring"? 09:06 < emzy> concept ACK. Sounds good to me. 09:06 < murch> hi 09:07 < pinheadmz> blanching connecting to more nodes and checking for more-work blocks 09:07 < michaelfolkson> Overkill as in we don't need to worry about eclipse attacks pinheadmz? 09:07 < jnewbery> *exploring 09:07 < dhruvm> (1) Mitigation against eclipse attacks by periodically creating new relay-only connections to look for higher PoW elsewhere and (2) Honest peers helping bridge other trapped honest nodes. 09:07 < pinheadmz> michaelfolkson overkill in that we already have block-only connections and 8 peers 09:07 < glozow> it could be better for eclipse attacks, provided you have a healthy/diverse addrman 09:07 < michaelfolkson> It is incremental, sure. But incremental better than nothing 09:07 < jnewbery> dappdever: yes, relaying transactions and addrs reveals information about the network topology, which could be used by a spy to map your connections 09:08 < jnewbery> There have been a couple of other PR review clubs about the concepts: https://bitcoincore.reviews/15759 https://bitcoincore.reviews/17428 09:08 < michaelfolkson> Can we just double check I have terminology right? Stale tip is when your tip has a new block on top of the previous tip right? 09:08 < michaelfolkson> *I 09:09 < michaelfolkson> Nothing to do with orphan blocks or anything like that 09:09 < pinheadmz> stale tip means no new blocks in a while 09:09 < pinheadmz> orphan blocks like the name suggests menas a block with no parent 09:09 < elle> even if an attacker can map your connections, how can they actually perform the eclipse attack? how can the attacker get you to disconnect you other peers and connect to them instead? 09:09 < michaelfolkson> Just missing the latest block? 09:09 < pinheadmz> doesnt happen anymore since headers first 09:09 < lightlike> I'm still a a bit sceptical: if your addrman has been corrupted, the additional periodic block-only connections don't seem to help much (because you likely won't connect to a good peer). But if your addrman is healthy, how did you get eclipsed in the first place? 09:09 < joelklabo> I was curious why block-only peers reveal less about network topology but I'll check if that was covered in the previous reviews 09:09 < jnewbery> michaelfolkson: that's a really great question. Terminology is important here, and people often get it wrong or get confused 09:09 < glozow> lightlike: that's my bit of skepticism as well 09:10 < jnewbery> This is a good summary of what is meant by stale and orphan blocks: https://bitcoin.stackexchange.com/a/5869/26940 09:10 < michaelfolkson> Great, thanks 09:10 < pinheadmz> elle i think a real eclipse involves filling up a peers addrman with "poinson" entries and just waiting and waiting for those to get rotated in, or possibly joinging with a DoS attack to force the node to restart ? 09:10 < pinheadmz> *poison 09:11 < jnewbery> I'd say that 'stale' doesn't mean the block is old. It means that we know about a competing chain with more work 09:11 < elle> oh! ok cool. thanks pinheadmz :) 09:11 < michaelfolkson> lightlike, glozow: I think you are thinking in boolean rather than on a continuum. Your addrman isn't either perfectly pure or perfectly poisoined. It could be partially poisoned or under attack 09:12 < michaelfolkson> Any bolstered defence is better than nothing 09:12 < dhruvm> Since certain nodes are probably larger targets for eclipse attacks, small incremental benefits for them should have significant network benefits? 09:13 < jnewbery> Right, defense in depth. But I think a good question is whether the potential improved protection is worth the additional complexity 09:14 < amiti> joelklabo: I gave a talk about eclipse attacks & it covered some of the differences between relaying txns / blocks / addrs. if you're interested: https://www.youtube.com/watch?v=H-wH6mY9pZo&feature=youtu.be&t=252 09:14 < jnewbery> We've already started touching on the next question: Describe a scenario in which periodic block-relay-only connections could help prevent an attack? 09:14 < joelklabo> thanks amiti, I'll check that out 09:14 -!- sergei-t [~sergei@2001:a18:0:b23::6] has left #bitcoin-core-pr-reviews [] 09:14 < sipa> one argument may be the symmetry: we create extra normal connections that may get promoted to normal ones... it feels natural to do the same with blockonly connections 09:15 < jnewbery> Thanks amiti! That was a great talk. 09:15 < jesseposner> This paper has a detailed description of an eclipse attack: https://www.usenix.org/system/files/conference/usenixsecurity15/sec15-paper-heilman.pdf 09:15 < ajonas> Would a case be when the node has been eclipsed but their addrman still has honest addresses in it? 09:16 < dappdever> A node could receive block-relay-only blocks that reveal the legit chain that was attempting to be obfuscated by an attacker 09:16 < amiti> building on the conversation so far- I guess if an adversary has been trying to poison your addrman, has taken over some of it eg. 60%. you could happen to have all your outbound connections to them, but then rotating through some of the others connects you to the honest network? 09:16 < jnewbery> ajonas: yes, I think that's the scenario where this would help. 09:16 < joelklabo> addrman, that's address manager I'm guessing? 09:17 < glozow> yup 09:17 < dhruvm> amiti: is the opposite true as well? if you are eclipsed, this makes it more likely that an honest node reaches out? 09:17 < jnewbery> joelklabo: right. It's where we store addresses of potential peers. 09:17 < willcl_ark> If we are worried about having a poisoned addrman, and then with a feeler we detect that we're on a stale tip, would we not want to promote that new ("honest") peer to a NODE_NETWORK peer, rather than blocksonly? 09:17 < sipa> ip addresses, to be clear; not bitcoin addresses 09:17 < pinheadmz> so this PR introduces kind of a slow churn through the addrman 09:17 < jnewbery> sipa: thanks for the clarification! 09:17 < willcl_ark> unless blocksonly also relays addrs? 09:18 < amiti> dhruvm: ah fair, but I think it would be harder to be eclipsed if you're also enabling inbounds? although maybe not that much harder if you were just spinning up 🤔 09:18 < pinheadmz> if theres at least one honest peer in there well eventually connect to it and learn the truth about reality 09:18 < jnewbery> willcl_ark: no. Just blocks. We want block-relay-only peers to be as undetectable as possible, so they don't take part in address relay 09:18 < jnewbery> (I assume you mean block-relay-only peers, not -blocksonly?) 09:18 < dappdever> Where do the block-relay-only inbound connections come from? How do they discover your address initially? 09:19 < willcl_ark> jnewbery: hmmm. So we would be OK with a "bad" Addrman as long as we keep receiving good blocks? 09:19 < willcl_ark> (and yes that's what I mean!0 09:19 < pinheadmz> dappdever all inbound connections just come from your public IP being gossiped by peers. 09:19 < sipa> dappdever: they're just normal nodes, and their IP address gets rumoured 09:19 < pinheadmz> from your nodes perspective, the block only peers that matter are always outbound conenctions 09:19 < dhruvm> amiti: That means this PR helps even when your adddrman is 100% poisoned? 09:19 < dappdever> got it, thanks 09:20 < sipa> dappdever: it's not that these are secret nodes; just rhe existance of the connection that is less observable 09:20 < joelklabo> sipa, what does rumoured mean? 09:20 < jnewbery> Just so everyone is clear, -blocksonly mode and block-relay-only peers are two different concepts. Again, I think it's important to be precise with terminology because they mean different things and it's easy to get confused between them otherwise. 09:20 < jnewbery> The thread from here lists some differences: https://github.com/bitcoin/bitcoin/pull/20217#issuecomment-727876700 09:20 < sipa> joelklabo: nodes sharing data with each other... "hey i know a cool IP address of a bitcoin node" 09:20 < sipa> joelklabo: address relay 09:21 < joelklabo> I see, thanks 09:21 < michaelfolkson> How long does it take for a newly mined block to propagate across the network? You would want your block-relay connection to have the chance to give you the latest block rather than being cut off instantly? 09:21 < sipa> jnewbery: ah yes, thanks - i may be getting the name wrong sometimes 09:21 < jnewbery> rumour/gossip/relay are mostly interchangeable 09:22 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Quit: ZNC - http://znc.sourceforge.net] 09:22 < jnewbery> propogation/dissemination/promulgation. What a rich language we have. 09:22 < dappdever> are the rumours just a different message type? Or does it happen on a different protocol? 09:22 < dappdever> or different socket? 09:22 < sipa> dappdever: all the same P2P protocol, same connections, same sockets 09:23 < jnewbery> dappdever: addresses are 'rumoured' or 'gossipped' or 'relayed' using the `addr` message in the P2P protocol 09:23 < dhruvm> dappdever: https://developer.bitcoin.org/reference/p2p_networking.html#addr 09:23 < joelklabo> so the nodes you are connecting to are all the same you just ask those two for block only 09:23 < dhruvm> dappdever: also https://developer.bitcoin.org/reference/p2p_networking.html#getaddr 09:23 < jonatack> dappdever: you might find helpful the enum class ConnectionType documentation in src/net.h 09:23 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #bitcoin-core-pr-reviews 09:23 < jnewbery> joelklabo: yes. In fact, you're connecting to them and saying "no transactions please". They may still gossip addresses to you but you'd ignore them 09:24 < willcl_ark> so what are the assumptions here: we are currently eclipsed, but we have _some_ honest addresses in our addrman. This PR will eventually rotate through enough block-relay-only peers that we will discover a more-work chain, then replace one of our current block-relay-only peers with the new peer? 09:24 < michaelfolkson> And all connections are taken from the addrman right? If your addrman is 100 percent poisoned and all connections are controlled by adversary you're done unless you restart 09:24 < jnewbery> willcl_ark: exactly right 09:24 < jnewbery> michaelfolkson: you're done even if you restart 09:24 < willcl_ark> jnewbery: ok thanks 09:25 < jnewbery> because when you restart you'll connect to peers from your addrman 09:25 < michaelfolkson> jnewbery: Restart with DNS seeds 09:25 < sipa> willcl_ark: another way to put it is that with more connections, even just occasionally, the threshold of how much of addrman needs to be under attacker control increases 09:25 < jonatack> michaelfolkson: or addnode an honest peer 09:25 < joelklabo> is the reason we ask for blocks only to save bandwidth? 09:25 < jnewbery> I'd need to look at it again, but I don't think we use DNS seeds if our addrman is already full 09:25 < sipa> joelklabo: less bandwidth, and less leaking of network topology 09:26 < sipa> joelklabo: from such a block-relay connection, the peer has a harder time inferring who you're connected to 09:26 < dhruvm> michaelfolkson: If you are 100% poisoned, does this PR increase the chance that an honest peer reaches out if you have inbounds enabled? 09:26 < jnewbery> joelklabo: I'd say mostly less leaking of network topology. The fact that they use less bandwidth/memory/CPU is a nice property that means we can potentially have more of them 09:26 < pinheadmz> jnewbery actually that makes me think - what if this PR actually queried the DNS seeds? they are centralized endpoints but are trusted 09:27 < sipa> pinheadmz: dns seeds are a last resort effort to find peers 09:27 < pinheadmz> like, use a completely different protocol besides bitcoin p2p to get new peers 09:27 < willcl_ark> sipa: thats an interesting way to look at it. To be honest, I am kind of suprised nodes don't handshake and request tip hash (or a getheaders) from more nodes before this PR 09:27 < pinheadmz> sipa yes but they exist outside the p2p network and cant be eclipesed 09:27 < pinheadmz> unless your DNS resolver is also under attack 09:27 < joelklabo> harder to infer network topology because just less traffic between the nodes? Sorry if I'm repeating questions 09:27 < pinheadmz> or a seed maintainer is under attack 09:27 < michaelfolkson> dhruvm: This PR has no effect on that. But yeah an inbound request could save you generally (I think...) 09:27 < jesseposner> There are many flavors of eclipse attacks, and I believe that the block-relay-only connections are trying to mitigate against attacks that target the connected peers directly. It doesn't protect against a 100% poisoned addrman state. It simply makes it more difficult to discover all the connected peers. 09:27 < jnewbery> pinheadmz: all of this stuff is easy if you just trust a centralized authority! 09:28 < jnewbery> you could just get your blocks directly from sipa 09:28 < willcl_ark> :D 09:29 < ajonas> joelklabo - transactions can be fingerprinted to see who is connected to who. It's easier to infer topology through tx relay. 09:29 < jnewbery> sorry, that was a bit flippant. But using DNS seeds does have a downside. One is that you're potentially telling people that you're running a bitcoin node that you wouldn't otherwise be doing 09:29 < emzy> I only trust blocks I mined myself ;) 09:29 < dhruvm> michaelfolkson: ah, ok. i am trying to make sense of https://github.com/bitcoin/bitcoin/pull/19858/files#r483713328 09:29 < lightlike> I'd hope that it would be much easier to corrupt the DNS nodes (that are on central places with actual, known operators) than our addrman. 09:29 < pinheadmz> jnewbery point taken! see Vitaliks recent post about "weak subjectivity" i.e. check a block exploerer now and then 😬 09:29 -!- mol [~mol@unaffiliated/molly] has joined #bitcoin-core-pr-reviews 09:30 < jnewbery> emzy: that's an impressive level of self-sovereignty 09:30 < jnewbery> ok, next question: What are potential downsides to the concept? Are there any ways it could reduce security guarantees? 09:30 < sipa> after altcoins, egocoins; everyone has their own 09:30 < ajonas> joelklabo see https://arxiv.org/abs/1812.00942 09:31 < joelklabo> ajonas, thanks 09:31 < michaelfolkson> dhruvm: "the behavior introduced here is beneficial to not just the node doing it, but to the whole network, as a node already connected to the honest network that is periodically connecting to new peers to sync tips with others is helping bridge the entire network" 09:31 < michaelfolkson> dhruvm: I think what Suhas means there is that the fewer nodes that are owned/eclipsed the better for the network 09:32 < sipa> joelklabo: tx relay and addr relay are relatively easy to use for identifying nodes and connectioms; there are various known techniques, and it's not so surprising... just construct a random tx or addr message, send it somewhere, and see where it appears 09:32 < sipa> joelklabo: this is not true for blocks, as they're very infrequent by design, and very expensive to construct 09:32 < sipa> joelklabo: look up txprobe 09:32 < joelklabo> I see, thanks sipa 09:33 -!- kristapsk_ is now known as kristapsk 09:33 < ajonas> michaelfolkson dhruvm - well there is also a subtle upgrade to find new block-only peers that are "useful" as in they deliver blocks that our current peers aren't delivering 09:33 < sipa> jnewbery: reducing open listening sockets on the network seems the most obvious downside 09:33 < jnewbery> We do make use of techniques to try to mask tx origination (e.g. randomized timer, batching invs), but I think those are really adding statistical noise. With a large enough sample set, I think it would still be possible to map the network 09:34 < sipa> jnewbery: exactly 09:34 < jonatack> joelklabo: if you run a node, you can observe the network interactions using -netinfo to help get an intuition for these things 09:34 < jonatack> watch ./src/bitcoin-cli -netinfo 4 09:34 < joelklabo> thanks, I'll check that out 09:34 < pinheadmz> jnewbery i dont see how this could affect security but yeah perhaps privacy 09:35 < lightlike> one downside would be that an attacker that we connect to and that can send us a new block would replace a potentially honest blocks-only connection - although this would be really costly for the attacker, because they would basically need to be a miner withholding a new block from the network just to eclipse us further. 09:35 < jnewbery> sipa: yes, that does seem to be the main downside. Also the cost/complexity/maintenance that's associated with any change 09:35 < michaelfolkson> ajonas dhruvm: So the more an individual node can find useful peers the easier it makes it for everybody else? 09:35 < pinheadmz> and honestly, if bandwidth were free computaitonally and financially - wouldnt there be a benefit to connecting to all 9,000 nodes? 09:35 < michaelfolkson> Not for privacy pinheadmz 09:35 < dhruvm> michaelfolkson: ajonas: How is a node already connected to an honest network, helping bridge the entire network by making new "feeler" block-relay-only connections? 09:35 < jnewbery> Next question. What happens if we learn about new blocks when opening one of these new block-relay-only connections? 09:36 < ajonas> If we learn a new block, we rotate out an existing next-youngest block-relay peer in favor of the new peer. 09:36 < blanching> batching invs only hides data flows / timing analysis which in some domains can obfuscate some topology but weakly, probabilistically (high resolution: eve knows x was sent before y and can conclude A) 09:36 < amiti> dhruvm, michaelfolkson: block-relay-only connections also help with grander partition attacks than just eclipse attacks (eg. a malicious party trying to create a network split). which is more a network level concern (and everybody loses if that happens) 09:36 < blanching> not sufficient to obfuscate topology well 09:37 < michaelfolkson> Good point amiti 09:37 < jnewbery> blanching: indeed. It can make mapping the topology more costly/difficult but can't prevent it entirely 09:37 < blanching> +1 09:37 < dhruvm> michaelfolkson: ajonas: amiti: So that comment is more about block propogation times rather than the honesty of the peer? 09:38 < ajonas> dhruvm: that would make sense to me 09:38 < jesseposner> ajonas: +1 09:38 < michaelfolkson> How long does it take for a newly mined block to propagate across the network? You would want your block-relay connection to have the chance to give you the latest block rather than being cut off instantly? 09:38 < murch> jnewbery: but without auth, how would you know they're from sipa? 09:39 < blanching> michaelfolkson IIRC something like 6-18 seconds is considered healthy propagation time 09:39 < jonatack> murch: i know when i get a block from sipa 09:39 < sipa> for non-miners the propagation time really doesn't matter 09:39 < michaelfolkson> So should you wait 18 seconds before kicking out a block-relay-only connection blanching? 09:39 < sipa> as long as it's not close to interblock time, of course, as you want to see confirmations 09:39 < blanching> 6-18 seconds. e.g. 1-3% of the average block creation window 09:40 < jnewbery> ajonas: if we receive a block from the additional block-relay-only peer while we're connected, will we _always_ disconnect the second youngest? 09:40 < sipa> these improvememts are primarily about making sure you get blocks at all 09:40 < sipa> not necessarily fast 09:40 < blanching> michaelfolkson I'd defer to someone else to answer that 09:40 < willcl_ark> jnewbery: there is a comparison to see who gave us a block most recently too 09:41 < blanching> sipa: agreed. it's about preventing peer ossification which would decrease attack costs 09:41 < michaelfolkson> You don't want to kick out your block-relay-only connection for no reason 09:41 < glozow> could someone elaborate on open listening sockets? how limited is the resource and what's the cost of taking up more? 09:41 < pinheadmz> glozow there is a maxInbound setting, right? 09:41 < blanching> for clarity: peer ossifying = decreases attack costs 09:42 < sipa> glozow: it hasn't been a problem for years, but early in bitcoin's history there was a time when inbound sockets were hard to find 09:42 < jnewbery> willcl_ark: right, we see which of our two youngest block-relay-only peers (which will be one of the existing block-relay-only peers and the 'additional' block-relay-only peer) gave us the block least recently and consider it for disconnection 09:42 < ajonas> jnewbery: I guess not if we're under the respective conn limits? 09:42 < sipa> this was when light clients with permanent p2p connections were far more popular than today 09:43 < jnewbery> so we're really getting into the meat of the new functionality here. https://github.com/bitcoin/bitcoin/pull/19858/files#diff-6875de769e90cec84d2e8a9c1b962cdbcda44d870d42e4215827e599e11e90e3R3962-R3999 if you want to follow along 09:43 < sipa> glozow: our prioritization logic for incoming connections makes it also harder to attack, unlesd you have lots of source IP networks 09:43 < glozow> oh, you mean like, there are fewer total available inbounds in the network? so it'd be harder for a new node to get a good set of outbound connections? 09:43 < sipa> glozow: yes 09:43 < willcl_ark> ajonas: I think only if we currently have a feeler active 09:43 < emzy> But we will free a conection if we find a better block-relay-only peer. So nothing changes. 09:44 < dhruvm> blanching: are anchors evidence that some ossification is useful? 09:44 < jnewbery> This function EvictExtraOutboundPeers() is called on a timer every 45 seconds 09:44 < lightlike> jonatack: In your test on mainnet, how often per hour would you actually succesfully connect to a new outbound peer and compare headers? In a short run on testnet yesterday with a fresh node (no peers.dat), that was rather rare, most of the time I'd get no response and the 5min poisson timer would start again. 09:44 < willcl_ark> as GetExtraBlockRelayCount will return > 0 09:44 < sipa> glozow: 128 incoming slots per publicly reachable peer, by default... it's a finite resource that can be used up 09:45 < sipa> years ago we'd just stop accepting incoming connections when the limit was reached... which made it kind of trivial to attack 09:45 < jnewbery> willcl_ark: feelers aren't relevant here. GetExtraBlockRelayCount() doesn't count feelers. 09:45 < emzy> btw. all my nodes have more then 128 connections incoming. 09:45 < amiti> for the questions around how long it takes blocks to propagate, this is a cool resource: 09:45 < amiti> https://dsn.tm.kit.edu/bitcoin/#propagation 09:45 < amiti> https://dsn.tm.kit.edu/bitcoin/videos.html#blocks 09:46 < willcl_ark> jnewbery: damn my terminology! I meant an extra block-relay-only peer over the limit (of two?) 09:46 < pinheadmz> emzy would you be willing to share how you host those/ what the cost is? 09:46 < glozow> sipa: ah okay i see, are inbounds less scarce now? does anyone ever have a problem getting 10 outbounds? 09:46 < blanching> amiti 09:46 < jonatack> lightlike: i've been busy with things for the 0.21 release and meaning to get back to the data, but IIRC, using this PR i did see greater peer diversity (e.g. higher peer id numbers) than usual. don't depend on that info though; i need to refresh and look at the data 09:46 -!- shesek [~shesek@unaffiliated/shesek] has quit [Remote host closed the connection] 09:46 < blanching> amiti i didnt realize they were still updating that thanks 09:46 < jnewbery> willcl_ark: ah! Terminology is important!!! 09:46 < sipa> that's no longer the case, but still... if there are more than 128*public_nodes/10 nodes making connections, not everyone would get them 09:46 < jnewbery> otherwise we're talking about different things and we don't realise it :) 09:46 < sipa> glozow: yeah, i'm talkimg about 2012 or so 09:47 < michaelfolkson> Thanks amiti 09:47 < jonatack> lightlike: i don't remember more but will contact you, if you don't mind, when i look at it 09:47 < sipa> glozow: i haven't seen evidence of scarcity since 09:47 < emzy> pinheadmz: all only HDD. Costs are about $10 to $50. 09:47 < sipa> jnewbery: misunderstandings are the basis of the best inventions 09:47 < jnewbery> ok, so if we call EvictExtraOutboundPeers(), and we currently have an additional block-relay-only peer (i.e. 3 in total), and the youngest gave us a block more recently than the second youngest, what do we do? 09:47 < amiti> sipa: 128 incoming? I thought 125 total - 8 (outbound) - 2 (block-relay-only) 09:47 < pinheadmz> emzy ahhhh HDD, must be slow to sync - but these are VPS? like amazon, digital ocean, etc? 09:48 < amiti> (searching code now) 09:48 < sipa> amiti: you're right, i was simplifying 09:48 < sipa> and/or misremembering 09:48 < emzy> pinheadmz: some are VPN others are dedicated servers. IBD is like one day. 09:48 < glozow> sipa: makes sense. i always thought the inbound : outbound ratio seemed really high, and people explained because not everyone enables inbounds, so the people who do have to make up for it. 09:48 < jesseposner> jnewbery: disconnect the second youngest 09:48 < lightlike> jonatack: if you had net logging enabled, you could also grep for the new log entries (e.g. "keeping block-relay-only peer=X chosen for eviction") 09:48 < michaelfolkson> sipa glozow: Something to monitor if Neutrino becomes widely used 09:48 < jnewbery> sipa: apparently so. The half-dose/full-dose oxford vaccine regime is more effective than the full-dose/full-dose regime and they only found out because they'd accidentally given the half dose to a bunch of people 09:49 < jesseposner> jnewbery: unless the youngest hasn't given us a block, in which case disconnect the youngest 09:49 < sipa> jnewbery: erlay was invented by gleb who misunderstood gmaxwell's idea 09:50 < jonatack> lightlike: yes, i did have it enabled and was grepping for "enabling\|block-relay-only" and -i "enabling\|block-relay-only\|extra" 09:50 < jonatack> lightlike: (based also on my added logging) 09:50 < jnewbery> jesseposner: are there any conditions where we wouldn't disconnect either? 09:50 < lightlike> jonatack: cool, I'd be interested in the statistics once you find the time for that! 09:51 < jonatack> lightlike: happy to, i'm sure you'll have good ideas for interpreting it 09:52 < willcl_ark> looks like we disconnect youngest_peer.first as default 09:52 -!- pinheadm_ [~pinheadmz@pool-100-33-69-78.nycmny.fios.verizon.net] has joined #bitcoin-core-pr-reviews 09:52 -!- odv [6c3e3198@unaffiliated/odv] has joined #bitcoin-core-pr-reviews 09:52 < willcl_ark> ah, if we are getting a block right now then we might not disconnect either 09:53 < jnewbery> willcl_ark: youngest_peer.first refers to the peer id. (youngest_peer is a std::pair of peer id and the time of the latest block from that peer) 09:54 < jnewbery> willcl_ark: yes, exactly. If we're currently downloading a block from that peer, then we won't disconnect it 09:54 < michaelfolkson> sipa jnewbery: Misunderstandings are great for inventing things. Not so great for analyzing a robustness of a system and avoiding bugs ;) 09:54 < jnewbery> which makes sense. We shouldn't disconnect a peer if it looks like it's about to give us a block 09:54 < ajonas> Am I reading this right that we need to have been connected to those peers for more than 30 seconds? 09:55 -!- pinheadmz [~pinheadmz@91.207.175.28] has quit [Ping timeout: 256 seconds] 09:55 < willcl_ark> ajonas: looks like that to me: more than 30 seconds connection time and not currently receiveing a block 09:55 < jnewbery> ajonas: exactly right. If the timer for EvictExtraOutboundPeers() pops a few seconds after connecting to the additional block-relay-only peer, then we haven't given it enough time to send us a block, so we don't disconnect 09:56 < jnewbery> Ok, time for one last question. How do we maintain the count for block-relay-only connections? What changes in this PR? 09:56 -!- rav3n [92732bd1@146.115.43.209] has joined #bitcoin-core-pr-reviews 09:57 < jonatack> lightlike: looking at it right now, a few (~4 to 10) times an hour 09:57 < jnewbery> hint: we're looking in ThreadOpenConnections() 09:58 < jnewbery> here: https://github.com/bitcoin/bitcoin/blob/1e9e4b68f3cbd1af3de5806aeff952e9bf0ab8fc/src/net.cpp#L1843 09:59 < jnewbery> ThreadOpenConnections() is running a loop to open new connections to peers. Whenever it runs through that loop it'll count the number of block-relay-only connections that we have here: https://github.com/bitcoin/bitcoin/blob/1e9e4b68f3cbd1af3de5806aeff952e9bf0ab8fc/src/net.cpp#L1904-L1929 09:59 -!- Talkless [~Talkless@mail.dargis.net] has joined #bitcoin-core-pr-reviews 09:59 < jnewbery> That function is really long and really confusing, but it's essential to understanding how we manage our outbound connections, so definitly worth digging into if this stuff interests you 10:00 < michaelfolkson> Cool 10:00 -!- odv [6c3e3198@unaffiliated/odv] has quit [Remote host closed the connection] 10:00 < jnewbery> That's time everyone. Thanks for coming and chatting about p2p! And thanks again to amiti for preparing the notes and questions. 10:00 < lightlike> jonatack: thanks, that sounds quite good to me, considering that at a 100% success rate we'd have 12 hits per hour on average. 10:00 < blanching> thanks amiti jnewbery 10:00 < lightlike> thanks! 10:00 < jesseposner> Thanks! 10:00 < ajonas> thanks jnewbery and amiti for notes 10:00 < jnewbery> next week, glozow is hosting. Don't miss it! 10:00 < jnewbery> #endmeeting 10:01 < glozow> thanks jnewbery!!! 10:01 < willcl_ark> thanks jnewbery! 10:01 < elle> thanks everyone! 10:01 < jonatack> thanks amiti and john! 10:01 < dhruvm> Thank you, jnewbery, amiti and suhas! 10:01 -!- elle [~ellemouto@155.93.252.70] has quit [Quit: leaving] 10:01 < jnewbery> thank you glozow!!!!! 10:01 < amiti> thanks jnewbery, thanks all :) 10:01 < emzy> tnx amiti and jnewbery 10:01 < glozow> and thank you amiti!!!!!!!!!!!! 10:01 < jnewbery> ok time out 10:01 < jnewbery> enough !s 10:01 < michaelfolkson> Cut the conversation log 10:02 -!- rav3n [92732bd1@146.115.43.209] has quit [Remote host closed the connection] 10:02 * jesseposner graps scissors 10:02 < joelklabo> thanks everyone! jnewbery definitely learned some new things today 10:02 < dappdever> thank you 10:02 < willcl_ark> and nice work on Brink too jnewbery, super cool! 10:03 < blanching> +1 10:03 < joelklabo> I mean I learned some new things today. Thanks jnewbery 10:03 < blanching> lol 10:03 < joelklabo> haha 10:03 -!- dappdever [~dappdever@pool-100-8-184-107.nwrknj.fios.verizon.net] has quit [] 10:03 < jnewbery> joelklabo: I learned some new things too! 10:04 < jnewbery> wellcl_ark: thanks. Exciting times 10:08 < glozow> willcl_ark: u inspired me to make dis https://github.com/glozow/bitcoin-notes/blob/master/confusing-terms.md 10:09 < jnewbery> glozow: so cool! Great idea 10:10 < jonatack> lightlike: (and anyone else) pushed the patch branch if you want to run it and compare: https://github.com/jonatack/bitcoin/tree/pr19858-patches 10:11 < jonatack> or suggest ideas for improving it 10:12 -!- andozw [496d3915@73.109.57.21] has quit [Ping timeout: 245 seconds] 10:12 < sipa> glozow: i had never heard of cyberphunk 10:12 < michaelfolkson> glozow: BCH is Bose Chaudhuri Hocquenghem codes https://bitcoin.stackexchange.com/questions/74573/how-is-bech32-based-on-bch-codes 10:13 < lightlike> glozow: I like it! One term that used to confused me lot is that a "coin" is not 1 bitcoin but a utxo :-) 10:13 < glozow> oo thanks for suggestions, i will add 10:13 < glozow> sipa: my B@Bies say it all the time 10:14 < sipa> if you read cryptography papers that mention they use "coins" it means (public) randomness 10:14 < sipa> glozow: your what? 10:15 < jonatack> also COIN = 100000000 10:15 < jonatack> 1E8 10:15 < glozow> sipa: my Blockchain@Berkeley (B@B) babies 10:15 -!- varioust_ [~varioust@gateway/tor-sasl/varioust] has joined #bitcoin-core-pr-reviews 10:16 < michaelfolkson> Better add that to the doc as well 10:16 < glozow> yeah, not to be confused with actual babies 10:16 < jonatack> glozow: what is next week's PR? 10:16 < emzy> glozow: sounds like they start early in life with lerning Blockchain. ;) 10:17 < glozow> jonatack: #19698 :) 10:17 -!- varioust [~varioust@gateway/tor-sasl/varioust] has quit [Ping timeout: 240 seconds] 10:18 < jonatack> "test: apply strict verification flags for transaction tests and assert backwards compatibility" ... cool, wow pretty big 10:18 < blanching> alluring pr :) 10:19 < glozow> jonatack: yeah i had a hard time shortening the name 10:19 < jonatack> ah i meant the diff size. bringing back a Johnson Lau inititive, nice. noticed he has some that could use new life. 10:20 < glozow> oh right yes, i split it in half too. most of the diff is in the json 10:21 < jonatack> glozow: good stuff. i'll look out for your notes and questions PR 10:21 < glozow> :) 10:23 < blanching> glozow the distinction wrt orphan, extinct, barren, blocks is another "highly confusing" point of discussion 10:24 < blanching> I guess mainly because early uses weren't accurate nomenclature 10:24 < jonatack> blanching: add uncle blocks to that 10:24 < blanching> 8) 10:25 < emzy> jonatack: just no ;) 10:25 < glozow> ahhh thank you 10:26 < sipa> glozow: what about cypherphunk? 10:26 < glozow> sipa: funky music about cryptography, duh 10:29 < michaelfolkson> Yeah you can use that StackExchange link John shared on orphan, stale etc https://bitcoin.stackexchange.com/questions/5859/what-are-orphaned-and-stale-blocks/ 10:29 < blanching> pinheadm_ will master the debut release of the BitcoinCore players 10:33 < blanching> sipa glozow https://www.mixcloud.com/cameronferguson3745/cypherphunk/ haha 10:34 < glozow> blanching: omg cool 10:38 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Quit: ZNC - http://znc.sourceforge.net] 10:39 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #bitcoin-core-pr-reviews 10:44 < jnewbery> Today's meeting log is posted: https://bitcoincore.reviews/19858 11:24 -!- blanching [~blanching@195.181.160.175.adsl.inet-telecom.org] has left #bitcoin-core-pr-reviews [] 11:40 -!- pinheadm_ [~pinheadmz@pool-100-33-69-78.nycmny.fios.verizon.net] has quit [Quit: pinheadm_] 11:40 -!- pinheadmz [~pinheadmz@pool-100-33-69-78.nycmny.fios.verizon.net] has joined #bitcoin-core-pr-reviews 11:40 -!- joelklabo [~textual@157-131-101-185.fiber.dynamic.sonic.net] has quit [Quit: My iMac has gone to sleep. ZZZzzz…] 11:41 < pinheadmz> blanching: heh the most "cypherpunk" music I've made is the intro to Trace Mayer's podcast :-) that's the specific word he used in the request 12:06 < sdaftuar> dhruvm: i saw your question about my comment here - https://github.com/bitcoin/bitcoin/pull/19858/files#r483713328 12:08 < sdaftuar> the point i was trying to make is that it's healthy for the whole network if any honest node is engaging in the proposed behavior 12:09 < sdaftuar> because even an eclipsed (listening) node will eventually get an inbound connection from the honest nodes who are syncing tips with everyone in their addrman, and at the point that happens they'll have a chance to learn of a block from the incoming connection 12:11 < sdaftuar> at any rate this ties right into the comment that amiti made, that by creating new edges in the network graph all the time, we are hardening the network against partition attacks (ie raising the cost to an attacker of being able to successfully partition the network for a given amount of time) 12:12 < sdaftuar> please feel free to ping me here if you have more questions 12:23 -!- Talkless [~Talkless@mail.dargis.net] has quit [Quit: Konversation terminated!] 12:30 -!- shesek [~shesek@164.90.217.137] has joined #bitcoin-core-pr-reviews 12:30 -!- shesek [~shesek@164.90.217.137] has quit [Changing host] 12:30 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-pr-reviews 13:00 -!- effexzi [uid474242@gateway/web/irccloud.com/x-sfrzhrhccjrtpvef] has joined #bitcoin-core-pr-reviews 13:02 -!- varioust_ [~varioust@gateway/tor-sasl/varioust] has quit [Ping timeout: 240 seconds] 13:05 -!- lightlike [~lightlike@p200300c7ef192500384a4babe11cbcaf.dip0.t-ipconnect.de] has quit [Quit: Leaving] 14:00 -!- thomasb06 [~user@eth-west-pareq2-46-193-0-224.wb.wifirst.net] has quit [Remote host closed the connection] 14:44 -!- joelklabo [~textual@157-131-101-185.fiber.dynamic.sonic.net] has joined #bitcoin-core-pr-reviews 14:52 -!- varioust_ [~varioust@gateway/tor-sasl/varioust] has joined #bitcoin-core-pr-reviews 15:37 -!- ctrlbreak [~ctrlbreak@159.2.182.106] has joined #bitcoin-core-pr-reviews 16:35 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:7ef3:bc9a:f509:3193:e14d] has joined #bitcoin-core-pr-reviews 16:46 -!- joelklabo [~textual@157-131-101-185.fiber.dynamic.sonic.net] has quit [Quit: My iMac has gone to sleep. ZZZzzz…] 16:58 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:7ef3:bc9a:f509:3193:e14d] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 17:06 -!- seven_ [~seven@cpe-90-157-197-248.static.amis.net] has quit [Ping timeout: 240 seconds] 17:12 -!- andrewtoth [~andrewtot@gateway/tor-sasl/andrewtoth] has joined #bitcoin-core-pr-reviews 17:20 -!- larryruane_ [uid473749@gateway/web/irccloud.com/x-vuztyjlkacjmkbdi] has quit [Quit: Connection closed for inactivity] 17:35 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has joined #bitcoin-core-pr-reviews 17:38 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Ping timeout: 240 seconds] 18:13 -!- pinheadmz [~pinheadmz@pool-100-33-69-78.nycmny.fios.verizon.net] has quit [Quit: pinheadmz] 18:17 -!- pinheadmz [~pinheadmz@pool-100-33-69-78.nycmny.fios.verizon.net] has joined #bitcoin-core-pr-reviews 18:34 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Quit: ZNC - http://znc.sourceforge.net] 18:36 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #bitcoin-core-pr-reviews 18:38 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:7ef3:25b7:f5b5:a852:f5c4] has joined #bitcoin-core-pr-reviews 18:56 -!- effexzi [uid474242@gateway/web/irccloud.com/x-sfrzhrhccjrtpvef] has quit [Quit: Connection closed for inactivity] 19:22 -!- joelklabo [~textual@157-131-101-185.fiber.dynamic.sonic.net] has joined #bitcoin-core-pr-reviews 19:26 -!- varioust_ [~varioust@gateway/tor-sasl/varioust] has quit [Ping timeout: 240 seconds] 19:31 -!- varioust_ [~varioust@gateway/tor-sasl/varioust] has joined #bitcoin-core-pr-reviews 19:54 -!- reallll [~belcher@unaffiliated/belcher] has joined #bitcoin-core-pr-reviews 19:58 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 260 seconds] 20:00 -!- varioust_ [~varioust@gateway/tor-sasl/varioust] has quit [Ping timeout: 240 seconds] 20:03 -!- seven_ [~seven@cpe-90-157-197-248.static.amis.net] has joined #bitcoin-core-pr-reviews 20:12 -!- mol_ [~mol@unaffiliated/molly] has joined #bitcoin-core-pr-reviews 20:15 -!- mol [~mol@unaffiliated/molly] has quit [Ping timeout: 240 seconds] 20:27 -!- Wild_Man_ [~Android@2600:1700:ebe0:2940:cc05:474c:1015:186b] has joined #bitcoin-core-pr-reviews 20:27 -!- Wild_Man_ [~Android@2600:1700:ebe0:2940:cc05:474c:1015:186b] has left #bitcoin-core-pr-reviews [] 20:36 -!- pinheadmz [~pinheadmz@pool-100-33-69-78.nycmny.fios.verizon.net] has quit [Quit: pinheadmz] 21:02 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:7ef3:25b7:f5b5:a852:f5c4] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 21:14 -!- da39a3ee5e6b4b0d [~da39a3ee5@2403:6200:8876:7ef3:25b7:f5b5:a852:f5c4] has joined #bitcoin-core-pr-reviews 21:23 -!- joelklabo [~textual@157-131-101-185.fiber.dynamic.sonic.net] has quit [Quit: My iMac has gone to sleep. ZZZzzz…] 22:02 -!- t37p4 [~t37p4@30q3.l.time4vps.cloud] has joined #bitcoin-core-pr-reviews 23:12 -!- jesseposner [~jesseposn@2601:643:8980:bfd2:f8c7:7086:b02c:8117] has quit [Quit: My Mac Mini has gone to sleep. ZZZzzz…]