--- Day changed Wed Nov 20 2019 00:40 -!- davterra [~dulyNoded@195.242.213.120] has quit [Ping timeout: 265 seconds] 00:41 -!- davterra [~dulyNoded@195.242.213.120] has joined #bitcoin-core-pr-reviews 01:01 -!- ironbutt [~LiberLive@144.49.211.130.bc.googleusercontent.com] has joined #bitcoin-core-pr-reviews 01:25 -!- nobody123 [~nobody123@ipservice-092-217-244-175.092.217.pools.vodafone-ip.de] has joined #bitcoin-core-pr-reviews 02:26 -!- diogosergio [~diogoserg@212.36.34.126] has joined #bitcoin-core-pr-reviews 04:53 -!- slivera [slivera@gateway/vpn/privateinternetaccess/slivera] has joined #bitcoin-core-pr-reviews 05:29 -!- jonatack [~jon@213.152.161.25] has joined #bitcoin-core-pr-reviews 05:49 -!- emilengler [~emilengle@unaffiliated/emilengler] has joined #bitcoin-core-pr-reviews 06:24 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer] 06:24 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-pr-reviews 06:31 -!- diogosergio [~diogoserg@212.36.34.126] has quit [Ping timeout: 240 seconds] 06:34 -!- raj_149 [~quassel@ec2-18-217-191-36.us-east-2.compute.amazonaws.com] has joined #bitcoin-core-pr-reviews 06:45 -!- diogosergio [~diogoserg@212.36.34.126] has joined #bitcoin-core-pr-reviews 06:46 -!- emil__ [~emilengle@unaffiliated/emilengler] has joined #bitcoin-core-pr-reviews 06:48 -!- emilengler [~emilengle@unaffiliated/emilengler] has quit [Ping timeout: 246 seconds] 07:07 -!- andrewtoth_ [~andrewtot@gateway/tor-sasl/andrewtoth] has joined #bitcoin-core-pr-reviews 07:08 -!- andrewtoth [~andrewtot@gateway/tor-sasl/andrewtoth] has quit [Ping timeout: 260 seconds] 07:10 -!- andrewtoth_ [~andrewtot@gateway/tor-sasl/andrewtoth] has quit [Remote host closed the connection] 07:25 -!- andrewtoth [~andrewtot@gateway/tor-sasl/andrewtoth] has joined #bitcoin-core-pr-reviews 07:34 -!- jonatack [~jon@213.152.161.25] has quit [Ping timeout: 240 seconds] 07:38 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Ping timeout: 260 seconds] 07:40 -!- andrewtoth [~andrewtot@gateway/tor-sasl/andrewtoth] has quit [Ping timeout: 260 seconds] 07:43 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined #bitcoin-core-pr-reviews 07:49 -!- slivera [slivera@gateway/vpn/privateinternetaccess/slivera] has quit [Remote host closed the connection] 08:01 -!- ajonas [sid385278@gateway/web/irccloud.com/x-roxvryzffjhjkyws] has quit [] 08:02 -!- fox2p [~fox2p@cpe-66-108-32-173.nyc.res.rr.com] has quit [Ping timeout: 240 seconds] 08:05 -!- jonatack [~jon@89-93-131-92.hfc.dyn.abo.bbox.fr] has joined #bitcoin-core-pr-reviews 08:06 -!- fox2p [~fox2p@185.9.18.99] has joined #bitcoin-core-pr-reviews 08:08 -!- diogosergio [~diogoserg@212.36.34.126] has quit [Ping timeout: 252 seconds] 08:12 -!- emil__ [~emilengle@unaffiliated/emilengler] has quit [Quit: Leaving] 08:12 -!- emilengler [~emilengle@unaffiliated/emilengler] has joined #bitcoin-core-pr-reviews 08:22 -!- andrewtoth [~andrewtot@gateway/tor-sasl/andrewtoth] has joined #bitcoin-core-pr-reviews 08:24 -!- diogosergio [~diogoserg@212.36.34.126] has joined #bitcoin-core-pr-reviews 08:29 -!- diogosergio [~diogoserg@212.36.34.126] has quit [Ping timeout: 265 seconds] 08:45 -!- diogosergio [~diogoserg@212.36.34.126] has joined #bitcoin-core-pr-reviews 08:50 -!- diogosergio [~diogoserg@212.36.34.126] has quit [Ping timeout: 240 seconds] 09:01 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has joined #bitcoin-core-pr-reviews 09:01 -!- Talkless [~Talkless@hst-227-49.splius.lt] has joined #bitcoin-core-pr-reviews 09:02 < jnewbery> Hi folks. We'll get started in about an hour. Looking forward to it! Notes and questions are here: https://bitcoincore.reviews/16442.html 09:03 * pinheadmz does the cabbage-patch 09:04 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has quit [Client Quit] 09:04 -!- Talkless [~Talkless@hst-227-49.splius.lt] has quit [Client Quit] 09:05 -!- Talkless [~Talkless@hst-227-49.splius.lt] has joined #bitcoin-core-pr-reviews 09:06 -!- diogosergio [~diogoserg@212.36.34.126] has joined #bitcoin-core-pr-reviews 09:07 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has joined #bitcoin-core-pr-reviews 09:11 -!- diogosergio [~diogoserg@212.36.34.126] has quit [Ping timeout: 265 seconds] 09:12 -!- Talkless [~Talkless@hst-227-49.splius.lt] has quit [Quit: Konversation terminated!] 09:13 -!- Talkless [~Talkless@hst-227-49.splius.lt] has joined #bitcoin-core-pr-reviews 09:51 -!- icota[m] [icotamatri@gateway/shell/matrix.org/x-byhwlcwqlrdmwoao] has joined #bitcoin-core-pr-reviews 09:58 -!- miketwenty1 [473b2311@c-71-59-35-17.hsd1.ga.comcast.net] has joined #bitcoin-core-pr-reviews 09:58 < miketwenty1> I'm here to learn about bitcoin 09:58 < pinheadmz> miketwenty1: this is a good place for that! 09:59 -!- sanoj [sid385278@gateway/web/irccloud.com/x-toqmhvdkwcptqtmz] has joined #bitcoin-core-pr-reviews 10:00 < jnewbery> #startmeeting 10:00 < jnewbery> hi 10:00 < pinheadmz> hi 10:00 < fjahr> hi 10:00 < amiti> hi 10:00 < sanoj> hi 10:00 < emzy> Hi 10:00 < jnewbery> hi folks. pinheadmz is hosting today's meeting. Notes and questions are in the usual place: https://bitcoincore.reviews/16442.html 10:01 -!- jimpo [~jimpo@ec2-13-57-39-52.us-west-1.compute.amazonaws.com] has joined #bitcoin-core-pr-reviews 10:01 < pinheadmz> Hi all, is this anyone's first review club meeting? 10:01 < michaelfolkson> Hi 10:01 < jnewbery> Thanks to pinheadmz for writing notes and hosting. I'll hand straight over to him 10:01 < jimpo> hi 10:01 < miketwenty1> This is my first meeting 10:02 < nobody123> hi 10:02 < andrewtoth> hi my first meeting 10:02 < pinheadmz> oh thanks jimpo for attending - he's the author's of todays PR 10:02 < fanquake> hi 10:02 -!- tuxcanfly [~tuxcanfly@68.183.231.167] has joined #bitcoin-core-pr-reviews 10:02 < jkczyz> hi 10:02 < tuxcanfly> hello 10:02 < pinheadmz> Ok so for new friends: don't be afraid to ask questions! at any time, we'll get to you. 10:03 < pinheadmz> and thanks tuxcanfly for attending, he's working on bip157/158 in bcoin and knows a little something about cfilters 10:04 < pinheadmz> Cool, so does someone want to summarize the PR in a few lines? 10:04 < fjahr> It makes the filters from bip157/158 available on the P2P layer 10:05 -!- headway-translat [~headway-t@195.181.160.175.adsl.inet-telecom.org] has joined #bitcoin-core-pr-reviews 10:05 < pinheadmz> fjahr: yes! 10:05 < pinheadmz> so bip158 filters are a compact way to represent the contents of a block 10:05 < jonatack> hi 10:05 < jonatack> from the bitcoiner meetup in Paris :) 10:06 < pinheadmz> and bip157 is the specification on how those filters can be requested and delivered around the netwrk 10:06 < pinheadmz> jonatack: ooh lala! 10:06 < pinheadmz> Did everyone get a chance to review the PR? How about a quick y/n from everyone 10:06 < pinheadmz> maybe tell us how far you got in review? 10:06 < jnewbery> for those who don't have time to read the full BIPs, here's a summary of what they do: https://bitcoin.stackexchange.com/a/86232/26940 :) 10:06 < jimpo> y 10:06 < michaelfolkson> y 10:07 < jnewbery> y. high-level review. Didn't do detailed review 10:07 < tuxcanfly> y 10:07 < jkczyz> y - most code but didn't get through the tests 10:07 < fjahr> y, had looked at it a while back, did it again this week, wish I had more time for testing 10:07 < fanquake> n 10:08 < amiti> ish- took a look but didn't dig in 10:08 < pinheadmz> Great! Lets go through the questions -- how does a node signal to the network that it is able to serve CFilters ? 10:08 < miketwenty1> n 10:08 < pinheadmz> michaelfolkson: do you wanna take this one? ^ 10:08 < emzy> n 10:08 < fjahr> there is a service bit signalling bip158 support 10:08 < andrewtoth> n 10:08 < andrewtoth> NODE_COMPACT_FILTERS service bit 10:09 < michaelfolkson> Signals NODE_COMPACT_FILTERS yup 10:09 < miketwenty1> cfilters = compact filters? 10:09 < pinheadmz> miketwenty1: yes 10:09 < jnewbery> (it's initially slightly confusing that that's defined in BIP 158) 10:10 < pinheadmz> miketwenty1: bip158 defines cfilters or compact filters or casually "neutrino filters" 10:10 < jimpo> jnewbery: Yeah... I agree :-/ 10:10 < miketwenty1> thanks pinheadmz 10:11 < pinheadmz> does anyone NOT understand what cfilters are or why they are important? maybe we should check in on that 10:11 < nobody123> is there a class which handles all the p2p networking? 10:12 < pinheadmz> nobody123: the net_processing.cpp file is where most of the messaging logic is handled - is that what you mean? 10:12 < jnewbery> nobody123: not really a class. Look in net.cpp for the low-level peers and connections management, and then in net_processing.cpp for the application-level logic 10:13 < nobody123> jnewbery, pinheadmz thx 10:13 < pinheadmz> Ok, so on the topic of the service bit -- Did anyone notice the sort of open-ended question about this service bit as it relates to this PR? 10:13 < pinheadmz> (hint: question #3) 10:14 < sanoj> in terms of how to do this without blocking net threads and not signal to peer peers until the indexing is in sync? 10:14 < pinheadmz> sanoj: yes! 10:14 < pinheadmz> As far as I understand, the service bits are all set on launch and aren't really designed to ever change during run time 10:14 < pinheadmz> jimpo: is that correct? ^ 10:15 < sanoj> so as is if block filter index is still syncing then we don't advertise the service bit (https://github.com/bitcoin/bitcoin/pull/16442/files#diff-9a82240fe7dfe86564178691cc57f2f1R2657) 10:15 < jimpo> yes 10:15 < pinheadmz> sanoj: right - so what happens once a node has finished indexing? Can it start serving filters right away? 10:15 < jimpo> this would be the first case of a dynamically changing service flag (if that idea makes it through) review, hence the discussion 10:15 < andrewtoth> From jimpo in the PR: "And if I understand correctly, we re-advertise our local address with service bits to each peer every day on average, so the network should see the change eventually without the node operator having to do anything." Where does this happen? 10:15 < headway-translat> pinheadmz: purpose of compact filters is to create a space-efficient proof of txns non-inclusion in a block so that light clients have cheaper access to truth. e.g. lightning light clients can space / bandwidth efficiently certify that channel closure txns etc aren't in blocks 10:16 < jnewbery> my point was that service bits should advertise *capabilities*, not *current node state*. That's my understanding at least 10:16 < jimpo> andrewtoth: looking 10:16 < pinheadmz> headway-translat: great summary, yes 10:16 < pinheadmz> jnewbery: thats a good point 10:17 < jnewbery> the service bits are somewhat long-lasting. They're gossipped in VERSION messages, and served on DNS seed responses I believe 10:18 < jimpo> andrewtoth: From my reading AdvertiseLocal + nNextLocalAddrSend 10:18 < jnewbery> so if I start by telling my peers that I don't support a feature, that'll get gossipped around the network for some time after I change my service bit flags 10:18 < andrewtoth> jimpo: thanks 10:18 < pinheadmz> I suppose the question for light clients would be: if you request a cfilter and the node respnds with notfound or null -- is that reason to disconnect or ban? Because that node could still just be syncing 10:18 < jimpo> jnewbery: My argument would be that that's blurry. A node is incapable of serving filters it hasn't indexed yet. 10:20 < andrewtoth> How can a light client verify that the cfilters are correct? 10:20 < michaelfolkson> So jamesob makes the comparison with IBD, is how that is dealt with instructive here? 10:21 < fjahr> andrewtoth: the client can compare cfilters from different nodes 10:21 < pinheadmz> andrewtoth: good question - until the cfilters are committed in the block somehow (and therefore verified by all network nodes and ensured by proof of work) - there is no way to know without checking the block yourself 10:22 < jnewbery> jimpo: am I right in saying your main objection is that it makes logic more complicated for clients: https://github.com/bitcoin/bitcoin/pull/16442#discussion_r336731315 ? 10:22 < pinheadmz> andrewtoth: fjahr: but yes, BIP157 specifies a process by which a light node can be "pretty sure" the filters they get are correct 10:22 < jimpo> pinheadz: Even checking the block yourself is insufficient. You'd need chainstate or undo data. :-/ 10:22 < michaelfolkson> You could have two flags. One saying that you intend to provide them and one saying you're not ready to provide them? After a certain period of time the gossip would ignore the second flag? 10:22 -!- davterra [~dulyNoded@195.242.213.120] has quit [Ping timeout: 276 seconds] 10:22 < pinheadmz> jimpo: interesting - why would that be? aren't the cfilters noncontextual ? 10:22 < jimpo> jnewbery: Yes, that is my concern. That allowing a valid case where a node advertises that it can serve filters and is unable to would complicate client logic. 10:22 -!- davterra [~dulyNoded@195.242.213.120] has joined #bitcoin-core-pr-reviews 10:23 < jimpo> pinheadmz: There was a change to the BIP last year that made them contextual. Specifically, the filters include the prevout output scripts, which are not included in blocks, only referenced by inputs. 10:23 < Talkless> hi, are these filters are against txids, scripts (addresses), both? 10:23 -!- sebastian_ [~sebastian@85.203.15.20] has joined #bitcoin-core-pr-reviews 10:23 < jimpo> just scripts. Block output script and input (prev output) scripts. 10:24 < Talkless> thanks 10:24 < pinheadmz> jimpo: wow ok! extra work for the node... tuxcanfly did you know that? lol 10:24 < jnewbery> I think you need that logic anyway. In an untrusted p2p network, you don't know who is advertising that they can serve the filters: malicious nodes, buggy implementations, stalling node, unreliable connections, ... A BIP 157 client has to be able to handle cases where it gets invalid or missng reponses 10:24 < andrewtoth> michaelfolkson: what would be the difference to just signaling the first bit and then disconnecting after the ignore time time if they don't start providing? 10:25 < jimpo> jnewbery: Sure, but you just ban. This is a case where you need to disconnect from a *properly behaving* node. 10:26 < pinheadmz> jnewbery: youre thinking the signal bit should always be active, and if a node can't reply with a filter, they get banned by the client? 10:26 < Talkless> jimpo: but that's only about tx'es inside actual block, transactions queued in mempool will not be delivered through this mechanism to the light wallets? 10:26 < sanoj> jimpo: what about a state that returns some sort of sync message. I wouldn't imagine it'd happen that often but it would avoid the disconnect for proper behavior and maintain the validity of the capabilities service bit 10:26 < jimpo> Probably what a client should do is fetch the cfcheckpts to see how in-sync the node is 10:26 < pinheadmz> Talkless: thats my understanding 10:26 < jimpo> if nodes advertise the service bit before they have an in-sync index 10:27 < jnewbery> A bit more on the contextual/non-contextual stuff: the new scripts created in the block can obviously be constructed from the block. The scripts spent in the block require a node that has the UTXO set or the 'undo data' for the block 10:27 < pinheadmz> Talkless: wasabi wallet, for example, uses neturino filters but ALSO connects to Tx relay to discover "0-conf" 10:27 < Talkless> pinheadmz: interesting. 10:27 < jnewbery> ('undo data' == outputs spent in a block) 10:28 < michaelfolkson> andrewtoth: I suppose it depends whether you are concerned about maintaining connections or not. If you're not then sure advertise that you offer something and get disconnected when you disappoint 10:28 < miketwenty1> undo data = data needed in case of a reorg? 10:29 < pinheadmz> miketwenty1: yep, the coins spent by a block - so they are "unspent" if a blockis disconnected 10:29 < michaelfolkson> I would think you would generally want to minimize disconnections when both parties are honest and responsive 10:29 < pinheadmz> I like jimpo idea that light clients should request cfheckpts first to determine how in sync a node is 10:30 < pinheadmz> then the service bit could stay on, like jnewbery said to "indicate capability" 10:30 < jimpo> just... but why? 10:30 < jimpo> like I don't understand why dynamically switching on a service flag is such a bad idea 10:30 -!- jasonzhouu [~jasonzhou@2409:8a28:c17:26a0:53bd:22c0:b30a:33f7] has joined #bitcoin-core-pr-reviews 10:31 < Talkless> will new light wallet have to download all filters from the beginning, or could start from it's own "birthday" if wanted? 10:31 < jimpo> if it could flap, sure that's not good. but it's just a one-way transition from off to on. 10:31 < miketwenty1> just a clarification q. 10:32 < miketwenty1> pinheadmz "checkpts"? 10:32 < pinheadmz> jimpo: I was wondering about the tiny amount of time it would take a fully synced node to produce the cfilter... there could be a moment where even a synced node can't provide the filter for the tip ? 10:32 < andrewtoth> Talkless: I believe this PR introduces a way to request a range of filters, so it can start at birthdate 10:32 < jnewbery> I personally don't like it for a couple of reasons: service bits are handled by our net layer, and I don't like coupling together net with the compact blocks index. second: I think there's a general assumption that service bits don't change and I'd prefer not to break that 10:32 < jimpo> nope, we block on the background thread in that case 10:32 < jimpo> until the filter is built 10:32 < pinheadmz> jimpo: 👍 10:32 < headway-translat> Talkless CMIIR but i believe how the encoding works, to have higher degree of soundness, client would need to request filters several blocks before their bday 10:32 < pinheadmz> miketwenty1: so we'll get to checkpts i none sec... 10:33 < headway-translat> CMIIW* 10:33 < pinheadmz> s/in one... 10:33 < Talkless> andrewtoth: thanks 10:33 < Talkless> headway-translat: interesting, any ideas why? 10:34 < pinheadmz> headway-translat: Talkless: There is also the filter-headers, which commit to the previous block 10:34 < headway-translat> Believe it's due to how the filter headers work but I could be wrong. https://bitcoin.stackexchange.com/questions/86231/whats-the-distinction-between-bip-157-and-bip-158-are-they-supported-by-bitcoi/86232#86232 10:35 < pinheadmz> IIUC, both filter headers and checkpoints are needed just because we can't commit the filter the in the block yet 10:35 < headway-translat> yeah I suppose I am combining the two proofs 10:35 < headway-translat> erroneously 10:35 < andrewtoth> jnewbery: re service bits not changing, what is relying on this assumption? 10:36 < pinheadmz> lets talk about the checkpoints - does anyone want to explain the purpose of the cfcheckpt message ? 10:36 < michaelfolkson> andrewtoth: I think jnewbery answered this earlier. The incorrect service bit would percolate around the gossip network and be a pain to replace 10:37 < jimpo> well, let's be clear. A service bit can change on a node -- it just requires a restart currently. 10:37 < jimpo> and still does take a while for the new setting to propagate. 10:37 < andrewtoth> yes, that's what I was thinking, a restart would change and gossip would still be out of date 10:38 < jimpo> the difference here is that a service bit cannot (yet) change over the lifetime of a *connection* 10:38 < pinheadmz> tuxcanfly: are you still with us? do you want to explain the cfcheckpt message? 10:38 < jnewbery> andrewtoth: when peer addresses are advertised in ADDR messages, that peer's service bits are included, so you can think of other nodes caching the peer's service capabilities 10:39 < jnewbery> jimpo: right. If you consider switching nodes off/on then anything can change! 10:39 < miketwenty1> q. are checkpoints ephemeral or forever? 10:40 < miketwenty1> im not clear on the purpose of the checkpoint 10:40 < jkczyz> pinheadmz: the checkpoints allow clients to parallelize getcfilters from peers 10:40 < miketwenty1> in the context of this BIP 10:40 < pinheadmz> jkczyz: ty! yes 10:40 < jnewbery> miketwenty1: https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki#getcfcheckpt 10:40 < pinheadmz> miketwenty1: so thats an interesting q, because the checkpoints are based on 1000-block intervals from the requested "stop" block 10:41 < pinheadmz> so its possible that every time you make that request, you get a different series back 10:41 < pinheadmz> (jimpo right?) 10:41 < jimpo> no, 1000-block intervals from the start block 10:41 < pinheadmz> sorry thanks 10:41 < jimpo> so they are the same on every request so they can be cached easily 10:41 < pinheadmz> https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki#getcfcheckpt 10:41 < pinheadmz> just mentions the stophash ...? 10:42 < jimpo> the start block is 0 10:42 < pinheadmz> ah ok I see in https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki#cfcheckpt yes 10:42 -!- ironbutt [~LiberLive@144.49.211.130.bc.googleusercontent.com] has quit [Read error: Connection reset by peer] 10:43 < pinheadmz> So if I'm a brand new light client, what order do I send the BIP157 messages in? 10:43 < pinheadmz> (hint: https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki#client-operation) 10:44 < tuxcanfly> pinheadmz: sorry afk a bit, but yeah I remember coming across getcfcheckpt as a way to sync cfilters in batches 10:44 < tuxcanfly> because earlier you had to make a request per block... for each cfilter! 10:44 < jimpo> eek, the client-operation in the BIP is still out of date and assumes the old outpoint filter spec 10:44 * jimpo covers eyes 10:44 < pinheadmz> tuxcanfly: ty! a nice optimization 10:44 < jimpo> Need to finish re-writing that 10:45 < tuxcanfly> yeah you even had two types of filters so that would have been double the number of p2p messages for each client 10:45 < pinheadmz> jimpo: 😆 10:46 < sanoj> do i understand correctly that if we are syncing to block 600,001, the last checkpoint we'd receive covers up to 600,000? 10:46 -!- ironbutt [~LiberLive@82.142.166.82] has joined #bitcoin-core-pr-reviews 10:47 < pinheadmz> sanoj: yes that sounds right, but then presumably when we request the actual filters (in parallel) we would get that last 1 as well 10:47 < sanoj> if so, what happens if there is a reorg that spans that 1000 filter header mark? 10:47 < pinheadmz> because the light client knows the block headers on the best-work-valid chain 10:49 < jnewbery> sanoj: 'checkpoints' might be a bit confusing terminology here because we also have checkpointed blocks in Bitcoin. Here, cfcheckpoints are just an optimization for downloading the full set of compact block filters for the chain 10:49 < miketwenty1> i fell into this group initially^ 10:49 < pinheadmz> Ok home stretch! lets start wrapping up -- 10:49 < jimpo> as a clarification, they help parallelize the download of the filter headers 10:50 < jimpo> then the filter headers help parallelize the download of the actual filters 10:50 < pinheadmz> Q#6 - does this PR enable bitcoin core to RETRIEVE filters from peers ? 10:50 < jnewbery> once a client has downloaded all the compact block filters initially, then checkpoints have no further function 10:50 < sanoj> i see. Thanks all. 10:50 < jnewbery> jimpo: right - thanks for the clarification! 10:51 < miketwenty1> so, @jn 10:51 < michaelfolkson> What was implementing Neutrino on Bcoin like pinheadmz? Did you consider similar issues? Anything you will revisit now it is (almost) fully implemented on Core? 10:51 < miketwenty1> jnewbery to answer @sanoj's question .. you would just undo data.. and redownload compact filters? 10:52 < pinheadmz> michaelfolkson: tuxcanfly might wanna field this one - so far we have bip158 implemented but actually not bip157 yet 10:52 < jkczyz> pinheadmz: Nope, with this PR it only sends filters. It does not request or handle receiving them 10:52 < michaelfolkson> pinheadmz: Oh interesting. Delayed because of Core timetable or just lack of resources? 10:52 < pinheadmz> jkczyz: thats right! 10:53 < jnewbery> miketwenty1: light clients are not keeping track of the UTXO set, so they don't need undo data to roll back during a reorg 10:53 < pinheadmz> We have the spec and the bip157 process, but since bitcoind is not intended to be a light client, it doesn't need to request the filters 10:53 < pinheadmz> (although in a former pr review club, we DID discuss how bip158 filters are useful internally for the wallet) 10:53 < jkczyz> jnewbery, jimpo: Peers can send notfound in response to getdata. Would something analogous be suitable for when the filters are not ready? Is there some general error mechanism in Core's P2P layer? 10:53 < jnewbery> they do need to keep track of confirmations, so they'd need to work out whether any transactions that they're interested in had been removed from the block chain during the reorg. That doesn't require undo data 10:54 < jimpo> jkczyz: I remember some discussion of getting rid of NOTFOUND. Where did that end up? 10:54 < jnewbery> jkczyz: I agree that NOTFOUND seems appropriate here 10:54 < jnewbery> jimpo: not that I'm aware of. Perhaps you're thinking of REJECT messages? 10:54 < jimpo> maybe 10:54 < pinheadmz> jimpo: i thought that discussion was about mempool leaks? 10:55 < tuxcanfly> michaelfol: to answer your bcoin question - it was pretty straightforward... however there were some bip updates which needed to be synced... the test vectors were helpful in that regard, to make sure we're core compatible for every edge case 10:55 < pinheadmz> er, nm - i might be confused 10:55 < fjahr> could also implement something like NOTREADY? 10:55 < jnewbery> NOTFOUND could be useful in cases like https://github.com/bitcoin/bitcoin/pull/15505 , so I hope they don't get removed! 10:55 -!- sebastian_ [~sebastian@85.203.15.20] has quit [Ping timeout: 265 seconds] 10:55 < fjahr> to make sure it is clear why it was not found 10:56 < pinheadmz> michaelfolkson: tuxcanfly actually had to re-write the PR! This is the one that landed, if you're interested: https://github.com/bcoin-org/bcoin/pull/797 10:56 < pinheadmz> OK five minutes left !! 10:56 < tuxcanfly> michaelfol: the p2p cfilter serve was part of the PR but didn't enough review so it isn't merged just yet... we will be getting that one soon 10:56 < pinheadmz> Did anyone play with the tests at all? Or try anything wacky with an actual lightning node? 10:56 < pinheadmz> I ran the test suite with wireshark open, sniffing loca host - and is really cool to see the dialog between the test nodes 10:57 < pinheadmz> *localhost 10:57 < tuxcanfly> I just have one question to jimpo or anyone interested 10:57 < jimpo> sorry, guys I have to go now 10:57 < jimpo> thanks everyone for you time reviewing! 10:57 < pinheadmz> jimpo: TY so much for attending! <3 10:57 < jonatack> thanks jimpo 10:57 < jnewbery> jimpo: thanks so much for dropping in and answering questions! 10:57 < miketwenty1> (y) 10:57 < fjahr> I just inspected some messages by logging, nothing much 10:58 < pinheadmz> tuxcanfly: whats your Q ? 10:58 < Talkless> thanks jimpo! 10:58 < jonatack> met laurent mt here and we've been discussing and following along 10:58 < tuxcanfly> ah well... I was just curious if more thought was given to committing cfilters to blocks via coinbase (something similar was mentioned in the btcd/bitcoind comments) 10:59 < fjahr> I thought for a moment the filters in the test might be all 0's because there are no transactions in the blocks but forgot about the coinbase of course... 10:59 < Talkless> Talkless: I believe this PR introduces a way to request a range of filters, so it can start at birthdate 10:59 < Talkless> I wonder this range is defined w.r.t block number, or UTC..? 10:59 < Talkless> if new light wallet is created, how does it define it's birthday? 11:00 < pinheadmz> Talkless: the day it generated the seed :-) 11:00 < Talkless> if it's block number, it has to check.. what..? Ask node whats latest block..? at that time..? 11:00 < Talkless> so the time? 11:00 < pinheadmz> yeah or block height would work 11:01 < jnewbery> Talkless: either is fine. If you generate a new private key randomly, you know for sure that there haven't been any sends to that key before that time 11:01 -!- braydonf [~braydon@gateway/tor-sasl/braydonf] has joined #bitcoin-core-pr-reviews 11:01 < pinheadmz> just to know at what point in the past history there will definitely not be any incoming transacations (bc the addresses didnt esit yet) 11:01 < Talkless> well if wallet knows time, and p2p services would need block indexes, some translation would be needed. 11:01 < jnewbery> whether you store that as a clock time or block number is an implementation detail 11:01 < pinheadmz> jnewbery: do you know how the discussion ended up on the mailing list re: commiting filters to the block? I know it was proposed... 11:01 < Talkless> jnewbery: my question is how would you ask a node a block filters since your birthday? Will node need time, or block index? 11:02 < jnewbery> pinheadmz: I haven't seen any discussion of that recently. I think we'd want non-committed BIP157 filters to be used for a long time before proposing a soft fork to commit to them 11:02 < pinheadmz> Talkless: you provide a start height 11:02 < pinheadmz> https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki#getcfilters 11:03 < pinheadmz> jnewbery: makes sense, might need updates to the protocol before making it consensus 11:03 < Talkless> pinheadmz: ok so if wallet knows UTC time of it's "birthday", how could it transalte it's UTC birthday to the height? 11:03 < pinheadmz> Talkless: ha, ok I guess block height is more relevant in this case :-) 11:03 -!- fox2p [~fox2p@185.9.18.99] has quit [Read error: Connection reset by peer] 11:03 < pinheadmz> although timestamps are commited in the block headers as well 11:03 < jnewbery> Talkless: sync all block headers and look at timestamps (and then add some buffer) 11:03 < pinheadmz> Ok that about does it! 11:03 < miketwenty1> is this the part of the meeting where you let the new guy merge the PR? 11:03 < pinheadmz> Anyone have any lingering questions or comments? 11:04 < pinheadmz> miketwenty1: 😂 11:04 < jnewbery> #action ship it 11:04 < Talkless> jnewbery: how much all block headers take space? 11:04 < Talkless> bandwth? 11:04 < pinheadmz> Talkless: headers are 8 bytes x ~600,000 blocks in the chain 11:04 < pinheadmz> *80 bytes sorry 11:04 < Talkless> 80 yeah 11:04 < Talkless> right, forgot that 11:04 < Talkless> simple math 11:05 < pinheadmz> ok im calling it! 11:05 < pinheadmz> #endmeeting 11:05 < pinheadmz> thanks everyone for hanging out! 11:05 < emzy> ca. 50 MB 11:05 < jnewbery> Thanks for hosting pinheadmz. Great job! 11:05 < emzy> about* 11:05 < tuxcanfly> thanks pinheadmz 11:05 < Talkless> thanks! 11:05 < michaelfolkson> Thanks pinheadmz jnewbery et al 11:05 * pinheadmz headbanging \m/ 11:05 < andrewtoth> thanks! 11:05 < emzy> tnx! 11:05 < jnewbery> I'll try to get notes up for next week before the weekend 11:06 < jnewbery> message me if you want to host! 11:06 < sanoj> Thanks pinheadmz 11:06 < miketwenty1> peace out.. thanks for all that you do 11:06 < Talkless> jnewbery: it would be cool to have ability to download range of headers *around* that time thought :P 11:06 < Talkless> to do kina a linear search of that your height based on your utc birthday 11:07 -!- fox2p [~fox2p@cpe-66-108-32-173.nyc.res.rr.com] has joined #bitcoin-core-pr-reviews 11:07 < pinheadmz> Talkless: well remember, neutriuno clients still have all the block headers, before they even start with filters 11:07 < pinheadmz> so theyd know which blocks were mined when 11:08 < Talkless> no search, just "please give me header of the LAST block which time is less or equel to the X timestamp" 11:08 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has quit [Quit: Sleep mode] 11:08 < Talkless> pinheadmz: oh :/ 11:08 < Talkless> ok so that settles it. 11:09 < Talkless> So right, you dowload all headers, than knowing your wallet's birthday, you can start download filters from the height you calculated from headers. 11:10 < pinheadmz> +1 11:11 < headway-translat> thanks pinheadmz jnewbery 11:12 -!- headway-translat [~headway-t@195.181.160.175.adsl.inet-telecom.org] has left #bitcoin-core-pr-reviews [] 11:12 < jonatack> great discussion, ty pinheadmz 11:12 < Talkless> I wonder, is there a difference, if you sync your lnd node, using your own bitcoin core node, using Neutirno VS whatever current lnd method is? 11:13 < pinheadmz> Talkless: https://github.com/lightningnetwork/lnd/blob/master/docs/INSTALL.md#available-backend-operating-modes 11:14 < pinheadmz> lnd can operate off a full node, or neutrino 11:14 < Talkless> pinheadmz: I knwo, I use full node 11:14 < pinheadmz> ah word! 11:14 < Talkless> I mean is there a advantage of some sort 11:14 < Talkless> if I would start using my own node via neutrino protocol? 11:14 < pinheadmz> the advantage is not needing a full node :-) 11:15 < Talkless> does lnd fetches full blcoks from my node currently, and analyzes using that btcd library? 11:15 < Talkless> or it gets json output of all tx'es in the blocks? 11:15 < pinheadmz> you're running lnd + bitcoind? 11:15 < Talkless> Basically, syncing your lnd with your node using current method, vs neutrino, is there a advatage in either case? 11:15 < pinheadmz> pretty sure bitcoind sends the entire block to lnd 11:15 < Talkless> pinheadmz: yes 11:16 < pinheadmz> the thing is - and we didnt really cover it - 11:16 < Talkless> pinheadmz: ok so theoretically neutrino mode should be faster, less traffic, less cpu time needed to swallow all? 11:16 < pinheadmz> if your wallet matches a filter THEN you need to download that ful lblock! 11:16 < pinheadmz> *block 11:16 < Talkless> yeh, right, now I remember that. 11:16 < pinheadmz> so neutrino is 11:16 < pinheadmz> oops 11:16 < Talkless> but that's not all blcoks, just some. 11:16 < pinheadmz> correct 11:16 < pinheadmz> so if you have a full node available, theres no reason to use neutrino 11:17 < pinheadmz> as far as i know 11:17 < Talkless> So again, neutridno mode would seem more performant, but that's just a guess. 11:17 < Talkless> pinheadmz: I just speculating maybe just less load on initial sync 11:17 < pinheadmz> hmm 11:17 < Talkless> pinheadmz: id does take a two or more days on my arm SBC... 11:17 < Talkless> thanks lnd introduced birthdays! 11:17 < pinheadmz> yeah 11:18 < pinheadmz> i run bitcoind + lnd on a raspberry pi as well :-) 11:18 < Talkless> cool :) 11:18 < pinheadmz> the new Pi4 + 4 GB ram is smokin hot, works great 11:18 < Talkless> oh I would like more ram, 2GB OdroidHC1 has not much ram left for fs caching :/ 11:19 < Talkless> or is it 4 too.. 11:19 * Talkless ssh... 11:19 < Talkless> right, only 2GB 11:20 < Talkless> Anyway, I should try to resync my lnd using neutrino with my own node, and compare results :P 11:20 < pinheadmz> yeah would be interesting to know 11:21 < pinheadmz> I wanna try just using public neutrino servers as well and spin up lnd without a full node 11:25 -!- miketwenty1 [473b2311@c-71-59-35-17.hsd1.ga.comcast.net] has quit [Remote host closed the connection] 11:26 -!- jonatack [~jon@89-93-131-92.hfc.dyn.abo.bbox.fr] has quit [Ping timeout: 276 seconds] 11:27 < jnewbery> meeting log is up: https://bitcoincore.reviews/16442.html 12:09 -!- Talkless [~Talkless@hst-227-49.splius.lt] has quit [Quit: Konversation terminated!] 12:15 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has joined #bitcoin-core-pr-reviews 12:29 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has quit [Quit: Sleep mode] 12:30 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has joined #bitcoin-core-pr-reviews 13:22 -!- emilengler [~emilengle@unaffiliated/emilengler] has quit [Quit: Leaving] 14:02 -!- jonatack [~jon@37.166.80.227] has joined #bitcoin-core-pr-reviews 14:05 -!- andrewtoth [~andrewtot@gateway/tor-sasl/andrewtoth] has quit [Ping timeout: 260 seconds] 14:10 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has quit [Quit: Sleep mode] 14:20 -!- diogosergio [~diogoserg@176.24.23.243] has joined #bitcoin-core-pr-reviews 14:41 -!- pinheadmz [~matthewzi@91.132.137.236] has quit [Quit: pinheadmz] 14:46 -!- diogosergio [~diogoserg@176.24.23.243] has quit [Ping timeout: 276 seconds] 14:57 -!- diogosergio [~diogoserg@176.24.23.243] has joined #bitcoin-core-pr-reviews 15:01 -!- diogosergio [~diogoserg@176.24.23.243] has quit [Ping timeout: 265 seconds] 15:29 -!- slivera [slivera@gateway/vpn/privateinternetaccess/slivera] has joined #bitcoin-core-pr-reviews 15:57 -!- pinheadmz [~matthewzi@91.132.137.236] has joined #bitcoin-core-pr-reviews 16:23 -!- provoostenator [~quassel@provoostenator.sprovoost.nl] has quit [Remote host closed the connection] 16:27 -!- provoostenator [~quassel@provoostenator.sprovoost.nl] has joined #bitcoin-core-pr-reviews 17:53 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has joined #bitcoin-core-pr-reviews 18:04 -!- soju [uid403160@gateway/web/irccloud.com/x-qqvgocvrdemjdsjf] has joined #bitcoin-core-pr-reviews 18:34 -!- jonatack [~jon@37.166.80.227] has quit [Read error: Connection reset by peer] 19:09 -!- felixfoertsch23 [~felixfoer@2001:16b8:5010:a500:9c8d:3c6c:8b6b:5814] has joined #bitcoin-core-pr-reviews 19:11 -!- felixfoertsch [~felixfoer@i6DFA6E9A.versanet.de] has quit [Ping timeout: 252 seconds] 19:11 -!- jasonzhouu [~jasonzhou@2409:8a28:c17:26a0:53bd:22c0:b30a:33f7] has quit [Remote host closed the connection] 19:11 -!- jasonzhouu [~jasonzhou@2409:8a28:c17:26a0:53bd:22c0:b30a:33f7] has joined #bitcoin-core-pr-reviews 19:32 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:9518:d7d3:7541:1eae] has quit [Quit: Sleep mode] 19:45 -!- jasonzhouu [~jasonzhou@2409:8a28:c17:26a0:53bd:22c0:b30a:33f7] has quit [Quit: Leaving] 21:11 -!- MarcoFalke [~none@198.12.116.246] has quit [Ping timeout: 276 seconds] 21:20 -!- felixfoertsch23 [~felixfoer@2001:16b8:5010:a500:9c8d:3c6c:8b6b:5814] has quit [Quit: ZNC 1.7.3 - https://znc.in] 21:20 -!- felixfoertsch [~felixfoer@2001:16b8:5010:a500:4d0d:7e55:e4c:24d5] has joined #bitcoin-core-pr-reviews 21:30 -!- provoostenator [~quassel@provoostenator.sprovoost.nl] has quit [Remote host closed the connection] 21:34 -!- provoostenator [~quassel@provoostenator.sprovoost.nl] has joined #bitcoin-core-pr-reviews 22:43 -!- diogosergio [~diogoserg@176.24.23.243] has joined #bitcoin-core-pr-reviews 22:47 -!- diogosergio [~diogoserg@176.24.23.243] has quit [Ping timeout: 240 seconds] 23:23 -!- ir0nbutt [~LiberLive@144.49.211.130.bc.googleusercontent.com] has joined #bitcoin-core-pr-reviews 23:26 -!- ironbutt [~LiberLive@82.142.166.82] has quit [Read error: Connection reset by peer] 23:27 -!- ironbutt [~LiberLive@82.142.166.82] has joined #bitcoin-core-pr-reviews 23:29 -!- ir0nbutt [~LiberLive@144.49.211.130.bc.googleusercontent.com] has quit [Ping timeout: 245 seconds] 23:30 -!- ir0nbutt [~LiberLive@144.49.211.130.bc.googleusercontent.com] has joined #bitcoin-core-pr-reviews 23:31 -!- ironbutt [~LiberLive@82.142.166.82] has quit [Read error: Connection reset by peer] 23:33 -!- ironbutt [~LiberLive@82.142.166.82] has joined #bitcoin-core-pr-reviews 23:34 -!- ir0nbutt [~LiberLive@144.49.211.130.bc.googleusercontent.com] has quit [Ping timeout: 252 seconds] 23:35 -!- molz [~molly@unaffiliated/molly] has quit [Ping timeout: 244 seconds] 23:40 -!- ir0nbutt [~LiberLive@144.49.211.130.bc.googleusercontent.com] has joined #bitcoin-core-pr-reviews 23:42 -!- ironbutt [~LiberLive@82.142.166.82] has quit [Ping timeout: 252 seconds] 23:46 -!- ironbutt [~LiberLive@82.142.166.82] has joined #bitcoin-core-pr-reviews 23:47 -!- ir0nbutt [~LiberLive@144.49.211.130.bc.googleusercontent.com] has quit [Ping timeout: 240 seconds]