--- Log opened Mon Feb 06 00:00:36 2023
00:01 < josie> the downside of using concatenation is we would need to give more data to the light client, so I think the speedup would have to be significant
00:07 < josie> regarding the outpoint hash, I think we could safely truncate this to 8 bytes. this means we would only have to give (8 bytes + 32 bytes) per tx to light clients if we find that concatenation is significantly faster than multiplication
00:10 < josie> how I came up with 8 bytes is: 64 bits means we can hash 6.1 x 10^6 items before getting a p = 10^-6 probability of at least one collision (https://en.wikipedia.org/wiki/Birthday_problem#Probability_table)
00:17 < josie> so someone would have to make roughly 6M payments to the same silent address using the same set of pubkeys for each transaction before having a 10^-6 chance of generating the same destination address again
00:22 < josie> I checked UTXO counts per address and the biggest one is 1.6M, followed by ~400,000, ~200,000 and then ~30,000 for everything less than the top 50, so it feels like it would be pretty difficult to make 6M payments using the same set of pubkeys
00:53 < josie> RubenSomsen: looking at bip158, the false positive rate is 1/784931, so lets say we have 2500 txs in a block and each tx has a p2tr unspent output. if the light client supports 5 labels per SP address, we would be checking 12500 outputs per block. so we would have 12500 * 1/p = 1.6% chance of getting a false positive in the block?
00:56 < josie> doesn't seem terrible to me, considering the actual number of txs in a block with at least one unspent p2tr will be much lower
04:54 <@RubenSomsen> josie: not sure where you landed on P2PK, P2MS, but ideally we allow for paying with a wide variety of inputs. I agree this probably isn't trivial. No matter what the script, the recipient basically has to figure out which keys are being signed with (i.e. interpret the script) and use those to generate the tweak.
05:00 <@RubenSomsen> Re: outpoint hash, thanks for crunching the numbers. Probably good to be conservative but happen to know how much worse it gets with 4 bytes?
05:04 <@RubenSomsen> Re: bip158 false positive rate, not bad, but if a light client has to download 1.6% of all blocks that still seems somewhat non-ideal. This is an implementation detail that won't affect the spec though, so for now probably fine?
05:06 < josie> re: p2pk,p2ms my original motivation was to limit the protocol to inputs where we can get the pubkey from the scriptSig. but since taproot outputs would also require us to get the prevout for the pubkey, we would still need logic for getting the prevouts. seems better to not exclude them?
05:09 < josie> re: 4 bytes, its pretty bad haha. all it would take is 93 payments from the same set of pubkeys for a p = 10^-6 chance of a collision. ~10,000 payments for a 1% chance of collision. while unlikely, it doesn't seem crazy that a large exchange might make 10k payments to the same silent payment address
05:13 < josie> re: bip158 I used the "worst case" scenario, which assumes that light clients are scanning for every tx, so the 1.6% would only be true once taproot adoption is at 100%. this tells me bip158 is certainly feasible for the short term (1 - 3 years), which also gives us plenty of time to research alternatives for light clients to request the data. and yes, doesn't affect the spec at all, more
05:13 < josie> just making sure its possible to build a light client with the tools that exist today
10:29 < w0xlt> ---
10:30 < w0xlt> I think the address reuse prevention variant `X' = hash(x*hash(txid1,index1,txid2,index2,..)*I)*G + X` might be incompatible with UTXO Set scan.
10:30 < w0xlt> UTXO Set has only the structures `COutPoint` and `Coin`, in which it is not possible to obtain all (txid,index) of the transaction.
10:30 < w0xlt> Currently, it doesn't matter because the index returns the sum of the public key according to the transaction id (which is available in the UTXO Set).
10:31 < w0xlt> If the index stores `hash(txid1,index1,txid2,index2,..)*sum_pub_key` instead of just `sum_pub_key` it might work, but it's not just a pubkey sum anymore, but also multiplication of a pub key by a scalar, which I presume is expensive.
10:45 < josie> w0xlt: yeah, I think the index would need the hash of the outpoints, which makes it more expensive to build the index
10:48 < josie> I'm working on a branch now which doesn't use the index to run some benchmarks. My guess is we might not need the index for normal node operation, but would need an index if the node plans to offer light client data
11:00 < w0xlt> josie: Great ! But I think `scantxoutset` RPC requires either an index or an unpruned node (to find the transactions).
11:01 < w0xlt> But I wonder if looking up for each transaction for each UTXO is feasible.
11:10 < w0xlt> The version without the index might not have much impact for receiving and sending silent payments normally. The problem is precisely in `scantxoutset`.
11:22 < josie> w0xlt: yep, you're right about `scantxoutset` needing a full node or an index. in that case, we could say you need to run a full node to recover your silent payment wallet, or we have the index optional for the pruned node use case and build the index during IBD
11:24 < josie> if you're already running a full node, I don't think you'd ever want to use `scantxoutset` anyways, since if you're recovering, you might as well recover the full wallet history 
11:26 < w0xlt> Yes, it makes sense.
11:28 < w0xlt> I think PR currently does not use index to receive silent payment normally. It expects the inputs to already be in UTXO Set . If it can't get them from the UTXO Set, then it uses the index.
11:28 < w0xlt> So it might work for the benchmark.
11:41 < w0xlt> https://github.com/w0xlt/bitcoin/blob/7a93c808a217fedc0199d4336588f28212dedeeb/src/wallet/wallet.cpp#L1375-L1387
11:54 < josie> w0xlt: ah nice! I didn't realize `FindPreviousCoin` was querying the UTXO set
11:57 < josie> right now, I'm trying to get some numbers for `rescanblockchain` and `importdescriptors`. at this point, I'm fairly convinced it will be just as fast without an index for a full node since the only way to get the prevouts for historic transactions is to use the block undo data, which is exactly what the index is doing
13:50 -!- gnusha_ [~gnusha@user/gnusha] has joined #silentpayments
13:50 [Users #silentpayments]
13:50 [@RubenSomsen] [ gnusha_    ] [ josie  ] [ LarryRuane] [ sipa ] 
13:50 [ gnusha     ] [ jesseposner] [ kanzure] [ Murch1    ] [ w0xlt] 
13:50 -!- Irssi: #silentpayments: Total of 10 nicks [1 ops, 0 halfops, 0 voices, 9 normal]
13:51 -!- kanzure_ [~kanzure@user/kanzure] has joined #silentpayments
13:51 -!- Channel #silentpayments created Tue Oct 18 06:05:11 2022
13:54 -!- Irssi: Join to #silentpayments was synced in 204 secs
13:56 -!- Netsplit *.net <-> *.split quits: gnusha, kanzure
17:19 < w0xlt> josie: Interesting. If we can remove the index, the code and UX are simpler. But it's curious that using the undo data on each scan has the same performance as the index.
17:19 < w0xlt> --
17:19 < w0xlt> I think the index doesn't need to store `hash(txid1,index1,txid2,index2,..)*sum_pub_key`. It can send `<sum_pub_key1><hash1(txid1,index1,txid2,index2,..)><sum_pub_key2><hash2(txid1,index1,txid2,index2,..)>...` to the client.
17:19 < w0xlt> The client already needs to perform an ECDH, so it can do `(i*hash)*sum_pub_key`, keeping just one EC multiplication per output.
17:20 < w0xlt> And that way, no extra expensive operations are added in node.
17:30 -!- kanzure_ is now known as kanzure
19:33 < w0xlt> Example:
19:34 < w0xlt> https://www.irccloud.com/pastebin/QlmFAe0I/
19:39 < w0xlt> It seems that the number of candidate transactions on mainnet has increased. Ordinals effect? In that case, perhaps a more refined `scriptPubKey` filter could be considered. Currently the node only checks if the type is `TxoutType::WITNESS_V1_TAPROOT`.
22:26 < w0xlt> I updated the PR. It now uses the address reuse prevention variant and support for light clients is implemented as described above(`<sum_pub_key1><hash1(txid1,index1,txid2,index2,..)><sum_pub_key2><hash2(txid1 ,index1,txid2,index2,..)>...`) without adding any further EC multiplication.
22:27 < w0xlt> Thus, this approach can be tested.
22:27 < w0xlt> I've also added a new `wallet_silentpayment_addr_reuse.py` functional test that checks that addresses derived from silent payments are unique.
--- Log closed Tue Feb 07 00:00:37 2023