--- Log opened Thu Sep 30 00:00:22 2021
00:46 -!- adiabat [~adiabat@63.209.32.102] has quit [Ping timeout: 252 seconds]
00:47 -!- adiabat [~adiabat@63.209.32.102] has joined #utreexo
02:54 -!- chuck_ [~chuck@116.193.133.8] has joined #utreexo
02:55 -!- chuck_ [~chuck@116.193.133.8] has quit [Client Quit]
09:33 < kcalvinalvin> Should ask but would it be better to just send the entire rev block over for the CSNs
09:33 < kcalvinalvin> And have the CSNs shift through it to get rid of same block spends
09:33 < kcalvinalvin> Since we're already making them do it for normal blocks
09:34 < kcalvinalvin> So I don't think it really takes up extra cpu
09:34 < adiabat> OK wait why
09:34 < kcalvinalvin> But it'd be a bit more data since the rev blocks aren't deduped
09:34 < adiabat> right now, are we sending leaf data about same block spends?
09:34 < kcalvinalvin> No we're not
09:35 < kcalvinalvin> But if we just straight up send the rev block over, we'd be sending leaf data for the same block spends
09:35 < adiabat> So the idea is it would save time from the server side, as it would just never generate leaf data at all?
09:35 < kcalvinalvin> Mostly space
09:36 < kcalvinalvin> But I guess a bit of cpu during proof generation as well
09:36 < kcalvinalvin> You can still make the server dedupe out the rev block before sending it over
09:36 < kcalvinalvin> but it'd be a lot of cpu
09:36 < adiabat> space in that the server right now has .rev blocks AND the leaf data which is redundant
09:36 < kcalvinalvin> Yes
09:37 < adiabat> OK got it... I guess in general I don't think it's worth it because
09:37 < adiabat> you're basically pushing more work onto the CSNs so the server does less work
09:37 < adiabat> but if the serer does the work once, then many CSNs can benefit
09:38 < kcalvinalvin> But the server has to keep a duplicate of the rev blocks
09:38 < adiabat> I guess if we wanted to get fancy, we could *replace* .rev files with leaf data, or something directly consumable by CSNs
09:38 < kcalvinalvin> We could and it work
09:38 < kcalvinalvin> Not sure if that'd pass by core
09:38 < adiabat> but even now, .rev is not huge
09:38 < adiabat> it's like, 10%?
09:38 < adiabat> i'll check
09:38 < kcalvinalvin> Heh should check
09:39 < kcalvinalvin> 47GB for me
09:39 < kcalvinalvin> So not inconsequential 
09:39 < adiabat> so more like 20%
09:40 < adiabat> but that's 20% compared to the blocks
09:40 < adiabat> with a bridge node there's also that whole proof thing, which is even bigger
09:40 < kcalvinalvin> Well
09:40 < kcalvinalvin> we don't know
09:40 < kcalvinalvin> heh cuz we only have the numbers for non-compact udata
09:40 < adiabat> right ideally we get disk stored proofs well under the blocks size
09:41 < kcalvinalvin> I should sync up mainnet overnight and check the numbers
09:41 < kcalvinalvin> Should be much better now with compact udata
09:41 < adiabat> My thinking is... CSN "experience" is the most important
09:41 < adiabat> CSNX?
09:41 < adiabat> like, gets data quick, small download, easy to verify, etc
09:42 < adiabat> and BNX (bridge node experience) is not as important
09:42 < kcalvinalvin> For testnet, 32GB block data, 8.8GB compact proofs, 5.4G undoblocks
09:42 < adiabat> I mean it's still important, if a bridge node needs some huge server people won't run it
09:42 < adiabat> hm.  another idea is you can just prune undo blocks
09:42 < adiabat> Like you're never going to use the old ones
09:43 < kcalvinalvin> They can be made smaller right?
09:43 < kcalvinalvin> Should look at that
09:43 < adiabat> the undo blocks themselves?
09:43 < kcalvinalvin> yes
09:43 < adiabat> could I guess but really
09:43 < kcalvinalvin> adiabat: lol at CSNX
09:43 < adiabat> if you just "rm rev00000.dat"
09:44 < adiabat> it might run fine
09:44 < adiabat> well... probably have to change a little code but really
09:44 < kcalvinalvin> bitcoind does checks for that
09:44 < adiabat> you're never going to *use* rev00000.dat
09:44 < kcalvinalvin> lol yes
09:44 < kcalvinalvin> Should nodes support reorging back to genesis?
09:44 < adiabat> so if we want to save space, we can just prune most of the rev files
09:45 < adiabat> what happens in bitcoind if a pruned node needs to reorg to before it has rev blocks?
09:45 < adiabat> I think we've asked that a few times and haven't tried it
09:46 < kcalvinalvin> Crashes..?
09:46 < adiabat> I know that in practice, none of the software will actually be able to reorg back to genesis
09:46 < adiabat> so having rev blocks back to genesis doesn't do anything
09:46 < adiabat> though actually, *we* use the rev data to build proofs
09:46 < kcalvinalvin> Like Core is pretty adamant about user having to do something if a node crashes so I'm guessing it errors out and tells you to reindex
09:47 < adiabat> utreexo is likely the largest (or only?) consumer of rev___.dat files :)
09:47 < kcalvinalvin> Also, rev blocks are now used to calculate fees or seomthing
09:47 < adiabat> really?
09:47 < adiabat> is that a new thing in bitcoind?
09:47 < kcalvinalvin> Ok I don't think it was fees but
09:48 < kcalvinalvin> It is now being used for something
09:48 < kcalvinalvin> yeah it's new
09:48 < adiabat> huh.  OK well we should look at that, if things are actually using rev00000.dat we can't delete it
09:48 < adiabat> Pruned nodes don't have rev00000.dat though and they seem to do OK
09:49 < adiabat> maybe we can make a function to regenerate rev blocks from the leaf data
09:49 < kcalvinalvin> Also, might not yet be actually merged
09:49 < kcalvinalvin> Can't find the PR right now :(
09:49 < adiabat> that would be slow, but also seems OK to be slow in something that never happens
09:49 < kcalvinalvin> > maybe we can make a function to regenerate rev blocks from the leaf data
09:49 < kcalvinalvin> Spicy
09:49 < kcalvinalvin> I like this
09:50 < kcalvinalvin> Ok for utcd
09:50 < kcalvinalvin> Would be terribly hard for bitcoind
09:50 < kcalvinalvin> (For bitcoin devs to accept it)
09:50 < adiabat> bitcoind who knows, maybe people want to keep rev00000.dat "just in case"
09:51 < adiabat> it's a waste of space but it's not huge, and it's on the bridge node size so I don't think it's the most urgent thing
09:51 < adiabat> but yeah it is kind of annoying to have 40+ GB that does nothing
09:53 < kcalvinalvin> Ok so I won't bother with it for now
09:53 < kcalvinalvin> but it is an improvement we can make
09:54 < adiabat> yeah definitely.  And to an extent it's a general bitcoin improvement, not just a utreexo improvement
09:54 < adiabat> every non-pruned node has 47GB of pretty much useless data
09:54 < adiabat> the first 45GB or so are useless I guess
09:56 < kcalvinalvin> Could also dedupe rev
09:56 < kcalvinalvin> Would have a lot of space
09:56 < adiabat> the rev blocks also store weird hashes that nothing uses right
09:57 < kcalvinalvin> yes
09:57 < kcalvinalvin> 32 bytes per rev block
10:11 -!- adiabat_ [~adiabat@63.209.32.102] has joined #utreexo
10:17 -!- Netsplit *.net <-> *.split quits: adiabat
12:08 -!- r-ush [~r-ush@52.187.184.81] has quit [Quit: ZNC 1.7.5+deb4 - https://znc.in]
12:09 -!- r-ush [~r-ush@52.187.184.81] has joined #utreexo
13:03 -!- pigeons [~pigeons@androzani.sysevolve.com] has quit [Ping timeout: 240 seconds]
13:04 -!- takinbo [~takinbo@user/takinbo] has quit [Ping timeout: 250 seconds]
13:06 -!- takinbo [~takinbo@user/takinbo] has joined #utreexo
13:07 -!- dunxen [~dunxen@gateway/tor-sasl/dunxen] has quit [Read error: Connection reset by peer]
13:09 -!- pigeons [~pigeons@androzani.sysevolve.com] has joined #utreexo
23:14 -!- Netsplit *.net <-> *.split quits: pigeons
23:16 -!- Netsplit over, joins: pigeons
23:26 -!- pigeons_ [~pigeons@androzani.sysevolve.com] has joined #utreexo
23:26 -!- pigeons [~pigeons@androzani.sysevolve.com] has quit [Ping timeout: 252 seconds]
--- Log closed Fri Oct 01 00:00:23 2021