--- Day changed Wed Jan 08 2020
01:32 < kcalvinalvin> I tried benchmarking up to the blk00002.dat file. The disk based version is 4% slower
01:32 < kcalvinalvin> pretty good
01:41 < kcalvinalvin> oh wait nvm I was looking at the wrong thing. It's 74% slower
01:41 < kcalvinalvin> not so good
01:42 < kcalvinalvin> I'm testing with an external ssd through usb 3.1
01:43 < kcalvinalvin> I guess it'll be real bad once you get to those heavy shuffles
09:05 < adiabat> kcalvinalvin: yeah there are definitely ways to make it faster.  Right now it uses "floor transform" which is...
09:06 < adiabat> simpler I guess for the forest remove method, but much slower because each individual 32 byte piece is swapped separately
09:07 < adiabat> and since it's a swap, it's like, read position a, read position b, write position b, write position a
09:07 < adiabat> for a big shuffle those are far away, and it does them all individually
09:08 < adiabat> the right way to do it, which might not be too hard, would be to move the whole chunk at once.  can take a lot of ram thogh
09:08 < adiabat> move at least up to a few megabytes at once; read positions a ... a+4096, then read b ... b+4096, then write b, then write a
09:10 < adiabat> hm.  I'm also thinking this is a bit of a mess for crashes / poweroff consistency... but can deal with that way later
09:32 < kcalvinalvin> I'm thinking maybe it'll be easier if genproofs flushes to disk once it fills up a certain amount of bytes
09:32 < kcalvinalvin> maybe have it like 128MB threshold and flush once it reaches
09:32 < kcalvinalvin> adiabat also, are you free sometime today?
09:53 < kcalvinalvin> Ok I guess I'll write it down here in case I forget.
09:54 < kcalvinalvin> I'm was pondering about merging to core today (again) and I think putting in the caching part is the logical step in doing so.
09:55 < kcalvinalvin> I really don't see any reason why Core would deny it since it doesn't assume any more security assumptions
09:55 < kcalvinalvin> It doesn't seem *that* hard to get it working on Core code too
09:57 < kcalvinalvin> I guess the question is if Core would accept clairvoyent version of the caching since it's uglier to get it working
09:57 < kcalvinalvin> Second was the PKscript thingy. I'm trying to figure out what isn't a "weird" tx. I came up with:
09:59 < kcalvinalvin> P2PK, P2PKH, P2SH, P2WPKH, P2WSH, and non-P2SH multisig
10:01 < kcalvinalvin> I guess it's easier to pick out what *is* a "weird" transaction and am a little tripped up about that
11:55 < adiabat> what do you mean about merging caching in to core?
11:55 < adiabat> like... including all caching with utreexo vs merging in a version that doesn't have any caching?
11:56 < adiabat> as for weirdness, I'd say non-P2SH multisig is weird
11:56 < adiabat> or at least, definitely non-recoverable
11:56 < adiabat> Maybe p2pk also counts... it's weird now, and I think unrecoverable
11:56 < adiabat> p2pkh, p2sh, p2wpkh and p2wsh are all OK
11:57 < adiabat> also op-return is "OK" in that it doesn't count as a utxo 
11:57 < adiabat> I think other than that, it'd have to be all the data stored
11:57 < adiabat> p2pk might not...?  not sure about that one
12:01 < kcalvinalvin> adiabat so don't include p2pk and non-p2sh multisig?
12:32 < adiabat> don't include them in the normal list I guess yeah
12:32 < adiabat> I'm really not sure about p2pk, maybe keep a separate count of those
12:32 < adiabat> guessing they're all in 2009/2010
12:33 < kcalvinalvin> ok sounds good. 
12:33 < kcalvinalvin> btw what are you working on currently?
12:34 < kcalvinalvin> I'm making some changes to forest stuff (and trying to fix hdmi on my chrombook gahhh)
13:51 < adiabat> right now still bug fixing undo branch.  should be almost done
13:51 < adiabat> then I guess will do network messages...
15:31 < adiabat> ok couple 100K runs & not finding any bugs, so maybe undo branch is good to go
18:29 < adiabat> OK merged everything into master.  Maybe see if it's slower / faster?  Probably slower... but it sets things up for being faster
18:29 < adiabat> also I think it still works but only tested for a few minutes