--- Log opened Mon May 04 00:00:11 2020
06:56 < adiabat> yeah it's a dedupe problem
06:56 < adiabat> almost have it fixed
06:56 < adiabat> (says in the commit message)
06:57 < adiabat> basically dedupe hashes shouldn't be in accumulator, as it's pretty bitcoin specific, and genProofs should catch that before calling Modiyu
06:57 < adiabat> *Modify
07:36 < adiabat> ok fixed and pushed to master
07:36 < adiabat> should work now, seems to work OK for me
07:47 < adiabat> ... ok except ibdsim doesn't work at all.  looking at that
07:52 < adiabat> ok fixed
07:52 < adiabat> cool now network PR.  Those last few should have been in their own PR I guess but oh well
07:52 < adiabat> also maybe I'll first merge the doc PR
07:53 < adiabat> also ibdsim is way faster than genproofs.  Which makes sense but feels like maybe too big of a difference
07:53 < adiabat> maybe there was some performance hit to genproofs somewhere.  anyway can look at later
09:43 < adiabat> mmm no it's working because it's never deleting anything.  Maybe 1 more fix to master...
10:08 < adiabat> heh yeah as far as I can tell there's no reader for proof data... so the IBDsim client never read the proofs that genproofs made
10:10 < kcalvinalvin> yeah... I've been debugging for the better part of yesterday afternoon
10:11 < kcalvinalvin> Code changed a lot haha
10:18 < adiabat> I'm changing a lot now, merging pFileWorker and pOffsetFileWorker
10:19 < adiabat> writing a proof reader.  I thought that existed...?
10:19 < adiabat> but I don't see anything that reads proof.dat / proofoffset.dat
11:49 < kcalvinalvin> There isn't a proof reader per se but ibdsim reads it during when it syncs
11:54 < kcalvinalvin> adiabat is there a name for forest.height that you thought of? 
11:55 < kcalvinalvin> I'll push the changes that ja asked for in the morning.
12:02 < kcalvinalvin> hmm ok the new changes seem to have fixed the issue I was running into today
12:12 < adiabat> where does IBDsim read proofs..?
12:12 < adiabat> I might have deleted it
12:13 < adiabat> there's no "height" anywhere in forest / pollard now, it's just rows
12:14 < adiabat> I couldn't find anywhere where IBDsim actually read from proof files or offsets or anything.
12:14 < adiabat> Possible I accidentally removed it a while ago.  Adding it in now
12:18 < kcalvinalvin> hmmm maybe I should spend an afternoon and make a docker build for testing or something...
12:18 < kcalvinalvin> would be more helpful in speeding things along
12:18 < adiabat> ah pFile was read like a month ago
12:18 < adiabat> I probably got rid of that accidentally, which made ibdsim work great!
12:18 < adiabat> because it never had to verify any proofs :)
12:19 < adiabat> anyway adding back in
12:19 < kcalvinalvin> oh ok. Hmmm.. Do you think leaving forest.height the same makes sense
12:19 < kcalvinalvin> because like... it isn't
12:20 < adiabat> huh?
12:20 < adiabat> where?
12:21 < adiabat> you'd rather it go back to forest.height instead of forest.rows?
12:21 < kcalvinalvin> Wait hold up maybe it isn't there anymore
12:21 < adiabat> the only "height" in all of accumulator is for the simchain testing thing
12:21 < kcalvinalvin> oh
12:21 < kcalvinalvin> noice
12:21 < kcalvinalvin> :)
12:21 < adiabat> yeah... ok I will get IBDsim reading proofs again, THEN can do network
12:22 < adiabat> I changed the proofs, so now it's all the rev data AND proof hashes
12:22 < adiabat> I guess that change is when I made it stop reading proofs entirele
12:22 < adiabat> *entirely
12:22 < kcalvinalvin> well urm.. I guess I could look at more documentation stuff
12:23 < kcalvinalvin> Would be tad nicer if master didn't break so often.. :)
12:25 < kcalvinalvin> Which... I'm guilty as well... I'll def look into more CI stuff
12:40 < adiabat> well none of the *tests* broke
12:40 < adiabat> we don't have any tests for cmd/, which is I guess the issue
12:40 < adiabat> accumulator/ which has tests had pretty much worked fine
12:41 < adiabat> I guess an automated sim-chain kind of thing for genproofs / ibdsim would be the way to test cmd side
12:44 < kcalvinalvin> The official go documentation says it's ok to add files go sole testing purposes
12:45 < kcalvinalvin> btcd does this too from what I see. What do you think about just hardcoding in the rev and blk files?
12:45 < kcalvinalvin> not hardcoding but having it in the repo
12:48 < adiabat> thing is they're huuuge
12:48 < adiabat> I think a simulator wouldn't be that hard
12:48 < adiabat> can either have a simulator that makes blk / rev, or one that creates a regtest and makes commands to bitcoind
12:49 < adiabat> and then goes through the blk/rev that generates
12:49 < adiabat> I like randomized also because ... we know it does OK with the couple gigs of testnet3 data, but what if it dies on some other weird thing...
12:50 < adiabat> a simulator might be able to find that, or like weird numbers of leaves, etc
21:23 < adiabat> man, yeah we thought things were working, but everything outside of accumulator was broken
21:23 < adiabat> just... no tests to fail I guess
21:24 < adiabat> anyway still doesn't work but I'm done for now... everything I fix just uncovers the next problem heh
23:27 < kcalvinalvin> oof
--- Log closed Tue May 05 00:00:11 2020