--- Day changed Mon Jan 13 2020
02:01 < kcalvinalvin> ah ok ProveBlock is working fine. One off bug by me
07:13 < adiabat> cool!  works is good.  Slow is not but slow & works is much better than fast & doesn't work
07:13 < adiabat> I'm going to start on network stuff for ibdsim
07:43 < kcalvinalvin> Are you ripping stuff from btcd?
07:43 < kcalvinalvin> I've been thinking that it might be better to just put in utreexo into btcd
08:05 < adiabat> what do you mean ripping stuff?
08:06 < kcalvinalvin> Like are you taking the libraries from btcd? (ex: signature verification lib)
08:09 < adiabat> right now no, not doing that
08:09 < adiabat> ah shoot I merged #49 but it doesn't compile
08:09 < adiabat> ./genproofs.go:56:43: not enough arguments in call to buildOffsetFile
08:09 < adiabat>  have (simutil.Hash, int, map[[32]byte]simutil.RawHeaderData, string, chan bool)
08:09 < adiabat>  want (simutil.Hash, int, map[[32]byte]simutil.RawHeaderData, string, string, chan bool)
08:10 < adiabat> where does currentOffsetHeightFilePath come from...
08:10 < adiabat> seems like it comes later at 84
08:10 < kcalvinalvin> oh that's the offsetfile stuff
08:11 < adiabat> is there a quick fix?
08:11 < kcalvinalvin> ah the extra string is based off the new path thingy
08:11 < adiabat> simutil.HeightFilePath?
08:12 < kcalvinalvin> basically it does filepath.Join() inside the function
08:12 < adiabat> I mean to fix it, if I put simutil.HeightFilePath in the call to buildOffsetFile is that right?
08:12 < adiabat> like that makes it compile
08:13 < kcalvinalvin> one sec
08:14 < kcalvinalvin> currentOffsetHeight, _ = buildOffsetFile(tip, height, nextMap, simutil.OffsetFilePath, simutil.CurrentOffsetFilePath, offsetfinished)
08:14 < kcalvinalvin> that's how I have it
08:14 < adiabat> hm     
08:14 < adiabat> should I revert the PR merge?
08:15 < kcalvinalvin> I could rebase it myself if that's easier
08:15 < adiabat> probably a bunch of differences then
08:15 < kcalvinalvin> I didn't change *that* much
08:15 < adiabat> no I mean the difference between PR and
08:15 < kcalvinalvin> but there are few minor differences here and there
08:16 < kcalvinalvin> yes     
08:16 < adiabat> like I reverted the commit I made
08:16 < adiabat> so it was back to 6faa939
08:17 < kcalvinalvin> Are you mentioning the commit on master or the commit on my PR?
08:18 < kcalvinalvin> ah wait yeah I see
08:18 < kcalvinalvin> I think it's fine. I can rebase onto that commit
08:18 < adiabat> I hate git
08:18 < kcalvinalvin> Why is github telling me about the force-push on master like it's on the PR? odd...
08:18 < kcalvinalvin> I like git. Github is meh haha
08:19 < adiabat> ok I think it's back to the way it was 24 hours ago
08:19 < adiabat> except it's not... ug
08:20 < kcalvinalvin> haha I have a fork of how it was 24 hours ago. Should I push it?
08:20 < adiabat> ok it's on 6faa939b7a60291c2e00abdc58ef961e11fb2272
08:20 < adiabat> githubs UI for PRs is all messed up but whatever
08:21 < kcalvinalvin> Cool. Is there something I need to do?
08:21 < adiabat> how do I delete a PR...?
08:22 < adiabat> the "Add paths for the files" PR didn't work
08:22 < adiabat> when applied to 6faa939b
08:22 < kcalvinalvin> git reset HEAD^
08:22 < adiabat> yeah master is back to working
08:22 < kcalvinalvin> git rebase -i "commit hash"
08:22 < kcalvinalvin> and drop from there
08:22 < adiabat> yeah the "git" ness is OK
08:22 < adiabat> the "github" ness is not
08:22 < kcalvinalvin> lol     
08:23 < adiabat> like how do I delete a PR from the web ui
08:23 < kcalvinalvin> like close my PR?
08:23 < adiabat> no resurrect it
08:23 < adiabat> like I tried to revert it
08:23 < kcalvinalvin> I don't think you can
08:24 < kcalvinalvin> here I'll push again
08:24 < adiabat> see this is why I have to delete everything and start over all the time with git
08:24 < adiabat> no don't push anything
08:24 < adiabat> that just makes it worse
08:24 < kcalvinalvin> I meant make a PR
08:24 < kcalvinalvin> should I not do that...?
08:24 < adiabat> I just want github to go back to how it was an hour ago
08:24 < kcalvinalvin> Oh like put the networking stuff back in?
08:25 < adiabat> no     
08:25 < adiabat> everything back to the way it was 1 hour ago
08:25 < adiabat> master on 6faa939b
08:25 < adiabat> PR #49 open, unmerged
08:25 < adiabat> PR #51 doesn't exist
08:25 < kcalvinalvin> so I think I'll have to push PR #49 again
08:26 < kcalvinalvin> oh ok
08:26 < kcalvinalvin> check out my master
08:26 < kcalvinalvin> it's how you want it
08:27 < kcalvinalvin> github.com/kcalvinalvin/utreexo
08:27 < adiabat> no, that's not what I mean
08:27 < adiabat> the "git" ness is OK
08:27 < kcalvinalvin> ok I'm on jitsi
08:27 < adiabat> but now github has stupid PRs that are broken
08:27 < adiabat> and apparently they won't fix that
08:28 < adiabat> so you have to make a whole new repo
08:28 < kcalvinalvin> ohhhhhhhhh I see
08:28 < kcalvinalvin> Isn't that just a bug
08:28 < kcalvinalvin> I don't think the code has changed?
08:28 < adiabat> https://github.com/mit-dci/utreexo/pull/51
08:29 < adiabat> yeah it's just stupid that there are "PRs"
08:29 < adiabat> and they say "merged"
08:29 < adiabat> but they're not
08:29 < kcalvinalvin> yea I think that's just a Github bug
08:29 < kcalvinalvin> I don't think you can have the merged sign turn into a close sign
08:30 < adiabat> I don't want either
08:30 < adiabat> I want it gone
08:30 < adiabat> like it never existed
08:30 < adiabat> like an hour ago
08:31 < kcalvinalvin> https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/reverting-a-pull-request
08:31 < kcalvinalvin> idk if this solves it
08:31 < adiabat> not revert, delete
08:31 < adiabat> so that there is no PR #51
08:31 < adiabat> anyway, you can open PR #52
08:31 < adiabat> which would just be #49
08:32 < kcalvinalvin> Yeah I don't think github supports that
08:32 < adiabat> that's so dumb, what if someone spams you with thousands of PRs... you have to close them all, but they just stay there?
08:32 < adiabat> you can't delete anything?
08:32 < adiabat> anyway whatever
08:32 < kcalvinalvin> I... think so?
08:33 < kcalvinalvin> there's always gitlab...
08:35 < adiabat> ok anyway so whatever the PR is
08:35 < adiabat> kcalvinalvin/add-paths doesn't compile
08:35 < adiabat> ff8a83c5b419f17f3397c3dd80c0645f45fe9da5
08:37 < kcalvinalvin> based off of 6faa939b7a60291c2e00abdc58ef961e11fb2272 right?
08:37 < kcalvinalvin> huh it's building on my end
08:37 < kcalvinalvin> adiabat: what's the error message?
08:37 < adiabat> ./genproofs.go:56:43: not enough arguments in call to buildOffsetFile
08:40 < adiabat> on mine that's
08:40 < adiabat> currentOffsetHeight, _ = buildOffsetFile(tip, height, nextMap, offsetfile, offsetfinished)
08:40 < adiabat> which, yeah, not enough args
08:40 < kcalvinalvin> hmm ok could you just try PR #50
08:40 < kcalvinalvin> it has all the changes #49 has anyways
08:41 < adiabat> that one compiles
08:42 < adiabat> ok I'll just merge that one then shuffle around the files to work on network
08:42 < kcalvinalvin> hrm weird #49 doesn't have a change on github that *is* on my local branch
08:42 < kcalvinalvin> odd     
08:42 < adiabat> and if it doesn't compile or work, whatever, I'm not dealing with git / github...
08:45 < kcalvinalvin> adiabat does that one work?
08:48 < adiabat> I mean it compiles so good enough :)
08:50 < adiabat> ok so moved file names and stuff, basically for client / server I'll have a server thread running to serve the IBD client thread
08:50 < adiabat> so first they run on the same computer, and it's channels with messages going back and forth
08:51 < adiabat> then when that works, make serialization for those messages and go over TCP
08:51 < kcalvinalvin> ok cool
08:52 < kcalvinalvin> Does this mean that the utreexo lib is stable for now?
08:52 < adiabat> heh, I mean, probably not working on it the next few days stable
08:52 < kcalvinalvin> haha ok
08:52 < adiabat> the *interface* to it should be stable... ish
08:53 < kcalvinalvin> Am eager to port it over to Rust or cpp
08:53 < adiabat> though I'm wondering about positionMap
08:53 < kcalvinalvin> cpp not so much but eh
08:53 < adiabat> there are still lots of TODOs in it.
08:53 < adiabat> hm     
08:53 < adiabat> if you're going to translate though, I should fix one part
08:54 < adiabat> the "floorTransform" ... need to get rid of that, which shouldn't be too bad
08:54 < kcalvinalvin> Isn't the positionMap thing also really implementation dependent?
08:54 < adiabat> then it's like, stable enough to leave for a while
08:54 < adiabat> yeah the positionMap is also like... should that even be in there...?
08:55 < adiabat> maybe that's an external library's problem?
08:55 < kcalvinalvin> imo that's the case
08:55 < adiabat> but it's actually tricky for an external library
08:55 < adiabat> like, how is it supposed to know where things moved
08:56 < adiabat> interface-wise, maybe you DO need it, because like
08:56 < adiabat> "position" is a totally internal thing
08:57 < adiabat> from the point of view of the program using utreexo, it throws things in
08:57 < adiabat> and then gets proofs, and deletes, etc
08:57 < adiabat> but if it's like "hey give me a proof for ffaab1b1", and utreexo "is like, sure, where is it?"
08:58 < adiabat> and upstream is like "I dunno, I put it in yesterday...?"
08:58 < kcalvinalvin> I guess the question is how much should Utreexo handle?
08:59 < kcalvinalvin> I guess the Core people want to treat it like leveldb?
08:59 < adiabat> levelDB definitely gives you a value if you give it a key
08:59 < adiabat> you don't have to give it a disk offset for the internal db
09:00 < adiabat> because that's like... not your problem, that's levelDBs problem which files it uses and where
09:00 < kcalvinalvin> I guess they won't accept something that's gonna break all the RPC calls
09:00 < adiabat> I'm not even thinking core, like
09:00 < adiabat> even if you moved positionMap
09:01 < adiabat> how would you update it?  what would Modify() return that let you update it...?
09:01 < adiabat> it's pretty tied in there
09:01 < kcalvinalvin> what do you mean by moved
09:01 < adiabat> I mean if you get rid of positionMap
09:02 < adiabat> then the calls to Prove/ ProveBlock change
09:02 < adiabat> right now it's func (f *Forest) Prove(wanted Hash) (Proof, error) 
09:02 < kcalvinalvin> Yeah I guess then it breaks
09:02 < adiabat> but without positionMap, you can't do that, you have to ask for a position, not a hash
09:03 < adiabat> so the code asking for the proof needs to know the position
09:03 < adiabat> so you need a positionMap somewhere
09:03 < adiabat> it'd just be in ibdsim somewhere
09:03 < adiabat> but... if it's somewhere else
09:03 < kcalvinalvin> leveldb seems perfect haha
09:03 < adiabat> there's a lot of calls to it
09:03 < adiabat> perfect for what?
09:04 < kcalvinalvin> replacement for positionMap
09:04 < adiabat> no...
09:04 < adiabat> we don't need to replace it
09:04 < adiabat> we don't need to put it on disk; it can be generated
09:05 < kcalvinalvin> but what about the rev stuff?
09:05 < adiabat> probably need levelDB for that
09:05 < adiabat> but maybe only for weird outputs
09:06 < kcalvinalvin> I did mean to ask again. Why only for weird outputs?
09:06 < adiabat> normal ones you only need a tiny amount of data
09:07 < adiabat> probably fits in 8 bytes
09:07 < adiabat> so can put it along side the forest probably
09:07 < kcalvinalvin> You don't mean normal == standard tx
09:07 < adiabat> yeha
09:07 < adiabat> well
09:07 < adiabat> not standardness but
09:07 < adiabat> "output standardness"
09:08 < adiabat> being p2pkh, p2sh, or segwit
09:08 < adiabat> if any of those, all the extra data needed fits in less than 16 bytes
09:09 < adiabat> so could just have another on disk data store, like for the leaves
09:09 < adiabat> or make the base of the forest 48 bytes instead of 32 bytes or whatever
09:09 < kcalvinalvin> so output > 16 bytes; mark as weird?
09:09 < adiabat> ?     
09:09 < adiabat> what's weird?
09:09 < kcalvinalvin> like "weird" as in bigger than your normal tx right
09:10 < adiabat> no     
09:10 < adiabat> a "weird" output is just one that's not p2pkh, p2sh, segwit
09:10 < adiabat> and so you need to keep more data about it, in some db or whatever
09:11 < adiabat> regular outputs though, you need very little extra data
09:11 < adiabat> you need *some*, but mostly the amount
09:11 < adiabat> which is 8 bytes max
09:11 < adiabat> so can store that in a similar structure as the forest as it's fixed length and small
09:12 < adiabat> anyway got to go... meetings n stuff
09:12 < kcalvinalvin> wouldn't you be able to make big multisig stuff with p2sh?
09:23 < kcalvinalvin> ah ok we only care about outputs
09:45 < adiabat> well, care about both but with p2sh you get the preimage when spending
09:45 < adiabat> so you can just hash that, see if it fits with the leaf given, and if it does, you're good
09:46 < adiabat> even though you "forgot" what the output's address actually was
10:33 < kcalvinalvin> ok so I'll make that script and then work on pollard resume
15:13 < adiabat> ok for pollard resume for now, I'd say only use tree tops, don't bother writing the whole pollard state to disk