--- Day changed Wed Sep 21 2016
00:18 <@waxwing> mustyoshi: i'll be back starting a few hours from now, just ping me again whenever you have a chance, thanks
00:21 -!- fqtw__ [~me@x590ce61b.dyn.telefonica.de] has joined #joinmarket
00:24 -!- fqtw_ [~me@x590e1139.dyn.telefonica.de] has quit [Ping timeout: 260 seconds]
00:36 -!- fqtw_ [~me@x590d5004.dyn.telefonica.de] has joined #joinmarket
00:39 -!- fqtw__ [~me@x590ce61b.dyn.telefonica.de] has quit [Ping timeout: 265 seconds]
00:39 -!- fqtw [~me@x590d2246.dyn.telefonica.de] has joined #joinmarket
00:42 -!- fqtw_ [~me@x590d5004.dyn.telefonica.de] has quit [Ping timeout: 244 seconds]
00:52 -!- fqtw_ [~me@x590c1580.dyn.telefonica.de] has joined #joinmarket
00:55 -!- fqtw [~me@x590d2246.dyn.telefonica.de] has quit [Ping timeout: 244 seconds]
01:04 -!- molz [~molly@unaffiliated/molly] has joined #joinmarket
01:07 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 250 seconds]
01:16 -!- HeySteve [~Lizard__W@unaffiliated/heysteve] has joined #joinmarket
01:38 -!- mrkent [~textual@unaffiliated/mrkent] has quit []
03:50 -!- coins123 [~coins123@unaffiliated/coins123] has quit [Read error: Connection reset by peer]
03:50 -!- coins123_ [~coins123@ip-244-225.sn1.clouditalia.com] has joined #joinmarket
04:52 < GithubBot5678> [joinmarket] AdamISZ pushed 1 new commit to develop: https://git.io/viHVr
04:52 < GithubBot5678> joinmarket/develop 93dfefc Adam Gibson: remove random_nick including from add-on scripts
05:07 -!- coins123_ [~coins123@ip-244-225.sn1.clouditalia.com] has quit [Read error: Connection reset by peer]
05:07 -!- coins123 [~coins123@unaffiliated/coins123] has joined #joinmarket
05:07 -!- coins123 [~coins123@unaffiliated/coins123] has quit [Remote host closed the connection]
05:56 -!- fqtw__ [~me@x4db1ca68.dyn.telefonica.de] has joined #joinmarket
05:59 -!- fqtw_ [~me@x590c1580.dyn.telefonica.de] has quit [Ping timeout: 255 seconds]
06:00 -!- fqtw_ [~me@x590e3273.dyn.telefonica.de] has joined #joinmarket
06:03 -!- fqtw__ [~me@x4db1ca68.dyn.telefonica.de] has quit [Ping timeout: 240 seconds]
06:18 -!- viasil [~viasil@198.12.75.25] has quit [Ping timeout: 240 seconds]
06:19 -!- viasil [~viasil@198.12.75.25] has joined #joinmarket
06:26 < mustyoshi> waxwing, okay
06:26 <@waxwing> mustyoshi: hi, here now
06:26 < mustyoshi> same, so have you thought of any reason it fails for me
06:26 <@waxwing> i wonder what we could do, hm.
06:27 < mustyoshi> Like I said, I have two versions running off the same wallet, but the other one reports the correct balance
06:27 <@waxwing> so when you said "loads of errors" in that last message, you meant not errors but wrong balances i guess?
06:27 < mustyoshi> Yeah
06:27 <@waxwing> ok, can you see the balance reports side by side at all?
06:28 < mustyoshi> Sure give me a bit to run it
06:28 <@waxwing> i'm wondering if we can get any clues. other ideas would involve mostly sending you a copy of a hacked blockchaininterface.py with extra debug statements in it. i'll do that now.
06:29 <@waxwing> i'd ask you to do bitcoin-cli listaddressgroupings and pipe it to file, but i don't see much point in doing that directly.
06:31 <@waxwing> btw i did tests with exactly your index_cache, where i did a sync from scratch and then with fast, it seemed to work fine, there were 31k+ imports in the first phase, took about 15 minutes maybe (on regtest,but still)
06:31 <@waxwing> i threw a bunch of coins in various locations throughout the mixdepths
06:32 < mustyoshi> So there must be something wrong with my personal setup. Would restarting my node with --reindex maybe fix it?
06:32 <@waxwing> well .. but it works the slow way? i'm pretty sure that's not necessary
06:33 <@waxwing> you have all the addresses imported into your joinmarket-XXXXX wallet.
06:33 <@waxwing> and you are able to see the unspent coins.
06:33 < mustyoshi> The normal version just completed, correct balance.
06:34 < mustyoshi> Yep, your version still gives incorrect balance even without --fast
06:34 <@waxwing> this is wallet-tool ?
06:34 < mustyoshi> yes
06:34 <@waxwing> what does `git log` show as the last commit?
06:35 < mustyoshi> commit ffcd3044fa350fae612a6785f5c0b6b826ec4e8d
06:35 < mustyoshi>     add fast sync option for Core wallets
06:36 <@waxwing> btw they're both my version :) i updated it before this
06:36 <@waxwing> hmm i am getting completely lost then.
06:36 <@waxwing> i see none of this....
06:37 < mustyoshi> I'll reindex my node
06:37 <@waxwing> it can't be that, if another version of the code works
06:37 <@waxwing> we never need that anyway
06:38 < mustyoshi> Is it testing for index_cache and acting on it even without --fast?
06:38 <@waxwing> we need to isolate this: running off that commit, you get a wrong balance even without --fast, in the same directory or different?
06:38 < mustyoshi> Same directory
06:38 <@waxwing> mustyoshi: no --fast will only activate if both --fast and index_cache
06:38 < mustyoshi> The other version (different directory, different copy of wallet file) works fine.
06:38 <@waxwing> but actually you can see that from "detailed wallet history" etc - that means not fast
06:39 <@waxwing> mustyoshi: can you compare the wallet files.
06:39 <@waxwing> i guess there are at least 3, and possibly 4 cases to consider here, you have 2 directories and we have 2 wallet sync modes.
06:40 < mustyoshi> Different index caches
06:40 < mustyoshi> Hmm
06:40 <@waxwing> so "the other version works fine" means, your live/original version?
06:40 <@waxwing> and does "it works fine" mean, presumably, you only used it without --fast (just sanity checking here)
06:41 < mustyoshi> Yeah, it's a copy of 1.4
06:41 <@waxwing> i have an idea - in the new, copied directory, you ran first a sync without --fast, is that right?
06:41 < mustyoshi> yes
06:42 <@waxwing> and do you happen to remember, at the end of it, did it say something about "restart, rescan if necessary"?
06:42 < mustyoshi> No I don't believe it did
06:42 <@waxwing> ok maybe i'm not right. so what's the difference in the index_cache?
06:43 < mustyoshi> The version that works has higher numbers in the second part of the arrays
06:43 <@waxwing> ah ok, i'm starting to see it i think.
06:44 <@waxwing> ah but - the difference is fairly small?
06:44 < mustyoshi> Yeah, just a few hundred maybe
06:44 <@waxwing> presumably it just represents the change since it's been running
06:44 <@waxwing> well not presumably i guess that's fairly obvious
06:44 < mustyoshi> Copied it over and fast sync is working
06:45 <@waxwing> really? i'm confused that it should sensitively depend on being perfectly up to date.
06:45 <@waxwing> i guess that's why i said originally "i'd be less confused if you reported a slight difference".
06:45 < mustyoshi> Yeah, that shouldn't have anything to do with running it without --fast
06:45 <@waxwing> a slight difference may make sense if the index_cache was stale. but not like 98%!
06:45 <@waxwing> or whatever you said
06:46 < mustyoshi> But shouldn't it be updating the index cache when I run it without --fast?
06:46 < mustyoshi> I'm just wondering what causes it to be incorrect even when it's going the "normal" way
06:46 <@waxwing> no, it's confusing, but sync doesn't actually update the index_cache, it only updates the index
06:46 < mustyoshi> Ah
06:47 <@waxwing> the index_cache is updated when transactions happen, so that there is a persistent memory of "used" addresses
06:47 <@waxwing> while the "index" is the live pointer to which address is used.
06:47 <@waxwing> it perhaps could be simplified, but that's how it is now.
06:47 < mustyoshi> Hmm
06:48 < mustyoshi> No that makes sense, so the first number is the index, and the second is the used portion?
06:48 <@waxwing> you mean in the list in the file?
06:48 < mustyoshi> Yeah
06:48 <@waxwing> no that's just external/internal branches of each mixdepth
06:48 <@waxwing> that whole thing is the cache of the index for every branch
06:49 < mustyoshi> o
06:49 <@waxwing> i certainly didn't mean for anyone to have to worry about this stuff, it should just be the case that if you have previously synced, run the prog for a while, then on restarting --fast should sync it quickly and "just work".
06:50 <@waxwing> it's nearly impossible for me to figure out what went wrong here, especially if it's now not going wrong!
06:50 < mustyoshi> Well, I'm still running both versions so presumably they will get out of sync again
06:50 <@waxwing> yeah but that's the bit i don't get, say it's 50 behind on one branch, but there were no coins there, why would it get confused and lose all your coins?
06:51 <@waxwing> oh, perhaps i'm being dumb about that: the coins are cycling forwards, so it could easily lose quite a lot quite quickly
06:51 < mustyoshi> Yeah, I use the deluxe generator in 1.4, so it's posting 5 orders, I guess the volume in 1.4 is still high enough
06:51 <@waxwing> yeah, that does sound plausible, but like 95% + is quite surprising, unless you're using almost all your coins quite quickly.
06:52 <@waxwing> mustyoshi: i guess technically you didn't follow my test instructions - i said "shut down your yield generator" but then i also said it shouldn't really matter :)
06:52 < mustyoshi> oh lol
06:52 < mustyoshi> Must have missed that part
06:52 <@waxwing> the thing is, it seems crazy that it should, if you ran a test over a period of a few minutes. but perhaps not.
06:53 <@waxwing> and there's the whole tangle about setting up, and then maybe syncing old style which takes time etc.
06:53 < mustyoshi> I'm about to tell you how many joins I was in over the last few days
06:53 <@waxwing> anyway, i'm using this new version now, still waiting for anyone else than you to test it :)
06:54 < mustyoshi> It's much faster I'll give you that. I'm still nowhere near done learning python multiprocessing for my electrum attempt lol
06:54 <@waxwing> does it take 30s-1min for you?
06:55 <@waxwing> i think i found it like sub 30s maybe. i dunno. anyway the old version is not acceptable for everyday operations.
06:55 <@waxwing> actually for small wallets it's a few seconds at worst.
06:56 < mustyoshi> Minute 30 for my wallet
06:56 < mustyoshi> But that's including sync_unspent, so about a minute, yeah
06:56 < mustyoshi> Much much better
06:56 -!- coins123 [~coins123@unaffiliated/coins123] has joined #joinmarket
06:57 < mustyoshi> Honestly, the waiting 15+ minutes for restarting scripts was terrible, especially if I'm trying to write a yieldgenerator
06:57 < mustyoshi> Yeah, 36 joins in the last 10 hours for me.
06:58 < mustyoshi> Which is more than in the 3 days prior...
06:58 <@waxwing> mustyoshi: you're getting somewhat of an unfair advantage by using the non-private form of the code :) i have about 10 in the last 24 hours.
06:58 < mustyoshi> Heh, the privacy isn't what I'm after ;)
06:59 <@waxwing> yes i understand. there is such a thing as short term and long term thinking though.
07:00 <@waxwing> mustyoshi: do you have a github handle
07:00 < mustyoshi> Yep, it's also mustyoshi
08:07 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has quit [Quit: Leaving.]
08:11 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has joined #joinmarket
09:22 -!- fqtw__ [~me@x590d9b0a.dyn.telefonica.de] has joined #joinmarket
09:26 -!- fqtw_ [~me@x590e3273.dyn.telefonica.de] has quit [Ping timeout: 240 seconds]
10:43 -!- grubles [~buttsniff@unaffiliated/grubles] has quit [Quit: Leaving]
12:49 -!- Elanthius [5e1f234e@gateway/web/freenode/ip.94.31.35.78] has quit [Ping timeout: 240 seconds]
13:26 <@waxwing> bit of fun: https://www.reddit.com/r/joinmarket/comments/53ult3/heres_one_go_at_a_logo_what_do_people_think/
13:37 < arubi> not bad
13:44 -!- megaddin [aladdin@gateway/shell/fnordserver.eu/x-fhguzqzesrbzuuwm] has quit [Remote host closed the connection]
14:00 -!- megaddin [aladdin@gateway/shell/fnordserver.eu/x-zioulpjzjkhghmca] has joined #joinmarket
15:48 -!- moli [~molly@unaffiliated/molly] has joined #joinmarket     
15:49 -!- molz [~molly@unaffiliated/molly] has quit [Ping timeout: 260 seconds]     
16:03 -!- HeySteve [~Lizard__W@unaffiliated/heysteve] has quit [Ping timeout: 250 seconds]     
16:09 -!- lnostdal [~lnostdal@27-245-11.connect.netcom.no] has joined #joinmarket     
16:09 -!- fqtw [~me@x4db1980a.dyn.telefonica.de] has joined #joinmarket     
16:13 -!- fqtw__ [~me@x590d9b0a.dyn.telefonica.de] has quit [Ping timeout: 244 seconds]     
17:44 -!- fqtw_ [~me@x4db0e4e4.dyn.telefonica.de] has joined #joinmarket     
17:48 -!- fqtw [~me@x4db1980a.dyn.telefonica.de] has quit [Ping timeout: 265 seconds]     
18:27 -!- arubi [~ese168@unaffiliated/arubi] has quit [Ping timeout: 265 seconds]     
18:32 -!- arubi [~ese168@unaffiliated/arubi] has joined #joinmarket     
19:13 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has quit [Ping timeout: 265 seconds]     
19:31 -!- fqtw__ [~me@x4db00312.dyn.telefonica.de] has joined #joinmarket     
19:34 -!- fqtw_ [~me@x4db0e4e4.dyn.telefonica.de] has quit [Ping timeout: 260 seconds]     
20:45 -!- lnostdal [~lnostdal@27-245-11.connect.netcom.no] has quit [Read error: Connection timed out]     
20:46 -!- lnostdal [~lnostdal@27-245-11.connect.netcom.no] has joined #joinmarket     
20:47 -!- lnostdal [~lnostdal@27-245-11.connect.netcom.no] has quit [Max SendQ exceeded]     
21:16 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 264 seconds]     
21:34 -!- moli [~molly@unaffiliated/molly] has joined #joinmarket     
23:19 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has quit [Quit: Leaving.]     
23:35  * adlai starts watching mr robot season finale; decides to practice self-control, reads fungibit's jm_unmixer first