--- Day changed Mon Apr 16 2018
02:07 -!- MaxSan [~user@91.207.102.163] has quit [Remote host closed the connection]     
03:17 < waxwing> alexcato doing a bunch of stuff but haven't forgotten your comment. your situation where clearing cache fixed it, i totally forgot, in the sense that that totally makes no sense. we need to find another explanation for what the hell happened there.
03:17 < waxwing> otoh there seems to be no doubt about the scenario i described; i was able to replicate it easily in testing/regtest.
05:12 -!- belcher_ [~belcher@unaffiliated/belcher] has joined #joinmarket     
05:55 -!- Giszmo [~leo@pc-37-38-86-200.cm.vtr.net] has quit [Ping timeout: 260 seconds]     
06:02 -!- lnostdal [~lnostdal@77.70.119.51] has quit [Ping timeout: 240 seconds]     
06:10 -!- Giszmo [~leo@pc-37-38-86-200.cm.vtr.net] has joined #joinmarket     
06:11 -!- lnostdal [~lnostdal@77.70.119.51] has joined #joinmarket     
06:16 -!- Lairem [~User@unaffiliated/lairem] has left #joinmarket []     
06:17 -!- Giszmo [~leo@pc-37-38-86-200.cm.vtr.net] has quit [Ping timeout: 260 seconds]     
06:31 -!- Giszmo [~leo@ip-252-233.219.201.nextelmovil.cl] has joined #joinmarket     
07:29 -!- Giszmo [~leo@ip-252-233.219.201.nextelmovil.cl] has quit [Ping timeout: 240 seconds]     
07:46 -!- Giszmo [~leo@ip-86-233.219.201.nextelmovil.cl] has joined #joinmarket     
07:56 -!- bsm1175321 [~mcelrath@173-9-124-61-NewEngland.hfc.comcastbusiness.net] has joined #joinmarket     
10:00 < arubi> what does the cache do wrt the import process?
10:05 -!- quitobro [~quitobro@pool-108-41-0-186.nycmny.fios.verizon.net] has joined #joinmarket     
10:10 -!- quitobro [~quitobro@pool-108-41-0-186.nycmny.fios.verizon.net] has quit [Ping timeout: 276 seconds]     
10:40 -!- Giszmo [~leo@ip-86-233.219.201.nextelmovil.cl] has quit [Quit: Leaving.]     
10:53 -!- Giszmo [~leo@ip-86-233.219.201.nextelmovil.cl] has joined #joinmarket     
11:00 < waxwing> it's basically giving the sync process knowledge of what addresses have already been used, so that even if there's no evidence of it being used (in the Core wallet), it makes sure you don't reuse addresses lower in the branch.
11:01 < waxwing> so theoretically it's possible that if there were keys at a higher index, let's say 1000, that were imported with importprivkey, they wouldn't be reached with a fresh sync when the cache had been wiped.
11:01 < waxwing> but that's a kind of weird scenario.
11:03 < arubi> iirc there were 760 keys being imported in AlexCato's log?  maybe I'm misremembering
11:06 < arubi> the paste is gone now
11:07 < AgoraRelay> [agora-irc/AlexCato] paste is gone, but it's also in the issue description: https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/129
11:07 < AgoraRelay> [agora-irc/AlexCato] so yeah, it was a huge amount
11:08 < AgoraRelay> [agora-irc/AlexCato] good memory, 760 was exactly right
11:08 < arubi> :)     
11:08 -!- Giszmo [~leo@ip-86-233.219.201.nextelmovil.cl] has quit [Quit: Leaving.]     
11:08 < AgoraRelay> [agora-irc/AlexCato] afk for a bit, but will be back later
11:09 < arubi> alright, so if there's a way to tell it to import "that many", maybe we can recreate the error just to be sure
11:24 < waxwing> sorry been a bit distracted, let's try to think about this carefully now.
11:24 < waxwing> https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/129 reopened (alexcato pls don't be shy to do that, as i said i was happy for it to be reopened :)
11:25 < waxwing> what i don't really get is, if you delete the index cache and then try wallet-tool again, it'll resync and should still see the already-used addresses. it can fail to see the entire set of already-used addresses, if there is a big gap.
11:26 < waxwing> in that case of course the import has, in some sense, "failed" - it will reuse addresses.
11:26 < waxwing> people in that situation have tended to know that something's wrong, because coins were missing. but if the wallet was already emptied, that warning sign would not occur.
11:27 < arubi> ah that's interesting, is the account name that is chosen for the new import process the same one that was before the cache was removed?
11:28 < waxwing> an extra point of confusion is, the --fast version of sync does not have this danger; because it insists on finding all the addresses that come out of addressgroupings.
11:29 < waxwing> i tried to lay out the details of that algorithm in this commentary: https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmclient/jmclient/blockchaininterface.py#L458-L478
11:29 < waxwing> but, i'm guessing that alexcato was not using this, so we can probably put it to one side.
11:29 < waxwing> arubi, the account name is chosen based on a hash of the first address in the first mixdepth, so it's deterministic to the wallet seed.
11:30 < arubi> alright, that's good then
11:30 < waxwing> (btw `addressgroupings` was `listaddressgroupings` but doesn't matter here)
11:31 < arubi> yea.  also I think your intuition about the gap thing might be right, there was a time when cj's were dropping for a while?
11:31 < arubi> maybe it's pre segwit, I don't remember (or maybe I'm making things up..)
11:32 < waxwing> well yes it was more at an earlier time. but it's a possibility to bear in mind.
11:35 < waxwing> so, the non-fast version of sync uses data that comes out of listtransactions; so, if there is a big gap at say 100-200 and index_cache goes up to 1000, it will import up to 1000, but if there is no index_cache, it will stop at 100+ gap limit. crudely speaking it's like that.
11:36 < waxwing> so the possibility is there. if `importprivkey` was used on the key at index say 900, then it would crash there, but if index_cache was removed it would not. but the sync would have failed.
11:36 < waxwing> well i'm repeating myself now, talked about that scenario already.
11:36 < arubi> so, regroup, wrt to this specific "wallet already has the key for this address", there are two possibilities, either the key was imported and there are no transactions associated with the address, or there are.  in either case, does it even matter?  if the key was imported, can't the import process just treat it as if it's used?
11:37 -!- dx25 [~dx25@67-3-137-174.omah.qwest.net] has quit [Remote host closed the connection]     
11:38 < waxwing> hmm that sounds entirely logical, why was i thinking that isn't correct? have to think a bit
11:40 < waxwing> ok, so the difference is whether transactions were done on the key or not.
11:40 -!- dx25 [~dx25@67-3-137-174.omah.qwest.net] has joined #joinmarket     
11:40 < waxwing> our `listtransactions` calls use wallet_name which filters to only the JM wallet.
11:41 < waxwing> what is a reasonable assumption is that we did transactions on it before importing to the Core wallet.
11:42 < waxwing> then it'd be OK, except, i worry again about - what if it was moved between Core instances, maybe a tx was done on one Core instance, then JM moved to another Core instance, then importprivkey is called, then JM reloaded and it's seen as unused when it is used (on-chain, if not in this Core instance)
11:46 < arubi> it would only miss it being used if rescan wasn't done after import
11:47 < arubi> but that's true for pure address import too, not just privkey import then migrate to a different wallet
11:51 < arubi> it's getting into wallet recovery territory which is basically hard for everyone.  the only real users who "suffer" from it are folks using full nodes because everyone else just end up querying some authority for all their history
12:27 -!- lnostdal [~lnostdal@77.70.119.51] has quit [Ping timeout: 256 seconds]     
12:28 -!- lnostdal [~lnostdal@77.70.119.51] has joined #joinmarket     
12:32 -!- lnostdal [~lnostdal@77.70.119.51] has quit [Excess Flood]     
12:32 -!- lnostdal [~lnostdal@77.70.119.51] has joined #joinmarket     
16:20 -!- quitobro [~quitobro@pool-108-41-0-186.nycmny.fios.verizon.net] has joined #joinmarket     
16:23 -!- Giszmo [~leo@pc-37-38-86-200.cm.vtr.net] has joined #joinmarket     
16:34 -!- belcher_ [~belcher@unaffiliated/belcher] has quit [Quit: Leaving]     
19:32 -!- Giszmo [~leo@pc-37-38-86-200.cm.vtr.net] has quit [Ping timeout: 268 seconds]     
19:52 -!- Giszmo [~leo@ip-37-237-219-201.nextelmovil.cl] has joined #joinmarket     
20:15 -!- quitobro [~quitobro@pool-108-41-0-186.nycmny.fios.verizon.net] has quit [Quit: quitobro]     
20:16 -!- AgoraRelay [~jmrelayfn@p5DE4A529.dip0.t-ipconnect.de] has quit [Ping timeout: 256 seconds]     
20:28 -!- AgoraRelay [~jmrelayfn@p54866E35.dip0.t-ipconnect.de] has joined #joinmarket     
20:34 -!- quitobro [~quitobro@pool-108-41-0-186.nycmny.fios.verizon.net] has joined #joinmarket     
20:34 -!- quitobro [~quitobro@pool-108-41-0-186.nycmny.fios.verizon.net] has quit [Client Quit]     
21:03 -!- GAit_ [~GAit@unaffiliated/gait] has joined #joinmarket     
21:03 -!- technonerd [~techno@unaffiliated/technonerd] has quit [Ping timeout: 256 seconds]     
21:03 -!- GAit [~GAit@unaffiliated/gait] has quit [Ping timeout: 256 seconds]     
21:03 -!- Giszmo [~leo@ip-37-237-219-201.nextelmovil.cl] has quit [Quit: Leaving.]     
21:03 -!- technonerd [~techno@unaffiliated/technonerd] has joined #joinmarket     
21:16 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has quit [Ping timeout: 266 seconds]     
21:18 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has joined #joinmarket     
23:23 -!- lnostdal [~lnostdal@77.70.119.51] has quit [Read error: Connection reset by peer]     
23:23 -!- lnostdal [~lnostdal@77.70.119.51] has joined #joinmarket