--- Day changed Sat Sep 29 2018 01:26 -!- beIcher is now known as belcher_ 01:26 -!- belcher [~belcher@unaffiliated/belcher] has joined #joinmarket 01:54 -!- maxxam [~maxxam@infra.isplevel.pro] has joined #joinmarket 01:55 < maxxam> ok, so it looks like ANY wallet generated by recovering from seed requires running `bitcoind -rescan` 01:55 < maxxam> HOWEVER this is not possible on a pruned node, and would require downloading the whole blockchain 01:55 < maxxam> given my wallet is the exact same as the wallet I had previously set up … the previous one was just corrupted due to MMD something … is there any way to avoid re-downloading the whole blockchain? 01:56 < maxxam> can I get wallet-tool / etc to accept the new wallet as containing the same addresses as the old? 01:58 < maxxam> …hmm, looks like there is something weird going on, because I previously ran wallet-tool on the new wallet, and then it worked. It wasn’t until I used `sendpayment.py -m 1 -n 0` to reshuffle funds on the wallet, and added more funds to the wallet, that the requirement to `restart Bitcoin Core with -rescan if you're recovering an existing wallet from backup seed` started cropping up every time I try to run wallet-ool 02:01 < maxxam> right, so now after restarting bitcoind (but not successfully running `-rescan`) wallet-tool works again 02:01 < maxxam> never mind, I guess 02:02 < maxxam> (yes, I tried running wallet-tool multiple times and got the rescan message each time) 02:18 < waxwing> the message doesn't always mean you actually have to rescan (i forget the exact wording but i believe it qualifies the need) 02:19 < waxwing> the need is there specifically and only when you recover a wallet into a Core node, where the wallet has been used before, but not with *that* Core node. 02:19 < waxwing> iirc it says 'otherwise just restart' 02:20 < waxwing> and that's what you should do, otherwise. in those cases (the cases where you don't need to rescan), the only thing that's happened is you've imported a few new addresses into your Core wallet. Then you just run the script again. 02:20 < waxwing> Re: pruned node differences, i'll let belcher explain to you any nuances (I always forget as I've never used it) 02:21 < belcher> rescan doesnt work if your rescan range goes beyond your prune depth 02:22 < belcher> if you want to rescan in that situation then you need to redownload blocks 02:22 < belcher> but it sounds like in your situation you dont? 02:23 < belcher> what rescan does is scan the blockchain using the watch-only wallets in wallet.dat, and adds found transactions to wallet.dat.... it sounds like your wallet.dat already has watch-only addresses and transactions? if so you dont need to rescan 02:24 < waxwing> belcher, depending on the algo, the wallet-tool sync sometimes throws up that message right (people sometimes interpret it as 'you must rescan', but it doesn't say that); in the latest versions, that comes up a bit more frequently, in my experience. 02:25 < belcher> yes maybe 02:25 < belcher> what might happen is yieldgen imports two or three more addresses because it made that many coinjoins, but when you run wallet-tool it imports 20 or 30 addresses, and because its imported then it gives that message, but in that situation you dont need to rescan because the 20 or 30 addresses are new and empty 02:26 < belcher> the only time you need to rescan is if addresses got imported which have historical transactions 02:38 -!- undeath [~undeath@hashcat/team/undeath] has joined #joinmarket 02:55 -!- maxxam [~maxxam@infra.isplevel.pro] has quit [Ping timeout: 245 seconds] 03:10 -!- maxxam [~maxxam@infra.isplevel.pro] has joined #joinmarket 03:56 -!- bad_duck [~arthur@area51.powaaa.com] has quit [Ping timeout: 240 seconds] 03:56 -!- core_ [~core@ilya.xxx] has quit [Ping timeout: 240 seconds] 03:58 -!- core [~core@unaffiliated/core] has joined #joinmarket 04:10 -!- maxxam [~maxxam@infra.isplevel.pro] has quit [Ping timeout: 244 seconds] 04:40 -!- maxxam [~maxxam@infra.isplevel.pro] has joined #joinmarket 05:14 -!- maxxam_ [~maxxam@116.66.197.134] has joined #joinmarket 05:15 -!- maxxam [~maxxam@infra.isplevel.pro] has quit [Ping timeout: 272 seconds] 05:15 -!- maxxam_ is now known as maxxam 05:52 -!- maxxam_ [~maxxam@infra.isplevel.pro] has joined #joinmarket 05:53 -!- maxxam [~maxxam@116.66.197.134] has quit [Ping timeout: 272 seconds] 05:53 -!- maxxam_ is now known as maxxam 06:19 < maxxam> waxwing: aha, so what you are saying is that it was importing (deterministically generating) new addresses into the wallet, which triggered the restart message. That makes sense! (and consider making the message more clear!) 06:21 < waxwing> yeah. re: message, probably (PRs welcome), thing is, i feel sure at some point in the last year or so someone re-jigged it to do the "re-start" part automatically, but i've got lost in all the myriad changes to sync. for now, just restart it once (or sometimes twice) and it should be fine, except for the special case mentioned above, where you do actually have to rescan. 06:21 < maxxam> belcher: that makes sense (about only needing to rescan if you import addresses with historical transactions) … I was just under the impression it had added a NEW watch-only wallet (with the same addresses) and needed a rescan for that reason. Hopefully this never actually happens! 06:22 < maxxam> waxwing: I’m not really set up to do github (embarassingly) — otherwise I would… 06:25 < waxwing> the wallet name is defined by a hash of the first address in it (basically), so it'll always create the same wallet-name/account/label .. actually i'm not even sure what would happen if you created a *different* account name, then imported the same addresses, whether that needs to be rescanned or not. 06:25 < waxwing> to complicate matters further, accounts are being deprecated as of 0.18 and so belcher is currently working on the fix needed to switch to using labels instead. 06:26 < waxwing> in 0.17 you can still use it but you need to start bitcoind with --deprecatedrpc=accounts 07:02 < belcher> maxxam that can never happen, if you import the same addresses again then it notices and does nothing 07:04 < maxxam> good :) … also to consider worth clarifying, since that was my whole point of confusion. that WAS what I had just done — imported via seed! — and the message said if you import via seed you have to rescan. Except in my case I guess… 07:04 < maxxam> either way the test tumble is running, and seems to be working 07:13 < belcher> thats a very special situation i guess 07:13 < belcher> its accurate most of the time, you only need to rescan when you recover an old wallet 07:13 < belcher> btw, core 0.17 has a new rpc called scantxoutset which could be used to avoid rescans, the tradeoff is you dont see the history (which is never used in joinmarket anyway) 07:16 -!- Cory [~Cory@unaffiliated/cory] has quit [Ping timeout: 252 seconds] 07:22 -!- Pasha [~Cory@unaffiliated/cory] has joined #joinmarket 07:25 -!- Pasha is now known as Cory 07:39 < maxxam> does this mean my tumble is dead? 07:39 < maxxam> ERROR not enough liquidity in the orderbook n=8 suitable-counterparties=5 amount=50000 totalorders=5 07:40 < maxxam> Taker not continuing after receipt of orderbook 07:40 < maxxam> (tumble.py is just hanging there, not doing anything) 07:52 < waxwing> for some transactions it will reduce the n value until it manages to find enough, and for all transactions it will keep trying. but as i said yesterday, figures like 50k are just too small. 07:52 < waxwing> maxxam, ^ 07:53 < maxxam> waxwing: well it got this far (mixdepth 2) like that :) … anyway, do I need to —restart it or should I wait? 07:55 < maxxam> (this may be how the wallet corruption happened… previously when this message — `Taker not continuing` appeared, for a different reason, I just ctrl-c’ed it) 07:56 < waxwing> it waits because of cousre in theory the number can increase when different bots turn up. i mean feel free to experiment :) 07:56 < waxwing> no ctrl-c is fine, it'll never corrupt the wallet. 07:56 < maxxam> aha. will it keep coming up with new sizes on each try, or is the joinamount fixed ahead of time in the schedule? 07:56 < waxwing> read the 'tweaking the schedule' section in https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/docs/tumblerguide.md for info on that 07:57 < maxxam> will do 07:57 < maxxam> …if only I had the time, I’d file the mother of all PRs reorganising the docs… 08:01 < maxxam> ok, so in TUMBLE.schedule I see lines where line[1] — ‚amount-in-satoshis‘ according to the docs — is a fractional amount, sometimes a very small fraction, or even zero 08:03 < maxxam> a previously completed tumble had an amount-in-satoshis of zero 08:03 < maxxam> but the one which is hanging has a particularly small fractional amount 08:06 < maxxam> (err, that should be „a previously completed coinjoin“) … if I rerun tumbler.py with a higher minimum size, will that corrupt the in-progress tumble, or will it just limit the small coinjoin to a more reasonable value, and continue? 08:16 < waxwing> maxxam, are you asking, can you do --restart with a changed -s parameter? 08:16 < maxxam> correct 08:17 < waxwing> hmm i'm not sure i'd have to investigate. i specified people should only do --restart with the same parameters, mostly to avoid having to think through each possible case. 08:17 < maxxam> glad I asked :) 08:18 < waxwing> at the level you're working at, as i said yesterday, you almost may as well start working with custom schedules (see the sendpayment notes in scripts/README and look at the custom schedule for testnet file example 08:18 < waxwing> i said that yesterday because, as i said, using very tiny amounts is just not going to work for various reasons, if you're determined to do tests with it you'll have to do a lot of fiddling about anyway. 08:19 < maxxam> I’m not using a particularly tiny amount, about 0.02 BTC 08:22 < maxxam> question: sample-schedule-for-testnet specifies `index 1: amount. integer value in satoshis to be spent.` 08:22 < maxxam> BUT there are definitely non-integer amounts in my auto generated schedule 08:23 < maxxam> right, never mind, I see the next line 08:23 < maxxam> (`For the tumbler, can be a decimal (0.0 (maybe add that part to this file, the original source of my confusion on the float issue: https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/docs/tumblerguide.md ) 08:27 < waxwing> ah yes maxxam good catch, it says 'amount in satoshis' there, it shouldn't for that file. 08:27 < belcher> ob-watcher.py has a graph iirc 08:27 < belcher> of the coinjoin sizes offered, so from there one could see which values are too small or big 08:29 < maxxam> belcher: is that different from https://joinmarket.me/ob/ ? 08:29 < belcher> its the same, that url is just a front end for ob-watcher.py 08:30 < belcher> you can run the same script on your own computer 08:30 < belcher> in case that site ever went down for example 08:30 < maxxam> cool 08:30 < maxxam> my server is currently occupied with the tumble, but good to know 08:33 < maxxam> belcher: so in this case I see a few orders with 20k minimums in the book (e.g by J5EH3nNobjsQhHBv ) so it should go thru, right? 08:34 < belcher> i guess so 08:34 < maxxam> aha, so the schedule entry seems to have randomly called for a large number of counterparties 08:35 < maxxam> so it has currently found 7 but needs 8 08:35 < maxxam> and it looks like when it regenerates the transaction, it does not re-randomise the number of counterparties 08:38 < maxxam> the other odd thing is, when the schedule called for ca 5% of the balance, it had to limit-up to 50k satoshis, but when the transaction was regenerated to call for ca 4% of the balance, it did not trigger the 50k limit but ended up with an order size around 60k 08:41 < waxwing> maxxam, as it says in that section i mentioned, it does tweak the number of counterparties for sweeps but not for non-sweeps. for non sweeps it tweaks the amount instead. it's just a best effort thing that'll be fine in most cases. in very low liquidity it's not really going to work, that's asking a bit much :) 08:44 < maxxam> waxwing: aha, I didn’t realise the file covered that 08:45 < waxwing> well, i did just point to that section a short while ago, but i don't blame you for having trouble keeping up :) 08:46 < maxxam> haha no worries 08:47 < maxxam> I am still curious about doing —restart with a different -s value though, it SEEMS like that would solve the stuck CJ and ASSUMING -s is just a limiter applied after the fact then THEORETICALLY it should be fine 08:53 < maxxam> so it looks like applying mincjamount happens in taker.py: https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/36c3fd9c57b834d2b4e687d9ed9e83d9381896e6/jmclient/jmclient/taker.py#L171 08:54 < maxxam> on the other hand, the stuck CJ finally went through, so I don’t (currently) need to risk it 14:43 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 252 seconds] 14:43 -!- belcher [~belcher@unaffiliated/belcher] has joined #joinmarket 15:12 -!- undeath [~undeath@hashcat/team/undeath] has quit [Quit: WeeChat 2.2] 18:12 -!- maxxam [~maxxam@infra.isplevel.pro] has quit [Ping timeout: 240 seconds] 20:04 -!- maxxam [~maxxam@infra.isplevel.pro] has joined #joinmarket 20:38 -!- Cory [~Cory@unaffiliated/cory] has quit [Ping timeout: 272 seconds] 20:43 -!- Pasha [~Cory@unaffiliated/cory] has joined #joinmarket 20:45 < maxxam> hmm, the test tumble got stuck again. If I am going to retry it with a changed -s param, what files should I back up in case it fails? The old TUMBLE.schedule presumably, anything else? 20:47 -!- Pasha is now known as Cory 21:32 < qubenix> exit 23:32 < maxxam> qubenix: I think you mean `/quit`