--- Day changed Sat Dec 07 2019
00:47 -!- MaxSan [~four@185.93.182.171] has quit [Ping timeout: 268 seconds]     
01:07 -!- MaxSan [~four@103.254.153.99] has joined #joinmarket     
03:04 -!- Kellie21Hermisto [~Kellie21H@ns334669.ip-5-196-64.eu] has joined #joinmarket     
03:12 -!- reallll [~belcher@unaffiliated/belcher] has joined #joinmarket     
03:16 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 268 seconds]     
03:26 -!- SourDoughB [027e398c@2.126.57.140] has joined #joinmarket     
04:12 < SourDoughB> Maker's dilemma: promote JM to attract more takers (fee revenue) <-> keep quiet to not attract more maker competition XD 
04:15 -!- Kellie21Hermisto [~Kellie21H@ns334669.ip-5-196-64.eu] has quit [Ping timeout: 268 seconds]     
04:17 -!- kristapsk [~KK@gateway/tor-sasl/kristapsk] has quit [Remote host closed the connection]     
04:18 -!- kristapsk [~KK@gateway/tor-sasl/kristapsk] has joined #joinmarket     
05:00 -!- mr_paz [~mr_paz@24.14.251.223] has joined #joinmarket     
05:22 -!- reallll is now known as belcher     
06:10 -!- onion_kirens [~onion_kir@104.37.30.44] has joined #joinmarket     
06:11 < onion_kirens> I have a tumbler session that has been stuck on "Transaction XYZ has a conflict, abandoning". Under what circumstances does this happen? (block re-org?) and how can I resolve it safely?
06:14 < belcher> yep its caused by a conflict https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/d3ca99ccfd992c4e24fbfc69f9b9de82f7dc0e6b/jmclient/jmclient/wallet_service.py#L206-L209
06:14 < belcher> perhaps it was a RBF
06:14 < belcher> try ending the tumbler run and then restarting, either adding --restart to the command line arguments to using joinmarket-qt to load the tumbler schedule file and restart that way
06:19 < onion_kirens> Okay should I start at the first mixdepth with confirmed utxos?
06:20 < belcher> restart mode handles all that
06:20 < belcher> it aims to continue where the last confirmed transaction is
06:20 < onion_kirens> I did not restart. 
06:20 < onion_kirens> opened joinmarket-qt
06:20 < onion_kirens> I canceled the tumbler.py session and started joimarket-qt
06:21 < belcher> if the tumbler run was originally with the CLI tumbler.py then you should restart also with CLI
06:22 < belcher> which is done by running exactly the same tumbler.py line but with --restart added
06:22 < onion_kirens> okay. I'll manually recover the utxo's if I get any stuck then
06:22 < onion_kirens> and keep this in mind in the future
06:23 < waxwing> what do you mean by 'stuck' utxo in this context?
06:24 < waxwing> utxos are never stuck in joinmarket or really any coinjoin software; either they get spent in a coinjoin or they don't. if they don't you can spend them freely (or do a different coinjoin with them)
06:24 < waxwing> in the past people talked about 'stuck' utxos because of very low fee payments. that's something else, though.
06:25 < onion_kirens> meaning I am assuming that, since I ran a joimmarket-qt tumble session after an aborted tumble.py, there is a possibility I will break something (e.g. that a change utxo might not tumble)
06:26 < waxwing> btw if you restart on Qt it should still work, you'll get a dialog saying "interrupted tumble detected, want to restart?".
06:26 < waxwing> it's crude but it works fine afaict; it just looks in TUMBLE.schedule in logs/
06:27 < onion_kirens> I see. Basically the sequence of events is this:
06:27 < waxwing> the base premise is that we only co-spend utxos from the same mixdepth. Joinmarket enforces that globally, so if a tumble is stopped halfway and then you do other spends, it will still respect that rule.
06:28 < onion_kirens> Okay so what happens when I just don't restart the tumble, and run a new tumble with change and cj-out utxo's at various mixdepths
06:28 < waxwing> i'm saying it still preserves that rule as per above
06:28 < onion_kirens> Because that's what I did, (because I do not quite know what I'm doing)
06:28 < onion_kirens> okay, thanks
06:29 < waxwing> also, on 'conflicted tx' errors, this was the purpose of the rearchitect i did in 2017; the code will always retry the coinjoin after a longish timeout (20 minutes by default) **whatever** the cause of failure.
06:29 < waxwing> if you got spend conflict errors it would have just continued, trying again (with tweaks) until a coinjoin went through.
06:29 < onion_kirens> The transaction conflict message was repeated over the course of an hour
06:30 < waxwing> yes fair enough; and a similar (worse) case would be if there was just no liquidity for your chosen amount, even when it gets automatically tweaked lower.
06:30 < waxwing> in other words, it is certainly possible for failure to repeat over and over for a while, in principle indefinitely.
06:30 < onion_kirens> I think the liquidity issue is a real possibility in this case
06:30 < waxwing> right. if N parties have the right amount and you want a coinjoin of N, then .. you *can* be just out of luck. software can't help there.
06:31 < waxwing> you can of course just leave it running for a long time until new liquidity shows up.
06:33 < onion_kirens> another unrelated mishap i had yesterday: I had three subsequent tumbles. In each case, I funded 3 addresses at mixdepth 0. When one tumble finished, I would load up three more addresses and tumble them. On the 6th deposit, the wallet never saw the TX. I increased my mixdepth, moved the Core wallet.dat so a  new one could be created, and then issued a rescanblockchain RPC command. I was able to fix my issue this way. But what could have happened?
06:34 < onion_kirens> Sorry, I mean "I increased my gap limit to 50" 
06:34 < onion_kirens> not "increased my mixdepth"
06:34 < onion_kirens> It's suspicious that the wallet failed to see the transaction beyond the gap limit, which is why I increased it
06:35 < onion_kirens> When I started investigating the issue, Core did not show anything beyond the 5th address when I called listaddressgroupings RPC
06:35 < waxwing> i'm not sure. that is indeed interesting. CLI or GUI?
06:35 < onion_kirens> sorry, I mean the 6th address
06:35 < onion_kirens> CLI     
06:36 < waxwing> version of joinmarket?
06:36 < onion_kirens> 0.6.0
06:37 < waxwing> ok. so, you're running wallet-tool to get addresses to deposit to, right?
06:37 < onion_kirens> Yes, on deposit, I would run wallet-tool to verify the transaction was in the mempool
06:38 < onion_kirens> block explorers showed that the transaction was broadcast and eventually confirmed, but wallet-tool never registered the address with Core
06:38 < onion_kirens> And yes, I woudld run wallet-tool.py to get the addresses to which I would deposit. There was no address re-use.
06:39 < onion_kirens> Maybe it did register the address with Core, but listaddressgroupings only shows those addresses that had/have balances?
06:39 < onion_kirens> Not sure.
06:41 < belcher> listaddressgroupings should show all imported addesses, it sounds like the new address wasnt imported for some reason
06:41 < waxwing> so after the first 3 deposits, did you see that the bip32 paths for mixdepth 0 looked like this? https://0bin.net/paste/+fisP8MaO7R3udZ2#3jcLujLcbon2i+JUvMMHIMddCiV6q0OgHDLI-wpr2nt
06:42 < onion_kirens> How many addresses at a time does joinmarket add to Core?
06:42 < onion_kirens> yes, they looked like that
06:43 < waxwing> so it was (a) deposit 3 (b) run tumbler (c) deposit 3 <-- and here the last of 3 didn't show up because Core hadn't imported it?
06:43 < onion_kirens> and after tumble, available deposit addresses started at index 3
06:44 < onion_kirens> (a) deposit 3 (b) run tumbler (c) deposit 3 (d) run tumbler (e) deposit 3 (f) deposit never displays
06:44 < waxwing> oh so 2 groups of 3 were OK, and the third group failed, even the first of that third group?
06:44 < onion_kirens> yes, even the first of the 3rd group
06:44 < onion_kirens> Now I will say that the only deviation in (e) that I made was that I manually constructed one transaction with 3 outputs. 
06:45 < waxwing> oh. well, that could cause an issue yes. which is unfortunate.
06:45 < onion_kirens> good to know!
06:45 < waxwing> your question re: how many imports is very pertinent, unfortunately the answer is complicated.
06:45 < waxwing> in these cases do consider using --recoversync
06:46 < onion_kirens> will --recoversync perform the rescanblockchain for me?
06:46 < waxwing> it tries to make less assumptions about the wallet. however i'm not stating anything categorically here, needs more thinking (at least for me!)
06:46 < waxwing> no it won't, which is a fair point.
06:46 < onion_kirens> okay
06:46 < waxwing> i wanted to say your question about how many imports is pertinent, but the answer is very complicated.
06:46 < waxwing> for a start, there is a bug in Bitcoin Core pre 0.19 such that large cached imports are practically not possible with HDD disks
06:47 < waxwing> because they have an unnecessary database lock which creates ludicrously large delays like 10s of seconds, breaking coinjoins.
06:47 < onion_kirens> I believe electrum-personal-server registers like 1000 addresses per wallet 
06:47 < onion_kirens> oh wow :O
06:47 < waxwing> hence we have some very complex decisions we have to make to try to allow users to use have sync work without any issues *most* of the time.
06:48 < waxwing> still enough waffle, i want to see if your exact scenario is reproducible. i don't *think* that should happen normally.
06:48 < waxwing> but with an extra manually created transaction, perhaps.
06:48 < waxwing> also i'm not sure either way whether --recoversync would have solved it for you, it's too late to find out now i guess.
06:48 < onion_kirens> manually created in Electrum 
06:49 < onion_kirens> using the "Pay to Many" dialog
06:49 < waxwing> but bear in mind in future that that option basically makes less assumptions. i guess you're right if we failed to import an address it wouldn't have helped, on its own.
06:52 < onion_kirens> Well thank you for your help. I will try to be less exotic in my use-cases when it counts, and reckless when it doesn't.
06:52 < waxwing> (also interesting about joinmarket: 1000 addresses isn't really "big" here; we have 12 branches and long running users use up 1000 addresses pretty quickly. if you call importmulti on 5000 on an HDD you're going to be waiting minutes or hours, believe it or not. we had a report that 60 imports took half a minute).
06:53 < onion_kirens> I have definitely experienced this wait time every time elctrum-personal-server starts up
06:53 < waxwing> doing your own txs shouldn't be a problem. i just need to investigate it though, i feel like we missed that part when going through all the details of getting sync to be as simple and fast as possible within reason.
06:53 < waxwing> if you're on SSD you can notice some delay, but it's not usually too big of a  deal.
06:54 < onion_kirens> just an external HDD
06:54 < onion_kirens> so yeah, it was noticeable 
06:54 < waxwing> right. the difference is like a factor of 30 or something.
06:54 < SourDoughB> sorry, newbie side question: what happens when all the 1000 addresses get used? Will I need to take some action?
06:55 < waxwing> and seems to be linear.
06:55 < waxwing> SourDoughB, no, it does actually import new address as you go (despite the discussion above - some kind of bug probably related to external txs)
06:55 < SourDoughB> ok thx :)
06:55 < waxwing> it's a good idea to move to a new wallet when it starts to get really huge; but a couple of thousand shouldn't trouble you at all.
06:58 < waxwing> onion_kirens, if you plan to stick around, i am going to try to reproduce your error both with and without external transactions
06:59 < waxwing> if you don't plan to stick around here, a github issue would be a good place to continue the discussion. i mean up to you, of course.
07:01 < waxwing> onion_kirens, also to be clearer, you say you created pay to many with three outputs from electrum, but how did it involve the joinmarket wallet? i'm guessing you mean paid to three JM deposit addresses from electrum?
07:02 < waxwing> if so then it seems like it's basically not relevant so the problem must not be connected to that.
07:04 < onion_kirens> Yes, I paid to three JM deposit addresses with one Pay-To-Many Electrum transaction
07:08 -!- belcher [~belcher@unaffiliated/belcher] has quit [Quit: Leaving]     
07:17 < waxwing> i can definitely reproduce this. it's quite a subtle and complex thing, but it's definitely a bug, thanks for the report onion_kirens 
07:18 < onion_kirens> wow! what was it related to?
07:19 < waxwing> wallet-tool is just read only, somehow we are displaying addresses gap-limit ahead, but not having imported them already. i need to investigate a bit, why that's happening.
07:19 < waxwing> the rule that shouldn't be broken (but is) is, we should not display addresses that are not imported.
07:20 < onion_kirens> How can I avoid this?
07:20 < waxwing> sorry i need time. if i say stuff now it's liable to be wrong somehow.
07:21 < onion_kirens> okay, thanks for looking into it
07:24 -!- onion_kirens [~onion_kir@104.37.30.44] has left #joinmarket ["Leaving"]     
08:17 -!- onion_kirens [~onion_kir@104.37.30.44] has joined #joinmarket     
08:19 -!- onion_kirens [~onion_kir@104.37.30.44] has quit [Client Quit]     
08:33 -!- MaxSan [~four@103.254.153.99] has quit [Ping timeout: 268 seconds]     
08:54 -!- oldseed [b9d4aa77@185.212.170.119] has joined #joinmarket     
08:57 < oldseed> Hi, I have an old style seed that I ran yield generator on a while back.  Is there any way to extract the private keys outside of resyncing with the full node and displayall.  I'm hoping for some other method to view the derivation path of the old style seed.  Please help anyone.  Thank you.
10:40 -!- moberneto [~moberneto@104.37.30.44] has joined #joinmarket     
10:40 -!- moberneto [~moberneto@104.37.30.44] has quit [Client Quit]     
10:57 < belcher_> oldseed: use the -H option in wallet-tool
10:58 < belcher_> for example: python3 wallet-tool.py -p -H m/0/0/0/000 yourwalletfile.jmdat
10:58 < belcher_> see wallet-tool.py --help for all the details
11:03 < waxwing> oldseed, this may or may not be helpful: https://gist.github.com/AdamISZ/d978a1fa06b808773cf14d08b16c6b6e
11:05 < waxwing> specifically for pre-segwit and pre-BIP39 (so derivation paths as per belcher_ 's comment above) and with no access to Bitcoin Core, and you want to get out private keys.
11:28 -!- belcher [~belcher@unaffiliated/belcher] has joined #joinmarket     
11:29 < waxwing> oldseed, and this is the instructions for how to set up to use that: https://www.reddit.com/r/joinmarket/comments/bd5fhu/my_joinmarket_story/el24sxr/ ... if you can't figure it out let us know
11:41 -!- SourDoughB [027e398c@2.126.57.140] has quit [Remote host closed the connection]     
12:12 -!- SourDoughB [027e398c@2.126.57.140] has joined #joinmarket     
13:17 < oldseed> Thank you so much guys.  Legends.
13:26 -!- MaxSan [~four@103.254.153.99] has joined #joinmarket     
13:53 -!- MaxSan [~four@103.254.153.99] has quit [Quit: Leaving.]     
16:26 -!- SourDoughB [027e398c@2.126.57.140] has quit [Remote host closed the connection]     
17:24 -!- CgRelayBot [~CgRelayBo@p548664F9.dip0.t-ipconnect.de] has quit [Ping timeout: 265 seconds]     
17:26 -!- AgoraRelay [~jmrelayfn@p548664F9.dip0.t-ipconnect.de] has quit [Ping timeout: 250 seconds]     
17:35 -!- AgoraRelay [~jmrelayfn@p548667A4.dip0.t-ipconnect.de] has joined #joinmarket     
17:37 -!- CgRelayBot [~CgRelayBo@p548667A4.dip0.t-ipconnect.de] has joined #joinmarket     
17:52 -!- mr_paz [~mr_paz@24.14.251.223] has quit [Read error: Connection reset by peer]     
18:04 -!- dopplergange [~dop@31.169.121.17] has quit [Ping timeout: 265 seconds]     
18:07 -!- dopplergange [~dop@196.244.191.166] has joined #joinmarket     
23:59 -!- slivera [slivera@gateway/vpn/privateinternetaccess/slivera] has joined #joinmarket