--- Log opened Sun Aug 01 00:00:25 2021
00:08 -!- k3tan [~pi@user/k3tan] has quit [Ping timeout: 256 seconds]
00:16 -!- k3tan [~pi@user/k3tan] has joined #joinmarket
00:32 -!- technonerd [~techno@user/technonerd] has quit [Remote host closed the connection]
00:33 -!- technonerd [~techno@user/technonerd] has joined #joinmarket
00:51 < waxwing> belcher_, 14 btc locked already, after 1 day.
00:54 -!- JMBridge [~CoinJoins@gateway/tor-sasl/jmbridge] has joined #joinmarket
00:55 -!- JMBridge [~CoinJoins@gateway/tor-sasl/jmbridge] has quit [Remote host closed the connection]
00:58 < waxwing> belcher_, the fidelity bond is only set in `Maker.try_to_create_my_orders`, right? so if a maker is running over the boundary (1st of month), it'll keep announcing the bond after it's expired?
01:10 -!- JMBridge [~CoinJoins@gateway/tor-sasl/jmbridge] has joined #joinmarket
01:10 -!- JMBridge [~CoinJoins@gateway/tor-sasl/jmbridge] has quit [Remote host closed the connection]
01:15 < waxwing> (separately, a nit: showutxos shows tries/tries remaining on timelocked. just noting it here in case i forget)
01:20 -!- JMBridge [~CoinJoins@gateway/tor-sasl/jmbridge] has joined #joinmarket
01:24 -!- JMBridge [~CoinJoins@gateway/tor-sasl/jmbridge] has quit [Ping timeout: 244 seconds]
01:51 -!- JMBridge [~CoinJoins@gateway/tor-sasl/jmbridge] has joined #joinmarket
03:09 -!- SomeFellow [~SomeFello@194.54.28.203] has joined #joinmarket
03:13 < SomeFellow> Recovered to an FB wallet, and now the vast majority of my UTXOs are missing when I run wallet-tool. I have maybe 3 or so left of close to a hundred before the recovery. So far, I'm suspecting either that the Core -rescan didn't go off properly (would that have such a result?), or that the default gap limit is too low (would that be an issue,
03:13 < SomeFellow> assuming normal YG usage?). Any other ideas are welcome.
03:21 < SomeFellow> Curiously, running a summary on the old wallet.jmdat reveals that it's missing about 2% of the total funds it had yesterday. I haven't run any CJs as either a maker or takes since then, so I'm not sure why that would be.
03:26 < SomeFellow> The money in the old and new wallets (using the same seed phrase) add up to only slightly more than what the grand total should be. It seems like there are two unused deposits in MD0 that are in the new one but not the old one (both several months old), and one unused deposit in MD0 that's present in both wallets (that's only a few days old).
03:28 < SomeFellow> Also, it seems like recovering unfreezes all your auto-frozen 0.00~547 spam UTXOs. Makes perfect sense that it would, since freeze status isn't part of the seed phrase, but still something I didn't consider. Worth keeping in mind if you're worried about privacy.
03:35 < waxwing> oh that's a good point.
03:35 < SomeFellow> Also found a couple of cj-outs in MD2 that are in the new but not the old. Weird.
03:36 < waxwing> if you switch between the two the syncing of the wallet can get funky. use --recoversync if it doesn't look right, usually that will correct it.
03:38 < SomeFellow> I've never used the old wallet for anything after creating the new one (except as a target for a summary). In fact, I've never used the new wallet for anything either (again, except for the summary).
03:38 < SomeFellow> I'll give recoversync a try.
03:48 < SomeFellow> Ran recoversync on the new wallet with a gap limit of 100, and if found another dozen or so small UTXOs. Guess I'll do it again with a GL of 9999 or whatever and hope for the best.
03:53 < SomeFellow> The balance shown for MD4 is slightly different between 'summary' and just a plain wallet-tool. Looks like the latter is counting both change-outs, but the former only one of them. Odd. It's not like they could be frozen, and it's been more than 24h since I last ran the YG, so unconfirmed also seems unlikely.
03:55 < SomeFellow> And I'm still a little unsettled by funds missing from the old wallet. The new one not finding everything due to gap limit issues or whatever makes sense, but how could untouched deposits that've been in the old one for months suddenly disappear and only exist in the new wallet (still as untouched deposits)?
04:07 < waxwing> i'm curious, when you talk about gap limits of 100+, how heavily used is the wallet? hundreds of addresses per branch? (which would be thousands in total)
04:11 < SomeFellow> Been running a YG for six months off the same wallet with a high maxsize and a low minsize.
04:13 < waxwing> i mean, recoversync + increased gap limit should in theory catch everything, but it may require multiple runs before it eventually finds literally everything. needing 100 or more is kind of outside anything i looked at.
04:15 < SomeFellow> Would multiple runs make a difference?
04:16 < waxwing> yes, although it's supposed to be the case that it tells you to restart if it knows the process isn't finished. i guess it shouldn't be needed otherwise.
04:17 < SomeFellow> Just checked the statement, and I've only done a total of about 350 CJs as a maker. Not nearly as many as I though, though I guess I only lowered my minsize to its current level fairly recently.
04:18 < SomeFellow> *as I thought
04:18 < waxwing> a couple more things: you do need to use --recoversync initially when doing the recover into FB wallet (but i guess that's clear already), and, you might experiment with increasing these values (for *non* recoversync sync): https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmclient/jmclient/wallet_service.py#L557-L558
04:19 < waxwing> 350 says one thing, but the not-completed attempts can add a lot more usages. but if there was no all-out DOS attack or something, it's unlikely you're into the several thousands.
05:03 <+JoinMarketRelay> [hackint/nikolai] congratulations and thank you for the awesome fidelity bonds release
05:04 <+JoinMarketRelay> [hackint/person] GL 9999 is gigantic, 500 should be way more than enough. but -g 500 --recoversync really might need to be run repeatedly as the tool tells you, and after each run you need to rescan blockchain. if you remember since when you started using this wallet then you can set the rescan range: rescanblockchain ( start_height stop_height )
05:05 < waxwing> you don't actually have to do the rescanning if the addresses were already imported, which they will be on the same Core instance and the same bitcoind wallet (rpc_wallet_file in config)
05:06 < waxwing> the real nasty case is when you recover a big, active wallet on a new Bitcoin Core. but even then if you know when to rescan from (which blockheight) it doesn't need to be *too* bad (nowadays scanning from a height isn't ridiculously slow)
05:07 <+JoinMarketRelay> [hackint/person] actually you do have to rescan even if the addresses were already imported using the previous jm wallet, i have this exact experience
05:07 < waxwing> not if on the same bitcoind wallet file
05:23 <+JoinMarketRelay> [hackint/nikolai] is it safe to recreate a fidelity bond wallet with same mnemonic seed that was used in previous versions?
05:54 < qubenix> nikolai: in here: https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/docs/fidelity-bonds.md it says that you can use the recover method on your existing native segwit wallet to make it a fb wallet.
05:57 <+JoinMarketRelay> [hackint/nikolai] qubenix: thank you. i just gave it a read now but wanted to make sure before playing around with priv keys
06:34 -!- MN-ca [~MN-ca@cpe589630a45781-cm589630a4577f.cpe.net.cable.rogers.com] has joined #joinmarket
06:37 -!- MN-ca [~MN-ca@cpe589630a45781-cm589630a4577f.cpe.net.cable.rogers.com] has quit [Client Quit]
06:40 -!- MN-ca [~MN-ca@cpe589630a45781-cm589630a4577f.cpe.net.cable.rogers.com] has joined #joinmarket
06:42 -!- MN-ca [~MN-ca@cpe589630a45781-cm589630a4577f.cpe.net.cable.rogers.com] has quit [Client Quit]
06:52 -!- MN-ca [~MN-ca@cpe589630a45781-cm589630a4577f.cpe.net.cable.rogers.com] has joined #joinmarket
06:54 -!- MN-ca [~MN-ca@cpe589630a45781-cm589630a4577f.cpe.net.cable.rogers.com] has quit [Client Quit]
07:08 <+JoinMarketRelay> [hackint/nikolai] hello
07:12 -!- MN-ca [~MN-ca@cpe589630a45781-cm589630a4577f.cpe.net.cable.rogers.com] has joined #joinmarket
07:13 <+JoinMarketRelay> [hackint/nikolai] i am trying to get 0.9.0 working with fidelity bonds, i've read some of the docs related to fidelity bonds excluding the one about financial mathemtics. i do not understand one part. bond_value = (locked_coins * exp(interest_rate * locktime))^2
07:14 -!- MN-ca [~MN-ca@cpe589630a45781-cm589630a4577f.cpe.net.cable.rogers.com] has left #joinmarket []
07:14 <+JoinMarketRelay> [hackint/nikolai] the interest_rate, which is set to 1.5% by default in newly generated config, what does this actually do for makers, does the cjfee_relative for YG have to be same as interest_rate?
07:46 < SomeFellow> A gap limit 9999 recoversync later, and everything is where it should be. There's no kill like overkill.
07:56 <+JoinMarketRelay> [hackint/nikolai] anyone that could give me a hand with interest_rate - bond_value = (2 * exp(0.015 * (60*60*24*356.25*0.0833333333333333)))^2   --- i'm trying to calculate bond_value for 1/12 of a year ~ 1 month
08:02 < waxwing> SomeFellow, what are the indices you see on the branches? this doesn't reveal private info, but it might help understand (might!)
08:04 < SomeFellow> Not sure what you mean. The m84 stuff?
08:06 < SomeFellow> First column when you run wallet-tool?
08:22 < waxwing> SomeFellow, oh sorry, the latest index that's got coins on it in each internal/external branch, in each account
08:22 < waxwing> the index is the very last figure in th m/84/.. list
08:24 < waxwing> for the FB wallets the 0th mixdepth is a bit complicated because it has two extra branches, but all the other mixdepths (accounts) have only two branches (0 - external, 1 - internal)
08:24 <+JoinMarketRelay> [hackint/nikolai] waxwing: i am one step from testing 0.9.0 with fidelity bonds. i do not understand the interest_rate configuration option in joinmarket.cfg. what happens with this interest_rate, who pays this?
08:35 < SomeFellow> waxwing The biggest one in the incomplete new wallet was 102, while the biggest one in the complete new wallet is 325.
08:39 < SomeFellow> nikolai I think it has something to do with how highly takers value the duration you lock funds up for. I don't think anyone actually 'pays' it. I assume if it's set close to 0, the taker('s coinjoin script) doesn't care if you lock 10 BTC up for a month or 100 years. Don't take my word for it though, I'm just guessing.
08:55 <+JoinMarketRelay> [hackint/nikolai] SomeFellow: thanks for the explanation, it's possible that it works how you said. will report back when tested.
09:16 < SomeFellow> Do locked coins need a certain amount of confirmations to show up properly? The first time I ran YG (with 0 conf on the FB funding transaction without thinking), it said I had no FB. Fair enough. Now I have >0, and the YG said it published a FB, but the FB column in the OB next to my offer still says 0.
09:23 < waxwing> SomeFellow is correct about interest rates.
09:23 < waxwing> afaik they need 1 conf.
09:23 < waxwing> but i'd have to double check.
09:24 < SomeFellow> The YG said I had one, and that it was published. Even gave me a nice long string of numbers and letters. Still 0 in the OB.
09:26 < waxwing> if it's shown as "Announcing fidelity bond" in your output of yg, it'll be announced, maybe your OB is the old version that doesn't recognize?
09:26 < waxwing> or restart it? been working fine for me
09:30 < SomeFellow> Restarted the OB, and that seems to have fixed it. Or maybe with was because it had more confirmations at that point. Who knows? Last question, hopefully: Is there any way I can confirm that the YG is running (only) over Tor? I uncommented the four and two line sections in the conf, but I'd like to be sure.
09:40 -!- jungly [~jungly@gateway/tor-sasl/jungly] has joined #joinmarket
09:40 -!- jungly [~jungly@gateway/tor-sasl/jungly] has quit [Remote host closed the connection]
09:40 <+JoinMarketRelay> [hackint/nikolai] had the same issue, restarted OB and FB amount shows. thanks!
09:44 <+JoinMarketRelay> [hackint/nikolai] SomeFellow: you could always setup additional firewall rules. using iptables here to  only allow connection to and from specific IP address that corresponds to a tor bridge ip. anything else, (input, output, forward) not corresponding to that rule is dropped.
09:47 <+JoinMarketRelay> [hackint/nikolai] SomeFellow: i hope this short iptables rules will help you get the idea :) http://ix.io/3uH7
09:48 < SomeFellow> nikolai Refreshed my OB after you said your FB started working, didn't notice any new ones. Restarted my OB, and a new FB appeared. It could have coincidentally appeared in the ~10 seconds that took, I guess. If not, maybe new FBs don't appear properly without an OB restart?
09:48 < SomeFellow> nikolai I'll look into it. Thanks.
09:50 <+JoinMarketRelay> [hackint/nikolai] of course, modify these to your own needs. if you do not use a custom tor bridge, the easiest might be to periodically get list of tor entry nodes and apply iptables rules with those ips, allowing traffic towards them and from them. you might want to include ports in iptables rules when possible to make it a bit more strict and viable to attacks
09:50 <+JoinMarketRelay> [hackint/nikolai] s/viable/not viable/
10:05 <+JoinMarketRelay> [hackint/nikolai] better said, less viable :)
10:14 <+JoinMarketRelay> [hackint/nikolai] woah. i didn't see all the options in the OB navigation. pretty nice. it's visible how big difference in bond value can be when locktime is a couple of years instead of few months.
10:22 <+JoinMarketRelay> [hackint/nikolai] i am looking around to understand how are bond values pushed. wouldn't it be safer so that every taker decodes redeemscript with 'decodescript' instead of trusting the data pushed by maker.
10:29 < waxwing> see FidelityBondMixin.get_validated_timelocked_fidelity_bond_utxo()
10:29 < waxwing> it regenerates the script from the pubkey and locktime and validates that the script matches. afair.
11:42 -!- JMBridged [~CoinJoins@gateway/tor-sasl/jmbridge] has joined #joinmarket
11:46 -!- JMBridge [~CoinJoins@gateway/tor-sasl/jmbridge] has quit [Remote host closed the connection]
11:46 -!- JMBridged is now known as JMBridge
12:12 <+JoinMarketRelay> [hackint/nikolai] thank you
14:08 <+JoinMarketRelay> [hackint/person] belcher waxwing once FB gets unlocked and unfrozen and JM's yg wants to use it the yg crashes with "Invalid private key", full log: https://pastebin.com/9aRztD7p
14:12 < waxwing> prob need to autofreeze in case it's yg running, 
14:16 <+JoinMarketRelay> [hackint/person] what are you saying?
14:18 <+JoinMarketRelay> [hackint/person] FB documentation says that once FB gets unlocked we can unfreeze it and yg will spend it. this is not true, it's crashing
14:22 < waxwing> i was saying (just first thought) that we need to freeze during yg running, but i was not aware the doc said that. it should definitely be possible to do it that way, so  yeah, i agree.
14:23 < waxwing> yeah my original thought made no sense, sorry
14:24 <+JoinMarketRelay> [hackint/person] once it got unlocked it automatically got frozen, that's not the issue. i stopped yg, unfrozen it, started yg and ever since whenever yg wants to use this unlocked&unfrozen FB utxo it crashes
14:24 < waxwing> yes, i understand what you're saying
14:24 <+JoinMarketRelay> [hackint/person] it is a huge problem
14:24 < waxwing> well not a *huge* problem. you can definitely spend it outside the yg.
14:25 <+JoinMarketRelay> [hackint/person] yes and pay for it, no thanks. i would prefer if what was written in the documentation was actually true
14:25 < waxwing> PRs welcome.
14:26 <+JoinMarketRelay> [hackint/person] great. you make a new feature, it doesn't work, now you ask the users to fix it for you
14:30 <+JoinMarketRelay> [hackint/person] i froze it again, maybe once more people will get their FB unfrozen and to their horror find themselves unable to spend it with their yg, maybe this bug will eventually gets fixed
14:32 <+JoinMarketRelay> [hackint/person] and i am not so sure if one "can definitely spend it outside the yg", since the same should be true for spending it inside the yg and that's not true. the error itself doesn't seem to be exclusive for yg
14:42 <+JoinMarketRelay> [hackint/kristapsk] person, JoinMarket is not a for-profit company, it's just a bunch of open source developers working on it in their spare time, so you can't expect immediate response on any support calls / bug reports you might have
14:44 <+JoinMarketRelay> [hackint/person] i don't expect immediate response, i expect a normal human reaction, i.e. acknowledging the huge problem and apology for incorrect documentation and misleading the users into gettings their money stuck, not insolence and arrogance
14:46 -!- belcher_ is now known as belcher
14:47 < belcher> waxwing fidelity bonds still have a value even after the locktime has expired
14:47 < belcher> the value exponentially decays with r=interest_rate
14:47 < belcher> so the user can wait a little bit to spend the coin into a new timelocked address, they dont need to pay huge miner fees to get it confirmed quickly for example
14:58 <+JoinMarketRelay> [hackint/person] spending of unlocked&unfrozen FB via sendpayment.py doesn't work either, it crashes with "witness script is not specified for segwit native p2wsh input at index 0", everyone who used FB gets their money stuck
14:58 <+JoinMarketRelay> [hackint/kristapsk] person, if you have a github account, I would recommend you to fill an issue at https://github.com/JoinMarket-Org/joinmarket-clientserver
14:59 <+JoinMarketRelay> [hackint/person] i don't have a github account and there are two main JM developers present
15:01 < waxwing> belcher, ah yes, forgot that point thanks.
15:02 < waxwing> re: spending, you definitely can spend after timelock via sendpayment.
15:02 <+JoinMarketRelay> [hackint/person] i just said i definitely cannot spend it via sendpayment
15:06 < waxwing> i'm not sure what the delta might be , person. which commit are you running off (or just v0.9.0)?
15:06 <+JoinMarketRelay> [hackint/person] i froze all other utxos in m0 and left only the FB unfrozen, then i did python sendpayment.py -N 0 --psbt xyz.jmdat 0 destination, it crashed at "joinmarket-clientserver/jmvenv/lib/python3.8/site-packages/bitcointx/core/psbt.py", line 1020, in sign" with the error i said above
15:07 < waxwing> oh, psbt. ah.
15:07 <+JoinMarketRelay> [hackint/person] i am using recent git master
15:07 <+JoinMarketRelay> [hackint/person] yes psbt, now you are going to say that it would "definitely" work without psbt, right?
15:08 < waxwing> i wasn't going to say *exactly* that, no.
15:08 < waxwing> also "recent", please be more specific on that, just in case.
15:08 <+JoinMarketRelay> [hackint/person] i used psbt because i don't want to actually spend it, i am still hoping it will get fixed so that yg can spend it, and maybe then i can test it
15:09 < belcher> how about we wait for someone more polite to report the bug :p
15:09 < waxwing> ok. so you were doing it as a test, i see.
15:09 <+JoinMarketRelay> [hackint/person] ebf9e550045e64bcc7308221b9710bc4e157e7d0
15:09 < waxwing> the thing is, while unfortunate, it isn't surprising to me that doing this with psbt doesn't work. it wasn't tested.
15:09 < waxwing> but consider: you can test in a different way.
15:10 < waxwing> just do the sendpayment run and choose 'n' instead of 'y' when it prompts.
15:10 < waxwing> then you can even save the hex tx if you feel like it.
15:10 < waxwing> there is even a 'testmempoolaccept' command in Core.
15:10 <+JoinMarketRelay> [hackint/person] ok, i never tried sendpayment.py so i didn't know if there was any further confirmation
15:10 < waxwing> yes as long as you don't choose the `--answeryes` option.
15:11 < waxwing> with -N0 it doesn't try to coinjoin but just builds the tx.
15:15 < waxwing> i went to considerable lengths to test that locked coins become spendable (it was harder than it might be even on regtest due to time not height based locks), but all my tests were just "try sendpayment", honestly didn't consider the obvious of 'leave it and yieldgen with it' (re: psbt our support is fairly primitive so it's not surprising, but it should be documented while it doesn't work)
15:16 < waxwing> so to state the obvious, that yg spending thing will be fixed shortly (assuming it is reproducible)
15:16 <+JoinMarketRelay> [hackint/person] sendpayment without psbt seemed ok, it gave me hex tx
15:17 < waxwing> for peace of mind try the testmempoolaccept via bitcoin-cli (check the help)
15:17 <+JoinMarketRelay> [hackint/person] do want me do any troubleshooting with yg spending or any more error logs?
15:18 <+JoinMarketRelay> [hackint/person] i did that, it gave "allowed": true
15:18 < waxwing> ok good
15:18 < belcher> "unfreeze and leave it in yieldgen to be spent" might be a bad idea if it ends up co-spent with other coins you didnt intend it to be linked with
15:19 < waxwing> yeah that did occur to me too but otoh it's not always so clear the right thing. you could coinjoin it out *on its own* but that's a bit expensive.
15:19 < waxwing> the obvious one is spend it out to a new timelocked output.
15:19 < waxwing> but that's only one way someone might want to do things.
15:21 <+JoinMarketRelay> [hackint/person] all my coins are already mixed many times, the FB was one of them, if it gents spent now via yg it's perfectly alright
15:21 <+JoinMarketRelay> [hackint/person] gets*
15:21 <+JoinMarketRelay> [hackint/kristapsk] https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/955
15:22 <+JoinMarketRelay> [hackint/person] this is actually difficult to test because some taker needs to ask for a specific size of mix so that my yg picks the FB utxo
15:22 < waxwing> no worries on testing, can do it on regtest easily
15:23 < waxwing> it's late here will look at it maybe late tomorrow unless someone else does it
15:23 <+JoinMarketRelay> [hackint/person] ok so i freeze it again, unfreeze everything else and let yg run normally for the time being
15:24 < waxwing> yes
15:27 <+JoinMarketRelay> [hackint/person] while we are at it, there is also a timezone issue probably. the wallet-tool.py started showing FB as unlocked even before its OP_CHECKLOCKTIMEVERIFY locktime and it also allowed me to unfreeze
15:38 <+JoinMarketRelay> [hackint/nikolai] person: 'great. you make a new feature, it doesn't work, now you ask the users to fix it for you' - then ... code your own joinmarket. be thankful somebody is spending their free time to make joinmarket better. thank you for the new feature devs.
15:48 <+JoinMarketRelay> [hackint/nikolai] anyways, so far yg with fb working and doing cjs well.
16:09 -!- SomeFellow [~SomeFello@194.54.28.203] has quit [Quit: Connection closed]
17:14 -!- belcher_ [~belcher@user/belcher] has joined #joinmarket
17:17 -!- belcher [~belcher@user/belcher] has quit [Ping timeout: 272 seconds]
23:41 -!- belcher_ is now known as belcher
--- Log closed Mon Aug 02 00:00:26 2021