--- Day changed Mon Jul 16 2018
00:51 -!- undeath [~undeath@unaffiliated/undeath] has joined #joinmarket     
01:32 -!- undeath [~undeath@unaffiliated/undeath] has quit [Quit: WeeChat 2.1]     
04:35 -!- GitHub71 [GitHub71@gateway/service/github.com/x-mbmsxnfvqzmllvsp] has joined #joinmarket     
04:35 < GitHub71> [joinmarket-clientserver] AlexCato opened pull request #164: Yield generator fee adjustments (master...yg_feeadjustments) https://git.io/fNOts
04:35 -!- GitHub71 [GitHub71@gateway/service/github.com/x-mbmsxnfvqzmllvsp] has left #joinmarket []     
04:37 < AgoraRelay> [agora-irc/CatoAlex] meh... one more commit in there than I wanted. Doesnt have any effect on the changes, but still is ugly. After all this time, still struggling with git ;(
04:52 < waxwing> catoalex you can use do `git rebase -S -i HEAD~2` then replace second one with "squash", then reset the comment and quit, and it'll update. then you `git push -f remote your-pr-branch`
04:53 < waxwing> the -S is for sign. but anyway there should be a simple-ish walkthrough somewhere on the web. keyword is 'squash'
05:11 < Sentineo> what does this error mean?: Counterparty commitment not accepted, reason: Invalid podle pubkey: <key>
05:14 < waxwing> Sentineo, it means that a taker provided an invalid commitment opening in the !auth message . the fact that it's the pubkey is unusual though, usually if it fails it's because the utxo is too new or too small.
05:15 < waxwing> i wonder if someone is trying to do joins with segwit makers with a non-segwit wallet (just a guess, it may not even make sense, would have to analyze).
05:15 < waxwing> never seen that error myself i think.
05:16 < waxwing> hmm looks like i had it exactly once, on the 3rd of May.
05:19 < Sentineo> interesting
05:37 -!- beIcher is now known as belcher_     
05:57 < waxwing> so no comments on this thought? https://twitter.com/waxwing__/status/1018506618210783234
05:58 < waxwing> i thought it was an interesting point that i don't believe we've ever discussed before
05:59 -!- GitHub16 [GitHub16@gateway/service/github.com/x-ljpoootmpompwars] has joined #joinmarket     
05:59 < GitHub16> [joinmarket-clientserver] AlexCato closed pull request #164: Yield generator default fee adjustments (master...yg_feeadjustments) https://git.io/fNOts
05:59 -!- GitHub16 [GitHub16@gateway/service/github.com/x-ljpoootmpompwars] has left #joinmarket []     
06:14 -!- undeath [~undeath@unaffiliated/undeath] has joined #joinmarket     
06:19 -!- GitHub197 [GitHub197@gateway/service/github.com/x-pvahjoqmhaxsshhh] has joined #joinmarket     
06:19 < GitHub197> [joinmarket-clientserver] AlexCato opened pull request #165: Yield generator default fee adjustments (master...yg-defaultfees) https://git.io/fNOnz
06:19 -!- GitHub197 [GitHub197@gateway/service/github.com/x-pvahjoqmhaxsshhh] has left #joinmarket []     
06:31 < arubi> waxwing, wouldn't this kinda thing ^ (the tweet) be watermarking makers?  we know most don't change defaults
06:31 < arubi> but hm, it would be "marking" them just for *this taker so maybe who cares?  interesting
06:33 < arubi> and not like it can causes privacy wrt outputs, which is what cj is about so..  I tend to think it's ok
06:33 < waxwing> well, about watermarking: we have the same issue with fees. currently a lot (most) use an algorithm where they re-randomize their fees between transactions.
06:33 < AgoraRelay> [agora-irc/CatoAlex] why would a maker change the utxo age he accepts anyways?
06:34 < waxwing> it's not that i was suggesting they change it, only they might have their own idea what level of rate limiting is acceptable to them. perhaps they think the age needs to be 50 to protect them more robustly from snoopers.
06:34 < AgoraRelay> [agora-irc/CatoAlex] well, to throttle snoop attacks, right
06:34 < waxwing> but, shrug, it's pretty easy to see why i originally coded it to be a fixed, consensus variable. much easier that way.
06:35 < waxwing> what sparked the thought is, i wondered, what if the current rejections we're seeing aren't actually malicious?
06:35 < arubi> if anybody's going to change the value, I'd guess they'll reduce it rather than increase
06:35 < AgoraRelay> [agora-irc/CatoAlex] but if the maker answers his accepted utxo age, this would indeed watermark him. Because snoops can always get the maker to return it
06:37 < waxwing> well but even more, the idea would be that he'd publish it in the offer, so it'd be public
06:37 < arubi> in my mind it'll be the taker telling its own utxo age and the maker either says "yes" or "no"
06:37 < waxwing> as i said, i think it has the same status as fees, where we already addressed this watermarking issue
06:38 < waxwing> arubi, i think the only way it can work is (a) maker includes his age requirement in the offer (b) taker filters the orderbook to only those he can fulfil (although that's a bit twisty, code wise)
06:38 < waxwing> i'm not really suggesting it tbh, at least not right now. it's a bit annoying. but it occurred to me we'd never really considered it.
06:39 < waxwing> and right now, makers can make this decision with an edit to the config file, don't even have to touch code.
06:39 < AgoraRelay> [agora-irc/CatoAlex] if the maker only says yes/no after the taker already wants to initiate a CJ, then the taker would have to restart quite often (burning his commitments) or risk lower privacy (by falling back to minimum_makers if only some makers accept his age)
06:39 < waxwing> it might not be irrational, at all.
06:39 < waxwing> catoalex that's why i'm saying taker filters the orderbook based on this, and why i'm saying maker advertises it in !*offer
06:39 < waxwing> to avoid that issue
06:40 < arubi> ah right, wasting commitments.
06:40 < undeath> to me offers already seem pretty easy to distinguish from all the variables available, is full maker anonymity really possible?
06:41 < waxwing> seems sort of half-true really. you can avoid re-announcements, you can randomize values, but mostly makers are linkable tx to tx. as long as they're not completely transparent in terms of *all* the utxos they hold, we're in a reasonable situation i think.
06:42 < AgoraRelay> [agora-irc/CatoAlex] yeah, i see where you're coming from waxwing. If included in the !offer, its easy to get it working. So no downside there. Wonder if enough makers care at all, if the change would be amde
06:42 < AgoraRelay> [agora-irc/CatoAlex] *made
06:42 < waxwing> hp2 announcements are passed around before broadcast also
06:42 < waxwing> catoalex i guess this is not an issue, i was just wondering really.
06:45 -!- GitHub191 [GitHub191@gateway/service/github.com/x-sssceeiqffbtojti] has joined #joinmarket     
06:45 < GitHub191> [joinmarket-clientserver] AdamISZ closed pull request #159: Allow Takers to restart with honest makers (master...restart-with-subset) https://git.io/fNJzP
06:45 -!- GitHub191 [GitHub191@gateway/service/github.com/x-sssceeiqffbtojti] has left #joinmarket []     
06:45 -!- GitHub29 [GitHub29@gateway/service/github.com/x-vlnsnvjxgpfzgbvz] has joined #joinmarket     
06:45 < GitHub29> [joinmarket-clientserver] AdamISZ pushed 1 new commit to master: https://git.io/fNOWn
06:45 < GitHub29> joinmarket-clientserver/master 8316683 AdamISZ: Merge #159: Allow Takers to restart with honest makers...
06:45 -!- GitHub29 [GitHub29@gateway/service/github.com/x-vlnsnvjxgpfzgbvz] has left #joinmarket []     
06:46 < waxwing> i assume no one objects to #165 .. bikeshedding not in evidence :)
06:48 < undeath> what's the difference in amount of cjs between the default fees and the newly proposed ones?
06:49 < undeath> if it's less than 1:10 it doesn't make sense
06:49 < waxwing> spoke too soon :)
06:49 < waxwing> grubles, ping, ^
06:50 < arubi> how many satoshis is the cjfee_factor after the order of magnitude fee decrease?
06:53 < arubi> oh +-50 satoshis based on the absolute fee I guess?  that's ok I think
06:53 < AgoraRelay> [agora-irc/CatoAlex] nothing really changed there, thats still the same 10% default for yg-privacyenhanced.
06:53 < AgoraRelay> [agora-irc/CatoAlex] so if absfee=500, its +/- 50 indeed
06:54 < waxwing> how about just random-under-a-maximum instead? :)
06:55 < waxwing> i honestly don't know, as of today i've found making that reduction in fees useful, not least because i want to support people's ability to do joins without blocking, but of course that changes over time. there isn't a "right" number.
06:55 < AgoraRelay> [agora-irc/CatoAlex] after having slept over this a few nights, i tend to agree with random-under-maximum
06:56 < waxwing> the restart-with-subset and complete-with-subset help quite a lot; you'll probably have noticed how on average the number of counterparties is less than it used, for that reason, but at least they go through
06:57 < undeath> would it make sense to increase the default number of counterparties then?
06:57 < undeath> what's the default?
06:57 < waxwing> yes alexcato i'm reasonably sure that for non-blocking, a random selection is kinda "optimal", so you just have to decide what fee you're willing to accept and make that a limit
06:57 < AgoraRelay> [agora-irc/CatoAlex] (a) better for taker privacy (b) can still be manually overriden if the user does not want it. Downside: (c) cjfee using defaults will be higher on average than it is now
06:57 < waxwing> undeath, the default is 5 to 7, randomly
06:57 < undeath> ah, ok
06:57 < undeath> that seems ok, even if one bails out
06:57 < waxwing> both for sendpayment and tumbler
06:58 < waxwing> yeah but i suspect we're getting average of about 2 bailing out, looking over lots of examples (including myself as taker)
06:58 < undeath> oh     
06:58 < undeath> maybe increase to 6-8 then?
06:58 < waxwing> it's possible. i wouldn't want to go above8 probably, without bikeshedding yet another number: maker_timeout_sec
06:59 < AgoraRelay> [agora-irc/CatoAlex] if you increase that too much, you'll run into "not enough counterparties to do joins" problems before you even find out which ones bail
06:59 < waxwing> oh that as well, liquidity
06:59 < undeath> true
06:59 < undeath> that's tough
06:59 < waxwing> 5-7 leave it for now? i think so
06:59 < AgoraRelay> [agora-irc/CatoAlex] also its quite easy for malicious makers to just add some more
06:59 < undeath> require a minimum after considering bailouts?
07:00 < AgoraRelay> [agora-irc/CatoAlex] i'd leave it and consider the random-under-maximum
07:00 < AgoraRelay> [agora-irc/CatoAlex] we have that minimum already
07:00 < undeath> what is it?
07:00 < AgoraRelay> [agora-irc/CatoAlex] it's configurable in minimum_makers in the config
07:00 < AgoraRelay> [agora-irc/CatoAlex] default=2
07:00 < undeath> I see
07:00 < waxwing> yes rand-under-max is the only real defence we have but combined with lots more liquidity really.
07:01 < undeath> rand-under-max sounds like an interesting thing, that even allows some maker anonymity
07:01 < waxwing> so, this was a bit controversial, as expected :)
07:01 < undeath> but I guess it's hard again to find a limit everyone agrees to
07:01 < AgoraRelay> [agora-irc/CatoAlex] well, good thing here is: not needed to have consensus here, because the takers can use it if they like, or not if they prefer it the way it is now
07:02 < AgoraRelay> [agora-irc/CatoAlex] it's just about defaults again... which arent changed that often
07:05 < waxwing> also my argument that rand-under-max is somehow 'optimal' is .. dubious at best :) i just couldn't think of a better way to deal with the current problem that, any malicious participant can simply observe current prices, and set themselves lower than that, then exploit the fact that the exp distribution will then massively favour them.
07:07 < waxwing> i also like it's dynamics: if a user is finding themselves too often getting "bad" cheap liquidity, and they bump up their maximum, it's very likely to broaden the scope of what they get, whereas with price-weighting, it has increasingly less effect whether you're willing to accept higher prices, because your algo won't choose them anyway
07:07 < waxwing> s/it's/its/
07:09 < undeath> where is the distribution calculated?
07:10 < waxwing> it's choose_orders code, i think in support.py, i'll check
07:10 < undeath> it's also possible to simply adjust it a bit to still do weighted selection but set a max probability difference
07:10 < waxwing> yes weighted_orders_choose in jmclient/support.py
07:11 < undeath> possibliy allowing to filter out makers that are too expensive for the user
07:11 < waxwing> undeath, yes that's quite logical (max probability difference)
07:11 < waxwing> we can code all of them though, then only have to bikeshed the default :)
07:11 < AgoraRelay> [agora-irc/CatoAlex] Working on the tx-fee defaults if the fee estimation of bitcoin core does not work: currently its 10k sat/kb. Imho that default is already acceptable. Does anybody disagree here?
07:12 < waxwing> yeah we just changed it to 10k, so i guess we agree :)
07:12 < waxwing> it was 100k but undeath noticed
07:12 < undeath> we just recently changed it from 100k to 10k
07:12 < AgoraRelay> [agora-irc/CatoAlex] oh. Alright :D
07:13 < AgoraRelay> [agora-irc/CatoAlex] one number still needs a change though, imho: absurd_fee_per_kb currently is 350 sat/b
07:14 < AgoraRelay> [agora-irc/CatoAlex] i'd reduce that to something in the 50-100 range?
07:14 < waxwing> well, probably 350 is too high, but you can debate that too: iirc Core had it at like 1 million (at least when i was looking at this)
07:14 < AgoraRelay> [agora-irc/CatoAlex] (as context: that number is used if bitcoin core does return a valid estimation, which might be very high though. This is the emergency stop if it surpasses this threshold)
07:15 < undeath> 350 is high, but the user gets to see it and can decide if it's worth it for them
07:15 < undeath> so I think it's ok
07:15 < AgoraRelay> [agora-irc/CatoAlex] its configurable in joinmarket.cfg anyways, this is just to make accidental takers aware imho; thinking of someone trying to mix 0.1 btc but would los like half of it to tx fees
07:15 < AgoraRelay> [agora-irc/CatoAlex] *lose
07:20 < AgoraRelay> [agora-irc/CatoAlex] i'll still prepare a PR for this, if you think it should stay at 350, thats fine with me as well. But interested in more comments
07:23 < waxwing> i'm ok with 350. fee environment changes rapidly. few scenarios where that would lose a large amount of money, i think.
07:23 < waxwing> afk     
07:42 < belcher_> how about "random-under-a-maximum" where that maximum is chosen by some algorithm, not by a default value 
07:43 < undeath> how would you imagine that algorithm to look like?
07:44 < belcher_> median of the cheapest N*3 makers in range
07:44 < belcher_> N is the number of makers you intend to do a coinjoin with
07:45 < belcher_> there's earlier discussion on the joinmarket issue #14 as well, including an idea to quantize prices (which is similar to random-under-maximum)
07:46 < belcher_> the chief problem i think is that attacking makers can auto-underbid by just a few satoshi, but users dont have any meaningful preferences for prices only a few sat cheaper
07:50 -!- qubenix [~qubenix@212.117.188.113] has joined #joinmarket     
07:52 < belcher_> another way (alluded to before) could be "random betwen max/min" and the min starts small, then if you get too many unrespnsive makers then the min goes up
07:58 < AgoraRelay> [agora-irc/CatoAlex] belcher_] median of the cheapest N*3 makers in range <-- so if i was malicious, I'd only have to sybil attack with 21 makers with fee=1 sat, and the fee ceiling is 1sat. This works right now as well (though 'only' with high probability), but would make it even worse imho
08:00 < AgoraRelay> [agora-irc/CatoAlex] "random betwen max/min" <-- better, but not much different from the current "restart with good subset of good makers if too few respondents"?
08:00 < AgoraRelay> [agora-irc/CatoAlex] thing is, i dont see a good solution either, so please dont take this as unfair criticism
08:14 -!- qubenix [~qubenix@212.117.188.113] has quit [Remote host closed the connection]     
08:16 < waxwing> belcher_, thanks for the quantization discussion pointer (lost in the mists of time now), i find that these are all slight variations on the theme: uniform over fixed range (rand under max), quantization (rand over fixed ranges), or undeath's tweak on weighted_order_choose: weighted but with min and max prob, so shallow curve instead of steep, and then what we have now
08:16 < waxwing> https://github.com/JoinMarket-Org/joinmarket/issues/14#issuecomment-143509788 is most relevant (irc log)
08:19 -!- qubenix [~qubenix@66.71.240.201] has joined #joinmarket     
08:25 < waxwing> i like random-under-max (essentially quantized with only 1 range) because easy to code and easy for user to understand.
08:25 < waxwing> one option to avoid the problem of a default maximum is to have the maximum change randomly from transaction to transaction
08:26 < waxwing> just prompt the user "is it OK to pay up to a maximum of 10283 satoshis to each counterparty?" or something
08:26 < undeath> not sure if that may end up actually seeming confusing
08:27 < undeath> ppl might assume the program is asking them whether they are going to pay that much
08:27 < waxwing> yes from UX perspective it's almost ridiculous :)
08:28 < waxwing> i'm more thinking the general idea. what i'm trying to get at is, i don't think a max/ceiling chosen per taker, should really be an issue in terms of fee-sniping or whatever it's called.
08:28 < waxwing> because it's not something globally agreed.
08:29 < undeath> and there would have to be some minimum again
08:29 < waxwing> subtle i guess, depends on how seriously you take the tyranny of defaults
08:29 < waxwing> minimum?
08:30 < undeath> if the random number ends up being 5 it wouldn't find any counterparties
08:30 < waxwing> oh you mean that. yes, a range of course, agreed, so it's fuzzing the edge not removing it.
08:31 < undeath> that only leaves to determine said edge :)
08:31 < waxwing> it's easy to get lost in the details; the core concept is we want to dilute the bad actors amongst the honest actors. there needs to not be a range where they can cluster to advantage (right now, the very bottom)
08:32 < waxwing> of course if bad actors >> honest actors we're (currently) completely stuffed, whatever we do
08:32 < undeath> I think the max prob diff could help a fair bit
08:32 < undeath> and is a pretty simple change
08:32 < waxwing> but if not, even up to say a 50/50 split the system can still kinda work if we don't give them a huge target to aim at (bottom of orderbook)
08:33 < waxwing> undeath, sure but essentially you're just proposing to make the curve shallower, and i'm just proposing to make it flat. not such a big difference :)
08:33 < undeath> there's still some number to figure out though :/
08:34 < undeath> that's true, that might actually be better
08:34 < waxwing> there are currently two params there: the lambda for the exp function, and a sanity check "max fee", i want to reduce that to one: "max fee", although i may fail if we want to randomize that max.
08:34 < undeath> it's even harder to game
08:35 < waxwing> right, that's what i've been saying: maximize randomness minimize gameability .. but we can't just accept any fee, hence all the above theories
08:41 < undeath> but what exactly will randomizing the max achieve?
08:42 < undeath> if I was malicious I'd just set the fee low enough to always be selected
08:42 < undeath> *considered
08:44 < waxwing> undeath, but then you're going to be selected 1/N so to speak
08:45 < waxwing> whereas with a weighting you get a much higher chance by going right to near-zero
08:45 < undeath> yes, but why randomizing the max?
08:45 < waxwing> oh, why randomizing the max. yes good point, i think i might have got confused there.
08:46 < waxwing> assuming malicious actors are not economically motivated, not sure it's helpful/needed
08:50 < waxwing> undeath, if you want to code random-under-max you're welcome, it's, as i think you already observed, easy. i would be inclined to make these changes: make it the default option for sendpayment and tumbler, but add a CLI option to change to weighted. for Qt, i think there's a field to choose that already, but with only one choice, again i would switch the default and keep weighted optional.
08:52 < waxwing> i'll do it if you don't want to, but do other people agree that this is a better default than the current weighted?
08:52 < undeath> I can do that, seems easy enough
08:53 < AgoraRelay> [agora-irc/CatoAlex] do other people agree that this is a better default than the current weighted? <-- i agree
08:53 -!- MaxSan [~user@185.9.19.107] has joined #joinmarket     
08:59 < waxwing> belcher_, ? i
08:59 < waxwing> .. feel like you're not convinced
09:00 < waxwing> your algorithmic idea kinda makes sense to me really
09:00 < waxwing> avoids a clustering issue (of honest participants) which could end up being a bit screwy
09:15 -!- grubles [~grubles@unaffiliated/grubles] has quit [Ping timeout: 264 seconds]     
09:33 -!- MaxSan [~user@185.9.19.107] has quit [Ping timeout: 248 seconds]     
09:41 -!- lnostdal [~lnostdal@77.70.119.51] has quit [Ping timeout: 240 seconds]     
09:45 -!- grubles [~grubles@unaffiliated/grubles] has joined #joinmarket     
09:58 < undeath> waxwing: how should the limit for rand-under-max be specified? config? hardcoded? because support.py does not (yet) use anything from jm_single
10:02 < waxwing> undeath, as i mentioned, it'd have to be a CLI in sendpayment and tumbler, you'll note (somewhere) that the choose_orders function is a callback (iirc in Taker), i'll take another look later, busy rn
10:13 < undeath> I don't mean the select function itself but the max fee limit for choosing orders
10:14 < undeath> as far as I can see select functions don't accept any kind of such arguments right now
10:27 < waxwing> yes, i think you can use a CLI to set a jm_single global var such that the replacement for weighted_order_choose just filters the set of offers to remove those above that threshold
10:27 < waxwing> i mean i guess that's the natural way
10:28 < undeath> ok     
10:28 < waxwing> maybe not architecturally too good i admit
10:28 < undeath> would that be required or should there be some default?
10:29 < waxwing> the current check like filter_order_callback just checks the total fee; maybe there's an argument it should be kept like that
10:29 < waxwing> oh, no, that doesn't make sense, ignore that
10:29 < waxwing> undeath, CLI settings have a default
10:30 < undeath> k     
10:30 < undeath> it's a little hard coming up with a good default here when you don't know the cj amount
10:31 < AgoraRelay> [agora-irc/CatoAlex] should be a % of total cj amount
10:31 < waxwing> oh right, we'd want this to be a percentage i'd think (calced whether offer was rel or abs)
10:32 < AgoraRelay> [agora-irc/CatoAlex] dont think we need to distinguish between rel/abs at this level. If the % is, just as example, 0.02% and for the instantiated CJ amount from a taker that is, 20k sat, then all abs-offers below that level should automatically be possible to be selected
10:32 < AgoraRelay> [agora-irc/CatoAlex] just like it currently is if you use the -P option
10:32 < AgoraRelay> [agora-irc/CatoAlex] eh, -C
10:41 -!- lnostdal [~lnostdal@77.70.119.51] has joined #joinmarket     
11:55 -!- beIcher [~user@unaffiliated/belcher] has joined #joinmarket     
11:57 -!- belcher_ [~user@unaffiliated/belcher] has quit [Ping timeout: 260 seconds]     
12:14 -!- MaxSan [~user@185.9.19.107] has joined #joinmarket     
12:35 -!- undeath [~undeath@unaffiliated/undeath] has quit [Quit: WeeChat 2.1]     
13:29 < waxwing> lol @ J5Bet77eF3HiKmBB
13:33 < waxwing> does anyone know if there are any non-segwit makers active? i haven't seen a single !reloffer or !absoffer after more than a week in the cgan pit (i've been lurking there)
13:33 < waxwing> and yet my obwatcher still claims there's 20 active there. i guess they're just sitting there the whole time doing nothing?
15:32 -!- MaxSan [~user@185.9.19.107] has quit [Ping timeout: 264 seconds]     
16:35 -!- grubles [~grubles@unaffiliated/grubles] has quit [Quit: Leaving]     
16:58 -!- grubles [~grubles@unaffiliated/grubles] has joined #joinmarket     
17:24 -!- AgoraRelay [~jmrelayfn@p5DE4AFC2.dip0.t-ipconnect.de] has quit [Ping timeout: 268 seconds]     
17:38 -!- AgoraRelay [~jmrelayfn@p5DE4AFF4.dip0.t-ipconnect.de] has joined #joinmarket     
19:24 -!- Giszmo [~leo@pc-72-54-46-190.cm.vtr.net] has joined #joinmarket     
19:35 -!- Giszmo [~leo@pc-72-54-46-190.cm.vtr.net] has quit [Quit: Leaving.]     
19:38 -!- Giszmo [~leo@pc-72-54-46-190.cm.vtr.net] has joined #joinmarket     
19:38 -!- Giszmo [~leo@pc-72-54-46-190.cm.vtr.net] has quit [Client Quit]