--- Day changed Wed Oct 19 2016
01:09 -!- mkarrer_ [~mkarrer@7.red-83-47-85.dynamicip.rima-tde.net] has quit [Read error: No route to host]     
01:10 -!- mkarrer [~mkarrer@7.red-83-47-85.dynamicip.rima-tde.net] has joined #joinmarket     
01:48 -!- mkarrer_ [~mkarrer@109.69.10.67] has joined #joinmarket     
01:51 -!- mkarrer [~mkarrer@7.red-83-47-85.dynamicip.rima-tde.net] has quit [Ping timeout: 252 seconds]     
04:24 < GithubBot5678> [joinmarket] AdamISZ pushed 2 new commits to develop: https://git.io/vPSuU
04:24 < GithubBot5678> joinmarket/develop e256672 AlexCato: Add quick install notes for tails...
04:24 < GithubBot5678> joinmarket/develop 38e271d Adam Gibson: Merge #635: Add quick install notes for tails...
04:34 < waxwing> a small follow up to yesterday: https://github.com/JoinMarket-Org/joinmarket/issues/628#issuecomment-254786814 ; i find myself wondering why this hasn't cropped up before, or has it?
04:43 < waxwing> ok, a further thought on adlai 's taker-sophistication proposal: his original proposal was more about using information about utxos to aid selection, but the extension that was discussed in Milan, and was prompted by the observation of (obvious) duplicate bots has an important problem i think:
04:44 < waxwing> consider that the list of utxos provided at the point of !ioauth is unverified; thus, if we discard Makers offering the same utxo as other makers, we create another vector for makers to attack each other
04:45 < waxwing> since a malicious maker can choose to send you a list of utxos including utxos he has observed (by running taker scripts, or by observing blockchain outputs perhaps) as belonging to other makers, thus getting them banned.
04:45 < waxwing> you can imagine someone deliberately running one entirely fictitious maker that does this, and thus removing itself + one or more other makers from consideration, thus bumping the chance for his real maker.
04:46 < waxwing> well, details, but generally it's introducing another dos vector unless all the utxos offered are verified with signatures at the time of !ioauth
04:46 < waxwing> which would be a ton of extra data
04:49 < waxwing> *maybe* as little as a few hundred bytes with pubkey recovery? (assuming say 2-5 utxos which is normal). can it be compactified using a schnorr type aggregation?
04:50 < waxwing> hmm, no you can't have it both ways; if you use pubkey recovery, you need the individual signatures right
05:11 < adlai> waxwing: have you had a chance to study the possibility of a set exclusion proof?
05:12 < adlai> aiui, if you merkelize an ordered tree, you can prove that the two branches sandwiching where the excluded item would be, are touching
05:13  * adlai still isn't entirely clear on how this would be used in a protocol
05:28 < waxwing> adlai: my earlier mumbling on this channel about it was something like "seems like fantasy" :) in the fantasy world, i'd want the makers each to publish a compact commitment to the taker such that the taker could, by some calculation, verify that none of his maker's commitments would open to sets of utxos having any utxos in common
05:28 < adlai> so strictly speaking, you're looking for a proof of null intersection
05:28 < adlai> not nonmembership
05:28 < waxwing> but this is orthogonal to what i was worrying about above, i.e. that the utxos provided in !ioauth are currently verified
05:28 < waxwing> adlai: yes
05:29 < waxwing> sloppy language/thinking, sorry
05:29 < waxwing> s/verified/unverified/
05:30  * adlai handwaves up some interactive protocol where makers publish the merkelized ordered list commitments, then once takers have utxos from each maker, they request the nonmembership proofs from each participant maker
05:30 < waxwing> as-is, the fact that they are unverified doesn't amount to much, since it can only result in what can be done crudely: fail to complete. if we take banning action based on that info, it changes it.
05:31 < adlai> so if the maker wants to get paid for participating in the transaction, they need to cough up the nonmembership proofs on other makers' utxos
05:31 < adlai> takers should naturally ask for these on their own utxos as well, so the maker can't distinguish between taker and maker utxos in the final tx
05:33  * adlai wonders whether we want to commit to the set of utxos, or the set of hp2s
05:33 < waxwing> yeah there might be something there. probably just use the one nums point rather than allowing multiple (gets complicated)
05:34 < waxwing> but that seems like the last step of the puzzle :)
05:34 < adlai> i'm not suggesting multiple nums points, i mean committing to the multiple h(podle(utxo.pubkey,numsgen))
05:35 < waxwing> right, understood. i'm just not quite up to speed on the whole picture yet :)
05:36 < waxwing> so first merkelized ordered list commitments: so this is a tree of the utxos with a deterministic ordering, is that it?
05:36 < adlai> yes     
05:37 < waxwing> if we want to make proofs like you said above "that the two branches are touching", doesn't this get very large; do we have to do this for every possibility?
05:37 < adlai> this lets makers prove two things: that their inputs were chosen in advance, and that arbitrary challenges provided by the taker were not part of the chosen set
05:38 < adlai> i think the pessimal case is still O(log(N)) where N is the number of commitments the maker includes
05:38 < adlai> specifically, it's ~4 times the tree height
05:39 < adlai> best case is 2h, if your sandwich branches are two leaves on the same merkle branch
05:39 < adlai> worst case is 4h, if they diverge at the root
05:41 < adlai> ... but there is quadratic blowup if the taker challenges every maker with every input
05:42 < waxwing> yeah was gonna ask that. it feels like you couldn't get this compact enough.
05:42 < adlai> then again - every maker already has to receive every input in the current protocol, in !tx
05:42 < waxwing> it's cool to have a compact commitment upfront (in the !reloffer or whatever), but needs somehow not to be crazy slow
05:42 < adlai> so this is only a constant factor worsening relative to the current transmissions
05:43 < adlai> er, log.
05:44 < adlai> the worsening seems to be concentrated on the taker
05:44 < waxwing> say average 3 utxos actually *used* per maker (but their tree would be bigger if commitment in offer, as it would include all they're offering, right?), and 5 makers, does it mean we've got like 12 proof requests for each maker? or am i way off track in how you imagine this working.
05:45 < adlai> that sounds correct.
05:46 < adlai> but again, we already have a 15fold transmission to each maker, because they need to get the whole tx, and send it back signed (with every one of their utxos)
05:47 < waxwing> it's not impossible i think, but at some point we may as well just zksnark everything :)
05:47  * adlai mumbles something about dependency bloat
05:49 < waxwing> so the proof request is, say, just a single H(P2) and you might send 12 to each maker. then maker has to send back a data structure to prove. how big is that?
05:50 < adlai> at least 2*log(count(maker_hp2s))*sizeof(hp2), at most twice that
05:51 < waxwing> ok, well it's feasible i guess. might finally push us to get a non-IRC MC actually working :)
05:51  * adlai isn't certain yet that hp2s are needed here, by this point in the protocol don't takers know maker inputs?
05:51 < waxwing> yeah i'm not certain either, but it doesn't make any difference space-wise
05:51 < adlai> ...... but using hp2s means the makers don't leak inputs that aren't used in this transaction
05:52 < waxwing> right .. if you didn't, the proof would contain utxo hashes right? which can be backsolved against utxo set.
05:52 < adlai> my thinking exactly.
05:53 < waxwing> yeah, let's say, handwave that that's right, it's the same kind of thinking as before
05:54 < waxwing> oh hang on your formula is for one proof, right, so don't you need 12 of those? (with my numbers)
05:54 < adlai> yes     
05:54 < adlai> sorry, overlooked that.
05:57 < adlai> 2X <= proof size <= 4X, where X = 2*log(length(thismaker.inputs))*length(vin)*sizeof(hp2), length(vin) ~= thismaker.inputs * makercount
05:58 < waxwing> i'm not sure about hp2 now, because it's useless unless you open it (it's a commitment).
05:58 < adlai> makers also must open hp2s for the inputs they use
05:58 < waxwing> yes; openings of those are very bloaty
05:59 < adlai> but linear, not quadratic
05:59 < adlai> they prove exclusion on every other input, but only open hp2s on their own
06:00 < adlai> low-hanging fruit: encode !hp2 in base64
06:01 < waxwing> yeah i get the idea, but it's more data: for 3 it's 4 32 byte values roughly (P2, P, s, e), not counting utxo itself because elsewhere
06:01 < waxwing> b64 yeah, that's a given
06:01 < waxwing> i mean, it *should* be :)
06:02 < waxwing> i should have said, for *1* it's 4 32 byte values, so 3 times that for 3 utxos
06:06 < adlai> so, for makercount = 4, 3 inputs used out of 64 total (number pulled from my ass), i calculate: 2*6*12*32 = 4608 bytes for the exclusion proofs, plus 3*4*32 = 384 bytes for the openings
06:07 < adlai> sorry, 4608 is X, so 9216 <= proof <= 18432
06:07 < adlai> (my formula was too high, instead of length(vin) it's (length(vin)-length(thismaker.inputs))
06:07 < adlai> )     
06:07 < adlai> doing this over irc will be... interesting
06:08 < adlai> given that some transactions are much, much larger than this
06:08 < waxwing> right, was going to make a joke about going back to my websockets implementation :)
06:09 < adlai> we could always have the challenges be probabilistic
06:10 < waxwing> true, just request a proof of 1 or 2, not exactly a 'solution' but worth considering
06:11 < adlai> random choosing is probably inferior to picking the largest inputs
06:12 < adlai> maybe. that was my intuition but i'm having a hard time justifying it properly.
06:16 < waxwing> now i'm thinking we've missed the point in this. consider, if you publish such a commitment early, in !Xreloffer, you haven't at that point given information that indicates that there is no overlap; each commitment will be unique except for the super-dumb completely identical bots.
06:17 < waxwing> and since when you reveal the utxos in !ioauth, you give the exact set of utxos you're going to use anyway, what is gained? if any banning were to occur it would only be at that step. there isn't any early-filtering achieved from doing this, right?
06:18 < adlai> it prevents makers who might use the same inputs, but don't.
06:19 < adlai> ie, are almost certainly controlled by the same entity
06:19 < waxwing> but if a banning rule is in place, they would be similarly dis-incented right
06:20  * adlai is imagining a "smart sybil" that runs multiple makers off the same wallet, but coordinated to provide distinct inputs
06:20 < waxwing> well as we discussed if they always provide different, that's fine, no?
06:21  * adlai thinxs
06:21 < waxwing> (assuming they had edited the code to make it safe)
06:21 < adlai> what do you mean by a "banning rule"?
06:22 < waxwing> basically what's in the gist, and what we mentioned in milan, that if common utxos were found in maker sets after ioauth, ban both
06:22 < adlai> ok. "ban" is a bit of a strong word there, unless we persist these bans (in what form?)
06:23 < waxwing> right, "don't use" is more accurate than ban, sorry
06:23 < adlai> re: distinct inputs from the same wallet... i do remember us mentioning this in milan as less of a problem, but i'm still not sure it's a situation we want to tolerate
06:23 < waxwing> at some point the difference between "smart bot that isolates utxo sets from same wallet" and "different wallets" is too blurred, not sure we can do much about that.
06:24 < adlai> there is a difference, which is that the latter don't automatically merge inputs, whereas the former could
06:24 < waxwing> although for sure we should strongly signal to people not to do that without totally rewriting the code, as it's going to be a mess and likely privacy-breaking
06:24 < adlai> think of this in terms of mixdepths. two bots running off distinct mixdepth ranges is OK, two bots running off the same ones - less so.
06:24 < waxwing> yes i don't think there's any lack of clarity in dis-recommending it, but it's not what we can work on at protocol level, right; there, we can only take action to prevent simultaneous offering of same coins
06:25 < adlai> what are you saying is privacy-breaking? i don't think this proposed challenge-response system exposes information beyond that which is currently exposed
06:26 < waxwing> no, i was referring to people running two bots off the same wallet; that's privacy-breaking (likely) without code-rewrite
06:27 < adlai> does a sybil attacker doing this in order to eliminate larger parts of the graph care about their own privacy?
06:27 < waxwing> no, but it's worth mentioning for those people who want to make more money but not so much that they want to kill JM for its users :)
06:31 < waxwing> heh funny, if the maker published a podle that was somehow tied to a unique transaction id for *all* of his offered utxos immediately in the offer, it would have the property i wanted (early-filter duplicates), but that would make the !offer like 1kB long :)
06:32 < waxwing> i say 'tied to a unique transaction id' as a handwave of, it has to keep private if it gets republished many times, no idea what i really mean there
06:32 < adlai> doesn't that leak when the maker utxos change?
06:32 < waxwing> no that idea is horrifyingly dumb, but funny anyway
06:33 < waxwing> well that's what i meant about 'unique to transaction' (actually it would have to be unique to offer, wouldn't it)
06:33 < waxwing> horrifying for many reasons i think, e.g. leaking the number of utxos even if perfectly hiding
06:33 < adlai> it seems to me to be an inferior solution... although you could pad with dummy commitments
06:33 < waxwing> yes, and the number of utxos changes at certain points, even worse
06:34 < waxwing> padding: yeah just what that idea needs, more data :) /jk
06:34 < waxwing> sometimes it seems like a good idea to mention bad ideas, though :) can spark new thoughts maybe
06:35  * adlai bbl, errands; but this was a good brainstorm!
06:37 < waxwing> pit seems fairly active, someone knows what they're doing, i guess
06:37 < waxwing> is anyone else finding agora acting flaky right now? seem to be reconnecting all the time
06:37 < waxwing> maybe smuggler is fed up with it
06:38 < JM-IRCRelay> [AlexCato] my agora-connection is rock-solid (connected to the tor hidden service though)
06:39 < JM-IRCRelay> [AlexCato] and yeah, pit is active. Since the sybils are gone, the completion-rate of started joins has skyrocketed
06:40 < JM-IRCRelay> [waxwing] AlexCato: are you going to twiddle the yg-pe/basic for randomisation?
06:40 < JM-IRCRelay> [waxwing] i can't remember what my latest thought on that was. it isn't so obvious the right approach
06:41 < JM-IRCRelay> [AlexCato] can do the randomization-YG, but it would get a new name, since you're absolutely right when you said that it's not about PE any more then
06:42 < JM-IRCRelay> [AlexCato] if you want to see that, i'll get it done somewhen tonight
06:42 < waxwing> isn't it? i'm confused. not reannouncing is one thing, randomising fee is another.
06:42 < waxwing> don't worry there's no rush.
06:42 < JM-IRCRelay> [waxwing] AlexCato: what's the port for the HS?
06:43 < JM-IRCRelay> [AlexCato] not just the fees (cj + tx), i'd randomize everything, including min and max offer sizes
06:43 < JM-IRCRelay> [AlexCato] port is 6667
06:43 < waxwing> yeah that's it, just found it. thanks
06:45 < JM-IRCRelay> [AlexCato] randomizing vs. not announcing: if a maker's fees change, isnt a re-announce necessary? I know it doesnt matter for sendpayments, because the privmsgs would contain updated offers
06:45 < JM-IRCRelay> [AlexCato] but afaik the tumbler doesnt request the !orderbook again in future steps
06:45 < JM-IRCRelay> [AlexCato] what would happen if the tumbler remembers offers, but the maker changed them without re-announcing?
06:45 < JM-IRCRelay> [waxwing] oh yeah that was it, contradictory intentions .. yeah, let's have a separate one
06:46 < JM-IRCRelay> [waxwing] i think we should make a new repo for custom scripts
06:46 < waxwing> alexcato what did you mean when you said sybil left?
06:47 < JM-IRCRelay> [AlexCato] the 5-7 guys with the 7.4x max size offers are gone, at least according to your orderbook-site. Last time i checked
06:47 < waxwing> i think back
06:48 < JM-IRCRelay> [AlexCato] bah. Everything went fine for a while. Still think they're responsible for many commitments to be burned
06:48 < waxwing> i might do a little prying right now. afaict the attack is economic, but it will lead to tx failures almost certainly, unless he's been clever.
06:58 < waxwing> hmm, seems i was wrong, no response
07:01 < waxwing> non-response to !auth from 3 of those 10
07:02 < waxwing> so perhaps yes, someone is deliberately just trying to block transactions. surprising.
07:10 < waxwing> to re-state the obvious: use -P and don't choose the guy using 7.49... as his maxsize; doesn't help for tumbler unfortunately. and yes a patch to reject equal sized min/max would "fix" it, until he changed it :) add-utxo.py can be used to get a buffer of commitments if needed.
07:11 < waxwing> maybe "certain parties" are doing this to try to force more leniency so they can get back to spying.
07:24 -!- imposter [uid57046@gateway/web/irccloud.com/x-avpeietnkrvhhyif] has joined #joinmarket     
07:32 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has joined #joinmarket     
08:40 -!- Anduck [~Anduck@unaffiliated/anduck] has quit [Quit: leaving]     
08:45 -!- mkarrer [~mkarrer@7.red-83-47-85.dynamicip.rima-tde.net] has joined #joinmarket     
08:49 -!- mkarrer_ [~mkarrer@109.69.10.67] has quit [Ping timeout: 256 seconds]     
08:53 -!- Anduck [~Anduck@unaffiliated/anduck] has joined #joinmarket     
09:25 < GithubBot5678> [joinmarket] AlexCato opened pull request #640: add yg-randomizer: YG to randomize all aspects of cj offers (develop...yg-randomizer) https://git.io/vP9ZF
10:13 -!- grubles [~grubles@unaffiliated/grubles] has joined #joinmarket     
10:26 < GithubBot5678> [joinmarket] AdamISZ opened pull request #641: Modify default fee to 0.02% (develop...default_fee_change) https://git.io/vP90O
10:41 < waxwing> yes, it seems reliable that they don't respond to !auth. note btw it's 5 bots with 6 orders each, not 10 with 3 each.
10:42 < waxwing> non-response to auth seems like a bannable offence :)
10:46 < waxwing> perhaps vitalik can give us some tips on how to resist attacks, seems to be gaining a lot of experience
10:47 < arubi> rm -rf ethereum; mkdir ethereum
10:47 < waxwing> heh. one day i'll have to prune all my jm logs
10:47 -!- imposter [uid57046@gateway/web/irccloud.com/x-avpeietnkrvhhyif] has quit [Quit: Connection closed for inactivity]     
10:47 < waxwing> 300M lol
10:48 < arubi> :O, are these logs from live joins?
10:48 < waxwing> logs were always bloaty, but when the spy arrived in earnest .. sheesh
10:48 < waxwing> btw alexcato made a nice PR that was merged the other day to clean out the terminal spam at least, but it's actually nice to have all the data in the log file
10:49 < arubi> I'd probably end up `tail -f log` like I do with core
10:49 < waxwing> well if you like the spam you can still have it of course
10:49 < waxwing> i mean, on the terminal (debug level)
10:50 < waxwing> even as a super-duper user i kind of appreciate the lack of noise with the default INFO level
10:50 < arubi> at least with core there isn't anything at all that happens in cli, so it's nice to have some sense of progress and sanity
10:50 < arubi> but info messages are better if log gets everything eventually
10:50 < waxwing> yeah i basically always ran tail -f on Core from day 1
10:52 -!- DeathShadow--666 [~IDSE@S0106a84e3f595813.vc.shawcable.net] has quit [Ping timeout: 250 seconds]     
11:01 -!- trustiee [~smuxi@204.187.100.19] has joined #joinmarket     
11:01 < trustiee> I ran tumbler with -M 6 and it stopped working so I killed it. Then I run wallet-tool with -m 7 and it gives IndexError: list index out of range
11:02 < trustiee> How do I fix this so the wallet can see beyond mixdepth 6 now?
11:02 < waxwing> trustiee: yes this was a bug, fixed in latest develop
11:03 < trustiee> Shall I use the latest or recover wallet from seed now?
11:03 < waxwing> sorry afk for a couple of hours now, if you're not sure how to pull the latest from the develop branch perhaps someone else can explain to you how to do it
11:03 < trustiee> I'll just use latest. Thanks
11:03 < waxwing> no you don't need to recover or anything, just use the latest version from the develop branch
11:14 -!- DeathShadow--666 [~IDSE@S0106a84e3f595813.vc.shawcable.net] has joined #joinmarket     
11:26 -!- DeathShadow--666 [~IDSE@S0106a84e3f595813.vc.shawcable.net] has quit [Ping timeout: 252 seconds]     
11:41 -!- DeathShadow--666 [~IDSE@S0106a84e3f595813.vc.shawcable.net] has joined #joinmarket     
12:09 -!- trustiee [~smuxi@204.187.100.19] has quit [Read error: Connection reset by peer]     
15:04 -!- puddinpop [~puddinpop@unaffiliated/puddinpop] has joined #joinmarket     
16:54 < belcher> waxwing found some time to read through the code for podle, multi message channels and 0.2.0 protocol
16:55 < belcher> looks nice
17:41 < Lightsword> what’s the difference between yg-pe.py and yield-generator-basic.py?
17:42 < grubles> read the code
17:42 < grubles> there's comments
17:47 -!- trustiee [~smuxi@204.187.100.19] has joined #joinmarket     
17:47 < trustiee> in the latest development build, how do I see the raw transaction? (sometimes it doesn't broadcast and I need the raw transaction to manually broadcast it)
17:48 < trustiee> In previous builds that is dumped to the console output, but in development branch that is not shown
17:53 < trustiee> waxwing sorry to bug you but it's urgent
18:13 < solace> CANT STUMP
18:13 < trustiee> I figured it out.
18:13 < trustiee> just use getrawtransaction with bitcoin-cli to get it
18:14 < trustiee> solace: what does that mean?
18:14 < solace> 💩  debate's on  💩
18:14 < trustiee> oh lol
18:19 < solace> 💩  ya lmao  💩
18:19 < solace> 💩  and instead of watching it im shitposting  💩
18:55 -!- trustiee [~smuxi@204.187.100.19] has quit [Remote host closed the connection]     
22:04 -!- Giszmo [~leo@pc-40-227-45-190.cm.vtr.net] has quit [Quit: Leaving.]     
22:25 < waxwing> trustiee's Q is answered by looking in the log file or setting the logging level to DEBUG in joinmarket.cfg
23:34 < GithubBot5678> [joinmarket] CohibAA opened pull request #642: add hostid to debug log (develop...patch-19) https://git.io/vPHMi
23:50 -!- molz [~molly@unaffiliated/molly] has quit [Read error: Connection reset by peer]     
23:50 -!- moli [~molly@unaffiliated/molly] has joined #joinmarket     
23:54 -!- solace [h@wowana.me] has quit [Ping timeout: 258 seconds]     
23:58 -!- solace [~h@wowana.me] has joined #joinmarket