--- Day changed Sat Aug 06 2016
01:13 < nkuttler> mh, whenever i use --mixdepth with sendpayment i get an IndexError, can somebody confirm this?
02:00 < waxwing> nkuttler: one scenario it's possible is where you set up a yg with one number of mixdepths, then index_cache in wallet is set to a list of that length, if you then try to start  with a different number (higher) you get an index error. just a thought, back soon
02:14 < waxwing> but sendpayment i think that doesn't apply to, so scratch that
02:44 -!- arubi_ [~ese168@unaffiliated/arubi] has joined #joinmarket
02:46 < nkuttler> i guess i'll create a ticket
02:48 -!- arubi [~ese168@unaffiliated/arubi] has quit [Ping timeout: 258 seconds]
02:49 < waxwing> nkuttler: sure if it's easier that way; it's probably something simple, though, i just haven't had a chance to look
02:53 < nkuttler> here's a traceback https://dpaste.de/3Eeo
02:53 < waxwing> suddenly occurs to me it might be related to what's at the end of this:
02:53 < waxwing> https://github.com/JoinMarket-Org/joinmarket/issues/171#issuecomment-236975683
02:53 < nkuttler> the yigen is using another wallet
02:54 < waxwing> what's the length of the index_cache array in the wallet.json?
02:55 < nkuttler> "index_cache": [[1, 23], [2, 15], [2, 3], [2, 3]], 
02:55 < waxwing> if it's less than 5, you can use -a N where N is the length
02:55 < waxwing> right, so 4
02:55 < nkuttler> same error with -N 4
02:55 < waxwing> -a 4
02:55 < nkuttler> oops
02:56 < nkuttler> that works, thanks
02:56 < waxwing> i'm not sure why you've got a wallet with 4 mixdepths though. i don't think that'd happen by default, unless i missed something.
02:57 < nkuttler> that's odd, must have happened recently
02:57 < waxwing> the TLDR on -a is, i added that option to sendpayment so you can source commitment utxos from the whole wallet, not just the one mixdepth you're spending from.
02:57 < nkuttler> can't even call wallet-tool
02:57 < waxwing> it'll be the same issue. use -m 4 on wallet-tool
02:58 < nkuttler> i don't think i did anything special when creating that wallet
02:58 < waxwing> did you use it previously as a yield gen wallet? i think you must have done.
02:59 < nkuttler> yeah
02:59 < waxwing> if you did, it still would stick with the default 5 unless you choose to use a different number
02:59 < waxwing> but if you changed the number for the yigen, that's when index_cache gets overwritten with a different size and this happens.
02:59 < nkuttler> let me check my bitcoind history, how much i sent into it
03:00  * nkuttler checks his backups
03:00 < waxwing> yes this is an option it would probably be better not to have, given it can cause annoyance, but otoh there are good reasons to have it too (the option is -m for yieldgenerators)
03:01 < nkuttler> it used to have 5
03:01 < nkuttler> i think i also sent 5 x 1000 tbtc into it
03:02 < waxwing> i may be wrong but you must have specified -m 4 on the yigen at some point, i don't see any other code path to update the index_cache
03:02 < waxwing> apart from tumbler, but you haven't used that i think
03:02 < nkuttler> i did try!
03:02 < nkuttler> not enough people in the market though. did that yesterday actually
03:02 < waxwing> ah. i guess you didn't keep a record of the options?
03:03 < nkuttler> i don't think i used any options
03:04  * nkuttler never figured out where his bash history ends up when inside screen
03:04 < waxwing> ok. it'll be tricky to work out exactly how it happened, since you used the wallet in several different modes, but at least we do have the options -a and -m to do what you need.
03:04 < nkuttler> can i re-create the fifth depth? i think there are coins in it
03:04 < waxwing> feel free to open a github issue, because we'd want to not crash on an index error but give help instead
03:05 < nkuttler> will do
03:05 < waxwing> nkuttler: yeah the simplest thing to do is recover, or even simpler is to delete index_cache in the json
03:05 < nkuttler> oh, let me do that
03:05 < waxwing> if it was a huge wallet that'd be a pain, but here re-syncing should be very quick
03:05 < nkuttler> but i'll report the bug first
03:05 < nkuttler> yeah, just a testnet wallet
03:05 -!- arubi__ [~ese168@unaffiliated/arubi] has joined #joinmarket
03:07 < waxwing> technically there are scenarios where this could result in using addresses that were already proposed in incomplete transactions, so while it's easy to "fix" this changed-number-of-mixdepths scenario by just sort of wiping the slate clean, a proper solution would probably involve some more code. but that's for another time.
03:09 -!- arubi_ [~ese168@unaffiliated/arubi] has quit [Ping timeout: 258 seconds]
03:22 < waxwing> so re: the issue of commitment broadcast discussed yesterday, what do people think about this: each YG in a transaction simply privmsgs to one other random YG, who pubmsgs on receipt. So if 3 participate in a tx, 3 will broadcast to the pit, but not the one that did the tx.
03:22 < waxwing> of course that mechanism would not be perfectly reliable, but we don't need it to be (not even close)
03:22 < waxwing> s/not the one/not the ones/
03:35 -!- rdymac [uid31665@gateway/web/irccloud.com/x-qksybecupraqktvs] has joined #joinmarket
03:59 -!- belcher [~user@unaffiliated/belcher] has joined #joinmarket
04:00 < belcher> waxwing ping
04:00 < waxwing> pong
04:00 < belcher> i read your blog, its good
04:01 < waxwing> belcher: thanks. but the analysis is a bit all over the place, i already thought of like 3 ways it's wrong :)
04:01 < belcher> have you thought about how the spy doesnt need to watch for re-annoucement of offers he could watch the bitcoin p2p network instead
04:01 < waxwing> to be fair, that's almost inevitable there's so many variables...
04:01 < belcher> for when one of the utxos he knows gets spent
04:02 < waxwing> well yeah. when he's succeeded in picking up the right ones, he'll know when it's spent.
04:05 < belcher> yeah well its just a small point really
04:05 < waxwing> is it? to be honest i find myself lost half the time :)
04:06 < waxwing> the part i didn't get into at all is the whole "well you can see all JM txs so the outputs which are not are all the analyst needs" argument
04:07 < waxwing> well alright that's not a good argument, but it's not completely irrelevant either.
04:07  * waxwing is putting together a patch for the "transfer once then broadcast" thing above, let me know if there's something wrong with it
04:08 < belcher> i meant this "Prefer algorithms that don't require re-announcement of offers after transaction completion, as this is a very useful marker for an attacker as to who to query."
04:08 < belcher> if the spy is watching the blockchain they can see when a transaction completes, no need to watch the pit
04:09 < waxwing> ok but i think they want to query the right bots after a transaction, so as to find the fresh utxos and assign them to pit members
04:09 < waxwing> i mean yeah they "find" them already on the bc but i think by assigning them it gives them more concrete evidence?
04:10 < waxwing> well, in the simplest possible case, outputs 1,2,3 are cjouts, if 2 and 3 are immediately re-advertised, job done, right?
04:10 < belcher> yep
04:10 < waxwing> but to do that it really helps if you know *who* to query
04:11 < belcher> if the spy has been spying he'll know that utxos 2 and 3 belong to those irc nicks
04:11 < belcher> unless the makers restart their bots in the meantime
04:11 < waxwing> if 1,2,3 are the cjouts, surely he can only discover the ownership by queries *after* the tx completes?
04:12 < belcher> yes, sorry i was thinking inputs
04:12 < belcher> so lets say
04:12 < belcher> the spy sees a tx on the bc, inputs A, B, C outputs 1, 2, 3, they know that A and B belong to some irc nicks
04:13 < belcher> then they get their utxos again and find the one which came from outputs 1, 2 or 3
04:14 < waxwing> yeah before and after, but i was handwaving in the blog saying we should focus on the cjouts cos that's the critical part. that means queries to the pit after the tx, and before they get consumed again.
04:14 -!- HostFat [~HostFat@2-235-224-2.ip230.fastwebnet.it] has joined #joinmarket
04:14 < belcher> what did you mean by the "transfer once then broadcast" thing above
04:15 < waxwing> <waxwing> so re: the issue of commitment broadcast discussed yesterday, what do people think about this: each YG in a transaction simply privmsgs to one other random YG, who pubmsgs on receipt. So if 3 participate in a tx, 3 will broadcast to the pit, but not the one that did the tx.
04:15 < waxwing> <waxwing> of course that mechanism would not be perfectly reliable, but we don't need it to be (not even close)
04:15 < waxwing> <waxwing> s/not the one/not the ones/
04:16 < belcher> ah interesting idea
04:16 < belcher> so you cant link irc nick to commitment/tx event
04:16 < belcher> yes that could be good
04:17 < waxwing> unfortunately we can't use the same technique for !reloffer :)
04:17 < belcher> i wonder if that gives an incentive for the spy to fill up the pit with irc bot
04:17 < belcher> brb
04:18 < waxwing> yes fair Q, it's not the same as sybiling maker bots generally, although arguably there is incentive for that in today's system.
04:19 < waxwing> (i mean sybiling without doing txs, as we know from earlier discussions, there is incentive for that also, but entirely different kind of attack)
04:21 < waxwing> i guess it's a lot like sybiling bitcoin p2p to find transaction origin (which as far as I know, is for sure a thing)
04:21 -!- arubi_ [~ese168@unaffiliated/arubi] has joined #joinmarket
04:25 -!- arubi__ [~ese168@unaffiliated/arubi] has quit [Ping timeout: 244 seconds]
04:28 < belcher> you could try relaying commitments
04:28 < belcher> the maker who gets it has some probability of broadcasting and some of privmsging again
04:29 < belcher> but as you said even that doesnt solve the problem, just means they need more bots
04:36 < waxwing> i'll try to code a one and done. done most of it, the only substantial code is to have makers maintain a list of recent other makers via a callback in on_pubmsg
04:36 < waxwing> yeah the original idea was to sort of "spread it around" but prefer simpler for now, we can see how it goes after
04:36 < belcher> wouldnt it be easier for makers to inherit from OrderbookWatcher
04:36 < waxwing> none of this is consensus-y so it can be changed
04:36 < waxwing> belcher: ah good point
04:37 < waxwing> is that going to cause any other issues? on the face of it that's by far the most sensible
04:37 < waxwing> one could argue adding the db stuff is unnecessary perhaps?
04:39 < waxwing> if just slapped in, it would result in makers requesting the orderbook
04:39 < waxwing> but could just overwrite that
04:39 < waxwing> oh, makers already do overwrite it, no problem
04:42 -!- arubi__ [~ese168@unaffiliated/arubi] has joined #joinmarket
04:45 < belcher> they do?
04:45 < belcher> since patientsendpayment is a taker and maker and it does !orderbook and !absorder 
04:45 < waxwing> yeah overwrite on_welcome for announcing orders and whatnot
04:46 < waxwing> sorry i was wittering without proper context
04:47 -!- arubi_ [~ese168@unaffiliated/arubi] has quit [Ping timeout: 276 seconds]
04:48 -!- akrmn [~akrmn@191.red-83-61-221.dynamicip.rima-tde.net] has quit [Read error: Connection reset by peer]
04:55 < waxwing> belcher: i'm not familiar with the db stuff, can i do something like orders = db.execute('SELECT * from orderbook').fetchall() and then orders is list of dicts each with a 'counterparty' key?
04:59 < belcher> its not a dict it something else specific to sqlite3
04:59 < belcher> but you can access it the same with row['ordertype'] for example
04:59 < belcher> best to do SELECT counterparty from orderbook
05:00 < belcher> see the code in taker.py that sends a fully signed tx to a random maker, it does a similar thing to what you're doing
05:00 < waxwing> ah, thanks, good thought
05:00 < belcher> i think it actually uses the random functions in sql, like SELECT counterparty FROM orderbook WITH RAND();
05:00 < belcher> or something
05:00 < belcher> LIMIT 1 is in there
05:01 < waxwing> yes that's nice and succinct, thanks
05:12 < belcher> bbl
05:13 -!- King_Rex [~King_Rex@unaffiliated/king-rex/x-3258444] has joined #joinmarket
06:11 < waxwing> belcher: what's the right way to handle the exceptional case of the query not returning any results? if it's a dict i'll just do "if not crow or 'counterparty' not in crow: return' but, not sure if that's right here.
06:24 < waxwing> ok just using "if crow is None: return", seems to be fine/correct, the above isn't right.
06:45 -!- Guest79503 is now known as pigeons
06:47 -!- arubi__ [~ese168@unaffiliated/arubi] has quit [Quit: Leaving]
06:47 -!- arubi [~ese168@unaffiliated/arubi] has joined #joinmarket
06:50 -!- arubi [~ese168@unaffiliated/arubi] has quit [Client Quit]
06:51 -!- arubi [~ese168@unaffiliated/arubi] has joined #joinmarket
06:52 -!- arubi [~ese168@unaffiliated/arubi] has quit [Remote host closed the connection]
06:55 -!- arubi [~ese168@unaffiliated/arubi] has joined #joinmarket
07:58 -!- mkarrer [~mkarrer@142.red-83-47-107.dynamicip.rima-tde.net] has quit [Remote host closed the connection]
07:58 -!- mkarrer [~mkarrer@142.red-83-47-107.dynamicip.rima-tde.net] has joined #joinmarket
09:39 < nkuttler> yay, it worked. sent faucet payouts through jm
09:41 < waxwing> \o/
09:42 < waxwing> (i did one half an hour ago btw). also, there seems to be at least one non-responsive bot in there, don't know who it is.
09:44 < waxwing> belcher: i'm kind of idly musing about what can be done to have yield generators not "give away" participation with immediate re-announcement. iirc -basic always does, and mixdepth only sometimes does.
09:56 < waxwing> https://www.reddit.com/r/joinmarket/comments/4wgnof/confirmation_times_on_my_yieldgen_bot_since/ <-- can't remember why i was curious, but plotted it so i thought i'd share
09:59 < nkuttler> not worried about your privacy there?
09:59 < waxwing> confirmation time?
09:59 < waxwing> it's smoothed 20 period also
10:00 < nkuttler> ah
10:00 < waxwing> you'd make a good employee for chainalysis with that kind of thinking :)
10:00 < nkuttler> yours confirm way faster than mine
10:00 < nkuttler> or my plotting code is buggy..
10:00 < waxwing> well, but they're not mine
10:01 < waxwing> i.e. the taker sets the txfee
10:01 < nkuttler> right
10:01 < waxwing> i'm assuming (maybe wrongly) that they leave the default setting of 3 blocks target
10:01 < waxwing> in which case it's interesting evidence that it's rather conservative; whether i mean Core or blockcypher, not sure which is dominant
10:02 < waxwing> curious, what kind of figures are you getting?
10:03 < nkuttler> i see a few > 100
10:03 < nkuttler> and plenty > 50
10:03 < waxwing> individually, sure, i think that's normal. but try some medium range average
10:03 < waxwing> hmm i take that back, i can see virtually none with 100
10:04 < waxwing> a couple in march. also, note this is only since january.
10:04 < nkuttler> hm, you wouldn't know how to get a ranged average with a pandas df?
10:05 < waxwing> sorry never used pandas. just used libreoffice calc.
10:05 < nkuttler> ah, there's a rolling_mean
10:06 < waxwing> it would have been better if i had timebased the y-axis but too much effort, just a minor bit of curiosity
10:06 < waxwing> x-axis sorry
11:06 < nkuttler> hm. do you guys think that publishing the txids of my testnet faucet payouts would be bad?
11:07 < pigeons> in my opinion it seems consistent with testing use of testnet
11:07 < nkuttler> yeah. was probably easy to track them before anyway
11:08 < waxwing> nkuttler: actually in the past we have even published real txids, although I don't make a habit of it; consider that elementary bc analysis can find all JM txs with high probability
11:08 < waxwing> as for testnet, i can't imagine it'd be an issue for anyone
11:09 < nkuttler> yeah
11:09 < nkuttler> darn, re-running the payout script starts at mixdepth 0 an errs..
11:09  * nkuttler should randomize that or something
11:09 < waxwing> what error?
11:10 < nkuttler> insufficient funds
11:10 < waxwing> oh that one, i remember
11:10 < waxwing> so do you have it cycling now?
11:10 < nkuttler> yeah
11:12 < waxwing> btw if it's active pls take today's update, i'd like to see the cross-maker commitment broadcast working.
11:12 < waxwing> well, it's already working i think i see, no panic
11:18 < nkuttler> 4e051c861343ea4344c45d21d7b0f3c584255f389a94c6513d1de1dd6855f761 with today's code
11:19 < waxwing> nkuttler: thanks
11:20 < nkuttler> yw
11:27 < nkuttler> sigh. my anti-spam measures are too good, can't even create new faucet requests myself.oO
11:37 < waxwing> you mean anti-spam on the faucet, not jm anti-spam right?
11:38 < nkuttler> yeah, internal stuff
12:02 < waxwing> randomized broadcast seems to be working from what i can see.
12:02 < nkuttler> yep
12:03 < waxwing> i think nkuttler is putting it through its paces
12:03 < nkuttler> just trying to capture that txid from the output
12:03 < waxwing> ah, i see. how about using the log?
12:03 < nkuttler> mmh
12:03 < nkuttler> that would need a delay
12:03 < waxwing> well, either way. a bit of a pain i can imagine.
12:03 < nkuttler> can't be that hard. just have to figure out pexpect
12:03 < waxwing> delay, well, or on shutdown?
12:39 < nkuttler> friggin pexpect, but i nailed it https://kuttler.eu/en/bitcoin/btc/faucet/
12:40 < nkuttler> first one is linked, more to follow
12:41 < waxwing> nkuttler: well yeah pexpect is a bit of a bear, but really it's our fault for not making it easy for you to write a custom client (nice API), but .. another battle :)
12:41 < waxwing> can i try it?
12:41 < nkuttler> sure
12:43 < waxwing> it says 'success' but i don't see it yet
12:43 < nkuttler> hm, nice, broke the template..
12:43 < nkuttler> oh, that's just a successful request, not sent yet
12:43 < nkuttler> i guess i'll update that text
12:44 < waxwing> well, it's ok, it says 'sometimes takes a few days' etc
12:45 < nkuttler> sending it now
12:45 < nkuttler> yep, appears on the page
12:46 < waxwing> received, thanks
12:48 < nkuttler> yw
13:09 -!- opticbit [~opticbit@ip68-108-125-25.lv.lv.cox.net] has joined #joinmarket
13:18 < waxwing> "Most of the people doing coinjoin, you think, why they do that? I think most of them because they are probably not very clean, so they use coinjoin, ...." <-- not so sure :)
13:19 < waxwing> (http://diyhpl.us/wiki/transcripts/2016-july-bitcoin-developers-miners-meeting/cali2016/)
13:19 < waxwing> http://diyhpl.us/wiki/transcripts/2016-july-bitcoin-developers-miners-meeting/cali2016/
13:19 < waxwing> "I have been running my joinmarket coinjoin client the whole time on my laptop for this whole event." <-- take a guess :)
13:24 < waxwing> the comments are not attributed, but if i'm reading it right (i might not be) then some mining representatives there seem to have never understood what bitcoin actually is.
13:27 < nkuttler> meh. i use coinjoin because privacy and fungibility matter
13:33 < waxwing> we should encourage people running yield generators to do all their retail payments via their joinmarket wallet (well, at least as much as is practical). i've been doing that for a while. i think it can really help the effectiveness of the system as a whole, not least because it breaks the assumption that "exits" from the JM network of transactions are non-yield generator entities.
14:10 -!- puddinpop [~puddinpop@unaffiliated/puddinpop] has quit [Ping timeout: 250 seconds]
15:00 -!- coins123 [~coins123@unaffiliated/coins123] has quit [Ping timeout: 250 seconds]
16:04 -!- puddinpop [~puddinpop@unaffiliated/puddinpop] has joined #joinmarket
16:08 <+JM-IRCRelay> [AlexCato] currently setting up the 0.2.0 branch. From the IRC post: "For setting up a wallet, testnet is (can be) different on joinmarket". So that means i cannot use a wallet.json file, or that i dont have to?
16:12 <+JM-IRCRelay> [AlexCato] nevermind, figured it out; just needed to set joinmarket.cfg to 'testnet' before creating a new wallet
16:13 <+JM-IRCRelay> [AlexCato] the testnet-faucet gives me an error when i request a few coins though, anyone can send a few testnet-coins to mi4ip8CB4dBpjPee3ikFbWEzg3sEDT58Zk ?
16:19 <+JM-IRCRelay> [AlexCato] thanks, whoever that was :)
16:20 <+JM-IRCRelay> [AlexCato] testnet 0.2.0 yieldgen now online on CGAN and rizon.net. Will keep it running and check logs, anything in particular you want me to look out for?
17:11 < nkuttler> \o/ yay
17:11 < nkuttler> Alex, make a request on mine https://kuttler.eu/en/bitcoin/btc/faucet/
17:12 < nkuttler> i'll up the coin amount for you, and send it through jm
17:22 < belcher> waxwing i plan to make it much easier to do retail payments while the yg is running
17:22 < belcher> it comes after the bitcoin-qt integration
17:22 < belcher> yes it should be good
17:26 < belcher> bitcoin-qt integration involves that hook that fires whenever walletnotify (or a wallet updating event) happens
17:27 < belcher> so if the yieldgenerator listens to that hook and just recreates its offers, then whenever sendpayment or JoinMarket-Qt send a coinjoin the yieldgenerator on that same wallet will update
17:39 < belcher> oh yes damn you're right that the broadcasting-tx-to-random-maker doesnt check if the query returns no results
17:54 < belcher> now that Maker also inherits from OrderbookWatcher, we should just combine the CoinJoinPeer and OrderbookWatcher classes
19:06 -!- King_Rex [~King_Rex@unaffiliated/king-rex/x-3258444] has quit [Remote host closed the connection]
19:18 -!- belcher [~user@unaffiliated/belcher] has quit [Ping timeout: 264 seconds]
20:38 -!- puddinpop [~puddinpop@unaffiliated/puddinpop] has quit [Ping timeout: 250 seconds]
23:30 -!- btcdrak [uid165369@gateway/web/irccloud.com/x-rmlmrflropeoozsh] has quit [Quit: Connection closed for inactivity]
23:37 -!- btcdrak [uid165369@gateway/web/irccloud.com/x-cowjvbohfpuqmmno] has joined #joinmarket