--- Day changed Mon May 27 2019
00:11 -!- takamatsu [~takamatsu@unaffiliated/takamatsu] has joined #joinmarket     
01:50 -!- takamatsu [~takamatsu@unaffiliated/takamatsu] has quit [Ping timeout: 258 seconds]     
01:58 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] i'm currently trying to debug a problem i found with nick signature verification, which creates this log message:
01:58 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1]  [DEBUG]  nick signature verification failed, ignoring: <makername>
01:59 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] it seems that it is *not* actually ignoring the "failinig" maker, it still does add it to the list of offers to be chosen from
01:59 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] this is the line in which the verification is requested: https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmdaemon/jmdaemon/message_channel.py#L915
02:00 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] i'm having trouble understanding how this works though, there seems to be no return value. How does it actually try to react to the successful or failinig nick verification? My limited python skills are the problem here
02:03 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] this is where this error message is created. We can see that it just continues as normal on the nick verification failure: https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmclient/jmclient/client_protocol.py#L109-L130 . This is where i get lost in calls and callbacks dont dont get how the return codes
02:03 < AgoraRelay> work
02:03 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] in general, its better not to start security code with "verif_result = True" but with the inverted logic :)
02:12 -!- takamatsu [~takamatsu@unaffiliated/takamatsu] has joined #joinmarket     
03:04 -!- berndj [~berndj@azna.co.za] has quit [Ping timeout: 246 seconds]     
03:25 -!- takamatsu [~takamatsu@unaffiliated/takamatsu] has quit [Ping timeout: 272 seconds]     
03:36 < waxwing> alexcato, the client calls back to the daemon with this: https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmclient/jmclient/client_protocol.py#L125
03:37 < waxwing> then the daemon only calls on_verified_privmsg if it succeeded: https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmdaemon/jmdaemon/daemon_protocol.py#L209-L213
03:37 < waxwing> note the 'log.msg(...)' there is actually twisted logging and is not actually displayed/triggered in prod code (twisted logging isn't enabled)
03:39 < waxwing> the overall structure of this verification functionality is quite tortuous; the reason is that i wanted to restrict the bitcoin code to the client side only, so signs and verifies get passed between. probably that could have been avoided but, another discussion from analyzing this apparent bug.
03:40 < waxwing> (i say 'apparent' because it's possible this is some strange behaviour by other code, but i think it's likely a bug in the code as-is)
03:42 < waxwing> one point that springs to mind is whether the hostid fits; the call assumes that the counterparty is using the same hostid as we are. if not the ecdsa verify ought to fail, right.
03:47 < waxwing> but are you def getting that error, and not the 'nick hash check failed' one?
03:47 -!- liead [~adlai@unaffiliated/adlai] has joined #joinmarket     
03:47 < waxwing> hmm let me try an expt now
03:47 -!- adlai [~adlai@unaffiliated/adlai] has quit [Read error: Connection reset by peer]     
03:49 -!- MaxSan [~four@134.19.179.203] has joined #joinmarket     
03:51 < waxwing> goodness, that's weird. when i start a yg it connects to cgan and darkscience fine; when i start a sendpayment, darkscience immediately boots me and continues to do so in a loop.
03:52 < waxwing> ok, worked on trying again; maybe just temporary conn. problem.
03:53 < waxwing> now i have 6 'nick sig verification failed' as you reported, on first attempt to start sendpayment. let me see if there's any pattern in where those bots are.
03:58 < waxwing> alexcato i think i got it
03:59 < waxwing> so the hostid is set here: https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmdaemon/jmdaemon/irc.py#L380-L394
04:00 < waxwing> i set a print statement to show the value of hostid ('v') being set here; then i tried on Tor and on non-Tor
04:00 -!- puddinpop [~puddinpop@unaffiliated/puddinpop] has quit [Remote host closed the connection]     
04:00 < waxwing> on the Tor version, it gets set to "Darkscience", and i see about 6 verification fails.
04:01 < waxwing> on the non-Tor cleartext server, it gets set to "darkscience" and i see tons of verification fails.
04:01 < waxwing> which just means there is an underlying but unfortunately incorrect assumption that IRC networks keep the "NETWORK=x" setting fixed across different modes of access.
04:01 < waxwing> we should double check Cgan but i think it is consistent here.
04:19 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] oh, thats great to know! Another part of the problem which I've found 1 or 2 days ago, which triggered the investigation: the taker is not actually ignoring these makers but still tries to coinjoin with them. During the actual coinjoin, it then treats these sig verification failure makers similar as if they were not
04:19 < AgoraRelay> responding (so the
04:19 < AgoraRelay> [agora-irc/CgRelayBot] MINIMUM_MAKER setting comes into effect). Imho they should never have been part of the set of potent
04:19 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] ial makers to join with in the first place
04:20 < waxwing> alexcato ok i'll look at that. that is indeed bad/weird!
04:23 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] that's what i was initially trying to solve, before i got stuck. BUt gotta leave house for now, will try again later tonight
04:25 < waxwing> alexcato oh i suddenly thought (i'm not looking at this now but it sprung to mind): if the counterparty is on both msgchans, then the fact that it fails to communicate on darkscience probably won't stop it taking part in a conversation
04:34 -!- takamatsu [~takamatsu@unaffiliated/takamatsu] has joined #joinmarket     
05:20 -!- berndj [~berndj@azna.co.za] has joined #joinmarket     
05:37 -!- MaxSan [~four@134.19.179.203] has quit [Ping timeout: 250 seconds]     
06:15 -!- takamatsu [~takamatsu@unaffiliated/takamatsu] has quit [Ping timeout: 248 seconds]     
06:42 -!- takamatsu [~takamatsu@unaffiliated/takamatsu] has joined #joinmarket     
06:56 < waxwing> so the situation according to me is: there is no bug in Joinmarket here per se; if a user is connecting on darkscience and cgan, as is the default, nothing should fail to work assuming cgan is up; if a user is using Tor there'll be no failure on darkscience either, for all the makers using Tor (which is like all but 6 or so).
06:56 < waxwing> what remains is: is it IRC standard to have the 'NETWORK' name be case insensitive? if so it's a trivial edit to make sure this doesn't crop up.
06:57 < waxwing> but i can't see anything from looking at IRC RFCs about 'NETWORK' at all, perhaps i haven't tried enough.
07:01 < waxwing> if i'm reading this right (and indeed, if it's in any way authoritative(?)) you could argue we're just leaning on that variable a bit too much: https://tools.ietf.org/html/draft-hardy-irc-isupport-00#page-13
07:01 < waxwing> i don't see anything better than making it case insensitive and hoping for the best.
07:56 -!- puddinpop [~puddinpop@unaffiliated/puddinpop] has joined #joinmarket     
08:27 -!- takamatsu [~takamatsu@unaffiliated/takamatsu] has quit [Read error: Connection reset by peer]     
08:58 -!- AbuseOfNotation [~AbuseOfNo@217.151.98.168] has joined #joinmarket     
09:10 < belcher> waxwing i think your analysis is correct, NETWORK probably isnt meant to be used that way
09:11 < belcher> but IMO i cant think of any other alternative, so maybe its best to just always convert the network to lowercase, and carry on
09:13 < belcher> perhaps another option is to add a new field in the config file which tells the name of the network... id say .lower() is easier/better
09:19 -!- MaxSan [~four@134.19.179.235] has joined #joinmarket     
10:21 -!- liead is now known as adlai     
11:25 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] hm. That .lower() change has a problem though: everyone, including all the makers, needs to upgrade to that fixed version or they cant verify each other's names?
11:31 < waxwing> yes, good point, it's only a sort of half-backward compatible solution (at best). i'm assuming this is not an issue in practice today because all the makers are on CgAn, but during downtime of that server it could well be causing issues occasionally.
11:32 < waxwing> but anyway that's somewhat tangential, you raise a good point .. so if the verification code allows both cases, it's better though right.
11:33 < waxwing> i mean, it's better than changing whatever the hostid is currently.
11:34 < waxwing> huh, i guess that doesn't (cleanly) work either, if i'm currently on "darkscience" and it doesn't work with that, do i start checking all possible case differences? :)
11:36 < waxwing> maybe it's reasonable to check 2 possibilities: (1) whatever i currently have as 'self.hostid', (2) the all lowercase version of the same thing. accept both as verified.
11:36 < waxwing> well 'either' not 'both', of course.
11:49 -!- Cory [Cory@unaffiliated/cory] has quit [Ping timeout: 245 seconds]     
11:50 < waxwing> alexcato you were asking about the logic flow of the code, it's (D): on_privmsg -> jm_request_msg_sig_verify -> (C): on_JM_REQUEST_MSGSIG_VERIFY (verifies), then sends JMMsgSigVerify -> (D): on_JM_MSGSIGNATURE_VERIFY, if verified, passes to on_verified_privmsg and continues.
11:50 < waxwing> C is client D is daemon
11:50 < waxwing> you can use the @**.responder decorators to see the match between the send and receive, the messages are the types defined in the amp commands.py in jmbase/
11:51 < waxwing> my guess is that last thing is what's not obvious from looking at the code; the functions have similar names and obviously you can't directly trace calling paths between the sends and receives over AMP
11:56 -!- Cory [Cory@unaffiliated/cory] has joined #joinmarket     
11:56 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] >> i'm assuming this is not an issue in practice today
11:57 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] just modified my code to use .lower(). Available makers was reduced to 6... and i connected to both cgan and darkscience on tor
11:58 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] 119 sig verification error messages
12:00 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] i believe this solves the other mystery though. Theory: it *does* not add offers of a sig failing maker to the list. But I have still seen them in my earlier tests, because on one of the irc servers the PMs by makers were fine (only failed on the other one).
12:10 < waxwing> yes that's what i meant above.
12:11 < waxwing> 'not an issue in practice today' - yes that was wrong, in the sense of 'if we change it to .lower()', see my last suggestion, that might be the best we can do for now.
12:12 < waxwing> (i corrected below "i guess that doesn't (cleanly) work either ..."
12:14 -!- johnhmay [sid110431@gateway/web/irccloud.com/x-nvnbcgifuupzbxko] has quit [Read error: Connection reset by peer]     
12:14 < waxwing> "thats what i meant above" - sorry need to be clearer, i mean when i said: "oh i suddenly thought..." ; we agree that that's why you still see these bots in joins, because they're on the other chan too.
12:16 -!- takinbo [sid19838@gateway/web/irccloud.com/x-kmjdifsocslcvitq] has quit [Ping timeout: 252 seconds]     
12:19 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] alright. I'll locally test the idea which makes .lower() an additional check... that would solve the most immediate problem
12:20 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] is there a list for things we might keep in mind for the next non-backward-compatible version? Switching to .lower() only might be a good thing to add to it
12:21 -!- Guest1469 [sid19838@gateway/web/irccloud.com/x-sojffothjulkiyil] has joined #joinmarket     
12:24 < waxwing> i don't have good answers here. best i can see right now is: add the second option to the verification side; then later (perhaps much later?) we could change the hostid setting assuming everyone had upgraded the verification so it would always work.
12:24 < waxwing> but that's yuck.
12:28 -!- Guest1469 [sid19838@gateway/web/irccloud.com/x-sojffothjulkiyil] has quit [Ping timeout: 248 seconds]     
12:39 < AgoraRelay> [agora-irc/CgRelayBot] [cgan/AlexCato1] hm. I'm missing something. hostid seems not to be part of the check anywhere? https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmclient/jmclient/client_protocol.py#L109-L130
12:42 < waxwing> alexcato see on_privmsg in message_channel.py ; the hostid (our one) is appended to what we receive from the counterparty as message.
12:42 -!- Guest1469 [sid19838@gateway/web/irccloud.com/x-khjxhymesieoomuc] has joined #joinmarket     
12:44 -!- johnhmay [sid110431@gateway/web/irccloud.com/x-gcbhjrvftkaoardw] has joined #joinmarket     
12:51 < belcher> how about use .lower() but only if the current date is after 1st september 2019
12:59 < belcher> so that assumes that almost-everyone has updated by that date
13:37 -!- AbuseOfNotation [~AbuseOfNo@217.151.98.168] has quit [Ping timeout: 258 seconds]     
14:08 -!- MaxSan [~four@134.19.179.235] has quit [Ping timeout: 248 seconds]     
16:30 -!- MaxSan [~four@134.19.179.235] has joined #joinmarket     
17:36 -!- viasil [~viasil@185.107.94.166] has quit [Ping timeout: 248 seconds]     
17:37 -!- viasil [~viasil@185.107.94.166] has joined #joinmarket     
17:39 -!- MaxSan [~four@134.19.179.235] has quit [Ping timeout: 272 seconds]     
17:39 -!- AgoraRelay [~jmrelayfn@p5DE4ABE9.dip0.t-ipconnect.de] has quit [Ping timeout: 272 seconds]     
17:39 -!- StopAndDecrypt [~StopAndDe@unaffiliated/stopanddecrypt] has quit [Ping timeout: 245 seconds]     
17:53 -!- MaxSan [~four@213.152.162.165] has joined #joinmarket     
17:54 -!- AgoraRelay [~jmrelayfn@p5DE4AB56.dip0.t-ipconnect.de] has joined #joinmarket     
18:19 -!- MaxSan [~four@213.152.162.165] has quit [Ping timeout: 246 seconds]     
18:38 -!- MaxSan [~four@193.148.16.211] has joined #joinmarket     
19:07 -!- Guest1469 is now known as takinbo     
19:07 -!- takinbo [sid19838@gateway/web/irccloud.com/x-khjxhymesieoomuc] has quit [Changing host]     
19:07 -!- takinbo [sid19838@unaffiliated/takinbo] has joined #joinmarket     
19:07 -!- takinbo [sid19838@unaffiliated/takinbo] has quit [Changing host]     
19:07 -!- takinbo [sid19838@gateway/web/irccloud.com/x-khjxhymesieoomuc] has joined #joinmarket     
21:21 -!- MaxSan [~four@193.148.16.211] has quit [Quit: Leaving.]     
23:04 -!- Cory [Cory@unaffiliated/cory] has quit [Ping timeout: 245 seconds]     
23:10 -!- Pasha [Cory@unaffiliated/cory] has joined #joinmarket     
23:13 -!- Pasha is now known as Cory