--- Day changed Thu Nov 22 2018
00:55 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Excess Flood]     
00:55 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #joinmarket     
03:12 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Excess Flood]     
03:12 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #joinmarket     
03:50 < waxwing> arubi, just a quick ping again in case you're around. i'm doing the release with that gpg thing included, it does "reliably" fail to pass on travis (see above).
03:50 < waxwing> is it reasonable to assume that this won't affect users? I tried it myself once, and posted the log on the PR, but i don't have feedback from others whether it works for them.
03:52 < waxwing> i know for sure as soon as i release i'll get a message "still doesn't work" because of this ... which is really annoying.
03:55 < arubi> waxwing, did you see my comment on the PR? : https://github.com/JoinMarket-Org/joinmarket-clientserver/pull/219#issuecomment-439347388   the issue with not passing on travis can indeed happen on a user's end if eg they used --no-gpg-verify up until now, but stop using the flag at a later point
03:55 < arubi> the fix would be, in case of travis delete the cache, and for a user would be removing the libsodium tar.gz from ./deps/cache
03:56 < waxwing> ah ok. can we add a note to the release notes perhaps?
03:56 < waxwing> that might be the first step, if we're not yet ready to completely streamline it.
03:56 < waxwing> if you just write a couple of sentences to tell people exactly what they should do, that would be fine, i can copy it into release notes.
03:57 < arubi> yes moment
03:57 < waxwing> arubi, ^
03:57 < waxwing> thx     
04:08 < arubi> added the text to #219 in a new comment
04:08 < arubi> waxwing, ^
04:11 < waxwing> thx arubi 
04:11 < arubi> np     
04:52 -!- GitHub62 [GitHub62@gateway/service/github.com/x-syoetblvlbnwtgov] has joined #joinmarket     
04:52 < GitHub62> [joinmarket-clientserver] AdamISZ pushed 1 new commit to master: https://git.io/fpBRu
04:52 < GitHub62> joinmarket-clientserver/master f18c67c AdamISZ: release notes for 0.4.2
04:52 -!- GitHub62 [GitHub62@gateway/service/github.com/x-syoetblvlbnwtgov] has left #joinmarket []     
05:25 < waxwing> release is done, followed all the usual communications procedures, feel free to spread it elsewhere
05:28 < belcher> tweeting, sec
05:29 < belcher> https://twitter.com/joinmarket/status/1065598205759561728
05:38 < waxwing> belcher, cheers
05:52 -!- belcher_ [~user@unaffiliated/belcher] has quit [Ping timeout: 246 seconds]     
06:07 -!- belcher_ [~user@unaffiliated/belcher] has joined #joinmarket     
06:11 < waxwing> arubi, so we do kinda need that travis thing fixed; for now i'll just run tests manually locally before merging as a stopgap.
06:23 < waxwing> Lightsword, re: 224, this is a dumb question, but are we using AMP v2? :) (relevance is, without bigstring, mainnet will puke in v1, although all tests will pass, due to large mainnet orderbooks)
06:23 < waxwing> it says here it's a 'work in progress' https://amp-protocol.net/conversations.html#encoding-amp-conversations
06:25 < waxwing> the twisted/protocol/amp.py i'm referencing doesn't mention versions.
06:27 < waxwing> i dunno not seeing it anywhere else than that page, i'll try on mainnet and see if it pukes
06:34 < waxwing> no, doesn't puke. hmm, perhaps it's that Unicode() handles the large stuff better.
06:41 -!- undeath [~undeath@hashcat/team/undeath] has joined #joinmarket     
06:53 < waxwing> looks like i'm getting aroun 20K not 65K in the string_orderbook so not conclusive unfortunately. might need to force it in a test to be sure.
06:55 < waxwing> hmm so the docs say values in key-value pairs are limited to 65K without bigstring, so i'm currently wary about this. even if in practice it's a way off it could happen any time.
07:20 < waxwing> Lightsword, written up what i found in detail here:  File "/home/adam/virtualenvs/csenv/local/lib/python2.7/site-packages/twisted/protocols/amp.py", line 701, in serialize
07:20 < waxwing> 	    raise TooLong(False, True, v, k)
07:20 < waxwing> 	twisted.protocols.amp.TooLong: 
07:20 < waxwing> damn sorry copy-paste fail, i meant here: https://github.com/JoinMarket-Org/joinmarket-clientserver/pull/224#issuecomment-441060679
07:54 < arubi> waxwing, do you mean that travis is broken even after cache cleanup?   to fix install.sh completely, we will have to start checking the sha256 of the .tar.gz.sig file as well, to decide whether we should download it or not (simply force downloading every time will break offline installs - but not sure if anybody does offline installs)
08:01 < arubi> I asked yesterday I think, Lightsword also brought this up, we might be overkilling it with the pgp sig verify there.  we're checking a file of a "trusted" hash, against a "trusted" pgp key, and now we'll have to also add the hash of the signature file itself.  of course it'll succeed at gpg verify if all file hashes + key fingerprint match
08:03 < arubi> in Core's depends build system, only hashes of dependency files are checked - pgp signatures aren't used
08:03 < undeath> I was actually thinking the same
08:04 < undeath> checksum + pgp is a bit overkill
08:07 < undeath> the checksum is already kinda pgp signed if the git signature is verified
08:08 < arubi> it is exactly.  otoh, I don't mind adding in checking the hash of the sig file.  yes it's overkill but can't actually hurt
08:09 < undeath> well, if it makes the install process more error prone it does hurt
08:10 < arubi> right that is true.  are we sure that there is no downside to stop checking pgp sigs?
08:11 < undeath> with the checksum being verified I can't think of any
08:12 < arubi> and basically if folks aren't checking pgp for jm release itself, then it's game over anyway
08:26 < arubi> I'll start working on removing gpg (and combining all the ${dependency}_get() functions - now it's easier) while we wait for waxwing to weigh in.  if we do eventually prefer to leave gpg in then I'll start working on adding the .sig file hashes.
08:48 < waxwing> i tend to defer to you guys on this, but: i know a lot of people do verify gpg sigs on downloads. qubenix ping, your thoughts?
08:49 < waxwing> what is the checksum/sha hash checked against?
08:49 < undeath> the thing is, we have a may times verified sha256 checksum already, signed using your pgp key
08:49 < waxwing> i see
08:49 < qubenix> i agree with the common sentiment: checksum is good since the project is signed
08:50 < waxwing> arubi, you said 'is travis broken even after cache cleanup?' is cache cleanup already done? i don't actually know what cache cleanup we're talking about
08:50 < waxwing> qubenix, thanks.
08:50 < belcher> i may have mentioned before but there is a way to sign releases too https://github.com/chris-belcher/electrum-personal-server/issues/10#issuecomment-386121030
08:51 < belcher> i could do that right now for joinmarket
08:51 < belcher> we could upload sigs from myself and others to the release page
08:52 < arubi> waxwing, I think only the repo owners can clean the cache.  you do it from travis' website from the "more options" dropdown on the upper right
08:52 < waxwing> belcher, oh you mean like sign the zip files and whatnot?
08:52 < belcher> yes     
08:52 < waxwing> i did that for binaries but not code, seemed not needed.
08:53 < waxwing> the tag is signed and all the commits are signed. well all the release commits
08:53 < undeath> but the tar/zip archives have no git metadata
08:53 < waxwing> but still, could be done yes. just upload them. it felt a bit overkill to me.
08:53 < belcher> those can only be seen if the user runs from master (or possible master and git reset tag)
08:53 < waxwing> i see.
08:53 < waxwing> arubi, thanks. i can give it a try now then.
08:53 < arubi> oke     
08:54 < belcher> if users either check the sig of the release .zip or check the sigs of the commits, then gpg doesnt need to be anywhere else in theory
08:55 < belcher> it is required for external projects actually, like the binary of libsodium say
08:55 < waxwing> right there are no hard and fast rules here. we have a set of dependencies, and yes *in theory* if we attest to sha256-es then ...
08:55 < arubi> that's actually what we're talking about removing
08:56 < arubi> er, not the sodium binary itself, just its gpg check
08:57 < waxwing> arubi, so i junked the cache for master; just try the build again, right?
08:58 < arubi> yep     
09:13 < Lightsword> so I guess amp v2 never happened?
09:15 < waxwing> Lightsword, it looks like it was just something planned, i didn't find any more details, maybe i didn't look hard enough
09:15 < waxwing> arubi, all the jobs still errored, here's the first one: https://travis-ci.org/JoinMarket-Org/joinmarket-clientserver/jobs/458397214
09:15 < Lightsword> ok, I’ll revert the bigstring removal part then
09:18 < arubi> waxwing, looks like "curl: (7) Failed to connect to download.libsodium.org port 443"  :(
09:19 < arubi> now it wants to download the dependencies again because we removed the cache, and correctly fails when it can't
09:19 < waxwing> Lightsword, so you're going to partially revert the revert of the revert? :)
09:19 < Lightsword> :P     
09:19 < undeath> lol     
09:20 < waxwing> arubi, so ... it seems we have consensus that we could actually remove the gpg part and just check the hashes.
09:20 < arubi> yep, already working on it
09:20 < waxwing> if this is as difficult as it seems to be, the tendency in that direction appears overwhelming. can it be made optional easily?
09:20 < waxwing> as in not default.
09:20 < waxwing> or if you prefer just ditching it i'm ok with that. we can always give people instructions/info on how to verify manually.
09:21 < undeath> with our signed commits I really see no use in keeping it
09:21 < arubi> I believe it's better to just remove.  if folks aren't gpg verifying the release itself, then no hash\key\sig in install.sh will help
09:23 < waxwing> well but a hash in install.sh surely does help since they're downloading stuff, right.
09:23 < waxwing> maybe i just parsed your sentence wrong.
09:23 < arubi> I mean, if install.sh isn't genuine, then any hash\url within it might also not be genuine and still pass hash + pgp verify
09:24 < waxwing> right
09:24 < arubi> so basically if we sign install.sh, and users verify that, then there's no reason for nested pgp verify.  hashes stay for consistency check of the files
09:25 < waxwing> well speaking only theoretically, trust isn't binary. but in practice this seems right.
09:25 < Lightsword> waxwing, can you test that new version I just pushed?
09:25 < Lightsword> it’s same as before but without the bigstring removal
09:25 < waxwing> Lightsword, ok. will do.
09:27 < arubi> waxwing, yea also this is basically how core's depends build works.  you verify pgp of the release source tar.gz then dependencies are only checked for the correct hash in the build process
09:28 < arubi> (hashes are in the Makefiles themselves, much like how they're in our install.sh)
09:28 < waxwing> ok     
09:35 < Lightsword> rebased just now to fix that merge conflict
09:45 -!- technonerd [~techno@gateway/tor-sasl/technonerd] has quit [Ping timeout: 256 seconds]     
10:28 < Lightsword> waxwing, this failure is just a travis bug right? https://travis-ci.org/JoinMarket-Org/joinmarket-clientserver/jobs/458509600
10:30 < arubi> not travis bug but install.sh bug.  I'm about to PR the fix
10:32 -!- GitHub29 [GitHub29@gateway/service/github.com/x-wvcwganhnygsuqji] has joined #joinmarket     
10:32 < GitHub29> [joinmarket-clientserver] fivepiece opened pull request #228: Remove gpg signature validation from install.sh (master...installsh_remove_gpg) https://git.io/fpBAM
10:32 -!- GitHub29 [GitHub29@gateway/service/github.com/x-wvcwganhnygsuqji] has left #joinmarket []     
10:49 -!- rdymac [uid31665@gateway/web/irccloud.com/x-vjqckaubtfrljexw] has joined #joinmarket     
12:48 -!- GitHub17 [GitHub17@gateway/service/github.com/x-dipipkolewpqefua] has joined #joinmarket     
12:48 < GitHub17> [joinmarket-clientserver] AdamISZ closed pull request #228: Remove gpg signature validation from install.sh (master...installsh_remove_gpg) https://git.io/fpBAM
12:48 -!- GitHub17 [GitHub17@gateway/service/github.com/x-dipipkolewpqefua] has left #joinmarket []     
12:48 -!- GitHub155 [GitHub155@gateway/service/github.com/x-zwnbtncvfhuzwiza] has joined #joinmarket     
12:48 < GitHub155> [joinmarket-clientserver] AdamISZ pushed 3 new commits to master: https://git.io/fpRvb
12:48 < GitHub155> joinmarket-clientserver/master fdc6687 fivepiece: remove gpg signature validation from install.sh...
12:48 < GitHub155> joinmarket-clientserver/master d353b5f fivepiece: remove gpg verify disable flag from docker scripts
12:48 < GitHub155> joinmarket-clientserver/master e92512f AdamISZ: Merge #228: Remove gpg signature validation from install.sh...
12:48 -!- GitHub155 [GitHub155@gateway/service/github.com/x-zwnbtncvfhuzwiza] has left #joinmarket []     
12:54 -!- GitHub177 [GitHub177@gateway/service/github.com/x-cmotedsajzjuoqcg] has joined #joinmarket     
12:54 < GitHub177> [joinmarket-clientserver] AdamISZ pushed 2 new commits to master: https://git.io/fpRfm
12:54 < GitHub177> joinmarket-clientserver/master 5a0d5fe James Hilliard: Revert "Revert "Python 3 style conversion""...
12:54 < GitHub177> joinmarket-clientserver/master 7c6b4f1 AdamISZ: Merge #224: Python 3 style conversion...
12:54 -!- GitHub177 [GitHub177@gateway/service/github.com/x-cmotedsajzjuoqcg] has left #joinmarket []     
12:54 -!- GitHub91 [GitHub91@gateway/service/github.com/x-qwpuljgzfogoqyha] has joined #joinmarket     
12:54 < GitHub91> [joinmarket-clientserver] AdamISZ closed pull request #224: Python 3 style conversion (master...future-fix) https://git.io/fp8q3
12:54 -!- GitHub91 [GitHub91@gateway/service/github.com/x-qwpuljgzfogoqyha] has left #joinmarket []     
12:54 < waxwing> i'll be running that on mainnet, pls join me if you can ^
12:59 < Lightsword> I’m rebasing my coincurve branch now
12:59 < Lightsword> btw, can joinmarket be used on testnet?
13:04 < waxwing> Lightsword, yes it can, but you need counterparties
13:04 < waxwing> set network = 'testnet' in the config
13:04 < Lightsword> is there an IRC channel set up for it?
13:04 < waxwing> Lightsword, also note that it appends -test to the name of the IRC channel it uses
13:04 < waxwing> :)     
13:05 < waxwing> in the past we have used cgan agora and also freenode for this (the latter without tor though)
13:08 < waxwing> Lightsword, i think 'revert revert ..' might not have been the best commit message :) it looks like nothing changed from the first version. nbd of course :)
13:09 < Lightsword> ah, do people have bots running there normally? that would probably be a good way to test for version incompatibilites
13:09 < Lightsword> waxwing, heh. yeah I probably should have changed that commit message
13:09 < belcher> nobody on cgan or agora runs testnet right now
13:09 < waxwing> exactly, as i say, it needs people running there, and they don't :) the only times we managed to get even a few people were (a) before the first mainnet release and (b) before the big protocol upgrade
13:10 < waxwing> also testnet3 is horror
13:11 < undeath> you need a bitcoind on testnet which makes the whole thing a lot more complicated
13:12 < Lightsword> hmm, maybe a good idea to keep the electrum protocol? then you can test with that instead of bitcoind
13:13 < waxwing> Lightsword, right, that was one of my motivations for keeping that running. problem is it was a bit shitty, i actually rather doubt it'd work for makers, maybe for takers.
13:13 < Lightsword> hmm, why would makers be different there?
13:13 < waxwing> i haven't tried it for a few months, it used to 'work' (with flakiness where you might have to restart to get a functioning electrum server connection)
13:14 < Lightsword> wonder if that’s related at all to electrum itself being python3
13:14 < waxwing> don't remember really. you can try it but, i make less than no guarantees of functionality :)
13:14 < waxwing> no, it's just talking to electrum servers. see electruminterface.py in jmclient
13:14 < Lightsword> just pushed rebased coincurve branch
13:15 < waxwing> oh god, i remembered how rubbish it was; it has two different types of connection for reasons i don't really want to remember :)
13:16 < Lightsword> waxwing, was the electruminterface.py using code from electrum or was it a fully independent implementation
13:17 < waxwing> iirc i just wrote it myself, but ofc i was doubtless copying code from here and there.
13:18 < waxwing> earlier i'd written a JM electrum plugin so was a bit familiar with a fair amount of electrum code.
13:19 < waxwing> Lightsword, bear in mind there was a strong negative impression of it even existing, because people who are into this stuff (rightly) think actually using an electrum connection for joinmarket is a terrible idea.
13:20 < waxwing> the first version of joinmarket used, by default mind you, a connection to blockr.io ... and it actually ended up being used a *lot* by people on Tails machines without a full node.
13:21 < Lightsword> yeah, still better using joinmarket with electrum than just using electrum by itself :P
13:21 < undeath> electrum is a cool thing, but you should be running your own server if you're using jm
13:21 < Lightsword> yeah
13:21 < waxwing> well yes. one might argue though that there's a polluting effect just from *others* using a non-private setup
13:21 < waxwing> but that's a general problem with anonymity tech
13:24 < waxwing> i guess i'll review coincurve stuff tomorrow then. i do like that one, but i'm ultra-ultra paranoid about bitcoin changes.
13:25 < waxwing> another one to do fairly quickly is get rid of btc.py. unfortunately it's not just 'delete the file' but it should be *fairly* simple i hope.
13:26 < Lightsword> coincurve I’m a little more confident in since it seems to be better tested and was essentially a drop in replacement
13:27 < Lightsword> the py3 stuff def requires a lot of care since it can have cascading side effects
13:28 < Lightsword> waxwing, so what’s the issue with btc.py?
13:28 < Lightsword> that it’s doing that weird import with fallback?
13:29 < waxwing> it was there to create an interface for other bitcoin backends, and specifically for electrum. but tbh i didn't even know the right way to construct it, and since i abandoned the electrum plugin, it's just a confusing/pointless thing at this point
13:30 < waxwing> not to say something *like* that mightn't be useful in future, it might
13:30 < waxwing> Lightsword, i'm also more confident about coincurve's correctness, but risk = probability * impact :)
13:31 < Lightsword> true
13:31 < undeath> it was there to allow running (some parts of) jmclient without jmbitcoin (?)
13:32 < waxwing> yes, exactly that. and that was the reason for the setup option of that excluded jmbitcoin
13:32 < undeath> but I'm not actually sure what part of jmclient you could safely run without it cracking
13:32 < Lightsword> stuff like this IMO is a bigger worry https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmbitcoin/jmbitcoin/secp256k1_transaction.py#L37-L44
13:32 < waxwing> well you've rewritten the wallet code now undeath , it did work the way it was before. anyway this is all irrelevant now.
13:33 < undeath> oh, I see
13:33 < Lightsword> basically all the code with isinstance() checks is a bit frightening
13:33 < waxwing> Lightsword, yes quite so, but that's why i'm trying to consider seriously whether to just rewrite the whole thing rather than port it.
13:33 < Lightsword> I’m planning to refactor as part of my next port attempt
13:33 < undeath> esp get rid of auto-detecting formats and such things :/
13:33 < Lightsword> and get rid of those checks
13:34 < undeath> it's a nice feature, but not in the backend code
13:34 < Lightsword> autodetection like that is generally considered bad practice since it swallows type exceptions
13:34 < Lightsword> which I lean on heavially for porting
13:35 < undeath> I mean, you should handle and convert it wherever a user would input it
13:35 < undeath> not throw at the backend and let it do… things
13:35 < Lightsword> yeah, user input is one thing, internals are different
13:36 < Lightsword> any automatic conversions should happen immediately as part of accepting user input, after that types should generally be strictly enforced
13:36 < waxwing> Lightsword, what do you think of gitlab? (if anything)
13:36 < Lightsword> waxwing, it’s fine, I use it for private repos, but for public I generally prefer github
15:41 -!- undeath [~undeath@hashcat/team/undeath] has quit [Quit: WeeChat 2.3]     
16:49 -!- lnostdal [~lnostdal@77.70.119.51] has quit [Quit: https://www.Quanto.ga/]     
16:54 -!- lnostdal [~lnostdal@77.70.119.51] has joined #joinmarket     
17:10 -!- AgoraRelay [~jmrelayfn@p5DE4AD38.dip0.t-ipconnect.de] has quit [Ping timeout: 244 seconds]     
17:20 -!- AgoraRelay [~jmrelayfn@p5DE4AB50.dip0.t-ipconnect.de] has joined #joinmarket     
18:21 -!- arubi [~ese168@gateway/tor-sasl/ese168] has quit [Remote host closed the connection]     
18:21 -!- arubi [~ese168@gateway/tor-sasl/ese168] has joined #joinmarket     
18:24 -!- arubi [~ese168@gateway/tor-sasl/ese168] has quit [Remote host closed the connection]     
18:25 -!- arubi [~ese168@gateway/tor-sasl/ese168] has joined #joinmarket     
19:15 -!- lnostdal [~lnostdal@77.70.119.51] has quit [Remote host closed the connection]     
19:18 -!- lnostdal [~lnostdal@77.70.119.51] has joined #joinmarket     
23:09 < Sentineo> I wonder if github stays the same now that it was bought or it will go down the road of skype