--- Day changed Tue Sep 19 2017
02:57 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 264 seconds]     
05:34 < GitHub24> [joinmarket-clientserver] AdamISZ pushed 1 new commit to master: https://git.io/v5Agq
05:34 < GitHub24> joinmarket-clientserver/master b221f1a AdamISZ: wif version byte change:...
05:36 < waxwing> it seems quite often that just restarting the build fixes it. i don't know if it's something weird our end (-p no:warnings *could* be related), or it could be travis' problem.
06:54 -!- coins123 [~coins123@unaffiliated/coins123] has quit [Ping timeout: 246 seconds]     
07:02 -!- fredrin [~fredrin@gore.copyleft.no] has quit [Ping timeout: 248 seconds]     
07:13 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
07:17 < waxwing> going to spend a bit of time looking at your PR belcher .. i want to tag a release now.
07:18 < waxwing> a shame that, as usual, get no response to requests for testers ...
07:19 < waxwing> tried really hard to make it ridiculously easy, at least on linux. installation is one line. and this thing can be tested even without bitcoin core, so in theory you don't need anything else, at all.
07:21 -!- coins123 [~coins123@unaffiliated/coins123] has joined #joinmarket     
07:23 < belcher> yes its a shame :\
07:24 < waxwing> so it doesn't affect an existing wallet, that makes sense. did you implement it as an extra field as you mentioned belcher ?
07:24 < waxwing> i will look at the code, just want to try it out a bit first
07:24 < belcher> yes     
07:25 < belcher> 'encrypted_mnemonic_passphrase' inside wallet.json
07:25 < belcher> if that doesnt exist then its the same as before
07:25 < waxwing> right that's helpful
07:26 < belcher> ping me if you need anything clarified, hopefully iv made it as readable as possible
07:26 < belcher> also i renamed some stuff, like entropy_to_seed() is now wallet_data_to_seed()
07:29 < waxwing> i think there'll just be a lot of comments that it's confusing.
07:29 < waxwing> i know you were asking about what to call it, i don't know, but 'mnemonic passphrase' is going to befuddle people, because i think everyone has in mind that 'mnemonic' is the from-dictionary multi-word phrase.
07:30 < belcher> i imagine it that its the passphrase for the mnemonic
07:32 < belcher> open to another name though, renaming is easy
07:32 < belcher> coming up with good names is hard, one of the hardest things sometimes
07:32 < waxwing> everything is working at the moment (generate, display, showseed, recover)
07:33 < waxwing> is it all (except showseed) done in Qt too?
07:33 < belcher> yes     
07:33 < waxwing> ok will test out a bit
07:50 < waxwing> i would go with 'recovery password'
07:51 < waxwing> either that, or something that further emphasises that it's an additional factor, so it's not a replacement or something.
07:51 < waxwing> without making the name too long ...
07:53 < belcher> 'recovery passphrase'
07:53 < belcher> so it would have 'mnemonic recovery phrase' + 'recovery passphrase'
07:54 < belcher> hmm yeah im happy with most any name
07:55 < belcher> electrum call it "seed extension" and actually tells you that you must write it down
07:55 < belcher> i suppose theres a danger that the user doesnt realize, then forgets the passphrase and loses their money
07:58 < waxwing> belcher, the thing is, i don't understand why you want to use the term 'passphrase', 'phrase' suggests multiword no?
07:58 < waxwing> this is generally going to be one word, although it doesn't have to be
07:58 < belcher> gpg calls its thing a passphrase
07:58 < belcher> usuallyaverylongstringmadeupseveralwords
07:59 < waxwing> in terms of danger, yes, it's more powerful (considerably, deniability etc), but more dangerous too.
07:59 < waxwing> i *probably* wouldn't use this system, although i'm undecided.
08:00 < waxwing> yes i see the argument for it being a phrase. but the situation is becoming super-confusing for the user: there are two separate sets of multiword phrases for them to remember. i do think it's kind of smart, but it's a big step up in complexity.
08:00 < belcher> im hoping newbs would default to not using it, with the "if you dont know what this is, write no"
08:00 < waxwing> yes, true, but it's hard for them to make that kind of decision.
08:01 < waxwing> damn, i find it hard to decide myself, even though i basically understand it :)
08:01 < belcher> id love to find a way to make it not confusing
08:01 < belcher> you can also write down the passphrase alongside the mnemonic
08:01 < belcher> get a steel sheet and use a dremel to write the mnemonic and passphrase, then bury it
08:01 < belcher> then on paper write the mnemonic without the passphrase and put that in your friends house
08:02 < waxwing> i didn't find any problems so far, but i did find a bug in my own code heh
08:02 < belcher> which bug?
08:02 < waxwing> generate with electrum hangs on trying to sync the new wallet; it's just a matter of forgetting to set self.wallet_synced = True in that branch of the sync code.
08:03 < waxwing> you'll see 'no tx used' and then it doesn't actually load into the wallet tab. just one line, no big deal.
08:03 < belcher> oooh
08:03 < belcher> yeah i noticed that, didnt even realize it was a bug
08:04 < waxwing> another cosmetic annoyance is you have to manually resize the wallet tree view
08:04 < waxwing> i really should have fixed that, looked at it once or twice but didn't find an immediate solution, there is some obscure way of resizing columns automatically, but it's tricky
08:05 < waxwing> because some of the entries are too long anyway (like xpubs), so you want it to expand in a very particular way.
08:05 < waxwing> well no matter, better focus on making stuff correct.
08:08 < belcher> have you heard that electrum wont be releasing a segwit/p2sh version but instead will go straight to bech32
08:09 < waxwing> yes i think i did. oh well; trezor and ledger (and i think samourai?) are doing bip49 already though.
08:10 < waxwing> but bech32 means you can't receive from non-updated wallets.
08:10 < waxwing> that's why i thought the transition would take quite a while.
08:10 < belcher> yep     
08:10 < waxwing> i'd have thought bech32 is more for the power usage today; like big orgs that do a lot of internal txs.
08:10 < belcher> first thing that will happen is wait for all wallet to be able to send to bech32
08:10 < belcher> yes exactly
08:10 < belcher> for change addresses and stuff
08:11 < waxwing> so electrum is being a bit weird if that's the case. they did some sterling work on handling forks, perhaps thomasv fell a bit behind with segwit because of that.
08:11 < waxwing> but going straight to bech32 seems a bit weird for a consumer wallet to me.
08:12 < waxwing> afk for an hour or two.
09:16 -!- quitobro [~quitobro@pool-68-129-86-134.nycmny.fios.verizon.net] has joined #joinmarket     
09:17 < quitobro> sup guys :)
09:17 < belcher> hello
09:18 < quitobro> how’s it going today?
09:19 -!- delinquentme [~delinquen@2602:306:ceb7:990:c5cf:32a3:c15:357b] has joined #joinmarket     
09:19 < belcher> another day
09:24 < quitobro> indeed. i’m still syncing my blockchain data :(
09:36 -!- headcase [~headcase@unaffiliated/headcase] has quit [Remote host closed the connection]     
09:37 -!- delinquentme [~delinquen@2602:306:ceb7:990:c5cf:32a3:c15:357b] has quit [Ping timeout: 252 seconds]     
09:50 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has joined #joinmarket     
10:13 -!- coins123 [~coins123@unaffiliated/coins123] has quit []     
10:25 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 260 seconds]     
10:43 -!- delinquentme [~delinquen@2602:306:ceb7:990:cdf7:d527:d537:2e8c] has joined #joinmarket     
11:05 -!- fredrin [~fredrin@cm-84.209.144.12.getinternet.no] has joined #joinmarket     
11:26 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
11:26 -!- cbits [~cbits@c83-250-193-158.bredband.comhem.se] has joined #joinmarket     
11:27 -!- headcase_ [~headcase@unaffiliated/headcase] has joined #joinmarket     
11:30 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 240 seconds]     
11:50 -!- StopAndDecrypt [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has quit []     
11:52 -!- cbits [~cbits@c83-250-193-158.bredband.comhem.se] has quit [Ping timeout: 240 seconds]     
11:52 -!- StopAndDecrypt [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has joined #joinmarket     
11:54 -!- StopAndDecrypt [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has quit [Client Quit]     
11:55 -!- StopAndDecrypt [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has joined #joinmarket     
11:55 < waxwing> belcher, so you don't like 'seed extension', hmm, i just have a strong opinion that calling it a 'mnemonic' anything is going to confuse people with the mnemonic seed (i think everyone has become familiar with that), and 'passphrase' is going to confuse people because they will be unsure whether they have to enter more than 1 word.
11:55 < waxwing> 'seed extension' does indeed sound a bit alien, but i'd prefer it.
11:56 < belcher> i see, so you'd use "mnemonic" as a noun instead of adjective
11:56 < belcher> id be happy with "password" then
11:56 < waxwing> but more than that, i think it might be necessary to do this: write a short, isolated document in the code repo explaining how it's used, and then hardcode a link to that document on the command line and GUI when the option is presented.
11:56 < quitobro> is it like a salt?
11:56 < waxwing> maybe that's the only sensible thing to do really, no short name is going to solve the confusion of a user who hasn't heard of the idea.
11:56 < belcher> what kind of things should be in that document? since its not much more than just the 12 word thing 
11:57 < waxwing> quitobro, well somewhat, but not solving the problem a salt solves
11:58 < waxwing> it's all variations on the same theme: when you make a commitment, you have to add randomness to the preimage of the hash to make it not guessable; when you make a password hash db, same thing, here, you don't strictly need it but it adds another security factor and creates a deniability possibility
11:58 < waxwing> belcher, well could be that wiki thing that (you?) wrote, but a little more specific
11:59 < waxwing> defining the terms, and explaining their purpose in this exact context.
11:59 < belcher> how about "mnemonic extension"? since right now "seed" is only used to mean the big number that is fed into bip32_master_key()
12:00 < belcher> so you have "mnemonic" or "mnemonic phrase" and then "mnemonic extension"
12:00 < waxwing> sounds ok to me.
12:00 < belcher> ok, ill rename everything now
12:00 < waxwing> but i think ultimately you should decide, you've actually looked into this.
12:01 < waxwing> so if you want to stick with what it was, that's ok with me, but i really think we should point the user at something to explain it, whatever it's called.
12:02 < belcher> its all explained here https://github.com/JoinMarket-Org/joinmarket/wiki/Using-the-JoinMarket-internal-wallet
12:02 < belcher> except for this new mnemonic extension part
12:02 < waxwing> sure, but that's the only part i think is confusing :) maybe it's just me, but i feel like everyone knows the '12/13/24/25 word seed' idea by now.
12:03 < waxwing> and everyone's familiar with the idea a wallet file can/should be encrypted with a password.
12:04 < belcher> so would adding an explanation to that wiki page be enough?
12:04 < waxwing> i'm probably overplaying how confusing it is because i just hadn't looked into it before. so ultimately i'll let you have the final call on it.
12:04 < waxwing> if you like. like i said before, i don't like relying on wiki pages for this, though.
12:05 < waxwing> just not secure, wouldn't be even if we flagged edits immediately.
12:05 < belcher> alright well thats a different issue
12:05 < belcher> if its not the wiki then its in some github .md document
12:05 < waxwing> yes that's why i said a hardcoded link to a doc in the repo
12:05 < belcher> either way, the docs could explain it in the same way they explain how to use the wallet generally
12:05 < belcher> ok     
12:06 < waxwing> (admittedly it's a bit shitty, i only suggest doing it that way because, while short, it has to be a paragraph or two. too much on a dialog or command line prompt i think (plus formatting)
12:07 < waxwing> i mean you could ignore all this bikeshedding if you like, given it already says 'if you don't know what this is choose no'
12:07 < waxwing> just having nightmares of users saying "i did this and now i don't know where my wallet is" :)
12:08 < belcher> im changing it to "mnemonic extension" since i was never 100% sure about the name "mnemonic passphrase"
12:09 < quitobro> waxwing: qq on this -> https://github.com/mecampbellsoup/joinmarket-clientserver/blob/5e43ad0b99eab894c701d42ae7cdb906c33c1cb5/docs/TESTING.md#L5
12:10 < quitobro> “root dir” == JM dir right
12:10 < waxwing> yeah sorry slightly sloppy mean joinmarket-clientserver/ in that case
12:10 < waxwing> underscore sorry
12:10 < quitobro> it’s hyphened no?
12:10 < waxwing> oh yeah lol sorry
12:11 < quitobro> lol ok
12:11 < quitobro> the dir w/ setupall.py in it etc
12:11 < waxwing> hmm requirements-dev.txt .. does that even exist in that repo, can't remember
12:11 < quitobro> ya     
12:11 < quitobro> (jmvenv) satoshi@hydrogen:~/joinmarket-clientserver$ cat requirements-dev.txt 
12:11 < quitobro> # matplotlib
12:11 < quitobro> # numpy
12:11 < quitobro> pexpect
12:11 < quitobro> coverage==4.2
12:11 < quitobro> pytest==3.0.5
12:11 < quitobro> pytest-cov==2.4.0
12:11 < quitobro> python-coveralls
12:11 < quitobro> mock
12:11 < waxwing> quitobro, lol no need for that :)
12:12 < quitobro> sry     
12:12 < quitobro> i meant, "yes"
12:12 < waxwing> also you keep doxing yourself satoshi, be careful :)
12:12 < quitobro> hehe
12:16 < quitobro> waxwing: in that same TESTING.md file where it reads “both files will need minor edits for your btc configuration”, those edits are described in the previous repo’s docs here, correct? https://github.com/Joinmarket-Org/joinmarket/wiki/Testing
12:17 < waxwing> quitobro, there is one important difference, which i probably should have noted:
12:17 < waxwing> you don't need the walletnotify lines in your bitcoin.conf for this repo; it no longer uses that feature
12:17 < waxwing> also means you don't need curl (although of course on linux you almost certainly have it anyway)
12:18 < quitobro> ok i’ll update the doc
12:18 < waxwing> yes if you go through all the steps and accumulate some edits that'd be good; i wrote that in a bit of a rush, it's just a few notes.
12:19 < waxwing> i used 'python -m py.test' in that command line instead of just 'py.test ..' for some obscure reason i've now forgot; and nowadays i find it works fine with just 'py.test ..' so, shrug, dunno.
12:20 < waxwing> ah so the 'curl is also needed' line should be deleted. not correct now.
12:20 < quitobro> ok yea i’m going thru it… “you'll first want to copy bitcoin.conf in the test/ directory to a place you choose” this isn’t necessary if you just set the flat `—btcconf=/home/user/.bitcoin/bitcoin.conf` no?
12:20 -!- [SaD] [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has joined #joinmarket     
12:20 < waxwing> quitobro, the reason i say that is because, you don't want (usually) to use your "normal" bitcoin.conf file for your tests, your normal one has your real (secure) password, usually.
12:21 -!- StopAndDecrypt [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has quit [Ping timeout: 240 seconds]     
12:21 < waxwing> and it may have various other custom configuration that you use for your real node.
12:21 < quitobro> can’t the regtest suite just default to using test/bitcoin.conf?
12:21 < waxwing> ah, hmm, yes i think that would be an improvement, i can't think of any reason why a tester has to customise (at least, usually)
12:21 < quitobro> waxwing: just saw your GH comment; yea i typed that before i realized that i can run regtest w/o synced blockchain
12:22 < quitobro> so we could put btcconf=~/joinmarket-clientserver/test/bitcoin.conf hardcoded maybe
12:22 < quitobro> not sure everyone will have installed to ~/joinmarket-cs tho
12:23 -!- delinquentme [~delinquen@2602:306:ceb7:990:cdf7:d527:d537:2e8c] has quit [Ping timeout: 255 seconds]     
12:23 -!- cbits [~cbits@c83-250-193-158.bredband.comhem.se] has joined #joinmarket     
12:23 < waxwing> yeah. so, edit test/bitcoin.conf (remove the old walletnotify lines), and then rejigging the file conftest.py to use it. you should be able to enforce relative path in conftest.py by referring to the working directory.
12:24 < waxwing> i want to spend a bit of time looking at belcher 's code now
12:24 < belcher> i just pushed an edit that renamed those things
12:25 < waxwing> thanks
12:26 < quitobro> waxwing: remove alertnotify as well? those were set to use wget and not curl anyway it looks like
12:27 < belcher> alerts have been removed from core since 0.12 iirc
12:27 < belcher> so that will never fire
12:28 < waxwing> yeah. and curl or wget were both usable. anyway right, we can ditch all that.
12:30 < quitobro> i edited this page to put OSX clearly in its own section
12:30 < quitobro> https://github.com/JoinMarket-Org/joinmarket/wiki/Testing
12:30 < quitobro> did i demarcate correctly
12:32 < waxwing> looks fine sure
12:33 < waxwing> belcher, so you're encrypting mnemonic_extension (ME for short) like sha256(ME)[:8] + '|' + ME
12:34 < belcher> yes     
12:34 < belcher> the reason is:
12:35 < waxwing> FAKESEED? :)
12:35 < belcher> the code here checks the length of the cleartext https://github.com/JoinMarket-Org/joinmarket/blob/master/joinmarket/wallet.py#L200-L203
12:35 < belcher> because sometimes that aes function doesnt throw ValueError if the password is wrong
12:35 < belcher> but for mnemonic extension we cant check the length since it could be any length, so i use a checksum instead
12:36 < belcher> its documented in the comments, a one-line comment explaining why
12:38 < waxwing> belcher, you mean the line that says 'has checksum' ?
12:38 < belcher> something like that yes
12:39 < waxwing> ok just checking, that's the only place i see a comment, maybe i missed it
12:40 < waxwing> iirc that'd mean for a password < 7 characters you'll get 2 aes blocks (1 iv, 1 ciphertext), and for > 7 it'll be more than 2 blocks
12:40 < waxwing> but, meh, this is not the wallet encryption only the recovery
12:41 < waxwing> for the wallet encryption you can just hash it as only going forwards
12:41 < belcher> i didnt work it out, yes they'll be some length where its two blocks
12:41 < belcher> we could add out own padding? checksum|extension|padding-until-3-aes-blocks ?
12:42 < belcher> idk how important it is
12:42 -!- [SaD] is now known as StopAndDecrypty     
12:42 -!- StopAndDecrypty is now known as StopAndDecrypt     
12:42 < waxwing> well i was musing about it, but as we know some people like to use crazy large passwords (well, not crazy perhaps!)
12:42 < belcher> im one of those
12:42 < belcher> depends what you mean by crazy though
12:42 < waxwing> but yes as mentioned, it clearly is not nearly as important as the wallet encryption, and that has fixed length
12:44 < waxwing> i think we should have a test function for a variety of MEs, from crazy short to crazy long and we should also make sure we explicitly reject non-ascii chars
12:46 < waxwing> padding out to say a 4 block minimum (which, discounting iv is 3x16-9 so 39) would seem pretty sensible too, not a big deal but it's icky to give away that it's a short phrase when it happens to be so.
12:47 < waxwing> even if 99% of the time that information wouldn't be useful
12:47 < belcher> ok, sounds wise
12:47 < belcher> comment that the PR too
12:47 < waxwing> yes, will do now
12:51 < waxwing> ignore build test failures in case you missed that.
12:51 < waxwing> arubi, ping on the above, any idea what causes that travis failure?
12:52 < arubi> yep I'll have a look
12:54 < belcher> how would you make it reject non-ascii chars waxwing ?
12:56 < waxwing> there's probably a smart way but you could just check ord() on the individual bytes i think?
12:56 < waxwing> i should really look that up :)
12:57 < waxwing> hmm according to stackexchange, my answer is indeed correct and indeed shitty :) https://stackoverflow.com/questions/196345/how-to-check-if-a-string-in-python-is-in-ascii#196391
12:59 < belcher> how would the padding bytes be generated? just make them all zeros?
13:00 < waxwing> hmm; if we require ascii we could start the padding (the additional-to-pkcs7-padding) with a non-ascii byte?
13:02 < waxwing> hmph, it gets a bit more confusing/complicated than we really want. like i said in that last comment, pkcs7 already fudges it a fair bit (whole 16 byte blocks), so maybe leave this out. not sure.
13:03 -!- fredrin [~fredrin@cm-84.209.144.12.getinternet.no] has quit [Ping timeout: 240 seconds]     
13:04 < GitHub89> [joinmarket-clientserver] mecampbellsoup opened pull request #86: [WIP] Update TESTING.md (master...update-testing-doc) https://git.io/v5xoP
13:08 < waxwing> it's a bit of a quandary. the lax user who chooses <=7 char ME exposes that fact in the file. for someone to 'crack' it they'd have to actually have the 12 word seed. but, if they do, the ME is basically worthless. this makes padding seem essential. i guess, you can pad the ME *before* encryption though, i think that is very easy no?
13:08 < waxwing> well no, what am i saying, of course it's before, not after, encryption ...
13:09 < waxwing> so something like, take one non-ascii byte like ff or something, and enforce minimum length of, i don't know, 30 chars, and then let pkcs7 do its normal padding to block size for 30 or more input bytes.
13:13  * waxwing wonders why there was never an AES block size debate
13:15 < arubi> so this PR running now, is that expected to fail?
13:15 < arubi> I see some succeeding and some failing
13:15 < waxwing> quitobro, yeah that looks good (the code change), let me know when you've tried it out. we could also edit the run_tests.sh (which contains the pytest command used in the travis test)
13:15 < belcher> debates happen when theres an inherent conflict :p which aes doesnt have
13:16 < waxwing> arubi, ah this is very inconsistent.
13:16 < waxwing> let me find one of the type i was seeing last couple of days
13:16 < belcher> waxwing the extension and entropy are encrypted with the same password, so if the adversary cracks the len<7 extension they can use the same password to get the entropy (words)
13:17 < arubi> waxwing, https://travis-ci.org/JoinMarket-Org/joinmarket-clientserver/builds
13:17 < belcher> although tbh if they have access to that computer and those skills they can grab the private keys from memory
13:18 < belcher> what would be wrong by padding the cleartext so its sha256^2(ME)[:8]|ME|ff*padding-count
13:18 < waxwing> belcher, oh! same password, right
13:18 < waxwing> belcher, nothing, i also came to that conclusion re padding
13:19 < waxwing> arubi, looking at the build history leaves me seriously confused. i have seen more than one kind of failure, and even more, i have seen builds fail and then automatically correct themselves later. don't know what's going on actually.
13:20 < quitobro> waxwing: the btcroot=/path/to/bitcoin/bin… the bitcoin bin is bitcoind? /usr/bin/bitcoind
13:20 < quitobro> in my case
13:20 < waxwing> quitobro, that directory is supposed to be the location of bitcoind and bitcoin-cli
13:20 < quitobro> seems like that could also be the default
13:20 < quitobro> oh so just /usr/bin ?
13:21 < waxwing> quitobro, yes there could be a default there, true, or the default could be for it to be in path. i forget, maybe it already is.
13:21 < quitobro> which file would that be set in, conftest.py?
13:21 < arubi> yea all errors seem completely different from each other
13:21 < arubi> travis alerts that they've updated their images, might be related..
13:22 < waxwing> belcher, so what happens step by step. suppose no padding. then, suppose attacker observes short encryptedME. then, tries 7char ME -> deduces key -> assuming correct key, decrypts seed -> deduces HD wallet and checks.
13:22 < waxwing> because same encryption key.
13:23 < belcher> yep     
13:23 < waxwing> quitobro, default is '' indeed; see --btcroot
13:24 < belcher> solution is to use padding to always make it like 5 blocks
13:24 < arubi> so waxwing maybe add : https://blog.travis-ci.com/2017-08-29-trusty-image-updates
13:24 < waxwing> so i think if a person has it in /usr/bin and it's in the path, it works.
13:24 < waxwing> quitobro, if you have bitcoind in your path, try the test without setting --btcroot
13:24 < quitobro> rgr     
13:25 < waxwing> belcher, yeah something like that (although we presumably allow > 5 also)
13:25 < waxwing> arubi, thanks
13:25 < arubi> quitobro, sorry, are you trying to run tests?
13:26 < quitobro> yes     
13:26 < arubi> oh I see you know about run_tests.sh , it's supposed to take care of everyting
13:26 < arubi> everything*
13:26 < quitobro> i’m doing the `python -m py.test …` approach rn
13:27 < waxwing> he's basically trying to figure out how to run them and also make additions to the testing doc.
13:27 < arubi> yea now I see
13:27 < quitobro> should i use the shell sciprt instead
13:27 < quitobro> everything failing atm
13:27 < waxwing> quitobro, well you could but i'd say no; that's more for travis, for automating.
13:27 < waxwing> you want to know how to do them for your own work
13:28 < arubi> if you're running from within your virtualenv, it should pretty much work if a pristine bitcoind\-cli are in your path
13:28 < quitobro> i did a couple extra steps
13:28 < arubi> so, none that are running anything, also take care of placing joinmarket.cfg and bitcoin.conf at the right places
13:28 < quitobro> mv test/regtest_joinmarket.cfg joinmarket.cfg
13:29 < arubi> and did you clone minircd too to ./miniircd ?
13:29 < quitobro> i did git clone of it it’s not in my path tho
13:29 < quitobro> it’s just a dir in joinmarket-cs
13:29 < arubi> that should be enough
13:30 < waxwing> quitobro, 'cp' better than 'mv' there :) you might need the original at some point
13:30 < arubi> do you have anything for bitcoind\-cli in your path right now that might interfere, or a ~/.bitcoin/bitcoin.conf ?
13:30 < quitobro> i meant cp sry
13:30 < quitobro> yea i have a bitcoin.conf but i moved it to bitcoin.conf.bak
13:31 < arubi> are you running from somewhere that can't bind to 127.0.0.1 ?
13:32 < quitobro> ok fixed it - i hadn’t actually done `cp test/bitcoin.conf ~/.bitcoin/bitcoin.conf`
13:32 < quitobro> now it’s working
13:32 < arubi> oh phew :)
13:33 < waxwing> quitobro, but i thought your PR assumed a default location of test/bitcoin.conf ?
13:33 < waxwing> (if no --btcconf= is used)
13:33 < quitobro> yea it does. i just hadn’t actually put the test-specific bitcoin.conf at that location
13:33 < quitobro> i’m running these tests w/o a —btconf= flag anyway
13:33 < quitobro> using my pr
13:33 < quitobro> branch*
13:35 < waxwing> belcher, it still feels off ... are we effectively giving the attacker a crib *if* the user uses a very weak password as an ME.
13:35 < waxwing> suppose joe user decides that the ME doesn't have to be very secure and uses 'monkey' as his password.
13:35 < waxwing> sorry 'monkey' as his ME i mean.
13:36 < belcher> right, then someone could bruteforce it
13:36 < belcher> whats the solution then, lots of stretching?
13:36 < waxwing> it's kind of tricky logic but i'm somehow arguing it's made it worse, because although most users won't do that, those that do have exposed the seed, in theory.
13:36 < waxwing> well yes there's that, but i find the logic of this rather difficult to disentangle :)
13:38 < waxwing> quitobro, i actually didn't understand that, if you're using your branch, and you're *not* specifying a --btcconf flag, then i thought your code takes from test/bitcoin.conf by default
13:39 < quitobro> waxwing: yea you’re right, good catch...
13:39 < quitobro> think i may need to remove this https://github.com/mecampbellsoup/joinmarket-clientserver/blob/97606573577b8671400cd273b98baca65f37bf16/conftest.py#L79
13:40 < waxwing> i wouldn't remove it; some people might want to use a specific non-default location for whatever reason
13:40 < waxwing> your idea is to have test/bitcoin.conf as the default, that's good i think
13:40 < quitobro> it seems like that line requires it to be passed as a flag tho `—btcconf`, no?
13:40 < waxwing> default here https://github.com/mecampbellsoup/joinmarket-clientserver/blob/97606573577b8671400cd273b98baca65f37bf16/conftest.py#L44
13:41 < quitobro> i can’t tell how/whether the pytest setup on line 74 invokes pytest_addoption
13:41 < quitobro> is that part of the pytest api?
13:42 < waxwing> btw good idea to use os.path.join(root_path(), 'test/bitcoin.conf') instead of using '+' it's more platform-independent
13:42 < quitobro> ok     
13:42 < waxwing> quitobro, yeah pytest handles that with the addoption thing
13:43 < quitobro> is there a way to run the tests and fail fast? 
13:44 < quitobro> otherwise it takes a while and i get tons of noisy failures that are all probably tthe same reason
13:44 < waxwing> quitobro, yes, you can do just one test for example
13:44 < waxwing> py.test .. options ... jmbitcoin/test/test_addresses.py -s
13:45 < waxwing> this will only run that one test file (which may contain more than one test)
13:45 < waxwing> and the `-s` flag prints the output to stdout, that's optional
13:45 < waxwing> but of course it's helpful sometimes when debugging
13:45 < waxwing> you can also go further and test only one specific test function, e.g.:
13:46 < waxwing> py.test .. options jmbitcoin/test/test_addresses.py -k name_of_test_function
13:46 < quitobro> great ty
13:46 < waxwing> where 'name of test function' is a string that matches the name of that one function (doesn't have to be total, just a string match)
13:47 < quitobro> waxwing: so i should be running these tests w/o passing a btcconf flag *and* without a bitcoin.conf at ~/.bitcoin right
13:48 < waxwing> well the idea would be that you don't need to touch (or move) the ~/.bitcoin/bitcoin.conf, if you allow the default, which is to read it from test/bitcoin.conf
13:48 < quitobro> ok     
13:50 < quitobro> waxwing: here is the test command as i’m invoking it:     python -m py.test --cov=jmclient --cov=jmbitcoin --cov=jmbase --cov=jmdaemon --cov-report html --btcpwd=123456abcdef --nirc=2
13:50 < quitobro> seems to be working more or less
13:51 < quitobro> had a few failures last time but will update
13:51 < waxwing> ok, so a couple of things: i believe you can replace `python -m py.test` with just `py.test` (i mentioned about that before), and also the --cov stuff can all be omitted, although in future it may be useful for reading the coverage reports
13:51 < waxwing> (it's just a pain when you're only testing it out)
13:53 < waxwing> also, one thing i have discovered: do `rm -rf ~/.bitcoin/regtest` (carefully!) before a full run, there is unfortunately an issue with test_wallets.py that requires the regtest blockchain to be empty on startup for some obscure reason i forgot.
13:53 < waxwing> honestly i think test_wallets.py needs to be completely rewritten anyway. but, another time.
13:54 < waxwing> hmm i should really make a one line bash script or alias for that command, i'm terrified i'm going to delete my blockchain with it ...
13:57 < arubi> waxwing, it's easy to add the 'datadir=..' line to your bitcoin.conf like the test script does
13:57 < arubi> so really your datadir can be anywhere and not in ~/.bitcoin (scary I agree)
13:57 < waxwing> ah yes, good point :)
13:58 < arubi> * bonus points for keeping it in /dev/shm too, way faster :)
14:01 < arubi> iirc a PR to core where you could set a "sub .conf" file for network specific settings.  that should be very useful for software like jm that wants to interface to a local client
14:02 < arubi> it could be made even more powerful if instead of just network specific, you could have "application specific" .conf, so alongside multiwallet and rpcauth, you might set some specific fees, alerts, etc
14:02 < arubi> network_jm_regtest=... , never worry about pytest again :)
14:03 < waxwing> so .. like for one daemon? would it like serve rpc on different ports ?
14:03 < waxwing> bah it's late, my brain no longer works :)
14:03 < arubi> maybe it doesn't have to really
14:04 < arubi> most stuff can deal with read only stuff like zmq and rest
14:04 < arubi> jm needs a wallet, multiwallet is awesome, but also needs some alerts
14:05 < arubi> bitcoind is a beast
14:06 -!- headcase_ [~headcase@unaffiliated/headcase] has quit [Remote host closed the connection]     
14:10 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
14:14 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 240 seconds]     
14:24 -!- belcher [~belcher@unaffiliated/belcher] has quit [Quit: Leaving]     
14:27 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has joined #joinmarket     
14:32 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 248 seconds]     
14:53 -!- MaxSan [~one@46.19.137.116] has joined #joinmarket     
15:32 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
15:33 -!- headcase [~headcase@unaffiliated/headcase] has quit [Client Quit]     
15:34 < quitobro> arubi woah cool - which test script are u referring to which adds `datadir=..` ? this… ? https://github.com/mecampbellsoup/joinmarket-clientserver/blob/ac64f89222ed69bee853a1ae822e8ba7cecb85a5/test/run_tests.sh#L47
15:35 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
15:50 -!- cbits [~cbits@c83-250-193-158.bredband.comhem.se] has quit [Read error: Connection reset by peer]     
15:51 -!- quitobro [~quitobro@pool-68-129-86-134.nycmny.fios.verizon.net] has quit [Quit: quitobro]     
16:12 -!- belcher [~belcher@unaffiliated/belcher] has joined #joinmarket     
16:30 -!- quitobro [~quitobro@104.247.38.147] has joined #joinmarket     
16:33 -!- headcase [~headcase@unaffiliated/headcase] has quit [Remote host closed the connection]     
16:38 -!- quitobro [~quitobro@104.247.38.147] has quit [Quit: quitobro]     
16:46 -!- quitobro [~quitobro@104.247.38.147] has joined #joinmarket     
16:46 -!- quitobro [~quitobro@104.247.38.147] has quit [Client Quit]     
16:48 -!- quitobro [~quitobro@104.247.38.147] has joined #joinmarket     
16:49 -!- quitobro [~quitobro@104.247.38.147] has quit [Client Quit]     
16:51 -!- quitobro [~quitobro@104.247.38.147] has joined #joinmarket     
16:51 -!- quitobro [~quitobro@104.247.38.147] has quit [Client Quit]     
16:52 -!- quitobro [~quitobro@104.247.38.147] has joined #joinmarket     
16:59 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
17:02 -!- quitobro [~quitobro@104.247.38.147] has quit [Quit: quitobro]     
17:03 -!- headcase [~headcase@unaffiliated/headcase] has quit [Remote host closed the connection]     
17:03 -!- StopAndDecrypty [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has joined #joinmarket     
17:03 -!- StopAndDecrypt [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has quit [Disconnected by services]     
17:03 -!- StopAndDecrypty is now known as StopAndDecrypt     
17:05 -!- quitobro [~quitobro@104.247.38.147] has joined #joinmarket     
17:09 -!- quitobro [~quitobro@104.247.38.147] has quit [Client Quit]     
17:10 -!- quitobro [~quitobro@104.247.38.147] has joined #joinmarket     
17:12 -!- quitobro [~quitobro@104.247.38.147] has quit [Client Quit]     
17:15 -!- quitobro [~quitobro@104.247.38.147] has joined #joinmarket     
17:18 -!- quitobro [~quitobro@104.247.38.147] has quit [Client Quit]     
17:23 -!- quitobro [~quitobro@104.247.38.147] has joined #joinmarket     
17:26 -!- quitobro [~quitobro@104.247.38.147] has quit [Client Quit]     
17:32 -!- MaxSan [~one@46.19.137.116] has quit [Quit: Leaving.]     
18:10 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
18:37 -!- poohaha [2d7a8a0f@gateway/web/freenode/ip.45.122.138.15] has joined #joinmarket     
19:14 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has joined #joinmarket     
19:47 -!- poohaha [2d7a8a0f@gateway/web/freenode/ip.45.122.138.15] has quit [Ping timeout: 260 seconds]     
19:57 -!- StopAndDecrypt [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has quit [Ping timeout: 248 seconds]     
19:57 -!- StopAndDecrypt [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has joined #joinmarket     
20:14 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 248 seconds]     
20:19 -!- StopAndDecrypty [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has joined #joinmarket     
20:20 -!- StopAndDecrypt [~StopAndDe@c-73-248-248-9.hsd1.nj.comcast.net] has quit [Ping timeout: 252 seconds]     
20:20 -!- headcase [~headcase@unaffiliated/headcase] has quit [Remote host closed the connection]     
20:23 -!- Giszmo [~leo@pc-204-28-214-201.cm.vtr.net] has quit [Ping timeout: 264 seconds]     
20:25 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
20:29 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 264 seconds]     
20:33 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
20:34 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has joined #joinmarket     
20:34 -!- headcase_ [~headcase@unaffiliated/headcase] has joined #joinmarket     
20:37 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 240 seconds]     
20:45 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 260 seconds]     
20:47 -!- Giszmo [~leo@pc-204-28-214-201.cm.vtr.net] has joined #joinmarket     
21:09 -!- headcase_ [~headcase@unaffiliated/headcase] has quit [Remote host closed the connection]     
21:31 -!- quitobro [~quitobro@pool-68-129-86-134.nycmny.fios.verizon.net] has joined #joinmarket     
21:35 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
21:41 -!- quitobro [~quitobro@pool-68-129-86-134.nycmny.fios.verizon.net] has quit [Quit: quitobro]     
21:43 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 240 seconds]     
21:49 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
21:51 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has joined #joinmarket     
21:53 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 240 seconds]     
21:58 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
22:01 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 260 seconds]     
22:02 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 248 seconds]     
22:07 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
22:10 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has joined #joinmarket     
22:11 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 240 seconds]     
22:20 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 255 seconds]     
22:39 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
22:44 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 260 seconds]     
22:48 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
22:57 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 255 seconds]     
23:07 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
23:17 -!- coins123 [~coins123@unaffiliated/coins123] has joined #joinmarket     
23:20 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 248 seconds]     
23:25 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
23:29 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 240 seconds]     
23:34 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
23:38 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has joined #joinmarket     
23:43 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 246 seconds]     
23:45 -!- delinquentme [~delinquen@108-235-112-153.lightspeed.sntcca.sbcglobal.net] has quit [Ping timeout: 240 seconds]     
23:48 -!- headcase [~headcase@unaffiliated/headcase] has joined #joinmarket     
23:52 -!- headcase [~headcase@unaffiliated/headcase] has quit [Ping timeout: 248 seconds]