--- Day changed Thu Jan 28 2016
00:11 -!- shorena [~shorena@unaffiliated/shorena] has joined #lightning-dev
02:08 -!- jtimon [~quassel@126.31.134.37.dynamic.jazztel.es] has joined #lightning-dev
02:23 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has joined #lightning-dev
04:11 -!- drnet [~drnett@178.165.129.207.wireless.dyn.drei.com] has joined #lightning-dev
04:12 -!- shorena [~shorena@unaffiliated/shorena] has quit [Ping timeout: 250 seconds]
04:45 -!- drnet [~drnett@178.165.129.207.wireless.dyn.drei.com] has quit [Quit: Leaving]
04:55 -!- cjcj [2e3b026a@gateway/web/freenode/ip.46.59.2.106] has joined #lightning-dev
05:21 -!- shorena [~shorena@unaffiliated/shorena] has joined #lightning-dev
05:47 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has joined #lightning-dev
05:50 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Ping timeout: 250 seconds]
06:17 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has joined #lightning-dev
06:42 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #lightning-dev
06:47 -!- shorena [~shorena@unaffiliated/shorena] has left #lightning-dev ["part"]
07:16 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Quit: leaving]
07:17 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has joined #lightning-dev
07:20 -!- dcousens [~anon@c110-22-219-15.sunsh4.vic.optusnet.com.au] has quit [Client Quit]
07:30 -!- Guest54430 [~chatzilla@97-90-24-187.dhcp.mtpk.ca.charter.com] has joined #lightning-dev
07:32 -!- Guest54430 is now known as roidster
07:32 -!- roidster is now known as Guest52040
07:33 -!- Guest52040 is now known as roidster
08:49 -!- zzyzx [~chatzilla@97-90-24-187.dhcp.mtpk.ca.charter.com] has joined #lightning-dev
08:49 -!- zzyzx is now known as Guest13742
08:50 -!- roidster [~chatzilla@97-90-24-187.dhcp.mtpk.ca.charter.com] has quit [Ping timeout: 276 seconds]
08:50 -!- Guest13742 is now known as roidster
08:50 -!- roidster is now known as Guest23846
08:53 -!- Guest23846 is now known as roidster
08:56 -!- cjcj [2e3b026a@gateway/web/freenode/ip.46.59.2.106] has quit [Quit: Page closed]
09:26 -!- ghtdak [~ghtdak@unaffiliated/ghtdak] has quit [Ping timeout: 256 seconds]
09:27 -!- ghtdak [~ghtdak@unaffiliated/ghtdak] has joined #lightning-dev
09:30 -!- CJP [~CJP@a83-163-77-195.adsl.xs4all.nl] has joined #lightning-dev
09:32 -!- bityogi [~textual@208-104-143-200.brvd.dsl.dyn.comporium.net] has joined #lightning-dev
12:09 -!- drnet [~drnett@62-46-154-124.adsl.highway.telekom.at] has joined #lightning-dev
12:36 -!- rusty2 [~rusty@pdpc/supporter/bronze/rusty] has joined #lightning-dev
12:38 < roasbeef> rusty2: heya so yeah by including the length, I mean adding the packet length to the AEAD construction under a separate key_2. 
12:38 < roasbeef> just typed up the reasoning for my sanity, but i'll copy pasta it here, sec 
12:38 < roasbeef> The use of independant keys for encrypting the length, and packet payload is required in order to avoid creating a decryption oracle for the packet payload.
12:39 < roasbeef> Although chachapoly-1305, by itself is IND-CPA (secure against a chosen plaintext attack), and IND-CCA2 (secure vs an adaptive chosen ciphertext attack), the *packet handling* itself can create a decryption oracle for the underlying plaintext payload.
12:39 < roasbeef> This would allow an attacker to launch a chosen plaintext attack, in the process decrypting a payload we've transmitted previously (!!) :x
12:40 < roasbeef> This attack was discovered under a augmented version of the decryption oracle model, wherein the oracle can be fed input *byte-by-byte*. 
12:40 < roasbeef>  So the attacker can take an old packet, feed the first 4-bytes of it to an attacker, then feed random bytes incrementally, until the oracle rejects the packet with a MAC error.
12:41 < roasbeef> With this attack, the attacker learns the decryption of the first 4 bytes of the ciphertext...
12:41 < roasbeef> It turns out that *any* protocol which encrypts the length, and runs over a transport that allow an attacker to feed bytes incrementally is susceptible to this attack. This vulnerability was discovered in an older version of OpenSSH
12:42 < roasbeef> One of the ways to disallow this attack completely is not encrypt the packet length at all. It's arguable that encrypting the length doesn't really provide anything since an attacker to deduce the packet lengths via a timing side-channel
12:42 < roasbeef>  Another way to prevent this attack is to include a *separate* MAC for the length under the same key. This MAC would be placed right after the length in the packet header. 
12:43 < roasbeef> Using a separate key also renders this attack obsolete, since the length and packet are encrypted with two independant streams. Hmm trying to dig up where I read that this is also an acceptable defense...
12:43 < roasbeef> Anyway, here's a good blog post on the subjcet: http://www.thoughtcrime.org/blog/the-cryptographic-doom-principle/
12:44 < roasbeef> and here's the original research paper (attacks the older openssh): http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf
12:45 < roasbeef> erm, assuming rusty2 is rusty-rusty, but I guess this channel is logged? 
12:45 < roasbeef> will post to the ML also replying the the laundry list w/ some other stuffs
12:46 < CJP> You're writing a lot for someone who doesn't know whether anyone is reading ;-)
12:46 < roasbeef> hehe :p 
12:47 < roasbeef> but yeah, so in essense if we want to encrypt the packet length we *must not* use the same stream as is used for the packet payload 
12:47 < CJP> So the subject is the encryption/authentication layer?
12:47 < roasbeef> yeah
12:48 < roasbeef> justifying why I recommened encrypting the length with a new stream, instead of using the stream for the packet payload 
12:48 < roasbeef> rusty2: oh and yeah, the length is protected under the MAC
12:48 < CJP> New stream = something like a separate TCP stream?
12:49 < roasbeef> a new chachapoly1305 stream
12:49 < roasbeef> as in, an independant key for encrypting the length
12:50 < CJP> Ah, that explains. I thought separate TCP streams wouldn't solve anything since an attacker might be able to intercept both.
12:50 < roasbeef> btw, for the folks checking out the paper, you can skip to section 6.1 where it discusses the counter measures 
12:52 -!- belcher [~user@unaffiliated/belcher] has joined #lightning-dev
12:52 < CJP> I haven't looked into the encryption layer yet, but it seems more complicated than I assumed. Apparently, you can't just take a non-encrypted version of the Lightning protocol, and blindly put an encryption layer underneath?
12:53 < roasbeef> well, this attack arises due to the way you've gotta process the packets if the length is encrypted 
12:53 < roasbeef> if you don't encrypt the length at all, everything is OK
12:54 < CJP> But if you encrypt everything the same way, in a single stream (say, like TLS probably does), that would not be OK?
12:56 < roasbeef> I don't think TLS encrypts the packet length, IIRC it's part of the AEAD scheme so protected under the MAC as associated data, but not encrypted
12:57 < roasbeef> yeh with TLS 1.3 "The length of the vector is not included in the encoded stream.
12:57 < roasbeef> " 
12:57 < roasbeef> err sorry, ignore that stopped reading to quickly :p 
12:57 < CJP> But if you had a (length,payload), ... protocol inside a byte stream that is encrypted with TLS, you would have a packet length encrypted by TLS. Or are we talking about different things?
12:58 < roasbeef> "
12:58 < roasbeef> Note in particular that the length of a record or absence of traffic itself is not protected by encryption unless the sender uses the supplied padding mechanism - see Section 5.2.3 for more details."
12:58 < roasbeef> ahh I see what you're getting at
12:59 < roasbeef> well if we we're using TLS, we wouldn't incrementally read the ciphertext, we'd just get the decrypted payload from the record 
13:02 < roasbeef> hmmm, i'd have to think about it more though, but under that scheme I think one would be protected by the attack due to TLS's input handling 
13:02 < CJP> I haven't looked in existing Lightning encryption designs yet, but the way I always saw it is "something like TLS, but simpler, with an alternative for the CA system that is tailor-made for Lightning, and maybe packet-based instead of stream-based".
13:02 < roasbeef> the attack exploits an implementation detail in input handling
13:03 < roasbeef> well, I think most would like to avoid using TLS in order to avoid depending on openssl 
13:04 < roasbeef> if the packet length isn't encrypted, or uses one of the other countermeasures this attack is irrelevant
13:04 < CJP> hehe yeah. And nobody wants GNUTLS because of GPL?
13:11 < roasbeef> heh yeh I guess :p
13:13 < CJP> I just realized today that in Amiko Pay, I've been spending a month or so trying to make just 100 lines of code work. 8-O
13:13 < roasbeef> what behavior in particular? 
13:14 < CJP> It sounds pretty bad, but that 100 lines of code is "standing on the shoulders of a giant", and the work is of course in building the giant.
13:14 < CJP> The 100 lines is a simple version of a payment channel.
13:15 < CJP> Actually it isn't even a real payment channel; it's just a placeholder that performs a similar function between trusted peers.
13:18 < CJP> The effort is mostly due to two different programming paradigms. Amiko Pay is almost completely asynchronous, but the Bitcoind RPC is modeled as function calls. Doing function calls asynchronously is a PITA, so I had to find something convenient for that.
13:19 < roasbeef> ahhh, Amiko Pay in written in Python right? 
13:19 < CJP> yes it is.
13:19 < roasbeef> have you considered using the asyncio lib, to create a coroutine wrapper around the sync calls? 
13:19 < roasbeef> or are you using 2.7? 
13:20 < CJP> I am using 2.7.
13:20 < roasbeef> ahh gotcha
13:20 < roasbeef> hmm I think it's been back-ported under tulip (the old name before asyncio)?
13:22 < CJP> Coroutines are not a complete solution, since they still depart from the core design of Amiko Pay. In-between event handling, Amiko Pay stores the entire state to disk, so it can be restored. Coroutine state is not easily stored to disk.
13:23 < CJP> In this case(!!), however, function calls always return before Amiko Pay attempts to store the state again, so, in this case(!!) it would be OK to temporarily have active coroutines.
13:26 < CJP> In fact, I am also exploiting that special case in the current solution. It may be better to explicitly relax the idea that "the entire node state must always be serializable", and define WHEN it has to be serializable.
13:28 < roasbeef> hmm, I see, so this is all part of some checkpointing system you have in place? 
13:29 < CJP> Yes, exactly.
13:30 < CJP> The general idea is: message received -> process message -> no exceptions: store state and emit messages. message received -> process message -> exceptions: go back to previous state, log the exception and maybe return an error message.
13:30 < roasbeef> gotcha, I haven't yet implemented such a system for lnd, but there're some TODO's in the code concerning it 
13:31 < roasbeef> though we don't have the bitcoind RPC problem since we embed our own wallet 
13:32 < roasbeef> although btcrpcclient does have async wrappers around the sync calls (they just spawn a goroutine which sends on a resp channel once the request returns) 
13:32 -!- rusty2 [~rusty@pdpc/supporter/bronze/rusty] has quit [Ping timeout: 276 seconds]
13:33 < CJP> With your own wallet, how do you detect incoming "normal" Bitcoin transactions? Do you have some asynchronous code that listens on the Bitcoin P2P network?
13:35 < roasbeef> atm it uses btcwallet which creates a websockets connection to a local btcd node, it recieves notifications over this websockets connect 
13:35 < roasbeef> go's channels make async stuff like this relatively straightforward, with the added bonus that the code reads as if it was syncorhnous 
13:36 -!- binns [sid105317@gateway/web/irccloud.com/x-pminevnysfizedxs] has quit []
13:36 < roasbeef> but tadge is currently writing an SPV lib, which will be integrated once it's ready 
13:37 < roasbeef> ultimately, for the wallet there'll be an interface set up, so users have an option (bitcoin core rpc wallet, our spv, btcwallet, etc) 
13:40 < roasbeef> the notifications stuff is also abstracted via an interface, so you'll be able to swap in say Core's ZeroMQ notifications instead of polling RPC (the current concrete notification implementation uses the websockets connection) 
13:41 < CJP> OK, cool. Having a convenient async interface to Bitcoin sounds really useful.
13:46 -!- CJP [~CJP@a83-163-77-195.adsl.xs4all.nl] has quit [Quit: Ik ga weg]
13:57 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Ping timeout: 250 seconds]
14:09 -!- drnet [~drnett@62-46-154-124.adsl.highway.telekom.at] has quit [Quit: Leaving]
14:14 -!- PaulCapestany [~PaulCapes@204.28.124.82] has quit [Quit: .]
14:14 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #lightning-dev
14:26 -!- zzyzx [~chatzilla@97-90-24-187.dhcp.mtpk.ca.charter.com] has joined #lightning-dev
14:26 -!- zzyzx is now known as Guest11957
14:30 -!- roidster [~chatzilla@97-90-24-187.dhcp.mtpk.ca.charter.com] has quit [Ping timeout: 276 seconds]
14:30 -!- Guest11957 is now known as roidster
14:30 -!- roidster is now known as Guest90250
14:37 -!- laurentmt [~Thunderbi@128-79-141-196.hfc.dyn.abo.bbox.fr] has quit [Quit: laurentmt]
15:01 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has joined #lightning-dev
15:01 -!- Guest90250 is now known as roidster
16:30 -!- rusty [~rusty@pdpc/supporter/bronze/rusty] has joined #lightning-dev
16:45 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Ping timeout: 250 seconds]
16:50  * rusty wakes up, backscrolls.... thanks roasbeef!
16:53 < rusty> roasbeef: OK, so assume chachapoly1305, AFAICT cryptobox makes it easiest to implement {length}{mac}{packet}{mac}.
16:53 < rusty> roasbeef: using the same key.  We pay a 16-byte penalty for obscuring the length, basically.
16:55 < rusty> roasbeef: BTW, is one of you reading the lightning-dev laundry-list thread?
17:04 < roasbeef> so i checked with a crypto expert (asked me crypto professor after class) and he agrees simply encrypting the length with a diff key foils the attack
17:05 < roasbeef> rusty: yes i have an unfinished reponse drafted, took a detour to check the literature on the packet input handling w/ encrypted length problem
17:06 < rusty> roasbeef: so you encrypt the length with key2, but roll it into the single HMAC tag?  Most simple crypto APIs won't net you do that, I think.
17:06 < rusty> s/net/let/
17:42 < roasbeef> rusty: yep encrypt with the second key, then wrap it up into the single HMAC tag
17:44 < rusty> roasbeef: Yeah, I thougth so.  An unencrypted length lends itself nicely to "just use cryptobox", as does "length + hmac".  This is more optimal, but I fear when people have to use lower-level crypto APIs....
17:44 < roasbeef> hmm yeah I think that might be the case for some APIs, fwiw the lib we're using implements chacha by itself, then composes it with a poly3015 implementation 
17:46 < rusty> roasbeef: mind you, I put the cleartext lenght in the hmac in my current construction...
17:47 < rusty> roasbeef: but that's partially because nothing in openssl is easy to use, so I was having to assemble routines out of matchsticks and snot anyway...
17:47 -!- jtimon [~quassel@126.31.134.37.dynamic.jazztel.es] has quit [Ping timeout: 265 seconds]
18:03 -!- rrrandom [~user@90.211.150.189] has joined #lightning-dev
18:03 -!- rrrandom [~user@90.211.150.189] has quit [Remote host closed the connection]
18:05 -!- belcher [~user@unaffiliated/belcher] has quit [Ping timeout: 240 seconds]
18:11 < roasbeef> rusty: having the cleartext in the mac is fine, there's no issue there
18:12 < roasbeef> rusty: hahah :p
18:48 -!- binns [~binns@21/bitcoin/binns] has joined #lightning-dev
18:57 -!- binns [~binns@21/bitcoin/binns] has quit [Quit: leaving]
19:25 -!- binns [~binns@21/bitcoin/binns] has joined #lightning-dev
19:31 -!- bityogi [~textual@208-104-143-200.brvd.dsl.dyn.comporium.net] has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
20:13 -!- binns [~binns@21/bitcoin/binns] has quit [Quit: brb]
20:15 -!- binns [~wb@21/bitcoin/binns] has joined #lightning-dev
20:40 < aj> rusty: [see mail to lightning-dev re: understanding shachain. here for the moment if you wanted to chat about it]
20:41 < rusty> aj: yes... you're agreeing with me.  I said a 1x1... 64-dimensional hypercube is a binary tree.
20:43 < aj> rusty: yeah but you don't need to /tell/ anyone it's a hypercube, it's only the tree that matters
20:43 < rusty> aj: yes.
20:43 < aj> rusty: (pretty sure the 1x1x1.. is redundant too, but anyway)
20:44 < rusty> aj: the hypercube construction was the scaffolding used to build the idea, it's not useful once it's constructed :)
20:46 < rusty> aj: hmm, I think the 64-deep merkle construction is slightly simpler though.
20:47 < rusty> aj: though less efficient if implemented naively.
20:49 < aj> rusty: the height calculations for the elkrem construction seemed confusing to me; it doesn't map to binary nicely, and i couldn't work out how to map it to anything particularly nicely
20:50 < aj> rusty: the only difference between elkrem and shachain to me is that elkrem is a binary tree, while shachain's has up to 64 branches from a node
20:51 < rusty> aj: val(N) { ret=seed; for (i = 63; i >= 0; i--) { if (N & (1 << i)) ret = SHA(ret || 1); else ret = SHA(ret || 0) };  return ret; }
20:51 < rusty> That's pretty trivial to produce.
20:52 < aj> erm, that's not right though is it?
20:52 < rusty> aj: AFAICT it's a valid variant.
20:54 < rusty> aj: in practice, of course, you need to start handing out upper nodes, so the simplicity of this breaks down.
20:55 < aj> rusty: every value returned from that involves 64 hash operations, so you need 2**64 storage to keep track of old hashes...
20:55 < rusty> aj: no, that's a naive implementation.
20:56 < rusty> aj: the generator can trivially cache upper nodes.  Of course, they ahve to hand the recipient those upper nodes.
20:56 < aj> rusty: no, the recipient needs to store the hashes; it can't derive an older hash from a newer one with that val() function
20:57 < rusty> aj: so instead of actually handing val(1) you hand the parent of val(0) and val(1).
20:58 < rusty> aj: as I said, it's trivial for the generator.  Just non-trivial for the recipient.
20:58 < aj> rusty: yes, you use an entirely different function, that's harder to work out :)
20:58 < rusty> aj: true, I don't think it's very useful, but it *is* easy to understand.
20:59 < aj> rusty: if you had 2**63 values, those would be the elkrem leaf nodes; but it's getting them in the right order with the parents so that the recipient only has to use log(2**64) storage that's "messy"
21:00 < rusty> aj: yes
21:31 < aj> rusty: (oh, that mail is waiting moderator approval, apparently the picture was too big)
21:32 < rusty> aj: thanks, done
22:22 -!- rusty [~rusty@pdpc/supporter/bronze/rusty] has quit [Ping timeout: 250 seconds]