--- Log opened Mon Feb 03 00:00:28 2020
02:34 -!- PaulTroon [~paultroon@h-5-150-248-150.NA.cust.bahnhof.se] has joined #c-lightning
02:40 -!- cryptoso- [~cryptosoa@gateway/tor-sasl/cryptosoap] has joined #c-lightning
02:41 -!- cryptosoap [~cryptosoa@gateway/tor-sasl/cryptosoap] has quit [Ping timeout: 240 seconds]
02:42 -!- zmnscpxj_ [~zmnscpxj@gateway/tor-sasl/zmnscpxj] has joined #c-lightning
02:45 -!- zmnscpxj [~zmnscpxj@gateway/tor-sasl/zmnscpxj] has quit [Ping timeout: 240 seconds]
02:46 -!- masterdonx2 [~masterdon@45.248.79.86] has quit [Ping timeout: 268 seconds]
02:46 -!- MasterdonX [~masterdon@152.89.160.122] has joined #c-lightning
02:54 -!- PaulTroon [~paultroon@h-5-150-248-150.NA.cust.bahnhof.se] has quit [Remote host closed the connection]
03:17 -!- dr-orlovsky [~dr-orlovs@77-58-192-184.dclient.hispeed.ch] has quit [Quit: My MacBook has gone to sleep. ZZZzzz...]
03:18 -!- kristapsk_ [~KK@gateway/tor-sasl/kristapsk] has joined #c-lightning
03:21 -!- kristapsk [~KK@gateway/tor-sasl/kristapsk] has quit [Ping timeout: 240 seconds]
03:24 -!- kristapsk_ [~KK@gateway/tor-sasl/kristapsk] has quit [Remote host closed the connection]
03:36 -!- PaulTroon [~paultroon@h-5-150-248-150.NA.cust.bahnhof.se] has joined #c-lightning
03:50 <@cdecker> Hm, you can always track non-public channels in a plugin and then use them to compute routes in the plugin (Dijkstra is not that hard), or use that info to fill gaps that `getroute` can't bridge (compute A->B, append B->C which is not public and then compute C->D using getroute again)
03:51 -!- belcher [~belcher@unaffiliated/belcher] has joined #c-lightning
04:46 < fiatjaf> https://github.com/ElementsProject/lightning/issues/3482#issuecomment-581388897
04:46 < fiatjaf> you were quicker this time
05:23 <@cdecker> Hehe
05:41 -!- bitdex [~bitdex@gateway/tor-sasl/bitdex] has quit [Quit: = ""]
06:01 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zkjjaoqsjfattawp] has left #c-lightning []
06:01 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zkjjaoqsjfattawp] has joined #c-lightning
06:06 -!- dr-orlovsky [~dr-orlovs@194.230.147.144] has joined #c-lightning
06:18 -!- vincenzopalazzo [~vincent@mob-109-113-241-127.net.vodafone.it] has joined #c-lightning
06:49 -!- vincenzopalazzo [~vincent@mob-109-113-241-127.net.vodafone.it] has quit [Remote host closed the connection]
06:50 -!- dr-orlovsky [~dr-orlovs@194.230.147.144] has quit [Quit: My MacBook has gone to sleep. ZZZzzz...]
06:55 -!- dr-orlovsky [~dr-orlovs@194.230.147.144] has joined #c-lightning
07:09 -!- dr-orlovsky [~dr-orlovs@194.230.147.144] has quit [Quit: My MacBook has gone to sleep. ZZZzzz...]
09:33 -!- fiatjaf [~fiatjaf@2804:7f2:2996:333e:ea40:f2ff:fe85:d2dc] has quit [Ping timeout: 265 seconds]
09:33 -!- fiatjaf [~fiatjaf@2804:7f2:2996:333e:ea40:f2ff:fe85:d2dc] has joined #c-lightning
10:01 -!- zmnscpxj_ [~zmnscpxj@gateway/tor-sasl/zmnscpxj] has quit [Quit: Leaving]
10:24 -!- Kostenko [~Kostenko@2001:8a0:7293:8f00:c82d:30c0:f65f:ddff] has quit [Ping timeout: 248 seconds]
10:33 <@niftynei> in theory, as long as you figure out a way to inject them into the gossip store they should be used for routing, no?
10:33 <@cdecker> Yes, but it'd require a restart otherwise the store isn't read by `gossipd`
10:34 <@cdecker> And since `gossipd` may be writing to that file concurrently you'd likely have to stop, modify, start
10:34 <@niftynei> tho you'd have to make sure they aren't eligible for being sent to peers as gossip lol
10:35 <@niftynei> or just inject it into the "recevied gossip" pipeline, and let gossipd write it to the file for you 
10:35 <@niftynei> set up a mock peer to connect to who's gossip is just all the private data you want to know :P
10:37 <@niftynei> provided there's a trivial-ish way to get it into the gossip store, i can't remember if there's a private flag on gossip... probably?
10:40 -!- Kostenko [~Kostenko@2001:8a0:7293:1200:68f5:2150:4d47:f16f] has joined #c-lightning
10:41 -!- kristapsk [~KK@gateway/tor-sasl/kristapsk] has joined #c-lightning
10:53 -!- vincenzopalazzo [~vincent@host98-32-dynamic.17-87-r.retail.telecomitalia.it] has joined #c-lightning
11:00 <@cdecker> Yep, there is indeed the concept of a local update (https://github.com/ElementsProject/lightning/blob/master/gossipd/gossip_store.csv) which means we can local_add a channel (this is mainly used for local channels that haven't reached 6 confs yet) and then update using private updates
11:09 -!- rusty [~rusty@pdpc/supporter/bronze/rusty] has joined #c-lightning
11:40 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has joined #c-lightning
11:42 -!- vasild_ [~vd@gateway/tor-sasl/vasild] has quit [Client Quit]
11:43 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Ping timeout: 240 seconds]
11:59 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Read error: Connection reset by peer]
12:06 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #c-lightning
12:35 -!- kristapsk [~KK@gateway/tor-sasl/kristapsk] has quit [Remote host closed the connection]
12:35 -!- kristapsk [~KK@gateway/tor-sasl/kristapsk] has joined #c-lightning
13:14 -!- rusty [~rusty@pdpc/supporter/bronze/rusty] has quit [Quit: Leaving.]
13:16 -!- vincenzopalazzo [~vincent@host98-32-dynamic.17-87-r.retail.telecomitalia.it] has quit [Ping timeout: 260 seconds]
14:25 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zkjjaoqsjfattawp] has left #c-lightning []
14:25 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zkjjaoqsjfattawp] has joined #c-lightning
14:56 -!- dmar198 [491226d9@c-73-18-38-217.hsd1.mi.comcast.net] has joined #c-lightning
14:57 < dmar198> I created a new node today and opened two public channels. Other people can see those channels, and they can see my peers, but they can't find a route to me. Also, if I create an invoice, other people can pay them, but there are route hints. What might cause these issues? Why would I need route hints when my channels are public?
15:00 < fiatjaf> dmar198: I can find a route to you now
15:02 < dmar198> oh
15:03 < dmar198> what command did you run?
15:04 < dmar198> I just ran getroute [my-node-id] 10 10 and I still can't find a route to my main node from my laptop node
15:06 < fiatjaf> you can't find a route for 10msat, but you can for 1000msat
15:07 < dmar198> woot! That worked! This is great fiatjaf, thank you so much. BTW, the default for "getroute" is 10 msats
15:09 < dmar198> actually I'm probably wrong about that
15:09 < dmar198> so nevermind. You've fully resolved my issue, thank you fiatjaf!
15:10 -!- rusty [~rusty@pdpc/supporter/bronze/rusty] has joined #c-lightning
15:32 -!- dmar198 [491226d9@c-73-18-38-217.hsd1.mi.comcast.net] has quit [Quit: dmar198]
16:29 -!- zmnscpxj [~zmnscpxj@gateway/tor-sasl/zmnscpxj] has joined #c-lightning
16:37 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:4517:9679:60e5:9978] has joined #c-lightning
16:37 -!- michaelfolkson [~textual@2a00:23c5:be04:e501:4517:9679:60e5:9978] has quit [Client Quit]
16:45 -!- DeanWeen [~dean@gateway/tor-sasl/deanguss] has quit [Remote host closed the connection]
16:45 -!- DeanWeen [~dean@gateway/tor-sasl/deanguss] has joined #c-lightning
17:17 < fiatjaf> rusty, zmnscpxj: every time I use 'pay' and try to read the code I come to different conclusions, but I don't know if I'm too volatile or it is changing all the time
17:17 < zmnscpxj> it is changing all the time, because we have no idea what the best `pay` algorithm is
17:17 < fiatjaf> can you tell me if it will wait forever until a payment fails completely or if it will exit before that
17:18 < zmnscpxj> for a payment attempt, it will wait for a failure to be reported back
17:18 < zmnscpxj> every forwarding node wants to report failure ASAP so they can free up their channel capacities for non-failing payment attempts
17:18 < fiatjaf> I wrote my own 'pay' implementation because 'pay' before would exit and my program would think the payment had failed but in fact there was a payment in flight still
17:18 < zmnscpxj> however, some forwarding nodes go offline and are unable to report failure back
17:18 < fiatjaf> then what happens?
17:18 < zmnscpxj> hmmmm looks like a change LOL
17:19 < zmnscpxj> then some *other* forwarding node, or your own node, realizes that the timelock is getting very near and drops the channel onchain and reports failure back
17:19 < zmnscpxj> `listpays` should tell you what is really happening.
17:19 < zmnscpxj> If it is still `pending` it can still succeed
17:19 < fiatjaf> ok, but if that takes a week?
17:20 < zmnscpxj> then `pay` should wait a week
17:20 < zmnscpxj> this is yet another griefing problem
17:20 < fiatjaf> I think it should be like that
17:20 < fiatjaf> what happens if you close the terminal or something like that while it's waiting?
17:20 < zmnscpxj> I think there was a `pay` bug before where it didn't wait for some cases, after `pay` was moved from `payalgo.c` and into a `pay` plugin
17:21 < zmnscpxj> command continues running and retrying
17:21 < zmnscpxj> `listpays` is what you should depend on
17:21 < fiatjaf> no, please
17:21 < fiatjaf> let me depend only on 'pay'
17:21 < zmnscpxj> then you close the terminal so how do you know what happened to `pay`?
17:22 < fiatjaf> I won't close the terminal, because there won't be a terminal, but a program talking to the lightning-rpc socket
17:22 < zmnscpxj> so your program crashes so how do you know what happened to `pay`?
17:23 < fiatjaf> in that case my program has marked that payment as pending (before calling 'pay') and it will remain like that until manual verification
17:23 < zmnscpxj> yes, and then your program can come back online, at startup it checks `listpays` and sees what happened
17:23 < zmnscpxj> without doing manual verification
17:24 < fiatjaf> no, because imagine if 'pay' is still trying?
17:24 < zmnscpxj> If `pay` is still trying, `listpays` will say `pending`
17:24 < fiatjaf> 'listpays' may return 'failed' but one msec later the pay command dispatches a new 'sendpay'
17:25 < zmnscpxj> You are thinking of `listsendpays` which has that behavior
17:25 < zmnscpxj> `listpays` knows about `pay` working
17:25 < fiatjaf> oh
17:25 < zmnscpxj> it's why `listpays` was addded
17:25 < fiatjaf> that is a good thing
17:25 < zmnscpxj> Because somebody (you?) complained that before about `listsendpays`, so we added `listpays`
17:26 < zmnscpxj> I do not know if we documented it properly though LOL
17:26 < fiatjaf> I know I've complained about this multiple times
17:26 < fiatjaf> now I can be happy
17:26 < fiatjaf> thank you
17:26 < zmnscpxj> Also we had `listpayments` which was an alias of `listsendpays` for even more confusion
17:27 < zmnscpxj> `listpayments` has been excised from the code, and you should use `listpays`, `listsendpays` is a low-level command you should use only if you use `sendpay` directly
17:27 < fiatjaf> wait, what if there's a pending payment and lightningd is restarted?
17:27 < fiatjaf> will 'listpays' know about that?
17:28 < zmnscpxj> If it is a `sendpay` in-flight, then `listpays` will ask `listsendpays` what happened
17:28 < zmnscpxj> If the `listsendpays` reports back that it has failed, and `listpays` knows that there is no `pay` command currently working on that payment hash, it reports failure
17:29 < fiatjaf> right
17:29 < zmnscpxj> So basically, what `listpays` does is ask `listsendpays`.  If `listsendpays` says it is failed, it checks the local `pay` implementation to see if it is still working and amends it to `pending` if there is one
17:29 < zmnscpxj> If `lightningd` restarts then the local `pay` implementation has stopped working so `listpays` will pass-through `listsendpays` for the query
17:30 < zmnscpxj> So if the `listsendpays` says it is still pending then `listpays` also says it is still pending
17:30 < fiatjaf> right, I get it
17:30 < fiatjaf> this is great
17:30 < zmnscpxj> ok ^^
17:31 < zmnscpxj> I think it was added last year
17:31 < zmnscpxj> maybe middle of 2019
17:37 < zmnscpxj> fiatjaf: relevant code if you are curious: https://github.com/ElementsProject/lightning/blob/55173a56b7f3626bdcd3934b4f58d50f74ccecb0/plugins/pay.c#L1627-L1634
17:38 < zmnscpxj> This is the branch where `listsendpays` reports it as `failed`, we upgrade it to `pending` if we know a `pay` algo is running for that payment
17:39 < fiatjaf> I'll now proceed to trash my pay implementation
17:39 < zmnscpxj> Yes, pay algorithms are hard because of all the weird edge cases
17:40 < fiatjaf> what about MPP?
17:40 < fiatjaf> maybe I'll try to write an MPP pay implementation now that I'm free from the burden of my previous thing
17:41 < zmnscpxj> No idea how best to implement it, though see the c-lightning list thread from Rene Pickhardt recently
17:41 < zmnscpxj> I suggested reusing his balancing algorithm to guide the initial split of the MPP
17:41 < zmnscpxj> But Jit Rebalance > MPP ............. bleah
17:42 < fiatjaf> you don't like MPP?
17:42 < zmnscpxj> There is no clear heuristic how to split payments (other than what I suggested)
17:42 < zmnscpxj> Whereas JIT rebalance has a clear heuristic
17:42 < zmnscpxj> "if you cannot forward, rebalance"
17:43 < zmnscpxj> MPP is useful if you know the balance of every channel along the way, which we do not want to know because privacy
17:43 < zmnscpxj> So JIT rebalance is better in this case because the forwarding node knows its own channel balances and can perform actions
17:43 < zmnscpxj> based on that knowledge
17:43 < zmnscpxj> without leaking it to anybody else
17:45 < zmnscpxj> Then the issue is also how to pilot the MPP.
17:45 < zmnscpxj> Because you do not get an indication "oh this part reached the payee"
17:45 < zmnscpxj> You only know "this part is still pending, the other part failed".
17:46 < zmnscpxj> Do you try to wait for the pending part to fail as well so you can maybe divide the payments in a different way?
17:46 < zmnscpxj> Or do you retry the failing part without trying to change how the payment is split?
17:46 < zmnscpxj> lots of little questions with no decent heuristics
17:46 < zmnscpxj> still.... maybe it can be doable with some simple heuristics or something
17:46 < zmnscpxj> hopefully
17:47 < zmnscpxj> Then there is the fee budget --- every split increases the total fees.
17:48 < zmnscpxj> ehhh maybe I am just too negative.  JIT rebalance is trivial compared to MPP but everybody is excited for MPP / AMP
17:51 < fiatjaf> because MPP in theory can make a big payment fit in a bunch of small channels and people like to have a ton of small channels instead of one big
17:51 < fiatjaf> I think
17:51 < fiatjaf> but JIT rebalance can be implemented as a plugin by anyone who wants it, right? or does it require protocol changes?
17:51 < zmnscpxj> JIT rebalance requires you to have just *one* channel big enough to pay the large amount
17:52 < zmnscpxj> and yes, there is already a plugin I think
17:52 < zmnscpxj> lemme dig it up
17:52 < zmnscpxj> https://github.com/lightningd/plugins/tree/master/jitrebalance
17:53 < zmnscpxj> Though it only rebalances for forwards, not for your own payments.
17:53 < fiatjaf> is there anyone using the jitrebalance plugin?
17:54 < fiatjaf> my htlc_accepted spots are already taken
17:54 < zmnscpxj> dunno
17:54 < zmnscpxj> yeah that is why we want to implement hook chaining
17:54 < fiatjaf> this implements it for your own payments: https://medium.com/@akumaigorodski/automatic-iterative-rebalancing-e9c6216969aa
17:54 < zmnscpxj> and why I made all hooks have `{'result':'continue'}` as the no-op condition
17:55 < zmnscpxj> With hook chaining you can load up on as many plugins as you want on `htlc_accepted`.
18:13 < fiatjaf> these were wise decisions
18:18 -!- belcher [~belcher@unaffiliated/belcher] has quit [Quit: Leaving]
19:02 -!- bitdex [~bitdex@gateway/tor-sasl/bitdex] has joined #c-lightning
19:45 -!- kristapsk [~KK@gateway/tor-sasl/kristapsk] has quit [Ping timeout: 240 seconds]
20:10 -!- rusty [~rusty@pdpc/supporter/bronze/rusty] has quit [Quit: Leaving.]
20:23 -!- Dean_Guss [~dean@gateway/tor-sasl/deanguss] has joined #c-lightning
20:26 -!- DeanWeen [~dean@gateway/tor-sasl/deanguss] has quit [Ping timeout: 240 seconds]
21:34 -!- justan0theruser is now known as justanotheruser
21:58 -!- zmnscpxj [~zmnscpxj@gateway/tor-sasl/zmnscpxj] has quit [Quit: Leaving]
22:34 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zkjjaoqsjfattawp] has left #c-lightning []
22:35 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zkjjaoqsjfattawp] has joined #c-lightning
23:08 -!- zmnscpxj [~zmnscpxj@gateway/tor-sasl/zmnscpxj] has joined #c-lightning
--- Log closed Tue Feb 04 00:00:29 2020