--- Log opened Thu Sep 03 00:00:07 2020
00:39 < m-schmoock> cdecker: so, the reason the chain openchannel_hook tests are ustable is, because the parameter order of "--plugin=..." does not define the hook chain order. They are not parallel but the execution order is undefined
00:40 < m-schmoock> this makes writing tests for a chained hook messy
00:40 < m-schmoock> ideas?
00:58 <@cdecker> Is it the testing framework mixing the order, or is it internal to `lightningd`?
00:58 < m-schmoock> From what I can see in the logs its the internal
00:59 < m-schmoock> I wrote a test that provocates that behaviour 
00:59 -!- mrostecki [~mrostecki@gateway/tor-sasl/mrostecki] has joined #c-lightning
01:00 <@cdecker> Interesting, that should not happen I think. At least not for different argument names (otherwise things like `--disable-plugin` would not work).
01:00 < m-schmoock> The logs show the correct plugin order at the top
01:00 <@cdecker> Might be that arguments with the same key are mixed
01:00 < m-schmoock> might it be relates to the actual execution speed of init/getmanifest methods
01:02 < m-schmoock> becuase my tests work most of the time and does not seem to a functional or valgrind troubles anymore Im sure the change is done. But I like the test to be stabel ;)
01:02 < m-schmoock> maybe I add a sleep(2) to the second plugin init method and that helps enforcing the chained hook order somehow :D
01:08 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Ping timeout: 260 seconds]
01:20 < m-schmoock> update: adding sleep to init did not help. at which point in time are plugin hooks registerd?
01:54 -!- jonatack_ [~jon@192.113.14.109.rev.sfr.net] has joined #c-lightning
01:59 -!- jonatack_ [~jon@192.113.14.109.rev.sfr.net] has quit [Ping timeout: 264 seconds]
01:59 -!- jonatack_ [~jon@213.152.162.10] has joined #c-lightning
02:09 < m-schmoock> cdecker: okay, im sure the plugin order is not messed up by pyln plugin testfrwamework. I wrote a test that specifically checks that. still the hook order is still undefined
02:11 <@cdecker> The hooks are registered as a result of the `getmanifest` call, which is handled internally in the `pyln-client` library
02:15 < m-schmoock> hm... so if I delay the getmanifest the tests should be stable (not a working solution)
02:20 < darosior> The getmanifest call are async fwiw
02:20 < darosior> So maybe the hook ordered is defined as "the order in which plugins are registered" ?
02:20 < darosior> Which is kinda random
02:21 < m-schmoock> question is: do we want a defined order or not?
02:21 < m-schmoock> I can maybe make my tests stable for both situations
02:21 < m-schmoock> but I assume having a defined order maybe helpful
02:22 < darosior> Don't know, to be reliable this incurs starting plugins synchronously which would significantly (not *that much*, but still if you have a lot of plugins) increase startup time
02:23 < darosior> But i did not follow the overall discussions around openchannel hook so i can't really opinion here :-)
02:25 < m-schmoock> thats one aspect. but on the other hand having a defined order can be helpful where shipped base plugins get called first 
02:33 < darosior> Yes, and maybe also one day for /*hides*/ interdependencies 
02:33 < m-schmoock> I'll workaround my unstable test around that fact for now
02:50 <@cdecker> Well, we can keep the async startup of plugins, just need to associate each getmanifest result with a priority, and process them in that order
02:51 <@cdecker> Or we can have each entry in the hook registry have an associated priority and just sort the after startup of all plugins completes
02:51 <@cdecker> We already have two phases that are separated by a sync barrier: pre-init and post-init
02:52 <@cdecker> So that'd be a good place to order the entries I guess
02:54 <@cdecker> I'll open an issue to have a permanent place to discuss how we can address this in future
02:54 < darosior> But then a misfunctionning could block the startup ?
02:54 < darosior> today if a plugin does not answer init it times out, but if we rely on it for some order don't we require it ?
02:55 < m-schmoock> I think the most transparent way woudl be to enforece plugin order as given via arguments. not sure if thats applicable
02:55 < m-schmoock> having priorities for various hooks makes debugging a mess
02:55 <@cdecker> Nope, if it times out on the getmanifest we drop it, init is just to inform the plugin it's good to talk to the RPC, we don't actually need the response
02:56 <@cdecker> I mean we assign a priority based on their position in the command line for now (we can discuss whether we'd want to be able to have plugins specify a prio, or add syntax to the command line to specofy priority later on)
02:56 <@cdecker> This is just to make the ordering deterministic for now
02:57 -!- vincenzopalazzo [~vincent@host-79-23-116-18.retail.telecomitalia.it] has quit [Quit: Leaving]
02:57 < m-schmoock> ahhh, that sounds good
02:57 < m-schmoock> didnt get that first
02:58 < m-schmoock> cdecker: also one questsion: should I prevent calling a second plugin hook if the first already rejected (for informal reasons). Or should I code it in a way that the first reject closes the loop
02:59 <@cdecker> The way I usually think about the hook chaining is that anything that doesn't return `{"result": "continue"}` is a final decision and should break out of the chain
03:00 <@cdecker> This was good and simple enough for all hooks so far, though it might not work for all hooks, so we need to determine that on a case-by-case basis
03:00 < m-schmoock> okay, so I need to update my PR on that. currently it 'asks' all plugins but goes for the first reject
03:01 < m-schmoock> the C code is not very suportive on that as when we return false on a deserialize, the final does not get called :D 
03:07 <@cdecker> Ok, filed https://github.com/ElementsProject/lightning/issues/4005 to track this including link to this discussion
03:10 < m-schmoock> thx
03:46 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zzgbwiizjmpgirfq] has left #c-lightning []
03:46 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zzgbwiizjmpgirfq] has joined #c-lightning
03:58 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Ping timeout: 240 seconds]
04:01 -!- vasild [~vd@gateway/tor-sasl/vasild] has joined #c-lightning
04:05 -!- jasan [~jasan@tunnel509499-pt.tunnel.tserv27.prg1.ipv6.he.net] has quit [Quit: WeeChat 2.7.1]
04:27 -!- jonatack_ [~jon@213.152.162.10] has quit [Ping timeout: 260 seconds]
05:10 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #c-lightning
05:12 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 258 seconds]
05:30 < m-schmoock> cdecker: this is now ready for review from my side: https://github.com/ElementsProject/lightning/pull/3960
05:30 <@cdecker> Will review asap
05:30 < m-schmoock> thx :D
05:37 -!- shesek [~shesek@unaffiliated/shesek] has joined #c-lightning
06:53 -!- mdunnio [~mdunnio@208.59.170.5] has joined #c-lightning
07:14 -!- jonatack_ [~jon@37.170.79.18] has joined #c-lightning
08:45 -!- jonatack_ [~jon@37.170.79.18] has quit [Ping timeout: 260 seconds]
09:10 -!- jonatack_ [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #c-lightning
09:13 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #c-lightning
09:15 -!- queip [~queip@unaffiliated/rezurus] has quit [Ping timeout: 265 seconds]
09:23 -!- queip [~queip@unaffiliated/rezurus] has joined #c-lightning
09:51 -!- jonatack_ [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Quit: jonatack_]
09:52 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #c-lightning
10:53 -!- Victor_sueca is now known as Victorsueca
10:58 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 256 seconds]
11:02 -!- liberliver [~Thunderbi@2a01:c22:ac7b:6b00:a2af:bdff:fe37:2d32] has quit [Ping timeout: 272 seconds]
11:35 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #c-lightning
11:36 -!- alko89 [~alko89@unaffiliated/alko89] has quit [Quit: ZNC 1.7.5 - https://znc.in]
11:38 -!- alko89 [~alko89@unaffiliated/alko89] has joined #c-lightning
12:11 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zzgbwiizjmpgirfq] has left #c-lightning []
12:11 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zzgbwiizjmpgirfq] has joined #c-lightning
12:14 -!- vincenzopalazzo [~vincent@host-79-23-116-18.retail.telecomitalia.it] has joined #c-lightning
12:36 -!- kristapsk [~KK@gateway/tor-sasl/kristapsk] has joined #c-lightning
12:41 -!- belcher_ [~belcher@unaffiliated/belcher] has joined #c-lightning
12:44 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 240 seconds]
14:48 -!- instagibbs [~instagibb@pool-71-178-191-230.washdc.fios.verizon.net] has quit [Ping timeout: 264 seconds]
14:51 -!- instagibbs [~instagibb@pool-71-178-191-230.washdc.fios.verizon.net] has joined #c-lightning
15:13 -!- mdunnio [~mdunnio@208.59.170.5] has quit [Remote host closed the connection]
15:37 -!- mrostecki [~mrostecki@gateway/tor-sasl/mrostecki] has quit [Remote host closed the connection]
15:59 -!- vasild [~vd@gateway/tor-sasl/vasild] has quit [Ping timeout: 240 seconds]
16:00 -!- vasild [~vd@gateway/tor-sasl/vasild] has joined #c-lightning
16:03 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has joined #c-lightning
16:06 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 260 seconds]
18:34 -!- Victorsueca [~Victorsue@unaffiliated/victorsueca] has joined #c-lightning
18:36 -!- Victor_sueca [~Victorsue@unaffiliated/victorsueca] has quit [Ping timeout: 260 seconds]
18:57 -!- mdunnio [~mdunnio@208.59.170.5] has joined #c-lightning
19:02 -!- mdunnio [~mdunnio@208.59.170.5] has quit [Ping timeout: 265 seconds]
19:37 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has quit [Ping timeout: 264 seconds]
19:45 -!- justanotheruser [~justanoth@unaffiliated/justanotheruser] has joined #c-lightning
20:33 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zzgbwiizjmpgirfq] has left #c-lightning []
20:33 -!- blockstream_bot [blockstrea@gateway/shell/sameroom/x-zzgbwiizjmpgirfq] has joined #c-lightning
21:13 -!- shesek [~shesek@unaffiliated/shesek] has quit [Remote host closed the connection]
21:49 -!- jb55 [~jb55@gateway/tor-sasl/jb55] has quit [Ping timeout: 240 seconds]
21:49 -!- jb551 [~jb55@gateway/tor-sasl/jb55] has joined #c-lightning
22:04 -!- shesek [~shesek@unaffiliated/shesek] has joined #c-lightning
22:24 -!- jb551 [~jb55@gateway/tor-sasl/jb55] has quit [Remote host closed the connection]
22:25 -!- jb551 [~jb55@gateway/tor-sasl/jb55] has joined #c-lightning
23:07 -!- mdunnio [~mdunnio@208.59.170.5] has joined #c-lightning
23:11 -!- mdunnio [~mdunnio@208.59.170.5] has quit [Ping timeout: 240 seconds]
23:32 -!- asoltys [~root@s207-81-214-2.bc.hsia.telus.net] has joined #c-lightning
--- Log closed Fri Sep 04 00:00:08 2020