--- Log opened Thu Mar 02 00:00:58 2023
00:19 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
00:20 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
00:36 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Read error: Connection reset by peer]
00:36 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
01:00 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
01:00 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
02:54 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
02:54 -!- opsec_x122 [~opsec_x12@44.25.143.46] has joined #c-lightning
02:59 -!- opsec_x122 [~opsec_x12@44.25.143.46] has quit [Read error: Connection reset by peer]
03:44 -!- denis2342 [~denis@p549d86f9.dip0.t-ipconnect.de] has joined #c-lightning
03:50 < denis2342> is the autoclean plugin really getting every failed forward, encoding it to json and then checking if the date is old enough to delete it. and then creating a sql query with several thousand parameter entries to delete them all instead of a clean “delete from forwards where state = failed && entry > $date” (pseudocode)
03:51 < denis2342> because I see the autoclean process is taking 100% and is talking to nobody but it is using several gigabytes of ram.
03:52 < denis2342> this looks really wrong to me
03:52 < vincenzopalazzo> denis2342: it is the first run of the autoclean?
03:52 < denis2342> vincenzopalazzo: yes
03:53 < denis2342> no, it was
03:53 < denis2342> in the meantime it is the third run, because I wanted to test it first
03:53 < denis2342> every run needs 90min 
03:53 < denis2342> that was the first one: lightning-cli autoclean-once failedforwards 41472000
03:53 < vincenzopalazzo> I think this deserve a github issue with a good description?
03:54 < vincenzopalazzo> if this appens also after the first run, yes there is something wrong
03:54 < denis2342> for a good description I need more infos and thats why Iam here! ;)
04:00 < denis2342> the way I see it is that failed forwards are in the forwards table with a state of 3 and an received_time entry
04:00 < denis2342> and with autoclean I can delete all entries older than x seconds
04:00 < denis2342> sql would do this in seconds
04:01 < denis2342> milliseconds mostly
04:01 < denis2342> I admit that I have 800000 failed entries
04:16 < denis2342> now it looks like autoclean gave postgresql the 5GB query ;)
04:17 < denis2342> maybe not 5GB, autoclean needs that for the complete data, postgresql only needs the id to delete it, but LOTS of ids
04:22 < denis2342> it cleaned 226581 entries from 799343 in about 90min, taking 9GB of ram
05:13 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
05:25 -!- opsec_x122 [~opsec_x12@44.25.143.46] has joined #c-lightning
05:25 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
05:54 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
05:57 -!- opsec_x122 [~opsec_x12@44.25.143.46] has quit [Ping timeout: 248 seconds]
06:12 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
06:12 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
06:47 < denis2342> also it seems that all the entries are deleted in their own transaction which needs to sync on disk, which takes ages
06:59 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Read error: Connection reset by peer]
07:00 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
07:01 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
07:01 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
07:20 -!- instagibbs [~instagibb@pool-100-15-126-231.washdc.fios.verizon.net] has quit [Quit: Ping timeout (120 seconds)]
07:20 -!- instagibbs [~instagibb@pool-100-15-126-231.washdc.fios.verizon.net] has joined #c-lightning
07:27 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
07:27 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
08:04 -!- ghost43 [~ghost43@gateway/tor-sasl/ghost43] has quit [Remote host closed the connection]
08:05 -!- ghost43 [~ghost43@gateway/tor-sasl/ghost43] has joined #c-lightning
08:18 -!- opsec_x122 [~opsec_x12@44.25.143.46] has joined #c-lightning
08:18 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
08:21 -!- opsec_x122 [~opsec_x12@44.25.143.46] has quit [Read error: Connection reset by peer]
08:21 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
08:22 -!- jarthur [~jarthur@user/jarthur] has joined #c-lightning
08:27 -!- denis2342 [~denis@p549d86f9.dip0.t-ipconnect.de] has quit [Quit: denis2342]
10:04 < vincenzopalazzo> I think you had already all the info to open a issue with a minimum accettable description?
10:06 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
10:06 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
10:10 -!- opsec_x122 [~opsec_x12@44.25.143.46] has joined #c-lightning
10:11 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
10:17 < cdecker[m]> There is a facility to batch changes in a single tx, that was introduced exactly for this case. But yeah, some form of chunking is likely a good idea
10:32 -!- PhyrexianNFM [~Phyrexian@104.202.159.143.dyn.plus.net] has joined #c-lightning
10:34 < PhyrexianNFM> Hi, I was wondering, what is the reason c-lightning does not support Windows? What are the main problems that would have to be overcome in order to add support?
11:06 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
11:06 -!- opsec_x122 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
11:35 -!- PhyrexianNFM [~Phyrexian@104.202.159.143.dyn.plus.net] has quit [Quit: Client closed]
11:49 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Read error: Connection reset by peer]
11:49 -!- opsec_x122 [~opsec_x12@44.25.143.46] has joined #c-lightning
11:52 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
11:53 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
11:54 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
11:55 -!- Guest6 [~Guest6@87.68.218.231.adsl.012.net.il] has joined #c-lightning
11:55 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
11:55 -!- opsec_x12 [~opsec_x12@44.25.143.46] has joined #c-lightning
11:56 -!- opsec_x122 [~opsec_x12@44.25.143.46] has quit [Ping timeout: 255 seconds]
11:58 -!- opsec_x122 [~opsec_x12@44.25.143.46] has joined #c-lightning
11:58 -!- instagibbs [~instagibb@pool-100-15-126-231.washdc.fios.verizon.net] has quit [Quit: Ping timeout (120 seconds)]
11:59 -!- instagibbs [~instagibb@pool-100-15-126-231.washdc.fios.verizon.net] has joined #c-lightning
11:59 < Guest6> any good ideas how to backup the SQLite DB, on a remote encrypted storage? I tried to use Keybase solution (KBFS), but it's not stable enough and CLN is crashing every now and than (maybe it shouldn't crash if backup is failing?), and KBFS can use a lot of resources (memory leak?). Any other stable solutions?
11:59 -!- opsec_x122 [~opsec_x12@44.25.143.46] has quit [Remote host closed the connection]
11:59 -!- opsec_x122 [~opsec_x12@44.25.143.46] has joined #c-lightning
12:01 -!- opsec_x12 [~opsec_x12@44.25.143.46] has quit [Ping timeout: 248 seconds]
12:17 -!- jonatack [~jonatack@user/jonatack] has quit [Quit: WeeChat 3.8]
12:51 -!- jonatack [~jonatack@user/jonatack] has joined #c-lightning
14:03 -!- pakaro [~quassel@142.243.254.224] has joined #c-lightning
14:50 -!- Guest6 [~Guest6@87.68.218.231.adsl.012.net.il] has quit [Quit: Client closed]
14:51 -!- pakaro [~quassel@142.243.254.224] has quit [Ping timeout: 268 seconds]
15:35 -!- Guest6 [~Guest6@87.68.218.231.adsl.012.net.il] has joined #c-lightning
15:35 -!- Guest6 [~Guest6@87.68.218.231.adsl.012.net.il] has quit [Client Quit]
16:12 -!- instagibbs1 [~instagibb@pool-100-15-126-231.washdc.fios.verizon.net] has joined #c-lightning
16:14 -!- instagibbs [~instagibb@pool-100-15-126-231.washdc.fios.verizon.net] has quit [Ping timeout: 268 seconds]
16:14 -!- instagibbs1 is now known as instagibbs
16:25 -!- cryptapus [~cryptapus@user/cryptapus] has quit [Read error: Connection reset by peer]
17:46 -!- cryptapus [~cryptapus@user/cryptapus] has joined #c-lightning
17:58 -!- ghost43 [~ghost43@gateway/tor-sasl/ghost43] has quit [Quit: Leaving]
17:58 -!- ghost43 [~ghost43@gateway/tor-sasl/ghost43] has joined #c-lightning
18:24 -!- jarthur [~jarthur@user/jarthur] has quit [Ping timeout: 248 seconds]
18:27 -!- jarthur [~jarthur@user/jarthur] has joined #c-lightning
20:11 -!- jon_atack [~jonatack@user/jonatack] has joined #c-lightning
20:14 -!- jonatack [~jonatack@user/jonatack] has quit [Ping timeout: 248 seconds]
22:55 -!- Netsplit *.net <-> *.split quits: jarthur, opsec_x122, roasbeef, warren
22:56 -!- Netsplit over, joins: warren, jarthur, roasbeef, opsec_x122
23:16 -!- jarthur [~jarthur@user/jarthur] has quit [Ping timeout: 248 seconds]
23:19 -!- jarthur [~jarthur@user/jarthur] has joined #c-lightning
--- Log closed Fri Mar 03 00:00:59 2023