--- Log opened Thu Dec 13 00:00:50 2018
00:07 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
00:08 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
00:08 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
00:08 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
00:13 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
00:13 -!- Murch [~murch@c-73-223-113-121.hsd1.ca.comcast.net] has quit [Quit: Snoozing.]
00:15 -!- nelsonhb [~Instantbi@49.124.42.114] has quit [Quit: Instantbird 1.5 -- http://www.instantbird.com]
00:49 -!- booyah_ [~bb@193.25.1.157] has joined #bitcoin-core-dev
00:49 -!- booyah [~bb@193.25.1.157] has quit [Read error: Connection reset by peer]
00:57 -!- TX1683 [~TX1683@unaffiliated/tx1683] has quit [Ping timeout: 244 seconds]
00:59 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
00:59 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
00:59 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
01:03 -!- TX1683 [~TX1683@unaffiliated/tx1683] has joined #bitcoin-core-dev
01:08 -!- owowo [~ovovo@unaffiliated/ovovo] has joined #bitcoin-core-dev
01:10 -!- TheRec [~toto@drupal.org/user/146860/view] has quit []
01:35 -!- BGL [eighty@75-149-171-58-Washington.hfc.comcastbusiness.net] has joined #bitcoin-core-dev
02:02 -!- Guyver2 [AdiIRC@guyver2.xs4all.nl] has joined #bitcoin-core-dev
02:04 -!- Sentineo [~Undefined@unaffiliated/sentineo] has quit [Quit: no more]
02:07 -!- TheRec [~toto@84-72-172-174.dclient.hispeed.ch] has joined #bitcoin-core-dev
02:07 -!- TheRec [~toto@84-72-172-174.dclient.hispeed.ch] has quit [Changing host]
02:07 -!- TheRec [~toto@drupal.org/user/146860/view] has joined #bitcoin-core-dev
02:27 -!- promag [~promag@bl6-24-70.dsl.telepac.pt] has joined #bitcoin-core-dev
02:34 -!- timothy [~tredaelli@redhat/timothy] has joined #bitcoin-core-dev
02:35 -!- spinza [~spin@155.93.246.187] has quit [Quit: Coyote finally caught up with me...]
03:05 -!- rh0nj [~rh0nj@136.243.139.96] has quit [Remote host closed the connection]
03:06 -!- rh0nj [~rh0nj@136.243.139.96] has joined #bitcoin-core-dev
03:25 -!- spinza [~spin@155.93.246.187] has joined #bitcoin-core-dev
03:29 -!- Sentineo [~Undefined@unaffiliated/sentineo] has joined #bitcoin-core-dev
03:35 -!- Guyver2_ [~Guyver@2001:985:f3f:1:1d5f:5a7b:8eb6:219] has joined #bitcoin-core-dev
03:37 < wumpus> unknown options in the config file are ignored, unknown command line options give an error, I think that's a good middle ground
03:38 < wumpus> if you explicitly provide -wallet= on *the command line* with a -disablewallet build I think it's fair to error out, you're requesting something it cannot do
03:38 -!- Guyver2 [AdiIRC@guyver2.xs4all.nl] has quit [Ping timeout: 264 seconds]
03:38 < wumpus> while if there's still left-over wallet configuration in the configuration file, well that probably shouldn't trip up too bad
03:42 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev
03:47 < promag> wumpus: I tend to agree with that
03:51 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
03:54 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
03:54 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
03:54 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
04:16 -!- fanquake [~fanquake@unaffiliated/fanquake] has joined #bitcoin-core-dev
04:16 < fanquake> wumpus I think #14741 and #14319 can go in
04:16 < gribble> https://github.com/bitcoin/bitcoin/issues/14741 | doc: Indicate -rpcauth option password hashing alg by dongcarl · Pull Request #14741 · bitcoin/bitcoin · GitHub
04:16 < gribble> https://github.com/bitcoin/bitcoin/issues/14319 | doc: Fix PSBT howto and example parameters by priscoan · Pull Request #14319 · bitcoin/bitcoin · GitHub
04:17 < fanquake> Both trivialish doc changes
04:18 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
04:20 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
04:20 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
04:20 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
04:25 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
04:26 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
04:27 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
04:28 < wumpus> fanquake: looks like it!
04:28 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
04:29 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 246 seconds]
04:43 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has quit [Ping timeout: 264 seconds]
04:46 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has joined #bitcoin-core-dev
04:52 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has quit [Ping timeout: 260 seconds]
04:55 -!- miknotauro [~miknotaur@201.114.143.145] has quit [Ping timeout: 250 seconds]
04:58 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev
05:00 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has joined #bitcoin-core-dev
05:03 < fanquake> Anyone found any issues testing rc1?
05:03 < wumpus> haven't seen any issues reported ye
05:15 < wumpus> not sure we've given it enough time yet, though
05:19 < wumpus> though rc period for backport releases needs to be less than for major releases
05:20 -!- promag [~promag@bl6-24-70.dsl.telepac.pt] has quit [Remote host closed the connection]
05:20 < fanquake> fair enough
05:20 < fanquake> was just wondering, because there's at least one more fix, plus the PSBT documentation I'm about to backport we could get into .1
05:21 < fanquake> Otherwise we'll just wait for .2
05:21 < wumpus> but could discuss that at the meeting, along with whether to drop vista support in 0.18.0
05:21 < wumpus> right
05:22 < fanquake> I was actually considering getting up for the meeting this morning. Had a few things to bring up
05:24 < fanquake> Also qt 5.12 for 0.18.0
05:26 -!- Guyver2__ [~Guyver@2001:985:f3f:1:1d5f:5a7b:8eb6:219] has joined #bitcoin-core-dev
05:28 -!- Guyver2_ [~Guyver@2001:985:f3f:1:1d5f:5a7b:8eb6:219] has quit [Ping timeout: 264 seconds]
05:30 < wumpus> right
05:31 < wumpus> ok that'd be awesome :)
05:32 < wumpus> re: qt, we probably want #14849 (5.9.7) first
05:32 < gribble> https://github.com/bitcoin/bitcoin/issues/14849 | depends: qt 5.9.7 by fanquake · Pull Request #14849 · bitcoin/bitcoin · GitHub
05:32 < wumpus> that looks ready
05:32 < fanquake> wumpus yep. I was going to add that as my "high priority" PR. Once that's in I'll backport to 0.17, then start looking at 5.12
05:36 < fanquake> Guess I'll add something else to HP
05:39 < wumpus> heh! it's good to have this in, I think, so that it (hopefully) gets some manual testing before release, qt relies mostly on manual testing afterall
05:41 < fanquake> hopefully nobody has been trying to use the (now disabled) lcd number functionality :p
05:43 -!- Guyver2__ [~Guyver@2001:985:f3f:1:1d5f:5a7b:8eb6:219] has quit [Quit: Going offline, see ya! (www.adiirc.com)]
05:44 < wumpus> hah just looked what that does and it really emulates a LCD display on the screen, no I don't think we plan on using that :p
05:44 < wumpus> always good to speed up qt compile by removing unnecessary modules
05:46 < wumpus> I'm happy how fast we got the boost compile already in the same way
05:47 < fanquake> Yes. A NO_QT=1 depends build is quite fast
05:47 < wumpus> yep even on slow hw
05:58 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
05:58 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 250 seconds]
05:58 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
06:12 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
06:13 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
06:14 < wumpus> speaking of which, my RISC-V node still runs fine <3
06:18 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
06:19 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
06:20 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
06:21 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
06:21 < fanquake> nice
06:26 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
06:26 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
06:27 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
06:29 -!- wxss [~user@190.2.132.87] has joined #bitcoin-core-dev
06:29 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
06:30 < fanquake> wumpus have you found a RISC-V board that will run Qt yet :o
06:33 < wumpus> fanquake: I'm fairly sure it would run qt (using remote X server) :-) but no, I don't have the extension board with PCI-E functionality so I can't actually connect a monitor
06:33 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
06:33 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
06:33 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
06:33 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
06:35 < wumpus> SiFive Unleashed + Extension board + AMD gfx card could run qt, but the former two are really hard to get hold of. I think that's going to improve though, more RISC-V cores are in the works, maybe next year...
06:35 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: No route to host]
06:35 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
06:39 < fanquake> wumpus cool. Will have to track something down to experiment with.
06:44 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
06:45 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
06:45 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
06:45 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
06:48 < wumpus> I really hope there will be more affordable boards that can run Linux, next
06:53 -!- belcher [~belcher@unaffiliated/belcher] has quit [Ping timeout: 252 seconds]
06:54 -!- belcher [~belcher@unaffiliated/belcher] has joined #bitcoin-core-dev
07:01 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
07:03 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
07:03 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
07:03 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
07:06 -!- millerti [~millerti@cpe-66-24-91-119.stny.res.rr.com] has joined #bitcoin-core-dev
07:08 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
07:09 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev
07:10 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
07:10 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
07:10 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
07:13 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
07:13 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
07:14 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
07:18 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
07:25 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
07:28 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
07:28 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
07:28 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
07:29 < luke-jr> would be nice to get POWER gitian builds merged, considering it's a much better deal than RISC-V etc
07:30 -!- jarthur [~jarthur@207.114.244.5] has joined #bitcoin-core-dev
07:30 < luke-jr> shesek: please fix your connection problems sometime in the next 4 hours
07:34 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has quit [Ping timeout: 246 seconds]
07:34 < wumpus> luke-jr: what's blocking that?
07:34 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
07:36 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
07:36 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
07:36 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
07:37 < luke-jr> wumpus: no idea; there's a minor rebase needed, but it sat idle for some time before that
07:39 < luke-jr> #14066
07:39 < gribble> https://github.com/bitcoin/bitcoin/issues/14066 | gitian-linux: Build binaries for 64-bit POWER by luke-jr · Pull Request #14066 · bitcoin/bitcoin · GitHub
07:40 < wumpus> I guess we need some more testers
07:40 < wumpus> also: did you fix cfields's remarks yet?
07:41 < luke-jr> I answered them, at least
07:41 < wumpus> ok
07:43 -!- davec [~davec@cpe-24-243-249-218.hot.res.rr.com] has quit [Quit: leaving]
07:44 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
07:45 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
07:45 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
07:45 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
07:46 -!- promag [~promag@83.223.234.98] has joined #bitcoin-core-dev
07:51 -!- davec [~davec@cpe-24-243-249-218.hot.res.rr.com] has joined #bitcoin-core-dev
07:52 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
07:53 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has joined #bitcoin-core-dev
07:53 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
07:53 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
07:53 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
08:02 < luke-jr> kinda wish I knew what was going on here: https://bugzilla.mozilla.org/show_bug.cgi?id=1512162
08:02 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
08:07 -!- jungly [~quassel@79.8.200.97] has joined #bitcoin-core-dev
08:08 < wumpus> that's a *weird* issue, why would that only happen on one platform
08:08 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
08:08 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
08:08 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
08:10 -!- promag [~promag@83.223.234.98] has quit [Remote host closed the connection]
08:11 -!- fanquake [~fanquake@unaffiliated/fanquake] has quit []
08:12 < wumpus> could be a compiler bug
08:17 < luke-jr> yes, it's not very clear if the compiler or Linux is screwing up here
08:18 < luke-jr> since the VDSO is provided at runtime, I would expect Linux
08:18 < luke-jr> but this VDSO code hasn't changed since 2010
08:21 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
08:24 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
08:25 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
08:26 < cjd> Could have been interesting to see the backtrace on the gettimeofday() to see if there is perhaps something weird about where the timeval is placed
08:26 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
08:26 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
08:26 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
08:27 < cjd> I could imagine something like timeval placed on a 32 bit boundry, linux thinks it's supposed to be on the 64, trashes the stack cookie
08:27 < cjd> but I only looked at it for 5 min so my 2c is probably overpriced
08:31 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
08:32 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
08:34 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
08:36 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
08:36 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
08:36 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
08:43 -!- promag [~promag@83.223.234.60] has joined #bitcoin-core-dev
08:47 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
08:49 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
08:52 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
08:52 -!- Tralfaz [~none@104.248.145.220] has joined #bitcoin-core-dev
08:55 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
08:55 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
08:55 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
08:56 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
08:58 -!- timothy [~tredaelli@redhat/timothy] has quit [Quit: Konversation terminated!]
08:58 -!- promag [~promag@83.223.234.60] has quit [Remote host closed the connection]
09:02 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
09:02 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
09:02 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
09:04 -!- miknotauro [~miknotaur@201.114.143.145] has joined #bitcoin-core-dev
09:13 -!- miknotauro [~miknotaur@201.114.143.145] has quit [Ping timeout: 246 seconds]
09:31 -!- wxss [~user@190.2.132.87] has quit [Ping timeout: 250 seconds]
09:39 -!- jungly [~quassel@79.8.200.97] has quit [Remote host closed the connection]
09:40 -!- promag [~promag@83.223.234.93] has joined #bitcoin-core-dev
09:45 -!- danra [5ee654fb@gateway/web/freenode/ip.94.230.84.251] has joined #bitcoin-core-dev
09:48 -!- booyah_ is now known as booyah
09:53 -!- spinza [~spin@155.93.246.187] has quit [Quit: Coyote finally caught up with me...]
09:54 -!- wxss [~user@190.2.132.87] has joined #bitcoin-core-dev
10:02 -!- mode/#bitcoin-core-dev [+o wumpus] by ChanServ
10:02 -!- mode/#bitcoin-core-dev [-o wumpus] by ChanServ
10:03 -!- kmels [~kmels@190.14.131.146] has joined #bitcoin-core-dev
10:10 -!- promag [~promag@83.223.234.93] has quit [Remote host closed the connection]
10:16 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 245 seconds]
10:17 -!- Murch [~murch@50-200-105-218-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev
10:20 -!- Giszmo [~leo@190.162.241.129] has quit [Ping timeout: 246 seconds]
10:25 -!- miknotauro [~miknotaur@201.114.143.145] has joined #bitcoin-core-dev
10:34 -!- bitcoinjunior [~bitcoinju@87.101.92.100] has joined #bitcoin-core-dev
10:39 -!- Giszmo [~leo@190.162.241.129] has joined #bitcoin-core-dev
10:41 -!- spinza [~spin@155.93.246.187] has joined #bitcoin-core-dev
10:42 < wumpus> looks like travis is failing on all new PRs
10:43 < wumpus> some extrememly scary error
10:43 < wumpus> https://travis-ci.org/bitcoin/bitcoin/jobs/467611982
10:44 < wumpus> (this is for #14951 which is only a RPC tests change and definitely shouldn't cause like this)
10:44 < gribble> https://github.com/bitcoin/bitcoin/issues/14951 | Revert "tests: Support calling add_nodes more than once" by MarcoFalke · Pull Request #14951 · bitcoin/bitcoin · GitHub
10:48 < gmaxwell> it appears to be saying that two threads are using the same fastrandomcontext at once.
10:49 -!- hebasto [~hebasto@95.164.65.194] has quit [Remote host closed the connection]
10:49 < gmaxwell> I didn't know we were running tsan in travis.
10:49 < wumpus> oof likely caused by #14624 then
10:49 < gribble> https://github.com/bitcoin/bitcoin/issues/14624 | Some simple improvements to the RNG code by sipa · Pull Request #14624 · bitcoin/bitcoin · GitHub
10:50 < jonasschnelli> probably...
10:50 -!- chenpo [~chenpo@118-168-207-62.dynamic-ip.hinet.net] has joined #bitcoin-core-dev
10:50 < gmaxwell> wumpus: looking at src/test/validation_block_tests.cpp it's totally misusing the rng in the test.
10:52 < sipa> why did travis not detect this in the pr?
10:54 < gmaxwell> I don't even really get why this paticular test exists-- like it's just starting lots of threads that call ProcessNewBlock at once. But ProcessNewBlock pretty much instantly takes a lock.
10:54 < wumpus> yeh it's clearly sharing a FastRandomContext between multiple threads
10:54 < sipa> actually, it's good to know tsan catches this
10:54 -!- brianhoffman_ [~brianhoff@pool-72-83-155-130.washdc.fios.verizon.net] has joined #bitcoin-core-dev
10:54 < wumpus> sipa: hmm, maybe because your PR was from before TSAN checking was added
10:54 -!- brianhoffman [~brianhoff@pool-72-83-155-130.washdc.fios.verizon.net] has quit [Ping timeout: 250 seconds]
10:54 -!- brianhoffman_ is now known as brianhoffman
10:55 < sipa> ah
10:55 < gmaxwell> if so, how come it wasn't caught when tsan checking was added?
10:55 < gmaxwell> simple fix, and just a test bug regardless.
10:55 -!- brianhoffman [~brianhoff@pool-72-83-155-130.washdc.fios.verizon.net] has quit [Client Quit]
10:55 < wumpus> it doesn't re-run all the travis runs for allPRs on every merge
10:55 < gmaxwell> ah.
10:55 < wumpus> might be an old cache, or who knows
10:55 < gmaxwell> though I still don't really get the utility of that test.
10:56 < wumpus> it's pretty nice that this was aught and that you were able to decipher the error :)
10:57 < gmaxwell> for some reason the tsan output is a little weird, it's only giving one side's backtrace. 
10:57 < gmaxwell> Not my first time decyphering tsan output. :P
10:57 -!- hex17or [~hex17or@HSI-KBW-046-005-137-223.hsi8.kabel-badenwuerttemberg.de] has quit [Ping timeout: 240 seconds]
11:00 -!- hex17or [~hex17or@HSI-KBW-046-005-137-223.hsi8.kabel-badenwuerttemberg.de] has joined #bitcoin-core-dev
11:00 < provoostenator> Meeting?
11:00 < wumpus> I guess it tests the locking in ProcessNewBlock in case that becomes less granular inthe future
11:00 < wumpus> #startmeeting
11:00 < lightningbot> Meeting started Thu Dec 13 19:00:20 2018 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
11:00 < lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
11:00 < moneyball> hi
11:00 < wumpus> proposed topic: drop Windows Vista support, make minimum supported Windows 7
11:01 < moneyball> just one topic proposed during the week - jamesob
11:01 < gmaxwell> wumpus: ping people.
11:01 < wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb
11:01 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev
11:01 < provoostenator> (ah well, good way to get more people to review that RNG PR post merge)
11:01 < achow101> hi
11:01 < meshcollider> hi
11:01 < jonasschnelli> hi
11:02 < luke-jr> hi
11:02 < chenpo> hi
11:02 < wumpus> #topic High priority for review
11:02 < provoostenator> Let's call the topic Asta la la Vista :-)
11:03 < wumpus> https://github.com/bitcoin/bitcoin/projects/8 three PRs left on the list right now: #14336 #14565 #14573
11:03 < gribble> https://github.com/bitcoin/bitcoin/issues/14336 | net: implement poll by pstratem · Pull Request #14336 · bitcoin/bitcoin · GitHub
11:03 < gribble> https://github.com/bitcoin/bitcoin/issues/14565 | Overhaul importmulti logic by sipa · Pull Request #14565 · bitcoin/bitcoin · GitHub
11:03 < gribble> https://github.com/bitcoin/bitcoin/issues/14573 | qt: Add Window menu by promag · Pull Request #14573 · bitcoin/bitcoin · GitHub
11:03 < wumpus> provoostenator: good idea
11:03 < sipa> hi
11:03 < wumpus> the poll one should be (almost?) ready to merge
11:04 < provoostenator> Nominating #11082
11:04 < gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
11:04 < provoostenator> Needs rebase, but really could use more review before that.
11:04 < wumpus> provoostenator: already needs rebase now
11:04 < wumpus> (and has, for 24 days)
11:05 < jonasschnelli> provoostenator: Maybe take that over from luke-jr
11:05 < sipa> does it need concept discussion?
11:05 < jonasschnelli> probably...
11:05 < provoostenator> I will eventually, but I have 15 other thing on my list. I also think it needs concept discussion.
11:06 < luke-jr> it doesn't need taking over, although if someone wants to rebase it sooner, I can push that
11:06 < provoostenator> Though we've had IRC chats about it before and everyone seemed to think it was at least a reasonable approach, compared to alterantives.
11:06 < wumpus> if it needs concept discussion, it definitely doesn't belong on the high priority for review list
11:06 < wumpus> yes, I think it's a reasonable approach
11:06 < provoostenator> Alright, maybe add it after rebase?
11:06 < wumpus> I dread making init option parsing even more involved, but it's better than the alternatives
11:07 < provoostenator> Dynamic wallet loading also needs this to make it good UX.
11:07 < sipa> yeah, fair
11:07 < wumpus> this needs *very* good tests
11:07 < provoostenator> Otherwise you'd have to manually load each time you start.
11:07 < sipa> provoostenator: i was expecting that to go in the qt settings
11:07 < wumpus> especially for the qt case ast hat's even crazier
11:08 < wumpus> how many levels of overlapping options can you have :)
11:08 < provoostenator> It has a bunch of Boost tests, but can probably use some QT tests and functional tests.
11:08 < wumpus> though this is supposed to get rid of most of the qt settings I guess?
11:08 < gmaxwell> obviously we should store what wallets get loaded in wallet.dat...
11:08 < jonasschnelli> Overlapping options is already problematic on the QT layer...
11:08 < provoostenator> Well, fewer levels thanks to my followup #12833 
11:08 < luke-jr> wumpus: in a subsequent PR
11:08 < sipa> gmaxwell: wahaha
11:08 < gribble> https://github.com/bitcoin/bitcoin/issues/12833 | [qt] move QSettings to bitcoin_rw.conf where possible by Sjors · Pull Request #12833 · bitcoin/bitcoin · GitHub
11:09 < wumpus> gmaxwell: yesss
11:09 < wumpus> luke-jr: right
11:09 < provoostenator> But I'm reluctant to add more settings, as that makes rebasing 12833 a pain.
11:09 < wumpus> luke-jr: so eventually it'll replace non-pure-user-interface settings in the qt settings, I mean?
11:10 < provoostenator> (and that one also needs tests, I haven't delved into how to write QT tests yet, can someone make me a Python framework?) 
11:10 < luke-jr> wumpus: ideally
11:10 < provoostenator> Yeah the idea is to get rid of QTsettings, with the exception of the datadir.
11:10 < jonasschnelli> I guess the QT settings files will always be there for window sizes and such... but non critical settings for sure
11:10 < wumpus> qtsettings is fine for *ui* settings
11:10 < wumpus> such as the last position of the window
11:10 < provoostenator> And what jonasschnelli says.
11:10 < wumpus> and things like that
11:10 < jonasschnelli> indeed
11:10 < wumpus> but not for settings shared with the core
11:11 < provoostenator> But not stuff that's shared with bitcoind like prune=
11:11 < jonasschnelli> yes
11:11 < wumpus> such as dbcache, etc
11:11 < wumpus> right.
11:11 < jonasschnelli> That was just a bypass of not having a way to write to a config section
11:11 < luke-jr> ?
11:12 < jonasschnelli> We wanted dbcache, proxys in Qt but didn't had a way to write to the config, so we added another layer
11:12 < wumpus> yep
11:12 < jonasschnelli> Which is no longer a clever approach once we have rw_config
11:12 < sipa> i'd be more comfortable if the set of modifiable settings (the list of wallets?) were distinct from those that aren't
11:12 < wumpus> we definitely don't want to write to bitcoin.conf directly, but having a separate configuration file that's writable is fine
11:13 < sipa> and have the GUI (where you really want everything to be modifable) directly modify bitcoin.conf
11:13 < wumpus> noooooo
11:13 < wumpus> don't write to bitcoin.conf, never
11:13 < sipa> if you use the GUI, the GUI manages the settings
11:13 < wumpus> conf should be read only
11:13 < sipa> if you don't, the config file does
11:13 < wumpus> we've been over this, please
11:13 < sipa> right, or have a separate config entirely
11:13 < wumpus> let's not change the idea now
11:13 < sipa> i really dislike having a UI edit things at the same level as the user is supposed to
11:13 < sipa> okay.
11:14  * sipa shuts up
11:14 < wumpus> we can have this discussion for a few years then never decide to do anything
11:14 < provoostenator> There's also a bunch of settings that can go straight into the wallet payload, like RBF and address type (though not their defaults).
11:14 < provoostenator> (or maybe even their defaults)
11:15 < provoostenator> #13044
11:15 < gribble> https://github.com/bitcoin/bitcoin/issues/13044 | [RFC] Long term plan for wallet command-line args · Issue #13044 · bitcoin/bitcoin · GitHub
11:15 < jonasschnelli> maybe this also raises the question how to distinct global settings between wallets
11:15 < provoostenator> That also simplifies QT, because wallet settings can be updated without any of the QTSettings stuff.
11:16 < wumpus> I'm unconfortable with writing to bitcoin.conf directly, as it is specified with -conf, which might point to a read-only configuration file, it should not be assumed it is writable
11:16 < wumpus> putting settings in the wallet again?
11:16 < gmaxwell> ugh
11:16 < sipa> ugh
11:16 < wumpus> yea deja vu...
11:16 < gmaxwell> putting settings in the wallet I think is even worse now that we have multiwallet.
11:16 < jonasschnelli> wumpus: there are the wallet flags now...
11:16 < provoostenator> wumpus: yes, that was the idea. Wallets have meta data entries. We already put binary flags in the wallet.
11:16 < jonasschnelli> but yeah...not settings
11:17 < sipa> provoostenator: originally all settings were in the wallet, it was pretty bad :)
11:17 < luke-jr> it might make sense for some subset of settings
11:17 < wumpus> sure, wallet-specific metadata should be stored in the wallet
11:17 < wumpus> but not settings
11:17 < gmaxwell> so you load another wallet and then your software starts behaving differently, madness!
11:17 < provoostenator> Yes, obviously only wallet settings.
11:17 < jamesob> write to wallet.conf.qt in datadir, have that override -conf settings?
11:17 < gmaxwell> yea sure wallet specific metadata sure fine whatever.
11:17 < jamesob> *bitcoin.conf.qt
11:17 < wumpus> jamesob: that's exactly the idea behind the bitcoin_rw.conf
11:17 < jonasschnelli> I just though about address type per wallet,... how do we handle different address types with different wallets?
11:17 < luke-jr> eg, you might have a non-segwit wallet, and a segwit wallet
11:17 < provoostenator> See the list in the issue I linked to above.
11:17 < jonasschnelli> *thought
11:18 < jamesob> wumpus: my bad, getting to the meetin glate
11:18 < wumpus> jamesob: it contains writable settings which override what is in bitcoin.conf
11:18 < wumpus> jamesob: this means being able to edit settings through RPC as well as the GUI
11:18 < jamesob> oh cool
11:18 < provoostenator> Some of these make sense int he wallet: addresstype, changetype, discardfee, fallbackfee, keypool, mintxfee, paytxfee, txconfirmtarget, etc. For others it doesn't make sense.
11:19 < gmaxwell> I don't see how fee settings make much sense in a wallet.
11:19 < wumpus> I'm not sure
11:19 < gmaxwell> Addresstype I could buy. maybe.
11:19 < provoostenator> gmaxwell: to make them behave consistently.
11:19 < jonasschnelli> Addresstype certenly,... fees, probably not
11:19 < sipa> addresstype will go away in the brave new world future anyway
11:19 < wumpus> yes
11:19 < gmaxwell> keypool no, it almost could go away.
11:19 < jonasschnelli> But I know a lot of people running a SW and a non-SW wallet in parallel
11:19 < provoostenator> But maybe fee preferences are more mempool weather dependent than wallet specific.
11:19 < gmaxwell> jonasschnelli: sounds brain damaged.
11:20 < gmaxwell> jonasschnelli: do you mean they just want to get keys with different address types?
11:20 < jonasschnelli> It may be,... but it's just how transition phases happens
11:20 < provoostenator> "keypool" might be replaced with a descriptor specific keypool, but I don't think that changes anything. Unless we stop using hardened derivation at the address level.
11:20 < wumpus> jamesob: please review #11082 :)
11:20 < gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
11:20 < gmaxwell> in any case, changing wallets doesn't change what network you're on, so changing txfee settings doesn't really follow logically.
11:20 < sipa> provoostenator: yes keypool will go away as well
11:21 < wumpus> gmaxwell: agree
11:21 < luke-jr> gmaxwell: maybe if you're worried the wallets will get linked by behaviour?
11:21 < gmaxwell> provoostenator: keypool was configurable in the first place because it interacted with backup durability.
11:21 < jonasschnelli> gmaxwell: I guess they want a SW wallet that derives P2SH(P2WPKH) and and their old wallet that derives P2PKH... (and not mix them)
11:21 < jamesob> wumpus: roger that
11:21 < provoostenator> I do like the idea of giving config / rw_config wallet specific sections.
11:21 < wumpus> scoping settings on the wallet will be confusing for users, should imo be avoided if possible
11:22 -!- riemann [~riemann@217.96.174.198.ipv4.supernova.orange.pl] has joined #bitcoin-core-dev
11:22 < gmaxwell> provoostenator: so it's not clear to me that forward caching of keys would be configurable in the future, or at least that it would be anything but an advanced thing that users usually shouldn't mess with.
11:22 < wumpus> if anything it's very difficult to come up with a good user interface for that
11:22 < sipa> i think all of those can either be reasonably done at the node level, or turn into descriptor-specific things in the wallet (and thus not need config)
11:22 < provoostenator> gmaxwell: ah I see, making it "just work" could make sense
11:22  * luke-jr suggests ignoring wallet-specific settings for initial rwconf purposes <.<
11:22 < wumpus> luke-jr: yes, I agree
11:22 < gmaxwell> yea agreed.
11:23 < wumpus> luke-jr: one step at a time
11:23 < wumpus> we've been on this step for years
11:23 < wumpus> every time it's the same discussion, though I haven't heard the idea of settings in the wallet seriously proposed for a while :)
11:23 < gmaxwell> to the extent that there are wallet specific things they probably should be in the wallet so they move with the wallet.  Regardless, they don't need to be part of the rwconf discussion.
11:24 < provoostenator> Agree regarding getting rwconf out the door without adding anything to it.
11:24 < jonasschnelli> wumpus: since its only addresstype that may be relevant and will go away over time,... I take back the argument that settings per wallet are relevant
11:24 < wumpus> jonasschnelli: thank you
11:25 < gmaxwell> (I'm just doubtful that there actually are more than a couple wallet specific things, addresstypes seem the most realistic of the ones mentioned here)
11:25 < provoostenator> Indeed, that seems the most important thing to keep consistent per wallet for privacy reasons.
11:25 < jonasschnelli> boolean things like disable-privatekeys can be handles with the 64bit wallet flags
11:25 < jonasschnelli> *handled
11:26 < provoostenator> One day everyone will send to bech32 addresses, and then we'll do v1 SegWit to start the problem over again :-)
11:26 < sipa> jonasschnelli: even that we don't need post descriptors
11:26 < wumpus> at the least, settings that are part of the wallet *should not* be part of the global options sytem
11:26 < wumpus> otherwise it just becomes too many levels
11:26 < sipa> you'd just not a have a descriptor with private keys if you don't want private keys
11:26 < gmaxwell> provoostenator: no, because v1 segwit doesnt' change the address type, bech32 already supports it.
11:26 < gmaxwell> (shocking, we already anticipated that problem... :P )
11:27 -!- timothy [~tredaelli@redhat/timothy] has joined #bitcoin-core-dev
11:27 < luke-jr> gmaxwell: but p2sh^2 could (independnetly of segwitv1)
11:27 < gmaxwell> provoostenator: I'm sure there will be some broken sites, but it should be much less of an issue.
11:27 < jonasschnelli> sipa: is that similar of saying "you don't need to import private keys if you don't want to have private keys"?
11:28 < jonasschnelli> disable-private key seems to be another layer,... a footgun protection. But seems like we are going OT
11:28 < sipa> jonasschnelli: it would turn no-private-keys into a flag for wallet creation perhaps, to determine what descriptors a new wallet is born with; but it wouldn't affect anything later
11:28 < provoostenator> gmaxwell: but you still don't want to e.g. consistently use or not use schnorr, so you still need to maintain a preference. Even if it's all bech32.
11:28 < provoostenator> *do want
11:28 < luke-jr> so back to rwconf.. <.<
11:28 < sipa> anyway, sure - we could keep it as a footgun protection, but it seems kind of pointless
11:28 < gmaxwell> provoostenator: yes but thats just a property of which keys/descriptors are in the wallet.
11:28 < sipa> yeah, getting offtopic, sorry
11:28 < provoostenator> gmaxwell: good point
11:28 < wumpus> I think it's time to move to the next topic
11:29 < provoostenator> Descriptors are very useful.
11:29 < meshcollider> sipa: I assume you'd still need a way to fail if you try and import private keys though
11:29 < provoostenator> That's the foodgun protection.
11:29 < provoostenator> footgun
11:30 < provoostenator> I see two topics proposed here https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a
11:30 < provoostenator> jamesob and moneyball
11:30 < wumpus> #topic Asta la la Vista
11:30 < jamesob> Terminator movies?
11:31 < provoostenator> Dropping Vista support
11:31 < wumpus> so the idea to move the minimum supported windows to W7 recently came up in #14922
11:31 < gribble> https://github.com/bitcoin/bitcoin/issues/14922 | [WIP] windows: Set _WIN32_WINNT to 0x0601 (Windows 7) by ken2812221 · Pull Request #14922 · bitcoin/bitcoin · GitHub
11:31 < sipa> vista extended support ended on april 11, 2017
11:31 < wumpus> apparently this was already changed in the release notes #12546
11:31 < gribble> https://github.com/bitcoin/bitcoin/issues/12546 | [docs] Minor improvements to Compatibility Notes by randolf · Pull Request #12546 · bitcoin/bitcoin · GitHub
11:31 < sipa> end of mainstream support was in 2009
11:31 < luke-jr> for Linux, we just support latest stable distros, so if we mirror that for Windows, we can move to Win10 :P
11:32 < sipa> it was also one of the least popular windows releases...
11:32 < wumpus> so I think dropping support for Vista is pretty non-controversial
11:32 < wumpus> yes
11:32 < wumpus> it wasn't even popular at the time
11:32 < achow101> +1
11:32 < jonasschnelli> ack
11:32 < meshcollider> Yep
11:32 < sipa> so i think there is no real to keep it
11:32 < provoostenator> I'm a Mac fanboy, so conflict of interest :-)
11:32 < sipa> i actually expect less opposition than the opposition we got from dropping XP support
11:33 < gmaxwell> People are still using XP, I think less so than vista.
11:33 < luke-jr> sipa: this may be the first time we actually break XP, note
11:33 < wumpus> luke-jr: W7 is still used quite a lot by people unwilling to move to W10
11:33 < gmaxwell> er more so than vista.
11:33 < gmaxwell> the bigger issue will be W10 which is kinda spyware land windows from what I'm told.
11:33 < wumpus> yes
11:33 < meshcollider> Most people consider W7 as a strict improvement over vista so there should be minimal resistance :)
11:33 < sipa> is there anything between w7 and w10?
11:33 < luke-jr> 8.1
11:34 < wumpus> there's a windows 8 but I don't think it ever caught on much
11:34 < sipa> ah yes
11:34 < sipa> anyway, ack dropping vista support
11:34 < wumpus> ok
11:35 < sipa> based on what i read; not a windows user myself
11:35 < gmaxwell> that PR will it make it actually not work when it does otherwise?
11:35 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-urmxuxhfxoqohcbo] has joined #bitcoin-core-dev
11:35 < bitcoin-git> [bitcoin] MarcoFalke opened pull request #14953: test: Make g_insecure_rand_ctx thread_local (master...Mf1812-testThreadLocal) https://github.com/bitcoin/bitcoin/pull/14953
11:35 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-urmxuxhfxoqohcbo] has left #bitcoin-core-dev []
11:35 < MarcoFalke> Should fix the thread sanitizer issue ^
11:35 < gmaxwell> That isn't what we mean by supported on other platforms, on linux if you want to try running on really old stuff, you can try but you're own your own.
11:35 < wumpus> yes it sets the minimum API level so that it won't start in <W7, and makes it possible to use newer APIs
11:35  * luke-jr pokes MarcoFalke
11:36 < gmaxwell> ah it gates newer APIs.  Is there a reason to not announce we don't support vista, but only bump that flag when we actually go to use one of those apis?
11:36 < gmaxwell> (just blocking software where it otherwise works doesn't really feel in the spirit of open source)
11:36 < wumpus> you're pedaling back now?
11:36 < luke-jr> gmaxwell: IIRC some PR wants to use Win7 API
11:37 < gmaxwell> luke-jr: okay.
11:37 < wumpus> FWIW qt is also going to break vista support 
11:37 < luke-jr> I did suggest making the change in *that* PR..;
11:37 < wumpus> this is for 0.18
11:37 < wumpus> which is still a few months away
11:37 < gmaxwell> wumpus: no wumpus, I'm not "pedaling back"   but the logic given above that we don't support older linux doesn't actually apply to actively breaking older windows.
11:37 < gmaxwell> If it's going to be broken by other changes in any case, thats fine then.
11:37 < luke-jr> eg, ionice_win bumps the Windows API version too, but wasn't merged yet
11:37 < wumpus> gmaxwell: fair enough but luke-jr was talking about supporting *only* windows 10
11:38 < wumpus> which is kind of extreme
11:38 < wumpus> W7 as bottom should be in line with all modern software
11:38 < luke-jr> wumpus: just mentioning it as a comparison to how we handle Linux distros ;)
11:38 < gmaxwell> from all I've heard running bitcoin on windows 10 sounds like it's probably a bad idea in general. 
11:38 < meshcollider> From the PR, "#14881 is using inet_pton and it's only for Vista or later. So I create this PR just for that."
11:38 < gribble> https://github.com/bitcoin/bitcoin/issues/14881 | Tests: Contract testing for the procedure AddTimeData by mmachicao · Pull Request #14881 · bitcoin/bitcoin · GitHub
11:39 < gmaxwell> wumpus: okay, concern withdrawn. I just thought it would be sad to gratitiously break it, but since we have a reason to to do, good to go.
11:39 < luke-jr> gmaxwell: but only Windows 10 is actually supports anymore IIRC
11:39 < wumpus> luke-jr: yes, I know it was not seriously, but would be in line with what we do for wsay, MacOSX, but mac users like upgrading a lot more than windows users
11:39 < luke-jr> supported*
11:39 < wumpus> luke-jr: what does microsoft still officially support? only W10? are you sure?
11:40  * luke-jr double checks
11:40 < luke-jr> Win 8.1: Mainstream support ended on January 9, 2018
11:40 < luke-jr> apparently you can pay for support until 2023
11:40 < achow101> windows 7 will have security updates until jan 2020
11:40 < wumpus> great
11:41 < luke-jr> maybe I misunderstand extended support
11:41 < luke-jr> Win 7: Mainstream support ended on January 13, 2015. Extended support until January 14, 2020 (January 2023 for Professional and Enterprise, users will need to pay for security updates).[5][6]
11:41 < wumpus> so w7 is still more or less relevant
11:41 < achow101> yes
11:42 < achow101> and i'm pretty sure a lot of people still use it too
11:42 < wumpus> right
11:42 < wumpus> let's move to next topic
11:42 < wumpus> #topic "add address index" PRs (jamesob)
11:43 < jamesob> I've talked to a number of companies who're clamoring for an address index and we've got four attempts buuuut
11:43 < jamesob> sipa has potentially disabused me of the notion that an addr index is a legitimate approach to just about anything that isn't debugging or analysis
11:43 < jonasschnelli> jamesob: you mean unspent address index?
11:44 < jamesob> sure, or a more general script descriptor index
11:44 < jonasschnelli> I agree with sipa
11:44 < achow101> jamesob: for what are the companies trying to use an address index for?
11:44 < wumpus> an address index over the full block chain never was a good idea, one over the utxo set was considered but never made itin yet
11:44 < jamesob> in any case, I think we should have some sort of recommendation (and ideally a maintained piece of software) to recommend to the folks who want to do addr-indexy thing
11:44 < jamesob> s
11:45 < luke-jr> jamesob: "don't do that" :p
11:45 < gmaxwell> I think unspents indexes are fine, generally, but shouldn't be a priority for the project over other concerns.. if someone wants to come and do the work for them, and to do them well, great.
11:45 < jamesob> achow101: afaict mostly watching for activity on various addresses
11:45 < jonasschnelli> jamesob: you could do the address index externaly via p2p..... look at https://github.com/jonasschnelli/bitcoincore-indexd (experiment)
11:45 < provoostenator> Always happy to test and review those address index PRs. I think that on top of the new index scheme it's not too bad to maintain.
11:45 < luke-jr> what if we have a full TXO/script index, and only expose it in the GUI as a block explorer? (ie, no RPC to be abused)
11:45 < gmaxwell> jamesob: our general recommendation for watching is to import them into a watching wallet.
11:45 < wumpus> *watcing activiity* shouldn't require an index of everything over the whole block chain
11:45 < provoostenator> Or we could come up with a plugin system for indexes, though that's also a can of worms.
11:45 < jamesob> jonasschnelli: indeed, as well as projects like electrs
11:46 < wumpus> wasn't there some external project addressing this?
11:46 < jonasschnelli> I think we should not load a complete address index on the shoulders of Core
11:46 < gmaxwell> jamesob: and I think if there are limitations on importing for watching, we'd like to improve those.
11:46 < luke-jr> provoostenator: I thought we had that now
11:46 < sipa> gmaxwell: yup!
11:46 < wumpus> it's a *huge* database in any case
11:46 < jonasschnelli> But I agree, externally makes it a bit more complex
11:46 < jamesob> in the next few weeks I'm going to be getting in touch with companies to get a more concrete idea of the usecases, so I'll report back with what I find
11:46 < provoostenator> luke-jr: have what? People can compile their own client of course, but that's pretty high bar and can easily lead to rebase nightmares.
11:47 < wumpus> bitcoin core is not meant as a chain analysis platform
11:47 < luke-jr> wumpus: I did it in ~2 GB a year or so ago, IIRC
11:47 < wumpus> you can do forensics using your own tools
11:47 < jamesob> wumpus: very much agree
11:47 < gmaxwell> jonasschnelli: right the problem for externally, is that implementing blockchain consistency is non-trivial and in my expirence most people who want to build an index don't want to deal with that.
11:47 < gmaxwell> jamesob: more information on the actual usecases would be very helpful.
11:47 < wumpus> it's non trivial but not rocket science either
11:47 < jonasschnelli> Sadly people expect fast BIP44 recovery (incl. history). This seems to be the most prominent real-world usecase for an address index
11:48 < wumpus> it shouldn't be that anything non-trivial needs to end up[ in bitcoin core
11:48 < wumpus> we're not the world of non-trivial solutions
11:48 < gmaxwell> jonasschnelli: most (I believe most, many at least) electrum servers don't do history, so I'm not entirely clear on the including history part of your comment.
11:48 < provoostenator> Let's also not forget to update #14053 with concept ACK / NACK so people don't waste time on trying it if it's undesirable.
11:48 < gmaxwell> wumpus: heh.
11:48 < gribble> https://github.com/bitcoin/bitcoin/issues/14053 | Add address-based index (attempt 4?) by marcinja · Pull Request #14053 · bitcoin/bitcoin · GitHub
11:48 < jonasschnelli> gmaxwell: IMO they do (index everything)
11:49 < jonasschnelli> (not electrum personal server though)
11:49 < wumpus> I mean this is another thing that's been *years*
11:49 < gmaxwell> jonasschnelli: no, I know that for a fact-- many electrum servers don't index history.
11:49 < wumpus> really, no one has been able to build a solution to this?
11:49 < gmaxwell> (I just don't know if its most of them)
11:49 < wumpus> not one of all those companies that want it?
11:49 < jamesob> provoostenator: and its sibling #14035
11:49 < luke-jr> wumpus: people capable of it don't want it :p
11:49 < gribble> https://github.com/bitcoin/bitcoin/issues/14035 | Utxoscriptindex by mgrychow · Pull Request #14035 · bitcoin/bitcoin · GitHub
11:49 < sipa> wumpus: blockstream.info uses electrs, which seems to work very well
11:49 < gmaxwell> wumpus: I think the process of learning enough to do it well does a pretty good job of convincing you that you don't want it.
11:49 < wumpus> gmaxwell: heh
11:49 < provoostenator> jonasschnelli: BIP44 recovery can be handled once we have descriptor support for importmulti and slightly more sane behavior (or a replacement for) the keypool.
11:50 < wumpus> sipa: that's a good suggestion then!
11:50 < gmaxwell> provoostenator: history recovery requires a scan of the chain, or a phenominally expensive index.
11:50 < jonasschnelli> provoostenator: you can't recover the history without an address index or a complete rescan (which is not possible for pruned peers)
11:50 < wumpus> https://github.com/romanz/electrs
11:50 < provoostenator> Ok, rescan for pruned nodes isn't there yet.
11:50 < gmaxwell> jonasschnelli: note that an index is also not compatible with pruning. :P
11:50 < wumpus> alrady had it bookmarked apparently :)
11:50 < jonasschnelli> The only light here is the scantxoutset where you can recover within seconds but not the spent-history
11:51 < provoostenator> Though you could use the neutrino filters for that and then refetch the relevant blocks.
11:51 < jonasschnelli> gmaxwell: it could be,... if we just keep the pointers and load the data via p2p once requested
11:51 < gmaxwell> jonasschnelli: I think we'd get a lot more bang for our buck working on making 'wallet without history before x' (pruned wallet) a first class supported thing.
11:51 < jonasschnelli> gmaxwell: completely agree on this
11:51 < gmaxwell> jonasschnelli: transfering 200GB of data to do the input is not really reasonable...
11:52 < gmaxwell> (and then not saving it... this would be pretty harmful to the p2p network if people were actually patient enough to use it. :) )
11:52 < jonasschnelli> gmaxwell: I mean there is a PR that enabled txindex with pruning... and fetches the tx over p2p once requested outside the kept block area
11:52 < jonasschnelli> which is slow.... but for debugging proposes okay
11:52 < luke-jr> (this makes me think again of the concept of prune-to-slow-storage where you can plug in a USB drive when/if you need old data..)
11:52 < gmaxwell> jonasschnelli: oh, I didn't recall that PR. 
11:53 < wumpus> luke-jr: prune-to-tape *hides*
11:53 < provoostenator> luke-jr: slow storage being the internet? :-)
11:53 < jonasschnelli> #13014
11:53 < gribble> https://github.com/bitcoin/bitcoin/issues/13014 | Allow txindex in prune mode by jonasschnelli · Pull Request #13014 · bitcoin/bitcoin · GitHub
11:53 < luke-jr> provoostenator: could be local
11:53 < gmaxwell> jonasschnelli: bip 157 filtering could be used similarly. 
11:54 < gmaxwell> jonasschnelli: e.g. saved the bip157 filters locally, and scan against them.
11:54 < jonasschnelli> yes... less disk space, more time spent on filtering... but same idea
11:54 < luke-jr> hmm
11:54 < luke-jr> what data do they want returned from the index? maybe we don't need to retain/fetch the block
11:54 -!- dviola [~diego@unaffiliated/dviola] has joined #bitcoin-core-dev
11:55 -!- rh0nj [~rh0nj@136.243.139.96] has quit [Remote host closed the connection]
11:55 < luke-jr> eg, if the client would be happy with a list of block numbers or txids..
11:55 < jamesob> are any of the BIP157/158 PRs on the high prio list? if not, they should be
11:55 < gmaxwell> wumpus: sadly, freeking tape costs about as much as HDDs today...  LTO7 tapes (6TB) cost about $70.
11:55 < wumpus> gmaxwell: ouch
11:55 < provoostenator> So the wallet recovery flow would be: add descriptors, download BIP-157 filters (if you didn't keep them), process filters, fetch a few blocks. User can immediately use their wallet to receive and send.
11:55 < wumpus> gmaxwell: I guess they're more reliable though
11:55 < luke-jr> jamesob: making light wallets stronger has a negative impact on Bitcoin :/
11:56 -!- rh0nj [~rh0nj@136.243.139.96] has joined #bitcoin-core-dev
11:56 < gmaxwell> provoostenator: I don't really think thats a viable long term workflow either...  rather if you couldn't afford the space to keep them you probably don't want the time/bandwidth to download them.
11:56 < sipa> luke-jr: BIP158 helps our local wallet too
11:56 < provoostenator> luke-jr: not making them possible sends most people to web wallets, not to full nodes. 
11:56 < jamesob> luke-jr: better light wallets might help adoption
11:56 < gmaxwell> luke-jr: I'm only interested in it for the local wallets.
11:57 < luke-jr> jamesob: better to not have that adoption..
11:57 < luke-jr> sipa / gmaxwell: yes, that's a point
11:57 < provoostenator> gmaxwell: recovery should be a rare thing anyway, I assume it only happens after a disaster. In which case you'd probably download the whole chain again anyway.
11:57 < sipa> provoostenator: it should be.
11:57 < gmaxwell> jamesob: history has shown otherwise, bip158 doesn't make lite wallets fundimentally more usable than they are now.  They're still massively worse than server driven wallets like electrum or web wallets.
11:58 < gmaxwell> provoostenator: right but thats also a reason that fetching them when you don't have them isn't that interesting, IMO.
11:58 < jamesob> good point - but I guess if existing light wallets switched to bip157 it'd at least ease load on existing full nodes
11:58 < gmaxwell> jamesob: what light wallets?
11:58 < provoostenator> gmawell is that _because_ of bip158 or just because there aren't that many developers working on light (non web, non electrum) wallets? That could change over time.
11:58 < wumpus> 1 minute to go
11:59 < jamesob> anything using bloom-filter-based SPV
11:59 < gmaxwell> jamesob: at least connections I see there is virtually no acutal usage of p2p lite wallets anymore. 
11:59 < wumpus> please wrap up the meeting
11:59 < jamesob> I'll report on any compelling usecases I find for addr index, but I suspect sipa et al are right that that's usually just the Wrong way
11:59 < gmaxwell> provoostenator: P2p lite wallets that scan the chain just end up with a very poor user expirence. 
12:00 < jamesob> in the meantime I recommend we give concept ACK/NACK to outstanding PRs which are related
12:00 < gmaxwell> and that doesn't really have anything to do with bip158 vs bip37.
12:00 -!- timothy [~tredaelli@redhat/timothy] has quit [Remote host closed the connection]
12:00 < gmaxwell> (in fact BIP158 is somewhat worse, but slightly less of a privacy disaster)
12:00 < wumpus> #endmeeting
12:00 < lightningbot> Meeting ended Thu Dec 13 20:00:37 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
12:00 < lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.html
12:00 < lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.txt
12:00 < lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-12-13-19.00.log.html
12:01 < provoostenator> gmaxwell: I was thinking mostly about Lightning wallets that rely on a lightweight wallet but don't have support for on chain (other than topping up channels). Those don't need to care about unconfirmed transactions.
12:01 < wumpus> jamesob: I think poeople have become tired of arguing, or even explicitly NACKing those things
12:01 < gmaxwell> (it's worse because it takes a client more time to scan the chain than with BIP37, as it has to get quite a bit more data)
12:01 < gmaxwell> wumpus: +1
12:02 < jamesob> wumpus: makes sense. I'll come up with a reply to both PRs that tries to explain why they're not a fit for core
12:02 < wumpus> at least I have, I mean, if you've been in this since 2011 or so, and have to keep telling people something is abad idea while it becomes ever a worse idea (due to block chain growth)
12:02 < jamesob> and maybe we can think about adding something to the developer guidelines about indexing
12:02 < gmaxwell> UTXO-iyx indexes I generally concept ack (at least if they're ones that aren't specific about 'addresses' but work for spk's generally), history ones, I have NAKed. 
12:02 < wumpus> gmaxwell: yep
12:03 < sipa> provoostenator: yes, i think BIP157 may be useful for a new class of clients that may become popular
12:03 < jamesob> gmaxwell: so a scriptpubkey index on UTXOs is worthwhile? is that a generally useful index to have for use by core?
12:03 < gmaxwell> jamesob: we have a problem that there aren't good instructions on watchign without an index (and we likely have bugs and limitations that make it unattractive that haven't been adequately reported)..  And for the analysis usecases what people want is an analysis platform, which we're not going to provide but don't really have much to recommend.
12:04 < jamesob> gmaxwell: agree, I don't want to spend my time on the analysis usecases
12:05 < sipa> jamesob: i dislike UTXO index equally, but it's certainly less of a burden
12:05 < gmaxwell> jamesob: It's both useful (esp if we gain pruned wallet support-- but also scantxoutset speedup), but also it's not incompatiable with scalablity.
12:05 < jamesob> good point
12:05 < sipa> gmaxwell: assuming having a local UTXO set is forever
12:05 < gmaxwell> jamesob: it's my personal belief that within 10 years most users won't ever bother fetching the far back history (or only doing so as a background test and not keeping the results). 
12:06 < gmaxwell> sipa: yes, well when otherwise becomes viable my thinking may change!
12:06 < sipa> (which at least for the foreseeable future it will be)
12:06 < jamesob> #utreexo :)
12:06 < gmaxwell> jamesob: and moreover, with the exception of ultra high end commercial installs, few users would have the resources or patience to fetch 15 years of bitcoin history.
12:07 < wumpus> doesn't need to be 'forever', it's useful *now*
12:07 < gmaxwell> jamesob: so any of indexes on that history are is trouble for the ecosystem, as the indexes are many times less viable resource wise than the history itself.
12:07 < jamesob> gmaxwell: so we'd expect "hobbyists" to run pruned nodes, or do you see some more sophisticated compaction of chain history?
12:07 < sipa> jamesob: providing bulk access to sequential data is *cheap*
12:08 < sipa> i expect nearly everyone to not bother with having the full chain
12:08 < gmaxwell> jamesob: My expirence is that commerical parties are often less tolerant of resource usage and more willing to accept "query blockchain.info" than hobbists.
12:08 < wumpus> UTXO db=sophisticaed compaction of chain history
12:08 < gmaxwell> jamesob: so I think you probably have that part backwards. :)
12:08 < jamesob> wumpus: right but you have to validate the construction of a utxo db...
12:09 < gmaxwell> jamesob: I expect that in the future nodes will download a historical UTXO set, verifying it against an assumevalid in their code, and then sync forward from that.
12:09 < wumpus> jamesob: oh sure if you really want to bypass validation
12:09 < sipa> (or in the even further future, in a utreexo or magical accumulator world, not even have that UTXO set...)
12:10 < gmaxwell> sipa: yes, but so far all those proposals are too slow to use or increase bandwidth 10x fold, which isn't a winning tradeoff.
12:10 < jamesob> gmaxwell wumpus: okay so that leads back into the discussion of some kind of utxo set commitment
12:10 < wumpus> jamesob: you can have my UTXO set if you want, no guarantees though :<
12:10 < jamesob> wumpus: ha generous
12:10 < gmaxwell> jamesob: no, it has nothing to do with a utxo set commitment in fact.
12:10 < sipa> gmaxwell: utreexo is at 2x bandwidth or so
12:11 < sipa> (according to simulation results i heard)
12:11 < jamesob> gmaxwell: you're talking about "assumevalid" in the existing sense and not an assumevalid on a utxo hash, right?
12:11 < provoostenator> What does "utreexo" stand for again?
12:11 < sipa> provoostenator: utxo + tree
12:11 < provoostenator> uTreeXO?
12:11 < jamesob> provoostenator: https://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-10-08-utxo-accumulators-and-utreexo/
12:11 < gmaxwell> jamesob: a utxo has, but "utxo commitment" has generally ment putting it in blocks which is irrelevant here.
12:12 < sipa> unfortunately it needs blocks that commit to the utxo *proofs* (not the root) for DoS resistance
12:13 < sipa> imo
12:13 < jamesob> gmaxwell: so what you're saying is that 15 years from now, you think that we'll have a hardcoded hash of some historical utxo snapshot (a la assumevalid) and users will start their chain validation from there?
12:13 < gmaxwell> jamesob: doing an assumevalid merely needs a hash, which we already have... though it would be better if nodes could record their hash for each block to make review easier.
12:14 < gmaxwell> jamesob: Yes, short of breakthroughs in ZKPs I don't think anything else will be viable.
12:15 < gmaxwell> We needed to constrain blocksize to a fraction of the current size to avoid time to initial sync constantly growing up even assuming computers/bandwidth/disk improving by 18% or whatever year over year. Since that isn't the case the validation time will continue to grow.
12:16 < jamesob> gmaxwell: makes sense
12:16 < gmaxwell> And it's already large enough that it pushes many companies to prefer hosted APIs... (even if they don't mind the initial sync time, the risk of having to take it again while the service is down is pretty unattractive).  Some end users are less tolerant of loading delays, some more.
12:17 < gmaxwell> And in any case, the same security argument for assumevalid-scripts would apply: if someone can get away with making really obvious bad changes to the code you're running, all bets are off.
12:18 < gmaxwell> Main gaps in implementation now are that it would be really good to make it easier to audit an assumevalid hash, which maybe implies adding an incrementally updatable hash.
12:18 < gmaxwell> And then just storing and fetching snapshots for the p2p protocol.
12:19 < jamesob> i.e. sipa's rolling MUHASH
12:20 < sipa> i'm leaning towards the ECMH approach now, as it's easier to cache for prevalidated txn etc
12:20 < sipa> it's more CPU, but in generally CPU time that can easily move to separate threads etc
12:21 < jamesob> (for anyone curious about rolling UTXO set hashes: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-May/014337.html)
12:24 < luke-jr> will it upset anyone's reviewing if I rebase #11082 now?
12:24 < gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
12:25 < gmaxwell> I don't disagree, however for the 'sync from a snapshot and everyone can validate the hash', we could do something more boring and just compute a conventional hash of the utxo set in the background every 25000 blocks.
12:25 < gmaxwell> There is no particular need to know the hash at every single block... sync logistics mean that it wouldn't be realistic to support syncing from an arbritary block anyways.
12:27 < gmaxwell> The other issue with using muash/ecmh with a sync is that it doesn't really lend itself to incremental validation of fetched chunks. So it would probably need to be used in addition to another hash in any case.
12:28 < gmaxwell> incremental validation of chunks-- I mean you're gonna download the 2gb utxo set from 8 peers... okay, great, you get the whole thing and the echm doesn't match. What now?  You don't even know what peer gave you the bad data.
12:28 < luke-jr> btw, it's not too late to reduce block sizes; if we get an organized dev-supported proposal following Lightning going production-ready, I think we can get enough support
12:29 < gmaxwell> luke-jr: it is too late because it already takes much larger to sync the chain than many people will tolerate.
12:29 < luke-jr> gmaxwell: we can catch up with time
12:29 < gmaxwell> Also, the harm from the significant reduction required would be considerable.
12:29 < luke-jr> not post-Lightning
12:30 < gmaxwell> luke-jr: yes post lightning too, since channel creations/closers still use capacity.
12:30 < luke-jr> less than we need
12:30 < gmaxwell> So you probably want a tree hash so that you can have each peer prove that the chunk they're sending you is consistent with the download you're expecting.
12:31 < sipa> luke-jr: i don't expect there to be a "lightning going production-ready"; there may or may not be a phase where it grows sufficiently mature, and there may or may not be a phase where large scale adoption happens
12:32 < sipa> don't put all eggs in one basket
12:32 < gmaxwell> luke-jr: I don't think it's at all obvious that the result is less than we need.
12:32 < luke-jr> one basket is better than none
12:32 < gmaxwell> Current blocksizes are perfectly survivable if we get out of this fetch the whole history on every install mode.
12:33 -!- Murch [~murch@50-200-105-218-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.]
12:33 < luke-jr> I don't think changing the security model is an appropriate avenue for that.
12:34 < gmaxwell> I don't agree that its a meaninful change to the security model, but even if it were, it would be a good tradeoff.
12:34 < gmaxwell> having users and businesses not willing to run nodes because bringing one up takes days of syncing is a change to the security model.
12:34 < gmaxwell> unambigiously.
12:34 -!- Murch [~murch@50-200-105-218-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev
12:35 < gmaxwell> and we're already there, and no reduction will eliminate that. (you're right that assuming (cough) continued computer speedups it would eventually get better if blocks were significantly limited today, but even that would take a long time)
12:36 -!- phwalkr [~phwalkr@res-sit20967.ppp.twt.it] has joined #bitcoin-core-dev
12:37 < gmaxwell> luke-jr: not to mention that the political viablity of a decrease is essentially 0, -- something you always knew, as that was also the reason that "the limit can be removed and restored if there are issues!" was a dumb idea on arrival.
12:39 < luke-jr> it's not zero. It just requires more education, less block space need (ie, Lightning), and ideally developer support.
12:39 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
12:40 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
12:40 < gwillen> given the massive political consequences of merely _not raising_ the size, I can hardly even imagine the idea of lowering it being taken seriously.
12:43 < luke-jr> gwillen: the trolls are off having fun with their altcoin now, no need to appease them anymore (in fact, they will probably encourage it because they think it's a bad idea)
12:44 < gmaxwell> The fundimental issue that many people can't wrap their heads around a limit having any purpose at all still remains. Also the issue of it being totally unclear how much is enough.
12:44 < luke-jr> gmaxwell: that's where education comes in
12:44 < gmaxwell> Education only goes so far.
12:45 -!- gabridome [~gabridome@host225-165-dynamic.211-62-r.retail.telecomitalia.it] has joined #bitcoin-core-dev
12:45 < gwillen> you've proposed reductions in size before, and I do not have the sense that they have ever gotten traction
12:45 < gmaxwell> in any case, the system inherently resists changes, so the deck is stacked against you... this is a good thing usually.
12:45 < gwillen> even among people opposed to increasing it
12:45 < luke-jr> gwillen: not really
12:45 < gmaxwell> luke-jr: I don't personally see a need or use to decrease it now.
12:46 < luke-jr> gmaxwell: because you've given up, it sounds like
12:46 < gmaxwell> Improvements to initial sync are required regardless! and I don't see them as regretable.
12:46 < luke-jr> real improvements to initial sync only work in combination with a block size decrease
12:46 < gmaxwell> and with them I'm not aware of an argument as to why the current size is particularly problematic,  we have _radically_ improved things like block propagation.
12:47 < gmaxwell> (for example)
12:47 < gmaxwell> and with the work on tx relay improvements we'll be able to cut relay overheads by maybe 40x. 
12:48 -!- Guyver2 [AdiIRC@guyver2.xs4all.nl] has joined #bitcoin-core-dev
12:48 < luke-jr> that's unrelated to initial sync..
12:49 < gmaxwell> exactly, I am pointing out that the current size is okay except for initial sync. And initial sync needs to be safely shortcutted regardless of the ongoing future size. And once it is, initial sync would no longer be a huge issue.
12:50 < luke-jr> I don't agree. That's what I mean by just giving up.
12:51 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-nqgqofiilnagoedl] has joined #bitcoin-core-dev
12:51 < bitcoin-git> [bitcoin] MarcoFalke opened pull request #14954: WIP: build: Require python 3.5 (master...Mf1811-buildPython3) https://github.com/bitcoin/bitcoin/pull/14954
12:51 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-nqgqofiilnagoedl] has left #bitcoin-core-dev []
12:51 -!- Murch [~murch@50-200-105-218-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.]
12:51 < gmaxwell> I don't think thats giving up at all.
12:52 < gmaxwell> unless you think that downloading bitcoin node software written by other people and not doing the work of writing it yourself is giving up. :P
12:52 < luke-jr> code can be verified
12:53 < gmaxwell> with an extreme amount of work, and an AV can be verified, with considerably less cost/work than code.
12:53 -!- rex4539 [~rex4539@ppp-2-84-161-2.home.otenet.gr] has joined #bitcoin-core-dev
12:54 < luke-jr> not if we give up on a full sync
12:55 < gmaxwell> I'm confused by that view.
12:56 < gmaxwell> set -assumevalid=0 and you'll not use it, you'll take a lot longer to sync.  It's still radically cheaper than doing pratically any code audit.  (especially since in the common case of someone bringing up a node when they already had one running, their review need only be to compare between their nodes)
12:56 -!- rockhouse [~rockhouse@unaffiliated/rockhouse] has quit [Remote host closed the connection]
12:56 -!- victorSN [~victorSN@unaffiliated/victorsn] has quit [Read error: Connection reset by peer]
12:56 < luke-jr> gmaxwell: but we're talking about leaving in place an infinitely growing initial sync time
12:56 < luke-jr> right _now_ it's easier, but it won't be for long
12:57 -!- phwalkr_ [~phwalkr@247.100.103.87.rev.vodafone.pt] has joined #bitcoin-core-dev
12:58 < gmaxwell> The cost of an outsider finding a subtly introduced 'bug' in the code is phenomial, maybe the cost of a review that could reliably catch something like that is on the order of $100,000  ... and that cost goes up over time as wages for experts seem to go up and the codebase increases in size.
12:58 < gmaxwell> It would take a LONG time of growth before the cost of a non-AV sync matches the cost of review that would reliably find a subtle bugdoor.
12:58 -!- Murch [~murch@50-200-105-218-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev
12:59 < luke-jr> therefore everyone should just trust developers /s
12:59 < gmaxwell> I don't see how that follows.
13:00 -!- phwalkr [~phwalkr@res-sit20967.ppp.twt.it] has quit [Ping timeout: 250 seconds]
13:00 < gmaxwell> Verification is costly, there isn't any way around that.
13:01 < luke-jr> it's something people can still do today, for the blockchain
13:05 -!- phwalkr [~phwalkr@res-anet36049d.ppp.twt.it] has joined #bitcoin-core-dev
13:08 -!- phwalkr_ [~phwalkr@247.100.103.87.rev.vodafone.pt] has quit [Ping timeout: 240 seconds]
13:09 < gmaxwell> luke-jr: and they will still be able to do it in 10 years at current growth rates. it will just continue to be burdensome enough that most people will choose not to do so, exactly as is the case for the rest of the code today.
13:10 < gmaxwell> Worrying that it'll take 40 hours of computer time instead of 8  seems penny-wise-and-pound foolish, when it'll take 40 hours of _human_ time to do even a light review of the code.
13:11 < gmaxwell> in terms of aggregate validation, political efforts would probably be better spent making implementations more verifyable.
13:20 < treyzania> inb4 bitcoin node in coq
13:21 < aj> fwiw, with the "do nothing" approach, assuming IBD speeds increase by 18% per year, and chain length grows by 200GB/year, IBD maxes out at around 2.6x current length in 5 years, and finally gets back to current time after ~18 years
13:23 -!- jarthur [~jarthur@207.114.244.5] has quit []
13:26 < gmaxwell> aj: thanks.
13:29 -!- phwalkr [~phwalkr@res-anet36049d.ppp.twt.it] has quit [Ping timeout: 245 seconds]
13:30 -!- phwalkr [~phwalkr@res-anet36049d.ppp.twt.it] has joined #bitcoin-core-dev
13:31 < aj> oh, except 100GB/year is more reasonable? 2MB * 1000 blks/week * 52 weeks/year?
13:32 < aj> that's max of 1.5x in 4 years, back to same time as now in 12 years
13:33 < phantomcircuit> gmaxwell, the only reason i can think you'd want an addrindex is if you have a pile of private keys and cant remember any metadata about them
13:34 < gmaxwell> phantomcircuit: even there, do you really want an addrindex or a utxo spk index? Also if it's actually a  pile, scantxout set will be faster.
13:35 < phantomcircuit> gmaxwell, for whatever accounting reason you need the history not just the current utxo
13:35 < aj> decreasing the weight limit to 2Mweight under those assumptions (so 50GB/year) keeps IBD time increase under 10% from now
13:36 < gmaxwell> phantomcircuit: indeed.  But for that case, you'd still probably be just as well off with bip157 filters on disk, and sequential scanning those.
13:45 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-ybefowaopxzuxqns] has joined #bitcoin-core-dev
13:45 < bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/9a4334443085...a5aea9654f36
13:45 < bitcoin-git> bitcoin/master faead93 MarcoFalke: test: Make g_insecure_rand_ctx thread_local
13:45 < bitcoin-git> bitcoin/master a5aea96 MarcoFalke: Merge #14953: test: Make g_insecure_rand_ctx thread_local...
13:45 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-ybefowaopxzuxqns] has left #bitcoin-core-dev []
13:45 < phantomcircuit> gmaxwell, that wasn't an option when people were clamoring for an addrindex though
13:46 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-ihnimxqterohktfb] has joined #bitcoin-core-dev
13:46 < bitcoin-git> [bitcoin] MarcoFalke closed pull request #14953: test: Make g_insecure_rand_ctx thread_local (master...Mf1812-testThreadLocal) https://github.com/bitcoin/bitcoin/pull/14953
13:46 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-ihnimxqterohktfb] has left #bitcoin-core-dev []
13:46 < gmaxwell> well it's not an option yet. yea, without more inforation on what people are asking for I'm not sure if people would be satisified by disk-filter powered rescan.
13:47 < phantomcircuit> gmaxwell, im 99% certain what people want is an easy way to import historic payments to an address for accounting reasons
13:47 < phantomcircuit> and have that work with whatever custom wallet stuff they've written
13:47 < phantomcircuit> (ie no creation timestamp)
13:47 < gmaxwell> that absoluely is a reason, I doubt its the only one.
13:48 < phantomcircuit> im sure the other 1% is blockexplorers calling rpc directly
13:48 < gmaxwell> e.g. like I've talked to an exchange who was asking for an index because they thought it would make the daemon into a block explorer and they could use it to answer 'sent from' for blacklisting purposes. 
13:48 < gmaxwell> But none of the addrindex proposals would have made that possible, regardless...
13:49 < gmaxwell> because they index outputs, not inputs.
13:52 -!- fabianfabian [~fabianfab@D9656CCE.cm-27.dynamic.ziggo.nl] has joined #bitcoin-core-dev
13:52 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
13:53 < adiabat> sorry just got to this but a couple maybe relevant points:
13:54 < kanzure> late hi
13:54 < adiabat> I'm hoping the utreexo thing will help clients sync faster, even without any checkpoints.  If you're running on an HDD, disk I/O is the main delay for IBD
13:55 -!- miknotauro [~miknotaur@201.114.143.145] has quit [Ping timeout: 268 seconds]
13:55 < adiabat> and with utreexo / other utxo accumulator, you can avoid touching the disk and stay in ram, so you are just cpu limited by signatures
13:55 < adiabat> (which fast computers with SSDs are now anyway, but lots of people run nodes with spinning drives)
13:56 -!- chenpo [~chenpo@118-168-207-62.dynamic-ip.hinet.net] has quit [Remote host closed the connection]
13:56 < adiabat> also separate point is that the need for a bridge node; these nodes can attach proofs to inputs for the nodes which hage pruned the utxos set away and only hold the accumulator
13:56 < treyzania> I wonder how long it would take to sync off of a bunch of RAIDed floppies
13:57 < adiabat> the bridge node needs a utxo index; not of addresses though.  But I'm not sure if it makes sense to build that into core or keep it as a separate server / node / software
13:57 < adiabat> so that might be a place to put address index software if people don't want it in bitcoind
13:58 < luke-jr> oh wow, I totally forgot about adiabat's thing after Tokyo >_<
13:58 < adiabat> yeah I need to put stuff up publicly.  I've been working on it but it takes longer than I would expect, like evertyhing else :)
13:58 < instagibbs> (btw he quoted 15% overhead during IBD in other channel)
13:59 < adiabat> instagibbs: yeah, overhead depends on how much ram you dedicate to caching the proofs
14:00 < adiabat> with no caching I have 250GB of proofs, so not great
14:00 < adiabat> but it goes down quickly with increased caching.  Also I have a maybe controversial way to use shorter hashes.
14:01 < luke-jr> did kanzure transcribe the discussions in Tokyo? I feel like I should review them now :/
14:01 < adiabat> in that you can argue that you're only needing 2nd preimage resistance, not collision resistance.  But I'm not 100% set on that, even though I'm pretty convinced
14:02 < adiabat> yeah kanzure's transcript is here: http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-10-08-utxo-accumulators-and-utreexo/
14:03 < luke-jr> thanks
14:05 < kanzure> :)
14:05 < kanzure> you beat me to it
14:05 < sipa> a rare event.
14:08 -!- Tralfaz [~none@104.248.145.220] has quit [Quit: Leaving]
14:11 < gmaxwell> adiabat: you can sync entirely in ram today, it just takes 8GB of cache... so you can measure the performance impact of that vs defaults.
14:12 < gmaxwell> adiabat: it's non-trivial, but I am not expecting that it would be better for sync time to double bandwidth usage but avoid the in memory state.
14:12 < sipa> gmaxwell: with a couple hundred MB of cache it seems the overhead is just 15%
14:13 < sipa> if that's true, it starts to sound appealing for far more deployments
14:13 < gmaxwell> well currently. 
14:14 < gmaxwell> I'm confused about that, it seems improbable to me. 15% would mean only on the order of _one_ extra hash per input.
14:15  * sipa looks at adiabat 
14:15 < gmaxwell> 2x I believed. :)
14:15 < treyzania> it works *really* great for embedded devices which might have a few hundred MiB memory and flash storage
14:15 -!- chenpo [~chenpo@118-160-51-208.dynamic-ip.hinet.net] has joined #bitcoin-core-dev
14:16 < treyzania> or like rpis
14:16 -!- phwalkr [~phwalkr@res-anet36049d.ppp.twt.it] has quit [Quit: Leaving...]
14:16 < adiabat> gmaxwell: it's because so many utxos are short-lived, and this is with relying on "hints" from the archive node you IBD from
14:17 < adiabat> the archive node knows what's happening next and can give TTL values for all the utxos created in a block
14:17 < gmaxwell> Oh I see the few hundred mb cache is basically a cache of recent outputs, plus the top of a tree for non-recent outputs (or similar)?
14:17 < adiabat> better than recent -- upcoming
14:17 < sipa> adiabat: but that doesn't apply for chain tip updates, only historical sync
14:17 < adiabat> that part is trusted, so they could lie and say long-lived utxos will be consumed soon
14:18 < adiabat> right, this is only for IBD
14:18 < sipa> i think the affect on bandwidth to keep in sync is more important
14:18 < adiabat> these optimizations don't help once you're synced up; I've mostly been focusing on IBD so far
14:19 < adiabat> there are other ideas for in-sync, but I haven't worked on that much; with IBD I have a nice data set to work off of to get performance numbers
14:20 < adiabat> archive nodes indicating which utxos will be consumed soon could potentially be useful for IBD with the current levelDB as well I guess
14:21 < adiabat> worst case if the node you're downloading from lies to you, your cache is full of the wrong things and you have to go to disk more and hang up on the lying node
14:22 < gmaxwell> I don't think this is currently an interesting route of optimizations, from an engineering perspective.   Right now the existing software doesn't implement a simple fifo for management of the cache, in part due to the complexity of doing so.  Fifoing inputs would probably have 98% of the locality gain (a question I guess your research could answer), but still less complexity than cache 
14:22 < gmaxwell> management hints.
14:23 < adiabat> yeah, fair, it makes more sense with the accumulator proofs in that you prevent having to download a lot of data
14:23 < gmaxwell> Right sounds more useful there.
14:25 < gmaxwell> But even there in IBD you're essentially trading off (say) 200gb * .15 = 30GB more data transfered, in order to avoid having to store 2GB durign sync.  I think under most relaistic models of the relative costs of storage and bandwidth, this tradeoff isn't that amazing.  Having the option of making it would be nice, but I would expect relatively few systems to actually be better off with that 
14:25 < gmaxwell> choice.
14:28 < adiabat> for powerful computers it doesn't really help, but for computers with HDDs / space constraints it can help
14:28 < sipa> or with I/O-starved VPSes
14:28 < sipa> and perhaps it's also just faster
14:28 < adiabat> but yeah if you have a fast computer and slow internet, it makes things worse
14:28 < sipa> (to be seen)
14:29 < sipa> and it also makes assumevalid-utxo models trivial
14:29 < sipa> (no snapshot to transfer...)
14:29 < adiabat> well a bech32-encodable snapshot
14:30 < adiabat> hm actually no, it's a few hundred bytes so not quite... :)
14:30 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
14:30 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
14:32 < gmaxwell> if you're assuming that the node knows the top of the tree, in order to avoid an absurd bandwidth blowup, then you'll have to transfer that.
14:33 < sipa> that just gets relayed along with the first transactions that need it
14:33 < adiabat> you could tranfer the top along with the root, but as soon as you see a few proofs in the mempool, you'll get the top
14:33 < adiabat> right, what sipa said
14:33 < sipa> it's of course still bandwidth; just saying it doesn't need a special store-and-transfer-a-snapshot protocol
14:34 < gmaxwell> Okay, but thats just hiding the cost.
14:34 < sipa> by trivial i meant complexity, not bandwidth - but even there you're probably right that it's complexity we're paying anyway in a more complex block/tx sync protocl
14:34 < gmaxwell> sipa: it does need a snapshot, because you cannot run from the tip without wreaking security, you'll need the state as of the point you know is valid.
14:35 < adiabat> yeah, network bandwidth is the big cost to this, CPU cost is low
14:35 < gmaxwell> (I almost agreed with you for a moment there though! :P )
14:35 < sipa> gmaxwell: ok, sure - but it's a few hundred bytes
14:38 -!- Guyver2 [AdiIRC@guyver2.xs4all.nl] has quit [Quit: Going offline, see ya! (www.adiirc.com)]
14:38 -!- michaelsdunn1 [~michaelsd@unaffiliated/michaelsdunn1] has quit [Remote host closed the connection]
14:38 < gmaxwell> really? there are 68m utxo or so, 1000 bytes would allow for 32 hashes.   for 68m hashes we need a tree of depth 29.  or 24 if 5 levels are saved from the top, so that results in 768 bytes per input of membership proof. 
14:39 < gmaxwell> so I think not a few hundreds bytes... certantly a lot smaller than the whole utxo set though... assuming you don't want to be able to offer proofs yourself.
14:40 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has quit [Ping timeout: 256 seconds]
14:41 -!- sipa [~pw@gateway/tor-sasl/sipa1024] has joined #bitcoin-core-dev
14:41 < sipa> what did i miss?
14:42 < luke-jr> pizza
14:42 < gmaxwell> dunno if you missed anything.
14:42 < sipa> that's okay
14:42 < gmaxwell> did you see
14:42 < gmaxwell> 14:38:47 < gmaxwell> really? there are 68m utxo or so, 1000 bytes would allow for 32 hashes.   for 68m hashes we need a tree of depth 29.  or 24 if 5 
14:42 < gmaxwell>                      levels are saved from the top, so that results in 768 bytes per input of membership proof. 
14:42 < gmaxwell> 14:39:16 < gmaxwell> so I think not a few hundreds bytes... certantly a lot smaller than the whole utxo set though... assuming you don't want to be 
14:42 < gmaxwell>                      able to offer proofs yourself.
14:42 < gmaxwell> ?
14:43 < adiabat> I'll clean up the code a bit, but the code I have is not a single tree; it's a forest of perfect binary trees
14:43 < sipa> gmaxwell: the "state" of the utxo set is just the roots of each of the trees in the forest
14:43 < adiabat> so the worst case proof yeah is like 28 high
14:43 < sipa> it's distinct from the cache of elements in the trees (which just gets transferred along with the blocks)
14:43 < adiabat> but you can try to put most of the old utxos on the left in the big tree, and most proofs that you need to transfer are in the smaller trees with high turnover
14:44 < gmaxwell> sipa ... adding 28 hashes per input is a non-starter for general usage. There are some applications, vps in a datacenter where internal bandwidth is free, where it might be attractive.  but for most systems _bandwidth_ is the most limited resource.
14:45 < sipa> gmaxwell: of course
14:45 < adiabat> gmaxwell: 28 hashes per input is too much, agreed.  There's ways to bring that number down significantly
14:45 < sipa> gmaxwell: i think you're missing some of the ideas
14:46 < adiabat> (which I haven't written up, so not you're fault :)
14:46 < gmaxwell> adiabat: it can be brought down by caching more of the top...
14:46 < sipa> but in general i think it's a semantics discussion about what you call state and what you call cache
14:46 < adiabat> but a big part is exploiting the power-law dureation of utxos
14:46 < sipa> gmaxwell: adiabat is well aware
14:47 < gmaxwell> adiabat: That doesn't come about as a result of any physical law or even economic incentive in the current system. :(
14:47 < adiabat> true, a lot of is is a heuristic, so who knows.  if you have uniform random access across the utxo set, it gets worse.
14:48 < adiabat> fitting it to the current 200GB data set we have though, it's quite consistent that many utxos persist for only a few blocks
14:49 < gmaxwell> sipa: please we cannot continue to fail to do something about sync times in favor of conjecture about future ideas with uncertian performance.  Like here "It's 2x" but only under assumptions about spending patterns, that is _really_ misleading and I think it undermines productive discussion.
14:49 < adiabat> (many get created & destroyed in the same block which means they never even touch the accumulator)
14:50 -!- wxss [~user@190.2.132.87] has quit [Quit: leaving]
14:50 < sipa> gmaxwell: for the sake of discussion, please just treat the caching we're talking about as what you're calling keeping the top levels of the tree as state
14:50 < adiabat> gmaxwell: I certainly don't mean to over-sell this; it's not a silver bullet, but I think it can help in some cases
14:50 < gmaxwell> adiabat: Sure.
14:51 < sipa> gmaxwell: fwiw, our current performance for sync with dbcache less than the entire UTXO set is also dependent on spending patterns
14:52 < gmaxwell> sipa: sure. (though less than it would be if the dbcache's managment weren't kinda stupid wrt locality)
14:53 < gmaxwell> But "oh look spending patterns changed and with no impact in fees, bandwidth needed by bitcoin nodes just went up 5x" isn't the same as saying dbcache works better with more locality.
14:53 < sipa> yeah, agree
14:54 < sipa> we need a separate average case and worst case analysis w.r.t. bandwidth impact
14:55 < gmaxwell> I am frustrated because these kind of far off researchy things seem to keep getting pulled out as a reason to not do something more near term about the initial synctime problem.
14:56 < sipa> i don't think so
14:56 < gmaxwell> Well, that is how I saw the discussion today.
14:57 < sipa> as far as i'm concerned, the potential prospect of utreexo or similar techniques are only a reason to not support a utxo-address index for me
14:57 < gmaxwell> That makes sense. I wasn't connecting it to that discussion.
14:58 -!- Tralfaz [~none@185.156.175.59] has joined #bitcoin-core-dev
14:59 < gmaxwell> We can debate on the necessity of users getting historical data except for research purposes, no such debate exists for utxo though... so some scanning mechenism must be provided even in a utxotree world. ... Ideally one that preserves user privacy. But there can be ways of doing that.
14:59 < adiabat> fwiw I think it's towards the practical / implementable side of research-land.  I certainly don't want to say "this solves everything, don't worry about sync times, I got this" though.
15:00 -!- Tralfaz [~none@185.156.175.59] has quit [Remote host closed the connection]
15:01 < gmaxwell> adiabat: Towards perhaps, but I think you overestimate that.  The gap between what we can imagine and build for testing and what we can safely put into production is pretty big.   Right now Bitcoind's dbcache grows up to some limit then is flushed completely, rather than acting as a fifo queue that exploits locality.  "Make it fifo" sounds trivial, but actually getting it done, and Right, and 
15:01 < gmaxwell> being confident that its right is not.
15:01 < sipa> adiabat: well, we'll see - i think you may be underestimating the amount of work needed to deploy archive nodes, and the practically implementing a dos-resistant stateful sync protocol for parts of the utxo tree
15:02 < sipa> adiabat: even after you've characterized all the memory and cpu requirements of a close to fully written up spec
15:02 < adiabat> agreed, it's always harder than it seems.  Also, I don't mean to put it into bitcoind, I'm going to start with more of a server / client model
15:02 < adiabat> I guess in comparison to the class group accumulator it's more implementable though
15:03 < sipa> wahaha
15:03 -!- Tralfaz [~none@185.156.175.59] has joined #bitcoin-core-dev
15:03 < sipa> yes
15:03 < gmaxwell> sipa: in any case, I was connecting this discussion back more to the assume valid discusion instead of the indexing discussion.  So to me it sounded like an argument to do nothing because utxotree will solve everything... and I'm dubious that the bandwidth overheads will make it a _general_ solution (obviously a big win in some cases), and I'm dubious as to how fast it can be made available.
15:03 < gmaxwell> adiabat: yea, funny I dunno about that! so like a class group accumulator is an isolated number theory programming project.
15:04 < adiabat> oh yeah definitely don't count on anything I'm doing time-wise, takes forever :)
15:04 < gmaxwell> so just from a software eng I think CGA might be more implementable, but it's just too slow to make useful regardless.
15:04 < sipa> gmaxwell: my current estimate for updating an archival node with a new block is about a year of CPU time
15:04 < sipa> gmaxwell: with class group based accumulators
15:04 < gmaxwell> sipa: lol yes, thats my 'too slow to make useful'
15:04 < sipa> oh of course
15:05 < gmaxwell> but like, _ignoring that_ (lol), in terms of actually switching bitcoin to use it, I expect it would be easier than utxotree. 
15:05 < sipa> also ignoring the novel cryptographic assumption? :)
15:05 < adiabat> huh, yeah the proofs are tiny, everything's constant sized, etc.  The utreexo accumulator stuff does have a whole bunch of weird logic.
15:05 < gmaxwell> Even including the cost of having to write a "fast" classgroup acc library (assuming fast doesn't mean anything more than well optimized)
15:06 < sipa> but sure, everything constant size makes a lot of logistical problems go away
15:07 < gmaxwell> including need for protocols that make roundtrips... the bane of everyone's existance.
15:08 -!- Giszmo [~leo@190.162.241.129] has quit [Ping timeout: 250 seconds]
15:11 < gmaxwell> sipa: in any case, going back to the indexing question... for history indexing part of my concern there is that for some use cases there is a "just don't do that" or "look in the utxo instead" option.  Like "We need a history index to look up from addresses!"  :P   for the utxo set that is much less the case.
15:11 < gmaxwell> so if you imagine that in the future everything is using utxotree, we'll still need to provide some way to look up scriptpubkeys... maybe it's PIR queries to nodes what have an N of M shared copy of the whole database or whatever... but something will have to be provided.
15:11 < gmaxwell> as rescan is already pretty unrealistic and contantly getting worse.
15:12 < sipa> gmaxwell: i'm not sure what use cases really need access to the full UTXO set
15:12 -!- promag [~promag@bl22-246-44.dsl.telepac.pt] has joined #bitcoin-core-dev
15:12 < sipa> of course there is a convenience in having it
15:12 < sipa> but there is convenience in having an address index for the whole chain too
15:13 < gmaxwell> sipa: I need to take my wallet offline for a long time or recover a backup and want to spend my funds.
15:13 < gmaxwell> To actually _use_ bitcoin I need that data... vs history is needed for analysis/auditing/etc purposes.
15:14 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 250 seconds]
15:14 < gmaxwell> (and for those analysis you probably don't just want an address index in history you want all sorts of other indexes)
15:15 < sipa> gmaxwell: assuming the UTXO set was equally expensive to look things up as in the blockchain (it's obviously not, but just as a hypothetical), would you still say that?
15:15 < gmaxwell> yes, because you can't spend your bitcoins without the utxo data!
15:16 < sipa> sure, but we need access to the chain anyway in recovery scenarios
15:16 < gmaxwell> (I mean if they were equal, I suppose you could just use the blockchain instead. but the utxo part is special because it's actually needed to make any use of all of your coins)
15:16 < gmaxwell> sipa: we don't, the fact that we recover tx history is an artifact of how the software was implemented.
15:16 < gmaxwell> Considering we can't recover tx medatadata, its not like we're actually recovering the real history.
15:17 < gmaxwell> The history can well be useful, for sure, but it is not needed to make use of your bitcoins.
15:17 < gmaxwell> (and other wallets, e.g. electrum, can run without history)
15:17 -!- lopp [c0ab1d67@gateway/web/freenode/ip.192.171.29.103] has joined #bitcoin-core-dev
15:17 < sipa> gmaxwell: for a business knowing history may be far more important than the ability to spend coins
15:18 < sipa> my point is that what you're talking about is pretty much a recovery scenario that should be rare; and the only reason you consider is more reasonable than seeing whole history is because it's currently (and presumably later) much cheaper
15:18 < gmaxwell> sipa: I disagree because the thing we can recover from the blockchain is not history, it's partial history-- it lacks metadata which is actually the important information.
15:18 < gmaxwell> moreover, that data is there-- go use a block explorer, there is no need a bitcoin wallet or node need to provide it.
15:18 < sipa> also true; generally you also need information you'd have backed up separately
15:18  * gmaxwell bbl
15:22 -!- Giszmo [~leo@ip-154-236-219-201.nextelmovil.cl] has joined #bitcoin-core-dev
15:23 < lopp> question regarding maintaining code integrity during the development process: is there any way that an adversarial maintainer could merge an altered PR commit? just trying to fully visualize the "chain of code custody" between the steps of peer review and actually being merged
15:23 -!- chenpo [~chenpo@118-160-51-208.dynamic-ip.hinet.net] has quit [Remote host closed the connection]
15:25 -!- fabianfabian [~fabianfab@D9656CCE.cm-27.dynamic.ziggo.nl] has quit [Quit: Textual IRC Client: www.textualapp.com]
15:25 -!- hex17or [~hex17or@HSI-KBW-046-005-137-223.hsi8.kabel-badenwuerttemberg.de] has quit [Quit: leaving]
15:38 -!- Giszmo [~leo@ip-154-236-219-201.nextelmovil.cl] has quit [Ping timeout: 250 seconds]
15:42 -!- shesek [~shesek@unaffiliated/shesek] has quit [Read error: Connection reset by peer]
15:59 -!- Giszmo [~leo@190.162.241.129] has joined #bitcoin-core-dev
16:07 -!- promag_ [~promag@bl22-246-44.dsl.telepac.pt] has joined #bitcoin-core-dev
16:09 -!- promag [~promag@bl22-246-44.dsl.telepac.pt] has quit [Ping timeout: 250 seconds]
16:12 -!- _cryptodesktop_i [~John@91.245.74.33] has joined #bitcoin-core-dev
16:17 < harding> lopp: under the default GitHub settings, anyone with commit access can push anything they want, so they could pull a PR into a private branch, modify it, merge it locally, and push it.  Some projects enable GitHub settings that prevent  direct pushes like that to the master branch, but I'm not sure whether Bitcoin Core uses them.  Alternatively, when you open a GitHub PR, a box is checked by default that allows people with commit 
16:17 < harding> access to the target repository to push to the branch, which is another way comitters could modify the code.  What comitter Mallory can't do is change contributor Alice's commits without removing any PGP signature she put on them, so if you trust Alice's key (and if she always signs her commits), you can confirm she provided those commits.
16:18 -!- miknotauro [~miknotaur@201.114.143.145] has joined #bitcoin-core-dev
16:21 < midnightmagic> code review would need to be compromised; and all the people who build it would have to miss it; and then following that the users upgrading would similarly have to miss it. wumpus does some weird things to double-check the merge process including comparing self-made branches with the PR results and so on.
16:23 < midnightmagic> lopp: anything not attached to an actual process including PR and discussion would be noticed (IMO) fairly rapidly and examined more closely.
16:23 < lopp> midnightmagic: it's mainly the "maintainer slips in a few lines to a PR" vector I'm trying to better understand
16:28 < midnightmagic> lopp: If I were a PR submitter, I would notice that. And the commits are tracked well, so even e.g. push -f could be easily forensically examined. Many of us keep full history immune to push -f tip disconnection.. A PR mod would be an obvious slip.
16:32 < midnightmagic> lopp: external verification of correct merging would likely be welcome though, if you have a script or something that you want to implement. Well. I mean I'd run it anyway.
16:35 < sipa> midnightmagic: i think a force push to the repo would be easily noticed yes
16:35 < sipa> i'm not so sure about a PR being modified by the maintained
16:36 < midnightmagic> Maybe that's just me then. But authorship would be contaminated at that point. I as a PR submitter would have noticed instantly if you guys committed something other than what I PR'd.
16:46 -!- rh0nj [~rh0nj@136.243.139.96] has quit [Remote host closed the connection]
16:46 < promag_> midnightmagic: how?
16:46 -!- promag_ is now known as promag
16:46 -!- _cryptodesktop_i [~John@91.245.74.33] has quit [Ping timeout: 250 seconds]
16:47 -!- rh0nj [~rh0nj@136.243.139.96] has joined #bitcoin-core-dev
16:47 -!- _cryptosignal_me [~John@91.245.74.33] has joined #bitcoin-core-dev
16:50 < gmaxwell> midnightmagic: I'm fairly confident that almost no submitters would notice a subtly different version.
16:51 < gmaxwell> A grossly different version, sure.
16:51 < gmaxwell> Esp since we don't force PRs to be rebased by the submitter before merging them.
16:51 < midnightmagic> weird.
16:51 < gmaxwell> (I mean, they usually have to apply cleanly, but thats it)
16:52 -!- spinza [~spin@155.93.246.187] has quit [Quit: Coyote finally caught up with me...]
16:52 < gmaxwell> many reviewers ack the specific merged commit, but that data is just lost in github land.
16:52 < promag> gmaxwell: never saw that (I think)
16:52 < gmaxwell> At least if the modified the change inside the merge commit, that would be apparent in the git history.
16:53 < midnightmagic> thus authorship contamination.
16:53 < sipa> no, changes in the merge commit wouldn't change the commit that comes from the author
16:53 < sipa> the commit id would be exactly the expected value
16:53 < gmaxwell> promag: never saw people ack specific commits? a lotof people do it consistently.  e.g. first merged commit of yours I clicked on https://github.com/bitcoin/bitcoin/pull/14885#issuecomment-445561532
16:54 < gmaxwell> sipa: right, but it would be apparent in the git history that the merge wasn't faithful, if anyone looked.
16:54 < gmaxwell> (I'm not saying that anyone looks)
16:54 < midnightmagic> the commit of the submitter could but the merge commit wouldn't.
16:54 < promag> gmaxwell: you said "many reviewers ack the specific merged commit"
16:54 < sipa> midnightmagic: i'm confused what you're referring to
16:54 < promag> I thought you meant the merge commit
16:55 < gmaxwell> promag: sorry, I didn't make the clear, no not the merge commit unfortunately that can't really happen logistically. :(
16:55 < gmaxwell> promag: but the commit that is eventually merged, not the merge commit itself.
16:55 < sipa> we actually could very reasonable verify that merge commits are faithful and conflictless
16:55 < sipa> but we don't have infrastructure for that
16:55 < sipa> it would be kinda pointless without also verifying e.g. that the author commit matches the value that was ACKed by reviewers etc
16:56 < gmaxwell> Someone could write automation that checks that merge commits are faithful, but I would WAG that not 100% of them are.
16:56 < midnightmagic> sipa: those commitids which are added to git history as a result of the merge would include a change by the maintainer, assuming we have a commit in the history which the PR submitter actually intended to be committed. Changes would be visible via git log and other historical examination
16:56 < gmaxwell> sipa: well the merge commit could copy existant acks from the github into the commit message.
16:56 < gmaxwell> The point I was trying to make and midnightmagic is making is that at least if it's the merge commit that screws it up there is a clear record.
16:56 < sipa> midnightmagic: yeah, it would be - i'm just doubtful that there is any infrasturcture or reviewer policy that would notice that
16:56 < promag> midnightmagic: do you periodically check history and merge commits?
16:57 < sipa> midnightmagic: because git doesn't tell you unless you go out of your way
16:57 < gmaxwell> vs editing the commit that gets merged itself, which would actually not leave a good record.
16:57 < promag> the same regarding merge os backports
16:57 < midnightmagic> sipa: I accept that. I'm just saying I would definitely notice contamination of any PR I submitted. And I think it's weird that other submitters don't verify. :(
16:57 < promag> s/os/of
16:57 < gmaxwell> midnightmagic: how would you even go check?
16:58 < gmaxwell> I guess do the merge yourself and compare the resulting trees?
16:58 < sipa> git show on the merge commit will show you the conflicts, i think
16:58 < sipa> (but it does depend on your local diff/conflict resolving strategy, which is a config setting)
16:59 < midnightmagic> git log, diffs, looking at the merge points and reverifying actual code committed. I don't really trust git either, after watching superior commercial systems screw up their merging in corner cases that wrecked large corp IP.
16:59 < gmaxwell> I'm no longer a commiter on bitcoin, I'd be interested in knowing how often commiters are manually resolving conflicts on master. I believe the answer is rarely but not never.
16:59 < sipa> gmaxwell: i think the answer is almost never
16:59 < gmaxwell> I think I recall doing so... but only like ... once.
16:59 < promag> gmaxwell: never noticed one in almost 2 years
17:00 < sipa> promag: how would you notice?
17:00 -!- spinza [~spin@155.93.246.187] has joined #bitcoin-core-dev
17:00 < gmaxwell> because our history is not linear there is virtually always conflict resolution, but it's automated.
17:00 < promag> right, I mean, it wasn't discuessed here at least XD
17:01 < gmaxwell> promag: yea, it wouldn't get discussed. like as a commiter I would have only done a manual resoution if it was utterly trivial and obvious, and otherwise I would have nagged someone to rebase.
17:01 < sipa> i personally would always ask for rebasing instead of merging something with conflicts, but i'm not using my merge power frequently
17:01 < sipa> and if that doesn't happen, submit a PR myself with the rebase
17:01 < gmaxwell> sipa: even if the PR author had been gone for two months ?
17:01 < gmaxwell> okay fair enough.
17:02 < sipa> i think in backport branches maintainer changes may be more common, but i shouldn't speak for others
17:02  * midnightmagic discovers taking something for granted.
17:02 < gmaxwell> Actually I think I've seen you do exactly that, open a rebase basically right before something gets merged.
17:02 -!- _cryptosignal_me [~John@91.245.74.33] has quit [Read error: Connection reset by peer]
17:02 < gmaxwell> sipa: yea, backports are a mess of manual resolution, thats why I was specific about master.
17:03 < gmaxwell> In any case, I expect we could pretty easily have a 'no manual resolution in merges' standard, but its kinda pointless without validation.
17:04 < gmaxwell> and I wouldn't be surprised if validation might be complicated by git version differences. :(
17:04 < promag> I think backports are more attractive to tamper, are on the verge of release
17:05 < gmaxwell> maybe. but they inherently get a lot less attention in a lot of regards.
17:06 < gmaxwell> Tampering in general is not the most attractive attack vector because its conspicuous. I think in general when we think about process improvements, it's best to think in terms of beneficial changes that make tampering harder as a side effect, rather than targeted changes against tampering.
17:23 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-cfwnfuwniikromza] has joined #bitcoin-core-dev
17:23 < bitcoin-git> [bitcoin] MeshCollider pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/a5aea9654f36...7a30e0f6c544
17:23 < bitcoin-git> bitcoin/master 0e75f44 Pieter Wuille: Replace CAffectedKeysVisitor with descriptor based logic
17:23 < bitcoin-git> bitcoin/master 7a30e0f MeshCollider: Merge #14821: Replace CAffectedKeysVisitor with descriptor based logic...
17:23 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-cfwnfuwniikromza] has left #bitcoin-core-dev []
17:24 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-lizewceuryjusvww] has joined #bitcoin-core-dev
17:24 < bitcoin-git> [bitcoin] MeshCollider closed pull request #14821: Replace CAffectedKeysVisitor with descriptor based logic (master...201810_die_caffectedkeysvisitor_die) https://github.com/bitcoin/bitcoin/pull/14821
17:24 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-lizewceuryjusvww] has left #bitcoin-core-dev []
17:25 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has quit [Ping timeout: 245 seconds]
17:26 < midnightmagic> lopp: check the merge commits. unclean commits result in diffs of the merging commit and the topmost merge point; clean merges result in a pure list of commits between the branchpoint of the PR branch and the topmost commit. Alternatively, for a forced merge commit (like what github does) you can diff the base of the merge and the commitid of the merging commit and the results should be 
17:26 < midnightmagic> identical. example;
17:26 < midnightmagic> er..  example:  git diff 4b4e9486af..86eddd466e, vs. git diff 4b4e9486af..4ad560dba9b3362b6639e1a1c7898ea9ca946af5
17:27 < gmaxwell> midnightmagic: there are no clean commits.
17:27 < gmaxwell> or almost none.
17:27 -!- Murch [~murch@50-200-105-218-static.hfc.comcastbusiness.net] has quit [Quit: Snoozing.]
17:27 < midnightmagic> good then. they're harder to detect.
17:27 < gmaxwell> the only time the merge does nothing is if the history was linear... which it usually isn't except for quick fixes.
17:28 < gmaxwell> like #14953 probably was merged clean, but few things are because there is just too much activity.
17:28 < midnightmagic> corner case.  So github doesn't force a merge commit?
17:28 < gribble> https://github.com/bitcoin/bitcoin/issues/14953 | test: Make g_insecure_rand_ctx thread_local by MarcoFalke · Pull Request #14953 · bitcoin/bitcoin · GitHub
17:28 < gmaxwell> midnightmagic: there is always a merge commit, nothing goes in without a merge commit.
17:29 < midnightmagic> You mean in github?
17:29 < gmaxwell> In the bitcoin project. We never commit directly to master. (I believe that no adays that is an absolute never)
17:29 < sipa> gmaxwell: indeed
17:29 < midnightmagic> Ah -- yes, good then.
17:29 < gmaxwell> For master there is always a PR and a merge.
17:29 < sipa> verify-commits would fail if you didn't
17:30 < sipa> so that's not just policy, it's actually enforced by CI
17:31 < promag> is this used contrib/devtools/github-merge.py ?
17:31 < meshcollider> I'm not sure thats true, I think wumpus sometimes pushes release note changes directly without a PR?
17:31 < gmaxwell> meshcollider: not on master, AFAIK
17:32 < sipa> promag: yes, all merges are done through that script
17:32 < meshcollider> https://github.com/bitcoin/bitcoin/commit/825f779dc715f0f006ce0f196dc834355d2e0a5a
17:32 < meshcollider> for example ^
17:33 < sipa> midnightmagic: still a merge commit above it; 024816d6cfc719c7109cf5f1aa02d41056df848d
17:33 < sipa> no?
17:33 < sipa> oh no
17:33 < aj> contrib/verify-commits.py has code to check merge commits are clean... there's an explicit list of non-clean merge commits as well (contrib/verify-commits/allow-unclean-merge-commits)
17:33 < sipa> you're right; hmm
17:33 < meshcollider> historical release notes are the only time I've seen a direct push to master IIRC
17:33 < sipa> ah, yes, there can be direct commits in masters, but they need to be signed by a maintainer
17:33 < sipa> and merge commits, which also need to be signed
17:34 < gmaxwell> aside, github output on the commit is weird, look what PR it attributes it to
17:34 < sipa> and merge commits cannot merge more than 2 branches
17:34 < gmaxwell> no octopus merges? aww
17:34 < midnightmagic> :-o
17:35 < meshcollider> yep thats enforced by verify-commits
17:35 < gmaxwell> I wonder if we shouldn't just have the verify also disallow direct commits on master. It doesn't much matter one way or another, since by norm they're not used (except for trivial things).
17:36 < gmaxwell> also not terribly important one way or another, I guess.
17:36 < gmaxwell> but the fact that sipa and I thought they weren't allowed maybe suggests that they shouldn't be.
17:36 < midnightmagic> "you man. don't eat those."
17:37 < meshcollider> its exactly the same as opening a PR and then merging it straight away, I don't think there is any security difference
17:37 < meshcollider> both will be alerted on IRC by the commit bot for example
17:39 < gmaxwell> meshcollider: indeed, but in terms of process flow, it's one fewer things to watch out for.
17:39 -!- luke-jr [~luke-jr@unaffiliated/luke-jr] has joined #bitcoin-core-dev
17:39 < gmaxwell> ::shrugs:: I don't have a strong opinion, but given that we almost never do it, it might be better to never do it.
17:40 < gmaxwell> If nothing else, it helps draw a stronger line between our practices and common but bad practices (lots of cryptocurrency projects commit everything directly to their repo with no review or oppturnity for review at all)
17:41 < gmaxwell> obviously its up to the people who do it. :)
17:42 < sipa> yeah, it's easy to discuss policies when you're not the person affected by them :)
17:43 < gmaxwell> Well, I am affected by them, at least in theory. I give at least a glance over look at _every_ commit in a release.  If I were doing that by going through the list on github, then I'd miss some.
17:43 < gmaxwell> In actuality, I do it via git, so I wouldn't ... but I could have been! :P
17:44 < midnightmagic> ha
17:44 < gmaxwell> (looking at changes on github is kind of a lost cause because of the automatic file hiding, among other things)
17:44 < meshcollider> gmaxwell: do you mean the list of PRs merged, or the list of commits itself
17:45 < meshcollider> because of course direct commits to master will be in the latter
17:45 -!- shesek [~shesek@141.226.218.112] has joined #bitcoin-core-dev
17:45 -!- shesek [~shesek@141.226.218.112] has quit [Changing host]
17:45 -!- shesek [~shesek@unaffiliated/shesek] has joined #bitcoin-core-dev
17:45 < gmaxwell> meshcollider: If I had moved to doing the review via github, I would have been doing it on PRs.
17:45 < meshcollider> fair enough
17:45 < gmaxwell> also with the mistaken impression that there was a Pr for everything.
17:46 < meshcollider> because you can still look at https://github.com/bitcoin/bitcoin/commits/master
17:46 < gmaxwell> Indeed. 
17:46 < aj> if nothing else, doing everything via a PR provides an easy place to capture comments on commits
17:46 < gmaxwell> aj: right thats why I would use PRs if I were doing it via the web.
17:46 < gmaxwell> In reality, I prefer to not see the comments.
17:46 < gmaxwell> because they can be misleading.
17:47 < gmaxwell> (A lesson I learned on this project a couple years back...)
17:47 < aj> gmaxwell: comments are helpful if you're looking at code that was merged a while ago, and want to understand some of the choices. in theory, anyway...
17:47 < gmaxwell> aj: what I like to do is look just at git, and then if I can't understand it, go look at the PR.
17:48 < aj> gmaxwell: yeah, exactly
17:48 < gmaxwell> less loading of expectations.
17:48 < aj> gmaxwell: (then find i can't understand it after reading the PR, then ask on irc :)
17:48 < gmaxwell> I couple times I've caught bugs by reading the change instead of the PR that I'm confident that I wouldn't have caught if I read the PR first.
17:49 < gmaxwell> (though for unmerged things I still end up always reading the PR first, ... lazy oh well)
17:53 < luke-jr> (#11082 is rebased btw)
17:53 < gribble> https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
17:54 < luke-jr> (if someone wants to add it to the high-prio list..)
17:54 < meshcollider> luke-jr: done
17:58 -!- Tralfaz [~none@185.156.175.59] has quit [Quit: Leaving]
18:02 -!- Murch [~murch@50-200-105-218-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev
18:04 -!- Murch [~murch@50-200-105-218-static.hfc.comcastbusiness.net] has quit [Client Quit]
18:08 < lopp> sipa: so is it accurate to say that maintainers can't alter commits that they merge because it would cause the verify-commits script to fail?
18:10 < promag> lopp: what if they change verify commits or related?
18:11 < gmaxwell> lopp: no that isn't the case at all.
18:11 < gmaxwell> lopp: they can change the commits they merge, but it won't agree with the commit ids that people ack on PRs.  Nothing automated checks this.
18:11 < gmaxwell> lopp: parties other than the maintainers (e.g. github) can't do that though, because merge commits are signed and verify commits checks the signatures.
18:12 < gmaxwell> promag: instructions on verify commits has you use it in a way where you fetch changes and then verify them before merging into your local tree.
18:12 -!- Giszmo [~leo@190.162.241.129] has quit [Ping timeout: 268 seconds]
18:12 < lopp> gotcha; I know there is a trust issue with the verify-commits script itself though I'm not familiar with how the CI runs it... hopefully it doesn't run whatever version is at HEAD
18:13 < gmaxwell> see the readme on verify commits "Using verify-commits.py safely"
18:17 < promag> gmaxwell: still don't understand.. a merge commit could remove/skip the check in travis right?
18:18 < gmaxwell> I have no idea what travis does. verify-commits is run by users (and should be run by you too), and if run like the readme says, no commit can defeat it.
18:19 < sipa> promag: if you mean that someone can modify the verify-commits script to be neutered... sure, but i'm sure that would be noticed by review
18:19 < promag> gmaxwell: thanks, will do!
18:20 < promag> meh, trust in maintainers \o/
18:21 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection]
18:22 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined #bitcoin-core-dev
18:24 < sipa> promag: s/maintainers/reviewers/
18:28 < luke-jr> lopp: don't trust CI
18:30 -!- Giszmo [~leo@ip-125-234-219-201.nextelmovil.cl] has joined #bitcoin-core-dev
18:34 -!- otoburb [~otoburb@unaffiliated/otoburb] has quit [Quit: leaving]
18:45 -!- fanquake [~fanquake@unaffiliated/fanquake] has joined #bitcoin-core-dev
18:58 < meshcollider> sipa: in #14565 is there a reason you moved pubkey_map and privkey_map outside of import_data
18:58 < gribble> https://github.com/bitcoin/bitcoin/issues/14565 | Overhaul importmulti logic by sipa · Pull Request #14565 · bitcoin/bitcoin · GitHub
18:59 < sipa> meshcollider: yes, ryanofsky asked for it
18:59 < sipa> :)
18:59 < meshcollider> oh
19:00 < sipa> after noticing that the recursive function doesn't use them, it felt cleaner to have it out of it
19:00 < meshcollider> its making my rebase more annoying :)
19:00 < meshcollider> all good
19:03 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-aejiphbegawnvoqi] has joined #bitcoin-core-dev
19:03 < bitcoin-git> [bitcoin] sipa opened pull request #14955: Switch all RNG code to the built-in PRNG (master...201812_generic_rand) https://github.com/bitcoin/bitcoin/pull/14955
19:03 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-aejiphbegawnvoqi] has left #bitcoin-core-dev []
19:05 -!- jhfrontz [~Adium@cpe-184-57-118-36.columbus.res.rr.com] has joined #bitcoin-core-dev
19:09 < gmaxwell> sipa: re 14955 on the strenghtening in the scheduler, do you hold exclusive access the whole time, or do you get randomness, strenghten and then mix back?
19:10 < sipa> gmaxwell: nope!
19:10 < sipa> only at the end
19:10 < sipa> the process is [gather entropy externally] [get entropy from local RNG] [strengthen] [feed back into local RNG]
19:11 < gmaxwell> but it isn't holding any locks during strengthen? right?
19:12 < gmaxwell> lol maybe you should use sha256d64 as the strengthening function. :P
19:13 < gmaxwell> lot more strenghtening per unit time to do 4-parallel sha256d64. :)
19:13 < sipa> gmaxwell: no, just when reading the local RNG, and when mixing back in
19:13 < sipa> gmaxwell: perhaps!
19:14 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit []
19:16 < gmaxwell> sipa: is there any way to arrange things so that no strong randomness will get gotten until one strengthening pass has happened?
19:17 < sipa> gmaxwell: yes, already done
19:17 < gmaxwell> sweet.
19:17 < sipa> the first invocation will run with 'startup' seed level, which includes everything else
19:18 < gmaxwell> we should like serialize out a random number on stop, and read it back in on start. (is there an existing dat file we could abuse for this, like peers or mempool :P ) so as to conserve our strenghtening efforts.
19:46 -!- rh0nj [~rh0nj@136.243.139.96] has quit [Ping timeout: 240 seconds]
19:48 -!- miknotauro [~miknotaur@201.114.143.145] has quit [Ping timeout: 250 seconds]
19:48 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev
19:50 -!- dviola [~diego@unaffiliated/dviola] has quit [Quit: WeeChat 2.3]
19:54 -!- rh0nj [~rh0nj@136.243.139.96] has joined #bitcoin-core-dev
20:00 -!- profmac [~ProfMac@2001:470:1f0f:226:113d:bde2:6907:1cf] has quit [Ping timeout: 268 seconds]
20:13 -!- miknotauro [~miknotaur@201.114.143.145] has joined #bitcoin-core-dev
20:13 -!- profmac [~ProfMac@2001:470:1f0f:226:a0ea:c3c6:55e3:a823] has joined #bitcoin-core-dev
20:32 -!- Giszmo [~leo@ip-125-234-219-201.nextelmovil.cl] has quit [Ping timeout: 244 seconds]
20:33 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 246 seconds]
20:59 < gmaxwell> Anyone heard any feedback on 0.17.1rc1? 
20:59 < gmaxwell> BlueMatt: ^ have you tested it to make sure it's all happy on ubuntu?
21:00 -!- SirPNut19 [~SirPNut@223.130.128.101.dy.bbexcite.jp] has joined #bitcoin-core-dev
21:00 < gmaxwell> warren: ^ have you tested it to make sure its all happy on the latest fedora?
21:00 -!- SirPNut19 [~SirPNut@223.130.128.101.dy.bbexcite.jp] has quit [Remote host closed the connection]
21:06 -!- grubles [~grubles@unaffiliated/grubles] has quit [Quit: Leaving]
21:12 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-qmupeaqajjbyvogp] has joined #bitcoin-core-dev
21:12 < bitcoin-git> [bitcoin] bpay opened pull request #14956: [qt] Ensure tx send error highlight is visible (master...show-highlighted-field) https://github.com/bitcoin/bitcoin/pull/14956
21:12 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-qmupeaqajjbyvogp] has left #bitcoin-core-dev []
21:20 -!- grubles [~grubles@unaffiliated/grubles] has joined #bitcoin-core-dev
21:29 < fanquake> gmaxwell trying to collect test feedback in #14902
21:29 < gribble> https://github.com/bitcoin/bitcoin/issues/14902 | v0.17.1 testing · Issue #14902 · bitcoin/bitcoin · GitHub
22:27 < kallewoof> Now that GetType() is gone in the source code, I'm not sure how to detect whether a serialization op is a hash operation or not. I need this for the block header in signet. Is there a new way to determine operation type somehow?
22:34 < sipa> kallewoof: i don't see what you need it for
22:35 < sipa> the hash should generally be the same as the network serialization
22:35 < kallewoof> sipa: I include the signature when (de)serializing, but not when getting the hash. The signature is a payload of the block header, by necessity
22:36 < kallewoof> I may be able to exclude the signature part when doing the actual sig check though. *checks*
22:36 < sipa> kallewoof: that seems ugly
22:36 < sipa> you can use a flag like the ones used for witness serialization
22:37 < sipa> actually, you can use exactly the same flag
22:37 < kallewoof> Oh!
22:51 -!- Zenton [~user@unaffiliated/vicenteh] has quit [Ping timeout: 244 seconds]
22:54 -!- aelxsam [3a98c470@gateway/web/freenode/ip.58.152.196.112] has joined #bitcoin-core-dev
23:19 -!- Murch [~murch@c-73-223-113-121.hsd1.ca.comcast.net] has joined #bitcoin-core-dev
23:22 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-buvytvvihpeooijg] has joined #bitcoin-core-dev
23:22 < bitcoin-git> [bitcoin] Empact opened pull request #14957: wallet: Initialize stop_block to nullptr in CWallet::ScanForWalletTransactions (master...stop-block-null) https://github.com/bitcoin/bitcoin/pull/14957
23:22 -!- bitcoin-git [bitcoin-gi@gateway/service/github.com/x-buvytvvihpeooijg] has left #bitcoin-core-dev []
23:55 -!- rh0nj [~rh0nj@136.243.139.96] has quit [Remote host closed the connection]
23:56 -!- rh0nj [~rh0nj@136.243.139.96] has joined #bitcoin-core-dev
--- Log closed Fri Dec 14 00:00:48 2018