--- Day changed Thu Jun 15 2017
00:00 -!- riemann [~riemann@ip-222-88.ists.pl] has quit [Ping timeout: 255 seconds]
00:03 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
00:08 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 246 seconds]
00:09 < wumpus> jonasschnelli: sorry if I'm completely off with https://github.com/bitcoin/bitcoin/pull/10251#issuecomment-308647720, has been a while I've touched that code, but I have some concerns
00:09 < jonasschnelli> wumpus: you mean the comment is wrong?
00:10 < wumpus> no, I think it's correct, that's why I'm worried
00:10 < wumpus> if you can convince me it's wrong that would be great
00:10 < wumpus> :)
00:10 < jonasschnelli> okay... let me see
00:11 < wumpus> but unconditional lock-taking on cs_main should be avoided as much as possible in the GUI thread
00:11 < wumpus> and the timer isn't some separate thread, and neither is the signal handler for updateBalance
00:13 -!- RubenSomsen [~RubenSoms@1.217.138.142] has joined #bitcoin-core-dev
00:14 -!- RubenSomsen [~RubenSoms@1.217.138.142] has quit [Remote host closed the connection]
00:14 < jonasschnelli> wumpus: the PR 10251 would remove the timer... and..
00:14 -!- RubenSomsen [~RubenSoms@1.217.138.142] has joined #bitcoin-core-dev
00:14 < jonasschnelli> wumpus: the wallet would push balance updates to the GUI
00:15 < wumpus> so where would the balance computation happen?
00:15 -!- riemann [~riemann@ip-222-88.ists.pl] has joined #bitcoin-core-dev
00:15 < wumpus> functions like GetUnconfirmedWatchOnlyBalance are called from the GUI thread
00:15 < wumpus> so if the balance is dirty, the computation happens in the GUI thread, after aquiring cs_main and cs_wallet unconditionally
00:16 < jonasschnelli> Hmm... yes. I see
00:16 < wumpus> up to six times - for every type of balance, this could take ~6 minutes on a slower machine during heavy lock contention
00:17 < jonasschnelli> But only if a tx gets added or block connected/disconnected, right?
00:17 < wumpus> yes, which is very often
00:17 < wumpus> ideally the GUI thread would *never* hang itself based on notifications
00:17 < jonasschnelli> You think the TRY_LOCK in a poll routine performs better?
00:17 < wumpus> this is currently the case for that timer
00:18 < jonasschnelli> Yeah.. it should not be in the GUI thread
00:18 < wumpus> it optimizes for a different user experience - it's slower, but it never hangs the GUI thread
00:18 < wumpus> it wouldn't be a big deal if wallet functions didn't need cs_main
00:18 < wumpus> taking the wallet lock itself is ok
00:19 < wumpus> but the cs_main lock is terrible, never grab it from the GUI thread if possible, and if you need to, and it's just to update information, use TRY_LOCK
00:19 < jonasschnelli> That was my main intention (remove cs_main from GUI), but I guess I have made it worse
00:20 < jonasschnelli> I thought pushing would perform better then constant polling (even with a TRY_LOCK)
00:20 < jonasschnelli> Because, if you can LOCK in a TRY_LOCK, I would have expected to also block further calls that want to aquire the LOCK during the time you calculate the balances
00:20 < bitcoin-git> [bitcoin] luke-jr opened pull request #10595: Bugfix: RPC/Mining: Use pre-segwit sigops and limits, when working with non-segwit GBT clients (master...gbt_nosegwit_fix) https://github.com/bitcoin/bitcoin/pull/10595
00:20 < wumpus> (for user-initiated things such as sending a transaction, it's more acceptable, though there too I'd prefer a solution that doesn't block the GUI thread)
00:21 < jonasschnelli> Can we recalculate the balance during MarkDirty (connect/disconnect block, etc.)?
00:21 < wumpus> the whole point of the dirty stuff is not to do that
00:22 < wumpus> e.g.: if you're going to recalculate the balance *every* time it gets dirty, you don't need a cache
00:22 < jonasschnelli> The idea behind the dirty/caching is to no recalculate it at every call,.. right?
00:23 < wumpus> the markdirty is lazy evaluation: we know that the balance might have changed, but computing it can be expensive, so only compute it when necessary
00:23 < jonasschnelli> Ideally the balance calls return the cache and the calls that have the power to invalidate the cache could recalculate?
00:23 < jonasschnelli> I see..
00:23 < jonasschnelli> [..] only compute it when necessary <-- make sense
00:23 < jonasschnelli> the PR does that,.. but on the GUI thread
00:24 < luke-jr> wumpus: I may not be able to make the meeting, but FWIW I am unsure if 10595 should be a blocker for v0.14.2 or not
00:25 < jonasschnelli> wumpus: what about calling the getBalance() calls within a QThread and Q_EMIT balanceChanged once it's done?
00:25 < wumpus> luke-jr: if we're unsure, it probably shouldn't be
00:26 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
00:26 < wumpus> jonasschnelli: but then we're adding a thread
00:27 < jonasschnelli> or use CScheduler?
00:27 < wumpus> jonasschnelli: I do think eventually that's a better solution though: have a thread to communicate with the core
00:28 < wumpus> never block in the GUI thread, have the GUI send commands/receive notifications from that thread
00:28 < jonasschnelli> Yes. Some calls are synchronous though,.. they need redesign. But that must be done at some point
00:28 < wumpus> but not a thread especially for updating balances, that's overkill, the current solution works ok
00:29 < jonasschnelli> I think a tool where we can see what lock gets triggered how much and how long it has spent time in there would be of great value for the GUI
00:29 < wumpus> the poll timer also offers congestion control: if a lot of transactions come in, it won't recompute the balance every time
00:32 < jonasschnelli> wumpus: thanks for having a look into this and it seems to have be a waste of time... let me work towards the GUI<->Core communication thread
00:33 < wumpus> jonasschnelli: yes, sorry for being encouraging at first then backpedalling, but I hadn't seen this
00:33 < jonasschnelli> Yes. Great review. I'm happy you brought that up
00:33 < gmaxwell> luke-jr: the gbt thing? I don't see how its a potential issue unless you are adding _more_ transactions to your block than the template gave you, which I doubt anyone does right now.
00:34 < luke-jr> gmaxwell: or a lot of sigops in the generation tx possibly, but a very unlikely case
00:35 < luke-jr> not sure even Eligius or p2pool would hit it
00:36 < luke-jr> yeah, seems like it shouldn't be a practical problem thinking about it more
00:36 < wumpus> jonasschnelli: and in general design direction you're right - direct notification is better than polling, but it works best if all the information can be passed in through the signal so that there's no need for a round-trip
00:37 < wumpus> jonasschnelli: in this case that won't work due to lazy evaluation
00:37 -!- cysm_ [cysm@gateway/shell/elitebnc/x-fffsqqnhtkmjkcvk] has quit [Ping timeout: 260 seconds]
00:37 < jonasschnelli> Yes. I see that point now...
00:41 -!- jtimon [~quassel@117.29.134.37.dynamic.jazztel.es] has quit [Ping timeout: 240 seconds]
00:41 -!- cysm [cysm@gateway/shell/elitebnc/x-ijltuqrlzqzyflko] has joined #bitcoin-core-dev
00:50 -!- arubi [~ese168@gateway/tor-sasl/ese168] has quit [Remote host closed the connection]
00:51 -!- JackH [~laptop@79-73-189-176.dynamic.dsl.as9105.com] has joined #bitcoin-core-dev
00:51 -!- arubi [~ese168@gateway/tor-sasl/ese168] has joined #bitcoin-core-dev
01:08 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 255 seconds]
01:10 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev
01:18 < bitcoin-git> [bitcoin] benma opened pull request #10596: Add vConnect to CConnman::Options (master...connmanoptions_connect) https://github.com/bitcoin/bitcoin/pull/10596
01:34 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev
01:39 -!- Gues_____ [~textual@2a02:908:1470:8a60:d4:84c:860b:3102] has joined #bitcoin-core-dev
01:51 -!- Gues_____ [~textual@2a02:908:1470:8a60:d4:84c:860b:3102] has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
01:53 -!- timothy [tredaelli@redhat/timothy] has joined #bitcoin-core-dev
01:54 -!- BashCo [~BashCo@unaffiliated/bashco] has joined #bitcoin-core-dev
02:17 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt]
02:19 -!- jannes [~jannes@095-097-246-234.static.chello.nl] has joined #bitcoin-core-dev
02:27 -!- beatrootfarmer [~Beatrootg@2a02:c7d:12e:100:8d63:63a2:2014:a44] has joined #bitcoin-core-dev
02:30 -!- goatturneer [~Beatrootg@2a02:c7d:12e:100:2577:d614:6112:217a] has quit [Ping timeout: 246 seconds]
02:39 < bitcoin-git> [bitcoin] practicalswift opened pull request #10597: scripted-diff: Make use of C++11:s improved handling of two consecutive right angle brackets (master...right-angle-brackets) https://github.com/bitcoin/bitcoin/pull/10597
02:48 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-xwdvzdtgbfamkkho] has joined #bitcoin-core-dev
02:56 -!- riemann [~riemann@ip-222-88.ists.pl] has quit [Quit: Leaving]
03:12 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 240 seconds]
03:13 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev
03:44 < bitcoin-git> [bitcoin] laanwj closed pull request #10588: doc: Note preexisting bug in display of fee calculation in coin control (0.14...notebug) https://github.com/bitcoin/bitcoin/pull/10588
03:52 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/228c319a944b...7c72fb99afba
03:52 < bitcoin-git> bitcoin/master e9cd778 Alex Morcos: Pass in smart fee slider value to coin control dialog...
03:52 < bitcoin-git> bitcoin/master 7c72fb9 Wladimir J. van der Laan: Merge #10582: Pass in smart fee slider value to coin control dialog...
03:53 < bitcoin-git> [bitcoin] laanwj closed pull request #10582: Pass in smart fee slider value to coin control dialog (master...fixcoincontrolfee) https://github.com/bitcoin/bitcoin/pull/10582
03:57 < ryanofsky> wumpus, jonasschnelli: related to this topic, not sure if you saw #10504 (also had a question for you in #10244)
03:57 < gribble> https://github.com/bitcoin/bitcoin/issues/10504 | GUI unresponsive during slow operations · Issue #10504 · bitcoin/bitcoin · GitHub
03:57 < gribble> https://github.com/bitcoin/bitcoin/issues/10244 | [qt] Add abstraction layer for accessing node and wallet functionality from gui by ryanofsky · Pull Request #10244 · bitcoin/bitcoin · GitHub
03:57 < wumpus> ryanofsky: no, hadn't seen yet, will take a look
04:26 -!- cryptapus_afk is now known as cryptapus
04:39 < fanquake> wumpus going to build/release 14.2 tonight?
04:41 < wumpus> fanquake: the release notes are so depressing now!
04:41 < wumpus> but yes, I think we should just get it done
04:41 < wumpus> 0.14.3 will be more exciting, I promise
04:46 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has joined #bitcoin-core-dev
04:46 < wumpus> so, one last time: did anyone hear of any issues with rc2?
05:05 < paveljanik> silent night, ...
05:06 < paveljanik> no issues
05:08 < wumpus> agreed
05:08 < wumpus> well, there we go
05:09 < wumpus> * [new tag] v0.14.2 -> v0.14.2
05:20 < bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/7c72fb99afba...c2ab38bdd57a
05:20 < bitcoin-git> bitcoin/master 1bebfc8 Alex Morcos: Output Fee Estimation Calculations in CreateTransaction
05:20 < bitcoin-git> bitcoin/master c2ab38b Wladimir J. van der Laan: Merge #10284: Always log debug information for fee calculation in CreateTransaction...
05:20 < bitcoin-git> [bitcoin] laanwj closed pull request #10284: Always log debug information for fee calculation in CreateTransaction (master...debugEstimates) https://github.com/bitcoin/bitcoin/pull/10284
05:37 -!- dabura667_ [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has quit [Ping timeout: 240 seconds]
05:45 < bitcoin-git> [bitcoin] paveljanik opened pull request #10598: Supress struct/class mismatch warnings introduced in #10284 (master...20170615_FeeCalculation_structclass) https://github.com/bitcoin/bitcoin/pull/10598
05:52 < fanquake> wumpus heh have to follow up with a quick 0.14.3 then.
05:53 < wumpus> yes - 0.14.2 just has to get out because of the upnp vuln.
05:54 -!- apll [~edgars@222.240.93.225] has quit [Ping timeout: 260 seconds]
05:55 -!- RubenSomsen [~RubenSoms@1.217.138.142] has quit [Ping timeout: 240 seconds]
05:56 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev
05:56 -!- apll [~edgars@222.240.93.225] has joined #bitcoin-core-dev
06:02 -!- apll [~edgars@222.240.93.225] has quit [Ping timeout: 255 seconds]
06:03 -!- apll [~edgars@222.240.93.225] has joined #bitcoin-core-dev
06:15 -!- RubenSomsen [~RubenSoms@1.217.138.142] has joined #bitcoin-core-dev
06:16 < jonasschnelli> ryanofsky: Yes. Saw it... as soon as I have a bit more time (currently occupied with DigitalBitbox work) i'll give you response...
06:17 < jonasschnelli> The GUI responsiveness is ugly right now...
06:17 < jonasschnelli> I wonder moving everything to ZMQ/RPC would be a viable approach (fully detach the GUI)
06:18 < jonasschnelli> Ideally a clone of qt/ (to /qtdetatched)
06:18 < jonasschnelli> Then it would allow to remove some functions which are not possible via ZMQ/RPC
06:22 < ryanofsky> is that different than what I am doing with 10244?
06:23 < ryanofsky> 10244 removes direct calls from qt -> node wallet and replaces then with calls through interfaces which could be implemented locally or through any rpc mechanism
06:23 < bitcoin-git> [bitcoin] NicolasDorier reopened pull request #9991: listreceivedbyaddress Filter Address (master...listreceivedbyaddress-filtered) https://github.com/bitcoin/bitcoin/pull/9991
06:24 < wumpus> ryanofsky: but in itself that won't improve responsiveness, or does it also make callbacks asynchronous?
06:26 < wumpus> I still have to look at it in detail, I think the idea makes sense though
06:26 < ryanofsky> it doesn't affect responsiveness, though i do think the improved code organization would make improvements easier
06:27 < ryanofsky> 10244 by itself doesn't change any behavior, all it does is replace direct calls with calls through interfaces
06:27 < wumpus> right
06:27 < wumpus> which makes sense
06:27 -!- mkarrer [~mkarrer@158.red-83-53-16.dynamicip.rima-tde.net] has quit []
06:28 < wumpus> and when there's interfaces, we can add signals to them, to subscribe to to make the UI updates asynchronous
06:29 < wumpus> why rename CFeeBumper to FeeBumper though?
06:29 < wumpus> it would be better if this was separated into GUI changes, and core changes, I think in current state this is way too large
06:29 < ryanofsky> or even if we don't do that, it makes it easier to identify where blocking calls are happening, because they will all look like m_ipc_node->something() or m_ipc_wallet->something()
06:31 < ryanofsky> the feebumper commit is the one commit that actually makes substantial core changes, i can pull that into a separate pr
06:31 < wumpus> but why rename it?
06:31 < ryanofsky> the other commits are almost entirely changes in src/qt and additions to src/ipc
06:32 < ryanofsky> i renamed it because the commit had to update every single reference to it anyway, so i figured it'd be good to follow the current naming convention
06:33 < wumpus> it seems a bit too much to try to do all that in one PR, in my opinion, but don't know what others think
06:33 < ryanofsky> anyway happy to pull that commit into a separate pr
06:34 < ryanofsky> happy to break it down
06:34 < ryanofsky> but just know that one commit is an anomoly
06:34 < ryanofsky> the other commits are uniform mechanical changes
06:34 < wumpus> I don't personally particularly care if there's a large GUI change, but core changes need to be reviewable easily, so combining functionality changes with a rename will make it harder
06:35 < wumpus> ok, that's good
06:35 < ryanofsky> ok, will pull that commit out
06:42 < ryanofsky> there is another smaller change which affects locking in wallet.cpp. will make a separate pr for that too. after this 10244 should only touch src/qt/ and src/ipc/ files, no core files
06:45 < ryanofsky> jonasschnelli, still curious to hear more about your qtdetached idea, and if 10244 would help with that
06:46 < jonasschnelli> 10244 can probably help, but my first hurdle is: +2,217 −1,131
06:46 < jonasschnelli> :)
06:47 < jonasschnelli> ryanofsky: The general abstraction may be good... but it may be also possible though wallet-/clientmodel
06:48 < ryanofsky> i guess i'd really appreciate feedback on the first commit: https://github.com/bitcoin/bitcoin/pull/10244/commits/7aeea1e0e54a773f8283605e9e4f9051cbf1ea87
06:48 < jonasschnelli> My (probably dumb) concept would be to have RPC calls in there...
06:48 < ryanofsky> because all the other commits (except feebumper which i will pull out) follow the same pattern as first commit
06:49 < jonasschnelli> I guess I would first tackle the asynchrony in the GUI layer...
06:49 < jonasschnelli> That needs to be done anyways and can have a quicker end user benefit when we would run all node communication in a designated thread
06:50 < wumpus> my preferred priority would also be improving responsiveness first
06:50 < jonasschnelli> Yes.
06:50 < jonasschnelli> For that, we first need to analze what functions/locks are the worst
06:50 < wumpus> though I think moving things to interfaces can help with that
06:50 < ryanofsky> can you guys explain your reasoning? to me it seems like this is basically unrelated, but could only make things easier not harder
06:50 < wumpus> as then it's clearer what the interface to the core thread should be
06:50 < jonasschnelli> Turn those expensive calls into async and use a general NODE<->GUI thread?
06:51 < ryanofsky> is there some way that my changes could make async improvments harder that i'm not seeing?
06:51 < jonasschnelli> No... i don't say you change is not good
06:51 < wumpus> not AFAIK
06:51 < jonasschnelli> I'm just arguing about priorities
06:51 < jonasschnelli> To get a +2,217 −1,131 change in, my experience tells me, ~0.5-1y
06:53 < jonasschnelli> but ryanofsky change makes sense.. the additional layer *IPC* <-> *wallet/client-model* <-> GUI is probably okay...
06:55 < ryanofsky> well hopefully it will take less than 0.5y to decide whether it's okay or not :)
06:56 < jonasschnelli> hehe... I think the concept is good.
06:57 < ryanofsky> feedback so far is helpful though, definitely makes sense to keep all core changes out of the pr and make it qt-only
06:58 < jonasschnelli> Yes. Ideally... this would reduce some risks
07:00 < wumpus> agree with doing it after the 0.15 split-off
07:01 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has joined #bitcoin-core-dev
07:02 < fanquake> to early for a 0.16 tag heh
07:03 < bitcoin-git> [bitcoin] practicalswift closed pull request #10597: scripted-diff: Make use of the improved handling of two consecutive right angle brackets in C++11 (master...right-angle-brackets) https://github.com/bitcoin/bitcoin/pull/10597
07:04 < jonasschnelli> ryanofsky wumpus: is it only me or is the responsiveness in the current GUI worse then 0.13?
07:04 < jonasschnelli> ryanofsky: I guess some minor fixes could speed up the GUI and there would still be time for 0.15
07:04 < fanquake> jonasschnelli in 0.14.2 or master?
07:05 -!- riemann [~riemann@84-10-11-234.static.chello.pl] has quit [Client Quit]
07:05 < jonasschnelli> fanquake: I always run master and I often have some freezes...
07:05 < jonasschnelli> I though my balance fix would reduce some... but sadly no
07:06 < ryanofsky> i can't say because i didn't really use the gui in 0.13, and even now tend to only use gui with small wallets, few transactions
07:06 < jonasschnelli> Would a LogPrint (with microseconds) in our LOCK macro make sense to analyse the lock behavior better?
07:07 < ryanofsky> that seems like would probably tell you where freezes are happening, maybe there is also a way to hook into qt to warn about event handlers that take a long time to run
07:08 < fanquake> jonasschnelli during any particular actions?
07:08 < jonasschnelli> fanquake: the freezes or the log print?
07:08 < jonasschnelli> ryanofsky: Yes...
07:08 < fanquake> the freezes
07:09 < jonasschnelli> fanquake: Yes. Calling generate 1 followed by a sendtoaddress feels blockish
07:10 < wumpus> I think there's too much overhead from logging to make that useful, also logging itself also locks
07:11 < wumpus> so you'd have to exclude that; but you could try
07:14 < jonasschnelli> I guess i'd use printf for a short hackish solution
07:15 < wumpus> in any case, I'm not sure how much you'd learn, all LOCKs in GUI code and calls to core functions that take locks on cs_main are a problem. THough possibly you can find the ones that occur most frequently if you also log where it's taken.
07:39 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 255 seconds]
07:45 < Lauda> aruby
07:45 < Lauda> oops wrong channel
07:48 -!- arowser [~quassel@106.120.101.38] has quit [Ping timeout: 246 seconds]
07:56 < bitcoin-git> [bitcoin] ryanofsky opened pull request #10600: Make feebumper class stateless (master...pr/ipc-bump) https://github.com/bitcoin/bitcoin/pull/10600
08:05 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 240 seconds]
08:06 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev
08:07 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev
08:19 -!- arowser [~quassel@106.120.101.38] has joined #bitcoin-core-dev
08:29 -!- fanquake [~fanquake@unaffiliated/fanquake] has quit [Quit: Leaving.]
08:33 -!- JackH [~laptop@79-73-189-176.dynamic.dsl.as9105.com] has quit [Ping timeout: 260 seconds]
08:35 < bitcoin-git> [bitcoin] practicalswift opened pull request #10602: Make clang-format use C++11 features (e.g. A> instead of A >) (master...clang-format-cpp11) https://github.com/bitcoin/bitcoin/pull/10602
08:47 -!- Dizzle [~dizzle@108.171.182.16] has joined #bitcoin-core-dev
08:50 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev
09:12 -!- PaulCapestany [~PaulCapes@ip72-209-228-52.dc.dc.cox.net] has quit [Quit: .]
09:12 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev
09:16 -!- PaulCapestany [~PaulCapes@ip72-209-228-52.dc.dc.cox.net] has joined #bitcoin-core-dev
09:17 -!- Guest___ [~textual@2a02:908:1470:8a60:c985:8604:30bb:be8b] has joined #bitcoin-core-dev
09:39 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev
09:39 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
09:43 -!- jtimon [~quassel@117.29.134.37.dynamic.jazztel.es] has joined #bitcoin-core-dev
09:53 -!- Guest___ [~textual@2a02:908:1470:8a60:c985:8604:30bb:be8b] has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
10:00 -!- ProfMac [43c671dc@gateway/web/freenode/ip.67.198.113.220] has joined #bitcoin-core-dev
10:20 -!- murch [6052501c@gateway/web/freenode/ip.96.82.80.28] has joined #bitcoin-core-dev
10:25 -!- JackH [~laptop@79-73-189-176.dynamic.dsl.as9105.com] has joined #bitcoin-core-dev
10:34 -!- davec [~davec@cpe-24-243-249-218.hot.res.rr.com] has quit [Ping timeout: 255 seconds]
10:35 < jnewbery> Is anyone using the 'comparison' part of the comparison test framework? Any objections to me retiring it? #10603
10:35 < gribble> https://github.com/bitcoin/bitcoin/issues/10603 | Retire the comparison test framework · Issue #10603 · bitcoin/bitcoin · GitHub
10:36 -!- davec [~davec@cpe-24-243-249-218.hot.res.rr.com] has joined #bitcoin-core-dev
10:36 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Quit: mining]
10:36 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Ping timeout: 255 seconds]
10:36 -!- To7 [~theo@2604:2000:1382:b7:f9cf:9d46:c6ac:9dbd] has joined #bitcoin-core-dev
10:40 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev
10:45 -!- goatpig [5a5c653a@gateway/web/freenode/ip.90.92.101.58] has quit [Ping timeout: 260 seconds]
10:45 < wumpus> jnewbery: I'm fairly sure people are using it
10:45 < wumpus> @(sdaftuar morcos BlueMatt)
10:46 < sdaftuar> wumpus: i've talked to jnewbery about it offline a bit; i mostly think we don't really use it
10:46 < wumpus> okay
10:46 < sdaftuar> the intention was to be able to compare many versions of the software
10:46 < wumpus> but that isn't that useful in practice?
10:46 < sdaftuar> but i don't think anyone (outside my proof of concept, way back when) actually tried to do that?
10:47 < wumpus> would be a better question to ask then "should someone be doing those tests?"
10:47 < sdaftuar> yeah i think that's the good question
10:48 < jnewbery> The way those test cases are actually used in practice is that test_runner.py runs them with a single node. In theory they could be run with multiple nodes and the states compared, but as far as I'm aware no-one does that
10:48 < wumpus> I mean we could always add tests, maybe even in a travis crontab
10:48 < sdaftuar> in practice, it seems like we've never really written tests where the outcome wasn't fixed and known, and therefore hardcoded in the test itself
10:49 < sdaftuar> i guess it's possibel we could introduce a change to bitcoind and the corresponding test and accidentally introduce a comparison failure
10:50 < sdaftuar> p2p-fullblocktest doesn't change very often though
10:50 < sdaftuar> the risk seems like it'd be in consensus changes not yet deployed, eg something like segwit
10:50 < sdaftuar> but those tests haven't changed much either
10:52 -!- timothy [tredaelli@redhat/timothy] has quit [Quit: Konversation terminated!]
10:52 < jnewbery> I think the risk seems low. The individual tests are written quite prescriptively, so in order to break a 'comparison' without breaking running the test with an individual node, someone would probably need to update the individual test case to change the expectation
10:53 < jnewbery> Anyway, I'd be interested to hear whether anyone out there is using them or has any input. Issue 10603
10:55 < jnewbery> The tests that use the comparison test framework are:
10:55 < jnewbery> 10457
10:55 < jnewbery> bip65-cltv-p2p.py bip68-112-113-p2p.py bip9-softforks.py bipdersig-p2p.py invalidblockrequest.py invalidtxrequest.py p2p-fullblocktest.py
11:01 < gmaxwell> sdaftuar: we used it extensively in the past.
11:01 < gmaxwell> rather the old bitcoinj tool. (to be clear)
11:01 < sdaftuar> gmaxwell: right, we're still going to have p2p-fullblocktest
11:01 < sdaftuar> which is approx. the same as the old bitcoinj tool
11:02 < sdaftuar> the question is just, do we try to support/improve the ComparisonTestFramework thing as the implementation
11:02 < sdaftuar> which i think has mostly been a not very helpful infrastructure... the tests in that framework are hard to read
11:02 < gmaxwell> OK.
11:03 < sdaftuar> and working around p2p changes that the ComparisonTestFramework isn't designed for is pretty hacky
11:03 < bitcoin-git> [bitcoin] jnewbery opened pull request #10604: Expose multiwallet in getwalletinfo and add multiwallet test (master...multiwallet_test) https://github.com/bitcoin/bitcoin/pull/10604
11:05 < gmaxwell> sorry, I tuned in to the end of the conversation. I mostly started commenting because I strongly disagree with the claim that that the existing test cases were at all close enough to comprehensive to guarentee consensus consistency with another implementation. We don't even come close to 100% branch coverage in script.
11:06 -!- murch [6052501c@gateway/web/freenode/ip.96.82.80.28] has quit [Ping timeout: 260 seconds]
11:06 < sdaftuar> oh, yeah definitely agree with that
11:07 < sdaftuar> i think the question is, should we be working towards comparison style tests, along the lines of what the comparisontestframework was intended for?
11:07 < kanzure> branch coverage would be a good set of tests..
11:07 < kanzure> er, script coverage. what is missing?
11:11 < gmaxwell> sdaftuar: I think they're secondary to other kinds of tests. Comparison is primarily useful if we have good random behavior generation and can verify consistency.
11:13 < sdaftuar> that makes sense to me... my thought was that it would be more helpful to rewrite our comparison tests (which are currently just evaluating a single node anyway) in an imperative style, to make them easier to maintain and debug
11:14 -!- Giszmo [~leo@ip-141-233.219.201.nextelmovil.cl] has joined #bitcoin-core-dev
11:14 < sdaftuar> and in retrospect, i think if we did want a comparison test framework, it probably makes more sense to scrap trying to do it over p2p, and just test consensus with submitblock or something. less prone to maintenance headache
11:14 < gmaxwell> er. I don't agree (sadly)--
11:16 < gmaxwell> for two reasons: p2p means its more duarable across versions (e.g. you can test an alternative implementation), and p2p means its testing the interface that actually matters; it would catch thing like failures in the block fetching state machine.
11:16 < gmaxwell> Comparing other implementations can increase the sensitivity of the tests even if we'd never expect someone to use the other implementation.
11:17 < sdaftuar> that's true in theory, but i think that has mostly failed in practice
11:18 < sdaftuar> at least, we have never implemented the full range of p2p behaviors in a single piece of python test harness code
11:18 < wumpus> what I'm not sure about is how the comparision test is better than just running the tests against the other implementation
11:19 < sdaftuar> so for instance, we end up implementing kludges and workarounds in the python test code in order to eg ensure block delivery to a particular implementation. see for instance how we handle announcing and relaying a block, after we turned off direct fetch on inv's in bitcoind
11:19 < sipa> wumpus: i guess the advantage is that you can randomly generate scenarios and compare them, rather than needing a test writer who knows what the result is supposed to be
11:19 < gmaxwell> wumpus: see my comment about randomly generated behavior. If you have a random sequence of operations you can test certian invariants, but the random generator doesn't know the exact result-- but it can let us know when two different implementations behaved differently.
11:19 < sipa> but that advantage does go away if you're just running a static scenario
11:20 < jnewbery> gmaxwell: I think really for stability across versions, you want to have a known good node that you use RPCs on to create blocks and transactions, then connect that node to nodes-under-test of different versions. Otherwise you need to update the tests whenever any p2p logic changes (eg headers first, etc)
11:20 < gmaxwell> It's an orthorgonal and very powerful method of testing.
11:20 -!- Giszmo [~leo@ip-141-233.219.201.nextelmovil.cl] has quit [Ping timeout: 240 seconds]
11:20 < sdaftuar> it seems to me that the more durable thing is to have specific p2p tests that exercise the code paths we want to test
11:20 < wumpus> right, if you can somehow generate random scenarios, it'd be useful
11:20 < ProfMac> Is this the best place to ask questions re: setting up the virtualbox / debian / gitian build environment according to https://github.com/bitcoin/bitcoin/blob/master/doc/gitian-building.md#create-a-new-virtualbox-vm The first question is at #installing-gitian, when I do the git clone https://github.com/bitcoin/bitcoin should I be in ~ or in vm-builder-0.12.4+bzr494
11:20 < gmaxwell> And yes, for static tests is not useful IMO.
11:20 -!- goatpig [5a5c653a@gateway/web/freenode/ip.90.92.101.58] has joined #bitcoin-core-dev
11:20 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Quit: mining]
11:20 < wumpus> yes, then it's like fuzzing two implementations at the same time and comparing
11:20 < gmaxwell> Rather if we get interesting patterns out of a comparison, we could seralize those into a static test (and add the exact comparison points)
11:21 < wumpus> ProfMac: where doesn't matter, as long as you consistently use same the directory, though cloning bitcoin under the vm-builder directory likely is not what you want
11:21 < gmaxwell> This is, e.g. how we tested the DER parser in libsecp256k1-- a harness with three implementations.
11:21 < jnewbery> ok, so how about I change those static tests to use the standard test framework? I won't delete the comparison test framework code so if someone wants to use it for randomized testing, they can still do that
11:22 < ProfMac> Thanks wumpus. I am doing a 2nd install from scratch, yesterday I could not get it to work.
11:22 < sipa> jnewbery: if all they are doing is testing a scenario where it's obvious from the test code what the expected behaviour is, sure
11:23 < gmaxwell> Would that mean moving the test stimulus for those things from p2p to rpc?
11:23 < sdaftuar> no, it shouldn't
11:23 < gmaxwell> okay!
11:24 < wumpus> that would mean that the comparison test code doesn't get tested itself anymore, and thus will code-rot
11:24 < sdaftuar> wumpus: i fear it already has rotted :(
11:24 < ProfMac> So I'm going to continue with the 3 git clone commands in "~/opt/gitian", anyone see any red flags or offer a better naming scheme?
11:24 < wumpus> ok...
11:25 < sdaftuar> i mean that in the sense, we already have hacky workarounds in eg p2p-fullblocktest to deal with shortcomings in the framework
11:27 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev
11:35 -!- jannes [~jannes@095-097-246-234.static.chello.nl] has quit [Quit: Leaving]
11:43 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Quit: mining]
11:46 -!- wangchun [~wangchun@li414-193.members.linode.com] has quit [Remote host closed the connection]
11:49 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev
11:50 < gmaxwell> meeting in 10 minutes.
11:55 -!- lichtamberg_ [~Adium@2a02:8388:2001:8600:2d41:72a9:750f:966e] has joined #bitcoin-core-dev
11:57 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt]
11:58 -!- twistedline [~quassel@unaffiliated/twistedline] has quit [Ping timeout: 255 seconds]
12:00 < jonasschnelli> hi
12:00 < wumpus> #startmeeting
12:00 < lightningbot> Meeting started Thu Jun 15 19:00:07 2017 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
12:00 < lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
12:00 < wumpus> PSA: v0.14.2 has been tagged, please start your gitian builders
12:00 < wumpus> topics?
12:00 < gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
12:00 < wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 instagibbs achow101
12:00 < kanzure> hi.
12:00 < paveljanik> Hi
12:00 < achow101> hi
12:00 < cfields> jonasschnelli: i just wrote a quick mutex locktime reporter, ping me after meeting if you'd like to discuss
12:00 < sdaftuar> here
12:00 < sipa> yow
12:00 < gmaxwell> (I guess I should update my list)
12:00 < jonasschnelli> cfields: awesome!
12:01 < sipa> wumpus: you're requesting the presence of 2 instagibbses?
12:01 < gmaxwell> cfields: finally someone did that, awesome.
12:01 < cfields> well it's very dumb, but it's something :)
12:01 < wumpus> sipa: yes!
12:01 < sipa> cfields: -DDEBUG_LOCKCONTENTION?
12:01 < luke-jr> lol
12:01 < jonasschnelli> cfields: dumb is good
12:01 < wumpus> #topic high priority for review
12:02 < gmaxwell> https://github.com/bitcoin/bitcoin/projects/8
12:02 < sipa> #10148 plz *puppyeyes*
12:02 < gribble> https://github.com/bitcoin/bitcoin/issues/10148 | Use non-atomic flushing with block replay by sipa · Pull Request #10148 · bitcoin/bitcoin · GitHub
12:02 < wumpus> 10148 is already on the list, I'm testing it
12:02 < sipa> cool
12:02 < sdaftuar> i'm working on an rpc test as well
12:02 < sipa> sdaftuar: awesome!
12:02 < sipa> can i help?
12:03 < sdaftuar> i'll let you know!
12:03 < sdaftuar> i think i almos thave it
12:03 -!- twistedline [~quassel@unaffiliated/twistedline] has joined #bitcoin-core-dev
12:03 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev
12:04 < sipa> having thought more about it, i don't think #10339 will have any significant performance impact
12:04 < gribble> https://github.com/bitcoin/bitcoin/issues/10339 | Optimization: Calculate block hash less times by jtimon · Pull Request #10339 · bitcoin/bitcoin · GitHub
12:04 < luke-jr> basic multiwallet was merged. I hope to have the next step (RPC support) later today. ACK to add to priority then?
12:04 < jonasschnelli> RPC support would be great
12:04 < sipa> agree
12:04 < jonasschnelli> You mean addressing wallet via RPC endpoint?
12:04 < achow101> luke-jr: yes please
12:05 < luke-jr> jonasschnelli: I mean each username has a different single wallet
12:05 < jonasschnelli> luke-jr: hmm...
12:05 < wumpus> sipa: ok, removing it from high priority then
12:05 < jonasschnelli> I'd prefere endpoints
12:05 < jonasschnelli> AUTH for wallet switching seems hackish
12:05 < luke-jr> jonasschnelli: endpoints can be done later; I'm just trying to get the simplest stuff done first
12:05 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 240 seconds]
12:05 < jonasschnelli> endpoint is 10lines of code
12:05 < wumpus> I'd also prefer endpoints, makes it easier to move wallets to external processes etc just by changing the url
12:05 < jonasschnelli> I can post it to you later
12:06 < luke-jr> jonasschnelli: not securely ;p
12:06 < luke-jr> jonasschnelli: I don't want JoinMarket to have access to my main wallet
12:06 < jonasschnelli> there is no security in our RPC implementation :/
12:06 < sipa> yay, wallet ACLs
12:06 * sipa hides
12:06 < wumpus> please don't use auth, it's not supposed to be a multi-user authentication mult-wallet, that just adds another nightmare difficult to support like accounts
12:06 < achow101> you could do some combination of both?
12:06 < luke-jr> wumpus: different username is also a simple change to the URI
12:06 < wumpus> we already support different usernames/passwords
12:06 < jonasschnelli> use a passphase as wallet name
12:06 < wumpus> it's an authentication feature, should not affect the wallet
12:06 < jonasschnelli> same security as basic auth?
12:07 < sipa> i think the first step should be either endpoint or a generic optional named parameter to select the wallet
12:07 < luke-jr> wumpus: I see no distinction
12:07 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Client Quit]
12:07 < wumpus> sipa: yes
12:07 < gmaxwell> endpoints won't be ten lines of code. After all, we'll need to add support for them to bitcoin-cli, the test framework, etc.
12:07 < achow101> sipa: agreed
12:07 < sipa> the choice of which user can access which wallets is orthogonal, i think
12:07 < wumpus> gmaxwell: which is easy, the underlying stuff (rpcproxy, libevent) obviously supports it
12:07 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev
12:07 < sipa> but i would prefer not to tie users to wallets at the auth level
12:07 < jonasschnelli> gmaxwell: yes Indeed
12:08 < jonasschnelli> first stage, each user should be able to access each wallet....
12:08 < gmaxwell> obviously we do want to have username/wallet binding, right? This lets you be more confident e.g. that your joinmarket install isn't going to screw up your ordinary wallet, for example.
12:08 < wumpus> sipa: me neither, it just seems a level violation, and causes wrong expectations that giving access to RPC to one wallet is secure in any way
12:08 < gmaxwell> (eventually)
12:08 < luke-jr> jonasschnelli: no -.-
12:09 < wumpus> gmaxwell: I really think that's going too far
12:09 < gmaxwell> then why are we bothering?
12:09 < wumpus> securing RPC for multiple users is absolutely a nightmare
12:09 < jonasschnelli> luke-jr: the first logical extendable step would be that, no? Adding wallet selecting via AUTH is something you need to throw away later
12:09 < luke-jr> well, if I can't isolate JoinMarket this way, I have no interest in doing it.. so I can just move on to GUI and leave RPC support in Knots only
12:09 < sipa> wumpus: i think it's inevitable that we'll need that
12:09 < wumpus> anyhow a security layer could always be added could be later if endpoint-based multiwallet is in place
12:09 < wumpus> sipa: I think it's a mistake
12:10 < wumpus> sipa: just like accounts was
12:10 < wumpus> it's something that bitcoind shouldn't handle
12:10 < gmaxwell> I think what luke would like to accomplish is making multiwallet immediately useful for the application of combining multiple applicatoins onto one bitcoind; rather than having to run seperate bitcoinds for each thing that needs a wallet that you're running.
12:10 < luke-jr> jonasschnelli: no?
12:10 < sipa> gmaxwell: i think that's an interesting use case; i don't think it should be the first step
12:11 < wumpus> that's just inviting bugs, there's no way we can make that secure, the RPC is not a secure endpoint and is regarded as compeltely trusted
12:11 < wumpus> it would escalate a bug in e.g. a single RPC command to a security issue, right now RPC access = fully trusted
12:11 < jonasschnelli> luke-jr: I don't know the JoinMarket use case very well.. but if you give it access to your node, it could shutdown, add peers, etc. (in case you don't trust that software)
12:11 < gmaxwell> This is also important to us at blockstream and we will end up maintaining a fork of Bitcoin with it. (though luke wasn't doing this work at our request).
12:11 < luke-jr> jonasschnelli: even if we add endpoint multiwallet and ACLs later, we still want a way to select a default wallet for each user
12:12 < luke-jr> wumpus: then why do we have auth at all?
12:12 < jonasschnelli> I really think we should keep hands away from multi-user/multi-wallet setup
12:12 < wumpus> luke-jr: to gain access
12:12 < wumpus> jonasschnelli: me too... seems something that needs to be a level on top, not handled by bitcoind itself
12:12 < jonasschnelli> For now we should focus on single-user/multi-wallet (1:n)
12:13 < jonasschnelli> n:n smells like a account-like-problem-re-incarnation
12:13 < wumpus> anyhow if we have endpoint multi-wallet access, it'spossible to slap on a wallet/user auth mapping later
12:13 < luke-jr> or vice-versa..
12:13 < wumpus> that's "just" a matter of access control
12:13 < jonasschnelli> yes. n:n may make sense.. but endpoint first seems much more logical
12:13 < wumpus> yes
12:13 < gmaxwell> jonasschnelli: I do not follow your comment with account like problems. The problem with accounts is that they weren't wallets but users expected them to be and treated them like ones.
12:14 < jonasschnelli> gmaxwell: Yes. Not directly related.
12:14 < sipa> i don't think access control is necessarily that complicated; have a global permission and wallet specific permission; configure which users have which
12:14 < wumpus> I just think that making bitcoind multi-user is a grave mistake
12:14 < wumpus> but I"ll shut up about it...
12:14 < jonasschnelli> I think the complexity is huge,.. leads to permission groups, etc.
12:14 < wumpus> yes, exactly, some people wnat everything in bitcoind
12:14 < gmaxwell> jonasschnelli: what? no it doesn't.
12:14 < sipa> well it seems that multiple people want multiwallet for multiple reasons
12:14 < sipa> i don't think that's a problem
12:15 < sipa> and should not be a blocker for the basic functionality
12:15 < wumpus> this is one the reason why the wallet should have been split off to a separate process / library I guess... now it all needs to be compounded
12:15 < wumpus> making bitcoind some kind of systemd
12:15 < jonasschnelli> if we start to use n:n, enterprises will probably use it for multi-user wallet backends...
12:15 < luke-jr> wumpus: user:wallet makes a split off later simpler
12:15 < jonasschnelli> and removing – if it gets to complicated – is hard or even impossible (like the accounting)
12:15 < wumpus> jonasschnelli: yes exactly... and what if there's a bug in that
12:15 < luke-jr> endpoints makes splitting off later complex
12:15 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 268 seconds]
12:15 < wumpus> it moves all the (perceived) responsiblity for managing multi-user setups secure to us
12:16 < jonasschnelli> luke-jr: endpoint would even work if each wallet runs in its own process
12:16 < gmaxwell> How do we split wallets if we are using endpoints?
12:16 < jonasschnelli> (though auth probably also)
12:16 < luke-jr> jonasschnelli: huh? not really..?
12:16 < jtimon> but multi-wallet doesn't imply multi-user, does it?
12:16 < wumpus> gmaxwell: what do you mean with "split wallets"?
12:16 < gmaxwell> split wallets int oseperate processes
12:16 < wumpus> gmaxwell: different wallets have different URLs then
12:17 < wumpus> gmaxwell: so it's just another change: change the port...
12:17 < luke-jr> wumpus: ALL of these options are simple URI changes..
12:17 < achow101> how would different endpoints work with bitcoin-cli or the debug console?
12:17 < luke-jr> although some tools don't allow changing the URI right now
12:17 < wumpus> http://127.0.0.1:8333/wallet1 versus http://127.0.0.1:8334/wallet2
12:17 < gmaxwell> achow101: thats why it isn't ten lines of code.
12:17 < jonasschnelli> achow101: you can add: -wallet=filename
12:17 < wumpus> achow101: just add an option
12:18 < sipa> s/filename/name/
12:18 < jonasschnelli> endpoints in bitcoin-cli is not really complex...
12:18 < luke-jr> achow101: debug console isn't via RPC anyway
12:18 < jonasschnelli> sipa: yes
12:18 < achow101> but it certainly wouldn't work with debug console
12:18 < gmaxwell> sipa +1
12:18 < wumpus> as for debug console: you could ask the same question about authentication
12:18 < gmaxwell> luke-jr: I think we'll need endpoints in any case regardless of auth to set a default.
12:18 < luke-jr> gmaxwell: maybe
12:18 < jonasschnelli> achow101: the whole GUI has no multiwallet interface
12:18 < gmaxwell> luke-jr: because that will be what you need to make it usable to work with multiple wallets as a single user.
12:18 < wumpus> debug console is not authenticated at all - so adding endpoint/auth support is likely the similar amount of work
12:19 < wumpus> if access control is implemented, a single user could want to have access to multiple wallets anyhow
12:19 < gmaxwell> debug console should perhaps get a dropdown, and yes, it will be the same work either way.. probably easier with endpoints.
12:19 < wumpus> so user=wallet is a bad abstraction
12:19 < luke-jr> I already have the GUI done BTW
12:19 < gmaxwell> yes you'll want to have access to multiple wallets from a single user regardless.
12:19 < luke-jr> it's just based on the RPC branch
12:19 < sipa> luke-jr: how does it let you select the wallet?
12:19 < gmaxwell> luke-jr: how does gui handle the debug console?
12:19 < jonasschnelli> A very hackish (and very old) endpoint impl
12:19 < luke-jr> sipa: comboboxes
12:20 < jonasschnelli> A very hackish (and very old) endpoint impl for bitcoin-cli: https://github.com/jonasschnelli/bitcoin/blob/2015/05/corewallet/src/bitcoin-cli.cpp#L134
12:20 < luke-jr> one in the main window, and one in the debug window
12:20 < gmaxwell> sounds more or less okay.
12:20 < gmaxwell> luke-jr: so why not implement endpoints first? surely even if your own use needs account you can carry a 5 line patch to allow accounts to select the default wallet.
12:20 < luke-jr> wumpus: even if a single user can access multiple wallets, we still want a way to choose the default
12:21 < gmaxwell> see above
12:21 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev
12:21 < luke-jr> gmaxwell: it's more code, and not done yet
12:21 < wumpus> well the default wallet could depend on the user, I don't really care
12:21 < luke-jr> I can implement it, but IMO it will delay things to make it the next step
12:21 < wumpus> though I'd prefer to get rid of 'default wallet', in time
12:21 < jonasschnelli> maybe the GUI should have a node window (network, peers) and a wallet-window per wallet...
12:21 < sipa> jonasschnelli: ugh
12:21 < luke-jr> the user=>wallet stuff is literally done and well-tested (in Knots), just needs to be rebased
12:21 < gmaxwell> jonasschnelli: that doesn't sound like a good UI. :P
12:22 < sipa> jonasschnelli: /me remembers browsers before tabs
12:22 < gmaxwell> at least not mandatory.
12:22 < gmaxwell> what sipa says. :P
12:22 < jonasschnelli> yeah... I like windows.. but I'm pretty alone nowadays with that
12:22 < jonasschnelli> Yeah. Tabs make more sense I guess.
12:22 < sipa> anyway, separate discussion
12:22 * jonasschnelli think sipa certainly browses with lynx
12:23 < sipa> i would really prefer endpoints or optional named argument to select a wallet, and deal with the authentication question later
12:23 < jonasschnelli> sipa: +1
12:23 < gmaxwell> luke-jr: in any case, seems to me the path forward is to do the endpoints thing, and having auth pick default is a simple change which is either sufficiently non-objectionable or at least a trivial patch to carry.
12:23 < achow101> sipa: ack
12:23 < wumpus> sipa: same for me
12:23 < luke-jr> would anyone NACK if I go forward with user->wallet mappings since they're basically ready, and then do endpoints based on that?
12:24 < achow101> probably
12:24 < gmaxwell> luke-jr: does it also support one user with many wallets?
12:24 < wumpus> well as we determined above, a user may want to have access to multiple wallets, so a single user->wallet mapping just doesn't cut it, even if you want to add access control
12:24 < gmaxwell> what wumpus says.
12:24 < luke-jr> gmaxwell: the current code does not, but there's no reason the endpoints couldn't add that
12:24 < wumpus> I really think we should just start with endpoints as sipa says
12:24 < jnewbery> Isn't rpcuser deprecated anyway?
12:24 < jonasschnelli> Yes. Let's start with endpoint.. I'll can write it next week because I already did once...
12:24 < wumpus> I'm not going to NACK anything that makes progress though
12:24 < luke-jr> jnewbery: it's rpcauth
12:25 < sipa> jnewbery: rpcauth isn't
12:25 < jonasschnelli> *I can
12:25 < gmaxwell> jnewbery: rpcuser is, but this is rpcauth (rpcuser doesn't even have multiple users)
12:25 < jtimon> jnewbery: is rpcuser deprecated? since when?
12:25 < gmaxwell> jtimon: a year?
12:25 < achow101> jtimon: it's deprecated since a long time ago
12:25 < gmaxwell> it prints out a notice!
12:25 < wumpus> rpcuser is deprecated, people are encouraged to use either rpcauth or cookie auth
12:25 < wumpus> we won't remove it just yet ofcourse
12:26 < jtimon> oh, deprecated as in "we want to remove this", but it actually still works, no?
12:26 < paveljanik> 26 minutes...
12:26 < luke-jr> right
12:26 < achow101> yes
12:26 < wumpus> that is what deprecated means, yes
12:26 < jtimon> yeah, sorry
12:26 < instagibbs> paveljanik, 34 to go :P
12:26 < wumpus> paveljanik: what's so special about 26?
12:26 < sipa> jonasschnelli: any progress on GUI for database upgrade?
12:26 < wumpus> #topic GUI for database upgrade?
12:27 < gmaxwell> And we've wasted a perfectly good half hour. :P luke should put up patches and we can yell at him on github, but I think I would really prefer if the first cut does multiple wallets for a user... (if nothing else, that is the easiest thing to test)
12:27 < jonasschnelli> sipa: I sadly had only little time last and this week
12:27 < luke-jr> wumpus: 21 is half of 42
12:27 < jonasschnelli> sipa: I looked into it and wanted to ask you how I get the max size of a db cursor (to calc progress)
12:27 < sipa> jonasschnelli: it's not hard to estimate as txids are randomly distributed
12:27 < gmaxwell> so you just look at the txid...
12:27 < sipa> i can add code for that
12:27 < gmaxwell> they're done in order.
12:28 < gmaxwell> if it's at 0x01... then it's done 1/256 of it.
12:28 < sipa> txid -> arith_uint256 -> * 100/2^256
12:28 -!- Aaronvan_ [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
12:28 < jonasschnelli> Okay. The rest is simple (debug.log non newline [10%] progress / GUI splash screen progress with abort)
12:28 < wumpus> BTW: jnewbery rebased the label API pull (#7729), a lot of thanks for that
12:28 < gribble> https://github.com/bitcoin/bitcoin/issues/7729 | rpc: introduce label API for wallet by laanwj · Pull Request #7729 · bitcoin/bitcoin · GitHub
12:28 < sipa> jonasschnelli: i'll write code for the progress estimation
12:29 < jonasschnelli> sipa: Okay. Pass me over a commit and I'll finish the rest
12:29 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev
12:29 < jnewbery> wumpus: no problem. I wanted to test drive it :)
12:29 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 268 seconds]
12:30 < gmaxwell> wumpus: \O/ on the label rebase.
12:30 < wumpus> must have been a nightmare
12:31 < wumpus> any other topics?
12:31 < jonasschnelli> what about the rpc splitting,... has that been discussed so far?
12:32 < jnewbery> There had been quite a few refactors. I think the rebase was good, but reviewers should look out for anything that looks off
12:32 < jonasschnelli> signrawwithkey, etc.?
12:32 < wumpus> rpc splitting?
12:32 < achow101> the PRs I made to unfuck signrawtx and validateaddress
12:32 < wumpus> ah https://github.com/bitcoin/bitcoin/pull/10583
12:32 < wumpus> #10583
12:32 < jonasschnelli> achow101: yes. Those..
12:32 < gribble> https://github.com/bitcoin/bitcoin/issues/10583 | [RPC] Split part of validateaddress into getaddressinfo by achow101 · Pull Request #10583 · bitcoin/bitcoin · GitHub
12:33 < wumpus> #topic split off wallet functionality from mixed wallet/non-wallet RPC calls
12:33 < wumpus> or something
12:33 < achow101> #10570
12:33 < gribble> https://github.com/bitcoin/bitcoin/issues/10570 | [RPC] Split signrawtransaction into multiple distinct RPCs · Issue #10570 · bitcoin/bitcoin · GitHub
12:33 < wumpus> concept ack (haven't gotten around to reviewing anything)
12:33 < gmaxwell> I think those changes all make sense. Someone commented about breaking compatibility, but its for a new major version and it will be easy for callers to update their behavior.
12:33 < morcos> i'm here now.. haven't caught up on backlog
12:34 < sipa> it breaks compatibility with a never documented or advertized feature :)
12:34 < wumpus> well we could allow both, for one major version
12:34 < gmaxwell> Though we might want to rename the old calls at the same time. (a suggestion for discussion)
12:34 < sipa> (concatenating multiple tx hex strings, yuck)
12:34 < wumpus> wut?
12:34 < wumpus> ok, we're talking about different things
12:34 < instagibbs> aside from mixing wallet/nonwallet, what's the issue with validateaddress?
12:34 < gmaxwell> sipa: it was certantly advertised and known.
12:34 < wumpus> I mean validateaddress/getaddressinfo
12:34 < instagibbs> (or is that the issue)
12:34 < achow101> instagibbs: that's the issue
12:34 < sipa> wumpus: ah, i'm talking about signrawtransaction
12:34 < gmaxwell> wumpus: he's talking about the signraw split to create the combine call.
12:34 < wumpus> instagibbs: that is the issue
12:34 < instagibbs> ok, are we killing off getinfo then too:
12:34 < instagibbs> :)
12:35 < achow101> hopefully :D
12:35 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 240 seconds]
12:35 < wumpus> instagibbs: yes, but that's not the topic now
12:35 < gmaxwell> I would miss getinfo. all the other commands take more typing. :P
12:35 < wumpus> we're already confusing two things, let's add more!
12:35 < gmaxwell> wumpus: okay!
12:35 < instagibbs> eh, we're talking about blowing away validateaddress as is, sorry
12:35 < wumpus> gmaxwell: hey I have a pull that implements it client side
12:35 < gmaxwell> wumpus: lol. can you name the rpc call "gi" :P even less typing.
12:35 < jonasschnelli> IMO having a non wallet sign rawtx where priv keys are passed aroung in a shell over a possible TCP channel is not ideal.. but we already have it... I though instead of splitting it off, move it to the tool
12:35 < jonasschnelli> But I see the point with getting the UTXOs
12:36 < sipa> let's rename all RPCs to get*info... for example s/sendtoaddress/getnewpaymenttxid/
12:36 < gmaxwell> jonasschnelli: we need the UTXOs or its all awfulsauce.
12:36 < instagibbs> sipa, lol
12:36 < wumpus> #8843
12:36 < gribble> https://github.com/bitcoin/bitcoin/issues/8843 | rpc: Handle `getinfo` client-side in bitcoin-cli w/ `-getinfo` by laanwj · Pull Request #8843 · bitcoin/bitcoin · GitHub
12:36 < luke-jr> lol
12:36 < luke-jr> getsignedtransaction
12:36 < jonasschnelli> gmaxwell: I though the node RPC can spitout what you need to pass it into bitcoin-tx or so...
12:36 < jonasschnelli> but I know... very inconvinient
12:36 < luke-jr> getthistransactionbroadcast
12:36 < luke-jr> :p
12:36 < sipa> jonasschnelli: sure it can; it's just more convenient to not need that
12:36 < wumpus> eventually closed it because the only person responding was luke-jr and he kept arguing against it
12:37 < gmaxwell> jonasschnelli: and that adds steps to the process.. which is already long enough that it's prone to error.
12:37 < sipa> jonasschnelli: it's called listunspent
12:37 < jonasschnelli> It's just another source how people can shoot themselfs with exposing priv keys
12:37 < gmaxwell> luke-jr: why do you hate freedom?
12:37 < luke-jr> gmaxwell: !⁈
12:37 < gmaxwell> haha
12:37 -!- murch [6052501c@gateway/web/freenode/ip.96.82.80.28] has joined #bitcoin-core-dev
12:38 < sipa> jonasschnelli: but the functionality already exists, and i very much like removing it from the wallet (so people at least won't accidentally mix up privkey based operations with wallet stuff)
12:38 < instagibbs> sipa, ok I see the motivation there
12:38 < gmaxwell> anyways, I can just say that I have tried to stop using getinfo and failed. Mostly because typing getnetworkinfo getblockchainfo getfooooooooooinfo and then wading through a bunch of things when I want to see: How many connections, which block am I at, and what wallet am I running (which I can tell via the balance). :P just personal feedback.
12:39 < luke-jr> gmaxwell: I didn't even NACK it :o
12:39 < jonasschnelli> Yes... I guess that makes sense. I kinda hoped once we touch that we could move it away from the node into a sep. process
12:39 < luke-jr> gmaxwell: use the GUI for that! :p
12:39 < sipa> jonasschnelli: i have a vague proposal for that too, but it's more complicated
12:39 < gmaxwell> luke-jr: again, why do you hate freedom? :P
12:39 < sipa> jonasschnelli: and involves a new format for partially signed transactions...
12:39 < wumpus> gmaxwell: anyhow we can easily reopen and rebase that PR, bitcoin-cli hardly changed since then
12:40 < sipa> wumpus: ack
12:40 < jonasschnelli> sipa: that contains everything you need to sign?
12:40 < luke-jr> gmaxwell: make a shell alias to all the calls ;)
12:40 < gmaxwell> it's so much easier to have a signing blob thing post segwit. :(
12:40 < bitcoin-git> [bitcoin] laanwj reopened pull request #8843: rpc: Handle `getinfo` client-side in bitcoin-cli w/ `-getinfo` (master...2016_09_getinfo_clientside) https://github.com/bitcoin/bitcoin/pull/8843
12:40 < sipa> jonasschnelli: yes, contains amounts, change info, prevouts being spent, ...
12:40 < wumpus> yes, let's activate segwit
12:40 < gmaxwell> luke-jr: I don't like customizing my expirence of bitcoin too much because then I'll just patch around everything that stinks.
12:40 * sipa revives segnet
12:41 < jonasschnelli> sipa: That's also something we could re-use for the detatched signing standard (a.k.a hardware wallet standard)
12:41 < luke-jr> we could just leave getinfo how it is ;)
12:41 < gmaxwell> in any case, we're offtopic. I think that achow's PRs are all nice incremental improvements and we should take them (after review)
12:41 < sipa> jonasschnelli: pre-segwit however, it also needs to contain the full spent transactions :(
12:41 < luke-jr> bitcoin-cli -getinfo only handles 1 sortof-use-case, and leaves the other 2 supported use cases unaddressed
12:41 < sipa> gmaxwell: agree
12:41 < wumpus> after promising to deprecate it for years... yeah, of course....
12:41 < sipa> (disclaimer: achow101'w my intern this summer, i asked him to work on those)
12:42 < jonasschnelli> sipa: [full spent transactions], I guess that's okay.
12:42 < sipa> jonasschnelli: bitcoind unfortunately can't do that generically (you need the wallet for that)
12:42 < wumpus> getinfo is going away, there's no going back now
12:42 < luke-jr> let's add a getallinfo then
12:42 < luke-jr> /s
12:42 < wumpus> I've exactly documented what information you can find on what get*info command
12:42 < gmaxwell> I am fine with it going away, but I don't believe we replaced it as well as we thought we did.
12:43 < wumpus> and the client-side getinfo is ther for user friendlyness, if people want it
12:43 < sipa> i have no problem with removing it, with or without 8843
12:43 < luke-jr> wumpus: it doesn't work in the debug window
12:43 < sipa> i'm sure i'll curse a bit that getinfo isn't around anymore, and then change my habits
12:43 < gmaxwell> sipa: I tried blocking it, you won't. the replacements right now are not usable.
12:43 < luke-jr> :/
12:43 < gmaxwell> But thats okay, wumpus suggestion would be fine, though luke has a point about the debug console.
12:43 < wumpus> luke-jr: isn't all the information in the debug window *without* typing anything?
12:43 < achow101> removing getinfo will mess with a ton of things that use getinfo for basic rpc connection checking too...
12:44 < wumpus> achow101: and you're starting to bring that up *now*?
12:44 < achow101> but I think it should be removed anyways
12:44 < sipa> let's merge the getuptime rpc thing
12:44 < gmaxwell> wumpus: if it isn't we should make it. problem solved.
12:44 < bitcoin-git> [bitcoin] ryanofsky opened pull request #10605: Add AssertLockHeld assertions in CWallet::ListCoins (master...pr/listlock) https://github.com/bitcoin/bitcoin/pull/10605
12:44 < instagibbs> achow101, move to dumpprivkey obv
12:44 < wumpus> gmaxwell: the first tab of the debug window pretty much shows everything, and indeed, if it isn't it could be added
12:44 < sipa> oh, another topic: non-hardened key derivation
12:44 < achow101> wumpus: well most of those things are old website scripts that were written once and never touched again by the authors
12:44 < instagibbs> sipa, ACK
12:44 < gmaxwell> luke-jr: I think ^ is how we should handle the gui. (also important to make it copy/pasteable if it isn't.)
12:45 < wumpus> I'm really disappointed that years after deciding to deprecate getinfo we're still having this discussion
12:45 < luke-jr> wumpus: lol maybe :D
12:45 < wumpus> anyhow next topic
12:45 < gmaxwell> wumpus: sometimes you have to try things out to know their effects completely!
12:45 < wumpus> #topic non-hardened key derivation
12:45 < sipa> so
12:45 < sipa> non-hardened key derivation has many use cases in addition to hardened
12:46 < jonasschnelli> I guess NicolasDorier made good work there
12:46 < wumpus> achow101: we should carefully note it in the release notes of course
12:46 < sipa> however, they also have a gaping wide security hole when child private keys are exposed
12:46 < wumpus> achow101: we could even make getinfo fail with a custom message
12:46 < instagibbs> that plus dumpwallet will give you sads
12:46 < sipa> thus, suggestion: allow a new wallet to be created with either harderned or unhardened keys
12:46 < sipa> when you choose unhardened, dumpprivkey is disabled
12:46 < instagibbs> xpub is only accessible through dumpwallet right now AFAIK
12:46 < achow101> wumpus: returning null would probably not mess with anything
12:46 < sipa> (but dumpmasterkey or whatever is still available)
12:47 < achow101> sipa: there is no dumpmasterkey
12:47 < jonasschnelli> instagibbs: dumpprivkey must be disabled anyways
12:47 < luke-jr> sipa: I'd want to be able to mix them..
12:47 < jonasschnelli> achow101: dumpmasterkey must be added
12:47 < instagibbs> luke-jr, .... why
12:47 < gmaxwell> this is a lot more interesting with multiwallet support in place, since the cases where you want that are mostly secondary wallets (like incoming payments with keys generated by your webserver)
12:47 < wumpus> dumpmasterkey, isn't there a PR for that?
12:47 < jonasschnelli> no
12:47 < luke-jr> instagibbs: to generate reusable payment tokens
12:47 < sipa> luke-jr: i guess that's fine; just disable dumpwallet, and dumpprivkey selectively for keys derived in a non-hardened fashion
12:47 < gmaxwell> sipa said or whatever for a reason. :P
12:47 < achow101> jonasschnelli: I'm stil lwaiting #9504
12:47 < gribble> https://github.com/bitcoin/bitcoin/issues/9504 | [RPC] dumpmasterprivkey command by achow101 · Pull Request #9504 · bitcoin/bitcoin · GitHub
12:48 < jonasschnelli> oh...
12:48 < luke-jr> sipa: sgtm
12:48 < wumpus> see, I wasn't crazy
12:48 < gmaxwell> wumpus: well, technically it doesn't prove that...
12:48 < jonasschnelli> achow101: how can I not be aware of that PR... sorry for the missinfo
12:48 < wumpus> gmaxwell: true
12:48 < sipa> getmissinfo
12:48 < jonasschnelli> :/
12:48 < jonasschnelli> (I even commited on the PR ^^)
12:48 < jonasschnelli> commented
12:49 < wumpus> getmisinfo would really fit with the spirit of the times
12:49 < gmaxwell> Anyways, I think that sounds okay. These applications will likely need a couple of extra RPCs too. no need to design here however. (e.g. it will need to export the extended public key.)
12:49 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev
12:49 < sipa> not much more to say about the topic - just pointing out that if we disable dumping child private keys, my concern with non-hardered derivation largely goes away
12:49 < wumpus> anyhow, no problem with non-hardened key support
12:49 < jonasschnelli> I guess with allowing xpub derivation, flexible keypath would be welcome..
12:50 < jonasschnelli> People want to use BIP44
12:50 < wumpus> as an option, not as default
12:50 < gmaxwell> w/ disabling and it not being a default thing. sounds great to me.
12:50 < achow101> sgtm
12:50 < jonasschnelli> Yes. Would be a great change...
12:50 < gmaxwell> also so long as we do the extra rpcs to make it actually useful (like extract the extended public keys, and whatever else is needed to handle an external address generator.)
12:51 < jonasschnelli> I guess keypool handling would be much simpler with xpub derivation
12:51 < gmaxwell> I doubt it?
12:51 < instagibbs> I think it's the same-ish
12:51 < jonasschnelli> Why would you need a keypool with xpub derivation?
12:51 < gmaxwell> for scanning.
12:51 < jonasschnelli> you derive when you need
12:51 < instagibbs> you have master seed already, you can already do that
12:51 < jonasschnelli> you always derive the lookup window in mem
12:51 < gmaxwell> you need to scan forward to know when you're paid.
12:52 < jonasschnelli> +1000 keys in mem should be ackish
12:52 < jonasschnelli> (pubkeys)
12:52 < gmaxwell> yes, the keypool doesn't technically need to be saved on disk, though it may be faster to do so then to generate thousands of addresses at every wallet load.
12:52 < gmaxwell> but otherwise I think its the same.
12:52 < jonasschnelli> I'd say loading the keypool at wallet load takes almost the same (or longer)
12:53 < jtimon> sipa: why would people want to dump child private keys?
12:53 < gmaxwell> okay, this would make the change much more intrusive then I think.
12:53 < instagibbs> jtimon, you want to stop them from doing so
12:53 < gmaxwell> jtimon: same reason some people eat paste. (they don't know better)
12:53 < jonasschnelli> heh
12:53 < sipa> jonasschnelli: the reason creating new keys is slow, is because we flush them all individually to disk
12:54 < wumpus> yes...
12:54 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt]
12:54 < jtimon> gmaxwell: I see
12:54 < jonasschnelli> Yes. In-mem non-bdb "keypool" would be much faster
12:54 < gmaxwell> that flushing is not required anymore with hdwallets regardless, I believe.
12:54 < sipa> gmaxwell: exactly
12:55 < jonasschnelli> With xpub derivation, would there be really a need to write the pool to disk? I doubt it
12:55 < gmaxwell> (we will still need a flush to know how many we've given out...)
12:55 < wumpus> gmaxwell: good point!
12:55 < jonasschnelli> All you need is the seed
12:55 < gmaxwell> jonasschnelli: no, not a need, though why result in more distinct codepaths? I would expect it to also make loading faster.
12:55 < achow101> you just need to write the path of the most recent key
12:55 < jonasschnelli> gmaxwell: yes. thats a good point
12:56 < jonasschnelli> achow101: Yes.
12:56 < gmaxwell> achow101: yes, every new address still needs to do a wallet flush. But on bulk key creation we do not need a flush for each operation.
12:56 < gmaxwell> e.g. starting up for the first time with a keypoool of 10000.
12:58 < wumpus> two minutes to go
12:58 < gmaxwell> jonasschnelli: did you ever get around to implementing the change where any key that is noticed used on the blockchain marks all the earlier keys in its chain as used?
12:58 < jonasschnelli> gmaxwell: HD restore?
12:59 < jonasschnelli> https://github.com/bitcoin/bitcoin/pull/10240
12:59 < gmaxwell> it's part of your restore pr? okay.
12:59 < jonasschnelli> That makes all keys as used up to the one found
12:59 < gmaxwell> great.
12:59 < jonasschnelli> It can even temp. halt the sync if you prune, etc.
12:59 < jonasschnelli> Needs overhaul and rebase...
12:59 < jonasschnelli> It's already (too) big
12:59 < jonasschnelli> ryanofsky gave me a hard time
13:00 < sipa> #getmeetingendinfo
13:00 < wumpus> #endmeeting
13:00 < lightningbot> Meeting ended Thu Jun 15 20:00:12 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
13:00 < lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-15-19.00.html
13:00 < lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-15-19.00.txt
13:00 < lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-15-19.00.log.html
13:02 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev
13:04 -!- lichtamberg_ [~Adium@2a02:8388:2001:8600:2d41:72a9:750f:966e] has quit [Quit: Leaving.]
13:07 < gmaxwell> wumpus: sorry to bog you with the getinfo stuff. :(
13:08 < wumpus> gmaxwell: yes, no problem, don't mind resurrecting #8843
13:08 < gribble> https://github.com/bitcoin/bitcoin/issues/8843 | rpc: Handle `getinfo` client-side in bitcoin-cli w/ `-getinfo` by laanwj · Pull Request #8843 · bitcoin/bitcoin · GitHub
13:09 < wumpus> gmaxwell: just don't want to talk with luke-jr about it :p
13:09 < gmaxwell> I think the main challenges for it in an interactive setting is that the replacements spam the screen with information which is not what I'm interested in, and it takes about three commands to check on them. Thats not reason to not get rid of getinfo at all, and I support getting rid of it, but think we need something like your PR. I'll give it a look.
13:09 < gmaxwell> haha
13:09 < gmaxwell> Well, he does hate freedom, after all.
13:09 < sipa> gmaxwell: perhaps you should write a getmyinfo.sh with just queries for the things you're typically interested in :)
13:09 < wumpus> I mean the reason for deprecating it is that we didn't want to add any more info to it, also it combines wallet and non-wallet information which doesn't work with multiwallet
13:10 < wumpus> sipa: I have tons of those scripts
13:10 < wumpus> e.g. to show # connections per interface in/out
13:11 < wumpus> getinfo's output isn't *that* useful
13:11 < luke-jr> maybe -getinfo should take a -getinfodata which can go in the config file, to tell it what data to fetch from what RPCs ;)
13:11 < wumpus> luke-jr: well in that case I'd prefer just a python script
13:11 < ProfMac> What git version introduced the file bitcoin/contrib/gitian-build.sh ?
13:12 < wumpus> oh wait I'm talking to luke-jr about getinfo
13:12 < luke-jr> :<
13:12 -!- owowo [ovovo@gateway/vpn/mullvad/x-orpjbholoumyozpv] has quit [Ping timeout: 260 seconds]
13:12 < morcos> at the risk of irritating everyone, i use getinfo all the time
13:13 < gmaxwell> sipa: so then I'm happy while typical users suffer, I've tried to avoid doing that.
13:13 < achow101> ProfMac: check the history in github
13:13 < ProfMac> morcos, yep, me too.
13:13 < gmaxwell> same reason I don't use the ncurses thing that I like.
13:13 < luke-jr> gmaxwell: throw it in contrib/?
13:13 < wumpus> e.g. scripts like this work, it's not a work of art but meh https://gist.github.com/laanwj/27241686db0db712cd94f692a491ad78
13:14 < wumpus> typical users may have very different wants, though
13:14 < ProfMac> achow101, I've been in Google, github, for days. Thought someone would just actually answer the question.
13:14 < instagibbs> I've been weaning myself off of getinfo
13:15 < wumpus> in any case, I'm all for keeping -getinfo client-side in bitcoin-cli, I'm rebasing 8843
13:15 < achow101> ProfMac: it was introduced in this commit: https://github.com/bitcoin/bitcoin/commit/eda4cfb992d140c8471f6cc2bf16c4a106439225
13:16 < Chris_Stewart_5> Why don't we return fee estimation information in sat/byte?
13:16 < ProfMac> Thanks, achow101
13:16 < jonasschnelli> gmaxwell: Yes. 20 is to little...
13:16 < instagibbs> ProfMac, you can always `git log -p -M --follow -- file/path` to track those kind of questions
13:16 < jonasschnelli> I guess I took BIP44 for a start.. but it's a single const
13:17 < instagibbs> Chris_Stewart_5, because legacy, mostly
13:17 -!- owowo [ovovo@gateway/vpn/mullvad/x-mscexuilulryezqa] has joined #bitcoin-core-dev
13:17 < wumpus> instagibbs: hah, 99% of the time that's the right answer to any 'why' question
13:18 < Chris_Stewart_5> I suppose that is hard to change too for fear of people paying large fees if they don't get the memo
13:19 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has quit [Ping timeout: 246 seconds]
13:20 < wumpus> once a custom is ingrained, it usually turns out the cost and risk of change doesn't weigh up against any possible benefits
13:21 < ProfMac> Thanks instagibbs. I'm just spinning up to speed with git. I made a deterministic vm description and a Ubuntu preseed file de novo and am now making a local git repository, then spinning up the deterministic bitcoin build. Lots of coffee and new thought patterns.
13:21 < instagibbs> Any new interface certainly should be sat/byte
13:21 < Chris_Stewart_5> ^
13:21 < wumpus> and then you have multiple different conventions on different RPC calls
13:23 -!- RubenSomsen [~RubenSoms@1.217.138.142] has quit [Ping timeout: 268 seconds]
13:24 < sipa> nanobitcoins per Hart
13:25 < wumpus> some of the output of getinfo is also blatantly deceptive by now, e.g. it shows one proxy, while every network can have its own proxy
13:25 < sipa> do we still show hashrate?
13:25 < wumpus> other things are only marginally useful; how many times have you wanted to know the the wallet version?
13:26 < wumpus> or "testnet":false, which is also false for regtest
13:26 < wumpus> sipa: no, that seems to be gone :
13:27 < instagibbs> new interface meaning RPC2.0 or whatever... not new calls
13:27 < gmaxwell> To be clear I am joking about luke hating freedom.
13:28 < wumpus> oh, I thought you were serious!
13:29 < cfields> jonasschnelli / gmaxwell / sipa: https://pastebin.com/raw/eKhEiJR3
13:29 < wumpus> https://github.com/bitcoin/bitcoin/pull/8843 rebased
13:29 < jonasschnelli> cfields: ahh.. that look like a beautiful girl
13:29 < cfields> :)
13:29 < jonasschnelli> cfields: I expected a code diff at the bottom.... :)
13:30 < cfields> guess that's what you had in mind, then?
13:30 < cfields> heh, 1 sec
13:30 < jonasschnelli> Yes. Have you tried to run it with the GUI?
13:30 < cfields> it's per-thread/per-mutex/per-lock
13:30 < cfields> no, just simple tests so far
13:30 < cfields> something about threadnames are busted, and it's not sorted yet. and the only threshold for display is that the time is non-zero
13:31 < jonasschnelli> To bed my wife calls me to bed.. I'll play with the code soon
13:31 < cfields> other than that, seems to work as expected
13:31 < cfields> ok, i'll push to a branch
13:31 < jonasschnelli> thanks
13:31 < cfields> np, nnite
13:31 < wumpus> cfields: nice!
13:32 -!- Giszmo [~leo@ip-140-233.219.201.nextelmovil.cl] has joined #bitcoin-core-dev
13:34 < cfields> https://github.com/theuni/bitcoin/commit/be49a294a240ec81a901af1aaabbba2172d38dc1
13:35 < cfields> jonasschnelli: for tomorrow ^^
13:43 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Quit: mining]
13:43 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Ping timeout: 260 seconds]
13:46 -!- lichtamberg_ [~Adium@2a02:8388:2001:8600:78d2:9597:f7c9:c336] has joined #bitcoin-core-dev
13:48 < cfields> er, that should s/locked/contended/ everywhere. it's showing any non-zero time taken to grab a lock.
13:50 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev
13:50 -!- RubenSomsen [~RubenSoms@1.217.138.142] has joined #bitcoin-core-dev
13:58 < bitcoin-git> [bitcoin] benma opened pull request #10607: scripted-diff: stop using the gArgs wrappers (master...gargs_wrappers) https://github.com/bitcoin/bitcoin/pull/10607
14:01 -!- goatpig [5a5c653a@gateway/web/freenode/ip.90.92.101.58] has quit [Ping timeout: 260 seconds]
14:07 -!- Aaronvan_ is now known as AaronvanW
14:08 -!- lichtamberg_ [~Adium@2a02:8388:2001:8600:78d2:9597:f7c9:c336] has left #bitcoin-core-dev []
14:10 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev
14:11 -!- cryptapus is now known as cryptapus_afk
14:11 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 240 seconds]
14:12 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Quit: mining]
14:12 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev
14:16 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Ping timeout: 240 seconds]
14:20 < ProfMac> pastebin or google docs?
14:20 -!- F-DK [57481815@gateway/web/freenode/ip.87.72.24.21] has joined #bitcoin-core-dev
14:22 -!- Giszmo [~leo@ip-140-233.219.201.nextelmovil.cl] has quit [Ping timeout: 268 seconds]
14:27 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev
14:29 -!- elkalamar [~elkalamar@84.126.69.179.dyn.user.ono.com] has joined #bitcoin-core-dev
14:30 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has joined #bitcoin-core-dev
14:32 -!- elkalamar [~elkalamar@84.126.69.179.dyn.user.ono.com] has quit [Remote host closed the connection]
14:33 -!- F-DK [57481815@gateway/web/freenode/ip.87.72.24.21] has quit [Quit: Page closed]
14:35 -!- Giszmo [~leo@ip-38-233.219.201.nextelmovil.cl] has joined #bitcoin-core-dev
14:36 < ProfMac> I am stuck trying to do a gitian build. I have made a Pastebin https://pastebin.com/eXPaQsLf
14:37 -!- talmai [~T@c-76-24-28-74.hsd1.ma.comcast.net] has quit [Ping timeout: 260 seconds]
14:39 -!- goatpig [68a01502@gateway/web/freenode/ip.104.160.21.2] has joined #bitcoin-core-dev
14:44 -!- Giszmo [~leo@ip-38-233.219.201.nextelmovil.cl] has quit [Read error: Connection reset by peer]
15:06 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]
15:10 -!- Dizzle [~dizzle@108.171.182.16] has quit [Quit: Leaving...]
15:10 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 240 seconds]
15:11 -!- murch [6052501c@gateway/web/freenode/ip.96.82.80.28] has quit [Ping timeout: 260 seconds]
15:25 -!- timothy [~tredaelli@redhat/timothy] has joined #bitcoin-core-dev
15:26 -!- nemgun [~nemgun@105.101.182.63] has joined #bitcoin-core-dev
15:27 -!- timothy [~tredaelli@redhat/timothy] has quit [Client Quit]
15:30 -!- marcoagner [~user@177.41.197.237] has quit [Ping timeout: 240 seconds]
15:34 -!- nemgun [~nemgun@105.101.182.63] has quit [Ping timeout: 240 seconds]
15:35 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev
15:38 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 240 seconds]
15:39 -!- jtimon [~quassel@117.29.134.37.dynamic.jazztel.es] has quit [Remote host closed the connection]
15:42 -!- marcoagner [~user@191.249.241.221] has joined #bitcoin-core-dev
15:46 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Ping timeout: 240 seconds]
15:46 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has joined #bitcoin-core-dev
15:52 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev
15:53 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev
15:59 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt]
15:59 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev
16:14 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has joined #bitcoin-core-dev
16:16 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 255 seconds]
16:18 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev
16:24 -!- chjj [~chjj@unaffiliated/chjj] has quit [Ping timeout: 268 seconds]
16:38 -!- chjj [~chjj@unaffiliated/chjj] has joined #bitcoin-core-dev
16:42 -!- elkalamar [~elkalamar@84.126.69.179.dyn.user.ono.com] has joined #bitcoin-core-dev
16:48 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt]
17:01 -!- abpa [~abpa@96-82-80-28-static.hfc.comcastbusiness.net] has quit [Quit: Textual IRC Client: www.textualapp.com]
17:14 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Remote host closed the connection]
17:15 -!- apll [~edgars@222.240.93.225] has quit [Ping timeout: 260 seconds]
17:17 -!- apll [~edgars@222.240.93.225] has joined #bitcoin-core-dev
17:21 -!- mol [~IRCIdent@unaffiliated/molly] has quit [Ping timeout: 245 seconds]
17:23 -!- mol [~IRCIdent@unaffiliated/molly] has joined #bitcoin-core-dev
17:46 -!- Chris_Stewart_5 [~chris@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 240 seconds]
17:47 < bitcoin-git> [bitcoin] gmaxwell opened pull request #10608: Add a comment explaining the use of MAX_BLOCK_BASE_SIZE. (master...size_comment) https://github.com/bitcoin/bitcoin/pull/10608
17:56 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has joined #bitcoin-core-dev
17:58 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has quit [Remote host closed the connection]
17:59 -!- dabura667 [~dabura667@p98110-ipngnfx01marunouchi.tokyo.ocn.ne.jp] has joined #bitcoin-core-dev
18:12 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-xwdvzdtgbfamkkho] has quit [Quit: Connection closed for inactivity]
18:27 < NicolasDorier> wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 instagibbs achow101 : Sorry to ping you all, but since some dev are in Tokyo around end of july for some conferences, I thought it can be a good idea to have such occasion to have 1 or 2 days of pure bitcoin core
18:27 < NicolasDorier> coding/review/idea sharing together. If you are interested, come in the channel ##tokyocore . My company Digital Garage can give us a nice place to go for coding. Note: I certaintly forgot some name, if you are contributor and wish to come, you are defintively welcome. The 25 June we freeze the list.
18:31 < cfields> NicolasDorier: thanks for arranging!
18:33 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 268 seconds]
18:39 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev
19:04 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
19:10 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 240 seconds]
19:18 < cfields> gitian builders: 0.14.2 detached sigs are up
19:21 -!- ProfMac [43c671dc@gateway/web/freenode/ip.67.198.113.220] has quit [Ping timeout: 260 seconds]
19:24 -!- elkalamar [~elkalamar@84.126.69.179.dyn.user.ono.com] has quit [Quit: Leaving]
20:28 -!- ProfMac [43c671dc@gateway/web/freenode/ip.67.198.113.220] has joined #bitcoin-core-dev
20:56 -!- RubenSomsen [~RubenSoms@1.217.138.142] has quit [Ping timeout: 240 seconds]
21:06 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has quit [Ping timeout: 246 seconds]
21:08 -!- Dyaheon [~Dya@a91-156-192-39.elisa-laajakaista.fi] has joined #bitcoin-core-dev
21:33 < cfields> NicolasDorier: I just rsvpd as I was going to be there already. Thanks again.
21:47 -!- ProfMac [43c671dc@gateway/web/freenode/ip.67.198.113.220] has quit [Ping timeout: 260 seconds]
21:57 < NicolasDorier> yay awesome
22:01 -!- harding [~harding@mail.dtrt.org] has quit [Remote host closed the connection]
22:37 -!- [Author] [~Author]@2401:a400:3200:5600:bad:d09:15:90d] has quit [Ping timeout: 276 seconds]
22:41 -!- [Author] [~Author]@2401:a400:3200:5600:bad:d09:15:90d] has joined #bitcoin-core-dev
22:42 -!- RubenSomsen [~RubenSoms@1.217.138.142] has joined #bitcoin-core-dev
23:07 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
23:11 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-ezafpzgerkxlyolo] has joined #bitcoin-core-dev
23:12 -!- AaronvanW [~AaronvanW@unaffiliated/aaronvanw] has quit [Ping timeout: 260 seconds]
23:29 -!- ClockCat [~CattieCat@75-109-228-219.stjocmtk01.res.dyn.suddenlink.net] has joined #bitcoin-core-dev
23:35 -!- goatpig [68a01502@gateway/web/freenode/ip.104.160.21.2] has quit [Quit: Page closed]
23:41 -!- SopaXorzTaker [~SopaXorzT@unaffiliated/sopaxorztaker] has joined #bitcoin-core-dev
23:44 -!- Cory [~Cory@unaffiliated/cory] has quit [Ping timeout: 260 seconds]
23:46 -!- BashCo [~BashCo@unaffiliated/bashco] has quit [Ping timeout: 240 seconds]
23:47 -!- Giszmo [~leo@pc-240-13-215-201.cm.vtr.net] has quit [Ping timeout: 240 seconds]
23:48 -!- ClockCat [~CattieCat@75-109-228-219.stjocmtk01.res.dyn.suddenlink.net] has quit [Quit: Leaving]
23:56 -!- Pasha [~Cory@unaffiliated/cory] has joined #bitcoin-core-dev
23:59 -!- Pasha is now known as Cory