--- Day changed Sat Apr 23 2016
00:13 -!- arowser_ [~quassel@106.120.101.38] has joined #bitcoin-core-dev
00:14 -!- arowser [~quassel@106.120.101.38] has quit [Ping timeout: 276 seconds]
00:23 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
00:29 -!- Alopex [~bitcoin@cyber.dealing.ninja] has quit [Remote host closed the connection]
00:30 -!- Alopex [~bitcoin@cyber.dealing.ninja] has joined #bitcoin-core-dev
00:34 < luke-jr> phantomcircuit: it's SSSE3 usually :/
00:34 < luke-jr> I had to build my glibc with -mno-ssse3 or stuff crashed
00:34 < phantomcircuit> luke-jr, ha
00:35  * luke-jr never figured out why Haswell's SSSE3 seems to be broken in 32-bit mode.
00:37 < GitHub187> [bitcoin] laanwj opened pull request #7927: Minor changes to dbwrapper to simplify support for other databases (master...2016_04_dbwrapper_modernization) https://github.com/bitcoin/bitcoin/pull/7927
00:39 < gmaxwell> you mean 'x32' right?
00:42 < luke-jr> no, regular 32-bit
00:42 < gmaxwell> weird.
00:42 < luke-jr> I suspect x32 would work
00:42 < luke-jr> my best guess so far has been pointer alignment stuff
00:42 < gmaxwell> I assume it's not haswell, but some GCC bug. 
00:42 < luke-jr> possible
00:42 < luke-jr> maybe Haswell is stricted on the spec than previous CPUs?
00:43 < sipa> i doubt the spec changed, but perhaps older cpus were more lenient with spec violations :)
00:43 < luke-jr> although it'd be a glibc *and* Mesa bug, as it turned up at least both implementations of memcpy (not sure why Mesa has a separate one, but oh well)
00:44 < luke-jr> or GCC, but I don't see how the compiler could be at fault for this
00:44 < luke-jr> IIRC it's inline asm
00:44 < luke-jr> (but I haven't looked at it in probably a year or so)
00:44 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has quit [Read error: Connection reset by peer]
00:50 -!- davec [~davec@cpe-24-243-251-52.hot.res.rr.com] has joined #bitcoin-core-dev
00:58 < phantomcircuit> luke-jr, that would explain why it does what i expect it to do
00:58 < luke-jr> ?
00:58 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has joined #bitcoin-core-dev
00:58 < phantomcircuit> gmaxwell, it seems like the only way to ensure sa isn't being violated would be to copy byte by byte
01:06 < sipa> phantomcircuit: void* is allowed to alias any pointer, i think
01:08 < luke-jr> in C, at least; not sure about C++ - it complains if you do it implicitly, at least
01:13 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-zprlrenqftfaswgo] has joined #bitcoin-core-dev
01:23 -!- PaulCape_ [~PaulCapes@204.28.124.82] has quit [Quit: .]
01:24 -!- PaulCapestany [~PaulCapes@204.28.124.82] has joined #bitcoin-core-dev
01:28 -!- frankenmint [~frankenmi@174-25-22-102.ptld.qwest.net] has joined #bitcoin-core-dev
01:34 -!- Thireus1 [~Thireus@vps-92.197.170.217.stwvps.net] has quit [Quit: Leaving.]
01:42 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has joined #bitcoin-core-dev
01:56 -!- p15x [~p15x@162.91.145.64.unassigned.bringover.net] has quit [Ping timeout: 260 seconds]
02:00 -!- KHCkjhv [~OUYOybgP@188.25.138.138] has joined #bitcoin-core-dev
02:08 -!- KHCkjhv [~OUYOybgP@188.25.138.138] has quit [Quit: Leaving]
02:21 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]
02:21 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has quit [Ping timeout: 260 seconds]
02:35 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has joined #bitcoin-core-dev
02:50 -!- arowser_ [~quassel@106.120.101.38] has quit [Ping timeout: 246 seconds]
02:50 -!- arowser [~quassel@106.120.101.38] has joined #bitcoin-core-dev
03:05 < sipa> i upgraded to xenial, did git -dfx, ccache -C, ./autogen.sh, ./configure, make -j5... and i get util.cpp:817:53: error: ‘COPYRIGHT_HOLDERS’ was not declared in this scope
03:06  * sipa tries again
03:07 < luke-jr> stale config.h somewhere?
03:07 < sipa> after git -dfx ?
03:07 < sipa> eh, git clean -dfx
03:09 < luke-jr> hm, assuming our gitignore is okay that should eliminate any
03:10 < sipa> git clean -dfx deletes all files not checked in
03:10 < sipa> including ignored files
03:10 < sipa> but also others
03:10 < sipa> so gitignore is not relevant
03:11 < luke-jr> hm
03:11 < sipa> i think it's working now...
03:12 < luke-jr> what changed?
03:12 < sipa> i did git clean -dfx again...
03:15 < luke-jr> strange
03:16 < NicolasDorier> btw I got also the same error yestersday
03:16 < luke-jr> I don't suppose either of you saved the config.h/log to compare?
03:16 < NicolasDorier> I can reproduce one sec
03:19 < NicolasDorier> luke-jr:  right now I have the error, what do you want I do to help fixing it ? 
03:19 < NicolasDorier> (I'm not linux wizard so step by step would be appreciated :p)
03:20 < luke-jr> NicolasDorier: ideally, copy the entire bitcoin/ dir somewhere, then re-do it successfully and compare
03:21 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has joined #bitcoin-core-dev
03:24 < NicolasDorier> k one sec.
03:39 < NicolasDorier> mmh, now getting another error after git clean -dfx... but that might be unrelated to the earrlier copyright error as I'm not on master branch right now
03:39 < NicolasDorier> https://usercontent.irccloud-cdn.com/file/BO3RdE0Q/
03:39 < NicolasDorier> will check out
03:47 < NicolasDorier> nevermind was a space in the path
03:47 < NicolasDorier> ah no
03:47 -!- arowser [~quassel@106.120.101.38] has quit [Quit: No Ping reply in 180 seconds.]
03:48 -!- arowser [~quassel@106.120.101.38] has joined #bitcoin-core-dev
04:04 < randy-waterhouse> recent commit 351abf9e035581e3320fbb72ec5b1fa452b09c2f (depends build) introduces following error http://pastebin.com/q5cP70y8 (and core dump) on bitcoin-qt
04:10 -!- AaronvanW [~ewout@172pc231.sshunet.nl] has joined #bitcoin-core-dev
04:10 -!- AaronvanW [~ewout@172pc231.sshunet.nl] has quit [Changing host]
04:10 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
04:14 < NicolasDorier> I could not reproduce the 'COPYRIGHT_HOLDERS' error after the first git clean -dfx. It compiled fine :/
04:18 -!- belcher [~user@unaffiliated/belcher] has joined #bitcoin-core-dev
04:18 < btcdrak> usually just a make clean;./autogen.sh solves this for me
04:35 -!- murch [~murch@p4FE38E39.dip0.t-ipconnect.de] has joined #bitcoin-core-dev
04:40 < luke-jr> NicolasDorier: ok, so now you have two dirs to compare, right?
04:40 < NicolasDorier> yes. one which compile and the other with the error
04:41 < NicolasDorier> but I wanted to replicate the bug wherer git clean -dfx does not solve the copyright error, it happened to me once as well
04:42 < luke-jr> NicolasDorier: diff -ur firstdir seconddir >diff
04:42 < luke-jr> then pastebin the diff file
04:42 < NicolasDorier> ok
04:42 < NicolasDorier> one sec
04:45 < NicolasDorier> luke-jr it has more than 512kb does not see to be able to copy
04:46 < luke-jr> dropbox?
04:46 < NicolasDorier> one sec yes I'll find a way
04:48 < NicolasDorier> luke-jr: https://aois.blob.core.windows.net/public/diff.txt
04:48 < NicolasDorier> I need to run
04:52 < NicolasDorier> in the bitcoin folder does not compile bitcoin2 does
04:53 < sipa> are you sure the source code is the same across the two?
04:53 < sipa> the dependencies are different, which is a bit suspicious
04:55 < luke-jr> -rpcserver.h:
04:55 < luke-jr> +rpc/server.h:
04:55 < luke-jr> definitely not the same..
04:56 < sipa> one looks 0.12, the other master
05:30 -!- jarret [~jarret@162.216.46.40] has quit [Ping timeout: 246 seconds]
05:37 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev
05:41 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit]
05:44 -!- randy-waterhouse [~kiwigb@opentransactions/dev/randy-waterhouse] has quit [Quit: Leaving.]
05:48 -!- BashCo [~BashCo@unaffiliated/bashco] has quit []
05:53 -!- airmac [~cheers@168.1.79.118] has joined #bitcoin-core-dev
05:54 < airmac> anyone wanna trade i got 29g of bitgold with trade for bitcoin intrested msg me for trade we can use escrow if you want
05:54 < sipa> airmac: not here
05:54 < sipa> #bitcoin-otc or similar channels please
06:11 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Remote host closed the connection]
06:30 -!- murch [~murch@p4FE38E39.dip0.t-ipconnect.de] has quit [Remote host closed the connection]
06:30 < nkuttler> sipa: already banned from everywhere
06:31 < sipa> nkuttler: ok
06:37 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has quit [Quit: Leaving.]
07:00 -!- frankenmint [~frankenmi@174-25-22-102.ptld.qwest.net] has quit [Remote host closed the connection]
07:12 -!- frankenmint [~frankenmi@174-25-22-102.ptld.qwest.net] has joined #bitcoin-core-dev
07:23 -!- AaronvanW [~ewout@172pc231.sshunet.nl] has joined #bitcoin-core-dev
07:23 -!- AaronvanW [~ewout@172pc231.sshunet.nl] has quit [Changing host]
07:23 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has joined #bitcoin-core-dev
07:41 -!- jarret [~jarret@162.216.46.36] has joined #bitcoin-core-dev
07:51 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev
07:51 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit]
07:58 < airmac> anyone wanna trade i got 29g of bitgold with trade for bitcoin intrested msg me for trade we can use escrow if you want\
07:58 < belcher> wrong channel, go to #bitcoin-otc or similar
07:59 -!- mode/#bitcoin-core-dev [+o sipa] by ChanServ
07:59 -!- mode/#bitcoin-core-dev [+b *!*cheers@168.1.79.*] by sipa
07:59 -!- airmac was kicked from #bitcoin-core-dev by sipa [go elsewhere]
08:00 -!- mode/#bitcoin-core-dev [-o sipa] by ChanServ
08:06 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev
08:20 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev
08:24 -!- TomMc [~tom@unaffiliated/tommc] has joined #bitcoin-core-dev
08:59 -!- BCBot_ [~BCBot@pc-5305.ethz.ch] has joined #bitcoin-core-dev
09:46 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Quit: laurentmt]
09:56 < Chris_Stewart_5> How is this a valid test case? Shouldn't the 0x00 be pushed onto the stack by just an OP_0? 
09:56 < Chris_Stewart_5> ["0x01 0x00", "1", "MINIMALDATA"]
09:58 < sipa> link?
09:58 < sipa> ah, i get it 
09:59 < sipa> no, OP_0 pushes the empty vector onto the stack, not 0x00
09:59 < Chris_Stewart_5> ooo. 
10:00 < Chris_Stewart_5> Tricky :-)
10:01 < Chris_Stewart_5> Is that what this comment is saying? I didn't really understand the comment
10:01 < Chris_Stewart_5> ["Numeric minimaldata rules are only applied when a stack item is numerically evaluated; the push itself is allowed"]
10:01 < Chris_Stewart_5> https://github.com/bitcoin/bitcoin/blob/master/src/test/data/script_tests.json#L598
10:03 < sipa> yes, that's exactly what it's saying
10:04 < sipa> MINIMALDATA includes 2 rules: for every push operation, the shortest possible form must be used 2) whenever a stack element is interpreter as a number, it must be in the shortest possible notation
10:04 < sipa> here, 0x00 is pushed onto the stack with a push operation that satisfies 1)
10:05 < sipa> 2) is not applicable, because it is not being interpreted as a number
10:09 < Chris_Stewart_5> Ok that makes sense. Thanks sipa. 
10:47 -!- molz [~molly@unaffiliated/molly] has quit [Ping timeout: 268 seconds]
10:52 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
11:17 -!- Samdney [~Samdney@dyn-ant666999.hawo.ipv6.uni-erlangen.de] has joined #bitcoin-core-dev
11:54 -!- molz [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
11:57 -!- moli [~molly@unaffiliated/molly] has quit [Ping timeout: 252 seconds]
12:20 -!- supasonic [~supasonic@172-11-188-117.lightspeed.rcsntx.sbcglobal.net] has joined #bitcoin-core-dev
12:26 < luke-jr> oops, we released 0.12.1 with no practical way to mine (GBT hasn't been updated for BIP 9) :x
12:26 < luke-jr> well, practical and correct I guess
12:26 < luke-jr> should I de-bundle BIP 9 GBT stuff from segwit, or just figure wait for that?
12:32 < luke-jr> not sure there's a way to support 0.12.1 properly, so basically means the softfork "needs" to be delayed?
12:32 -!- JackH [~Jack@79-73-185-113.dynamic.dsl.as9105.com] has joined #bitcoin-core-dev
12:32 < sipa> how do you mean?
12:33 < sipa> it's already being used by miners
12:33 < luke-jr> sipa: there's no way for GBT clients to know what the rules are
12:33 < sipa> there's no need for that if they're connecting to a trusted bitcoind
12:34 < luke-jr> GBT requires it
12:34 < luke-jr> since it could very well be needed
12:34 < luke-jr> eg, with segwit the handling of txid/hash
12:35 < sipa> so let's say there is a softfork that adds UTXO commitments
12:35 < sipa> what you going to do? provide the entire utxo set in the GBT response?
12:35 < luke-jr> I could hack libblkmaker to understand that the new "version" can be treated as the previous ones, but this will break as soon as that bit is reused
12:35 < sipa> your view of reality is a dream
12:35 < sipa> GBT is not being used the way you envision it
12:36 < luke-jr> in such a case, the GBT client needs to be aware the commitment is required and where
12:36 < luke-jr> so it can place the commitment from the local bitcoind in the right part of the block
12:36 < sipa> or it could just not bother, and use a full node to construct the block proposals, which is in a much better position to do so, and already knows everything necessary (apart from coinbase outputs)
12:36 < luke-jr> sipa: I wouldn't have even noticed this if it wasn't.
12:37 < sipa> but no need to have thing discussion again
12:37 < sipa> i'm fine with reporting what bip9 deployments are active and available in GBT
12:37 < sipa> but please don't go claim that it's not possible to mine without
12:38 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has joined #bitcoin-core-dev
12:38 < sipa> there is no real world mining now that needs such information
12:38 < luke-jr> per spec, GBT clients cannot work with 0.12.1 as-is today.
12:38 < luke-jr> and de facto miners implementing it do not work with it
12:39 < sipa> why not? all deployed GBT clients in the world work fine with it
12:39 < luke-jr> I'm just trying to find a clean way to fix this real-world problem miners are encountering
12:39 < luke-jr> nope
12:39 < sipa> how so?
12:39 < luke-jr> the version field is unrecognised, so they throw an error
12:41 < sipa> how do you mean?
12:41 < luke-jr> https://github.com/bitcoin/libblkmaker/blob/master/blkmaker_jansson.c#L273
12:42 < sipa> ok, so the software needs to be updated
12:43 < sipa> i see what you're saying
12:43 -!- frankenmint [~frankenmi@174-25-22-102.ptld.qwest.net] has quit [Remote host closed the connection]
12:44 < sipa> though i think most gbt clients just pass through the version field, and they would be fine, because bitcoind's block template always corresponds to a valid block
12:44 < luke-jr> those would be fine now (albeit not per spec), but they'll break worse with segwit
12:45 < luke-jr> I suppose libblkmaker can treat the case of (currentsoftforkversion && !bip9)
12:45 < luke-jr> as long as the next softfork uses BIP9 GBT changes, that would protect against bit reuse
12:45 < sipa> i think it may be best to disentangle the segwit changes from bip9 changes to gbt
12:45 < luke-jr> does that sound reasonable?
12:45 < luke-jr> ok
12:46 < luke-jr> so today I will focus/prioritise 1) finish revising GBT BIP changes, 2) split BIP9 GBT code from segwit, 3) update libblkmaker to detect 0.12.1 situation
12:48 < sipa> so i think we at least agreed that gbt should report the list of currently active rules, probably by name
12:48 < sipa> (trying to remember the discussion we had before)
12:49 < luke-jr> version: bits the server likes you to set, vbavailable: {name:bit} mapping array the server allows you to set, vbrequired: bits the server demands you to set
12:50 < sipa> apart from that there were two other functions: 1) policy about which bits the server allows you to set/not set, and a suggestion 3) providing information about linkage between available bits and the deployment they correspond to
12:50 < sipa> i'm a bit unconfortable with the last thing, as it's only useful in the case the client trusts the server
12:51 < luke-jr> so libblkmaker will have a special case for 0x20000000 && !vbavailable, to handle 0.12.1 correctly
12:51 < sipa> how will you deal with 0x20000001?
12:52 < luke-jr> hm, I'm confused
12:53 < luke-jr> I pulled 0x20000000 out of a recent block to be lazy, but that indicates no softforks, doesn't it? :x
12:54 < luke-jr> oh, the begin time..
12:54 < sipa> yes, after may 1st it will set 0x20000001
12:55 < luke-jr> so I need to check for & 0xFFFFFFFE == 0x20000000
12:55 < sipa> cfields: ^ relevant discussion
12:56 < luke-jr> sipa: re vbrequired: the server can always reject submissions that clear bits it insists on, so this merely informs clients in advance so they can choose not to use that server
13:02 < sipa> luke-jr: right, no problem with saying "these bits you must set, and these you may set"
13:03 < sipa> i'm unconvinced about providing information about what bits corresponds with whicj deployment, as that is ourely informational with nonrelevance to what answers are acceptable
13:04 < sipa> *which *purely
13:04 < sipa> *no relevance
13:05 < luke-jr> in the sense that you wish to discuss it further before proceeding that direction, or just a comment?
13:05 < sipa> like, we also don't provide information about what the coinbase maturity is, while it could just as easily change in a softfork
13:06 < luke-jr> things like that are why GBT says clients shouldn't make assumptions about block versions they don't understand ;)
13:06 < sipa> my point is that this mapping information is going beyond the scope of the GBT call, and if clients which tovparticipate in voting, well, then they are responsible for knowing what means what
13:06 < sipa> *wish to participate
13:06 < luke-jr> "what means what" depends on blockchain context they lack
13:06 < sipa> nno, it does not
13:06 < sipa> it's in the bips
13:07 < sipa> ah, there is one piece of context information they lack
13:07 < sipa> mediantimepast
13:08 < luke-jr> also testnet vs mainnet, but that's admittedly less important
13:08 < sipa> i see
13:09 < luke-jr> possibly becomes important if we try to use GBT for sidechains (but I'm not convinced that makes sense)
13:09 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has quit [Quit: Leaving.]
13:10 -!- frankenmint [~frankenmi@174-25-22-102.ptld.qwest.net] has joined #bitcoin-core-dev
13:16 -!- frankenmint [~frankenmi@174-25-22-102.ptld.qwest.net] has quit []
13:23 < luke-jr> ok, step 1 done I hope: https://github.com/bitcoin/bips/pull/365
13:25 -!- zlover [uid34185@gateway/web/irccloud.com/x-wplywvjztxkodvkc] has joined #bitcoin-core-dev
13:27 < btcdrak> luke-jr: your changes dont render properly https://i.imgur.com/Qmwqh0k.png
13:28 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has joined #bitcoin-core-dev
13:29 < instagibbs> sipa, "Otherwise, going from WITNESS->P2SH+WITNESS would be possible, which is not a softfork." I don't grok this. Could you briefly explain it?
13:30 < luke-jr> btcdrak: looks like none of the GBT stuff does, due to GitHub migration
13:31 < luke-jr> https://help.github.com/articles/supported-mediawiki-formats/#unsupported-mediawiki-syntaxes :/
13:33 < luke-jr> btcdrak: fixed by avoiding the templates
13:35 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has joined #bitcoin-core-dev
13:35 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has quit [Client Quit]
14:04 < sipa> instagibbs: say you have a normal P2SH (not witness) output being spent
14:05 < sipa> oh, no
14:06 < sipa> instagibbs: say you have an invalid witness-in-p2sh spending
14:07 < sipa> with just witness enabled, that's valid
14:07 < instagibbs> wait what, is it invalid or invalid?
14:07 -!- arubi [~ese168@unaffiliated/arubi] has joined #bitcoin-core-dev
14:07 < instagibbs> err valid
14:09 < sipa> say you have a p2sh output, and a corresponding input to spend it, which provides redeemscript which is a witness script, and then witness program that corresponds to it, but is invalid (say it's  wrong signature)
14:09 < kanzure> why would enabling witness make that valid?
14:10 < kanzure> wrong signatures should always be wrong
14:10 < sipa> if you evaluate that with WITNESS, it is valid, because it's a p2sh outout, which is an anyone can spend
14:11 < sipa> if you evaluate with WITNESS+P2SH, the P2SH evaluation happens, which triggers the witness logic, which is invalid
14:11 < sipa> eh
14:11 < sipa> that's all true, but not the reason why it's not a softfork
14:12 < kanzure> oops just checked context. didn't see the question re: the comment. nevermind.
14:15 < instagibbs> "say you have a p2sh output" are you saying this is a p2wsh or something similar
14:16 < instagibbs> or just a plain p2sh like today
14:17 < instagibbs> oh i see
14:18 < instagibbs> I'm not quite getting the comment then, although I agree with what you're saying here
14:20 < sipa> let me think again
14:20 < instagibbs> it sounds like you're saying SOFTFORK1 this is valid, then adding SOFTFORK2 it's now invalid
14:20 < instagibbs> which is def of softfork?
14:21 < sipa> there is a similar rule for cleanstack, and that one i can explain
14:21 < sipa> no, the problem is that when validation with  flags1 fails, but with flags1+flags2 succeeds, then it is not a aoftfork
14:21 < sipa> and all flags should be softforks
14:23 < sipa> mempool validations relies on that
14:24 < instagibbs> I'm clearly missing something important here.
14:24 < instagibbs> probably what the flag semantics are or something
14:26 -!- achow101 [~achow101@pool-108-2-202-49.phlapa.fios.verizon.net] has joined #bitcoin-core-dev
14:27 < instagibbs> oh... it's just talking about the actual implementation and program flow...
14:28  * instagibbs smacks forehead 
14:28 < sipa>  so, for ok, for WWITNESS and CLEANSTACK
14:28 < sipa> any witness spend would fail CLEANSTACK validation
14:28 < sipa> because it is just pushes
14:29 < sipa> so it clearly does not satisfy CLEANSTACK
14:29 < instagibbs> I misunderstood the comment, as I thought
14:29 < sipa> unless when WITNESS is enabled, then we ignore the CLEANSTACK rule
14:30 < sipa> thus, a change in flags from CLEANSTACK to CLEANSTACK+WITNESS is not a softfork
14:30 < instagibbs> I read it as "BIP141 by itself is somehow a hardfork"
14:30 < sipa> oh, no
14:30 < instagibbs> totally get it now
14:30 < sipa> we just make CLEANSTACK without WITNESS invalid, so such a change can never occur
14:30 < sipa> ok!
14:31 < luke-jr> sipa: does BIP68+112+113 have a VB name?
14:31 < sipa> luke-jr: csv
14:31 < luke-jr> k
14:33 < instagibbs> thanks for the explanation!
14:33 < luke-jr> sipa: am I correct in understanding that the block time itself is entirely unrelated to the meaning of bits?
14:34 -!- achow101 [~achow101@pool-108-2-202-49.phlapa.fios.verizon.net] has quit [Ping timeout: 246 seconds]
14:35 -!- achow101 [~achow101@pool-108-2-202-49.phlapa.fios.verizon.net] has joined #bitcoin-core-dev
14:41 < sipa> luke-jr: the rules that apply to a block are purely a function of its ancestor block header
14:41 -!- phantomcircuit [~phantomci@192.241.205.97] has quit [Max SendQ exceeded]
14:41 < sipa> not of its own timestamp or version
14:41 -!- phantomcircuit [~phantomci@192.241.205.97] has joined #bitcoin-core-dev
14:44 < luke-jr> k
14:51 < luke-jr> wtf does this GCC thing mean: http://codepad.org/BfJgyPwQ :/
14:54 < gmaxwell> luke-jr: it's pointing out that you should use MAX_VERSION_BITS_DEPLOYMENTS instead of MAX_VERSION_BITS_DEPLOYMENTS.
14:54 < luke-jr> am I blind or something? those look the same to me :|
14:54 -!- Guyver2 [~Guyver2@guyver2.xs4all.nl] has quit [Quit: :)]
14:55 < sipa> different namespace?
14:55 < luke-jr> aha that could be it
14:55 < luke-jr> yup
14:55 < luke-jr> thanks
15:10 < Lightsword> luke-jr, why not version GBT separately from the block version?
15:10 < luke-jr> Lightsword: it doesn't make sense to do so
15:12 < Lightsword> most of the time soft forks don’t require mining software changes though, so it would be a good way to signal that mining software needs to be upgraded
15:12 -!- Don_John [~Don@251-223-114-134.nat.resnet.nau.edu] has joined #bitcoin-core-dev
15:12 -!- Don_John [~Don@251-223-114-134.nat.resnet.nau.edu] has quit [Remote host closed the connection]
15:13 < Lightsword> and in general mining software has no reason to care about the rules being enforced on the network since it assumes the template is valid anyways
15:13 < luke-jr> Lightsword: well, previously, GBT had the version/force mutation for that, but nobody ever considered it worth using
15:14 < Lightsword> luke-jr, so the mining software could actually enforce soft fork rules different from bitcoind?
15:14 < luke-jr> the new BIP 9 stuff doesn't really break that either
15:14 < luke-jr> Lightsword: ?
15:14 < Lightsword> what is the purpose of version/force mutation?
15:15 < luke-jr> Lightsword: tells clients to use the provided version despite their ignorance of its meaning
15:15 < luke-jr> so the server could look at what the client supports (in the GBT request), and decide it was close enough
15:15 < Lightsword> oh, is that for GBT pooled mining?
15:17 < luke-jr> mutations are defined in BIP 23, yes, but GBT itself has no distinction between pools and bitcoind
15:23 < Lightsword> luke-jr, so that would allow say embedded GBT clients to mine on a pool without support for segwit or only soft forks that have no GBT format changes like CSV?
15:24 < sipa> Lightsword: the choice that miner clients have is choosing to vote for a deployment or not
15:24 < sipa> not about rules that are actually enforced
15:24 < luke-jr> Lightsword: there is no way to support segwit in GBT clients unless the GBT client is aware of the segwit changes.
15:24 < sipa> those are set by the full node/server that has to accept them
15:27 < Lightsword> so the miners connecting to the pool would be able to modify their block version number to whatever they want?
15:27 < sipa> if the server supports it
15:28 -!- zlover [uid34185@gateway/web/irccloud.com/x-wplywvjztxkodvkc] has quit [Quit: Connection closed for inactivity]
15:30 < luke-jr> Lightsword: see https://github.com/bitcoin/bips/pull/365
15:30 < Lightsword> ok, well from a practical point of view why wouldn’t it make sense for GBT to be versioned separately from the block version so that mining clients can easially determine if they have to upgrade or if they can just use the provided block version as is?
15:31 < luke-jr> Lightsword: because that's not a simple boolean question, and provides no benefits over how GBT presently supports it
15:31 < luke-jr> the CSV deployment *could* have used version/force for old GBT clients, but it did not.
15:33 < Lightsword> I’m confused on what the purpose of the “name” field is, if miners have to be updated to recognize it why can’t they just decode the provided block version number and infer it themselves?
15:33 -!- aquentson [~aquentson@unaffiliated/aquentson] has quit [Ping timeout: 268 seconds]
15:34 < luke-jr> the block version number indicates the pending proposals, not the ones in effect
15:35 < Lightsword> why would the mining software need to care about that specifically as opposed to just caring about whether it broke compatibility?
15:37 -!- achow101 [~achow101@pool-108-2-202-49.phlapa.fios.verizon.net] has quit [Ping timeout: 240 seconds]
15:38 < luke-jr> Lightsword: compatibility is only guaranteed with the known rules
15:39 < Lightsword> luke-jr, but most soft fork rules are enforced by bitcoind by just not including transactions in the template, so in thsoe cases the GBT version would not need to be changed
15:41 < Lightsword> am I missing something here? I don’t see why the GBT client would need to care about those types of rule changes since they are enforced by bitcoind simply excluding invalid transactions
15:43 < luke-jr> Lightsword: eg, when the transactions and generation tx come from different servers
15:48 < Lightsword> luke-jr, wouldn’t both of those be versioned? are you talking about a GBT client mining off of two different GBT sources at once?
15:48 < luke-jr> yes
15:49 < Lightsword> so wouldn’t the client be able to recognize that the GBT version from the transactions server and generation tx server are different?
15:50 < luke-jr> …
15:50 < luke-jr> you're suggesting they be the same
15:53 < Lightsword> luke-jr, have previous soft forks restricted generation transaction validity?
15:53 -!- molz [~molly@unaffiliated/molly] has quit [Ping timeout: 252 seconds]
15:53 -!- moli [~molly@unaffiliated/molly] has joined #bitcoin-core-dev
15:54 < sipa> Lightsword: by definition, all of them
15:54 < sipa> ah, generation transaction, sorry
15:54 < sipa> bip30 and bip34
15:54 < luke-jr> ^
15:55 < sipa> and bip141
15:55 < Lightsword> ok, but some don’t as well right?
15:56 < Lightsword> like CSV and BIP66?
15:57 < luke-jr> Lightsword: GBT code exists which will merge transaction sets from multiple servers as well, if we want to get pedantic
15:58 -!- achow101 [~achow101@pool-108-2-202-49.phlapa.fios.verizon.net] has joined #bitcoin-core-dev
15:59 < Lightsword> basically what I’m thinking is that we should have a GBT version indicator that should change to signal to the miner that additional rules need to be enforced, but only for rules that aren’t simply enforced by the excluding the transaction from the template
15:59 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has quit [Ping timeout: 244 seconds]
16:00 < Lightsword> since the mining client doesn’t really need to care about a rule that is enforced simply by exclusion of a transaction
16:00 < luke-jr> it might
16:02 < Lightsword> in what sort of situation would that be?
16:02 < luke-jr> [22:57:03] <luke-jr> Lightsword: GBT code exists which will merge transaction sets from multiple servers as well, if we want to get pedantic
16:03 < Lightsword> does anybody actually do that?
16:03 < luke-jr> dunno, doubt it
16:04 < Lightsword> I mean, merging transactions means you have to support a lot of consensus logic within the mining software itself
16:04 < luke-jr> not much
16:05 < sipa> it also means you probably want a GBT that reports the entire mempool (or part of it), instead of something already tailored for a single block
16:07 < luke-jr> perhaps a "rules/force" mutation would make sense
16:07 -!- laurentmt [~Thunderbi@176.158.157.202] has joined #bitcoin-core-dev
16:07 < luke-jr> then the server can evaluate if the client's supported rules are sufficient and override that check
16:07 < Lightsword> since most mining software wouldn’t have the neccesary consensus logic to actually handle merging to begin with I would say it would be best to ignore that case for having a GBT version
16:08 < Lightsword> basically the issue right now is we don’t have a way to signal when mining software needs to upgrade or even likely needs to upgrade, at least for the most recent soft forks
16:09 < luke-jr> "version" did that pre-BIP9, and "rules" does it with BIP 9
16:09 -!- laurentmt [~Thunderbi@176.158.157.202] has quit [Client Quit]
16:11 -!- pmienk [~pmienk@c-71-227-177-179.hsd1.wa.comcast.net] has joined #bitcoin-core-dev
16:11 < Lightsword> if the signaling method gets false positives on telling the mining software that it needs to upgrade then people will ignore those rules, the problem with using rules the way you have it right now is that fairly often the miner doesn’t actually care about the rules since it’s simply enforced by exclusion of transactions
16:11 -!- Thireus [~Thireus@vps-92.197.170.217.stwvps.net] has joined #bitcoin-core-dev
16:12 < Lightsword> IMO we need a way to signal to the miner that they need to enforce a rule that is not simply enforced by transaction exclusion otherwise miners will outright ignore rules all together
16:12 < luke-jr> at their own risk
16:14 < Lightsword> well if we have a soft fork that we know is enforced by transaction exclusion we should have a way to tell the mining client that they are ok as long as they are only including the transaction provided in the template
16:14 < luke-jr> ok, that'd be a "rules/force" mutation..
16:15 < sipa> luke-jr: ext filesystems have an extension mechanism that lists the enabled extensions in an FS in 3 classes 1) if you don't know about this, you're fine 2) if you don't know about this, you can only mount ro 3) if you don't know this, you can't mount
16:15 < sipa> luke-jr: perhaps the active rules can use a similar classification
16:16 < Lightsword> I’m confused on how mutable actually works
16:16  * sipa doubts anyone uses mutable at all in GBT
16:17 < luke-jr> server-side, mutable is typically static in practice, but the server could evaluate the client's capabilities and set a rules/force to tell them they're okay
16:18 < sipa> i think you're overdesigning it, trying to cater to every possible use, without any real world demand
16:18 < luke-jr> client-side, it tells the client what they're allowed to do with the template
16:18 < luke-jr> sipa: yes, GBT is overdesigned and should be thrown out, but first we'd need a new protocol
16:18 < Lightsword> well in practical sense bitcoind is the server and the stratum server is the client
16:18 < luke-jr> Lightsword: you're only thinking about your one use case.
16:18 < Lightsword> the use case that basically the entire network runs on though :P
16:18 < sipa> the only one that matters
16:19 < luke-jr> …
16:19 < luke-jr> except the miners who don't.
16:19 < Lightsword> which is almost certainly well below 1% of the network
16:20 < Lightsword> hell, even eligius doesn’t allow GBT mining :P
16:20 < luke-jr> so let's favour the centralised miners abusing their position, at the expense of the 1% who actually care to solo mine?
16:20 < luke-jr> Lightsword: yes it does
16:20 < Lightsword> it just redirects to stratum last I checked
16:20 < luke-jr> you can disable the redirection
16:20 < Lightsword> does anybody actually do that?
16:21 < luke-jr> no idea, ask wizkid057
16:21 < Lightsword> ok, well IMO it’s better to design a signaling method that’s actually going to be useful in practice rather than just getting ignored like it is right now
16:22 < luke-jr> anyone know what testnet blocks made csv change states?
16:22 < Lightsword> huh? i didn’t know it changed status
16:23 < midnightmagic> Much of that ignored stance comes from the undeserved reputational damage that luke's suffered over the years, and the deliberate work that's gone *specifically* into being contrarian.
16:23 < midnightmagic> like, re-written and re-designed *as a specific and incompatible alternative* to things like GBT.
16:24 < luke-jr> Lightsword: splitting the rules list up like sipa suggested sounds like a possible approach, but would require approximately the same code to implement as rules/force would
16:24 < sipa> not sure what you mean by rules/force
16:24 < luke-jr> sipa: server evaluates the client's supported rules, and tells it that it can safely ignore the rules it doesn't know
16:24 < Lightsword> midnightmagic, to be fair GBT is pretty much impractical for pooled mining
16:25  * midnightmagic shrugs.
16:25 < midnightmagic> And instead of making GBT better or including it in a superset, people did other things.
16:26 < phantomcircuit> luke-jr, i'm some large % of testnet and running 73fc922ed64333d45f18d8a448f30cfa2ae0281e
16:26 < luke-jr> midnightmagic: meh, GBT is fundamentally incompatible with sidechains, so it needs to be replaced anyway
16:26 < luke-jr> phantomcircuit: ?
16:26 < Lightsword> midnightmagic, well that’s what happens when something doesn’t fit people’s needs and redesigning it is easier than trying to fix it
16:26 < phantomcircuit> you asked about testnet and csv; i did not answer your question but did provide additional information
16:27 < luke-jr> phantomcircuit: I don't see an answer to my question >_
16:27 < luke-jr> >_<
16:27 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 276 seconds]
16:27 < phantomcircuit> luke-jr, i'm so helpful right
16:27 -!- AaronvanW [~ewout@unaffiliated/aaronvanw] has quit [Remote host closed the connection]
16:27 < gmaxwell> Lightsword: consider what you're saying, that it's impratical to send the miner the block they're working on, once per block?  This should be setting off red alarm bells for you.
16:28 < phantomcircuit> gmaxwell, i think you need more specific nouns
16:28 < Lightsword> gmaxwell, when a “miner” is a server farm with 10k miners all directly connected to a pool…it’s going to be a problem in most cases
16:28 < phantomcircuit> (actually i think this entire conversation needs more specific nouns)
16:28 < Lightsword> even if the block size is 50k
16:30 < Lightsword> it’s really a matter of what can send out the block update faster, miners aren’t likely to give up profit margin unless they have a really good reason to
16:30 < gmaxwell> Lightsword: GBT isn't a protocol for dispatching to random hardware. But at the same time having 50k tcp connections from devices at one site to a pool makes no sense either.
16:30 < phantomcircuit> gmaxwell, (switching topics) i'm finding the afl hang detection to be pretty annoying, it refuses to start a new run if a test case is even 1ms above the hang threshold... which means moving tests between a server and my laptop results in having to purge a bunch of tests (the server being much faster than my laptop)
16:30 < phantomcircuit> any suggestion?
16:30 -!- achow101 [~achow101@pool-108-2-202-49.phlapa.fios.verizon.net] has quit [Ping timeout: 252 seconds]
16:31 < gmaxwell> phantomcircuit: up your threshold on the laptop.
16:31 < Lightsword> gmaxwell, well that’s the reason stratum happened, because it was desgined for dispatching to random hardware more or less :P
16:31 < gmaxwell> No, it's a crappy protocol for that too.
16:31 < phantomcircuit> gmaxwell, it would be nice if it counted instructions instead but i assume that would be both harder and more expensive?
16:32 < phantomcircuit> i thought x86 systems had performance counters that could efficiently do that though
16:32 -!- bitcoinfr [4ecf1f0c@gateway/web/freenode/ip.78.207.31.12] has joined #bitcoin-core-dev
16:32 < sipa> phantomcircuit: rdtsc
16:32 < Lightsword> gmaxwell, sure but it was better than GBT for that and there wasn’t an alternative people could choose
16:32 < sipa> doesn't count instructions, just clock cycles
16:32 < bitcoinfr> .
16:32 < phantomcircuit> sipa, that seems better actually
16:33 < midnightmagic> Lightsword: they explicitly ignored most of the feedback about their new design, and in at least one case, it was done in a *trivially exploitable way.*
16:33 < luke-jr> and made no effort to collaborate with others already working on GBT
16:33 < luke-jr> but that's history, and irrelevant to what we do in BIP 9
16:33 -!- bitcoinfr [4ecf1f0c@gateway/web/freenode/ip.78.207.31.12] has quit [Client Quit]
16:33 < luke-jr> which is something that needs to get wrapped up sooner rather than later so solo mining is practical again
16:33 < gmaxwell> Lightsword: You're suffering history rewrite in terms of the current state of the world, back when these protocols were created the world was very different than now.
16:34 -!- achow101 [~achow101@pool-108-2-202-49.phlapa.fios.verizon.net] has joined #bitcoin-core-dev
16:36 < Lightsword> was there much effort on either side to collaborate? at the time I thought luke was just arguing something along the lines of you shouldn’t use stratum you should use GBT instead
16:39 < luke-jr> Lightsword: GBT was developed and revised over the course of a year or two, before stratum was even mentioned publicly
16:39 < luke-jr> developed openly, with collaboration from multiple people
16:53 < midnightmagic> yes, lots of attempts at collaboration and unification.
16:58 < luke-jr> why does BOOST_FOREACH not work on UniValues? :<
17:09 < sipa> we should fix that
17:10 < phantomcircuit> gmaxwell, when fuzzing deserialization, should i treat throwing exceptions as a crash or as an expected result
17:11 < gmaxwell> phantomcircuit: depends on the exception.
17:11 < gmaxwell> The deceralization uses exceptions (spit) to do error reporting. So you shouldn't treat as crash ordinary errors during deseralization.
17:12 < phantomcircuit> gmaxwell, the serialization stuff will throw std::ios_base::failure when there's a short read
17:14 < gmaxwell> Not a crash.
17:17 < phantomcircuit> ok that's what i thought
17:18 < gmaxwell> basically, you only want to treat as crashes things that you might possibly want to go fix.
17:18 < gmaxwell> :)
17:21 -!- TomMc [~tom@unaffiliated/tommc] has quit [Ping timeout: 244 seconds]
17:22 < phantomcircuit> gmaxwell, that reminds me (although i dont know why) that i wanted to do a vm with the glibcxx_debug flag
17:23 < gmaxwell> phantomcircuit: that would be very handy to have. 
17:23 < phantomcircuit> also fun fact
17:23 < phantomcircuit> the sort command is threaded
17:23 < gmaxwell> I know. it uses a merge sort with storage in /tmp.
17:23 < gmaxwell> it's actually quite good.
17:24 < midnightmagic> Un-fun fact. Sort uses the locale for sorting order, which if it's not 'c' and is e.g. en_CA.UTF-8, puts capital 'H' *after* lowercase 'i'.
17:25 < gmaxwell> yea, the standard posix string functions all became localized in the 90s. And this means that e.g. number parsing functions change behavior (comma vs dot handling) ... resuting in lots of broken software that was using them for handling file formats and network protocols.
17:25 < gmaxwell> Which goes back to my standard advice of not using strings anywhere normative. :)
17:26 < luke-jr> anyone know what testnet blocks made csv change states?
17:26 < phantomcircuit> luke-jr, something with timestamp around 1456790400,
17:27 < luke-jr> gmaxwell: don't you need to explicitly enable locales for POSIX?
17:27 < phantomcircuit> maybe?
17:47 < phantomcircuit> gmaxwell, oops deleted my afl-cmin results
17:47 < phantomcircuit> :|
17:48 < phantomcircuit> https://github.com/pstratem/bitcoin-public/blob/2016-04-20-fuzzing-framework/src/bitcoin-fuzzy.cpp
17:48 < phantomcircuit> suggestions for more things to fuzz appreciated
17:59 -!- gevs [~greg@unaffiliated/gevs] has quit [Ping timeout: 250 seconds]
18:12 -!- gevs [~greg@unaffiliated/gevs] has joined #bitcoin-core-dev
18:22 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has joined #bitcoin-core-dev
18:28 < gmaxwell> phantomcircuit: CBlockHeader? CBanEntry? CTxUndo? CBlockUndo? CCoin? CNetAddr? CSubNet? CService? CMessageHeader? CAddress? CInv? CBloomFilter? CDiskBlockIndex? CTxOutCompressor? CBlockFileInfo? COutPoint? CTxIn? CTxOut?  wallet-- CMasterKey? CKeyPool? CWalletKey? CWalletTx? CAccountingEntry? CAccount? CKeyMetadata?
18:28 < gmaxwell> phantomcircuit: some of these are used internally in others, so I'm not sure if they should be added at the top level or not.
18:28 < phantomcircuit> gmaxwell, CBlockHeader is kind of covered by CBlock, but yeah
18:28 < gmaxwell> yea, well the rule should be if a hash or something is going to prevent AFL from finding an input that lets it reach thoe whole thing, you should go directly.
18:33 -!- Ylbam [uid99779@gateway/web/irccloud.com/x-zprlrenqftfaswgo] has quit [Quit: Connection closed for inactivity]
18:54 -!- belcher [~user@unaffiliated/belcher] has quit [Quit: Leaving]
19:12 -!- TomMc [~tom@unaffiliated/tommc] has joined #bitcoin-core-dev
19:14 < gmaxwell> phantomcircuit: the next improvement you could do is, assuming the deseralization was successful you could try calling some methods on the object. (e.g. whatever we'd normally do after deseralizing it).
19:20 < phantomcircuit> gmaxwell, heh one of these is crazy slow
19:35 -!- BCB [424180ce@gateway/web/freenode/ip.66.65.128.206] has joined #bitcoin-core-dev
19:37 -!- TomMc [~tom@unaffiliated/tommc] has quit [Ping timeout: 250 seconds]
19:38 -!- achow101 [~achow101@pool-108-2-202-49.phlapa.fios.verizon.net] has quit [Read error: Connection reset by peer]
19:42 -!- fedruantine [fedruantin@2600:3c03::f03c:91ff:fe55:c675] has joined #bitcoin-core-dev
21:09 -!- Amnez777- [~Amnez777@37.157.216.173] has joined #bitcoin-core-dev
21:12 -!- Amnez777 [~Amnez777@37.157.216.175] has quit [Ping timeout: 260 seconds]
21:18 -!- Amnez777 [~Amnez777@37.157.216.175] has joined #bitcoin-core-dev
21:18 -!- Amnez777- [~Amnez777@37.157.216.173] has quit [Ping timeout: 276 seconds]
21:32 -!- skyraider [uid41097@gateway/web/irccloud.com/x-hpagikakarqcekxf] has joined #bitcoin-core-dev
22:23 < GitHub117> [bitcoin] pstratem opened pull request #7930: CAddrMan::Deserialize Initialize to NULL on insane input. (master...2014-04-23-addrman-deserialize-limits) https://github.com/bitcoin/bitcoin/pull/7930
22:24 -!- Chris_Stewart_5 [~Chris_Ste@unaffiliated/chris-stewart-5/x-3612383] has quit [Ping timeout: 244 seconds]
22:24 < phantomcircuit> sipa, does ^ make any sense?
22:27 -!- justanotheruser [~Justan@unaffiliated/justanotheruser] has quit [Ping timeout: 240 seconds]
23:05 -!- BCB [424180ce@gateway/web/freenode/ip.66.65.128.206] has quit [Quit: Page closed]
23:08 -!- Amnez777 [~Amnez777@37.157.216.175] has quit [Ping timeout: 276 seconds]
23:29 -!- Amnez777 [~Amnez777@37.157.216.133] has joined #bitcoin-core-dev
23:41 -!- skyraider [uid41097@gateway/web/irccloud.com/x-hpagikakarqcekxf] has quit [Quit: Connection closed for inactivity]