* [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
@ 2015-07-11 8:05 Nathan Wilcox
2015-07-11 9:24 ` Jorge Timón
2015-07-11 15:34 ` Jeff Garzik
0 siblings, 2 replies; 13+ messages in thread
From: Nathan Wilcox @ 2015-07-11 8:05 UTC (permalink / raw)
To: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 2180 bytes --]
Thesis: The disincentive miners have for verifying transactions is
problematic and weakens the network's robustness against forks.
According to the 2015-07-04 bitcoin.org alert [1]_ so-called "SPV Mining"
has become popular across a large portion of miners, and this enabled the
consensus-violating forks to persist. Peter Todd provides an explanation
of the incentive for SPV Mining over in another thread [2]_.
.. [1] https://bitcoin.org/en/alert/2015-07-04-spv-mining#cause
.. [2]
https://www.mail-archive.com/bitcoin-dev@lists.linuxfoundation.org/msg00404.html
If there is a cost to verifying transactions in a received block, then
there is an incentive to *not verify transactions*. However, this is
balanced by the a risk of mining atop an invalid block.
If we imagine all miners verify all transactions, except Charlie the
Cheapskate, then it's in Charlie's interest to forego transaction
verification. If all miners make a similar wager, then in the extreme,
no miners verify any transactions, and the expected cost of skipping
transaction verification becomes very high.
Unfortunately, it's difficult to measure how many miners are not
validating transactions, since there's no evidence of this until they
mine atop on invalid block. Because of this, I worry that over time,
more and more miners cut this particular corner, to save on costs.
If true, then the network continues to grow more brittle towards the kind
of forking-persistence behavior we saw from the July 4th (and 5th) forks.
This gets weird. For example, a malicious miner which suspects a large
fraction of miners are neglecting transaction verification may choose to
forego a block reward by throwing an erroneous transaction into their
winning block, then, as all the "SPV Miners" run off along a worthless
chain, they can reap a higher reward rate due to controlling a larger
network capacity fraction on the valid chain.
Can we fix this?
--
Nathan Wilcox
Least Authoritarian
email: nathan@leastauthority•com
twitter: @least_nathan
Standard Disclaimer: I'm behind on dev archives, irc logs, bitcointalk,
the wiki... if this has been discussed before I appreciate mentions of
that fact.
[-- Attachment #2: Type: text/html, Size: 2648 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-11 8:05 [bitcoin-dev] SPV Mining reveals a problematic incentive issue Nathan Wilcox
@ 2015-07-11 9:24 ` Jorge Timón
2015-07-11 10:39 ` Tier Nolan
2015-07-13 16:04 ` Peter Todd
2015-07-11 15:34 ` Jeff Garzik
1 sibling, 2 replies; 13+ messages in thread
From: Jorge Timón @ 2015-07-11 9:24 UTC (permalink / raw)
To: Nathan Wilcox; +Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 3176 bytes --]
All miners should validate transactions precisely because of the latest
attack you've described. Full miners can gain a lot from this attack to
leverage their full validation against spv miners who blindly spend energy
hashing on top of something that may be worthless crap. SPV mining makes no
sense, but some miners claim they're doind it for very short periods of
time, which shouldn't be as bad as doing it all the time.
I think it would be more rational for them to keep mining on top of the old
block until they've fully validated the new block (which shouldn't take so
long anyway), even if this slightly increases the orphan rate.
On Jul 11, 2015 10:05 AM, "Nathan Wilcox" <nathan@leastauthority•com> wrote:
> Thesis: The disincentive miners have for verifying transactions is
> problematic and weakens the network's robustness against forks.
>
> According to the 2015-07-04 bitcoin.org alert [1]_ so-called "SPV Mining"
> has become popular across a large portion of miners, and this enabled the
> consensus-violating forks to persist. Peter Todd provides an explanation
> of the incentive for SPV Mining over in another thread [2]_.
>
> .. [1] https://bitcoin.org/en/alert/2015-07-04-spv-mining#cause
>
> .. [2]
> https://www.mail-archive.com/bitcoin-dev@lists.linuxfoundation.org/msg00404.html
>
> If there is a cost to verifying transactions in a received block, then
> there is an incentive to *not verify transactions*. However, this is
> balanced by the a risk of mining atop an invalid block.
>
> If we imagine all miners verify all transactions, except Charlie the
> Cheapskate, then it's in Charlie's interest to forego transaction
> verification. If all miners make a similar wager, then in the extreme,
> no miners verify any transactions, and the expected cost of skipping
> transaction verification becomes very high.
>
> Unfortunately, it's difficult to measure how many miners are not
> validating transactions, since there's no evidence of this until they
> mine atop on invalid block. Because of this, I worry that over time,
> more and more miners cut this particular corner, to save on costs.
>
> If true, then the network continues to grow more brittle towards the kind
> of forking-persistence behavior we saw from the July 4th (and 5th) forks.
>
> This gets weird. For example, a malicious miner which suspects a large
> fraction of miners are neglecting transaction verification may choose to
> forego a block reward by throwing an erroneous transaction into their
> winning block, then, as all the "SPV Miners" run off along a worthless
> chain, they can reap a higher reward rate due to controlling a larger
> network capacity fraction on the valid chain.
>
> Can we fix this?
>
> --
> Nathan Wilcox
> Least Authoritarian
>
> email: nathan@leastauthority•com
> twitter: @least_nathan
>
> Standard Disclaimer: I'm behind on dev archives, irc logs, bitcointalk,
> the wiki... if this has been discussed before I appreciate mentions of
> that fact.
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
[-- Attachment #2: Type: text/html, Size: 4092 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-11 9:24 ` Jorge Timón
@ 2015-07-11 10:39 ` Tier Nolan
2015-07-11 12:09 ` Nathan Wilcox
2015-07-12 18:37 ` Jorge Timón
2015-07-13 16:04 ` Peter Todd
1 sibling, 2 replies; 13+ messages in thread
From: Tier Nolan @ 2015-07-11 10:39 UTC (permalink / raw)
Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 1067 bytes --]
On Sat, Jul 11, 2015 at 10:24 AM, Jorge Timón <jtimon@jtimon•cc> wrote:
> I think it would be more rational for them to keep mining on top of the
> old block until they've fully validated the new block (which shouldn't take
> so long anyway), even if this slightly increases the orphan rate.
Increased orphan rate means that the network is (slightly) less secure.
If miners have a 5% orphan rate, then an attacker can launch a 51% attack
with 49% of the network.
It isn't a massive difference, but it is there.
As long as miners switch back to non-SPV mining after a timeout, SPV-mining
is safe for everyone.
The average cost to the miner from building on an invalid block is small,
as long as invalid blocks only happen rarely.
Miners still have an incentive to do full validation, so that they can
include transactions and get transaction fees.
SPV-mining is to prevent hashing hardware from having to waste power when
it isn't needed.
It may be less of a problem if (when?) electricity costs dominate hardware
capital costs.
[-- Attachment #2: Type: text/html, Size: 1529 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-11 10:39 ` Tier Nolan
@ 2015-07-11 12:09 ` Nathan Wilcox
2015-07-11 13:17 ` Tier Nolan
2015-07-12 18:37 ` Jorge Timón
1 sibling, 1 reply; 13+ messages in thread
From: Nathan Wilcox @ 2015-07-11 12:09 UTC (permalink / raw)
To: Tier Nolan; +Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 3815 bytes --]
On Sat, Jul 11, 2015 at 2:24 AM, Jorge Timón <jtimon@jtimon•cc> wrote:
> All miners should validate transactions precisely because of the latest
attack you've described.
The problem is one of individual incentives leading to a systemic problem.
"All X should do..." solutions which are oblivious to individual incentives
don't scale, eg: "If all factories avoid emitting pollution they don't pay
for..." does not work if individual factories save their own costs by
dumping into the environment. No one wants environmental catastrophe, and
no one wants a blockchain where miners don't validate transactions, but
there may be a systemic problem of incentives.
The bitcoin.org claim that "about half" of miner capacity does SPV Mining,
is consistent with the incentives problem as I described it. However, I
don't claim the state of mining is certainly due to these incentives and
not other incentives we haven't discussed. Also, there may be other
incentives that outweigh this issue.
On Sat, Jul 11, 2015 at 3:39 AM, Tier Nolan <tier.nolan@gmail•com> wrote:
>
>
> On Sat, Jul 11, 2015 at 10:24 AM, Jorge Timón <jtimon@jtimon•cc> wrote:
>
>> I think it would be more rational for them to keep mining on top of the
>> old block until they've fully validated the new block (which shouldn't take
>> so long anyway), even if this slightly increases the orphan rate.
>
>
> Increased orphan rate means that the network is (slightly) less secure.
>
> If miners have a 5% orphan rate, then an attacker can launch a 51% attack
> with 49% of the network.
>
> It isn't a massive difference, but it is there.
>
> As long as miners switch back to non-SPV mining after a timeout,
> SPV-mining is safe for everyone.
>
>
If there's any cost to non-SPV mining, and the chance that an incoming
block contains only valid transactions is very high, isn't there still an
incentive to make this timeout longer and longer?
The average cost to the miner from building on an invalid block is small,
> as long as invalid blocks only happen rarely.
>
>
Yes. If it's rare enough, then skipping transaction validation saves more
cost than the expected cost of mining atop an invalid block. ... but if
everyone follows that logic, the chance is no longer rare.
> Miners still have an incentive to do full validation, so that they can
> include transactions and get transaction fees.
>
>
This is a good point. If the benefit to skipping verification outweighs
expected transaction costs, then a non-validating miner might also choose
to mine empty blocks with only their coinbase. (I recall hearing this
occurred somewhat regularly around 2013, but I haven't paid attention since
then. How common are empty blocks these days?)
This is a benefit of the world where transaction fees approach or surpass
block reward.
SPV-mining is to prevent hashing hardware from having to waste power when
> it isn't needed.
>
>
It may be less of a problem if (when?) electricity costs dominate hardware
> capital costs.
>
Oh. This is a different explanation than Peter Todd's I linked to above,
which claimed it was network latency in receiving transactions.
Could you explain this a bit more? What is not needed, the hashing
hardware or the power? How does SPV Mining affect that?
I'll bet there are many individual rationales for SPV-Mining, but
ultimately they come down to dropping a cost based on a bet about other
miners' behavior.
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
--
Nathan Wilcox
Least Authoritarian
email: nathan@leastauthority•com
twitter: @least_nathan
[-- Attachment #2: Type: text/html, Size: 6379 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-11 12:09 ` Nathan Wilcox
@ 2015-07-11 13:17 ` Tier Nolan
2015-07-11 15:04 ` Dave Hudson
0 siblings, 1 reply; 13+ messages in thread
From: Tier Nolan @ 2015-07-11 13:17 UTC (permalink / raw)
To: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 6265 bytes --]
On Sat, Jul 11, 2015 at 1:09 PM, Nathan Wilcox <nathan@leastauthority•com>
wrote:
>
> If there's any cost to non-SPV mining, and the chance that an incoming
> block contains only valid transactions is very high, isn't there still an
> incentive to make this timeout longer and longer?
>
The benefit drops off pretty quickly as the timeout increases (and
eventually goes negative).
You could look at it that headers having 4 states.
1) Valid
2) Probably Valid
3) Probably Invalid
4) Invalid
SPV mining puts newly received headers into the "probably valid" category.
It builds empty (coinbase only) blocks on top of probably valid headers and
build empty blocks on the header.
Once it receives the full block, it can change the state to Valid. At that
point, it can build full blocks on top of the header.
As time passes without the full block being received/validated, it becomes
less and less likely that the block is actually valid.
The timeout is to recognize that fact. Making the timeout 24 hours is not
likely to give the miner much benefit over making it 1-2 minutes.
Setting the timeout to low means that the miner sometimes switches away
from a header that turns out to be valid.
Setting it to high means that the miner ends up staying to long on a header
that turns out to be invalid.
At some point, the second effect is stronger than the first effect. The
timeout needs to be high enough so switching away from valid headers is
rare but low enough so that it doesn't stay on invalid headers for ages.
If 99% of full blocks are received (and validated) within 30 seconds of the
header arriving, then it is reasonable for the miner to assume that if the
full block hasn't arrived within 60 seconds of the header arriving, then
the header is for an invalid block.
Yes. If it's rare enough, then skipping transaction validation saves more
> cost than the expected cost of mining atop an invalid block. ... but if
> everyone follows that logic, the chance is no longer rare.
>
SPV miners don't actually produce invalid blocks though (other than
building on invalid blocks). The full blocks they produce are still fully
checked blocks.
> SPV-mining is to prevent hashing hardware from having to waste power when
>> it isn't needed.
>>
>>
> It may be less of a problem if (when?) electricity costs dominate hardware
>> capital costs.
>>
>
> Oh. This is a different explanation than Peter Todd's I linked to above,
> which claimed it was network latency in receiving transactions.
>
SPV mining is mainly to protect against latency. The reason that matters
is that latency means that hashers end up building on blocks even though a
new block has been found.
You can look at it as wasting hashing power due to latency.
In the world where minting fees are very low, there is no point in SPV
mining.
I assume at the point, the memory pool/queue is a few blocks deep. This
means that the pool can create a full block without having to wait for new
transactions to be sent in.
It still needs to wait for the new full block before it knows which
transactions to remove from its memory pool.
Pools have to pay their hashers for hashing power. When minting fees are
tiny, pools only get income only from tx fees.
There is no point in creating empty blocks, since they don't pay anything.
Between when the block is found and the pool has a new block ready to mine,
there is no incentive for the pool to give out new work. The stratum
protocol could be modified so pools can say. (It might already support
this)
<Send work to hashers>
-- block is found by some other pool --
<Cancel work for miners>
-- pool builds new block --
<Send new work to hashers>
The cancel command says that the pool will not accept any of the work that
has been made obsolete.
This gives a window of 20-30 seconds after each block where the pool has
invalidated the old work, but does not send new work. Hashers' hardware
would be stalled.
On the other hand, the pool that found the block could create a new block
straight away. There is an incentive for hashers to have a connection to
multiple pools.
Pools might go pure pay per share. The protocol would say how much they
are willing to pay for a share and the local mining proxy would pick the
most profitable pool. This eliminates pools having lots of ways of saying
how they charge fees, you just connect to lots of pools and go with the one
that pays the most.
More interestingly, the average fee per byte for transactions in the memory
pool is likely to increase as time passes since the last block.
When two blocks are found very fast one after another, the second block is
likely to have lower average fees. This is because the first block would
have included most of the high value transactions and there wasn't enough
time for new ones to arrive before the second block was found.
Hashers would end up getting variable payouts based on how long since the
last block was received. If some miners increase/decrease their output
based on the fees the pools offer, then as time passes since the last
block, the hashing rate would increase. This reduces the variation in
block to block times.
For example, if there was 1.5MB of transactions in the memory pool and they
all paid the same fee per byte, then the block would be a full block at
that rate. However, there would only be 0.5MB of transactions left. This
means that the next block would be half full and only be able to pay out
50% of the fee, but the difficulty would be the same. The pay per hash
would be 50% lower. Once 0.5MB of new transactions arrive, the fee would
be back up to the same as the previous block.
If there are major SHA256 altcoins (or side chains), then miners might end
up switching between coins. The longer a coin went without a new block
being found, the more tx fees available in the memory pool, so the more
hashing power would decide to switch to that coin.
There could be a situation where adding a few more transactions to the
memory pool of a coin would cause a 100X increasing in hashing until the
block was found and then it stalling again as the hashing power switches
away. This is similar to alt coins getting blasted by coin switching pools
and then dropping to almost no power.
[-- Attachment #2: Type: text/html, Size: 8259 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-11 13:17 ` Tier Nolan
@ 2015-07-11 15:04 ` Dave Hudson
2015-07-11 16:26 ` Tier Nolan
0 siblings, 1 reply; 13+ messages in thread
From: Dave Hudson @ 2015-07-11 15:04 UTC (permalink / raw)
To: Tier Nolan; +Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 5122 bytes --]
> On 11 Jul 2015, at 14:17, Tier Nolan <tier.nolan@gmail•com> wrote:
>
> On Sat, Jul 11, 2015 at 1:09 PM, Nathan Wilcox <nathan@leastauthority•com <mailto:nathan@leastauthority•com>> wrote:
>
> If there's any cost to non-SPV mining, and the chance that an incoming block contains only valid transactions is very high, isn't there still an incentive to make this timeout longer and longer?
>
> The benefit drops off pretty quickly as the timeout increases (and eventually goes negative).
>
> Hashers would end up getting variable payouts based on how long since the last block was received. If some miners increase/decrease their output based on the fees the pools offer, then as time passes since the last block, the hashing rate would increase. This reduces the variation in block to block times.
>
> For example, if there was 1.5MB of transactions in the memory pool and they all paid the same fee per byte, then the block would be a full block at that rate. However, there would only be 0.5MB of transactions left. This means that the next block would be half full and only be able to pay out 50% of the fee, but the difficulty would be the same. The pay per hash would be 50% lower. Once 0.5MB of new transactions arrive, the fee would be back up to the same as the previous block.
This would probably be worse. The 1 MB would include the highest fee transactions, leaving the lowest fees in the remaining 0.5 MB.
> If there are major SHA256 altcoins (or side chains), then miners might end up switching between coins. The longer a coin went without a new block being found, the more tx fees available in the memory pool, so the more hashing power would decide to switch to that coin.
>
> There could be a situation where adding a few more transactions to the memory pool of a coin would cause a 100X increasing in hashing until the block was found and then it stalling again as the hashing power switches away. This is similar to alt coins getting blasted by coin switching pools and then dropping to almost no power.
If hashing isn't constantly applied all the time then the inter-block times will expand and the difficulty will reduce. This means that a pool that decides to use all of its available hashing 100% of the time now has a distinct advantage over those who are playing nicely. This is the same problem that the "proof of idle" idea had; it only works if no-one chooses to try to exploit it (which seems very unlikely).
One might ask why a pool might wish to try to exploit this, but let's assume we have a fees-only reward, so here's a quick thought experiment - the numbers are only approximate and would need a thorough simulation but serve for discussion:
Say, for the sake of argument that over a nominal 10 minute period we see 10 BTC worth of transaction fees. If the mempool is empty of interesting fees at the start of a block then we might like to imagine that rational miners will power down their hashing to save energy costs until the fees are worthwhile. Let's assume, for the sake of argument, that this nominally takes 5 minutes.
After 5 minutes we go from 0% to 100% as all hashing engines switch on. The difficulty will have corrected to mean that 100% of the work will nominally happen in the next 5 minutes, but that means that a malicious miner has a 2x amplification of their nominal hashing rate to do mischief in the preceding 5 minutes.
Such a malicious miner would choose to spend their 5 minutes re-mining the previous block, but dropping some amount of the transactions from it. Let's say that they try to re-mine only 9.5 BTC out the previous 10 BTC. If they succeed then they're offering everyone else an extra 0.5 BTC (5%) if they mine on top of their re-mined block and as an incentive to orphan the original block. Rational miners would definitely choose to build on the re-mined block because they get more reward from doing so.
Of course the extra hashing that our malicious miner is doing will actually push the difficulty back up somewhat, but they're still running at an advantage over the long-term. I've also ignored some of the other security implications of the hashing amplification effects (e.g. 25% of the hash rate can end up controlling 50% of the blocks in the scenario above).
An obvious objection to this scenario is that everyone would notice the malicious mining. Statistically, yes, the orphan rate would be much higher, but there's no guarantee that anyone could ever work out who was behind this. It's all too easy to create an "unknown" pool, or a series of "unknown" pools. Of course our malicious miner might choose to only target blocks that had particularly high fees associated with it, in which case the orphan rate might barely change.
The only way I can see that this wouldn't be the case would be if there were always useful fees available to mine immediately after a block is found. If block space is kept moderately scarce then immediately a block is found then everyone will keep mining and the incentives to try to deliberately orphan the last block are dramatically reduced.
[-- Attachment #2: Type: text/html, Size: 6598 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-11 8:05 [bitcoin-dev] SPV Mining reveals a problematic incentive issue Nathan Wilcox
2015-07-11 9:24 ` Jorge Timón
@ 2015-07-11 15:34 ` Jeff Garzik
1 sibling, 0 replies; 13+ messages in thread
From: Jeff Garzik @ 2015-07-11 15:34 UTC (permalink / raw)
To: Nathan Wilcox; +Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 2621 bytes --]
The miners with invalid blocks were punished with a loss of bitcoin
income...
On Sat, Jul 11, 2015 at 4:05 AM, Nathan Wilcox <nathan@leastauthority•com>
wrote:
> Thesis: The disincentive miners have for verifying transactions is
> problematic and weakens the network's robustness against forks.
>
> According to the 2015-07-04 bitcoin.org alert [1]_ so-called "SPV Mining"
> has become popular across a large portion of miners, and this enabled the
> consensus-violating forks to persist. Peter Todd provides an explanation
> of the incentive for SPV Mining over in another thread [2]_.
>
> .. [1] https://bitcoin.org/en/alert/2015-07-04-spv-mining#cause
>
> .. [2]
> https://www.mail-archive.com/bitcoin-dev@lists.linuxfoundation.org/msg00404.html
>
> If there is a cost to verifying transactions in a received block, then
> there is an incentive to *not verify transactions*. However, this is
> balanced by the a risk of mining atop an invalid block.
>
> If we imagine all miners verify all transactions, except Charlie the
> Cheapskate, then it's in Charlie's interest to forego transaction
> verification. If all miners make a similar wager, then in the extreme,
> no miners verify any transactions, and the expected cost of skipping
> transaction verification becomes very high.
>
> Unfortunately, it's difficult to measure how many miners are not
> validating transactions, since there's no evidence of this until they
> mine atop on invalid block. Because of this, I worry that over time,
> more and more miners cut this particular corner, to save on costs.
>
> If true, then the network continues to grow more brittle towards the kind
> of forking-persistence behavior we saw from the July 4th (and 5th) forks.
>
> This gets weird. For example, a malicious miner which suspects a large
> fraction of miners are neglecting transaction verification may choose to
> forego a block reward by throwing an erroneous transaction into their
> winning block, then, as all the "SPV Miners" run off along a worthless
> chain, they can reap a higher reward rate due to controlling a larger
> network capacity fraction on the valid chain.
>
> Can we fix this?
>
> --
> Nathan Wilcox
> Least Authoritarian
>
> email: nathan@leastauthority•com
> twitter: @least_nathan
>
> Standard Disclaimer: I'm behind on dev archives, irc logs, bitcointalk,
> the wiki... if this has been discussed before I appreciate mentions of
> that fact.
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
[-- Attachment #2: Type: text/html, Size: 3560 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-11 15:04 ` Dave Hudson
@ 2015-07-11 16:26 ` Tier Nolan
0 siblings, 0 replies; 13+ messages in thread
From: Tier Nolan @ 2015-07-11 16:26 UTC (permalink / raw)
Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 5391 bytes --]
On Sat, Jul 11, 2015 at 4:04 PM, Dave Hudson <dave@hashingit•com> wrote:
> This would probably be worse. The 1 MB would include the highest fee
> transactions, leaving the lowest fees in the remaining 0.5 MB.
>
Right, in the example, I was assuming all transactions had the same fee per
byte.
> If hashing isn't constantly applied all the time then the inter-block
> times will expand and the difficulty will reduce. This means that a pool
> that decides to use all of its available hashing 100% of the time now has a
> distinct advantage over those who are playing nicely. This is the same
> problem that the "proof of idle" idea had; it only works if no-one chooses
> to try to exploit it (which seems very unlikely).
>
Uh, I don't think so. Pools would offer a price per hash based on how much
tx fees that they can get at that moment.
Offering more than that would mean they make a loss on average.
Say, for the sake of argument that over a nominal 10 minute period we see
> 10 BTC worth of transaction fees. If the mempool is empty of interesting
> fees at the start of a block then we might like to imagine that rational
> miners will power down their hashing to save energy costs until the fees
> are worthwhile. Let's assume, for the sake of argument, that this nominally
> takes 5 minutes.
>
I think the hardware would be able to power down nearly instantly.
Granted, if they have generators or similar, they may not be able to do it
so fast.
Switching to an altcoin is pretty much instant though.
> After 5 minutes we go from 0% to 100% as all hashing engines switch on.
> The difficulty will have corrected to mean that 100% of the work will
> nominally happen in the next 5 minutes, but that means that a malicious
> miner has a 2x amplification of their nominal hashing rate to do mischief
> in the preceding 5 minutes.
>
I think you need to split it into hashers and pools, rather than miners.
Hashers have to pay electricity costs to keep their equipment running.
Powering down for 5 minutes is cheaper than using that hashing for other
things.
The ratio of capital costs and electricity determines which wins out.
In the example given, it would work out as something like this.
0 mins: pool offers 0
2 mins: pool offers 20% of average
4 mins: pool offers 40% of average
6 mins: pool offers 60% of average
8 mins: pool offers 80% of average
10 mins: pool offers 100% of average
12 mins: pool offers 120% of average
14 mins: pool offers 140% of average
...
This means that more and more miners will accept the offer as time passes.
If a miner was using solar power for their miners, then they might as well
run it for the full 10 mins, since it is pointless to leave the equipment
off. With batteries they could shift some of their output to the more
profitable period.
Such a malicious miner would choose to spend their 5 minutes re-mining the
> previous block, but dropping some amount of the transactions from it. Let's
> say that they try to re-mine only 9.5 BTC out the previous 10 BTC. If they
> succeed then they're offering everyone else an extra 0.5 BTC (5%) if they
> mine on top of their re-mined block and as an incentive to orphan the
> original block. Rational miners would definitely choose to build on the
> re-mined block because they get more reward from doing so.
>
If they find that block, it will be a tie with the other block, but created
much later. That means that nobody will build on the block they found.
> Of course the extra hashing that our malicious miner is doing will
> actually push the difficulty back up somewhat, but they're still running at
> an advantage over the long-term. I've also ignored some of the other
> security implications of the hashing amplification effects (e.g. 25% of the
> hash rate can end up controlling 50% of the blocks in the scenario above).
>
> I think mining strategy once minting fees are greatly reduced is an
interesting question. We can't assume that miners are all going to build
on the tip.
In your example, you can bribe the miner of the next block by paying to
OP_TRUE.
A <- B <-C
Assume that C pays 1BTC in fees and the miner creates a new version of C
that pays 1.1BTC in fees.
C' pays 1.1 BTC in fees and also pays 0.05BTC to OP_TRUE.
This means that miners who build on C' instead of C get a 0.05BTC 'bribe'
to ignore the fact that C' was found much later.
It isn't clear if other miners would be better off building D anyway, since
they could take 100% of the fees.
If the average fees per block was 1BTC and someone send a block that pays
10BTC in fees, it could stall the block chain. You can do the same bribery
trick.
If C has the 10BTC transaction, you can create a C' block and pay 1BTC to
OP_TRUE. The miner who builds on C' include a transaction which pays that
money to him.
Another miner can produce C'' that pays 2BTC to OP_TRUE.
> The only way I can see that this wouldn't be the case would be if there
> were always useful fees available to mine immediately after a block is
> found. If block space is kept moderately scarce then immediately a block is
> found then everyone will keep mining and the incentives to try to
> deliberately orphan the last block are dramatically reduced.
>
True, if there is multiple blocks worth of transactions in the memory pool,
then losing one block's worth of transactions won't drop the total fees
down to zero.
[-- Attachment #2: Type: text/html, Size: 7552 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-11 10:39 ` Tier Nolan
2015-07-11 12:09 ` Nathan Wilcox
@ 2015-07-12 18:37 ` Jorge Timón
2015-07-12 18:54 ` Tier Nolan
1 sibling, 1 reply; 13+ messages in thread
From: Jorge Timón @ 2015-07-12 18:37 UTC (permalink / raw)
To: Tier Nolan; +Cc: bitcoin-dev
On Sat, Jul 11, 2015 at 12:39 PM, Tier Nolan <tier.nolan@gmail•com> wrote:
>
>
> On Sat, Jul 11, 2015 at 10:24 AM, Jorge Timón <jtimon@jtimon•cc> wrote:
>>
>> I think it would be more rational for them to keep mining on top of the
>> old block until they've fully validated the new block (which shouldn't take
>> so long anyway), even if this slightly increases the orphan rate.
>
>
> Increased orphan rate means that the network is (slightly) less secure.
>
> If miners have a 5% orphan rate, then an attacker can launch a 51% attack
> with 49% of the network.
>
> It isn't a massive difference, but it is there.
If miners aren't validating the blocks they mine on top of, an
attacker can do more nasty things I think.
> As long as miners switch back to non-SPV mining after a timeout, SPV-mining
> is safe for everyone.
>
> The average cost to the miner from building on an invalid block is small, as
> long as invalid blocks only happen rarely.
>
> Miners still have an incentive to do full validation, so that they can
> include transactions and get transaction fees.
>
> SPV-mining is to prevent hashing hardware from having to waste power when it
> isn't needed.
As long as miners switch back to the new longest chain after they
validate the block, mining on top of the
non-most-work-but-surely-valid may be less risky than mining on top of
a most-work-but-potentially-invalid block.
This has risks too. In both cases, if they don't mine a block during
the block validation, everything is fine.
If they successfully SPV mine, they risk having mined on top of an
invalid block, which not only means lost coins for them but high risk
for regular SPV users.
If they successfully mine on top of the previous block, they start a
mini-race that they can win or not, but the impact to regular SPV
users is much lower.
The later may be slightly less profitable, but I bet the difference is
negligible. It would be interesting to know if miners actually did
this numbers and how (in case their model is incomplete or flawed).
It is important to note that while SPV mining requires you to produce
empty blocks, mining on the previous on top of the previous block
allows you to include transactions and earn fees.
In a future where block rewards aren't so overwhelmingly dominated by
subsidies, the numbers will run against SPV mining.
In a future without (or with negligible) subsidy, SPV mining is always
inferior to just keep mining on top of the same block you were mining
until you fully validate the next one.
> It may be less of a problem if (when?) electricity costs dominate hardware
> capital costs.
This seems correct (for both cases).
It's also less worrying the shorter the full validation time of a block is.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-12 18:37 ` Jorge Timón
@ 2015-07-12 18:54 ` Tier Nolan
0 siblings, 0 replies; 13+ messages in thread
From: Tier Nolan @ 2015-07-12 18:54 UTC (permalink / raw)
Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 1897 bytes --]
On Sun, Jul 12, 2015 at 7:37 PM, Jorge Timón <jtimon@jtimon•cc> wrote:
> As long as miners switch back to the new longest chain after they
> validate the block, mining on top of the
> non-most-work-but-surely-valid may be less risky than mining on top of
> a most-work-but-potentially-invalid block.
>
It depends on how long they are waiting. If they receive a header, it is
very likely to be part of a valid block.
The more time that passes, the more likely that the header's block was
invalid after all.
This tradeoff is what the timeout takes into account. For a short period
of time after the header is received, it is probably valid but eventually,
as time passes without it being fully validated, it is more likely to be
false after all.
If they successfully SPV mine, they risk having mined on top of an
> invalid block, which not only means lost coins for them but high risk
> for regular SPV users.
>
With a 1 minute timeout, there is only a 10% chance they will find another
block.
It is important that when a header is marked as "probably invalid" that all
the header's children are also updated too. The whole chain times out.
It is important to note that while SPV mining requires you to produce
> empty blocks, mining on the previous on top of the previous block
> allows you to include transactions and earn fees.
> In a future where block rewards aren't so overwhelmingly dominated by
> subsidies, the numbers will run against SPV mining.
>
Agreed. Transaction only fees changes the whole incentive structure.
A fee pool has been suggested to keep things as they are now. All fees
(mint & tx fees) are paid into a fee pool. 1% of the total pool fund is
paid to the coinbase.
This keeps the total payout per block reasonably stable. On the other
hand, it removes the incentive to actually include transactions at all.
[-- Attachment #2: Type: text/html, Size: 2644 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-11 9:24 ` Jorge Timón
2015-07-11 10:39 ` Tier Nolan
@ 2015-07-13 16:04 ` Peter Todd
2015-07-13 17:33 ` Eric Lombrozo
2015-07-13 17:49 ` Eric Lombrozo
1 sibling, 2 replies; 13+ messages in thread
From: Peter Todd @ 2015-07-13 16:04 UTC (permalink / raw)
To: Jorge Timón; +Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 2525 bytes --]
On Sat, Jul 11, 2015 at 11:24:48AM +0200, Jorge Timón wrote:
> All miners should validate transactions precisely because of the latest
> attack you've described. Full miners can gain a lot from this attack to
> leverage their full validation against spv miners who blindly spend energy
> hashing on top of something that may be worthless crap. SPV mining makes no
> sense, but some miners claim they're doind it for very short periods of
> time, which shouldn't be as bad as doing it all the time.
>
> I think it would be more rational for them to keep mining on top of the old
> block until they've fully validated the new block (which shouldn't take so
> long anyway), even if this slightly increases the orphan rate.
You're missing something really critical about what F2Pool/AntPool were
(are?) doing: They're finding out about new blocks not by getting block
headers from just anywhere, but by connecting to other pools' via
stratum anonymously and determining what block hash they're telling the
hashers at the pool to work on. (e.g. what prevblockhash is in the block
header of shares being generated)
If other pools try to fake this information they're immediately and
directly losing money, because they're telling their own hashers to make
invalid blocks. This of course has a high chance of being detected, and
can easily be FUDed into "STOP MINING AT FOO POOL!" reardless of what
the ivory tower game theory might say. The only hope the pools have is
to somehow identify which connections correspond to other pools with
high reliability and target just those connections - good luck on that.
Anyway, all this concern about SPV mining is misguided: relying purely
on SPV w/ low #'s of confirmations just isn't very smart. What SPV can
do - at least while the inflation subsidy is still high - is give
reasonable protection against your third-party-run trusted full nodes
from lying to you, simply because doing so has well-defined costs in
terms of energy to create fake blocks. Targetting enough people at once
to make a fake block a worthwhile investment is difficult, particularly
when you take into account how timing works in the defenders favor - the
attacker probably only has a small % of hashing power, so they're going
to wait a long time to find their fake block. Between that and a trusted
third party-run full node you're probably reasonably safe, for now.
--
'peter'[:-1]@petertodd.org
0000000000000000086007e31decd6eb80e07f77271ef50c69e1e6342161f4e5
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-13 16:04 ` Peter Todd
@ 2015-07-13 17:33 ` Eric Lombrozo
2015-07-13 17:49 ` Eric Lombrozo
1 sibling, 0 replies; 13+ messages in thread
From: Eric Lombrozo @ 2015-07-13 17:33 UTC (permalink / raw)
To: Peter Todd; +Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 2816 bytes --]
>
> On Sat, Jul 11, 2015 at 11:24:48AM +0200, Jorge Timón wrote:
> > All miners should validate transactions precisely because of the latest
> > attack you've described. Full miners can gain a lot from this attack to
> > leverage their full validation against spv miners who blindly spend
> energy
> > hashing on top of something that may be worthless crap. SPV mining makes
> no
> > sense, but some miners claim they're doind it for very short periods of
> > time, which shouldn't be as bad as doing it all the time.
> >
> > I think it would be more rational for them to keep mining on top of the
> old
> > block until they've fully validated the new block (which shouldn't take
> so
> > long anyway), even if this slightly increases the orphan rate.
>
> You're missing something really critical about what F2Pool/AntPool were
> (are?) doing: They're finding out about new blocks not by getting block
> headers from just anywhere, but by connecting to other pools' via
> stratum anonymously and determining what block hash they're telling the
> hashers at the pool to work on. (e.g. what prevblockhash is in the block
> header of shares being generated)
>
> If other pools try to fake this information they're immediately and
> directly losing money, because they're telling their own hashers to make
> invalid blocks. This of course has a high chance of being detected, and
> can easily be FUDed into "STOP MINING AT FOO POOL!" reardless of what
> the ivory tower game theory might say. The only hope the pools have is
> to somehow identify which connections correspond to other pools with
> high reliability and target just those connections - good luck on that.
>
>
> Anyway, all this concern about SPV mining is misguided: relying purely
> on SPV w/ low #'s of confirmations just isn't very smart. What SPV can
> do - at least while the inflation subsidy is still high - is give
> reasonable protection against your third-party-run trusted full nodes
> from lying to you, simply because doing so has well-defined costs in
> terms of energy to create fake blocks. Targetting enough people at once
> to make a fake block a worthwhile investment is difficult, particularly
> when you take into account how timing works in the defenders favor - the
> attacker probably only has a small % of hashing power, so they're going
> to wait a long time to find their fake block. Between that and a trusted
> third party-run full node you're probably reasonably safe, for now.
>
> --
> 'peter'[:-1]@petertodd.org
> 0000000000000000086007e31decd6eb80e07f77271ef50c69e1e6342161f4e5
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
[-- Attachment #2: Type: text/html, Size: 3361 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
2015-07-13 16:04 ` Peter Todd
2015-07-13 17:33 ` Eric Lombrozo
@ 2015-07-13 17:49 ` Eric Lombrozo
1 sibling, 0 replies; 13+ messages in thread
From: Eric Lombrozo @ 2015-07-13 17:49 UTC (permalink / raw)
To: Peter Todd; +Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 3497 bytes --]
My other email client messed up. I apologize for the blank message.
Anyhow...
Even though the cost of mining bad blocks is high enough to deter most
deliberate attacks, if we're not properly validating blocks, this
deterrence does not stop bugs nor version issues and it opens up attack
vectors like someone hacking into a mining pool server.
It is imperative we continue to look for ways to make secure validation
cheaper. I would make this #1 priority. Not only is it crucial to the
integrity of our security model - it is crucial for scalability and
decentralization as well.
- Eric Lombrozo
On Jul 13, 2015 11:05 AM, "Peter Todd" <pete@petertodd•org> wrote:
> On Sat, Jul 11, 2015 at 11:24:48AM +0200, Jorge Timón wrote:
> > All miners should validate transactions precisely because of the latest
> > attack you've described. Full miners can gain a lot from this attack to
> > leverage their full validation against spv miners who blindly spend
> energy
> > hashing on top of something that may be worthless crap. SPV mining makes
> no
> > sense, but some miners claim they're doind it for very short periods of
> > time, which shouldn't be as bad as doing it all the time.
> >
> > I think it would be more rational for them to keep mining on top of the
> old
> > block until they've fully validated the new block (which shouldn't take
> so
> > long anyway), even if this slightly increases the orphan rate.
>
> You're missing something really critical about what F2Pool/AntPool were
> (are?) doing: They're finding out about new blocks not by getting block
> headers from just anywhere, but by connecting to other pools' via
> stratum anonymously and determining what block hash they're telling the
> hashers at the pool to work on. (e.g. what prevblockhash is in the block
> header of shares being generated)
>
> If other pools try to fake this information they're immediately and
> directly losing money, because they're telling their own hashers to make
> invalid blocks. This of course has a high chance of being detected, and
> can easily be FUDed into "STOP MINING AT FOO POOL!" reardless of what
> the ivory tower game theory might say. The only hope the pools have is
> to somehow identify which connections correspond to other pools with
> high reliability and target just those connections - good luck on that.
>
>
> Anyway, all this concern about SPV mining is misguided: relying purely
> on SPV w/ low #'s of confirmations just isn't very smart. What SPV can
> do - at least while the inflation subsidy is still high - is give
> reasonable protection against your third-party-run trusted full nodes
> from lying to you, simply because doing so has well-defined costs in
> terms of energy to create fake blocks. Targetting enough people at once
> to make a fake block a worthwhile investment is difficult, particularly
> when you take into account how timing works in the defenders favor - the
> attacker probably only has a small % of hashing power, so they're going
> to wait a long time to find their fake block. Between that and a trusted
> third party-run full node you're probably reasonably safe, for now.
>
> --
> 'peter'[:-1]@petertodd.org
> 0000000000000000086007e31decd6eb80e07f77271ef50c69e1e6342161f4e5
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
[-- Attachment #2: Type: text/html, Size: 4226 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2015-07-13 17:49 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-07-11 8:05 [bitcoin-dev] SPV Mining reveals a problematic incentive issue Nathan Wilcox
2015-07-11 9:24 ` Jorge Timón
2015-07-11 10:39 ` Tier Nolan
2015-07-11 12:09 ` Nathan Wilcox
2015-07-11 13:17 ` Tier Nolan
2015-07-11 15:04 ` Dave Hudson
2015-07-11 16:26 ` Tier Nolan
2015-07-12 18:37 ` Jorge Timón
2015-07-12 18:54 ` Tier Nolan
2015-07-13 16:04 ` Peter Todd
2015-07-13 17:33 ` Eric Lombrozo
2015-07-13 17:49 ` Eric Lombrozo
2015-07-11 15:34 ` Jeff Garzik
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox