* [bitcoin-dev] Hash of UTXO set as consensus-critical @ 2015-09-18 19:05 Rune Kjær Svendsen 2015-09-18 19:43 ` Patrick Strateman 0 siblings, 1 reply; 14+ messages in thread From: Rune Kjær Svendsen @ 2015-09-18 19:05 UTC (permalink / raw) To: bitcoin-dev Currently, when a new node wants to join the network, it needs to retrieve the entire blockchain history, starting from January 2009 and up until now, in order to derive a UTXO set that it can verify new blocks/transactions against. With a blockchain size of 40GB and a UTXO size of around 1GB, the extra bandwidth required is significant, and will keep increasing indefinitely. If a newly mined block were to include the UTXO set hash of the chain up until the previous block — the hash of the UTXO set on top of which this block builds — then new nodes, who want to know whether a transaction is valid, would be able to acquire the UTXO set in a trustless manner, by only verifying proof-of-work headers, and knowing that a block with an invalid UTXO set hash would be rejected. I’m not talking about calculating a complicated tree structure from the UTXO set, which would put further burden on already burdened Bitcoin Core nodes. We simply include the hash of the current UTXO set in a newly created block, such that the transactions in the new block build *on top* of the UTXO set whose hash is specified. This actually alleviates Bitcoin Core nodes, as it will now become possible for nodes without the entire blockchain to answer SPV queries (by retrieving the UTXO set trustlessly and using this to answer queries). It also saves bandwidth for Bitcore Core nodes, who only need to send roughly 1GB of data, in order to synchronise a node, rather than 40GB+. I will continue to run a full Bitcoin Core node, saving the entire blockchain history, but it shouldn’t be a requirement to hold the entire transaction history in order to start verifying new transactions. As far as I can see, this also forces miners to actually maintain an UTXO set, rather than just build on top of the chain with the most proof-of-work. Producing a UTXO set and verifying a block against a chain is the same thing, so by including the hash of the UTXO set we force miners to verify the block that they want to build on top of. Am I missing something obvious, because as far as I can see, this solves the problem of quadratic time complexity for initial sync: http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s The only added step to verifying a block is to hash the UTXO set. So it does require additional computation, but most modern CPUs have a SHA256 throughput of around 500 MB/s, which means it takes only two seconds to hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). A small sacrifice for the added ease of initial syncing, in my opinion. /Rune ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-18 19:05 [bitcoin-dev] Hash of UTXO set as consensus-critical Rune Kjær Svendsen @ 2015-09-18 19:43 ` Patrick Strateman 2015-09-18 20:07 ` Alex Morcos 2015-09-18 20:17 ` Rune Kjær Svendsen 0 siblings, 2 replies; 14+ messages in thread From: Patrick Strateman @ 2015-09-18 19:43 UTC (permalink / raw) To: bitcoin-dev Full nodes using UTXO set commitments is a change to the bitcoin security model. Currently an attacker with >50% of the network hashrate can rewrite history. If full nodes rely on UTXO set commitments such an attacker could create an infinite number of bitcoins (as in many times more than the current 21 million bitcoin limit). Before we consider mechanisms for UTXO set commitments, we should seriously discuss whether the security model reduction is reasonable. On 09/18/2015 12:05 PM, Rune Kjær Svendsen via bitcoin-dev wrote: > Currently, when a new node wants to join the network, it needs to retrieve the entire blockchain history, starting from January 2009 and up until now, in order to derive a UTXO set that it can verify new blocks/transactions against. With a blockchain size of 40GB and a UTXO size of around 1GB, the extra bandwidth required is significant, and will keep increasing indefinitely. If a newly mined block were to include the UTXO set hash of the chain up until the previous block — the hash of the UTXO set on top of which this block builds — then new nodes, who want to know whether a transaction is valid, would be able to acquire the UTXO set in a trustless manner, by only verifying proof-of-work headers, and knowing that a block with an invalid UTXO set hash would be rejected. > > I’m not talking about calculating a complicated tree structure from the UTXO set, which would put further burden on already burdened Bitcoin Core nodes. We simply include the hash of the current UTXO set in a newly created block, such that the transactions in the new block build *on top* of the UTXO set whose hash is specified. This actually alleviates Bitcoin Core nodes, as it will now become possible for nodes without the entire blockchain to answer SPV queries (by retrieving the UTXO set trustlessly and using this to answer queries). It also saves bandwidth for Bitcore Core nodes, who only need to send roughly 1GB of data, in order to synchronise a node, rather than 40GB+. I will continue to run a full Bitcoin Core node, saving the entire blockchain history, but it shouldn’t be a requirement to hold the entire transaction history in order to start verifying new transactions. > > As far as I can see, this also forces miners to actually maintain an UTXO set, rather than just build on top of the chain with the most proof-of-work. Producing a UTXO set and verifying a block against a chain is the same thing, so by including the hash of the UTXO set we force miners to verify the block that they want to build on top of. > > Am I missing something obvious, because as far as I can see, this solves the problem of quadratic time complexity for initial sync: http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s > > The only added step to verifying a block is to hash the UTXO set. So it does require additional computation, but most modern CPUs have a SHA256 throughput of around 500 MB/s, which means it takes only two seconds to hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). A small sacrifice for the added ease of initial syncing, in my opinion. > > /Rune > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-18 19:43 ` Patrick Strateman @ 2015-09-18 20:07 ` Alex Morcos 2015-09-18 20:11 ` Matt Corallo 2015-09-18 20:17 ` Rune Kjær Svendsen 1 sibling, 1 reply; 14+ messages in thread From: Alex Morcos @ 2015-09-18 20:07 UTC (permalink / raw) To: Patrick Strateman; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 4004 bytes --] I guess I always assumed that UTXO set commitments were an alternative security model (between SPV and full-node), not that they would cause the existing security model to be deprecated. On Fri, Sep 18, 2015 at 3:43 PM, Patrick Strateman via bitcoin-dev < bitcoin-dev@lists•linuxfoundation.org> wrote: > Full nodes using UTXO set commitments is a change to the bitcoin > security model. > > Currently an attacker with >50% of the network hashrate can rewrite > history. > > If full nodes rely on UTXO set commitments such an attacker could create > an infinite number of bitcoins (as in many times more than the current > 21 million bitcoin limit). > > Before we consider mechanisms for UTXO set commitments, we should > seriously discuss whether the security model reduction is reasonable. > > On 09/18/2015 12:05 PM, Rune Kjær Svendsen via bitcoin-dev wrote: > > Currently, when a new node wants to join the network, it needs to > retrieve the entire blockchain history, starting from January 2009 and up > until now, in order to derive a UTXO set that it can verify new > blocks/transactions against. With a blockchain size of 40GB and a UTXO size > of around 1GB, the extra bandwidth required is significant, and will keep > increasing indefinitely. If a newly mined block were to include the UTXO > set hash of the chain up until the previous block — the hash of the UTXO > set on top of which this block builds — then new nodes, who want to know > whether a transaction is valid, would be able to acquire the UTXO set in a > trustless manner, by only verifying proof-of-work headers, and knowing that > a block with an invalid UTXO set hash would be rejected. > > > > I’m not talking about calculating a complicated tree structure from the > UTXO set, which would put further burden on already burdened Bitcoin Core > nodes. We simply include the hash of the current UTXO set in a newly > created block, such that the transactions in the new block build *on top* > of the UTXO set whose hash is specified. This actually alleviates Bitcoin > Core nodes, as it will now become possible for nodes without the entire > blockchain to answer SPV queries (by retrieving the UTXO set trustlessly > and using this to answer queries). It also saves bandwidth for Bitcore Core > nodes, who only need to send roughly 1GB of data, in order to synchronise a > node, rather than 40GB+. I will continue to run a full Bitcoin Core node, > saving the entire blockchain history, but it shouldn’t be a requirement to > hold the entire transaction history in order to start verifying new > transactions. > > > > As far as I can see, this also forces miners to actually maintain an > UTXO set, rather than just build on top of the chain with the most > proof-of-work. Producing a UTXO set and verifying a block against a chain > is the same thing, so by including the hash of the UTXO set we force miners > to verify the block that they want to build on top of. > > > > Am I missing something obvious, because as far as I can see, this solves > the problem of quadratic time complexity for initial sync: > http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s > > > > The only added step to verifying a block is to hash the UTXO set. So it > does require additional computation, but most modern CPUs have a SHA256 > throughput of around 500 MB/s, which means it takes only two seconds to > hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). > A small sacrifice for the added ease of initial syncing, in my opinion. > > > > /Rune > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists•linuxfoundation.org > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 4836 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-18 20:07 ` Alex Morcos @ 2015-09-18 20:11 ` Matt Corallo 0 siblings, 0 replies; 14+ messages in thread From: Matt Corallo @ 2015-09-18 20:11 UTC (permalink / raw) To: Alex Morcos, Patrick Strateman; +Cc: Bitcoin Dev I believe the discussion here is on improving initial-sync time by simply skipping initial-sync and getting a committed-to utxo set. This is obviously a new security model in between SPV and full-node (I would call it SPV with future validation). Still, I'm not convinced it buys us anything, we really should just tweak Bitcoin Core to do spv mode at startup and validate backwards in the background. I think this would alleviate most of the concerns raised, given the chain growth is not entirely unreasonable going forward. On 09/18/15 20:07, Alex Morcos via bitcoin-dev wrote: > I guess I always assumed that UTXO set commitments were an alternative > security model (between SPV and full-node), not that they would cause > the existing security model to be deprecated. > > > On Fri, Sep 18, 2015 at 3:43 PM, Patrick Strateman via bitcoin-dev > <bitcoin-dev@lists•linuxfoundation.org > <mailto:bitcoin-dev@lists•linuxfoundation.org>> wrote: > > Full nodes using UTXO set commitments is a change to the bitcoin > security model. > > Currently an attacker with >50% of the network hashrate can rewrite > history. > > If full nodes rely on UTXO set commitments such an attacker could create > an infinite number of bitcoins (as in many times more than the current > 21 million bitcoin limit). > > Before we consider mechanisms for UTXO set commitments, we should > seriously discuss whether the security model reduction is reasonable. > > On 09/18/2015 12:05 PM, Rune Kjær Svendsen via bitcoin-dev wrote: > > Currently, when a new node wants to join the network, it needs to > retrieve the entire blockchain history, starting from January 2009 > and up until now, in order to derive a UTXO set that it can verify > new blocks/transactions against. With a blockchain size of 40GB and > a UTXO size of around 1GB, the extra bandwidth required is > significant, and will keep increasing indefinitely. If a newly mined > block were to include the UTXO set hash of the chain up until the > previous block — the hash of the UTXO set on top of which this block > builds — then new nodes, who want to know whether a transaction is > valid, would be able to acquire the UTXO set in a trustless manner, > by only verifying proof-of-work headers, and knowing that a block > with an invalid UTXO set hash would be rejected. > > > > I’m not talking about calculating a complicated tree structure > from the UTXO set, which would put further burden on already > burdened Bitcoin Core nodes. We simply include the hash of the > current UTXO set in a newly created block, such that the > transactions in the new block build *on top* of the UTXO set whose > hash is specified. This actually alleviates Bitcoin Core nodes, as > it will now become possible for nodes without the entire blockchain > to answer SPV queries (by retrieving the UTXO set trustlessly and > using this to answer queries). It also saves bandwidth for Bitcore > Core nodes, who only need to send roughly 1GB of data, in order to > synchronise a node, rather than 40GB+. I will continue to run a full > Bitcoin Core node, saving the entire blockchain history, but it > shouldn’t be a requirement to hold the entire transaction history in > order to start verifying new transactions. > > > > As far as I can see, this also forces miners to actually maintain > an UTXO set, rather than just build on top of the chain with the > most proof-of-work. Producing a UTXO set and verifying a block > against a chain is the same thing, so by including the hash of the > UTXO set we force miners to verify the block that they want to build > on top of. > > > > Am I missing something obvious, because as far as I can see, this > solves the problem of quadratic time complexity for initial sync: > http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s > > > > The only added step to verifying a block is to hash the UTXO set. > So it does require additional computation, but most modern CPUs have > a SHA256 throughput of around 500 MB/s, which means it takes only > two seconds to hash the UTXO set. And this can be improved further > (GPUs can do 2-3 GB/s). A small sacrifice for the added ease of > initial syncing, in my opinion. > > > > /Rune > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists•linuxfoundation.org > <mailto:bitcoin-dev@lists•linuxfoundation.org> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > <mailto:bitcoin-dev@lists•linuxfoundation.org> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-18 19:43 ` Patrick Strateman 2015-09-18 20:07 ` Alex Morcos @ 2015-09-18 20:17 ` Rune Kjær Svendsen 2015-09-18 20:37 ` Jorge Timón 2015-09-19 2:30 ` Justus Ranvier 1 sibling, 2 replies; 14+ messages in thread From: Rune Kjær Svendsen @ 2015-09-18 20:17 UTC (permalink / raw) To: Patrick Strateman; +Cc: bitcoin-dev There are a couple of points I’d like to address. Firstly, yes, >50% attacks are a problem for Bitcoin. Bitcoin does not function if the majority of mining power is dishonest. There is no way around that. It’s how proof-of-work functions. And if we lose proof-of-work, we lose Bitcoin. Secondly, I’m not suggesting that UTXO set hashes *replace* block hashes, or even that it should be in the block header (probably in the coinbase somewhere). I suggest it as an *addition* to the existing consensus rules. Full nodes can still verify the chain with the added step of hashing the UTXO set for every block. Of course, this can easily be deferred to after proof-of-work has been verified already, such that no work is wasted. Unless a 51% attack is in effect. But I argue that this is a moot point, since Bitcoin is useless anyway under such circumstances. Lastly, I’m not suggesting miners discard the blockchain history. A miner has an incentive to be absolutely sure that the chain he’s building on is the right one. If he’s wrong, he loses money/income. There’s simply no reason for a professional miner *not* to do the full initial sync, which only needs to be done once. Non-miners, who just want to check the balance of their wallet, however, really don’t need to retrieve information about Hal Finney sending bitcoins to Satoshi in 2010. In any case, this practice isn’t sustainable. In the end, it isn’t possible to control whether a miner verifies the entire blockchain anyway (anyone can send the UTXO set over the wire). Not letting the proof-of-work cover the UTXO hash doesn’t solve this problem, it only makes it impossible to know whether a given UTXO set is the one that the majority is mining on without retrieving the entire blockchain, and doing the verification yourself. People can choose to skip that regardless of what we do. Furthermore, all nodes have the option of deciding which level of security they want. We’re not lessening security of the protocol, we’re strengthening the security of something that’s already possible to do (build on top of an unverified blockchain), but we’d rather want that people not do. /Rune > On 18 Sep 2015, at 21:43, Patrick Strateman via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote: > > Full nodes using UTXO set commitments is a change to the bitcoin > security model. > > Currently an attacker with >50% of the network hashrate can rewrite history. > > If full nodes rely on UTXO set commitments such an attacker could create > an infinite number of bitcoins (as in many times more than the current > 21 million bitcoin limit). > > Before we consider mechanisms for UTXO set commitments, we should > seriously discuss whether the security model reduction is reasonable. > > On 09/18/2015 12:05 PM, Rune Kjær Svendsen via bitcoin-dev wrote: >> Currently, when a new node wants to join the network, it needs to retrieve the entire blockchain history, starting from January 2009 and up until now, in order to derive a UTXO set that it can verify new blocks/transactions against. With a blockchain size of 40GB and a UTXO size of around 1GB, the extra bandwidth required is significant, and will keep increasing indefinitely. If a newly mined block were to include the UTXO set hash of the chain up until the previous block — the hash of the UTXO set on top of which this block builds — then new nodes, who want to know whether a transaction is valid, would be able to acquire the UTXO set in a trustless manner, by only verifying proof-of-work headers, and knowing that a block with an invalid UTXO set hash would be rejected. >> >> I’m not talking about calculating a complicated tree structure from the UTXO set, which would put further burden on already burdened Bitcoin Core nodes. We simply include the hash of the current UTXO set in a newly created block, such that the transactions in the new block build *on top* of the UTXO set whose hash is specified. This actually alleviates Bitcoin Core nodes, as it will now become possible for nodes without the entire blockchain to answer SPV queries (by retrieving the UTXO set trustlessly and using this to answer queries). It also saves bandwidth for Bitcore Core nodes, who only need to send roughly 1GB of data, in order to synchronise a node, rather than 40GB+. I will continue to run a full Bitcoin Core node, saving the entire blockchain history, but it shouldn’t be a requirement to hold the entire transaction history in order to start verifying new transactions. >> >> As far as I can see, this also forces miners to actually maintain an UTXO set, rather than just build on top of the chain with the most proof-of-work. Producing a UTXO set and verifying a block against a chain is the same thing, so by including the hash of the UTXO set we force miners to verify the block that they want to build on top of. >> >> Am I missing something obvious, because as far as I can see, this solves the problem of quadratic time complexity for initial sync: http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s >> >> The only added step to verifying a block is to hash the UTXO set. So it does require additional computation, but most modern CPUs have a SHA256 throughput of around 500 MB/s, which means it takes only two seconds to hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). A small sacrifice for the added ease of initial syncing, in my opinion. >> >> /Rune >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-18 20:17 ` Rune Kjær Svendsen @ 2015-09-18 20:37 ` Jorge Timón 2015-09-18 20:38 ` Jorge Timón 2015-09-19 2:30 ` Justus Ranvier 1 sibling, 1 reply; 14+ messages in thread From: Jorge Timón @ 2015-09-18 20:37 UTC (permalink / raw) To: Rune Kjær Svendsen; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 7168 bytes --] Well, with utxo commitments at some point maybe is enough to validate the full headers history but only the last 5 years of ttansaction history (assuming utxo commitments are buried 5 years worth of blocks in the past). This scales much better than validating the full history and if we get a 5 year reorg something is going really wrong anyway... Maybe after validating the last 5 years you also want to validate the rest of the history backards to get the "fully-full node" security. Of course 5 years it's just an arbitrary number: 2 or maybe even 1 would probably be secure enough for most people. I've referred to this idea as "hard checkpoints" or "moving the genesis block forward" in the past. On Sep 18, 2015 4:18 PM, "Rune Kjær Svendsen" < bitcoin-dev@lists•linuxfoundation.org> wrote: > There are a couple of points I’d like to address. > > Firstly, yes, >50% attacks are a problem for Bitcoin. Bitcoin does not > function if the majority of mining power is dishonest. There is no way > around that. It’s how proof-of-work functions. And if we lose > proof-of-work, we lose Bitcoin. > > Secondly, I’m not suggesting that UTXO set hashes *replace* block hashes, > or even that it should be in the block header (probably in the coinbase > somewhere). I suggest it as an *addition* to the existing consensus rules. > Full nodes can still verify the chain with the added step of hashing the > UTXO set for every block. Of course, this can easily be deferred to after > proof-of-work has been verified already, such that no work is wasted. > Unless a 51% attack is in effect. But I argue that this is a moot point, > since Bitcoin is useless anyway under such circumstances. > > Lastly, I’m not suggesting miners discard the blockchain history. A miner > has an incentive to be absolutely sure that the chain he’s building on is > the right one. If he’s wrong, he loses money/income. There’s simply no > reason for a professional miner *not* to do the full initial sync, which > only needs to be done once. Non-miners, who just want to check the balance > of their wallet, however, really don’t need to retrieve information about > Hal Finney sending bitcoins to Satoshi in 2010. In any case, this practice > isn’t sustainable. > > In the end, it isn’t possible to control whether a miner verifies the > entire blockchain anyway (anyone can send the UTXO set over the wire). Not > letting the proof-of-work cover the UTXO hash doesn’t solve this problem, > it only makes it impossible to know whether a given UTXO set is the one > that the majority is mining on without retrieving the entire blockchain, > and doing the verification yourself. People can choose to skip that > regardless of what we do. > > Furthermore, all nodes have the option of deciding which level of security > they want. We’re not lessening security of the protocol, we’re > strengthening the security of something that’s already possible to do > (build on top of an unverified blockchain), but we’d rather want that > people not do. > > /Rune > > > > On 18 Sep 2015, at 21:43, Patrick Strateman via bitcoin-dev < > bitcoin-dev@lists•linuxfoundation.org> wrote: > > > > Full nodes using UTXO set commitments is a change to the bitcoin > > security model. > > > > Currently an attacker with >50% of the network hashrate can rewrite > history. > > > > If full nodes rely on UTXO set commitments such an attacker could create > > an infinite number of bitcoins (as in many times more than the current > > 21 million bitcoin limit). > > > > Before we consider mechanisms for UTXO set commitments, we should > > seriously discuss whether the security model reduction is reasonable. > > > > On 09/18/2015 12:05 PM, Rune Kjær Svendsen via bitcoin-dev wrote: > >> Currently, when a new node wants to join the network, it needs to > retrieve the entire blockchain history, starting from January 2009 and up > until now, in order to derive a UTXO set that it can verify new > blocks/transactions against. With a blockchain size of 40GB and a UTXO size > of around 1GB, the extra bandwidth required is significant, and will keep > increasing indefinitely. If a newly mined block were to include the UTXO > set hash of the chain up until the previous block — the hash of the UTXO > set on top of which this block builds — then new nodes, who want to know > whether a transaction is valid, would be able to acquire the UTXO set in a > trustless manner, by only verifying proof-of-work headers, and knowing that > a block with an invalid UTXO set hash would be rejected. > >> > >> I’m not talking about calculating a complicated tree structure from the > UTXO set, which would put further burden on already burdened Bitcoin Core > nodes. We simply include the hash of the current UTXO set in a newly > created block, such that the transactions in the new block build *on top* > of the UTXO set whose hash is specified. This actually alleviates Bitcoin > Core nodes, as it will now become possible for nodes without the entire > blockchain to answer SPV queries (by retrieving the UTXO set trustlessly > and using this to answer queries). It also saves bandwidth for Bitcore Core > nodes, who only need to send roughly 1GB of data, in order to synchronise a > node, rather than 40GB+. I will continue to run a full Bitcoin Core node, > saving the entire blockchain history, but it shouldn’t be a requirement to > hold the entire transaction history in order to start verifying new > transactions. > >> > >> As far as I can see, this also forces miners to actually maintain an > UTXO set, rather than just build on top of the chain with the most > proof-of-work. Producing a UTXO set and verifying a block against a chain > is the same thing, so by including the hash of the UTXO set we force miners > to verify the block that they want to build on top of. > >> > >> Am I missing something obvious, because as far as I can see, this > solves the problem of quadratic time complexity for initial sync: > http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s > >> > >> The only added step to verifying a block is to hash the UTXO set. So it > does require additional computation, but most modern CPUs have a SHA256 > throughput of around 500 MB/s, which means it takes only two seconds to > hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). > A small sacrifice for the added ease of initial syncing, in my opinion. > >> > >> /Rune > >> _______________________________________________ > >> bitcoin-dev mailing list > >> bitcoin-dev@lists•linuxfoundation.org > >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > > > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists•linuxfoundation.org > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > [-- Attachment #2: Type: text/html, Size: 8244 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-18 20:37 ` Jorge Timón @ 2015-09-18 20:38 ` Jorge Timón 2015-09-18 22:22 ` Vincent Truong 0 siblings, 1 reply; 14+ messages in thread From: Jorge Timón @ 2015-09-18 20:38 UTC (permalink / raw) To: Rune Kjær Svendsen; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 7455 bytes --] s/move the genesis block forward/move your genesis checkpoint forward/ On Sep 18, 2015 4:37 PM, "Jorge Timón" <jtimon@jtimon•cc> wrote: > Well, with utxo commitments at some point maybe is enough to validate the > full headers history but only the last 5 years of ttansaction history > (assuming utxo commitments are buried 5 years worth of blocks in the past). > This scales much better than validating the full history and if we get a 5 > year reorg something is going really wrong anyway... > Maybe after validating the last 5 years you also want to validate the rest > of the history backards to get the "fully-full node" security. > Of course 5 years it's just an arbitrary number: 2 or maybe even 1 would > probably be secure enough for most people. I've referred to this idea as > "hard checkpoints" or "moving the genesis block forward" in the past. > On Sep 18, 2015 4:18 PM, "Rune Kjær Svendsen" < > bitcoin-dev@lists•linuxfoundation.org> wrote: > >> There are a couple of points I’d like to address. >> >> Firstly, yes, >50% attacks are a problem for Bitcoin. Bitcoin does not >> function if the majority of mining power is dishonest. There is no way >> around that. It’s how proof-of-work functions. And if we lose >> proof-of-work, we lose Bitcoin. >> >> Secondly, I’m not suggesting that UTXO set hashes *replace* block hashes, >> or even that it should be in the block header (probably in the coinbase >> somewhere). I suggest it as an *addition* to the existing consensus rules. >> Full nodes can still verify the chain with the added step of hashing the >> UTXO set for every block. Of course, this can easily be deferred to after >> proof-of-work has been verified already, such that no work is wasted. >> Unless a 51% attack is in effect. But I argue that this is a moot point, >> since Bitcoin is useless anyway under such circumstances. >> >> Lastly, I’m not suggesting miners discard the blockchain history. A miner >> has an incentive to be absolutely sure that the chain he’s building on is >> the right one. If he’s wrong, he loses money/income. There’s simply no >> reason for a professional miner *not* to do the full initial sync, which >> only needs to be done once. Non-miners, who just want to check the balance >> of their wallet, however, really don’t need to retrieve information about >> Hal Finney sending bitcoins to Satoshi in 2010. In any case, this practice >> isn’t sustainable. >> >> In the end, it isn’t possible to control whether a miner verifies the >> entire blockchain anyway (anyone can send the UTXO set over the wire). Not >> letting the proof-of-work cover the UTXO hash doesn’t solve this problem, >> it only makes it impossible to know whether a given UTXO set is the one >> that the majority is mining on without retrieving the entire blockchain, >> and doing the verification yourself. People can choose to skip that >> regardless of what we do. >> >> Furthermore, all nodes have the option of deciding which level of >> security they want. We’re not lessening security of the protocol, we’re >> strengthening the security of something that’s already possible to do >> (build on top of an unverified blockchain), but we’d rather want that >> people not do. >> >> /Rune >> >> >> > On 18 Sep 2015, at 21:43, Patrick Strateman via bitcoin-dev < >> bitcoin-dev@lists•linuxfoundation.org> wrote: >> > >> > Full nodes using UTXO set commitments is a change to the bitcoin >> > security model. >> > >> > Currently an attacker with >50% of the network hashrate can rewrite >> history. >> > >> > If full nodes rely on UTXO set commitments such an attacker could create >> > an infinite number of bitcoins (as in many times more than the current >> > 21 million bitcoin limit). >> > >> > Before we consider mechanisms for UTXO set commitments, we should >> > seriously discuss whether the security model reduction is reasonable. >> > >> > On 09/18/2015 12:05 PM, Rune Kjær Svendsen via bitcoin-dev wrote: >> >> Currently, when a new node wants to join the network, it needs to >> retrieve the entire blockchain history, starting from January 2009 and up >> until now, in order to derive a UTXO set that it can verify new >> blocks/transactions against. With a blockchain size of 40GB and a UTXO size >> of around 1GB, the extra bandwidth required is significant, and will keep >> increasing indefinitely. If a newly mined block were to include the UTXO >> set hash of the chain up until the previous block — the hash of the UTXO >> set on top of which this block builds — then new nodes, who want to know >> whether a transaction is valid, would be able to acquire the UTXO set in a >> trustless manner, by only verifying proof-of-work headers, and knowing that >> a block with an invalid UTXO set hash would be rejected. >> >> >> >> I’m not talking about calculating a complicated tree structure from >> the UTXO set, which would put further burden on already burdened Bitcoin >> Core nodes. We simply include the hash of the current UTXO set in a newly >> created block, such that the transactions in the new block build *on top* >> of the UTXO set whose hash is specified. This actually alleviates Bitcoin >> Core nodes, as it will now become possible for nodes without the entire >> blockchain to answer SPV queries (by retrieving the UTXO set trustlessly >> and using this to answer queries). It also saves bandwidth for Bitcore Core >> nodes, who only need to send roughly 1GB of data, in order to synchronise a >> node, rather than 40GB+. I will continue to run a full Bitcoin Core node, >> saving the entire blockchain history, but it shouldn’t be a requirement to >> hold the entire transaction history in order to start verifying new >> transactions. >> >> >> >> As far as I can see, this also forces miners to actually maintain an >> UTXO set, rather than just build on top of the chain with the most >> proof-of-work. Producing a UTXO set and verifying a block against a chain >> is the same thing, so by including the hash of the UTXO set we force miners >> to verify the block that they want to build on top of. >> >> >> >> Am I missing something obvious, because as far as I can see, this >> solves the problem of quadratic time complexity for initial sync: >> http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s >> >> >> >> The only added step to verifying a block is to hash the UTXO set. So >> it does require additional computation, but most modern CPUs have a SHA256 >> throughput of around 500 MB/s, which means it takes only two seconds to >> hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). >> A small sacrifice for the added ease of initial syncing, in my opinion. >> >> >> >> /Rune >> >> _______________________________________________ >> >> bitcoin-dev mailing list >> >> bitcoin-dev@lists•linuxfoundation.org >> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > >> > >> > _______________________________________________ >> > bitcoin-dev mailing list >> > bitcoin-dev@lists•linuxfoundation.org >> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > [-- Attachment #2: Type: text/html, Size: 8665 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-18 20:38 ` Jorge Timón @ 2015-09-18 22:22 ` Vincent Truong 0 siblings, 0 replies; 14+ messages in thread From: Vincent Truong @ 2015-09-18 22:22 UTC (permalink / raw) To: Jorge Timón; +Cc: bitcoin-dev [-- Attachment #1: Type: text/plain, Size: 9009 bytes --] This way lets us protect from 51% overwrites for a whole year: 1. Hash utxo set as is today, H1, and store it in a block. 2. At the same time, store a copy of the utxo set for H1 on disk, D1 3. After 1 year, create D2, then wait for H2 (if a fork happens then recreate D2 and see which wins) 4. The block with H2 must hash on top of H1 4. Blocks before H1 can be safely pruned by the network, only keeping D1 for reference/validation, plus blocks the node wants to keep (data/colored coins) 5. After 1 year, repeat for H3. 7. D1 can also be dropped after a few days once D3 is up, since the H1 securing D1 would have been pruned with H3's usage of D2 by then. This reduces the security model from 'always secure' to 'secure, as of last year'. An attacker will need to have hidden hashing power to overwrite a years worth of blocks. Which I think would be hard enough. The attacker can also try to undo a freshly built Hn, but because we can build Dn first and wait for Hn, the nodes should be expecting the same hash. They also serve as automated checkpoints to prevent them from overwriting all of it. On Sep 19, 2015 6:38 AM, "Jorge Timón" < bitcoin-dev@lists•linuxfoundation.org> wrote: > s/move the genesis block forward/move your genesis checkpoint forward/ > On Sep 18, 2015 4:37 PM, "Jorge Timón" <jtimon@jtimon•cc> wrote: > >> Well, with utxo commitments at some point maybe is enough to validate the >> full headers history but only the last 5 years of ttansaction history >> (assuming utxo commitments are buried 5 years worth of blocks in the past). >> This scales much better than validating the full history and if we get a 5 >> year reorg something is going really wrong anyway... >> Maybe after validating the last 5 years you also want to validate the >> rest of the history backards to get the "fully-full node" security. >> Of course 5 years it's just an arbitrary number: 2 or maybe even 1 would >> probably be secure enough for most people. I've referred to this idea as >> "hard checkpoints" or "moving the genesis block forward" in the past. >> On Sep 18, 2015 4:18 PM, "Rune Kjær Svendsen" < >> bitcoin-dev@lists•linuxfoundation.org> wrote: >> >>> There are a couple of points I’d like to address. >>> >>> Firstly, yes, >50% attacks are a problem for Bitcoin. Bitcoin does not >>> function if the majority of mining power is dishonest. There is no way >>> around that. It’s how proof-of-work functions. And if we lose >>> proof-of-work, we lose Bitcoin. >>> >>> Secondly, I’m not suggesting that UTXO set hashes *replace* block >>> hashes, or even that it should be in the block header (probably in the >>> coinbase somewhere). I suggest it as an *addition* to the existing >>> consensus rules. Full nodes can still verify the chain with the added step >>> of hashing the UTXO set for every block. Of course, this can easily be >>> deferred to after proof-of-work has been verified already, such that no >>> work is wasted. Unless a 51% attack is in effect. But I argue that this is >>> a moot point, since Bitcoin is useless anyway under such circumstances. >>> >>> Lastly, I’m not suggesting miners discard the blockchain history. A >>> miner has an incentive to be absolutely sure that the chain he’s building >>> on is the right one. If he’s wrong, he loses money/income. There’s simply >>> no reason for a professional miner *not* to do the full initial sync, which >>> only needs to be done once. Non-miners, who just want to check the balance >>> of their wallet, however, really don’t need to retrieve information about >>> Hal Finney sending bitcoins to Satoshi in 2010. In any case, this practice >>> isn’t sustainable. >>> >>> In the end, it isn’t possible to control whether a miner verifies the >>> entire blockchain anyway (anyone can send the UTXO set over the wire). Not >>> letting the proof-of-work cover the UTXO hash doesn’t solve this problem, >>> it only makes it impossible to know whether a given UTXO set is the one >>> that the majority is mining on without retrieving the entire blockchain, >>> and doing the verification yourself. People can choose to skip that >>> regardless of what we do. >>> >>> Furthermore, all nodes have the option of deciding which level of >>> security they want. We’re not lessening security of the protocol, we’re >>> strengthening the security of something that’s already possible to do >>> (build on top of an unverified blockchain), but we’d rather want that >>> people not do. >>> >>> /Rune >>> >>> >>> > On 18 Sep 2015, at 21:43, Patrick Strateman via bitcoin-dev < >>> bitcoin-dev@lists•linuxfoundation.org> wrote: >>> > >>> > Full nodes using UTXO set commitments is a change to the bitcoin >>> > security model. >>> > >>> > Currently an attacker with >50% of the network hashrate can rewrite >>> history. >>> > >>> > If full nodes rely on UTXO set commitments such an attacker could >>> create >>> > an infinite number of bitcoins (as in many times more than the current >>> > 21 million bitcoin limit). >>> > >>> > Before we consider mechanisms for UTXO set commitments, we should >>> > seriously discuss whether the security model reduction is reasonable. >>> > >>> > On 09/18/2015 12:05 PM, Rune Kjær Svendsen via bitcoin-dev wrote: >>> >> Currently, when a new node wants to join the network, it needs to >>> retrieve the entire blockchain history, starting from January 2009 and up >>> until now, in order to derive a UTXO set that it can verify new >>> blocks/transactions against. With a blockchain size of 40GB and a UTXO size >>> of around 1GB, the extra bandwidth required is significant, and will keep >>> increasing indefinitely. If a newly mined block were to include the UTXO >>> set hash of the chain up until the previous block — the hash of the UTXO >>> set on top of which this block builds — then new nodes, who want to know >>> whether a transaction is valid, would be able to acquire the UTXO set in a >>> trustless manner, by only verifying proof-of-work headers, and knowing that >>> a block with an invalid UTXO set hash would be rejected. >>> >> >>> >> I’m not talking about calculating a complicated tree structure from >>> the UTXO set, which would put further burden on already burdened Bitcoin >>> Core nodes. We simply include the hash of the current UTXO set in a newly >>> created block, such that the transactions in the new block build *on top* >>> of the UTXO set whose hash is specified. This actually alleviates Bitcoin >>> Core nodes, as it will now become possible for nodes without the entire >>> blockchain to answer SPV queries (by retrieving the UTXO set trustlessly >>> and using this to answer queries). It also saves bandwidth for Bitcore Core >>> nodes, who only need to send roughly 1GB of data, in order to synchronise a >>> node, rather than 40GB+. I will continue to run a full Bitcoin Core node, >>> saving the entire blockchain history, but it shouldn’t be a requirement to >>> hold the entire transaction history in order to start verifying new >>> transactions. >>> >> >>> >> As far as I can see, this also forces miners to actually maintain an >>> UTXO set, rather than just build on top of the chain with the most >>> proof-of-work. Producing a UTXO set and verifying a block against a chain >>> is the same thing, so by including the hash of the UTXO set we force miners >>> to verify the block that they want to build on top of. >>> >> >>> >> Am I missing something obvious, because as far as I can see, this >>> solves the problem of quadratic time complexity for initial sync: >>> http://www.youtube.com/watch?v=TgjrS-BPWDQ&t=2h02m12s >>> >> >>> >> The only added step to verifying a block is to hash the UTXO set. So >>> it does require additional computation, but most modern CPUs have a SHA256 >>> throughput of around 500 MB/s, which means it takes only two seconds to >>> hash the UTXO set. And this can be improved further (GPUs can do 2-3 GB/s). >>> A small sacrifice for the added ease of initial syncing, in my opinion. >>> >> >>> >> /Rune >>> >> _______________________________________________ >>> >> bitcoin-dev mailing list >>> >> bitcoin-dev@lists•linuxfoundation.org >>> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> > >>> > >>> > _______________________________________________ >>> > bitcoin-dev mailing list >>> > bitcoin-dev@lists•linuxfoundation.org >>> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists•linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > [-- Attachment #2: Type: text/html, Size: 10593 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-18 20:17 ` Rune Kjær Svendsen 2015-09-18 20:37 ` Jorge Timón @ 2015-09-19 2:30 ` Justus Ranvier 2015-09-19 15:45 ` Rune Kjær Svendsen 1 sibling, 1 reply; 14+ messages in thread From: Justus Ranvier @ 2015-09-19 2:30 UTC (permalink / raw) To: bitcoin-dev [-- Attachment #1.1: Type: text/plain, Size: 1230 bytes --] On 18/09/15 15:17, Rune Kjær Svendsen via bitcoin-dev wrote: > Bitcoin does not function if the majority of mining power is dishonest. There is no way around that. It’s how proof-of-work functions. None of those statements are true. If a majority of Bitcoin miners are mining invalid blocks, then they aren't Bitcoin miners any more and are no longer relevant to the Bitcoin consensus. There does exist a problem that light clients aren't always able to tell the difference between chains that are valid and chains that are not valid, but it's is possible to create simple proofs that would do so: https://gist.github.com/justusranvier/451616fa4697b5f25f60 If those changes would be implemented, then any node that knew a chain was invalid could produce a compact proof that anyone else in the network could verify, regardless of how much proof of work was used to create the invalid chain. Committed UTXO sets would need safe to rely upon if a similar set of proofs that a particular set was invalid existed. -- Justus Ranvier Open Bitcoin Privacy Project http://www.openbitcoinprivacyproject.org/ justus@openbitcoinprivacyproject•org E7AD 8215 8497 3673 6D9E 61C4 2A5F DA70 EAD9 E623 [-- Attachment #1.2: 0xEAD9E623.asc --] [-- Type: application/pgp-keys, Size: 18729 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-19 2:30 ` Justus Ranvier @ 2015-09-19 15:45 ` Rune Kjær Svendsen 2015-09-19 17:19 ` Justus Ranvier 0 siblings, 1 reply; 14+ messages in thread From: Rune Kjær Svendsen @ 2015-09-19 15:45 UTC (permalink / raw) To: Justus Ranvier; +Cc: bitcoin-dev We need to distinguish between two different things here: 1) A 51% attack, where the majority of mining power is *malicious* (hence “attack”) and 2) A fork that exists because of a disagreement in the network, with total mining power split in two camps, each camp mining peacefully on their own chain These are two very different scenarios. Some claim that including the UTXO set hash in the block creates a vulnerability where miners can include the wrong UTXO hash, and mine on that, but this is only possible if a 51% attack is in effect. And if a 51% attack is in effect, it’s a moot point, because Bitcoin is useless anyway, because that 51% malicious mining power could just as well be used for mining empty blocks on top of the official chain. Or blocks containing randomly generated transactions, without confirming any legitimate transactions. This is why we say that a majority of honest miners is a hard requirement for Bitcoin. A majority of dishonest miners can only be circumvented through centralisation, and then we don’t have Bitcoin any longer. Scenario 2 is unproblematic regardless of whether we include the UTXO hash in the block (and make it consensus-critical) or not, since the majority mining power on either chain isn’t malicious. It is correct that if you’re a full node, and a 51% attack is in effect, you are able to verify that miners are honest (ie. you know whether a 51% attack is in effect or not). But this doesn’t change the fact that the Bitcoin network is unreliable, at best, when a majority of mining power is used for malicious purposes. /Rune > On 19 Sep 2015, at 04:30, Justus Ranvier via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote: > > On 18/09/15 15:17, Rune Kjær Svendsen via bitcoin-dev wrote: >> Bitcoin does not function if the majority of mining power is dishonest. There is no way around that. It’s how proof-of-work functions. > > > None of those statements are true. > > If a majority of Bitcoin miners are mining invalid blocks, then they > aren't Bitcoin miners any more and are no longer relevant to the Bitcoin > consensus. > > There does exist a problem that light clients aren't always able to tell > the difference between chains that are valid and chains that are not > valid, but it's is possible to create simple proofs that would do so: > > https://gist.github.com/justusranvier/451616fa4697b5f25f60 > > > If those changes would be implemented, then any node that knew a chain > was invalid could produce a compact proof that anyone else in the > network could verify, regardless of how much proof of work was used to > create the invalid chain. > > Committed UTXO sets would need safe to rely upon if a similar set of > proofs that a particular set was invalid existed. > > -- > Justus Ranvier > Open Bitcoin Privacy Project > http://www.openbitcoinprivacyproject.org/ > justus@openbitcoinprivacyproject•org > E7AD 8215 8497 3673 6D9E 61C4 2A5F DA70 EAD9 E623 > <0xEAD9E623.asc>_______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-19 15:45 ` Rune Kjær Svendsen @ 2015-09-19 17:19 ` Justus Ranvier 2015-09-19 20:11 ` Rune K. Svendsen 0 siblings, 1 reply; 14+ messages in thread From: Justus Ranvier @ 2015-09-19 17:19 UTC (permalink / raw) To: Rune Kjær Svendsen; +Cc: bitcoin-dev [-- Attachment #1.1: Type: text/plain, Size: 586 bytes --] On 19/09/15 10:45, Rune Kjær Svendsen wrote: > We need to distinguish between two different things here: > > 1) A 51% attack, where the majority of mining power is *malicious* (hence “attack”) What does "malicious" mean? In other words, If miner A is mining honestly, and miner B is mining maliciously, what are some of the possible difference in their behaviour we would observe? -- Justus Ranvier Open Bitcoin Privacy Project http://www.openbitcoinprivacyproject.org/ justus@openbitcoinprivacyproject•org E7AD 8215 8497 3673 6D9E 61C4 2A5F DA70 EAD9 E623 [-- Attachment #1.2: 0xEAD9E623.asc --] [-- Type: application/pgp-keys, Size: 18729 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-19 17:19 ` Justus Ranvier @ 2015-09-19 20:11 ` Rune K. Svendsen 2015-09-20 0:48 ` Dave Scotese 2015-09-21 17:15 ` Justus Ranvier 0 siblings, 2 replies; 14+ messages in thread From: Rune K. Svendsen @ 2015-09-19 20:11 UTC (permalink / raw) To: Justus Ranvier; +Cc: bitcoin-dev An honest miner is a miner that supports the network by building on top of the best valid chain. A malicious miner is one who wants to disrupt the Bitcoin network, not support it, for example by executing a 51% attack which mines empty blocks on top of the best chain. /Rune > Den 19/09/2015 kl. 19.19 skrev Justus Ranvier <justus@openbitcoinprivacyproject•org>: > >> On 19/09/15 10:45, Rune Kjær Svendsen wrote: >> We need to distinguish between two different things here: >> >> 1) A 51% attack, where the majority of mining power is *malicious* (hence “attack”) > > What does "malicious" mean? > > In other words, If miner A is mining honestly, and miner B is mining > maliciously, what are some of the possible difference in their behaviour > we would observe? > > > -- > Justus Ranvier > Open Bitcoin Privacy Project > http://www.openbitcoinprivacyproject.org/ > justus@openbitcoinprivacyproject•org > E7AD 8215 8497 3673 6D9E 61C4 2A5F DA70 EAD9 E623 > <0xEAD9E623.asc> ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-19 20:11 ` Rune K. Svendsen @ 2015-09-20 0:48 ` Dave Scotese 2015-09-21 17:15 ` Justus Ranvier 1 sibling, 0 replies; 14+ messages in thread From: Dave Scotese @ 2015-09-20 0:48 UTC (permalink / raw) To: Rune K. Svendsen; +Cc: bitcoin-dev [-- Attachment #1: Type: text/plain, Size: 4323 bytes --] It seems there should be a practical limit to the size of a re-org - I mean a practical limit that is smaller than the current height. Vincent's proposal suggests that a year's worth of blocks is such a practical limit. I agree. There are probably lower limits that are practical too, but I like an entire year just to be conservative. As Vincent points out, "An attacker will need to have hidden hashing power to overwrite a years worth of blocks." TL;DR for the rest of this: Txns that lose confirmations from a reorg and then show up in the mempool but not in any of the next few blocks indicate malicious mining. I see a blind spot here. We are seeing the rule that says the longest chain is the valid chain as impossible to break, but it isn't. We broke it to fix the BerkelyDB problem. The code itself would have prevented us from doing that IF 51% of the hashpower had been used to build on the wrong chain, but it wasn't. Justus' question about what malicious means is key here. The blind spot is a bit more complex than just viewing the longest chain as impossible to break except with more than 51% of the hash power. The blind spot is our inability to distinguish between malicious blocks and honest blocks. Rune suggests that empty blocks indicate malice. I like that (which is why I advocate using BitcoinDaysDestroyed to decide between blocks at the same height that appear at nearly the same time, rather than first-seen). There are other methods we can use to distinguish between malicious blocks and honest ones. I'm inventing one right now, but I'm sure better ones can be found. Here's mine: Once a transaction has been confirmed, its originator generally takes on the responsibility of re-broadcasting it if it gets re-org'd out of its confirmation(s). Many mempools will see that re-broadcast, *if it happens*. Any malice in a 51% attack would come in the form of failing to include such transactions. If we have a history of orphaned blocks, then we can check to see which ones have been included in non-orphaned blocks since they got reorg'd out. Such transactions should be top-priority after a reorg, even if they have zero fees. When there is a transaction that doesn't appear in a new block within a couple hours of a reorg, that indicates dishonesty, usually in the sender (but that could be negligence), but possibly in the miner. Looking at the mempool would determine which, wouldn't it? notplato On Sat, Sep 19, 2015 at 1:11 PM, Rune K. Svendsen via bitcoin-dev < bitcoin-dev@lists•linuxfoundation.org> wrote: > An honest miner is a miner that supports the network by building on top of > the best valid chain. A malicious miner is one who wants to disrupt the > Bitcoin network, not support it, for example by executing a 51% attack > which mines empty blocks on top of the best chain. > > > /Rune > > > Den 19/09/2015 kl. 19.19 skrev Justus Ranvier < > justus@openbitcoinprivacyproject•org>: > > > >> On 19/09/15 10:45, Rune Kjær Svendsen wrote: > >> We need to distinguish between two different things here: > >> > >> 1) A 51% attack, where the majority of mining power is *malicious* > (hence “attack”) > > > > What does "malicious" mean? > > > > In other words, If miner A is mining honestly, and miner B is mining > > maliciously, what are some of the possible difference in their behaviour > > we would observe? > > > > > > -- > > Justus Ranvier > > Open Bitcoin Privacy Project > > http://www.openbitcoinprivacyproject.org/ > > justus@openbitcoinprivacyproject•org > > E7AD 8215 8497 3673 6D9E 61C4 2A5F DA70 EAD9 E623 > > <0xEAD9E623.asc> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > -- I like to provide some work at no charge to prove my value. Do you need a techie? I own Litmocracy <http://www.litmocracy.com> and Meme Racing <http://www.memeracing.net> (in alpha). I'm the webmaster for The Voluntaryist <http://www.voluntaryist.com> which now accepts Bitcoin. I also code for The Dollar Vigilante <http://dollarvigilante.com/>. "He ought to find it more profitable to play by the rules" - Satoshi Nakamoto [-- Attachment #2: Type: text/html, Size: 5648 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [bitcoin-dev] Hash of UTXO set as consensus-critical 2015-09-19 20:11 ` Rune K. Svendsen 2015-09-20 0:48 ` Dave Scotese @ 2015-09-21 17:15 ` Justus Ranvier 1 sibling, 0 replies; 14+ messages in thread From: Justus Ranvier @ 2015-09-21 17:15 UTC (permalink / raw) To: Rune K. Svendsen; +Cc: bitcoin-dev [-- Attachment #1.1: Type: text/plain, Size: 1697 bytes --] On 19/09/15 15:11, Rune K. Svendsen wrote: > An honest miner is a miner that supports the network by building on top of the best valid chain. A malicious miner is one who wants to disrupt the Bitcoin network, not support it, for example by executing a 51% attack which mines empty blocks on top of the best chain. This isn't a particularly good definition. "An honest miner is a miner that supports the network by building on top of the best valid chain." What is the "best valid chain"? The one with the most proof of work? The one that meets some other definition of "best"? "A malicious miner is one who wants to disrupt the Bitcoin network, not support it" This is a tautology, the equivalent of saying "a malicious miner is a miner that is malicious" A true, but entirely useless, statement. "for example by executing a 51% attack which mines empty blocks on top of the best chain." Again, you're begging the question with the word "attack", because that's what you're supposed to demonstrate. Apparently the difference between honest mining and malicious mining is empty blocks? You've said in both cases the miners are extending the "best valid chain". Is extending the best valid chain with an empty block always a malicious act? What's the significance of 51% in this definition? Is the same empty block which extended the best valid chain honest if the miner who produced it has 49% of the network hashing power and malicious if they add a few more ASIC units? -- Justus Ranvier Open Bitcoin Privacy Project http://www.openbitcoinprivacyproject.org/ justus@openbitcoinprivacyproject•org E7AD 8215 8497 3673 6D9E 61C4 2A5F DA70 EAD9 E623 [-- Attachment #1.2: 0xEAD9E623.asc --] [-- Type: application/pgp-keys, Size: 18729 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2015-09-21 17:15 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-09-18 19:05 [bitcoin-dev] Hash of UTXO set as consensus-critical Rune Kjær Svendsen 2015-09-18 19:43 ` Patrick Strateman 2015-09-18 20:07 ` Alex Morcos 2015-09-18 20:11 ` Matt Corallo 2015-09-18 20:17 ` Rune Kjær Svendsen 2015-09-18 20:37 ` Jorge Timón 2015-09-18 20:38 ` Jorge Timón 2015-09-18 22:22 ` Vincent Truong 2015-09-19 2:30 ` Justus Ranvier 2015-09-19 15:45 ` Rune Kjær Svendsen 2015-09-19 17:19 ` Justus Ranvier 2015-09-19 20:11 ` Rune K. Svendsen 2015-09-20 0:48 ` Dave Scotese 2015-09-21 17:15 ` Justus Ranvier
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox