* [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets @ 2015-08-29 16:43 Daniele Pinna 2015-08-29 17:30 ` Jorge Timón 2015-08-29 21:07 ` Matt Corallo 0 siblings, 2 replies; 9+ messages in thread From: Daniele Pinna @ 2015-08-29 16:43 UTC (permalink / raw) To: bitcoin-dev [-- Attachment #1: Type: text/plain, Size: 1596 bytes --] I'd like to submit this paper to the dev-list which analyzes how miner advantages scale with network and mempool properties in a scenario of uncapped block sizes. The work proceeds, in a sense, from where Peter R's work left off correcting a mistake and addressing the critiques made by the community to his work. The main result of the work is a detailed analysis of mining advantages (defined as the added profit per unit of hash) as a function of miner hashrate. In it, I show how large block subsidies (or better, low mempool fees-to-subsidy ratios) incentivize the pooling of large hashrates due to the steady increasing of marginal profits as hashrates grow. The paper also shows that part of the large advantage the large miners have today is due to there being a barrier to entry into a high-efficiency mining class which has access to expected profits an order of magnitude larger than everyone else. As block subsidies decrease, this high-efficiency class is expected to vanish leading to a marginal profit structure which decreases as a function of hashrate. This work has vacuumed my entire life for the past two weeks leading me to lag behind on a lot of work. I apologize for typos which I may not have seen. I stand by for any comments the community may have and look forward to reigniting consideration of a block size scaling proposal (BIP101) which, due to the XT fork drama, I believe has been placed hastily and undeservedly on the chopping block. https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets Regards, Daniele [-- Attachment #2: Type: text/html, Size: 1869 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets 2015-08-29 16:43 [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets Daniele Pinna @ 2015-08-29 17:30 ` Jorge Timón 2015-08-29 21:07 ` Matt Corallo 1 sibling, 0 replies; 9+ messages in thread From: Jorge Timón @ 2015-08-29 17:30 UTC (permalink / raw) To: Daniele Pinna; +Cc: Bitcoin Dev [-- Attachment #1: Type: text/plain, Size: 1300 bytes --] On Aug 29, 2015 9:43 AM, "Daniele Pinna via bitcoin-dev" < bitcoin-dev@lists•linuxfoundation.org> wrote: > > This work has vacuumed my entire life for the past two weeks leading me to lag behind on a lot of work. I apologize for typos which I may not have seen. I stand by for any comments the community may have and look forward to reigniting consideration of a block size scaling proposal (BIP101) which, due to the XT fork drama, I believe has been placed hastily and undeservedly on the chopping block. I don't like relying on exponential growth (that's why I don't like neither Gavin's 101 nor Pieter's 103). But I don't think it's too late to turn bip101 into just another proposal for an uncontroversial hardfork (changing the 75% to 95% would be the first step) and xt into just another software fork. My favorite one so far is bip102 (even though I still consider "2mb now" arbitrary and I'm worried about making mining centralization even worse than it is now), but if it was framed as a schism hardfork like bip101 I would also warn about the dangers of a schism hardfork for it. > https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets I'll read it to try to understand your claims. Are you presentung this in the scaling workshop? [-- Attachment #2: Type: text/html, Size: 1669 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets 2015-08-29 16:43 [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets Daniele Pinna 2015-08-29 17:30 ` Jorge Timón @ 2015-08-29 21:07 ` Matt Corallo 2015-08-29 23:17 ` Peter R 1 sibling, 1 reply; 9+ messages in thread From: Matt Corallo @ 2015-08-29 21:07 UTC (permalink / raw) To: Daniele Pinna, bitcoin-dev I believe it was pointed out previously in the discussion of the Peter R paper, but I'll repeat it here so that its visible - this seems to ignore the effects of transaction validation caches and block compression protocols. Many large miners already have their own network to relay blocks around the globe with only a few bytes on the wire at block-time, and there is also the bitcoinrelaynetwork.org network, which does the same for smaller miners, albeit with slightly less efficiency. Also, transaction validation time upon receiving a block can be rather easily made negligible (ie the only validation time you should have is the DB modify-utxo-set time). Thus, the increased orphan risk for including a transaction can be reduced to a very, very tiny amount, making the optimal blocksize, essentially, including everything that you're confident is in the mempool of other reasonably large miners. Matt On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote: > I'd like to submit this paper to the dev-list which analyzes how miner > advantages scale with network and mempool properties in a scenario of > uncapped block sizes. The work proceeds, in a sense, from where Peter > R's work left off correcting a mistake and addressing the critiques made > by the community to his work. > > The main result of the work is a detailed analysis of mining advantages > (defined as the added profit per unit of hash) as a function of miner > hashrate. In it, I show how large block subsidies (or better, low > mempool fees-to-subsidy ratios) incentivize the pooling of large > hashrates due to the steady increasing of marginal profits as hashrates > grow. > > The paper also shows that part of the large advantage the large miners > have today is due to there being a barrier to entry into a > high-efficiency mining class which has access to expected profits an > order of magnitude larger than everyone else. As block subsidies > decrease, this high-efficiency class is expected to vanish leading to a > marginal profit structure which decreases as a function of hashrate. > > This work has vacuumed my entire life for the past two weeks leading me > to lag behind on a lot of work. I apologize for typos which I may not > have seen. I stand by for any comments the community may have and look > forward to reigniting consideration of a block size scaling proposal > (BIP101) which, due to the XT fork drama, I believe has been placed > hastily and undeservedly on the chopping block. > > https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets > > > Regards, > Daniele > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets 2015-08-29 21:07 ` Matt Corallo @ 2015-08-29 23:17 ` Peter R 2015-08-30 2:33 ` Matt Corallo 0 siblings, 1 reply; 9+ messages in thread From: Peter R @ 2015-08-29 23:17 UTC (permalink / raw) To: Matt Corallo; +Cc: bitcoin-dev, Daniele Pinna [-- Attachment #1: Type: text/plain, Size: 4804 bytes --] Hello Matt and Daniele, > this seems to ignore the effects of transaction validation caches and block > compression protocols. The effect of block compression protocols is included. This is what I call the "coding gain" and use the Greek letter "gamma" to represent. As long as the block solution announcements contain information (i.e., Shannon Entropy) about the transactions included in a block, then the fee market will be "healthy" according to the definitions given in the linked paper (see below). This is the case right now, this is the case with your relay network, and this would be the case using any implementation of IBLTs that I can imagine, so long as miners can still construct blocks according to their own volition. The "healthy fee market" result follows from the Shannon-Hartley theorem; the SH-theorem describes the maximum rate at which information (Shannon Entropy) can be transmitted over a physical communication channel. https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf I've exchanged emails with Greg Maxwell about (what IMO is) an academic scenario where the block solutions announcements contain no information at all about the transactions included in the blocks. Although the fee market would not be healthy in such a scenario, it is my feeling that this also requires miners to relinquish their ability to construct blocks according to their own volition (i.e., the system would already be centralized). I look forward to a white paper demonstrating otherwise! Best regards, Peter On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote: > I believe it was pointed out previously in the discussion of the Peter R > paper, but I'll repeat it here so that its visible - this seems to > ignore the effects of transaction validation caches and block > compression protocols. Many large miners already have their own network > to relay blocks around the globe with only a few bytes on the wire at > block-time, and there is also the bitcoinrelaynetwork.org network, which > does the same for smaller miners, albeit with slightly less efficiency. > Also, transaction validation time upon receiving a block can be rather > easily made negligible (ie the only validation time you should have is > the DB modify-utxo-set time). Thus, the increased orphan risk for > including a transaction can be reduced to a very, very tiny amount, > making the optimal blocksize, essentially, including everything that > you're confident is in the mempool of other reasonably large miners. > > Matt > > On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote: >> I'd like to submit this paper to the dev-list which analyzes how miner >> advantages scale with network and mempool properties in a scenario of >> uncapped block sizes. The work proceeds, in a sense, from where Peter >> R's work left off correcting a mistake and addressing the critiques made >> by the community to his work. >> >> The main result of the work is a detailed analysis of mining advantages >> (defined as the added profit per unit of hash) as a function of miner >> hashrate. In it, I show how large block subsidies (or better, low >> mempool fees-to-subsidy ratios) incentivize the pooling of large >> hashrates due to the steady increasing of marginal profits as hashrates >> grow. >> >> The paper also shows that part of the large advantage the large miners >> have today is due to there being a barrier to entry into a >> high-efficiency mining class which has access to expected profits an >> order of magnitude larger than everyone else. As block subsidies >> decrease, this high-efficiency class is expected to vanish leading to a >> marginal profit structure which decreases as a function of hashrate. >> >> This work has vacuumed my entire life for the past two weeks leading me >> to lag behind on a lot of work. I apologize for typos which I may not >> have seen. I stand by for any comments the community may have and look >> forward to reigniting consideration of a block size scaling proposal >> (BIP101) which, due to the XT fork drama, I believe has been placed >> hastily and undeservedly on the chopping block. >> >> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets >> >> >> Regards, >> Daniele >> >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [-- Attachment #2: Type: text/html, Size: 5729 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets 2015-08-29 23:17 ` Peter R @ 2015-08-30 2:33 ` Matt Corallo 2015-08-30 2:35 ` Matt Corallo 2015-08-30 2:49 ` Peter R 0 siblings, 2 replies; 9+ messages in thread From: Matt Corallo @ 2015-08-30 2:33 UTC (permalink / raw) To: Peter R; +Cc: bitcoin-dev, Daniele Pinna It is not a purely academic scenario that blocks contain effectively no information (that was not previously relayed). I'm not aware of any public code to do so, but I know several large miners who pre-relay the block(s) they are working on to other nodes of theirs around the globe. This means at announce-time you have only a few bytes to broadcast (way less than a packet, and effects of using smaller packets to relay things vs larger packets are very small, if anything). After you've broadcast to all of your nodes, hops to other mining nodes are probably only a handful of ms away with very low packet loss, so relay time is no longer connected to transaction inclusion at all (unless you're talking about multi-GB blocks). Of course, this is relay time for large miners who can invest time and money to build such systems. Small miners are completely screwed in such a system. Thus, the orphan risk for including a transaction is related to the validation time (which is only DB modify-utxo-set time, essentially, which maybe you can optimize much of that away, too, and only have to pass over mempool or so). Anyway, my point, really, is that though miners will have an incentive to not include transactions which will trigger validation by other nodes (ie things not already in their mempool), the incentive to not include transactions which have already been relayed around sufficiently is, while not theoretically zero, as near to zero in practice as you can get. Matt On 08/29/15 23:17, Peter R wrote: > Hello Matt and Daniele, > >> this seems to ignore the effects of transaction validation caches and >> *block >> compression protocols. * > > The effect of block compression protocols is included. This is what I > call the "coding gain" and use the Greek letter "gamma" to represent. > > As long as the block solution announcements contain information (i.e., > Shannon Entropy) about the transactions included in a block, then the > fee market will be "healthy" according to the definitions given in the > linked paper (see below). This is the case right now, this is the case > with your relay network, and this would be the case using any > implementation of IBLTs that I can imagine, so long as miners can still > construct blocks according to their own volition. The "healthy fee > market" result follows from the Shannon-Hartley theorem; the SH-theorem > describes the maximum rate at which information (Shannon Entropy) can be > transmitted over a physical communication channel. > > https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf > > I've exchanged emails with Greg Maxwell about (what IMO is) an academic > scenario where the block solutions announcements contain *no information > at all* about the transactions included in the blocks. Although the fee > market would not be healthy in such a scenario, it is my feeling that > this also requires miners to relinquish their ability to construct > blocks according to their own volition (i.e., the system would already > be centralized). I look forward to a white paper demonstrating otherwise! > > Best regards, > Peter > > > > On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev > <bitcoin-dev@lists•linuxfoundation.org > <mailto:bitcoin-dev@lists•linuxfoundation.org>> wrote: > >> I believe it was pointed out previously in the discussion of the Peter R >> paper, but I'll repeat it here so that its visible - this seems to >> ignore the effects of transaction validation caches and block >> compression protocols. Many large miners already have their own network >> to relay blocks around the globe with only a few bytes on the wire at >> block-time, and there is also the bitcoinrelaynetwork.org >> <http://bitcoinrelaynetwork.org> network, which >> does the same for smaller miners, albeit with slightly less efficiency. >> Also, transaction validation time upon receiving a block can be rather >> easily made negligible (ie the only validation time you should have is >> the DB modify-utxo-set time). Thus, the increased orphan risk for >> including a transaction can be reduced to a very, very tiny amount, >> making the optimal blocksize, essentially, including everything that >> you're confident is in the mempool of other reasonably large miners. >> >> Matt >> >> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote: >>> I'd like to submit this paper to the dev-list which analyzes how miner >>> advantages scale with network and mempool properties in a scenario of >>> uncapped block sizes. The work proceeds, in a sense, from where Peter >>> R's work left off correcting a mistake and addressing the critiques made >>> by the community to his work. >>> >>> The main result of the work is a detailed analysis of mining advantages >>> (defined as the added profit per unit of hash) as a function of miner >>> hashrate. In it, I show how large block subsidies (or better, low >>> mempool fees-to-subsidy ratios) incentivize the pooling of large >>> hashrates due to the steady increasing of marginal profits as hashrates >>> grow. >>> >>> The paper also shows that part of the large advantage the large miners >>> have today is due to there being a barrier to entry into a >>> high-efficiency mining class which has access to expected profits an >>> order of magnitude larger than everyone else. As block subsidies >>> decrease, this high-efficiency class is expected to vanish leading to a >>> marginal profit structure which decreases as a function of hashrate. >>> >>> This work has vacuumed my entire life for the past two weeks leading me >>> to lag behind on a lot of work. I apologize for typos which I may not >>> have seen. I stand by for any comments the community may have and look >>> forward to reigniting consideration of a block size scaling proposal >>> (BIP101) which, due to the XT fork drama, I believe has been placed >>> hastily and undeservedly on the chopping block. >>> >>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets >>> >>> >>> Regards, >>> Daniele >>> >>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists•linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation.org >> <mailto:bitcoin-dev@lists•linuxfoundation.org> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets 2015-08-30 2:33 ` Matt Corallo @ 2015-08-30 2:35 ` Matt Corallo 2015-08-30 3:08 ` Peter R 2015-08-30 2:49 ` Peter R 1 sibling, 1 reply; 9+ messages in thread From: Matt Corallo @ 2015-08-30 2:35 UTC (permalink / raw) To: Peter R; +Cc: bitcoin-dev, Daniele Pinna Of course this assumes the network does not change any as a result of such a system. But such a system provides strong incentives for the network to centralize in other ways (put all the mining nodes in one DC for all miners, etc). Matt On 08/30/15 02:33, Matt Corallo via bitcoin-dev wrote: > It is not a purely academic scenario that blocks contain effectively no > information (that was not previously relayed). I'm not aware of any > public code to do so, but I know several large miners who pre-relay the > block(s) they are working on to other nodes of theirs around the globe. > This means at announce-time you have only a few bytes to broadcast (way > less than a packet, and effects of using smaller packets to relay things > vs larger packets are very small, if anything). After you've broadcast > to all of your nodes, hops to other mining nodes are probably only a > handful of ms away with very low packet loss, so relay time is no longer > connected to transaction inclusion at all (unless you're talking about > multi-GB blocks). Of course, this is relay time for large miners who can > invest time and money to build such systems. Small miners are completely > screwed in such a system. > > Thus, the orphan risk for including a transaction is related to the > validation time (which is only DB modify-utxo-set time, essentially, > which maybe you can optimize much of that away, too, and only have to > pass over mempool or so). Anyway, my point, really, is that though > miners will have an incentive to not include transactions which will > trigger validation by other nodes (ie things not already in their > mempool), the incentive to not include transactions which have already > been relayed around sufficiently is, while not theoretically zero, as > near to zero in practice as you can get. > > Matt > > On 08/29/15 23:17, Peter R wrote: >> Hello Matt and Daniele, >> >>> this seems to ignore the effects of transaction validation caches and >>> *block >>> compression protocols. * >> >> The effect of block compression protocols is included. This is what I >> call the "coding gain" and use the Greek letter "gamma" to represent. >> >> As long as the block solution announcements contain information (i.e., >> Shannon Entropy) about the transactions included in a block, then the >> fee market will be "healthy" according to the definitions given in the >> linked paper (see below). This is the case right now, this is the case >> with your relay network, and this would be the case using any >> implementation of IBLTs that I can imagine, so long as miners can still >> construct blocks according to their own volition. The "healthy fee >> market" result follows from the Shannon-Hartley theorem; the SH-theorem >> describes the maximum rate at which information (Shannon Entropy) can be >> transmitted over a physical communication channel. >> >> https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf >> >> I've exchanged emails with Greg Maxwell about (what IMO is) an academic >> scenario where the block solutions announcements contain *no information >> at all* about the transactions included in the blocks. Although the fee >> market would not be healthy in such a scenario, it is my feeling that >> this also requires miners to relinquish their ability to construct >> blocks according to their own volition (i.e., the system would already >> be centralized). I look forward to a white paper demonstrating otherwise! >> >> Best regards, >> Peter >> >> >> >> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev >> <bitcoin-dev@lists•linuxfoundation.org >> <mailto:bitcoin-dev@lists•linuxfoundation.org>> wrote: >> >>> I believe it was pointed out previously in the discussion of the Peter R >>> paper, but I'll repeat it here so that its visible - this seems to >>> ignore the effects of transaction validation caches and block >>> compression protocols. Many large miners already have their own network >>> to relay blocks around the globe with only a few bytes on the wire at >>> block-time, and there is also the bitcoinrelaynetwork.org >>> <http://bitcoinrelaynetwork.org> network, which >>> does the same for smaller miners, albeit with slightly less efficiency. >>> Also, transaction validation time upon receiving a block can be rather >>> easily made negligible (ie the only validation time you should have is >>> the DB modify-utxo-set time). Thus, the increased orphan risk for >>> including a transaction can be reduced to a very, very tiny amount, >>> making the optimal blocksize, essentially, including everything that >>> you're confident is in the mempool of other reasonably large miners. >>> >>> Matt >>> >>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote: >>>> I'd like to submit this paper to the dev-list which analyzes how miner >>>> advantages scale with network and mempool properties in a scenario of >>>> uncapped block sizes. The work proceeds, in a sense, from where Peter >>>> R's work left off correcting a mistake and addressing the critiques made >>>> by the community to his work. >>>> >>>> The main result of the work is a detailed analysis of mining advantages >>>> (defined as the added profit per unit of hash) as a function of miner >>>> hashrate. In it, I show how large block subsidies (or better, low >>>> mempool fees-to-subsidy ratios) incentivize the pooling of large >>>> hashrates due to the steady increasing of marginal profits as hashrates >>>> grow. >>>> >>>> The paper also shows that part of the large advantage the large miners >>>> have today is due to there being a barrier to entry into a >>>> high-efficiency mining class which has access to expected profits an >>>> order of magnitude larger than everyone else. As block subsidies >>>> decrease, this high-efficiency class is expected to vanish leading to a >>>> marginal profit structure which decreases as a function of hashrate. >>>> >>>> This work has vacuumed my entire life for the past two weeks leading me >>>> to lag behind on a lot of work. I apologize for typos which I may not >>>> have seen. I stand by for any comments the community may have and look >>>> forward to reigniting consideration of a block size scaling proposal >>>> (BIP101) which, due to the XT fork drama, I believe has been placed >>>> hastily and undeservedly on the chopping block. >>>> >>>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets >>>> >>>> >>>> Regards, >>>> Daniele >>>> >>>> >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists•linuxfoundation.org >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists•linuxfoundation.org >>> <mailto:bitcoin-dev@lists•linuxfoundation.org> >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets 2015-08-30 2:35 ` Matt Corallo @ 2015-08-30 3:08 ` Peter R 0 siblings, 0 replies; 9+ messages in thread From: Peter R @ 2015-08-30 3:08 UTC (permalink / raw) To: Matt Corallo; +Cc: bitcoin-dev, Daniele Pinna [-- Attachment #1: Type: text/plain, Size: 8086 bytes --] > Of course this assumes the network does not change any as a result of > such a system. But such a system provides strong incentives for the > network to centralize in other ways (put all the mining nodes in one DC > for all miners, etc). If all the mining nodes are in one data center, and if all the nodes are programmed to build blocks in essentially the same way, then I would agree that the orphan cost would be negligible! I will add this as an example of a network configuration where the results of my paper would be less relevant. Peter On 2015-08-29, at 7:35 PM, Matt Corallo <lf-lists@mattcorallo•com> wrote: > Of course this assumes the network does not change any as a result of > such a system. But such a system provides strong incentives for the > network to centralize in other ways (put all the mining nodes in one DC > for all miners, etc). > > Matt > > On 08/30/15 02:33, Matt Corallo via bitcoin-dev wrote: >> It is not a purely academic scenario that blocks contain effectively no >> information (that was not previously relayed). I'm not aware of any >> public code to do so, but I know several large miners who pre-relay the >> block(s) they are working on to other nodes of theirs around the globe. >> This means at announce-time you have only a few bytes to broadcast (way >> less than a packet, and effects of using smaller packets to relay things >> vs larger packets are very small, if anything). After you've broadcast >> to all of your nodes, hops to other mining nodes are probably only a >> handful of ms away with very low packet loss, so relay time is no longer >> connected to transaction inclusion at all (unless you're talking about >> multi-GB blocks). Of course, this is relay time for large miners who can >> invest time and money to build such systems. Small miners are completely >> screwed in such a system. >> >> Thus, the orphan risk for including a transaction is related to the >> validation time (which is only DB modify-utxo-set time, essentially, >> which maybe you can optimize much of that away, too, and only have to >> pass over mempool or so). Anyway, my point, really, is that though >> miners will have an incentive to not include transactions which will >> trigger validation by other nodes (ie things not already in their >> mempool), the incentive to not include transactions which have already >> been relayed around sufficiently is, while not theoretically zero, as >> near to zero in practice as you can get. >> >> Matt >> >> On 08/29/15 23:17, Peter R wrote: >>> Hello Matt and Daniele, >>> >>>> this seems to ignore the effects of transaction validation caches and >>>> *block >>>> compression protocols. * >>> >>> The effect of block compression protocols is included. This is what I >>> call the "coding gain" and use the Greek letter "gamma" to represent. >>> >>> As long as the block solution announcements contain information (i.e., >>> Shannon Entropy) about the transactions included in a block, then the >>> fee market will be "healthy" according to the definitions given in the >>> linked paper (see below). This is the case right now, this is the case >>> with your relay network, and this would be the case using any >>> implementation of IBLTs that I can imagine, so long as miners can still >>> construct blocks according to their own volition. The "healthy fee >>> market" result follows from the Shannon-Hartley theorem; the SH-theorem >>> describes the maximum rate at which information (Shannon Entropy) can be >>> transmitted over a physical communication channel. >>> >>> https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf >>> >>> I've exchanged emails with Greg Maxwell about (what IMO is) an academic >>> scenario where the block solutions announcements contain *no information >>> at all* about the transactions included in the blocks. Although the fee >>> market would not be healthy in such a scenario, it is my feeling that >>> this also requires miners to relinquish their ability to construct >>> blocks according to their own volition (i.e., the system would already >>> be centralized). I look forward to a white paper demonstrating otherwise! >>> >>> Best regards, >>> Peter >>> >>> >>> >>> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev >>> <bitcoin-dev@lists•linuxfoundation.org >>> <mailto:bitcoin-dev@lists•linuxfoundation.org>> wrote: >>> >>>> I believe it was pointed out previously in the discussion of the Peter R >>>> paper, but I'll repeat it here so that its visible - this seems to >>>> ignore the effects of transaction validation caches and block >>>> compression protocols. Many large miners already have their own network >>>> to relay blocks around the globe with only a few bytes on the wire at >>>> block-time, and there is also the bitcoinrelaynetwork.org >>>> <http://bitcoinrelaynetwork.org> network, which >>>> does the same for smaller miners, albeit with slightly less efficiency. >>>> Also, transaction validation time upon receiving a block can be rather >>>> easily made negligible (ie the only validation time you should have is >>>> the DB modify-utxo-set time). Thus, the increased orphan risk for >>>> including a transaction can be reduced to a very, very tiny amount, >>>> making the optimal blocksize, essentially, including everything that >>>> you're confident is in the mempool of other reasonably large miners. >>>> >>>> Matt >>>> >>>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote: >>>>> I'd like to submit this paper to the dev-list which analyzes how miner >>>>> advantages scale with network and mempool properties in a scenario of >>>>> uncapped block sizes. The work proceeds, in a sense, from where Peter >>>>> R's work left off correcting a mistake and addressing the critiques made >>>>> by the community to his work. >>>>> >>>>> The main result of the work is a detailed analysis of mining advantages >>>>> (defined as the added profit per unit of hash) as a function of miner >>>>> hashrate. In it, I show how large block subsidies (or better, low >>>>> mempool fees-to-subsidy ratios) incentivize the pooling of large >>>>> hashrates due to the steady increasing of marginal profits as hashrates >>>>> grow. >>>>> >>>>> The paper also shows that part of the large advantage the large miners >>>>> have today is due to there being a barrier to entry into a >>>>> high-efficiency mining class which has access to expected profits an >>>>> order of magnitude larger than everyone else. As block subsidies >>>>> decrease, this high-efficiency class is expected to vanish leading to a >>>>> marginal profit structure which decreases as a function of hashrate. >>>>> >>>>> This work has vacuumed my entire life for the past two weeks leading me >>>>> to lag behind on a lot of work. I apologize for typos which I may not >>>>> have seen. I stand by for any comments the community may have and look >>>>> forward to reigniting consideration of a block size scaling proposal >>>>> (BIP101) which, due to the XT fork drama, I believe has been placed >>>>> hastily and undeservedly on the chopping block. >>>>> >>>>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets >>>>> >>>>> >>>>> Regards, >>>>> Daniele >>>>> >>>>> >>>>> _______________________________________________ >>>>> bitcoin-dev mailing list >>>>> bitcoin-dev@lists•linuxfoundation.org >>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>>> >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists•linuxfoundation.org >>>> <mailto:bitcoin-dev@lists•linuxfoundation.org> >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> [-- Attachment #2: Type: text/html, Size: 8981 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets 2015-08-30 2:33 ` Matt Corallo 2015-08-30 2:35 ` Matt Corallo @ 2015-08-30 2:49 ` Peter R 2015-08-30 3:56 ` Matt Corallo 1 sibling, 1 reply; 9+ messages in thread From: Peter R @ 2015-08-30 2:49 UTC (permalink / raw) To: Matt Corallo; +Cc: bitcoin-dev, Daniele Pinna [-- Attachment #1: Type: text/plain, Size: 6874 bytes --] Hello Matt, > It is not a purely academic scenario that blocks contain effectively no > information (that was not previously relayed). The results of my paper hold unless the information about the included transactions is identically zero. If this information is extremely small, then I agree that something other than the orphan cost would drive the fee market. I don't think this will ever by the case, however. > I'm not aware of any > public code to do so, but I know several large miners who pre-relay the > block(s) they are working on to other nodes of theirs around the globe…. In this paragraph, you are talking about intra-miner communication (you refer to other nodes "of theirs"). I agree that this can be very fast because there's only one entity involved. > Thus, the orphan risk for including a transaction is related to the > validation time … the incentive to not include transactions which have already > been relayed around sufficiently is, while not theoretically zero, as > near to zero in practice as you can get. I look forward to hearing your talk about the Relay Network. Let's image there was no block size limit. Using the Relay Network as it currently operates now, how big would the block solution announcement be: (1) if a miner published a 10 MB block where 100% of the transactions are known by the other nodes? (2) if a miner published a 10 MB block where 90% of the transactions are known by the other nodes? (3) if a miner published a 100 MB spam block where 0% of the transactions are known by the other nodes? Best regards, Peter > > Matt > > On 08/29/15 23:17, Peter R wrote: >> Hello Matt and Daniele, >> >>> this seems to ignore the effects of transaction validation caches and >>> *block >>> compression protocols. * >> >> The effect of block compression protocols is included. This is what I >> call the "coding gain" and use the Greek letter "gamma" to represent. >> >> As long as the block solution announcements contain information (i.e., >> Shannon Entropy) about the transactions included in a block, then the >> fee market will be "healthy" according to the definitions given in the >> linked paper (see below). This is the case right now, this is the case >> with your relay network, and this would be the case using any >> implementation of IBLTs that I can imagine, so long as miners can still >> construct blocks according to their own volition. The "healthy fee >> market" result follows from the Shannon-Hartley theorem; the SH-theorem >> describes the maximum rate at which information (Shannon Entropy) can be >> transmitted over a physical communication channel. >> >> https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf >> >> I've exchanged emails with Greg Maxwell about (what IMO is) an academic >> scenario where the block solutions announcements contain *no information >> at all* about the transactions included in the blocks. Although the fee >> market would not be healthy in such a scenario, it is my feeling that >> this also requires miners to relinquish their ability to construct >> blocks according to their own volition (i.e., the system would already >> be centralized). I look forward to a white paper demonstrating otherwise! >> >> Best regards, >> Peter >> >> >> >> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev >> <bitcoin-dev@lists•linuxfoundation.org >> <mailto:bitcoin-dev@lists•linuxfoundation.org>> wrote: >> >>> I believe it was pointed out previously in the discussion of the Peter R >>> paper, but I'll repeat it here so that its visible - this seems to >>> ignore the effects of transaction validation caches and block >>> compression protocols. Many large miners already have their own network >>> to relay blocks around the globe with only a few bytes on the wire at >>> block-time, and there is also the bitcoinrelaynetwork.org >>> <http://bitcoinrelaynetwork.org> network, which >>> does the same for smaller miners, albeit with slightly less efficiency. >>> Also, transaction validation time upon receiving a block can be rather >>> easily made negligible (ie the only validation time you should have is >>> the DB modify-utxo-set time). Thus, the increased orphan risk for >>> including a transaction can be reduced to a very, very tiny amount, >>> making the optimal blocksize, essentially, including everything that >>> you're confident is in the mempool of other reasonably large miners. >>> >>> Matt >>> >>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote: >>>> I'd like to submit this paper to the dev-list which analyzes how miner >>>> advantages scale with network and mempool properties in a scenario of >>>> uncapped block sizes. The work proceeds, in a sense, from where Peter >>>> R's work left off correcting a mistake and addressing the critiques made >>>> by the community to his work. >>>> >>>> The main result of the work is a detailed analysis of mining advantages >>>> (defined as the added profit per unit of hash) as a function of miner >>>> hashrate. In it, I show how large block subsidies (or better, low >>>> mempool fees-to-subsidy ratios) incentivize the pooling of large >>>> hashrates due to the steady increasing of marginal profits as hashrates >>>> grow. >>>> >>>> The paper also shows that part of the large advantage the large miners >>>> have today is due to there being a barrier to entry into a >>>> high-efficiency mining class which has access to expected profits an >>>> order of magnitude larger than everyone else. As block subsidies >>>> decrease, this high-efficiency class is expected to vanish leading to a >>>> marginal profit structure which decreases as a function of hashrate. >>>> >>>> This work has vacuumed my entire life for the past two weeks leading me >>>> to lag behind on a lot of work. I apologize for typos which I may not >>>> have seen. I stand by for any comments the community may have and look >>>> forward to reigniting consideration of a block size scaling proposal >>>> (BIP101) which, due to the XT fork drama, I believe has been placed >>>> hastily and undeservedly on the chopping block. >>>> >>>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets >>>> >>>> >>>> Regards, >>>> Daniele >>>> >>>> >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists•linuxfoundation.org >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists•linuxfoundation.org >>> <mailto:bitcoin-dev@lists•linuxfoundation.org> >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> [-- Attachment #2: Type: text/html, Size: 8011 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets 2015-08-30 2:49 ` Peter R @ 2015-08-30 3:56 ` Matt Corallo 0 siblings, 0 replies; 9+ messages in thread From: Matt Corallo @ 2015-08-30 3:56 UTC (permalink / raw) To: Peter R; +Cc: bitcoin-dev, Daniele Pinna On 08/30/15 02:49, Peter R wrote: > Hello Matt, > >> It is not a purely academic scenario that blocks contain *effectively* no >> information (that was not previously relayed). > > The results of my paper hold unless the information about the included > transactions is *identically zero*. If this information is extremely > small, then I agree that something other than the orphan cost would > drive the fee market. I don't think this will ever by the case, however. You missed one key point - information transmitted across networks is not slowed down on a per-bit basis, its (largely) slowed down on a per-packet basis! You have some 1400+ bytes to work with before you notice the increase in information in relay cost. Of course this only applies if you are talking about long-haul links (across oceans, between countries, etc), if you're talking about short links it is a per-byte cost, but your time is probably negligible for anything but VERY large blocks. >> I'm not aware of any >> public code to do so, but I know several large miners who pre-relay the >> block(s) they are working on to other nodes *of theirs* around the globe…. > > In this paragraph, you are talking about intra-miner communication (you > refer to other nodes "of theirs"). I agree that this can be very fast > because there's only one entity involved. Very fast is an understatement. My point was that it is literally always a single packet, and a very small packet, at that. >> Thus, the orphan risk for including a transaction is related to the >> validation time … the incentive to not include transactions which have >> already >> been relayed around sufficiently is, while not theoretically zero, *as >> near to zero in practice as you can get*. > > I look forward to hearing your talk about the Relay Network. Let's > image there was no block size limit. Using the Relay Network*as it > currently operates now*, how big would the block solution announcement be: I think you missed it, this mail was not about the relay network. Let's ignore the relay network entirely for now. Several miners have nodes well distributed all over the globe which peer either directly with other miners in the area, or just connect to a ton of nodes, probably hitting other miners. The cost to transmit from their pool server to their relay server is one packet, always. They pre-relay everything they're gonna include, so, at max, they might have to pick between a few pre-relayed blocks (what, like, 2/5 bits?). From there, you assume they are transmitting to a node in the same DC, or at least very close (say, local within AWS-US-East/maybe between AWS-US-East and somewhere in NYC), where transmitting even 100MB is really fast. > (1) if a miner published a 10 MB block where 100% of the transactions > are known by the other nodes? > > (2) if a miner published a 10 MB block where 90% of the transactions are > known by the other nodes? > > (3) if a miner published a 100 MB spam block where 0% of the > transactions are known by the other nodes? > > Best regards, > Peter > > >> >> Matt >> >> On 08/29/15 23:17, Peter R wrote: >>> Hello Matt and Daniele, >>> >>>> this seems to ignore the effects of transaction validation caches and >>>> *block >>>> compression protocols. * >>> >>> The effect of block compression protocols is included. This is what I >>> call the "coding gain" and use the Greek letter "gamma" to represent. >>> >>> As long as the block solution announcements contain information (i.e., >>> Shannon Entropy) about the transactions included in a block, then the >>> fee market will be "healthy" according to the definitions given in the >>> linked paper (see below). This is the case right now, this is the case >>> with your relay network, and this would be the case using any >>> implementation of IBLTs that I can imagine, so long as miners can still >>> construct blocks according to their own volition. The "healthy fee >>> market" result follows from the Shannon-Hartley theorem; the SH-theorem >>> describes the maximum rate at which information (Shannon Entropy) can be >>> transmitted over a physical communication channel. >>> >>> https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf >>> >>> I've exchanged emails with Greg Maxwell about (what IMO is) an academic >>> scenario where the block solutions announcements contain *no information >>> at all* about the transactions included in the blocks. Although the fee >>> market would not be healthy in such a scenario, it is my feeling that >>> this also requires miners to relinquish their ability to construct >>> blocks according to their own volition (i.e., the system would already >>> be centralized). I look forward to a white paper demonstrating >>> otherwise! >>> >>> Best regards, >>> Peter >>> >>> >>> >>> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev >>> <bitcoin-dev@lists•linuxfoundation.org >>> <mailto:bitcoin-dev@lists•linuxfoundation.org> >>> <mailto:bitcoin-dev@lists•linuxfoundation.org>> wrote: >>> >>>> I believe it was pointed out previously in the discussion of the Peter R >>>> paper, but I'll repeat it here so that its visible - this seems to >>>> ignore the effects of transaction validation caches and block >>>> compression protocols. Many large miners already have their own network >>>> to relay blocks around the globe with only a few bytes on the wire at >>>> block-time, and there is also the bitcoinrelaynetwork.org >>>> <http://bitcoinrelaynetwork.org> >>>> <http://bitcoinrelaynetwork.org> network, which >>>> does the same for smaller miners, albeit with slightly less efficiency. >>>> Also, transaction validation time upon receiving a block can be rather >>>> easily made negligible (ie the only validation time you should have is >>>> the DB modify-utxo-set time). Thus, the increased orphan risk for >>>> including a transaction can be reduced to a very, very tiny amount, >>>> making the optimal blocksize, essentially, including everything that >>>> you're confident is in the mempool of other reasonably large miners. >>>> >>>> Matt >>>> >>>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote: >>>>> I'd like to submit this paper to the dev-list which analyzes how miner >>>>> advantages scale with network and mempool properties in a scenario of >>>>> uncapped block sizes. The work proceeds, in a sense, from where Peter >>>>> R's work left off correcting a mistake and addressing the critiques >>>>> made >>>>> by the community to his work. >>>>> >>>>> The main result of the work is a detailed analysis of mining advantages >>>>> (defined as the added profit per unit of hash) as a function of miner >>>>> hashrate. In it, I show how large block subsidies (or better, low >>>>> mempool fees-to-subsidy ratios) incentivize the pooling of large >>>>> hashrates due to the steady increasing of marginal profits as hashrates >>>>> grow. >>>>> >>>>> The paper also shows that part of the large advantage the large miners >>>>> have today is due to there being a barrier to entry into a >>>>> high-efficiency mining class which has access to expected profits an >>>>> order of magnitude larger than everyone else. As block subsidies >>>>> decrease, this high-efficiency class is expected to vanish leading to a >>>>> marginal profit structure which decreases as a function of hashrate. >>>>> >>>>> This work has vacuumed my entire life for the past two weeks leading me >>>>> to lag behind on a lot of work. I apologize for typos which I may not >>>>> have seen. I stand by for any comments the community may have and look >>>>> forward to reigniting consideration of a block size scaling proposal >>>>> (BIP101) which, due to the XT fork drama, I believe has been placed >>>>> hastily and undeservedly on the chopping block. >>>>> >>>>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets >>>>> >>>>> >>>>> Regards, >>>>> Daniele >>>>> >>>>> >>>>> _______________________________________________ >>>>> bitcoin-dev mailing list >>>>> bitcoin-dev@lists•linuxfoundation.org >>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>>> >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists•linuxfoundation.org >>>> <mailto:bitcoin-dev@lists•linuxfoundation.org> >>>> <mailto:bitcoin-dev@lists•linuxfoundation.org> >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> > ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2015-08-30 3:56 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-08-29 16:43 [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets Daniele Pinna 2015-08-29 17:30 ` Jorge Timón 2015-08-29 21:07 ` Matt Corallo 2015-08-29 23:17 ` Peter R 2015-08-30 2:33 ` Matt Corallo 2015-08-30 2:35 ` Matt Corallo 2015-08-30 3:08 ` Peter R 2015-08-30 2:49 ` Peter R 2015-08-30 3:56 ` Matt Corallo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox