Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

From: Billy Tetrud <billy.tetrud@gmail•com>
To: vjudeu@gazeta•pl
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools
Date: Mon, 20 Dec 2021 09:18:44 -0800	[thread overview]
Message-ID: <CAGpPWDbGTET27Hq8kKrQQoOavtOUEGzYkohVurSqZJ5r+o4gpQ@mail.gmail.com> (raw)
In-Reply-To: <150492262-e89b67dc2c010e65008ad976e2647ec1@pmq4v.m5r2.onet>

[-- Attachment #1: Type: text/plain, Size: 17616 bytes --]

> you can assign some minimal difficulty,

I was assuming that would be part of the plan.

> Signing sounds more like Proof of Stake

Associating signing with proof of stake and thereby concluding that signing
is something to avoid sounds like superstitious thinking. A signing scheme
with proof of work would clearly not be proof of stake. I would suggest not
dismissing a design out of hand like that. The benefit of that over merge
mining would be no extra on chain foot print. What do you think the
downsides might be?

> if you use just hashes, then they could be random.

You're right. Nodes would of course need to validate the Bitcoin block
headers being included, so i concede hashing them doesn't gain you anything.

On Thu, Dec 16, 2021, 22:37 <vjudeu@gazeta•pl> wrote:

> > I was thinking that this would be a separate blockchain with separate
> headers that progress linearly like a normal blockchain.
>
> Exactly, that's what I called "superblocks", where you have a separate
> chain, just to keep block headers instead of transactions.
>
> > A block creator would collect together as many blocks that haven't been
> collected yet into the next superblock (and maybe receive a reward
> proportional to how many / how much weight they include).
>
> You cannot "catch them all". If you do, you can end up with a lot of block
> headers, where each of them has difficulty equal to one. You need some
> limit, you can limit amount of blocks, you can assign some minimal
> difficulty, it does not matter that much, but some limit is needed, also
> because mining on top of the latest superblock should be more profitable
> than replacing someone else's reward in the previous superblock by your own
> reward and getting a bigger share in the previous superblock.
>
> > This could be done using merge mining, or it could be done using a
> signing scheme (eg where the block creator signs to say "I created this
> superblock" and have mechanisms to punish those who sign multiple
> superblocks at the same height.
>
> I would pick merge mining, because it is more compatible with existing
> mining scheme. Signing sounds more like Proof of Stake and I am trying to
> avoid that solution. Also, there is no need to sign anything, because you
> are solo mining where you have your own coinbase transaction or you are
> mining in a pool, where you have some shared address, and then you cannot
> produce any incompatible superblock, because the protocol can tell you,
> which address you should use (and if it is N-of-N taproot multisig and you
> have some closing transaction, then you can safely mine it).
>
> > Really, you could even just use hashes of the block headers.
>
> Replacing transactions with block headers will do the same trick. Each
> transaction is first hashed with double SHA-256, in exactly the same way as
> block headers are. If you replace transactions with block headers, you
> would get a superblock header, then varint saying how many block headers
> are there, and then you can place all block headers. During superblock
> merkle tree construction, you will hash all block headers (so you will get
> block hashes as leaves), and then you will combine block hashes in the same
> way as transaction hashes are combined.
>
> From the Script point of view, you can always use "OP_SIZE 80
> OP_EQUALVERIFY OP_HASH256 <hash> OP_EQUAL". Then, you can just change the
> size, just to show which object is hashed. Value 80 will work for block
> headers, small values below 520 will work for small transactions, value 64
> will work for any merkle tree proof, no matter if it is for superblock or
> normal block. Also, by using block headers instead of hashes, you can prove
> that at least a proper amount of work was done to produce it, because if
> you use just hashes, then they could be random.
>
> On 2021-12-16 17:57:23 user Billy Tetrud <billy.tetrud@gmail•com> wrote:
>
> @Jeremy
> >   for top-level pool participants there is never any central custody.
>
> I definitely see that. That was actually what I meant when I said the
> goals aren't the same as benefits. While your idea definitely satisfies all
> your goals in a modular way, the fact that it relies on pools means that
> unless the pools can also satisfy the goals, the total system also doesn't
> satisfy the goals (even tho the piece of that system you designed does).
>
> > Thus it doesn't "hurt" anyone except for the miners who are taking the
> not fully locked in funds risk
>
> True, it only potentially hurts whoever the channel partner is accepting
> the unspendable coins. And no one can really stop anyone from taking that
> risk if they really want to. But in that case, its not exactly a fully
> functional channel, since recourse mechanisms couldn't be performed.
> Wouldn't that open such a channel up to a pretty bad theft possibility?
>
> @Bob
> > Increased payout regularity does not lower the viable size of mining
> pools, because smaller mining pools using this mechanism still have higher
> variance.
>
> Yes, smaller mining pools will always have higher variance. However, lower
> variance has diminishing benefits. Below a certain amount of variance, less
> variance isn't very valuable. So increased payout regularity does indeed
> lower the viable size of mining pools because a given low-enough level of
> variance can be achieved with less pool hashpower.
>
> > The on-chain footprint is *higher* due to the increased payout
> regularity.
>
> That's a reasonable point. However, I think there is a difference here
> between the regularity of rewards vs payouts. Rewards for each miner can be
> more regular without necessarily increasing the number of on-chain payouts.
> In fact, theoretically, an individual miner could let their rewards
> accumulate in a pool over many rewards and only redeem when they need the
> coins for something. The incentive is there for each miner to be judicious
> on how much onchain space they take up.
>
> @vjudeu
>
> > how many block headers should be stored per one "superblock"?
>
> I was thinking that this would be a separate blockchain with separate
> headers that progress linearly like a normal blockchain. A block creator
> would collect together as many blocks that haven't been collected yet into
> the next superblock (and maybe receive a reward proportional to how many /
> how much weight they include). This could be done using merge mining, or it
> could be done using a signing scheme (eg where the block creator signs to
> say "I created this superblock" and have mechanisms to punish those who
> sign multiple superblocks at the same height. For merge mining, I could
> even imagine the data necessary to validate that it has been merge mined
> could be put into a taproot script branch (creating an invalid script, but
> a valid hash of the superblock).
>
> > we can collect all headers with the same previous block hash, and
> distribute block reward between all coinbase transactions in those headers
>
> Exactly.
>
> > we would just have block headers instead of transactions
>
> Yeah, I think that would be the way to go. Really, you could even just use
> hashes of the block headers. But the size doesn't matter much because it
> would be both a small blockchain and an ephemeral one (which can be fully
> discarded after all parties have been paid out, or at least their payout
> has been committed to on the bitcoin blockchain).
>
> On Thu, Dec 16, 2021 at 1:35 AM <vjudeu@gazeta•pl
> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchEuGRE%2BJkYAEBM5BgkBElIaQgpBQUFBVCVGX18dBRtTEQcBF1UyQUoDEQ0TRQYNQUdjI1hCU0cyajZIblhVZRQcVlEe>>
> wrote:
>
>> > The missing piece here would be an ordering of weak blocks to make the
>> window possible. Or at least a way to determine what blocks should
>> definitely be part of a particular block's pay out. I could see this being
>> done by a separate ephemeral blockchain (which starts fresh after each
>> Bitcoin block) that keeps track of which weak blocks have been submitted,
>> potentially using the pow already in each block to secure it. Granted that
>> piece is a bit half baked, but it seems quite solvable. Wdyt?
>>
>> I thought about something like that, but there is one problem: how many
>> block headers should be stored per one "superblock"? Currently, we have
>> single block header, where the whole coinbase transaction is taken by some
>> mining pool or solo miner. But instead, each miner could submit its own
>> block header. Then, we can collect all headers with the same previous block
>> hash, and distribute block reward between all coinbase transactions in
>> those headers. One "superblock" then would be created in a similar way as
>> existing blocks, we would just have block headers instead of transactions.
>> If most transactions inside those blocks will be the same, then each block
>> could be expressed just as a set of transaction hashes, only coinbase
>> transactions or custom, non-broadcasted transactions included by miners
>> will be revealed, everything else will be known.
>>
>> > One thing that jumped out at me as not safe is throwing block rewards
>> into a channel and being able to spend them immediately. There's a reason
>> block rewards aren't spendable for a while, and channels don't solve that
>> problem, do they? Why not simply reduce the on chain wait time for spending
>> block rewards at that point? Seems like the consequences would be the same.
>>
>> All coinbase rewards are unspendable for 100 blocks, it is enforced by
>> consensus. It does not matter if there are outputs owned directly by
>> miners, or if there is one huge N-of-N taproot multisig for the whole pool,
>> where every miner signed the closing transaction. The only option to take
>> coins faster I can see is swapping the coins by some LN transaction. But
>> then, the other party can check if some deposit to the LN channel is a part
>> of the coinbase transaction or not, and then decide if it is acceptable to
>> do the swap.
>>
>> On 2021-12-15 19:00:44 user Billy Tetrud via bitcoin-dev <
>> bitcoin-dev@lists•linuxfoundation.org
>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchE6GhA5LFpLCUc7EVZQVl9dQRIXXR8NCBMbCwIGChJXQFxcXEgcFh8UVVVDEyBdVkE9JVRdEwFhYXVlblhVIkosEAszLR5BQVV7U0MID0BAQUgIGh0RHgAMGAMXBQJfW1sdXRQUQUoDQlAiBFY8>>
>> wrote:
>>
>> Looks like an interesting proposal, but it doesn't seem to quite match
>> the goals you mentioned. As you do mention, this mining pool coordination
>> doesn't get rid of the need for mining pools in the first place. So it
>> doesn't satisfy item 1 on your goal list afaict.
>>
>> The primary benefits over what we have today that I can see are:
>> 1. increased payout regularity, which lowers the viable size of mining
>> pools, and
>> 2. Lower on chain footprint through combining pay outs from multiple
>> pools.
>>
>> Am I missing some?
>>
>> These are interesting benefits, but it would be nice if your post was
>> clearer on that, since the goals list is not the same as the list of
>> potential benefits of this kind of design.
>>
>> As far as enabling solo mining, what if this concept were used off chain?
>> Have a public network of solo miners who publish "weak blocks" to that
>> network, and the next 100 (or 1000 etc) nice miners pay you out as long as
>> you're also being nice by following the protocol? All the nice
>> optimizations you mentioned about eg combined taproot payouts would apply i
>> think. The only goals this wouldn't satisfy are 3 and 5 since an extra
>> network is needed, but to be fair, your proposal requires pools which all
>> need their own extra network anyways.
>>
>> The missing piece here would be an ordering of weak blocks to make the
>> window possible. Or at least a way to determine what blocks should
>> definitely be part of a particular block's pay out. I could see this being
>> done by a separate ephemeral blockchain (which starts fresh after each
>> Bitcoin block) that keeps track of which weak blocks have been submitted,
>> potentially using the pow already in each block to secure it. Granted that
>> piece is a bit half baked, but it seems quite solvable. Wdyt?
>>
>> One thing that jumped out at me as not safe is throwing block rewards
>> into a channel and being able to spend them immediately. There's a reason
>> block rewards aren't spendable for a while, and channels don't solve that
>> problem, do they? Why not simply reduce the on chain wait time for spending
>> block rewards at that point? Seems like the consequences would be the same.
>>
>> On Tue, Dec 14, 2021, 16:12 Bob McElrath via bitcoin-dev <
>> bitcoin-dev@lists•linuxfoundation.org
>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchE6GhA5LFpLCUc7EVZQVl9dQRIXXR8NCBMbCwIGChJXQFxcXEgcFh8UVVVDEyBdVkE9JVRdEwFhYXVlblhVIkosEAszLR5BQVV7U0MID0BAQUgIGh0RHgAMGAMXBQJfW1sdXRQUQUoDQlAiBFY8>>
>> wrote:
>>
>>> You are hand waving. Attempting to redefine terms to justify your
>>> argument is
>>> intellectually dishonest. Bitcoin pools have *always* been about variance
>>> reduction. Your window function fundamentally CANNOT be used to hedge
>>> hashrate.
>>> Various suggestions below introduce dangerous new games that might be
>>> played by
>>> miners.
>>>
>>> The fact is that the half-baked design you posted is less than useless,
>>> and
>>> doesn't do anything that anyone wants.
>>>
>>> You are trying to justify CTV by making it be all things to all people.
>>> "When
>>> all you have is a hammer, every problem looks like a nail".  Instead I
>>> humbly
>>> suggest that you pick ONE problem for which CTV is demonstrably the
>>> right and
>>> best solution, instead of snowing us with a ton of half-baked things that
>>> *could* be done, and often don't even require CTV, and some (like this
>>> one)
>>> fundamentally don't work. I do like some of your ideas, but if you had
>>> to pick
>>> just one "use case", which would it be?
>>>
>>> Jeremy [jlrubin@mit•edu
>>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchEyHxYvIVpLARduChoQSFZQR0NWQVZWJUNRXwMSCRMTBgcWASdWVkpbCxUTQwoWQUdjKVBMGFY3MWMWeU9QBAZtNw%3D%3D>]
>>> wrote:
>>> > Bitcoin didn't invent the concept of pooling:
>>> https://en.wikipedia.org/wiki/
>>> > Pooling_(resource_management). This is a Bitcoin Mining Pool, although
>>> it may
>>> > not be your favorite kind, which is fixated on specific properties of
>>> computing
>>> > contributions before finding a block. Pooling is just a general
>>> technique for
>>> > aggregating resources to accomplish something. If you have another
>>> name like
>>> > pooling that is in common use for this type of activity I would be
>>> more than
>>> > happy to adopt it.
>>> >
>>> > This sort of pool can hedge not only against fee rates but also against
>>> > increases in hashrate since your historical rate 'carries' into the
>>> future as a
>>> > function of the window. Further, windows and reward functions can be
>>> defined in
>>> > a myriad of ways that could, e.g., pay less to blocks found in more
>>> rapid
>>> > succession, contributing to the smoothing functionality.
>>> >
>>> > With respect to sub-block pooling, as described in the article, this
>>> sort of
>>> > design also helps with micro-pools being able to split resources
>>> > non-custodially in every block as a part of the higher order DCFMP.
>>> The point
>>> > is not, as noted, to enable solo mining an S9, but to decrease the
>>> size of the
>>> > minimum viable pool. It's also possible to add, without much
>>> validation or
>>> > data, some 'uncle block' type mechanism in an incentive compatible way
>>> (e.g.,
>>> > add 10 pow-heavy headers on the last block for cost 48 bytes header +
>>> 32 bytes
>>> > payout key) such that there's an incentive to include the heaviest
>>> ones you've
>>> > seen, not just your own, that are worth further study and consideration
>>> > (particularly because it's non-consensus, only for opt-in
>>> participation in the
>>> > pool).
>>> >
>>> > With respect to space usage, it seems you wholly reject the viability
>>> of a
>>> > payment pool mechanism to cut-through chain space. Is this a critique
>>> that
>>> > holds for all Payment Pools, or just in the context of mining? Is
>>> there a
>>> > particular reason why you think it infeasible that "strongly online"
>>> > counterparties would be able to coordinate more efficiently? Is it
>>> preferable
>>> > for miners, the nexus of decentralization for Bitcoin, to prefer to use
>>> > custodial services for pooling (which may require KYC/AM) over bearing
>>> a cost
>>> > of some extra potential chainload?
>>> >
>>> > Lastly, with respect to complexity, the proposal is actually
>>> incredibly simple
>>> > when you take it in a broader context. Non Interactive Channels and
>>> Payment
>>> > Pools are useful by themselves, so are the operations to merge them
>>> and swap
>>> > balance across them. Therefore most of the complexity in this proposal
>>> is
>>> > relying on tools we'll likely see in everyday use in any case, DCFMP
>>> or no.
>>> >
>>> > Jeremy
>>> > !DSPAM:61b8f2f5321461582627336!
>>> --
>>> Cheers, Bob McElrath
>>>
>>> "For every complex problem, there is a solution that is simple, neat,
>>> and wrong."
>>>     -- H. L. Mencken
>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists•linuxfoundation.org
>>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchE6GhA5LFpLCUc7EVZQVl9dQRIXXR8NCBMbCwIGChJXQFxcXEgcFh8UVVVDEyBdVkE9JVRdEwFhYXVlblhVIkosEAszLR5BQVV7U0MID0BAQUgIGh0RHgAMGAMXBQJfW1sdXRQUQUoDQlAiBFY8>
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>

[-- Attachment #2: Type: text/html, Size: 20638 bytes --]