[bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 1046 bytes --]

Howdy, welcome to day 15!

Today's post covers a form of a mining pool that can be operated as sort of
a map-reduce over blocks without any "infrastructure".

https://rubin.io/bitcoin/2021/12/12/advent-15/

There's still some really open-ended questions (perhaps for y'all to
consider) around how to select an analyze the choice of window and payout
functions, but something like this could alleviate a lot of the
centralization pressures typically faced by pools.

Notably, compared to previous attempts, combining the payment pool payout
with this concept means that there is practically very little on-chain
overhead from this approach as the chain-load
for including payouts in every block is deferred for future cooperation
among miners. Although that can be considered cooperation itself, if you
think of it like a pipeline, the cooperation happens out of band from
mining and block production so it really is coordination free to mine.


Cheers,

Jeremy

--
@JeremyRubin <https://twitter.com/JeremyRubin>
<https://twitter.com/JeremyRubin>

[-- Attachment #2: Type: text/html, Size: 2726 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[vjudeu]

[-- Attachment #1: Type: text/plain, Size: 4469 bytes --]

> how to select an analyze the choice of window
Currently, we need 100 blocks to spend the coinbase transaction and I think that should be our "window".
> and payout functions
Something like "miner-based difficulty" should do the trick. So, each miner is trying to produce its own block, with its own transactions, and its own coinbase reward (based on those transactions, if we want to think ahead and do it right from the start, we should be ready for situation where the basic block reward is zero and the whole coinbase is based only on transaction fees). So, each miner can mine a block with its own coinbase amount (based on transaction fees). Then, that miner should multiply the target by the number of satoshis collected in the coinbase transaction to get "target per satoshi". Then, by dividing this target by its block hash, it would produce the number of satoshis that miner should receive.
Some example:
difficulty: 170ba21f
target: 0000000000000000000ba21f0000000000000000000000000000000000000000
coinbase: 6.27930034 BTC (627930034 satoshis = 0x256d73b2 satoshis)
targetPerSatoshi: 0000000000000000000ba21f0000000000000000000000000000000000000000*0x256d73b2
targetPerSatoshi: 000000000001b367c41da68e0000000000000000000000000000000000000000
sampleShare: 0000000000000000b613738816247a7f4d357cae555996519cf5b543e9b3554b
minerReward: targetPerSatoshi/sampleShare=0x2642e (156718 satoshis = 0.00156718 BTC for this share)
Because we assume that the basic reward will be zero, we assume that all miners will include their own set of transactions. That means, to check if the miner really should receive that reward, checking all transactions is required. Assuming that most of the miners will have similar transactions in their mempools, for each share there is a need to only check transactions that were unknown by that miner. For all other previously validated transactions, miners can store a table like: "<txid> <fee>" and then quickly validate if the amount specified in the coinbase transaction is correct.
To avoid "share spam", we can use something like "miner-based difficulty" mentioned above. Everyone knows the network difficulty, but not all miners are directly connected. So, for each connection with each miner in our decentralized pool, we can define a difficulty for each connection. In this way, each node can specify the absolute minimum difficulty, where paying any reward is above the dust limit, and where including that miner makes sense. Then, each miner can produce shares and adjust miner-based difficulty, just to produce for example one share per 10 minutes (or per 30 seconds if we have enough resources to fully validate each share from each miner we are connected with in that time).
If we want to include really small miners (like CPU miners), then we need a way to allow sub-satoshi payments. That means, each small miner should mine to a single N-of-N taproot-based multisig, where the whole pot is then splitted between N miners in LN. That means, for example one output of 1000 satoshis can be shared between one million small CPU miners. Then, our target from example above is denominated in millisatoshis.
targetPerSatoshi: 000000000001b367c41da68e0000000000000000000000000000000000000000*0x3e8 (1000 in decimal)
targetPerMillisatoshi: 0000000006a4cd5613d29ab00000000000000000000000000000000000000000
On 2021-12-12 17:43:39 user Jeremy via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
Howdy, welcome to day 15!
 
Today's post covers a form of a mining pool that can be operated as sort of a map-reduce over blocks without any "infrastructure".
 
https://rubin.io/bitcoin/2021/12/12/advent-15/
 
There's still some really open-ended questions (perhaps for y'all to consider) around how to select an analyze the choice of window and payout functions, but something like this could alleviate a lot of the centralization pressures typically faced by pools.
 
Notably, compared to previous attempts, combining the payment pool payout with this concept means that there is practically very little on-chain overhead from this approach as the chain-load
for including payouts in every block is deferred for future cooperation among miners. Although that can be considered cooperation itself, if you think of it like a pipeline, the cooperation happens out of band from mining and block production so it really is coordination free to mine.
 
Cheers,
 
Jeremy
--
@JeremyRubin

[-- Attachment #2: Type: text/html, Size: 6496 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 1102 bytes --]

Hey there!

Thanks for your response!

One of the reasons to pick a longer window of, say, a couple difficulty
periods would be that you can make participation in the pool hedge you
against hashrate changes.

You're absolutely spot on to think about the impact of pooling w.r.t.
variance when fees > subsidy. That's not really in the analysis I had in
the (old) post, but when the block revenues swing, dcfmp over longer
periods can really smooth out the revenues for miners in a great way. This can
also help with the "mind the gap" problem when there isn't a backlog of
transactions, since producing an empty block still has some value (in order
to incentivize mining transaction at all and not cheating, we need to
reward txn inclusion as I think you're trying to point out.

Sadly, I've read the rest of your email a couple times and I don't really
get what you're proposing at all. It jumps right into "things you could
compute". Can you maybe try stating the goals of your payout function, and
then demonstrate how what you're proposing meets that? E.g., we want to pay
more to miners that do x?

[-- Attachment #2: Type: text/html, Size: 3817 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 2566 bytes --]

Bitcoin didn't invent the concept of pooling:
https://en.wikipedia.org/wiki/Pooling_(resource_management). This is a
Bitcoin Mining Pool, although it may not be your favorite kind, which is
fixated on specific properties of computing contributions before finding a
block. Pooling is just a general technique for aggregating resources to
accomplish something. If you have another name like pooling that is in
common use for this type of activity I would be more than happy to adopt it.

This sort of pool can hedge not only against fee rates but also against
increases in hashrate since your historical rate 'carries' into the future
as a function of the window. Further, windows and reward functions can be
defined in a myriad of ways that could, e.g., pay less to blocks found in
more rapid succession, contributing to the smoothing functionality.

With respect to sub-block pooling, as described in the article, this sort
of design also helps with micro-pools being able to split resources
non-custodially in every block as a part of the higher order DCFMP. The
point is not, as noted, to enable solo mining an S9, but to decrease the
size of the minimum viable pool. It's also possible to add, without much
validation or data, some 'uncle block' type mechanism in an incentive
compatible way (e.g., add 10 pow-heavy headers on the last block for cost
48 bytes header + 32 bytes payout key) such that there's an incentive to
include the heaviest ones you've seen, not just your own, that are worth
further study and consideration (particularly because it's non-consensus,
only for opt-in participation in the pool).

With respect to space usage, it seems you wholly reject the viability of a
payment pool mechanism to cut-through chain space. Is this a critique that
holds for all Payment Pools, or just in the context of mining? Is there a
particular reason why you think it infeasible that "strongly online"
counterparties would be able to coordinate more efficiently? Is it
preferable for miners, the nexus of decentralization for Bitcoin, to prefer
to use custodial services for pooling (which may require KYC/AM) over
bearing a cost of some extra potential chainload?

Lastly, with respect to complexity, the proposal is actually incredibly
simple when you take it in a broader context. Non Interactive Channels and
Payment Pools are useful by themselves, so are the operations to merge them
and swap balance across them. Therefore most of the complexity in this
proposal is relying on tools we'll likely see in everyday use in any case,
DCFMP or no.

Jeremy

[-- Attachment #2: Type: text/html, Size: 4079 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 1716 bytes --]

I've received some confused messages that whatever I was replying to didn't
come through, I've reproduced Bob's e-mail below that I was responding to
for context:

























*This, quite simply, is not a "pool". A pool is by definition a tool to
reduceprofit variance by miners by collecting "weak blocks" that do not
meet thedifficulty target, so as to get a better statistical measure of
each miner'shashrate, which is used to subdivide profits. These are called
"shares" and areentirely absent here.The only available information here to
decide payouts is the blocks themselves,I do not have any higher statistics
measurement to subdivide payments. If Iexpect to earn 3 blocks within the
window, sometimes I will earn 2 and sometimesI will earn 4. Whether I keep
the entire coinbase in those 2-4 blocks, or I have100 other miners paying
me 1/100 as much 100 times, my payment is the same andmust be proportional
to the number of blocks I mine in the window.  My varianceis not
reduced.Further, by making miners pay other miners within the window N,
this results inN^2 payments to miners which otherwise would have had N
coinbase payments. So,this is extremely block-space inefficient for no good
reason. P2Pool had thesame problem and generated giant coinbases which
competed with fee revenue."Congestion control" makes this somewhat worse
since is it is an absoluteincrease in the block space consumed for these
N^2 payments.The only thing this proposal does do is smooth out fee
revenue. While hedging onfee revenue is valuable, this is an extremely
complicated and expensive way togo about it, that simultaneously *reduces*
fee revenue due to all the extrablock space used for miner payouts.*

>

[-- Attachment #2: Type: text/html, Size: 5919 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Bob McElrath]

You are hand waving. Attempting to redefine terms to justify your argument is
intellectually dishonest. Bitcoin pools have *always* been about variance
reduction. Your window function fundamentally CANNOT be used to hedge hashrate.
Various suggestions below introduce dangerous new games that might be played by
miners.

The fact is that the half-baked design you posted is less than useless, and
doesn't do anything that anyone wants.

You are trying to justify CTV by making it be all things to all people. "When
all you have is a hammer, every problem looks like a nail".  Instead I humbly
suggest that you pick ONE problem for which CTV is demonstrably the right and
best solution, instead of snowing us with a ton of half-baked things that
*could* be done, and often don't even require CTV, and some (like this one)
fundamentally don't work. I do like some of your ideas, but if you had to pick
just one "use case", which would it be?

Jeremy [jlrubin@mit.edu] wrote:
> Bitcoin didn't invent the concept of pooling: https://en.wikipedia.org/wiki/
> Pooling_(resource_management). This is a Bitcoin Mining Pool, although it may
> not be your favorite kind, which is fixated on specific properties of computing
> contributions before finding a block. Pooling is just a general technique for
> aggregating resources to accomplish something. If you have another name like
> pooling that is in common use for this type of activity I would be more than
> happy to adopt it.
> 
> This sort of pool can hedge not only against fee rates but also against
> increases in hashrate since your historical rate 'carries' into the future as a
> function of the window. Further, windows and reward functions can be defined in
> a myriad of ways that could, e.g., pay less to blocks found in more rapid
> succession, contributing to the smoothing functionality.
> 
> With respect to sub-block pooling, as described in the article, this sort of
> design also helps with micro-pools being able to split resources
> non-custodially in every block as a part of the higher order DCFMP. The point
> is not, as noted, to enable solo mining an S9, but to decrease the size of the
> minimum viable pool. It's also possible to add, without much validation or
> data, some 'uncle block' type mechanism in an incentive compatible way (e.g.,
> add 10 pow-heavy headers on the last block for cost 48 bytes header + 32 bytes
> payout key) such that there's an incentive to include the heaviest ones you've
> seen, not just your own, that are worth further study and consideration
> (particularly because it's non-consensus, only for opt-in participation in the
> pool).
> 
> With respect to space usage, it seems you wholly reject the viability of a
> payment pool mechanism to cut-through chain space. Is this a critique that
> holds for all Payment Pools, or just in the context of mining? Is there a
> particular reason why you think it infeasible that "strongly online"
> counterparties would be able to coordinate more efficiently? Is it preferable
> for miners, the nexus of decentralization for Bitcoin, to prefer to use
> custodial services for pooling (which may require KYC/AM) over bearing a cost
> of some extra potential chainload?
> 
> Lastly, with respect to complexity, the proposal is actually incredibly simple
> when you take it in a broader context. Non Interactive Channels and Payment
> Pools are useful by themselves, so are the operations to merge them and swap
> balance across them. Therefore most of the complexity in this proposal is
> relying on tools we'll likely see in everyday use in any case, DCFMP or no.
> 
> Jeremy
> !DSPAM:61b8f2f5321461582627336!
--
Cheers, Bob McElrath

"For every complex problem, there is a solution that is simple, neat, and wrong."
    -- H. L. Mencken 

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Billy Tetrud]

[-- Attachment #1: Type: text/plain, Size: 6368 bytes --]

Looks like an interesting proposal, but it doesn't seem to quite match the
goals you mentioned. As you do mention, this mining pool coordination
doesn't get rid of the need for mining pools in the first place. So it
doesn't satisfy item 1 on your goal list afaict.

The primary benefits over what we have today that I can see are:
1. increased payout regularity, which lowers the viable size of mining
pools, and
2. Lower on chain footprint through combining pay outs from multiple pools.

Am I missing some?

These are interesting benefits, but it would be nice if your post was
clearer on that, since the goals list is not the same as the list of
potential benefits of this kind of design.

As far as enabling solo mining, what if this concept were used off chain?
Have a public network of solo miners who publish "weak blocks" to that
network, and the next 100 (or 1000 etc) nice miners pay you out as long as
you're also being nice by following the protocol? All the nice
optimizations you mentioned about eg combined taproot payouts would apply i
think. The only goals this wouldn't satisfy are 3 and 5 since an extra
network is needed, but to be fair, your proposal requires pools which all
need their own extra network anyways.

The missing piece here would be an ordering of weak blocks to make the
window possible. Or at least a way to determine what blocks should
definitely be part of a particular block's pay out. I could see this being
done by a separate ephemeral blockchain (which starts fresh after each
Bitcoin block) that keeps track of which weak blocks have been submitted,
potentially using the pow already in each block to secure it. Granted that
piece is a bit half baked, but it seems quite solvable. Wdyt?

One thing that jumped out at me as not safe is throwing block rewards into
a channel and being able to spend them immediately. There's a reason block
rewards aren't spendable for a while, and channels don't solve that
problem, do they? Why not simply reduce the on chain wait time for spending
block rewards at that point? Seems like the consequences would be the same.

On Tue, Dec 14, 2021, 16:12 Bob McElrath via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> You are hand waving. Attempting to redefine terms to justify your argument
> is
> intellectually dishonest. Bitcoin pools have *always* been about variance
> reduction. Your window function fundamentally CANNOT be used to hedge
> hashrate.
> Various suggestions below introduce dangerous new games that might be
> played by
> miners.
>
> The fact is that the half-baked design you posted is less than useless, and
> doesn't do anything that anyone wants.
>
> You are trying to justify CTV by making it be all things to all people.
> "When
> all you have is a hammer, every problem looks like a nail".  Instead I
> humbly
> suggest that you pick ONE problem for which CTV is demonstrably the right
> and
> best solution, instead of snowing us with a ton of half-baked things that
> *could* be done, and often don't even require CTV, and some (like this one)
> fundamentally don't work. I do like some of your ideas, but if you had to
> pick
> just one "use case", which would it be?
>
> Jeremy [jlrubin@mit.edu] wrote:
> > Bitcoin didn't invent the concept of pooling:
> https://en.wikipedia.org/wiki/
> > Pooling_(resource_management). This is a Bitcoin Mining Pool, although
> it may
> > not be your favorite kind, which is fixated on specific properties of
> computing
> > contributions before finding a block. Pooling is just a general
> technique for
> > aggregating resources to accomplish something. If you have another name
> like
> > pooling that is in common use for this type of activity I would be more
> than
> > happy to adopt it.
> >
> > This sort of pool can hedge not only against fee rates but also against
> > increases in hashrate since your historical rate 'carries' into the
> future as a
> > function of the window. Further, windows and reward functions can be
> defined in
> > a myriad of ways that could, e.g., pay less to blocks found in more rapid
> > succession, contributing to the smoothing functionality.
> >
> > With respect to sub-block pooling, as described in the article, this
> sort of
> > design also helps with micro-pools being able to split resources
> > non-custodially in every block as a part of the higher order DCFMP. The
> point
> > is not, as noted, to enable solo mining an S9, but to decrease the size
> of the
> > minimum viable pool. It's also possible to add, without much validation
> or
> > data, some 'uncle block' type mechanism in an incentive compatible way
> (e.g.,
> > add 10 pow-heavy headers on the last block for cost 48 bytes header + 32
> bytes
> > payout key) such that there's an incentive to include the heaviest ones
> you've
> > seen, not just your own, that are worth further study and consideration
> > (particularly because it's non-consensus, only for opt-in participation
> in the
> > pool).
> >
> > With respect to space usage, it seems you wholly reject the viability of
> a
> > payment pool mechanism to cut-through chain space. Is this a critique
> that
> > holds for all Payment Pools, or just in the context of mining? Is there a
> > particular reason why you think it infeasible that "strongly online"
> > counterparties would be able to coordinate more efficiently? Is it
> preferable
> > for miners, the nexus of decentralization for Bitcoin, to prefer to use
> > custodial services for pooling (which may require KYC/AM) over bearing a
> cost
> > of some extra potential chainload?
> >
> > Lastly, with respect to complexity, the proposal is actually incredibly
> simple
> > when you take it in a broader context. Non Interactive Channels and
> Payment
> > Pools are useful by themselves, so are the operations to merge them and
> swap
> > balance across them. Therefore most of the complexity in this proposal is
> > relying on tools we'll likely see in everyday use in any case, DCFMP or
> no.
> >
> > Jeremy
> > !DSPAM:61b8f2f5321461582627336!
> --
> Cheers, Bob McElrath
>
> "For every complex problem, there is a solution that is simple, neat, and
> wrong."
>     -- H. L. Mencken
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 7846 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 6395 bytes --]

Hi Billy!

Thanks for your response. Some replies inline:


On Wed, Dec 15, 2021 at 10:01 AM Billy Tetrud via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Looks like an interesting proposal, but it doesn't seem to quite match the
> goals you mentioned. As you do mention, this mining pool coordination
> doesn't get rid of the need for mining pools in the first place. So it
> doesn't satisfy item 1 on your goal list afaict.
>

It does, actually :) Point 1 was

   1. Funds should not be centrally custodied, ever, if at all

And for top-level pool participants there is never any central custody.
What the windows are there (100 blocks, 2016, 4032, etc) is up to the
specific implementation which sets limits on how small you can be to
participate.

Further, for the entities that are too small:

from the article:
*> **The blocks that they mine should use a taproot address/key which is a
multisig of some portion of the workshares, that gets included in the
top-level pool as a part of Payment Pool.*

The micro-pools embed a multisig of top-contributors, 'reputable' members,
or on a rotating basis, as a leaf node to the parent. They then opt-out of
having their leaf channel-ized, as noted.

This would be fully non-custodial if we always included all miners. The
issue is that opens up DoS if one miner goes away, so you do want to anchor
around a few.

In this mode, you can set the protocol up such that immediately after
getting a reward in a block, you should see the chosen nodes for multi-sigs
distribute the spoils according to the schedule that is agreed on in the
block causing the share to be granted.

the main issue is data availability, without extra in-band storage local
mining pools have to track the work shares (which can be committed to in a
block) locally for auditing.

This is not fully non-custodial, but it doesn't have to be centrally
custodied by one party. We can multisig immediately after every block (and
nodes should quit their pool if they don't get sigs quickly perhaps).
Further, nodes can hash into multiple pools dividing their risk (modulo
sybil attack) across many pools.

If we had stronger covenants (CAT, AMOUNT, DIVIDE/MUL), we could make every
leaf node commit to payment pools that operate on percents instead of fixed
amounts and we'd be able to handle this in a manner that the payment pools
work no matter what amount is assigned to them.



The primary benefits over what we have today that I can see are:
> 1. increased payout regularity, which lowers the viable size of mining
> pools, and
> 2. Lower on chain footprint through combining pay outs from multiple pools.
>
> Am I missing some?
>
> These are interesting benefits, but it would be nice if your post was
> clearer on that, since the goals list is not the same as the list of
> potential benefits of this kind of design.
>

I think I hit all the benefits mentioned:

1. Funds should not be centrally custodied, ever, if at all.
see above -- we can do better for smaller miners, but we hit this for
miners above the threshold.

2. No KYC/AML.
see above, payouts are done 'decentralized' by every miner mining to the
payout

3. No “Extra network” software required.
you need the WASM, but do not need any networked software to participate,
so there are no DoS concerns from participating.

You do need extra software to e.g. use channels or cut-through multiple
pools, but only after the fact of minding.

4. No blockchain bloat.

Very little, if cut-through + LN works.


5. No extra infrastructure.

Not much needed, if anything. I don't really know what 'infrastructure'
means, but I kind of imagined it to mean 'big expensive things' that would
make it hard to partake.


6. The size of a viable pool should be smaller. Remember our singer – if
you just pool with one other songwriter it doesn’t make your expected time
till payout in your lifetime. So bigger the pools, more regular the
payouts. We want the smallest possible “units of control” with the most
regular payouts possible.

I think this works, roughly?


> As far as enabling solo mining, what if this concept were used off chain?
> Have a public network of solo miners who publish "weak blocks" to that
> network, and the next 100 (or 1000 etc) nice miners pay you out as long as
> you're also being nice by following the protocol? All the nice
> optimizations you mentioned about eg combined taproot payouts would apply i
> think. The only goals this wouldn't satisfy are 3 and 5 since an extra
> network is needed, but to be fair, your proposal requires pools which all
> need their own extra network anyways.
>
> The missing piece here would be an ordering of weak blocks to make the
> window possible. Or at least a way to determine what blocks should
> definitely be part of a particular block's pay out. I could see this being
> done by a separate ephemeral blockchain (which starts fresh after each
> Bitcoin block) that keeps track of which weak blocks have been submitted,
> potentially using the pow already in each block to secure it. Granted that
> piece is a bit half baked, but it seems quite solvable. Wdyt?
>
>
Yeah, it's worth thinking more about 100%. This post wasn't a deployable
thing, more an exposition of a technique. I'd love to see a weak-block
based pool, the main issue as noted is the extra software component + data
availability, but perhaps that's solvable!



> One thing that jumped out at me as not safe is throwing block rewards into
> a channel and being able to spend them immediately. There's a reason block
> rewards aren't spendable for a while, and channels don't solve that
> problem, do they? Why not simply reduce the on chain wait time for spending
> block rewards at that point? Seems like the consequences would be the same.
>

Miners could already do this if they mine to e.g. a multisig (trustlessly
if they form blocks with their counterparty and pre-sign before hashing).
Also in lightning we don't generally have to check that our routes channels
exist, we don't care as long as they are happy. Thus it doesn't "hurt"
anyone except for the miners who are taking the not fully locked in funds
risk, a risk they already take. But that risk can't infect the rest of
Bitcoin's users.

[-- Attachment #2: Type: text/html, Size: 14303 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Bob McElrath]

You basically described Braidpool:
    https://github.com/pool2win/braidpool
    https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-August/019371.html

We're working on this actively and will have some updates soon. Additional
contributors are most welcome.

To your points below:
1. Increased payout regularity does not lower the viable size of mining pools,
    because smaller mining pools using this mechanism still have higher variance.
2. The on-chain footprint is *higher* due to the increased payout regularity.

For a talk a while back I computed that if you want to have a 1% annual variance
on your profits, you need a pool that is about 20% of the network. The only way
to get this is with smaller, faster blocks or "shares". 
    https://www.youtube.com/watch?v=91WKy7RYHD4

There is a discussion forum at:
    https://matrix.to/#/#braidpool:matrix.org
and a mailing list:
    https://sourceforge.net/p/braidpool/mailman/

All of the existing discussion has been happening privately unfortunately but
I'll try to start using Matrix. ;-)

We've been discussing alternatives for both fee-rate and hashrate derivatives
lately. I'm not opposed to using CTV for some of the things in braidpool, if it
makes sense. Payment pools and unilateral channel openings may be interesting in
this context.

P.S. if anyone wants me to write up a blurb of exactly *why* a construction
without shares cannot be used for hashrate derivatives I can do that, just ask.
It comes down to maximum likelihood estimators for the Poisson distribution...

Billy Tetrud [billy.tetrud@gmail.com] wrote:
> Looks like an interesting proposal, but it doesn't seem to quite match the
> goals you mentioned. As you do mention, this mining pool coordination doesn't
> get rid of the need for mining pools in the first place. So it doesn't satisfy
> item 1 on your goal list afaict. 
> 
> The primary benefits over what we have today that I can see are:
> 1. increased payout regularity, which lowers the viable size of mining pools,
> and
> 2. Lower on chain footprint through combining pay outs from multiple pools.
> 
> Am I missing some?
> 
> These are interesting benefits, but it would be nice if your post was clearer
> on that, since the goals list is not the same as the list of potential benefits
> of this kind of design.
> 
> As far as enabling solo mining, what if this concept were used off chain? Have
> a public network of solo miners who publish "weak blocks" to that network, and
> the next 100 (or 1000 etc) nice miners pay you out as long as you're also being
> nice by following the protocol? All the nice optimizations you mentioned about
> eg combined taproot payouts would apply i think. The only goals this wouldn't
> satisfy are 3 and 5 since an extra network is needed, but to be fair, your
> proposal requires pools which all need their own extra network anyways. 
> 
> The missing piece here would be an ordering of weak blocks to make the window
> possible. Or at least a way to determine what blocks should definitely be part
> of a particular block's pay out. I could see this being done by a separate
> ephemeral blockchain (which starts fresh after each Bitcoin block) that keeps
> track of which weak blocks have been submitted, potentially using the pow
> already in each block to secure it. Granted that piece is a bit half baked, but
> it seems quite solvable. Wdyt?
> 
> One thing that jumped out at me as not safe is throwing block rewards into a
> channel and being able to spend them immediately. There's a reason block
> rewards aren't spendable for a while, and channels don't solve that problem, do
> they? Why not simply reduce the on chain wait time for spending block rewards
> at that point? Seems like the consequences would be the same.
> 
> On Tue, Dec 14, 2021, 16:12 Bob McElrath via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
>     You are hand waving. Attempting to redefine terms to justify your argument
>     is
>     intellectually dishonest. Bitcoin pools have *always* been about variance
>     reduction. Your window function fundamentally CANNOT be used to hedge
>     hashrate.
>     Various suggestions below introduce dangerous new games that might be
>     played by
>     miners.
> 
>     The fact is that the half-baked design you posted is less than useless, and
>     doesn't do anything that anyone wants.
> 
>     You are trying to justify CTV by making it be all things to all people.
>     "When
>     all you have is a hammer, every problem looks like a nail".  Instead I
>     humbly
>     suggest that you pick ONE problem for which CTV is demonstrably the right
>     and
>     best solution, instead of snowing us with a ton of half-baked things that
>     *could* be done, and often don't even require CTV, and some (like this one)
>     fundamentally don't work. I do like some of your ideas, but if you had to
>     pick
>     just one "use case", which would it be?
> 
>     Jeremy [jlrubin@mit.edu] wrote:
>     > Bitcoin didn't invent the concept of pooling: https://en.wikipedia.org/
>     wiki/
>     > Pooling_(resource_management). This is a Bitcoin Mining Pool, although it
>     may
>     > not be your favorite kind, which is fixated on specific properties of
>     computing
>     > contributions before finding a block. Pooling is just a general technique
>     for
>     > aggregating resources to accomplish something. If you have another name
>     like
>     > pooling that is in common use for this type of activity I would be more
>     than
>     > happy to adopt it.
>     >
>     > This sort of pool can hedge not only against fee rates but also against
>     > increases in hashrate since your historical rate 'carries' into the
>     future as a
>     > function of the window. Further, windows and reward functions can be
>     defined in
>     > a myriad of ways that could, e.g., pay less to blocks found in more rapid
>     > succession, contributing to the smoothing functionality.
>     >
>     > With respect to sub-block pooling, as described in the article, this sort
>     of
>     > design also helps with micro-pools being able to split resources
>     > non-custodially in every block as a part of the higher order DCFMP. The
>     point
>     > is not, as noted, to enable solo mining an S9, but to decrease the size
>     of the
>     > minimum viable pool. It's also possible to add, without much validation
>     or
>     > data, some 'uncle block' type mechanism in an incentive compatible way
>     (e.g.,
>     > add 10 pow-heavy headers on the last block for cost 48 bytes header + 32
>     bytes
>     > payout key) such that there's an incentive to include the heaviest ones
>     you've
>     > seen, not just your own, that are worth further study and consideration
>     > (particularly because it's non-consensus, only for opt-in participation
>     in the
>     > pool).
>     >
>     > With respect to space usage, it seems you wholly reject the viability of
>     a
>     > payment pool mechanism to cut-through chain space. Is this a critique
>     that
>     > holds for all Payment Pools, or just in the context of mining? Is there a
>     > particular reason why you think it infeasible that "strongly online"
>     > counterparties would be able to coordinate more efficiently? Is it
>     preferable
>     > for miners, the nexus of decentralization for Bitcoin, to prefer to use
>     > custodial services for pooling (which may require KYC/AM) over bearing a
>     cost
>     > of some extra potential chainload?
>     >
>     > Lastly, with respect to complexity, the proposal is actually incredibly
>     simple
>     > when you take it in a broader context. Non Interactive Channels and
>     Payment
>     > Pools are useful by themselves, so are the operations to merge them and
>     swap
>     > balance across them. Therefore most of the complexity in this proposal is
>     > relying on tools we'll likely see in everyday use in any case, DCFMP or
>     no.
>     >
>     > Jeremy
>     >
>     --
>     Cheers, Bob McElrath
> 
>     "For every complex problem, there is a solution that is simple, neat, and
>     wrong."
>         -- H. L. Mencken
> 
>     _______________________________________________
>     bitcoin-dev mailing list
>     bitcoin-dev@lists.linuxfoundation.org
>     https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 
> !DSPAM:61ba2512470948607217095!
--
Cheers, Bob McElrath

"For every complex problem, there is a solution that is simple, neat, and wrong."
    -- H. L. Mencken 

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[vjudeu]

[-- Attachment #1: Type: text/plain, Size: 8618 bytes --]

> The missing piece here would be an ordering of weak blocks to make the window possible. Or at least a way to determine what blocks should definitely be part of a particular block's pay out. I could see this being done by a separate ephemeral blockchain (which starts fresh after each Bitcoin block) that keeps track of which weak blocks have been submitted, potentially using the pow already in each block to secure it. Granted that piece is a bit half baked, but it seems quite solvable. Wdyt?
 
I thought about something like that, but there is one problem: how many block headers should be stored per one "superblock"? Currently, we have single block header, where the whole coinbase transaction is taken by some mining pool or solo miner. But instead, each miner could submit its own block header. Then, we can collect all headers with the same previous block hash, and distribute block reward between all coinbase transactions in those headers. One "superblock" then would be created in a similar way as existing blocks, we would just have block headers instead of transactions. If most transactions inside those blocks will be the same, then each block could be expressed just as a set of transaction hashes, only coinbase transactions or custom, non-broadcasted transactions included by miners will be revealed, everything else will be known.
> One thing that jumped out at me as not safe is throwing block rewards into a channel and being able to spend them immediately. There's a reason block rewards aren't spendable for a while, and channels don't solve that problem, do they? Why not simply reduce the on chain wait time for spending block rewards at that point? Seems like the consequences would be the same.
All coinbase rewards are unspendable for 100 blocks, it is enforced by consensus. It does not matter if there are outputs owned directly by miners, or if there is one huge N-of-N taproot multisig for the whole pool, where every miner signed the closing transaction. The only option to take coins faster I can see is swapping the coins by some LN transaction. But then, the other party can check if some deposit to the LN channel is a part of the coinbase transaction or not, and then decide if it is acceptable to do the swap.
On 2021-12-15 19:00:44 user Billy Tetrud via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
Looks like an interesting proposal, but it doesn't seem to quite match the goals you mentioned. As you do mention, this mining pool coordination doesn't get rid of the need for mining pools in the first place. So it doesn't satisfy item 1 on your goal list afaict.   
The primary benefits over what we have today that I can see are:
1. increased payout regularity, which lowers the viable size of mining pools, and
2. Lower on chain footprint through combining pay outs from multiple pools.
 
Am I missing some?
 
These are interesting benefits, but it would be nice if your post was clearer on that, since the goals list is not the same as the list of potential benefits of this kind of design.
 
As far as enabling solo mining, what if this concept were used off chain? Have a public network of solo miners who publish "weak blocks" to that network, and the next 100 (or 1000 etc) nice miners pay you out as long as you're also being nice by following the protocol? All the nice optimizations you mentioned about eg combined taproot payouts would apply i think. The only goals this wouldn't satisfy are 3 and 5 since an extra network is needed, but to be fair, your proposal requires pools which all need their own extra network anyways. 
 
The missing piece here would be an ordering of weak blocks to make the window possible. Or at least a way to determine what blocks should definitely be part of a particular block's pay out. I could see this being done by a separate ephemeral blockchain (which starts fresh after each Bitcoin block) that keeps track of which weak blocks have been submitted, potentially using the pow already in each block to secure it. Granted that piece is a bit half baked, but it seems quite solvable. Wdyt?
 
One thing that jumped out at me as not safe is throwing block rewards into a channel and being able to spend them immediately. There's a reason block rewards aren't spendable for a while, and channels don't solve that problem, do they? Why not simply reduce the on chain wait time for spending block rewards at that point? Seems like the consequences would be the same.
On Tue, Dec 14, 2021, 16:12 Bob McElrath via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
You are hand waving. Attempting to redefine terms to justify your argument is
intellectually dishonest. Bitcoin pools have *always* been about variance
reduction. Your window function fundamentally CANNOT be used to hedge hashrate.
Various suggestions below introduce dangerous new games that might be played by
miners.
The fact is that the half-baked design you posted is less than useless, and
doesn't do anything that anyone wants.
You are trying to justify CTV by making it be all things to all people. "When
all you have is a hammer, every problem looks like a nail".  Instead I humbly
suggest that you pick ONE problem for which CTV is demonstrably the right and
best solution, instead of snowing us with a ton of half-baked things that
*could* be done, and often don't even require CTV, and some (like this one)
fundamentally don't work. I do like some of your ideas, but if you had to pick
just one "use case", which would it be?
Jeremy [jlrubin@mit.edu] wrote:
> Bitcoin didn't invent the concept of pooling: https://en.wikipedia.org/wiki/
> Pooling_(resource_management). This is a Bitcoin Mining Pool, although it may
> not be your favorite kind, which is fixated on specific properties of computing
> contributions before finding a block. Pooling is just a general technique for
> aggregating resources to accomplish something. If you have another name like
> pooling that is in common use for this type of activity I would be more than
> happy to adopt it.
>
> This sort of pool can hedge not only against fee rates but also against
> increases in hashrate since your historical rate 'carries' into the future as a
> function of the window. Further, windows and reward functions can be defined in
> a myriad of ways that could, e.g., pay less to blocks found in more rapid
> succession, contributing to the smoothing functionality.
>
> With respect to sub-block pooling, as described in the article, this sort of
> design also helps with micro-pools being able to split resources
> non-custodially in every block as a part of the higher order DCFMP. The point
> is not, as noted, to enable solo mining an S9, but to decrease the size of the
> minimum viable pool. It's also possible to add, without much validation or
> data, some 'uncle block' type mechanism in an incentive compatible way (e.g.,
> add 10 pow-heavy headers on the last block for cost 48 bytes header + 32 bytes
> payout key) such that there's an incentive to include the heaviest ones you've
> seen, not just your own, that are worth further study and consideration
> (particularly because it's non-consensus, only for opt-in participation in the
> pool).
>
> With respect to space usage, it seems you wholly reject the viability of a
> payment pool mechanism to cut-through chain space. Is this a critique that
> holds for all Payment Pools, or just in the context of mining? Is there a
> particular reason why you think it infeasible that "strongly online"
> counterparties would be able to coordinate more efficiently? Is it preferable
> for miners, the nexus of decentralization for Bitcoin, to prefer to use
> custodial services for pooling (which may require KYC/AM) over bearing a cost
> of some extra potential chainload?
>
> Lastly, with respect to complexity, the proposal is actually incredibly simple
> when you take it in a broader context. Non Interactive Channels and Payment
> Pools are useful by themselves, so are the operations to merge them and swap
> balance across them. Therefore most of the complexity in this proposal is
> relying on tools we'll likely see in everyday use in any case, DCFMP or no.
>
> Jeremy
> !DSPAM:61b8f2f5321461582627336!
--
Cheers, Bob McElrath
"For every complex problem, there is a solution that is simple, neat, and wrong."
    -- H. L. Mencken
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 10774 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Billy Tetrud]

[-- Attachment #1: Type: text/plain, Size: 13022 bytes --]

@Jeremy
>   for top-level pool participants there is never any central custody.

I definitely see that. That was actually what I meant when I said the goals
aren't the same as benefits. While your idea definitely satisfies all your
goals in a modular way, the fact that it relies on pools means that unless
the pools can also satisfy the goals, the total system also doesn't satisfy
the goals (even tho the piece of that system you designed does).

> Thus it doesn't "hurt" anyone except for the miners who are taking the
not fully locked in funds risk

True, it only potentially hurts whoever the channel partner is accepting
the unspendable coins. And no one can really stop anyone from taking that
risk if they really want to. But in that case, its not exactly a fully
functional channel, since recourse mechanisms couldn't be performed.
Wouldn't that open such a channel up to a pretty bad theft possibility?

@Bob
> Increased payout regularity does not lower the viable size of mining
pools, because smaller mining pools using this mechanism still have higher
variance.

Yes, smaller mining pools will always have higher variance. However, lower
variance has diminishing benefits. Below a certain amount of variance, less
variance isn't very valuable. So increased payout regularity does indeed
lower the viable size of mining pools because a given low-enough level of
variance can be achieved with less pool hashpower.

> The on-chain footprint is *higher* due to the increased payout regularity.

That's a reasonable point. However, I think there is a difference here
between the regularity of rewards vs payouts. Rewards for each miner can be
more regular without necessarily increasing the number of on-chain payouts.
In fact, theoretically, an individual miner could let their rewards
accumulate in a pool over many rewards and only redeem when they need the
coins for something. The incentive is there for each miner to be judicious
on how much onchain space they take up.

@vjudeu

> how many block headers should be stored per one "superblock"?

I was thinking that this would be a separate blockchain with separate
headers that progress linearly like a normal blockchain. A block creator
would collect together as many blocks that haven't been collected yet into
the next superblock (and maybe receive a reward proportional to how many /
how much weight they include). This could be done using merge mining, or it
could be done using a signing scheme (eg where the block creator signs to
say "I created this superblock" and have mechanisms to punish those who
sign multiple superblocks at the same height. For merge mining, I could
even imagine the data necessary to validate that it has been merge mined
could be put into a taproot script branch (creating an invalid script, but
a valid hash of the superblock).

> we can collect all headers with the same previous block hash, and
distribute block reward between all coinbase transactions in those headers

Exactly.

> we would just have block headers instead of transactions

Yeah, I think that would be the way to go. Really, you could even just use
hashes of the block headers. But the size doesn't matter much because it
would be both a small blockchain and an ephemeral one (which can be fully
discarded after all parties have been paid out, or at least their payout
has been committed to on the bitcoin blockchain).

On Thu, Dec 16, 2021 at 1:35 AM <vjudeu@gazeta.pl> wrote:

> > The missing piece here would be an ordering of weak blocks to make the
> window possible. Or at least a way to determine what blocks should
> definitely be part of a particular block's pay out. I could see this being
> done by a separate ephemeral blockchain (which starts fresh after each
> Bitcoin block) that keeps track of which weak blocks have been submitted,
> potentially using the pow already in each block to secure it. Granted that
> piece is a bit half baked, but it seems quite solvable. Wdyt?
>
> I thought about something like that, but there is one problem: how many
> block headers should be stored per one "superblock"? Currently, we have
> single block header, where the whole coinbase transaction is taken by some
> mining pool or solo miner. But instead, each miner could submit its own
> block header. Then, we can collect all headers with the same previous block
> hash, and distribute block reward between all coinbase transactions in
> those headers. One "superblock" then would be created in a similar way as
> existing blocks, we would just have block headers instead of transactions.
> If most transactions inside those blocks will be the same, then each block
> could be expressed just as a set of transaction hashes, only coinbase
> transactions or custom, non-broadcasted transactions included by miners
> will be revealed, everything else will be known.
>
> > One thing that jumped out at me as not safe is throwing block rewards
> into a channel and being able to spend them immediately. There's a reason
> block rewards aren't spendable for a while, and channels don't solve that
> problem, do they? Why not simply reduce the on chain wait time for spending
> block rewards at that point? Seems like the consequences would be the same.
>
> All coinbase rewards are unspendable for 100 blocks, it is enforced by
> consensus. It does not matter if there are outputs owned directly by
> miners, or if there is one huge N-of-N taproot multisig for the whole pool,
> where every miner signed the closing transaction. The only option to take
> coins faster I can see is swapping the coins by some LN transaction. But
> then, the other party can check if some deposit to the LN channel is a part
> of the coinbase transaction or not, and then decide if it is acceptable to
> do the swap.
>
> On 2021-12-15 19:00:44 user Billy Tetrud via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> Looks like an interesting proposal, but it doesn't seem to quite match the
> goals you mentioned. As you do mention, this mining pool coordination
> doesn't get rid of the need for mining pools in the first place. So it
> doesn't satisfy item 1 on your goal list afaict.
>
> The primary benefits over what we have today that I can see are:
> 1. increased payout regularity, which lowers the viable size of mining
> pools, and
> 2. Lower on chain footprint through combining pay outs from multiple pools.
>
> Am I missing some?
>
> These are interesting benefits, but it would be nice if your post was
> clearer on that, since the goals list is not the same as the list of
> potential benefits of this kind of design.
>
> As far as enabling solo mining, what if this concept were used off chain?
> Have a public network of solo miners who publish "weak blocks" to that
> network, and the next 100 (or 1000 etc) nice miners pay you out as long as
> you're also being nice by following the protocol? All the nice
> optimizations you mentioned about eg combined taproot payouts would apply i
> think. The only goals this wouldn't satisfy are 3 and 5 since an extra
> network is needed, but to be fair, your proposal requires pools which all
> need their own extra network anyways.
>
> The missing piece here would be an ordering of weak blocks to make the
> window possible. Or at least a way to determine what blocks should
> definitely be part of a particular block's pay out. I could see this being
> done by a separate ephemeral blockchain (which starts fresh after each
> Bitcoin block) that keeps track of which weak blocks have been submitted,
> potentially using the pow already in each block to secure it. Granted that
> piece is a bit half baked, but it seems quite solvable. Wdyt?
>
> One thing that jumped out at me as not safe is throwing block rewards into
> a channel and being able to spend them immediately. There's a reason block
> rewards aren't spendable for a while, and channels don't solve that
> problem, do they? Why not simply reduce the on chain wait time for spending
> block rewards at that point? Seems like the consequences would be the same.
>
> On Tue, Dec 14, 2021, 16:12 Bob McElrath via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org
> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchE6GhA5LFpLCUc7EVZQVl9dQRIXXR8NCBMbCwIGChJXQFxcXEgcFh8UVVVDEyBdVkE9JVRdEwFhYXVlblhVIkosEAszLR5BQVV7U0MID0BAQUgIGh0RHgAMGAMXBQJfW1sdXRQUQUoDQlAiBFY8>>
> wrote:
>
>> You are hand waving. Attempting to redefine terms to justify your
>> argument is
>> intellectually dishonest. Bitcoin pools have *always* been about variance
>> reduction. Your window function fundamentally CANNOT be used to hedge
>> hashrate.
>> Various suggestions below introduce dangerous new games that might be
>> played by
>> miners.
>>
>> The fact is that the half-baked design you posted is less than useless,
>> and
>> doesn't do anything that anyone wants.
>>
>> You are trying to justify CTV by making it be all things to all people.
>> "When
>> all you have is a hammer, every problem looks like a nail".  Instead I
>> humbly
>> suggest that you pick ONE problem for which CTV is demonstrably the right
>> and
>> best solution, instead of snowing us with a ton of half-baked things that
>> *could* be done, and often don't even require CTV, and some (like this
>> one)
>> fundamentally don't work. I do like some of your ideas, but if you had to
>> pick
>> just one "use case", which would it be?
>>
>> Jeremy [jlrubin@mit.edu
>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchEyHxYvIVpLARduChoQSFZQR0NWQVZWJUNRXwMSCRMTBgcWASdWVkpbCxUTQwoWQUdjKVBMGFY3MWMWeU9QBAZtNw%3D%3D>]
>> wrote:
>> > Bitcoin didn't invent the concept of pooling:
>> https://en.wikipedia.org/wiki/
>> > Pooling_(resource_management). This is a Bitcoin Mining Pool, although
>> it may
>> > not be your favorite kind, which is fixated on specific properties of
>> computing
>> > contributions before finding a block. Pooling is just a general
>> technique for
>> > aggregating resources to accomplish something. If you have another name
>> like
>> > pooling that is in common use for this type of activity I would be more
>> than
>> > happy to adopt it.
>> >
>> > This sort of pool can hedge not only against fee rates but also against
>> > increases in hashrate since your historical rate 'carries' into the
>> future as a
>> > function of the window. Further, windows and reward functions can be
>> defined in
>> > a myriad of ways that could, e.g., pay less to blocks found in more
>> rapid
>> > succession, contributing to the smoothing functionality.
>> >
>> > With respect to sub-block pooling, as described in the article, this
>> sort of
>> > design also helps with micro-pools being able to split resources
>> > non-custodially in every block as a part of the higher order DCFMP. The
>> point
>> > is not, as noted, to enable solo mining an S9, but to decrease the size
>> of the
>> > minimum viable pool. It's also possible to add, without much validation
>> or
>> > data, some 'uncle block' type mechanism in an incentive compatible way
>> (e.g.,
>> > add 10 pow-heavy headers on the last block for cost 48 bytes header +
>> 32 bytes
>> > payout key) such that there's an incentive to include the heaviest ones
>> you've
>> > seen, not just your own, that are worth further study and consideration
>> > (particularly because it's non-consensus, only for opt-in participation
>> in the
>> > pool).
>> >
>> > With respect to space usage, it seems you wholly reject the viability
>> of a
>> > payment pool mechanism to cut-through chain space. Is this a critique
>> that
>> > holds for all Payment Pools, or just in the context of mining? Is there
>> a
>> > particular reason why you think it infeasible that "strongly online"
>> > counterparties would be able to coordinate more efficiently? Is it
>> preferable
>> > for miners, the nexus of decentralization for Bitcoin, to prefer to use
>> > custodial services for pooling (which may require KYC/AM) over bearing
>> a cost
>> > of some extra potential chainload?
>> >
>> > Lastly, with respect to complexity, the proposal is actually incredibly
>> simple
>> > when you take it in a broader context. Non Interactive Channels and
>> Payment
>> > Pools are useful by themselves, so are the operations to merge them and
>> swap
>> > balance across them. Therefore most of the complexity in this proposal
>> is
>> > relying on tools we'll likely see in everyday use in any case, DCFMP or
>> no.
>> >
>> > Jeremy
>> > !DSPAM:61b8f2f5321461582627336!
>> --
>> Cheers, Bob McElrath
>>
>> "For every complex problem, there is a solution that is simple, neat, and
>> wrong."
>>     -- H. L. Mencken
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchE6GhA5LFpLCUc7EVZQVl9dQRIXXR8NCBMbCwIGChJXQFxcXEgcFh8UVVVDEyBdVkE9JVRdEwFhYXVlblhVIkosEAszLR5BQVV7U0MID0BAQUgIGh0RHgAMGAMXBQJfW1sdXRQUQUoDQlAiBFY8>
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 15772 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 745 bytes --]

high level response:

including a small number of block headers (10?) directly as op_return
metadata (or something) doesn't have that high overhead necessarily, but
could be super effective at helping miners participate with lower hashrate.
the reason to include this as on-chain data is so that the mining pool
doesn't require any external network software.

this would balance out the issues if the data is somewhat bounded (e.g., 10
headers). what's nice is this data has no consensus meaning as it's client
side validated by the DCFMP block filter.

interestingly, the participating pools could 'vote' on how difficult shares
should be as a metaparameter to the pool over blocks... but analysis gets
more complex with that.

cheers,

jeremy

[-- Attachment #2: Type: text/html, Size: 2046 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[vjudeu]

[-- Attachment #1: Type: text/plain, Size: 15153 bytes --]

> I was thinking that this would be a separate blockchain with separate headers that progress linearly like a normal blockchain.
Exactly, that's what I called "superblocks", where you have a separate chain, just to keep block headers instead of transactions.
> A block creator would collect together as many blocks that haven't been collected yet into the next superblock (and maybe receive a reward proportional to how many / how much weight they include).
You cannot "catch them all". If you do, you can end up with a lot of block headers, where each of them has difficulty equal to one. You need some limit, you can limit amount of blocks, you can assign some minimal difficulty, it does not matter that much, but some limit is needed, also because mining on top of the latest superblock should be more profitable than replacing someone else's reward in the previous superblock by your own reward and getting a bigger share in the previous superblock.
> This could be done using merge mining, or it could be done using a signing scheme (eg where the block creator signs to say "I created this superblock" and have mechanisms to punish those who sign multiple superblocks at the same height.
I would pick merge mining, because it is more compatible with existing mining scheme. Signing sounds more like Proof of Stake and I am trying to avoid that solution. Also, there is no need to sign anything, because you are solo mining where you have your own coinbase transaction or you are mining in a pool, where you have some shared address, and then you cannot produce any incompatible superblock, because the protocol can tell you, which address you should use (and if it is N-of-N taproot multisig and you have some closing transaction, then you can safely mine it).
> Really, you could even just use hashes of the block headers.
Replacing transactions with block headers will do the same trick. Each transaction is first hashed with double SHA-256, in exactly the same way as block headers are. If you replace transactions with block headers, you would get a superblock header, then varint saying how many block headers are there, and then you can place all block headers. During superblock merkle tree construction, you will hash all block headers (so you will get block hashes as leaves), and then you will combine block hashes in the same way as transaction hashes are combined.
From the Script point of view, you can always use "OP_SIZE 80 OP_EQUALVERIFY OP_HASH256 <hash> OP_EQUAL". Then, you can just change the size, just to show which object is hashed. Value 80 will work for block headers, small values below 520 will work for small transactions, value 64 will work for any merkle tree proof, no matter if it is for superblock or normal block. Also, by using block headers instead of hashes, you can prove that at least a proper amount of work was done to produce it, because if you use just hashes, then they could be random.
On 2021-12-16 17:57:23 user Billy Tetrud <billy.tetrud@gmail.com> wrote:
@Jeremy
>   for top-level pool participants there is never any central custody.
 
I definitely see that. That was actually what I meant when I said the goals aren't the same as benefits. While your idea definitely satisfies all your goals in a modular way, the fact that it relies on pools means that unless the pools can also satisfy the goals, the total system also doesn't satisfy the goals (even tho the piece of that system you designed does). 
 
> Thus it doesn't "hurt" anyone except for the miners who are taking the not fully locked in funds risk
 
True, it only potentially hurts whoever the channel partner is accepting the unspendable coins. And no one can really stop anyone from taking that risk if they really want to. But in that case, its not exactly a fully functional channel, since recourse mechanisms couldn't be performed. Wouldn't that open such a channel up to a pretty bad theft possibility?
 
@Bob
> Increased payout regularity does not lower the viable size of mining pools, because smaller mining pools using this mechanism still have higher variance.
 
Yes, smaller mining pools will always have higher variance. However, lower variance has diminishing benefits. Below a certain amount of variance, less variance isn't very valuable. So increased payout regularity does indeed lower the viable size of mining pools because a given low-enough level of variance can be achieved with less pool hashpower.
 
> The on-chain footprint is *higher* due to the increased payout regularity.
 
That's a reasonable point. However, I think there is a difference here between the regularity of rewards vs payouts. Rewards for each miner can be more regular without necessarily increasing the number of on-chain payouts. In fact, theoretically, an individual miner could let their rewards accumulate in a pool over many rewards and only redeem when they need the coins for something. The incentive is there for each miner to be judicious on how much onchain space they take up.
 
@vjudeu
 
> how many block headers should be stored per one "superblock"?
 
I was thinking that this would be a separate blockchain with separate headers that progress linearly like a normal blockchain. A block creator would collect together as many blocks that haven't been collected yet into the next superblock (and maybe receive a reward proportional to how many / how much weight they include). This could be done using merge mining, or it could be done using a signing scheme (eg where the block creator signs to say "I created this superblock" and have mechanisms to punish those who sign multiple superblocks at the same height. For merge mining, I could even imagine the data necessary to validate that it has been merge mined could be put into a taproot script branch (creating an invalid script, but a valid hash of the superblock). 
 
> we can collect all headers with the same previous block hash, and distribute block reward between all coinbase transactions in those headers
 
Exactly.
 
> we would just have block headers instead of transactions
 
Yeah, I think that would be the way to go. Really, you could even just use hashes of the block headers. But the size doesn't matter much because it would be both a small blockchain and an ephemeral one (which can be fully discarded after all parties have been paid out, or at least their payout has been committed to on the bitcoin blockchain). 
On Thu, Dec 16, 2021 at 1:35 AM <vjudeu@gazeta.pl> wrote:
> The missing piece here would be an ordering of weak blocks to make the window possible. Or at least a way to determine what blocks should definitely be part of a particular block's pay out. I could see this being done by a separate ephemeral blockchain (which starts fresh after each Bitcoin block) that keeps track of which weak blocks have been submitted, potentially using the pow already in each block to secure it. Granted that piece is a bit half baked, but it seems quite solvable. Wdyt?
 
I thought about something like that, but there is one problem: how many block headers should be stored per one "superblock"? Currently, we have single block header, where the whole coinbase transaction is taken by some mining pool or solo miner. But instead, each miner could submit its own block header. Then, we can collect all headers with the same previous block hash, and distribute block reward between all coinbase transactions in those headers. One "superblock" then would be created in a similar way as existing blocks, we would just have block headers instead of transactions. If most transactions inside those blocks will be the same, then each block could be expressed just as a set of transaction hashes, only coinbase transactions or custom, non-broadcasted transactions included by miners will be revealed, everything else will be known.
> One thing that jumped out at me as not safe is throwing block rewards into a channel and being able to spend them immediately. There's a reason block rewards aren't spendable for a while, and channels don't solve that problem, do they? Why not simply reduce the on chain wait time for spending block rewards at that point? Seems like the consequences would be the same.
All coinbase rewards are unspendable for 100 blocks, it is enforced by consensus. It does not matter if there are outputs owned directly by miners, or if there is one huge N-of-N taproot multisig for the whole pool, where every miner signed the closing transaction. The only option to take coins faster I can see is swapping the coins by some LN transaction. But then, the other party can check if some deposit to the LN channel is a part of the coinbase transaction or not, and then decide if it is acceptable to do the swap.
On 2021-12-15 19:00:44 user Billy Tetrud via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
Looks like an interesting proposal, but it doesn't seem to quite match the goals you mentioned. As you do mention, this mining pool coordination doesn't get rid of the need for mining pools in the first place. So it doesn't satisfy item 1 on your goal list afaict.   
The primary benefits over what we have today that I can see are:
1. increased payout regularity, which lowers the viable size of mining pools, and
2. Lower on chain footprint through combining pay outs from multiple pools.
 
Am I missing some?
 
These are interesting benefits, but it would be nice if your post was clearer on that, since the goals list is not the same as the list of potential benefits of this kind of design.
 
As far as enabling solo mining, what if this concept were used off chain? Have a public network of solo miners who publish "weak blocks" to that network, and the next 100 (or 1000 etc) nice miners pay you out as long as you're also being nice by following the protocol? All the nice optimizations you mentioned about eg combined taproot payouts would apply i think. The only goals this wouldn't satisfy are 3 and 5 since an extra network is needed, but to be fair, your proposal requires pools which all need their own extra network anyways. 
 
The missing piece here would be an ordering of weak blocks to make the window possible. Or at least a way to determine what blocks should definitely be part of a particular block's pay out. I could see this being done by a separate ephemeral blockchain (which starts fresh after each Bitcoin block) that keeps track of which weak blocks have been submitted, potentially using the pow already in each block to secure it. Granted that piece is a bit half baked, but it seems quite solvable. Wdyt?
 
One thing that jumped out at me as not safe is throwing block rewards into a channel and being able to spend them immediately. There's a reason block rewards aren't spendable for a while, and channels don't solve that problem, do they? Why not simply reduce the on chain wait time for spending block rewards at that point? Seems like the consequences would be the same.
On Tue, Dec 14, 2021, 16:12 Bob McElrath via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
You are hand waving. Attempting to redefine terms to justify your argument is
intellectually dishonest. Bitcoin pools have *always* been about variance
reduction. Your window function fundamentally CANNOT be used to hedge hashrate.
Various suggestions below introduce dangerous new games that might be played by
miners.
The fact is that the half-baked design you posted is less than useless, and
doesn't do anything that anyone wants.
You are trying to justify CTV by making it be all things to all people. "When
all you have is a hammer, every problem looks like a nail".  Instead I humbly
suggest that you pick ONE problem for which CTV is demonstrably the right and
best solution, instead of snowing us with a ton of half-baked things that
*could* be done, and often don't even require CTV, and some (like this one)
fundamentally don't work. I do like some of your ideas, but if you had to pick
just one "use case", which would it be?
Jeremy [jlrubin@mit.edu] wrote:
> Bitcoin didn't invent the concept of pooling: https://en.wikipedia.org/wiki/
> Pooling_(resource_management). This is a Bitcoin Mining Pool, although it may
> not be your favorite kind, which is fixated on specific properties of computing
> contributions before finding a block. Pooling is just a general technique for
> aggregating resources to accomplish something. If you have another name like
> pooling that is in common use for this type of activity I would be more than
> happy to adopt it.
>
> This sort of pool can hedge not only against fee rates but also against
> increases in hashrate since your historical rate 'carries' into the future as a
> function of the window. Further, windows and reward functions can be defined in
> a myriad of ways that could, e.g., pay less to blocks found in more rapid
> succession, contributing to the smoothing functionality.
>
> With respect to sub-block pooling, as described in the article, this sort of
> design also helps with micro-pools being able to split resources
> non-custodially in every block as a part of the higher order DCFMP. The point
> is not, as noted, to enable solo mining an S9, but to decrease the size of the
> minimum viable pool. It's also possible to add, without much validation or
> data, some 'uncle block' type mechanism in an incentive compatible way (e.g.,
> add 10 pow-heavy headers on the last block for cost 48 bytes header + 32 bytes
> payout key) such that there's an incentive to include the heaviest ones you've
> seen, not just your own, that are worth further study and consideration
> (particularly because it's non-consensus, only for opt-in participation in the
> pool).
>
> With respect to space usage, it seems you wholly reject the viability of a
> payment pool mechanism to cut-through chain space. Is this a critique that
> holds for all Payment Pools, or just in the context of mining? Is there a
> particular reason why you think it infeasible that "strongly online"
> counterparties would be able to coordinate more efficiently? Is it preferable
> for miners, the nexus of decentralization for Bitcoin, to prefer to use
> custodial services for pooling (which may require KYC/AM) over bearing a cost
> of some extra potential chainload?
>
> Lastly, with respect to complexity, the proposal is actually incredibly simple
> when you take it in a broader context. Non Interactive Channels and Payment
> Pools are useful by themselves, so are the operations to merge them and swap
> balance across them. Therefore most of the complexity in this proposal is
> relying on tools we'll likely see in everyday use in any case, DCFMP or no.
>
> Jeremy
> !DSPAM:61b8f2f5321461582627336!
--
Cheers, Bob McElrath
"For every complex problem, there is a solution that is simple, neat, and wrong."
    -- H. L. Mencken
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 19249 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Billy Tetrud]

[-- Attachment #1: Type: text/plain, Size: 17616 bytes --]

> you can assign some minimal difficulty,

I was assuming that would be part of the plan.

> Signing sounds more like Proof of Stake

Associating signing with proof of stake and thereby concluding that signing
is something to avoid sounds like superstitious thinking. A signing scheme
with proof of work would clearly not be proof of stake. I would suggest not
dismissing a design out of hand like that. The benefit of that over merge
mining would be no extra on chain foot print. What do you think the
downsides might be?

> if you use just hashes, then they could be random.

You're right. Nodes would of course need to validate the Bitcoin block
headers being included, so i concede hashing them doesn't gain you anything.


On Thu, Dec 16, 2021, 22:37 <vjudeu@gazeta.pl> wrote:

> > I was thinking that this would be a separate blockchain with separate
> headers that progress linearly like a normal blockchain.
>
> Exactly, that's what I called "superblocks", where you have a separate
> chain, just to keep block headers instead of transactions.
>
> > A block creator would collect together as many blocks that haven't been
> collected yet into the next superblock (and maybe receive a reward
> proportional to how many / how much weight they include).
>
> You cannot "catch them all". If you do, you can end up with a lot of block
> headers, where each of them has difficulty equal to one. You need some
> limit, you can limit amount of blocks, you can assign some minimal
> difficulty, it does not matter that much, but some limit is needed, also
> because mining on top of the latest superblock should be more profitable
> than replacing someone else's reward in the previous superblock by your own
> reward and getting a bigger share in the previous superblock.
>
> > This could be done using merge mining, or it could be done using a
> signing scheme (eg where the block creator signs to say "I created this
> superblock" and have mechanisms to punish those who sign multiple
> superblocks at the same height.
>
> I would pick merge mining, because it is more compatible with existing
> mining scheme. Signing sounds more like Proof of Stake and I am trying to
> avoid that solution. Also, there is no need to sign anything, because you
> are solo mining where you have your own coinbase transaction or you are
> mining in a pool, where you have some shared address, and then you cannot
> produce any incompatible superblock, because the protocol can tell you,
> which address you should use (and if it is N-of-N taproot multisig and you
> have some closing transaction, then you can safely mine it).
>
> > Really, you could even just use hashes of the block headers.
>
> Replacing transactions with block headers will do the same trick. Each
> transaction is first hashed with double SHA-256, in exactly the same way as
> block headers are. If you replace transactions with block headers, you
> would get a superblock header, then varint saying how many block headers
> are there, and then you can place all block headers. During superblock
> merkle tree construction, you will hash all block headers (so you will get
> block hashes as leaves), and then you will combine block hashes in the same
> way as transaction hashes are combined.
>
> From the Script point of view, you can always use "OP_SIZE 80
> OP_EQUALVERIFY OP_HASH256 <hash> OP_EQUAL". Then, you can just change the
> size, just to show which object is hashed. Value 80 will work for block
> headers, small values below 520 will work for small transactions, value 64
> will work for any merkle tree proof, no matter if it is for superblock or
> normal block. Also, by using block headers instead of hashes, you can prove
> that at least a proper amount of work was done to produce it, because if
> you use just hashes, then they could be random.
>
> On 2021-12-16 17:57:23 user Billy Tetrud <billy.tetrud@gmail.com> wrote:
>
> @Jeremy
> >   for top-level pool participants there is never any central custody.
>
> I definitely see that. That was actually what I meant when I said the
> goals aren't the same as benefits. While your idea definitely satisfies all
> your goals in a modular way, the fact that it relies on pools means that
> unless the pools can also satisfy the goals, the total system also doesn't
> satisfy the goals (even tho the piece of that system you designed does).
>
> > Thus it doesn't "hurt" anyone except for the miners who are taking the
> not fully locked in funds risk
>
> True, it only potentially hurts whoever the channel partner is accepting
> the unspendable coins. And no one can really stop anyone from taking that
> risk if they really want to. But in that case, its not exactly a fully
> functional channel, since recourse mechanisms couldn't be performed.
> Wouldn't that open such a channel up to a pretty bad theft possibility?
>
> @Bob
> > Increased payout regularity does not lower the viable size of mining
> pools, because smaller mining pools using this mechanism still have higher
> variance.
>
> Yes, smaller mining pools will always have higher variance. However, lower
> variance has diminishing benefits. Below a certain amount of variance, less
> variance isn't very valuable. So increased payout regularity does indeed
> lower the viable size of mining pools because a given low-enough level of
> variance can be achieved with less pool hashpower.
>
> > The on-chain footprint is *higher* due to the increased payout
> regularity.
>
> That's a reasonable point. However, I think there is a difference here
> between the regularity of rewards vs payouts. Rewards for each miner can be
> more regular without necessarily increasing the number of on-chain payouts.
> In fact, theoretically, an individual miner could let their rewards
> accumulate in a pool over many rewards and only redeem when they need the
> coins for something. The incentive is there for each miner to be judicious
> on how much onchain space they take up.
>
> @vjudeu
>
> > how many block headers should be stored per one "superblock"?
>
> I was thinking that this would be a separate blockchain with separate
> headers that progress linearly like a normal blockchain. A block creator
> would collect together as many blocks that haven't been collected yet into
> the next superblock (and maybe receive a reward proportional to how many /
> how much weight they include). This could be done using merge mining, or it
> could be done using a signing scheme (eg where the block creator signs to
> say "I created this superblock" and have mechanisms to punish those who
> sign multiple superblocks at the same height. For merge mining, I could
> even imagine the data necessary to validate that it has been merge mined
> could be put into a taproot script branch (creating an invalid script, but
> a valid hash of the superblock).
>
> > we can collect all headers with the same previous block hash, and
> distribute block reward between all coinbase transactions in those headers
>
> Exactly.
>
> > we would just have block headers instead of transactions
>
> Yeah, I think that would be the way to go. Really, you could even just use
> hashes of the block headers. But the size doesn't matter much because it
> would be both a small blockchain and an ephemeral one (which can be fully
> discarded after all parties have been paid out, or at least their payout
> has been committed to on the bitcoin blockchain).
>
> On Thu, Dec 16, 2021 at 1:35 AM <vjudeu@gazeta.pl
> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchEuGRE%2BJkYAEBM5BgkBElIaQgpBQUFBVCVGX18dBRtTEQcBF1UyQUoDEQ0TRQYNQUdjI1hCU0cyajZIblhVZRQcVlEe>>
> wrote:
>
>> > The missing piece here would be an ordering of weak blocks to make the
>> window possible. Or at least a way to determine what blocks should
>> definitely be part of a particular block's pay out. I could see this being
>> done by a separate ephemeral blockchain (which starts fresh after each
>> Bitcoin block) that keeps track of which weak blocks have been submitted,
>> potentially using the pow already in each block to secure it. Granted that
>> piece is a bit half baked, but it seems quite solvable. Wdyt?
>>
>> I thought about something like that, but there is one problem: how many
>> block headers should be stored per one "superblock"? Currently, we have
>> single block header, where the whole coinbase transaction is taken by some
>> mining pool or solo miner. But instead, each miner could submit its own
>> block header. Then, we can collect all headers with the same previous block
>> hash, and distribute block reward between all coinbase transactions in
>> those headers. One "superblock" then would be created in a similar way as
>> existing blocks, we would just have block headers instead of transactions.
>> If most transactions inside those blocks will be the same, then each block
>> could be expressed just as a set of transaction hashes, only coinbase
>> transactions or custom, non-broadcasted transactions included by miners
>> will be revealed, everything else will be known.
>>
>> > One thing that jumped out at me as not safe is throwing block rewards
>> into a channel and being able to spend them immediately. There's a reason
>> block rewards aren't spendable for a while, and channels don't solve that
>> problem, do they? Why not simply reduce the on chain wait time for spending
>> block rewards at that point? Seems like the consequences would be the same.
>>
>> All coinbase rewards are unspendable for 100 blocks, it is enforced by
>> consensus. It does not matter if there are outputs owned directly by
>> miners, or if there is one huge N-of-N taproot multisig for the whole pool,
>> where every miner signed the closing transaction. The only option to take
>> coins faster I can see is swapping the coins by some LN transaction. But
>> then, the other party can check if some deposit to the LN channel is a part
>> of the coinbase transaction or not, and then decide if it is acceptable to
>> do the swap.
>>
>> On 2021-12-15 19:00:44 user Billy Tetrud via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org
>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchE6GhA5LFpLCUc7EVZQVl9dQRIXXR8NCBMbCwIGChJXQFxcXEgcFh8UVVVDEyBdVkE9JVRdEwFhYXVlblhVIkosEAszLR5BQVV7U0MID0BAQUgIGh0RHgAMGAMXBQJfW1sdXRQUQUoDQlAiBFY8>>
>> wrote:
>>
>> Looks like an interesting proposal, but it doesn't seem to quite match
>> the goals you mentioned. As you do mention, this mining pool coordination
>> doesn't get rid of the need for mining pools in the first place. So it
>> doesn't satisfy item 1 on your goal list afaict.
>>
>> The primary benefits over what we have today that I can see are:
>> 1. increased payout regularity, which lowers the viable size of mining
>> pools, and
>> 2. Lower on chain footprint through combining pay outs from multiple
>> pools.
>>
>> Am I missing some?
>>
>> These are interesting benefits, but it would be nice if your post was
>> clearer on that, since the goals list is not the same as the list of
>> potential benefits of this kind of design.
>>
>> As far as enabling solo mining, what if this concept were used off chain?
>> Have a public network of solo miners who publish "weak blocks" to that
>> network, and the next 100 (or 1000 etc) nice miners pay you out as long as
>> you're also being nice by following the protocol? All the nice
>> optimizations you mentioned about eg combined taproot payouts would apply i
>> think. The only goals this wouldn't satisfy are 3 and 5 since an extra
>> network is needed, but to be fair, your proposal requires pools which all
>> need their own extra network anyways.
>>
>> The missing piece here would be an ordering of weak blocks to make the
>> window possible. Or at least a way to determine what blocks should
>> definitely be part of a particular block's pay out. I could see this being
>> done by a separate ephemeral blockchain (which starts fresh after each
>> Bitcoin block) that keeps track of which weak blocks have been submitted,
>> potentially using the pow already in each block to secure it. Granted that
>> piece is a bit half baked, but it seems quite solvable. Wdyt?
>>
>> One thing that jumped out at me as not safe is throwing block rewards
>> into a channel and being able to spend them immediately. There's a reason
>> block rewards aren't spendable for a while, and channels don't solve that
>> problem, do they? Why not simply reduce the on chain wait time for spending
>> block rewards at that point? Seems like the consequences would be the same.
>>
>> On Tue, Dec 14, 2021, 16:12 Bob McElrath via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org
>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchE6GhA5LFpLCUc7EVZQVl9dQRIXXR8NCBMbCwIGChJXQFxcXEgcFh8UVVVDEyBdVkE9JVRdEwFhYXVlblhVIkosEAszLR5BQVV7U0MID0BAQUgIGh0RHgAMGAMXBQJfW1sdXRQUQUoDQlAiBFY8>>
>> wrote:
>>
>>> You are hand waving. Attempting to redefine terms to justify your
>>> argument is
>>> intellectually dishonest. Bitcoin pools have *always* been about variance
>>> reduction. Your window function fundamentally CANNOT be used to hedge
>>> hashrate.
>>> Various suggestions below introduce dangerous new games that might be
>>> played by
>>> miners.
>>>
>>> The fact is that the half-baked design you posted is less than useless,
>>> and
>>> doesn't do anything that anyone wants.
>>>
>>> You are trying to justify CTV by making it be all things to all people.
>>> "When
>>> all you have is a hammer, every problem looks like a nail".  Instead I
>>> humbly
>>> suggest that you pick ONE problem for which CTV is demonstrably the
>>> right and
>>> best solution, instead of snowing us with a ton of half-baked things that
>>> *could* be done, and often don't even require CTV, and some (like this
>>> one)
>>> fundamentally don't work. I do like some of your ideas, but if you had
>>> to pick
>>> just one "use case", which would it be?
>>>
>>> Jeremy [jlrubin@mit.edu
>>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchEyHxYvIVpLARduChoQSFZQR0NWQVZWJUNRXwMSCRMTBgcWASdWVkpbCxUTQwoWQUdjKVBMGFY3MWMWeU9QBAZtNw%3D%3D>]
>>> wrote:
>>> > Bitcoin didn't invent the concept of pooling:
>>> https://en.wikipedia.org/wiki/
>>> > Pooling_(resource_management). This is a Bitcoin Mining Pool, although
>>> it may
>>> > not be your favorite kind, which is fixated on specific properties of
>>> computing
>>> > contributions before finding a block. Pooling is just a general
>>> technique for
>>> > aggregating resources to accomplish something. If you have another
>>> name like
>>> > pooling that is in common use for this type of activity I would be
>>> more than
>>> > happy to adopt it.
>>> >
>>> > This sort of pool can hedge not only against fee rates but also against
>>> > increases in hashrate since your historical rate 'carries' into the
>>> future as a
>>> > function of the window. Further, windows and reward functions can be
>>> defined in
>>> > a myriad of ways that could, e.g., pay less to blocks found in more
>>> rapid
>>> > succession, contributing to the smoothing functionality.
>>> >
>>> > With respect to sub-block pooling, as described in the article, this
>>> sort of
>>> > design also helps with micro-pools being able to split resources
>>> > non-custodially in every block as a part of the higher order DCFMP.
>>> The point
>>> > is not, as noted, to enable solo mining an S9, but to decrease the
>>> size of the
>>> > minimum viable pool. It's also possible to add, without much
>>> validation or
>>> > data, some 'uncle block' type mechanism in an incentive compatible way
>>> (e.g.,
>>> > add 10 pow-heavy headers on the last block for cost 48 bytes header +
>>> 32 bytes
>>> > payout key) such that there's an incentive to include the heaviest
>>> ones you've
>>> > seen, not just your own, that are worth further study and consideration
>>> > (particularly because it's non-consensus, only for opt-in
>>> participation in the
>>> > pool).
>>> >
>>> > With respect to space usage, it seems you wholly reject the viability
>>> of a
>>> > payment pool mechanism to cut-through chain space. Is this a critique
>>> that
>>> > holds for all Payment Pools, or just in the context of mining? Is
>>> there a
>>> > particular reason why you think it infeasible that "strongly online"
>>> > counterparties would be able to coordinate more efficiently? Is it
>>> preferable
>>> > for miners, the nexus of decentralization for Bitcoin, to prefer to use
>>> > custodial services for pooling (which may require KYC/AM) over bearing
>>> a cost
>>> > of some extra potential chainload?
>>> >
>>> > Lastly, with respect to complexity, the proposal is actually
>>> incredibly simple
>>> > when you take it in a broader context. Non Interactive Channels and
>>> Payment
>>> > Pools are useful by themselves, so are the operations to merge them
>>> and swap
>>> > balance across them. Therefore most of the complexity in this proposal
>>> is
>>> > relying on tools we'll likely see in everyday use in any case, DCFMP
>>> or no.
>>> >
>>> > Jeremy
>>> > !DSPAM:61b8f2f5321461582627336!
>>> --
>>> Cheers, Bob McElrath
>>>
>>> "For every complex problem, there is a solution that is simple, neat,
>>> and wrong."
>>>     -- H. L. Mencken
>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> <http://../NowaWiadomosc/Do/QlIkBFQ6QUFhIVRZX192dnQBeCtCchE6GhA5LFpLCUc7EVZQVl9dQRIXXR8NCBMbCwIGChJXQFxcXEgcFh8UVVVDEyBdVkE9JVRdEwFhYXVlblhVIkosEAszLR5BQVV7U0MID0BAQUgIGh0RHgAMGAMXBQJfW1sdXRQUQUoDQlAiBFY8>
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>

[-- Attachment #2: Type: text/html, Size: 20638 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[vjudeu]

[-- Attachment #1: Type: text/plain, Size: 17887 bytes --]

> Associating signing with proof of stake and thereby concluding that signing is something to avoid sounds like superstitious thinking.
If you introduce signing into mining, then you will have cases, where someone is powerful enough to produce blocks, but cannot, because signing is needed. Then, your consensus is no longer "the heaviest chain", but "the heaviest signed chain". That means, your computing power is no longer enough by itself (as today), because to make a block, you also need some kind of "permission to mine", because first you sign things (like in signet) and then you mine them. That kind of being "reliably unreliable" may be ok for testing, but not for the main network.
> A signing scheme with proof of work would clearly not be proof of stake.
It is not that far from Proof of Stake as you may think. In Proof of Stake, you sign your whale balance with thousands of BTC, and then you can mine a new block, just by putting your coins at stake. In Proof of Work with signing, you also make a signature, and even if there is no amount in satoshis for your signing key, it is still somehow checked, who can sign a block and how many signatures are needed. So, is your private key that allows you to extend the chain is really worth zero satoshis, because there is no balance connected with your address/public key? Not really, it is worth a lot of coins, because it can be used to control the chain.
> The benefit of that over merge mining would be no extra on chain foot print.
Your signatures would be that "extra on chain foot print". In signet, by default you have 1-of-2 multisig, so you have one signature for every block. In the main chain, there is no need for any foot print, because you can reveal taproot public key and hide other things in tapscript, so your foot print is present only when people will not cooperate.
On 2021-12-20 18:18:58 user Billy Tetrud <billy.tetrud@gmail.com> wrote:
> you can assign some minimal difficulty,
 
I was assuming that would be part of the plan.
 
> Signing sounds more like Proof of Stake
 
Associating signing with proof of stake and thereby concluding that signing is something to avoid sounds like superstitious thinking. A signing scheme with proof of work would clearly not be proof of stake. I would suggest not dismissing a design out of hand like that. The benefit of that over merge mining would be no extra on chain foot print. What do you think the downsides might be?
 
> if you use just hashes, then they could be random.
 
You're right. Nodes would of course need to validate the Bitcoin block headers being included, so i concede hashing them doesn't gain you anything.
On Thu, Dec 16, 2021, 22:37 <vjudeu@gazeta.pl> wrote:
> I was thinking that this would be a separate blockchain with separate headers that progress linearly like a normal blockchain.
Exactly, that's what I called "superblocks", where you have a separate chain, just to keep block headers instead of transactions.
> A block creator would collect together as many blocks that haven't been collected yet into the next superblock (and maybe receive a reward proportional to how many / how much weight they include).
You cannot "catch them all". If you do, you can end up with a lot of block headers, where each of them has difficulty equal to one. You need some limit, you can limit amount of blocks, you can assign some minimal difficulty, it does not matter that much, but some limit is needed, also because mining on top of the latest superblock should be more profitable than replacing someone else's reward in the previous superblock by your own reward and getting a bigger share in the previous superblock.
> This could be done using merge mining, or it could be done using a signing scheme (eg where the block creator signs to say "I created this superblock" and have mechanisms to punish those who sign multiple superblocks at the same height.
I would pick merge mining, because it is more compatible with existing mining scheme. Signing sounds more like Proof of Stake and I am trying to avoid that solution. Also, there is no need to sign anything, because you are solo mining where you have your own coinbase transaction or you are mining in a pool, where you have some shared address, and then you cannot produce any incompatible superblock, because the protocol can tell you, which address you should use (and if it is N-of-N taproot multisig and you have some closing transaction, then you can safely mine it).
> Really, you could even just use hashes of the block headers.
Replacing transactions with block headers will do the same trick. Each transaction is first hashed with double SHA-256, in exactly the same way as block headers are. If you replace transactions with block headers, you would get a superblock header, then varint saying how many block headers are there, and then you can place all block headers. During superblock merkle tree construction, you will hash all block headers (so you will get block hashes as leaves), and then you will combine block hashes in the same way as transaction hashes are combined.
From the Script point of view, you can always use "OP_SIZE 80 OP_EQUALVERIFY OP_HASH256 <hash> OP_EQUAL". Then, you can just change the size, just to show which object is hashed. Value 80 will work for block headers, small values below 520 will work for small transactions, value 64 will work for any merkle tree proof, no matter if it is for superblock or normal block. Also, by using block headers instead of hashes, you can prove that at least a proper amount of work was done to produce it, because if you use just hashes, then they could be random.
On 2021-12-16 17:57:23 user Billy Tetrud <billy.tetrud@gmail.com> wrote:
@Jeremy
>   for top-level pool participants there is never any central custody.
 
I definitely see that. That was actually what I meant when I said the goals aren't the same as benefits. While your idea definitely satisfies all your goals in a modular way, the fact that it relies on pools means that unless the pools can also satisfy the goals, the total system also doesn't satisfy the goals (even tho the piece of that system you designed does). 
 
> Thus it doesn't "hurt" anyone except for the miners who are taking the not fully locked in funds risk
 
True, it only potentially hurts whoever the channel partner is accepting the unspendable coins. And no one can really stop anyone from taking that risk if they really want to. But in that case, its not exactly a fully functional channel, since recourse mechanisms couldn't be performed. Wouldn't that open such a channel up to a pretty bad theft possibility?
 
@Bob
> Increased payout regularity does not lower the viable size of mining pools, because smaller mining pools using this mechanism still have higher variance.
 
Yes, smaller mining pools will always have higher variance. However, lower variance has diminishing benefits. Below a certain amount of variance, less variance isn't very valuable. So increased payout regularity does indeed lower the viable size of mining pools because a given low-enough level of variance can be achieved with less pool hashpower.
 
> The on-chain footprint is *higher* due to the increased payout regularity.
 
That's a reasonable point. However, I think there is a difference here between the regularity of rewards vs payouts. Rewards for each miner can be more regular without necessarily increasing the number of on-chain payouts. In fact, theoretically, an individual miner could let their rewards accumulate in a pool over many rewards and only redeem when they need the coins for something. The incentive is there for each miner to be judicious on how much onchain space they take up.
 
@vjudeu
 
> how many block headers should be stored per one "superblock"?
 
I was thinking that this would be a separate blockchain with separate headers that progress linearly like a normal blockchain. A block creator would collect together as many blocks that haven't been collected yet into the next superblock (and maybe receive a reward proportional to how many / how much weight they include). This could be done using merge mining, or it could be done using a signing scheme (eg where the block creator signs to say "I created this superblock" and have mechanisms to punish those who sign multiple superblocks at the same height. For merge mining, I could even imagine the data necessary to validate that it has been merge mined could be put into a taproot script branch (creating an invalid script, but a valid hash of the superblock). 
 
> we can collect all headers with the same previous block hash, and distribute block reward between all coinbase transactions in those headers
 
Exactly.
 
> we would just have block headers instead of transactions
 
Yeah, I think that would be the way to go. Really, you could even just use hashes of the block headers. But the size doesn't matter much because it would be both a small blockchain and an ephemeral one (which can be fully discarded after all parties have been paid out, or at least their payout has been committed to on the bitcoin blockchain). 
On Thu, Dec 16, 2021 at 1:35 AM <vjudeu@gazeta.pl> wrote:
> The missing piece here would be an ordering of weak blocks to make the window possible. Or at least a way to determine what blocks should definitely be part of a particular block's pay out. I could see this being done by a separate ephemeral blockchain (which starts fresh after each Bitcoin block) that keeps track of which weak blocks have been submitted, potentially using the pow already in each block to secure it. Granted that piece is a bit half baked, but it seems quite solvable. Wdyt?
 
I thought about something like that, but there is one problem: how many block headers should be stored per one "superblock"? Currently, we have single block header, where the whole coinbase transaction is taken by some mining pool or solo miner. But instead, each miner could submit its own block header. Then, we can collect all headers with the same previous block hash, and distribute block reward between all coinbase transactions in those headers. One "superblock" then would be created in a similar way as existing blocks, we would just have block headers instead of transactions. If most transactions inside those blocks will be the same, then each block could be expressed just as a set of transaction hashes, only coinbase transactions or custom, non-broadcasted transactions included by miners will be revealed, everything else will be known.
> One thing that jumped out at me as not safe is throwing block rewards into a channel and being able to spend them immediately. There's a reason block rewards aren't spendable for a while, and channels don't solve that problem, do they? Why not simply reduce the on chain wait time for spending block rewards at that point? Seems like the consequences would be the same.
All coinbase rewards are unspendable for 100 blocks, it is enforced by consensus. It does not matter if there are outputs owned directly by miners, or if there is one huge N-of-N taproot multisig for the whole pool, where every miner signed the closing transaction. The only option to take coins faster I can see is swapping the coins by some LN transaction. But then, the other party can check if some deposit to the LN channel is a part of the coinbase transaction or not, and then decide if it is acceptable to do the swap.
On 2021-12-15 19:00:44 user Billy Tetrud via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
Looks like an interesting proposal, but it doesn't seem to quite match the goals you mentioned. As you do mention, this mining pool coordination doesn't get rid of the need for mining pools in the first place. So it doesn't satisfy item 1 on your goal list afaict.   
The primary benefits over what we have today that I can see are:
1. increased payout regularity, which lowers the viable size of mining pools, and
2. Lower on chain footprint through combining pay outs from multiple pools.
 
Am I missing some?
 
These are interesting benefits, but it would be nice if your post was clearer on that, since the goals list is not the same as the list of potential benefits of this kind of design.
 
As far as enabling solo mining, what if this concept were used off chain? Have a public network of solo miners who publish "weak blocks" to that network, and the next 100 (or 1000 etc) nice miners pay you out as long as you're also being nice by following the protocol? All the nice optimizations you mentioned about eg combined taproot payouts would apply i think. The only goals this wouldn't satisfy are 3 and 5 since an extra network is needed, but to be fair, your proposal requires pools which all need their own extra network anyways. 
 
The missing piece here would be an ordering of weak blocks to make the window possible. Or at least a way to determine what blocks should definitely be part of a particular block's pay out. I could see this being done by a separate ephemeral blockchain (which starts fresh after each Bitcoin block) that keeps track of which weak blocks have been submitted, potentially using the pow already in each block to secure it. Granted that piece is a bit half baked, but it seems quite solvable. Wdyt?
 
One thing that jumped out at me as not safe is throwing block rewards into a channel and being able to spend them immediately. There's a reason block rewards aren't spendable for a while, and channels don't solve that problem, do they? Why not simply reduce the on chain wait time for spending block rewards at that point? Seems like the consequences would be the same.
On Tue, Dec 14, 2021, 16:12 Bob McElrath via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
You are hand waving. Attempting to redefine terms to justify your argument is
intellectually dishonest. Bitcoin pools have *always* been about variance
reduction. Your window function fundamentally CANNOT be used to hedge hashrate.
Various suggestions below introduce dangerous new games that might be played by
miners.
The fact is that the half-baked design you posted is less than useless, and
doesn't do anything that anyone wants.
You are trying to justify CTV by making it be all things to all people. "When
all you have is a hammer, every problem looks like a nail".  Instead I humbly
suggest that you pick ONE problem for which CTV is demonstrably the right and
best solution, instead of snowing us with a ton of half-baked things that
*could* be done, and often don't even require CTV, and some (like this one)
fundamentally don't work. I do like some of your ideas, but if you had to pick
just one "use case", which would it be?
Jeremy [jlrubin@mit.edu] wrote:
> Bitcoin didn't invent the concept of pooling: https://en.wikipedia.org/wiki/
> Pooling_(resource_management). This is a Bitcoin Mining Pool, although it may
> not be your favorite kind, which is fixated on specific properties of computing
> contributions before finding a block. Pooling is just a general technique for
> aggregating resources to accomplish something. If you have another name like
> pooling that is in common use for this type of activity I would be more than
> happy to adopt it.
>
> This sort of pool can hedge not only against fee rates but also against
> increases in hashrate since your historical rate 'carries' into the future as a
> function of the window. Further, windows and reward functions can be defined in
> a myriad of ways that could, e.g., pay less to blocks found in more rapid
> succession, contributing to the smoothing functionality.
>
> With respect to sub-block pooling, as described in the article, this sort of
> design also helps with micro-pools being able to split resources
> non-custodially in every block as a part of the higher order DCFMP. The point
> is not, as noted, to enable solo mining an S9, but to decrease the size of the
> minimum viable pool. It's also possible to add, without much validation or
> data, some 'uncle block' type mechanism in an incentive compatible way (e.g.,
> add 10 pow-heavy headers on the last block for cost 48 bytes header + 32 bytes
> payout key) such that there's an incentive to include the heaviest ones you've
> seen, not just your own, that are worth further study and consideration
> (particularly because it's non-consensus, only for opt-in participation in the
> pool).
>
> With respect to space usage, it seems you wholly reject the viability of a
> payment pool mechanism to cut-through chain space. Is this a critique that
> holds for all Payment Pools, or just in the context of mining? Is there a
> particular reason why you think it infeasible that "strongly online"
> counterparties would be able to coordinate more efficiently? Is it preferable
> for miners, the nexus of decentralization for Bitcoin, to prefer to use
> custodial services for pooling (which may require KYC/AM) over bearing a cost
> of some extra potential chainload?
>
> Lastly, with respect to complexity, the proposal is actually incredibly simple
> when you take it in a broader context. Non Interactive Channels and Payment
> Pools are useful by themselves, so are the operations to merge them and swap
> balance across them. Therefore most of the complexity in this proposal is
> relying on tools we'll likely see in everyday use in any case, DCFMP or no.
>
> Jeremy
> !DSPAM:61b8f2f5321461582627336!
--
Cheers, Bob McElrath
"For every complex problem, there is a solution that is simple, neat, and wrong."
    -- H. L. Mencken
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 23198 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 868 bytes --]

> If you introduce signing into mining, then you will have cases, where
> someone is powerful enough to produce blocks, but cannot, because signing
> is needed. Then, your consensus is no longer "the heaviest chain", but "the
> heaviest signed chain". That means, your computing power is no longer
> enough by itself (as today), because to make a block, you also need some
> kind of "permission to mine", because first you sign things (like in
> signet) and then you mine them. That kind of being "reliably unreliable"
> may be ok for testing, but not for the main network.


this is a really great point worth underscoring. this is the 'key
ingredient' for DCFMP, which is that there is no signing or other network
system that is 'in the way' of normal bitcoin mining, just an opt-in set of
rules for sharing the bounties of your block in exchange for future shares.

[-- Attachment #2: Type: text/html, Size: 1393 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Billy Tetrud]

[-- Attachment #1: Type: text/plain, Size: 1710 bytes --]

@vjudeu
>  If you introduce signing into mining, then you will have cases, where
someone is powerful enough to produce blocks, but cannot, because signing
is needed..  your consensus is no longer "the heaviest chain"

You've misunderstood my suggestion. This would not be possible with what I
suggested. Why do you think of the signature as some kind of barrier? What
I was suggesting was that, when a miner participating in this protocol
mines a valid bitcoin block, they then sign a superblock with a public key
that can be verified alongside the coinbase output (eg say with data in the
first tapleaf of the output address). The block is still connected to
something secured by PoW. You really made a lot of incorrect assumptions
about what I suggested.

On Thu, Dec 23, 2021 at 1:05 PM Jeremy <jlrubin@mit.edu> wrote:

> If you introduce signing into mining, then you will have cases, where
>> someone is powerful enough to produce blocks, but cannot, because signing
>> is needed. Then, your consensus is no longer "the heaviest chain", but "the
>> heaviest signed chain". That means, your computing power is no longer
>> enough by itself (as today), because to make a block, you also need some
>> kind of "permission to mine", because first you sign things (like in
>> signet) and then you mine them. That kind of being "reliably unreliable"
>> may be ok for testing, but not for the main network.
>
>
> this is a really great point worth underscoring. this is the 'key
> ingredient' for DCFMP, which is that there is no signing or other network
> system that is 'in the way' of normal bitcoin mining, just an opt-in set of
> rules for sharing the bounties of your block in exchange for future shares.
>

[-- Attachment #2: Type: text/html, Size: 2559 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Bob McElrath]

This, quite simply, is not a "pool". A pool is by definition a tool to reduce
profit variance by miners by collecting "weak blocks" that do not meet the
difficulty target, so as to get a better statistical measure of each miner's
hashrate, which is used to subdivide profits. These are called "shares" and are
entirely absent here.

The only available information here to decide payouts is the blocks themselves,
I do not have any higher statistics measurement to subdivide payments. If I
expect to earn 3 blocks within the window, sometimes I will earn 2 and sometimes
I will earn 4. Whether I keep the entire coinbase in those 2-4 blocks, or I have
100 other miners paying me 1/100 as much 100 times, my payment is the same and
must be proportional to the number of blocks I mine in the window.  My variance
is not reduced.

Further, by making miners pay other miners within the window N, this results in
N^2 payments to miners which otherwise would have had N coinbase payments. So,
this is extremely block-space inefficient for no good reason. P2Pool had the
same problem and generated giant coinbases which competed with fee revenue.
"Congestion control" makes this somewhat worse since is it is an absolute
increase in the block space consumed for these N^2 payments.

The only thing this proposal does do is smooth out fee revenue. While hedging on
fee revenue is valuable, this is an extremely complicated and expensive way to
go about it, that simultaneously *reduces* fee revenue due to all the extra
block space used for miner payouts.

Jeremy via bitcoin-dev [bitcoin-dev@lists.linuxfoundation.org] wrote:
> Howdy, welcome to day 15!
> 
> Today's post covers a form of a mining pool that can be operated as sort of a
> map-reduce over blocks without any "infrastructure".
> 
> https://rubin.io/bitcoin/2021/12/12/advent-15/
> 
> There's still some really open-ended questions (perhaps for y'all to consider)
> around how to select an analyze the choice of window and payout functions, but
> something like this could alleviate a lot of the centralization pressures
> typically faced by pools.
> 
> Notably, compared to previous attempts, combining the payment pool payout with
> this concept means that there is practically very little on-chain overhead from
> this approach as the chain-load
> for including payouts in every block is deferred for future cooperation among
> miners. Although that can be considered cooperation itself, if you think of it
> like a pipeline, the cooperation happens out of band from mining and block
> production so it really is coordination free to mine.
> 
> 
> Cheers,
> 
> Jeremy
> 
> --
> @JeremyRubin 
> !DSPAM:61b626be345321821816715!

> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 
> 
> !DSPAM:61b626be345321821816715!

--
Cheers, Bob McElrath

"For every complex problem, there is a solution that is simple, neat, and wrong."
    -- H. L. Mencken 

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[yanmaani]

How does this differ from p2pool?

If you've just re-invented p2pool, shouldn't you credit their prior art?

Monero is doing their implementation of p2pool. They have viable solo 
mining, as far as I understand. The basic idea is you have several 
P2pools. If you have a block time of 10 minutes, p2pool has 20% of 
hashrate, and there's 100 p2pool chains, each chain gets 0.2% of net 
hash. If you're OK with 20s block times (orphans aren't really a big 
problem), you need (20/600) * (0.02/100) = 0.00067% of network hash to 
get a payout every 10m.

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[Jeremy]

[-- Attachment #1: Type: text/plain, Size: 2201 bytes --]

I could add a comparison to p2pool if you want, but bear in mind this is a
blog post designed to introduce a complex topic to a wide audience, not a
literature review of all possible designs and prior art.

In particular, while P2Pool and DCFMP share a goal (decentralize mining),
the approaches to them bear very little similarity as DCFMP is focused on
making the pooling a pure client side validatable function of the existing
chain, and not create a major risk to mining centralization with a reliance
on a new network running on top of Bitcoin. DCFMP also lacks the core value
prop of P2Pool which is higher resolution on share assignment.

Further, DCFMP's core innovations are Payment Pool and non interactive
channel based, something the P2Pool does not have, but could adopt, in
theory, to solve their payout problems[^note]. I still believe that making
a unified layer of networked software all miners are running on top of
Bitcoin in the loop of mining is a major risk and architecturally bad idea,
hence my advocacy for doing such designs as micro pools inside a DCFMP; It
would be possible to make the "micropools" run on a P2Pool like software,
the DCFMP allows for smaller P2Pools to aggregate their hashrate
trustlessly with the main DCFMP shares.



[^note]: for what it's worth, I was not familiar with p2pool very much
before I came up with DCFMP. The lineage of my conceptual work was
determinism, payment pools, and then realizing they could do something for
mining.
--
@JeremyRubin <https://twitter.com/JeremyRubin>
<https://twitter.com/JeremyRubin>


On Wed, Dec 15, 2021 at 1:11 PM <yanmaani@cock.li> wrote:

> How does this differ from p2pool?
>
> If you've just re-invented p2pool, shouldn't you credit their prior art?
>
> Monero is doing their implementation of p2pool. They have viable solo
> mining, as far as I understand. The basic idea is you have several
> P2pools. If you have a block time of 10 minutes, p2pool has 20% of
> hashrate, and there's 100 p2pool chains, each chain gets 0.2% of net
> hash. If you're OK with 20s block times (orphans aren't really a big
> problem), you need (20/600) * (0.02/100) = 0.00067% of network hash to
> get a payout every 10m.
>

[-- Attachment #2: Type: text/html, Size: 3763 bytes --]

Re: [bitcoin-dev] [Bitcoin Advent Calendar] Decentralized Coordination Free Mining Pools

[vjudeu]

[-- Attachment #1: Type: text/plain, Size: 2689 bytes --]

> Can you maybe try stating the goals of your payout function, and then demonstrate how what you're proposing meets that?
 
The goals are quite simple: if you are a solo miner, you are trying to mine a block that meets the network difficulty. If you are using some kind of pool, then you are trying to mine N times easier blocks and receive N times lower reward for doing that. If many miners work on similar transactions, then each miner can validate each transaction once and assign transaction fee to transaction id, in this way the coinbase reward can be quickly checked, because you have to check only those transactions, which were unknown to you and for example included only by this miner and not broadcasted. Assuming that most of the transactions will be the same and included by most of the miners, that verification would be quick and can be simplified only to checking "what is different from what I am mining".
Also, to determine the proper amount of shares received, you can assign a difficulty for each miner. So, if you are connected to eight mining nodes, you can assign a difficulty to each of them, just to limit how much work for each share they can produce to have it accepted and included for payments. It is needed to avoid spamming by producing a lot of shares at difficulty one by bigger miners, they should find it more profitable to create bigger shares, because by accumulating them, it is cheaper to receive one bigger payment than a lot of smaller payments.
On 2021-12-13 14:59:58 user Jeremy <jlrubin@mit.edu> wrote:
Hey there!
 
Thanks for your response!
 
One of the reasons to pick a longer window of, say, a couple difficulty periods would be that you can make participation in the pool hedge you against hashrate changes.
 
You're absolutely spot on to think about the impact of pooling w.r.t. variance when fees > subsidy. That's not really in the analysis I had in the (old) post, but when the block revenues swing, dcfmp over longer periods can really smooth out the revenues for miners in a great way. This can also help with the "mind the gap" problem when there isn't a backlog of transactions, since producing an empty block still has some value (in order to incentivize mining transaction at all and not cheating, we need to reward txn inclusion as I think you're trying to point out.
 
Sadly, I've read the rest of your email a couple times and I don't really get what you're proposing at all. It jumps right into "things you could compute". Can you maybe try stating the goals of your payout function, and then demonstrate how what you're proposing meets that? E.g., we want to pay more to miners that do x?
 
 
 
 

[-- Attachment #2: Type: text/html, Size: 5340 bytes --]