[bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[Daniele Pinna]

[-- Attachment #1: Type: text/plain, Size: 1596 bytes --]

I'd like to submit this paper to the dev-list which analyzes how miner
advantages scale with network and mempool properties in a scenario of
uncapped block sizes. The work proceeds, in a sense, from where Peter R's
work left off correcting a mistake and addressing the critiques made by the
community to his work.

The main result of the work is a detailed analysis of mining advantages
(defined as the added profit per unit of hash) as a function of miner
hashrate. In it, I show how large block subsidies (or better, low mempool
fees-to-subsidy ratios) incentivize the pooling of large hashrates due to
the steady increasing of marginal profits as hashrates grow.

The paper also shows that part of the large advantage the large miners have
today is due to there being a barrier to entry into a high-efficiency
mining class which has access to expected profits an order of magnitude
larger than everyone else. As block subsidies decrease, this
high-efficiency class is expected to vanish leading to a marginal profit
structure which decreases as a function of hashrate.

This work has vacuumed my entire life for the past two weeks leading me to
lag behind on a lot of work. I apologize for typos which I may not have
seen. I stand by for any comments the community may have and look forward
to reigniting consideration of a block size scaling proposal (BIP101)
which, due to the XT fork drama, I believe has been placed hastily and
undeservedly on the chopping block.

https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets

Regards,
Daniele

[-- Attachment #2: Type: text/html, Size: 1869 bytes --]

Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[Jorge Timón]

[-- Attachment #1: Type: text/plain, Size: 1300 bytes --]

On Aug 29, 2015 9:43 AM, "Daniele Pinna via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> This work has vacuumed my entire life for the past two weeks leading me
to lag behind on a lot of work. I apologize for typos which I may not have
seen. I stand by for any comments the community may have and look forward
to reigniting consideration of a block size scaling proposal (BIP101)
which, due to the XT fork drama, I believe has been placed hastily and
undeservedly on the chopping block.

I don't like relying on exponential growth (that's why I don't like neither
Gavin's 101 nor Pieter's 103).

But I don't think it's too late to turn bip101 into just another proposal
for an uncontroversial hardfork (changing the 75% to 95% would be the first
step) and xt into just another software fork.

My favorite one so far is bip102 (even though I still consider "2mb now"
arbitrary and I'm worried about making mining centralization even worse
than it is now), but if it was framed as a schism hardfork like bip101 I
would also warn about the dangers of a schism hardfork for it.

>
https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets

I'll read it to try to understand your claims. Are you presentung this in
the scaling workshop?

[-- Attachment #2: Type: text/html, Size: 1669 bytes --]

Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[Matt Corallo]

I believe it was pointed out previously in the discussion of the Peter R
paper, but I'll repeat it here so that its visible - this seems to
ignore the effects of transaction validation caches and block
compression protocols. Many large miners already have their own network
to relay blocks around the globe with only a few bytes on the wire at
block-time, and there is also the bitcoinrelaynetwork.org network, which
does the same for smaller miners, albeit with slightly less efficiency.
Also, transaction validation time upon receiving a block can be rather
easily made negligible (ie the only validation time you should have is
the DB modify-utxo-set time). Thus, the increased orphan risk for
including a transaction can be reduced to a very, very tiny amount,
making the optimal blocksize, essentially, including everything that
you're confident is in the mempool of other reasonably large miners.

Matt

On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote:
> I'd like to submit this paper to the dev-list which analyzes how miner
> advantages scale with network and mempool properties in a scenario of
> uncapped block sizes. The work proceeds, in a sense, from where Peter
> R's work left off correcting a mistake and addressing the critiques made
> by the community to his work.
> 
> The main result of the work is a detailed analysis of mining advantages
> (defined as the added profit per unit of hash) as a function of miner
> hashrate. In it, I show how large block subsidies (or better, low
> mempool fees-to-subsidy ratios) incentivize the pooling of large
> hashrates due to the steady increasing of marginal profits as hashrates
> grow. 
> 
> The paper also shows that part of the large advantage the large miners
> have today is due to there being a barrier to entry into a
> high-efficiency mining class which has access to expected profits an
> order of magnitude larger than everyone else. As block subsidies
> decrease, this high-efficiency class is expected to vanish leading to a
> marginal profit structure which decreases as a function of hashrate.
> 
> This work has vacuumed my entire life for the past two weeks leading me
> to lag behind on a lot of work. I apologize for typos which I may not
> have seen. I stand by for any comments the community may have and look
> forward to reigniting consideration of a block size scaling proposal
> (BIP101) which, due to the XT fork drama, I believe has been placed
> hastily and undeservedly on the chopping block.
> 
> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets
> 
> 
> Regards,
> Daniele
> 
> 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 

Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[Peter R]

[-- Attachment #1: Type: text/plain, Size: 4804 bytes --]

Hello Matt and Daniele,

>  this seems to ignore the effects of transaction validation caches and block
> compression protocols. 

The effect of block compression protocols is included.  This is what I call the "coding gain" and use the Greek letter "gamma" to represent. 

As long as the block solution announcements contain information (i.e., Shannon Entropy) about the transactions included in a block, then the fee market will be "healthy" according to the definitions given in the linked paper (see below).  This is the case right now, this is the case with your relay network, and this would be the case using any implementation of IBLTs that I can imagine, so long as miners can still construct blocks according to their own volition.  The "healthy fee market" result follows from the Shannon-Hartley theorem; the SH-theorem describes the maximum rate at which information (Shannon Entropy) can be transmitted over a physical communication channel.   

 https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf

I've exchanged emails with Greg Maxwell about (what IMO is) an academic scenario where the block solutions announcements contain no information at all about the transactions included in the blocks.  Although the fee market would not be healthy in such a scenario, it is my feeling that this also requires miners to relinquish their ability to construct blocks according to their own volition (i.e., the system would already be centralized).  I look forward to a white paper demonstrating otherwise!

Best regards,
Peter



On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:

> I believe it was pointed out previously in the discussion of the Peter R
> paper, but I'll repeat it here so that its visible - this seems to
> ignore the effects of transaction validation caches and block
> compression protocols. Many large miners already have their own network
> to relay blocks around the globe with only a few bytes on the wire at
> block-time, and there is also the bitcoinrelaynetwork.org network, which
> does the same for smaller miners, albeit with slightly less efficiency.
> Also, transaction validation time upon receiving a block can be rather
> easily made negligible (ie the only validation time you should have is
> the DB modify-utxo-set time). Thus, the increased orphan risk for
> including a transaction can be reduced to a very, very tiny amount,
> making the optimal blocksize, essentially, including everything that
> you're confident is in the mempool of other reasonably large miners.
> 
> Matt
> 
> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote:
>> I'd like to submit this paper to the dev-list which analyzes how miner
>> advantages scale with network and mempool properties in a scenario of
>> uncapped block sizes. The work proceeds, in a sense, from where Peter
>> R's work left off correcting a mistake and addressing the critiques made
>> by the community to his work.
>> 
>> The main result of the work is a detailed analysis of mining advantages
>> (defined as the added profit per unit of hash) as a function of miner
>> hashrate. In it, I show how large block subsidies (or better, low
>> mempool fees-to-subsidy ratios) incentivize the pooling of large
>> hashrates due to the steady increasing of marginal profits as hashrates
>> grow. 
>> 
>> The paper also shows that part of the large advantage the large miners
>> have today is due to there being a barrier to entry into a
>> high-efficiency mining class which has access to expected profits an
>> order of magnitude larger than everyone else. As block subsidies
>> decrease, this high-efficiency class is expected to vanish leading to a
>> marginal profit structure which decreases as a function of hashrate.
>> 
>> This work has vacuumed my entire life for the past two weeks leading me
>> to lag behind on a lot of work. I apologize for typos which I may not
>> have seen. I stand by for any comments the community may have and look
>> forward to reigniting consideration of a block size scaling proposal
>> (BIP101) which, due to the XT fork drama, I believe has been placed
>> hastily and undeservedly on the chopping block.
>> 
>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets
>> 
>> 
>> Regards,
>> Daniele
>> 
>> 
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


[-- Attachment #2: Type: text/html, Size: 5729 bytes --]

Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[Matt Corallo]

It is not a purely academic scenario that blocks contain effectively no
information (that was not previously relayed). I'm not aware of any
public code to do so, but I know several large miners who pre-relay the
block(s) they are working on to other nodes of theirs around the globe.
This means at announce-time you have only a few bytes to broadcast (way
less than a packet, and effects of using smaller packets to relay things
vs larger packets are very small, if anything). After you've broadcast
to all of your nodes, hops to other mining nodes are probably only a
handful of ms away with very low packet loss, so relay time is no longer
connected to transaction inclusion at all (unless you're talking about
multi-GB blocks). Of course, this is relay time for large miners who can
invest time and money to build such systems. Small miners are completely
screwed in such a system.

Thus, the orphan risk for including a transaction is related to the
validation time (which is only DB modify-utxo-set time, essentially,
which maybe you can optimize much of that away, too, and only have to
pass over mempool or so). Anyway, my point, really, is that though
miners will have an incentive to not include transactions which will
trigger validation by other nodes (ie things not already in their
mempool), the incentive to not include transactions which have already
been relayed around sufficiently is, while not theoretically zero, as
near to zero in practice as you can get.

Matt

On 08/29/15 23:17, Peter R wrote:
> Hello Matt and Daniele,
> 
>>  this seems to ignore the effects of transaction validation caches and
>> *block
>> compression protocols. *
> 
> The effect of block compression protocols is included.  This is what I
> call the "coding gain" and use the Greek letter "gamma" to represent. 
> 
> As long as the block solution announcements contain information (i.e.,
> Shannon Entropy) about the transactions included in a block, then the
> fee market will be "healthy" according to the definitions given in the
> linked paper (see below).  This is the case right now, this is the case
> with your relay network, and this would be the case using any
> implementation of IBLTs that I can imagine, so long as miners can still
> construct blocks according to their own volition.  The "healthy fee
> market" result follows from the Shannon-Hartley theorem; the SH-theorem
> describes the maximum rate at which information (Shannon Entropy) can be
> transmitted over a physical communication channel.   
> 
>  https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf
> 
> I've exchanged emails with Greg Maxwell about (what IMO is) an academic
> scenario where the block solutions announcements contain *no information
> at all* about the transactions included in the blocks.  Although the fee
> market would not be healthy in such a scenario, it is my feeling that
> this also requires miners to relinquish their ability to construct
> blocks according to their own volition (i.e., the system would already
> be centralized).  I look forward to a white paper demonstrating otherwise!
> 
> Best regards,
> Peter
> 
> 
> 
> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org
> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
> 
>> I believe it was pointed out previously in the discussion of the Peter R
>> paper, but I'll repeat it here so that its visible - this seems to
>> ignore the effects of transaction validation caches and block
>> compression protocols. Many large miners already have their own network
>> to relay blocks around the globe with only a few bytes on the wire at
>> block-time, and there is also the bitcoinrelaynetwork.org
>> <http://bitcoinrelaynetwork.org> network, which
>> does the same for smaller miners, albeit with slightly less efficiency.
>> Also, transaction validation time upon receiving a block can be rather
>> easily made negligible (ie the only validation time you should have is
>> the DB modify-utxo-set time). Thus, the increased orphan risk for
>> including a transaction can be reduced to a very, very tiny amount,
>> making the optimal blocksize, essentially, including everything that
>> you're confident is in the mempool of other reasonably large miners.
>>
>> Matt
>>
>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote:
>>> I'd like to submit this paper to the dev-list which analyzes how miner
>>> advantages scale with network and mempool properties in a scenario of
>>> uncapped block sizes. The work proceeds, in a sense, from where Peter
>>> R's work left off correcting a mistake and addressing the critiques made
>>> by the community to his work.
>>>
>>> The main result of the work is a detailed analysis of mining advantages
>>> (defined as the added profit per unit of hash) as a function of miner
>>> hashrate. In it, I show how large block subsidies (or better, low
>>> mempool fees-to-subsidy ratios) incentivize the pooling of large
>>> hashrates due to the steady increasing of marginal profits as hashrates
>>> grow.
>>>
>>> The paper also shows that part of the large advantage the large miners
>>> have today is due to there being a barrier to entry into a
>>> high-efficiency mining class which has access to expected profits an
>>> order of magnitude larger than everyone else. As block subsidies
>>> decrease, this high-efficiency class is expected to vanish leading to a
>>> marginal profit structure which decreases as a function of hashrate.
>>>
>>> This work has vacuumed my entire life for the past two weeks leading me
>>> to lag behind on a lot of work. I apologize for typos which I may not
>>> have seen. I stand by for any comments the community may have and look
>>> forward to reigniting consideration of a block size scaling proposal
>>> (BIP101) which, due to the XT fork drama, I believe has been placed
>>> hastily and undeservedly on the chopping block.
>>>
>>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets
>>>
>>>
>>> Regards,
>>> Daniele
>>>
>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> <mailto:bitcoin-dev@lists.linuxfoundation.org>
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 

Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[Matt Corallo]

Of course this assumes the network does not change any as a result of
such a system. But such a system provides strong incentives for the
network to centralize in other ways (put all the mining nodes in one DC
for all miners, etc).

Matt

On 08/30/15 02:33, Matt Corallo via bitcoin-dev wrote:
> It is not a purely academic scenario that blocks contain effectively no
> information (that was not previously relayed). I'm not aware of any
> public code to do so, but I know several large miners who pre-relay the
> block(s) they are working on to other nodes of theirs around the globe.
> This means at announce-time you have only a few bytes to broadcast (way
> less than a packet, and effects of using smaller packets to relay things
> vs larger packets are very small, if anything). After you've broadcast
> to all of your nodes, hops to other mining nodes are probably only a
> handful of ms away with very low packet loss, so relay time is no longer
> connected to transaction inclusion at all (unless you're talking about
> multi-GB blocks). Of course, this is relay time for large miners who can
> invest time and money to build such systems. Small miners are completely
> screwed in such a system.
> 
> Thus, the orphan risk for including a transaction is related to the
> validation time (which is only DB modify-utxo-set time, essentially,
> which maybe you can optimize much of that away, too, and only have to
> pass over mempool or so). Anyway, my point, really, is that though
> miners will have an incentive to not include transactions which will
> trigger validation by other nodes (ie things not already in their
> mempool), the incentive to not include transactions which have already
> been relayed around sufficiently is, while not theoretically zero, as
> near to zero in practice as you can get.
> 
> Matt
> 
> On 08/29/15 23:17, Peter R wrote:
>> Hello Matt and Daniele,
>>
>>>  this seems to ignore the effects of transaction validation caches and
>>> *block
>>> compression protocols. *
>>
>> The effect of block compression protocols is included.  This is what I
>> call the "coding gain" and use the Greek letter "gamma" to represent. 
>>
>> As long as the block solution announcements contain information (i.e.,
>> Shannon Entropy) about the transactions included in a block, then the
>> fee market will be "healthy" according to the definitions given in the
>> linked paper (see below).  This is the case right now, this is the case
>> with your relay network, and this would be the case using any
>> implementation of IBLTs that I can imagine, so long as miners can still
>> construct blocks according to their own volition.  The "healthy fee
>> market" result follows from the Shannon-Hartley theorem; the SH-theorem
>> describes the maximum rate at which information (Shannon Entropy) can be
>> transmitted over a physical communication channel.   
>>
>>  https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf
>>
>> I've exchanged emails with Greg Maxwell about (what IMO is) an academic
>> scenario where the block solutions announcements contain *no information
>> at all* about the transactions included in the blocks.  Although the fee
>> market would not be healthy in such a scenario, it is my feeling that
>> this also requires miners to relinquish their ability to construct
>> blocks according to their own volition (i.e., the system would already
>> be centralized).  I look forward to a white paper demonstrating otherwise!
>>
>> Best regards,
>> Peter
>>
>>
>>
>> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev
>> <bitcoin-dev@lists.linuxfoundation.org
>> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>>
>>> I believe it was pointed out previously in the discussion of the Peter R
>>> paper, but I'll repeat it here so that its visible - this seems to
>>> ignore the effects of transaction validation caches and block
>>> compression protocols. Many large miners already have their own network
>>> to relay blocks around the globe with only a few bytes on the wire at
>>> block-time, and there is also the bitcoinrelaynetwork.org
>>> <http://bitcoinrelaynetwork.org> network, which
>>> does the same for smaller miners, albeit with slightly less efficiency.
>>> Also, transaction validation time upon receiving a block can be rather
>>> easily made negligible (ie the only validation time you should have is
>>> the DB modify-utxo-set time). Thus, the increased orphan risk for
>>> including a transaction can be reduced to a very, very tiny amount,
>>> making the optimal blocksize, essentially, including everything that
>>> you're confident is in the mempool of other reasonably large miners.
>>>
>>> Matt
>>>
>>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote:
>>>> I'd like to submit this paper to the dev-list which analyzes how miner
>>>> advantages scale with network and mempool properties in a scenario of
>>>> uncapped block sizes. The work proceeds, in a sense, from where Peter
>>>> R's work left off correcting a mistake and addressing the critiques made
>>>> by the community to his work.
>>>>
>>>> The main result of the work is a detailed analysis of mining advantages
>>>> (defined as the added profit per unit of hash) as a function of miner
>>>> hashrate. In it, I show how large block subsidies (or better, low
>>>> mempool fees-to-subsidy ratios) incentivize the pooling of large
>>>> hashrates due to the steady increasing of marginal profits as hashrates
>>>> grow.
>>>>
>>>> The paper also shows that part of the large advantage the large miners
>>>> have today is due to there being a barrier to entry into a
>>>> high-efficiency mining class which has access to expected profits an
>>>> order of magnitude larger than everyone else. As block subsidies
>>>> decrease, this high-efficiency class is expected to vanish leading to a
>>>> marginal profit structure which decreases as a function of hashrate.
>>>>
>>>> This work has vacuumed my entire life for the past two weeks leading me
>>>> to lag behind on a lot of work. I apologize for typos which I may not
>>>> have seen. I stand by for any comments the community may have and look
>>>> forward to reigniting consideration of a block size scaling proposal
>>>> (BIP101) which, due to the XT fork drama, I believe has been placed
>>>> hastily and undeservedly on the chopping block.
>>>>
>>>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets
>>>>
>>>>
>>>> Regards,
>>>> Daniele
>>>>
>>>>
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev@lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> <mailto:bitcoin-dev@lists.linuxfoundation.org>
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 

Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[Peter R]

[-- Attachment #1: Type: text/plain, Size: 8086 bytes --]

> Of course this assumes the network does not change any as a result of
> such a system. But such a system provides strong incentives for the
> network to centralize in other ways (put all the mining nodes in one DC
> for all miners, etc).

If all the mining nodes are in one data center, and if all the nodes are programmed to build blocks in essentially the same way, then I would agree that the orphan cost would be negligible!  I will add this as an example of a network configuration where the results of my paper would be less relevant.  

Peter  


On 2015-08-29, at 7:35 PM, Matt Corallo <lf-lists@mattcorallo.com> wrote:

> Of course this assumes the network does not change any as a result of
> such a system. But such a system provides strong incentives for the
> network to centralize in other ways (put all the mining nodes in one DC
> for all miners, etc).
> 
> Matt
> 
> On 08/30/15 02:33, Matt Corallo via bitcoin-dev wrote:
>> It is not a purely academic scenario that blocks contain effectively no
>> information (that was not previously relayed). I'm not aware of any
>> public code to do so, but I know several large miners who pre-relay the
>> block(s) they are working on to other nodes of theirs around the globe.
>> This means at announce-time you have only a few bytes to broadcast (way
>> less than a packet, and effects of using smaller packets to relay things
>> vs larger packets are very small, if anything). After you've broadcast
>> to all of your nodes, hops to other mining nodes are probably only a
>> handful of ms away with very low packet loss, so relay time is no longer
>> connected to transaction inclusion at all (unless you're talking about
>> multi-GB blocks). Of course, this is relay time for large miners who can
>> invest time and money to build such systems. Small miners are completely
>> screwed in such a system.
>> 
>> Thus, the orphan risk for including a transaction is related to the
>> validation time (which is only DB modify-utxo-set time, essentially,
>> which maybe you can optimize much of that away, too, and only have to
>> pass over mempool or so). Anyway, my point, really, is that though
>> miners will have an incentive to not include transactions which will
>> trigger validation by other nodes (ie things not already in their
>> mempool), the incentive to not include transactions which have already
>> been relayed around sufficiently is, while not theoretically zero, as
>> near to zero in practice as you can get.
>> 
>> Matt
>> 
>> On 08/29/15 23:17, Peter R wrote:
>>> Hello Matt and Daniele,
>>> 
>>>> this seems to ignore the effects of transaction validation caches and
>>>> *block
>>>> compression protocols. *
>>> 
>>> The effect of block compression protocols is included.  This is what I
>>> call the "coding gain" and use the Greek letter "gamma" to represent. 
>>> 
>>> As long as the block solution announcements contain information (i.e.,
>>> Shannon Entropy) about the transactions included in a block, then the
>>> fee market will be "healthy" according to the definitions given in the
>>> linked paper (see below).  This is the case right now, this is the case
>>> with your relay network, and this would be the case using any
>>> implementation of IBLTs that I can imagine, so long as miners can still
>>> construct blocks according to their own volition.  The "healthy fee
>>> market" result follows from the Shannon-Hartley theorem; the SH-theorem
>>> describes the maximum rate at which information (Shannon Entropy) can be
>>> transmitted over a physical communication channel.   
>>> 
>>> https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf
>>> 
>>> I've exchanged emails with Greg Maxwell about (what IMO is) an academic
>>> scenario where the block solutions announcements contain *no information
>>> at all* about the transactions included in the blocks.  Although the fee
>>> market would not be healthy in such a scenario, it is my feeling that
>>> this also requires miners to relinquish their ability to construct
>>> blocks according to their own volition (i.e., the system would already
>>> be centralized).  I look forward to a white paper demonstrating otherwise!
>>> 
>>> Best regards,
>>> Peter
>>> 
>>> 
>>> 
>>> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev
>>> <bitcoin-dev@lists.linuxfoundation.org
>>> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>>> 
>>>> I believe it was pointed out previously in the discussion of the Peter R
>>>> paper, but I'll repeat it here so that its visible - this seems to
>>>> ignore the effects of transaction validation caches and block
>>>> compression protocols. Many large miners already have their own network
>>>> to relay blocks around the globe with only a few bytes on the wire at
>>>> block-time, and there is also the bitcoinrelaynetwork.org
>>>> <http://bitcoinrelaynetwork.org> network, which
>>>> does the same for smaller miners, albeit with slightly less efficiency.
>>>> Also, transaction validation time upon receiving a block can be rather
>>>> easily made negligible (ie the only validation time you should have is
>>>> the DB modify-utxo-set time). Thus, the increased orphan risk for
>>>> including a transaction can be reduced to a very, very tiny amount,
>>>> making the optimal blocksize, essentially, including everything that
>>>> you're confident is in the mempool of other reasonably large miners.
>>>> 
>>>> Matt
>>>> 
>>>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote:
>>>>> I'd like to submit this paper to the dev-list which analyzes how miner
>>>>> advantages scale with network and mempool properties in a scenario of
>>>>> uncapped block sizes. The work proceeds, in a sense, from where Peter
>>>>> R's work left off correcting a mistake and addressing the critiques made
>>>>> by the community to his work.
>>>>> 
>>>>> The main result of the work is a detailed analysis of mining advantages
>>>>> (defined as the added profit per unit of hash) as a function of miner
>>>>> hashrate. In it, I show how large block subsidies (or better, low
>>>>> mempool fees-to-subsidy ratios) incentivize the pooling of large
>>>>> hashrates due to the steady increasing of marginal profits as hashrates
>>>>> grow.
>>>>> 
>>>>> The paper also shows that part of the large advantage the large miners
>>>>> have today is due to there being a barrier to entry into a
>>>>> high-efficiency mining class which has access to expected profits an
>>>>> order of magnitude larger than everyone else. As block subsidies
>>>>> decrease, this high-efficiency class is expected to vanish leading to a
>>>>> marginal profit structure which decreases as a function of hashrate.
>>>>> 
>>>>> This work has vacuumed my entire life for the past two weeks leading me
>>>>> to lag behind on a lot of work. I apologize for typos which I may not
>>>>> have seen. I stand by for any comments the community may have and look
>>>>> forward to reigniting consideration of a block size scaling proposal
>>>>> (BIP101) which, due to the XT fork drama, I believe has been placed
>>>>> hastily and undeservedly on the chopping block.
>>>>> 
>>>>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets
>>>>> 
>>>>> 
>>>>> Regards,
>>>>> Daniele
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> bitcoin-dev mailing list
>>>>> bitcoin-dev@lists.linuxfoundation.org
>>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>> 
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev@lists.linuxfoundation.org
>>>> <mailto:bitcoin-dev@lists.linuxfoundation.org>
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>> 
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> 


[-- Attachment #2: Type: text/html, Size: 8981 bytes --]

Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[Peter R]

[-- Attachment #1: Type: text/plain, Size: 6874 bytes --]

Hello Matt,

> It is not a purely academic scenario that blocks contain effectively no
> information (that was not previously relayed).

The results of my paper hold unless the information about the included transactions is identically zero.  If this information is extremely small, then I agree that something other than the orphan cost would drive the fee market. I don't think this will ever by the case, however.

> I'm not aware of any
> public code to do so, but I know several large miners who pre-relay the
> block(s) they are working on to other nodes of theirs around the globe….

In this paragraph, you are talking about intra-miner communication (you refer to other nodes "of theirs").   I agree that this can be very fast because there's only one entity involved. 

> Thus, the orphan risk for including a transaction is related to the
> validation time … the incentive to not include transactions which have already
> been relayed around sufficiently is, while not theoretically zero, as
> near to zero in practice as you can get.

I look forward to hearing your talk about the Relay Network.  Let's image there was no block size limit.  Using the Relay Network as it currently operates now, how big would the block solution announcement be:

(1) if a miner published a 10 MB block where 100% of the transactions are known by the other nodes?  

(2) if a miner published a 10 MB block where 90% of the transactions are known by the other nodes?  

(3) if a miner published a 100 MB spam block where 0% of the transactions are known by the other nodes?

Best regards,
Peter


> 
> Matt
> 
> On 08/29/15 23:17, Peter R wrote:
>> Hello Matt and Daniele,
>> 
>>> this seems to ignore the effects of transaction validation caches and
>>> *block
>>> compression protocols. *
>> 
>> The effect of block compression protocols is included.  This is what I
>> call the "coding gain" and use the Greek letter "gamma" to represent. 
>> 
>> As long as the block solution announcements contain information (i.e.,
>> Shannon Entropy) about the transactions included in a block, then the
>> fee market will be "healthy" according to the definitions given in the
>> linked paper (see below).  This is the case right now, this is the case
>> with your relay network, and this would be the case using any
>> implementation of IBLTs that I can imagine, so long as miners can still
>> construct blocks according to their own volition.  The "healthy fee
>> market" result follows from the Shannon-Hartley theorem; the SH-theorem
>> describes the maximum rate at which information (Shannon Entropy) can be
>> transmitted over a physical communication channel.   
>> 
>> https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf
>> 
>> I've exchanged emails with Greg Maxwell about (what IMO is) an academic
>> scenario where the block solutions announcements contain *no information
>> at all* about the transactions included in the blocks.  Although the fee
>> market would not be healthy in such a scenario, it is my feeling that
>> this also requires miners to relinquish their ability to construct
>> blocks according to their own volition (i.e., the system would already
>> be centralized).  I look forward to a white paper demonstrating otherwise!
>> 
>> Best regards,
>> Peter
>> 
>> 
>> 
>> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev
>> <bitcoin-dev@lists.linuxfoundation.org
>> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>> 
>>> I believe it was pointed out previously in the discussion of the Peter R
>>> paper, but I'll repeat it here so that its visible - this seems to
>>> ignore the effects of transaction validation caches and block
>>> compression protocols. Many large miners already have their own network
>>> to relay blocks around the globe with only a few bytes on the wire at
>>> block-time, and there is also the bitcoinrelaynetwork.org
>>> <http://bitcoinrelaynetwork.org> network, which
>>> does the same for smaller miners, albeit with slightly less efficiency.
>>> Also, transaction validation time upon receiving a block can be rather
>>> easily made negligible (ie the only validation time you should have is
>>> the DB modify-utxo-set time). Thus, the increased orphan risk for
>>> including a transaction can be reduced to a very, very tiny amount,
>>> making the optimal blocksize, essentially, including everything that
>>> you're confident is in the mempool of other reasonably large miners.
>>> 
>>> Matt
>>> 
>>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote:
>>>> I'd like to submit this paper to the dev-list which analyzes how miner
>>>> advantages scale with network and mempool properties in a scenario of
>>>> uncapped block sizes. The work proceeds, in a sense, from where Peter
>>>> R's work left off correcting a mistake and addressing the critiques made
>>>> by the community to his work.
>>>> 
>>>> The main result of the work is a detailed analysis of mining advantages
>>>> (defined as the added profit per unit of hash) as a function of miner
>>>> hashrate. In it, I show how large block subsidies (or better, low
>>>> mempool fees-to-subsidy ratios) incentivize the pooling of large
>>>> hashrates due to the steady increasing of marginal profits as hashrates
>>>> grow.
>>>> 
>>>> The paper also shows that part of the large advantage the large miners
>>>> have today is due to there being a barrier to entry into a
>>>> high-efficiency mining class which has access to expected profits an
>>>> order of magnitude larger than everyone else. As block subsidies
>>>> decrease, this high-efficiency class is expected to vanish leading to a
>>>> marginal profit structure which decreases as a function of hashrate.
>>>> 
>>>> This work has vacuumed my entire life for the past two weeks leading me
>>>> to lag behind on a lot of work. I apologize for typos which I may not
>>>> have seen. I stand by for any comments the community may have and look
>>>> forward to reigniting consideration of a block size scaling proposal
>>>> (BIP101) which, due to the XT fork drama, I believe has been placed
>>>> hastily and undeservedly on the chopping block.
>>>> 
>>>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets
>>>> 
>>>> 
>>>> Regards,
>>>> Daniele
>>>> 
>>>> 
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev@lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>> 
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> <mailto:bitcoin-dev@lists.linuxfoundation.org>
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> 


[-- Attachment #2: Type: text/html, Size: 8011 bytes --]

Re: [bitcoin-dev] On the Nature of Miner Advantages in Uncapped Block Size Fee Markets

[Matt Corallo]

On 08/30/15 02:49, Peter R wrote:
> Hello Matt,
> 
>> It is not a purely academic scenario that blocks contain *effectively* no
>> information (that was not previously relayed). 
> 
> The results of my paper hold unless the information about the included
> transactions is *identically zero*.  If this information is extremely
> small, then I agree that something other than the orphan cost would
> drive the fee market. I don't think this will ever by the case, however.

You missed one key point - information transmitted across networks is
not slowed down on a per-bit basis, its (largely) slowed down on a
per-packet basis! You have some 1400+ bytes to work with before you
notice the increase in information in relay cost. Of course this only
applies if you are talking about long-haul links (across oceans, between
countries, etc), if you're talking about short links it is a per-byte
cost, but your time is probably negligible for anything but VERY large
blocks.

>> I'm not aware of any
>> public code to do so, but I know several large miners who pre-relay the
>> block(s) they are working on to other nodes *of theirs* around the globe….
> 
> In this paragraph, you are talking about intra-miner communication (you
> refer to other nodes "of theirs").   I agree that this can be very fast
> because there's only one entity involved. 

Very fast is an understatement. My point was that it is literally always
a single packet, and a very small packet, at that.

>> Thus, the orphan risk for including a transaction is related to the
>> validation time … the incentive to not include transactions which have
>> already
>> been relayed around sufficiently is, while not theoretically zero, *as
>> near to zero in practice as you can get*.
> 
> I look forward to hearing your talk about the Relay Network.  Let's
> image there was no block size limit.  Using the Relay Network*as it
> currently operates now*, how big would the block solution announcement be:

I think you missed it, this mail was not about the relay network. Let's
ignore the relay network entirely for now. Several miners have nodes
well distributed all over the globe which peer either directly with
other miners in the area, or just connect to a ton of nodes, probably
hitting other miners. The cost to transmit from their pool server to
their relay server is one packet, always. They pre-relay everything
they're gonna include, so, at max, they might have to pick between a few
pre-relayed blocks (what, like, 2/5 bits?). From there, you assume they
are transmitting to a node in the same DC, or at least very close (say,
local within AWS-US-East/maybe between AWS-US-East and somewhere in
NYC), where transmitting even 100MB is really fast.

> (1) if a miner published a 10 MB block where 100% of the transactions
> are known by the other nodes?  
> 
> (2) if a miner published a 10 MB block where 90% of the transactions are
> known by the other nodes?  
> 
> (3) if a miner published a 100 MB spam block where 0% of the
> transactions are known by the other nodes?
> 
> Best regards,
> Peter
> 
> 
>>
>> Matt
>>
>> On 08/29/15 23:17, Peter R wrote:
>>> Hello Matt and Daniele,
>>>
>>>> this seems to ignore the effects of transaction validation caches and
>>>> *block
>>>> compression protocols. *
>>>
>>> The effect of block compression protocols is included.  This is what I
>>> call the "coding gain" and use the Greek letter "gamma" to represent.
>>>
>>> As long as the block solution announcements contain information (i.e.,
>>> Shannon Entropy) about the transactions included in a block, then the
>>> fee market will be "healthy" according to the definitions given in the
>>> linked paper (see below).  This is the case right now, this is the case
>>> with your relay network, and this would be the case using any
>>> implementation of IBLTs that I can imagine, so long as miners can still
>>> construct blocks according to their own volition.  The "healthy fee
>>> market" result follows from the Shannon-Hartley theorem; the SH-theorem
>>> describes the maximum rate at which information (Shannon Entropy) can be
>>> transmitted over a physical communication channel.   
>>>
>>> https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf
>>>
>>> I've exchanged emails with Greg Maxwell about (what IMO is) an academic
>>> scenario where the block solutions announcements contain *no information
>>> at all* about the transactions included in the blocks.  Although the fee
>>> market would not be healthy in such a scenario, it is my feeling that
>>> this also requires miners to relinquish their ability to construct
>>> blocks according to their own volition (i.e., the system would already
>>> be centralized).  I look forward to a white paper demonstrating
>>> otherwise!
>>>
>>> Best regards,
>>> Peter
>>>
>>>
>>>
>>> On 2015-08-29, at 2:07 PM, Matt Corallo via bitcoin-dev
>>> <bitcoin-dev@lists.linuxfoundation.org
>>> <mailto:bitcoin-dev@lists.linuxfoundation.org>
>>> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>>>
>>>> I believe it was pointed out previously in the discussion of the Peter R
>>>> paper, but I'll repeat it here so that its visible - this seems to
>>>> ignore the effects of transaction validation caches and block
>>>> compression protocols. Many large miners already have their own network
>>>> to relay blocks around the globe with only a few bytes on the wire at
>>>> block-time, and there is also the bitcoinrelaynetwork.org
>>>> <http://bitcoinrelaynetwork.org>
>>>> <http://bitcoinrelaynetwork.org> network, which
>>>> does the same for smaller miners, albeit with slightly less efficiency.
>>>> Also, transaction validation time upon receiving a block can be rather
>>>> easily made negligible (ie the only validation time you should have is
>>>> the DB modify-utxo-set time). Thus, the increased orphan risk for
>>>> including a transaction can be reduced to a very, very tiny amount,
>>>> making the optimal blocksize, essentially, including everything that
>>>> you're confident is in the mempool of other reasonably large miners.
>>>>
>>>> Matt
>>>>
>>>> On 08/29/15 16:43, Daniele Pinna via bitcoin-dev wrote:
>>>>> I'd like to submit this paper to the dev-list which analyzes how miner
>>>>> advantages scale with network and mempool properties in a scenario of
>>>>> uncapped block sizes. The work proceeds, in a sense, from where Peter
>>>>> R's work left off correcting a mistake and addressing the critiques
>>>>> made
>>>>> by the community to his work.
>>>>>
>>>>> The main result of the work is a detailed analysis of mining advantages
>>>>> (defined as the added profit per unit of hash) as a function of miner
>>>>> hashrate. In it, I show how large block subsidies (or better, low
>>>>> mempool fees-to-subsidy ratios) incentivize the pooling of large
>>>>> hashrates due to the steady increasing of marginal profits as hashrates
>>>>> grow.
>>>>>
>>>>> The paper also shows that part of the large advantage the large miners
>>>>> have today is due to there being a barrier to entry into a
>>>>> high-efficiency mining class which has access to expected profits an
>>>>> order of magnitude larger than everyone else. As block subsidies
>>>>> decrease, this high-efficiency class is expected to vanish leading to a
>>>>> marginal profit structure which decreases as a function of hashrate.
>>>>>
>>>>> This work has vacuumed my entire life for the past two weeks leading me
>>>>> to lag behind on a lot of work. I apologize for typos which I may not
>>>>> have seen. I stand by for any comments the community may have and look
>>>>> forward to reigniting consideration of a block size scaling proposal
>>>>> (BIP101) which, due to the XT fork drama, I believe has been placed
>>>>> hastily and undeservedly on the chopping block.
>>>>>
>>>>> https://www.scribd.com/doc/276849939/On-the-Nature-of-Miner-Advantages-in-Uncapped-Block-Size-Fee-Markets
>>>>>
>>>>>
>>>>> Regards,
>>>>> Daniele
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> bitcoin-dev mailing list
>>>>> bitcoin-dev@lists.linuxfoundation.org
>>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>>
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev@lists.linuxfoundation.org
>>>> <mailto:bitcoin-dev@lists.linuxfoundation.org>
>>>> <mailto:bitcoin-dev@lists.linuxfoundation.org>
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>