[bitcoin-dev] Miners are struggling with blocks far smaller than 750KB blocks and resorting to SPV mining

[bitcoin-dev] Miners are struggling with blocks far smaller than 750KB blocks and resorting to SPV mining

[Luv Khemani]

[-- Attachment #1: Type: text/plain, Size: 2496 bytes --]

Hi all,
  I previously mentioned in a post that i believe that technically nodes are capable of handling blocks an order of magnitude larger than the current blocksize limit, the only missing thing was an incentive to run them. I have been monitoring the blockchain for the past couple of weeks and am seeing that even miners who have all the incentives are for whatever reason struggling to download and validate much smaller blocks.
The data actually paints a very grim picture of the current bandwidth/validating capacity of the global mining network.
See the following empty blocks mined despite a non-trivial elapsed time from the previous block just from the past couple of days alone (Data from insight.bitpay.com):

EmptyBlock /Time since previous block/ Size of previous block(bytes)/Mined by====================================================370165  29s  720784  Antpool370160  31s  50129    BTCChinaPool370076  49s  469988  F2Pool370059  34s  110994  Antpool370057  73s  131603  Antpool
We have preceding blocks as small as 50KB with 30s passing and the miner continues to mine empty blocks via SPV mining.
The most glaring case is Block 370057 where despite 73s elapsing and the preceding block being a mere 131KB, the miner is unable to download/validate fast enough to include transactions in his block. Unless ofcourse the miner is mining empty blocks on purpose, which does not make sense as all of these pools do mine blocks with transactions when the elapsed time is greater.

This is a cause for great concern, because if miners are SPV mining for a whole minute for <750KB blocks, at 8MB blocks, the network will just fall apart as a significant portion of the hashing power SPV mines throughout. All a single malicious miner has to do is mine an invalid block on purpose, let these pools SPV mine on top of them while it mines a valid block free of their competition. Yes, these pools deserve to lose money in that event, but the impact of reorgs and many block orphans for anyone not running a full node could be disastrous, especially more so in the XT world where Mike wants everyone to be running SPV nodes. I simply don't see the XT fork having any chance of surviving if SPV nodes are unreliable. 
And if these pools go out of business, it will lead to even more mining centralization which is already too centralized today.

Can anyone representing these pools comment on why this is happening? Are these pools on Matt's relay network?



 		 	   		  

[-- Attachment #2: Type: text/html, Size: 3080 bytes --]

Re: [bitcoin-dev] Miners are struggling with blocks far smaller than 750KB blocks and resorting to SPV mining

[jl2012]

The traffic between the pool server and individual hashers is far busier 
than 50kB/30s. If their bandwidth is so limited, hashers would have 
switched to other pools already.

All these data may prove is they have very bad mining codes. For 
example, their hashers may not be required to update the transaction 
list regularly. I don't think they are struggling. They are just too 
lazy or think that's too risky to improve their code. After all, they 
are generating half million USD per day and a few seconds of downtime 
would hurt.

By the way, vast majority of the full blocks (>0.99MB) on the blockchain 
are generated by Chinese pools.

Luv Khemani via bitcoin-dev 於 2015-08-17 04:42 寫到:
> Hi all,
> 
>  I previously mentioned in a post that i believe that technically
> nodes are capable of handling blocks an order of magnitude larger than
> the current blocksize limit, the only missing thing was an incentive
> to run them. I have been monitoring the blockchain for the past couple
> of weeks and am seeing that even miners who have all the incentives
> are for whatever reason struggling to download and validate much
> smaller blocks.
> 
> The data actually paints a very grim picture of the current
> bandwidth/validating capacity of the global mining network.
> 
> See the following empty blocks mined despite a non-trivial elapsed
> time from the previous block just from the past couple of days alone
> (Data from insight.bitpay.com):
> 
> EmptyBlock /Time since previous block/ Size of previous
> block(bytes)/Mined by
> ====================================================370165 29s 720784
> Antpool
> 370160 31s 50129 BTCChinaPool
> 370076 49s 469988 F2Pool
> 370059 34s 110994 Antpool
> 370057 73s 131603 Antpool
> 
> We have preceding blocks as small as 50KB with 30s passing and the
> miner continues to mine empty blocks via SPV mining.
> The most glaring case is Block 370057 where despite 73s elapsing and
> the preceding block being a mere 131KB, the miner is unable to
> download/validate fast enough to include transactions in his block.
> Unless ofcourse the miner is mining empty blocks on purpose, which
> does not make sense as all of these pools do mine blocks with
> transactions when the elapsed time is greater.
> 
> This is a cause for great concern, because if miners are SPV mining
> for a whole minute for <750KB blocks, at 8MB blocks, the network will
> just fall apart as a significant portion of the hashing power SPV
> mines throughout. All a single malicious miner has to do is mine an
> invalid block on purpose, let these pools SPV mine on top of them
> while it mines a valid block free of their competition. Yes, these
> pools deserve to lose money in that event, but the impact of reorgs
> and many block orphans for anyone not running a full node could be
> disastrous, especially more so in the XT world where Mike wants
> everyone to be running SPV nodes. I simply don't see the XT fork
> having any chance of surviving if SPV nodes are unreliable.
> 
> And if these pools go out of business, it will lead to even more
> mining centralization which is already too centralized today.
> 
> Can anyone representing these pools comment on why this is happening?
> Are these pools on Matt's relay network?
> 
> 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Miners are struggling with blocks far smaller than 750KB blocks and resorting to SPV mining

[Luv Khemani]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb2312", Size: 4486 bytes --]

It could be laziness but i doubt it, especially when the business is so competitive and margins ever shrinking.Half a million dollars in revenue mean little if your running costs are also in the same region.
Also apologies for the bad formatting, outlook must have screwed it up.
EmptyBlock /Time since previous block/ Size of previous block(bytes)/Mined by====================================================370165 29s 720784 Antpool370160 31s 50129 BTCChinaPool370076 49s 469988 F2Pool370059 34s 110994 Antpool370057 73s 131603 Antpool


> Date: Mon, 17 Aug 2015 05:15:15 -0400
> From: jl2012@xbt.hk
> To: luvb@hotmail.com
> CC: bitcoin-dev@lists.linuxfoundation.org
> Subject: Re: [bitcoin-dev] Miners are struggling with blocks far smaller than 750KB blocks and resorting to SPV mining
> 
> The traffic between the pool server and individual hashers is far busier 
> than 50kB/30s. If their bandwidth is so limited, hashers would have 
> switched to other pools already.
> 
> All these data may prove is they have very bad mining codes. For 
> example, their hashers may not be required to update the transaction 
> list regularly. I don't think they are struggling. They are just too 
> lazy or think that's too risky to improve their code. After all, they 
> are generating half million USD per day and a few seconds of downtime 
> would hurt.
> 
> By the way, vast majority of the full blocks (>0.99MB) on the blockchain 
> are generated by Chinese pools.
> 
> Luv Khemani via bitcoin-dev ì¶ 2015-08-17 04:42 Œ‘µ½:
> > Hi all,
> > 
> >  I previously mentioned in a post that i believe that technically
> > nodes are capable of handling blocks an order of magnitude larger than
> > the current blocksize limit, the only missing thing was an incentive
> > to run them. I have been monitoring the blockchain for the past couple
> > of weeks and am seeing that even miners who have all the incentives
> > are for whatever reason struggling to download and validate much
> > smaller blocks.
> > 
> > The data actually paints a very grim picture of the current
> > bandwidth/validating capacity of the global mining network.
> > 
> > See the following empty blocks mined despite a non-trivial elapsed
> > time from the previous block just from the past couple of days alone
> > (Data from insight.bitpay.com):
> > 
> > EmptyBlock /Time since previous block/ Size of previous
> > block(bytes)/Mined by
> > ====================================================370165 29s 720784
> > Antpool
> > 370160 31s 50129 BTCChinaPool
> > 370076 49s 469988 F2Pool
> > 370059 34s 110994 Antpool
> > 370057 73s 131603 Antpool
> > 
> > We have preceding blocks as small as 50KB with 30s passing and the
> > miner continues to mine empty blocks via SPV mining.
> > The most glaring case is Block 370057 where despite 73s elapsing and
> > the preceding block being a mere 131KB, the miner is unable to
> > download/validate fast enough to include transactions in his block.
> > Unless ofcourse the miner is mining empty blocks on purpose, which
> > does not make sense as all of these pools do mine blocks with
> > transactions when the elapsed time is greater.
> > 
> > This is a cause for great concern, because if miners are SPV mining
> > for a whole minute for <750KB blocks, at 8MB blocks, the network will
> > just fall apart as a significant portion of the hashing power SPV
> > mines throughout. All a single malicious miner has to do is mine an
> > invalid block on purpose, let these pools SPV mine on top of them
> > while it mines a valid block free of their competition. Yes, these
> > pools deserve to lose money in that event, but the impact of reorgs
> > and many block orphans for anyone not running a full node could be
> > disastrous, especially more so in the XT world where Mike wants
> > everyone to be running SPV nodes. I simply don't see the XT fork
> > having any chance of surviving if SPV nodes are unreliable.
> > 
> > And if these pools go out of business, it will lead to even more
> > mining centralization which is already too centralized today.
> > 
> > Can anyone representing these pools comment on why this is happening?
> > Are these pools on Matt's relay network?
> > 
> > 
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 
 		 	   		  

[-- Attachment #2: Type: text/html, Size: 5571 bytes --]

Re: [bitcoin-dev] Miners are struggling with blocks far smaller than 750KB blocks and resorting to SPV mining

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 3448 bytes --]

As jl2012 indicated, this is an insufficient analysis.

You cannot assume that because X time passed since the last block, the
miner's internal block maker has updated the template, and from there, is
shipped out to the hashers in the field.  Further, even if you update the
block template at time Y, a hasher might return with a solution for the
block at time X, and the miner will of course publish solution X.

There are several steps of latency involved, even ignoring things such as
the miner relay network or getting US pool stratum links to a sub-pool in
China.






On Mon, Aug 17, 2015 at 4:42 AM, Luv Khemani via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi all,
>
>   I previously mentioned in a post that i believe that technically nodes
> are capable of handling blocks an order of magnitude larger than the
> current blocksize limit, the only missing thing was an incentive to run
> them. I have been monitoring the blockchain for the past couple of weeks
> and am seeing that even miners who have all the incentives are for whatever
> reason struggling to download and validate much smaller blocks.
>
> The data actually paints a very grim picture of the current
> bandwidth/validating capacity of the global mining network.
>
> See the following empty blocks mined despite a non-trivial elapsed time
> from the previous block just from the past couple of days alone (Data from
> insight.bitpay.com):
>
> EmptyBlock /Time since previous block/ Size of previous block(bytes)/Mined
> by
> ====================================================
> 370165  29s  720784  Antpool
> 370160  31s  50129    BTCChinaPool
> 370076  49s  469988  F2Pool
> 370059  34s  110994  Antpool
> 370057  73s  131603  Antpool
>
> We have preceding blocks as small as 50KB with 30s passing and the miner
> continues to mine empty blocks via SPV mining.
> The most glaring case is Block 370057 where despite 73s elapsing and the
> preceding block being a mere 131KB, the miner is unable to
> download/validate fast enough to include transactions in his block. Unless
> ofcourse the miner is mining empty blocks on purpose, which does not make
> sense as all of these pools do mine blocks with transactions when the
> elapsed time is greater.
>
> This is a cause for great concern, because if miners are SPV mining for a
> whole minute for <750KB blocks, at 8MB blocks, the network will just fall
> apart as a significant portion of the hashing power SPV mines throughout.
> All a single malicious miner has to do is mine an invalid block on purpose,
> let these pools SPV mine on top of them while it mines a valid block free
> of their competition. Yes, these pools deserve to lose money in that event,
> but the impact of reorgs and many block orphans for anyone not running a
> full node could be disastrous, especially more so in the XT world where
> Mike wants everyone to be running SPV nodes. I simply don't see the XT fork
> having any chance of surviving if SPV nodes are unreliable.
>
> And if these pools go out of business, it will lead to even more mining
> centralization which is already too centralized today.
>
> Can anyone representing these pools comment on why this is happening? Are
> these pools on Matt's relay network?
>
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 4385 bytes --]

Re: [bitcoin-dev] Miners are struggling with blocks far smaller than 750KB blocks and resorting to SPV mining

[Timo Hanke]

[-- Attachment #1: Type: text/plain, Size: 3634 bytes --]

That's not a valid conclusion. Just because you observe a miner producing
empty blocks doesn't mean he is SPV mining.

There can be many reasons for mining on an empty block even after having
fully verified the previous block. And therefore these reasons would be
completely independent of block size. You cannot conclude that miners are
struggling with a certain block size.

For example, there are reasons rooted in the mining hardware and mining
software itself, which have nothing to do with the node software, in
particular not with block propagation, verification or transaction
selection. See also https://github.com/BlockheaderNonce2/bitcoin/wiki where
this was warned about. The effect can be expected to become more pronounced
in the future.

Timo


On Mon, Aug 17, 2015 at 10:42 AM, Luv Khemani via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi all,
>
>   I previously mentioned in a post that i believe that technically nodes
> are capable of handling blocks an order of magnitude larger than the
> current blocksize limit, the only missing thing was an incentive to run
> them. I have been monitoring the blockchain for the past couple of weeks
> and am seeing that even miners who have all the incentives are for whatever
> reason struggling to download and validate much smaller blocks.
>
> The data actually paints a very grim picture of the current
> bandwidth/validating capacity of the global mining network.
>
> See the following empty blocks mined despite a non-trivial elapsed time
> from the previous block just from the past couple of days alone (Data from
> insight.bitpay.com):
>
> EmptyBlock /Time since previous block/ Size of previous block(bytes)/Mined
> by
> ====================================================
> 370165  29s  720784  Antpool
> 370160  31s  50129    BTCChinaPool
> 370076  49s  469988  F2Pool
> 370059  34s  110994  Antpool
> 370057  73s  131603  Antpool
>
> We have preceding blocks as small as 50KB with 30s passing and the miner
> continues to mine empty blocks via SPV mining.
> The most glaring case is Block 370057 where despite 73s elapsing and the
> preceding block being a mere 131KB, the miner is unable to
> download/validate fast enough to include transactions in his block. Unless
> ofcourse the miner is mining empty blocks on purpose, which does not make
> sense as all of these pools do mine blocks with transactions when the
> elapsed time is greater.
>
> This is a cause for great concern, because if miners are SPV mining for a
> whole minute for <750KB blocks, at 8MB blocks, the network will just fall
> apart as a significant portion of the hashing power SPV mines throughout.
> All a single malicious miner has to do is mine an invalid block on purpose,
> let these pools SPV mine on top of them while it mines a valid block free
> of their competition. Yes, these pools deserve to lose money in that event,
> but the impact of reorgs and many block orphans for anyone not running a
> full node could be disastrous, especially more so in the XT world where
> Mike wants everyone to be running SPV nodes. I simply don't see the XT fork
> having any chance of surviving if SPV nodes are unreliable.
>
> And if these pools go out of business, it will lead to even more mining
> centralization which is already too centralized today.
>
> Can anyone representing these pools comment on why this is happening? Are
> these pools on Matt's relay network?
>
>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 4573 bytes --]