[Bitcoin-development] Proposal for extra nonce in block header

[Bitcoin-development] Proposal for extra nonce in block header

[Timo Hanke]

I'd like to put the following draft of a BIP up for discussion.

Timo

# Abstract
There are incentives for miners to find cheap, non-standard ways to generate new work, which are not necessarily in the best interest of the protocol.
In order to reduce these incentives this proposal re-assigns 2 bytes from the version field of the block header to a new extra nonce field. 
# Copyright
# Specification
The block version number field in the block header is reduced in size from 4 to 2 bytes. 
The third and fourth byte in the block header are assigned to the new extra nonce field inside the block header.
# Motivation
The motivation of this proposal is to provide miners with a cheap constant-complexity method to create new work that does not require altering the transaction tree.

Furthermore, the motivation is to protect the version and timestamp fields in the block header from abuse.
# Rationale
Traditionally, the extra nonce is part of the coinbase field of the generation transaction, which is always the very first transaction of a block.
After incrementing the extra nonce the minimum amount of work a miner has to do to re-calculate the block header is a) to hash the coinbase transaction and b) to re-calculate the left-most branch of the merkle tree all the way to the merkle root.
This is necessary overhead a miner has to do besides hashing the block header itself.
We shall call the process that leads to a new block header from the same transaction set the _pre-hashing_.

First it should be noted that the relative cost of pre-hashing in its traditional form depends
on the block size, which may create an unwanted incentive for miners
to keep the block size small. However, this is not the main motivation for
the current proposal.

While the block header is hashed by ASICs, pre-hashing typically happens on a CPU because of the greater flexibility required.
Consequently, as ASIC cost per hash performance drops the relative cost of pre-hashing increases.

This creates an incentive for miners to find cheaper ways to create new work than by means of pre-hashing.
An example of this currently happening is the on-device rolling of the timestamp into the future.
These ways of creating new work are unlikely to be in the best interest of the protocol.
For example, rolling the timestamp faster than the real time is unwanted (more so on faster blockchains).

The version number in the block header is a possible target for alteration with the goal of cheaply creating new work.
Currently, blocks with arbitrarily large version numbers get relayed and accepted by the network.
As this is unwanted behaviour, there should not exist any incentive for a miner to abuse the version number in this way. 

The solution is to reduce the range of version numbers from 2^32 to 2^16 and to declare the third and forth bytes of the block header as legitimate space for an extra nonce.
This will reduce the incentive for a miner to abuse the shortened version number by a factor in the order of 2^16. 

As a side effect, this proposal greatly reduces the bandwidth requirements of a blind pool protocol by only submitting the block header to the miner.
# Backwards Compatibility
Old versions of the client will accept blocks of this kind but will throw an alert at the user to upgrade.
The only code change would be a cast of the version number to a short.
Besides the upgrade alert, old and new versions of the client can co-exist and there is no need to introduce a new block version number or to phase-out old block versions.
# Reference Implementation
# Final implementation

-- 
Timo Hanke
PGP 1EFF 69BC 6FB7 8744 14DB  631D 1BB5 D6E3 AB96 7DA8

Re: [Bitcoin-development] Proposal for extra nonce in block header

[Melvin Carvalho]

[-- Attachment #1: Type: text/plain, Size: 4561 bytes --]

On 27 April 2014 09:07, Timo Hanke <timo.hanke@web.de> wrote:

> I'd like to put the following draft of a BIP up for discussion.
>
> Timo
>
> # Abstract
> There are incentives for miners to find cheap, non-standard ways to
> generate new work, which are not necessarily in the best interest of the
> protocol.
> In order to reduce these incentives this proposal re-assigns 2 bytes from
> the version field of the block header to a new extra nonce field.
> # Copyright
> # Specification
> The block version number field in the block header is reduced in size from
> 4 to 2 bytes.
> The third and fourth byte in the block header are assigned to the new
> extra nonce field inside the block header.
> # Motivation
> The motivation of this proposal is to provide miners with a cheap
> constant-complexity method to create new work that does not require
> altering the transaction tree.
>
> Furthermore, the motivation is to protect the version and timestamp fields
> in the block header from abuse.
> # Rationale
> Traditionally, the extra nonce is part of the coinbase field of the
> generation transaction, which is always the very first transaction of a
> block.
> After incrementing the extra nonce the minimum amount of work a miner has
> to do to re-calculate the block header is a) to hash the coinbase
> transaction and b) to re-calculate the left-most branch of the merkle tree
> all the way to the merkle root.
> This is necessary overhead a miner has to do besides hashing the block
> header itself.
> We shall call the process that leads to a new block header from the same
> transaction set the _pre-hashing_.
>
> First it should be noted that the relative cost of pre-hashing in its
> traditional form depends
> on the block size, which may create an unwanted incentive for miners
> to keep the block size small. However, this is not the main motivation for
> the current proposal.
>
> While the block header is hashed by ASICs, pre-hashing typically happens
> on a CPU because of the greater flexibility required.
> Consequently, as ASIC cost per hash performance drops the relative cost of
> pre-hashing increases.
>
> This creates an incentive for miners to find cheaper ways to create new
> work than by means of pre-hashing.
> An example of this currently happening is the on-device rolling of the
> timestamp into the future.
> These ways of creating new work are unlikely to be in the best interest of
> the protocol.
> For example, rolling the timestamp faster than the real time is unwanted
> (more so on faster blockchains).
>
> The version number in the block header is a possible target for alteration
> with the goal of cheaply creating new work.
> Currently, blocks with arbitrarily large version numbers get relayed and
> accepted by the network.
> As this is unwanted behaviour, there should not exist any incentive for a
> miner to abuse the version number in this way.
>
> The solution is to reduce the range of version numbers from 2^32 to 2^16
> and to declare the third and forth bytes of the block header as legitimate
> space for an extra nonce.
> This will reduce the incentive for a miner to abuse the shortened version
> number by a factor in the order of 2^16.
>
> As a side effect, this proposal greatly reduces the bandwidth requirements
> of a blind pool protocol by only submitting the block header to the miner.
> # Backwards Compatibility
> Old versions of the client will accept blocks of this kind but will throw
> an alert at the user to upgrade.
> The only code change would be a cast of the version number to a short.
> Besides the upgrade alert, old and new versions of the client can co-exist
> and there is no need to introduce a new block version number or to
> phase-out old block versions.
> # Reference Implementation
> # Final implementation
>

If changing the structure of the block header, wouldnt you also need to
increment the version number to 3?


>
> --
> Timo Hanke
> PGP 1EFF 69BC 6FB7 8744 14DB  631D 1BB5 D6E3 AB96 7DA8
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 5378 bytes --]

Re: [Bitcoin-development] Proposal for extra nonce in block header

[Timo Hanke]

> If changing the structure of the block header, wouldnt you also need to
> increment the version number to 3?

No, in this case I don't think so. Incrementing the version number has
two purposes:

1. inform old clients that something new is going on
2. be able to phase out old version numbers and block them once the new
version number becomes a supermajority.

None of these two is necessary here. Old clients already recognize the
new block headers as something new because they look like very high
version numbers to them. And there is no reason to ever phase out blocks
that have zero in the MSBs of the version.

On Sun, Apr 27, 2014 at 10:17:11AM +0200, Melvin Carvalho wrote:
> On 27 April 2014 09:07, Timo Hanke <timo.hanke@web.de> wrote:
> 
>     I'd like to put the following draft of a BIP up for discussion.
> 
>     Timo
> 
>     # Abstract
>     There are incentives for miners to find cheap, non-standard ways to
>     generate new work, which are not necessarily in the best interest of the
>     protocol.
>     In order to reduce these incentives this proposal re-assigns 2 bytes from
>     the version field of the block header to a new extra nonce field.
>     # Copyright
>     # Specification
>     The block version number field in the block header is reduced in size from
>     4 to 2 bytes.
>     The third and fourth byte in the block header are assigned to the new extra
>     nonce field inside the block header.
>     # Motivation
>     The motivation of this proposal is to provide miners with a cheap
>     constant-complexity method to create new work that does not require
>     altering the transaction tree.
> 
>     Furthermore, the motivation is to protect the version and timestamp fields
>     in the block header from abuse.
>     # Rationale
>     Traditionally, the extra nonce is part of the coinbase field of the
>     generation transaction, which is always the very first transaction of a
>     block.
>     After incrementing the extra nonce the minimum amount of work a miner has
>     to do to re-calculate the block header is a) to hash the coinbase
>     transaction and b) to re-calculate the left-most branch of the merkle tree
>     all the way to the merkle root.
>     This is necessary overhead a miner has to do besides hashing the block
>     header itself.
>     We shall call the process that leads to a new block header from the same
>     transaction set the _pre-hashing_.
> 
>     First it should be noted that the relative cost of pre-hashing in its
>     traditional form depends
>     on the block size, which may create an unwanted incentive for miners
>     to keep the block size small. However, this is not the main motivation for
>     the current proposal.
> 
>     While the block header is hashed by ASICs, pre-hashing typically happens on
>     a CPU because of the greater flexibility required.
>     Consequently, as ASIC cost per hash performance drops the relative cost of
>     pre-hashing increases.
> 
>     This creates an incentive for miners to find cheaper ways to create new
>     work than by means of pre-hashing.
>     An example of this currently happening is the on-device rolling of the
>     timestamp into the future.
>     These ways of creating new work are unlikely to be in the best interest of
>     the protocol.
>     For example, rolling the timestamp faster than the real time is unwanted
>     (more so on faster blockchains).
> 
>     The version number in the block header is a possible target for alteration
>     with the goal of cheaply creating new work.
>     Currently, blocks with arbitrarily large version numbers get relayed and
>     accepted by the network.
>     As this is unwanted behaviour, there should not exist any incentive for a
>     miner to abuse the version number in this way.
> 
>     The solution is to reduce the range of version numbers from 2^32 to 2^16
>     and to declare the third and forth bytes of the block header as legitimate
>     space for an extra nonce.
>     This will reduce the incentive for a miner to abuse the shortened version
>     number by a factor in the order of 2^16.
> 
>     As a side effect, this proposal greatly reduces the bandwidth requirements
>     of a blind pool protocol by only submitting the block header to the miner.
>     # Backwards Compatibility
>     Old versions of the client will accept blocks of this kind but will throw
>     an alert at the user to upgrade.
>     The only code change would be a cast of the version number to a short.
>     Besides the upgrade alert, old and new versions of the client can co-exist
>     and there is no need to introduce a new block version number or to
>     phase-out old block versions.
>     # Reference Implementation
>     # Final implementation
> 
> 
> If changing the structure of the block header, wouldnt you also need to
> increment the version number to 3?

Re: [Bitcoin-development] Proposal for extra nonce in block header

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 6351 bytes --]

Although I agree 32 bits for a version is overkill, I really don't like the
idea of you simply ignoring the protocol spec to try and reduce your own
costs. Especially because in future we should make unknown versions a
validation rule, so we can easily trigger hard forks.

If this change was introduced through a proper process and software was
properly upgraded to understand the new header format, that'd be one thing.
Arbitrarily exploiting what is IMHO a missing rule in the rule set to shave
a bit more profit is something else.


On Sun, May 4, 2014 at 5:14 PM, Timo Hanke <timo.hanke@web.de> wrote:

> > If changing the structure of the block header, wouldnt you also need to
> > increment the version number to 3?
>
> No, in this case I don't think so. Incrementing the version number has
> two purposes:
>
> 1. inform old clients that something new is going on
> 2. be able to phase out old version numbers and block them once the new
> version number becomes a supermajority.
>
> None of these two is necessary here. Old clients already recognize the
> new block headers as something new because they look like very high
> version numbers to them. And there is no reason to ever phase out blocks
> that have zero in the MSBs of the version.
>
> On Sun, Apr 27, 2014 at 10:17:11AM +0200, Melvin Carvalho wrote:
> > On 27 April 2014 09:07, Timo Hanke <timo.hanke@web.de> wrote:
> >
> >     I'd like to put the following draft of a BIP up for discussion.
> >
> >     Timo
> >
> >     # Abstract
> >     There are incentives for miners to find cheap, non-standard ways to
> >     generate new work, which are not necessarily in the best interest of
> the
> >     protocol.
> >     In order to reduce these incentives this proposal re-assigns 2 bytes
> from
> >     the version field of the block header to a new extra nonce field.
> >     # Copyright
> >     # Specification
> >     The block version number field in the block header is reduced in
> size from
> >     4 to 2 bytes.
> >     The third and fourth byte in the block header are assigned to the
> new extra
> >     nonce field inside the block header.
> >     # Motivation
> >     The motivation of this proposal is to provide miners with a cheap
> >     constant-complexity method to create new work that does not require
> >     altering the transaction tree.
> >
> >     Furthermore, the motivation is to protect the version and timestamp
> fields
> >     in the block header from abuse.
> >     # Rationale
> >     Traditionally, the extra nonce is part of the coinbase field of the
> >     generation transaction, which is always the very first transaction
> of a
> >     block.
> >     After incrementing the extra nonce the minimum amount of work a
> miner has
> >     to do to re-calculate the block header is a) to hash the coinbase
> >     transaction and b) to re-calculate the left-most branch of the
> merkle tree
> >     all the way to the merkle root.
> >     This is necessary overhead a miner has to do besides hashing the
> block
> >     header itself.
> >     We shall call the process that leads to a new block header from the
> same
> >     transaction set the _pre-hashing_.
> >
> >     First it should be noted that the relative cost of pre-hashing in its
> >     traditional form depends
> >     on the block size, which may create an unwanted incentive for miners
> >     to keep the block size small. However, this is not the main
> motivation for
> >     the current proposal.
> >
> >     While the block header is hashed by ASICs, pre-hashing typically
> happens on
> >     a CPU because of the greater flexibility required.
> >     Consequently, as ASIC cost per hash performance drops the relative
> cost of
> >     pre-hashing increases.
> >
> >     This creates an incentive for miners to find cheaper ways to create
> new
> >     work than by means of pre-hashing.
> >     An example of this currently happening is the on-device rolling of
> the
> >     timestamp into the future.
> >     These ways of creating new work are unlikely to be in the best
> interest of
> >     the protocol.
> >     For example, rolling the timestamp faster than the real time is
> unwanted
> >     (more so on faster blockchains).
> >
> >     The version number in the block header is a possible target for
> alteration
> >     with the goal of cheaply creating new work.
> >     Currently, blocks with arbitrarily large version numbers get relayed
> and
> >     accepted by the network.
> >     As this is unwanted behaviour, there should not exist any incentive
> for a
> >     miner to abuse the version number in this way.
> >
> >     The solution is to reduce the range of version numbers from 2^32 to
> 2^16
> >     and to declare the third and forth bytes of the block header as
> legitimate
> >     space for an extra nonce.
> >     This will reduce the incentive for a miner to abuse the shortened
> version
> >     number by a factor in the order of 2^16.
> >
> >     As a side effect, this proposal greatly reduces the bandwidth
> requirements
> >     of a blind pool protocol by only submitting the block header to the
> miner.
> >     # Backwards Compatibility
> >     Old versions of the client will accept blocks of this kind but will
> throw
> >     an alert at the user to upgrade.
> >     The only code change would be a cast of the version number to a
> short.
> >     Besides the upgrade alert, old and new versions of the client can
> co-exist
> >     and there is no need to introduce a new block version number or to
> >     phase-out old block versions.
> >     # Reference Implementation
> >     # Final implementation
> >
> >
> > If changing the structure of the block header, wouldnt you also need to
> > increment the version number to 3?
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 7757 bytes --]

Re: [Bitcoin-development] Proposal for extra nonce in block header

[Timo Hanke]

On Sun, May 04, 2014 at 05:26:06PM +0200, Mike Hearn wrote:
> Although I agree 32 bits for a version is overkill, I really don't like the
> idea of you simply ignoring the protocol spec to try and reduce your own costs.

The purpose of the proposal is to change the protocol spec, not to
ignore it.

The argument for the proposal is explained in the Rationale section, and
in abstracted form means precisely to make everybody follow the protocol
spec by reducing incentives to ignore it. Specifically, it is about
protecting the timestamp field. 

I talked about relative costs involved in hashing, and how those will
change, and what incentives that creates. This development cannot be
ignored.

> Especially because in future we should make unknown versions a validation rule,
> so we can easily trigger hard forks.

Why does it require 32 bits?

> If this change was introduced through a proper process and software was
> properly upgraded to understand the new header format, that'd be one thing.
> Arbitrarily exploiting what is IMHO a missing rule in the rule set to shave a
> bit more profit is something else.

Again, this is a BIP. I am proposing a software upgrade, which is
absolutely required. When I said that version 3 is not required I meant
that the software upgrade (which basically just turns the nonce into a
short) does not have to be accompanied by a new version number for any
technical reason. Is there another reason why it should be incremented?

> On Sun, May 4, 2014 at 5:14 PM, Timo Hanke <timo.hanke@web.de> wrote:
> 
>     > If changing the structure of the block header, wouldnt you also need to
>     > increment the version number to 3?
> 
>     No, in this case I don't think so. Incrementing the version number has
>     two purposes:
> 
>     1. inform old clients that something new is going on
>     2. be able to phase out old version numbers and block them once the new
>     version number becomes a supermajority.
> 
>     None of these two is necessary here. Old clients already recognize the
>     new block headers as something new because they look like very high
>     version numbers to them. And there is no reason to ever phase out blocks
>     that have zero in the MSBs of the version.
> 
>     On Sun, Apr 27, 2014 at 10:17:11AM +0200, Melvin Carvalho wrote:
>     > On 27 April 2014 09:07, Timo Hanke <timo.hanke@web.de> wrote:
>     >
>     >     I'd like to put the following draft of a BIP up for discussion.
>     >
>     >     Timo
>     >
>     >     # Abstract
>     >     There are incentives for miners to find cheap, non-standard ways to
>     >     generate new work, which are not necessarily in the best interest of
>     the
>     >     protocol.
>     >     In order to reduce these incentives this proposal re-assigns 2 bytes
>     from
>     >     the version field of the block header to a new extra nonce field.
>     >     # Copyright
>     >     # Specification
>     >     The block version number field in the block header is reduced in size
>     from
>     >     4 to 2 bytes.
>     >     The third and fourth byte in the block header are assigned to the new
>     extra
>     >     nonce field inside the block header.
>     >     # Motivation
>     >     The motivation of this proposal is to provide miners with a cheap
>     >     constant-complexity method to create new work that does not require
>     >     altering the transaction tree.
>     >
>     >     Furthermore, the motivation is to protect the version and timestamp
>     fields
>     >     in the block header from abuse.
>     >     # Rationale
>     >     Traditionally, the extra nonce is part of the coinbase field of the
>     >     generation transaction, which is always the very first transaction of
>     a
>     >     block.
>     >     After incrementing the extra nonce the minimum amount of work a miner
>     has
>     >     to do to re-calculate the block header is a) to hash the coinbase
>     >     transaction and b) to re-calculate the left-most branch of the merkle
>     tree
>     >     all the way to the merkle root.
>     >     This is necessary overhead a miner has to do besides hashing the
>     block
>     >     header itself.
>     >     We shall call the process that leads to a new block header from the
>     same
>     >     transaction set the _pre-hashing_.
>     >
>     >     First it should be noted that the relative cost of pre-hashing in its
>     >     traditional form depends
>     >     on the block size, which may create an unwanted incentive for miners
>     >     to keep the block size small. However, this is not the main
>     motivation for
>     >     the current proposal.
>     >
>     >     While the block header is hashed by ASICs, pre-hashing typically
>     happens on
>     >     a CPU because of the greater flexibility required.
>     >     Consequently, as ASIC cost per hash performance drops the relative
>     cost of
>     >     pre-hashing increases.
>     >
>     >     This creates an incentive for miners to find cheaper ways to create
>     new
>     >     work than by means of pre-hashing.
>     >     An example of this currently happening is the on-device rolling of
>     the
>     >     timestamp into the future.
>     >     These ways of creating new work are unlikely to be in the best
>     interest of
>     >     the protocol.
>     >     For example, rolling the timestamp faster than the real time is
>     unwanted
>     >     (more so on faster blockchains).
>     >
>     >     The version number in the block header is a possible target for
>     alteration
>     >     with the goal of cheaply creating new work.
>     >     Currently, blocks with arbitrarily large version numbers get relayed
>     and
>     >     accepted by the network.
>     >     As this is unwanted behaviour, there should not exist any incentive
>     for a
>     >     miner to abuse the version number in this way.
>     >
>     >     The solution is to reduce the range of version numbers from 2^32 to 2
>     ^16
>     >     and to declare the third and forth bytes of the block header as
>     legitimate
>     >     space for an extra nonce.
>     >     This will reduce the incentive for a miner to abuse the shortened
>     version
>     >     number by a factor in the order of 2^16.
>     >
>     >     As a side effect, this proposal greatly reduces the bandwidth
>     requirements
>     >     of a blind pool protocol by only submitting the block header to the
>     miner.
>     >     # Backwards Compatibility
>     >     Old versions of the client will accept blocks of this kind but will
>     throw
>     >     an alert at the user to upgrade.
>     >     The only code change would be a cast of the version number to a
>     short.
>     >     Besides the upgrade alert, old and new versions of the client can
>     co-exist
>     >     and there is no need to introduce a new block version number or to
>     >     phase-out old block versions.
>     >     # Reference Implementation
>     >     # Final implementation
>     >
>     >
>     > If changing the structure of the block header, wouldnt you also need to
>     > increment the version number to 3?
> 
>     ------------------------------------------------------------------------------
>     "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>     Instantly run your Selenium tests across 300+ browser/OS combos.  Get
>     unparalleled scalability from the best Selenium testing platform available.
>     Simple to use. Nothing to install. Get started now for free."
>     http://p.sf.net/sfu/SauceLabs
>     _______________________________________________
>     Bitcoin-development mailing list
>     Bitcoin-development@lists.sourceforge.net
>     https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 
> 

-- 
Timo Hanke
PGP 1EFF 69BC 6FB7 8744 14DB  631D 1BB5 D6E3 AB96 7DA8

Re: [Bitcoin-development] Proposal for extra nonce in block header

[Timo Hanke]

Greg,

I'd like to ask you to assign a BIP number to this proposal and open
another round of discussion.

There is now a reference implementation available as pull request #5102
(https://github.com/bitcoin/bitcoin/pull/5102).

It introduces a new version number (3) to properly distinguish the
interpretation of the version number and allow for a clean upgrade
process.

Unittests are included.

The updated BIP draft in .mediawiki format is available here:
https://github.com/BlockheaderNonce2/bitoin/wiki

Thanks,
Timo

On Sun, May 04, 2014 at 05:26:06PM +0200, Mike Hearn wrote:
> Although I agree 32 bits for a version is overkill, I really don't like the
> idea of you simply ignoring the protocol spec to try and reduce your own costs.
> Especially because in future we should make unknown versions a validation rule,
> so we can easily trigger hard forks.
> 
> If this change was introduced through a proper process and software was
> properly upgraded to understand the new header format, that'd be one thing.
> Arbitrarily exploiting what is IMHO a missing rule in the rule set to shave a
> bit more profit is something else.
> 
> 
> On Sun, May 4, 2014 at 5:14 PM, Timo Hanke <timo.hanke@web.de> wrote:
> 
>     > If changing the structure of the block header, wouldnt you also need to
>     > increment the version number to 3?
> 
>     No, in this case I don't think so. Incrementing the version number has
>     two purposes:
> 
>     1. inform old clients that something new is going on
>     2. be able to phase out old version numbers and block them once the new
>     version number becomes a supermajority.
> 
>     None of these two is necessary here. Old clients already recognize the
>     new block headers as something new because they look like very high
>     version numbers to them. And there is no reason to ever phase out blocks
>     that have zero in the MSBs of the version.
> 
>     On Sun, Apr 27, 2014 at 10:17:11AM +0200, Melvin Carvalho wrote:
>     > On 27 April 2014 09:07, Timo Hanke <timo.hanke@web.de> wrote:
>     >
>     >     I'd like to put the following draft of a BIP up for discussion.
>     >
>     >     Timo
>     >
>     >     # Abstract
>     >     There are incentives for miners to find cheap, non-standard ways to
>     >     generate new work, which are not necessarily in the best interest of
>     the
>     >     protocol.
>     >     In order to reduce these incentives this proposal re-assigns 2 bytes
>     from
>     >     the version field of the block header to a new extra nonce field.
>     >     # Copyright
>     >     # Specification
>     >     The block version number field in the block header is reduced in size
>     from
>     >     4 to 2 bytes.
>     >     The third and fourth byte in the block header are assigned to the new
>     extra
>     >     nonce field inside the block header.
>     >     # Motivation
>     >     The motivation of this proposal is to provide miners with a cheap
>     >     constant-complexity method to create new work that does not require
>     >     altering the transaction tree.
>     >
>     >     Furthermore, the motivation is to protect the version and timestamp
>     fields
>     >     in the block header from abuse.
>     >     # Rationale
>     >     Traditionally, the extra nonce is part of the coinbase field of the
>     >     generation transaction, which is always the very first transaction of
>     a
>     >     block.
>     >     After incrementing the extra nonce the minimum amount of work a miner
>     has
>     >     to do to re-calculate the block header is a) to hash the coinbase
>     >     transaction and b) to re-calculate the left-most branch of the merkle
>     tree
>     >     all the way to the merkle root.
>     >     This is necessary overhead a miner has to do besides hashing the
>     block
>     >     header itself.
>     >     We shall call the process that leads to a new block header from the
>     same
>     >     transaction set the _pre-hashing_.
>     >
>     >     First it should be noted that the relative cost of pre-hashing in its
>     >     traditional form depends
>     >     on the block size, which may create an unwanted incentive for miners
>     >     to keep the block size small. However, this is not the main
>     motivation for
>     >     the current proposal.
>     >
>     >     While the block header is hashed by ASICs, pre-hashing typically
>     happens on
>     >     a CPU because of the greater flexibility required.
>     >     Consequently, as ASIC cost per hash performance drops the relative
>     cost of
>     >     pre-hashing increases.
>     >
>     >     This creates an incentive for miners to find cheaper ways to create
>     new
>     >     work than by means of pre-hashing.
>     >     An example of this currently happening is the on-device rolling of
>     the
>     >     timestamp into the future.
>     >     These ways of creating new work are unlikely to be in the best
>     interest of
>     >     the protocol.
>     >     For example, rolling the timestamp faster than the real time is
>     unwanted
>     >     (more so on faster blockchains).
>     >
>     >     The version number in the block header is a possible target for
>     alteration
>     >     with the goal of cheaply creating new work.
>     >     Currently, blocks with arbitrarily large version numbers get relayed
>     and
>     >     accepted by the network.
>     >     As this is unwanted behaviour, there should not exist any incentive
>     for a
>     >     miner to abuse the version number in this way.
>     >
>     >     The solution is to reduce the range of version numbers from 2^32 to 2
>     ^16
>     >     and to declare the third and forth bytes of the block header as
>     legitimate
>     >     space for an extra nonce.
>     >     This will reduce the incentive for a miner to abuse the shortened
>     version
>     >     number by a factor in the order of 2^16.
>     >
>     >     As a side effect, this proposal greatly reduces the bandwidth
>     requirements
>     >     of a blind pool protocol by only submitting the block header to the
>     miner.
>     >     # Backwards Compatibility
>     >     Old versions of the client will accept blocks of this kind but will
>     throw
>     >     an alert at the user to upgrade.
>     >     The only code change would be a cast of the version number to a
>     short.
>     >     Besides the upgrade alert, old and new versions of the client can
>     co-exist
>     >     and there is no need to introduce a new block version number or to
>     >     phase-out old block versions.
>     >     # Reference Implementation
>     >     # Final implementation
>     >
>     >
>     > If changing the structure of the block header, wouldnt you also need to
>     > increment the version number to 3?
> 
>     ------------------------------------------------------------------------------
>     "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>     Instantly run your Selenium tests across 300+ browser/OS combos.  Get
>     unparalleled scalability from the best Selenium testing platform available.
>     Simple to use. Nothing to install. Get started now for free."
>     http://p.sf.net/sfu/SauceLabs
>     _______________________________________________
>     Bitcoin-development mailing list
>     Bitcoin-development@lists.sourceforge.net
>     https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 
> 

-- 
Timo Hanke
PGP 1EFF 69BC 6FB7 8744 14DB  631D 1BB5 D6E3 AB96 7DA8

Re: [Bitcoin-development] Proposal for extra nonce in block header

[Mark Friedenbach]

I'm not convinced of the necessity of this idea in general, but if it
were to be implemented I would recommend serializing the nVersion field
as a VarInt (Pieter Wuille's multi-byte serialization format) and using
the remaining space of the 4 bytes as your extra nonce.

That would allow serialization of numbers up to 0x1020407f (slightly
over 28 bits) before the 4-byte field is exhausted. For version numbers
less than 0x204080 there will be at least one byte of padding space left
over for extra-nonce usage (two bytes if less than 0x4080, three bytes
if less than 0x80). For version values up to 127, the format is exactly
identical when the padding bytes are zero.

On 04/27/2014 12:07 AM, Timo Hanke wrote:
> I'd like to put the following draft of a BIP up for discussion.
> 
> Timo
> 
> # Abstract
> There are incentives for miners to find cheap, non-standard ways to generate new work, which are not necessarily in the best interest of the protocol.
> In order to reduce these incentives this proposal re-assigns 2 bytes from the version field of the block header to a new extra nonce field. 
> # Copyright
> # Specification
> The block version number field in the block header is reduced in size from 4 to 2 bytes. 
> The third and fourth byte in the block header are assigned to the new extra nonce field inside the block header.
> # Motivation
> The motivation of this proposal is to provide miners with a cheap constant-complexity method to create new work that does not require altering the transaction tree.
> 
> Furthermore, the motivation is to protect the version and timestamp fields in the block header from abuse.
> # Rationale
> Traditionally, the extra nonce is part of the coinbase field of the generation transaction, which is always the very first transaction of a block.
> After incrementing the extra nonce the minimum amount of work a miner has to do to re-calculate the block header is a) to hash the coinbase transaction and b) to re-calculate the left-most branch of the merkle tree all the way to the merkle root.
> This is necessary overhead a miner has to do besides hashing the block header itself.
> We shall call the process that leads to a new block header from the same transaction set the _pre-hashing_.
> 
> First it should be noted that the relative cost of pre-hashing in its traditional form depends
> on the block size, which may create an unwanted incentive for miners
> to keep the block size small. However, this is not the main motivation for
> the current proposal.
> 
> While the block header is hashed by ASICs, pre-hashing typically happens on a CPU because of the greater flexibility required.
> Consequently, as ASIC cost per hash performance drops the relative cost of pre-hashing increases.
> 
> This creates an incentive for miners to find cheaper ways to create new work than by means of pre-hashing.
> An example of this currently happening is the on-device rolling of the timestamp into the future.
> These ways of creating new work are unlikely to be in the best interest of the protocol.
> For example, rolling the timestamp faster than the real time is unwanted (more so on faster blockchains).
> 
> The version number in the block header is a possible target for alteration with the goal of cheaply creating new work.
> Currently, blocks with arbitrarily large version numbers get relayed and accepted by the network.
> As this is unwanted behaviour, there should not exist any incentive for a miner to abuse the version number in this way. 
> 
> The solution is to reduce the range of version numbers from 2^32 to 2^16 and to declare the third and forth bytes of the block header as legitimate space for an extra nonce.
> This will reduce the incentive for a miner to abuse the shortened version number by a factor in the order of 2^16. 
> 
> As a side effect, this proposal greatly reduces the bandwidth requirements of a blind pool protocol by only submitting the block header to the miner.
> # Backwards Compatibility
> Old versions of the client will accept blocks of this kind but will throw an alert at the user to upgrade.
> The only code change would be a cast of the version number to a short.
> Besides the upgrade alert, old and new versions of the client can co-exist and there is no need to introduce a new block version number or to phase-out old block versions.
> # Reference Implementation
> # Final implementation
> 

Re: [Bitcoin-development] Proposal for extra nonce in block header

[Timo Hanke]

On Sun, Apr 27, 2014 at 02:38:06AM -0700, Mark Friedenbach wrote:
> I'm not convinced of the necessity of this idea in general, but if it
> were to be implemented I would recommend serializing the nVersion field
> as a VarInt (Pieter Wuille's multi-byte serialization format) and using
> the remaining space of the 4 bytes as your extra nonce.
> 
> That would allow serialization of numbers up to 0x1020407f (slightly
> over 28 bits) before the 4-byte field is exhausted. For version numbers
> less than 0x204080 there will be at least one byte of padding space left
> over for extra-nonce usage (two bytes if less than 0x4080, three bytes
> if less than 0x80). For version values up to 127, the format is exactly
> identical when the padding bytes are zero.

Neat idea. It might somewhat reduce the flexibility in which the version
field can be used in the future, though. 

For the sake of simplicity I lean towards a fixed length version field,
but would be ok with either. Keep in mind that version numbers can be
recycled after a couple of years. So there is effectively zero benefit
in allowing varints to extend beyond 2 bytes. For that reason personally
I think 1 byte of version number would be enough.

> On 04/27/2014 12:07 AM, Timo Hanke wrote:
> > I'd like to put the following draft of a BIP up for discussion.
> > 
> > Timo
> > 
> > # Abstract
> > There are incentives for miners to find cheap, non-standard ways to generate new work, which are not necessarily in the best interest of the protocol.
> > In order to reduce these incentives this proposal re-assigns 2 bytes from the version field of the block header to a new extra nonce field. 
> > # Copyright
> > # Specification
> > The block version number field in the block header is reduced in size from 4 to 2 bytes. 
> > The third and fourth byte in the block header are assigned to the new extra nonce field inside the block header.
> > # Motivation
> > The motivation of this proposal is to provide miners with a cheap constant-complexity method to create new work that does not require altering the transaction tree.
> > 
> > Furthermore, the motivation is to protect the version and timestamp fields in the block header from abuse.
> > # Rationale
> > Traditionally, the extra nonce is part of the coinbase field of the generation transaction, which is always the very first transaction of a block.
> > After incrementing the extra nonce the minimum amount of work a miner has to do to re-calculate the block header is a) to hash the coinbase transaction and b) to re-calculate the left-most branch of the merkle tree all the way to the merkle root.
> > This is necessary overhead a miner has to do besides hashing the block header itself.
> > We shall call the process that leads to a new block header from the same transaction set the _pre-hashing_.
> > 
> > First it should be noted that the relative cost of pre-hashing in its traditional form depends
> > on the block size, which may create an unwanted incentive for miners
> > to keep the block size small. However, this is not the main motivation for
> > the current proposal.
> > 
> > While the block header is hashed by ASICs, pre-hashing typically happens on a CPU because of the greater flexibility required.
> > Consequently, as ASIC cost per hash performance drops the relative cost of pre-hashing increases.
> > 
> > This creates an incentive for miners to find cheaper ways to create new work than by means of pre-hashing.
> > An example of this currently happening is the on-device rolling of the timestamp into the future.
> > These ways of creating new work are unlikely to be in the best interest of the protocol.
> > For example, rolling the timestamp faster than the real time is unwanted (more so on faster blockchains).
> > 
> > The version number in the block header is a possible target for alteration with the goal of cheaply creating new work.
> > Currently, blocks with arbitrarily large version numbers get relayed and accepted by the network.
> > As this is unwanted behaviour, there should not exist any incentive for a miner to abuse the version number in this way. 
> > 
> > The solution is to reduce the range of version numbers from 2^32 to 2^16 and to declare the third and forth bytes of the block header as legitimate space for an extra nonce.
> > This will reduce the incentive for a miner to abuse the shortened version number by a factor in the order of 2^16. 
> > 
> > As a side effect, this proposal greatly reduces the bandwidth requirements of a blind pool protocol by only submitting the block header to the miner.
> > # Backwards Compatibility
> > Old versions of the client will accept blocks of this kind but will throw an alert at the user to upgrade.
> > The only code change would be a cast of the version number to a short.
> > Besides the upgrade alert, old and new versions of the client can co-exist and there is no need to introduce a new block version number or to phase-out old block versions.
> > # Reference Implementation
> > # Final implementation
> > 
> 
> 

-- 
Timo Hanke
PGP 1EFF 69BC 6FB7 8744 14DB  631D 1BB5 D6E3 AB96 7DA8