[bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson]

[-- Attachment #1: Type: text/plain, Size: 2400 bytes --]

# BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

I admit, with my limited experience in the operation of the protocol, the section entitled 'Solution operation' may not be entirely correct but you will get the idea. If I have it wrong, please correct it back to the list.


## The problem:


Everybody wants value. Miners want to maximize revenue from fees. Consumers want transaction reliability and, (we presume) low fees.

Current transaction bandwidth limit is a limiting factor for both.


## Solution summary:


Provide each transaction with a transaction weight, being a function of the fee paid (on a curve), and the time waiting in the transaction pool (also on a curve) out to n days (n=30 ?); the transaction weight serving as the likelihood of a transaction being included in the current block, and then use a target block size.

Protocol enforcement to prevent high or low blocksize cheating to be handled by having the protocol determine the target size for the current block using; current transaction pool size x ( 1 / (144 x n days ) ) = transactions to be included in the current block.

The curves used for the weight of transactions would have to be appropriate.


## Pros:

* Maximizes transaction reliability.
* Maximizes possibility for consumer and business uptake.
* Maximizes total fees paid per block without reducing reliability; because of reliability, confidence and uptake are greater; therefore, more transactions and more transactions total at priority fees.
* Market determines fee paid for transaction priority.

* Fee recommendations work all the way out to 30 days or greater.

* Provides additional block entropy and greater security since there is less probability of predicting the next block.


## Cons:

* ?
* Must be first be programmed.
* Anything else?


## Solution operation:


As I have said, my simplistic view of the operation. If I have this wrong, please correct it back to the list.

1. The protocol determines the target block size.

2. Assign each transaction in the pool a transaction weight based on fee and time waiting in the transaction pool.

3. Begin selecting transactions to include in the current block using transaction weight as the likelihood of inclusion until target block size is met.

4. Solve block.


Regards,

Damian Williamson

[-- Attachment #2: Type: text/html, Size: 3550 bytes --]

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Jim Renkel]

[-- Attachment #1: Type: text/plain, Size: 3015 bytes --]

As i understand it, the transactions to be included in a block are 
entirely up to the miner of that block.


What prevents a miner from implementing the proposal on their own?


If this is adopted as some kind of "policy", what forces a miner to 
follow it?

Jim Renkel

On 12/2/2017 10:07 PM, Damian Williamson via bitcoin-dev wrote:
>
> # BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering 
> Transactions In Blocks
>
>
> I admit, with my limited experience in the operation of the protocol, 
> the section entitled 'Solution operation' may not be entirely correct 
> but you will get the idea. If I have it wrong, please correct it back 
> to the list.
>
> ## The problem:
>
>
> Everybody wants value. Miners want to maximize revenue from fees. 
> Consumers want transaction reliability and, (we presume) low fees.
>
>
> Current transaction bandwidth limit is a limiting factor for both.
>
> ## Solution summary:
>
>
> Provide each transaction with a transaction weight, being a function 
> of the fee paid (on a curve), and the time waiting in the transaction 
> pool (also on a curve) out to n days (n=30 ?); the transaction weight 
> serving as the likelihood of a transaction being included in the 
> current block, and then use a target block size.
>
>
> Protocol enforcement to prevent high or low blocksize cheating to be 
> handled by having the protocol determine the target size for the 
> current block using; current transaction pool size x ( 1 / (144 x n 
> days ) ) = transactions to be included in the current block.
>
> The curves used for the weight of transactions would have to be 
> appropriate.
>
> ## Pros:
>
>
> * Maximizes transaction reliability.
> * Maximizes possibility for consumer and business uptake.
> * Maximizes total fees paid per block without reducing reliability; 
> because of reliability, confidence and uptake are greater; therefore, 
> more transactions and more transactions total at priority fees.
> * Market determines fee paid for transaction priority.
>
> * Fee recommendations work all the way out to 30 days or greater.
>
> * Provides additional block entropy and greater security since there 
> is less probability of predicting the next block.
>
> ## Cons:
>
>
> * ?
> * Must be first be programmed.
> * Anything else?
>
> ## Solution operation:
>
>
> As I have said, my simplistic view of the operation. If I have this 
> wrong, please correct it back to the list.
>
>
> 1. The protocol determines the target block size.
>
> 2. Assign each transaction in the pool a transaction weight based on 
> fee and time waiting in the transaction pool.
>
> 3. Begin selecting transactions to include in the current block using 
> transaction weight as the likelihood of inclusion until target block 
> size is met.
>
> 4. Solve block.
>
> Regards,
>
> Damian Williamson
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


[-- Attachment #2: Type: text/html, Size: 5533 bytes --]

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[ZmnSCPxj]

[-- Attachment #1: Type: text/plain, Size: 4125 bytes --]

Good morning Damian,

The primary problem in your proposal, as I understand it, is that the "transaction pool" is never committed to and is not part of consensus currently.  It is unlikely that the transaction pool will ever be part of consensus, as putting the transaction pool into consensus is effectively lifting the block size to include the entire transaction pool into each block.  The issue is that the transaction pool is transient and future fullnodes cannot find the contents of the transaction pool at the time blocks were made and cannot verify the correctness of historical blocks.  Also, fullnodes using -blocksonly mode have no transaction pool and cannot verify incoming blocks for these new rules.

Applying a patch that follows this policy into Bitcoin Core without enforcing it in all fullnodes will simply lead to miners removing the patch in software they run, making it an exercise in futility to write, review, and test this code in the first place.

In addition, you reuse the term "weight" for something different than its current use.  Current use, is that the "weight" of a transaction, is the computed weight from the SegWit weight equation, measured in virtual units called "sipa", using the equation (4sipa / non-witness byte + 1sipa/witness byte).

Regards,
ZmnSCPxj

Sent with [ProtonMail](https://protonmail.com) Secure Email.

> -------- Original Message --------
> Subject: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks
> Local Time: December 6, 2017 3:38 AM
> UTC Time: December 5, 2017 7:38 PM
> From: bitcoin-dev@lists.linuxfoundation.org
> To: bitcoin-dev@lists.linuxfoundation.org <bitcoin-dev@lists.linuxfoundation.org>
>
> # BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks
>
> I admit, with my limited experience in the operation of the protocol, the section entitled 'Solution operation' may not be entirely correct but you will get the idea. If I have it wrong, please correct it back to the list.
>
> ## The problem:
>
> Everybody wants value. Miners want to maximize revenue from fees. Consumers want transaction reliability and, (we presume) low fees.
>
> Current transaction bandwidth limit is a limiting factor for both.
>
> ## Solution summary:
>
> Provide each transaction with a transaction weight, being a function of the fee paid (on a curve), and the time waiting in the transaction pool (also on a curve) out to n days (n=30 ?); the transaction weight serving as the likelihood of a transaction being included in the current block, and then use a target block size.
>
> Protocol enforcement to prevent high or low blocksize cheating to be handled by having the protocol determine the target size for the current block using; current transaction pool size x ( 1 / (144 x n days ) ) = transactions to be included in the current block.
>
> The curves used for the weight of transactions would have to be appropriate.
>
> ## Pros:
>
> * Maximizes transaction reliability.
> * Maximizes possibility for consumer and business uptake.
> * Maximizes total fees paid per block without reducing reliability; because of reliability, confidence and uptake are greater; therefore, more transactions and more transactions total at priority fees.
> * Market determines fee paid for transaction priority.
>
> * Fee recommendations work all the way out to 30 days or greater.
>
> * Provides additional block entropy and greater security since there is less probability of predicting the next block.
>
> ## Cons:
>
> * ?
> * Must be first be programmed.
> * Anything else?
>
> ## Solution operation:
>
> As I have said, my simplistic view of the operation. If I have this wrong, please correct it back to the list.
>
> 1. The protocol determines the target block size.
>
> 2. Assign each transaction in the pool a transaction weight based on fee and time waiting in the transaction pool.
>
> 3. Begin selecting transactions to include in the current block using transaction weight as the likelihood of inclusion until target block size is met.
>
> 4. Solve block.
>
> Regards,
>
> Damian Williamson

[-- Attachment #2: Type: text/html, Size: 5966 bytes --]

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson]

[-- Attachment #1: Type: text/plain, Size: 7384 bytes --]

Good afternoon ZmnSCPxj,


I have posted some discussion on the need for this proposal and, some refinements to the proposal explanation (not changes to the intended operation) to the bitcoin-discuss list. I didn't exactly mean to double post but thought it could use the discussion and, not to post it again, I will link to it when (if) it turns up, or will post it back here as an update on request. Currently, that post is awaiting moderator approval. I have also rewritten the solution operation section a bit in that post, not the idea that is being conveyed. I have added an additional step, reject blocks that do not meet the target block size for the current block.

I suggest it still should be added to the solution operation, to broadcast the next target block size with the current block when it is solved. Using that method may answer a part of your concern.

As I understand it, each node would be aware independently of x transactions waiting for confirmation, the transaction pool. Each node would no doubt have its own idea about how many waiting transactions there are and which particular transactions exist. I do not see why each node could not just work with the information at hand, it is my understanding that is what happens now, even with solved blocks vs the longest chain. I have not followed why you foresee from my proposal the need for fullnodes to back confirm the previous blocks in that manner.

If next blocksize is broadcast with the completed block it would be a simple matter to back confirm that. With transaction weight (transaction priority) I am suggesting that value gives the likelihood of a transaction being included, presuming an element of randomness as to whether any particular transaction is then included or not. Back confirmations on a transaction basis would be impossible anyway, all that could be confirmed is that a particular block has transactions that conform to a probability curve, if the variables are known, fee amount and time waiting in the pool, then the transaction priority can be recreated to establish that the probability of a particular block conforms. I certainly do not foresee including the full transaction pool in each block.

I am also presuming blocksize as a number of transactions rather than KB.

My suggestion, if adopted, is to directly make the operation of transaction priority a part of the operational standard - even without verification that it is being followed. The result of full transactional reliability is actually in the interests of miners as much as anyone.

The benefit of the proposal, not directly stated, is variable sized blocks (uncapped block size).

Yes, I have learned not to recycle terminology. My apologies, I had not been made aware that terminology already had use. Perhaps it would be simpler to call the proposal that I am putting forward here Transaction Priority.

Regards,
Damian Williamson


________________________________
From: ZmnSCPxj <ZmnSCPxj@protonmail.com>
Sent: Wednesday, 6 December 2017 4:46:45 PM
To: Damian Williamson
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

Good morning Damian,

The primary problem in your proposal, as I understand it, is that the "transaction pool" is never committed to and is not part of consensus currently.  It is unlikely that the transaction pool will ever be part of consensus, as putting the transaction pool into consensus is effectively lifting the block size to include the entire transaction pool into each block.  The issue is that the transaction pool is transient and future fullnodes cannot find the contents of the transaction pool at the time blocks were made and cannot verify the correctness of historical blocks.  Also, fullnodes using -blocksonly mode have no transaction pool and cannot verify incoming blocks for these new rules.

Applying a patch that follows this policy into Bitcoin Core without enforcing it in all fullnodes will simply lead to miners removing the patch in software they run, making it an exercise in futility to write, review, and test this code in the first place.

In addition, you reuse the term "weight" for something different than its current use.  Current use, is that the "weight" of a transaction, is the computed weight from the SegWit weight equation, measured in virtual units called "sipa", using the equation (4sipa / non-witness byte + 1sipa/witness byte).

Regards,
ZmnSCPxj




Sent with ProtonMail<https://protonmail.com> Secure Email.

-------- Original Message --------
Subject: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks
Local Time: December 6, 2017 3:38 AM
UTC Time: December 5, 2017 7:38 PM
From: bitcoin-dev@lists.linuxfoundation.org
To: bitcoin-dev@lists.linuxfoundation.org <bitcoin-dev@lists.linuxfoundation.org>



# BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

I admit, with my limited experience in the operation of the protocol, the section entitled 'Solution operation' may not be entirely correct but you will get the idea. If I have it wrong, please correct it back to the list.



## The problem:


Everybody wants value. Miners want to maximize revenue from fees. Consumers want transaction reliability and, (we presume) low fees.

Current transaction bandwidth limit is a limiting factor for both.



## Solution summary:


Provide each transaction with a transaction weight, being a function of the fee paid (on a curve), and the time waiting in the transaction pool (also on a curve) out to n days (n=30 ?); the transaction weight serving as the likelihood of a transaction being included in the current block, and then use a target block size.

Protocol enforcement to prevent high or low blocksize cheating to be handled by having the protocol determine the target size for the current block using; current transaction pool size x ( 1 / (144 x n days ) ) = transactions to be included in the current block.

The curves used for the weight of transactions would have to be appropriate.



## Pros:

* Maximizes transaction reliability.
* Maximizes possibility for consumer and business uptake.
* Maximizes total fees paid per block without reducing reliability; because of reliability, confidence and uptake are greater; therefore, more transactions and more transactions total at priority fees.
* Market determines fee paid for transaction priority.


* Fee recommendations work all the way out to 30 days or greater.

* Provides additional block entropy and greater security since there is less probability of predicting the next block.



## Cons:

* ?
* Must be first be programmed.
* Anything else?



## Solution operation:


As I have said, my simplistic view of the operation. If I have this wrong, please correct it back to the list.

1. The protocol determines the target block size.


2. Assign each transaction in the pool a transaction weight based on fee and time waiting in the transaction pool.

3. Begin selecting transactions to include in the current block using transaction weight as the likelihood of inclusion until target block size is met.

4. Solve block.



Regards,

Damian Williamson


[-- Attachment #2: Type: text/html, Size: 10550 bytes --]

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson]

[-- Attachment #1: Type: text/plain, Size: 4395 bytes --]

Hello Jim,


The variable block sizes would not, as I understand it, be easily implemented by a solo miner.


You are right, there is presently nothing stopping a miner from ordering the transactions included by a priority that is not entirely based on the fee.


It may be possible to verify blocks conform to the proposal by showing that the probability for all transactions included in the block statistically conform to a probability distribution curve, if that is necessary and,  *if* the individual transaction priority can be recreated. I am not that deep into the mathematics, however, it may also be possible to use a similar method to do this just based on the fee, that statistically, the blocks conform to a fee distribution. Needs a clever mathematician.


It is certainly possible to verify that blocks conform to the expected size.


Honour is why people follow policy without enforcement. I may be in the wrong group. (sic)


Regards,

Damian Williamson

________________________________
From: bitcoin-dev-bounces@lists.linuxfoundation.org <bitcoin-dev-bounces@lists.linuxfoundation.org> on behalf of Jim Renkel via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Wednesday, 6 December 2017 4:18:11 PM
To: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks


As i understand it, the transactions to be included in a block are entirely up to the miner of that block.


What prevents a miner from implementing the proposal on their own?


If this is adopted as some kind of "policy", what forces a miner to follow it?

Jim Renkel


On 12/2/2017 10:07 PM, Damian Williamson via bitcoin-dev wrote:

# BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

I admit, with my limited experience in the operation of the protocol, the section entitled 'Solution operation' may not be entirely correct but you will get the idea. If I have it wrong, please correct it back to the list.


## The problem:


Everybody wants value. Miners want to maximize revenue from fees. Consumers want transaction reliability and, (we presume) low fees.

Current transaction bandwidth limit is a limiting factor for both.


## Solution summary:


Provide each transaction with a transaction weight, being a function of the fee paid (on a curve), and the time waiting in the transaction pool (also on a curve) out to n days (n=30 ?); the transaction weight serving as the likelihood of a transaction being included in the current block, and then use a target block size.

Protocol enforcement to prevent high or low blocksize cheating to be handled by having the protocol determine the target size for the current block using; current transaction pool size x ( 1 / (144 x n days ) ) = transactions to be included in the current block.

The curves used for the weight of transactions would have to be appropriate.


## Pros:

* Maximizes transaction reliability.
* Maximizes possibility for consumer and business uptake.
* Maximizes total fees paid per block without reducing reliability; because of reliability, confidence and uptake are greater; therefore, more transactions and more transactions total at priority fees.
* Market determines fee paid for transaction priority.

* Fee recommendations work all the way out to 30 days or greater.

* Provides additional block entropy and greater security since there is less probability of predicting the next block.


## Cons:

* ?
* Must be first be programmed.
* Anything else?


## Solution operation:


As I have said, my simplistic view of the operation. If I have this wrong, please correct it back to the list.

1. The protocol determines the target block size.

2. Assign each transaction in the pool a transaction weight based on fee and time waiting in the transaction pool.

3. Begin selecting transactions to include in the current block using transaction weight as the likelihood of inclusion until target block size is met.

4. Solve block.


Regards,

Damian Williamson



_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org<mailto:bitcoin-dev@lists.linuxfoundation.org>
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev



[-- Attachment #2: Type: text/html, Size: 7235 bytes --]

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[ZmnSCPxj]

[-- Attachment #1: Type: text/plain, Size: 2084 bytes --]

Good morning Damian,

>As I understand it, each node would be aware independently of x transactions waiting for confirmation, the transaction pool.

Each long-running node would have a view that is roughly the same as the view of every other long-running node.

However, suppose a node, Sleeping Beauty, was temporarily stopped for a day (for various reasons) then is started again.  That node cannot verify what the "consensus" transaction pool was during the time it was stopped -- it has no view of that.  It can only trust that the longest chain is valid -- but that means it is SPV for this particular rule.

>If next blocksize is broadcast with the completed block it would be a simple matter to back confirm that.

It would not. Suppose Sleeping Beauty slept at block height 500,000.  On awakening, some node provides some purported block at height 500,001.  This block indicates some "next blocksize" for the block at height 500,002.  How does Sleeping Beauty know that the transaction pool at block 500,001 was of the correct size to provide the given "next blocksize"?  The only way, would be to look if there is some other chain with greater height that includes or does not include that block: that is, SPV confirmation.

How does Sleeping Beauty know it has caught up, and that its transaction pool is similar to that of its neighbors (who might be lying to it, for that matter), and that it should therefore stop using SPV confirmation and switch to strict fullnode rejection of blocks that indicate a "next blocksize" that is too large or too small according to your equation?  OR will it simply follow the longest chain always, in which case, it trusts miners to be honest about how loaded (or unloaded) the transaction pool is?

-------

As a general rule, consensus rules should restrict themselves to:

1.  The characteristics of the block.
2.  The characteristics of the transactions within the block.

The transaction pool is specifically those transaction that are NOT in any block, and thus, are not safe to depend on for any consensus rules.

Regards,
ZmnSCPxj

[-- Attachment #2: Type: text/html, Size: 2501 bytes --]

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson]

[-- Attachment #1: Type: text/plain, Size: 3773 bytes --]

Good morning ZmnSCPxj, it must be where you are,


I suppose that we are each missing each other's point some.


I understand that nodes would not be expected to agree on the transaction pool and do not propose validating that the correct transactions are included in a block. I speak of probability and likelihood of a transaction being included in a block, implying a random element. I do not propose rejecting blocks on the basis that the next block size is stated too large or too small for the transaction pool, only that the block received conforms to the next block size given on the previous block. Yes, it could be cheated. Also, various nodes may have at times wildly different amounts of transactions waiting in the transaction pool compared to each other and there could be a great disparity between them. It would not be possible in any case I can think of to validate the next block size is correct for the current transaction pool. Even as it is now, nodes may include transactions in a block that no other nodes have even heard of, nodes have no way to validate that either. If the block is built on sufficiently, it is the blockchain.


I will post back the revised proposal to the list. I have fleshed parts of it out more, given more explanation and, tried this time not to recycle terminology.


Regards,

Damian Williamson

________________________________
From: ZmnSCPxj <ZmnSCPxj@protonmail.com>
Sent: Thursday, 7 December 2017 5:46:08 PM
To: Damian Williamson
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

Good morning Damian,

>As I understand it, each node would be aware independently of x transactions waiting for confirmation, the transaction pool.

Each long-running node would have a view that is roughly the same as the view of every other long-running node.

However, suppose a node, Sleeping Beauty, was temporarily stopped for a day (for various reasons) then is started again.  That node cannot verify what the "consensus" transaction pool was during the time it was stopped -- it has no view of that.  It can only trust that the longest chain is valid -- but that means it is SPV for this particular rule.

>If next blocksize is broadcast with the completed block it would be a simple matter to back confirm that.

It would not. Suppose Sleeping Beauty slept at block height 500,000.  On awakening, some node provides some purported block at height 500,001.  This block indicates some "next blocksize" for the block at height 500,002.  How does Sleeping Beauty know that the transaction pool at block 500,001 was of the correct size to provide the given "next blocksize"?  The only way, would be to look if there is some other chain with greater height that includes or does not include that block: that is, SPV confirmation.

How does Sleeping Beauty know it has caught up, and that its transaction pool is similar to that of its neighbors (who might be lying to it, for that matter), and that it should therefore stop using SPV confirmation and switch to strict fullnode rejection of blocks that indicate a "next blocksize" that is too large or too small according to your equation?  OR will it simply follow the longest chain always, in which case, it trusts miners to be honest about how loaded (or unloaded) the transaction pool is?

-------

As a general rule, consensus rules should restrict themselves to:

1.  The characteristics of the block.
2.  The characteristics of the transactions within the block.

The transaction pool is specifically those transaction that are NOT in any block, and thus, are not safe to depend on for any consensus rules.

Regards,
ZmnSCPxj


[-- Attachment #2: Type: text/html, Size: 5434 bytes --]

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Damian Williamson]

[-- Attachment #1: Type: text/plain, Size: 5090 bytes --]

Good morning ZmnSCPxj,


Actually, there is no incentive to cheat target block size by providing a next block size that is higher or lower than the proposal would give. Under the proposal the transaction pool can grow quite large. A low next block size just defers collecting transaction fees, while a high next block size shrinks the transaction pool and thereby lowers fees. It seems like a standoff. This is especially true if the curve for time waiting in the transaction pool is extended beyond n days, since it is a curve, after waiting longer than 60 days (if n = 60 days) a transaction would have a priority greater than one-hundred and would therfore be the first transaction included with no possibility of failing the likelihood, so, even low fee paying transactions would be included first if the pool size is growing through incorrectly providing the next block size.


As it is now, I presume, a miner could include exactly one transaction in a block and pad?


Regards,

Damian Williamson

________________________________
From: Damian Williamson <willtech@live.com.au>
Sent: Thursday, 7 December 2017 7:13:14 PM
To: ZmnSCPxj
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks


Good morning ZmnSCPxj, it must be where you are,


I suppose that we are each missing each other's point some.


I understand that nodes would not be expected to agree on the transaction pool and do not propose validating that the correct transactions are included in a block. I speak of probability and likelihood of a transaction being included in a block, implying a random element. I do not propose rejecting blocks on the basis that the next block size is stated too large or too small for the transaction pool, only that the block received conforms to the next block size given on the previous block. Yes, it could be cheated. Also, various nodes may have at times wildly different amounts of transactions waiting in the transaction pool compared to each other and there could be a great disparity between them. It would not be possible in any case I can think of to validate the next block size is correct for the current transaction pool. Even as it is now, nodes may include transactions in a block that no other nodes have even heard of, nodes have no way to validate that either. If the block is built on sufficiently, it is the blockchain.


I will post back the revised proposal to the list. I have fleshed parts of it out more, given more explanation and, tried this time not to recycle terminology.


Regards,

Damian Williamson

________________________________
From: ZmnSCPxj <ZmnSCPxj@protonmail.com>
Sent: Thursday, 7 December 2017 5:46:08 PM
To: Damian Williamson
Cc: bitcoin-dev@lists.linuxfoundation.org
Subject: Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

Good morning Damian,

>As I understand it, each node would be aware independently of x transactions waiting for confirmation, the transaction pool.

Each long-running node would have a view that is roughly the same as the view of every other long-running node.

However, suppose a node, Sleeping Beauty, was temporarily stopped for a day (for various reasons) then is started again.  That node cannot verify what the "consensus" transaction pool was during the time it was stopped -- it has no view of that.  It can only trust that the longest chain is valid -- but that means it is SPV for this particular rule.

>If next blocksize is broadcast with the completed block it would be a simple matter to back confirm that.

It would not. Suppose Sleeping Beauty slept at block height 500,000.  On awakening, some node provides some purported block at height 500,001.  This block indicates some "next blocksize" for the block at height 500,002.  How does Sleeping Beauty know that the transaction pool at block 500,001 was of the correct size to provide the given "next blocksize"?  The only way, would be to look if there is some other chain with greater height that includes or does not include that block: that is, SPV confirmation.

How does Sleeping Beauty know it has caught up, and that its transaction pool is similar to that of its neighbors (who might be lying to it, for that matter), and that it should therefore stop using SPV confirmation and switch to strict fullnode rejection of blocks that indicate a "next blocksize" that is too large or too small according to your equation?  OR will it simply follow the longest chain always, in which case, it trusts miners to be honest about how loaded (or unloaded) the transaction pool is?

-------

As a general rule, consensus rules should restrict themselves to:

1.  The characteristics of the block.
2.  The characteristics of the transactions within the block.

The transaction pool is specifically those transaction that are NOT in any block, and thus, are not safe to depend on for any consensus rules.

Regards,
ZmnSCPxj


[-- Attachment #2: Type: text/html, Size: 7723 bytes --]

Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction Weight For Ordering Transactions In Blocks

[Erik Aronesty]

[-- Attachment #1: Type: text/plain, Size: 5656 bytes --]

You can feel free to write this version and try to get miners to use it.
 That's the nice thing about Bitcoin.

On Thu, Dec 7, 2017 at 3:49 PM, Damian Williamson via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Good morning ZmnSCPxj,
>
>
> Actually, there is no incentive to cheat target block size by providing a
> next block size that is higher or lower than the proposal would give. Under
> the proposal the transaction pool can grow quite large. A low next block
> size just defers collecting transaction fees, while a high next block size
> shrinks the transaction pool and thereby lowers fees. It seems like a
> standoff. This is especially true if the curve for time waiting in the
> transaction pool is extended beyond n days, since it is a curve, after
> waiting longer than 60 days (if n = 60 days) a transaction would have a
> priority greater than one-hundred and would therfore be the first
> transaction included with no possibility of failing the likelihood, so,
> even low fee paying transactions would be included first if the pool size
> is growing through incorrectly providing the next block size.
>
>
> As it is now, I presume, a miner could include exactly one transaction in
> a block and pad?
>
>
> Regards,
>
> Damian Williamson
> ------------------------------
> *From:* Damian Williamson <willtech@live.com.au>
> *Sent:* Thursday, 7 December 2017 7:13:14 PM
> *To:* ZmnSCPxj
>
> *Cc:* bitcoin-dev@lists.linuxfoundation.org
> *Subject:* Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction
> Weight For Ordering Transactions In Blocks
>
>
> Good morning ZmnSCPxj, it must be where you are,
>
>
> I suppose that we are each missing each other's point some.
>
>
> I understand that nodes would not be expected to agree on the transaction
> pool and do not propose validating that the correct transactions are
> included in a block. I speak of probability and likelihood of a transaction
> being included in a block, implying a random element. I do not propose
> rejecting blocks on the basis that the next block size is stated too large
> or too small for the transaction pool, only that the block received
> conforms to the next block size given on the previous block. Yes, it could
> be cheated. Also, various nodes may have at times wildly different amounts
> of transactions waiting in the transaction pool compared to each other and
> there could be a great disparity between them. It would not be possible in
> any case I can think of to validate the next block size is correct for the
> current transaction pool. Even as it is now, nodes may include transactions
> in a block that no other nodes have even heard of, nodes have no way to
> validate that either. If the block is built on sufficiently, it is the
> blockchain.
>
>
> I will post back the revised proposal to the list. I have fleshed parts of
> it out more, given more explanation and, tried this time not to recycle
> terminology.
>
>
> Regards,
>
> Damian Williamson
> ------------------------------
> *From:* ZmnSCPxj <ZmnSCPxj@protonmail.com>
> *Sent:* Thursday, 7 December 2017 5:46:08 PM
> *To:* Damian Williamson
> *Cc:* bitcoin-dev@lists.linuxfoundation.org
> *Subject:* Re: [bitcoin-dev] BIP Proposal: UTWFOTIB - Use Transaction
> Weight For Ordering Transactions In Blocks
>
> Good morning Damian,
>
> >As I understand it, each node would be aware independently of x
> transactions waiting for confirmation, the transaction pool.
>
> Each long-running node would have a view that is roughly the same as the
> view of every other long-running node.
>
> However, suppose a node, Sleeping Beauty, was temporarily stopped for a
> day (for various reasons) then is started again.  That node cannot verify
> what the "consensus" transaction pool was during the time it was stopped --
> it has no view of that.  It can only trust that the longest chain is valid
> -- but that means it is SPV for this particular rule.
>
> >If next blocksize is broadcast with the completed block it would be a
> simple matter to back confirm that.
>
> It would not. Suppose Sleeping Beauty slept at block height 500,000.  On
> awakening, some node provides some purported block at height 500,001.  This
> block indicates some "next blocksize" for the block at height 500,002.  How
> does Sleeping Beauty know that the transaction pool at block 500,001 was of
> the correct size to provide the given "next blocksize"?  The only way,
> would be to look if there is some other chain with greater height that
> includes or does not include that block: that is, SPV confirmation.
>
> How does Sleeping Beauty know it has caught up, and that its transaction
> pool is similar to that of its neighbors (who might be lying to it, for
> that matter), and that it should therefore stop using SPV confirmation and
> switch to strict fullnode rejection of blocks that indicate a "next
> blocksize" that is too large or too small according to your equation?  OR
> will it simply follow the longest chain always, in which case, it trusts
> miners to be honest about how loaded (or unloaded) the transaction pool is?
>
> -------
>
> As a general rule, consensus rules should restrict themselves to:
>
> 1.  The characteristics of the block.
> 2.  The characteristics of the transactions within the block.
>
> The transaction pool is specifically those transaction that are NOT in any
> block, and thus, are not safe to depend on for any consensus rules.
>
> Regards,
> ZmnSCPxj
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 8704 bytes --]