[bitcoin-dev] New difficulty algorithm part 2

[bitcoin-dev] New difficulty algorithm part 2

[Scott Roberts]

(This is new thread because I'm having trouble getting yahoo mail 

to use "reply-to", copy-pasting the subject did not work, and the 
list has not approved my gmail)
A hard fork in the near term is feasible only post-disaster (in my mind, 
that means Core failing from long transaction delays that destroys 
confidence and therefore price). A hard fork attempt to fix the situation 
will not work unless the difficulty is fixed to let price guide hash power 
instead of vice versa. We seem to be headed towards letting the tail wag 
the dog. BTC may find itself in the same position as BCH and all alts: the 
current difficulty algorithm is untenable and will require a fork. 

Current difficulty algorithm in presence of higher hashrate coin with 
the same POW: 
lower hashpower => wait times => lost confidence => lower price => defeat 

Difficulty algorithms that alts find absolutely necessary when there 
is a higher hash rate coin with the same POW: 
hodler faith => price => hashpower => survivable coin 

Alt experience time and time again is that Core will have to fork to a 
faster responding difficulty algorithm if it finds itself suddenly (and 
for the first time) with a lower hashrate. 


Mark Friedenbach wrote: 
> changing the difficulty adjustment algorithm doesn’t solve the underlying 
> issue, hashpower not being aligned with users’ (or owners') interests. 

I define "users" as those who it it for value transfer (including 
purchases) without concern for long-term value. If SegWit2x reduces fees 
per coin, then hashpower is being aligned with their short-term interests. 

It does not solve it, but it is a pre-requisite if the coin has a lower 
hashrate (BTC at end of November). A faster responding diffulty is a 
pre-requisite in minority hashrate coins for letting price (hodlers) 
dictate hashpower instead of vice versa. This is the experience of alts. 

ZmnSCPxj wrote: 
> Hodlers have much greater power in hardfork situations than miners 

Not when hodlers are more evenly split between coins. Miners will prefer 
the coin with higher transaction fees which will erode hodler confidence 
via longer delays. This means transaction fees will evolve to the highest 
that common marketplace users can accepet (they are not intereseted in 
hodler security), not the lowest technologically feasible fee that provides 
the greatest security. Large blocks reduce network security while giving 
the higher total transaction fees to miners even as it can reduce fees per 
coin for users. The mining "lobby" will always describe this as "best for 
users". Non-hodling users and miners logically prefer SegWit2x. 

ZmnSCPxj wrote: 
> BCH changed its difficulty algorithm, and it is often considered to be to 
its detriment due to sudden hashpower oscillations 

BCH has survived this long because they did NOT use the bitcoin difficulty 
algorithm. Granted, it is a bad design that included an asymmetry that has 
resulted in too many coins being issued. If they had inverted the decrease 
rule to create a symmetrically fast increase rule instead of keeping 
bitcoin's increase logic, they would be in much better shape, much better 
than the bitcoin difficulty algorithm. Making it symmetrical and fast would 
have resulted in more obvious fast oscillations, but this would have helped 
price discovery to settle the oscillations to an acceptable level that 
could stabilize the price by preventing too many coins from being issued. 

Oscillations require: 1) comparable price and 2) miners having the option 
to go back and forth to a larger coin. Bitcoin's long, jumping difficulty 
averaging window may destroy the minority hashrate coin faster in fewer 
oscillations thanks to a first-to-market effect more than reason. In 
persuit of higher total transacton fees, miners are deciding SegWit2x is 
"first-to-market" to cause Core to have long delays. This is not a 
conspiracy, but simply seeking profit. Since fees per coin can also be 
reduced, they can convince themselves and others that it is the best 
option. 

A shorter difficulty algorithm averaging window enables more, faster 
oscillations to enable better price discovery before a winner is chosen. 
The design I'm proposing should be close to the ideal.  For example, Mark 
Friedenbach suggested a difficulty adjustment every 18 blocks by averaging 
the past 36 blocks. If a coin using that has the minority hashrate, then it 
could quickly develop into a sudden influx from the majority change for 18 
blocks, then they exit back to the majority chain for 36 blocks before 
doing it again. They get 1/3 of the blocks at "zero excess cost" 
(difficulty will be 1/10 the correct value if they are 10x base hashrate) 
and then they will leave the constant miners with a higher difficulty for 
36 blocks (at 3.33x higher difficulty if the "attackers" are 10x the base 
hashrate). This forces constant miners to start copying them, amplifying 
the oscillations and delays of the minority hashrate coin. A rolling 
average window of any length does not theoretically prevent this, unless 
the window is short enough to be comparable to the time cost of switching 
coins, if there is a time cost. A say this because in testing I was able 
to design an attack algorithm that always gets 1/3 of the coins at "zero 
excess cost".  But a rolling average with a shorter window should make the 
"accidental collusion" of miners seeking profit more unlikely to occur. 
The reward function I've proposed appears to reduce it to 1/6 total coins 
obtainable at "zero excess cost", and similarly reduce oscillations and 
assist better price discovery.

Re: [bitcoin-dev] New difficulty algorithm part 2

[ZmnSCPxj]

[-- Attachment #1: Type: text/plain, Size: 3823 bytes --]

Good morning,

>ZmnSCPxj wrote:
>> Hodlers have much greater power in hardfork situations than miners
>
>Not when hodlers are more evenly split between coins. Miners will prefer
>the coin with higher transaction fees which will erode hodler confidence
>via longer delays. This means transaction fees will evolve to the highest
>that common marketplace users can accepet (they are not intereseted in
>hodler security), not the lowest technologically feasible fee that provides
>the greatest security. Large blocks reduce network security while giving
>the higher total transaction fees to miners even as it can reduce fees per
>coin for users. The mining "lobby" will always describe this as "best for
>users". Non-hodling users and miners logically prefer SegWit2x.

Hodlers still have greater power than non-hodling users, whether miners or day-traders.

Hodlers holding millions in coin, total, can greatly drop price of any undesired hardfork.

Miners will prefer the coin with higher transaction fees as measured in real-world value.  Thus even if the unwanted chain provides 2 tokens as fee per block, whereas the wanted chain provides 1 token as fee per block, if the unwanted chain tokens are valued at 1/4 the wanted chain tokens, miners will still prefer the wanted chain regardless.  What the chains will compete in is the real-world value of the total mining reward.

Hodlers hodl all the cards here.

>ZmnSCPxj wrote:
>> BCH changed its difficulty algorithm, and it is often considered to be to
>its detriment due to sudden hashpower oscillations
>
>BCH has survived this long because they did NOT use the bitcoin difficulty
<snip>

All this speculation seems to suggest to me simply that a difficulty change leads to keeping a chain alive unnecessarily, when by rights, it should be dead and laid to rest.

If Bitcoin needs some sudden change in difficulty algorithm to survive, then it has failed.  Feel free to do your own hardfork into yet another derivative altcoin.

>Bitcoin has developers!
>
>And those developers can publish a contingency plan!
>
>And that contingency plan can be an emergency hard fork to a different retarget algorithm.
>
>And that emergency hard fork can gain consensus if it is broadly preferred over the status quo.

Every hardfork is an invitation to shatter the community even further than it already is.  There is no need for further shattering.

The idea that an emergency hardfork to a different difficulty algorithm is necessary arises from a lack of understanding of just how much power hodlers have over the destiny of a coin.

Every hodler who rejects a hardfork, will sell the hardfork coin, increasing its supply and reducing its price.

Every miner who mines a rejected hardfork, creates new tokens in the hardfork, increasing its supply and reducing its price.

Coins that are hedl will remain hedl, and are not part of the supply.  Thus coins on the desired chain will remain high in price, regardless of available transaction rate.  The lack of freshly-minted coins also contracts the supply.

In the end, this will result in the same behavior as in BCH, where hodlers sodl the unwanted hardfork as quickly as they could.  Indeed, due exactly to miner support, 2X is much more likely to quickly drop in price than BCH did.  I am sure you have seen the images pointing out how easy it was to determine when BCH blocks arrived: they arrived a little before each dip in BCH price.  Fast 2X block rate will lead to faster 2X death, with its miners becoming bagholders.

As most Core developers hodl vast amounts, it is far more likely that any hardfork that goes against what Core wishes will collapse, simply by Core developers acting in their capacity as hodlers of Bitcoin, without needing to do any special action in their capacity as developers.

Regards,
ZmnSCPxj

[-- Attachment #2: Type: text/html, Size: 4815 bytes --]

Re: [bitcoin-dev] New difficulty algorithm part 2

[Mark Friedenbach]

> On Oct 12, 2017, at 3:40 AM, ZmnSCPxj via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
> As most Core developers hodl vast amounts, it is far more likely that any hardfork that goes against what Core wishes will collapse, simply by Core developers acting in their capacity as hodlers of Bitcoin, without needing to do any special action in their capacity as developers.

While this might be true of some, it is most certainly not true of many, and it is a very dangerous men to the safety and well being of people on this list.

You don’t get bitcoin for being a bitcoin developer, and there is no reason to suppose a developer has any more or less bitcoin than anyone else in the industry.

It is certainly the case that a large number of people and organizations who are not developers hold massive amounts of bitcoin (hundred of thousands each, millions in aggregate). 

Re: [bitcoin-dev] New difficulty algorithm part 2

[Scott Roberts]

Yes, the current price ratio indicates there is no need for a new
difficulty algorithm. I do not desire to fork before a disaster, or to
otherwise employ a new difficulty before a fork is otherwise needed.

A 2-week delay in difficulty response is a 2 week error in
measurement. Slow response generally means less intelligence.

My goal is not to have a bunch of BTC clones that merchants and buyers
use equally, but to have a  better difficulty algorithm in place to be
used in the next BTC "Core" fork. If not for the current situation,
then for future security.

>  This is a massive departure from the conception of Bitcoin as having a fixed limit and effectively becoming deflationary.

You mean multiple forks is inflationary. The current limit in quantity
is deflationary because the use of the coin is rising faster than its
mining is producing (see velocity of money). Constant value is defined
as being neither. Bitcoin's deflationary quality created a massive
marketing advantage as well as paid the creator about million dollars
an hour. If it suddenly were able to be a constant value coin, its use
in the marketplace and as a real store of value would skyrocket and
the cries of "Ponzi scheme" would stop. The trick is in determining
constant value without a 3rd party such as an index of a basket of
commodities (which both Keynes and von Mises wanted, but was scuttled
by the U.S. at Bretton Woods).

On Fri, Oct 13, 2017 at 12:45 AM, ZmnSCPxj <ZmnSCPxj@protonmail.com> wrote:
> Good morning,
>
>
>>ZmnSCPxj wrote:
>>> Thus even if the unwanted chain provides 2 tokens as fee per block,
>>> whereas the wanted chain provides 1 token as fee per block, if the
>>> unwanted chain tokens are valued at 1/4 the wanted chain tokens, miners
>>> will still prefer the wanted chain regardless.
>>
>>This is a good point I was not thinking about, but your math assumes
>>1/2 price for a coin that can do 2x more transactions. Holders like
>>Roger Ver have an interest in low price and more transactions. A coin
>>with 2x more transactions, 22% lower price, and 22% lower fees per
>>coin transferred will attract more merchants, customers, and miners
>>(they get 50% more total fees) and this will in turn attract more
>>hodlers and devs. This assumes it outweighs hodler security concerns.
>>Merchants and customers, to the extent they are not long term hodlers,
>>are not interested in price as much as stability, so they are somewhat
>>at odds with hodlers.
>
> As of this moment, BT1 / BT2 price ratio in BitFinex is slightly higher than
> 7 : 1.  Twice the transaction rate cannot overcome this price ratio
> difference.  Even if you were to claim that the BitFinex data is off by a
> factor of 3, twice the transaction rate still cannot overcome the price
> ratio difference.  Do you have stronger data than what is available on
> BitFinex?  If not, your assumptions are incorrect and all conclusions
> suspect.
>
>
>>Bitcoin consensus truth is based on "might is right". Buyers and
>>sellers of goods and services ("users") can shift some might to miners
>>via fees, to the chagrin of hodlers who have more interest in security
>>and price increases. Some hodlers think meeting user needs is the
>>source of long term value. Others think mining infrastructure is.
>
> Mining infrastructure follows price.  If bitcoins were still trading at 1
> USD per coin, nobody will build mining infrastructure to the same level as
> today, with 5000 USD per coin.
>
> Price will follow user needs, i.e. demand.
>
>>You
>>seem to require hodlers to correctly identify and rely solely on good
>>developers.
>
> For the very specific case of 2X, it is very easy to make this
> identification.  Even without understanding the work being done, one can
> reasonably say that it is far more likely that a loose group of 100 or more
> developers will contain a few good or excellent developers, than a group of
> a few developers containing a similar number of good or excellent
> developers.
>
> User needs will get met only on the chain that good developers work on.
> Bitcoin today has too many limitations: viruses on Windows can steal all
> your money, fee estimates consistently overestimate, fees rise during
> spamming attacks, easy to lose psuedonymity, tiny UTXOs are infeasible to
> spend, cannot support dozens of thousands of transactions per second.
> Rationally, long-term hodlers will select a chain with better developers who
> are more likely to discover or innovate methods to reduce, eliminate, or
> sidestep those limitations.  Perhaps the balance will change in the future,
> but it is certainly not the balance now, and thus any difficulty algorithm
> change in response to the current situation will be premature, and far more
> likely to cause disaster than avert one.
>
>>Whatever combination of these is the case, bad money can
>>still drive out good, especially if the market determination is not
>>efficient.
>>
>>A faster measurement of hashrate for difficulty enables the economic
>>determination to be more efficient and correct. It prevents the
>>biggest coin from bullying forks that have better ideas. Conversely,
>>it prevents miners from switching to an inferior coin simply because
>>it provides them with more "protection money" from fees that enables
>>them to bully Bitcoin Core out of existence, even in the presence of a
>>slightly larger hodler support.
>
> This requires that all chains follow the same difficulty adjustment: after
> all, it is also entirely the possibility that 2X will be the lower-hashrate
> coin in a few months, with the Core chain bullying them out of existence.
> Perhaps you should cross-post your analysis to bitcoin-segwit2x also.  After
> all, the 2X developers should also want to have faster price discovery of
> the true price of 2X, away from the unfavorable (incorrect?) pricing on
> BitFinex.
>
>>Devs are a governing authority under the influence of users, hodlers,
>>and miners. Miners are like banks lobbying government for higher total
>>fees. Hodlers are the new 1%, holding 90% of the coin, lobbying both
>>devs and users for security, but equally interested in price
>>increases. Users are "the people" that devs need to protect against
>>both hodlers and miners. They do not care about price as long as it is
>>stable. They do not want to become the 99% owning 10% of the coin or
>>have to pay unecessary fees merely for their coin to be the biggest
>>bully on the block. A faster responding difficulty will take a lot of
>>hot air out of the bully. It prevents miners from being able to
>>dictate that only coins with high fees are allowed. They are less
>>able to destroy small coins that have a fast defense.
>>
>>The 1% and banks would starve the people that feed them to death if
>>they were allowed complete control of the government. Are hodlers and
>>miners any wiser?
>
> Are developers any wiser, either?
>
> Then consider this wisdom: The fewer back-incompatible changes to a coin,
> the better.  Hardforks of any kind are an invitation to disaster and, at
> this point, require massive coordination effort which cannot be feasibly
> done within a month.  Fast market determination can be done using off-chain
> methods (such as on-exchange trades), and are generally robust against
> temporary problems on-chain, although admittedly there is a counterparty
> risk involved.  The coin works, and in general there is usually very little
> need to fix it, especially using dangerous hardforks.
>
>>Devs need to strive for an expansion of the coin
>>quantity to keep value constant which is the foundation of the 5
>>characteristics of an ideal currency.
>
> Is that your goal?  This is a massive departure from the conception of
> Bitcoin as having a fixed limit and effectively becoming deflationary.  It
> will also lead to massive economic distortions in favor of those who receive
> newly-minted coins.  I doubt any developer would want to have this property.
>
> Regards,
> ZmnSCPxj

Re: [bitcoin-dev] New difficulty algorithm part 2

[ZmnSCPxj]

[-- Attachment #1: Type: text/plain, Size: 6309 bytes --]

Good morning,

>ZmnSCPxj wrote:
>> Thus even if the unwanted chain provides 2 tokens as fee per block,
>> whereas the wanted chain provides 1 token as fee per block, if the
>> unwanted chain tokens are valued at 1/4 the wanted chain tokens, miners
>> will still prefer the wanted chain regardless.
>
>This is a good point I was not thinking about, but your math assumes
>1/2 price for a coin that can do 2x more transactions. Holders like
>Roger Ver have an interest in low price and more transactions. A coin
>with 2x more transactions, 22% lower price, and 22% lower fees per
>coin transferred will attract more merchants, customers, and miners
>(they get 50% more total fees) and this will in turn attract more
>hodlers and devs. This assumes it outweighs hodler security concerns.
>Merchants and customers, to the extent they are not long term hodlers,
>are not interested in price as much as stability, so they are somewhat
>at odds with hodlers.

As of this moment, BT1 / BT2 price ratio in BitFinex is slightly higher than 7 : 1.  Twice the transaction rate cannot overcome this price ratio difference.  Even if you were to claim that the BitFinex data is off by a factor of 3, twice the transaction rate still cannot overcome the price ratio difference.  Do you have stronger data than what is available on BitFinex?  If not, your assumptions are incorrect and all conclusions suspect.

>Bitcoin consensus truth is based on "might is right". Buyers and
>sellers of goods and services ("users") can shift some might to miners
>via fees, to the chagrin of hodlers who have more interest in security
>and price increases. Some hodlers think meeting user needs is the
>source of long term value. Others think mining infrastructure is.

Mining infrastructure follows price.  If bitcoins were still trading at 1 USD per coin, nobody will build mining infrastructure to the same level as today, with 5000 USD per coin.

Price will follow user needs, i.e. demand.

>You
>seem to require hodlers to correctly identify and rely solely on good
>developers.

For the very specific case of 2X, it is very easy to make this identification.  Even without understanding the work being done, one can reasonably say that it is far more likely that a loose group of 100 or more developers will contain a few good or excellent developers, than a group of a few developers containing a similar number of good or excellent developers.

User needs will get met only on the chain that good developers work on.  Bitcoin today has too many limitations: viruses on Windows can steal all your money, fee estimates consistently overestimate, fees rise during spamming attacks, easy to lose psuedonymity, tiny UTXOs are infeasible to spend, cannot support dozens of thousands of transactions per second.  Rationally, long-term hodlers will select a chain with better developers who are more likely to discover or innovate methods to reduce, eliminate, or sidestep those limitations.  Perhaps the balance will change in the future, but it is certainly not the balance now, and thus any difficulty algorithm change in response to the current situation will be premature, and far more likely to cause disaster than avert one.

>Whatever combination of these is the case, bad money can
>still drive out good, especially if the market determination is not
>efficient.
>
>A faster measurement of hashrate for difficulty enables the economic
>determination to be more efficient and correct. It prevents the
>biggest coin from bullying forks that have better ideas. Conversely,
>it prevents miners from switching to an inferior coin simply because
>it provides them with more "protection money" from fees that enables
>them to bully Bitcoin Core out of existence, even in the presence of a
>slightly larger hodler support.

This requires that all chains follow the same difficulty adjustment: after all, it is also entirely the possibility that 2X will be the lower-hashrate coin in a few months, with the Core chain bullying them out of existence.  Perhaps you should cross-post your analysis to bitcoin-segwit2x also.  After all, the 2X developers should also want to have faster price discovery of the true price of 2X, away from the unfavorable (incorrect?) pricing on BitFinex.

>Devs are a governing authority under the influence of users, hodlers,
>and miners. Miners are like banks lobbying government for higher total
>fees. Hodlers are the new 1%, holding 90% of the coin, lobbying both
>devs and users for security, but equally interested in price
>increases. Users are "the people" that devs need to protect against
>both hodlers and miners. They do not care about price as long as it is
>stable. They do not want to become the 99% owning 10% of the coin or
>have to pay unecessary fees merely for their coin to be the biggest
>bully on the block. A faster responding difficulty will take a lot of
>hot air out of the bully. It prevents miners from being able to
>dictate that only coins with high fees are allowed. They are less
>able to destroy small coins that have a fast defense.
>
>The 1% and banks would starve the people that feed them to death if
>they were allowed complete control of the government. Are hodlers and
>miners any wiser?

Are developers any wiser, either?

Then consider this wisdom: The fewer back-incompatible changes to a coin, the better.  Hardforks of any kind are an invitation to disaster and, at this point, require massive coordination effort which cannot be feasibly done within a month.  Fast market determination can be done using off-chain methods (such as on-exchange trades), and are generally robust against temporary problems on-chain, although admittedly there is a counterparty risk involved.  The coin works, and in general there is usually very little need to fix it, especially using dangerous hardforks.

>Devs need to strive for an expansion of the coin
>quantity to keep value constant which is the foundation of the 5
>characteristics of an ideal currency.

Is that your goal?  This is a massive departure from the conception of Bitcoin as having a fixed limit and effectively becoming deflationary.  It will also lead to massive economic distortions in favor of those who receive newly-minted coins.  I doubt any developer would want to have this property.

Regards,
ZmnSCPxj

[-- Attachment #2: Type: text/html, Size: 7749 bytes --]

Re: [bitcoin-dev] New difficulty algorithm part 2

[Scott Roberts]

ZmnSCPxj wrote:
> Thus even if the unwanted chain provides 2 tokens as fee per block,
> whereas the wanted chain provides 1 token as fee per block, if the
> unwanted chain tokens are valued at 1/4 the wanted chain tokens, miners
> will still prefer the wanted chain regardless.

This is a good point I was not thinking about, but your math assumes
1/2 price for a coin that can do 2x more transactions. Holders like
Roger Ver have an interest in low price and more transactions. A coin
with 2x more transactions, 22% lower price, and 22% lower fees per
coin transferred will attract more merchants, customers, and miners
(they get 50% more total fees) and this will in turn attract more
hodlers and devs. This assumes it outweighs hodler security concerns.
Merchants and customers, to the extent they are not long term hodlers,
are not interested in price as much as stability, so they are somewhat
at odds with hodlers.

Bitcoin consensus truth is based on "might is right". Buyers and
sellers of goods and services ("users") can shift some might to miners
via fees, to the chagrin of hodlers who have more interest in security
and price increases. Some hodlers think meeting user needs is the
source of long term value. Others think mining infrastructure is. You
seem to require hodlers to correctly identify and rely solely on good
developers. Whatever combination of these is the case, bad money can
still drive out good, especially if the market determination is not
efficient.

A faster measurement of hashrate for difficulty enables the economic
determination to be more efficient and correct. It prevents the
biggest coin from bullying forks that have better ideas. Conversely,
it prevents miners from switching to an inferior coin simply because
it provides them with more "protection money" from fees that enables
them to bully Bitcoin Core out of existence, even in the presence of a
slightly larger hodler support.

Devs are a governing authority under the influence of users, hodlers,
and miners. Miners are like banks lobbying government for higher total
fees. Hodlers are the new 1%, holding 90% of the coin, lobbying both
devs and users for security, but equally interested in price
increases. Users are "the people" that devs need to protect against
both hodlers and miners. They do not care about price as long as it is
stable. They do not want to become the 99% owning 10% of the coin or
have to pay unecessary fees merely for their coin to be the biggest
bully on the block.  A faster responding difficulty will take a lot of
hot air out of the bully. It prevents miners from being able to
dictate that only coins with high fees are allowed.  They are less
able to destroy small coins that have a fast defense.

The 1% and banks would starve the people that feed them to death if
they were allowed complete control of the government. Are hodlers and
miners any wiser? Devs need to strive for an expansion of the coin
quantity to keep value constant which is the foundation of the 5
characteristics of an ideal currency.  Therefore devs should seek
peaceful and sustainable forks of bitcoin. This will enable constant
value, security, and low transaction fees per coin transfer. Alts
aside, the current situation of discouraging forks forbids constant
value via limited quantity.  It also forces a choice between high
security and low fees.  Forks with a faster difficulty will be more
capable of retaining value.

Users, devs, hodlers, and miners are naturally aligned and at odds in
different ways. A flow chart of the checks and balances should enable
better development towards a self-controlling feedback system, but the
goals need to be known before it could be designed and implemented.
Hodlers say price increases is the goal. Users say efficient transfer
of value. Miners say fees (at least that's the end game after mining).
I'm with users despite trying to be the 1% (which reminds me of a book
about how people often vote based on feeling good about their morality
and concern for society as a whole, despite it being contrary to their
personal best interests if that vote wins.)