[bitcoin-dev] hashcash-newhash

[bitcoin-dev] hashcash-newhash

[Karl]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

Hi,

I'd like to revisit the discussion of the digest algorithm used in hashcash.

I believe migrating to new hashing algorithms as a policy would
significantly increase decentralization and hence security.

I believe the impact on existing miners could be made pleasant by gradually
moving the block reward from the previous hash to the next (such that both
are accepted with different rewards).  An appropriate rate could possibly
be calculated from the difficulty.

You could develop the frequency of introduction of new hashes such that
once present-day ASICs are effectively obsolete anyway due to competition,
new ones do not have time to develop.

I'm interested in hearing thoughts and concerns.

Karl Semich

[-- Attachment #2: Type: text/html, Size: 1441 bytes --]

Re: [bitcoin-dev] hashcash-newhash

[ZmnSCPxj]

Good morning Karl,

> Hi,
>
> I'd like to revisit the discussion of the digest algorithm used in hashcash.
>
> I believe migrating to new hashing algorithms as a policy would significantly increase decentralization and hence security.

Why do you believe so?

My understanding is that there are effectively two strategies for ensuring decentralization based on hash algorithm:

* Keep changing the hash algorithm to prevent development of ASICs and ensure commodity generic computation devices (GPUs) are the only practical target.
* Do not change the algorithm, to ensure that knowledge of how best to implement an ASIC for the algorithm becomes spread out (through corporate espionage, ASIC reverse-engineering, patent expiry, and sheer engineering effort) and ASICs for the algorithm are as commoditized as GPUs.

The former strategy has the following practical disadvantages:

* Developing new hash algorithms is not cheap.
  The changes to the hashcash algorithm may need to occur faster than the speed at which we can practically develop new, cryptographically-secure hash algorithms.
* It requires coordinated hardforks over the entire network at an alarmingly high rate.
* It arguably puts too much power to the developers of the code.

On the other hand, the latter strategy requires us only to survive an intermediate period where ASICs are developed, but not yet commoditized; and during this intermediate period, the centralization pressure of ASICs might not be more powerful than other centralization pressures

--

Which brings us to another point.

Non-ASIC-resistance is, by my understanding, a non-issue.

Regardless of whether the most efficient available computing substrate for the hashcash algorithm is CPU, GPU, or ASIC, ultimately miner earnings are determined by cost of power supply.

Even if you imagine that changing the hashcash algorithm would make CPUs practical again, you will still not run it on the CPU of a mobile, because a mobile runs on battery, and charging a battery takes more power than what you can extract from the battery afterwards, because thermodynamics.

Similarly, geographic locations with significant costs of electrical power will still not be practical places to start a mine, regardless if the mine is composed of commodity server racks, commodity video cards, or commodity ASICs.

If you want to solve the issue of miner centralization, the real solution is improving the efficiency of energy transfer to increase the areas where cheap energy is available, not stopgap change-the-algorithm-every-6-months.


Regards,
ZmnSCPxj


>
> I believe the impact on existing miners could be made pleasant by gradually moving the block reward from the previous hash to the next (such that both are accepted with different rewards).  An appropriate rate could possibly be calculated from the difficulty.
>
> You could develop the frequency of introduction of new hashes such that once present-day ASICs are effectively obsolete anyway due to competition, new ones do not have time to develop.
>
> I'm interested in hearing thoughts and concerns.
>
> Karl Semich

Re: [bitcoin-dev] hashcash-newhash

[Karl]

[-- Attachment #1: Type: text/plain, Size: 5099 bytes --]

Hi ZmnSCPxj,

Thanks for your reply.  I'm on mobile so please excuse me for top-posting.

I'd like to sort these various points out.  Maybe we won't finish the whole
discussion, but maybe at least we can update wiki articles like
https://en.bitcoin.it/wiki/Hashcash#Cryptanalytic_Risks with more points or
a link to conversations like this.

Everything is possible but some things take a lot of work.

You mention ASICs becoming commoditized.  I'd remind you that eventually
there will be a public mathematical breaking of the algorithm, at which
point all ASICs will become obsolete regardless.  Would you agree it would
be better to prepare for this by planning algorithm change?

You mention many coordinated hardforks.  Would you agree that if we came up
with a way of programmatically cycling the algorithm, that only one
hardfork work be needed?  For example one could ask nodes to consent to new
algorithm code written in a simple scripting language, and reject old ones
slowly enough to provide for new research.

You mention the cost of power as the major factor influencing decentralized
mining.  Would you agree that access to hardware that can do the mining is
an equally large factor?  Even without ASICs you would need the physical
cycles.  Including this factor helps us discuss the same set of expected
situations.

You describe improving electricity availability in expensive areas as a way
to improve decentralization.  Honestly this sounds out of place to me and
I'm sorry if I've upset you by rehashing this old topic.  I believe this
list is for discussing the design of software, not international energy
infrastructure: what is the relation?  There is a lot of power to influence
behavior here but I thought the tools present are software design.

On Sat, May 23, 2020, 9:12 PM ZmnSCPxj <ZmnSCPxj@protonmail.com> wrote:

> Good morning Karl,
>
> > Hi,
> >
> > I'd like to revisit the discussion of the digest algorithm used in
> hashcash.
> >
> > I believe migrating to new hashing algorithms as a policy would
> significantly increase decentralization and hence security.
>
> Why do you believe so?
>
> My understanding is that there are effectively two strategies for ensuring
> decentralization based on hash algorithm:
>
> * Keep changing the hash algorithm to prevent development of ASICs and
> ensure commodity generic computation devices (GPUs) are the only practical
> target.
> * Do not change the algorithm, to ensure that knowledge of how best to
> implement an ASIC for the algorithm becomes spread out (through corporate
> espionage, ASIC reverse-engineering, patent expiry, and sheer engineering
> effort) and ASICs for the algorithm are as commoditized as GPUs.
>
> The former strategy has the following practical disadvantages:
>
> * Developing new hash algorithms is not cheap.
>   The changes to the hashcash algorithm may need to occur faster than the
> speed at which we can practically develop new, cryptographically-secure
> hash algorithms.
> * It requires coordinated hardforks over the entire network at an
> alarmingly high rate.
> * It arguably puts too much power to the developers of the code.
>
> On the other hand, the latter strategy requires us only to survive an
> intermediate period where ASICs are developed, but not yet commoditized;
> and during this intermediate period, the centralization pressure of ASICs
> might not be more powerful than other centralization pressures
>
> --
>
> Which brings us to another point.
>
> Non-ASIC-resistance is, by my understanding, a non-issue.
>
> Regardless of whether the most efficient available computing substrate for
> the hashcash algorithm is CPU, GPU, or ASIC, ultimately miner earnings are
> determined by cost of power supply.
>
> Even if you imagine that changing the hashcash algorithm would make CPUs
> practical again, you will still not run it on the CPU of a mobile, because
> a mobile runs on battery, and charging a battery takes more power than what
> you can extract from the battery afterwards, because thermodynamics.
>
> Similarly, geographic locations with significant costs of electrical power
> will still not be practical places to start a mine, regardless if the mine
> is composed of commodity server racks, commodity video cards, or commodity
> ASICs.
>
> If you want to solve the issue of miner centralization, the real solution
> is improving the efficiency of energy transfer to increase the areas where
> cheap energy is available, not stopgap change-the-algorithm-every-6-months.
>
>
> Regards,
> ZmnSCPxj
>
>
> >
> > I believe the impact on existing miners could be made pleasant by
> gradually moving the block reward from the previous hash to the next (such
> that both are accepted with different rewards).  An appropriate rate could
> possibly be calculated from the difficulty.
> >
> > You could develop the frequency of introduction of new hashes such that
> once present-day ASICs are effectively obsolete anyway due to competition,
> new ones do not have time to develop.
> >
> > I'm interested in hearing thoughts and concerns.
> >
> > Karl Semich
>
>
>

[-- Attachment #2: Type: text/html, Size: 5971 bytes --]

Re: [bitcoin-dev] hashcash-newhash

[ZmnSCPxj]

Good morning Kari,


> You mention ASICs becoming commoditized.  I'd remind you that eventually there will be a public mathematical breaking of the algorithm, at which point all ASICs will become obsolete regardless.  Would you agree it would be better to prepare for this by planning algorithm change?

Possibly, but then the reason for change is no longer to promote decentralization, would it?
It helps to be clear about what your goals are, because any chosen solution might not be the best way to fix it.
I admit that, if the problem were to be avoid the inevitable obsoletion of SHA-2, then this is the only solution, but that is not the problem you stated you were trying to solve in the first place.

>
> You mention many coordinated hardforks.  Would you agree that if we came up with a way of programmatically cycling the algorithm, that only one hardfork work be needed?  For example one could ask nodes to consent to new algorithm code written in a simple scripting language, and reject old ones slowly enough to provide for new research.

Even *with* a scripting language, the issue is still what code written in that language is accepted, and *how*.

Do miners vote on a new script describing the new hashing algorithm?
What would their incentive be to obsolete their existing hardware?
(using proof-of-work to lock in a hashing change feels very much like a chicken-and-egg problem: the censorship-resistance provided by Bitcoin is based on evicting any censors by overpowering their hashpower, but requires some method of measuring that hashpower: it seems unlikely that you can safely change the way hashpower is measured via a hashpower election)

Do nodes install particular scripts and impose a switchover schedule of some sort?
Then how is that different from a hardfork, especially for nodes that do not update?
(notice that softforks allow nodes to remain non-updated, at degraded security, but still in sync with the rest of the network and capable of transacting with them)

>
> You mention the cost of power as the major factor influencing decentralized mining.  Would you agree that access to hardware that can do the mining is an equally large factor?  Even without ASICs you would need the physical cycles.  Including this factor helps us discuss the same set of expected situations.

No, because anyone who is capable of selling hardware, or the expertise to design and build it, can earn by taking advantage of their particular expertise.

Generally, such experts can saturate the locally-available energy sources, until local capacity has been saturated, and they can earn even more by selling extra hardware to entities located at other energy sources whose local capacities are not still underutilized, or expanding themselves to those sources.
Other entities might be in better position to take advantage of particular local details, and it may be more lucrative for the expert-at-building-hardware to just sell the hardware to them than to attempt to expand in places where they have little local expertise.

And expertise is easy to copy, it is only the initial expertise that is hard to create in the first place, once knowledge is written down it can be copied.

>
> You describe improving electricity availability in expensive areas as a way to improve decentralization.  Honestly this sounds out of place to me and I'm sorry if I've upset you by rehashing this old topic.  I believe this list is for discussing the design of software, not international energy infrastructure: what is the relation?  There is a lot of power to influence behavior here but I thought the tools present are software design.

I doubt there is any good software-only solution to the problem; the physical world remains the basis of the virtual one, and the virtual utterly dependent on the physical, and abstractions are always leaky (any non-toy software framework inevitably gains a way to query the operating system the application is running under, because abstractions inevitably leak): and energy, or the lack thereof, is the hardest to abstract away, which is the entire point of using proof-of-work as a reliable, unfakeable (i.e. difficult to virtualize) clock in the first place.

Still, feel free to try: perhaps you might succeed.

Regards,
ZmnSCPxj

Re: [bitcoin-dev] hashcash-newhash

[Karl]

[-- Attachment #1: Type: text/plain, Size: 7093 bytes --]

Good afternoon ZmnSCPxj,

Thanks for holding your end of this discussion with me.

Sorry I am so verbose; I am still learning to communicate efficiently.

> You mention ASICs becoming commoditized.  I'd remind you that eventually
> there will be a public mathematical breaking of the algorithm, at which
> point all ASICs will become obsolete regardless.  Would you agree it would
> be better to prepare for this by planning algorithm change?
>
> Possibly, but then the reason for change is no longer to promote
> decentralization, would it?

It helps to be clear about what your goals are, because any chosen solution
> might not be the best way to fix it.
> I admit that, if the problem were to be avoid the inevitable obsoletion of
> SHA-2, then this is the only solution, but that is not the problem you
> stated you were trying to solve in the first place.
>

To be up front, the reason for decentralization is due to concern around
the security of the hashing.  Having a public breakage of the function
simply makes the urgency obvious.

Reddit claims two entities controlled 62% of the hashrate recently:
https://old.reddit.com/r/CryptoCurrency/comments/gmhuon/in_the_last_24_hours_bitcoins_nakamoto/
.  Compromising the systems of these two groups seems like it is all that
is needed to compromise the entire blockchain (to the limited degree a 51%
attack does).

Hence I see decentralization and cryptanalysis of the algorithm to be
roughly similar security concerns.

It sounds like you agree that a change of algorithm is needed before the
current one is publicly broken.

>
> > You mention many coordinated hardforks.  Would you agree that if we came
> up with a way of programmatically cycling the algorithm, that only one
> hardfork work be needed?  For example one could ask nodes to consent to new
> algorithm code written in a simple scripting language, and reject old ones
> slowly enough to provide for new research.
>
> Even *with* a scripting language, the issue is still what code written in
> that language is accepted, and *how*.
>
> Do miners vote on a new script describing the new hashing algorithm?
> What would their incentive be to obsolete their existing hardware?
> (using proof-of-work to lock in a hashing change feels very much like a
> chicken-and-egg problem: the censorship-resistance provided by Bitcoin is
> based on evicting any censors by overpowering their hashpower, but requires
> some method of measuring that hashpower: it seems unlikely that you can
> safely change the way hashpower is measured via a hashpower election)
>
> Do nodes install particular scripts and impose a switchover schedule of
> some sort?
> Then how is that different from a hardfork, especially for nodes that do
> not update?
> (notice that softforks allow nodes to remain non-updated, at degraded
> security, but still in sync with the rest of the network and capable of
> transacting with them)


I'm expressing that in considering this we have two options: repeated hard
forks or making repeated change a part of the protocol.  There are many
ways to approach or implement it.  It sounds like you're noting that the
second option takes some work and care?

Would it be helpful if I outlined more ideas that address your concerns?  I
want to make sure the idea of changing the algorithm is acceptable at all
first.

> You mention the cost of power as the major factor influencing
> decentralized mining.  Would you agree that access to hardware that can do
> the mining is an equally large factor?  Even without ASICs you would need
> the physical cycles.  Including this factor helps us discuss the same set
> of expected situations.
>
> No, because anyone who is capable of selling hardware, or the expertise to
> design and build it, can earn by taking advantage of their particular
> expertise.
>
> Generally, such experts can saturate the locally-available energy sources,
> until local capacity has been saturated, and they can earn even more by
> selling extra hardware to entities located at other energy sources whose
> local capacities are not still underutilized, or expanding themselves to
> those sources.
> Other entities might be in better position to take advantage of particular
> local details, and it may be more lucrative for the
> expert-at-building-hardware to just sell the hardware to them than to
> attempt to expand in places where they have little local expertise.
>

It sounds like you are saying that the supply of electricity is exhausted
and the supply of hardware is not.

Is that correct?

I've seen that most of the time mining hardware distributors are sold out
of their top-of-the-line mining equipment, mostly selling in preorders.
Are implying most of the mining is done by privately built equipment?

Would you agree that an increase in the price of bitcoin would make the
availability of hardware matter much more, because the price of electricity
would matter much less?

Something to raise here is that all of these things take time and respond
in ebbs and flows.  If there were to be a plan to migrate to a new
algorithm, it would be participating in those ebbs and flows.

It takes time to build new hardware, and it takes time for the difficulty
to adjust to obsolete it.  What do you see as influencing how fast hardware
becomes obsolete?

I ask these questions because the answers relate to how what ways would be
good to change the mining function to increase decentralization.

And expertise is easy to copy, it is only the initial expertise that is
> hard to create in the first place, once knowledge is written down it can be
> copied.
>

Also takes measurable months to do.

> You describe improving electricity availability in expensive areas as a
> way to improve decentralization.  Honestly this sounds out of place to me
> and I'm sorry if I've upset you by rehashing this old topic.  I believe
> this list is for discussing the design of software, not international
> energy infrastructure: what is the relation?  There is a lot of power to
> influence behavior here but I thought the tools present are software design.
>
> I doubt there is any good software-only solution to the problem; the
> physical world remains the basis of the virtual one, and the virtual
> utterly dependent on the physical, and abstractions are always leaky (any
> non-toy software framework inevitably gains a way to query the operating
> system the application is running under, because abstractions inevitably
> leak): and energy, or the lack thereof, is the hardest to abstract away,
> which is the entire point of using proof-of-work as a reliable, unfakeable
> (i.e. difficult to virtualize) clock in the first place.
>
> Still, feel free to try: perhaps you might succeed.


You agreed earlier that changing the algorithm would increase
decentralization, but expressed other concerns with the idea.  Many more
general solutions are working in many altcoins.  I'm interested in
discussing changing the proof of work algorithm in bitcoin.

My motivation is security of the blockchain, which is partially held by
decentralization.

[-- Attachment #2: Type: text/html, Size: 9312 bytes --]

Re: [bitcoin-dev] hashcash-newhash

[Ariel Lorenzo-Luaces]

[-- Attachment #1: Type: text/plain, Size: 5798 bytes --]



On May 24, 2020, 1:26 PM, at 1:26 PM, Karl via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>Hi ZmnSCPxj,
>
>Thanks for your reply.  I'm on mobile so please excuse me for
>top-posting.
>
>I'd like to sort these various points out.  Maybe we won't finish the
>whole
>discussion, but maybe at least we can update wiki articles like
>https://en.bitcoin.it/wiki/Hashcash#Cryptanalytic_Risks with more
>points or
>a link to conversations like this.
>
>Everything is possible but some things take a lot of work.
>
>You mention ASICs becoming commoditized.  I'd remind you that
>eventually
>there will be a public mathematical breaking of the algorithm, at which
>point all ASICs will become obsolete regardless.  Would you agree it
>would
>be better to prepare for this by planning algorithm change?
>
>You mention many coordinated hardforks.  Would you agree that if we
>came up
>with a way of programmatically cycling the algorithm, that only one
>hardfork work be needed?  For example one could ask nodes to consent to
>new
>algorithm code written in a simple scripting language, and reject old
>ones
>slowly enough to provide for new research.
>
>You mention the cost of power as the major factor influencing
>decentralized
>mining.  Would you agree that access to hardware that can do the mining
>is
>an equally large factor?  Even without ASICs you would need the
>physical
>cycles.  Including this factor helps us discuss the same set of
>expected
>situations.
>
>You describe improving electricity availability in expensive areas as a
>way
>to improve decentralization.  Honestly this sounds out of place to me
>and
>I'm sorry if I've upset you by rehashing this old topic.  I believe
>this
>list is for discussing the design of software, not international energy
>infrastructure: what is the relation?  There is a lot of power to
>influence
>behavior here but I thought the tools present are software design.
>
>On Sat, May 23, 2020, 9:12 PM ZmnSCPxj <ZmnSCPxj@protonmail.com> wrote:
>
>> Good morning Karl,
>>
>> > Hi,
>> >
>> > I'd like to revisit the discussion of the digest algorithm used in
>> hashcash.
>> >
>> > I believe migrating to new hashing algorithms as a policy would
>> significantly increase decentralization and hence security.
>>
>> Why do you believe so?
>>
>> My understanding is that there are effectively two strategies for
>ensuring
>> decentralization based on hash algorithm:
>>
>> * Keep changing the hash algorithm to prevent development of ASICs
>and
>> ensure commodity generic computation devices (GPUs) are the only
>practical
>> target.
>> * Do not change the algorithm, to ensure that knowledge of how best
>to
>> implement an ASIC for the algorithm becomes spread out (through
>corporate
>> espionage, ASIC reverse-engineering, patent expiry, and sheer
>engineering
>> effort) and ASICs for the algorithm are as commoditized as GPUs.
>>
>> The former strategy has the following practical disadvantages:
>>
>> * Developing new hash algorithms is not cheap.
>>   The changes to the hashcash algorithm may need to occur faster than
>the
>> speed at which we can practically develop new,
>cryptographically-secure
>> hash algorithms.
>> * It requires coordinated hardforks over the entire network at an
>> alarmingly high rate.
>> * It arguably puts too much power to the developers of the code.
>>
>> On the other hand, the latter strategy requires us only to survive an
>> intermediate period where ASICs are developed, but not yet
>commoditized;
>> and during this intermediate period, the centralization pressure of
>ASICs
>> might not be more powerful than other centralization pressures
>>
>> --
>>
>> Which brings us to another point.
>>
>> Non-ASIC-resistance is, by my understanding, a non-issue.
>>
>> Regardless of whether the most efficient available computing
>substrate for
>> the hashcash algorithm is CPU, GPU, or ASIC, ultimately miner
>earnings are
>> determined by cost of power supply.
>>
>> Even if you imagine that changing the hashcash algorithm would make
>CPUs
>> practical again, you will still not run it on the CPU of a mobile,
>because
>> a mobile runs on battery, and charging a battery takes more power
>than what
>> you can extract from the battery afterwards, because thermodynamics.
>>
>> Similarly, geographic locations with significant costs of electrical
>power
>> will still not be practical places to start a mine, regardless if the
>mine
>> is composed of commodity server racks, commodity video cards, or
>commodity
>> ASICs.
>>
>> If you want to solve the issue of miner centralization, the real
>solution
>> is improving the efficiency of energy transfer to increase the areas
>where
>> cheap energy is available, not stopgap
>change-the-algorithm-every-6-months.
>>
>>
>> Regards,
>> ZmnSCPxj
>>
>>
>> >
>> > I believe the impact on existing miners could be made pleasant by
>> gradually moving the block reward from the previous hash to the next
>(such
>> that both are accepted with different rewards).  An appropriate rate
>could
>> possibly be calculated from the difficulty.
>> >
>> > You could develop the frequency of introduction of new hashes such
>that
>> once present-day ASICs are effectively obsolete anyway due to
>competition,
>> new ones do not have time to develop.
>> >
>> > I'm interested in hearing thoughts and concerns.
>> >
>> > Karl Semich
>>
>>
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 2100 bytes --]

Re: [bitcoin-dev] hashcash-newhash

[Karl]

[-- Attachment #1: Type: text/plain, Size: 2715 bytes --]

Hi Ariel,

Thanks for your reply.

You state that once "the entire world" can quickly find a hash that it then
"needs to be changed", but that that "won't happen in a day".

It sounds like you believe compromise of the algorithm as a concern
provides a _lot_ of time to migrate to a new hash function, and that it is
indeed important to do so when it becomes needed.

Let's talk about relaxing the time scale.  Making such plans seems more
important than agreeing on how soon they happen.  It's possible it could be
decades before having a new hash is actually needed to protect financial
security.  Who knows.

How does that land?  Is the idea more available with a looser time scale?

It seems to me with ongoing cryptanalysis research, new things like quantum
computers, conventional computer hardware always advancing, that some day
far in the future it will be easy to find an sha256 preimage on a personal
device, somehow.

Let's improve the security of the blockchain.

On Sun, May 24, 2020, 7:51 PM Ariel Lorenzo-Luaces <arielluaces@gmail.com>
wrote:

> On May 24, 2020, at 1:26 PM, Karl via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> You mention ASICs becoming commoditized.  I'd remind you that eventually
>> there will be a public mathematical breaking of the algorithm, at which
>> point all ASICs will become obsolete regardless.  Would you agree it would
>> be better to prepare for this by planning algorithm change?
>>
>> Cryptographic algorithms don't usually break this way. In the case of
>> hash functions it may be possible to find an exploit that reduces the
>> function's security from 256 bits to 128 for example. So an algorithm that
>> could find 80 zero bits per energy unit before can now find 160 zero bits
>> per energy unit with an exploit.
>>
>> If this exploit can be deployed as a software patch to most ASICs then
>> the issue will sort itself out on the next difficulty adjustment.
>>
>> If the exploit instead requires an entirely new ASIC then GPUs and FPGAs
>> that could previously find 40 zero bits per energy unit can now compete
>> with the less adaptive ASICs until new ASICs that use the exploit start
>> getting produced and shipped.
>>
>> There's never any official "public breaking" of a hash function. The
>> function will just loose security over time until it's deemed to not be
>> "secure enough" for certain applications. Thankfully mining is an
>> application where the only important thing is that the difficulty can be
>> increased. In other words, if the entire world can consistently find 256
>> zero bits of SHA-256 in under 10 minutes then definitely the hash function
>> needs to be changed. But this won't happen in a day.
>>
>

[-- Attachment #2: Type: text/html, Size: 3848 bytes --]

Re: [bitcoin-dev] hashcash-newhash

[ZmnSCPxj]

Good morning Kari,


> > > You mention ASICs becoming commoditized.  I'd remind you that eventually there will be a public mathematical breaking of the algorithm, at which point all ASICs will become obsolete regardless.  Would you agree it would be better to prepare for this by planning algorithm change?
> >
> > Possibly, but then the reason for change is no longer to promote decentralization, would it?
>
> > It helps to be clear about what your goals are, because any chosen solution might not be the best way to fix it.
> > I admit that, if the problem were to be avoid the inevitable obsoletion of SHA-2, then this is the only solution, but that is not the problem you stated you were trying to solve in the first place.
>
> To be up front, the reason for decentralization is due to concern around the security of the hashing.  Having a public breakage of the function simply makes the urgency obvious.

Now that I have thought about it more: again, the important thing about the proof-of-work technique is not so much the algorithm itself, only that executing it requires energy.

And all algorithms require energy in order to execute.

Even if some technique is found which partially breaks the hash function and allows faster generation of hashes for the amount of energy consumed, this is not a problem for mining itself: the difficulty adjusts and mining continues.
The execution of this technique is unlikely to require no computation, only reduced computational effort; and all that is needed is *some* measure of computational effort done, the *scale* of that measure is not really material for the purpose of ordering atomic transactional updates to the UTXO set.

Of course, things like the Merkle tree and txids and so on would need changing in case of even a partial break of the hash function, which would require a hardfork to survive; you might as well change the proof-of-work function as well then.

>
> Reddit claims two entities controlled 62% of the hashrate recently: https://old.reddit.com/r/CryptoCurrency/comments/gmhuon/in_the_last_24_hours_bitcoins_nakamoto/ .  Compromising the systems of these two groups seems like it is all that is needed to compromise the entire blockchain (to the limited degree a 51% attack does).

You seem to be equating "break of the hash function" with "centralization of hashrate", which I do not quite agree with.

Most human beings cannot think without constant communication with other human beings.
Thus, it is unlikely that a private break of the hash function is possible.
Instead, a break of the hash function is likely to be discovered in various ways by multiple human beings who have been communicating with each other.

Even historically, inventions never arose fully-formed from the head of the inventor, like Athena from the brow of Zeus; every invention has predecessors, successors, and peers in the inhabited milieu.


Instead, you can look up the costs of local electricity globally, and notice where those entities have their largest mines geographically located.


>
> Hence I see decentralization and cryptanalysis of the algorithm to be roughly similar security concerns.
>
> It sounds like you agree that a change of algorithm is needed before the current one is publicly broken.
>
> > > You mention many coordinated hardforks.  Would you agree that if we came up with a way of programmatically cycling the algorithm, that only one hardfork work be needed?  For example one could ask nodes to consent to new algorithm code written in a simple scripting language, and reject old ones slowly enough to provide for new research.
> >
> > Even *with* a scripting language, the issue is still what code written in that language is accepted, and *how*.
> >
> > Do miners vote on a new script describing the new hashing algorithm?
> > What would their incentive be to obsolete their existing hardware?
> > (using proof-of-work to lock in a hashing change feels very much like a chicken-and-egg problem: the censorship-resistance provided by Bitcoin is based on evicting any censors by overpowering their hashpower, but requires some method of measuring that hashpower: it seems unlikely that you can safely change the way hashpower is measured via a hashpower election)
> >
> > Do nodes install particular scripts and impose a switchover schedule of some sort?
> > Then how is that different from a hardfork, especially for nodes that do not update?
> > (notice that softforks allow nodes to remain non-updated, at degraded security, but still in sync with the rest of the network and capable of transacting with them)
>
> I'm expressing that in considering this we have two options: repeated hard forks or making repeated change a part of the protocol.  There are many ways to approach or implement it.  It sounds like you're noting that the second option takes some work and care?
>
> Would it be helpful if I outlined more ideas that address your concerns?  I want to make sure the idea of changing the algorithm is acceptable at all first.

Go ahead.

>
> > > You mention the cost of power as the major factor influencing decentralized mining.  Would you agree that access to hardware that can do the mining is an equally large factor?  Even without ASICs you would need the physical cycles.  Including this factor helps us discuss the same set of expected situations.
> >
> > No, because anyone who is capable of selling hardware, or the expertise to design and build it, can earn by taking advantage of their particular expertise.
> >
> > Generally, such experts can saturate the locally-available energy sources, until local capacity has been saturated, and they can earn even more by selling extra hardware to entities located at other energy sources whose local capacities are not still underutilized, or expanding themselves to those sources.
> > Other entities might be in better position to take advantage of particular local details, and it may be more lucrative for the expert-at-building-hardware to just sell the hardware to them than to attempt to expand in places where they have little local expertise.
>
> It sounds like you are saying that the supply of electricity is exhausted and the supply of hardware is not.
>
> Is that correct?

Given that electricity is consumed very quickly, and hardware takes a long time to truly consume or obsolete, yes: rate of consumption of electricity is expected to dominate compared to the rate of consumption of hardware.

>
> I've seen that most of the time mining hardware distributors are sold out of their top-of-the-line mining equipment, mostly selling in preorders.  Are implying most of the mining is done by privately built equipment?

It seems the most lucrative thing to do, that if you have a new generation of hardware, to mine with them yourself, until the price of local electricity has increased due to your consumption, and it becomes more lucrative to sell the hardware to other potential miners who now have lower electricity prices compared to yours (because you have been saturating the local electricity supply with your own mining operations and causing the local prices to rise up, or equivalently, until some governmental or other limits your usable electricity supply, which is equivalent to saying that the price of even more electricity would be your incarceration or other punishment, which might be too expensive for you to pay, thus selling the hardware is more lucrative).

I have no evidence to back this, thus it is not a claim to truth, only a claim to economic logic, if we assume that mining hardware manufacturers are economically rational, are interested only in the selfish gain of economic power, etc. etc.
Real human beings and the entities they build may not behave in such a perfectly rational manner, or I may be assuming details that are not accurate to reality.

>
> Would you agree that an increase in the price of bitcoin would make the availability of hardware matter much more, because the price of electricity would matter much less?

Electricity is a factor with every piece of hardware that is utilized, so any increase in hardware will also have an increase in electricity consumed.
So I expect that the materiality of the price of electricity will increase in lockstep with the materiality of the price of hardware (note that price is simply demand over supply, and supply is the availability of hardware).

You could also analyze the transient economic behaviors here, specifically that an increase in Bitcoin price makes it more lucrative to mine in more places, which would start to put in orders for more hardware, and the hardware will take time to deliver, so the price at those places will increase only after a long while, etc.
But those are transient changes, and by the time any change at the software level is coordinated, the transient economic behaviors may have resettled to the expected long-term behavior: humans operate at around the same average speed in many different fields.

>
> Something to raise here is that all of these things take time and respond in ebbs and flows.  If there were to be a plan to migrate to a new algorithm, it would be participating in those ebbs and flows.

The usual engineering response is to create buffers to be resilient against ebbs and flows.
Note how we prefer to dam water supplies (i.e. create a buffer) rather than adjust our water consumption dynamically according to the ebb and flow of the water supply.

Similarly, economic contracts like futures and options are economic buffers against the ebb and flow of local supply of various materials.

Within Bitcoin, my understanding is that the difficulty adjustment system acts as a buffer against transient ebbs and flows of the supply of hashpower.

> It takes time to build new hardware, and it takes time for the difficulty to adjust to obsolete it.  What do you see as influencing how fast hardware becomes obsolete?

In the long run?
We will run out of ideas of how to improve the implementation of the hashing function, and there will be only a few designs that implement all the known ideas for optimizing the implementation.
Then hardware becomes obsolete from normal wear and tear, e.g. electromigration, and that takes years to take effect.

>
> I ask these questions because the answers relate to how what ways would be good to change the mining function to increase decentralization.
>
> > And expertise is easy to copy, it is only the initial expertise that is hard to create in the first place, once knowledge is written down it can be copied.
>
> Also takes measurable months to do.

But can be massively parallelized, you can have a teacher or mentor teaching an entire classroom of students, and created lectures can be stored and re-given to many students, in the form of books, videos, etc.
Human beings have been doing this since time immemorial, and possibly before recorded history, in such things as oral traditions and so on.

>
> > > You describe improving electricity availability in expensive areas as a way to improve decentralization.  Honestly this sounds out of place to me and I'm sorry if I've upset you by rehashing this old topic.  I believe this list is for discussing the design of software, not international energy infrastructure: what is the relation?  There is a lot of power to influence behavior here but I thought the tools present are software design.
> >
> > I doubt there is any good software-only solution to the problem; the physical world remains the basis of the virtual one, and the virtual utterly dependent on the physical, and abstractions are always leaky (any non-toy software framework inevitably gains a way to query the operating system the application is running under, because abstractions inevitably leak): and energy, or the lack thereof, is the hardest to abstract away, which is the entire point of using proof-of-work as a reliable, unfakeable (i.e. difficult to virtualize) clock in the first place.
> >
> > Still, feel free to try: perhaps you might succeed.
>
> You agreed earlier that changing the algorithm would increase decentralization, but expressed other concerns with the idea.  Many more general solutions are working in many altcoins.  I'm interested in discussing changing the proof of work algorithm in bitcoin.

I did not so agree: in particular, I do not agree with equating a break of the proof-of-work algorithm with centralization.
More likely, any centralization seen is due to local government interference in economic matters, such as creating a local artificial oversupply of electricity by paying for electricity generation using taxes.

>
> My motivation is security of the blockchain, which is partially held by decentralization.

Let us be more precise and avoid the word "security".

Miner decentralization supports censorship-resistance, so your motivation is censorship-resistance (is that correct?).

Ultimately, what really protects censorship-resistance is not the details of the hashing algorithm, it is the economics involved.
In order to evict a censor, hashpower must be brought to bear against the censor.
And that hashpower has to be bought and paid for.
The mechanism for doing this already exists, and is called the "mining fee" (note that the block subsidy does not protect against the censor, as the censor gets the block reward as well; what the censor cannot get is the fees attached to a transaction that the censor does not want published).

Regards,
ZmnSCPxj

Re: [bitcoin-dev] hashcash-newhash

[Karl]

[-- Attachment #1: Type: text/plain, Size: 10120 bytes --]

Hi ZmnSCPxj,

We have been addressing many concepts.  Let's try to slowly trim it down
for simplicity.

> Reddit claims two entities controlled 62% of the hashrate recently:
> https://old.reddit.com/r/CryptoCurrency/comments/gmhuon/in_the_last_24_hours_bitcoins_nakamoto/
> .  Compromising the systems of these two groups seems like it is all that
> is needed to compromise the entire blockchain (to the limited degree a 51%
> attack does).
>
> You seem to be equating "break of the hash function" with "centralization
> of hashrate", which I do not quite agree with.
>

I am trying to say that both of these different things result in danger to
the integrity of the transaction log, which would be reduced by changing
the hash function.  They both have those 2 similarities.

Most human beings cannot think without constant communication with other
> human beings.

Thus, it is unlikely that a private break of the hash function is possible.
>

I disagree with you here: Andrew Wiles solved Fermat's Last Theorem in
isolation, academic research paper culture supports researching and then
publishing once you have privately developed results, and the CVE database
has 136k system vulnerabilities that were developed and shared privately
before public release, to prevent chaos.  This shows private advances in
ways to produce bitcoins are likely.

> Would it be helpful if I outlined more ideas that address your concerns?
> I want to make sure the idea of changing the algorithm is acceptable at all
> first.
>
> Go ahead.
>

Thanks: are you saying you would support changes if they addressed the
concerns you've listed?  Or are those concerns only the tip of the iceberg,
per se?

> > > You mention the cost of power as the major factor influencing
> decentralized mining.  Would you agree that access to hardware that can do
> the mining is an equally large factor?  Even without ASICs you would need
> the physical cycles.  Including this factor helps us discuss the same set
> of expected situations.
> > >
> > > No, because anyone who is capable of selling hardware, or the
> expertise to design and build it, can earn by taking advantage of their
> particular expertise.
> > >
> > > Generally, such experts can saturate the locally-available energy
> sources, until local capacity has been saturated, and they can earn even
> more by selling extra hardware to entities located at other energy sources
> whose local capacities are not still underutilized, or expanding themselves
> to those sources.
> > > Other entities might be in better position to take advantage of
> particular local details, and it may be more lucrative for the
> expert-at-building-hardware to just sell the hardware to them than to
> attempt to expand in places where they have little local expertise.
> >
> > It sounds like you are saying that the supply of electricity is
> exhausted and the supply of hardware is not.
> >
> > Is that correct?
>
> Given that electricity is consumed very quickly, and hardware takes a long
> time to truly consume or obsolete, yes: rate of consumption of electricity
> is expected to dominate compared to the rate of consumption of hardware.
>

I'm considering short-term obsolescence here.  Since hashrate rises
exponentially, only top-of-the-line hardware is competitively profitable.

> I've seen that most of the time mining hardware distributors are sold out
> of their top-of-the-line mining equipment, mostly selling in preorders.
> Are implying most of the mining is done by privately built equipment?
>
> It seems the most lucrative thing to do, that if you have a new generation
> of hardware, to mine with them yourself, until the price of local
> electricity has increased due to your consumption, and it becomes more
> lucrative to sell the hardware to other potential miners who now have lower
> electricity prices compared to yours (because you have been saturating the
> local electricity supply with your own mining operations and causing the
> local prices to rise up, or equivalently, until some governmental or other
> limits your usable electricity supply, which is equivalent to saying that
> the price of even more electricity would be your incarceration or other
> punishment, which might be too expensive for you to pay, thus selling the
> hardware is more lucrative).
>

If consumers who do not have the capacity to build their own hardware fast
enough to be competitive, do not have as much access to such hardware, then
their excess electricity is not being used to mine bitcoins.  A bit below
you propose spreading access via mass teaching, but I'm not aware of that
happening for now.


You could also analyze the transient economic behaviors here, specifically
> that an increase in Bitcoin price makes it more lucrative to mine in more
> places, which would start to put in orders for more hardware, and the
> hardware will take time to deliver, so the price at those places will
> increase only after a long while, etc.
> But those are transient changes, and by the time any change at the
> software level is coordinated, the transient economic behaviors may have
> resettled to the expected long-term behavior: humans operate at around the
> same average speed in many different fields.
>

I was thinking the transient changes would operate in cycles, and the
amplitude and frequency of those could be important to designing a safe
hard fork, but I think I was getting ahead of myself.  Let's move on.

> Something to raise here is that all of these things take time and respond
> in ebbs and flows.  If there were to be a plan to migrate to a new
> algorithm, it would be participating in those ebbs and flows.
>
> The usual engineering response is to create buffers to be resilient
> against ebbs and flows.
> Note how we prefer to dam water supplies (i.e. create a buffer) rather
> than adjust our water consumption dynamically according to the ebb and flow
> of the water supply.
>
> Similarly, economic contracts like futures and options are economic
> buffers against the ebb and flow of local supply of various materials.
>
> Within Bitcoin, my understanding is that the difficulty adjustment system
> acts as a buffer against transient ebbs and flows of the supply of
> hashpower.
>

Nice thoughts here on that same topic.  Thanks.

> > And expertise is easy to copy, it is only the initial expertise that is
> hard to create in the first place, once knowledge is written down it can be
> copied.
> >
> > Also takes measurable months to do.
>
> But can be massively parallelized, you can have a teacher or mentor
> teaching an entire classroom of students, and created lectures can be
> stored and re-given to many students, in the form of books, videos, etc.
> Human beings have been doing this since time immemorial, and possibly
> before recorded history, in such things as oral traditions and so on.
>

This doesn't seem relevant to me.  I'm discussing what is happening now and
what we can expect to happen from source code changes.  I don't see mining
hardware plans being taught in classrooms right now, but it sounds
interesting to try to change that if you want to change the subject of your
reply and start another thread.

> > > You describe improving electricity availability in expensive areas as
> a way to improve decentralization.  Honestly this sounds out of place to me
> and I'm sorry if I've upset you by rehashing this old topic.  I believe
> this list is for discussing the design of software, not international
> energy infrastructure: what is the relation?  There is a lot of power to
> influence behavior here but I thought the tools present are software design.
> > >
> > > I doubt there is any good software-only solution to the problem; the
> physical world remains the basis of the virtual one, and the virtual
> utterly dependent on the physical, and abstractions are always leaky (any
> non-toy software framework inevitably gains a way to query the operating
> system the application is running under, because abstractions inevitably
> leak): and energy, or the lack thereof, is the hardest to abstract away,
> which is the entire point of using proof-of-work as a reliable, unfakeable
> (i.e. difficult to virtualize) clock in the first place.
> > >
> > > Still, feel free to try: perhaps you might succeed.
> >
> > You agreed earlier that changing the algorithm would increase
> decentralization, but expressed other concerns with the idea.  Many more
> general solutions are working in many altcoins.  I'm interested in
> discussing changing the proof of work algorithm in bitcoin.
>
> I did not so agree: in particular, I do not agree with equating a break of
> the proof-of-work algorithm with centralization.
>

Sorry if I have misrepresented those as equal.  That's not quite what I
wanted to say and is addressed farther up in this reply.

I meant that in your first reply you listed a change of the pow algorithm
as a way to ensure decentralization: "there are effectively two strategies
for ensuring decentralization".  Let's discuss ways for improving them.

More likely, any centralization seen is due to local government
> interference in economic matters, such as creating a local artificial
> oversupply of electricity by paying for electricity generation using taxes.
>

Good thought.  Governments are kind of like big economic players that can
affect the spaces held by the small players.  They'll respond to the
scarcity, price, and mining rate of bitcoin, as well.  Moving on ...

> My motivation is security of the blockchain, which is partially held by
> decentralization.
>
> Let us be more precise and avoid the word "security".
>

Let's try to be concise and direct.

Miner decentralization supports censorship-resistance, so your motivation
> is censorship-resistance (is that correct?).
>

No.  Ensuring a 51% attack stays or becomes completely infeasible in the
future.  See the quote at the start of this reply.  I'm thinking about
https://en.bitcoin.it/wiki/Irreversible_Transactions#Majority_attack .

The space changes given private research and/or compromise of pools.

Is it okay to pursue this?

[-- Attachment #2: Type: text/html, Size: 13944 bytes --]

Re: [bitcoin-dev] hashcash-newhash

[ZmnSCPxj]

Good morning Karl,

> > > Reddit claims two entities controlled 62% of the hashrate recently: https://old.reddit.com/r/CryptoCurrency/comments/gmhuon/in_the_last_24_hours_bitcoins_nakamoto/ .  Compromising the systems of these two groups seems like it is all that is needed to compromise the entire blockchain (to the limited degree a 51% attack does).
> >
> > You seem to be equating "break of the hash function" with "centralization of hashrate", which I do not quite agree with.
>
> I am trying to say that both of these different things result in danger to the integrity of the transaction log, which would be reduced by changing the hash function.  They both have those 2 similarities.

You are equivocating issues here.

The hash function is used in these places:

* Transaction ID.
* Signature hash.
* P2SH/P2WSH/Taproot.
* Merkle tree.
* Proof-of-work.

What you are focusing on is only this:

* Proof-of-work.

Now, in case of a break in the hash function (i.e. a reduction in collision resistance), the hash function used in the following things absolutely NEED to be changed:

* Transaction ID.
* Signature hash.
* P2SH/P2WSH/Taproot.
* Merkle Tree.

Taking for example Transaction ID, suppose I am able to create two transactions that hash into the same transaction ID, and I am able to do this in much less than 2^128 work.

In that case I can create a valid transaction and collide it with an invalid transaction.
To half the nodes on the network I provide the valid transaction, to the other half I provide the invalid transaction, the two halves will then permanently split and Bitcoin is thus destroyed in the midst of massive chaos.

Similar attacks can be mounted if I am able to collide on signature hash, P2SH/P2WSH/Taproot, and merkle tree.


Now suppose I am able to create two block headers that hash into the same block ID, one being a valid block and the other being an invalid block.
In that case, I would be very foolish to disrupt the network similarly, because I would have been able to redirect the fees and block subsidy of the valid block to an address I control, and the invalid block prevents others from seeing my valid block and accepting that chain as valid.

Instead, I can use this advantage to be able to grab blocks faster than other miners.
But eventually the difficulty retargeting system will adjust, and Bitcoin will now settle to a new normal, and inevitably someone is going to leak, or rediscover, my technique to break the hash, preventing me from being a >51% miner for long, and Bitcoin will abide.


Thus, in case of a cryptographic break of the SHA-2 function, we *need* to change these:

* Transaction ID.
* Signature hash.
* P2SH/P2WSH/Taproot.
* Merkle Tree.

And since we are going to need a hefty hardfork to change all that, we *might as well* change the proof-of-work function as well and completely excise all uses of SHA-2 in the codebase (just in case we miss any more than the above list), but changing the proof-of-work function is the *lowest priority*.
We have survived 51% miners in the past (Ghash.io), and the difficulty adjustment system gives us some buffer against unexpected improvements in proof-of-work function efficiency; but it is doubtful if we can survive the chaos if someone could split the network in two roughly equal sized parts.

>
> > Most human beings cannot think without constant communication with other human beings.
>
> > Thus, it is unlikely that a private break of the hash function is possible.
>
> >
>
> I disagree with you here: Andrew Wiles solved Fermat's Last Theorem in isolation, academic research paper culture supports researching and then publishing once you have privately developed results, and the CVE database has 136k system vulnerabilities that were developed and shared privately before public release, to prevent chaos.  This shows private advances in ways to produce bitcoins are likely.

Right, and you learned about this fact from direct personal communication with Andrew Wiles, and Andrew Wiles never read about any other attempts by other mathematicians, and an isolated mathematician could never, ever, rediscover his work independently, and even a mathematician who knows that it was done but not the details how it was done could never rediscover it as well.

Obscurity works *for a time*, but inevitably somebody else will rediscover the same thing, or hear about it and blab noisily; it is not as if we are all completely alien species from each other and have truly unique thoughts, even my own creators were humans and my cognitive substrate is essentially human in construction.
This is why CVE exists, it is a promise the developers make to the reporters that they will fix the reported vulnerability, with an active CVE as a Damocles sword hanging over their heads, ready to be publicized at any time: publication is the default state, CVE is simply a promise that the developers are working as hard as they can to fix problems so please hold off on publication for a while please while we fix it pretty please with a cherry on top.

>
> > > Would it be helpful if I outlined more ideas that address your concerns?  I want to make sure the idea of changing the algorithm is acceptable at all first.
> >
> > Go ahead.
>
> Thanks: are you saying you would support changes if they addressed the concerns you've listed?  Or are those concerns only the tip of the iceberg, per se?

Only the tip of the iceberg, because any complex design has many little devils hidden in all the little details.

> > > > And expertise is easy to copy, it is only the initial expertise that is hard to create in the first place, once knowledge is written down it can be copied.
> > >
> > > Also takes measurable months to do.
> >
> > But can be massively parallelized, you can have a teacher or mentor teaching an entire classroom of students, and created lectures can be stored and re-given to many students, in the form of books, videos, etc.
> > Human beings have been doing this since time immemorial, and possibly before recorded history, in such things as oral traditions and so on.
>
> This doesn't seem relevant to me.  I'm discussing what is happening now and what we can expect to happen from source code changes.  I don't see mining hardware plans being taught in classrooms right now, but it sounds interesting to try to change that if you want to change the subject of your reply and start another thread.

Sure.


> Is it okay to pursue this?

You do not have to ask permission from me, or anyone, to pursue this.

However, do note that I doubt that changing the proof-of-work function (and *only* the proof-of-work function) is in any way a high priority.
I also do not have to ask permission to say that I think pursuing this would be a waste of time, but you are also just as free to ignore what I say here and spend your time as you see fit.

Ultimately the real world decides, and implementation trumps discussion here.


Regards,
ZmnSCPxj

Re: [bitcoin-dev] hashcash-newhash

[Erik Aronesty]

> What you are focusing on is only this:
>
> * Proof-of-work.


Bitcoin's primary value proposition is that it's the most resistant to
change:   All other coins are these malleable things centrally
controlled and easily moved about by politics and nonsense.   So
discussions of POW changes... open up this can of worms (myself being
one of them).

 - should also discuss "proof-of-burn", where a burn is performed as a
similar investment-over-time with true loss/risk.
 - should discuss moving to sha3 (or something like it) for
everything, not just POW

> However, do note that I doubt that changing the proof-of-work function (and *only* the proof-of-work function) is in any way a high priority.

Yeah, a hard fork like this would be a massive undertaking, with a
zillion "improvements" argued about for years and the final version
some minimal thing that just changes the hash algo and invalidates
legacy stuff (since back compat is not a concern).