[Bitcoin-development] Near-term scalability

[Gregory Maxwell <gmaxwell@gmail.com>]

[I originally sent an earlier version of this message to Mike off
list, but I figure it's worth adding to the public discussion]

On Fri, Jun 15, 2012 at 7:29 AM, Mike Hearn <mike@plan99.net> wrote:
> (4) Making the block size limit float is better than picking a new
> arbitrary threshold.
> On the forums Matt stated that block chain pruning was a no-go because
> "it makes bitcoin more centralized". I think we've thrashed this one
> out sufficiently well by now that there should be a united opinion on
> it.

By itself letting the size float has non-trivial existential risk.  A
Bitcoin with expensive transactions due to competition for space in
blocks can be front-ended with fast payment systems and still provide
the promised decentralized currency. Bitcoin with a very large
blockchain and blocks does not.  It would do the bitcoin users no good
to increase the transaction volume while concurrently making Bitcoin
more or less pointless over the alternatives.

Scalability must be improved, we can unite on that opinion.  But
scalability can't come at the expense of what made Bitcoin worth
having in the first place.

Fortunately it appear to be possible to greatly increase the
scalability without compromising on keeping the costs of operating a
fully validating node very low,  for example Pieter's experimentation
with txout+txid indexing (for the 'flip the chain' proposals)
indicates that the data required right now to validate further
transactions is only about 85MiB— and that would be somewhat smaller
with compression and with clients which intentionally try to reduce
the set of unspent transactions.   Commitments to these indexes in the
chain would allow almost-full validating nodes with fairly limited
resources.  (Almost-full meaning they would not validate the history
long before they started, they'd trusted header difficulty for that. They
could still mine and otherwise act as full nodes).

Achieving scalability improvements without breaking the radical
decentralization will be a lot harder than just improving scalability
but it's effort that is justified if the scalability is actually
needed.

How much decentralization is needed in the end?  That isn't clear— "As
much as possible" should generally be the goal.  Modern currencies
aren't controlled by single parties but by tens of thousands of
parties locked in economic, legal, and political compromise that
limits their control.  In Bitcoin the traditional controls that keep
parties honest are non-existent and if they were just directly applied
we'd potentially lose the properties that make Bitcoin distinct and
useful (e.g. make all miners mine only with FED permission and you
just have a really bandwidth inefficient interface to the dollar).
Instead we have aggressive decentralization and autonomous rule
enforcement.

Mike pointed out that  "Before he left Satoshi made a comment saying
he used to think Bitcoin would need millions of nodes if it became
really popular, but in the end he thought it could do fine with just
tens of thousands"    I'm not so sure— and I think the truth is in
between.  Tens of thousands of nodes— run by a self-selecting bunch of
people who reap the greatest rewards from controlling the validation
of Bitcoin, who by that criteria necessarily have a lot in common with
each other and perhaps not with the regular users— could easily be an
outcome where control is _less_ publicly vested than popular
government controlled currencies.   We probably don't need the raw
numbers of nodes, but we need a distribution of ownership and a
distribution of interest (e.g. not a system by bankers for bankers) of
those nodes which I think can only be achieved by making them cheap to
operate and having a lot more than we actually need. — though not so
much that it has to run on every laptop.

The core challenge is that the only obvious ways to justify the cost
of maintaining expensive validation infrastructure is because you
intend to manipulate the currency using it or because you intend to
prevent other people from manipulating the currency.  The latter
motivation is potentially subject to a tragedy of the commons— you
don't need to run a full validating node as long as 'enough' other
people do, and enough is a nice slippery slope to zero.   Right now
just the random computers I— some random geek— had at home prior to
Bitcoin could store over a hundred years of max size blocks and
process the maximum rate of transactions.   With the costs so low
there isn't any real question about a consolidation of validation
making Bitcoin pointless.  You could probably increase the scale 10x
without breaking that analysis  but beyond that unless the
cost-per-scale goes down a highly consolidated future seems likely.
40 years from now why would people use Bitcoin over centralized
private banknotes like paypal or democratic government controlled
currencies?

Perhaps Bitcoin transaction could transition to being more of the
same— controlled by a consortium of banks, exchanging gigabyte blocks
over terabit ethernet, but I think that would be sad.  An alternative
which was autonomous and decentralized even if the transactions were
somewhat slow or costly would be excellent competition for everything
else, and it's something I think man kind ought to have.