From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Yq9pC-0000cM-5U for bitcoin-development@lists.sourceforge.net; Thu, 07 May 2015 00:38:02 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.223.179 as permitted sender) client-ip=209.85.223.179; envelope-from=gmaxwell@gmail.com; helo=mail-ie0-f179.google.com; Received: from mail-ie0-f179.google.com ([209.85.223.179]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1Yq9pA-0000ql-4h for bitcoin-development@lists.sourceforge.net; Thu, 07 May 2015 00:38:02 +0000 Received: by iedfl3 with SMTP id fl3so30980552ied.1 for ; Wed, 06 May 2015 17:37:54 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.42.129.73 with SMTP id p9mr1165073ics.48.1430959074800; Wed, 06 May 2015 17:37:54 -0700 (PDT) Received: by 10.107.15.82 with HTTP; Wed, 6 May 2015 17:37:54 -0700 (PDT) In-Reply-To: <554A91BE.6060105@bluematt.me> References: <554A91BE.6060105@bluematt.me> Date: Thu, 7 May 2015 00:37:54 +0000 Message-ID: From: Gregory Maxwell To: Matt Corallo Content-Type: text/plain; charset=UTF-8 X-Spam-Score: -1.6 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (gmaxwell[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 AWL AWL: Adjusted score from AWL reputation of From: address X-Headers-End: 1Yq9pA-0000ql-4h Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Block Size Increase X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 May 2015 00:38:02 -0000 On Wed, May 6, 2015 at 10:12 PM, Matt Corallo wrote: > Recently there has been a flurry of posts by Gavin at > http://gavinandresen.svbtle.com/ which advocate strongly for increasing > the maximum block size. However, there hasnt been any discussion on this > mailing list in several years as far as I can tell. Thanks Matt; I was actually really confused by this sudden push with not a word here or on Github--so much so that I responded on Reddit to people pointing to commits in Gavin's personal repository saying they were reading too much into it. So please forgive me for the more than typical disorganization in this message; I've been caught a bit flatfooted on this and I'm trying to catch up. I'm juggling a fair amount of sudden pressure in my mailbox, and trying to navigate complex discussions in about eight different forums concurrently. There have been about a kazillion pages of discussion elsewhere (e.g. public IRC and Bitcointalk; private discussions in the past), not all of which is well known, and I can't hope to summarize even a tiny fraction of it in a single message-- but that's no reason to not start on it. > Block size is a question to which there is no answer, but which > certainly has a LOT of technical tradeoffs to consider. There are several orthogonal angles from which block size is a concern (both increases and non-increases). Most of them have subtle implications and each are worth its own research paper or six, so it can be difficult to only touch them slightly without creating a gish gallop that is hard to respond to. We're talking about tuning one of the fundamental scarcities of the Bitcoin Economy and cryptosystem--leaving the comfort of "rule by math" and venturing into the space of political decisions; elsewhere you'd expect to see really in-depth neutral analysis of the risks and tradeoffs, technically and economically. And make no mistake: there are real tradeoffs here, though we don't know their exact contours. Fundamentally this question exposes ideological differences between people interested in Bitcoin. Is Bitcoin more of a digital gold or is it more of a competitor to Square? Is Bitcoin something that should improve personal and commercial autonomy from central banks? From commercial banks? Or from just the existing status-quo commercial banks? What are people's fundamental rights with Bitcoin? Do participants have a right to mine? How much control should third parties have over their transactions? How much security must be provided? Is there a deadline for world domination or bust? Is Bitcoin only for the developed world? Must it be totally limited by the most impoverished parts of the world? Bitcoin exists at the intersection of many somewhat overlapping belief systems--and people of many views can find that Bitcoin meets their needs even when they don't completely agree politically. When Bitcoin is changed fundamentally, via a hard fork, to have different properties, the change can create winners or losers (if nothing else, then in terms of the kind of ideology supported by it). There are non-trivial number of people who hold extremes on any of these general belief patterns; Even among the core developers there is not a consensus on Bitcoin's optimal role in society and the commercial marketplace. To make it clear how broad the views go, even without getting into monetary policy... some people even argue that Bitcoin should act as censor-resistant storage system for outlawed content; -- I think this view is unsound, not achievable with the technology, and largely incompatible with Bitcoin's use as a money (because it potentially creates an externalized legal/harassment liability for node operators); but these are my personal value judgments; the view is earnestly held by more than a few; and that's a group that certainly wants the largest possible blocksizes (though even then that won't be enough). The subject is complicated even more purely on the technical side by the fact that Bitcoin has a layered security model which is not completely defined or understood: Bitcoin is secure if a majority of hashrate is "honest" (where "honesty" is a technical term which means "follows the right rules" without fail, even at a loss), but why might it be honest? That sends us into complex economic and social arguments, and the security thresholds start becoming worse when we assume some miners are economically rational instead of "honest". > increase in the near future. Long-term incentive compatibility requires > that there be some fee pressure, and that blocks be relatively > consistently full or very nearly full. What we see today are To elaborate, in my view there is a at least a two fold concern on this particular ("Long term Mining incentives") front: One is that the long-held argument is that security of the Bitcoin system in the long term depends on fee income funding autonomous, anonymous, decentralized miners profitably applying enough hash-power to make reorganizations infeasible. For fees to achieve this purpose, there seemingly must be an effective scarcity of capacity. The fact that verifying and transmitting transactions has a cost isn't enough, because all the funds go to pay that cost and none to the POW "artificial" cost; e.g., if verification costs 1 then the market price for fees should converge to 1, and POW cost will converge towards zero because they adapt to whatever is being applied. Moreover, the transmission and verification costs can be perfectly amortized by using large centralized pools (and efficient differential block transmission like the "O(1)" idea) as you can verify one time instead of N times, so to the extent that verification/bandwidth is a non-negligible cost to miners at all, it's a strong pressure to centralize. You can understand this intuitively: think for example of carbon credit cap-and-trade: the trade part doesn't work without an actual cap; if everyone was born with a 1000 petaton carbon balance, the market price for credits would be zero and the program couldn't hope to share behavior. In the case of mining, we're trying to optimize the social good of POW security. (But the analogy applies in other ways too: increases to the chain side are largely an externality; miners enjoy the benefits, everyone else takes the costs--either in reduced security or higher node operating else.) This area has been subject to a small amount of academic research (e.g. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2400519). But there is still much that is unclear. The second is that when subsidy has fallen well below fees, the incentive to move the blockchain forward goes away. An optimal rational miner would be best off forking off the current best block in order to capture its fees, rather than moving the blockchain forward, until they hit the maximum. That's where the "backlog" comment comes from, since when there is a sufficient backlog it's better to go forward. I'm not aware of specific research into this subquestion; it's somewhat fuzzy because of uncertainty about the security model. If we try to say that Bitcoin should work even in the face of most miners being profit-maximizing instead of altruistically-honest, we must assume the chain will not more forward so long as a block isn't full. In reality there is more altruism than zero; there are public pressures; there is laziness, etc. One potential argument is that maybe miners would be _regulated_ to behave correctly. But this would require undermining the openness of the system--where anyone can mine anonymously--in order to enforce behavior, and that same enforcement mechanism would leave a political level to impose additional rules that violate the extra properties of the system. So far the mining ecosystem has become incredibly centralized over time. I believe I am the only remaining committer who mines, and only a few of the regular contributors to Bitcoin Core do. Many participants have never mined or only did back in 2010/2011... we've basically ignored the mining ecosystem, and this has had devastating effects, causing a latent undermining of the security model: hacking a dozen or so computers--operated under totally unknown and probably not strong security policies--could compromise the network at least at the tip... Rightfully we should be regarding this an an emergency, and probably should have been have since 2011. This doesn't bode well for our ability to respond if a larger blocksize goes poorly. In kicking the can with the trivial change to just bump the size, are we making an implicit decision to go down a path that has a conclusion we don't want? (There are also shorter term mining incentives concerns; which Peter Todd has written more about, that I'll omit for now) > pretending these systems scale. Thus, instead of working on technologies > which bring Bitcoin's trustlessness to systems which scale beyond a I made a few relevant points back in 2011 (https://en.bitcoin.it/w/index.php?title=Scalability&action=historysubmit&diff=14273&oldid=14112) after Dan Kaminsky argued that Bitcoin's decentralization was pretext: that it was patently centralized since scaling directly in the network would undermine decentralization, that the Bitcoin network necessarily makes particular tradeoffs which prevent it from concurrently being all things to all people. But tools like the Lightning network proposal could well allow us to hit a greater spectrum of demands at once--including secure zero-confirmation (something that larger blocksizes reduce if anything), which is important for many applications. With the right technology I believe we can have our cake and eat it too, but there needs to be a reason to build it; the security and decentralization level of Bitcoin imposes a _hard_ upper limit on anything that can be based on it. Another key point here is that the small bumps in blocksize which wouldn't clearly knock the system into a largely centralized mode--small constants--are small enough that they don't quantitatively change the operation of the system; they don't open up new applications that aren't possible today. Deathandtaxes on the forum argued that Bitcoin needs a several hundred megabyte blocksize to directly meet the worldwide transaction needs _without retail_... Why without retail? Retail needs near instant soft security, which cannot be achieved directly with a global decentralized blockchain. I don't think 1MB is magic; it always exists relative to widely-deployed technology, sociology, and economics. But these factors aren't a simple function; the procedure I'd prefer would be something like this: if there is a standing backlog, we-the-community of users look to indicators to gauge if the network is losing decentralization and then double the hard limit with proper controls to allow smooth adjustment without fees going to zero (see the past proposals for automatic block size controls that let miners increase up to a hard maximum over the median if they mine at quadratically harder difficulty), and we don't increase if it appears it would be at a substantial increase in centralization risk. Hardfork changes should only be made if they're almost completely uncontroversial--where virtually everyone can look at the available data and say "yea, that isn't undermining my property rights or future use of Bitcoin; it's no big deal". Unfortunately, every indicator I can think of except fee totals has been going in the wrong direction almost monotonically along with the blockchain size increase since 2012 when we started hitting full blocks and responded by increasing the default soft target. This is frustrating; from a clean slate analysis of network health I think my conclusion would be to _decrease_ the limit below the current 300k/txn/day level. This is obviously not acceptable, so instead many people--myself included--have been working feverishly hard behind the scenes on Bitcoin Core to increase the scalability. This work isn't small-potatoes boring software engineering stuff; I mean even my personal contributions include things like inventing a wholly new generic algebraic optimization applicable to all EC signature schemes that increases performance by 4%, and that is before getting into the R&D stuff that hasn't really borne fruit yet, like fraud proofs. Today Bitcoin Core is easily >100 times faster to synchronize and relay than when I first got involved on the same hardware, but these improvements have been swallowed by the growth. The ironic thing is that our frantic efforts to keep ahead and not lose decentralization have both not been enough (by the best measures, full node usage is the lowest its been since 2011 even though the user base is huge now) and yet also so much that people could seriously talk about increasing the block size to something gigantic like 20MB. This sounds less reasonable when you realize that even at 1MB we'd likely have a smoking hole in the ground if not for existing enormous efforts to make scaling not come at a loss of decentralization. I'm curious as to what discussions people have seen; e.g., are people even here aware of these concerns? Are you aware of things like the hashcash mediated dynamic blocksize limiting? About proposals like lightning network (instant transactions and massive scale, in exchange for some short term DOS risk if a counterparty opts out)? Do people (other than Mike Hearn; I guess) think a future where everyone depends on a small number of "Google scale" node operations for the system is actually okay? (I think not, and if so we're never going to agree--but it can be helpful to understand when a disagreement is ideological).