From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 0499D267 for ; Wed, 29 Jul 2015 10:43:55 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pd0-f171.google.com (mail-pd0-f171.google.com [209.85.192.171]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A9644FA for ; Wed, 29 Jul 2015 10:43:53 +0000 (UTC) Received: by pdrg1 with SMTP id g1so3955453pdr.2 for ; Wed, 29 Jul 2015 03:43:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=vxmLhsGPl7q8y1KI3loPi6Ldudcdqy1A6jCVPcczfi4=; b=ZiltEJzsDspwvJvJtlRtV6Vg/cZQdv4ysgwNvp4ndXXT/aYrG0vBONpwtCxhckRqC9 zzLM1x0FXUplc25N9KnHEuCU0NyYzmTOsVNlJJs+uvdb0g//oA7yuYmkun9RyLWlOwM6 0AkUHvFaBoNsCFxJ9xs12ceOrjjtxAH88Z6AUbugtCL8rPPm9XW6TADpUA621To5jZyv s12ca/NzNC42TnPqIkvSBYq/vJjuO10ioexe0ey4o1bbYiKoyw2iroG7CQ3CUUdrczai mm4LfzcGCwa98blVLQYdh1e2UgZhLnoqI5vVKUe3+3ARiG6x+T6+M7oBGriIHGd1O7hQ dtZg== X-Received: by 10.70.55.165 with SMTP id t5mr91239295pdp.102.1438166633394; Wed, 29 Jul 2015 03:43:53 -0700 (PDT) Received: from [192.168.1.107] (cpe-76-167-237-202.san.res.rr.com. [76.167.237.202]) by smtp.gmail.com with ESMTPSA id oi17sm40006530pdb.74.2015.07.29.03.43.51 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 29 Jul 2015 03:43:52 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2098\)) Content-Type: multipart/signed; boundary="Apple-Mail=_28A5597D-F863-4CFD-ABFE-D62D74B591CB"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5 From: Eric Lombrozo In-Reply-To: Date: Wed, 29 Jul 2015 03:43:50 -0700 Message-Id: References: <1B7F00D3-41AE-44BF-818D-EC4EF279DC11@gmail.com> To: Mike Hearn X-Mailer: Apple Mail (2.2098) X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev Subject: Re: [bitcoin-dev] Why Satoshi's temporary anti-spam measure isn't temporary X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jul 2015 10:43:55 -0000 --Apple-Mail=_28A5597D-F863-4CFD-ABFE-D62D74B591CB Content-Type: multipart/alternative; boundary="Apple-Mail=_3003F103-A845-4886-A060-406FA39685D3" --Apple-Mail=_3003F103-A845-4886-A060-406FA39685D3 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jul 29, 2015, at 2:59 AM, Mike Hearn wrote: >=20 > I do love history lessons from people who weren't actually there. >=20 > Let me correct your misconceptions. >=20 >=20 > Initially there was no block size limit - it was thought that the fee = market would naturally develop and would impose economic constraints on = growth. >=20 > The term "fee market" was never used back then, and Satoshi did not = ever postulate economic constraints on growth. Back then the talk was = (quite sensibly) how to grow faster, not how to slow things down! Irrelevant what term was used - and as brilliant as Satoshi might have = been at some things, he obviously got this one wrong. >=20 > But this hypothesis failed after a sudden influx of new uses. It was = still too easy to attack the network. This idea had to wait until the = network was more mature to handle things. >=20 > No such event happened, and the hypothesis of which you talk never = existed. >=20 Nobody threatened to start mining huge blocks given how relatively = inexpensive it was to mine back then? >=20 > Enter a =E2=80=9Ctemporary=E2=80=9D anti-spam measure - a one megabyte = block size limit. >=20 > The one megabyte limit was nothing to do with anti spam. It was a = quick kludge to try and avoid the user experience degrading = significantly in the event of a "DoS block", back when everyone used = Bitcoin-Qt. The fear was that some malicious miner would generate = massive blocks and make the wallet too painful to use, before there were = any alternatives. I thought I clarified this in an earlier post - I meant DoS. Please = don=E2=80=99t digress on such stupid technicalities. > The plan was to remove it once SPV wallets were widespread. But = Satoshi left before that happened. >=20 Guess what? SPV wallets are still not particularly widespread=E2=80=A6and = those that are out there are notoriously terrible at detecting network = forks and making sure they are on the right one. >=20 > Now on to your claims: >=20 > 1) We never really got to test things out=E2=80=A6a fee market never = really got created, we never got to see how fees would really work in = practice. >=20 > The limit had nothing to do with fees. Satoshi explicitly wanted free = transactions to last as long as possible. Something has to limit block sizes in practice. Perhaps Satoshi was not = constrained by finite computational resources, but the rest of us sure = are. The fact that without imposing a hardcoded limit Satoshi couldn=E2=80= =99t figure out a way to keep the DoS-block guys away suggests he = didn=E2=80=99t have this fully worked out. I understand that initially it was desirable that transactions be = free=E2=80=A6but surely even Satoshi understood this couldn=E2=80=99t be = perpetually self-sustaining=E2=80=A6and that the ability to bid for = inclusion in blocks would eventually become a crucial component of the = network. Or were fees just added for decoration? We=E2=80=99re already more than six years into this. When were these = mechanisms going to be developed and tested? After 10 years? 20? Perhaps = after 1024 = years?(https://github.com/bitcoin/bips/blob/master/bip-0042.mediawiki = ) >=20 > 2) Turns out the vast majority of validation nodes have little if = anything to do with mining - validators do not get = compensated=E2=80=A6validation cost is externalized to the entire = network. >=20 > Satoshi explicitly envisioned a future where only miners ran nodes, so = it had nothing to do with this either. And Satoshi was dead wrong. As others have pointed out in this thread, = while this is certainly of historical interest, it is irrelevant from an = engineering perspective. > Validators validate for themselves. Calculating a local UTXO set and = then not using it for anything doesn't help anyone. SPV wallets need = filtering and serving capability, but a computer can filter and serve = the chain without validating it. Right. Turns out the ledger structure is terrible for constructing the = kinds of proofs that are most important to validators - i.e. whether an = output exists, what its script and amounts are, whether it=E2=80=99s = been spent, etc=E2=80=A6 Despite Satoshi=E2=80=99s brilliance, software architecture was = obviously not his strongest suit. But it didn=E2=80=99t really matter at = the beginning since this was really an experiment=E2=80=A6and he = succeeded in making his point. > The only purposes non-mining, non-rpc-serving, = non-Qt-wallet-sustaining full nodes are needed for with today's network = are: > Filtering the chain for bandwidth constrained SPV wallets (nb: you can = run an SPV wallet that downloads all transactions if you want). But this = could be handled by specialised nodes, just like we always imagined in = future not every node will serve the entire chain but only special = "archival nodes" >=20 > Relaying validated transactions so SPV wallets can stick a thumb into = the wind and heuristically guess whether a transaction is valid or not. = This is useful for a better user interface. >=20 > Storing the mempool and filtering/serving it so SPV wallets can find = transactions that were broadcast before they started, but not yet = included in a block. This is useful for a better user interface. > Outside of serving lightweight P2P wallets there's no purpose in = running a P2P node if you aren't mining, or using it as a trusted node = for your own operations. >=20 > And if one day there aren't enough network nodes being run by = volunteers to service all the lightweight wallets, then we can easily = create an incentive scheme to fix that. Yes, let=E2=80=99s wait until things are about to break before even = beginning to address the issue=E2=80=A6because we can =E2=80=9Ceasily = create=E2=80=9D anything we haven=E2=80=99t invented yet at the last = minute. >=20 >=20 > 3) Miners don=E2=80=99t even properly validate blocks. And the bigger = the blocks get, the greater the propensity to skip this step. Oops! >=20 > Miners who don't validate have a habit of bleeding money: that's the = system working as designed. >=20 Erm=E2=80=A6most miners just trust mining pool operators to validate = blocks for them=E2=80=A6and some of the biggest pools have been = blatantly cutting corners. Yes, a few pools might have temporarily bled = a little=E2=80=A6but properly validating is probably not the equilibrium = strategy=E2=80=A6and as time goes on, they are likely to start cutting = corners again. Whether they ultimately bleed money isn=E2=80=99t really = the point - many believe that cutting corners is actually a rational = strategy. If you want to discuss the game theory behind this, fine=E2=80=A6= but the fact some of the biggest mining pool operators are on record = saying they are likely to continue doing this is enough to seriously put = to question one of the most fundamental assumptions behind the network = security model. >=20 > 4) A satisfactory mechanism for thin clients to be able to securely = obtain reasonably secure, short proofs for their transactions never = materialized. >=20 > It did. I designed it. The proofs are short and "reasonably secure" in = that it would be a difficult and expensive attack to mount. You have my respect for BIP37, Mike. I know you can do amazing work. You = actually made SPV semi-useful despite inheriting such crappy data = structures. This is indeed to be respected. >=20 > But as is so often the case with Bitcoin Core these days, someone who = came along much later has retroactively decided that the work done so = far fails to meet some arbitrary and undefined level of perfection. = "Satisfactory" and "reasonably secure" don't mean anything, especially = not coming from someone who hasn't done the work, so why should anyone = care about that opinion of yours? Not done the work? I=E2=80=99m one of the very few developers in this space that has = actually tried *hard* to make your BIP37 work. Amongst the desktop = wallets listed on bitcoin.org , there are only two = that have always supported SPV (or at least I think MultiBit has always = supported it, perhaps I=E2=80=99m wrong). One is MultiBit, the other one = is mine. I give you credit for your work=E2=80=A6perhaps you could be = generous enough to extend me some credit too? --Apple-Mail=_3003F103-A845-4886-A060-406FA39685D3 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On Jul 29, 2015, at 2:59 AM, Mike Hearn <hearn@vinumeris.com>= wrote:

I do love history lessons from people who weren't = actually there.

Let = me correct your misconceptions.


Initially there = was no block size limit - it was thought that the fee market would = naturally develop and would impose economic constraints on growth. =

The = term "fee market" was never used back then, and Satoshi did not ever = postulate economic constraints on growth. Back then the talk was (quite = sensibly) how to grow faster, not how to slow things = down!

Irrelevant what term was used - and as brilliant as = Satoshi might have been at some things, he obviously got this one = wrong.
 

But this hypothesis failed after a sudden influx = of new uses. It was still too easy to attack the network. This idea had = to wait until the network was more mature to handle things.

No such event happened, and the hypothesis of which you talk = never existed.


Nobody threatened to start mining huge blocks = given how relatively inexpensive it was to mine back then?

 
Enter a =E2=80=9Ctemporary=E2=80=9D anti-spam = measure - a one megabyte block size limit.
The one megabyte limit was nothing to = do with anti spam. It was a quick kludge to try and avoid the user = experience degrading significantly in the event of a "DoS block", back = when everyone used Bitcoin-Qt. The fear was that some malicious miner = would generate massive blocks and make the wallet too painful to use, = before there were any = alternatives.

I thought I clarified this in an earlier post - I = meant DoS. Please don=E2=80=99t digress on such stupid = technicalities.


The plan was to remove it once SPV wallets were widespread. = But Satoshi left before that happened.


Guess what? SPV wallets are still not particularly = widespread=E2=80=A6and those that are out there are notoriously terrible = at detecting network forks and making sure they are on the right = one.

 
Now on to your claims:

1) We never really got to test things out=E2=80=A6= a fee market never really got created, we never got to see how fees = would really work in practice.

The limit had nothing to = do with fees. Satoshi explicitly wanted free transactions to last as = long as = possible.

Something has to limit block sizes in practice. = Perhaps Satoshi was not constrained by finite computational resources, = but the rest of us sure are. The fact that without imposing a hardcoded = limit Satoshi couldn=E2=80=99t figure out a way to keep the DoS-block = guys away suggests he didn=E2=80=99t have this fully worked = out.

I understand that initially it = was desirable that transactions be free=E2=80=A6but surely even Satoshi = understood this couldn=E2=80=99t be perpetually self-sustaining=E2=80=A6an= d that the ability to bid for inclusion in blocks would eventually = become a crucial component of the network. Or were fees just added for = decoration?

We=E2=80=99re already = more than six years into this. When were these mechanisms going to be = developed and tested? After 10 years? 20? Perhaps after 1024 years?(https://github.com/bitcoin/bips/blob/master/bip-0042.mediawiki<= /a>)

 
2) Turns out the = vast majority of validation nodes have little if anything to do with = mining - validators do not get compensated=E2=80=A6validation cost is = externalized to the entire network.

Satoshi explicitly envisioned a future where only miners ran = nodes, so it had nothing to do with this = either.

And Satoshi was dead wrong. As others have pointed out = in this thread, while this is certainly of historical interest, it is = irrelevant from an engineering perspective.


Validators validate for themselves. Calculating a local UTXO = set and then not using it for anything doesn't help anyone. SPV wallets = need filtering and serving capability, but a computer can filter and = serve the chain without validating = it.

Right. Turns out the ledger structure is terrible = for constructing the kinds of proofs that are most important to = validators - i.e. whether an output exists, what its script and amounts = are, whether it=E2=80=99s been spent, etc=E2=80=A6

Despite Satoshi=E2=80=99s brilliance, software = architecture was obviously not his strongest suit. But it didn=E2=80=99t = really matter at the beginning since this was really an experiment=E2=80=A6= and he succeeded in making his point.

The only purposes non-mining, = non-rpc-serving, non-Qt-wallet-sustaining full nodes are needed for with = today's network are:
  1. Filtering the chain for bandwidth constrained SPV wallets = (nb: you can run an SPV wallet that downloads all transactions if you = want). But this could be handled by specialised nodes, just like we = always imagined in future not every node will serve the entire chain but = only special "archival nodes"

  2. Relaying validated transactions so SPV wallets can stick a = thumb into the wind and heuristically guess whether a transaction is = valid or not. This is useful for a better user interface.

  3. Storing the mempool and = filtering/serving it so SPV wallets can find transactions that were = broadcast before they started, but not yet included in a block. This is = useful for a better user interface.
Outside of = serving lightweight P2P wallets there's no purpose in running a P2P node = if you aren't mining, or using it as a trusted node for your own = operations.

And if one day there aren't enough network nodes being run by = volunteers to service all the lightweight wallets, then we can easily = create an incentive scheme to fix = that.

Yes, let=E2=80=99s wait until things are about to = break before even beginning to address the issue=E2=80=A6because we can = =E2=80=9Ceasily create=E2=80=9D anything we haven=E2=80=99t invented yet = at the last minute.

 

3) Miners don=E2=80=99t even properly validate = blocks. And the bigger the blocks get, the greater the propensity to = skip this step. Oops!

Miners who don't validate have a habit = of bleeding money:   that's the system working as = designed.


Erm=E2=80=A6most miners just trust mining pool = operators to validate blocks for them=E2=80=A6and some of the biggest = pools have been blatantly cutting corners. Yes, a few pools might have = temporarily bled a little=E2=80=A6but properly validating is probably = not the equilibrium strategy=E2=80=A6and as time goes on, they are = likely to start cutting corners again. Whether they ultimately bleed = money isn=E2=80=99t really the point - many believe that cutting corners = is actually a rational strategy. If you want to discuss the game theory = behind this, fine=E2=80=A6but the fact some of the biggest mining pool = operators are on record saying they are likely to continue doing this is = enough to seriously put to question one of the most fundamental = assumptions behind the network security model.

 
4) A satisfactory mechanism for thin clients to = be able to securely obtain reasonably secure, short proofs for their = transactions never materialized.

It did. I designed it. = The proofs are short and "reasonably secure" in that it would be a = difficult and expensive attack to = mount.

You have my respect for BIP37, Mike. I know you = can do amazing work. You actually made SPV semi-useful despite = inheriting such crappy data structures. This is indeed to be = respected.


But as is so often the case with = Bitcoin Core these days, someone who came along much later has = retroactively decided that the work done so far fails to meet some = arbitrary and undefined level of perfection. "Satisfactory" and = "reasonably secure" don't mean anything, especially not coming from = someone who hasn't done the work, so why should anyone care about that = opinion of = yours?

Not done the work?

I=E2=80=99m one of the very few developers in this = space that has actually tried *hard* to make your BIP37 work. Amongst = the desktop wallets listed on bitcoin.org, there are only two that have always = supported SPV (or at least I think MultiBit has always supported it, = perhaps I=E2=80=99m wrong). One is MultiBit, the other one is mine. I = give you credit for your work=E2=80=A6perhaps you could be generous = enough to extend me some credit too?

= --Apple-Mail=_3003F103-A845-4886-A060-406FA39685D3-- --Apple-Mail=_28A5597D-F863-4CFD-ABFE-D62D74B591CB Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJVuK5mAAoJEJNAI64YFENUzcMP/iQeASDxU/sifAJa/FqBoVTK 7rR4XYXDKxc4NutUW7VbVxF0GNwMMU3t8Fre5MAuYdDW54bRiIaSLSBW3r+x4QKl EdKiWnEtIhRGuc2r6tIlAH8arZLaZMr9Oh0UrahgToWTKm0wmnv0tIWfFu3dnWpK wQgCAZMJaIkOHNoFqVftBIzMncaJCNH3GHj6rWJNoK7vjOPzIqeVZsUdEpSIovJ/ 8juEbwWM9XZYO/K4UOrjkJm964G4d/PAV4fOMW4AZ4L1aPGM4ICPMdUDUfFQAxiu BamUk5Uz5sKTOuUyEyKj/uLWgtBxifZfkk6kmV328B+KtrWel1OReJEJnwi8Q4U7 gxr0OSsR77VFNMH4w2rgU2K8SX9BdD0I+xJR6yCmfTGriD+f28VWhxaXZuADKunb sOo1U14xeJeTRwSRqTjj26sb9RntBXSp1y9jTPAPQjoy3zSq0HVDncqzLWqhcwJ6 Z6BoHBN2zAz1axExbOtBIjK0tLmfKGSIfAK7v2yMRRFKW9efZ3ZYgQwHK8d22C0y gQMlCJxwh5X4ZAHto2FxtBd5hnC3xSUzt5S2Fcs2qJfxvAJmPjVWqlehucuPFmel JMnm7LkK+wGFebdig7idSy23FlCMQmn8+AYZJ7C1fq/gzll33mBo9R2misGOFdki K6LNXZWckyDfc5Guy91O =W/My -----END PGP SIGNATURE----- --Apple-Mail=_28A5597D-F863-4CFD-ABFE-D62D74B591CB--