From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1W66AV-0000Ng-RT for bitcoin-development@lists.sourceforge.net; Wed, 22 Jan 2014 22:21:08 +0000 X-ACL-Warn: Received: from imap2-1.ox.registrar-servers.com ([198.187.29.240]) by sog-mx-4.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1W66AT-0005Ux-IB for bitcoin-development@lists.sourceforge.net; Wed, 22 Jan 2014 22:21:07 +0000 Received: from [93.154.133.148] (ip-93-154-133-148.free.aero2.net.pl [93.154.133.148]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by oxmail.registrar-servers.com (Postfix) with ESMTPSA id 64A355A0051; Wed, 22 Jan 2014 17:20:57 -0500 (EST) Message-ID: <52E04446.4020106@bcdev.net> Date: Wed, 22 Jan 2014 23:20:54 +0100 From: bc User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Jorge_Tim=F3n?= , Christophe Biocca References: <52E032BD.4020206@bcdev.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. X-Headers-End: 1W66AT-0005Ux-IB Cc: "bitcoin-development@lists.sourceforge.net" Subject: Re: [Bitcoin-development] Combining big transactions with hash-only blocks to improve tps. X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 22:21:08 -0000 Jorge Tim=F3n: The node would need to first verify a block before mining on top of it. Basically a receiver would ask a sender for missing transactions if he=20 doesn't know them already before propagating or mining the block. Christophe Biocca: You're right, my idea doesn't offer any real advantage over=20 prebroadcasting of the tree and including only it's header in a block. Thanks, Eric On 22.01.2014 23:10, Jorge Tim=F3n wrote: > Maybe I'm missing something. > How do miners validate blocks if they only receive the hashes of the > transactions? > Will they mine on top of a block when they don't know if it's valid? > > > On 1/22/14, Christophe Biocca wrote: >> Comments: >> >> bc: >> - Ultimately, this helps with block propagation latency, but not with >> the bandwidth constraints themselves, because all transactions do need >> to be broadcast. >> - Most of the benefits of your approach can be obtained simply by >> prebroadcasting the entire merkle tree while you're working on it. You >> can get even bigger gains by the miners reusing large chunks of each >> other's merkle trees (which they could if they had similar transaction >> selection policies). Then there's just the headers to broadcast. >> >> Natanael: >> - Most of the block's content is important though, because I don't >> just want to know that the block is valid, I also want to know what >> changes to make to my local copy of the UTXO. So I don't know how much >> space/bandwidth you'd save. You would definitely save on signature >> checking and independent validation, but that's CPU time. >> >> On Wed, Jan 22, 2014 at 4:43 PM, Natanael wrote= : >>> Couldn't we also use the type of zkSNARK's that Zerocoin adopted to >>> prove that the hash-only blocks only have valid transactions in it, >>> since they are small and quite efficient to verify? The trouble is >>> that they're still inefficient to generate, but given powerful enough >>> computers that compiles the hashes for the block and it could likely >>> still be done fast enough to handle large amounts of transactions. Th= e >>> computer is likely not going to be the most expensive part anyway by = a >>> far margin. >>> >>> zkSNARK =3D zero-knowledge Succinct Non-interactive ARgument of Knowl= edge >>> >>> On Wed, Jan 22, 2014 at 10:06 PM, bc wrote: >>>> Pdf version: >>>> http://bcdev.net/data/bitcoin_big_tx_with_coin_join.pdf >>>> >>>> >>>> =3D=3D Combining big transactions with hash-only blocks to improve t= ps. =3D=3D >>>> >>>> =3D=3D=3D=3D Abstract: =3D=3D=3D=3D >>>> I've heard people talk about including only hashes in a block to spe= ed >>>> up the network and also about using CoinJoin to improve privacy. I'v= e >>>> not heard anyone talk about implications of combining these two >>>> techniques. I think that it would both improve network's anonymity, = but >>>> also improve tps by a few orders of magnitude. >>>> >>>> I propose two optimizations: >>>> 1. Keep only hashes of transactions included in a block. Transfer al= l tx >>>> separately. >>>> 2. Use CoinJoin to merge transactions from many users for online >>>> shopping and banking. >>>> 3. Use Jumbo transactions as a fallback for applications where CoinJ= oin >>>> is inappropriate. >>>> >>>> =3D=3D=3D=3D Keeping only hashes of tx in a block: =3D=3D=3D=3D >>>> Currently every bitcoin block includes a copy of all transactions. T= his >>>> is redundant and unnecessary, since after the transaction gets >>>> transmitted, every node learns about it in seconds. >>>> By keeping only transaction hashes in block, we can keep block >>>> propagation time from increasing. >>>> Assuming a typical tx with one or two inputs and two outputs [typica= lly >>>> 300 bytes], current 1MiB block can contain about [assuming a block e= very >>>> 10 minutes]: >>>> 1MiB / 300 bytes =3D 3300tx =3D 5.5tps >>>> >>>> By keeping only hashes in a block [32 bytes per hash]: >>>> 1MiB / 32 bytes =3D 31000tx =3D 50tps >>>> >>>> =3D=3D Benefits: =3D=3D >>>> This method allows to achieve more tps without increasing the block >>>> propagation time, which is critical for mining decentralization. >>>> It removes redundancy, since every tx has to be transmitted only onc= e. >>>> It leads to a more consistent bandwidth utilization [large transacti= ons >>>> are transmitted all the time, while blocks are kept small and easy t= o >>>> propagate]. >>>> Because a block size is a constant, mining fees would not depend on = the >>>> size of a transaction. Obviously to limit the network flood, there >>>> should be a transaction size limit. >>>> >>>> =3D=3D Problems: =3D=3D >>>> Selfish miner can keep a subset of transactions only for yourself an= d >>>> release them only with a new block. This problem can be mitigated by >>>> making nodes verify all transactions before propagating a block. The >>>> incentive will then be to mine only a well-distributed transactions = to >>>> lower orphan rate. >>>> The miner can try to sneak up invalid transaction in a block. This >>>> problem is also mitigated by not accepting a block before it gets >>>> verified. >>>> >>>> =3D=3D=3D=3D CoinJoin: =3D=3D=3D=3D >>>> If the block size keeps only hashes, a transaction can be much bigge= r. >>>> Since CoinJoin allows many people to send coins with one transaction= , >>>> the effective transaction rate can be increased considerably. >>>> >>>> =3D=3D Example: =3D=3D >>>> Let's assume the transaction size limit of 50KiB. Limit of this size >>>> allows for a CoinJoin transaction between 50KiB / 300b =3D 170 >>>> participants. >>>> So for a block of 1MiB, it would allow for 50tps * >>>> 170effective_transactions/tx =3D 8500tps. >>>> >>>> =3D=3D Benefits: =3D=3D >>>> There would be an incentive for users to use CoinJoin by default [lo= wer >>>> tx fees per effective transaction], which would greatly increase >>>> anonymity of the network. >>>> Since block size stays the same, block propagation time also stays t= he >>>> same. >>>> It doesn't require any changes to the protocol. CoinJoin transaction= s >>>> were always supported in bitcoin. >>>> >>>> =3D=3D Problems: =3D=3D >>>> 1) CoinJoin requires collaboration between many users in real-time. = It >>>> means, that transaction must be distributed to every CoinJoin >>>> participant, and every participant has to sign it before it can be >>>> released. Therefore it induces delays, which can take some time. >>>> It wouldn't be an issue with Internet banking or on-line shopping [w= here >>>> even 10 minutes per transaction is fast enough], however even 20 sec= onds >>>> can make the system unsuitable for POS payments. >>>> Potential solution: Use bigger CoinJoin user base for online payment= s >>>> [with smaller fees], and a smaller one for POS payments [with larger >>>> fees]. >>>> >>>> 2) Signing a CoinJoin transaction requires to transfer a whole >>>> transaction for a user to sign. >>>> This can sometimes take up to a few minutes on a very slow networks. >>>> >>>> 3) CoinJoin transactions are limited. They are good enough for money >>>> transfer, but for more advanced appliances CoinJoin might be inadequ= ate. >>>> >>>> =3D=3D=3D=3D Jumbo transactons: =3D=3D=3D=3D >>>> I propose another tx type as a fallback where CoinJoin is not Combin= ing >>>> big transactions with hash-only blocks to improve tps.applicable. It >>>> would remove the CoinJoin induced delays, while keeping transaction >>>> sizes big. >>>> >>>> Image: http://bcdev.net/data/jubo_transaction_description.png >>>> >>>> Transaction joiner is a service that collects transactions from clie= nts >>>> and publishes them as a Jumbo transaction. >>>> Jumbo pubkey prevents transaction from being modified. It can only b= e >>>> accepted or rejected by the miner as a whole, which should limit >>>> discrimination. >>>> >>>> =3D=3D Algorithm: =3D=3D >>>> 1) Transaction joiner sends a Jumbo pubkey hash to the client. >>>> 2) Client creates a transaction, includes a Jumbo pubkey hash and si= gns >>>> it. >>>> 3) Transaction joiner waits until there are enough transactions and >>>> releases a Jumbo transaction to the network. >>>> 4) A miner includes only a hash of a Jumbo transaction in a block, h= e >>>> cannot cherry-pick individual transactions from the bulk. >>>> 5) The network checks if every transaction inside a Jumbo transactio= n >>>> includes a Jumbo pubkey hash and if every transaction inside is vali= d. >>>> >>>> =3D=3D Benefits: =3D=3D >>>> Since the block size stays the same, block propagation time also sta= y >>>> the same. >>>> There is no need to wait for every participant to sign the transacti= on. >>>> It's therefore more suitable for POS payments. >>>> No additional network overhead for a thin client compared to a stand= ard >>>> tx. >>>> Backwards compatibility with current transaction system. >>>> >>>> =3D=3D Problems: =3D=3D >>>> 1) Jumbo transactions don't mix coins. Anonymity of the network is n= ot >>>> increased. >>>> 2) There would be an incentive to use this transaction type by defau= lt >>>> [compared to CoinJoin]. >>>> >>>> Potential solution: >>>> Make Jumbo transaction size limit lower than CoinJoin. That would ma= ke >>>> fees for these transactions higher, thus creating an incentive to on= ly >>>> use them when necessary. >>>> >>>> 3) Transaction joiner has to wait for a Jumbo transaction to be big >>>> enough before it gets released. >>>> It's not a big problem. When the network load is low, the fee requir= ed >>>> for a tx to be included should be lower, allowing for smaller Jumbo >>>> transactions. When the network load is high, it takes less time to f= ill >>>> a Jumbo transaction. >>>> >>>> =3D=3D=3D=3D References: =3D=3D=3D=3D >>>> Increasing the Network Hashing Power by reducing block propagation t= ime >>>> https://bitcointalk.org/index.php?topic=3D145066.0 >>>> >>>> CoinJoin: Bitcoin privacy for the real world >>>> https://bitcointalk.org/index.php?topic=3D279249.0 >>>> >>>> Bitcoin: A Peer-to-Peer Electronic Cash System >>>> http://bitcoin.org/bitcoin.pdf >>>> >>>> --------------------------------------------------------------------= ---------- >>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For >>>> Critical Workloads, Development Environments & Everything In Between= . >>>> Get a Quote or Start a Free Trial Today. >>>> http://pubads.g.doubleclick.net/gampad/clk?id=3D119420431&iu=3D/4140= /ostg.clktrk >>>> _______________________________________________ >>>> Bitcoin-development mailing list >>>> Bitcoin-development@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >>> >>> ---------------------------------------------------------------------= --------- >>> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >>> Learn Why More Businesses Are Choosing CenturyLink Cloud For >>> Critical Workloads, Development Environments & Everything In Between. >>> Get a Quote or Start a Free Trial Today. >>> http://pubads.g.doubleclick.net/gampad/clk?id=3D119420431&iu=3D/4140/= ostg.clktrk >>> _______________________________________________ >>> Bitcoin-development mailing list >>> Bitcoin-development@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> >> ----------------------------------------------------------------------= -------- >> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >> Learn Why More Businesses Are Choosing CenturyLink Cloud For >> Critical Workloads, Development Environments & Everything In Between. >> Get a Quote or Start a Free Trial Today. >> http://pubads.g.doubleclick.net/gampad/clk?id=3D119420431&iu=3D/4140/o= stg.clktrk >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> > >