From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UnaH4-0003Rv-Ju for bitcoin-development@lists.sourceforge.net; Fri, 14 Jun 2013 20:07:06 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of petertodd.org designates 62.13.148.96 as permitted sender) client-ip=62.13.148.96; envelope-from=pete@petertodd.org; helo=outmail148096.authsmtp.net; Received: from outmail148096.authsmtp.net ([62.13.148.96]) by sog-mx-1.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1UnaH3-0003mb-4E for bitcoin-development@lists.sourceforge.net; Fri, 14 Jun 2013 20:07:06 +0000 Received: from mail-c232.authsmtp.com (mail-c232.authsmtp.com [62.13.128.232]) by punt9.authsmtp.com (8.14.2/8.14.2/Kp) with ESMTP id r5EK6vTb058478; Fri, 14 Jun 2013 21:06:57 +0100 (BST) Received: from petertodd.org (petertodd.org [174.129.28.249]) (authenticated bits=128) by mail.authsmtp.com (8.14.2/8.14.2/) with ESMTP id r5EK6skj054432 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 14 Jun 2013 21:06:56 +0100 (BST) Date: Fri, 14 Jun 2013 16:06:54 -0400 From: Peter Todd To: Luke-Jr Message-ID: <20130614200654.GB11509@petertodd.org> References: <20130527111149.GB8955@tilt> <20130531165758.GA29135@petertodd.org> <20130610210913.GA17242@petertodd.org> <201306102123.15732.luke@dashjr.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="QTprm0S8XgL7H0Dt" Content-Disposition: inline In-Reply-To: <201306102123.15732.luke@dashjr.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Server-Quench: f70e0877-d52d-11e2-b10b-0025903375e2 X-AuthReport-Spam: If SPAM / abuse - report it at: http://www.authsmtp.com/abuse X-AuthRoute: OCd2Yg0TA1ZNQRgX IjsJECJaVQIpKltL GxAVKBZePFsRUQkR aAdMdwYUEkAaAgsB AmUbWVVeUFt7XGU7 ag1VcwRfa1RMVxto VEFWR1pVCwQmQxl0 fRhdCUNycgFCe3w+ Z0ZjX3UVCkN6IEZ1 RUlJQWgEMXphaTUd TRJZd1FJcANIexZF aVN4USYPLwdSbGoL NQ4vNDcwO3BTJTpY RgYVKF8UXXNDMjM3 ShYeBzwrHF8EQSp7 Kh0gK1gTdAAI X-Authentic-SMTP: 61633532353630.1019:706 X-AuthFastPath: 0 (Was 255) X-AuthSMTP-Origin: 174.129.28.249/587 X-AuthVirus-Status: No virus detected - but ensure you scan with your own anti-virus system. X-Spam-Score: -1.5 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain -0.0 SPF_PASS SPF: sender matches SPF record X-Headers-End: 1UnaH3-0003mb-4E Cc: bitcoin-development@lists.sourceforge.net Subject: Re: [Bitcoin-development] Decentralizing mining X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Jun 2013 20:07:06 -0000 --QTprm0S8XgL7H0Dt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 10, 2013 at 09:23:14PM +0000, Luke-Jr wrote: > > They always (usually?) work on the subset of transactions common to both > > the pool's getblocktemplate and their local one. When they find a share > > if it doesn't meet difficulty they just hand it to the pool. Currently > > that is done by handing the whole block over, correct? I know the BIP > > says otherwise, but we should optimize this to just hand over tx hashes > > where possible. >=20 > The plan was to tell the pool it doesn't need to send transactions at all= , and=20 > only work on the ones from bitcoind. Currently, share submissions are jus= t the=20 > block header and coinbase transaction; in this case, however, the miner w= ill=20 > need to send merkle links also, possibly just once via a block proposal i= n=20 > advance. It strikes me that this would work best if the pool has a mempool with child-pays-for-parent support where conflicts *are* allowed. IE you record whatever transactions you know about, conflicting or not, calculate which ones gives you the most fees/kb, and then figure out which set of non-conflicting ones are optimal. Of course, "optimal" is the knapsack problem... Now you can easilly tell the miners working on shares for you which tx's would be optimal if they wish to know, and at the same time whatever shares they send you are most likely to include transactions in your mempool inventory, and thus they can send just tx hashes to reduce bandwidth. Part of the broader issue that when we send peers INV advertisements we should be telling them what the fee/kb is so our peers can prioritize properly. That'd also help for the case where you want to broadcast two transactions in a row, but the pair is only profitable because the second is paying the fee for the first. Speaking of, the way we tell peers about new blocks is really suboptimal: we tell every peer, in no particular order, about a new block via a block INV message, and then we give them the new block in parallel. I was looking through comp-sci papers on optimal flood-fill/gossip algorithms for random graph networks and it appears that optimal is to spend all your bandwidth to send the message to your fastest peer first, followed by your next fastest and so on. This works best because you get the exponential growth scaling faster by propagating the message as "deep" as possible in the network, and it then can flood outwards from there. Just sorting the peer list by #inv-recevied/time when doing INV pushes and when attending to incoming messages would probably be a big improvement. > > If the share does meet difficulty, hand it to both the pool and the > > local bitcoind. Should hand it to the pool first though, because the > > pool likely has the fastest block propagation, then hand it to local > > bitcoind. An optimized version may want to have some record of measured > > bandwidth - this applies Bitcoin in general too, although also has other > > issues. >=20 > Currently, BFGMiner is doing submission to the pool, waiting for a respon= se,=20 > then submitting to a local bitcoind. This is because the pool might need = to=20 > receive/record the share before it processes the block on bitcoind, or yo= u=20 > could lose credit for it. The response from the pool is rather small (a s= ingle=20 > TCP packet), so this shouldn't delay much longer. Right, I guess the pool wants to be sure you were actually the one who found the share, rather than just someone who was lucky enough to see it on the network and submitted it as your own. > > ## Reducing bandwidth > >=20 > > How about for normal shares we just pass the block header, and have the > > pool randomly pick a subset of transactions to audit? Any fraud cancels > > the users shares. This will work best in conjunction with a UTXO proof > > tree to prove fees, or by just picking whole shares randomly to audit. >=20 > Might as well just use higher difficulty shares (each one audited) for th= e=20 > same effect. Block proposals allow the miner to tell the pool its transac= tion=20 > set once (per txset change) for any number of shares. That's a good point - the current practice most pools seem to follow of about a share per second seems very excessive to me. On the other hand, it does have good user optics. The right solution might be something akin to P2Pool where the UI level is telling the user shares are being found so it's clear "stuff is happening", but under the hood only a small subset are ever sent to the pool. > > # Pool work > >=20 > > So does eliopool already accept arbitrary shares like this and do the > > correct accounting already? (IE adjust share amount based on fees?) What > > happens when the pool doesn't get the share directly, but does see the > > new block? > >=20 > > + possible protocol extensions >=20 > I don't follow. What part don't you follow? > > # Miner work > >=20 > > Basically we need code to merge the two block templates together to find > > commonality. I guess you probably want to implement this in bfgminer > > first, so add the code to libblkmaker first, then maybe python-blkmaker. >=20 > libblkmaker's API was designed for this from the start, so it should be f= airly=20 > easily implemented. Good. > > We also want an automatic fallback to local solo mining if the pool > > can't be contacted. > >=20 > > + possible protocol extensions >=20 > Failover already functions, but probably could use a rewrite... Sounds good. --=20 'peter'[:-1]@petertodd.org 000000000000006ed83d97e7266a66dcb057cd53750795bd209076fa0a34d4ec --QTprm0S8XgL7H0Dt Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlG7d94ACgkQpEFN739thozcBwCffMU//scgRaKgy69KipZgmMnU +lEAoI4YI6dnZ3LanuuIP0e2Q+R4yRLP =T0RA -----END PGP SIGNATURE----- --QTprm0S8XgL7H0Dt--