From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Wed, 31 Jul 2024 11:02:39 -0700 Received: from mail-qk1-f191.google.com ([209.85.222.191]) by mail.fairlystable.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sZDeo-0004tw-Ql for bitcoindev@gnusha.org; Wed, 31 Jul 2024 11:02:39 -0700 Received: by mail-qk1-f191.google.com with SMTP id af79cd13be357-7a1df9ced6fsf795281185a.2 for ; Wed, 31 Jul 2024 11:02:38 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1722448952; cv=pass; d=google.com; s=arc-20160816; b=sz83lEng27VtEK7cDuK3gJo6Ex08z/9NfIrYKXyUvWqrMaUHf8gZQjtfkE40IxvCJk 2S5jZQpiM2pM2IbLEkOUsAwjpd8Spdk9SAMaexTlfoTT2/OuccrBFaPvMZ9kTMSa19ij I2uDhB7HjCSKErdUVcZnl0bqrW8jl12a+JPKRX/S7FLr7mPMHlbW0eh8lQ12HVMgJ0fL GRx+RlWFggMXjqZ0vHm+fa4Fd547jMxLCvUiXASKGyepALMWz4tI+WT3f01NF6NqXPEC CEnuEDbvYOHYSwxa/n2Fw/kUSDp0AacD28N3CXnfdBxIauDIz2G0BM/r4JHbEV6kwk19 LWZg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:in-reply-to:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :dkim-signature; bh=bqzSBrj1l29XKylgF6w1TUa/8nuExWAb+9pSD0nnCSc=; fh=YtxbUr8q8vJ1vj7I8yIw8GNQYIrgEEDMA4i2Y+uO6Vs=; b=JC+mty4dDJ/7+383gMHuHYW5iXT4BwpVbCt+Sk4d2qkI3iCORnQ/QUwldJS6vgWyhQ elnAfeOnsOtqjM4dp4SpC5z2jfii7raodQTEsr6IGnZovts1/CP02je3R5VrdSfmHwSg VT5IPdPhsK7crJArI5W9XZjT/OP/evUWHFcG69J4teyAOLjV3wiclzunNUH0d1XrFfQl m7zEXnHIBg3QztTrbnLqrmffTM7KJvxPZ84NgLDk3aGGPsujqiYtv+Ucvm9FW/ZJI0GL vbZZqJEtpREWG+LWiR06JUu/47kfCFajg/gf8dzNqNQDMJgAUyv1ycY1p4NWsUrNcgie 9t8A==; darn=gnusha.org ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of aj@erisian.com.au designates 172.104.61.193 as permitted sender) smtp.mailfrom=aj@erisian.com.au DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1722448952; x=1723053752; darn=gnusha.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:sender:from:to:cc :subject:date:message-id:reply-to; bh=bqzSBrj1l29XKylgF6w1TUa/8nuExWAb+9pSD0nnCSc=; b=rxfF7COVM+hdbnc+8LEr80JIISpRU3BOIo9+Rrl45BMFxC/YwM3UYg1LSRxdWL/b2T nKTAf1xoQFWOkiduUy/1PVYYJVK++9NEFEfBhFtmqZMl3Xqoozo95xeiSTQ1mSVA8/nX KNCh0BEy4gJ+ybxQ8AbL/8gXAt3nf2NOMxdY3y6ORc9qVNecZsHTbg6W0ZL+4zW/OQ9Z A3mGwcpceFU8GXgWNl3I65hXAr/Lmh0E9Z91jY6Fa4tjHrHl4mjctHNN++lh0K3Nxy8A kTuN2iq0M+aktKK+3kWtTYwEOOmci0Jzb3M0uxSQ+s1ZNaAc4onDZFtuoZFoctgmBAO9 ZrRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722448952; x=1723053752; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:x-beenthere :x-gm-message-state:sender:from:to:cc:subject:date:message-id :reply-to; bh=bqzSBrj1l29XKylgF6w1TUa/8nuExWAb+9pSD0nnCSc=; b=I0RiKMdE3bzDSJ0z2HusJV8BivTNNgCpF2/DYTq7s5sz3wAwCHdTBDA1EA+DGXZtKY PlNkt3hYbYFzoiOvFleqgjFQHYKLxUFUFT2/4u/3fH7jv283n5KT6y4z/LZSrwlauDLN HtiN4fHLrZZ0SLF4WKw220MwohUBoLcTO3PLIr45KRV318X2c8ZTs5Kmpv0+creXalO2 6b7gGckJ+vsRgtAFA+vx28Sxw3kDK+8DiQOSZunPj4+y7JE8rgWxBSiIyzZE9Blo2kU/ aBiEEJoGdaxKsOrhl9H0hdYZV9G6/LU+GMlM0GKk2G2u7NULHXXGCGq7fIsBhuK/LZVH xejg== Sender: bitcoindev@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUBF9ZQ+4dYJFNjZ3FPdz73UNq5XdyjDdxxa7HtzEQgqtZT8M70I/tmyDiB/y1V3cz2iUnD4ynA2O8+1G3InW2vKYTic04= X-Gm-Message-State: AOJu0Yx1b7IWZSs9i7HueOcJJ69gwFIEmYf7rNzn1TKym6HaNKe8p9Eb M/bCtOCpGkcO+CbGY4izJQlFYTLfxWgEU2ta5HdLjK3zYcB0l4z/ X-Google-Smtp-Source: AGHT+IGVDqDdEpU90Zk6qiod/mnTlLCrW04h29ncpe8OACKvrJO9JHhCNSHJ541I4HllPRSKo7KW/g== X-Received: by 2002:ac8:5dc8:0:b0:44f:ea31:cdba with SMTP id d75a77b69052e-4514f9cfbdemr1255771cf.16.1722448951917; Wed, 31 Jul 2024 11:02:31 -0700 (PDT) X-BeenThere: bitcoindev@googlegroups.com Received: by 2002:ac8:580e:0:b0:444:f4c3:34ef with SMTP id d75a77b69052e-44fe2ece76bls127096541cf.0.-pod-prod-04-us; Wed, 31 Jul 2024 11:02:30 -0700 (PDT) X-Received: by 2002:a05:620a:46a8:b0:7a2:1bc:fbbc with SMTP id af79cd13be357-7a201bcfc6amr28816385a.11.1722448950128; Wed, 31 Jul 2024 11:02:30 -0700 (PDT) Received: by 2002:a05:620a:72a1:b0:7a1:c409:aa2c with SMTP id af79cd13be357-7a2086ba1b7ms85a; Wed, 31 Jul 2024 11:00:21 -0700 (PDT) X-Received: by 2002:a05:6122:d8d:b0:4f5:1a43:de4f with SMTP id 71dfb90a1353d-4f896625faemr192895e0c.13.1722448820423; Wed, 31 Jul 2024 11:00:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1722448820; cv=none; d=google.com; s=arc-20160816; b=MBgGBn/OW7I6BU6cyhYflBy/1m0s5PMuObykjeFORLyuTj+flnvt90WQoC9RM9AkgC bbNh4aIHyPbeDSMNddi69UUBqYsQplrel6IrLjmLjhxo+x8qeEmML2l9msx9ZmWd2LYC zrS8hFx7U/LfU/zXFt67vRRx31HnsOYVmbbT2aD4uM/oe5LE4COjlRx8GpcaqrYUNNOI rW2FHbxih2txBeWRpns1Ml5r4EczKlCJ3642acILzJC5wnjCjTgJoU2HMbfnH6AysQS7 O7915n0zC0XetinOMDKrs2kIw9VLk2Lj87R4iDHdyReTd0krL/u7+rMkdIawp7gqqzvP 1e+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date; bh=9dCiXex2pAOegHKvw3pUB8ZjnvUBWRHWt1ZFqd3n4yI=; fh=jRYg04Pl0IOQLxD6rA0ou4c50cbDWVvY9M7F1jDuQR0=; b=mTBuqK8a5T+o/Yrk/lPUbGbBQOBZAiqWEBak1On1CfCuri9JRARL29HaNUgZyaUu2f tslEcysy4l6TRRY9fgS/cRGXpqzSJ4ftzAReczsB+NpyMA9Mod+sUnGoVFlTSquQAKV3 xDA4qv6AiejpFm+C7kjYgoj/KDLLKKV5sJMX5TZM0YzosYSYBdKrlg7xZ7zRAvFtYiMa EknYan4N3FJNOGSQSXbdH+IGyK7KEtYukvsn/tAxpQ8VY17qlG9MdNpZE15N4Gv6kz2u hL+M/FJz0YEDB0h51P54AgDwSau2kqvlBba0gX1OZByexSW/v4ch5/DhBA2DDfvMIR8C Hmxw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of aj@erisian.com.au designates 172.104.61.193 as permitted sender) smtp.mailfrom=aj@erisian.com.au Received: from cerulean.erisian.com.au (azure.erisian.com.au. [172.104.61.193]) by gmr-mx.google.com with ESMTPS id af79cd13be357-7a1d72e8701si64361485a.0.2024.07.31.11.00.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jul 2024 11:00:20 -0700 (PDT) Received-SPF: pass (google.com: domain of aj@erisian.com.au designates 172.104.61.193 as permitted sender) client-ip=172.104.61.193; Received: from aj@azure.erisian.com.au by cerulean.erisian.com.au with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sZDcR-0001Ue-Mg; Thu, 01 Aug 2024 04:00:16 +1000 Received: by email (sSMTP sendmail emulation); Thu, 01 Aug 2024 04:00:07 +1000 Date: Thu, 1 Aug 2024 04:00:07 +1000 From: Anthony Towns To: Luke Dashjr Cc: bitcoindev@googlegroups.com Subject: Re: [bitcoindev] Mining pools, stratumv2 and oblivious shares Message-ID: References: <6f7feb2b-2e24-4081-b555-db69f34d308e@dashjr.org> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Disposition: inline In-Reply-To: <6f7feb2b-2e24-4081-b555-db69f34d308e@dashjr.org> X-Spam_score: -0.0 X-Spam_bar: / X-Original-Sender: aj@erisian.com.au X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of aj@erisian.com.au designates 172.104.61.193 as permitted sender) smtp.mailfrom=aj@erisian.com.au Precedence: list Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com List-ID: X-Google-Group-Id: 786775582512 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Score: -0.8 (/) I think "decentralised" isn't quite a specific enough term here; this scheme requires that the pool has a "trusted coordinator", who can be relied upon to (a) construct sensible templates for people to mine against, (b) reveal the secret they included in the template when a share has enough work to be a valid block, (c) not reveal the secret prior to work being submitted, ie, don't collaborate with people attacking the pool. But pools generally require trusted coordinators anyway: a) to hold custody over funds until miner income reaches the on-chain payment threshold b) to hold the funds prior to paying shares rewards over lightning c) to act as a validating proxy so that each participant in the pool doesn't have to check that every other share submitted to the pool had valid work If we want hashpower to be very widely distributed, those coordinators seem pretty important/essential; it makes for generally low payouts (ie, rarely reaching the onchain threshold), many payouts (ie, if you don't want to spam the chain, better pay them via lightning etc), and a lot of shares to check. I can see it making sense to have a coordinator-free pool if you have a high minimum hashrate requirement to be a member, perhaps 0.1%; at that point some of the members might themselves be pools with a central coordinator, etc. With at most ~1000 members, though, it's not clear to me that a pool like that wouldn't be better off doing identity-based membership and some form of federated governance and just banning attackers, rather than complex crypto protocols and game theory things. I think if you wanted to fix the problem for a pool-of-pools that does have a trusted coordinator, you'd need to do three PoW tests instead of two: work X from the miner demonstrates a valid pool share, additional work Y when you add in the pool's secret demonstrates a valid pool-of-pools share, and additional work Z when you add in the pool-of-pool's secret demonstrates a valid block. But nodes also need to calculate each of X, Y and Z and check X+Y+Z matches the chain difficulty. If you skip out Y, the miners can withhold from the pool; if you skip out Z, the pool can withhold from the pool-of-pools. Not really convinced that would be worthwhile. So I think making things better for pools with coordinators is worthwhile, anyway, even in an ideal world where there's a successful decentralised pool. I think this is the case even if the pool does some light KYC: if you're accepting shares from a mass of "lottery miners" it's probably easy for an attacker to bypass your KYC via identity theft etc, and still hurt you. Conversely, even in the absence of a decentralised pool, I don't think making it easier for pools that do have a coordinator to prevent block withholding is a bad thing: we're already in a situation where most hashpower goes to a few heavily-KYC pools that control the templates being worked on. Making it easier to setup a small non-KYC pools that can compete with these seems a step forward; and even if one of those pools sees huge success and becomes the new dominant pool, despite it being easier to compete with that pool, and uses that weight to dictate what transactions are included in the majority of block templates, that's not even any worse than what we have now... Anyway, I spent some time thinking about the problem for decentralised pools, where you don't have a coordinator and can't/don't keep secrets from miners/attackers. I did get a bit confused because [0] (expanding on "Luke-Jr's two-stage target mechanism" from [1]) seems to describe a very different proposal to the June 2012 description in [2]. [0] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/012069.html [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/012046.html [2] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2012-June/001506.html Unfortunately, I don't think approaches like that described in [2] actually do what we want. My understanding of that proposal is that it would go something like this: * a block header is changed to include: - version, nbits, timestamp, prevhash, merkleroot, nonce (as currently) - next_version, next_timestamp, next_merkleroot, next_nonce (additional) which is the combination of a share of the current block and a share for the following block * share work is based on (version, nbits, timestamp, prevprevhash, merkleroot, nonce) ie, when you're working on a share, you're only committing to the grandparent block, not the parent block * given a block header, you can check the share work for the current block, and the "next" block. both must have sufficient work * **additionally**, the hash of the entire block header must have work X * the total difficulty of a block is then the share work plus X, which must satisfy nbits. * (the next block's transaction tree is not expanded as part of consensus unless it happens to actually be the next block, though the pool members might still verify the details) In that scenario, miners who want to find a block at height N+1 probably take three steps: a) mine a bunch of shares at height N+1 b) mine a bunch of shares at height N+2 c) pair each N+1 share with each N+2 share until they find a pair that has work X, where the share work plus X matches the target difficulty For simplicity, imagine that every block with an odd height is empty -- otherwise it gets complicated to ensure there aren't double spends between some blocks at height N+1 and some blocks at height N+2. Note that once you've found block N+1, you'll have already done most of the work for step (a) at height N+2. The problem is that for miners/pools that don't publish their shares to the world prior to finding a block, larger miners get an advantage: having found K shares in each of steps (a) and (b), they have K^2 chances of having a pair that has work X in step (c), and thus a valid block. So a miner with twice the hashrate has four times the chances of finding a block in any given time period. I'd thought of a similar approach, where instead of taking a share for the current block and a share for the next block, you simply pair two shares for the current block. That seems a bit simpler, in that you don't have to worry about conflicting transaction sets, because once a block is found you simply discard all the existing shares, and you also don't need to worry about balancing the shares you have in step (a) and (b). But it still has the same K^2 problem benefiting large miners. I think in that scenario, the optimal pool size (in one sense, anyway) is about 70% (sqrt(0.5)?) -- you have h^2/(h^2 + (1-h)^2) odds of getting the next block, which amounts to about 85%, while the remaining 30% of hashrate only gets the remaining 15% of blocks. If you both have the same expenses, and they're running just break-even, then your expenses are paid by the first 35% of blocks, and the remaining 50% of blocks that you mine are pure profit. (If you instead accepted everyone into your pool, everyone would be getting the same profits, encouraging more people to buy mining hardware, driving up the difficulty; in this case, anyone who bought hardware would have to join the competing pool, and given they had the same expenses, would not end up making a profit, so you're effectively making excess profits by keeping Bitcoin's total hashrate lower than what it would be with today's system) But the main issue is that if the dominant pool is "evil" (it's centralised, censors txs, and charges high fees eg), a small competing pool starts out with a huge disadvantage -- if the 30% of competing hashrate was split into two pools of 15% instead, they'd get 4% of blocks each, eg. That's a much worse situation than today, and I don't think it's salvageable. For a decentralised pool, you also have the problem that this only fixes half the withholding attack -- in the decentralised case, the attacker can be assumed to know all the shares that have been found so far, so for all the possible blocks that would include a share found by the attacker, if the later of the two shares in that block was found by the attacker, they can simply withhold that share. Depending on how much of the pool's hashrate belongs to the attacker, that's somewhere above 50% of the blocks that would include a share from the attacker. I'd guess you could extend the approach further, so that rather than a block being made up of 2 shares, you have it be made up of n shares, so that effectively the additional O(H**N) benefit of getting additional hashrate outweighs the cost of the last share getting withheld. That's even worse at hurting small pools, but I think at that point you're effectively switching from Nakomoto consensus to more of a DAG-chain design anyway, so perhaps it's fair to design for your "pool" being 100% of miners. In some sense I think this is in conflict with the "progress-free" [3] nature of bitcoin mining. If you want the chance of finding a block to be proportional to hashrate, you want it to be independent of how much previous work has been done, but in that case you can't make one share's chance at being a valid block depend on other shares... [3] eg https://bitcoin.stackexchange.com/a/107401/30971 So pretty disappointed to conclude that I'm not seeing anywhere here where progress could usefully be made. The issue with the "invalid blocks" approach is that if you're not checking template validity but still rewarding payouts for submitted shares, then there is a trivial approach to the block withholding attack: just mine invalid blocks, and don't withhold any of them. At that point it doesn't matter how clever a scheme you have to make it difficult to tell which shares will be a valid block; they just submit all of them, knowing none of them will be a valid block. If you're associating hashrate with identities, you can still ban them when you find they've sent an invalid block, but if that's good enough, you can already ban identities that are fall below some threshold of unlucky-ness. As far as zero-knowledge proofs making template validation easy goes; I think that makes sense, but we're a long way off. First, I think you'd need to have utreexo hashes widely available to get a handle on the utxo set at all; and second, I think the state of the art is zerosync which is still in the "parity with `-assumevalid=TIP`" phase, and beyond that generating proofs is fairly compute intensive, so not something that you'd want to have as a blocking step in announcing a share to your pool. Cheers, aj On Tue, Jul 23, 2024 at 02:44:46PM -0400, Luke Dashjr wrote: > Block withholding is already trivial, and annoying to detect. Decentralised > mining / invalid blocks doesn't increase the risk at all - it only makes it > easier to detect if done that way. > > While it isn't made any worse, fixing it is still a good idea. > Unfortunately, I think your proposal still requires checking every template, > and only fixes block withholding for centralised mining. It would risk > *creating* the very "centralised advantage over decentralised" problem > you're trying to mitigate. > > Given that block withholding is trivial regardless of approach, and there's > no advantage to taking the invalid-block approach, the risk of invalid > blocks would stem from buggy nodes or softfork disagreements/lagging > upgrades. That risk can be largely addressed by spot-checking random > templates. > > Another possibility would be to use zero-knowledge proofs of block validity, > but I don't know if we're there yet. At that point, I think the hardfork to > solve the last remaining avenue would be a good idea. (Bonus points if > someone can think up a way to do it without a centrally-held > secret/server...) -- You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/bitcoindev/Zqp7p/j25tJI4zn9%40erisian.com.au.