From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Tue, 27 Aug 2024 06:56:09 -0700 Received: from mail-oo1-f62.google.com ([209.85.161.62]) by mail.fairlystable.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1siwg4-0000q5-8N for bitcoindev@gnusha.org; Tue, 27 Aug 2024 06:56:09 -0700 Received: by mail-oo1-f62.google.com with SMTP id 006d021491bc7-5de842f4435sf4306829eaf.1 for ; Tue, 27 Aug 2024 06:56:08 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1724766962; cv=pass; d=google.com; s=arc-20160816; b=WBPo6TKH6Hlrh7JimlBsHpxtQ2JFQ/Esp2Tg2foyVl6HCNBQeNiFUNkrW5gceUVqK+ SntBWWIvRsI/6yMi+aLpp14BUIvoP/FOeL+o9QCl3V7BvvjJYh41cCNfN3Qz0DMsJOBD 88UzSu5VRMk61Ftjhq8QY4vcJBnlUltQ41UXtoJxuxY1X2fvdsU/aUlAypNpgiiqnK8h xrb5seyIJxyH2iN26hxUXWpkWd/IFHaJg5q8ErBqBWzMQ18m2gdaDcIne9Kh7gz82qMP Lmj7kCol4X07KfSINJ4skPMHK0OeZWtXMA7nP9FQhhiE/+cUWPtmixBF7r1ntBGIf89s /BlA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:in-reply-to:from:content-language :references:cc:to:subject:mime-version:date:message-id:sender :dkim-signature; bh=13jFk99tzc38P4tMMnsbB1xqV6ey9gxajSeCMWFfH68=; fh=MqSmjcSw743LpeG9kjrsn/dKeCUXekXSxeEYim3GEFQ=; b=NYLq4REb7WAB2ftJ/nVFd0LyW4ILWGAR7nJ6za3ogN9UTX8rd8cRSlYnEHdM0RWINz OFY5O9/OiPXDndR487iVsBVoIAQ20ZKNChO0rufj2c5vPEOrevNC7pR97l1dvAP756cY 4wfU337LmpQUjwyDA43nll61DN3B3edAcfEnenAylPQ6Q5wXGUcKVA1IrNvZEcwv/ss6 Fm5SnuYjwxh7pmRZtdpFrw4G0fS0i0tY1Y51rUtF9jw2xDHLCibirGL9CJ5s19AlRGvK Rzd20sPvO5HK0iyniX7ICGzMw+l6T75fjKsnCfskZM4XwD7KvG6XZqq9YGJKhOJCIw7E 1/MA==; darn=gnusha.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@mattcorallo.com header.s=1724764862 header.b=Xqgm4Odg; dkim=pass header.i=@clients.mail.as397444.net header.s=1724764863 header.b=mXoIfXWw; spf=pass (google.com: domain of lf-lists@mattcorallo.com designates 2620:6e:a000:1::99 as permitted sender) smtp.mailfrom=lf-lists@mattcorallo.com; dmarc=pass (p=NONE sp=REJECT dis=NONE) header.from=mattcorallo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1724766962; x=1725371762; darn=gnusha.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:in-reply-to:from:content-language:references:cc :to:subject:mime-version:date:message-id:sender:from:to:cc:subject :date:message-id:reply-to; bh=13jFk99tzc38P4tMMnsbB1xqV6ey9gxajSeCMWFfH68=; b=DDuv+YPs5hrmsjV+vyToL/iyBaCyEVbvR2oHCV8ybrfcLAfAfH0lb/GAy1sM/wBCev /kMLeLtmrYR1OPvl1w5qM4BNtiiziT/S6+3UWpk4k1W6ReQTEDtfVcI3gTSRL6SYbPj5 E104/thI4/pxEZOu61aXh8dQYexQsAIKxCpG8vJOnsoITqTK+vWovKgdTX7doIn+xrZy nf1NxtQyuZwln1g8pH6amKh7VtD2jQM7fvKjAWqKFR1akKhUZ3+4kDX5+6C5iV/si0Kh eCKuuGIXCkb2mWY+YWwvINYAmIQjgqasuva3Nvc+WOzOP1ohHc8uAEU/mKjmwpDAd5S5 j+Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724766962; x=1725371762; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:in-reply-to:from:content-language:references:cc :to:subject:mime-version:date:message-id:x-beenthere :x-gm-message-state:sender:from:to:cc:subject:date:message-id :reply-to; bh=13jFk99tzc38P4tMMnsbB1xqV6ey9gxajSeCMWFfH68=; b=uSIV/Z8W+zZKs2QaosmdbdpAGgd0bjNfX1ocfwP1i4i3mtb3WghmJBy/Qkp2N2eE5/ KbBgmtj4O6j+JOWi6xFCujBhiYiLDDry76bdz9PLfRcqk2nrshZOVcBYW5SqD+NPuqrJ 0HjXjYjGp+UiTqsdYMRDxxdI7sU5wGLmHvFLtGi3QdB3Lhc5sTOHrYGl6m6oJcltuF1Q YCAjwW5UNLqGsiC2stCYOe1XE8bv6M0n/gEFwtus8ntZbidsnJqjM4VePvxeuZVuAUhx yyPATOU1z1yTdoEAzWchZApk/wujB07OxzlhwmG//d8s9IMTmqzefBT/3Ckao8oCpy7F UIxA== Sender: bitcoindev@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUtDje1q9VdnuKDd/BiSdkE97BbpdstczXGZvDejw3wpb9pwaeFBpsnCstGzeXzifz3v8ItrBNkmi5i@gnusha.org X-Gm-Message-State: AOJu0YwAunEm+0c9y4EdoxaQol9QR4IPzpXzNfiggvNC0NMqrwNoPcED yg26Vpf/fSwLB1LwKBjlDIxl450mqjaeYPbq1L0JJQ6DfhvjGRy2 X-Google-Smtp-Source: AGHT+IHydhDeLfDF+8a/PzyB1eL4uSOrgNClHGC1pWdaW307dBH/b7ziQHJQTBbC2yYzylzdlr348A== X-Received: by 2002:a05:6820:822:b0:5dd:c467:4ea9 with SMTP id 006d021491bc7-5dea67e003bmr2847468eaf.7.1724766961919; Tue, 27 Aug 2024 06:56:01 -0700 (PDT) X-BeenThere: bitcoindev@googlegroups.com Received: by 2002:a05:6820:600b:b0:5de:8561:8893 with SMTP id 006d021491bc7-5de856188b7ls1910964eaf.1.-pod-prod-08-us; Tue, 27 Aug 2024 06:55:59 -0700 (PDT) X-Received: by 2002:a05:6808:11c8:b0:3da:a6b7:4803 with SMTP id 5614622812f47-3def3f21188mr3615361b6e.2.1724766959444; Tue, 27 Aug 2024 06:55:59 -0700 (PDT) Received: by 2002:a05:6808:114d:b0:3db:178d:6ee6 with SMTP id 5614622812f47-3de28d345cbmsb6e; Tue, 27 Aug 2024 06:52:57 -0700 (PDT) X-Received: by 2002:a05:6358:889:b0:1ac:6662:36a1 with SMTP id e5c5f4694b2df-1b5ebf0e35amr348955255d.10.1724766776433; Tue, 27 Aug 2024 06:52:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1724766776; cv=none; d=google.com; s=arc-20160816; b=oi+1QrJEjeGps4GY9Ly5Db0UwQn7Ir4M0ZB4SnaSxVaxSYyJE6ksYgXmL7iDgCcLvl mkAVaF/uL+O5PknkZHe5Z6e/UPzmpI13DXTZriumxgptPtR7dzXHQ3sjLlIhdZyQ60ha hUkXKkr+VIokXVUxsdl+UzAZtslMHrykafFKau8xqa1Fr+lI6XZQQcui/YW50IdlDZCv 8UvgT+2Vwh2Smk2DcnLJEpFXhdzmv2FLtQVVNR2jruHF5L2RrnzHw1v4dWzXaHuDMbxF +z0INpdI0kCGpjHkJC3GQCbW9n5DRMLQv4UVV/Ze2jE6HRyBI5FxXUWe8anEWmFw6R5g y0jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:mime-version:date:message-id :dkim-signature:dkim-signature; bh=J20wP7FJmf57qnqsI6NkdcHUDc8ru7hN8EQ8cVIiN1M=; fh=ZhIePvtzD2qBgm9PadABf9spV2moFvlPRwwE8o08T1o=; b=znMmdEO1r8PjAcaEUBuHPDF+D+nsC1e94++ERMa1wqyiYY87Ce1lPhew1gZDn363YF 6Eqg6BaJdk0nzMiubxUx6PQ1bTjMWFwvssHyRKatf/TWGdK2paKkV6oD5Fbgo5vqcxb4 X/mXAePcck1Dj/K/N9eu39nx5uSgG7/jEWhzfGINVLqLkAx2xuX4n98dh+d4BzOLBY3G oauzDRj+gicfM0XkL/hkx2jIpLzIdAevio4mtG/01vDlQQ61QbueiSq8b4aMVqk0s8mS 6BZ7quUtcV8Vay/8AUhkakmDt+tWjdNgJsYt1RKAlL0hEc8CHUq02jLvzf6FfR0aSq/t mtIQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@mattcorallo.com header.s=1724764862 header.b=Xqgm4Odg; dkim=pass header.i=@clients.mail.as397444.net header.s=1724764863 header.b=mXoIfXWw; spf=pass (google.com: domain of lf-lists@mattcorallo.com designates 2620:6e:a000:1::99 as permitted sender) smtp.mailfrom=lf-lists@mattcorallo.com; dmarc=pass (p=NONE sp=REJECT dis=NONE) header.from=mattcorallo.com Received: from mail.as397444.net (mail.as397444.net. [2620:6e:a000:1::99]) by gmr-mx.google.com with ESMTPS id af79cd13be357-7a67f30ccf1si53135285a.1.2024.08.27.06.52.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Aug 2024 06:52:56 -0700 (PDT) Received-SPF: pass (google.com: domain of lf-lists@mattcorallo.com designates 2620:6e:a000:1::99 as permitted sender) client-ip=2620:6e:a000:1::99; X-DKIM-Note: Keys used to sign are likely public at X-DKIM-Note: https://as397444.net/dkim/mattcorallo.com and X-DKIM-Note: https://as397444.net/dkim/clients.mail.as397444.net X-DKIM-Note: For more info, see https://as397444.net/dkim/ Received: by mail.as397444.net with esmtpsa (TLS1.3) (Exim) (envelope-from ) id 1siwcw-005tTG-2R; Tue, 27 Aug 2024 13:52:54 +0000 Message-ID: <13cec02f-9012-4ba2-bb45-e00907a55357@mattcorallo.com> Date: Tue, 27 Aug 2024 09:52:53 -0400 MIME-Version: 1.0 Subject: Re: [bitcoindev] Mining pools, stratumv2 and oblivious shares To: Anthony Towns Cc: bitcoindev@googlegroups.com References: <26322ee8-08e6-4718-8d1c-60bca8c13c6a@mattcorallo.com> Content-Language: en-US From: Matt Corallo In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed X-Original-Sender: lf-lists@mattcorallo.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@mattcorallo.com header.s=1724764862 header.b=Xqgm4Odg; dkim=pass header.i=@clients.mail.as397444.net header.s=1724764863 header.b=mXoIfXWw; spf=pass (google.com: domain of lf-lists@mattcorallo.com designates 2620:6e:a000:1::99 as permitted sender) smtp.mailfrom=lf-lists@mattcorallo.com; dmarc=pass (p=NONE sp=REJECT dis=NONE) header.from=mattcorallo.com Precedence: list Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com List-ID: X-Google-Group-Id: 786775582512 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Score: -0.7 (/) On 8/27/24 5:07 AM, Anthony Towns wrote: > On Wed, Aug 21, 2024 at 10:28:35AM -0400, Matt Corallo wrote: > Right, but that just sets some threshold where low hashrate members of a > pool are indistinguishable from people attacking the pool. If there's no > attack going on, that's fine, of course. To put some numbers to this: Ocean > reportedly paid out ~10% more in reward for the same work compared to other > pools [0]. > > [0] https://x.com/ocean_mining/status/1825943407736533008 > > To reverse that, you could do something like: > > * Take Ocean's current (honest) hashrate of 2160 PH/s, ie about 0.33% of global > hashrate, or 3.35 blocks/week > * Find ~22% of that to attack with, ie 475 PH/s (0.74 blocks/week) > * Run the attack: > - Ocean's new hashrate is 2635 PH/s > - Ocean still only achieves 3.35 blocks/week > - You collect ~18% of their reward (0.6 blocks/week) > - Honest ocean miners collect the remainder 2.75 blocks/week) > - If 3.35 blocks/week was making 1.1x the FPPS reward, 2.75 blocks/week > is now only 0.9x the FPPS reward > - Publish a google doc saying Ocean sucks, FPPS is much better > > If submitted through a single account, 475PH/s would be the third > largest member of Ocean; and relatively easy to do statistical analysis > on. If split up across perhaps 500 accounts, with less than a PH/s each, > those accounts would not be in Ocean's top 80, and each individual account > wouldn't be expected to find a block more often than once a decade or > so. Submitting shares via tor, spending some time creating the accounts > and making them look normal (ie, actually submitting the 0.6 blocks/week > they're collectively expected to find), and receiving payouts over > lightning seems like it would close out many of the obvious ways of > telling that the accounts are all sybils. > > Maybe it's okay if the answer is just "analyse as best you can to find > the real culprit, and if you can't, just drop all the low hashrate pool > members". With an approach like that, you could decide something like > "if there's an attack that lasts for two months, I'll boot out everyone > who didn't find a block over that period" [1]. For someone with 0.02% > of global hashrate (130 PH/s), there's about an 80% chance they'll find > a block over two months, whereas for someone with 0.0012% (7.8 PH/s), > there's a 90% chance they won't. So at that point, even if you're honest > and you've invested $4M in ASICs (482x S21XP, 130 PH/s), you've still got > a 20% chance of being booted out of the pool; and if your investment is > only $240,000 (29x S21 XP, 7.8 PH/s), you've got a 90% chance of being > booted out. For comparison, only the top three users on Ocean's dashboard > report more than 130 PH/s, and 7.8 PH/s would put you in the top 25. > > (For comparison, each miner having to have at least 0.01% of global > hashpower to be viable means there's at most 10k miners worldwide, > which is about the same as the number of members in the SWIFT > system. Alternatively, the low figure above was 29x S21's; the new ESMA > recommendation [2] seems to consider you a threat to the environment / > large scale miner with just 17x S21's...) > > Or you could look at it the other way round: Ocean accepting anonymous > small miners seems like a nice thing now, but if it's also setting the > pool up for failure by providing a way for an attacker to hide its attack, > it might not actually be something that's good for the network. Yep, I generally agree with all of this. It kinda is what it is, we can't do much to change it, and pools do run the risk of getting attacked. For a PPLNS pool like Ocean, it'd mean the miners would lose out. For a PPS pool, like nearly all others, it'd mean the pool goes out of business. Pick your poison, I guess. The one other thing to point out that you can do is pay out a bonus for the miner that finds a block. This is (IIRC) what p2pool did, and what any decentralized pool worth its salt will do. That at least adds some (hopefully non-trivial) cost to block withholding to disincentivize it, though of course it comes at the cost of stable rewards...which was kinda the whole point of a pool. > >>> What I'm interested in is >>> a pool that doesn't do those things: for example, a world where 5% of >>> hashrate is from 60M BitAxe devices owned by 10M people, say. > > I'm interested in the scenario where there's a pool that supports large > numbers of low hashrate miners, even in the face of an attack, and I > just don't see a way in which statistical analysis can be good enough > in that scenario. > > (I'm also not sure there's much difference between the thresholds for > "can be confirmed to not be an attacker via statistical means" and "can > viably solo mine". Pools being only for the miners that don't really > need them doesn't seem great) Another way to look at this is pooled mining isn't useful for miners that will find a block approximately never :). > With a PoW algorithm supporting oblivious shares, that's not the case: > block withholding simply stops being an attack: if you withhold n shares, > your payout drops by the value of n shares, and the pool's revenue drops > by the expected value of n shares, the same as if you'd just turned > your miner off briefly. Sure, there are various schemes to prevent block withholding. Last I looked into it, though, they all require a major PoW change, which I'm not sure is worth doing just to fix block withholding, sadly. >>>> Adding more explicit "negotiation" to Stratum V2 work selection would defeat >>>> the purpose - if the pool is able to tell a miner not to work on some work >>>> it wants to, ... >>> A pool is always able to do that -- they can simply mark the share as >>> invalid after the fact and refuse to pay out on it, and perhaps make >>> a blog post explaining their policy. The over-the-wire protocol isn't >>> what provides that ability. >> A pool can decline to pay out, yes, but the miner will still work on that >> block. The point of custom work selection is that the miner will *always* >> work on the block they want, no matter what. And if they mind it, they >> broadcast it directly themselves. Anything else would defeat the point. > > If you mine a block, paying to a pool, but the pool is not paying you for > shares, all you're doing is making a large donation to the pool operator, > that at best might get passed on to other members of the pool, but won't > be passed on to you. That's even less profitable than solo mining. Sure, but this has nothing to do with custom work selection or StratumV2. You can always mine a block for a pool and the pool can always decide to not pay out for that work. You should probably use pools that aren't run by scammers. > You can certainly say "the miners will never tell the pool the contents > of their shares, and will never join a pool that expects that; pools > only find out about txs when a successful block is mined and broadcast", > and do a statistical analysis of shares vs blocks, but that again only > works for miners with large hashrates. > > Without miners sharing the block templates for their shares with the > pool, I don't see how a pool could preference miners who are picking > "good" templates, vs mining empty blocks; if you're only getting the > same reward as someone who's mining an empty block, it seems locally > optimal to mine an empty block yourself, though that certainly doesn't > seem globally optimal. If you're not validating the templates, but > rewarding shares with templates that claim to give high fees, that also > seems exploitable in ways that are harmful for the pool. I never said anything about refusing to tell a pool the contents of shares. Quite the opposite, in fact, in order to ensure StratumV2 work custom selection doesn't result in a degradation in pool block propagation performance, pools should be getting the contents of shares from miners and should prepare to forward that. They should also probably spot-check for validity, but more to detect misconfiguration than active attacks. >>>> The only >>>> way any kind of centralized pooling with custom work selection adds any >>>> value to Bitcoin's decentralization is if the clients insist on mining the >>>> work they want to - whether on the pool or solo mining if the pool doesn't >>>> want it. >>> >>> If you're really expecting miners are going to be constantly telling >>> their pool "do exactly what I want or I solo mine", I think you're pretty >>> likely to be disappointed by whatever the future holds... By its nature, >>> solo mining is something that can only be done profitably by relatively >>> few players at any given time; it's only potentially decentralised if the >>> total market for Bitcoin ownership/usage is itself very small. >> >> You're totally missing the point that pools can just...pay out properly? > > Giving "why can't we all just get along" vibes there... They certainly > could, but incentives for them to do otherwise don't go away just by > wishing they would. I don't buy that its in a pool's best interest to not pay out for the contract they have with their users? How is trashing your business and getting sued in their best interest? >> Or >> if they don't people will create new pools that do? Not sure why you think >> that's a far-fetched outcome. > > Sure, making it easy to create new pools is ideal. I think "just do > statistical analysis to detect/prevent attacks" is already a pretty big > impediment to that, though: it's hard to do, not automated, and likely > requires some degree of secrecy to do well. Same as "oh, we just use > heuristics to detect credit card fraud" vs "sign with your private key, > and no one can reuse your signature or create a fake one": one requires > a bunch of specialist knowledge that's hard to duplicate and is often > ineffective, the other is something that can be done reliably by freely > downloadable open source code. Sure, but this again has nothing to do with StratumV2 or custom work selection... > My chain of logic is: > > * I'd like to see mass-market mining be more viable (ie, lots of people each > with small amounts of hashrate, vs a smaller number of large operations > with large amounts of hashrate) Yea, I would too, but before we rush to go do a fork to change the PoW to add oblivious PoW, I'd really like to be convinced that its actually realistic that these kinds of miners can have more than a few % of total network hashrate. If we can only get them up to a few %....who cares? > * Small hashrate mining is only viable via altruism ("I'm supporting > the network, and it's cheap"), irrationality ("I know it's bad odds > for everyone else, but I'm lucky"), or pooled mining. > > * Altruism and irrationality don't work at scale, so I'd like to > see pooled mining work well for small amounts of hashrate. > > * Pools that accept small miners will have a very hard problem > preventing/addressing block withholding attacks, because statistical > methods are not viable for people making a mining investment that's > not in the six figure range. > > * We could make a very intrusive change to fix block kwithholding > attacks entirely, by supporting oblivious shares, so that the miner > can't tell which share will be a valid block. > > * Even if we did that, we'd still have to prevent rewarding miners > for producing shares based on invalid templates, Yea, one step at a time, mostly. But if the market did change somehow and small miners were shooting for more than a few % of hashrate (I dunno, ASIC heaters become a thing?) and then we decide we should fork in oblivious mining.... then I think statistical checking is much, much, much easier here. StratumV2 pools are already expected to spot-check work and generally expected to always see block templates, so it becomes a problem of "throw more CPU at it to verify more clients and reject more shares" (which if you have a ton of shares could probably be done kinda efficiently through script validity and UTXO caching). You could very reasonably rate-limit new template generation without breaking things, and require some minimum threshold of miner hashrate (yay PoW for anti-DoS) and probably you'd be fine. > if templates aren't > being provided by the pool. Sure, and if the templates are being provided by the pool then there's no value in having lots of small miners :). Doesn't change your argument, just worth pointing out, I think. Matt -- You received this message because you are subscribed to the Google Groups "Bitcoin Development Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an email to bitcoindev+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/bitcoindev/13cec02f-9012-4ba2-bb45-e00907a55357%40mattcorallo.com.