From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 34A68C0032 for ; Tue, 7 Nov 2023 16:12:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 854044185D for ; Tue, 7 Nov 2023 16:12:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 854044185D Authentication-Results: smtp4.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=achow101.com header.i=@achow101.com header.a=rsa-sha256 header.s=protonmail2 header.b=G9Gm0pYV X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -2.1 X-Spam-Level: X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HgeVlJFW8wET for ; Tue, 7 Nov 2023 16:12:44 +0000 (UTC) Received: from mail-40136.proton.ch (mail-40136.proton.ch [185.70.40.136]) by smtp4.osuosl.org (Postfix) with ESMTPS id B9195415FA for ; Tue, 7 Nov 2023 16:12:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org B9195415FA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=achow101.com; s=protonmail2; t=1699373558; x=1699632758; bh=RoEplILoQzxAS7Nc/aAKFIhnvrBir8Lq+0CghEhh3M0=; h=Date:To:From:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=G9Gm0pYVmtTWCYKKIk/a54ju8k1dfs7WF7J7FVU7m13trITSK6aY0gyuiD+jTUcqc Ov/ZOaMvQMspZMbIm+v7QDREDgj0Jg7Jqj3OmCoTW2aVq5MKuU5VmdVFiB+b02lQn7 NzkfnPXj+gmqgkOmlw5B1CJjyXJGz0sQAiS3S+KaF3JZ8+sVOyNte4fmZGVwyOMfsY t0o+QuAzzA76JSd20c5juvvkfVQEhfB4kUFN41you6Mw+SuZw73zOeEBwLr/Fa7qLZ C5KNxvkCJHa13VxoGspMhFuU2PD9rL3lzt0QfjryoyIRRMEnyNE80MDDpI5dbnZZpN nL9tVfGOUMbuA== Date: Tue, 07 Nov 2023 16:12:25 +0000 To: bitcoin-dev@lists.linuxfoundation.org From: Andrew Chow Message-ID: In-Reply-To: References: Feedback-ID: 53660394:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Tue, 07 Nov 2023 16:16:25 +0000 Subject: Re: [bitcoin-dev] Future of the bitcoin-dev mailing list X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Nov 2023 16:12:49 -0000 Thanks for writing this up. I would prefer that we continue to have a mailing list where email is a=20 functional and first class user interface. So that would be to migrate=20 to groups.io or Google Groups. I think Google Groups is probably the=20 better choice of the two. Although there are concerns that Google would shut down Google Groups or=20 specifically target a bitcoin-dev group, I think both are unlikely to=20 happen. Both Chromium and Android use Google Groups for their mailing=20 lists, so unless those go somewhere else, I doubt Google would=20 unceremoniously kill Google Groups. As for shutting down a bitcoin-dev=20 group specifically, given that there appears to be several thousand=20 groups whose sole purpose is to distribute spam, I don't think Google is=20 in the habit of shutting down groups. Regardless of what we do, there's always the risk that someone will shut=20 down or stop maintaining the servers for any discussion medium. Even=20 self hosting requires someone to keep the servers up and do maintenance,=20 and that person (or people) could get bored of it, run out of money, get=20 hit by a bus, etc. We're experiencing that right now with the Linux=20 Foundation, and I don't think fear of that should prevent us from moving=20 to a different third party host. Andrew Chow On 11/07/2023 10:37 AM, Bryan Bishop via bitcoin-dev wrote: > Hello, >=20 > We would like to request community=C2=A0feedback and proposals on the fut= ure=20 > of the mailing list. >=20 > Our current mailing list host, Linux Foundation, has indicated for years= =20 > that they have wanted to stop hosting mailing lists, which would mean=20 > the bitcoin-dev mailing list would need to move somewhere else. We=20 > temporarily avoided that, but recently LF has informed a moderator that= =20 > they will cease hosting any mailing lists later this year. >=20 > In this email, we will go over some of the history, options, and invite= =20 > discussion ahead of the cutoff. We have some ideas but want to solicit=20 > feedback and proposals. >=20 > Background > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > The bitcoin-dev mailing list was originally hosted on Sourceforge.net.=20 > The bitcoin development mailing list has been a source of proposals,=20 > analysis, and developer discussion for many years in the bitcoin=20 > community, with many thousands of participants. Later, the mailing list= =20 > was migrated to the Linux Foundation, and after that OSUOSL began to help= . >=20 > Linux Foundation first asked us to move the mailing list in 2017. They=20 > internally attempted to migrate all LF mailing lists from mailman2 to=20 > mailman3, but ultimately gave up. There were reports of scalability=20 > issues with mailman3 for large email communities. Ours definitely=20 > qualifies as.. large. >=20 > 2019 migration plan: LF was to turn off mailman and all lists would=20 > migrate to the paid service provider groups.io . Back= =20 > then we were given accounts to try the groups.io =20 > interface and administration features. Apparently we were not the only=20 > dev community who resisted change. To our surprise LF gave us several=20 > years of reprieve by instead handing the subdomain and server-side data= =20 > to the non-profit OSUOSL lab who instead operated mailman2 for the past= =20 > ~4 years. >=20 > OSUOSL has for decades been well known for providing server=20 > infrastructure for Linux and Open Source development so they were a good= =20 > fit. This however became an added maintenance burden for the small=20 > non-profit with limited resources. Several members of the Bitcoin dev=20 > community contributed funding to the lab in support of their Open Source= =20 > development infrastructure goals. But throwing money at the problem=20 > isn=E2=80=99t going to fix the ongoing maintenance burden created by anti= quated=20 > limitations of mailman2. >=20 > Permalinks > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > Linux Foundation has either offered or agreed to maintain archive=20 > permalinks so that content of historic importance is not lost.=20 > Fortunately for us while lists.linuxfoundation.org=20 > mailman will go down, they have=20 > agreed the read-only pipermail archives will remain online. So all old=20 > URLs will continue to remain valid. However, the moderators strongly=20 > advise that the community supplements with public-inbox instances to=20 > have canonical archive urls that are separate from any particular email= =20 > software host. >=20 > Public-Inbox > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > https://public-inbox.org/README.html >=20 > =E2=80=9CPublic Inbox=E2=80=9D decentralized archiving - no matter what m= ailing list=20 > server solution is used, anyone can use git to maintain their own=20 > mailing list archive and make it available to read on the web. >=20 > Public Inbox is a tool that you can run yourself. You can transform your= =20 > mbox file and it makes it browsable and viewable online. It commits=20 > every post to a git repository. It's kind of like a decentralized mail=20 > archiving tool. Anyone can publish the mail archive to any web server=20 > they wish. >=20 > We should try to have one or more canonical archives that are served=20 > using public-inbox. But it doesn't matter if these are lost because=20 > anyone else can archive the mailing list in the same way and re-publish= =20 > the archives. >=20 > These git commits can also be stamped using opentimestamps, inserting=20 > their hashes into the bitcoin blockchain. >=20 > LKML mailing list readers often use public-inbox's web interface, and=20 > they use the reply-to headers to populate their mail client and reply to= =20 > threads of interest. This allows their reply to be properly threaded=20 > even if they were not a previous subscriber to that mailing list to=20 > receive the headers. >=20 > public-inbox makes it so that it doesn't really matter where the list is= =20 > hosted, as pertaining to reading the mailing list. There is still a=20 > disruption if the mailing list goes away, but the archives live on and=20 > then people can post elsewhere. The archive gets disconnected from the=20 > mailing list host in terms of posting. We could have a few canonical=20 > URLs for the hosts, separate from the mailing list server. >=20 > mailman problems > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > Over the years we have identified a number of problems with mailman2=20 > especially as it pertains to content moderation. There are presently a=20 > handful of different moderators, but mailman2 only has a single password= =20 > for logging into the email management interface. There are no moderator= =20 > audit logs to see which user (there is no concept of different users)=20 > acted on an email. There is no way to mark an email as being=20 > investigated by one or more of the moderators. Sometimes, while=20 > investigating the veracity of an email, another moderator would come in= =20 > and approve a suspect email by accident. >=20 > Anti spam has been an issue for the moderators. It's relentless. Without= =20 > access to the underlying server, it has been difficult to fight spam.=20 > There is some support for filters in mailman2 but it's not great. >=20 > 100% active moderation and approval of every email is unsustainable for= =20 > volunteer moderators. A system that requires moderation is a heavy=20 > burden on the moderators and it slows down overall communication and=20 > productivity. There's lots of problems with this. Also, moderators can=20 > be blamed when they are merely slow while they are not actually censoring= . >=20 > Rejection emails can optionally be sent to=20 > bitcoin-dev-moderation@lists.ozlabs.org=20 > but this is an option=20 > that a moderator has to remember to type in each time. >=20 > Not to mention numerous bugs and vulnerabilities that have accumulated=20 > over the years for relatively unmaintained software. (Not disclosed here) >=20 > Requirements and considerations > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D >=20 > Looking towards the future, there are a number of properties that seem=20 > to be important for the bitcoin-dev mailing list community. First, it is= =20 > important that backups of the entire archive should be easy for the=20 > public to copy or verify so that the system can be brought up elsewhere= =20 > if necessary. >=20 > Second, there seems to be demand for both an email threading interface=20 > (using mailing list software) as well as web-accessible interfaces (such= =20 > as forum software). There seems to be very few options that cater to=20 > both email and web. Often, in forum software, email support is limited=20 > to email notifications and there is limited if any support for email=20 > user participation. >=20 > Third, there should be better support for moderator tools and management= =20 > of the mailing list. See above for complaints about problems with the=20 > mailman2 system. >=20 > Burdens of running your own mailing list and email server > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D >=20 > If you have never operated your own MTA you have no idea how difficult=20 > it is to keep secure and functional in the face of numerous challenges=20 > to deliverability. Anti-spam filtering is essential to prevent=20 > forwarding spam. The moment you forward even a single spam message you=20 > run the risk of the server IP address being added to blacklists. >=20 > The problem of spam filtering is so bad that most IP addresses are=20 > presumed guilty even if they have no prior spam history, such as if=20 > their network or subnetwork had spam issues in the past. >=20 > Even if you put unlimited time into managing your own email server,=20 > other people may not accept your email. Or you make one mistake, and=20 > then you get into permanent blacklists and it's hard to remove. The spam= =20 > problem is so bad that most IPs are automatically on a=20 > guilty-until-proven-innocent blacklist. >=20 > Often there is nothing you can do to get server IP addresses removed=20 > from spam blacklists or from "bad reputation" lists. >=20 > Ironically, hashcash-style proof-of-work stamps to prevent spam are an=20 > appealing solution but not widely used in this community. Or anywhere. >=20 > Infinite rejection or forwarding loops happen. They often need to be=20 > detected through vigilance and require manual sysadmin intervention to=20 > solve. >=20 > Bitcoin's dev lists being hosted alongside other Open Source projects=20 > was previously protective. If that mailing list server became=20 > blacklisted there were a lot of other people who would notice and=20 > complain. If we run a Bitcoin-specific mail server we are on our own.=20 > 100% of the administrative burden falls upon our own people. There is=20 > also nothing we can do if some unknown admin decides they don't like us. >=20 > Options > =3D=3D=3D=3D=3D=3D=3D >=20 > Web forums are an interesting option, but often don't have good email=20 > user integration. At most you can usually hope for email notifications=20 > and an ability to reply by email. It changes the model of the community= =20 > from push (email) to pull (logging into a forum to read). RSS feeds can= =20 > help a little bit. >=20 > Many other projects have moved from mailing lists to forums (eg=20 > https://discuss.python.org/ =E2=80=93 see= =20 > https://lwn.net/Articles/901744/ ; or= =20 > https://ethresear.ch/ ), which seem easier to=20 > maintain and moderate, and can have lots of advanced features beyond=20 > plaintext, maybe-threading and maybe-HTML-markup. >=20 > Who would host the forum? Would there be agreement around which forum=20 > software to use or which forum host? What about bitcointalk.org=20 > or delvingbitcoin.org=20 > ? There are many options available. Maybe=20 > what we actually want isn=E2=80=99t so much a discussion forum, as an 'ar= xiv of=20 > our own' where anons can post BIP drafts and the like? >=20 > Given the problems with mailman2, and the decline of email communities=20 > in general, it seems that moving to mailman3 would not be a viable=20 > long-term option. This leaves us with Google Groups or groups.io=20 > as two remaining options. >=20 > groups.io is an interesting option: they are a paid=20 > service that implements email communities along with online web forum=20 > support. However, their public changelog indicates it has been a few=20 > years since their last public change. They might be a smaller company=20 > and it is unclear how long they will be around or if this would be the=20 > right fit for hosting sometimes contentious bitcoin development=20 > discussions... >=20 > Google Groups is another interesting option, and comes with different=20 > tradeoffs. It's the lowest effort to maintain option, and has both an=20 > email interface and web forum interface. Users can choose which mode=20 > they want to interact with. >=20 > For the Google Groups web interface, you can use it with a non-gmail=20 > account, but you must create a Google Account which is free to do. it=20 > does not require any personal information to do so. This also allows you= =20 > to add 2FA. Non-gmail non-google users are able to subscribe and post=20 > email from their non-gmail non-google email accounts. Tor seems to work= =20 > for the web interface. >=20 > Will Google shut it down, will they cut us off, will they shut down=20 > non-google users? The same problem exists with other third-party hosts. >=20 > The moderation capabilities for Google Groups and groups.io=20 > seem to be comparable. It seems more likely that=20 > Google Groups will be able to handle email delivery issues far better=20 > than a small resource-constrained operation like groups.io=20 > . ((During feedback for this draft, luke-jr indicates= =20 > that Google Workspaces has been known to use blacklisted IPs for=20 > business email!)) >=20 > On the other hand, groups.io is a paid service and=20 > you get what you pay for... hopefully? >=20 > Finally, another option is to do literally nothing. It's less work=20 > overall. Users can switch to forums or other websites, or private=20 > one-on-one communication. It would remove a point of semi-centralization= =20 > from the bitcoin ecosystem. It would hasten ossification, but on the=20 > other hand it would hasten ossification and this could be a negative=20 > too. Moderators would be less of a target. >=20 > Unfortunately, by doing nothing, there would be no more widely used=20 > group email communication system between bitcoin developers. Developers= =20 > become less coordinated, mayhem and chaos as people go to different=20 > communication platforms, a divided community is more vulnerable, etc.=20 > BIP1 and BIP2 would need to be revised for other venues. >=20 > The main categories of what to move to are: web forums, mailing lists,=20 > and hybrids of those two options. Most everything is either self-hosted= =20 > or you pay someone else to host it. It's kind of the same problem=20 > though. It largely depends on how good is the software and unfortunately= =20 > running your own MTA that forwards mail is not a good option. >=20 > Going forward > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > We'd like to invite feedback and proposals from the community, and see=20 > what options are available. One potential option is a migration to=20 > Google Groups, but we're open to ideas at this point. We apologize for=20 > any inconvenience this disruption has caused. >=20 >=20 > Bitcoin-dev mailing list moderation team >=20 > Bryan Bishop > Ruben Somsen > Warren Togami > various others. >=20 > --=20 > - Bryan > https://twitter.com/kanzure