From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id B50FFC0032 for ; Tue, 7 Nov 2023 18:14:37 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9039F408D8 for ; Tue, 7 Nov 2023 18:14:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9039F408D8 Authentication-Results: smtp4.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=achow101.com header.i=@achow101.com header.a=rsa-sha256 header.s=protonmail2 header.b=Qc72Nq8B X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -2.1 X-Spam-Level: X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hXQsD-FYAoAB for ; Tue, 7 Nov 2023 18:14:35 +0000 (UTC) Received: from mail-40136.proton.ch (mail-40136.proton.ch [185.70.40.136]) by smtp4.osuosl.org (Postfix) with ESMTPS id 109AC408D6 for ; Tue, 7 Nov 2023 18:14:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 109AC408D6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=achow101.com; s=protonmail2; t=1699380868; x=1699640068; bh=VI0UeUCGvLl/EviBM9CWtZSgZBmsgI0NpmLmBNaVRY4=; h=Date:To:From:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=Qc72Nq8BphFVVsx7OXaV8bhkfJ1zwPLz/1ZWXvhTpABoOdIl07oOaGfyfYAB7r/bw DKa3HvV8U0turrGf5J9NUEBoNwniF5nYVjGcVWzvhWWdqvRK6O2vKcxsS6ZHlJdtDg newbkVvJW+DBtGpEZXuoaEMFydK1Ta6ZDODuFvvjY7KrDXCYHtFjwHW9qKKyl+Vwm0 7MBcBV5jAX2G4E/Fz8Ry4quO2G6syXwFhFdhtH22Inw1M+0iuiaf9suo8zUnJyOS6X E3KWAZ2EGb+Al8CePGUyZQ1UzPVhIJw+mM3MZqedqIhyLXjfmx/JypdRKwF13Mk/un +Ayeu7ckh1Bgg== Date: Tue, 07 Nov 2023 18:14:23 +0000 To: bitcoin-dev@lists.linuxfoundation.org From: Andrew Chow Message-ID: <2099470b-cca4-4fbe-99c3-ee2d2ed20157@achow101.com> In-Reply-To: References: Feedback-ID: 53660394:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Tue, 07 Nov 2023 19:05:43 +0000 Subject: Re: [bitcoin-dev] Future of the bitcoin-dev mailing list X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Nov 2023 18:14:37 -0000 Hi Dan, I don't think nostr would be a suitable replacement for the mailing=20 list, although this opinion is biased by the fact that I do not use=20 nostr and find it to be uninteresting. From my limited understanding of how nostr works, it's not clear to me=20 how a distributed system that uses message broadcast would work in the=20 same way as a mailing list. How would people "subscribe"? How would=20 archives be searched or otherwise be available to people who are not on=20 nostr? How do you distinguish and filter between legitimate dev posts=20 intended for discussion and random crap and shitposting as shows up on=20 social media? I also don't think that long form text on nostr (or any similar=20 platform) can sufficiently replace email. None of these things seem to=20 contain a way to have a separate subject line as email does. Subjects=20 are immensely important for me as it provides a quick and easy way to=20 filter out things I don't care about reading. I don't want to have read=20 something in before I can decide that I don't care about reading it. In general, I strongly prefer email, or a platform that has email as a=20 first class user interface, over platforms such as nostr, matrix, or web=20 forums. Email is universal - everyone has one and everyone knows how it=20 works. It dramatically lowers the barrier of entry. Having to make an=20 account somewhere or download some specific client in order to=20 participate will simply result in only the most dedicated participating.=20 Development in open source must be an open process and the barriers to=20 entry should be low. Lastly, IIRC the plan is to shut down the list by the end of this year.=20 Any solution that requires custom software and bridges to be created are=20 not going to be viable in this time frame. Andrew On 11/07/2023 12:03 PM, Ademan via bitcoin-dev wrote: > Hi Bryan, >=20 > I don't really want my first (and last?) devlist message to be a fairly= =20 > off-the-cuff post on this topic, but here we go anyway. >=20 > At the risk of sounding like a nostr evangelist (I promise I'm not), I=20 > want to suggest nostr as a potential replacement to the mailing list. A= =20 > decent chunk of software would need to be written, but none of the=20 > alternatives seem particularly attractive to me. I particularly dislike= =20 > the idea of locking into a single siloed forum service like the=20 > bitcointalk forums. I realize I may be in the minority of course. >=20 >=20 > Nostr enables the ML team to outsource all of its biggest burdens, if it= =20 > chooses: >=20 > - mail server blocking is N/A to nostr >=20 > - Hosting costs are completely outsourced unless the ML team chooses to= =20 > host a relay. >=20 > - Archives and web portal access can be similarly outsourced because any= =20 > nostr archive is self-authenticating. >=20 > - The ML team can also choose to completely outsource moderation, as=20 > nostr is (more or less) permissionless by nature. > =C2=A0 I understand if there is a "blessed" communication system, the ML= =20 > team would probably prefer to keep it high quality. To that end there=20 > are proposals for proof-of-work, and web of trust based blocklists for=20 > nostr which are optional for end users. There are other options such as= =20 > the "moderated communities" proposal which would provide tighter control. >=20 >=20 > On the user side, the optional moderation is very attractive, allowing=20 > controversial threads to exist and continue, without requiring everyone= =20 > to see them. >=20 >=20 > The following do not currently exist (to my knowledge) and would need to= =20 > be implemented to meet the ML's requirements: >=20 > - an email gateway to satisfy the bulk of existing ML subscribers > =C2=A0 This reintroduces issues with mail server blocking of course. > - a long-form oriented nostr client (current plain text clients could be= =20 > used in the meantime) >=20 > That admittedly is quite a lot of work, but the second item can be=20 > deferred, and the first is not particularly technically challenging, the= =20 > complications are all on the administration side. >=20 > I expect some reflexive NACKs based on the immaturity of the ecosystem=20 > but if we have months to prepare, I believe the core requirements can be= =20 > solidly satisfied in time, the rest can be developed over time, and I=20 > believe the advantages are worth careful consideration. >=20 > Cheers, > Dan >=20 > On Tue, Nov 7, 2023 at 9:38=E2=80=AFAM Bryan Bishop via bitcoin-dev=20 > > wrote: >=20 > Hello, >=20 > We would like to request community=C2=A0feedback and proposals on the > future of the mailing list. >=20 > Our current mailing list host, Linux Foundation, has indicated for > years that they have wanted to stop hosting mailing lists, which > would mean the bitcoin-dev mailing list would need to move somewhere > else. We temporarily avoided that, but recently LF has informed a > moderator that they will cease hosting any mailing lists later this > year. >=20 > In this email, we will go over some of the history, options, and > invite discussion ahead of the cutoff. We have some ideas but want > to solicit feedback and proposals. >=20 > Background > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > The bitcoin-dev mailing list was originally hosted on > Sourceforge.net. The bitcoin development mailing list has been a > source of proposals, analysis, and developer discussion for many > years in the bitcoin community, with many thousands of participants. > Later, the mailing list was migrated to the Linux Foundation, and > after that OSUOSL began to help. >=20 > Linux Foundation first asked us to move the mailing list in 2017. > They internally attempted to migrate all LF mailing lists from > mailman2 to mailman3, but ultimately gave up. There were reports of > scalability issues with mailman3 for large email communities. Ours > definitely qualifies as.. large. >=20 > 2019 migration plan: LF was to turn off mailman and all lists would > migrate to the paid service provider groups.io . > Back then we were given accounts to try the groups.io > interface and administration features. Apparently > we were not the only dev community who resisted change. To our > surprise LF gave us several years of reprieve by instead handing the > subdomain and server-side data to the non-profit OSUOSL lab who > instead operated mailman2 for the past ~4 years. >=20 > OSUOSL has for decades been well known for providing server > infrastructure for Linux and Open Source development so they were a > good fit. This however became an added maintenance burden for the > small non-profit with limited resources. Several members of the > Bitcoin dev community contributed funding to the lab in support of > their Open Source development infrastructure goals. But throwing > money at the problem isn=E2=80=99t going to fix the ongoing maintenan= ce > burden created by antiquated limitations of mailman2. >=20 > Permalinks > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > Linux Foundation has either offered or agreed to maintain archive > permalinks so that content of historic importance is not lost. > Fortunately for us while lists.linuxfoundation.org > mailman will go down, they have > agreed the read-only pipermail archives will remain online. So all > old URLs will continue to remain valid. However, the moderators > strongly advise that the community supplements with public-inbox > instances to have canonical archive urls that are separate from any > particular email software host. >=20 > Public-Inbox > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > https://public-inbox.org/README.html > >=20 > =E2=80=9CPublic Inbox=E2=80=9D decentralized archiving - no matter wh= at mailing list > server solution is used, anyone can use git to maintain their own > mailing list archive and make it available to read on the web. >=20 > Public Inbox is a tool that you can run yourself. You can transform > your mbox file and it makes it browsable and viewable online. It > commits every post to a git repository. It's kind of like a > decentralized mail archiving tool. Anyone can publish the mail > archive to any web server they wish. >=20 > We should try to have one or more canonical archives that are served > using public-inbox. But it doesn't matter if these are lost because > anyone else can archive the mailing list in the same way and > re-publish the archives. >=20 > These git commits can also be stamped using opentimestamps, > inserting their hashes into the bitcoin blockchain. >=20 > LKML mailing list readers often use public-inbox's web interface, > and they use the reply-to headers to populate their mail client and > reply to threads of interest. This allows their reply to be properly > threaded even if they were not a previous subscriber to that mailing > list to receive the headers. >=20 > public-inbox makes it so that it doesn't really matter where the > list is hosted, as pertaining to reading the mailing list. There is > still a disruption if the mailing list goes away, but the archives > live on and then people can post elsewhere. The archive gets > disconnected from the mailing list host in terms of posting. We > could have a few canonical URLs for the hosts, separate from the > mailing list server. >=20 > mailman problems > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > Over the years we have identified a number of problems with mailman2 > especially as it pertains to content moderation. There are presently > a handful of different moderators, but mailman2 only has a single > password for logging into the email management interface. There are > no moderator audit logs to see which user (there is no concept of > different users) acted on an email. There is no way to mark an email > as being investigated by one or more of the moderators. Sometimes, > while investigating the veracity of an email, another moderator > would come in and approve a suspect email by accident. >=20 > Anti spam has been an issue for the moderators. It's relentless. > Without access to the underlying server, it has been difficult to > fight spam. There is some support for filters in mailman2 but it's > not great. >=20 > 100% active moderation and approval of every email is unsustainable > for volunteer moderators. A system that requires moderation is a > heavy burden on the moderators and it slows down overall > communication and productivity. There's lots of problems with this. > Also, moderators can be blamed when they are merely slow while they > are not actually censoring. >=20 > Rejection emails can optionally be sent to > bitcoin-dev-moderation@lists.ozlabs.org > but this is an > option that a moderator has to remember to type in each time. >=20 > Not to mention numerous bugs and vulnerabilities that have > accumulated over the years for relatively unmaintained software. > (Not disclosed here) >=20 > Requirements and considerations > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D >=20 > Looking towards the future, there are a number of properties that > seem to be important for the bitcoin-dev mailing list community. > First, it is important that backups of the entire archive should be > easy for the public to copy or verify so that the system can be > brought up elsewhere if necessary. >=20 > Second, there seems to be demand for both an email threading > interface (using mailing list software) as well as web-accessible > interfaces (such as forum software). There seems to be very few > options that cater to both email and web. Often, in forum software, > email support is limited to email notifications and there is limited > if any support for email user participation. >=20 > Third, there should be better support for moderator tools and > management of the mailing list. See above for complaints about > problems with the mailman2 system. >=20 > Burdens of running your own mailing list and email server > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > If you have never operated your own MTA you have no idea how > difficult it is to keep secure and functional in the face of > numerous challenges to deliverability. Anti-spam filtering is > essential to prevent forwarding spam. The moment you forward even a > single spam message you run the risk of the server IP address being > added to blacklists. >=20 > The problem of spam filtering is so bad that most IP addresses are > presumed guilty even if they have no prior spam history, such as if > their network or subnetwork had spam issues in the past. >=20 > Even if you put unlimited time into managing your own email server, > other people may not accept your email. Or you make one mistake, and > then you get into permanent blacklists and it's hard to remove. The > spam problem is so bad that most IPs are automatically on a > guilty-until-proven-innocent blacklist. >=20 > Often there is nothing you can do to get server IP addresses removed > from spam blacklists or from "bad reputation" lists. >=20 > Ironically, hashcash-style proof-of-work stamps to prevent spam are > an appealing solution but not widely used in this community. Or > anywhere. >=20 > Infinite rejection or forwarding loops happen. They often need to be > detected through vigilance and require manual sysadmin intervention > to solve. >=20 > Bitcoin's dev lists being hosted alongside other Open Source > projects was previously protective. If that mailing list server > became blacklisted there were a lot of other people who would notice > and complain. If we run a Bitcoin-specific mail server we are on our > own. 100% of the administrative burden falls upon our own people. > There is also nothing we can do if some unknown admin decides they > don't like us. >=20 > Options > =3D=3D=3D=3D=3D=3D=3D >=20 > Web forums are an interesting option, but often don't have good > email user integration. At most you can usually hope for email > notifications and an ability to reply by email. It changes the model > of the community from push (email) to pull (logging into a forum to > read). RSS feeds can help a little bit. >=20 > Many other projects have moved from mailing lists to forums (eg > https://discuss.python.org/ =E2=80= =93 see > https://lwn.net/Articles/901744/ > ; or https://ethresear.ch/ ), which seem > easier to maintain and moderate, and can have lots of advanced > features beyond plaintext, maybe-threading and maybe-HTML-markup. >=20 > Who would host the forum? Would there be agreement around which > forum software to use or which forum host? What about > bitcointalk.org or delvingbitcoin.org > ? There are many options available. Maybe > what we actually want isn=E2=80=99t so much a discussion forum, as an= 'arxiv > of our own' where anons can post BIP drafts and the like? >=20 > Given the problems with mailman2, and the decline of email > communities in general, it seems that moving to mailman3 would not > be a viable long-term option. This leaves us with Google Groups or > groups.io as two remaining options. >=20 > groups.io is an interesting option: they are a > paid service that implements email communities along with online web > forum support. However, their public changelog indicates it has been > a few years since their last public change. They might be a smaller > company and it is unclear how long they will be around or if this > would be the right fit for hosting sometimes contentious bitcoin > development discussions... >=20 > Google Groups is another interesting option, and comes with > different tradeoffs. It's the lowest effort to maintain option, and > has both an email interface and web forum interface. Users can > choose which mode they want to interact with. >=20 > For the Google Groups web interface, you can use it with a non-gmail > account, but you must create a Google Account which is free to do. > it does not require any personal information to do so. This also > allows you to add 2FA. Non-gmail non-google users are able to > subscribe and post email from their non-gmail non-google email > accounts. Tor seems to work for the web interface. >=20 > Will Google shut it down, will they cut us off, will they shut down > non-google users? The same problem exists with other third-party host= s. >=20 > The moderation capabilities for Google Groups and groups.io > seem to be comparable. It seems more likely that > Google Groups will be able to handle email delivery issues far > better than a small resource-constrained operation like groups.io > . ((During feedback for this draft, luke-jr > indicates that Google Workspaces has been known to use blacklisted > IPs for business email!)) >=20 > On the other hand, groups.io is a paid service > and you get what you pay for... hopefully? >=20 > Finally, another option is to do literally nothing. It's less work > overall. Users can switch to forums or other websites, or private > one-on-one communication. It would remove a point of > semi-centralization from the bitcoin ecosystem. It would hasten > ossification, but on the other hand it would hasten ossification and > this could be a negative too. Moderators would be less of a target. >=20 > Unfortunately, by doing nothing, there would be no more widely used > group email communication system between bitcoin developers. > Developers become less coordinated, mayhem and chaos as people go to > different communication platforms, a divided community is more > vulnerable, etc. BIP1 and BIP2 would need to be revised for other > venues. >=20 > The main categories of what to move to are: web forums, mailing > lists, and hybrids of those two options. Most everything is either > self-hosted or you pay someone else to host it. It's kind of the > same problem though. It largely depends on how good is the software > and unfortunately running your own MTA that forwards mail is not a > good option. >=20 > Going forward > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > We'd like to invite feedback and proposals from the community, and > see what options are available. One potential option is a migration > to Google Groups, but we're open to ideas at this point. We > apologize for any inconvenience this disruption has caused. >=20 >=20 > Bitcoin-dev mailing list moderation team >=20 > Bryan Bishop > Ruben Somsen > Warren Togami > various others. >=20 > --=20 > - Bryan > https://twitter.com/kanzure > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >=20