Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Ali Sherief]

I find Google Groups especially repugnant not not only because what has already been mentioned, but Google Groups has a quite clunky and annoying user interface that makes it difficult for me to find anything or interest in there.

Usenet was migrated to Google Groups for some reason, and it's very difficult to search for anything of particular interest using that site.

Not to mention that Google Groups also contains a larger amount of spam (w.r.t value), so arguably the moderation burden will be higher.

It is necessary to try to find a way to keep the discussion on a mail server, since a migration off of it will render many users' email clients useless for this purpose.

- Ali

> On Mon, 13 Nov 2023 18:51:26 +0000, alicexbt <alicexbt@protonmail.com> wrote:
>
> Hi Overthefalls,
>
> +1
>
> Using google for bitcoin mailing list is not good. It feels embarrassing that some developers that built and maintained the only decentralized network used to settle uncensored payments and some of them even working on nostr, can't build their own mailing list which is better than present mailing list. I have some ideas but it seems the influential developers have already decided and wont accept anything.
>
> Nostr can be used to build a mailing list which also allows anyone to send emails apart from publishing events from different clients. We just need a new NIP so that nostr relays understand its a different event. There can be multiple front end with different levels of moderation to hide some emails and ultimately one will be used the most. It can use multiple relays and relays share some information in NIP 11 which can include an email address.
>
> /dev/fd0
> floppy disk guy
>
> Sent with Proton Mail secure email.
>
> On Monday, November 13th, 2023 at 8:35 PM, Overthefalls via bitcoin-dev bitcoin-dev@lists.linuxfoundation.org wrote:
>
> > On Tue, 2023-11-07 at 09:37 -0600, Bryan Bishop via bitcoin-dev wrote:
> >
> > > Google Groups is another interesting option,
> >
> > I don't think I'm the only person on this list that is strongly opposed to using google for anything. They are too big and they have their hand in everything, and their eyes (and analytics) on everything.
> >
> > I remember when there were virtually no gmail email addresses that posted to this list. Suddenly in 2020 or 2021, we had an influx of gmail subscribers and posters. That didn't escape me then and it is not lost on me now.
> >
> > Email is great for public discussion for many reasons. The fact that everyone gets a copy of the data, there is no single central authority that can edit emails once they have been sent out. Anyone can archive email messages, they can generally store or publish the data anywhere they like. That is not the case with web forum content.
> >
> > I like the lightning anti-spam fee idea. That would encourage me to finally adopt lightning, and it would, I'm sure, produce some interesting results for the list.
> >
> > I don't think email should be out of the question. Does anyone besides kanzure@gmail.com think that sticking with email is out of the question?
> >
> > Let's do what's necessary to stick with email.

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Brad Morrison]

[-- Attachment #1: Type: text/plain, Size: 3572 bytes --]

Hi all, 

It looks like there are only a few mailing lists left on
https://lists.linuxfoundation.org/mailman/listinfo and all of the
remaining ones are using Mailman version 2.1.15, which is not the
current version - https://www.gnu.org/software/mailman/   

Was there any decision made on where to move the bitcoin-dev mailing
list to? 

Thanks, 

Brad

On 2023-11-11 02:54, vjudeu via bitcoin-dev wrote:

> What about using Signet, or some separate P2P network, to handle all of that? 
> 
> 1. All e-mails could be sent in a pure P2P way, just each "mailing list node" would receive it, and include to its mempool.
> 2. The inclusion of some message would be decided by signing a block. Moderators would pick the proper messages, and publish them by broadcasting a new block to all nodes.
> 3. Each message will be signed by some public key. It could be changed each time, or even derived from some HD wallet. Only those owning "master public keys" would know, which messages were sent by the same person.
> 4. The time of the block could be much longer than 10 minutes. It could be for example one hour, one day, or even longer. Or, the commitment to all of that could be just included "every sometimes" to the existing Signet chain, because it would take no additional on-chain bytes, and can be easily done in the coinbase transaction.
> 5. If there will be too much spam in the mempool, then hashcash-based Proof of Work can be used to filter messages. Instead of fee-based filtering, it could be Proof-of-Work-based filtering. Even better: because of "master public keys", the regular participants could be allowed anyway, without providing additional Proof of Work. Their signature would be sufficient in that case.
> 6. The code is almost there. Maybe there are even altcoins, designed specifically for storing data, and we could just use them?
> 7. This kind of decision would push things like Silent Payments forward. Because then, you could develop scanners, to know, who wrote which message. You could enter some "master public key", scan the whole chain, and find out all messages written by that particular participant.
> 8. It would push commitments forward. Because then, it would be possible to send some message to the "P2P mailing list network", and reveal it later. Of course, it is not mandatory to accept commitments at all, which means, they could be easily disabled, if they would be misused. Or we could start with no commitments, and introduce them later if needed.
> 9. Because Signet challenge can contain some multisig, or even some Taproot address, there will be no issue with using the same password to access the moderation panel. Also, in that case, it is possible to prove later, which moderator accepted which message. And also, it is still possible to use some shared key, if revealing that is not desirable, or even it is possible to easily reach "approved by all moderators" messages, because their Schnorr signatures could be combined. Also, any K-of-N multisig can be battle-tested in that way. 
> 
> So, I can see two options: reusing some existing P2P network, or making a new one, designed specifically for handling mailing list messages in a pure P2P way. I guess we can try some existing chains first, and if there is no promising altcoin, or if we don't want to be associated with any altcoin, then our own Signet-like network could solve it. 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 4448 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Overthefalls]

[-- Attachment #1: Type: text/plain, Size: 3029 bytes --]

Hi floppy disk guy, thanks for prompting me to look closer at Nostr,
it's very interesting. 

I hope that whatever solution is chosen doesn't involve handing power
over to a centralized entity that wants collect as much information on
every living person as possible, and lock everyone and everything into
using it's services forever.

On Mon, 2023-11-13 at 18:51 +0000, alicexbt wrote:
> Hi Overthefalls,
> 
> 
> +1
> 
> 
> Using google for bitcoin mailing list is not good. It feels
> embarrassing that some developers that built and maintained the only
> decentralized network used to settle uncensored payments and some of
> them even working on nostr, can't build their own mailing list which
> is better than present mailing list. I have some ideas but it seems
> the influential developers have already decided and wont accept
> anything.
> 
> Nostr can be used to build a mailing list which also allows anyone to
> send emails apart from publishing events from different clients. We
> just need a new NIP so that nostr relays understand its a different
> event. There can be multiple front end with different levels of
> moderation to hide some emails and ultimately one will be used the
> most. It can use multiple relays and relays share some information in
> NIP 11 which can include an email address.
> 
> 
> /dev/fd0
> floppy disk guy
> 
> 
> 
>     
>         
>             
>     
>             
>         Sent with Proton Mail secure email.
>     
> 
> 
> 
> 
>         On Monday, November 13th, 2023 at 8:35 PM, Overthefalls via
> bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
> 
>         
> >             On Tue, 2023-11-07 at 09:37 -0600, Bryan Bishop via
> > bitcoin-dev wrote:
> > > Google Groups is another interesting option, 
> > 
> > I don't think I'm the only person on this list that is strongly
> > opposed to using google for anything. They are too big and they
> > have their hand in everything, and their eyes (and analytics) on
> > everything.
> > I remember when there were virtually no gmail email addresses that
> > posted to this list. Suddenly in 2020 or 2021, we had an influx of
> > gmail subscribers and posters. That didn't escape me then and it is
> > not lost on me now. 
> > Email is great for public discussion for many reasons. The fact
> > that everyone gets a copy of the data, there is no single central
> > authority that can edit emails once they have been sent out. Anyone
> > can archive email messages, they can generally store or publish the
> > data anywhere they like. That is not the case with web forum
> > content. 
> > I like the lightning anti-spam fee idea. That would encourage me to
> > finally adopt lightning, and it would, I'm sure, produce some
> > interesting results for the list. 
> > I don't think email should be out of the question. Does anyone
> > besides kanzure@gmail.com think that sticking with email is out of
> > the question?
> > Let's do what's necessary to stick with email. 
> > 
> > 
> > 
> > 
> > 
> >         
> 
>     



[-- Attachment #2: Type: text/html, Size: 5105 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[alicexbt]

[-- Attachment #1: Type: text/plain, Size: 2352 bytes --]

Hi Overthefalls,

+1

Using google for bitcoin mailing list is not good. It feels embarrassing that some developers that built and maintained the only decentralized network used to settle uncensored payments and some of them even working on nostr, can't build their own mailing list which is better than present mailing list. I have some ideas but it seems the influential developers have already decided and wont accept anything.

Nostr can be used to build a mailing list which also allows anyone to send emails apart from publishing events from different clients. We just need a new NIP so that nostr relays understand its a different event. There can be multiple front end with different levels of moderation to hide some emails and ultimately one will be used the most. It can use multiple relays and relays share some information in NIP 11 which can include an email address.

/dev/fd0
floppy disk guy

Sent with [Proton Mail](https://proton.me/) secure email.

On Monday, November 13th, 2023 at 8:35 PM, Overthefalls via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Tue, 2023-11-07 at 09:37 -0600, Bryan Bishop via bitcoin-dev wrote:
>
>> Google Groups is another interesting option,
>
> I don't think I'm the only person on this list that is strongly opposed to using google for anything. They are too big and they have their hand in everything, and their eyes (and analytics) on everything.
>
> I remember when there were virtually no gmail email addresses that posted to this list. Suddenly in 2020 or 2021, we had an influx of gmail subscribers and posters. That didn't escape me then and it is not lost on me now.
>
> Email is great for public discussion for many reasons. The fact that everyone gets a copy of the data, there is no single central authority that can edit emails once they have been sent out. Anyone can archive email messages, they can generally store or publish the data anywhere they like. That is not the case with web forum content.
>
> I like the lightning anti-spam fee idea. That would encourage me to finally adopt lightning, and it would, I'm sure, produce some interesting results for the list.
>
> I don't think email should be out of the question. Does anyone besides kanzure@gmail.com think that sticking with email is out of the question?
>
> Let's do what's necessary to stick with email.

[-- Attachment #2: Type: text/html, Size: 4379 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Overthefalls]

[-- Attachment #1: Type: text/plain, Size: 1250 bytes --]

On Tue, 2023-11-07 at 09:37 -0600, Bryan Bishop via bitcoin-dev wrote:
Google Groups is another interesting option, 

I don't think I'm the only person on this list that is strongly opposed
to using google for anything. They are too big and they have their hand
in everything, and their eyes (and analytics) on everything.

I remember when there were virtually no gmail email addresses that
posted to this list. Suddenly in 2020 or 2021, we had an influx of
gmail subscribers and posters. That didn't escape me then and it is not
lost on me now. 

Email is great for public discussion for many reasons. The fact that
everyone gets a copy of the data, there is no single central authority
that can edit emails once they have been sent out. Anyone can archive
email messages, they can generally store or publish the data anywhere
they like. That is not the case with web forum content. 

I like the lightning anti-spam fee idea. That would encourage me to
finally adopt lightning, and it would, I'm sure, produce some
interesting results for the list. 

I don't think email should be out of the question. Does anyone besides 
kanzure@gmail.com think that sticking with email is out of the
question?

Let's do what's necessary to stick with email. 





[-- Attachment #2: Type: text/html, Size: 1699 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Antoine Riard]

[-- Attachment #1: Type: text/plain, Size: 14819 bytes --]

Thanks for the write up and thanks to the bitcoin-dev mailing list
moderation team for their work along the years.

If we can pick up a communication platform where platform moderators /
infra maintainers have low-risk of being targeted by subpoena + gag order
or "injonction administrative" (the equivalent in some civil law systems)
due to lack of moderators discretionary decisions, I think this is a good
outcome.

I don't know of such a communication platform or set of protocols as of
today. Nostr is promising though realistically weak until half a decade of
work is poured in.

Personally, I'll be more present on the Delving Bitcoin forum, though it
sounds more a temporary solution than a long-term ideal. Being hosted by
kernels or other old open-sources project mailing list infra sounds like a
good idea.

Best,
Antoine

Le mar. 7 nov. 2023 à 15:37, Bryan Bishop via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> a écrit :

> Hello,
>
> We would like to request community feedback and proposals on the future of
> the mailing list.
>
> Our current mailing list host, Linux Foundation, has indicated for years
> that they have wanted to stop hosting mailing lists, which would mean the
> bitcoin-dev mailing list would need to move somewhere else. We temporarily
> avoided that, but recently LF has informed a moderator that they will cease
> hosting any mailing lists later this year.
>
> In this email, we will go over some of the history, options, and invite
> discussion ahead of the cutoff. We have some ideas but want to solicit
> feedback and proposals.
>
> Background
> ==========
>
> The bitcoin-dev mailing list was originally hosted on Sourceforge.net. The
> bitcoin development mailing list has been a source of proposals, analysis,
> and developer discussion for many years in the bitcoin community, with many
> thousands of participants. Later, the mailing list was migrated to the
> Linux Foundation, and after that OSUOSL began to help.
>
> Linux Foundation first asked us to move the mailing list in 2017. They
> internally attempted to migrate all LF mailing lists from mailman2 to
> mailman3, but ultimately gave up. There were reports of scalability issues
> with mailman3 for large email communities. Ours definitely qualifies as..
> large.
>
> 2019 migration plan: LF was to turn off mailman and all lists would
> migrate to the paid service provider groups.io. Back then we were given
> accounts to try the groups.io interface and administration features.
> Apparently we were not the only dev community who resisted change. To our
> surprise LF gave us several years of reprieve by instead handing the
> subdomain and server-side data to the non-profit OSUOSL lab who instead
> operated mailman2 for the past ~4 years.
>
> OSUOSL has for decades been well known for providing server infrastructure
> for Linux and Open Source development so they were a good fit. This however
> became an added maintenance burden for the small non-profit with limited
> resources. Several members of the Bitcoin dev community contributed funding
> to the lab in support of their Open Source development infrastructure
> goals. But throwing money at the problem isn’t going to fix the ongoing
> maintenance burden created by antiquated limitations of mailman2.
>
> Permalinks
> ==========
>
> Linux Foundation has either offered or agreed to maintain archive
> permalinks so that content of historic importance is not lost. Fortunately
> for us while lists.linuxfoundation.org mailman will go down, they have
> agreed the read-only pipermail archives will remain online. So all old URLs
> will continue to remain valid. However, the moderators strongly advise that
> the community supplements with public-inbox instances to have canonical
> archive urls that are separate from any particular email software host.
>
> Public-Inbox
> ============
>
> https://public-inbox.org/README.html
>
> “Public Inbox” decentralized archiving - no matter what mailing list
> server solution is used, anyone can use git to maintain their own mailing
> list archive and make it available to read on the web.
>
> Public Inbox is a tool that you can run yourself. You can transform your
> mbox file and it makes it browsable and viewable online. It commits every
> post to a git repository. It's kind of like a decentralized mail archiving
> tool. Anyone can publish the mail archive to any web server they wish.
>
> We should try to have one or more canonical archives that are served using
> public-inbox. But it doesn't matter if these are lost because anyone else
> can archive the mailing list in the same way and re-publish the archives.
>
> These git commits can also be stamped using opentimestamps, inserting
> their hashes into the bitcoin blockchain.
>
> LKML mailing list readers often use public-inbox's web interface, and they
> use the reply-to headers to populate their mail client and reply to threads
> of interest. This allows their reply to be properly threaded even if they
> were not a previous subscriber to that mailing list to receive the headers.
>
> public-inbox makes it so that it doesn't really matter where the list is
> hosted, as pertaining to reading the mailing list. There is still a
> disruption if the mailing list goes away, but the archives live on and then
> people can post elsewhere. The archive gets disconnected from the mailing
> list host in terms of posting. We could have a few canonical URLs for the
> hosts, separate from the mailing list server.
>
> mailman problems
> ================
>
> Over the years we have identified a number of problems with mailman2
> especially as it pertains to content moderation. There are presently a
> handful of different moderators, but mailman2 only has a single password
> for logging into the email management interface. There are no moderator
> audit logs to see which user (there is no concept of different users) acted
> on an email. There is no way to mark an email as being investigated by one
> or more of the moderators. Sometimes, while investigating the veracity of
> an email, another moderator would come in and approve a suspect email by
> accident.
>
> Anti spam has been an issue for the moderators. It's relentless. Without
> access to the underlying server, it has been difficult to fight spam. There
> is some support for filters in mailman2 but it's not great.
>
> 100% active moderation and approval of every email is unsustainable for
> volunteer moderators. A system that requires moderation is a heavy burden
> on the moderators and it slows down overall communication and productivity.
> There's lots of problems with this. Also, moderators can be blamed when
> they are merely slow while they are not actually censoring.
>
> Rejection emails can optionally be sent to
> bitcoin-dev-moderation@lists.ozlabs.org but this is an option that a
> moderator has to remember to type in each time.
>
> Not to mention numerous bugs and vulnerabilities that have accumulated
> over the years for relatively unmaintained software. (Not disclosed here)
>
> Requirements and considerations
> ===============================
>
> Looking towards the future, there are a number of properties that seem to
> be important for the bitcoin-dev mailing list community. First, it is
> important that backups of the entire archive should be easy for the public
> to copy or verify so that the system can be brought up elsewhere if
> necessary.
>
> Second, there seems to be demand for both an email threading interface
> (using mailing list software) as well as web-accessible interfaces (such as
> forum software). There seems to be very few options that cater to both
> email and web. Often, in forum software, email support is limited to email
> notifications and there is limited if any support for email user
> participation.
>
> Third, there should be better support for moderator tools and management
> of the mailing list. See above for complaints about problems with the
> mailman2 system.
>
> Burdens of running your own mailing list and email server
> =========================================================
>
> If you have never operated your own MTA you have no idea how difficult it
> is to keep secure and functional in the face of numerous challenges to
> deliverability. Anti-spam filtering is essential to prevent forwarding
> spam. The moment you forward even a single spam message you run the risk of
> the server IP address being added to blacklists.
>
> The problem of spam filtering is so bad that most IP addresses are
> presumed guilty even if they have no prior spam history, such as if their
> network or subnetwork had spam issues in the past.
>
> Even if you put unlimited time into managing your own email server, other
> people may not accept your email. Or you make one mistake, and then you get
> into permanent blacklists and it's hard to remove. The spam problem is so
> bad that most IPs are automatically on a guilty-until-proven-innocent
> blacklist.
>
> Often there is nothing you can do to get server IP addresses removed from
> spam blacklists or from "bad reputation" lists.
>
> Ironically, hashcash-style proof-of-work stamps to prevent spam are an
> appealing solution but not widely used in this community. Or anywhere.
>
> Infinite rejection or forwarding loops happen. They often need to be
> detected through vigilance and require manual sysadmin intervention to
> solve.
>
> Bitcoin's dev lists being hosted alongside other Open Source projects was
> previously protective. If that mailing list server became blacklisted there
> were a lot of other people who would notice and complain. If we run a
> Bitcoin-specific mail server we are on our own. 100% of the administrative
> burden falls upon our own people. There is also nothing we can do if some
> unknown admin decides they don't like us.
>
> Options
> =======
>
> Web forums are an interesting option, but often don't have good email user
> integration. At most you can usually hope for email notifications and an
> ability to reply by email. It changes the model of the community from push
> (email) to pull (logging into a forum to read). RSS feeds can help a little
> bit.
>
> Many other projects have moved from mailing lists to forums (eg
> https://discuss.python.org/ – see https://lwn.net/Articles/901744/ ; or
> https://ethresear.ch/), which seem easier to maintain and moderate, and
> can have lots of advanced features beyond plaintext, maybe-threading and
> maybe-HTML-markup.
>
> Who would host the forum? Would there be agreement around which forum
> software to use or which forum host? What about bitcointalk.org or
> delvingbitcoin.org? There are many options available. Maybe what we
> actually want isn’t so much a discussion forum, as an 'arxiv of our own'
> where anons can post BIP drafts and the like?
>
> Given the problems with mailman2, and the decline of email communities in
> general, it seems that moving to mailman3 would not be a viable long-term
> option. This leaves us with Google Groups or groups.io as two remaining
> options.
>
> groups.io is an interesting option: they are a paid service that
> implements email communities along with online web forum support. However,
> their public changelog indicates it has been a few years since their last
> public change. They might be a smaller company and it is unclear how long
> they will be around or if this would be the right fit for hosting sometimes
> contentious bitcoin development discussions...
>
> Google Groups is another interesting option, and comes with different
> tradeoffs. It's the lowest effort to maintain option, and has both an email
> interface and web forum interface. Users can choose which mode they want to
> interact with.
>
> For the Google Groups web interface, you can use it with a non-gmail
> account, but you must create a Google Account which is free to do. it does
> not require any personal information to do so. This also allows you to add
> 2FA. Non-gmail non-google users are able to subscribe and post email from
> their non-gmail non-google email accounts. Tor seems to work for the web
> interface.
>
> Will Google shut it down, will they cut us off, will they shut down
> non-google users? The same problem exists with other third-party hosts.
>
> The moderation capabilities for Google Groups and groups.io seem to be
> comparable. It seems more likely that Google Groups will be able to handle
> email delivery issues far better than a small resource-constrained
> operation like groups.io. ((During feedback for this draft, luke-jr
> indicates that Google Workspaces has been known to use blacklisted IPs for
> business email!))
>
> On the other hand, groups.io is a paid service and you get what you pay
> for... hopefully?
>
> Finally, another option is to do literally nothing. It's less work
> overall. Users can switch to forums or other websites, or private
> one-on-one communication. It would remove a point of semi-centralization
> from the bitcoin ecosystem. It would hasten ossification, but on the other
> hand it would hasten ossification and this could be a negative too.
> Moderators would be less of a target.
>
> Unfortunately, by doing nothing, there would be no more widely used group
> email communication system between bitcoin developers. Developers become
> less coordinated, mayhem and chaos as people go to different communication
> platforms, a divided community is more vulnerable, etc. BIP1 and BIP2 would
> need to be revised for other venues.
>
> The main categories of what to move to are: web forums, mailing lists, and
> hybrids of those two options. Most everything is either self-hosted or you
> pay someone else to host it. It's kind of the same problem though. It
> largely depends on how good is the software and unfortunately running your
> own MTA that forwards mail is not a good option.
>
> Going forward
> ===========
>
> We'd like to invite feedback and proposals from the community, and see
> what options are available. One potential option is a migration to Google
> Groups, but we're open to ideas at this point. We apologize for any
> inconvenience this disruption has caused.
>
>
> Bitcoin-dev mailing list moderation team
>
> Bryan Bishop
> Ruben Somsen
> Warren Togami
> various others.
>
> --
> - Bryan
> https://twitter.com/kanzure
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 16205 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[vjudeu]

[-- Attachment #1: Type: text/plain, Size: 2970 bytes --]

What about using Signet, or some separate P2P network, to handle all of that?
 
1. All e-mails could be sent in a pure P2P way, just each "mailing list node" would receive it, and include to its mempool.
2. The inclusion of some message would be decided by signing a block. Moderators would pick the proper messages, and publish them by broadcasting a new block to all nodes.
3. Each message will be signed by some public key. It could be changed each time, or even derived from some HD wallet. Only those owning "master public keys" would know, which messages were sent by the same person.
4. The time of the block could be much longer than 10 minutes. It could be for example one hour, one day, or even longer. Or, the commitment to all of that could be just included "every sometimes" to the existing Signet chain, because it would take no additional on-chain bytes, and can be easily done in the coinbase transaction.
5. If there will be too much spam in the mempool, then hashcash-based Proof of Work can be used to filter messages. Instead of fee-based filtering, it could be Proof-of-Work-based filtering. Even better: because of "master public keys", the regular participants could be allowed anyway, without providing additional Proof of Work. Their signature would be sufficient in that case.
6. The code is almost there. Maybe there are even altcoins, designed specifically for storing data, and we could just use them?
7. This kind of decision would push things like Silent Payments forward. Because then, you could develop scanners, to know, who wrote which message. You could enter some "master public key", scan the whole chain, and find out all messages written by that particular participant.
8. It would push commitments forward. Because then, it would be possible to send some message to the "P2P mailing list network", and reveal it later. Of course, it is not mandatory to accept commitments at all, which means, they could be easily disabled, if they would be misused. Or we could start with no commitments, and introduce them later if needed.
9. Because Signet challenge can contain some multisig, or even some Taproot address, there will be no issue with using the same password to access the moderation panel. Also, in that case, it is possible to prove later, which moderator accepted which message. And also, it is still possible to use some shared key, if revealing that is not desirable, or even it is possible to easily reach "approved by all moderators" messages, because their Schnorr signatures could be combined. Also, any K-of-N multisig can be battle-tested in that way.
 
So, I can see two options: reusing some existing P2P network, or making a new one, designed specifically for handling mailing list messages in a pure P2P way. I guess we can try some existing chains first, and if there is no promising altcoin, or if we don't want to be associated with any altcoin, then our own Signet-like network could solve it.

[-- Attachment #2: Type: text/html, Size: 3082 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[William Casarin]

On Tue, Nov 07, 2023 at 06:14:23PM +0000, Andrew Chow via bitcoin-dev wrote:
>Hi Dan,
>
>I don't think nostr would be a suitable replacement for the mailing
>list, although this opinion is biased by the fact that I do not use
>nostr and find it to be uninteresting.

email-like functionality over nostr isn't really explored yet, but it is
something I'm interesting in. So I agree nostr isn't a suitable email
replacement *yet*.

From my limited understanding of how nostr works, it's not clear to me
>how a distributed system that uses message broadcast would work in the
>same way as a mailing list.

My idea was to have a mailing list relay, the only thing missing is To:
and Cc: tags on notes so that the relay can reject notes not destined
for the mailing list

>How would people "subscribe"? How would archives be searched or
>otherwise be available to people who are not on nostr?

You would subscribe by connecting to the relay and pulling down the
notes. your client could cache notes and only pull new ones.

>How do you distinguish and filter between legitimate dev posts
>intended for discussion and random crap and shitposting as shows up on
>social media?

You would need to have metadata on the note that specifies that the note
is destined for that specific mailing list relay (To, Cc, etc). Then the
client sending the message can send it to that specific relay during
note composition. Again, this is different than then current model that
exists with social networking clients designed for blasting your note to
as many people as possible.

>I also don't think that long form text on nostr (or any similar
>platform) can sufficiently replace email. None of these things seem to
>contain a way to have a separate subject line as email does. Subjects
>are immensely important for me as it provides a quick and easy way to
>filter out things I don't care about reading. I don't want to have read
>something in before I can decide that I don't care about reading it.

Subject lines already exist in nostr and are a part of some email-like
clients like https://github.com/unclebob/more-speech . it's just a tag
like every other piece of metadata.

>In general, I strongly prefer email, or a platform that has email as a
>first class user interface, over platforms such as nostr, matrix, or web
>forums. Email is universal - everyone has one and everyone knows how it
>works. It dramatically lowers the barrier of entry. Having to make an
>account somewhere or download some specific client in order to
>participate will simply result in only the most dedicated participating.
>Development in open source must be an open process and the barriers to
>entry should be low.

I definitely prefer email at the moment as well, but it is also a pain
in the ass to run email infra. As someone who runs both email servers
and nostr relays I can say nostr is much more pleasant.

So yeah, it's a bit too early for a nostr replacement, but it's
definitely possible, and you get proper cryptographic identities and
schnorr signed notes which is a bonus. For dealing with spam you could
have a sat entrance fee via lightning. I will start looking into this!

Cheers,

	Will

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Emil Pfeffer]

On Tue, Nov 07, 2023 at 11:08:58PM +0000, Peter Todd via bitcoin-dev wrote:
> On Tue, Nov 07, 2023 at 09:37:22AM -0600, Bryan Bishop via bitcoin-dev wrote:
> > Anti spam has been an issue for the moderators. It's relentless. Without
> > access to the underlying server, it has been difficult to fight spam. There
> > is some support for filters in mailman2 but it's not great.
> 
> Since this is a technical mailing list it would be fine to require people to
> pay a non-refundable anti-spam fee, eg via lightning, to gain the ability to
> send messages. While this would require some custom software, it's probably
> even possible to implement this if a third party is used for hosting, provided
> they have some kind of API.
> 

Could be done but it's overkill.
Running your own mail server is not that scary as the initial post makes it out
to be. Sending out spam is problematic only when you allow imap/pop3 access
which is not the case when running a mailling list.

My suggestion is to get the Bitcoin CEO hook us up on https://lists.freebsd.org
Failling that I volunteer to get the same setup going for a community run
project.
-- 

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[email]

[-- Attachment #1: Type: text/plain, Size: 15212 bytes --]


On 2023-11-07 17:12, Andrew Chow via bitcoin-dev wrote:

> I would prefer that we continue to have a mailing list where email is a
> functional and first class user interface. So that would be to migrate
> to groups.io or Google Groups. I think Google Groups is probably the
> better choice of the two.

+1 to migrating to a different email service.  That seems like the most 
straightforward solution.

On 2023-11-08 04:56, Anthony Towns via bitcoin-dev wrote:

> delvingbitcoin.org is something I setup

Cool, thanks!  It's great to have more channels of discussion.  Same 
with IRC etc.

Cheers,
-Yancy

> Andrew Chow
> 
> On 11/07/2023 10:37 AM, Bryan Bishop via bitcoin-dev wrote:
> 
>> Hello,
>> 
>> We would like to request community feedback and proposals on the 
>> future
>> of the mailing list.
>> 
>> Our current mailing list host, Linux Foundation, has indicated for 
>> years
>> that they have wanted to stop hosting mailing lists, which would mean
>> the bitcoin-dev mailing list would need to move somewhere else. We
>> temporarily avoided that, but recently LF has informed a moderator 
>> that
>> they will cease hosting any mailing lists later this year.
>> 
>> In this email, we will go over some of the history, options, and 
>> invite
>> discussion ahead of the cutoff. We have some ideas but want to solicit
>> feedback and proposals.
>> 
>> Background
>> ==========
>> 
>> The bitcoin-dev mailing list was originally hosted on Sourceforge.net.
>> The bitcoin development mailing list has been a source of proposals,
>> analysis, and developer discussion for many years in the bitcoin
>> community, with many thousands of participants. Later, the mailing 
>> list
>> was migrated to the Linux Foundation, and after that OSUOSL began to 
>> help.
>> 
>> Linux Foundation first asked us to move the mailing list in 2017. They
>> internally attempted to migrate all LF mailing lists from mailman2 to
>> mailman3, but ultimately gave up. There were reports of scalability
>> issues with mailman3 for large email communities. Ours definitely
>> qualifies as.. large.
>> 
>> 2019 migration plan: LF was to turn off mailman and all lists would
>> migrate to the paid service provider groups.io <http://groups.io>. 
>> Back
>> then we were given accounts to try the groups.io <http://groups.io>
>> interface and administration features. Apparently we were not the only
>> dev community who resisted change. To our surprise LF gave us several
>> years of reprieve by instead handing the subdomain and server-side 
>> data
>> to the non-profit OSUOSL lab who instead operated mailman2 for the 
>> past
>> ~4 years.
>> 
>> OSUOSL has for decades been well known for providing server
>> infrastructure for Linux and Open Source development so they were a 
>> good
>> fit. This however became an added maintenance burden for the small
>> non-profit with limited resources. Several members of the Bitcoin dev
>> community contributed funding to the lab in support of their Open 
>> Source
>> development infrastructure goals. But throwing money at the problem
>> isn't going to fix the ongoing maintenance burden created by 
>> antiquated
>> limitations of mailman2.
>> 
>> Permalinks
>> ==========
>> 
>> Linux Foundation has either offered or agreed to maintain archive
>> permalinks so that content of historic importance is not lost.
>> Fortunately for us while lists.linuxfoundation.org
>> <http://lists.linuxfoundation.org> mailman will go down, they have
>> agreed the read-only pipermail archives will remain online. So all old
>> URLs will continue to remain valid. However, the moderators strongly
>> advise that the community supplements with public-inbox instances to
>> have canonical archive urls that are separate from any particular 
>> email
>> software host.
>> 
>> Public-Inbox
>> ============
>> 
>> https://public-inbox.org/README.html 
>> <https://public-inbox.org/README.html>
>> 
>> "Public Inbox" decentralized archiving - no matter what mailing list
>> server solution is used, anyone can use git to maintain their own
>> mailing list archive and make it available to read on the web.
>> 
>> Public Inbox is a tool that you can run yourself. You can transform 
>> your
>> mbox file and it makes it browsable and viewable online. It commits
>> every post to a git repository. It's kind of like a decentralized mail
>> archiving tool. Anyone can publish the mail archive to any web server
>> they wish.
>> 
>> We should try to have one or more canonical archives that are served
>> using public-inbox. But it doesn't matter if these are lost because
>> anyone else can archive the mailing list in the same way and 
>> re-publish
>> the archives.
>> 
>> These git commits can also be stamped using opentimestamps, inserting
>> their hashes into the bitcoin blockchain.
>> 
>> LKML mailing list readers often use public-inbox's web interface, and
>> they use the reply-to headers to populate their mail client and reply 
>> to
>> threads of interest. This allows their reply to be properly threaded
>> even if they were not a previous subscriber to that mailing list to
>> receive the headers.
>> 
>> public-inbox makes it so that it doesn't really matter where the list 
>> is
>> hosted, as pertaining to reading the mailing list. There is still a
>> disruption if the mailing list goes away, but the archives live on and
>> then people can post elsewhere. The archive gets disconnected from the
>> mailing list host in terms of posting. We could have a few canonical
>> URLs for the hosts, separate from the mailing list server.
>> 
>> mailman problems
>> ================
>> 
>> Over the years we have identified a number of problems with mailman2
>> especially as it pertains to content moderation. There are presently a
>> handful of different moderators, but mailman2 only has a single 
>> password
>> for logging into the email management interface. There are no 
>> moderator
>> audit logs to see which user (there is no concept of different users)
>> acted on an email. There is no way to mark an email as being
>> investigated by one or more of the moderators. Sometimes, while
>> investigating the veracity of an email, another moderator would come 
>> in
>> and approve a suspect email by accident.
>> 
>> Anti spam has been an issue for the moderators. It's relentless. 
>> Without
>> access to the underlying server, it has been difficult to fight spam.
>> There is some support for filters in mailman2 but it's not great.
>> 
>> 100% active moderation and approval of every email is unsustainable 
>> for
>> volunteer moderators. A system that requires moderation is a heavy
>> burden on the moderators and it slows down overall communication and
>> productivity. There's lots of problems with this. Also, moderators can
>> be blamed when they are merely slow while they are not actually 
>> censoring.
>> 
>> Rejection emails can optionally be sent to
>> bitcoin-dev-moderation@lists.ozlabs.org
>> <mailto:bitcoin-dev-moderation@lists.ozlabs.org> but this is an option
>> that a moderator has to remember to type in each time.
>> 
>> Not to mention numerous bugs and vulnerabilities that have accumulated
>> over the years for relatively unmaintained software. (Not disclosed 
>> here)
>> 
>> Requirements and considerations
>> ===============================
>> 
>> Looking towards the future, there are a number of properties that seem
>> to be important for the bitcoin-dev mailing list community. First, it 
>> is
>> important that backups of the entire archive should be easy for the
>> public to copy or verify so that the system can be brought up 
>> elsewhere
>> if necessary.
>> 
>> Second, there seems to be demand for both an email threading interface
>> (using mailing list software) as well as web-accessible interfaces 
>> (such
>> as forum software). There seems to be very few options that cater to
>> both email and web. Often, in forum software, email support is limited
>> to email notifications and there is limited if any support for email
>> user participation.
>> 
>> Third, there should be better support for moderator tools and 
>> management
>> of the mailing list. See above for complaints about problems with the
>> mailman2 system.
>> 
>> Burdens of running your own mailing list and email server
>> =========================================================
>> 
>> If you have never operated your own MTA you have no idea how difficult
>> it is to keep secure and functional in the face of numerous challenges
>> to deliverability. Anti-spam filtering is essential to prevent
>> forwarding spam. The moment you forward even a single spam message you
>> run the risk of the server IP address being added to blacklists.
>> 
>> The problem of spam filtering is so bad that most IP addresses are
>> presumed guilty even if they have no prior spam history, such as if
>> their network or subnetwork had spam issues in the past.
>> 
>> Even if you put unlimited time into managing your own email server,
>> other people may not accept your email. Or you make one mistake, and
>> then you get into permanent blacklists and it's hard to remove. The 
>> spam
>> problem is so bad that most IPs are automatically on a
>> guilty-until-proven-innocent blacklist.
>> 
>> Often there is nothing you can do to get server IP addresses removed
>> from spam blacklists or from "bad reputation" lists.
>> 
>> Ironically, hashcash-style proof-of-work stamps to prevent spam are an
>> appealing solution but not widely used in this community. Or anywhere.
>> 
>> Infinite rejection or forwarding loops happen. They often need to be
>> detected through vigilance and require manual sysadmin intervention to
>> solve.
>> 
>> Bitcoin's dev lists being hosted alongside other Open Source projects
>> was previously protective. If that mailing list server became
>> blacklisted there were a lot of other people who would notice and
>> complain. If we run a Bitcoin-specific mail server we are on our own.
>> 100% of the administrative burden falls upon our own people. There is
>> also nothing we can do if some unknown admin decides they don't like 
>> us.
>> 
>> Options
>> =======
>> 
>> Web forums are an interesting option, but often don't have good email
>> user integration. At most you can usually hope for email notifications
>> and an ability to reply by email. It changes the model of the 
>> community
>> from push (email) to pull (logging into a forum to read). RSS feeds 
>> can
>> help a little bit.
>> 
>> Many other projects have moved from mailing lists to forums (eg
>> https://discuss.python.org/ <https://discuss.python.org/> - see
>> https://lwn.net/Articles/901744/ <https://lwn.net/Articles/901744/> ; 
>> or
>> https://ethresear.ch/ <https://ethresear.ch/>), which seem easier to
>> maintain and moderate, and can have lots of advanced features beyond
>> plaintext, maybe-threading and maybe-HTML-markup.
>> 
>> Who would host the forum? Would there be agreement around which forum
>> software to use or which forum host? What about bitcointalk.org
>> <http://bitcointalk.org> or delvingbitcoin.org
>> <http://delvingbitcoin.org>? There are many options available. Maybe
>> what we actually want isn't so much a discussion forum, as an 'arxiv 
>> of
>> our own' where anons can post BIP drafts and the like?
>> 
>> Given the problems with mailman2, and the decline of email communities
>> in general, it seems that moving to mailman3 would not be a viable
>> long-term option. This leaves us with Google Groups or groups.io
>> <http://groups.io> as two remaining options.
>> 
>> groups.io <http://groups.io> is an interesting option: they are a paid
>> service that implements email communities along with online web forum
>> support. However, their public changelog indicates it has been a few
>> years since their last public change. They might be a smaller company
>> and it is unclear how long they will be around or if this would be the
>> right fit for hosting sometimes contentious bitcoin development
>> discussions...
>> 
>> Google Groups is another interesting option, and comes with different
>> tradeoffs. It's the lowest effort to maintain option, and has both an
>> email interface and web forum interface. Users can choose which mode
>> they want to interact with.
>> 
>> For the Google Groups web interface, you can use it with a non-gmail
>> account, but you must create a Google Account which is free to do. it
>> does not require any personal information to do so. This also allows 
>> you
>> to add 2FA. Non-gmail non-google users are able to subscribe and post
>> email from their non-gmail non-google email accounts. Tor seems to 
>> work
>> for the web interface.
>> 
>> Will Google shut it down, will they cut us off, will they shut down
>> non-google users? The same problem exists with other third-party 
>> hosts.
>> 
>> The moderation capabilities for Google Groups and groups.io
>> <http://groups.io> seem to be comparable. It seems more likely that
>> Google Groups will be able to handle email delivery issues far better
>> than a small resource-constrained operation like groups.io
>> <http://groups.io>. ((During feedback for this draft, luke-jr 
>> indicates
>> that Google Workspaces has been known to use blacklisted IPs for
>> business email!))
>> 
>> On the other hand, groups.io <http://groups.io> is a paid service and
>> you get what you pay for... hopefully?
>> 
>> Finally, another option is to do literally nothing. It's less work
>> overall. Users can switch to forums or other websites, or private
>> one-on-one communication. It would remove a point of 
>> semi-centralization
>> from the bitcoin ecosystem. It would hasten ossification, but on the
>> other hand it would hasten ossification and this could be a negative
>> too. Moderators would be less of a target.
>> 
>> Unfortunately, by doing nothing, there would be no more widely used
>> group email communication system between bitcoin developers. 
>> Developers
>> become less coordinated, mayhem and chaos as people go to different
>> communication platforms, a divided community is more vulnerable, etc.
>> BIP1 and BIP2 would need to be revised for other venues.
>> 
>> The main categories of what to move to are: web forums, mailing lists,
>> and hybrids of those two options. Most everything is either 
>> self-hosted
>> or you pay someone else to host it. It's kind of the same problem
>> though. It largely depends on how good is the software and 
>> unfortunately
>> running your own MTA that forwards mail is not a good option.
>> 
>> Going forward
>> ===========
>> 
>> We'd like to invite feedback and proposals from the community, and see
>> what options are available. One potential option is a migration to
>> Google Groups, but we're open to ideas at this point. We apologize for
>> any inconvenience this disruption has caused.
>> 
>> Bitcoin-dev mailing list moderation team
>> 
>> Bryan Bishop
>> Ruben Somsen
>> Warren Togami
>> various others.
>> 
>> --
>> - Bryan
>> https://twitter.com/kanzure <https://twitter.com/kanzure>
> 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 18888 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Anthony Towns]

On Tue, Nov 07, 2023 at 09:37:22AM -0600, Bryan Bishop via bitcoin-dev wrote:
> Web forums are an interesting option, but often don't have good email user
> integration.

> What about bitcointalk.org or delvingbitcoin.org?

delvingbitcoin.org is something I setup; it's a self-hosted discourse
instance. (You don't have to self-host discourse, but not doing so limits
the number of admins/moderators, the plugins you can use, and the APIs you
can access)

For what it's worth, I think (discourse) forums have significant
advantages over email for technical discussion:

 * much better markup: you can write LaTeX for doing maths, you
   can have graphviz or mermaid diagrams generated directly from text,
   you can do formatting without having to worry about HTML email.
   because that's done direct from markup, you can also quote such
   things in replies, or easily create a modified equation/diagram
   if desired, things that are much harder if equations/diagrams are
   image/pdf attachments.

 * consistent threading/quoting: you don't have to rely on email clients
   to get threading/quoting correct in order to link replies with the
   original message

 * having topics/replies, rather than everything being an individual
   email, tends to make it easier to avoid being distracted by followups
   to a topic you're not interested in.

 * you can do reactions (heart / thumbs up / etc) instead of "me too"
   posts, minimising the impact of low-content responses on readers,
   without doing away with those responses entirely.

 * after the fact moderation: with mailing lists, moderation can only
   be a choice between "send this post to every subscriber" or not,
   and the choice obviously has to be made before anyone sees the posts;
   forums allow off-topic/unconstructive posts to be removed or edited.

Compared to mailing-lists-as-a-service, a self-hosted forum has a few
other possible benefits:

 * it's easier to setup areas for additional topics, without worrying
   you're going to be forced into an arbitrarily higher pricing tier

 * you can setup spaces for private working groups. (and those groups can
   make their internal discussions public after the fact, if desired)

 * you can use plugin interfaces/APIs to link up with external resources

There are a few disadvantages too:

 * discourse isn't lightweight -- you need a whole bunch of infrastructure
   to go from the markdown posts to the actual rendered posts/comments;
   so backups of just the markdown text isn't really "complete"

 * discourse is quite actively developed -- so it could be possible
   that posts that use particular features/plugins (eg to generate
   diagrams) will go stale eventually as the software changes, and stop
   being rendered correctly

 * discourse gathers a moderate amount of non-public/potentially private
   data (eg email addresses, passwords, IP addresses, login times) that
   may make backups and admin access sensitive (which is why there's a
   git archive generated by a bot for delvingbitcoin, rather than raw
   database dumps)

There are quite a few open source projects using discourse instances, eg:

  Python: https://discuss.python.org/
  Ruby on Rails: https://discuss.rubyonrails.org/
  LLVM: https://discourse.llvm.org/
  Jupyter: https://discourse.jupyter.org/
  Fedora: https://discussion.fedoraproject.org/
  Ubuntu: https://discourse.ubuntu.com/
  Haskell: https://discourse.haskell.org/

There's also various crypto projects using it:

  Eth research: https://ethresear.ch/
  Chia: https://developers.chia.net/

There's a couple of LWN articles on Python's adoption of discourse
that I found interesting, fwiw:

  https://lwn.net/Articles/901744/  [2022-07-20]
  https://lwn.net/Articles/674271/  [2016-02-03]

I don't think this needs to be an "either-or" question -- better to
have technical discussions about bitcoin in many places and in many
formats, rather than just one -- but I thought I'd take the opportunity
to write out why I thought discourse was worth spending some time on in
this context.

Cheers,
aj

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

On Tue, Nov 07, 2023 at 09:37:22AM -0600, Bryan Bishop via bitcoin-dev wrote:
> Anti spam has been an issue for the moderators. It's relentless. Without
> access to the underlying server, it has been difficult to fight spam. There
> is some support for filters in mailman2 but it's not great.

Since this is a technical mailing list it would be fine to require people to
pay a non-refundable anti-spam fee, eg via lightning, to gain the ability to
send messages. While this would require some custom software, it's probably
even possible to implement this if a third party is used for hosting, provided
they have some kind of API.

-- 
https://petertodd.org 'peter'[:-1]@petertodd.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]

On Tue, Nov 07, 2023 at 11:03:30AM -0600, Ademan via bitcoin-dev wrote:
> Hi Bryan,
> 
> I don't really want my first (and last?) devlist message to be a fairly
> off-the-cuff post on this topic, but here we go anyway.
> 
> At the risk of sounding like a nostr evangelist (I promise I'm not), I want
> to suggest nostr as a potential replacement to the mailing list. A decent
> chunk of software would need to be written, but none of the alternatives
> seem particularly attractive to me. I particularly dislike the idea of
> locking into a single siloed forum service like the bitcointalk forums. I
> realize I may be in the minority of course.

Strong NACK on nostr. It's a badly designed, centralized, protocol that needs a
significant redesign to be usable. While off topic for this mailing list, some
of its many issues include:

* Reliance on single-key, cryptography that often results in people having
  their keys compromised. This is a serious problem in the context of
  bitcoin-dev, where faked messages published could easily have market-moving
  results.

* Inability to mirror relays: since nostr deliberately ignores the lessons of
  blockchains, there is no way to be sure that you have a complete set of
  messages from a given person, for a given topic, etc.

* Highly centralized design: since mirroring relays isn't reliable, in reality
  nostr operates in a highly centralized fashion, dependent on a tiny number of
  relays that can't be easily replaced if taken down.

-- 
https://petertodd.org 'peter'[:-1]@petertodd.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1374 bytes --]

On Tue, Nov 07, 2023 at 11:41:59AM -0800, Christopher Allen via bitcoin-dev wrote:
> As Bitcoin-Core already uses GitHub, another possibility is to use the new
> GitHub discussions feature. We increasingly have been using this at
> Blockchain Commons as everyone is using already using GitHub. We have also
> created some GitHub actions to backup discussions so that GitHub will not
> be a central point of failure -should be possible to create a static page
> archive using GitHub pages (but have not had budget for that).
> 
> For instance, here is the GitHub discussion area for wallet developers
> working together on Bitcoin wallet interoperability specifications:
> https://github.com/BlockchainCommons/Gordian-Developer-Community

Strong NACK.

bitcoin-dev should be independent of Bitcoin Core.

Also, a very useful thing that a mailing list does that GitHub does not is
cryptographic signatures, both obvious like PGP, and less obvious like DKIM. We
should not be moving even more discussion to mediums where authors aren't
properly signing their messages.

The user experience of GitHub and similar web forums is poor too. It's much
nicer to be able to reply to messages offline, asyncronously, regardless of
whether or not you happen to have a good internet connection at the time.

-- 
https://petertodd.org 'peter'[:-1]@petertodd.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Ryan Breen]

[-- Attachment #1: Type: text/plain, Size: 1735 bytes --]

I think GitHub Discussions is a great idea. If we are considering proprietary options like Google Groups, then we should definitely consider Discussions.

1. Guaranteed that nearly everyone participating here already has a GH account.
2. Offers many moderation options.
3. Good formatting abilities(tables!).
4. Can @ people.
5. Ability to categorize, close, lock discussions, etc.
6. Many great potential opportunities for automation via Actions.
7. Comes with added benefits such as a wiki, issues, etc.

My one catch is that I do not know what kind of interactions you can have with Discussions via email. This seems to be an important feature for many. What level of email notifications can you receive? Can you respond via email?

> On Nov 7, 2023, at 3:16 PM, Christopher Allen via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
> 
> As Bitcoin-Core already uses GitHub, another possibility is to use the new GitHub discussions feature. We increasingly have been using this at Blockchain Commons as everyone is using already using GitHub. We have also created some GitHub actions to backup discussions so that GitHub will not be a central point of failure -should be possible to create a static page archive using GitHub pages (but have not had budget for that).
> 
> For instance, here is the GitHub discussion area for wallet developers working together on Bitcoin wallet interoperability specifications: 
> https://github.com/BlockchainCommons/Gordian-Developer-Community
> 
> — Christopher Allen
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 2453 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[ponymontana]

[-- Attachment #1.1: Type: text/plain, Size: 15606 bytes --]

Hi, 

This impellent deadline could be took with enthusiasm from people that are anxious to experiment with new protocols and platforms that can replicate mailing lists and offer, in theory, better solutions.
I think this enthusiasm is totally positive and I encourage them to work on that ideas.

But I also think that this mailing list fills a very particoular need of communication in the bitcoin space. 
The stream of ideas hosted here is strictly dependant on the form it assumes when formalized in the peculiar format of mails-threads. 
Migrating these technical discussions to a forum or a pseudo-group-chat wouldn't replace this mailing list, even if the moderators behind and most of the participants would be the same.
It would eventually be a new and unstable solution, with no-guarantee to preserve the same goals reached here.

Today exist a lot of places where people can exchange ideas about bitcoin;
if new platforms will emerge as better suited to hosts BIP drafts and technical discussions, people will move organically through them.
In my opinion, "finding a new platform" is only marginally correlated to our main topic here.


If our problem is helping decide the "future of bitcoin-dev mailing list", the only two solutions to me appear to tautologically be:

1) Give continuity to bitcoin-dev mailing list with a ready drop-in replacement. 

2) Don't give continuity the bitcoin-dev mailing list.


In the case 1) a solution could be find a new host for the mailing list and work around the problems exposed.

In the case 2) is possible to do nothing OR to propose a new solution as a sort of "spiritual continuation" of bitcoin-dev mailing list, and eventually see if people will converge on it.


Understanding all the difficulty behind the management of the bitcoin-dev mailing list, I think it has worked very well for many years, and I hope it will work for the years to come.
I also want to say thanks to all the people behind this mailing list for all your work and effort.


---PM

Il 7 novembre 2023 16:37:22 CET, Bryan Bishop via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> ha scritto:
>Hello,
>
>We would like to request community feedback and proposals on the future of
>the mailing list.
>
>Our current mailing list host, Linux Foundation, has indicated for years
>that they have wanted to stop hosting mailing lists, which would mean the
>bitcoin-dev mailing list would need to move somewhere else. We temporarily
>avoided that, but recently LF has informed a moderator that they will cease
>hosting any mailing lists later this year.
>
>In this email, we will go over some of the history, options, and invite
>discussion ahead of the cutoff. We have some ideas but want to solicit
>feedback and proposals.
>
>Background
>==========
>
>The bitcoin-dev mailing list was originally hosted on Sourceforge.net. The
>bitcoin development mailing list has been a source of proposals, analysis,
>and developer discussion for many years in the bitcoin community, with many
>thousands of participants. Later, the mailing list was migrated to the
>Linux Foundation, and after that OSUOSL began to help.
>
>Linux Foundation first asked us to move the mailing list in 2017. They
>internally attempted to migrate all LF mailing lists from mailman2 to
>mailman3, but ultimately gave up. There were reports of scalability issues
>with mailman3 for large email communities. Ours definitely qualifies as..
>large.
>
>2019 migration plan: LF was to turn off mailman and all lists would migrate
>to the paid service provider groups.io. Back then we were given accounts to
>try the groups.io interface and administration features. Apparently we were
>not the only dev community who resisted change. To our surprise LF gave us
>several years of reprieve by instead handing the subdomain and server-side
>data to the non-profit OSUOSL lab who instead operated mailman2 for the
>past ~4 years.
>
>OSUOSL has for decades been well known for providing server infrastructure
>for Linux and Open Source development so they were a good fit. This however
>became an added maintenance burden for the small non-profit with limited
>resources. Several members of the Bitcoin dev community contributed funding
>to the lab in support of their Open Source development infrastructure
>goals. But throwing money at the problem isn’t going to fix the ongoing
>maintenance burden created by antiquated limitations of mailman2.
>
>Permalinks
>==========
>
>Linux Foundation has either offered or agreed to maintain archive
>permalinks so that content of historic importance is not lost. Fortunately
>for us while lists.linuxfoundation.org mailman will go down, they have
>agreed the read-only pipermail archives will remain online. So all old URLs
>will continue to remain valid. However, the moderators strongly advise that
>the community supplements with public-inbox instances to have canonical
>archive urls that are separate from any particular email software host.
>
>Public-Inbox
>============
>
>https://public-inbox.org/README.html
>
>“Public Inbox” decentralized archiving - no matter what mailing list server
>solution is used, anyone can use git to maintain their own mailing list
>archive and make it available to read on the web.
>
>Public Inbox is a tool that you can run yourself. You can transform your
>mbox file and it makes it browsable and viewable online. It commits every
>post to a git repository. It's kind of like a decentralized mail archiving
>tool. Anyone can publish the mail archive to any web server they wish.
>
>We should try to have one or more canonical archives that are served using
>public-inbox. But it doesn't matter if these are lost because anyone else
>can archive the mailing list in the same way and re-publish the archives.
>
>These git commits can also be stamped using opentimestamps, inserting their
>hashes into the bitcoin blockchain.
>
>LKML mailing list readers often use public-inbox's web interface, and they
>use the reply-to headers to populate their mail client and reply to threads
>of interest. This allows their reply to be properly threaded even if they
>were not a previous subscriber to that mailing list to receive the headers.
>
>public-inbox makes it so that it doesn't really matter where the list is
>hosted, as pertaining to reading the mailing list. There is still a
>disruption if the mailing list goes away, but the archives live on and then
>people can post elsewhere. The archive gets disconnected from the mailing
>list host in terms of posting. We could have a few canonical URLs for the
>hosts, separate from the mailing list server.
>
>mailman problems
>================
>
>Over the years we have identified a number of problems with mailman2
>especially as it pertains to content moderation. There are presently a
>handful of different moderators, but mailman2 only has a single password
>for logging into the email management interface. There are no moderator
>audit logs to see which user (there is no concept of different users) acted
>on an email. There is no way to mark an email as being investigated by one
>or more of the moderators. Sometimes, while investigating the veracity of
>an email, another moderator would come in and approve a suspect email by
>accident.
>
>Anti spam has been an issue for the moderators. It's relentless. Without
>access to the underlying server, it has been difficult to fight spam. There
>is some support for filters in mailman2 but it's not great.
>
>100% active moderation and approval of every email is unsustainable for
>volunteer moderators. A system that requires moderation is a heavy burden
>on the moderators and it slows down overall communication and productivity.
>There's lots of problems with this. Also, moderators can be blamed when
>they are merely slow while they are not actually censoring.
>
>Rejection emails can optionally be sent to
>bitcoin-dev-moderation@lists.ozlabs.org but this is an option that a
>moderator has to remember to type in each time.
>
>Not to mention numerous bugs and vulnerabilities that have accumulated over
>the years for relatively unmaintained software. (Not disclosed here)
>
>Requirements and considerations
>===============================
>
>Looking towards the future, there are a number of properties that seem to
>be important for the bitcoin-dev mailing list community. First, it is
>important that backups of the entire archive should be easy for the public
>to copy or verify so that the system can be brought up elsewhere if
>necessary.
>
>Second, there seems to be demand for both an email threading interface
>(using mailing list software) as well as web-accessible interfaces (such as
>forum software). There seems to be very few options that cater to both
>email and web. Often, in forum software, email support is limited to email
>notifications and there is limited if any support for email user
>participation.
>
>Third, there should be better support for moderator tools and management of
>the mailing list. See above for complaints about problems with the mailman2
>system.
>
>Burdens of running your own mailing list and email server
>=========================================================
>
>If you have never operated your own MTA you have no idea how difficult it
>is to keep secure and functional in the face of numerous challenges to
>deliverability. Anti-spam filtering is essential to prevent forwarding
>spam. The moment you forward even a single spam message you run the risk of
>the server IP address being added to blacklists.
>
>The problem of spam filtering is so bad that most IP addresses are presumed
>guilty even if they have no prior spam history, such as if their network or
>subnetwork had spam issues in the past.
>
>Even if you put unlimited time into managing your own email server, other
>people may not accept your email. Or you make one mistake, and then you get
>into permanent blacklists and it's hard to remove. The spam problem is so
>bad that most IPs are automatically on a guilty-until-proven-innocent
>blacklist.
>
>Often there is nothing you can do to get server IP addresses removed from
>spam blacklists or from "bad reputation" lists.
>
>Ironically, hashcash-style proof-of-work stamps to prevent spam are an
>appealing solution but not widely used in this community. Or anywhere.
>
>Infinite rejection or forwarding loops happen. They often need to be
>detected through vigilance and require manual sysadmin intervention to
>solve.
>
>Bitcoin's dev lists being hosted alongside other Open Source projects was
>previously protective. If that mailing list server became blacklisted there
>were a lot of other people who would notice and complain. If we run a
>Bitcoin-specific mail server we are on our own. 100% of the administrative
>burden falls upon our own people. There is also nothing we can do if some
>unknown admin decides they don't like us.
>
>Options
>=======
>
>Web forums are an interesting option, but often don't have good email user
>integration. At most you can usually hope for email notifications and an
>ability to reply by email. It changes the model of the community from push
>(email) to pull (logging into a forum to read). RSS feeds can help a little
>bit.
>
>Many other projects have moved from mailing lists to forums (eg
>https://discuss.python.org/ – see https://lwn.net/Articles/901744/ ; or
>https://ethresear.ch/), which seem easier to maintain and moderate, and can
>have lots of advanced features beyond plaintext, maybe-threading and
>maybe-HTML-markup.
>
>Who would host the forum? Would there be agreement around which forum
>software to use or which forum host? What about bitcointalk.org or
>delvingbitcoin.org? There are many options available. Maybe what we
>actually want isn’t so much a discussion forum, as an 'arxiv of our own'
>where anons can post BIP drafts and the like?
>
>Given the problems with mailman2, and the decline of email communities in
>general, it seems that moving to mailman3 would not be a viable long-term
>option. This leaves us with Google Groups or groups.io as two remaining
>options.
>
>groups.io is an interesting option: they are a paid service that implements
>email communities along with online web forum support. However, their
>public changelog indicates it has been a few years since their last public
>change. They might be a smaller company and it is unclear how long they
>will be around or if this would be the right fit for hosting sometimes
>contentious bitcoin development discussions...
>
>Google Groups is another interesting option, and comes with different
>tradeoffs. It's the lowest effort to maintain option, and has both an email
>interface and web forum interface. Users can choose which mode they want to
>interact with.
>
>For the Google Groups web interface, you can use it with a non-gmail
>account, but you must create a Google Account which is free to do. it does
>not require any personal information to do so. This also allows you to add
>2FA. Non-gmail non-google users are able to subscribe and post email from
>their non-gmail non-google email accounts. Tor seems to work for the web
>interface.
>
>Will Google shut it down, will they cut us off, will they shut down
>non-google users? The same problem exists with other third-party hosts.
>
>The moderation capabilities for Google Groups and groups.io seem to be
>comparable. It seems more likely that Google Groups will be able to handle
>email delivery issues far better than a small resource-constrained
>operation like groups.io. ((During feedback for this draft, luke-jr
>indicates that Google Workspaces has been known to use blacklisted IPs for
>business email!))
>
>On the other hand, groups.io is a paid service and you get what you pay
>for... hopefully?
>
>Finally, another option is to do literally nothing. It's less work overall.
>Users can switch to forums or other websites, or private one-on-one
>communication. It would remove a point of semi-centralization from the
>bitcoin ecosystem. It would hasten ossification, but on the other hand it
>would hasten ossification and this could be a negative too. Moderators
>would be less of a target.
>
>Unfortunately, by doing nothing, there would be no more widely used group
>email communication system between bitcoin developers. Developers become
>less coordinated, mayhem and chaos as people go to different communication
>platforms, a divided community is more vulnerable, etc. BIP1 and BIP2 would
>need to be revised for other venues.
>
>The main categories of what to move to are: web forums, mailing lists, and
>hybrids of those two options. Most everything is either self-hosted or you
>pay someone else to host it. It's kind of the same problem though. It
>largely depends on how good is the software and unfortunately running your
>own MTA that forwards mail is not a good option.
>
>Going forward
>===========
>
>We'd like to invite feedback and proposals from the community, and see what
>options are available. One potential option is a migration to Google
>Groups, but we're open to ideas at this point. We apologize for any
>inconvenience this disruption has caused.
>
>
>Bitcoin-dev mailing list moderation team
>
>Bryan Bishop
>Ruben Somsen
>Warren Togami
>various others.
>
>-- 
>- Bryan
>https://twitter.com/kanzure

[-- Attachment #1.2: Type: text/html, Size: 16359 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Keagan McClelland]

[-- Attachment #1: Type: text/plain, Size: 2970 bytes --]

I also think that good archives are extremely important. Far more important
than being a medium of discussion is capturing all of that discussion for
posterity. An unbelievable amount of knowledge capital has been built up in
the mailing list over the years and given that Bitcoin is a system that
needs to survive complete turnover in its contributor base, it's of extreme
importance that we have a system that can capture the archive.

While Nostr might be good towards the end of being very resilient it isn't
mature enough to have good UX's built up around it wherein people with a
variety of backgrounds can engage it. Personally, I think the email UX
leaves a lot to be desired but at least it's accessible to a lot of people.
I don't think I can say the same for Nostr yet.

I won't opine much further on the solution but I think the properties we
need to solve for are:

1. Archive is effectively permanent
2. Accessible to a wide audience
3. Data format is not proprietary and isn't tied to the success or failure
of a particular organization

In principle I think that Nostr can offer a lot in the long term towards
this goal, but it isn't really an immediate solution to this problem.

Keags

On Tue, Nov 7, 2023 at 12:07 PM David A. Harding via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On 2023-11-07 05:37, Bryan Bishop via bitcoin-dev wrote:
> > What about [...] delvingbitcoin.org?
>
> I'm only willing to consider discussion groups that provide good
> archives, so I think it's worth noting that James O'Beirne has written
> code[1] and is currently maintaining a git repo[2] with a backup of
> Delving Bitcoin discussion.  See his post[3] for additional details.
>
> In addition to providing an archive, I currently find it to be nice way
> to quickly skim all posts made to the forum since I last checked (plus I
> see edits)[4]:
>
> $ cd delving-bitcoin-archive/
> $ git pull
> $ git log -p archive/rendered-topics/
>
> I think some technical discussions were already migrating to Delving
> Bitcoin before the shutdown notice and I expect more discussions to move
> there in the future even if the current mailing list is relocated to a
> new platform.  Knowing that discussions are archived in a way that I can
> easily replicate was key to me feeling comfortable putting significant
> time into reading and writing posts on Delving Bitcoin, so I wanted to
> share that information here.
>
> -Dave
>
> [1] https://github.com/jamesob/discourse-archive
> [2] https://github.com/jamesob/delving-bitcoin-archive
> [3] https://delvingbitcoin.org/t/public-archive-for-delving-bitcoin/87/6
> [4] Plus every commit makes me laugh.  James O'Beirne's commit robot is
> called "jamesobot"
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 4109 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Tao Effect]

[-- Attachment #1: Type: text/plain, Size: 1453 bytes --]

Hi, I also have faced this same problem, and here’s my solution to it:

Use the latest version of https://www.simplemachines.org/ .

This is the same forum software that powered Bitcointalk, Silk Road, etc.

It has many advantages over every other platform out there:

1. It has great anti-spam prevention tools.
2. It gives you the tools you’ve requested with respect to moderation (individual moderation accounts with customizable permissions).
3. It is simple to use.
4. It is pretty and well designed, and allows you to organize threads and forums really well (unlike Discourse).
5. It doesn’t make unnecessary use of JavaScript (unlike Discourse), and therefore works well with all search engines (present and future).
6. You can self-host it, and migrate it to another server if needed, allowing the community to maintain full control over the data (unlike Microsoft/Github).
7. It supports great search functionality.
8. It is open source, and has many community plugins.
9. It runs anywhere PHP runs.
10. It is fast.
11. It has a well established history of powering Bitcoin communities. It has been with us from Day 1.

After using phpBB for years, researching other forums software, I switched to SMF, and stuck with it. I’m happy I did so. I recommend it.

Kind regards,
Greg Slepak

> On Nov 7, 2023, at 7:37 AM, Bryan Bishop via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
> [moving]


[-- Attachment #2: Type: text/html, Size: 2074 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Ademan]

[-- Attachment #1: Type: text/plain, Size: 24171 bytes --]

Hi Andrew,
    Thanks for the thoughtful response. I don't know that you'll find my
responses satisfactory (particularly around moderation), but there are at
least solutions to the objections. Except of course the timeline, which I
got wrong ;-) and means this would be half-baked at best by the time it's
needed.

On Tue, Nov 7, 2023 at 1:06 PM Andrew Chow via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi Dan,
>
> I don't think nostr would be a suitable replacement for the mailing
> list, although this opinion is biased by the fact that I do not use
> nostr and find it to be uninteresting.
>

I felt that way for a long time. I still have a number of reservations
about it technically, but I'm increasingly impressed, though not for
reasons relevant to the discussion.


>  From my limited understanding of how nostr works, it's not clear to me
> how a distributed system that uses message broadcast would work in the
> same way as a mailing list. How would people "subscribe"?


This is already accomplished by existing apps in various ways. There are
multiple options, from using labels, or topic tags (
https://github.com/nostr-protocol/nips/tree/master#standardized-tags ) (my
preference) or the moderated community approach (see below on moderation)


> How would
> archives be searched or otherwise be available to people who are not on
> nostr?


Dumb web portals with a familiar interface, existing nostr apps are
available as web clients in the interim.


> How do you distinguish and filter between legitimate dev posts
> intended for discussion and random crap and shitposting as shows up on
> social media?
>

I lean strongly towards the topic tag or label approach to constructing the
list, which means anyone can post "to the list". Moderation would be
applied client side like all nostr clients already do. This is where PoW (
https://github.com/nostr-protocol/nips/blob/master/13.md ) , and/or
web-of-trust and labeling (
https://github.com/nostr-protocol/nips/blob/master/32.md ) come in.

Moderation comes "for free" in the moderated communities model (
https://github.com/nostr-protocol/nips/blob/master/72.md ) Note that this
approach creates exactly the same moderation burden as the list team
already shoulders, and is overall less desirable imho, but it provides the
same level of control the current list does.


> I also don't think that long form text on nostr (or any similar
> platform) can sufficiently replace email. None of these things seem to
> contain a way to have a separate subject line as email does.


"Subject tag" ( https://github.com/nostr-protocol/nips/blob/master/14.md )


> Subjects
> are immensely important for me as it provides a quick and easy way to
> filter out things I don't care about reading. I don't want to have read
> something in before I can decide that I don't care about reading it.
>
> In general, I strongly prefer email, or a platform that has email as a
> first class user interface, over platforms such as nostr, matrix, or web
> forums. Email is universal - everyone has one and everyone knows how it
> works. It dramatically lowers the barrier of entry.


I think you'd be surprised just how low the barrier to entry in nostr is.
You can navigate to the website of any number of nostr apps and click
"generate key", but your point about the universality of email is well
taken. I think an email bridge would be an essential part of this system.
An email bridge would only be less than first-class because email is not as
rich as nostr* , from the email-user's perspective it could be no worse
than using an existing list server. FWIW I also strongly prefer email over
any web forum.

* Unless you want to go wild with non-standard headers


> Having to make an
> account somewhere or download some specific client in order to
> participate will simply result in only the most dedicated participating.
>

I actually think the barrier to entry is exceedingly low, which is part of
why we have concern about the spam problem.


> Development in open source must be an open process and the barriers to
> entry should be low.
>
> Lastly, IIRC the plan is to shut down the list by the end of this year.
> Any solution that requires custom software and bridges to be created are
> not going to be viable in this time frame.
>

I have to admit I misread the OP as shutdown at the end of 2024, not 2023,
but you're right, barely 2 months is a very tight schedule. Truly
addressing the needs of the list in that time frame is unlikely to be
possible. FWIW a workable "close enough" is basically possible today, but
that is probably unsatisfying.

Certainly if I take the lead, 2024-01-01 is an impossible timeline
(especially given my other obligations), but if there is even moderate
interest (part of why I bothered to share my ramblings), I could start
working on an MVP of the bridge, and a web client for evaluation down the
road.

Cheers,
Dan


>
> Andrew
>
> On 11/07/2023 12:03 PM, Ademan via bitcoin-dev wrote:
> > Hi Bryan,
> >
> > I don't really want my first (and last?) devlist message to be a fairly
> > off-the-cuff post on this topic, but here we go anyway.
> >
> > At the risk of sounding like a nostr evangelist (I promise I'm not), I
> > want to suggest nostr as a potential replacement to the mailing list. A
> > decent chunk of software would need to be written, but none of the
> > alternatives seem particularly attractive to me. I particularly dislike
> > the idea of locking into a single siloed forum service like the
> > bitcointalk forums. I realize I may be in the minority of course.
> >
> >
> > Nostr enables the ML team to outsource all of its biggest burdens, if it
> > chooses:
> >
> > - mail server blocking is N/A to nostr
> >
> > - Hosting costs are completely outsourced unless the ML team chooses to
> > host a relay.
> >
> > - Archives and web portal access can be similarly outsourced because any
> > nostr archive is self-authenticating.
> >
> > - The ML team can also choose to completely outsource moderation, as
> > nostr is (more or less) permissionless by nature.
> >    I understand if there is a "blessed" communication system, the ML
> > team would probably prefer to keep it high quality. To that end there
> > are proposals for proof-of-work, and web of trust based blocklists for
> > nostr which are optional for end users. There are other options such as
> > the "moderated communities" proposal which would provide tighter control.
> >
> >
> > On the user side, the optional moderation is very attractive, allowing
> > controversial threads to exist and continue, without requiring everyone
> > to see them.
> >
> >
> > The following do not currently exist (to my knowledge) and would need to
> > be implemented to meet the ML's requirements:
> >
> > - an email gateway to satisfy the bulk of existing ML subscribers
> >    This reintroduces issues with mail server blocking of course.
> > - a long-form oriented nostr client (current plain text clients could be
> > used in the meantime)
> >
> > That admittedly is quite a lot of work, but the second item can be
> > deferred, and the first is not particularly technically challenging, the
> > complications are all on the administration side.
> >
> > I expect some reflexive NACKs based on the immaturity of the ecosystem
> > but if we have months to prepare, I believe the core requirements can be
> > solidly satisfied in time, the rest can be developed over time, and I
> > believe the advantages are worth careful consideration.
> >
> > Cheers,
> > Dan
> >
> > On Tue, Nov 7, 2023 at 9:38 AM Bryan Bishop via bitcoin-dev
> > <bitcoin-dev@lists.linuxfoundation.org
> > <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
> >
> >     Hello,
> >
> >     We would like to request community feedback and proposals on the
> >     future of the mailing list.
> >
> >     Our current mailing list host, Linux Foundation, has indicated for
> >     years that they have wanted to stop hosting mailing lists, which
> >     would mean the bitcoin-dev mailing list would need to move somewhere
> >     else. We temporarily avoided that, but recently LF has informed a
> >     moderator that they will cease hosting any mailing lists later this
> >     year.
> >
> >     In this email, we will go over some of the history, options, and
> >     invite discussion ahead of the cutoff. We have some ideas but want
> >     to solicit feedback and proposals.
> >
> >     Background
> >     ==========
> >
> >     The bitcoin-dev mailing list was originally hosted on
> >     Sourceforge.net. The bitcoin development mailing list has been a
> >     source of proposals, analysis, and developer discussion for many
> >     years in the bitcoin community, with many thousands of participants.
> >     Later, the mailing list was migrated to the Linux Foundation, and
> >     after that OSUOSL began to help.
> >
> >     Linux Foundation first asked us to move the mailing list in 2017.
> >     They internally attempted to migrate all LF mailing lists from
> >     mailman2 to mailman3, but ultimately gave up. There were reports of
> >     scalability issues with mailman3 for large email communities. Ours
> >     definitely qualifies as.. large.
> >
> >     2019 migration plan: LF was to turn off mailman and all lists would
> >     migrate to the paid service provider groups.io <http://groups.io>.
> >     Back then we were given accounts to try the groups.io
> >     <http://groups.io> interface and administration features. Apparently
> >     we were not the only dev community who resisted change. To our
> >     surprise LF gave us several years of reprieve by instead handing the
> >     subdomain and server-side data to the non-profit OSUOSL lab who
> >     instead operated mailman2 for the past ~4 years.
> >
> >     OSUOSL has for decades been well known for providing server
> >     infrastructure for Linux and Open Source development so they were a
> >     good fit. This however became an added maintenance burden for the
> >     small non-profit with limited resources. Several members of the
> >     Bitcoin dev community contributed funding to the lab in support of
> >     their Open Source development infrastructure goals. But throwing
> >     money at the problem isn’t going to fix the ongoing maintenance
> >     burden created by antiquated limitations of mailman2.
> >
> >     Permalinks
> >     ==========
> >
> >     Linux Foundation has either offered or agreed to maintain archive
> >     permalinks so that content of historic importance is not lost.
> >     Fortunately for us while lists.linuxfoundation.org
> >     <http://lists.linuxfoundation.org> mailman will go down, they have
> >     agreed the read-only pipermail archives will remain online. So all
> >     old URLs will continue to remain valid. However, the moderators
> >     strongly advise that the community supplements with public-inbox
> >     instances to have canonical archive urls that are separate from any
> >     particular email software host.
> >
> >     Public-Inbox
> >     ============
> >
> >     https://public-inbox.org/README.html
> >     <https://public-inbox.org/README.html>
> >
> >     “Public Inbox” decentralized archiving - no matter what mailing list
> >     server solution is used, anyone can use git to maintain their own
> >     mailing list archive and make it available to read on the web.
> >
> >     Public Inbox is a tool that you can run yourself. You can transform
> >     your mbox file and it makes it browsable and viewable online. It
> >     commits every post to a git repository. It's kind of like a
> >     decentralized mail archiving tool. Anyone can publish the mail
> >     archive to any web server they wish.
> >
> >     We should try to have one or more canonical archives that are served
> >     using public-inbox. But it doesn't matter if these are lost because
> >     anyone else can archive the mailing list in the same way and
> >     re-publish the archives.
> >
> >     These git commits can also be stamped using opentimestamps,
> >     inserting their hashes into the bitcoin blockchain.
> >
> >     LKML mailing list readers often use public-inbox's web interface,
> >     and they use the reply-to headers to populate their mail client and
> >     reply to threads of interest. This allows their reply to be properly
> >     threaded even if they were not a previous subscriber to that mailing
> >     list to receive the headers.
> >
> >     public-inbox makes it so that it doesn't really matter where the
> >     list is hosted, as pertaining to reading the mailing list. There is
> >     still a disruption if the mailing list goes away, but the archives
> >     live on and then people can post elsewhere. The archive gets
> >     disconnected from the mailing list host in terms of posting. We
> >     could have a few canonical URLs for the hosts, separate from the
> >     mailing list server.
> >
> >     mailman problems
> >     ================
> >
> >     Over the years we have identified a number of problems with mailman2
> >     especially as it pertains to content moderation. There are presently
> >     a handful of different moderators, but mailman2 only has a single
> >     password for logging into the email management interface. There are
> >     no moderator audit logs to see which user (there is no concept of
> >     different users) acted on an email. There is no way to mark an email
> >     as being investigated by one or more of the moderators. Sometimes,
> >     while investigating the veracity of an email, another moderator
> >     would come in and approve a suspect email by accident.
> >
> >     Anti spam has been an issue for the moderators. It's relentless.
> >     Without access to the underlying server, it has been difficult to
> >     fight spam. There is some support for filters in mailman2 but it's
> >     not great.
> >
> >     100% active moderation and approval of every email is unsustainable
> >     for volunteer moderators. A system that requires moderation is a
> >     heavy burden on the moderators and it slows down overall
> >     communication and productivity. There's lots of problems with this.
> >     Also, moderators can be blamed when they are merely slow while they
> >     are not actually censoring.
> >
> >     Rejection emails can optionally be sent to
> >     bitcoin-dev-moderation@lists.ozlabs.org
> >     <mailto:bitcoin-dev-moderation@lists.ozlabs.org> but this is an
> >     option that a moderator has to remember to type in each time.
> >
> >     Not to mention numerous bugs and vulnerabilities that have
> >     accumulated over the years for relatively unmaintained software.
> >     (Not disclosed here)
> >
> >     Requirements and considerations
> >     ===============================
> >
> >     Looking towards the future, there are a number of properties that
> >     seem to be important for the bitcoin-dev mailing list community.
> >     First, it is important that backups of the entire archive should be
> >     easy for the public to copy or verify so that the system can be
> >     brought up elsewhere if necessary.
> >
> >     Second, there seems to be demand for both an email threading
> >     interface (using mailing list software) as well as web-accessible
> >     interfaces (such as forum software). There seems to be very few
> >     options that cater to both email and web. Often, in forum software,
> >     email support is limited to email notifications and there is limited
> >     if any support for email user participation.
> >
> >     Third, there should be better support for moderator tools and
> >     management of the mailing list. See above for complaints about
> >     problems with the mailman2 system.
> >
> >     Burdens of running your own mailing list and email server
> >     =========================================================
> >
> >     If you have never operated your own MTA you have no idea how
> >     difficult it is to keep secure and functional in the face of
> >     numerous challenges to deliverability. Anti-spam filtering is
> >     essential to prevent forwarding spam. The moment you forward even a
> >     single spam message you run the risk of the server IP address being
> >     added to blacklists.
> >
> >     The problem of spam filtering is so bad that most IP addresses are
> >     presumed guilty even if they have no prior spam history, such as if
> >     their network or subnetwork had spam issues in the past.
> >
> >     Even if you put unlimited time into managing your own email server,
> >     other people may not accept your email. Or you make one mistake, and
> >     then you get into permanent blacklists and it's hard to remove. The
> >     spam problem is so bad that most IPs are automatically on a
> >     guilty-until-proven-innocent blacklist.
> >
> >     Often there is nothing you can do to get server IP addresses removed
> >     from spam blacklists or from "bad reputation" lists.
> >
> >     Ironically, hashcash-style proof-of-work stamps to prevent spam are
> >     an appealing solution but not widely used in this community. Or
> >     anywhere.
> >
> >     Infinite rejection or forwarding loops happen. They often need to be
> >     detected through vigilance and require manual sysadmin intervention
> >     to solve.
> >
> >     Bitcoin's dev lists being hosted alongside other Open Source
> >     projects was previously protective. If that mailing list server
> >     became blacklisted there were a lot of other people who would notice
> >     and complain. If we run a Bitcoin-specific mail server we are on our
> >     own. 100% of the administrative burden falls upon our own people.
> >     There is also nothing we can do if some unknown admin decides they
> >     don't like us.
> >
> >     Options
> >     =======
> >
> >     Web forums are an interesting option, but often don't have good
> >     email user integration. At most you can usually hope for email
> >     notifications and an ability to reply by email. It changes the model
> >     of the community from push (email) to pull (logging into a forum to
> >     read). RSS feeds can help a little bit.
> >
> >     Many other projects have moved from mailing lists to forums (eg
> >     https://discuss.python.org/ <https://discuss.python.org/> – see
> >     https://lwn.net/Articles/901744/ <https://lwn.net/Articles/901744/>
> >     ; or https://ethresear.ch/ <https://ethresear.ch/>), which seem
> >     easier to maintain and moderate, and can have lots of advanced
> >     features beyond plaintext, maybe-threading and maybe-HTML-markup.
> >
> >     Who would host the forum? Would there be agreement around which
> >     forum software to use or which forum host? What about
> >     bitcointalk.org <http://bitcointalk.org> or delvingbitcoin.org
> >     <http://delvingbitcoin.org>? There are many options available. Maybe
> >     what we actually want isn’t so much a discussion forum, as an 'arxiv
> >     of our own' where anons can post BIP drafts and the like?
> >
> >     Given the problems with mailman2, and the decline of email
> >     communities in general, it seems that moving to mailman3 would not
> >     be a viable long-term option. This leaves us with Google Groups or
> >     groups.io <http://groups.io> as two remaining options.
> >
> >     groups.io <http://groups.io> is an interesting option: they are a
> >     paid service that implements email communities along with online web
> >     forum support. However, their public changelog indicates it has been
> >     a few years since their last public change. They might be a smaller
> >     company and it is unclear how long they will be around or if this
> >     would be the right fit for hosting sometimes contentious bitcoin
> >     development discussions...
> >
> >     Google Groups is another interesting option, and comes with
> >     different tradeoffs. It's the lowest effort to maintain option, and
> >     has both an email interface and web forum interface. Users can
> >     choose which mode they want to interact with.
> >
> >     For the Google Groups web interface, you can use it with a non-gmail
> >     account, but you must create a Google Account which is free to do.
> >     it does not require any personal information to do so. This also
> >     allows you to add 2FA. Non-gmail non-google users are able to
> >     subscribe and post email from their non-gmail non-google email
> >     accounts. Tor seems to work for the web interface.
> >
> >     Will Google shut it down, will they cut us off, will they shut down
> >     non-google users? The same problem exists with other third-party
> hosts.
> >
> >     The moderation capabilities for Google Groups and groups.io
> >     <http://groups.io> seem to be comparable. It seems more likely that
> >     Google Groups will be able to handle email delivery issues far
> >     better than a small resource-constrained operation like groups.io
> >     <http://groups.io>. ((During feedback for this draft, luke-jr
> >     indicates that Google Workspaces has been known to use blacklisted
> >     IPs for business email!))
> >
> >     On the other hand, groups.io <http://groups.io> is a paid service
> >     and you get what you pay for... hopefully?
> >
> >     Finally, another option is to do literally nothing. It's less work
> >     overall. Users can switch to forums or other websites, or private
> >     one-on-one communication. It would remove a point of
> >     semi-centralization from the bitcoin ecosystem. It would hasten
> >     ossification, but on the other hand it would hasten ossification and
> >     this could be a negative too. Moderators would be less of a target.
> >
> >     Unfortunately, by doing nothing, there would be no more widely used
> >     group email communication system between bitcoin developers.
> >     Developers become less coordinated, mayhem and chaos as people go to
> >     different communication platforms, a divided community is more
> >     vulnerable, etc. BIP1 and BIP2 would need to be revised for other
> >     venues.
> >
> >     The main categories of what to move to are: web forums, mailing
> >     lists, and hybrids of those two options. Most everything is either
> >     self-hosted or you pay someone else to host it. It's kind of the
> >     same problem though. It largely depends on how good is the software
> >     and unfortunately running your own MTA that forwards mail is not a
> >     good option.
> >
> >     Going forward
> >     ===========
> >
> >     We'd like to invite feedback and proposals from the community, and
> >     see what options are available. One potential option is a migration
> >     to Google Groups, but we're open to ideas at this point. We
> >     apologize for any inconvenience this disruption has caused.
> >
> >
> >     Bitcoin-dev mailing list moderation team
> >
> >     Bryan Bishop
> >     Ruben Somsen
> >     Warren Togami
> >     various others.
> >
> >     --
> >     - Bryan
> >     https://twitter.com/kanzure <https://twitter.com/kanzure>
> >     _______________________________________________
> >     bitcoin-dev mailing list
> >     bitcoin-dev@lists.linuxfoundation.org
> >     <mailto:bitcoin-dev@lists.linuxfoundation.org>
> >     https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >     <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>
> >
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 32484 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[David A. Harding]

On 2023-11-07 05:37, Bryan Bishop via bitcoin-dev wrote:
> What about [...] delvingbitcoin.org?

I'm only willing to consider discussion groups that provide good 
archives, so I think it's worth noting that James O'Beirne has written 
code[1] and is currently maintaining a git repo[2] with a backup of 
Delving Bitcoin discussion.  See his post[3] for additional details.

In addition to providing an archive, I currently find it to be nice way 
to quickly skim all posts made to the forum since I last checked (plus I 
see edits)[4]:

$ cd delving-bitcoin-archive/
$ git pull
$ git log -p archive/rendered-topics/

I think some technical discussions were already migrating to Delving 
Bitcoin before the shutdown notice and I expect more discussions to move 
there in the future even if the current mailing list is relocated to a 
new platform.  Knowing that discussions are archived in a way that I can 
easily replicate was key to me feeling comfortable putting significant 
time into reading and writing posts on Delving Bitcoin, so I wanted to 
share that information here.

-Dave

[1] https://github.com/jamesob/discourse-archive
[2] https://github.com/jamesob/delving-bitcoin-archive
[3] https://delvingbitcoin.org/t/public-archive-for-delving-bitcoin/87/6
[4] Plus every commit makes me laugh.  James O'Beirne's commit robot is 
called "jamesobot"

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Christopher Allen]

[-- Attachment #1: Type: text/plain, Size: 669 bytes --]

As Bitcoin-Core already uses GitHub, another possibility is to use the new
GitHub discussions feature. We increasingly have been using this at
Blockchain Commons as everyone is using already using GitHub. We have also
created some GitHub actions to backup discussions so that GitHub will not
be a central point of failure -should be possible to create a static page
archive using GitHub pages (but have not had budget for that).

For instance, here is the GitHub discussion area for wallet developers
working together on Bitcoin wallet interoperability specifications:
https://github.com/BlockchainCommons/Gordian-Developer-Community

— Christopher Allen

[-- Attachment #2: Type: text/html, Size: 863 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Andrew Chow]

Hi Dan,

I don't think nostr would be a suitable replacement for the mailing 
list, although this opinion is biased by the fact that I do not use 
nostr and find it to be uninteresting.

 From my limited understanding of how nostr works, it's not clear to me 
how a distributed system that uses message broadcast would work in the 
same way as a mailing list. How would people "subscribe"? How would 
archives be searched or otherwise be available to people who are not on 
nostr? How do you distinguish and filter between legitimate dev posts 
intended for discussion and random crap and shitposting as shows up on 
social media?

I also don't think that long form text on nostr (or any similar 
platform) can sufficiently replace email. None of these things seem to 
contain a way to have a separate subject line as email does. Subjects 
are immensely important for me as it provides a quick and easy way to 
filter out things I don't care about reading. I don't want to have read 
something in before I can decide that I don't care about reading it.

In general, I strongly prefer email, or a platform that has email as a 
first class user interface, over platforms such as nostr, matrix, or web 
forums. Email is universal - everyone has one and everyone knows how it 
works. It dramatically lowers the barrier of entry. Having to make an 
account somewhere or download some specific client in order to 
participate will simply result in only the most dedicated participating. 
Development in open source must be an open process and the barriers to 
entry should be low.

Lastly, IIRC the plan is to shut down the list by the end of this year. 
Any solution that requires custom software and bridges to be created are 
not going to be viable in this time frame.


Andrew

On 11/07/2023 12:03 PM, Ademan via bitcoin-dev wrote:
> Hi Bryan,
> 
> I don't really want my first (and last?) devlist message to be a fairly 
> off-the-cuff post on this topic, but here we go anyway.
> 
> At the risk of sounding like a nostr evangelist (I promise I'm not), I 
> want to suggest nostr as a potential replacement to the mailing list. A 
> decent chunk of software would need to be written, but none of the 
> alternatives seem particularly attractive to me. I particularly dislike 
> the idea of locking into a single siloed forum service like the 
> bitcointalk forums. I realize I may be in the minority of course.
> 
> 
> Nostr enables the ML team to outsource all of its biggest burdens, if it 
> chooses:
> 
> - mail server blocking is N/A to nostr
> 
> - Hosting costs are completely outsourced unless the ML team chooses to 
> host a relay.
> 
> - Archives and web portal access can be similarly outsourced because any 
> nostr archive is self-authenticating.
> 
> - The ML team can also choose to completely outsource moderation, as 
> nostr is (more or less) permissionless by nature.
>    I understand if there is a "blessed" communication system, the ML 
> team would probably prefer to keep it high quality. To that end there 
> are proposals for proof-of-work, and web of trust based blocklists for 
> nostr which are optional for end users. There are other options such as 
> the "moderated communities" proposal which would provide tighter control.
> 
> 
> On the user side, the optional moderation is very attractive, allowing 
> controversial threads to exist and continue, without requiring everyone 
> to see them.
> 
> 
> The following do not currently exist (to my knowledge) and would need to 
> be implemented to meet the ML's requirements:
> 
> - an email gateway to satisfy the bulk of existing ML subscribers
>    This reintroduces issues with mail server blocking of course.
> - a long-form oriented nostr client (current plain text clients could be 
> used in the meantime)
> 
> That admittedly is quite a lot of work, but the second item can be 
> deferred, and the first is not particularly technically challenging, the 
> complications are all on the administration side.
> 
> I expect some reflexive NACKs based on the immaturity of the ecosystem 
> but if we have months to prepare, I believe the core requirements can be 
> solidly satisfied in time, the rest can be developed over time, and I 
> believe the advantages are worth careful consideration.
> 
> Cheers,
> Dan
> 
> On Tue, Nov 7, 2023 at 9:38 AM Bryan Bishop via bitcoin-dev 
> <bitcoin-dev@lists.linuxfoundation.org 
> <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
> 
>     Hello,
> 
>     We would like to request community feedback and proposals on the
>     future of the mailing list.
> 
>     Our current mailing list host, Linux Foundation, has indicated for
>     years that they have wanted to stop hosting mailing lists, which
>     would mean the bitcoin-dev mailing list would need to move somewhere
>     else. We temporarily avoided that, but recently LF has informed a
>     moderator that they will cease hosting any mailing lists later this
>     year.
> 
>     In this email, we will go over some of the history, options, and
>     invite discussion ahead of the cutoff. We have some ideas but want
>     to solicit feedback and proposals.
> 
>     Background
>     ==========
> 
>     The bitcoin-dev mailing list was originally hosted on
>     Sourceforge.net. The bitcoin development mailing list has been a
>     source of proposals, analysis, and developer discussion for many
>     years in the bitcoin community, with many thousands of participants.
>     Later, the mailing list was migrated to the Linux Foundation, and
>     after that OSUOSL began to help.
> 
>     Linux Foundation first asked us to move the mailing list in 2017.
>     They internally attempted to migrate all LF mailing lists from
>     mailman2 to mailman3, but ultimately gave up. There were reports of
>     scalability issues with mailman3 for large email communities. Ours
>     definitely qualifies as.. large.
> 
>     2019 migration plan: LF was to turn off mailman and all lists would
>     migrate to the paid service provider groups.io <http://groups.io>.
>     Back then we were given accounts to try the groups.io
>     <http://groups.io> interface and administration features. Apparently
>     we were not the only dev community who resisted change. To our
>     surprise LF gave us several years of reprieve by instead handing the
>     subdomain and server-side data to the non-profit OSUOSL lab who
>     instead operated mailman2 for the past ~4 years.
> 
>     OSUOSL has for decades been well known for providing server
>     infrastructure for Linux and Open Source development so they were a
>     good fit. This however became an added maintenance burden for the
>     small non-profit with limited resources. Several members of the
>     Bitcoin dev community contributed funding to the lab in support of
>     their Open Source development infrastructure goals. But throwing
>     money at the problem isn’t going to fix the ongoing maintenance
>     burden created by antiquated limitations of mailman2.
> 
>     Permalinks
>     ==========
> 
>     Linux Foundation has either offered or agreed to maintain archive
>     permalinks so that content of historic importance is not lost.
>     Fortunately for us while lists.linuxfoundation.org
>     <http://lists.linuxfoundation.org> mailman will go down, they have
>     agreed the read-only pipermail archives will remain online. So all
>     old URLs will continue to remain valid. However, the moderators
>     strongly advise that the community supplements with public-inbox
>     instances to have canonical archive urls that are separate from any
>     particular email software host.
> 
>     Public-Inbox
>     ============
> 
>     https://public-inbox.org/README.html
>     <https://public-inbox.org/README.html>
> 
>     “Public Inbox” decentralized archiving - no matter what mailing list
>     server solution is used, anyone can use git to maintain their own
>     mailing list archive and make it available to read on the web.
> 
>     Public Inbox is a tool that you can run yourself. You can transform
>     your mbox file and it makes it browsable and viewable online. It
>     commits every post to a git repository. It's kind of like a
>     decentralized mail archiving tool. Anyone can publish the mail
>     archive to any web server they wish.
> 
>     We should try to have one or more canonical archives that are served
>     using public-inbox. But it doesn't matter if these are lost because
>     anyone else can archive the mailing list in the same way and
>     re-publish the archives.
> 
>     These git commits can also be stamped using opentimestamps,
>     inserting their hashes into the bitcoin blockchain.
> 
>     LKML mailing list readers often use public-inbox's web interface,
>     and they use the reply-to headers to populate their mail client and
>     reply to threads of interest. This allows their reply to be properly
>     threaded even if they were not a previous subscriber to that mailing
>     list to receive the headers.
> 
>     public-inbox makes it so that it doesn't really matter where the
>     list is hosted, as pertaining to reading the mailing list. There is
>     still a disruption if the mailing list goes away, but the archives
>     live on and then people can post elsewhere. The archive gets
>     disconnected from the mailing list host in terms of posting. We
>     could have a few canonical URLs for the hosts, separate from the
>     mailing list server.
> 
>     mailman problems
>     ================
> 
>     Over the years we have identified a number of problems with mailman2
>     especially as it pertains to content moderation. There are presently
>     a handful of different moderators, but mailman2 only has a single
>     password for logging into the email management interface. There are
>     no moderator audit logs to see which user (there is no concept of
>     different users) acted on an email. There is no way to mark an email
>     as being investigated by one or more of the moderators. Sometimes,
>     while investigating the veracity of an email, another moderator
>     would come in and approve a suspect email by accident.
> 
>     Anti spam has been an issue for the moderators. It's relentless.
>     Without access to the underlying server, it has been difficult to
>     fight spam. There is some support for filters in mailman2 but it's
>     not great.
> 
>     100% active moderation and approval of every email is unsustainable
>     for volunteer moderators. A system that requires moderation is a
>     heavy burden on the moderators and it slows down overall
>     communication and productivity. There's lots of problems with this.
>     Also, moderators can be blamed when they are merely slow while they
>     are not actually censoring.
> 
>     Rejection emails can optionally be sent to
>     bitcoin-dev-moderation@lists.ozlabs.org
>     <mailto:bitcoin-dev-moderation@lists.ozlabs.org> but this is an
>     option that a moderator has to remember to type in each time.
> 
>     Not to mention numerous bugs and vulnerabilities that have
>     accumulated over the years for relatively unmaintained software.
>     (Not disclosed here)
> 
>     Requirements and considerations
>     ===============================
> 
>     Looking towards the future, there are a number of properties that
>     seem to be important for the bitcoin-dev mailing list community.
>     First, it is important that backups of the entire archive should be
>     easy for the public to copy or verify so that the system can be
>     brought up elsewhere if necessary.
> 
>     Second, there seems to be demand for both an email threading
>     interface (using mailing list software) as well as web-accessible
>     interfaces (such as forum software). There seems to be very few
>     options that cater to both email and web. Often, in forum software,
>     email support is limited to email notifications and there is limited
>     if any support for email user participation.
> 
>     Third, there should be better support for moderator tools and
>     management of the mailing list. See above for complaints about
>     problems with the mailman2 system.
> 
>     Burdens of running your own mailing list and email server
>     =========================================================
> 
>     If you have never operated your own MTA you have no idea how
>     difficult it is to keep secure and functional in the face of
>     numerous challenges to deliverability. Anti-spam filtering is
>     essential to prevent forwarding spam. The moment you forward even a
>     single spam message you run the risk of the server IP address being
>     added to blacklists.
> 
>     The problem of spam filtering is so bad that most IP addresses are
>     presumed guilty even if they have no prior spam history, such as if
>     their network or subnetwork had spam issues in the past.
> 
>     Even if you put unlimited time into managing your own email server,
>     other people may not accept your email. Or you make one mistake, and
>     then you get into permanent blacklists and it's hard to remove. The
>     spam problem is so bad that most IPs are automatically on a
>     guilty-until-proven-innocent blacklist.
> 
>     Often there is nothing you can do to get server IP addresses removed
>     from spam blacklists or from "bad reputation" lists.
> 
>     Ironically, hashcash-style proof-of-work stamps to prevent spam are
>     an appealing solution but not widely used in this community. Or
>     anywhere.
> 
>     Infinite rejection or forwarding loops happen. They often need to be
>     detected through vigilance and require manual sysadmin intervention
>     to solve.
> 
>     Bitcoin's dev lists being hosted alongside other Open Source
>     projects was previously protective. If that mailing list server
>     became blacklisted there were a lot of other people who would notice
>     and complain. If we run a Bitcoin-specific mail server we are on our
>     own. 100% of the administrative burden falls upon our own people.
>     There is also nothing we can do if some unknown admin decides they
>     don't like us.
> 
>     Options
>     =======
> 
>     Web forums are an interesting option, but often don't have good
>     email user integration. At most you can usually hope for email
>     notifications and an ability to reply by email. It changes the model
>     of the community from push (email) to pull (logging into a forum to
>     read). RSS feeds can help a little bit.
> 
>     Many other projects have moved from mailing lists to forums (eg
>     https://discuss.python.org/ <https://discuss.python.org/> – see
>     https://lwn.net/Articles/901744/ <https://lwn.net/Articles/901744/>
>     ; or https://ethresear.ch/ <https://ethresear.ch/>), which seem
>     easier to maintain and moderate, and can have lots of advanced
>     features beyond plaintext, maybe-threading and maybe-HTML-markup.
> 
>     Who would host the forum? Would there be agreement around which
>     forum software to use or which forum host? What about
>     bitcointalk.org <http://bitcointalk.org> or delvingbitcoin.org
>     <http://delvingbitcoin.org>? There are many options available. Maybe
>     what we actually want isn’t so much a discussion forum, as an 'arxiv
>     of our own' where anons can post BIP drafts and the like?
> 
>     Given the problems with mailman2, and the decline of email
>     communities in general, it seems that moving to mailman3 would not
>     be a viable long-term option. This leaves us with Google Groups or
>     groups.io <http://groups.io> as two remaining options.
> 
>     groups.io <http://groups.io> is an interesting option: they are a
>     paid service that implements email communities along with online web
>     forum support. However, their public changelog indicates it has been
>     a few years since their last public change. They might be a smaller
>     company and it is unclear how long they will be around or if this
>     would be the right fit for hosting sometimes contentious bitcoin
>     development discussions...
> 
>     Google Groups is another interesting option, and comes with
>     different tradeoffs. It's the lowest effort to maintain option, and
>     has both an email interface and web forum interface. Users can
>     choose which mode they want to interact with.
> 
>     For the Google Groups web interface, you can use it with a non-gmail
>     account, but you must create a Google Account which is free to do.
>     it does not require any personal information to do so. This also
>     allows you to add 2FA. Non-gmail non-google users are able to
>     subscribe and post email from their non-gmail non-google email
>     accounts. Tor seems to work for the web interface.
> 
>     Will Google shut it down, will they cut us off, will they shut down
>     non-google users? The same problem exists with other third-party hosts.
> 
>     The moderation capabilities for Google Groups and groups.io
>     <http://groups.io> seem to be comparable. It seems more likely that
>     Google Groups will be able to handle email delivery issues far
>     better than a small resource-constrained operation like groups.io
>     <http://groups.io>. ((During feedback for this draft, luke-jr
>     indicates that Google Workspaces has been known to use blacklisted
>     IPs for business email!))
> 
>     On the other hand, groups.io <http://groups.io> is a paid service
>     and you get what you pay for... hopefully?
> 
>     Finally, another option is to do literally nothing. It's less work
>     overall. Users can switch to forums or other websites, or private
>     one-on-one communication. It would remove a point of
>     semi-centralization from the bitcoin ecosystem. It would hasten
>     ossification, but on the other hand it would hasten ossification and
>     this could be a negative too. Moderators would be less of a target.
> 
>     Unfortunately, by doing nothing, there would be no more widely used
>     group email communication system between bitcoin developers.
>     Developers become less coordinated, mayhem and chaos as people go to
>     different communication platforms, a divided community is more
>     vulnerable, etc. BIP1 and BIP2 would need to be revised for other
>     venues.
> 
>     The main categories of what to move to are: web forums, mailing
>     lists, and hybrids of those two options. Most everything is either
>     self-hosted or you pay someone else to host it. It's kind of the
>     same problem though. It largely depends on how good is the software
>     and unfortunately running your own MTA that forwards mail is not a
>     good option.
> 
>     Going forward
>     ===========
> 
>     We'd like to invite feedback and proposals from the community, and
>     see what options are available. One potential option is a migration
>     to Google Groups, but we're open to ideas at this point. We
>     apologize for any inconvenience this disruption has caused.
> 
> 
>     Bitcoin-dev mailing list moderation team
> 
>     Bryan Bishop
>     Ruben Somsen
>     Warren Togami
>     various others.
> 
>     -- 
>     - Bryan
>     https://twitter.com/kanzure <https://twitter.com/kanzure>
>     _______________________________________________
>     bitcoin-dev mailing list
>     bitcoin-dev@lists.linuxfoundation.org
>     <mailto:bitcoin-dev@lists.linuxfoundation.org>
>     https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>     <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>
> 

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Andreas Schildbach]

On 07/11/2023 16.37, Bryan Bishop via bitcoin-dev wrote:

> We would like to request community feedback and proposals on the future 
> of the mailing list.
 >
 > [...]

Have you considered switching to Matrix? It's federated, much like 
e-mail. It's censorship resistant, in the sense that any participating 
homeserver keeps a copy of a room. And everyone can run their own 
homeserver in the Matrix network.

Rooms can be E2E encrypted. For an email list replacement this is 
probably not relevant, however any person-2-person communication would 
benefit from being encrypted.

Synapse is the currently best developed home server that implements the 
Matrix specification. It's much easier to run than an email server + 
mailing list combo.

Much like E-Mail and IRC, there is a huge ecosystem of Matrix clients to 
choose from.

I'm aware the UI of most Matrix clients is more suitable for a chat 
system than lengthy email discussions. However, thanks to "threads", 
there is no reason it could be used like that.

https://matrix.org/
https://github.com/matrix-org/synapse

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Ademan]

[-- Attachment #1: Type: text/plain, Size: 16352 bytes --]

Hi Bryan,

I don't really want my first (and last?) devlist message to be a fairly
off-the-cuff post on this topic, but here we go anyway.

At the risk of sounding like a nostr evangelist (I promise I'm not), I want
to suggest nostr as a potential replacement to the mailing list. A decent
chunk of software would need to be written, but none of the alternatives
seem particularly attractive to me. I particularly dislike the idea of
locking into a single siloed forum service like the bitcointalk forums. I
realize I may be in the minority of course.


Nostr enables the ML team to outsource all of its biggest burdens, if it
chooses:

- mail server blocking is N/A to nostr

- Hosting costs are completely outsourced unless the ML team chooses to
host a relay.

- Archives and web portal access can be similarly outsourced because any
nostr archive is self-authenticating.

- The ML team can also choose to completely outsource moderation, as nostr
is (more or less) permissionless by nature.
  I understand if there is a "blessed" communication system, the ML team
would probably prefer to keep it high quality. To that end there are
proposals for proof-of-work, and web of trust based blocklists for nostr
which are optional for end users. There are other options such as the
"moderated communities" proposal which would provide tighter control.


On the user side, the optional moderation is very attractive, allowing
controversial threads to exist and continue, without requiring everyone to
see them.


The following do not currently exist (to my knowledge) and would need to be
implemented to meet the ML's requirements:

- an email gateway to satisfy the bulk of existing ML subscribers
  This reintroduces issues with mail server blocking of course.
- a long-form oriented nostr client (current plain text clients could be
used in the meantime)

That admittedly is quite a lot of work, but the second item can be
deferred, and the first is not particularly technically challenging, the
complications are all on the administration side.

I expect some reflexive NACKs based on the immaturity of the ecosystem but
if we have months to prepare, I believe the core requirements can be
solidly satisfied in time, the rest can be developed over time, and I
believe the advantages are worth careful consideration.

Cheers,
Dan

On Tue, Nov 7, 2023 at 9:38 AM Bryan Bishop via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hello,
>
> We would like to request community feedback and proposals on the future of
> the mailing list.
>
> Our current mailing list host, Linux Foundation, has indicated for years
> that they have wanted to stop hosting mailing lists, which would mean the
> bitcoin-dev mailing list would need to move somewhere else. We temporarily
> avoided that, but recently LF has informed a moderator that they will cease
> hosting any mailing lists later this year.
>
> In this email, we will go over some of the history, options, and invite
> discussion ahead of the cutoff. We have some ideas but want to solicit
> feedback and proposals.
>
> Background
> ==========
>
> The bitcoin-dev mailing list was originally hosted on Sourceforge.net. The
> bitcoin development mailing list has been a source of proposals, analysis,
> and developer discussion for many years in the bitcoin community, with many
> thousands of participants. Later, the mailing list was migrated to the
> Linux Foundation, and after that OSUOSL began to help.
>
> Linux Foundation first asked us to move the mailing list in 2017. They
> internally attempted to migrate all LF mailing lists from mailman2 to
> mailman3, but ultimately gave up. There were reports of scalability issues
> with mailman3 for large email communities. Ours definitely qualifies as..
> large.
>
> 2019 migration plan: LF was to turn off mailman and all lists would
> migrate to the paid service provider groups.io. Back then we were given
> accounts to try the groups.io interface and administration features.
> Apparently we were not the only dev community who resisted change. To our
> surprise LF gave us several years of reprieve by instead handing the
> subdomain and server-side data to the non-profit OSUOSL lab who instead
> operated mailman2 for the past ~4 years.
>
> OSUOSL has for decades been well known for providing server infrastructure
> for Linux and Open Source development so they were a good fit. This however
> became an added maintenance burden for the small non-profit with limited
> resources. Several members of the Bitcoin dev community contributed funding
> to the lab in support of their Open Source development infrastructure
> goals. But throwing money at the problem isn’t going to fix the ongoing
> maintenance burden created by antiquated limitations of mailman2.
>
> Permalinks
> ==========
>
> Linux Foundation has either offered or agreed to maintain archive
> permalinks so that content of historic importance is not lost. Fortunately
> for us while lists.linuxfoundation.org mailman will go down, they have
> agreed the read-only pipermail archives will remain online. So all old URLs
> will continue to remain valid. However, the moderators strongly advise that
> the community supplements with public-inbox instances to have canonical
> archive urls that are separate from any particular email software host.
>
> Public-Inbox
> ============
>
> https://public-inbox.org/README.html
>
> “Public Inbox” decentralized archiving - no matter what mailing list
> server solution is used, anyone can use git to maintain their own mailing
> list archive and make it available to read on the web.
>
> Public Inbox is a tool that you can run yourself. You can transform your
> mbox file and it makes it browsable and viewable online. It commits every
> post to a git repository. It's kind of like a decentralized mail archiving
> tool. Anyone can publish the mail archive to any web server they wish.
>
> We should try to have one or more canonical archives that are served using
> public-inbox. But it doesn't matter if these are lost because anyone else
> can archive the mailing list in the same way and re-publish the archives.
>
> These git commits can also be stamped using opentimestamps, inserting
> their hashes into the bitcoin blockchain.
>
> LKML mailing list readers often use public-inbox's web interface, and they
> use the reply-to headers to populate their mail client and reply to threads
> of interest. This allows their reply to be properly threaded even if they
> were not a previous subscriber to that mailing list to receive the headers.
>
> public-inbox makes it so that it doesn't really matter where the list is
> hosted, as pertaining to reading the mailing list. There is still a
> disruption if the mailing list goes away, but the archives live on and then
> people can post elsewhere. The archive gets disconnected from the mailing
> list host in terms of posting. We could have a few canonical URLs for the
> hosts, separate from the mailing list server.
>
> mailman problems
> ================
>
> Over the years we have identified a number of problems with mailman2
> especially as it pertains to content moderation. There are presently a
> handful of different moderators, but mailman2 only has a single password
> for logging into the email management interface. There are no moderator
> audit logs to see which user (there is no concept of different users) acted
> on an email. There is no way to mark an email as being investigated by one
> or more of the moderators. Sometimes, while investigating the veracity of
> an email, another moderator would come in and approve a suspect email by
> accident.
>
> Anti spam has been an issue for the moderators. It's relentless. Without
> access to the underlying server, it has been difficult to fight spam. There
> is some support for filters in mailman2 but it's not great.
>
> 100% active moderation and approval of every email is unsustainable for
> volunteer moderators. A system that requires moderation is a heavy burden
> on the moderators and it slows down overall communication and productivity.
> There's lots of problems with this. Also, moderators can be blamed when
> they are merely slow while they are not actually censoring.
>
> Rejection emails can optionally be sent to
> bitcoin-dev-moderation@lists.ozlabs.org but this is an option that a
> moderator has to remember to type in each time.
>
> Not to mention numerous bugs and vulnerabilities that have accumulated
> over the years for relatively unmaintained software. (Not disclosed here)
>
> Requirements and considerations
> ===============================
>
> Looking towards the future, there are a number of properties that seem to
> be important for the bitcoin-dev mailing list community. First, it is
> important that backups of the entire archive should be easy for the public
> to copy or verify so that the system can be brought up elsewhere if
> necessary.
>
> Second, there seems to be demand for both an email threading interface
> (using mailing list software) as well as web-accessible interfaces (such as
> forum software). There seems to be very few options that cater to both
> email and web. Often, in forum software, email support is limited to email
> notifications and there is limited if any support for email user
> participation.
>
> Third, there should be better support for moderator tools and management
> of the mailing list. See above for complaints about problems with the
> mailman2 system.
>
> Burdens of running your own mailing list and email server
> =========================================================
>
> If you have never operated your own MTA you have no idea how difficult it
> is to keep secure and functional in the face of numerous challenges to
> deliverability. Anti-spam filtering is essential to prevent forwarding
> spam. The moment you forward even a single spam message you run the risk of
> the server IP address being added to blacklists.
>
> The problem of spam filtering is so bad that most IP addresses are
> presumed guilty even if they have no prior spam history, such as if their
> network or subnetwork had spam issues in the past.
>
> Even if you put unlimited time into managing your own email server, other
> people may not accept your email. Or you make one mistake, and then you get
> into permanent blacklists and it's hard to remove. The spam problem is so
> bad that most IPs are automatically on a guilty-until-proven-innocent
> blacklist.
>
> Often there is nothing you can do to get server IP addresses removed from
> spam blacklists or from "bad reputation" lists.
>
> Ironically, hashcash-style proof-of-work stamps to prevent spam are an
> appealing solution but not widely used in this community. Or anywhere.
>
> Infinite rejection or forwarding loops happen. They often need to be
> detected through vigilance and require manual sysadmin intervention to
> solve.
>
> Bitcoin's dev lists being hosted alongside other Open Source projects was
> previously protective. If that mailing list server became blacklisted there
> were a lot of other people who would notice and complain. If we run a
> Bitcoin-specific mail server we are on our own. 100% of the administrative
> burden falls upon our own people. There is also nothing we can do if some
> unknown admin decides they don't like us.
>
> Options
> =======
>
> Web forums are an interesting option, but often don't have good email user
> integration. At most you can usually hope for email notifications and an
> ability to reply by email. It changes the model of the community from push
> (email) to pull (logging into a forum to read). RSS feeds can help a little
> bit.
>
> Many other projects have moved from mailing lists to forums (eg
> https://discuss.python.org/ – see https://lwn.net/Articles/901744/ ; or
> https://ethresear.ch/), which seem easier to maintain and moderate, and
> can have lots of advanced features beyond plaintext, maybe-threading and
> maybe-HTML-markup.
>
> Who would host the forum? Would there be agreement around which forum
> software to use or which forum host? What about bitcointalk.org or
> delvingbitcoin.org? There are many options available. Maybe what we
> actually want isn’t so much a discussion forum, as an 'arxiv of our own'
> where anons can post BIP drafts and the like?
>
> Given the problems with mailman2, and the decline of email communities in
> general, it seems that moving to mailman3 would not be a viable long-term
> option. This leaves us with Google Groups or groups.io as two remaining
> options.
>
> groups.io is an interesting option: they are a paid service that
> implements email communities along with online web forum support. However,
> their public changelog indicates it has been a few years since their last
> public change. They might be a smaller company and it is unclear how long
> they will be around or if this would be the right fit for hosting sometimes
> contentious bitcoin development discussions...
>
> Google Groups is another interesting option, and comes with different
> tradeoffs. It's the lowest effort to maintain option, and has both an email
> interface and web forum interface. Users can choose which mode they want to
> interact with.
>
> For the Google Groups web interface, you can use it with a non-gmail
> account, but you must create a Google Account which is free to do. it does
> not require any personal information to do so. This also allows you to add
> 2FA. Non-gmail non-google users are able to subscribe and post email from
> their non-gmail non-google email accounts. Tor seems to work for the web
> interface.
>
> Will Google shut it down, will they cut us off, will they shut down
> non-google users? The same problem exists with other third-party hosts.
>
> The moderation capabilities for Google Groups and groups.io seem to be
> comparable. It seems more likely that Google Groups will be able to handle
> email delivery issues far better than a small resource-constrained
> operation like groups.io. ((During feedback for this draft, luke-jr
> indicates that Google Workspaces has been known to use blacklisted IPs for
> business email!))
>
> On the other hand, groups.io is a paid service and you get what you pay
> for... hopefully?
>
> Finally, another option is to do literally nothing. It's less work
> overall. Users can switch to forums or other websites, or private
> one-on-one communication. It would remove a point of semi-centralization
> from the bitcoin ecosystem. It would hasten ossification, but on the other
> hand it would hasten ossification and this could be a negative too.
> Moderators would be less of a target.
>
> Unfortunately, by doing nothing, there would be no more widely used group
> email communication system between bitcoin developers. Developers become
> less coordinated, mayhem and chaos as people go to different communication
> platforms, a divided community is more vulnerable, etc. BIP1 and BIP2 would
> need to be revised for other venues.
>
> The main categories of what to move to are: web forums, mailing lists, and
> hybrids of those two options. Most everything is either self-hosted or you
> pay someone else to host it. It's kind of the same problem though. It
> largely depends on how good is the software and unfortunately running your
> own MTA that forwards mail is not a good option.
>
> Going forward
> ===========
>
> We'd like to invite feedback and proposals from the community, and see
> what options are available. One potential option is a migration to Google
> Groups, but we're open to ideas at this point. We apologize for any
> inconvenience this disruption has caused.
>
>
> Bitcoin-dev mailing list moderation team
>
> Bryan Bishop
> Ruben Somsen
> Warren Togami
> various others.
>
> --
> - Bryan
> https://twitter.com/kanzure
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 17895 bytes --]

Re: [bitcoin-dev] Future of the bitcoin-dev mailing list

[Andrew Chow]

Thanks for writing this up.

I would prefer that we continue to have a mailing list where email is a 
functional and first class user interface. So that would be to migrate 
to groups.io or Google Groups. I think Google Groups is probably the 
better choice of the two.

Although there are concerns that Google would shut down Google Groups or 
specifically target a bitcoin-dev group, I think both are unlikely to 
happen. Both Chromium and Android use Google Groups for their mailing 
lists, so unless those go somewhere else, I doubt Google would 
unceremoniously kill Google Groups. As for shutting down a bitcoin-dev 
group specifically, given that there appears to be several thousand 
groups whose sole purpose is to distribute spam, I don't think Google is 
in the habit of shutting down groups.

Regardless of what we do, there's always the risk that someone will shut 
down or stop maintaining the servers for any discussion medium. Even 
self hosting requires someone to keep the servers up and do maintenance, 
and that person (or people) could get bored of it, run out of money, get 
hit by a bus, etc. We're experiencing that right now with the Linux 
Foundation, and I don't think fear of that should prevent us from moving 
to a different third party host.


Andrew Chow

On 11/07/2023 10:37 AM, Bryan Bishop via bitcoin-dev wrote:
> Hello,
> 
> We would like to request community feedback and proposals on the future 
> of the mailing list.
> 
> Our current mailing list host, Linux Foundation, has indicated for years 
> that they have wanted to stop hosting mailing lists, which would mean 
> the bitcoin-dev mailing list would need to move somewhere else. We 
> temporarily avoided that, but recently LF has informed a moderator that 
> they will cease hosting any mailing lists later this year.
> 
> In this email, we will go over some of the history, options, and invite 
> discussion ahead of the cutoff. We have some ideas but want to solicit 
> feedback and proposals.
> 
> Background
> ==========
> 
> The bitcoin-dev mailing list was originally hosted on Sourceforge.net. 
> The bitcoin development mailing list has been a source of proposals, 
> analysis, and developer discussion for many years in the bitcoin 
> community, with many thousands of participants. Later, the mailing list 
> was migrated to the Linux Foundation, and after that OSUOSL began to help.
> 
> Linux Foundation first asked us to move the mailing list in 2017. They 
> internally attempted to migrate all LF mailing lists from mailman2 to 
> mailman3, but ultimately gave up. There were reports of scalability 
> issues with mailman3 for large email communities. Ours definitely 
> qualifies as.. large.
> 
> 2019 migration plan: LF was to turn off mailman and all lists would 
> migrate to the paid service provider groups.io <http://groups.io>. Back 
> then we were given accounts to try the groups.io <http://groups.io> 
> interface and administration features. Apparently we were not the only 
> dev community who resisted change. To our surprise LF gave us several 
> years of reprieve by instead handing the subdomain and server-side data 
> to the non-profit OSUOSL lab who instead operated mailman2 for the past 
> ~4 years.
> 
> OSUOSL has for decades been well known for providing server 
> infrastructure for Linux and Open Source development so they were a good 
> fit. This however became an added maintenance burden for the small 
> non-profit with limited resources. Several members of the Bitcoin dev 
> community contributed funding to the lab in support of their Open Source 
> development infrastructure goals. But throwing money at the problem 
> isn’t going to fix the ongoing maintenance burden created by antiquated 
> limitations of mailman2.
> 
> Permalinks
> ==========
> 
> Linux Foundation has either offered or agreed to maintain archive 
> permalinks so that content of historic importance is not lost. 
> Fortunately for us while lists.linuxfoundation.org 
> <http://lists.linuxfoundation.org> mailman will go down, they have 
> agreed the read-only pipermail archives will remain online. So all old 
> URLs will continue to remain valid. However, the moderators strongly 
> advise that the community supplements with public-inbox instances to 
> have canonical archive urls that are separate from any particular email 
> software host.
> 
> Public-Inbox
> ============
> 
> https://public-inbox.org/README.html <https://public-inbox.org/README.html>
> 
> “Public Inbox” decentralized archiving - no matter what mailing list 
> server solution is used, anyone can use git to maintain their own 
> mailing list archive and make it available to read on the web.
> 
> Public Inbox is a tool that you can run yourself. You can transform your 
> mbox file and it makes it browsable and viewable online. It commits 
> every post to a git repository. It's kind of like a decentralized mail 
> archiving tool. Anyone can publish the mail archive to any web server 
> they wish.
> 
> We should try to have one or more canonical archives that are served 
> using public-inbox. But it doesn't matter if these are lost because 
> anyone else can archive the mailing list in the same way and re-publish 
> the archives.
> 
> These git commits can also be stamped using opentimestamps, inserting 
> their hashes into the bitcoin blockchain.
> 
> LKML mailing list readers often use public-inbox's web interface, and 
> they use the reply-to headers to populate their mail client and reply to 
> threads of interest. This allows their reply to be properly threaded 
> even if they were not a previous subscriber to that mailing list to 
> receive the headers.
> 
> public-inbox makes it so that it doesn't really matter where the list is 
> hosted, as pertaining to reading the mailing list. There is still a 
> disruption if the mailing list goes away, but the archives live on and 
> then people can post elsewhere. The archive gets disconnected from the 
> mailing list host in terms of posting. We could have a few canonical 
> URLs for the hosts, separate from the mailing list server.
> 
> mailman problems
> ================
> 
> Over the years we have identified a number of problems with mailman2 
> especially as it pertains to content moderation. There are presently a 
> handful of different moderators, but mailman2 only has a single password 
> for logging into the email management interface. There are no moderator 
> audit logs to see which user (there is no concept of different users) 
> acted on an email. There is no way to mark an email as being 
> investigated by one or more of the moderators. Sometimes, while 
> investigating the veracity of an email, another moderator would come in 
> and approve a suspect email by accident.
> 
> Anti spam has been an issue for the moderators. It's relentless. Without 
> access to the underlying server, it has been difficult to fight spam. 
> There is some support for filters in mailman2 but it's not great.
> 
> 100% active moderation and approval of every email is unsustainable for 
> volunteer moderators. A system that requires moderation is a heavy 
> burden on the moderators and it slows down overall communication and 
> productivity. There's lots of problems with this. Also, moderators can 
> be blamed when they are merely slow while they are not actually censoring.
> 
> Rejection emails can optionally be sent to 
> bitcoin-dev-moderation@lists.ozlabs.org 
> <mailto:bitcoin-dev-moderation@lists.ozlabs.org> but this is an option 
> that a moderator has to remember to type in each time.
> 
> Not to mention numerous bugs and vulnerabilities that have accumulated 
> over the years for relatively unmaintained software. (Not disclosed here)
> 
> Requirements and considerations
> ===============================
> 
> Looking towards the future, there are a number of properties that seem 
> to be important for the bitcoin-dev mailing list community. First, it is 
> important that backups of the entire archive should be easy for the 
> public to copy or verify so that the system can be brought up elsewhere 
> if necessary.
> 
> Second, there seems to be demand for both an email threading interface 
> (using mailing list software) as well as web-accessible interfaces (such 
> as forum software). There seems to be very few options that cater to 
> both email and web. Often, in forum software, email support is limited 
> to email notifications and there is limited if any support for email 
> user participation.
> 
> Third, there should be better support for moderator tools and management 
> of the mailing list. See above for complaints about problems with the 
> mailman2 system.
> 
> Burdens of running your own mailing list and email server
> =========================================================
> 
> If you have never operated your own MTA you have no idea how difficult 
> it is to keep secure and functional in the face of numerous challenges 
> to deliverability. Anti-spam filtering is essential to prevent 
> forwarding spam. The moment you forward even a single spam message you 
> run the risk of the server IP address being added to blacklists.
> 
> The problem of spam filtering is so bad that most IP addresses are 
> presumed guilty even if they have no prior spam history, such as if 
> their network or subnetwork had spam issues in the past.
> 
> Even if you put unlimited time into managing your own email server, 
> other people may not accept your email. Or you make one mistake, and 
> then you get into permanent blacklists and it's hard to remove. The spam 
> problem is so bad that most IPs are automatically on a 
> guilty-until-proven-innocent blacklist.
> 
> Often there is nothing you can do to get server IP addresses removed 
> from spam blacklists or from "bad reputation" lists.
> 
> Ironically, hashcash-style proof-of-work stamps to prevent spam are an 
> appealing solution but not widely used in this community. Or anywhere.
> 
> Infinite rejection or forwarding loops happen. They often need to be 
> detected through vigilance and require manual sysadmin intervention to 
> solve.
> 
> Bitcoin's dev lists being hosted alongside other Open Source projects 
> was previously protective. If that mailing list server became 
> blacklisted there were a lot of other people who would notice and 
> complain. If we run a Bitcoin-specific mail server we are on our own. 
> 100% of the administrative burden falls upon our own people. There is 
> also nothing we can do if some unknown admin decides they don't like us.
> 
> Options
> =======
> 
> Web forums are an interesting option, but often don't have good email 
> user integration. At most you can usually hope for email notifications 
> and an ability to reply by email. It changes the model of the community 
> from push (email) to pull (logging into a forum to read). RSS feeds can 
> help a little bit.
> 
> Many other projects have moved from mailing lists to forums (eg 
> https://discuss.python.org/ <https://discuss.python.org/> – see 
> https://lwn.net/Articles/901744/ <https://lwn.net/Articles/901744/> ; or 
> https://ethresear.ch/ <https://ethresear.ch/>), which seem easier to 
> maintain and moderate, and can have lots of advanced features beyond 
> plaintext, maybe-threading and maybe-HTML-markup.
> 
> Who would host the forum? Would there be agreement around which forum 
> software to use or which forum host? What about bitcointalk.org 
> <http://bitcointalk.org> or delvingbitcoin.org 
> <http://delvingbitcoin.org>? There are many options available. Maybe 
> what we actually want isn’t so much a discussion forum, as an 'arxiv of 
> our own' where anons can post BIP drafts and the like?
> 
> Given the problems with mailman2, and the decline of email communities 
> in general, it seems that moving to mailman3 would not be a viable 
> long-term option. This leaves us with Google Groups or groups.io 
> <http://groups.io> as two remaining options.
> 
> groups.io <http://groups.io> is an interesting option: they are a paid 
> service that implements email communities along with online web forum 
> support. However, their public changelog indicates it has been a few 
> years since their last public change. They might be a smaller company 
> and it is unclear how long they will be around or if this would be the 
> right fit for hosting sometimes contentious bitcoin development 
> discussions...
> 
> Google Groups is another interesting option, and comes with different 
> tradeoffs. It's the lowest effort to maintain option, and has both an 
> email interface and web forum interface. Users can choose which mode 
> they want to interact with.
> 
> For the Google Groups web interface, you can use it with a non-gmail 
> account, but you must create a Google Account which is free to do. it 
> does not require any personal information to do so. This also allows you 
> to add 2FA. Non-gmail non-google users are able to subscribe and post 
> email from their non-gmail non-google email accounts. Tor seems to work 
> for the web interface.
> 
> Will Google shut it down, will they cut us off, will they shut down 
> non-google users? The same problem exists with other third-party hosts.
> 
> The moderation capabilities for Google Groups and groups.io 
> <http://groups.io> seem to be comparable. It seems more likely that 
> Google Groups will be able to handle email delivery issues far better 
> than a small resource-constrained operation like groups.io 
> <http://groups.io>. ((During feedback for this draft, luke-jr indicates 
> that Google Workspaces has been known to use blacklisted IPs for 
> business email!))
> 
> On the other hand, groups.io <http://groups.io> is a paid service and 
> you get what you pay for... hopefully?
> 
> Finally, another option is to do literally nothing. It's less work 
> overall. Users can switch to forums or other websites, or private 
> one-on-one communication. It would remove a point of semi-centralization 
> from the bitcoin ecosystem. It would hasten ossification, but on the 
> other hand it would hasten ossification and this could be a negative 
> too. Moderators would be less of a target.
> 
> Unfortunately, by doing nothing, there would be no more widely used 
> group email communication system between bitcoin developers. Developers 
> become less coordinated, mayhem and chaos as people go to different 
> communication platforms, a divided community is more vulnerable, etc. 
> BIP1 and BIP2 would need to be revised for other venues.
> 
> The main categories of what to move to are: web forums, mailing lists, 
> and hybrids of those two options. Most everything is either self-hosted 
> or you pay someone else to host it. It's kind of the same problem 
> though. It largely depends on how good is the software and unfortunately 
> running your own MTA that forwards mail is not a good option.
> 
> Going forward
> ===========
> 
> We'd like to invite feedback and proposals from the community, and see 
> what options are available. One potential option is a migration to 
> Google Groups, but we're open to ideas at this point. We apologize for 
> any inconvenience this disruption has caused.
> 
> 
> Bitcoin-dev mailing list moderation team
> 
> Bryan Bishop
> Ruben Somsen
> Warren Togami
> various others.
> 
> -- 
> - Bryan
> https://twitter.com/kanzure <https://twitter.com/kanzure>

[bitcoin-dev] Future of the bitcoin-dev mailing list

[Bryan Bishop]

[-- Attachment #1: Type: text/plain, Size: 13173 bytes --]

Hello,

We would like to request community feedback and proposals on the future of
the mailing list.

Our current mailing list host, Linux Foundation, has indicated for years
that they have wanted to stop hosting mailing lists, which would mean the
bitcoin-dev mailing list would need to move somewhere else. We temporarily
avoided that, but recently LF has informed a moderator that they will cease
hosting any mailing lists later this year.

In this email, we will go over some of the history, options, and invite
discussion ahead of the cutoff. We have some ideas but want to solicit
feedback and proposals.

Background
==========

The bitcoin-dev mailing list was originally hosted on Sourceforge.net. The
bitcoin development mailing list has been a source of proposals, analysis,
and developer discussion for many years in the bitcoin community, with many
thousands of participants. Later, the mailing list was migrated to the
Linux Foundation, and after that OSUOSL began to help.

Linux Foundation first asked us to move the mailing list in 2017. They
internally attempted to migrate all LF mailing lists from mailman2 to
mailman3, but ultimately gave up. There were reports of scalability issues
with mailman3 for large email communities. Ours definitely qualifies as..
large.

2019 migration plan: LF was to turn off mailman and all lists would migrate
to the paid service provider groups.io. Back then we were given accounts to
try the groups.io interface and administration features. Apparently we were
not the only dev community who resisted change. To our surprise LF gave us
several years of reprieve by instead handing the subdomain and server-side
data to the non-profit OSUOSL lab who instead operated mailman2 for the
past ~4 years.

OSUOSL has for decades been well known for providing server infrastructure
for Linux and Open Source development so they were a good fit. This however
became an added maintenance burden for the small non-profit with limited
resources. Several members of the Bitcoin dev community contributed funding
to the lab in support of their Open Source development infrastructure
goals. But throwing money at the problem isn’t going to fix the ongoing
maintenance burden created by antiquated limitations of mailman2.

Permalinks
==========

Linux Foundation has either offered or agreed to maintain archive
permalinks so that content of historic importance is not lost. Fortunately
for us while lists.linuxfoundation.org mailman will go down, they have
agreed the read-only pipermail archives will remain online. So all old URLs
will continue to remain valid. However, the moderators strongly advise that
the community supplements with public-inbox instances to have canonical
archive urls that are separate from any particular email software host.

Public-Inbox
============

https://public-inbox.org/README.html

“Public Inbox” decentralized archiving - no matter what mailing list server
solution is used, anyone can use git to maintain their own mailing list
archive and make it available to read on the web.

Public Inbox is a tool that you can run yourself. You can transform your
mbox file and it makes it browsable and viewable online. It commits every
post to a git repository. It's kind of like a decentralized mail archiving
tool. Anyone can publish the mail archive to any web server they wish.

We should try to have one or more canonical archives that are served using
public-inbox. But it doesn't matter if these are lost because anyone else
can archive the mailing list in the same way and re-publish the archives.

These git commits can also be stamped using opentimestamps, inserting their
hashes into the bitcoin blockchain.

LKML mailing list readers often use public-inbox's web interface, and they
use the reply-to headers to populate their mail client and reply to threads
of interest. This allows their reply to be properly threaded even if they
were not a previous subscriber to that mailing list to receive the headers.

public-inbox makes it so that it doesn't really matter where the list is
hosted, as pertaining to reading the mailing list. There is still a
disruption if the mailing list goes away, but the archives live on and then
people can post elsewhere. The archive gets disconnected from the mailing
list host in terms of posting. We could have a few canonical URLs for the
hosts, separate from the mailing list server.

mailman problems
================

Over the years we have identified a number of problems with mailman2
especially as it pertains to content moderation. There are presently a
handful of different moderators, but mailman2 only has a single password
for logging into the email management interface. There are no moderator
audit logs to see which user (there is no concept of different users) acted
on an email. There is no way to mark an email as being investigated by one
or more of the moderators. Sometimes, while investigating the veracity of
an email, another moderator would come in and approve a suspect email by
accident.

Anti spam has been an issue for the moderators. It's relentless. Without
access to the underlying server, it has been difficult to fight spam. There
is some support for filters in mailman2 but it's not great.

100% active moderation and approval of every email is unsustainable for
volunteer moderators. A system that requires moderation is a heavy burden
on the moderators and it slows down overall communication and productivity.
There's lots of problems with this. Also, moderators can be blamed when
they are merely slow while they are not actually censoring.

Rejection emails can optionally be sent to
bitcoin-dev-moderation@lists.ozlabs.org but this is an option that a
moderator has to remember to type in each time.

Not to mention numerous bugs and vulnerabilities that have accumulated over
the years for relatively unmaintained software. (Not disclosed here)

Requirements and considerations
===============================

Looking towards the future, there are a number of properties that seem to
be important for the bitcoin-dev mailing list community. First, it is
important that backups of the entire archive should be easy for the public
to copy or verify so that the system can be brought up elsewhere if
necessary.

Second, there seems to be demand for both an email threading interface
(using mailing list software) as well as web-accessible interfaces (such as
forum software). There seems to be very few options that cater to both
email and web. Often, in forum software, email support is limited to email
notifications and there is limited if any support for email user
participation.

Third, there should be better support for moderator tools and management of
the mailing list. See above for complaints about problems with the mailman2
system.

Burdens of running your own mailing list and email server
=========================================================

If you have never operated your own MTA you have no idea how difficult it
is to keep secure and functional in the face of numerous challenges to
deliverability. Anti-spam filtering is essential to prevent forwarding
spam. The moment you forward even a single spam message you run the risk of
the server IP address being added to blacklists.

The problem of spam filtering is so bad that most IP addresses are presumed
guilty even if they have no prior spam history, such as if their network or
subnetwork had spam issues in the past.

Even if you put unlimited time into managing your own email server, other
people may not accept your email. Or you make one mistake, and then you get
into permanent blacklists and it's hard to remove. The spam problem is so
bad that most IPs are automatically on a guilty-until-proven-innocent
blacklist.

Often there is nothing you can do to get server IP addresses removed from
spam blacklists or from "bad reputation" lists.

Ironically, hashcash-style proof-of-work stamps to prevent spam are an
appealing solution but not widely used in this community. Or anywhere.

Infinite rejection or forwarding loops happen. They often need to be
detected through vigilance and require manual sysadmin intervention to
solve.

Bitcoin's dev lists being hosted alongside other Open Source projects was
previously protective. If that mailing list server became blacklisted there
were a lot of other people who would notice and complain. If we run a
Bitcoin-specific mail server we are on our own. 100% of the administrative
burden falls upon our own people. There is also nothing we can do if some
unknown admin decides they don't like us.

Options
=======

Web forums are an interesting option, but often don't have good email user
integration. At most you can usually hope for email notifications and an
ability to reply by email. It changes the model of the community from push
(email) to pull (logging into a forum to read). RSS feeds can help a little
bit.

Many other projects have moved from mailing lists to forums (eg
https://discuss.python.org/ – see https://lwn.net/Articles/901744/ ; or
https://ethresear.ch/), which seem easier to maintain and moderate, and can
have lots of advanced features beyond plaintext, maybe-threading and
maybe-HTML-markup.

Who would host the forum? Would there be agreement around which forum
software to use or which forum host? What about bitcointalk.org or
delvingbitcoin.org? There are many options available. Maybe what we
actually want isn’t so much a discussion forum, as an 'arxiv of our own'
where anons can post BIP drafts and the like?

Given the problems with mailman2, and the decline of email communities in
general, it seems that moving to mailman3 would not be a viable long-term
option. This leaves us with Google Groups or groups.io as two remaining
options.

groups.io is an interesting option: they are a paid service that implements
email communities along with online web forum support. However, their
public changelog indicates it has been a few years since their last public
change. They might be a smaller company and it is unclear how long they
will be around or if this would be the right fit for hosting sometimes
contentious bitcoin development discussions...

Google Groups is another interesting option, and comes with different
tradeoffs. It's the lowest effort to maintain option, and has both an email
interface and web forum interface. Users can choose which mode they want to
interact with.

For the Google Groups web interface, you can use it with a non-gmail
account, but you must create a Google Account which is free to do. it does
not require any personal information to do so. This also allows you to add
2FA. Non-gmail non-google users are able to subscribe and post email from
their non-gmail non-google email accounts. Tor seems to work for the web
interface.

Will Google shut it down, will they cut us off, will they shut down
non-google users? The same problem exists with other third-party hosts.

The moderation capabilities for Google Groups and groups.io seem to be
comparable. It seems more likely that Google Groups will be able to handle
email delivery issues far better than a small resource-constrained
operation like groups.io. ((During feedback for this draft, luke-jr
indicates that Google Workspaces has been known to use blacklisted IPs for
business email!))

On the other hand, groups.io is a paid service and you get what you pay
for... hopefully?

Finally, another option is to do literally nothing. It's less work overall.
Users can switch to forums or other websites, or private one-on-one
communication. It would remove a point of semi-centralization from the
bitcoin ecosystem. It would hasten ossification, but on the other hand it
would hasten ossification and this could be a negative too. Moderators
would be less of a target.

Unfortunately, by doing nothing, there would be no more widely used group
email communication system between bitcoin developers. Developers become
less coordinated, mayhem and chaos as people go to different communication
platforms, a divided community is more vulnerable, etc. BIP1 and BIP2 would
need to be revised for other venues.

The main categories of what to move to are: web forums, mailing lists, and
hybrids of those two options. Most everything is either self-hosted or you
pay someone else to host it. It's kind of the same problem though. It
largely depends on how good is the software and unfortunately running your
own MTA that forwards mail is not a good option.

Going forward
===========

We'd like to invite feedback and proposals from the community, and see what
options are available. One potential option is a migration to Google
Groups, but we're open to ideas at this point. We apologize for any
inconvenience this disruption has caused.


Bitcoin-dev mailing list moderation team

Bryan Bishop
Ruben Somsen
Warren Togami
various others.

-- 
- Bryan
https://twitter.com/kanzure

[-- Attachment #2: Type: text/html, Size: 14194 bytes --]