On Wednesday, November 13, 2024 at 2:55:58 AM UTC+7 Bryan Bishop wrote:

Hi,

lists.linuxfoundation.org is no longer hosting the static HTML email
archives for bitcoin-dev or any other mailing list. We knew that this
might happen and prepared months ago with the mailing list migration
and also with various backups:
https://gnusha.org/pi/bitcoindev/CABaSBaxDjj6ySBx4v+rmpfrw4pE9b=JZJPzPQj_Z...@mail.gmail.com/

However, many thousands of links around the web are still pointing to
the old archive. This is really unfortunate as there is a wealth of
content in the archive and these links were considered canonical. This
is a general plea to update any broken links that you might be able to
fix.

What you can do, options
========================

There are a few options for actions that effected content providers can take:

1) Use `gnusha.org/url`. Explained below.

2) Replace lists.linuxfoundation.org urls with resolved urls to a new
location for mailing list archives. More on this below.

3) Do nothing. Wait and see. It is possible that we can plead our case
to Linux Foundation and get them to host static archives indefinitely,
or at least host redirects.

4) Replace broken links with Wayback Machine urls. Someone told me
apparently their archive is incomplete? I don't recommend this one
because I am concerned about stress and strain on the Internet Archive
Wayback Machine service beyond the additional activity that these
links would incur. Besides, Wayback is already a highly centralized
point for internet archival activity and therefore some
decenteralization is warranted here.

gnusha.org/url redirect service
===============================

This is a redirect service I made that takes a URL parameter and
redirects. To use it, get a previous url like:

`https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-June/014472.html`

your new url will be:

`https://gnusha.org/url/https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-June/014472.html`

and it will redirect to a current mailing list archive location for
that message:

`https://gnusha.org/pi/bitcoindev/ogoa6o$o4s$1...@blaine.gmane.org/`

It uses a mapping file based on the mailing list archives. I have
published scripts to make this mapping here:
https://gist.github.com/kanzure/4e7bcc58344ceaa1a668e65a434adb2b

See https://gnusha.org/url for more information. Thank you to Jonas
Schnelli for suggesting the creation of this redirect service.

Manual resolution
=================

How to "manually" resolve "without" the gnusha.org/url redirect
service. Use resolver.py here:
https://gist.github.com/kanzure/4e7bcc58344ceaa1a668e65a434adb2b#file-resolver-py
and set RESOLVER_HOST to `https://gnusha.org/url`, also set `resolver
= resolve_locally` to `resolver = resolve_remotely`.

This will convert urls from the old LF format to resolved urls like:
`https://gnusha.org/pi/bitcoindev/CABaSBaxDjj6ySBx4v+rmpfrw4pE9b=JZJPzPQj_Z...@mail.gmail.com/`
or
`https://mailing-list.bitcoindevs.xyz/bitcoindev/CABaSBaxDjj6ySBx4v+rmpfrw4pE9b=JZJPzPQj_Z...@mail.gmail.com/`
depending on which public-inbox mirror you choose to use.

Why resolve or not
==================

I'll elaborate on `gnusha.org/url` vs `gnusha.org/pi/bitcoindev` style urls.

It is nice and simple to use a find-and-replace with `gnusha.org/url/`
because anyone can easily repeat that exercise and verify the git diff
before merging the pull request. However, the actual routing is not
something that is verified by just checking that the `git diff` looks
sane.

If I was to provide my mapping (I haven't uploaded this yet, but I did
publish the scripts I used to make it, hopefully someone will
re-generate it to verify my work), then someone could write a script
and/or cross-verify that all of the replacements are valid according
to the mapping, if we were to use the destination urls instead of the
redirect service for this pull request. Then someone would be able to
verify that kind of pull request.

There are other further issues though, like content malleability,
wrong redirect link injection, service provider compromise... instead
of referencing by Message-ID maybe in the future we can reference by
email hash.... But now I'm dreaming.

- Bryan
https://x.com/kanzure