It might be helpful to reflect on how reality has changed since the OP_RETURN behavior was originally rolled out in February 2014.

At the time blockfile pruning would not be deployed to the public for another year and a half,  so everyone that ran a node was forced to store and serve anything that made it into a block.

In 2014 people were proposing protocol additions that would allow basically random access to the transactions/txouts over the P2P network, potentially outright allowing file trading applications that abused bitcoin nodes as their server.  Today there are no such proposals and any would be DOA because the abuse potential is well understood.

At the time miners were almost entirely driven by subsidy, fees were fairly insignificant.

With the exception of a rare joker block standardness rules were highly effective, and short of mining a block yourself or talking a highly community connected mining pool operator into it  (or being that operator) you couldn't bypass any of them.

Blocks were on average running at 16% of the consensus capacity limit.  Absent some kind of adhoc non-consensus restriction the cost to dump in large amounts of data would have been nothing.  Today the _average_ block is essentially full and the costs are considerable.

A minimum feerate of 1s/vb has increased in buying power by a factor of 171 from Feb 2014 to now.

Users were much more ignorant about what they got out of putting data in, it was common to hear things that would better be served with a commitment (or something like OTS, which wouldn't be available for another two and a half)-- they didn't have much reason to think hard about their application because stuffing in data was cheap (or, even free if not for limits).  Today's data stuffers cases generally do make sense on their own terms even if many would agree that they're not an appropriate use of Bitcoin.  There are more alternative communications channels than ever now, ipfs, nostr, other blockchains.  Stuffing data in is expensive. Those who still do it are willing to pay, which also means miners are willing to let them. The parties that do it are technically sophisticated or at the very least can afford the services of people who are and can and have implemented evasions to adhoc restrictions.

Today there are implementations of concepts like assumeutxo, which while subject to their own limitations would allow people to bring up fully validating nodes without ever downloading or processing spam data (and pruining as mentioned is real now and lets you not store it).  While not yet completed and deployed it clearly could be, but back then that sort of idea was highly theoretical.  Today the highly theoretical thing are stuff like ZKPs over the history.

At the time compact blocks as a protocol feature hadn't been conceived and earlier block relay improvements were only just being developed--- any additional data in blocks slowed their propagation.  Today the extra data only slows their propagation when it wasn't well relayed in advance.  So the limit back then benefited block propagation, today it hurts it.

At the time the understanding that delays in propagation don't hurt the miner that added that data as much as they hurt small miners relative to very large miners was less well understood. Eyal&Sirer's paper had only been published a couple months before and it contributed to understanding of communications delay as a factor in centralization pressure, though even today many don't understand it or don't think about it.

At the time, many of us had an understanding that these sorts of limit were unstable equilibrium at best-- and wouldn't stand once miners were being driven by fees, wouldn't stand if there was significant demand that wasn't just confused, ... today we've long since reached that world and found that that understanding was correct.

At the time no one had any reason to be concerned that they might be hauled before congress or even prosecuted because some state blacklist wasn't also a standarness rule or because it wasn't working well enough to block transactions.

The limit was somewhat popularly unpopular at the time and became extraordinarily unpopular later,  I feel a little irony that for merely defending the limit on its merits I was multiply subjected to threats of physical violence not too many years ago.  A common thread underlies much of the opposition to remove it: a knee jerk reaction to some perception of authority, which was largely unaware of or indifferent to the reasoning and the same false smears of conflicts of interest and whatnot have been deployed.  Fortunately for OP_RETURN it's initial addition added something entirely knew so even though it was limited it was more than nothing.  The really vigorous attacks on it came only after it was forgotten that it wasn't previously unlimited.  Similarly, instead of being based on the substance the driver appeared to be parties weaponizing it for other purposes.  Back then as a general means to attack the bitcoin project to promote forks,  this time because of anger about nft/shitcoin traffic which this change actually has little to no bearing on.

I fear at times there is a tendency to look at anything that came before you and assuming that it was some kind of holy solution.  But standardness limits aren't holy, they're a hack... sometimes a very useful hack. They should only exist when their benefits well outweigh their costs. Given the history I don't think it was wrong to limit at the time, but it was a very different world then.