Hey ZmnSCPxj,

As to your first point.  I wasn't aware there was so much volatility at the tip, also 100 blocks is quite the difference!  I agree no one could references a transaction in a newly formed blocks, but I'm curious how this number was chosen. Do you have any documentation or code that you can share related to how re-orgs are handled? Do we have a kind of 'consensus checkpoint' when a re-org is no longer possible? This is a very interesting topic.

 > * It strongly encourages pubkey reuse, reducing privacy.
Privacy-aware users are free to have single-use p2sh transactions, and they are free to use the same SCRIPT opcodes we have now.  Adding an extra opcode helps with the utility of SCRIPT by compressing the smallest SegWit transactions by a further 40% from 233 bytes to 148 bytes.  Cost savings is a great utility - and it need not undermine anyones privacy. The resulting p2sh SCRIPT could end up using public key material that could be compressed with a PubRef - everyone wins.

 > * There is a design-decision wherein a SCRIPT can only access data in the transaction that triggers its execution.
In order for a compression algorithm like LZ78 to be written in a stack-based language like SCRIPT, there needs to be pointer arithmetic to refer back to the dictionary or a larger application state.  If Bitcoin's entire stack was made available to the SCRIPT language as an application state, then LZ78-like compression could be accomplished using PubRef. If a Script can reuse a PUSHDATA, then transactions will be less repetitious...  and this isn't free.  There is a cost in supporting this opcode.

Giving the SCRIPT language access to more data opens the door for interesting algorithms, not just LZ78.  This is interesting to discuss how this application state could be brought to the language.  It strikes me that your concerns(ZmnSCPxj), as well as the topic of pruning brought up by others (including Pieter Wuille) could be fixed by the creation of a side-chain of indexes.  A validator would not need a hash table which is only needed for O(1) PUBREF creation, these nodes need not be burdened with this added index.  A validator only needs an array of PUSHDATA elements and can then validate any given SCRIPT at O(1).  

Just a thought.

Best Regards,
Mike


On Fri, Jul 19, 2019 at 11:08 AM ZmnSCPxj <ZmnSCPxj@protonmail.com> wrote:
Good morning Mike,

> PubRef is not susceptible to malleability attacks because the blockchain is immutable.

This is not quite accurate.
While very old blocks are indeed immutable-in-practice, chain tips are not, and are often replaced.
At least specify that such data can only be referred to if buried under 100 blocks.

--

There are a number of other issues:

* It strongly encourages pubkey reuse, reducing privacy.
* There is a design-decision wherein a SCRIPT can only access data in the transaction that triggers its execution.
  In particular, it cannot access data in the block the transaction is in, or in past blocks.
  For example, `OP_CHECKLOCKTIMEVERIFY` does not check the blockheight of the block that the transaction is confirmed in, but instead checks only `nLockTime`, a field in the transaction.
  * This lets us run SCRIPT in isolation on a transaction, exactly one time, when the transaction is about to be put into our mempool.
    When a new block arrives, transactions in our mempool that are in the block do not need to have their SCRIPTs re-executed or re-validated.

> In order for a client to make use of the PUBREF operations they’ll need access to a database that look up public-keys and resolve their PUBREF index.  A value can be resolved to an index with a hash-table lookup in O(1) constant time. Additionally, all instances of PUSHDATA can be indexed as an ordered list, resolution of a PUBREF index to the intended value would be an O(1) array lookup.  Although the data needed to build and resolve public references is already included with every full node, additional computational effort is needed to build and maintain these indices - a tradeoff which provides smaller transaction sizes and relieving the need to store repetitive data on the blockchain.

This is not only necessary at the creator of the transaction --- it is also necessary at every validator.

In particular, consider existing pruning nodes, which cannot refer to previous block data.

We would need to have another new database containing every `PUSHDATA` in existence.
And existing pruning nodes would need to restart from genesis, as this database would not exist yet.

Regards,
ZmnSCPxj