[bitcoin-dev] Three hardfork-related BIPs

[bitcoin-dev] Three hardfork-related BIPs

[Luke Dashjr]

I've put together three hardfork-related BIPs. This is parallel to the ongoing 
research into the MMHF/SHF WIP BIP, which might still be best long-term.

1) The first is a block size limit protocol change. It also addresses three 
criticisms of segwit: 1) segwit increases the block size limit which is 
already considered by many to be too large; 2) segwit treats pre-segwit 
transactions “unfairly” by giving the witness discount only to segwit 
transactions; and 3) that spam blocks can be larger than blocks mining 
legitimate transactions. This proposal may (depending on activation date) 
initially reduce the block size limit to a more sustainable size in the short-
term, and gradually increase it up over the long-term to 31 MB; it will also 
extend the witness discount to non-segwit transactions. Should the initial 
block size limit reduction prove to be too controversial, miners can simply 
wait to activate it until closer to the point where it becomes acceptable 
and/or increases the limit. However, since the BIP includes a hardfork, the 
eventual block size increase needs community consensus before it can be 
deployed. Proponents of block size increases should note that this BIP does 
not interfere with another more aggressive block size increase hardfork in the 
meantime. I believe I can immediately recommend this for adoption; however, 
peer and community review are welcome to suggest changes.
Text: https://github.com/luke-jr/bips/blob/bip-blksize/bip-blksize.mediawiki
Code: https://github.com/bitcoin/bitcoin/compare/master...luke-jr:bip-blksize 
(consensus code changes only)

2) The second is a *preparatory* change, that should allow trivially 
transforming certain classes of hardforks into softforks in the future. It 
essentially says that full nodes should relax their rule enforcement, after 
sufficient time that would virtually guarantee they have ceased to be 
enforcing the full set of rules anyway. This allows these relaxed rules to be 
modified or removed in a softfork, provided the proposal to do so is accepted 
and implemented with enough advance notice. Attempting to implement this has 
proven more complicated than I originally expected, and it may make more sense 
for full nodes to simply stop functioning (with a user override) after the 
cut-off date). In light of this, I do not yet recommend its adoption, but am 
posting it for review and comments only.
Text: https://github.com/luke-jr/bips/blob/bip-hfprep/bip-hfprep.mediawiki

3) Third is an anti-replay softfork which can be used to prevent replay 
attacks whether induced by a hardfork-related chain split, or even in ordinary 
operation. It does this by using a new opcode (OP_CHECKBLOCKATHEIGHT) for the 
Bitcoin scripting system that allows construction of transactions which are 
valid only on specific blockchains.
Text: https://github.com/luke-jr/bips/blob/bip-noreplay/bip-noreplay.mediawiki

Luke

Re: [bitcoin-dev] Three hardfork-related BIPs

[Andrew Johnson]

[-- Attachment #1: Type: text/plain, Size: 3723 bytes --]

Comment on #1.  You're dropping the blocksize limit to 300KB and only
reaching the limit that we have in place today 7 years later?  We're
already at capacity today, surely you're not serious with this proposal?
When you promised code for a hard forking block size increase in the HK
agreement I don't believe that a decrease first was made apparent.  While
not technically in violation of the letter of the agreement, I think this
is a pretty obviously not in the spirit of it.

On Jan 26, 2017 7:07 PM, "Luke Dashjr via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org> wrote:

I've put together three hardfork-related BIPs. This is parallel to the
ongoing
research into the MMHF/SHF WIP BIP, which might still be best long-term.

1) The first is a block size limit protocol change. It also addresses three
criticisms of segwit: 1) segwit increases the block size limit which is
already considered by many to be too large; 2) segwit treats pre-segwit
transactions “unfairly” by giving the witness discount only to segwit
transactions; and 3) that spam blocks can be larger than blocks mining
legitimate transactions. This proposal may (depending on activation date)
initially reduce the block size limit to a more sustainable size in the
short-
term, and gradually increase it up over the long-term to 31 MB; it will also
extend the witness discount to non-segwit transactions. Should the initial
block size limit reduction prove to be too controversial, miners can simply
wait to activate it until closer to the point where it becomes acceptable
and/or increases the limit. However, since the BIP includes a hardfork, the
eventual block size increase needs community consensus before it can be
deployed. Proponents of block size increases should note that this BIP does
not interfere with another more aggressive block size increase hardfork in
the
meantime. I believe I can immediately recommend this for adoption; however,
peer and community review are welcome to suggest changes.
Text: https://github.com/luke-jr/bips/blob/bip-blksize/bip-blksize.mediawiki
Code: https://github.com/bitcoin/bitcoin/compare/master...luke-
jr:bip-blksize
(consensus code changes only)

2) The second is a *preparatory* change, that should allow trivially
transforming certain classes of hardforks into softforks in the future. It
essentially says that full nodes should relax their rule enforcement, after
sufficient time that would virtually guarantee they have ceased to be
enforcing the full set of rules anyway. This allows these relaxed rules to
be
modified or removed in a softfork, provided the proposal to do so is
accepted
and implemented with enough advance notice. Attempting to implement this has
proven more complicated than I originally expected, and it may make more
sense
for full nodes to simply stop functioning (with a user override) after the
cut-off date). In light of this, I do not yet recommend its adoption, but am
posting it for review and comments only.
Text: https://github.com/luke-jr/bips/blob/bip-hfprep/bip-hfprep.mediawiki

3) Third is an anti-replay softfork which can be used to prevent replay
attacks whether induced by a hardfork-related chain split, or even in
ordinary
operation. It does this by using a new opcode (OP_CHECKBLOCKATHEIGHT) for
the
Bitcoin scripting system that allows construction of transactions which are
valid only on specific blockchains.
Text: https://github.com/luke-jr/bips/blob/bip-noreplay/bip-
noreplay.mediawiki

Luke
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 4963 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Luke Dashjr]

On Friday, January 27, 2017 3:04:50 AM Andrew Johnson wrote:
> Comment on #1.  You're dropping the blocksize limit to 300KB and only
> reaching the limit that we have in place today 7 years later?

The limit only drops all the way to 300k if it activates before 2017 April. 
Considering that this requires the consensus of a hardfork, followed by a 
release in software, and then actual activation by miners using BIP9, I think 
it's extremely unlikely to activate by then.

But more importantly: such a drop would probably be good for the network in 
the long-term. As explained in the Rationale section, 300k is necessary to 
maintain our *current* IBD (first-time node sync) costs even with 
technological improvements (which appear to be slowing lately).

> We're already at capacity today, surely you're not serious with this
> proposal?

We are only at capacity because the space is available below actual costs, 
and/or because efficient alternatives are not yet widely supported. A 
reduction of block size will likely squeeze out spam, and perhaps some 
unsustainable microtransaction use, but the volume which actually *benefits 
from* the blockchain's security should continue along fine. Furthermore, once 
Lightning is widely implemented as well-tested, at least microtransactions are 
likely to gain a huge improvement in efficiency, reducing legitimate usage of 
block sizes well below 300k naturally - that is frankly when I first expect 
this proposal to be seriously considered for activation (which is independent 
from the consensus to include support for it in nodes).

> When you promised code for a hard forking block size increase in the HK
> agreement I don't believe that a decrease first was made apparent.  While
> not technically in violation of the letter of the agreement, I think this
> is a pretty obviously not in the spirit of it.

I did not mention the HK "roundtable", because this is indeed not in the 
spirit of what we set out to do, and do not wish this to be interpreted as 
some kind of slap in the face of the honest participants of that discussion.

This proposal is, however, the best I am currently able to honestly recommend 
that meets the hard criteria outlined at Hong Kong a year ago. (Continued work 
on the MMHF/SHF concept may eventually deliver a better solution, but it is 
not yet ready.)

Luke

Re: [bitcoin-dev] Three hardfork-related BIPs

[Johnson Lau]

[-- Attachment #1: Type: text/plain, Size: 6182 bytes --]

I can’t recommend your first 2 proposals. But I only have the time to talk about the first one for now.

There are 2 different views on this topic:

1. “The block size is too small and people can’t buy a coffee with an on-chain transaction. Let’s just remove the limit”

2. “The block size is too big and people can’t run full nodes or do initial blockchain download (IBD). Let’s just reduce the limit”

For me, both approaches just show the lack of creativity, and lack of responsibility. Both just try to solve one problem, disregarding all the other consequences.

The 1MB is here, no matter you like it or not, it’s the current consensus. Any attempts to change this limit (up or down) require wide consensus of the whole community, which might be difficult.

Yes, I agree with you that the current 1MB block size is already too big for many people to run a full node. That’s bad, but it doesn’t mean we have no options other than reducing the block size. Just to cite some:

1. Blockchain pruning is already available, so the storage of blockchain is already an O(1) problem. The block size is not that important for this part
2. UTXO size is an O(n) problem, but we could limit its growth without limit the block size, by charging more for UTXO creation, and offer incentive for UTXO spending  **
3. For non-mining full node, latency is not critical. 1MB per 10 minutes is not a problem unless with mobile network. But I don’t think mobile network is ever considered as a suitable way for running a full node
4. For mining nodes, we already have compact block and xthin block, and FIBRE
5. For IBD, reducing the size won’t help much as it is already too big for many people. The right way to solve the IBD issue is to implement long latency UTXO commitment. Nodes will calculate a UTXO commitment every 1000 block, and commit to the UTXO status of the previous 1000 block (e.g. block 11000 will commit to the UTXO of block 10000). This is a background process and the overhead is negligible. When such commitments are confirmed for sufficiently long (e.g. 1 year), people will assume it is correct, and start IBD from that point by downloading UTXO from some untrusted sources. That will drastically reduce the time for IBD
6. No matter we change the block size limit or not, we need to implement a fraud-proof system to allow probabilistic validation by SPV nodes. So even a smartphone may validate 0.1% of the blockchain, and with many people using phone wallet, it will only be a net gain to the network security 

For points 2 and 6 above, I have some idea implemented in my experimental hardfork.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/013472.html <https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/013472.html>


> On 27 Jan 2017, at 09:06, Luke Dashjr via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
> I've put together three hardfork-related BIPs. This is parallel to the ongoing 
> research into the MMHF/SHF WIP BIP, which might still be best long-term.
> 
> 1) The first is a block size limit protocol change. It also addresses three 
> criticisms of segwit: 1) segwit increases the block size limit which is 
> already considered by many to be too large; 2) segwit treats pre-segwit 
> transactions “unfairly” by giving the witness discount only to segwit 
> transactions; and 3) that spam blocks can be larger than blocks mining 
> legitimate transactions. This proposal may (depending on activation date) 
> initially reduce the block size limit to a more sustainable size in the short-
> term, and gradually increase it up over the long-term to 31 MB; it will also 
> extend the witness discount to non-segwit transactions. Should the initial 
> block size limit reduction prove to be too controversial, miners can simply 
> wait to activate it until closer to the point where it becomes acceptable 
> and/or increases the limit. However, since the BIP includes a hardfork, the 
> eventual block size increase needs community consensus before it can be 
> deployed. Proponents of block size increases should note that this BIP does 
> not interfere with another more aggressive block size increase hardfork in the 
> meantime. I believe I can immediately recommend this for adoption; however, 
> peer and community review are welcome to suggest changes.
> Text: https://github.com/luke-jr/bips/blob/bip-blksize/bip-blksize.mediawiki
> Code: https://github.com/bitcoin/bitcoin/compare/master...luke-jr:bip-blksize 
> (consensus code changes only)
> 
> 2) The second is a *preparatory* change, that should allow trivially 
> transforming certain classes of hardforks into softforks in the future. It 
> essentially says that full nodes should relax their rule enforcement, after 
> sufficient time that would virtually guarantee they have ceased to be 
> enforcing the full set of rules anyway. This allows these relaxed rules to be 
> modified or removed in a softfork, provided the proposal to do so is accepted 
> and implemented with enough advance notice. Attempting to implement this has 
> proven more complicated than I originally expected, and it may make more sense 
> for full nodes to simply stop functioning (with a user override) after the 
> cut-off date). In light of this, I do not yet recommend its adoption, but am 
> posting it for review and comments only.
> Text: https://github.com/luke-jr/bips/blob/bip-hfprep/bip-hfprep.mediawiki
> 
> 3) Third is an anti-replay softfork which can be used to prevent replay 
> attacks whether induced by a hardfork-related chain split, or even in ordinary 
> operation. It does this by using a new opcode (OP_CHECKBLOCKATHEIGHT) for the 
> Bitcoin scripting system that allows construction of transactions which are 
> valid only on specific blockchains.
> Text: https://github.com/luke-jr/bips/blob/bip-noreplay/bip-noreplay.mediawiki
> 
> Luke
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


[-- Attachment #2: Type: text/html, Size: 8101 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Andrew Johnson]

[-- Attachment #1: Type: text/plain, Size: 3283 bytes --]

On Jan 26, 2017 10:15 PM, "Luke Dashjr" <luke@dashjr.org> wrote:

On Friday, January 27, 2017 3:04:50 AM Andrew Johnson wrote:
> Comment on #1.  You're dropping the blocksize limit to 300KB and only
> reaching the limit that we have in place today 7 years later?

The limit only drops all the way to 300k if it activates before 2017 April.
Considering that this requires the consensus of a hardfork, followed by a
release in software, and then actual activation by miners using BIP9, I
think
it's extremely unlikely to activate by then.

But more importantly: such a drop would probably be good for the network in
the long-term. As explained in the Rationale section, 300k is necessary to
maintain our *current* IBD (first-time node sync) costs even with
technological improvements (which appear to be slowing lately).


Other researchers have come to the conservative conclusion that we could
handle 4MB blocks today.  Imagine bitcoin had been invented in 1987 and had
a block size correspondent to the internet connections and hard drive sizes
of the day.  Your proposal would have probably brought us from 1Kb(then
reduced to 300 bytes) and up to a whopping 20Kb or so today.  Yet even you
think we can handle 15x that today.

You drastically underestimate the speed of technological progression, and
seem to fancy yourself the central planner of bitcoin.  Isn't that one of
the things we're trying to get away from, centrally planned economics?


> We're already at capacity today, surely you're not serious with this
> proposal?

We are only at capacity because the space is available below actual costs,
and/or because efficient alternatives are not yet widely supported. A
reduction of block size will likely squeeze out spam, and perhaps some
unsustainable microtransaction use, but the volume which actually *benefits
from* the blockchain's security should continue along fine. Furthermore,
once
Lightning is widely implemented as well-tested, at least microtransactions
are
likely to gain a huge improvement in efficiency, reducing legitimate usage
of
block sizes well below 300k naturally - that is frankly when I first expect
this proposal to be seriously considered for activation (which is
independent
from the consensus to include support for it in nodes).


Legitimate usage is a transaction that pays the appropriate fee to be
included.  The term legitimate transaction should be stricken from one's
vocabulary when describing a censorship resistant system such as bitcoin.


> When you promised code for a hard forking block size increase in the HK
> agreement I don't believe that a decrease first was made apparent.  While
> not technically in violation of the letter of the agreement, I think this
> is a pretty obviously not in the spirit of it.

I did not mention the HK "roundtable", because this is indeed not in the
spirit of what we set out to do, and do not wish this to be interpreted as
some kind of slap in the face of the honest participants of that discussion.


Too late for that, I suspect.


This proposal is, however, the best I am currently able to honestly
recommend
that meets the hard criteria outlined at Hong Kong a year ago. (Continued
work
on the MMHF/SHF concept may eventually deliver a better solution, but it is
not yet ready.)

Luke

[-- Attachment #2: Type: text/html, Size: 4764 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[t. khan]

[-- Attachment #1: Type: text/plain, Size: 5169 bytes --]

Regarding #1, I agree with Johnson Lau and others who have responded since
then—this proposal is not appropriate and should not be adopted for the
following reasons:

1. Miners will view it as way too little, delivered way too late. And as
soon as you say 300kb blocks, you've lost them all.

2. "Spam" - You're very fixated on this concept of spam transactions, but
the transactions that you deem as spam are legitimate, fee-paying
transactions. They're not a problem for miners. It's only a problem to you
as you've arbitrarily decided some transactions are legit and some are not.
It's an imaginary problem and we should focus on designs that solve real
problems instead.

Also, even if you changed the max size to 300kb, transactions that you (and
as far as I can tell, only you) consider spam will still be in there!
They'll just be paying a ridiculous fee along with everyone else.

3. 17% per year growth rate - This is making the assumption that the
current 1MB limit is already at the upper limit supportable by the network.
This isn't even remotely true, and starting this rate at the current limit
would cause the system to lag far behind the actual capability of the
network for no reason.

4. Nodes - Individuals have no incentive to run full nodes and we've
already passed the time where it makes any sense for them to do so.
Therefore restricting the blockchain size in an attempt to keep individuals
running nodes is futile at best and likely very damaging. Miners and
businesses using Bitcoin do have an incentive to run nodes and over the
years we've seen a migration of nodes from weak hands (individuals) to
strong hands (businesses).

Overall, this proposal would hamstring Bitcoin Core and would drive miners
towards Unlimited.

- t.k.

On Thu, Jan 26, 2017 at 8:06 PM, Luke Dashjr via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> I've put together three hardfork-related BIPs. This is parallel to the
> ongoing
> research into the MMHF/SHF WIP BIP, which might still be best long-term.
>
> 1) The first is a block size limit protocol change. It also addresses three
> criticisms of segwit: 1) segwit increases the block size limit which is
> already considered by many to be too large; 2) segwit treats pre-segwit
> transactions “unfairly” by giving the witness discount only to segwit
> transactions; and 3) that spam blocks can be larger than blocks mining
> legitimate transactions. This proposal may (depending on activation date)
> initially reduce the block size limit to a more sustainable size in the
> short-
> term, and gradually increase it up over the long-term to 31 MB; it will
> also
> extend the witness discount to non-segwit transactions. Should the initial
> block size limit reduction prove to be too controversial, miners can simply
> wait to activate it until closer to the point where it becomes acceptable
> and/or increases the limit. However, since the BIP includes a hardfork, the
> eventual block size increase needs community consensus before it can be
> deployed. Proponents of block size increases should note that this BIP does
> not interfere with another more aggressive block size increase hardfork in
> the
> meantime. I believe I can immediately recommend this for adoption; however,
> peer and community review are welcome to suggest changes.
> Text: https://github.com/luke-jr/bips/blob/bip-blksize/bip-
> blksize.mediawiki
> Code: https://github.com/bitcoin/bitcoin/compare/master...luke-
> jr:bip-blksize
> (consensus code changes only)
>
> 2) The second is a *preparatory* change, that should allow trivially
> transforming certain classes of hardforks into softforks in the future. It
> essentially says that full nodes should relax their rule enforcement, after
> sufficient time that would virtually guarantee they have ceased to be
> enforcing the full set of rules anyway. This allows these relaxed rules to
> be
> modified or removed in a softfork, provided the proposal to do so is
> accepted
> and implemented with enough advance notice. Attempting to implement this
> has
> proven more complicated than I originally expected, and it may make more
> sense
> for full nodes to simply stop functioning (with a user override) after the
> cut-off date). In light of this, I do not yet recommend its adoption, but
> am
> posting it for review and comments only.
> Text: https://github.com/luke-jr/bips/blob/bip-hfprep/bip-hfprep.mediawiki
>
> 3) Third is an anti-replay softfork which can be used to prevent replay
> attacks whether induced by a hardfork-related chain split, or even in
> ordinary
> operation. It does this by using a new opcode (OP_CHECKBLOCKATHEIGHT) for
> the
> Bitcoin scripting system that allows construction of transactions which are
> valid only on specific blockchains.
> Text: https://github.com/luke-jr/bips/blob/bip-noreplay/bip-
> noreplay.mediawiki
>
> Luke
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 6418 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Russell O'Connor]

[-- Attachment #1: Type: text/plain, Size: 718 bytes --]

On Jan 27, 2017 03:03, "Andrew Johnson via bitcoin-dev" <bitcoin-dev@lists.
linuxfoundation.org> wrote:

Other researchers have come to the conservative conclusion that we could
handle 4MB blocks today.


I believe this is a mischaracterization of the research conclusions.  The
actual conclusion was that the maximum value for the blocksize that the
network can safely handle (at that time) is some value that is
(conservatively) no more than 4MB.  This is because the research only
studies one aspect of the effect of blocksize on the network at a time and
the true safe value is the minimum of all aspects.  For example, the 4MB
doesn't cover the aspect of quadratic hashing for large transactions in
large blocks.

[-- Attachment #2: Type: text/html, Size: 1187 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Greg Sanders]

[-- Attachment #1: Type: text/plain, Size: 1774 bytes --]

Note that the 4MB number comes from a single network metric.

Quotes directly from the paper in question:
http://fc16.ifca.ai/bitcoin/papers/CDE+16.pdf

>Our results hinge on the key metric of effective throughput in the overlay
network, which we define here as which blocks propagate within an average
block interval period the percentage of nodes to.
...
>Note that as we consider only a subset of possible metrics (due to
difficulty in accurately measuring others), our results on
reparametrization may be viewed as upper bounds: additional metrics could
reveal even stricter limits.

It says nothing about any mining centralization pressure, DoS attacks, etc.
A single metric among many we have to contend with.






On Fri, Jan 27, 2017 at 3:34 PM, Russell O'Connor via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

>
>
> On Jan 27, 2017 03:03, "Andrew Johnson via bitcoin-dev" <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> Other researchers have come to the conservative conclusion that we could
> handle 4MB blocks today.
>
>
> I believe this is a mischaracterization of the research conclusions.  The
> actual conclusion was that the maximum value for the blocksize that the
> network can safely handle (at that time) is some value that is
> (conservatively) no more than 4MB.  This is because the research only
> studies one aspect of the effect of blocksize on the network at a time and
> the true safe value is the minimum of all aspects.  For example, the 4MB
> doesn't cover the aspect of quadratic hashing for large transactions in
> large blocks.
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 3008 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Christian Decker]

On Fri, Jan 27, 2017 at 03:47:20PM -0500, Greg Sanders via bitcoin-dev wrote:
> Note that the 4MB number comes from a single network metric.
> 
> Quotes directly from the paper in question:
> http://fc16.ifca.ai/bitcoin/papers/CDE+16.pdf
> 
> >Our results hinge on the key metric of effective throughput in the overlay
> network, which we define here as which blocks propagate within an average
> block interval period the percentage of nodes to.
> ...
> >Note that as we consider only a subset of possible metrics (due to
> difficulty in accurately measuring others), our results on
> reparametrization may be viewed as upper bounds: additional metrics could
> reveal even stricter limits.
> 
> It says nothing about any mining centralization pressure, DoS attacks, etc.
> A single metric among many we have to contend with.
>

As one of the authors of that paper and the source of the measurement
data I'd also like to point out that the 4MB number is indeed intended
as an optimistic upper bound on todays network capacity.

More importantly it's not a black and white situation, where there is
a magic number beyond which Bad Things (TM) happen, it's a spectrum on
which we can see a few threshold beyond which we _know_ Bad Things
definitely happen. Miner centralization pressure is felt earlier.

Re: [bitcoin-dev] Three hardfork-related BIPs

[Andrew Johnson]

[-- Attachment #1: Type: text/plain, Size: 3028 bytes --]

Thanks for replying, I'd be interested to see what you would come up with
today using the same methodology, seeing as max single hard drive capacity
has roughly doubled, global average internet bandwidth has increased 31%
from 4.8Mbps to 6.3Mbps(sourced from Akamai State of the Internet reports
2014q4 and 2016q3), and we now have xThin and compact blocks to help
significantly with block propagation time.  Not to mention the usual
improvements in CPUs(not that we're anywhere near a CPU bottleneck today
anyway save for quadratic hashing when raising the blocksize, but I don't
think that anyone would seriously suggest an increase without addressing
that).

I don't think that the 17% yearly increase is too far off base considering
current global trends(although I still don't particularly like the idea of
centrally planning the limit, especially not that far into the future), but
the 66% decrease first seems completely out of touch with reality.

I'd also like to point out to Luke that Satoshi envisioned most full nodes
running in data centers in the white paper, not every single user needs to
run a full node to use bitcoin.  Not to present this as an argument from
authority, but rather to remind us what the intention of the system was to
be(p2p cash, not a settlement layer only afforded by the wealthiest and
largest value transactions).  That a lot of people want to continue to move
in that direction shouldn't be a surprise.

On Jan 27, 2017 3:28 PM, "Christian Decker via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org> wrote:

On Fri, Jan 27, 2017 at 03:47:20PM -0500, Greg Sanders via bitcoin-dev
wrote:
> Note that the 4MB number comes from a single network metric.
>
> Quotes directly from the paper in question:
> http://fc16.ifca.ai/bitcoin/papers/CDE+16.pdf
>
> >Our results hinge on the key metric of effective throughput in the
overlay
> network, which we define here as which blocks propagate within an average
> block interval period the percentage of nodes to.
> ...
> >Note that as we consider only a subset of possible metrics (due to
> difficulty in accurately measuring others), our results on
> reparametrization may be viewed as upper bounds: additional metrics could
> reveal even stricter limits.
>
> It says nothing about any mining centralization pressure, DoS attacks,
etc.
> A single metric among many we have to contend with.
>

As one of the authors of that paper and the source of the measurement
data I'd also like to point out that the 4MB number is indeed intended
as an optimistic upper bound on todays network capacity.

More importantly it's not a black and white situation, where there is
a magic number beyond which Bad Things (TM) happen, it's a spectrum on
which we can see a few threshold beyond which we _know_ Bad Things
definitely happen. Miner centralization pressure is felt earlier.
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 4064 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Luke Dashjr]

On Friday, January 27, 2017 11:53:02 PM Andrew Johnson via bitcoin-dev wrote:
> I don't think that the 17% yearly increase is too far off base considering
> current global trends(although I still don't particularly like the idea of
> centrally planning the limit, especially not that far into the future), but
> the 66% decrease first seems completely out of touch with reality.

Assume as a premise (despite your apparent disagreement below) that for 
Bitcoin to function, a supermajority of economic activity needs to be verified 
using full nodes operated by the recipient. Evidence suggests that at this 
current time, at best 10% of economic activity is in fact using a full node to 
verify the transaction. On this basis, it seems pretty clear that serious 
action must be taken to change the status quo, and so for efforts to do so 
without dropping the block size have proven ineffective.

> I'd also like to point out to Luke that Satoshi envisioned most full nodes
> running in data centers in the white paper, not every single user needs to
> run a full node to use bitcoin.

Satoshi envisioned a system where full nodes could publish proofs of invalid 
blocks that would be automatically verified by SPV nodes and used to ensure 
even they maintained the equivalent of full node security so long as they were 
not isolated. But as a matter of fact, this vision has proven impossible, and 
there is to date no viable theory on how it might be fixed. As a result, the 
only way for nodes to have full-node-security is to actually be a true full 
node, and therefore the plan of only having full nodes in datacenters is 
simply not realistic without transforming Bitcoin into a centralised system.

> That a lot of people want to continue to move in that direction shouldn't
> be a surprise.

I think it's likely safe to say that if this were a possibility, everyone 
would want to continue to move in that direction. But as the facts stand, it 
simply isn't possible.

Luke

Re: [bitcoin-dev] Three hardfork-related BIPs

[Natanael]

[-- Attachment #1: Type: text/plain, Size: 1413 bytes --]

Den 28 jan. 2017 05:04 skrev "Luke Dashjr via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org>:

Satoshi envisioned a system where full nodes could publish proofs of invalid
blocks that would be automatically verified by SPV nodes and used to ensure
even they maintained the equivalent of full node security so long as they
were
not isolated. But as a matter of fact, this vision has proven impossible,
and
there is to date no viable theory on how it might be fixed. As a result, the
only way for nodes to have full-node-security is to actually be a true full
node, and therefore the plan of only having full nodes in datacenters is
simply not realistic without transforming Bitcoin into a centralised system.


Beside Zero-knowledge proofs, which is capable of proving much so more than
just validity, there are multi types of fraud proofs that only rely on the
format of the blocks. Such as publishing the block header + the two
colliding transactions included in it (in the case of double spending), or
if the syntax or logic is broken then you just publish that single
transaction.

There aren't all  that many cases where fraud proofs are unreasonably large
for a networked system like in Bitcoin. If Zero-knowledge proofs can be
applied securely, then I can't think of any exceptions at all for when the
proofs would be unmanageable. (Remember that full Zero-knowledge proofs can
be chained together!)

[-- Attachment #2: Type: text/html, Size: 2091 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1433 bytes --]



On 27 January 2017 15:53:02 GMT-08:00, Andrew Johnson via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>I'd also like to point out to Luke that Satoshi envisioned most full
>nodes
>running in data centers in the white paper, not every single user needs
>to
>run a full node to use bitcoin.  Not to present this as an argument
>from
>authority, but rather to remind us what the intention of the system was
>to
>be(p2p cash, not a settlement layer only afforded by the wealthiest and
>largest value transactions).  That a lot of people want to continue to
>move
>in that direction shouldn't be a surprise.

Satoshi also thought that SPV clients would be able to use fraud proofs (called "alerts" in the white paper) to detect fraudulent behavior by miners, and thus not have to completely trust those nodes in those datacenters. Unfortunately it turns out that fraud proofs are both a very difficult engineering challenge to implement, and also offer much less security than once thought. In fact, as per Satoshi's vision, SPV clients don't currently exist; what's called SPV isn't what Satoshi was envisioning.

Of course, this wouldn't be the first time that aspects of Satoshi's vision for Bitcoin turned out to be wrong: the white paper also refers to the "longest chain" rather than most-work chain, something that had to be fixed in what's technically a hardfork after Bitcoin's initial release.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 500 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2116 bytes --]



On 28 January 2017 02:36:16 GMT-08:00, Natanael via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>Den 28 jan. 2017 05:04 skrev "Luke Dashjr via bitcoin-dev" <
>bitcoin-dev@lists.linuxfoundation.org>:
>
>Satoshi envisioned a system where full nodes could publish proofs of
>invalid
>blocks that would be automatically verified by SPV nodes and used to
>ensure
>even they maintained the equivalent of full node security so long as
>they
>were
>not isolated. But as a matter of fact, this vision has proven
>impossible,
>and
>there is to date no viable theory on how it might be fixed. As a
>result, the
>only way for nodes to have full-node-security is to actually be a true
>full
>node, and therefore the plan of only having full nodes in datacenters
>is
>simply not realistic without transforming Bitcoin into a centralised
>system.
>
>
>Beside Zero-knowledge proofs, which is capable of proving much so more
>than
>just validity, there are multi types of fraud proofs that only rely on
>the
>format of the blocks. Such as publishing the block header + the two
>colliding transactions included in it (in the case of double spending),
>or
>if the syntax or logic is broken then you just publish that single
>transaction.

That's a perfect example of why fraud proofs aren't as secure as expected: the miner who created such a block wouldn't even give you the data necessary to prove the fraud in the first place.

What you actually need are validity challenges, where someone makes a challenge claiming that part of the block is invalid. A failure to meet the challenge with proof that the rules are followed is considered defacto evidence of fraud.

But validity challenges don't scale well and pose DoS attacks issues; it's far from clear that they can be implemented in a useful way. Even if validity challenges work, they also don't solve censorship: a world of nodes in large datacenters is a world where it's very easy to force the few Bitcoin nodes remaining to follow AML/KYC rules for instance, a risk we wouldn't be able to mitigate with a PoW change.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 500 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Luke Dashjr]

On Saturday, January 28, 2017 10:36:16 AM Natanael wrote:
> Den 28 jan. 2017 05:04 skrev "Luke Dashjr via bitcoin-dev" <
> bitcoin-dev@lists.linuxfoundation.org>:
> > Satoshi envisioned a system where full nodes could publish proofs of
> > invalid blocks that would be automatically verified by SPV nodes and used
> > to ensure even they maintained the equivalent of full node security so
> > long as they were not isolated. But as a matter of fact, this vision has
> > proven impossible, and there is to date no viable theory on how it might
> > be fixed. As a result, the only way for nodes to have full-node-security
> > is to actually be a true full node, and therefore the plan of only having
> > full nodes in datacenters is simply not realistic without transforming
> > Bitcoin into a centralised system.
> 
> Beside Zero-knowledge proofs, which is capable of proving much so more than
> just validity, there are multi types of fraud proofs that only rely on the
> format of the blocks. Such as publishing the block header + the two
> colliding transactions included in it (in the case of double spending), or
> if the syntax or logic is broken then you just publish that single
> transaction.

Why would someone malicious follow the format sufficiently to make those 
proofs possible?

> There aren't all  that many cases where fraud proofs are unreasonably large
> for a networked system like in Bitcoin. If Zero-knowledge proofs can be
> applied securely, then I can't think of any exceptions at all for when the
> proofs would be unmanageable. (Remember that full Zero-knowledge proofs can
> be chained together!)

ZK proofs do to a large extent simplify the situation, but still fail in one 
case (and one case is all an attacker needs, since he can choose how he 
attacks). Specifically, the attacker can create a block which is 100% well-
formed and valid (or not - nobody will really ever know), and simply withhold 
a single transaction in it, such that nobody ever has the complete block's 
data. Full nodes will of course not accept such a block until they have the 
complete data to validate, but they similarly cannot prove it is invalid 
without the complete data, and any non-full nodes cannot prove there is data 
missing without fetching and (to an extent) checking the entire block 
themselves.

Luke

Re: [bitcoin-dev] Three hardfork-related BIPs

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2795 bytes --]

On Sat, Jan 28, 2017 at 07:43:48PM +0000, Luke Dashjr via bitcoin-dev wrote:
> On Saturday, January 28, 2017 10:36:16 AM Natanael wrote:
> > There aren't all  that many cases where fraud proofs are unreasonably large
> > for a networked system like in Bitcoin. If Zero-knowledge proofs can be
> > applied securely, then I can't think of any exceptions at all for when the
> > proofs would be unmanageable. (Remember that full Zero-knowledge proofs can
> > be chained together!)
> 
> ZK proofs do to a large extent simplify the situation, but still fail in one 
> case (and one case is all an attacker needs, since he can choose how he 
> attacks). Specifically, the attacker can create a block which is 100% well-
> formed and valid (or not - nobody will really ever know), and simply withhold 
> a single transaction in it, such that nobody ever has the complete block's 
> data. Full nodes will of course not accept such a block until they have the 
> complete data to validate, but they similarly cannot prove it is invalid 
> without the complete data, and any non-full nodes cannot prove there is data 
> missing without fetching and (to an extent) checking the entire block 
> themselves.

So, in that particular type of case, the ZK proof may show that the block
itself is valid and follows all the rules; there'd be no need to get the block
data to know that.

The issue here is other miners being able to mine. Exactly what happens here
depends on the exact construction of the ZK proofs, but at best the missing
data will mean that part of the UTXO state can no longer be updated by other
miners, and thus they can't mine all transactions; at worst they'd be
completely preventing from mining at all.

This is why part of the economic pressure that users exert on miners is
subverted by SPV/lite-clients: users that can transact without sufficient
blockchain data to allow others to mine aren't exerting pressure on miners to
allow other miners to mine - particularly new entrants to mining. In that
respect, ZK proofs are in fact quite harmful to the security of the system if
applied naively.

Equally, I'll point out that if ZK proofs can be made sufficiently powerful to
do all the above, genuinely scalable sharded systems like my own Treechains are
far easier to implement, changing the discussion entirely. Currently it is far
from proven that ZK proofs can in fact accomplish this; I hear that Zcash will
soon have to upgrade their ZK-SNARK scheme due to advances in cryptographic
analysis that may result in a full system break in the near future. We really
don't want to be depending on that technology for Bitcoin's security until
events like that become much less common.

-- 
https://petertodd.org 'peter'[:-1]@petertodd.org

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Tom Harding]

On 1/28/2017 10:29 AM, Peter Todd via bitcoin-dev wrote:
> a world of nodes in large datacenters is a world where it's very easy
> to force the few Bitcoin nodes remaining to follow AML/KYC rules

If that's true, why haven't we already seen AML/KYC required of mining
pools?  That would be comparatively trivial.

Re: [bitcoin-dev] Three hardfork-related BIPs

[Eric Voskuil]

> On Jan 29, 2017, at 11:15 AM, Tom Harding via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
>> On 1/28/2017 10:29 AM, Peter Todd via bitcoin-dev wrote:
>> a world of nodes in large datacenters is a world where it's very easy
>> to force the few Bitcoin nodes remaining to follow AML/KYC rules
> 
> If that's true, why haven't we already seen AML/KYC required of mining
> pools?  That would be comparatively trivial.

It is true, there is no question. The fact that an attack does not appear to have occurred does not mean that the vulnerability exists. It is as you say a trivial exploit, which means it will happen when the economic incentive is great enough. Analogous attacks on other points of centralization are already well underway.

e

Re: [bitcoin-dev] Three hardfork-related BIPs

[David Vorick]

[-- Attachment #1: Type: text/plain, Size: 975 bytes --]

On Jan 29, 2017 2:28 PM, "Tom Harding via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org> wrote:

If that's true, why haven't we already seen AML/KYC required of mining
pools?  That would be comparatively trivial.



Some regulators are already looking into it. Even at this point you'd
either need multinational cooperation or you'd need China to decide that
51% attacking a budding technology is a good thing to do, something that
would be sure to increase tensions across the world.

But there are two bigger reasons. The first is that regulators are used to
doing regulation at exchange points, regulating mining is new and
unfamiliar and requires a decent understanding of blockchains. And the
second is that Bitcoin is tiny potatoes at this point. To the best of my
knowledge, organized crime outside of DNMs doesn't use Bitcoin. There's
minimal reason to target it while it's so small.

Regulated mining I believe is going to be a genuine risk as Bitcoin grows.

[-- Attachment #2: Type: text/html, Size: 1509 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[mbtc-dev]

On 27.01.2017 20:03, Luke Dashjr via bitcoin-dev wrote:
> Assume as a premise (despite your apparent disagreement below) that for
> Bitcoin to function, a supermajority of economic activity needs to be 
> verified
> using full nodes operated by the recipient. Evidence suggests that at 
> this
> current time, at best 10% of economic activity is in fact using a full 
> node to
> verify the transaction. On this basis, it seems pretty clear that 
> serious
> action must be taken to change the status quo, and so for efforts to do 
> so
> without dropping the block size have proven ineffective.
> 
Lets think like people in sales and marketing for a moment.

There's an implicit assumption here that ANY protocol or consensus-rule 
based solution exists that would reverse the trend of diminishing full 
node verified economic activity. Since there's no economic advantage to 
running a full node, there's no inherent motivation for implementation 
(or outright purchase) of full nodes by the very large percentage of 
people who fall in the non-technical "I just want it to work, and I 
don't want my money stolen" category. Yes, anyone on this list 
understands that "don't want my money stolen" is inherently connected to 
running your own node and using it for transactions, but the average 
user does not, and even if they did, they don't have the resources (time 
and/or money) to do anything about it. Running your own full node 
increases the protection agains double spend attacks and other protocol 
bases shenanigans, but now you've taken on another set of security 
exposures related to the physical box that is running the node. 
Anti-virus, off and on-site backups, multiple boxes/devices for 
multi-sig, backup of key seeds.

Reducing (or even maintaining) the block size doesn't somehow increase 
the number of people who are capable of running full nodes, and it 
doesn't add any incentive for people already in that "capable" set to 
suddenly take up the task of running and transacting via a full node. 
I'd argue that the size of the block-chain and the time to download it 
are the least concerning aspects to anyone faced with running their own 
node and actually storing some of their wealth on it and using it for 
transactions.

You're looking for a (maybe dangerous/maybe impossible) balance between 
choking off casual (not full node) usage of bitcoin and yet trying to 
make it more popular among the people (and organizations) who have the 
capability and resources to run and transact on full nodes.

We should sit with this for a moment.

On one hand, Bitcoin may ultimately end up as digital currency "only for 
geeks and B2B transactions." I'd speculate we'd loose a big subset of 
the geeks that way too, unless they happen to do a lot of transactions 
with medium to large size businesses. (Small businesses won't be able to 
afford the expense of or the time to maintain the node.) There's some 
level of risk that this pushes bitcoin into oblivion. And is it really a 
decentralized P2P currency if it's only used by medium and large 
businesses and a small set of technically capable individuals that 
transact with those entities directly in BTC? And is it really a 
decentralized currency in this scenario if its used mainly by medium and 
large businesses, banks, and exchanges? (I've purposely excluded small 
businesses because while they like the benefits of flexible payment 
systems, more don't have the time or skill (or resources to hire the 
skill) needed to do a full node implementation.)

I feel inherent cognitive dissonance between "keep it decentralized" and 
"not useful to small business and individuals." One can make the 
argument that L2 solutions will be available for the small businesses 
and individuals but that doesn't solve the stated intent of reversing 
the trend of transactions not originating from or being received by full 
nodes. I guess you're saying bitcoin will be stronger, more resistant to 
outside power agency and censorship if its only used by exchanges, 
banks, large businesses, and die-hard technically inclined people.


On the other hand, maybe there's a scenario where an average person 
walks into a big box electronics store in any developed country and buys 
a "personal digital bank" appliance. I frame it this way because the 
majority of the worlds population is never going to run a full node on 
their desktop or laptop. There's no viable scenario where that happens. 
Laptops and desktops are already diminishing in market share due to the 
introduction of tablets and smartphones. General purpose OS's are also 
inherently un-secure, so  going down this route means we are immediately 
in the realm of lots of theft. Preventing theft (or loss due to errors) 
requires additional digital key devices, or additional devices for 
multi-sig transactions just for basic financial safety, not to mention a 
functioning backup plan, including off-site backups. 
Ransomeware/phishing protection? Checking email and surfing the web on 
the computer that holds your standard (non-multi-sig) wallet? 
Forgetaboutit. It'll never reach critical mass. It's not a viable 
proposal. Not to mention, you can't physically carry your laptop with 
you when you go to the shopping mall. In order for this appliance model 
to function, smartphone based implementations will need to interact with 
your personal or family server/appliance, and you'll need to be able to 
do multi-sig with a smartphone and another physical token you carry with 
you. Imagine a 2 of 5 multi-sig wallet where your phone and an NFC or LE 
bluetooth device are sufficient to create a transaction on your home 
node while shopping. Or your phone has a single sig wallet and you top 
it up from your appliance and it never has a high balance. In any case, 
I've made the argument before that the definition of "bitcoin protocol" 
should, in addition to the consensus protocol, probably include a secure 
API protocol between wallet client and full node, and it still seems to 
be an important missing piece. I want to be able to travel and spend BTC 
and I DON'T want to do general purpose computing like email and web 
surfing on the same computer where I have a big chunk of life savings 
stored! I think defining this API will actually really support the use 
of user controlled full nodes for transactions! Imagine Trezor owners 
using their own node for transactions! Bitpay is the only player I know 
of that provides enough of a software stack to set this up for yourself.

I think reversing the non-full node transaction trend will have to be 
based the appliance usage model. You buy a new 200-500Gb nvme SSD every 
year and put it in one of the free slots. You upgrade when all slots are 
full. This is one scenario that could put us on a trend of increasing 
transactions originating and being received by personal full nodes, i.e. 
reversing centralization trends.


If there is any solution to this problem, it will need to recognize the 
fact that the supermajority of people on the planet are not technically 
savvy nor are they inclined to take the time to learn how to protect 
themselves with basic computer security much less how to use a full node 
for bitcoin transactions. The solution, if it exists, will need to be 
handed to them, and they'll need a reason to buy it. Any solution will 
also need to recognize the fact that it will cost resources (time and 
money) to run a full node. Lots of people spend a huge portion of their 
income just to get a smartphone because it's a useful communication 
device that does lots of other useful things. There's not nearly the 
same level of need to spend on a full node for bitcoin security.

Any solution to this problem should also recognize the fact that there's 
a significant amount of work to do to have a functioning personal 
implementation of a node and to use it for transactions. Even in my 
imagined future of polished and easy to use appliances, if you have 
enough capital in BTC that you need it and you can afford to buy it, 
you're now only starting to deal with implementation issues. You've now 
become your own bank. Now you have to secure that appliance physically, 
secure and back up the key seed material, secure the devices used to 
access it, connect an app on your smartphone to the appliance so you can 
create transactions while out of your home, connect your home 
computer(s) to the appliance, do key exchange with the app/PC and the 
appliance or implement some sort of PKI on all devices. You've just 
taken on the responsibility of a bank and a sysadmin! The higher the 
balance, the more of a target you are, and the more time/money you have 
to spend mitigating risk. This is a huge centralizing force that no one 
really seems to talk about. If you're the average person, you want to 
find a trustworthy company or trusted friend/family to take care of that 
stuff for you. If you're a technically inclined person AND maybe there's 
a way to reap some of the mining reward on a small scale, you're 
slightly more interested.

As a sysadmin for many years, I've seen first hand that most people want 
tools that just work, whether its software to make spreadsheets, 
operating systems, phones, or thermostats. My point here is that the 
number of people in the world who have the technical chops to run a node 
is ALWAYS going to be vastly lower than the number of people who will be 
using bitcoin (or cryptocurrency).

Of course we can make the argument that the definition of "bitcoin" is 
by design something to be used exclusively by institutions and geeks, 
and that this definition falls out of the necessity to ensure that it 
remains decentralized and censorship resistant. However, I'm not sure 
that logic holds or that it doesn't introduce risk that that sort of 
definition drives bitcoin toward diminished relevance.

At the end of all this though experiment, I'm still convinced that if 
the tools are built to enable flexible usage of full nodes (i.e. my 
phone, tablet or desktop app interfaces with the full node) then there's 
a large potential for increased usage of full nodes.

Thanks,
G

Re: [bitcoin-dev] Three hardfork-related BIPs

[Eric Voskuil]

The semantics of a necessarily secure and private client-server protocol differ from that of a necessarily distributed and public P2P protocol. I realize you refer to the C/S as a distinct API, but this point is worthy of clarification and emphasis.

The introduction of client-server sub-protocols into the Bitcoin P2P protocol has resulted in large scale privacy loss, weakened end-user security and reduced access to the public network. Plans to mitigate these issues stand to make matters worse by restricting access to the public network through the introduction of strong identity to the P2P protocol.

It is not the case that C/S APIs against private full nodes do not exist. Electrum (stratum) and Libbitcoin (zeromq) are notable examples. The management difficulties are not small, but there are also fundamental issues that must first be addressed.

In your example you imagine pluggsble SSD space, but Satoshi derivatives have scale deficiencies unrelated to storage. If we are going to get to reliable, cheap, performant personal full nodes (which I agree is essential to Bitcoin survival) we need nodes that scale (i.e. to the available hardware). We also require a robust, reliable and performant node/server development stack, not just the impossible choice between a fragile monolith and centralizing web APIs/wallets.

All centralized interfaces to Bitcoin (wallets, web APIs, payment services) shrink the economic consensus and thereby weaken its defense of sound and fungible money. The only solution is personally-controlled full nodes, as you say. The incentives for running a full node are sufficient if the cost of doing so is low. Getting there requires a node/server architecture intended for this outcome. Then maybe appliances are feasible.

e


> On Feb 6, 2017, at 8:24 AM, netkn0t (marcus) via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
>> On 27.01.2017 20:03, Luke Dashjr via bitcoin-dev wrote:
>> Assume as a premise (despite your apparent disagreement below) that for
>> Bitcoin to function, a supermajority of economic activity needs to be verified
>> using full nodes operated by the recipient. Evidence suggests that at this
>> current time, at best 10% of economic activity is in fact using a full node to
>> verify the transaction. On this basis, it seems pretty clear that serious
>> action must be taken to change the status quo, and so for efforts to do so
>> without dropping the block size have proven ineffective.
> Lets think like people in sales and marketing for a moment.
> 
> There's an implicit assumption here that ANY protocol or consensus-rule based solution exists that would reverse the trend of diminishing full node verified economic activity. Since there's no economic advantage to running a full node, there's no inherent motivation for implementation (or outright purchase) of full nodes by the very large percentage of people who fall in the non-technical "I just want it to work, and I don't want my money stolen" category. Yes, anyone on this list understands that "don't want my money stolen" is inherently connected to running your own node and using it for transactions, but the average user does not, and even if they did, they don't have the resources (time and/or money) to do anything about it. Running your own full node increases the protection agains double spend attacks and other protocol bases shenanigans, but now you've taken on another set of security exposures related to the physical box that is running the node. Anti-virus, off and on-site backups, multiple boxes/devices for multi-sig, backup of key seeds.
> 
> Reducing (or even maintaining) the block size doesn't somehow increase the number of people who are capable of running full nodes, and it doesn't add any incentive for people already in that "capable" set to suddenly take up the task of running and transacting via a full node. I'd argue that the size of the block-chain and the time to download it are the least concerning aspects to anyone faced with running their own node and actually storing some of their wealth on it and using it for transactions.
> 
> You're looking for a (maybe dangerous/maybe impossible) balance between choking off casual (not full node) usage of bitcoin and yet trying to make it more popular among the people (and organizations) who have the capability and resources to run and transact on full nodes.
> 
> We should sit with this for a moment.
> 
> On one hand, Bitcoin may ultimately end up as digital currency "only for geeks and B2B transactions." I'd speculate we'd loose a big subset of the geeks that way too, unless they happen to do a lot of transactions with medium to large size businesses. (Small businesses won't be able to afford the expense of or the time to maintain the node.) There's some level of risk that this pushes bitcoin into oblivion. And is it really a decentralized P2P currency if it's only used by medium and large businesses and a small set of technically capable individuals that transact with those entities directly in BTC? And is it really a decentralized currency in this scenario if its used mainly by medium and large businesses, banks, and exchanges? (I've purposely excluded small businesses because while they like the benefits of flexible payment systems, more don't have the time or skill (or resources to hire the skill) needed to do a full node implementation.)
> 
> I feel inherent cognitive dissonance between "keep it decentralized" and "not useful to small business and individuals." One can make the argument that L2 solutions will be available for the small businesses and individuals but that doesn't solve the stated intent of reversing the trend of transactions not originating from or being received by full nodes. I guess you're saying bitcoin will be stronger, more resistant to outside power agency and censorship if its only used by exchanges, banks, large businesses, and die-hard technically inclined people.
> 
> 
> On the other hand, maybe there's a scenario where an average person walks into a big box electronics store in any developed country and buys a "personal digital bank" appliance. I frame it this way because the majority of the worlds population is never going to run a full node on their desktop or laptop. There's no viable scenario where that happens. Laptops and desktops are already diminishing in market share due to the introduction of tablets and smartphones. General purpose OS's are also inherently un-secure, so  going down this route means we are immediately in the realm of lots of theft. Preventing theft (or loss due to errors) requires additional digital key devices, or additional devices for multi-sig transactions just for basic financial safety, not to mention a functioning backup plan, including off-site backups. Ransomeware/phishing protection? Checking email and surfing the web on the computer that holds your standard (non-multi-sig) wallet? Forgetaboutit. It'll never reach critical mass. It's not a viable proposal. Not to mention, you can't physically carry your laptop with you when you go to the shopping mall. In order for this appliance model to function, smartphone based implementations will need to interact with your personal or family server/appliance, and you'll need to be able to do multi-sig with a smartphone and another physical token you carry with you. Imagine a 2 of 5 multi-sig wallet where your phone and an NFC or LE bluetooth device are sufficient to create a transaction on your home node while shopping. Or your phone has a single sig wallet and you top it up from your appliance and it never has a high balance. In any case, I've made the argument before that the definition of "bitcoin protocol" should, in addition to the consensus protocol, probably include a secure API protocol between wallet client and full node, and it still seems to be an important missing piece. I want to be able to travel and spend BTC and I DON'T want to do general purpose computing like email and web surfing on the same computer where I have a big chunk of life savings stored! I think defining this API will actually really support the use of user controlled full nodes for transactions! Imagine Trezor owners using their own node for transactions! Bitpay is the only player I know of that provides enough of a software stack to set this up for yourself.
> 
> I think reversing the non-full node transaction trend will have to be based the appliance usage model. You buy a new 200-500Gb nvme SSD every year and put it in one of the free slots. You upgrade when all slots are full. This is one scenario that could put us on a trend of increasing transactions originating and being received by personal full nodes, i.e. reversing centralization trends.
> 
> 
> If there is any solution to this problem, it will need to recognize the fact that the supermajority of people on the planet are not technically savvy nor are they inclined to take the time to learn how to protect themselves with basic computer security much less how to use a full node for bitcoin transactions. The solution, if it exists, will need to be handed to them, and they'll need a reason to buy it. Any solution will also need to recognize the fact that it will cost resources (time and money) to run a full node. Lots of people spend a huge portion of their income just to get a smartphone because it's a useful communication device that does lots of other useful things. There's not nearly the same level of need to spend on a full node for bitcoin security.
> 
> Any solution to this problem should also recognize the fact that there's a significant amount of work to do to have a functioning personal implementation of a node and to use it for transactions. Even in my imagined future of polished and easy to use appliances, if you have enough capital in BTC that you need it and you can afford to buy it, you're now only starting to deal with implementation issues. You've now become your own bank. Now you have to secure that appliance physically, secure and back up the key seed material, secure the devices used to access it, connect an app on your smartphone to the appliance so you can create transactions while out of your home, connect your home computer(s) to the appliance, do key exchange with the app/PC and the appliance or implement some sort of PKI on all devices. You've just taken on the responsibility of a bank and a sysadmin! The higher the balance, the more of a target you are, and the more time/money you have to spend mitigating risk. This is a huge centralizing force that no one really seems to talk about. If you're the average person, you want to find a trustworthy company or trusted friend/family to take care of that stuff for you. If you're a technically inclined person AND maybe there's a way to reap some of the mining reward on a small scale, you're slightly more interested.
> 
> As a sysadmin for many years, I've seen first hand that most people want tools that just work, whether its software to make spreadsheets, operating systems, phones, or thermostats. My point here is that the number of people in the world who have the technical chops to run a node is ALWAYS going to be vastly lower than the number of people who will be using bitcoin (or cryptocurrency).
> 
> Of course we can make the argument that the definition of "bitcoin" is by design something to be used exclusively by institutions and geeks, and that this definition falls out of the necessity to ensure that it remains decentralized and censorship resistant. However, I'm not sure that logic holds or that it doesn't introduce risk that that sort of definition drives bitcoin toward diminished relevance.
> 
> At the end of all this though experiment, I'm still convinced that if the tools are built to enable flexible usage of full nodes (i.e. my phone, tablet or desktop app interfaces with the full node) then there's a large potential for increased usage of full nodes.
> 
> Thanks,
> G
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Three hardfork-related BIPs

[Staf Verhaegen]

[-- Attachment #1: Type: text/plain, Size: 2023 bytes --]

Eric Voskuil via bitcoin-dev schreef op zo 29-01-2017 om 11:37 [-0800]:
> > On Jan 29, 2017, at 11:15 AM, Tom Harding via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> > 
> >> On 1/28/2017 10:29 AM, Peter Todd via bitcoin-dev wrote:
> >> a world of nodes in large datacenters is a world where it's very easy
> >> to force the few Bitcoin nodes remaining to follow AML/KYC rules
> > 
> > If that's true, why haven't we already seen AML/KYC required of mining
> > pools?  That would be comparatively trivial.
> 
> It is true, there is no question. The fact that an attack does not appear to have occurred does not mean that the vulnerability exists. It is as you say a trivial exploit, which means it will happen when the economic incentive is great enough. Analogous attacks on other points of centralization are already well underway.

What on first sight seems trivial may be totally different when looking
deeper. People here seem to not realise that a lot of data centers (the
IAAS ones) just are just grouping the computers and provide remote
access. The client have full control over the machines. The center thus
just provides the hardware, the power and the internet access. They
typically don't inspect your internet traffic only reduce the speed if
you are going above certain threshold. Additionally there are countries
like Iceland that specifically make laws to not let the government get
power over data and network traffic in data centers.
Domestic ISP services typically want to prioritize traffic and thus have
most of the time network traffic deep packet inspection (DPI)
capabilities. These are thus much easier forced by government to filter
certain traffic. Additionally these companies often fall under
telecommunication laws also given government more control over them than
in a typical data center.

I host my Bitcoin node in a German datacenter and am sure it is more
censorship resistant that a node going through any American ISP.

greets,
Staf.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [bitcoin-dev] Three hardfork-related BIPs

[Daniele Pinna]

[-- Attachment #1: Type: text/plain, Size: 4003 bytes --]

Your BIP implementation should stress the capacity to softfork the rate of
blocksize increase if necessary. You briefly mention that:

*If over time, this growth factor is beyond what the actual technology
offers, the intention should be to soft fork a tighter limit.*

However this can work both ways so that the rate can potentially be
increased also. I think just mentioning this will soothe a lot of future
critiques.

Daniele























































*Message: 5Date: Fri, 27 Jan 2017 01:06:59 +0000From: Luke Dashjr
<luke@dashjr.org
<luke@dashjr.org>>To: bitcoin-dev@lists.linuxfoundation.org
<bitcoin-dev@lists.linuxfoundation.org>Subject: [bitcoin-dev] Three
hardfork-related BIPsMessage-ID: <201701270107.01092.luke@dashjr.org
<201701270107.01092.luke@dashjr.org>>Content-Type: Text/Plain;
charset="utf-8"I've put together three hardfork-related BIPs. This is
parallel to the ongoingresearch into the MMHF/SHF WIP BIP, which might
still be best long-term.1) The first is a block size limit protocol change.
It also addresses threecriticisms of segwit: 1) segwit increases the block
size limit which isalready considered by many to be too large; 2) segwit
treats pre-segwittransactions ?unfairly? by giving the witness discount
only to segwittransactions; and 3) that spam blocks can be larger than
blocks mininglegitimate transactions. This proposal may (depending on
activation date)initially reduce the block size limit to a more sustainable
size in the short-term, and gradually increase it up over the long-term to
31 MB; it will alsoextend the witness discount to non-segwit transactions.
Should the initialblock size limit reduction prove to be too controversial,
miners can simplywait to activate it until closer to the point where it
becomes acceptableand/or increases the limit. However, since the BIP
includes a hardfork, theeventual block size increase needs community
consensus before it can bedeployed. Proponents of block size increases
should note that this BIP doesnot interfere with another more aggressive
block size increase hardfork in themeantime. I believe I can immediately
recommend this for adoption; however,peer and community review are welcome
to suggest
changes.Text: https://github.com/luke-jr/bips/blob/bip-blksize/bip-blksize.mediawiki
<https://github.com/luke-jr/bips/blob/bip-blksize/bip-blksize.mediawiki>Code:
https://github.com/bitcoin/bitcoin/compare/master...luke-jr:bip-blksize
<https://github.com/bitcoin/bitcoin/compare/master...luke-jr:bip-blksize>(consensus
code changes only)2) The second is a *preparatory* change, that should
allow triviallytransforming certain classes of hardforks into softforks in
the future. Itessentially says that full nodes should relax their rule
enforcement, aftersufficient time that would virtually guarantee they have
ceased to beenforcing the full set of rules anyway. This allows these
relaxed rules to bemodified or removed in a softfork, provided the proposal
to do so is acceptedand implemented with enough advance notice. Attempting
to implement this hasproven more complicated than I originally expected,
and it may make more sensefor full nodes to simply stop functioning (with a
user override) after thecut-off date). In light of this, I do not yet
recommend its adoption, but amposting it for review and comments
only.Text: https://github.com/luke-jr/bips/blob/bip-hfprep/bip-hfprep.mediawiki
<https://github.com/luke-jr/bips/blob/bip-hfprep/bip-hfprep.mediawiki>3)
Third is an anti-replay softfork which can be used to prevent replayattacks
whether induced by a hardfork-related chain split, or even in
ordinaryoperation. It does this by using a new opcode
(OP_CHECKBLOCKATHEIGHT) for theBitcoin scripting system that allows
construction of transactions which arevalid only on specific
blockchains.Text:
https://github.com/luke-jr/bips/blob/bip-noreplay/bip-noreplay.mediawiki
<https://github.com/luke-jr/bips/blob/bip-noreplay/bip-noreplay.mediawiki>Luke*
Daniele Pinna, Ph.D

[-- Attachment #2: Type: text/html, Size: 8942 bytes --]