Re: [Bitcoin-development] Modularizing Bitcoin

Re: [Bitcoin-development] Modularizing Bitcoin

[bitcoingrant]

[-- Attachment #1: Type: text/plain, Size: 434 bytes --]

our estimates: ~8000
----- Original Message -----
From: Addy Yeow
Sent: 05/16/13 06:27 AM
To: bitcoingrant@gmx.com
Subject: Re: [Bitcoin-development] Modularizing Bitcoin

> Such developments would significantly strengthen the system. Modularization would make cancer attacks less likely and increase the node count, which, currently, is fairly low. Do we know for certain or at least a rough figure of the node count in the network?

[-- Attachment #2: Type: text/html, Size: 1414 bytes --]

Re: [Bitcoin-development] Modularizing Bitcoin

[Addy Yeow]

Is the number representing the count for the client nodes?

I was curious of the count myself earlier this week and started to
traverse down the network using getaddr message starting from seed
nodes and found upward to 57k nodes running protocol >= 70001 with
timestamp no older than 24 hours.

On Thu, May 16, 2013 at 8:48 PM,  <bitcoingrant@gmx.com> wrote:
> our estimates: ~8000
>
>
>
> ----- Original Message -----
>
> From: Addy Yeow
>
> Sent: 05/16/13 06:27 AM
>
> To: bitcoingrant@gmx.com
>
> Subject: Re: [Bitcoin-development] Modularizing Bitcoin
>
>
>
>> Such developments would significantly strengthen the system.
>> Modularization would make cancer attacks less likely and increase the node
>> count, which, currently, is fairly low.
>
> Do we know for certain or at least a rough figure of the node count in
> the network?
>

Re: [Bitcoin-development] Modularizing Bitcoin

[Luke-Jr]

On Thursday, May 16, 2013 10:55:44 AM Addy Yeow wrote:
> Is the number representing the count for the client nodes?
> 
> I was curious of the count myself earlier this week and started to
> traverse down the network using getaddr message starting from seed
> nodes and found upward to 57k nodes running protocol >= 70001 with
> timestamp no older than 24 hours.

This sounds accurate for listening nodes, and similar to what my own system 
counts: http://luke.dashjr.org/programs/bitcoin/files/charts/security.html

Of course, it doesn't include the (many?) connect-only nodes (eg, mobile or 
firewall/NAT'd) or non-p2p nodes (eg, Electrum).

Luke

Re: [Bitcoin-development] Modularizing Bitcoin

[Tamas Blummer]

[-- Attachment #1: Type: text/plain, Size: 372 bytes --]

There is a modular, modern, open source implementation of the BItcoin protocol with properties, e.g. remote wallet, you look for at bitcoingrant. It is Bits of Proof. A supported and hosted product launched at the BItcoin2013. 

You find the source at https://github.com/bitsofproof/supernode and supporting documentation at http://bitsofproof.com

Tamás Blummer



[-- Attachment #2: Type: text/html, Size: 3816 bytes --]

Re: [Bitcoin-development] Modularizing Bitcoin

[Ron]

[-- Attachment #1: Type: text/plain, Size: 3086 bytes --]

From: <bitcoingrant@gm...> - 2013-05-16 10:02
	  	


            
            
            One of the primary upcoming priorities for bitcoin’s infrastructure, beyond the bloom filter, will be the continued modularization of the system.
Here at the Bitcoin Grant, we would like to jump start this development with a financial incentive and initiate an ongoing conversation on how we can work together towards developing a smarter, more efficient system of tomorrow, today.
Up for grabs: 500 bitcoins or $500,000; whichever is greater.
Taking on a project of this scope is a highly intensive, technical undertaking and we believe excellent developers should be compensated as such, especially when it comes to open source projects.
One of the main goals will be to separate the wallet from the node, as we have already done with mining. This way, the wallet, which will only hold private keys and create transactions, would pass transactions directly to a relay node, based on the bloom filter. Meanwhile, the block node will maintain the block chain and validate and relay new blocks.
Such developments would significantly strengthen the system. Modularization would make cancer attacks less likely and increase the node count, which, currently, is fairly low.
This is by no means is a feature request, merely ideas as to initiate a discussion. We welcome any feedback or suggestions. And of course, let us know if you would like to contribute to this project by submiting a grant proposal.
http://bitcoingrant.org http://bitcoingrant.org/&lang=en


Hello

I don't know if this is the proper method of replying or even if 
I am allowed to reply!

Modularization can have many meanings, depending upon one's past! 
The code is somewhat compartmentalized/modularized now. But if one 
forces complete separation of the parts, with a 'loose coupling', 
etc., I find that the performance tends to suffer and the size 
increases. 

In the Java world there is the notion of refactoring one's code. 
This would be too much, I think, in this case. When I developed 
with a team and alone, I would make what used to be called 
'step-wise refinements' on existing working code. To me, one of 
things this meant was doing a one to one transformation of the 
source code, in such a way as to have the identical, byte for 
byte, executable file, but a 'better' set of source files. A 
similar process would seem appropriate here. Especially since 
there is much in the code that I don't understand :) 

As to 'separating the wallet from the node', do you mean allowing 
the wallet.dat file to be anywhere, and not restricted to the 'OS default' 
or 'datadir' directory? If so, I have done that with no change to the 
current behavior, and also the wallet.dat now can have any legal 
filename too! I haven't tested it yet on bitcoin-qt, but it runs 
on bitcoind. I am still 'ramping up' on github to get the code into 
view. After testing on bitcoin-qt of course.

What may I ask, is a cancer attack?

If any of this inappropriate, forgive me.

Ron

[-- Attachment #2: Type: text/html, Size: 4785 bytes --]

[Bitcoin-development] Modularizing Bitcoin

[bitcoingrant]

[-- Attachment #1: Type: text/plain, Size: 1459 bytes --]

One of the primary upcoming priorities for bitcoin’s infrastructure, beyond the bloom filter, will be the continued modularization of the system.
Here at the Bitcoin Grant, we would like to jump start this development with a financial incentive and initiate an ongoing conversation on how we can work together towards developing a smarter, more efficient system of tomorrow, today.
Up for grabs: 500 bitcoins or $500,000; whichever is greater.
Taking on a project of this scope is a highly intensive, technical undertaking and we believe excellent developers should be compensated as such, especially when it comes to open source projects.
One of the main goals will be to separate the wallet from the node, as we have already done with mining. This way, the wallet, which will only hold private keys and create transactions, would pass transactions directly to a relay node, based on the bloom filter. Meanwhile, the block node will maintain the block chain and validate and relay new blocks.
Such developments would significantly strengthen the system. Modularization would make cancer attacks less likely and increase the node count, which, currently, is fairly low.
This is by no means is a feature request, merely ideas as to initiate a discussion. We welcome any feedback or suggestions. And of course, let us know if you would like to contribute to this project by submiting a grant proposal.
http://bitcoingrant.org http://bitcoingrant.org/&lang=en

[-- Attachment #2: Type: text/html, Size: 5286 bytes --]

Re: [Bitcoin-development] Modularizing Bitcoin

[Addy Yeow]

> Such developments would significantly strengthen the system. Modularization would make cancer attacks less likely and increase the node count, which, currently, is fairly low.

Do we know for certain or at least a rough figure of the node count in
the network?

On Thu, May 16, 2013 at 8:02 PM,  <bitcoingrant@gmx.com> wrote:
> One of the primary upcoming priorities for bitcoin’s infrastructure, beyond
> the bloom filter, will be the continued modularization of the system.
>
> Here at the Bitcoin Grant, we would like to jump start this development with
> a financial incentive and initiate an ongoing conversation on how we can
> work together towards developing a smarter, more efficient system of
> tomorrow, today.
>
> Up for grabs: 500 bitcoins or $500,000; whichever is greater.
>
> Taking on a project of this scope is a highly intensive, technical
> undertaking and we believe excellent developers should be compensated as
> such, especially when it comes to open source projects.
>
> One of the main goals will be to separate the wallet from the node, as we
> have already done with mining. This way, the wallet, which will only hold
> private keys and create transactions, would pass transactions directly to a
> relay node, based on the bloom filter. Meanwhile, the block node will
> maintain the block chain and validate and relay new blocks.
>
> Such developments would significantly strengthen the system. Modularization
> would make cancer attacks less likely and increase the node count, which,
> currently, is fairly low.
>
> This is by no means is a feature request, merely ideas as to initiate a
> discussion. We welcome any feedback or suggestions. And of course, let us
> know if you would like to contribute to this project by submiting a grant
> proposal.
>
> http://bitcoingrant.org
>
>
>
>
>
> ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers complete
> security visibility with the essential security capabilities. Easily and
> efficiently configure, manage, and operate all of your security controls
> from a single console and one unified framework. Download a free trial.
> http://p.sf.net/sfu/alienvault_d2d
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Re: [Bitcoin-development] Modularizing Bitcoin

[Brenton Camac]

[-- Attachment #1: Type: text/plain, Size: 3729 bytes --]

It would be nice if that modularization effort would first and foremost focus on defining the protocols and APIs of the various modules (their responsibilities and patterns of interaction), rather than merely refactoring existing code.

Such an approach has many benefits.  

First, it promotes diversity of implementations.  Diverse implementations are now possible because the correctness of an implementation is now determined entirely by the compliance of its external behavior with the stated protocols and not its internal design. Thus this approach allows for equivalent but alternate implementations. Consequently, this approach -

1. increases the available pool of developers
2. reduces the impact any one implementation defect can have on the overall Bitcoin infrastructure
3. allows enhancement/optimization work of modules to proceed more easily as coupling with external modules is reduced

Second, and just as important, it allows analysis and critiquing of Bitcoin's infrastructure to be undertaken at a higher level than source code: i.e. abstract entities of the protocols and APIs.  Such scrutiny is important to being able to effectively manage the evolution of a system's architecture.

Its been my first-hand experience across many projects that this strategy contributes directly to significant improvements to quality when developing large, distributed, complex software systems.  Indeed, its considered a best practice when developing enterprise-grade software.

I would be happy to collaborate with others in such an undertaking.

- Brenton



On May 16, 2013, at 3:02 AM, bitcoingrant@gmx.com wrote:

> One of the primary upcoming priorities for bitcoin’s infrastructure, beyond the bloom filter, will be the continued modularization of the system.
> Here at the Bitcoin Grant, we would like to jump start this development with a financial incentive and initiate an ongoing conversation on how we can work together towards developing a smarter, more efficient system of tomorrow, today.
> Up for grabs: 500 bitcoins or $500,000; whichever is greater.
> Taking on a project of this scope is a highly intensive, technical undertaking and we believe excellent developers should be compensated as such, especially when it comes to open source projects.
> One of the main goals will be to separate the wallet from the node, as we have already done with mining. This way, the wallet, which will only hold private keys and create transactions, would pass transactions directly to a relay node, based on the bloom filter. Meanwhile, the block node will maintain the block chain and validate and relay new blocks.
> Such developments would significantly strengthen the system. Modularization would make cancer attacks less likely and increase the node count, which, currently, is fairly low.
> This is by no means is a feature request, merely ideas as to initiate a discussion. We welcome any feedback or suggestions. And of course, let us know if you would like to contribute to this project by submiting a grant proposal.
> http://bitcoingrant.org
>  
> 
>   ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers complete
> security visibility with the essential security capabilities. Easily and
> efficiently configure, manage, and operate all of your security controls
> from a single console and one unified framework. Download a free trial.
> http://p.sf.net/sfu/alienvault_d2d_______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


[-- Attachment #2: Type: text/html, Size: 8374 bytes --]

Re: [Bitcoin-development] Modularizing Bitcoin

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 3371 bytes --]

I'm all for funding of Bitcoin development, but I suggest talking to Gavin
to find out what efforts would be the biggest win right now. I don't see
why separating wallet code from the main Bitcoin process would increase
node count, as the cost of running the node is almost all in keeping up
with transaction traffic and time spent in the wallet is likely to dominate
only for large merchants or exchanges.

That said, you can already do this today - just run an SPV wallet like
bitcoinj connected to your personal node. The wallet code in bitcoind won't
be used for anything.

There are lots of things that can be done, but the best way to approach
this is to get tightly written technical requirements from people in the
know, and then contract with developers. Bounty style development has the
risk of uncoordinated development that duplicates work and puts pressure on
Gavin or other maintainers to accept shoddy code due to the "first past the
post" winning criteria. Finding developers you trust and contracting with
them for well specified improvements minimises risk for everyone.


On Thu, May 16, 2013 at 3:02 AM, <bitcoingrant@gmx.com> wrote:

> One of the primary upcoming priorities for bitcoin’s infrastructure,
> beyond the bloom filter, will be the continued modularization of the system.
>
> Here at the Bitcoin Grant, we would like to jump start this development
> with a financial incentive and initiate an ongoing conversation on how we
> can work together towards developing a smarter, more efficient system of
> tomorrow, today.
>
> Up for grabs: 500 bitcoins or $500,000; whichever is greater.
>
> Taking on a project of this scope is a highly intensive, technical
> undertaking and we believe excellent developers should be compensated as
> such, especially when it comes to open source projects.
>
> One of the main goals will be to separate the wallet from the node, as we
> have already done with mining. This way, the wallet, which will only hold
> private keys and create transactions, would pass transactions directly to a
> relay node, based on the bloom filter. Meanwhile, the block node will
> maintain the block chain and validate and relay new blocks.
>
> Such developments would significantly strengthen the system.
> Modularization would make cancer attacks less likely and increase the node
> count, which, currently, is fairly low.
>
> This is by no means is a feature request, merely ideas as to initiate a
> discussion. We welcome any feedback or suggestions. And of course, let us
> know if you would like to contribute to this project by submiting a grant
> proposal.
>
> http://bitcoingrant.org <http://bitcoingrant.org/&lang=en>
>
>
>
>
>
> ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers complete
> security visibility with the essential security capabilities. Easily and
> efficiently configure, manage, and operate all of your security controls
> from a single console and one unified framework. Download a free trial.
> http://p.sf.net/sfu/alienvault_d2d
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

[-- Attachment #2: Type: text/html, Size: 7120 bytes --]

Re: [Bitcoin-development] Modularizing Bitcoin

[Quinn Harris]

[-- Attachment #1: Type: text/plain, Size: 3289 bytes --]

Could this modularization effort lead to a special compiled bitcoind 
simulator that runs many virtual instances of a node on the same system 
(possibly same process)? The simulator would cache crypto computation 
results (ECDSA, SSH-256) to significantly speed up processing 
transactions and blocks between the virtual nodes.  The virtual nodes 
would also need to share the same block data in memory when possible.  
This would also require significant additional work to simulate the 
network and possibly computational limits.

The transactions on the existing block chain could be replayed into 
randomly chosen virtual nodes to help ensure any code changes would not 
have caused problems with historic transactions.  You could even record 
all the messages for some time on many real nodes and use that as test 
data for the simulator.  Many other test programs could be devised to 
quickly simulate other network activity.

This wouldn't completely replace the test network but could provide 
greater and quicker confidence that code changes are safe.

- Quinn


On 05/16/2013 04:02 AM, bitcoingrant@gmx.com wrote:
> One of the primary upcoming priorities for bitcoin's infrastructure, 
> beyond the bloom filter, will be the continued modularization of the 
> system.
>
> Here at the Bitcoin Grant, we would like to jump start this 
> development with a financial incentive and initiate an ongoing 
> conversation on how we can work together towards developing a smarter, 
> more efficient system of tomorrow, today.
>
> Up for grabs: 500 bitcoins or $500,000; whichever is greater.
>
> Taking on a project of this scope is a highly intensive, technical 
> undertaking and we believe excellent developers should be compensated 
> as such, especially when it comes to open source projects.
>
> One of the main goals will be to separate the wallet from the node, as 
> we have already done with mining. This way, the wallet, which will 
> only hold private keys and create transactions, would pass 
> transactions directly to a relay node, based on the bloom filter. 
> Meanwhile, the block node will maintain the block chain and validate 
> and relay new blocks.
>
> Such developments would significantly strengthen the system. 
> Modularization would make cancer attacks less likely and increase the 
> node count, which, currently, is fairly low.
>
> This is by no means is a feature request, merely ideas as to initiate 
> a discussion. We welcome any feedback or suggestions. And of course, 
> let us know if you would like to contribute to this project by 
> submiting a grant proposal.
>
> http://bitcoingrant.org <http://bitcoingrant.org/&lang=en>
>
>
>
>
> ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers complete
> security visibility with the essential security capabilities. Easily and
> efficiently configure, manage, and operate all of your security controls
> from a single console and one unified framework. Download a free trial.
> http://p.sf.net/sfu/alienvault_d2d
>
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development


[-- Attachment #2: Type: text/html, Size: 9306 bytes --]

Re: [Bitcoin-development] Modularizing Bitcoin

[Luke-Jr]

On Thursday, May 16, 2013 10:02:05 AM bitcoingrant@gmx.com wrote:
> One of the main goals will be to separate the wallet from the node, as we
> have already done with mining.

How is this different from what Electrum has already done?

Luke