public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: Tomas <tomas@tomasvdw.nl>
To: Greg Sanders <gsanders87@gmail.com>,
	Bitcoin Protocol Discussion
	<bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Using a storage engine without UTXO-index
Date: Fri, 07 Apr 2017 18:02:35 +0200	[thread overview]
Message-ID: <1491580955.73830.937628696.4EEA507C@webmail.messagingengine.com> (raw)
In-Reply-To: <CAB3F3DvG4NSBEk1vS-KWQjn3PXWOwF3xNP7_txUpwTdDMHettg@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 4114 bytes --]

Thank you,



The benches are running in Google Cloud Engine; currently on 8 vCPU
32gb, but I tend to switch hardware regularly.


Roughly, the results are better for Bitcrust with high end hardware and
the difference for total block validations is mostly diminished at 2
vCPU, 7,5 gb.


Note that the spend-tree optimization primarily aims to improve peak
load order validation; when a block with pre-synced transactions comes
in, but this is tricky to accurately bench with Core using this simple
method of comparison by logs.


I will upgrade to, and show the results against 0.14 in the next weeks.


Best,

Tomas





On Fri, Apr 7, 2017, at 16:14, Greg Sanders wrote:

> Interesting work.

> 

> I was wondering if you could tellank  us what specs for the machine
> being used as preliminary benchmark is here:
> https://bitcrust.org/results ?
> 

> I'd be interested to also see comparisons with 0.14 which has some
> improvements for script validation with more cores.
> 

> On Fri, Apr 7, 2017 at 4:47 AM, Tomas via bitcoin-dev <bitcoin-
> dev@lists.linuxfoundation.org> wrote:
>> Thank you Marcos,

>> 

>>  Though written in Rust, bitcrust-db is definitely usable as
>>  pluggable
>>  module as its interface will be roughly some queries, add_tx and

>>  add_block with blobs and flags. (Bitcrust internally uses a

>>  deserialize-only model, keeping references to the blobs with the
>>  parsed
>>  data).

>> 

>>  However, from Core's side I believe network and storage are
>>  currently
>>  rather tightly coupled, which will make this far from trivial.

>> 

>>  Regardless, I am also hoping (with funding & a team) to build a
>>  Bitcrust
>>  networking component as well to bring a strong competitor to the
>>  market.
>> 

>>  best,

>>  Tomas

>> 

>> 

>> 

>> 

>> On Fri, Apr 7, 2017, at 09:55, Marcos mayorga wrote:

>>  > Hi Tomas,

>>  >

>>  > I've read it and think it is an excellent work, I'd like to see it
>>  > integrated into bitcoin-core as a 'kernel module'.

>>  >

>>  > I see there are a lot of proof of concepts out there, IMO
>>  > every one
>>  > deserve a room in the bitcoin client as a selectable feature, to
>>  > make the
>>  > software more flexible and less dictatorial, an user could easily
>>  > select
>>  > which features she wants to run.

>>  >

>>  > Best regards,

>>  > Marcos

>>  >

>>  > > I have been working on a bitcoin implementation that uses a
>>  > > different
>>  > > approach to indexing for verifying the order of transactions.
>>  > > Instead of
>>  > > using an index of unspent outputs, double spends are verified by
>>  > > using a
>>  > > spend-tree where spends are scanned against spent outputs
>>  > > instead of
>>  > > unspent outputs.

>>  > >

>>  > > This allows for much better concurrency, as not only blocks, but
>>  > > also
>>  > > individual inputs can be verified fully in parallel.

>>  > >

>>  > > I explain the approach at https://bitcrust.org, source code is
>>  > > available
>>  > > at https://github.com/tomasvdw/bitcrust

>>  > >

>>  > > I am sharing this not only to ask for your feedback, but also to
>>  > > call
>>  > > for a clear separation of protocol and implementations: As this
>>  > > solution, reversing the costs of outputs and inputs, seems to
>>  > > have
>>  > > excellent performance characteristics (as shown in the test
>>  > > results),
>>  > > updates to the protocol addressing the UTXO growth, might not be
>>  > > worth
>>  > > considering *protocol improvements* and it might be best to
>>  > > address
>>  > > these concerns as implementation details.

>>  > >

>>  > > Kind regards,

>>  > > Tomas van der Wansem

>>  > > tomas@bitcrust.org

>>  > > Bitcrust

>>  > > _______________________________________________

>>  > > bitcoin-dev mailing list

>>  > > bitcoin-dev@lists.linuxfoundation.org

>>  > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

>>  > >

>>  >

>>  >

>>  _______________________________________________

>>  bitcoin-dev mailing list

>> bitcoin-dev@lists.linuxfoundation.org

>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev



[-- Attachment #2: Type: text/html, Size: 6443 bytes --]

  reply	other threads:[~2017-04-07 16:02 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-04-06 22:12 [bitcoin-dev] Using a storage engine without UTXO-index Tomas
2017-04-06 23:38 ` Eric Voskuil
2017-04-07  0:17   ` Tomas
2017-04-08 22:37     ` Eric Voskuil
2017-04-08 23:58       ` Tomas
2017-04-11  1:44         ` Eric Voskuil
2017-04-11  8:43           ` Tomas
2017-04-11  9:41             ` Eric Voskuil
2017-04-11 10:04               ` Tomas
     [not found] ` <CAAS2fgTEMCkDWdhCWt1EsUrnt3+Z_8m+Y1PTsff5Rc0CBnCKWQ@mail.gmail.com>
2017-04-07  0:48   ` Tomas
2017-04-07  1:09     ` Gregory Maxwell
2017-04-07  1:29       ` Tomas
2017-04-07 18:52         ` Tom Harding
2017-04-07 19:42           ` Gregory Maxwell
2017-04-08 18:27             ` Tom Harding
2017-04-08 19:23               ` Tomas
2017-04-07  7:55 ` Marcos mayorga
2017-04-07  8:47   ` Tomas
2017-04-07 14:14     ` Greg Sanders
2017-04-07 16:02       ` Tomas [this message]
2017-04-07 18:18 ` Gregory Maxwell
2017-04-07 18:39   ` Bram Cohen
2017-04-07 19:55     ` Eric Voskuil
2017-04-07 21:44       ` Tomas
2017-04-07 23:51         ` Eric Voskuil
2017-04-07 21:14     ` Tomas
2017-04-08  0:44       ` Gregory Maxwell
2017-04-08  7:28         ` Tomas
2017-04-08 19:23           ` Johnson Lau
2017-04-08 19:56             ` Tomas
2017-04-08 20:21               ` Johnson Lau
2017-04-08 20:42                 ` Tomas
2017-04-08 22:12                 ` Gregory Maxwell
2017-04-08 22:34                   ` Tomas
2017-04-08 21:22     ` Troy Benjegerdes

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1491580955.73830.937628696.4EEA507C@webmail.messagingengine.com \
    --to=tomas@tomasvdw.nl \
    --cc=bitcoin-dev@lists.linuxfoundation.org \
    --cc=gsanders87@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox