[bitcoin-dev] Building a Bitcoin API and query system.

[bitcoin-dev] Building a Bitcoin API and query system.

[Blockchain Group]

[-- Attachment #1: Type: text/plain, Size: 855 bytes --]

Hello everyone,

I am C++ & Node.js developer. I want to propose making a new Bitcoin API
that supports fast quering of Bitcoin blocks and transactions without the
need for syncing with all previous nodes.

In a typical case where I want to build a full fleged Bitcoin explorer cum
wallet system on my end with external APIs, I need to sync my node and then
query for the information I need to show separately. I am proposing a
unified method of finding/quering the blockchain data with a standardized
template containing minimal information about the actual mined block or
transaction yet satify the need of what I want to query.

I am working on making a template and a support mechanism on Node.js. I
want to propose it as an improvement (BIP). It will be a great help to
future web developers who want to make something similar.

Thanks
Sumit Lahiri.

[-- Attachment #2: Type: text/html, Size: 1097 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Joseph Gleason ⑈]

[-- Attachment #1: Type: text/plain, Size: 1563 bytes --]

For what it is worth, electrum has a lot or possibly all of what you are
talking about since the electrum servers are designed to quickly answer the
queries of light clients.  So right now, you could sync up an electrum
server or use an existing public one and send queries to it with json-rpc.

https://github.com/kyuupichan/electrumx/blob/master/docs/protocol-methods.rst


On Tue, Aug 28, 2018 at 5:36 AM Blockchain Group via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hello everyone,
>
> I am C++ & Node.js developer. I want to propose making a new Bitcoin API
> that supports fast quering of Bitcoin blocks and transactions without the
> need for syncing with all previous nodes.
>
> In a typical case where I want to build a full fleged Bitcoin explorer cum
> wallet system on my end with external APIs, I need to sync my node and then
> query for the information I need to show separately. I am proposing a
> unified method of finding/quering the blockchain data with a standardized
> template containing minimal information about the actual mined block or
> transaction yet satify the need of what I want to query.
>
> I am working on making a template and a support mechanism on Node.js. I
> want to propose it as an improvement (BIP). It will be a great help to
> future web developers who want to make something similar.
>
> Thanks
> Sumit Lahiri.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 2359 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Matias Alejo Garcia]

[-- Attachment #1: Type: text/plain, Size: 2262 bytes --]

Hi Sumit,

Take a look at https://github.com/bitpay/bitcore/tree/v8.0.0, it is a
bitcoin indexing API server, with several modules, like a block explorer, a
wallet module, etc. It is built using Node.js.

matías

On Tue, Aug 28, 2018 at 12:43 PM Joseph Gleason ⑈ via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> For what it is worth, electrum has a lot or possibly all of what you are
> talking about since the electrum servers are designed to quickly answer the
> queries of light clients.  So right now, you could sync up an electrum
> server or use an existing public one and send queries to it with json-rpc.
>
>
> https://github.com/kyuupichan/electrumx/blob/master/docs/protocol-methods.rst
>
>
> On Tue, Aug 28, 2018 at 5:36 AM Blockchain Group via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Hello everyone,
>>
>> I am C++ & Node.js developer. I want to propose making a new Bitcoin API
>> that supports fast quering of Bitcoin blocks and transactions without the
>> need for syncing with all previous nodes.
>>
>> In a typical case where I want to build a full fleged Bitcoin explorer
>> cum wallet system on my end with external APIs, I need to sync my node and
>> then query for the information I need to show separately. I am proposing a
>> unified method of finding/quering the blockchain data with a standardized
>> template containing minimal information about the actual mined block or
>> transaction yet satify the need of what I want to query.
>>
>> I am working on making a template and a support mechanism on Node.js. I
>> want to propose it as an improvement (BIP). It will be a great help to
>> future web developers who want to make something similar.
>>
>> Thanks
>> Sumit Lahiri.
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>


-- 
Matías Alejo Garcia
@ematiu
Roads? Where we're going, we don't need roads!

[-- Attachment #2: Type: text/html, Size: 3694 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Blockchain Group]

[-- Attachment #1: Type: text/plain, Size: 2442 bytes --]

Thanks, I'll check it out.

On Tue, Aug 28, 2018, 9:17 PM Matias Alejo Garcia <ematiu@gmail.com> wrote:

> Hi Sumit,
>
> Take a look at https://github.com/bitpay/bitcore/tree/v8.0.0, it is a
> bitcoin indexing API server, with several modules, like a block explorer, a
> wallet module, etc. It is built using Node.js.
>
> matías
>
> On Tue, Aug 28, 2018 at 12:43 PM Joseph Gleason ⑈ via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> For what it is worth, electrum has a lot or possibly all of what you are
>> talking about since the electrum servers are designed to quickly answer the
>> queries of light clients.  So right now, you could sync up an electrum
>> server or use an existing public one and send queries to it with json-rpc.
>>
>>
>> https://github.com/kyuupichan/electrumx/blob/master/docs/protocol-methods.rst
>>
>>
>> On Tue, Aug 28, 2018 at 5:36 AM Blockchain Group via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>> Hello everyone,
>>>
>>> I am C++ & Node.js developer. I want to propose making a new Bitcoin API
>>> that supports fast quering of Bitcoin blocks and transactions without the
>>> need for syncing with all previous nodes.
>>>
>>> In a typical case where I want to build a full fleged Bitcoin explorer
>>> cum wallet system on my end with external APIs, I need to sync my node and
>>> then query for the information I need to show separately. I am proposing a
>>> unified method of finding/quering the blockchain data with a standardized
>>> template containing minimal information about the actual mined block or
>>> transaction yet satify the need of what I want to query.
>>>
>>> I am working on making a template and a support mechanism on Node.js. I
>>> want to propose it as an improvement (BIP). It will be a great help to
>>> future web developers who want to make something similar.
>>>
>>> Thanks
>>> Sumit Lahiri.
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
>
> --
> Matías Alejo Garcia
> @ematiu
> Roads? Where we're going, we don't need roads!
>

[-- Attachment #2: Type: text/html, Size: 4221 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Guido Dassori]

[-- Attachment #1: Type: text/plain, Size: 3358 bytes --]

Hi Sumit,

There's no protocol api to fetch a single transaction over the p2p network.
You can query peers for mempool transactions, but once there are into a
block, no way to obtain them without fetching the entire block.

Also, you may want to take a look to spruned (
https://github.com/gdassori/spruned), it uses both the electrum & the p2p
network to do the same job you are researching on. It serves the data by
emulating some of the bitcoind rpc api, to be suitable by bitcoind backed
applications.

Guido

Il giorno mar 28 ago 2018 alle ore 19:41 Blockchain Group via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> ha scritto:

> Thanks, I'll check it out.
>
> On Tue, Aug 28, 2018, 9:17 PM Matias Alejo Garcia <ematiu@gmail.com>
> wrote:
>
>> Hi Sumit,
>>
>> Take a look at https://github.com/bitpay/bitcore/tree/v8.0.0, it is a
>> bitcoin indexing API server, with several modules, like a block explorer, a
>> wallet module, etc. It is built using Node.js.
>>
>> matías
>>
>> On Tue, Aug 28, 2018 at 12:43 PM Joseph Gleason ⑈ via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>> For what it is worth, electrum has a lot or possibly all of what you are
>>> talking about since the electrum servers are designed to quickly answer the
>>> queries of light clients.  So right now, you could sync up an electrum
>>> server or use an existing public one and send queries to it with json-rpc.
>>>
>>>
>>> https://github.com/kyuupichan/electrumx/blob/master/docs/protocol-methods.rst
>>>
>>>
>>> On Tue, Aug 28, 2018 at 5:36 AM Blockchain Group via bitcoin-dev <
>>> bitcoin-dev@lists.linuxfoundation.org> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I am C++ & Node.js developer. I want to propose making a new Bitcoin
>>>> API that supports fast quering of Bitcoin blocks and transactions without
>>>> the need for syncing with all previous nodes.
>>>>
>>>> In a typical case where I want to build a full fleged Bitcoin explorer
>>>> cum wallet system on my end with external APIs, I need to sync my node and
>>>> then query for the information I need to show separately. I am proposing a
>>>> unified method of finding/quering the blockchain data with a standardized
>>>> template containing minimal information about the actual mined block or
>>>> transaction yet satify the need of what I want to query.
>>>>
>>>> I am working on making a template and a support mechanism on Node.js. I
>>>> want to propose it as an improvement (BIP). It will be a great help to
>>>> future web developers who want to make something similar.
>>>>
>>>> Thanks
>>>> Sumit Lahiri.
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev@lists.linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>
>>
>> --
>> Matías Alejo Garcia
>> @ematiu
>> Roads? Where we're going, we don't need roads!
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 5692 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Eric Voskuil]

[-- Attachment #1: Type: text/plain, Size: 170 bytes --]

https://libbitcoin.org

> On Aug 28, 2018, at 10:34, Blockchain Group via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
> Thanks, I'll check it out.

[-- Attachment #2: Type: text/html, Size: 591 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Jonas Schnelli]

[-- Attachment #1: Type: text/plain, Size: 1666 bytes --]

Hi

To give a critical viewpoint on a such API:

Such APIs usually result in central validation, meaning that users trust API services rather the validating their own data. It break some of the fundamental properties of Bitcoin (avoid trusted third parties).
Systems or applications depending on a full indexed blockchain (a thus such API) do usually scale pretty bad.

I’d like to hear some concrete use-cases for a such block explorer(ish) API.

Thanks
—
Jonas

> Am 26.08.2018 um 21:58 schrieb Blockchain Group via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>:
> 
> Hello everyone,
> 
> I am C++ & Node.js developer. I want to propose making a new Bitcoin API that supports fast quering of Bitcoin blocks and transactions without the need for syncing with all previous nodes.
> 
> In a typical case where I want to build a full fleged Bitcoin explorer cum wallet system on my end with external APIs, I need to sync my node and then query for the information I need to show separately. I am proposing a unified method of finding/quering the blockchain data with a standardized template containing minimal information about the actual mined block or transaction yet satify the need of what I want to query.
> 
> I am working on making a template and a support mechanism on Node.js. I want to propose it as an improvement (BIP). It will be a great help to future web developers who want to make something similar.
> 
> Thanks
> Sumit Lahiri.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Blockchain Group]

[-- Attachment #1: Type: text/plain, Size: 1870 bytes --]

Thanks, I'll check it out.

On Wed, Aug 29, 2018, 12:06 AM Jonas Schnelli <dev@jonasschnelli.ch> wrote:

> Hi
>
> To give a critical viewpoint on a such API:
>
> Such APIs usually result in central validation, meaning that users trust
> API services rather the validating their own data. It break some of the
> fundamental properties of Bitcoin (avoid trusted third parties).
> Systems or applications depending on a full indexed blockchain (a thus
> such API) do usually scale pretty bad.
>
> I’d like to hear some concrete use-cases for a such block explorer(ish)
> API.
>
> Thanks
> —
> Jonas
>
> > Am 26.08.2018 um 21:58 schrieb Blockchain Group via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org>:
> >
> > Hello everyone,
> >
> > I am C++ & Node.js developer. I want to propose making a new Bitcoin API
> that supports fast quering of Bitcoin blocks and transactions without the
> need for syncing with all previous nodes.
> >
> > In a typical case where I want to build a full fleged Bitcoin explorer
> cum wallet system on my end with external APIs, I need to sync my node and
> then query for the information I need to show separately. I am proposing a
> unified method of finding/quering the blockchain data with a standardized
> template containing minimal information about the actual mined block or
> transaction yet satify the need of what I want to query.
> >
> > I am working on making a template and a support mechanism on Node.js. I
> want to propose it as an improvement (BIP). It will be a great help to
> future web developers who want to make something similar.
> >
> > Thanks
> > Sumit Lahiri.
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 2503 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Eric Voskuil]

The API implementation is not what is centralizing, nor is full indexation non-scalable. The centralization is in not running the API from a node under your own control. This is of course implied by the comment, “without the need for syncing”. In other words it is the deployment cost of the node that is centralizing.

Yet if people relied only on bitcoind and never centralized services there would be *no* block explorers (and no secure light wallets), because it does not provide remote query and does not fully index.

Block explorers and light wallets are pretty useful, so presumably some API must provide these features (ideally with reduced deployment cost). That will either be centralized or decentralized services. As such it seems wise to encourage the latter, as opposed to questioning whether there is any valid block explorer use case.

e


> On Aug 28, 2018, at 11:36, Jonas Schnelli via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> 
> Hi
> 
> To give a critical viewpoint on a such API:
> 
> Such APIs usually result in central validation, meaning that users trust API services rather the validating their own data. It break some of the fundamental properties of Bitcoin (avoid trusted third parties).
> Systems or applications depending on a full indexed blockchain (a thus such API) do usually scale pretty bad.
> 
> I’d like to hear some concrete use-cases for a such block explorer(ish) API.
> 
> Thanks
> —
> Jonas
> 
>> Am 26.08.2018 um 21:58 schrieb Blockchain Group via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>:
>> 
>> Hello everyone,
>> 
>> I am C++ & Node.js developer. I want to propose making a new Bitcoin API that supports fast quering of Bitcoin blocks and transactions without the need for syncing with all previous nodes.
>> 
>> In a typical case where I want to build a full fleged Bitcoin explorer cum wallet system on my end with external APIs, I need to sync my node and then query for the information I need to show separately. I am proposing a unified method of finding/quering the blockchain data with a standardized template containing minimal information about the actual mined block or transaction yet satify the need of what I want to query.
>> 
>> I am working on making a template and a support mechanism on Node.js. I want to propose it as an improvement (BIP). It will be a great help to future web developers who want to make something similar.
>> 
>> Thanks
>> Sumit Lahiri.
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Blockchain Group]

[-- Attachment #1: Type: text/plain, Size: 3145 bytes --]

Thanks! That is what my main point is.

On Wed, Aug 29, 2018, 8:10 PM Eric Voskuil <eric@voskuil.org> wrote:

> The API implementation is not what is centralizing, nor is full indexation
> non-scalable. The centralization is in not running the API from a node
> under your own control. This is of course implied by the comment, “without
> the need for syncing”. In other words it is the deployment cost of the node
> that is centralizing.
>
> Yet if people relied only on bitcoind and never centralized services there
> would be *no* block explorers (and no secure light wallets), because it
> does not provide remote query and does not fully index.
>
> Block explorers and light wallets are pretty useful, so presumably some
> API must provide these features (ideally with reduced deployment cost).
> That will either be centralized or decentralized services. As such it seems
> wise to encourage the latter, as opposed to questioning whether there is
> any valid block explorer use case.
>
> e
>
>
> > On Aug 28, 2018, at 11:36, Jonas Schnelli via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
> >
> > Hi
> >
> > To give a critical viewpoint on a such API:
> >
> > Such APIs usually result in central validation, meaning that users trust
> API services rather the validating their own data. It break some of the
> fundamental properties of Bitcoin (avoid trusted third parties).
> > Systems or applications depending on a full indexed blockchain (a thus
> such API) do usually scale pretty bad.
> >
> > I’d like to hear some concrete use-cases for a such block explorer(ish)
> API.
> >
> > Thanks
> > —
> > Jonas
> >
> >> Am 26.08.2018 um 21:58 schrieb Blockchain Group via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org>:
> >>
> >> Hello everyone,
> >>
> >> I am C++ & Node.js developer. I want to propose making a new Bitcoin
> API that supports fast quering of Bitcoin blocks and transactions without
> the need for syncing with all previous nodes.
> >>
> >> In a typical case where I want to build a full fleged Bitcoin explorer
> cum wallet system on my end with external APIs, I need to sync my node and
> then query for the information I need to show separately. I am proposing a
> unified method of finding/quering the blockchain data with a standardized
> template containing minimal information about the actual mined block or
> transaction yet satify the need of what I want to query.
> >>
> >> I am working on making a template and a support mechanism on Node.js. I
> want to propose it as an improvement (BIP). It will be a great help to
> future web developers who want to make something similar.
> >>
> >> Thanks
> >> Sumit Lahiri.
> >> _______________________________________________
> >> bitcoin-dev mailing list
> >> bitcoin-dev@lists.linuxfoundation.org
> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 4191 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Jonas Schnelli]

[-- Attachment #1: Type: text/plain, Size: 2340 bytes --]


> The API implementation is not what is centralizing, nor is full indexation non-scalable. The centralization is in not running the API from a node under your own control. This is of course implied by the comment, “without the need for syncing”. In other words it is the deployment cost of the node that is centralizing.

IMO an API that serves non verifiable data is supporting centralised validation. The „API" which supports one of the most important properties in Bitcoin – the ability to self-validate – is the data available via the p2p network.

> 
> Yet if people relied only on bitcoind and never centralized services there would be *no* block explorers (and no secure light wallets), because it does not provide remote query and does not fully index.
> 
> Block explorers and light wallets are pretty useful, so presumably some API must provide these features (ideally with reduced deployment cost). That will either be centralized or decentralized services. As such it seems wise to encourage the latter, as opposed to questioning whether there is any valid block explorer use case.

Bitcoin-Core has all required features to partially „index“ data (called the wallet) and provides them via the RPC API. If you don’t need to serve thousands of wallets (which smells after centralised validation), selective indexing (wallets) are the right choice. Also, if you have a proper light client architecture, you can use Bitcoin Core in pruned mode (<10GB of data) to serve an endless amount of wallets (client/server mode, I guess that is what you are referring to with "light clients").

I fail to see the use-cases where a fully index blockchain makes sense (the only one I can come up with is instant backup recovery where the transaction history needs to be preserved rather then recovering the UTXOs only).

Also, the p2p protocol has built in light client support with BIP37 (bloom filters) and soon BIP158 will be available on the network which does allow privacy-preserving "light clients" in a way where no trusted layer is required (client <-> p2p network rather then client <-> API provider <-> p2p network).

I don’t want to advocate against a full-index blockexplorer-like API. I just think its important to define the use case and be aware of the consequences and downsides.

/jonas

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Blockchain Group]

[-- Attachment #1: Type: text/plain, Size: 2613 bytes --]

Awesome, thanks for the information. I will work on it and keep it in mind.

On Wed, Aug 29, 2018, 11:57 PM Jonas Schnelli <dev@jonasschnelli.ch> wrote:

>
> > The API implementation is not what is centralizing, nor is full
> indexation non-scalable. The centralization is in not running the API from
> a node under your own control. This is of course implied by the comment,
> “without the need for syncing”. In other words it is the deployment cost of
> the node that is centralizing.
>
> IMO an API that serves non verifiable data is supporting centralised
> validation. The „API" which supports one of the most important properties
> in Bitcoin – the ability to self-validate – is the data available via the
> p2p network.
>
> >
> > Yet if people relied only on bitcoind and never centralized services
> there would be *no* block explorers (and no secure light wallets), because
> it does not provide remote query and does not fully index.
> >
> > Block explorers and light wallets are pretty useful, so presumably some
> API must provide these features (ideally with reduced deployment cost).
> That will either be centralized or decentralized services. As such it seems
> wise to encourage the latter, as opposed to questioning whether there is
> any valid block explorer use case.
>
> Bitcoin-Core has all required features to partially „index“ data (called
> the wallet) and provides them via the RPC API. If you don’t need to serve
> thousands of wallets (which smells after centralised validation), selective
> indexing (wallets) are the right choice. Also, if you have a proper light
> client architecture, you can use Bitcoin Core in pruned mode (<10GB of
> data) to serve an endless amount of wallets (client/server mode, I guess
> that is what you are referring to with "light clients").
>
> I fail to see the use-cases where a fully index blockchain makes sense
> (the only one I can come up with is instant backup recovery where the
> transaction history needs to be preserved rather then recovering the UTXOs
> only).
>
> Also, the p2p protocol has built in light client support with BIP37 (bloom
> filters) and soon BIP158 will be available on the network which does allow
> privacy-preserving "light clients" in a way where no trusted layer is
> required (client <-> p2p network rather then client <-> API provider <->
> p2p network).
>
> I don’t want to advocate against a full-index blockexplorer-like API. I
> just think its important to define the use case and be aware of the
> consequences and downsides.
>
> /jonas
>

[-- Attachment #2: Type: text/html, Size: 2876 bytes --]

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Eric Voskuil]

You have created a straw man.

And light clients working against the P2P network (anonymous nodes) implies they are not fully validating, so you are contradicting yourself.

e

> On Aug 29, 2018, at 11:27, Jonas Schnelli <dev@jonasschnelli.ch> wrote:
> 
> 
>> The API implementation is not what is centralizing, nor is full indexation non-scalable. The centralization is in not running the API from a node under your own control. This is of course implied by the comment, “without the need for syncing”. In other words it is the deployment cost of the node that is centralizing.
> 
> IMO an API that serves non verifiable data is supporting centralised validation. The „API" which supports one of the most important properties in Bitcoin – the ability to self-validate – is the data available via the p2p network.
> 
>> 
>> Yet if people relied only on bitcoind and never centralized services there would be *no* block explorers (and no secure light wallets), because it does not provide remote query and does not fully index.
>> 
>> Block explorers and light wallets are pretty useful, so presumably some API must provide these features (ideally with reduced deployment cost). That will either be centralized or decentralized services. As such it seems wise to encourage the latter, as opposed to questioning whether there is any valid block explorer use case.
> 
> Bitcoin-Core has all required features to partially „index“ data (called the wallet) and provides them via the RPC API. If you don’t need to serve thousands of wallets (which smells after centralised validation), selective indexing (wallets) are the right choice. Also, if you have a proper light client architecture, you can use Bitcoin Core in pruned mode (<10GB of data) to serve an endless amount of wallets (client/server mode, I guess that is what you are referring to with "light clients").
> 
> I fail to see the use-cases where a fully index blockchain makes sense (the only one I can come up with is instant backup recovery where the transaction history needs to be preserved rather then recovering the UTXOs only).
> 
> Also, the p2p protocol has built in light client support with BIP37 (bloom filters) and soon BIP158 will be available on the network which does allow privacy-preserving "light clients" in a way where no trusted layer is required (client <-> p2p network rather then client <-> API provider <-> p2p network).
> 
> I don’t want to advocate against a full-index blockexplorer-like API. I just think its important to define the use case and be aware of the consequences and downsides.
> 
> /jonas

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Aymeric Vitte]

Le 28/08/2018 à 20:36, Jonas Schnelli via bitcoin-dev a écrit :
> I’d like to hear some concrete use-cases for a such block explorer(ish) API.

https://github.com/Ayms/bitcoin-transactions which is somewhere
bitcoin-cli outside of bitcoin core with no wallet, which implies that
you don't want to mix/provide your wallet with/to the app creating your
transactions and/or you don't want to use wallet sw

Problem: quasi nobody succeeds to use it (and probably this trend is
unlikely to revert), that's why there is https://peersm.com/wallet which
is querying the info outside and output the right command to use with
the tool (or output the transaction if people put their keys, which is
of course not advised unless they are sure that the corresponding
addresses are dead ones)

It is planned to put the app for the advanced mode (ie people must know
all the parameters) as an offline one inside browsers, then back to the
above problem...

So probably the offline mode should include a phase where the tool
connects to some APIs/explorers like the one suggested here before
switching to the offline mode to enter the keys, this will always be not
very secure for the query phase unless it can become something
decentralized (and usable the same way on different networks), which as
far as I understand is envisioned here

-- 
Bitcoin transactions made simple: https://github.com/Ayms/bitcoin-transactions
Zcash wallets made simple: https://github.com/Ayms/zcash-wallets
Bitcoin wallets made simple: https://github.com/Ayms/bitcoin-wallets
Get the torrent dynamic blocklist: http://peersm.com/getblocklist
Check the 10 M passwords list: http://peersm.com/findmyass
Anti-spies and private torrents, dynamic blocklist: http://torrent-live.org
Peersm : http://www.peersm.com
torrent-live: https://github.com/Ayms/torrent-live
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms

Re: [bitcoin-dev] Building a Bitcoin API and query system.

[Blockchain Group]

[-- Attachment #1: Type: text/plain, Size: 2150 bytes --]

Thanks, I'll check it out.

On Thu, Aug 30, 2018, 3:33 PM Aymeric Vitte <vitteaymeric@gmail.com> wrote:

>
>
> Le 28/08/2018 à 20:36, Jonas Schnelli via bitcoin-dev a écrit :
> > I’d like to hear some concrete use-cases for a such block explorer(ish)
> API.
>
> https://github.com/Ayms/bitcoin-transactions which is somewhere
> bitcoin-cli outside of bitcoin core with no wallet, which implies that
> you don't want to mix/provide your wallet with/to the app creating your
> transactions and/or you don't want to use wallet sw
>
> Problem: quasi nobody succeeds to use it (and probably this trend is
> unlikely to revert), that's why there is https://peersm.com/wallet which
> is querying the info outside and output the right command to use with
> the tool (or output the transaction if people put their keys, which is
> of course not advised unless they are sure that the corresponding
> addresses are dead ones)
>
> It is planned to put the app for the advanced mode (ie people must know
> all the parameters) as an offline one inside browsers, then back to the
> above problem...
>
> So probably the offline mode should include a phase where the tool
> connects to some APIs/explorers like the one suggested here before
> switching to the offline mode to enter the keys, this will always be not
> very secure for the query phase unless it can become something
> decentralized (and usable the same way on different networks), which as
> far as I understand is envisioned here
>
> --
> Bitcoin transactions made simple:
> https://github.com/Ayms/bitcoin-transactions
> Zcash wallets made simple: https://github.com/Ayms/zcash-wallets
> Bitcoin wallets made simple: https://github.com/Ayms/bitcoin-wallets
> Get the torrent dynamic blocklist: http://peersm.com/getblocklist
> Check the 10 M passwords list: http://peersm.com/findmyass
> Anti-spies and private torrents, dynamic blocklist:
> http://torrent-live.org
> Peersm : http://www.peersm.com
> torrent-live: https://github.com/Ayms/torrent-live
> node-Tor : https://www.github.com/Ayms/node-Tor
> GitHub : https://www.github.com/Ayms
>
>
>

[-- Attachment #2: Type: text/html, Size: 3590 bytes --]