Re: [bitcoin-dev] Long-term vision for bitcoind (was libconsensus and bitcoin development process)

From: "Eric Lombrozo" <elombrozo@gmail•com>
To: "Jorge Timón" <jtimon@jtimon•cc>,
	"Wladimir J. van der Laan" <laanwj@gmail•com>
Cc: Bitcoin development mailing list <bitcoin-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] Long-term vision for bitcoind (was libconsensus and bitcoin development process)
Date: Wed, 23 Sep 2015 00:10:50 +0000	[thread overview]
Message-ID: <em19621644-ea81-4030-bf85-0b3dbca635a7@platinum> (raw)
In-Reply-To: <em9a15c95a-cdb0-451f-8b69-e73572a722d2@platinum>

I should also add that the mempool should exist in (2). This way the 
peer services layer can manage all relay policy and mempool management.

------ Original Message ------
From: "Eric Lombrozo" <elombrozo@gmail•com>
To: "Jorge Timón" <jtimon@jtimon•cc>; "Wladimir J. van der Laan" 
<laanwj@gmail•com>
Cc: "Bitcoin development mailing list" 
<bitcoin-dev@lists•linuxfoundation.org>
Sent: 9/22/2015 5:07:22 PM
Subject: Re: [bitcoin-dev] Long-term vision for bitcoind (was 
libconsensus and bitcoin development process)

>Here's what I propose as a long-term plan:
>
>1) libconsensus
>==========
>We should probably start by defining an API for libconsensus. It should 
>support an abstract DB model, track chain state, provide query 
>mechanisms for blocks and transactions with optional pruning and 
>indexing, expose a subscription mechanism for events such as NEW_TIP, 
>REORG, etc, and contain a script interpreter.
>
>We can develop the library in parallel with Bitcoin Core without too 
>much refactoring of Bitcoin Core itself...just moving pieces of Bitcoin 
>Core's consensus code into the new library, tracking code movements to 
>make merging easier. Yes, this is a bit ugly as it requires code 
>duplication...but it will temporarily avoid much of the downstream 
>pushback we're getting. The idea is that we can prove out the library 
>with some simple projects, then start removing the consensus stuff from 
>Bitcoin Core once we have greater acceptance of the library and better 
>documentation.
>
>
>2) peer services
>==========
>We develop a peer services library that performs the tasks of peer 
>discovery and relay, with the ability to connect to appropriate peers 
>and queue messages. It uses libconsensus for all validation 
>functionality and as a datastore for the consensus state but maintains 
>its own database for peer history and statistics. I believe Cory has 
>been working on this already using libevent. I've already developed an 
>async library for this as well.
>
>
>3) API/RPC
>=======
>We provide high level calls and pub/sub mechanisms. ZMQ has been 
>implemented and added already, but we could support other transports as 
>well.
>
>
>4) Wallet
>======
>The wallet is split out into a separate process that connects to the 
>stack via the API/RPC layer.
>
>
>- Eric
>
>------ Original Message ------
>From: "Jorge Timón" <bitcoin-dev@lists•linuxfoundation.org>
>To: "Wladimir J. van der Laan" <laanwj@gmail•com>
>Cc: "Bitcoin development mailing list" 
><bitcoin-dev@lists•linuxfoundation.org>
>Sent: 9/22/2015 11:36:14 AM
>Subject: [bitcoin-dev] Long-term vision for bitcoind (was libconsensus 
>and bitcoin development process)
>
>>On Fri, Sep 18, 2015 at 2:07 AM, Wladimir J. van der Laan via
>>bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote:
>>>  My long-term vision of bitcoind is a P2P node with validation and 
>>>blockchain store, with a couple of data sources that can be 
>>>subscribed to or pulled from.
>>
>>I agree with this long term vision.
>>Here's how I think it could happen:
>>
>>1) Libconsensus is completed and moved to a subtree (which has libsecp
>>as an internal subtree)
>>
>>2) Bitcoind becomes a subtree of bitcoin-wallet (which has
>>bitcoin-wallet and bitcoin-qt)
>>
>>Without aggressively changing it for this purpose, libconsensus should
>>tend to become C, like libsecp, which is better for proving
>>correctness.
>>Hopefully at some point it won't take much to move to C.
>>
>>Upper layers should move to C++11
>>
>>Don't focus on the git subtrees, the basic architecture is bitcoin-qt
>>on top of bitcoin-wallet, bitcoin-wallet on top of bitcoind (and
>>friends like bitcoin-cli and bitcoin-tx), bitcoind on top of
>>libconsensus on top of libsecp256k1.
>>
>>I believe this would maximize the number of people who can safely
>>contribute to the project.
>>I also believe this is the architecture most contributors have in mind
>>for the long term, but I may be wrong about it.
>>
>>Criticisms to this plan?
>>_______________________________________________
>>bitcoin-dev mailing list
>>bitcoin-dev@lists•linuxfoundation.org
>>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev