* [Bitcoin-development] Suggestion for enhancements to getblock
@ 2011-07-07 9:49 Andy Parkins
2011-07-07 15:42 ` Mike Hearn
0 siblings, 1 reply; 6+ messages in thread
From: Andy Parkins @ 2011-07-07 9:49 UTC (permalink / raw)
To: bitcoin-development
Hello,
This is a suggestion with a mind to the future. In particular, I'm slowly
working on an alternative client and library (I know, everyone says that).
I've got a feature that I'd like to have that would need a change in the
protocol. It's a change that I think would improve the official client as
well, so I'm bringing it up here.
It's actually two changes; and both could be acheived by adding new commands
to the protocol. I think that would be overkill though; as they fit quite
nicely into an extended getblocks command.
(1) The getblocks message is a list of inventory items (type MSG_BLOCK) being
requested.
Imagine this situation though. I am a light weight client. I store the block
headers only. I am only interested in the history of my own wallet addresses.
I receive a block broadcast with a transaction that sends coins to one of my
addresses. That transaction references other transactions (of course), but I
haven't stored any transactions. So; I want to request those transactions and
ensure they are all valid and in blocks. I can't.
I can request the transactions themselves; but I have no way of finding out
what block they were in without downloading the entire full block chain
myself. The thing is, a peer with the full block chain is able to do this
instantly.
Here then is my suggestion: getblocks should accept inventory items of type
MSG_TX as well as MSG_BLOCK. When it finds a MSG_TX request, it shouldn't
send the transaction (after all that is what the getdata message is for);
instead it should return the block that contains that transaction. It's an
alternative way of requesting a block -- by transaction in that block.
It should be obvious then that it would be easy for a lightweight client to
request the transaction chain transactions its interested in to create a list
of relevant hashes for the history it's after; and then put those hashes in a
getblocks request and have to look only at a few full blocks instead of the
full block chain.
(2) If you are offline when new transactions are broadcast, there is no way to
know they are pending.
Transactions that have been broadcast but not yet accepted into a block are
never resent (nor should they be). But if I am on a mobile client say, or a
light-weight, intermittently used client; I have no way of checking if a
transaction sending coins to one of my addresses is pending.
It should be possible to request the current pending transaction list.
My sugestion then is that a special virtual block request be possible. The
all zero block hash can never exist (it's used as the parent for the genesis
block, so it had better not). If I send a getblocks that contains an all-zero
hash, then the virtual block should be returned in a block message. That
virtual block will list all the current pending transactions. Obviously there
would be a problem that it's actual hash would not be zero; so it would have
to be marked as the virtual block in some other way (perhaps a zero timestamp
plus a zero parent hash, or similar).
The combination of the above two protocol changes makes it possible for any
client to have a large chunk of the facilities available in blockexplorer.com
and the very useful http://bitcoincharts.com/bitcoin/ information.
Andy
--
Dr Andy Parkins
andyparkins@gmail•com
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Bitcoin-development] Suggestion for enhancements to getblock
2011-07-07 9:49 [Bitcoin-development] Suggestion for enhancements to getblock Andy Parkins
@ 2011-07-07 15:42 ` Mike Hearn
2011-07-07 16:19 ` Andy Parkins
2011-07-07 17:45 ` Gregory Maxwell
0 siblings, 2 replies; 6+ messages in thread
From: Mike Hearn @ 2011-07-07 15:42 UTC (permalink / raw)
To: Andy Parkins; +Cc: bitcoin-development
On Thu, Jul 7, 2011 at 11:49 AM, Andy Parkins <andyparkins@gmail•com> wrote:
> Imagine this situation though. I am a light weight client. I store the block
> headers only. I am only interested in the history of my own wallet addresses.
> I receive a block broadcast with a transaction that sends coins to one of my
> addresses. That transaction references other transactions (of course), but I
> haven't stored any transactions. So; I want to request those transactions and
> ensure they are all valid and in blocks. I can't.
Everyone writing an alternative client goes through this thought
process :-) There's no point in doing it, you cannot prove your
transaction is not a double spend. That requires knowledge (ie, an
index) of all transactions.
You have to treat appearing deep in the chain as ipso-facto proof of
validity. Lightweight/SPV clients simply must have that trust, it
cannot be done any other way. See this article:
http://code.google.com/p/bitcoinj/wiki/SecurityModel
Currently this is pretty safe due to the crazy speeds. In future when
speeds are likely to be lower, it will be less safe and you'd have to
wait longer or use a trusted node.
> It should be possible to request the current pending transaction list.
I think it'd be better to implement the filtering suggestions that
have been made. It doesn't scale to download the entire memory pool -
a better approach is to give the remote node a filter to match against
transactions then have it only relay those. After setting a filter,
transactions pending and matching would be sent in one big inv and you
can then keep the connection open to learn about new transactions
without needing to "drink from the firehose". Filters can be
probabilistic and set on many different nodes to reduce the privacy
implications.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Bitcoin-development] Suggestion for enhancements to getblock
2011-07-07 15:42 ` Mike Hearn
@ 2011-07-07 16:19 ` Andy Parkins
2011-07-07 16:44 ` Mike Hearn
2011-07-07 17:45 ` Gregory Maxwell
1 sibling, 1 reply; 6+ messages in thread
From: Andy Parkins @ 2011-07-07 16:19 UTC (permalink / raw)
To: bitcoin-development
[-- Attachment #1: Type: Text/Plain, Size: 3029 bytes --]
On 2011 July 07 Thursday, Mike Hearn wrote:
> On Thu, Jul 7, 2011 at 11:49 AM, Andy Parkins <andyparkins@gmail•com> wrote:
> > Imagine this situation though. I am a light weight client. I store the
> > block headers only. I am only interested in the history of my own
> > wallet addresses. I receive a block broadcast with a transaction that
> > sends coins to one of my addresses. That transaction references other
> > transactions (of course), but I haven't stored any transactions. So; I
> > want to request those transactions and ensure they are all valid and in
> > blocks. I can't.
>
> Everyone writing an alternative client goes through this thought
> process :-) There's no point in doing it, you cannot prove your
> transaction is not a double spend. That requires knowledge (ie, an
> index) of all transactions.
Ah; you mistake me. I'm not interested in double spend prevention, in this
case I'd be willing to trust the full node to return whatever block it thinks
contains that transaction, and that it has already done double spend
prevention.
What I want to be able to do though is calculate a balance for an aribtrary
address. Not every address; just the particular ones that the client is
interested in. It's complete overkill to require the whole block chain just
to calculate the balance of a few addresses.
> You have to treat appearing deep in the chain as ipso-facto proof of
> validity. Lightweight/SPV clients simply must have that trust, it
> cannot be done any other way. See this article:
Not entirely. If I ask for "the block that contains transaction with hash
12345678abcd..." then when I get that full block, I can verify the merkle tree
myself. I do have to trust that the peer hasn't been adding double spends in,
but not that the transaction is actually in the chain.
> > It should be possible to request the current pending transaction list.
>
> I think it'd be better to implement the filtering suggestions that
> have been made. It doesn't scale to download the entire memory pool -
I'm sorry, I've only started watching this list in the last few days. I'm not
familiar with the filter suggestions.
I'm not entirely sure I see how a filter helps. If I've been offline for ten
minutes then I need all the transactions pending in the last ten minutes. No
amount of filtering makes that list any smaller.
> a better approach is to give the remote node a filter to match against
> transactions then have it only relay those. After setting a filter,
> transactions pending and matching would be sent in one big inv and you
> can then keep the connection open to learn about new transactions
> without needing to "drink from the firehose". Filters can be
> probabilistic and set on many different nodes to reduce the privacy
> implications.
That would be fine. My reason for suggesting using getblocks was that it
didn't introduce a new command.
Andy
--
Dr Andy Parkins
andyparkins@gmail•com
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Bitcoin-development] Suggestion for enhancements to getblock
2011-07-07 16:19 ` Andy Parkins
@ 2011-07-07 16:44 ` Mike Hearn
2011-07-07 19:02 ` Andy Parkins
0 siblings, 1 reply; 6+ messages in thread
From: Mike Hearn @ 2011-07-07 16:44 UTC (permalink / raw)
To: Andy Parkins; +Cc: bitcoin-development
> What I want to be able to do though is calculate a balance for an aribtrary
> address. Not every address; just the particular ones that the client is
> interested in. It's complete overkill to require the whole block chain just
> to calculate the balance of a few addresses.
But what is that for? You said it's for a lightweight client to do
that when it receives a transaction, to verify that all the
dependencies are in blocks recursively. But why?
> Not entirely. If I ask for "the block that contains transaction with hash
> 12345678abcd..." then when I get that full block, I can verify the merkle tree
> myself.
Well, it's more efficient to just verify the merkle branch. But yes.
> I'm not entirely sure I see how a filter helps. If I've been offline for ten
> minutes then I need all the transactions pending in the last ten minutes. No
> amount of filtering makes that list any smaller.
Why do you need all of them? You just care about the ones sending
coins to you, surely?
> That would be fine. My reason for suggesting using getblocks was that it
> didn't introduce a new command.
IMHO it's fine to introduce new commands. They'll just be ignored by
old clients in any event.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Bitcoin-development] Suggestion for enhancements to getblock
2011-07-07 15:42 ` Mike Hearn
2011-07-07 16:19 ` Andy Parkins
@ 2011-07-07 17:45 ` Gregory Maxwell
1 sibling, 0 replies; 6+ messages in thread
From: Gregory Maxwell @ 2011-07-07 17:45 UTC (permalink / raw)
To: Mike Hearn; +Cc: bitcoin-development
On Thu, Jul 7, 2011 at 11:42 AM, Mike Hearn <mike@plan99•net> wrote:
[snip]
> You have to treat appearing deep in the chain as ipso-facto proof of
> validity. Lightweight/SPV clients simply must have that trust, it
> cannot be done any other way. See this article:
>
> http://code.google.com/p/bitcoinj/wiki/SecurityModel
It _could_ be done another way, with a protocol change:
http://forum.bitcoin.org/index.php?topic=21995.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Bitcoin-development] Suggestion for enhancements to getblock
2011-07-07 16:44 ` Mike Hearn
@ 2011-07-07 19:02 ` Andy Parkins
0 siblings, 0 replies; 6+ messages in thread
From: Andy Parkins @ 2011-07-07 19:02 UTC (permalink / raw)
To: bitcoin-development
On Thursday 07 July 2011 17:44:48 Mike Hearn wrote:
> > What I want to be able to do though is calculate a balance for an
> > aribtrary address. Not every address; just the particular ones that
> > the client is interested in. It's complete overkill to require the
> > whole block chain just to calculate the balance of a few addresses.
>
> But what is that for? You said it's for a lightweight client to do
> that when it receives a transaction, to verify that all the
> dependencies are in blocks recursively. But why?
There is no way for a client to know in advance whether any broadcast
transaction contains a send to an address in its wallet. So every incoming
transaction has to be examined.
Then, there is no way to know if while you were offline any of the
transactions in the blocks you missed contained transactions for an address
in your wallet.
Also, a feature I am interested in supporting is a split wallet -- where the
private key is held elsewhere. I'd still want to be able to report the
current balance in a particular address though. That address can be added
at any time.
Also, I would like to make some blockexplorer-like facilities available to
lightweight clients.
> > Not entirely. If I ask for "the block that contains transaction with
> > hash 12345678abcd..." then when I get that full block, I can verify
> > the merkle tree myself.
>
> Well, it's more efficient to just verify the merkle branch. But yes.
We're only talking about one verifying one (or minimal numbers of) blocks;
"efficient" isn't really going to matter much in that context. Also, if
we're talking about a situation where we don't necessarily trust the remote,
we've got to verify the whole block, not just the one transaction we're
interested in, since we told the remote which one we were interested in when
we requested it.
> > I'm not entirely sure I see how a filter helps. If I've been offline
> > for ten minutes then I need all the transactions pending in the last
> > ten minutes. No amount of filtering makes that list any smaller.
>
> Why do you need all of them? You just care about the ones sending
> coins to you, surely?
Is the filter going to be filter-by-address then? I misunderstood in that
case, I thought you were talking about filter-by-hash, which obviously tells
you nothing about the contents of the transaction.
> > That would be fine. My reason for suggesting using getblocks was that
> > it didn't introduce a new command.
>
> IMHO it's fine to introduce new commands. They'll just be ignored by
> old clients in any event.
That's good to know. I'm trying to be circumspect in what my client does; I
want to be 100% compatible, which means if I need a new feature, it's got to
be in the official client first.
I accept that this is all big talk, and there are plenty of people who start
new clients and then give up; which might still happen to me.
Andy
--
Dr Andy Parkins
andyparkins@gmail•com
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-07-07 19:02 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-07-07 9:49 [Bitcoin-development] Suggestion for enhancements to getblock Andy Parkins
2011-07-07 15:42 ` Mike Hearn
2011-07-07 16:19 ` Andy Parkins
2011-07-07 16:44 ` Mike Hearn
2011-07-07 19:02 ` Andy Parkins
2011-07-07 17:45 ` Gregory Maxwell
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox