* [Bitcoin-development] bitcoind multiplexing proxy - request/response routing problem
@ 2011-09-08 2:11 Steve diy-eco-life
0 siblings, 0 replies; 4+ messages in thread
From: Steve diy-eco-life @ 2011-09-08 2:11 UTC (permalink / raw)
To: Gavin Andresen, Bitcoin Dev
This a reworking of a post I made on the bitcoinj list under a
different topic but it's something I'd like to throw out there for
input.
I'm going to build a proof of concept this weekend of the multiplexing
proxy that Gavin and Mike were talking about here:
http://sourceforge.net/mailarchive/message.php?msg_id=28049519
Initially just a dumb as doornails proxy between one local bitcoind
and one remote node. Once I've got that far the next step is to work
out how deal with request-response exchanges from multiple remote
nodes. I discussed this tatcm on IRC last night. The problem is
after relaying several requests from different remote nodes to the
local daemon you expect multiple responses to come back. How identify
which response matches which client's request. Bitcoind can implicitly
identify the recipient based on which connection made the request. By
piling all the requests onto one channel we lose this identifier. I
can think of 3 approaches to dealing with this:
1/ Generate a unique key from the request and can also be generated
from the response. e.g. getheaders key could be "headers" +
hash_start. We locally store a mapping to client (or clients) that
requested it and pass it to bitcoind. When we get a headers packet
back unique key = "headers" + hash_of_first_header, so we can lookup
the clients who requested it and send it back.
The unique key should have the following properties:
- can be reliably generated from both the request and the response.
- identical requests from different clients should generate the same
unique key (this allows us to recycle responses)
Problems:
This is dependent on each pair of request/response messages being
guaranteed to contain information needed to create an identical unique
key. I haven't looked in detail at every request/response pair yet to
confirm this. If it is the case then this is an onerous obligation to
place on the protocol to fulfil this condition for all future protocol
changes.
To obtain guaranteed uniqueness may require large keys. Is
'almost-unique' an option? e.g. generating a key off a getblocks
request using the first n bytes of each block_locator_hash would be
much smaller/faster and very likely to be unique. Are the risks of
sending back the wrong response to a request acceptable?
2/ Modify bitcoind to accept sequence numbers for request/response
type messages, similar to 'id' field in json-rpc. This is more
reliable but potentially quite invasive to the bitcoin protocol. It
also loses the inherent de-duplication of requests that we get with
the previous solution. If it were to be implemented I'd suggest
something like a separate sequence number message. i.e. proxy sends
seq message containing a unique ID. The contract is that the seq
message refers to the next message that comes over the wire. When the
response is ready the bitcoind sends a seq message with matching id
then sends the response message. A separate seq message at least
leaves the existing protocol untouched as the handling will remain
unchanged unless a request type message is preceded by a seq message.
This approach allows the proxy remain a lot thinner and dumber but we
lose a lot of the de-duplicating efficiencies from the first approach.
If we want to add this capability as well we essentially need to
implement option 1 as well although in this case we have a reliable
fallback. i.e. If we can't generate a unique key then we just use
single request/response matching via the seq id.
3/ Make the proxy intelligent enough to handle these requests itself.
Using getheaders as an example again. Proxy maintains it's own local
cache of headers. when a getheaders message comes in the proxy checks
if it has all requested. If not it requests the missing ones from the
local daemon, adds to it's cache and builds a headers response itself.
In this case the proxy definately has to be protocol version aware...
Advantages:
- This probably achieves the best combination of request/response
matching reliability and de-duplication of work.
Disadvantages:
- Complexity. The proxy needs to be far more protocol aware which
creates a maintenance dependency for future protocol changes.
Having spent the last couple of days studying the protocol I'm
inclined toward the first approach as an initial easy implementation
with a view to moving to the 3rd approach. It appears that most
response type messages could be relatively easily constructed from a
local cache. Before I looked at the protocol I would have said no way
to the 3rd but the depth of protocol awareness for 1 or 3 is not
really much different. Option 2 allows for a much dumber and thinner
proxy but loses a lot of potential efficiencies and if those were to
be regained it would require the same level of protocol awareness
inherent in 1 and 3 anyway. It would also require someone on the
bitcoind side to put their hand up to add the seq message
functionality as I don't have any c skills to speak of.
Ultimately option 3 is where I was seeing the proxy progressing to at
some point far in the future but the routing problem needs to be
solved right from the beginning as I see it.
I hope I'm not over complicating it. If anyone can think of a simpler
approach to the request/response routing problem I'm all ears.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Bitcoin-development] bitcoind multiplexing proxy - request/response routing problem
2011-09-08 9:29 ` Mike Hearn
@ 2011-09-08 10:29 ` Steve
0 siblings, 0 replies; 4+ messages in thread
From: Steve @ 2011-09-08 10:29 UTC (permalink / raw)
To: Mike Hearn; +Cc: Bitcoin Dev
> It's probably best to keep this discussion on just one mailing list.
> It's confusing to have duplicate threads in different places. People
> will end up making the same points.
>
Fair enough I'll take it to the bitcoinj list. I wanted to post here in
case I got any nibbles from c developers about option 2. If anyone
want's the follow this discussion on the other list it's here:
https://groups.google.com/forum/#!topic/bitcoinj/kqiBq9VxL-k
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Bitcoin-development] bitcoind multiplexing proxy - request/response routing problem
2011-09-08 8:15 Steve
@ 2011-09-08 9:29 ` Mike Hearn
2011-09-08 10:29 ` Steve
0 siblings, 1 reply; 4+ messages in thread
From: Mike Hearn @ 2011-09-08 9:29 UTC (permalink / raw)
To: shadders.del; +Cc: Bitcoin Dev
[-- Attachment #1: Type: text/plain, Size: 935 bytes --]
It's probably best to keep this discussion on just one mailing list. It's
confusing to have duplicate threads in different places. People will end up
making the same points.
To repeat what I posted elsewhere, for now I'd just start with the simplest
possible approach:
- Ignore version skew for now (disconnect older clients)
- Don't send received transactions/blocks to the bitcoind. Let it hear about
them from its own p2p connections. That way you will always receive all
valid transactions/blocks which you can then relay/cache/drop inbound
duplicates.
- Parse/handle inv/getblocks/getheaders requests so clients that connect and
catch up with the chain don't place any load on the bitcoind. If a client
requests data the proxy doesn't have in RAM, it can go fetch it from the
underlying bitcoind.
If you can make v1 work and demonstrate actual scalability improvements,
then you can always go back and make it smarter in v2.
[-- Attachment #2: Type: text/html, Size: 1079 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Bitcoin-development] bitcoind multiplexing proxy - request/response routing problem
@ 2011-09-08 8:15 Steve
2011-09-08 9:29 ` Mike Hearn
0 siblings, 1 reply; 4+ messages in thread
From: Steve @ 2011-09-08 8:15 UTC (permalink / raw)
To: Bitcoin Dev
4a/ Serialize all request/response exchanges. i.e. request comes in
from remote node, proxy aquires lock on the proxy-localdaemon channel
and sends request. Channel remains locked until response is received or
timeout (in which case remote node gets no response). Unlock channel
after response received and send to client.
Possibly messages that don't expect a response (e.g. relaying a tx
broadcast from remote node) can be pushed down a locked channel to
improve performance as they won't interfere with sequencing. Locked
channels may also receive other unsolicited messages from local daemon
before the expected response message which would be dealt with the same
as if they came from an unlocked channel.
Disadvantages: Idle time for channel while waiting for response. As
per option 2 this allows the proxy to stay dumb/thin but loses
opportunity for de-duplicating/caching unless option 1 is layered on top.
4b/ As per 4a but use all 125 available bitcoind connections in a
channel pool. Acquiring a lock on a channel consists of checking for an
unlocked channel first then waiting in a queue for one to become available.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2011-09-08 10:29 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-09-08 2:11 [Bitcoin-development] bitcoind multiplexing proxy - request/response routing problem Steve diy-eco-life
2011-09-08 8:15 Steve
2011-09-08 9:29 ` Mike Hearn
2011-09-08 10:29 ` Steve
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox