Subject: Re: [Bitcoin-development] Building a node crawler to map network

From: Steve <[email-masked]>
Date: Wed, 07 Sep 2011 00:17:29 +1000

Hi Mike,

I expect I'll be submitting patches for bitcoinj sometime in the future 
but I'm not really across it yet to the point where I'd be confident 
submitting patches right now...

This proxy sound like a good match for what I've been up to lately 
though so long as it wouldn't involve direct changes to bitcoind on my 
part.  My c/c++ skills are non-existent.

However I have been building a pool protocol using protobufs and netty 
for non-blocking IO and I'd imagine the kind of multiplexing proxy 
you're talking about could be easily implemented using netty.

I'm not really understanding the use case though.  I believe most 
bitcoind's have a default max connections of 8.  Is the goal to increase 
this without fundamentally altering the bitcoind concurrency model?  Or 
is it to provide capactity for a more hub/client oriented network?  If 
the latter then presumably this is functionality that should ideally be 
native to the client in the long term in the form of NIO?

On 06/09/11 23:31, Mike Hearn wrote:
>
>     I've looked but can't find a post like you're talking about.  Can
>     you point me to it?
>
> https://groups.google.com/forum/?pli=1#!topic/bitcoinj/LSlZdUWcCdk 
> <https://groups.google.com/forum/?pli=1#%21topic/bitcoinj/LSlZdUWcCdk>
>
>     If so then bollocks... I'm looking for something useful to do atm.
>      PoolServerJ is in a holding pattern atm as I've stabilisied all
>     the bugs I know about and am waiting for several pools to finish
>     testing and move into production so I'm twiddling thumbs trying to
>     figure out how to spend my time.
>
>
> Patches to BitCoinJ are always welcome :-)
>
> If you'd rather do your own thing, you could experiment with writing a 
> proxy that sits in front of bitcoind and multiplexes connections. 
> Gavin is concerned about socket exhaustion as users move to 
> lightweight clients. Multiplexing proxies are a battle-tested 
> technique for reducing the strain of this type of thing. BitCoinJ uses 
> thread-per-connection so wouldn't do a good job of that right now, but 
> allowing it to use a mix of async io and multi-threading would be a 
> nice improvement. It'd need some changes to bitcoind as well for a 
> really good effort, to allow for IPs to be forwarded. I'm happy to 
> discuss it more with you over on the bitcoinj list if wanted.