* [Bitcoin-development] Bootstrapping via BitTorrent trackers @ 2011-06-13 8:55 Christian Decker 2011-06-13 9:09 ` Jeff Garzik 0 siblings, 1 reply; 10+ messages in thread From: Christian Decker @ 2011-06-13 8:55 UTC (permalink / raw) To: bitcoin-development [-- Attachment #1: Type: text/plain, Size: 849 bytes --] Hi all. Just wanted to carry the discussion from the Forum over to the dev-list. We have quite a few bootstrapping mechanisms, starting with the overly complex (IMHO) IRC bootstrapping, which is often suspected as bot-activity. Then we have a few hardcoded nodes and some fallback nodes. I was wondering why we didn't adopt BitTorrent tracker bootstrapping until now? It's basically all it does. Given a hash (SHA1 hash of the genesis bloc would be nice ^^) it gives you a list of other nodes with the same hash. Given that there are quite a few open trackers (accepting and tracking any hash you throw at them) we could just decide to use 2-3 of those to bootstrap. The downside would be that they return bencoded data, which has to be interpreted first, but it's easier than implementing the IRC stuff, I think. Any comments? Regards, Chris [-- Attachment #2: Type: text/html, Size: 913 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers 2011-06-13 8:55 [Bitcoin-development] Bootstrapping via BitTorrent trackers Christian Decker @ 2011-06-13 9:09 ` Jeff Garzik 2011-06-13 9:38 ` Christian Decker 0 siblings, 1 reply; 10+ messages in thread From: Jeff Garzik @ 2011-06-13 9:09 UTC (permalink / raw) To: Christian Decker; +Cc: bitcoin-development On Mon, Jun 13, 2011 at 4:55 AM, Christian Decker <decker.christian@gmail•com> wrote: > We have quite a few bootstrapping mechanisms, starting with the overly > complex (IMHO) IRC bootstrapping, which is often suspected as bot-activity. > Then we have a few hardcoded nodes and some fallback nodes. I was wondering > why we didn't adopt BitTorrent tracker bootstrapping until now? It's > basically all it does. Given a hash (SHA1 hash of the genesis bloc would be > nice ^^) it gives you a list of other nodes with the same hash. It seems to offer few benefits over DNS seeding, while potentially potentially creating a vulnerable hot spot in the DHT. Sybil attacks on DHTs are well documented. -- Jeff Garzik exMULTI, Inc. jgarzik@exmulti•com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers 2011-06-13 9:09 ` Jeff Garzik @ 2011-06-13 9:38 ` Christian Decker 2011-06-13 9:56 ` Jeff Garzik 0 siblings, 1 reply; 10+ messages in thread From: Christian Decker @ 2011-06-13 9:38 UTC (permalink / raw) To: Jeff Garzik; +Cc: bitcoin-development [-- Attachment #1: Type: text/plain, Size: 1593 bytes --] Don't get me wrong, DNS Seeding is an excellent way to bootstrap via trusted nodes, I'm not trying to replace it. What I'm trying to get rid of is the IRC bootstrapping and the hardcoded nodes in the client, they're easy targets. BitTorrent trackers are used to handle several thousands of requests, so they would probably scale well enough. I'm not even talking about using the DHT trackers, but using old fashioned HTTP based trackers. The fact that each bitcoin client would contact the tracker would make it very hard for an attacker to get bootstrapping clients to exclusively connect to his compromised clients. I would say that using a tracker such as OpenBittorrent provides the same advantages as using an IRC channel. On Mon, Jun 13, 2011 at 11:09 AM, Jeff Garzik <jgarzik@exmulti•com> wrote: > On Mon, Jun 13, 2011 at 4:55 AM, Christian Decker > <decker.christian@gmail•com> wrote: > > We have quite a few bootstrapping mechanisms, starting with the overly > > complex (IMHO) IRC bootstrapping, which is often suspected as > bot-activity. > > Then we have a few hardcoded nodes and some fallback nodes. I was > wondering > > why we didn't adopt BitTorrent tracker bootstrapping until now? It's > > basically all it does. Given a hash (SHA1 hash of the genesis bloc would > be > > nice ^^) it gives you a list of other nodes with the same hash. > > It seems to offer few benefits over DNS seeding, while potentially > potentially creating a vulnerable hot spot in the DHT. Sybil attacks > on DHTs are well documented. > > -- > Jeff Garzik > exMULTI, Inc. > jgarzik@exmulti•com > [-- Attachment #2: Type: text/html, Size: 2082 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers 2011-06-13 9:38 ` Christian Decker @ 2011-06-13 9:56 ` Jeff Garzik 2011-06-13 10:54 ` Vladimir Marchenko 0 siblings, 1 reply; 10+ messages in thread From: Jeff Garzik @ 2011-06-13 9:56 UTC (permalink / raw) To: Christian Decker; +Cc: bitcoin-development On Mon, Jun 13, 2011 at 5:38 AM, Christian Decker <decker.christian@gmail•com> wrote: > BitTorrent trackers are used to handle several thousands of requests, so > they would probably scale well enough. I'm not even talking about using the > DHT trackers, but using old fashioned HTTP based trackers. The fact that > each bitcoin client would contact the tracker would make it very hard for an > attacker to get bootstrapping clients to exclusively connect to his > compromised clients. I would say that using a tracker such as OpenBittorrent > provides the same advantages as using an IRC channel. And how does the client discover HTTP trackers? You're either hardcoding -those- into the client, or adding an additional bootstrap step to discover them. Either way, it has the same problems as other current methods. The history and experience of gnutella's web caches vs. UDP host caches seems highly relevant here. -- Jeff Garzik exMULTI, Inc. jgarzik@exmulti•com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers 2011-06-13 9:56 ` Jeff Garzik @ 2011-06-13 10:54 ` Vladimir Marchenko 2011-06-13 11:48 ` Christian Decker 0 siblings, 1 reply; 10+ messages in thread From: Vladimir Marchenko @ 2011-06-13 10:54 UTC (permalink / raw) To: Jeff Garzik; +Cc: bitcoin-development one possible bootstrap method of last resort, 1. create a convention of bitcoind listening on a specific last octest of IPv4 address, let's say, .14 when possible. Those of us who have access to IP space would use .14's. 2. if no other bootstrap method works, client could start scanning x.x.x.14 addresses, perhaps in some semi-intelligent order (starting from more pobable /8's and /16's), if enough people place bitcoind on x.x.x.14 than after a 10-100 thousand checks it bound to find a bitcoind peer. It's messy, with all the excessive scanning etc... but it does not depend on anything except a bunch of bitcoind by convention preferring listening on x.x.x.14's. Given that this is a method of last resort in bootrap chain it whould hopefully not lead to DDOS on those unlucky to own *.14 and not running bitcoind there. Also the more people are running bitcoind on .14, the quicker it would find a peer, the less scanning to do. It is kind of self-regualting. For whatever it worth... On 13 June 2011 10:56, Jeff Garzik <jgarzik@exmulti•com> wrote: > On Mon, Jun 13, 2011 at 5:38 AM, Christian Decker > <decker.christian@gmail•com> wrote: >> BitTorrent trackers are used to handle several thousands of requests, so >> they would probably scale well enough. I'm not even talking about using the >> DHT trackers, but using old fashioned HTTP based trackers. The fact that >> each bitcoin client would contact the tracker would make it very hard for an >> attacker to get bootstrapping clients to exclusively connect to his >> compromised clients. I would say that using a tracker such as OpenBittorrent >> provides the same advantages as using an IRC channel. > > And how does the client discover HTTP trackers? You're either > hardcoding -those- into the client, or adding an additional bootstrap > step to discover them. Either way, it has the same problems as other > current methods. > > The history and experience of gnutella's web caches vs. UDP host > caches seems highly relevant here. > > -- > Jeff Garzik > exMULTI, Inc. > jgarzik@exmulti•com > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers 2011-06-13 10:54 ` Vladimir Marchenko @ 2011-06-13 11:48 ` Christian Decker 2011-06-13 16:51 ` Jeff Garzik 0 siblings, 1 reply; 10+ messages in thread From: Christian Decker @ 2011-06-13 11:48 UTC (permalink / raw) To: Vladimir Marchenko; +Cc: bitcoin-development [-- Attachment #1: Type: text/plain, Size: 4245 bytes --] Yes, those trackers would be hard coded, just like the IRC servers and channels are hardcoded right now. The advantages over IRC and DNS Seeds are: - sporadic HTTP requests to a tracker, as opposed to keeping an IRC connection open at all times - no virus/botnet like behaviour (automatically join IRC channel with cryptic name), ISPs tend to bother network admins (like myself) with alerts when they see this... - adapts faster than DNS Seeds which require configuration changes on seed should the nodes become unreachable - we already use HTTP to determine our external IP, so it would be a consolidation of transports - more peers than DNS Seeds (better load balancing) As for Vladimirs proposal, seems like an extreme measure, that is not really practical. Also it leads to network partitions since nodes will prefer their own /8 and /16 networks. IPv6 will also soon be a problem for this method. On Mon, Jun 13, 2011 at 12:54 PM, Vladimir Marchenko < vladimir@marchenko•co.uk> wrote: > one possible bootstrap method of last resort, > > 1. create a convention of bitcoind listening on a specific last octest > of IPv4 address, let's say, .14 when possible. Those of us who have > access to IP space would use .14's. > > 2. if no other bootstrap method works, client could start scanning > x.x.x.14 addresses, perhaps in some semi-intelligent order (starting > from more pobable /8's and /16's), if enough people place bitcoind on > x.x.x.14 than after a 10-100 thousand checks it bound to find a > bitcoind peer. > > It's messy, with all the excessive scanning etc... but it does not > depend on anything except a bunch of bitcoind by convention preferring > listening on x.x.x.14's. > > Given that this is a method of last resort in bootrap chain it whould > hopefully not lead to DDOS on those unlucky to own *.14 and not > running bitcoind there. Also the more people are running bitcoind on > .14, the quicker it would find a peer, the less scanning to do. It is > kind of self-regualting. > > For whatever it worth... > > > On 13 June 2011 10:56, Jeff Garzik <jgarzik@exmulti•com> wrote: > > On Mon, Jun 13, 2011 at 5:38 AM, Christian Decker > > <decker.christian@gmail•com> wrote: > >> BitTorrent trackers are used to handle several thousands of requests, so > >> they would probably scale well enough. I'm not even talking about using > the > >> DHT trackers, but using old fashioned HTTP based trackers. The fact that > >> each bitcoin client would contact the tracker would make it very hard > for an > >> attacker to get bootstrapping clients to exclusively connect to his > >> compromised clients. I would say that using a tracker such as > OpenBittorrent > >> provides the same advantages as using an IRC channel. > > > > And how does the client discover HTTP trackers? You're either > > hardcoding -those- into the client, or adding an additional bootstrap > > step to discover them. Either way, it has the same problems as other > > current methods. > > > > The history and experience of gnutella's web caches vs. UDP host > > caches seems highly relevant here. > > > > -- > > Jeff Garzik > > exMULTI, Inc. > > jgarzik@exmulti•com > > > > > ------------------------------------------------------------------------------ > > EditLive Enterprise is the world's most technically advanced content > > authoring tool. Experience the power of Track Changes, Inline Image > > Editing and ensure content is compliant with Accessibility Checking. > > http://p.sf.net/sfu/ephox-dev2dev > > _______________________________________________ > > Bitcoin-development mailing list > > Bitcoin-development@lists•sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > [-- Attachment #2: Type: text/html, Size: 5522 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers 2011-06-13 11:48 ` Christian Decker @ 2011-06-13 16:51 ` Jeff Garzik 2011-06-13 18:00 ` Vladimir Marchenko 0 siblings, 1 reply; 10+ messages in thread From: Jeff Garzik @ 2011-06-13 16:51 UTC (permalink / raw) To: Christian Decker; +Cc: bitcoin-development On Mon, Jun 13, 2011 at 7:48 AM, Christian Decker <decker.christian@gmail•com> wrote: > The advantages over IRC and DNS Seeds are: > - sporadic HTTP requests to a tracker, as opposed to keeping an IRC > connection open at all times No advantage over DNS (you're talking about IRC) > - no virus/botnet like behaviour (automatically join IRC channel with > cryptic name), ISPs tend to bother network admins (like myself) with alerts > when they see this... No advantage over DNS (you're talking about IRC) > - adapts faster than DNS Seeds which require configuration changes on seed > should the nodes become unreachable False. A proper DNS seed actively samples the bitcoin network for live nodes. > - we already use HTTP to determine our external IP, so it would be a > consolidation of transports Irrelevant -- it's still a bunch of additional code. We "already use" DNS lookups as well, so I could apply similar logic to DNS seeding. > - more peers than DNS Seeds (better load balancing) This is an invalid assumption, based on an overly simplistic sampling. DNS seeds are perfectly capable of returning new and useful peers on each query. -- Jeff Garzik exMULTI, Inc. jgarzik@exmulti•com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers 2011-06-13 16:51 ` Jeff Garzik @ 2011-06-13 18:00 ` Vladimir Marchenko 2011-06-13 18:41 ` Gavin 0 siblings, 1 reply; 10+ messages in thread From: Vladimir Marchenko @ 2011-06-13 18:00 UTC (permalink / raw) To: bitcoin-development Jeff is absolutely correct, stating that DNS bootstrapping can potentially be very robust. Consider, for example, that seeds can be hosted by services like zerigo, who provide decent management API's, as well as by thousands of small VPS operated by people all over the world. Moreover, if namecoin lives up to it's promises than some seeds can be hosted using it's tlds, and as such we would get around centralised ICANN/registrar weakness. The most practical way I see how to improve DNS bootstrapping is publishing build instructions for a simple DNS server build and a script which feeds it with node data. Something simple like djbdns plus a perl script, or similar zerigo API script, would do. Than anyone can host it. All left to do than is to hardcode a bunch of such DNS seeds into the client. Hardcoding seeds is not good enough? Get a convention that anyone owning a bitcoin.* domain should point seed.bitcoin.* to a decent DNS seed. Than clients simply scan 100-200 tld's to find working seed.bitcoin.* I am not quite sure, though, is if this would enable some attacks by poisoned seeds. Vladimir. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers 2011-06-13 18:00 ` Vladimir Marchenko @ 2011-06-13 18:41 ` Gavin 2011-06-13 20:16 ` Jeff Garzik 0 siblings, 1 reply; 10+ messages in thread From: Gavin @ 2011-06-13 18:41 UTC (permalink / raw) To: Vladimir Marchenko; +Cc: bitcoin-development > The most practical way I see how to improve DNS bootstrapping is > publishing build instructions for a simple DNS server build and a > script which feeds it with node data. Something simple like djbdns plus > a perl script, or similar zerigo API script, would do. Than anyone can > host it. All left to do than is to hardcode a bunch of such DNS seeds > into the client. Adding that code to bitcoin in a contrib/dns folder seems like a good idea. I'd be happy to run a dns seed. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers 2011-06-13 18:41 ` Gavin @ 2011-06-13 20:16 ` Jeff Garzik 0 siblings, 0 replies; 10+ messages in thread From: Jeff Garzik @ 2011-06-13 20:16 UTC (permalink / raw) To: Gavin; +Cc: bitcoin-development On Mon, Jun 13, 2011 at 2:41 PM, Gavin <gavinandresen@gmail•com> wrote: >> The most practical way I see how to improve DNS bootstrapping is >> publishing build instructions for a simple DNS server build and a >> script which feeds it with node data. Something simple like djbdns plus >> a perl script, or similar zerigo API script, would do. Than anyone can >> host it. All left to do than is to hardcode a bunch of such DNS seeds >> into the client. > > Adding that code to bitcoin in a contrib/dns folder seems like a good idea. Agreed. I'll see if I can put together a couple different solutions, and stick them in there. ETA 3-4 weeks though, given current bugfix workload + personal issues (moving to a new house). I had a thought related to this, but readily admit I haven't thought this through: create an RPC in bitcoin that returns a random selection of fresh P2P node addresses, from addr.dat. One problem I was running into was accessing this data from outside bitcoin. Using ArtForz' half-a-node as a base was a possibility, but creating "getnodes" RPC seemed far easier. djbdns (tinydns) is great for this: it is highly scalable and stable, and its input is a -very- simple text format, easier to generate than BIND zone files. > I'd be happy to run a dns seed. Great! A big missing piece is simply trusted people running DNS seeds on stable connectivity, once the DNS software is there. -- Jeff Garzik exMULTI, Inc. jgarzik@exmulti•com ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2011-06-13 20:16 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-06-13 8:55 [Bitcoin-development] Bootstrapping via BitTorrent trackers Christian Decker 2011-06-13 9:09 ` Jeff Garzik 2011-06-13 9:38 ` Christian Decker 2011-06-13 9:56 ` Jeff Garzik 2011-06-13 10:54 ` Vladimir Marchenko 2011-06-13 11:48 ` Christian Decker 2011-06-13 16:51 ` Jeff Garzik 2011-06-13 18:00 ` Vladimir Marchenko 2011-06-13 18:41 ` Gavin 2011-06-13 20:16 ` Jeff Garzik
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox