public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed

* [Bitcoin-development] DNS seeds returning gone peers
@ 2011-08-03 10:04 Mike Hearn
  2011-08-03 11:38 ` Matt Corallo
  2011-08-03 14:00 ` Rick Wesson
  0 siblings, 2 replies; 15+ messages in thread
From: Mike Hearn @ 2011-08-03 10:04 UTC (permalink / raw)
  To: Bitcoin Dev

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

This is expected to happen from time to time of course as it's inherently
racy, but there are a *lot* of bad nodes appearing in the DNS seeds.

$ nmap -oG /tmp/x -p 8333 `dig +short bitseed.bitcoin.org.uk
dnsseed.bluematt.me bitseed.xf2.org`
...
Nmap done: 48 IP addresses (25 hosts up) scanned in 9.80 seconds

$ grep -c 'closed' /tmp/x
6

So of 48 IPs returned only 19 are actually usable. This is slowing down peer
bringup for the Android apps, which don't currently save the addresses of
last-used peers (yes, I know we should fix this).

I was talking to a friend a few days ago about Bitcoin, he seemed
interested. I'm hoping he might take on DNS seeding as a project. A custom
DNS server that watches the network to find long-lived peers that run the
latest version would be helpful for resolving this kind of thing.

[-- Attachment #2: Type: text/html, Size: 1601 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

• * Re: [Bitcoin-development] DNS seeds returning gone peers
    2011-08-03 10:04 [Bitcoin-development] DNS seeds returning gone peers Mike Hearn
  @ 2011-08-03 11:38 ` Matt Corallo
    2011-08-03 12:00   ` Mike Hearn
    2011-08-03 12:27   ` Caleb James DeLisle
    2011-08-03 14:00 ` Rick Wesson
    1 sibling, 2 replies; 15+ messages in thread
  From: Matt Corallo @ 2011-08-03 11:38 UTC (permalink / raw)
    To: bitcoin-development
  
  [-- Attachment #1: Type: text/plain, Size: 2038 bytes --]
  
  On Wed, 2011-08-03 at 12:04 +0200, Mike Hearn wrote:
  > This is expected to happen from time to time of course as it's
  > inherently racy, but there are a lot of bad nodes appearing in the DNS
  > seeds.
  
  > $ nmap -oG /tmp/x -p 8333 `dig +short bitseed.bitcoin.org.uk
  > dnsseed.bluematt.me bitseed.xf2.org`
  > ...
  > Nmap done: 48 IP addresses (25 hosts up) scanned in 9.80 seconds
  
  > $ grep -c 'closed' /tmp/x
  > 6
  
  > So of 48 IPs returned only 19 are actually usable. This is slowing
  > down peer bringup for the Android apps, which don't currently save the
  > addresses of last-used peers (yes, I know we should fix this).
  Its actually much, much less.  You forgot to grep for filtered, which
  are also worthless and you didn't make an actual connection to the node,
  meaning there is no way to tell if the node has its connection slots
  full (a node which has the maximum connection count will ack a syn, but
  will drop the connection after the first message, so nmap thinks the
  port is open).
  I just tested and I show 0 accepting from bitseed.xf2.org and 0 from
  bitcoin.bitcoin.co.uk.  dnsseed.bluematt.me rotates every 2 minutes to
  the most recently checked so it tends to be pretty good if you get it
  right after a rotate, if you wait to long, those slots fill up quick.
  > 
  > I was talking to a friend a few days ago about Bitcoin, he seemed
  > interested. I'm hoping he might take on DNS seeding as a project. A
  > custom DNS server that watches the network to find long-lived peers
  > that run the latest version would be helpful for resolving this kind
  > of thing.
  Point him to https://github.com/TheBlueMatt/dnsseed it could use a bit
  of cleanup, but it works.
  If a different DNS Server were used to could pull directly from the
  database in a more dynamic way it would probably work better too (it was
  originally set up on MySQL and PowerDNS, but that is quite a resource
  hog compared to SQLite and BIND, but the original backend is still there
  and could work if you have a beefy enough server).
  
  Matt
  
  [-- Attachment #2: This is a digitally signed message part --]
  [-- Type: application/pgp-signature, Size: 836 bytes --]
  
  ^ permalink raw reply	[flat|nested] 15+ messages in thread

• • * Re: [Bitcoin-development] DNS seeds returning gone peers
      2011-08-03 11:38 ` Matt Corallo
    @ 2011-08-03 12:00   ` Mike Hearn
      2011-08-03 12:15     ` Johannes Henninger
      2011-08-03 12:27   ` Caleb James DeLisle
      1 sibling, 1 reply; 15+ messages in thread
    From: Mike Hearn @ 2011-08-03 12:00 UTC (permalink / raw)
      To: Matt Corallo; +Cc: bitcoin-development
    
    [-- Attachment #1: Type: text/plain, Size: 2077 bytes --]
    
    Yeah, the limited number of slots doesn't help either. I wonder if the
    current settings are too conservative.
    
    Your seed returns quite a few IPs that don't respond to ICMP pings - not
    sure what is happening there.
    
    03:01:53 scott:~$ for ip in `dig +short dnsseed.bluematt.me`; do echo -n -e
    "Testing $ip:   "; ping -c 1 $ip|grep 'packet loss'; done
    Testing 24.7.158.162:   1 packets transmitted, 1 received, 0% packet loss,
    time 0ms
    *Testing 50.19.225.254:   1 packets transmitted, 0 received, 100% packet
    loss, time 0ms
    Testing 67.242.10.199:   1 packets transmitted, 0 received, 100% packet
    loss, time 0ms
    Testing 72.223.56.138:   1 packets transmitted, 0 received, 100% packet
    loss, time 0ms
    Testing 76.92.171.255:   1 packets transmitted, 0 received, 100% packet
    loss, time 0ms
    Testing 76.123.10.117:   1 packets transmitted, 0 received, 100% packet
    loss, time 0ms
    Testing 80.3.173.28:   1 packets transmitted, 0 received, 100% packet loss,
    time 0ms
    *Testing 80.101.109.52:   1 packets transmitted, 1 received, 0% packet loss,
    time 0ms
    Testing 93.186.32.117:   1 packets transmitted, 1 received, 0% packet loss,
    time 0ms
    *Testing 94.19.17.167:   1 packets transmitted, 0 received, 100% packet
    loss, time 0ms
    *Testing 97.86.39.50:   1 packets transmitted, 1 received, 0% packet loss,
    time 0ms
    Testing 113.255.177.241:   1 packets transmitted, 1 received, 0% packet
    loss, time 0ms
    Testing 118.208.226.66:   1 packets transmitted, 1 received, 0% packet loss,
    time 0ms
    Testing 173.180.141.86:   1 packets transmitted, 1 received, 0% packet loss,
    time 0ms
    *Testing 174.119.14.66:   1 packets transmitted, 0 received, 100% packet
    loss, time 0ms
    *Testing 178.18.129.133:   1 packets transmitted, 1 received, 0% packet
    loss, time 0ms
    *Testing 193.86.163.77:   1 packets transmitted, 0 received, 100% packet
    loss, time 0ms
    Testing 206.255.99.164:   1 packets transmitted, 0 received, 100% packet
    loss, time 0ms
    Testing 216.8.180.85:   1 packets transmitted, 0 received, 100% packet loss,
    time 0ms
    Testing 24.1.117.3:   1 packets transmitted, 0 received, 100% packet loss,
    time 0ms*
    
    [-- Attachment #2: Type: text/html, Size: 2981 bytes --]
    
    ^ permalink raw reply	[flat|nested] 15+ messages in thread

  • • * Re: [Bitcoin-development] DNS seeds returning gone peers
        2011-08-03 12:00   ` Mike Hearn
      @ 2011-08-03 12:15     ` Johannes Henninger
        2011-08-03 12:17       ` Mike Hearn
        0 siblings, 1 reply; 15+ messages in thread
      From: Johannes Henninger @ 2011-08-03 12:15 UTC (permalink / raw)
        To: bitcoin-development
      
      On 08/03/2011 02:00 PM, Mike Hearn wrote:
      > Yeah, the limited number of slots doesn't help either. I wonder if the
      > current settings are too conservative.
      >
      > Your seed returns quite a few IPs that don't respond to ICMP pings -
      > not sure what is happening there.
      Many home routers ignore ICMP requests (mine does) and as far as I know
      Windows Vista/7 only responds to pings that originate from a LAN
      address/private address range in order to stay more "hidden".
      
      
      
      ^ permalink raw reply	[flat|nested] 15+ messages in thread

    • • * Re: [Bitcoin-development] DNS seeds returning gone peers
          2011-08-03 12:15     ` Johannes Henninger
        @ 2011-08-03 12:17       ` Mike Hearn
          0 siblings, 0 replies; 15+ messages in thread
        From: Mike Hearn @ 2011-08-03 12:17 UTC (permalink / raw)
          To: Johannes Henninger; +Cc: bitcoin-development
        
        [-- Attachment #1: Type: text/plain, Size: 1309 bytes --]
        
        OK, but, <50% show as port open .... maybe a bug in the seed?
        
        
        05:16:52 scott:~$ nmap -p 8333 `dig +short dnsseed.bluematt.me`
        
        Starting Nmap 5.00 ( http://nmap.org ) at 2011-08-03 14:17 CEST
        Interesting ports on 83.220.45.22:
        PORT     STATE SERVICE
        8333/tcp open  unknown
        
        Interesting ports on
        c-6de0e055.27-2-64736c11.cust.bredbandsbolaget.se(85.224.224.109):
        PORT     STATE SERVICE
        8333/tcp open  unknown
        
        Interesting ports on mackila.com (88.168.105.251):
        PORT     STATE SERVICE
        8333/tcp open  unknown
        
        Interesting ports on 93-81-112-85.broadband.corbina.ru (93.81.112.85):
        PORT     STATE SERVICE
        8333/tcp open  unknown
        
        Interesting ports on kons-5f710a2a.pool.mediaWays.net (95.113.10.42):
        PORT     STATE SERVICE
        8333/tcp open  unknown
        
        Interesting ports on 173-218-216-132.atw.suddenlink.net (173.218.216.132):
        PORT     STATE SERVICE
        8333/tcp open  unknown
        
        Interesting ports on
        dynamicip-188-232-23-24.pppoe.omsk.ertelecom.ru(188.232.23.24):
        PORT     STATE SERVICE
        8333/tcp open  unknown
        
        Interesting ports on c-71-229-116-166.hsd1.fl.comcast.net (71.229.116.166):
        PORT     STATE    SERVICE
        8333/tcp filtered unknown
        
        Interesting ports on c-76-25-209-23.hsd1.co.comcast.net (76.25.209.23):
        PORT     STATE SERVICE
        8333/tcp open  unknown
        
        Nmap done: 20 IP addresses (9 hosts up) scanned in 3.50 seconds
        
        [-- Attachment #2: Type: text/html, Size: 2412 bytes --]
        
        ^ permalink raw reply	[flat|nested] 15+ messages in thread

  • * Re: [Bitcoin-development] DNS seeds returning gone peers
      2011-08-03 11:38 ` Matt Corallo
      2011-08-03 12:00   ` Mike Hearn
    @ 2011-08-03 12:27   ` Caleb James DeLisle
      2011-08-03 12:40     ` Mike Hearn
      1 sibling, 1 reply; 15+ messages in thread
    From: Caleb James DeLisle @ 2011-08-03 12:27 UTC (permalink / raw)
      To: bitcoin-development
    
    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1
    
    
    
    On 08/03/2011 07:38 AM, Matt Corallo wrote:
    > On Wed, 2011-08-03 at 12:04 +0200, Mike Hearn wrote:
    >> This is expected to happen from time to time of course as it's
    >> inherently racy, but there are a lot of bad nodes appearing in the DNS
    >> seeds.
    > 
    >> $ nmap -oG /tmp/x -p 8333 `dig +short bitseed.bitcoin.org.uk
    >> dnsseed.bluematt.me bitseed.xf2.org`
    >> ...
    >> Nmap done: 48 IP addresses (25 hosts up) scanned in 9.80 seconds
    > 
    >> $ grep -c 'closed' /tmp/x
    >> 6
    > 
    >> So of 48 IPs returned only 19 are actually usable. This is slowing
    >> down peer bringup for the Android apps, which don't currently save the
    >> addresses of last-used peers (yes, I know we should fix this).
    > Its actually much, much less.  You forgot to grep for filtered, which
    > are also worthless and you didn't make an actual connection to the node,
    > meaning there is no way to tell if the node has its connection slots
    > full (a node which has the maximum connection count will ack a syn, but
    > will drop the connection after the first message, so nmap thinks the
    > port is open).
    > I just tested and I show 0 accepting from bitseed.xf2.org and 0 from
    > bitcoin.bitcoin.co.uk.  dnsseed.bluematt.me rotates every 2 minutes to
    > the most recently checked so it tends to be pretty good if you get it
    > right after a rotate, if you wait to long, those slots fill up quick.
    
    Someone I know who runs a moderately large website told me that some ISPs cache DNS for as long as a week without regard to TTL.
    If your DNS seeds are not pointing to your own dedicated boxen then you might want to do a lookup on a random cookie as a subdomain.
    
    Caleb
    
    >>
    >> I was talking to a friend a few days ago about Bitcoin, he seemed
    >> interested. I'm hoping he might take on DNS seeding as a project. A
    >> custom DNS server that watches the network to find long-lived peers
    >> that run the latest version would be helpful for resolving this kind
    >> of thing.
    > Point him to https://github.com/TheBlueMatt/dnsseed it could use a bit
    > of cleanup, but it works.
    > If a different DNS Server were used to could pull directly from the
    > database in a more dynamic way it would probably work better too (it was
    > originally set up on MySQL and PowerDNS, but that is quite a resource
    > hog compared to SQLite and BIND, but the original backend is still there
    > and could work if you have a beefy enough server).
    > 
    > Matt
    > 
    > 
    > 
    > ------------------------------------------------------------------------------
    > BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
    > The must-attend event for mobile developers. Connect with experts. 
    > Get tools for creating Super Apps. See the latest technologies.
    > Sessions, hands-on labs, demos & much more. Register early & save!
    > http://p.sf.net/sfu/rim-blackberry-1
    > 
    > 
    > 
    > _______________________________________________
    > Bitcoin-development mailing list
    > Bitcoin-development@lists•sourceforge.net
    > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)
    
    iQIcBAEBAgAGBQJOOT64AAoJECYAmptlsgnWNNYP/3pk/61AOaTo8577XF2E3RWU
    hqPclOYMveBZveo7Hz0k/Vf3TMf5p7CFGtFllApVmmR5ck4hXwow+tXwaYvBuf9b
    4QwDlNtBelZ+7DqOdDTxRjaRzBo7PlsuiEp+6B3+oggfjKWnkWlIighfM/6LOtMO
    kv+MsC9xulqCrX96FrQBERVknkvza8NWfVblAFCM0uxECC5Hd52W1Okx1cDqUsIj
    MAp6T6IDwy1u0WtYIZBbD3KR802tqTpx/gzElo2AAz4ZR8P9yATBbAjmd9vZDscB
    SRF5yg+BIAzTQzz84c0cno8Q6hFyieRLTu9x0AWUqAZPTL3OgYV7hyl9WXMzcQGY
    LgnG9hP5N9qn2S+rYJuNJwvTJhzaLgUwOdRgyisI8v86R5vEjOpAAPVStAgCtvw2
    6BJJvit3iZ74fK37kPb4iEljd11ibb8rOoiZzKSuS6LMJXkCplQQ06Uhy9MFd/Wn
    UZbSwbXqizJmQUPnHfCvvJc9fmoPFbj4SpYagxXFbUXtQsuB84h1e/jKAf5cvRed
    n0fLfKBZJuAfK1B/lV8+R6+oJj6F2OSGdCpdGzMlrxKX3JPcDJGK56/JmFzaqXyA
    ScaYja4yps83l80zt7H9Fadl642tNWXsdETniyDt2ADnqr1u/nWr9m+aT7oskbZA
    SfO+U22w9JC0CW8u5fAa
    =7WNK
    -----END PGP SIGNATURE-----
    
    
    
    
    ^ permalink raw reply	[flat|nested] 15+ messages in thread

  • • * Re: [Bitcoin-development] DNS seeds returning gone peers
        2011-08-03 12:27   ` Caleb James DeLisle
      @ 2011-08-03 12:40     ` Mike Hearn
        0 siblings, 0 replies; 15+ messages in thread
      From: Mike Hearn @ 2011-08-03 12:40 UTC (permalink / raw)
        To: Caleb James DeLisle; +Cc: bitcoin-development
      
      [-- Attachment #1: Type: text/plain, Size: 484 bytes --]
      
      >
      > Someone I know who runs a moderately large website told me that some ISPs
      > cache DNS for as long as a week without regard to TTL.
      >
      
      We track DNS disobeyers at Google, as we use it for load balancing (along
      with many other large sites).
      
      I'd be kind of surprised if any large/professional ISP disobeyed the TTL
      that badly, because it would cause frequent problems reaching popular sites
      like anything hosted on Google or Akamai. But randomizing the DNS request
      isn't a bad idea.
      
      [-- Attachment #2: Type: text/html, Size: 688 bytes --]
      
      ^ permalink raw reply	[flat|nested] 15+ messages in thread

• * Re: [Bitcoin-development] DNS seeds returning gone peers
    2011-08-03 10:04 [Bitcoin-development] DNS seeds returning gone peers Mike Hearn
    2011-08-03 11:38 ` Matt Corallo
  @ 2011-08-03 14:00 ` Rick Wesson
    2011-08-03 14:10   ` Mike Hearn
    1 sibling, 1 reply; 15+ messages in thread
  From: Rick Wesson @ 2011-08-03 14:00 UTC (permalink / raw)
    To: Mike Hearn; +Cc: Bitcoin Dev
  
  [-- Attachment #1: Type: text/plain, Size: 1630 bytes --]
  
  Mike,
  
  I think I can contribute to your DNS seeding project. Could you help define
  long-lived peers?
  
  -rick
  
  
  On Wed, Aug 3, 2011 at 3:04 AM, Mike Hearn <mike@plan99•net> wrote:
  
  > This is expected to happen from time to time of course as it's inherently
  > racy, but there are a *lot* of bad nodes appearing in the DNS seeds.
  >
  > $ nmap -oG /tmp/x -p 8333 `dig +short bitseed.bitcoin.org.uk
  > dnsseed.bluematt.me bitseed.xf2.org`
  > ...
  > Nmap done: 48 IP addresses (25 hosts up) scanned in 9.80 seconds
  >
  > $ grep -c 'closed' /tmp/x
  > 6
  >
  > So of 48 IPs returned only 19 are actually usable. This is slowing down
  > peer bringup for the Android apps, which don't currently save the addresses
  > of last-used peers (yes, I know we should fix this).
  >
  > I was talking to a friend a few days ago about Bitcoin, he seemed
  > interested. I'm hoping he might take on DNS seeding as a project. A custom
  > DNS server that watches the network to find long-lived peers that run the
  > latest version would be helpful for resolving this kind of thing.
  >
  >
  > ------------------------------------------------------------------------------
  > BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
  > The must-attend event for mobile developers. Connect with experts.
  > Get tools for creating Super Apps. See the latest technologies.
  > Sessions, hands-on labs, demos & much more. Register early & save!
  > http://p.sf.net/sfu/rim-blackberry-1
  > _______________________________________________
  > Bitcoin-development mailing list
  > Bitcoin-development@lists•sourceforge.net
  > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
  >
  >
  
  [-- Attachment #2: Type: text/html, Size: 2831 bytes --]
  
  ^ permalink raw reply	[flat|nested] 15+ messages in thread

• • * Re: [Bitcoin-development] DNS seeds returning gone peers
      2011-08-03 14:00 ` Rick Wesson
    @ 2011-08-03 14:10   ` Mike Hearn
      2011-08-03 14:18     ` Rick Wesson
      0 siblings, 1 reply; 15+ messages in thread
    From: Mike Hearn @ 2011-08-03 14:10 UTC (permalink / raw)
      To: Rick Wesson; +Cc: Bitcoin Dev
    
    [-- Attachment #1: Type: text/plain, Size: 2142 bytes --]
    
    There's no project currently :-)
    
    Starting from Matts code is probably the way to go. It's written in PHP.
    Alternatively, you could write a Java app for it, as there are drop-in DNS
    serving libraries you could link with BitCoinJ+sqlite. It probably wouldn't
    be that hard. You'd want to sort nodes by version, how long they've been
    observed to exist, the last polling time, etc.
    
    On Wed, Aug 3, 2011 at 4:00 PM, Rick Wesson
    <rick@support-intelligence•com>wrote:
    
    > Mike,
    >
    > I think I can contribute to your DNS seeding project. Could you help define
    > long-lived peers?
    >
    > -rick
    >
    >
    > On Wed, Aug 3, 2011 at 3:04 AM, Mike Hearn <mike@plan99•net> wrote:
    >
    >> This is expected to happen from time to time of course as it's inherently
    >> racy, but there are a *lot* of bad nodes appearing in the DNS seeds.
    >>
    >> $ nmap -oG /tmp/x -p 8333 `dig +short bitseed.bitcoin.org.uk
    >> dnsseed.bluematt.me bitseed.xf2.org`
    >> ...
    >> Nmap done: 48 IP addresses (25 hosts up) scanned in 9.80 seconds
    >>
    >> $ grep -c 'closed' /tmp/x
    >> 6
    >>
    >> So of 48 IPs returned only 19 are actually usable. This is slowing down
    >> peer bringup for the Android apps, which don't currently save the addresses
    >> of last-used peers (yes, I know we should fix this).
    >>
    >> I was talking to a friend a few days ago about Bitcoin, he seemed
    >> interested. I'm hoping he might take on DNS seeding as a project. A custom
    >> DNS server that watches the network to find long-lived peers that run the
    >> latest version would be helpful for resolving this kind of thing.
    >>
    >>
    >> ------------------------------------------------------------------------------
    >> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
    >> The must-attend event for mobile developers. Connect with experts.
    >> Get tools for creating Super Apps. See the latest technologies.
    >> Sessions, hands-on labs, demos & much more. Register early & save!
    >> http://p.sf.net/sfu/rim-blackberry-1
    >> _______________________________________________
    >> Bitcoin-development mailing list
    >> Bitcoin-development@lists•sourceforge.net
    >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
    >>
    >>
    >
    
    [-- Attachment #2: Type: text/html, Size: 3747 bytes --]
    
    ^ permalink raw reply	[flat|nested] 15+ messages in thread

  • • * Re: [Bitcoin-development] DNS seeds returning gone peers
        2011-08-03 14:10   ` Mike Hearn
      @ 2011-08-03 14:18     ` Rick Wesson
        2011-08-03 14:28       ` Douglas Huff
        2011-08-03 14:37       ` Christian Decker
        0 siblings, 2 replies; 15+ messages in thread
      From: Rick Wesson @ 2011-08-03 14:18 UTC (permalink / raw)
        To: Mike Hearn; +Cc: Bitcoin Dev
      
      [-- Attachment #1: Type: text/plain, Size: 2498 bytes --]
      
      Starting from bitcoinj, I have plenty of ways to publish DNS. Why sort them
      by version? Ordering from highest to lowest?
      
      how about publishing addresses under version.example.com if you version has
      a perfrence?
      
      -rick
      
      
      On Wed, Aug 3, 2011 at 7:10 AM, Mike Hearn <mike@plan99•net> wrote:
      
      > There's no project currently :-)
      >
      > Starting from Matts code is probably the way to go. It's written in PHP.
      > Alternatively, you could write a Java app for it, as there are drop-in DNS
      > serving libraries you could link with BitCoinJ+sqlite. It probably wouldn't
      > be that hard. You'd want to sort nodes by version, how long they've been
      > observed to exist, the last polling time, etc.
      >
      > On Wed, Aug 3, 2011 at 4:00 PM, Rick Wesson <rick@support-intelligence•com
      > > wrote:
      >
      >> Mike,
      >>
      >> I think I can contribute to your DNS seeding project. Could you help
      >> define long-lived peers?
      >>
      >> -rick
      >>
      >>
      >> On Wed, Aug 3, 2011 at 3:04 AM, Mike Hearn <mike@plan99•net> wrote:
      >>
      >>> This is expected to happen from time to time of course as it's inherently
      >>> racy, but there are a *lot* of bad nodes appearing in the DNS seeds.
      >>>
      >>> $ nmap -oG /tmp/x -p 8333 `dig +short bitseed.bitcoin.org.uk
      >>> dnsseed.bluematt.me bitseed.xf2.org`
      >>> ...
      >>> Nmap done: 48 IP addresses (25 hosts up) scanned in 9.80 seconds
      >>>
      >>> $ grep -c 'closed' /tmp/x
      >>> 6
      >>>
      >>> So of 48 IPs returned only 19 are actually usable. This is slowing down
      >>> peer bringup for the Android apps, which don't currently save the addresses
      >>> of last-used peers (yes, I know we should fix this).
      >>>
      >>> I was talking to a friend a few days ago about Bitcoin, he seemed
      >>> interested. I'm hoping he might take on DNS seeding as a project. A custom
      >>> DNS server that watches the network to find long-lived peers that run the
      >>> latest version would be helpful for resolving this kind of thing.
      >>>
      >>>
      >>> ------------------------------------------------------------------------------
      >>> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
      >>> The must-attend event for mobile developers. Connect with experts.
      >>> Get tools for creating Super Apps. See the latest technologies.
      >>> Sessions, hands-on labs, demos & much more. Register early & save!
      >>> http://p.sf.net/sfu/rim-blackberry-1
      >>> _______________________________________________
      >>> Bitcoin-development mailing list
      >>> Bitcoin-development@lists•sourceforge.net
      >>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
      >>>
      >>>
      >>
      >
      
      [-- Attachment #2: Type: text/html, Size: 4400 bytes --]
      
      ^ permalink raw reply	[flat|nested] 15+ messages in thread

    • • * Re: [Bitcoin-development] DNS seeds returning gone peers
          2011-08-03 14:18     ` Rick Wesson
        @ 2011-08-03 14:28       ` Douglas Huff
          2011-08-03 14:39         ` Mike Hearn
          2011-08-03 14:37       ` Christian Decker
          1 sibling, 1 reply; 15+ messages in thread
        From: Douglas Huff @ 2011-08-03 14:28 UTC (permalink / raw)
          To: Rick Wesson; +Cc: Bitcoin Dev
        
        [-- Attachment #1: Type: text/plain, Size: 3069 bytes --]
        
        There's a bigger problem here honestly. The p2p network is just starved for
        connectable slots.
        
        You can start up a bitcoind, on a host with sufficient connectivity, with
        -maxconnections=512 and they will fill in <15 minutes.
        
        As to why sort by version: <=3.23 has problems serving the blocks from the
        last 2 months. It can take days to weeks to get the whole chain if you're
        connecting to those nodes.
        
        -- 
        Douglas Huff
        On Aug 3, 2011 9:18 AM, "Rick Wesson" <rick@support-intelligence•com> wrote:
        > Starting from bitcoinj, I have plenty of ways to publish DNS. Why sort
        them
        > by version? Ordering from highest to lowest?
        >
        > how about publishing addresses under version.example.com if you version
        has
        > a perfrence?
        >
        > -rick
        >
        >
        > On Wed, Aug 3, 2011 at 7:10 AM, Mike Hearn <mike@plan99•net> wrote:
        >
        >> There's no project currently :-)
        >>
        >> Starting from Matts code is probably the way to go. It's written in PHP.
        >> Alternatively, you could write a Java app for it, as there are drop-in
        DNS
        >> serving libraries you could link with BitCoinJ+sqlite. It probably
        wouldn't
        >> be that hard. You'd want to sort nodes by version, how long they've been
        >> observed to exist, the last polling time, etc.
        >>
        >> On Wed, Aug 3, 2011 at 4:00 PM, Rick Wesson <
        rick@support-intelligence•com
        >> > wrote:
        >>
        >>> Mike,
        >>>
        >>> I think I can contribute to your DNS seeding project. Could you help
        >>> define long-lived peers?
        >>>
        >>> -rick
        >>>
        >>>
        >>> On Wed, Aug 3, 2011 at 3:04 AM, Mike Hearn <mike@plan99•net> wrote:
        >>>
        >>>> This is expected to happen from time to time of course as it's
        inherently
        >>>> racy, but there are a *lot* of bad nodes appearing in the DNS seeds.
        >>>>
        >>>> $ nmap -oG /tmp/x -p 8333 `dig +short bitseed.bitcoin.org.uk
        >>>> dnsseed.bluematt.me bitseed.xf2.org`
        >>>> ...
        >>>> Nmap done: 48 IP addresses (25 hosts up) scanned in 9.80 seconds
        >>>>
        >>>> $ grep -c 'closed' /tmp/x
        >>>> 6
        >>>>
        >>>> So of 48 IPs returned only 19 are actually usable. This is slowing down
        >>>> peer bringup for the Android apps, which don't currently save the
        addresses
        >>>> of last-used peers (yes, I know we should fix this).
        >>>>
        >>>> I was talking to a friend a few days ago about Bitcoin, he seemed
        >>>> interested. I'm hoping he might take on DNS seeding as a project. A
        custom
        >>>> DNS server that watches the network to find long-lived peers that run
        the
        >>>> latest version would be helpful for resolving this kind of thing.
        >>>>
        >>>>
        >>>>
        ------------------------------------------------------------------------------
        >>>> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
        >>>> The must-attend event for mobile developers. Connect with experts.
        >>>> Get tools for creating Super Apps. See the latest technologies.
        >>>> Sessions, hands-on labs, demos & much more. Register early & save!
        >>>> http://p.sf.net/sfu/rim-blackberry-1
        >>>> _______________________________________________
        >>>> Bitcoin-development mailing list
        >>>> Bitcoin-development@lists•sourceforge.net
        >>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
        >>>>
        >>>>
        >>>
        >>
        
        [-- Attachment #2: Type: text/html, Size: 4625 bytes --]
        
        ^ permalink raw reply	[flat|nested] 15+ messages in thread

      • • * Re: [Bitcoin-development] DNS seeds returning gone peers
            2011-08-03 14:28       ` Douglas Huff
          @ 2011-08-03 14:39         ` Mike Hearn
            2011-08-03 14:48           ` Gregory Maxwell
            0 siblings, 1 reply; 15+ messages in thread
          From: Mike Hearn @ 2011-08-03 14:39 UTC (permalink / raw)
            To: Douglas Huff; +Cc: Bitcoin Dev
          
          [-- Attachment #1: Type: text/plain, Size: 426 bytes --]
          
          >
          > There's a bigger problem here honestly. The p2p network is just starved for
          > connectable slots.
          >
          Suggestions:
          
          - massively increasing all the anti-DoS limits in 0.4, so far they've caused
          a lot more damage than they solved.
          
          - broadcasting an "upgrade now" type announcement. The alert system is
          intended for informing users when there's a threat to system stability and
          that's exactly what has been happening recently.
          
          [-- Attachment #2: Type: text/html, Size: 654 bytes --]
          
          ^ permalink raw reply	[flat|nested] 15+ messages in thread

        • • * Re: [Bitcoin-development] DNS seeds returning gone peers
              2011-08-03 14:39         ` Mike Hearn
            @ 2011-08-03 14:48           ` Gregory Maxwell
              2011-08-03 15:32             ` Matt Corallo
              0 siblings, 1 reply; 15+ messages in thread
            From: Gregory Maxwell @ 2011-08-03 14:48 UTC (permalink / raw)
              To: Mike Hearn; +Cc: Bitcoin Dev
            
            On Wed, Aug 3, 2011 at 10:39 AM, Mike Hearn <mike@plan99•net> wrote:
            >> There's a bigger problem here honestly. The p2p network is just starved
            >> for connectable slots.
            >
            > Suggestions:
            > - massively increasing all the anti-DoS limits in 0.4, so far they've caused
            > a lot more damage than they solved.
            
            You can't "massively increase" the number of available connection
            slots without risking running nodes on lower memory systems (e.g. VMs)
            out of memory.
            
            Moreover, 125 slots should be more than enough.  We need to figure out
            why it isn't.
            
            
            
            ^ permalink raw reply	[flat|nested] 15+ messages in thread

          • • * Re: [Bitcoin-development] DNS seeds returning gone peers
                2011-08-03 14:48           ` Gregory Maxwell
              @ 2011-08-03 15:32             ` Matt Corallo
                0 siblings, 0 replies; 15+ messages in thread
              From: Matt Corallo @ 2011-08-03 15:32 UTC (permalink / raw)
                To: bitcoin-development
              
              On Wed, 2011-08-03 at 10:48 -0400, Gregory Maxwell wrote:
              > On Wed, Aug 3, 2011 at 10:39 AM, Mike Hearn <mike@plan99•net> wrote:
              > >> There's a bigger problem here honestly. The p2p network is just starved
              > >> for connectable slots.
              > >
              > > Suggestions:
              > > - massively increasing all the anti-DoS limits in 0.4, so far they've caused
              > > a lot more damage than they solved.
              They were redone in .24 to the point that they should not cause any such
              issues in the future.
              > 
              > You can't "massively increase" the number of available connection
              > slots without risking running nodes on lower memory systems (e.g. VMs)
              > out of memory.
              > 
              > Moreover, 125 slots should be more than enough.  We need to figure out
              > why it isn't.
              Agreed.
              
              
              
              
              ^ permalink raw reply	[flat|nested] 15+ messages in thread

      • * Re: [Bitcoin-development] DNS seeds returning gone peers
          2011-08-03 14:18     ` Rick Wesson
          2011-08-03 14:28       ` Douglas Huff
        @ 2011-08-03 14:37       ` Christian Decker
          1 sibling, 0 replies; 15+ messages in thread
        From: Christian Decker @ 2011-08-03 14:37 UTC (permalink / raw)
          To: Bitcoin Dev
        
        [-- Attachment #1: Type: text/plain, Size: 3844 bytes --]
        
        I'm curious whether my Non-Blocking network stack (BitDroid) isn't better
        suited for detecting and tracking available peers. I have implemented
        several benchmarks, including a simple peer counter listener, which would
        have to be adapted to fit the DNS needs (open and check if a real peer is
        listening). Being non-blocking it can open several hundreds of connections
        to check reachability of the peers and at the same time keep a pool of peers
        connected to listen for address broadcasts, with minimal overhead (single
        thread, close to no context switches).
        
        Just an idea :-)
        
        Regards,
        Chris
        
        On Wed, Aug 3, 2011 at 4:18 PM, Rick Wesson
        <rick@support-intelligence•com>wrote:
        
        > Starting from bitcoinj, I have plenty of ways to publish DNS. Why sort them
        > by version? Ordering from highest to lowest?
        >
        > how about publishing addresses under version.example.com if you version
        > has a perfrence?
        >
        > -rick
        >
        >
        > On Wed, Aug 3, 2011 at 7:10 AM, Mike Hearn <mike@plan99•net> wrote:
        >
        >> There's no project currently :-)
        >>
        >> Starting from Matts code is probably the way to go. It's written in PHP.
        >> Alternatively, you could write a Java app for it, as there are drop-in DNS
        >> serving libraries you could link with BitCoinJ+sqlite. It probably wouldn't
        >> be that hard. You'd want to sort nodes by version, how long they've been
        >> observed to exist, the last polling time, etc.
        >>
        >> On Wed, Aug 3, 2011 at 4:00 PM, Rick Wesson <
        >> rick@support-intelligence•com> wrote:
        >>
        >>> Mike,
        >>>
        >>> I think I can contribute to your DNS seeding project. Could you help
        >>> define long-lived peers?
        >>>
        >>> -rick
        >>>
        >>>
        >>> On Wed, Aug 3, 2011 at 3:04 AM, Mike Hearn <mike@plan99•net> wrote:
        >>>
        >>>> This is expected to happen from time to time of course as it's
        >>>> inherently racy, but there are a *lot* of bad nodes appearing in the
        >>>> DNS seeds.
        >>>>
        >>>> $ nmap -oG /tmp/x -p 8333 `dig +short bitseed.bitcoin.org.uk
        >>>> dnsseed.bluematt.me bitseed.xf2.org`
        >>>> ...
        >>>> Nmap done: 48 IP addresses (25 hosts up) scanned in 9.80 seconds
        >>>>
        >>>> $ grep -c 'closed' /tmp/x
        >>>> 6
        >>>>
        >>>> So of 48 IPs returned only 19 are actually usable. This is slowing down
        >>>> peer bringup for the Android apps, which don't currently save the addresses
        >>>> of last-used peers (yes, I know we should fix this).
        >>>>
        >>>> I was talking to a friend a few days ago about Bitcoin, he seemed
        >>>> interested. I'm hoping he might take on DNS seeding as a project. A custom
        >>>> DNS server that watches the network to find long-lived peers that run the
        >>>> latest version would be helpful for resolving this kind of thing.
        >>>>
        >>>>
        >>>> ------------------------------------------------------------------------------
        >>>> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
        >>>> The must-attend event for mobile developers. Connect with experts.
        >>>> Get tools for creating Super Apps. See the latest technologies.
        >>>> Sessions, hands-on labs, demos & much more. Register early & save!
        >>>> http://p.sf.net/sfu/rim-blackberry-1
        >>>> _______________________________________________
        >>>> Bitcoin-development mailing list
        >>>> Bitcoin-development@lists•sourceforge.net
        >>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
        >>>>
        >>>>
        >>>
        >>
        >
        >
        > ------------------------------------------------------------------------------
        > BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
        > The must-attend event for mobile developers. Connect with experts.
        > Get tools for creating Super Apps. See the latest technologies.
        > Sessions, hands-on labs, demos & much more. Register early & save!
        > http://p.sf.net/sfu/rim-blackberry-1
        > _______________________________________________
        > Bitcoin-development mailing list
        > Bitcoin-development@lists•sourceforge.net
        > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
        >
        >
        
        [-- Attachment #2: Type: text/html, Size: 6331 bytes --]
        
        ^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2011-08-03 15:32 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-08-03 10:04 [Bitcoin-development] DNS seeds returning gone peers Mike Hearn
2011-08-03 11:38 ` Matt Corallo
2011-08-03 12:00   ` Mike Hearn
2011-08-03 12:15     ` Johannes Henninger
2011-08-03 12:17       ` Mike Hearn
2011-08-03 12:27   ` Caleb James DeLisle
2011-08-03 12:40     ` Mike Hearn
2011-08-03 14:00 ` Rick Wesson
2011-08-03 14:10   ` Mike Hearn
2011-08-03 14:18     ` Rick Wesson
2011-08-03 14:28       ` Douglas Huff
2011-08-03 14:39         ` Mike Hearn
2011-08-03 14:48           ` Gregory Maxwell
2011-08-03 15:32             ` Matt Corallo
2011-08-03 14:37       ` Christian Decker

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox