[techtalk] I need an algorithm!

[Michelle Murrain]

Hi folks,

I have this search engine idea, and I need to search for domain names - 
ultimately, I need the full listing of all domain names, but I don't want 
it all at once, just in small chunks at a time. And I only need the domain 
names, nothing else. To further make this difficult, the whois servers 
abort searches after 50 records are returned on global searches (for good 
reason!).

So, I need an algorithm that will let me eventually get all of the domain 
names. I know this must have been done before, I just have no idea how to 
do it. I looked at the source code for the whois client

So, for example, if I do a whois lookup on aa*, I get the first 50 
records.  (a* is too general, I guess) Looks like:

AA-ACCOUNTANTS.COM
..
AA-1.NET

Unfortunately, the last entry doesn't necessarily give me a lot of clues as 
to what's next.

I know there are millions and millions of domains, and I'm probably stupid 
to try this, but I know I'll be discarding data from at least 90% of them, 
so I don't have to keep information about them, just plug them in, check 
some things, and move to the next one.

Ideas?

TIA,
Michelle

Michelle Murrain
michelle at murrain.net
AIM:pearlbear0