I am looking for a guy who can create a polite python crawler to find expired domains.
Following functionality:
(1) Indexing domains
- Crawler crawls through the web and saves domainnames in a database
- There shall be the setting opportunity just to crawl domains with a certain ending
(2) Checking website status
- Another crawler regularly and automatically visits the domainnames saved in database
- By using HTTP-Header-Status or other ways, it shall be checked whether is domain is indexed in the nameservers
(3) Displayment of domains
- In case the domain is not in the DNS entries anymore, the pagerank of the domain is checked
- Furthermore, the validity of the pagerank is checked
These two parameters are then set in a list.
PLEASE JUST BID IF YOU CAN COMPLETE FULFILL THE REQUIREMENTS!
PLEASE READ!!! I am an experienced programmer, and I can create the web crawler you require. I have written spiders and crawlers in the past, many of which were more complicated than the one you have described in your project. If you are not satisfied with my results, I will not ask you to pay. Thank you for your consideration, and I hope to work with you soon.
I think i can finish all of them. I know how to go through links with scripts and i know how to check pagerank. But i also have some questions. Shall we discuss a bit through message?