Crawl website and copy all records (minus one condition)
$30-250 USD
Σε Εξέλιξη
Αναρτήθηκε πάνω από 10 χρόνια πριν
$30-250 USD
Πληρωμή κατά την παράδοση
I need a website's records copied by crawling. The website currently has about 250,000 pages. I need you to crawl all the pages and copy them into an organized database. There is one condition where you should not copy pages if the title of the page contains a comma (this will be explained in greater detail via PM).
You MUST have a lot of experience crawling websites.
You must create a fast crawler so that the website is copied in a reasonable time.
The website must be crawled correctly so that proper database is created.
Please PM your experience with crawler. I will PM you the website that needs to be copied and give you all details.