Advanced Proxy Configuration (Lighttp, Squid, HAProxy or similiar)

Ακυρώθηκε Αναρτήθηκε Aug 16, 2011 Πληρώθηκε κατά την παράδοση
Ακυρώθηκε Πληρώθηκε κατά την παράδοση

I am looking for someone with experience in setting up and configuring proxy servers.

I need a proxy setup that allows me to scrape Google without worrying about blocking. Basically Google will block requests if one IP is sending to many requests and it seems automated to them. We can prevent this by using a large number of proxies.

To simplify interaction we would like to setup one proxy server that handles distributing requests to anonymous, free proxies and then returns only non-blocked results.

The projects requires:

1) Setup of a proxy sever on Amazon EC2 and proxy configuration

2) Write Script that logs in to Proxybonanza, generate new list of proxies, download new proxy list and update proxy server configuration with the new proxies. Cron job to execute this monthly. Some simple interface to manually change list of proxies (update/modify).

3) Configure proxy to inspect result pages and pause all proxies for 30 minutes that Google has blocked. Retry initial blocked request to return correct page from Google.

4) Change User Agent of initial request automatically to a random user agent form this list [url removed, login to view]

It would be good if you have done something similar in the past and if you have a lot of experience configuring proxy servers.

Cisco Ηλεκτρονικά PHP SEO Αρχιτεκτονική Λογισμικού Δοκιμή Λογισμικού Φιλοξενία Ιστοσελίδων Διαχείριση Ιστοσελίδων Έλεγχος Ιστοσελίδων

Ταυτότητα Εργασίας: #3513369

Σχετικά με την εργασία

1 πρόταση Απομακρυσμένη εργασία Ενεργό Aug 21, 2011

1 freelancer κάνει προσφορά κατά μέσο όρο $84 για αυτή τη δουλειά

UFS

See private message.

$84.15 USD σε 14 μέρες
(14 Αξιολογήσεις)
3.8