Find Jobs
Hire Freelancers

Docker - Multi-Thread crawling- Microsoft Azure-AWS-Google Cloud-- Expert (100$) -- 14792

$30-250 USD

Κλειστή
Αναρτήθηκε πάνω από 4 χρόνια πριν

$30-250 USD

Πληρωμή κατά την παράδοση
Hello. thank you for having interest in this topic. *Please read description first thoroughly, if you're copy-pasting meaningless words I will instantly dump the proposal. Thank you * If you have experience in scraping/cralwing work, You know there's a capacity which one scraper one thread can be running. And in order to compensate this problem , we often adopt multi-threaded system inside of the script, But even when adopting this, We could be still having the same problem. This is why most of crawlers end up with using Cloud computer/servers , setting multiple virtual computer(server,cloud), setting multi-threaded script inside. Not necessarily to explain long about this, If you have experience, You'd definitely understand what I'm saying. I'm here to find decent future co-worker who has related skills to setup such environment and have experience to setup and build such models. Project price 100$ is related with the right next project we're going to jump on, that I will inform you through DM. (Direct message) or sending official offer. but beforehand, let me ask you questions. 1) Do you have experience multiple server/cloud/VM based scraping work? 2) If you have answers yes to 1), Could you show me a simple demonstration video about how it gets to be run? (No need to show script or inside-tricks, just on surface, simply the scenes that should be enough to prove it's working) 3) Did you also have experience with proxy-rotation adoptation? Please fill in the form with sending proposal. If our communication is not in exact form, We can not proceed any further. and I will kindly reject it all once it's not in the question form. Thank you.
Ταυτότητα εργασίας: 21073803

Σχετικά με την εργασία

1 proposal
Απομακρυσμένη Εργασία
Ενεργός/ή 4 χρόνια πριν

Ψάχνεις τρόπο για να κερδίσεις μερικά χρήματα;

Πλεονεκτήματα πλειοδοσίας στο Freelancer

Καθόρισε τον προϋπολογισμό σου και το χρονοδιάγραμμα
Πληρώσου για τη δουλειά σου
Περίγραψε την πρόταση σου
Η εγγραφή και η πλειοδοσία σε εργασίες είναι δωρεάν
1 freelancer δίνει μέση προσφορά $100 USD για αυτή τη δουλειά
Avatar Χρήστη
Hello, I am a fullstack .Net devloper having 5 years experience and multiple Microsoft Certificates. I've read your project details and I understand what's the problem you are facing. To answer your question first. 1. I do have experience on scraping work with Docker, with different networks. Database is internal access and I also have web front-end hosting open for public and have internal access to database. I have crawler running as a back-end worker and applications. Not sure what you mean by multiple server/cloud/VM based scraping work. There are so many ways of doing the same thing, I may need more clarification on what you mean by this. 2. I can show you a application I wrote having 30 threads crawling a website using different types of proxies. The application is not currently running since there are some limitations I am facing. The target website limit request by IP. Each IP only allows 100 requests per day and I don't have that much proxies to run with. But if you are interested, I am ok to show you. 3. Yep. I do know how to do proxy rotation. For me to make a proposal, I will need to understand what website you want to scrape and what's the server structure you have. The framework I am going to use will be .Net Core. If you are interested to talk more about the project, please let me know. I am putting 200 in 5 days for now since I need further discussion but usually scraping project only takes 1-2 days.
$100 USD σε 4 ημέρες
0,0 (0 αξιολογήσεις)
0,0
0,0

Σχετικά με τον πελάτη

Σημαία της KOREA, REPUBLIC OF
Incheon, Korea, Republic of
5,0
15
Επαληθευμένη μέθοδος πληρωμής
Μέλος από Φεβ 9, 2019

Επαλήθευση Πελάτη

Ευχαριστούμε! Σου έχουμε στείλει ένα email με ένα σύνδεσμο για να διεκδικήσεις τη δωρεάν πίστωση σου.
Κάτι πήγε στραβά κατά την προσπάθεια αποστολής του email σου. Παρακαλούμε δοκίμασε ξανά.
Εγγεγραμμένοι Χρήστες Συνολικές Αναρτημένες Δουλειές
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Φόρτωση προεπισκόπησης
Δόθηκε πρόσβαση για Geolocation.
Η σύνδεση σου έχει λήξει και τώρα έχεις αποσυνδεθεί. Παρακαλούμε συνδέσου ξανά.