Find Jobs
Hire Freelancers

php curl function based on sending headers

$250-750 USD

Κλειστή
Αναρτήθηκε πάνω από 10 χρόνια πριν

$250-750 USD

Πληρωμή κατά την παράδοση
The task is to write a function that gets all the contents from a website based on sending all the headers that a browser would normally send. Using FF live http headers you can capture all relevant headers and paste them into a below format. Just after visiting a website a session should be still active on the remote server and yet the captured phpsessionid should still be working. However the below script is not working and I am looking for a function that is simple robuust, and always works when all the headers are sent to the remote server that are also sent by the browser. Probably something is wrong or something is missing in the below script. However I checked and it sends exactly the same headers to the server as the browser, but the browser does return results and the below function doesn't. ===> CODE: echo get_contents('[login to view URL]', ''); function get_contents($url,$post){ if($post == ''){ $headers = array(); $headers[] = 'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0'; $headers[] = 'Accept: */*'; $headers[] = 'Accept-Language: nl,en-us;q=0.7,en;q=0.3'; $headers[] = 'Accept-Encoding: gzip, deflate'; $headers[] = 'X-Requested-With: XMLHttpRequest'; $headers[] = 'Referer: [login to view URL]'; $headers[] = 'Cookie: PHPSESSID=zxddfkojsdaofiguja09fgjad'; $headers[] = 'Connection: keep-alive'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10); curl_setopt($ch, CURLOPT_TIMEOUT, 10); curl_setopt($ch, CURLOPT_HEADER, 1); curl_setopt($ch, CURLINFO_HEADER_OUT, true); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); $body = curl_exec($ch); curl_close($ch); } return $body; }
Ταυτότητα εργασίας: 5328716

Σχετικά με την εργασία

26 προτάσεις
Απομακρυσμένη Εργασία
Ενεργός/ή 10 χρόνια πριν

Ψάχνεις τρόπο για να κερδίσεις μερικά χρήματα;

Πλεονεκτήματα πλειοδοσίας στο Freelancer

Καθόρισε τον προϋπολογισμό σου και το χρονοδιάγραμμα
Πληρώσου για τη δουλειά σου
Περίγραψε την πρόταση σου
Η εγγραφή και η πλειοδοσία σε εργασίες είναι δωρεάν
26 freelancers δίνουν μια μέση προσφορά $343 USD για αυτή τη δουλειά
Avatar Χρήστη
Hi! I'm interested in it, I've tonnes of experience with scraping n bots, let me know what pages u want to scrape and what data is there, what I need to do? I'll adjust the cost accordingly
$515 USD σε 5 ημέρες
5,0 (58 αξιολογήσεις)
7,1
7,1
Avatar Χρήστη
Dear Customer! I am an expert PHP developer with over 6 years of experience and very interested to work on this project. Available to start immediately and finish as soon as possible. My bid is for fast professional service exciting my customers. Please contact in PMB to discuss details. Best Regards, Zeke
$515 USD σε 10 ημέρες
4,7 (168 αξιολογήσεις)
7,0
7,0
Avatar Χρήστη
Ready to start =================================================================================================
$250 USD σε 5 ημέρες
5,0 (143 αξιολογήσεις)
6,7
6,7
Avatar Χρήστη
Hi, Well, it can't really be general as it is browser-linked. For what website do you want to use it? I'm sure your error is with cookies. Regards Sorin
$526 USD σε 10 ημέρες
4,8 (207 αξιολογήσεις)
6,5
6,5
Avatar Χρήστη
Dear sir i am keen to complete your goal. i am enough confident to play with curl and will be able to achieve the quality final result very soon.. sincerely
$250 USD σε 5 ημέρες
5,0 (26 αξιολογήσεις)
6,3
6,3
Avatar Χρήστη
Hi Scraping expert here. Show me the website and let me help you please. ............................................... Best Regards Bill
$263 USD σε 3 ημέρες
5,0 (49 αξιολογήσεις)
6,2
6,2
Avatar Χρήστη
Hello, The session is IP bound, so if your browser uses one session Id and you try to curl from your server, of course it won't work. You actually need two transactions, I assume the content is behind a login field? You need to login via PHP , save the cookies in a jar and send the correct session ID back to the site.
$333 USD σε 5 ημέρες
4,9 (57 αξιολογήσεις)
6,0
6,0
Avatar Χρήστη
Hi, I have a lot of experience scraping tricky sites with php/curl. Sometimes the error is that you're forgetting an hidden field or sending the submit button, others need the P3P header, the sites with ASP.net creates a _viewstate field which you have to parse and send together with your request. Anyway, there are many tricks and each site must be analysed separately. I can deliver you the function to scrape all the content from the site that you want. Please let me know any comment or doubt about my proposal, Regards, David.
$250 USD σε 5 ημέρες
4,9 (138 αξιολογήσεις)
5,4
5,4
Avatar Χρήστη
I can start as soon you provide the target site address and the target php version. I expect to be done within the next 24 hours but the deadline is still 2 days. Thanks Aydin
$250 USD σε 2 ημέρες
5,0 (23 αξιολογήσεις)
5,3
5,3
Avatar Χρήστη
A proposal has not yet been provided
$555 USD σε 10 ημέρες
5,0 (15 αξιολογήσεις)
5,3
5,3
Avatar Χρήστη
Hello, I am very good at getting data from websites using curl and headers. Please send me the link to the site and I will create a demo for you. Thanks Brian
$277 USD σε 2 ημέρες
5,0 (43 αξιολογήσεις)
5,0
5,0
Avatar Χρήστη
Hello, Is it possible to know what address is not working ? I have used cURL a lot in previous projects so I think I can help you. Best regards, Laurent.
$250 USD σε 1 ημέρα
4,7 (27 αξιολογήσεις)
5,0
5,0
Avatar Χρήστη
Hi. I've coded many web scrappers that simulate the actual behavior of web browsers in order to deliver better results (you can check the feedback for a few of them in my profile). I can do what you ask for. However, have in mind that, if you're trying to download a page which is not meant to be accessed from the browser location bar (ie not submitted by the user, like a home page), the headers needed are different from each other, on a per case basis. Especially, when a website performs an XHR request, it may send some special headers in order to identify itself as legitimate. Apart from this, letting cURL handle the cookies, if possible, is always a better idea than sending them oneself alongside other headers.
$277 USD σε 5 ημέρες
5,0 (11 αξιολογήσεις)
4,9
4,9
Avatar Χρήστη
Dear bwesteinde, I am very pleased to provide the service you need, please send the URL to check the code. Thanks. Look forward to working with you soon. Best Regards, Rose Office
$250 USD σε 7 ημέρες
5,0 (9 αξιολογήσεις)
3,7
3,7
Avatar Χρήστη
Hello! I have lots of experience developing web scrapers. I am also quite good at debugging the HTTP stack. Please send me more details about what you need done. Thanks
$555 USD σε 5 ημέρες
4,5 (6 αξιολογήσεις)
3,6
3,6
Avatar Χρήστη
Well sir i can do this and write the function and i am good experience with php , curl and web scraping and i'll edit this bid again to post the function.
$333 USD σε 2 ημέρες
5,0 (1 αξιολόγηση)
3,0
3,0
Avatar Χρήστη
Hi Bastiaan, I think you've missed curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); I've forgot this line to a scraper and it didn't return anything. Not sure if this will fix your problem, but it might work. Best Regards, Constantin Plaiasu
$250 USD σε 10 ημέρες
5,0 (6 αξιολογήσεις)
2,5
2,5
Avatar Χρήστη
- Have long experience (7+ year experience) in web development using PHP. - Have coded many similar projects using curl library. - No need to pay even the milestone till you see your code working. - I just need more details (the url of the site and any further needed details) to make a demo for you to show my ability to do the job exactly as you want ;) - I will be very glad to hear back from you :)
$250 USD σε 1 ημέρα
5,0 (1 αξιολόγηση)
0,8
0,8
Avatar Χρήστη
Hello I am interested in your job. I am a software expert with rich experiences in web scraping using php curl. I have my own php curl function library which can be used for web scraping and parsing. I have already done several web scraping projects successfully with my own library. And I have ever made aol account auto creator a few months ago. As you know, it isn't easy to crawl the aol.com. I hope you will discuss about your details with me. I will look forward to your message. Thanks
$515 USD σε 7 ημέρες
0,0 (0 αξιολογήσεις)
0,0
0,0
Avatar Χρήστη
Hi, I'm new here, but I'm expert in scraping sites with PHP and cURL, I do it for at least 10 years.. I scraped over 500 different sites last year. I can deliver a simple and robust function that will do scrape any site you need. Also, if you'll need to extract/parse/standardize data in the future, or need any help or tips, I can help you too. I just created my account a few hours ago, and this is my first bid.. I wanna start building my reputation.. so pick me and you'll won't regret.. thanks
$250 USD σε 2 ημέρες
0,0 (0 αξιολογήσεις)
0,0
0,0

Σχετικά με τον πελάτη

Σημαία της NETHERLANDS
Goes, Netherlands
4,8
64
Επαληθευμένη μέθοδος πληρωμής
Μέλος από Οκτ 19, 2011

Επαλήθευση Πελάτη

Ευχαριστούμε! Σου έχουμε στείλει ένα email με ένα σύνδεσμο για να διεκδικήσεις τη δωρεάν πίστωση σου.
Κάτι πήγε στραβά κατά την προσπάθεια αποστολής του email σου. Παρακαλούμε δοκίμασε ξανά.
Εγγεγραμμένοι Χρήστες Συνολικές Αναρτημένες Δουλειές
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Φόρτωση προεπισκόπησης
Δόθηκε πρόσβαση για Geolocation.
Η σύνδεση σου έχει λήξει και τώρα έχεις αποσυνδεθεί. Παρακαλούμε συνδέσου ξανά.