Ruby (on Rails) expert for Web Scraper / Web Scraping Job needed

Intro.

I am looking for a multi-talented Ruby (on Rails) expert in web scraping. You are going to build an entire webapplication with Ruby (on Rails) - using the newest Rails 6.0.0 version and the latest compatible Ruby version. You will find all the links to Frameworks, Templates, Gems etc. below.

Target website.

The first website will be AirBnB's room / housing listings. Please take a very careful look at a few of those listings. AirBnB's is randomly generating id and name HTML-attributes to protect their site from scraping. The structure seems to stay the same though. You will always be presented with 300 search results - even in cities like New York City. To see all listings you have to adjust the price (increase, decrease). In addition to that, use the map to get all the results (zoom in and out, move the map).

Backend.

• You will use the Kimura framework for scraping jobs (link below)

• You will create a "command script" that triggers an AWS EC2 server to be setup, configured and destroyed for every single scraping job (see tutorial / links below). This server will function as a proxy. You are going to use Terraform for that.

• Authorization through CanCanCan (Guest, User, Manager, Admin)

• Authentication through Devise (Basic User table with login is enough)

• Historization of all listings through Papertrail

Scraping job.

The scraping job needs to run when triggered (e.g. if I want to have a new city to be scraped). Also, the job needs to re-scrape an existing listing - the old version can be saved through Papertrail, or you just create a new listing and reference the old with the new one.

Frontend.

I will provide you with a Bootstrap admin template that needs to be used for the frontend. This Bootstrap admin template is built on top of the Bootstrap framework, SASS, jQuery, HTML5, and CSS. So you need to know how to work with those!

Links.

[login to view URL] (Ruby Framework)

[login to view URL] (Create AWS servers tutorial)

[login to view URL] (EC2 proxies)

[login to view URL]

[login to view URL]

[login to view URL]

Υπηρεσίες Ιστού Amazon CSS HTML5 PostgreSQL Ruby on Rails

Ταυτότητα Εργασίας: #20015968

Σχετικά με την εργασία