- Show my products, and show competitor (29 sites) prices of that same product.
(!!! there needs to be a solution to link productname of theirs to mine productnames. I have 'Brand', and 'Modeltype' to make a link easyer!)
- Visualize price progression over time with kibana per product / store. (Schedule a scraper every night!)
- Show if a price is dropping or going higher per product.
Use the opensource python scraper 'Scrapy'.
Adding sales of my shop, adding positions of product in categories of competitors and mine.
The scraper will run on a DigitalOcean VPS. You install the server + scraper etc.
The Kibana Logstash etc. server will run on other server.