The idea here is to build a simple monitoring service for users to track websites for the appearance of certain keywords.
I have a small bare-bones prototype running as a bash script (curl gets the source code, this is stored in a variable. Grep then checks the variable contents for a specific keyword. If the keyword is found an email is sent). A cronjob then runs this all every minute. This needs to be scaled up so that it can be used by multiple users on multiple sites for differing keywords (each user should be able to track a set of websites for keywords/changes).
The final system should allow users to log in/sign up. Sign up would require a creditcard and optional phone number (to send the notifications to - users have the option between email and sms notifications). Creditcard payments are to be processed through Stripe. SMS are sent through Twilio.
Then there'd be the generic settings panel that allows users to set what sites they want to track, what keyword they want to look out for, and the interval at which they want to track (x seconds if possible/ x minutes / x hours / daily / etc. )
Before starting this project I'd like to discuss the minimum possible interval for such a service and the server requirements (eg. what if 1000 people want to track 10 websites at 10 second intervals?).