The project involves building an application for Mac OS X (as well as being compatible for Windows, if possible) to scrape information of a password protected member directory on a web page. We have legal access to the site and the forums in question, which is located on LinkedIn. We basically need to recreate the personal information of fellow members on different forums, but in another and more user-friendly format.
The initial assignment is to convert all the information into Excel. Before starting the work, you will be furnished with an example of how the data should be organized in the excel spread sheets.
For more information on what type of information is available through a LinkedIn profile, plz refer to LinkedIn directly.
The application provided, must have an easy user interphase, so it easily can be managed without any need for programming experience and without the need of using CMD and MySQL commands.
The bot/application used for scraping information of the web, should also be easily modified for use on other sites as well. The application should, therefore, have programming options, where LinkedIn is already provided, but the opportunity to modify the settings with respect to what "categories" to convert both from and to.
This should also be built in a user-friendly interphase, without the need to do any reprogramming and use of MySQL commands.
If successfull implementation, we consider expanding the project, by establishing a database where different data will be uploaded and where we will try to build a platform where everyone can analyze the material and view it, both using mapping technology and other relevant tools and applications. But this phase is NOT a part of the current project. However, the programming language used, and the format the data is converted into, should take into account that it should meet the requirements of being fed into a new database, preferably Bento, since Access currently is unavailable for Mac.