This is the scraper for the scheduler website

#In order to run:

Authentication

You must either get the proper credientials from be or josh Alan to connect to the live databse

else inside of pipeline and the connect function, you must declare your own db and credentials

Libraries

Uses a combination of scrapy and Beautiful soup

A beta using selenium has been provided inside of srjc_selenium

This shows how to properly yield and use the twisted-scrapy framework, more to come involving middleware specficially for other colleges support and hopefuly a net to just scrape all at once for multiple colleges

TODOS

1. Syntax simplification
2. Support for classes that do not meet at a set time
3. Support for classes that do not have sections names

In progress: Current version is 0.1.0