A simple & efficient Web-Scraper implemented using Python libraries like 'requests' & 'BeautifulSoup4' that easily extracts the desired product details & dumps the scraped data automatically into a 'JSON' File.
The Web-Scraper is coded in Python so ensure whether the following pre-requisites are met, in order to run it on your system.
Python 3.5.7 - Minimum version required to run the scraper is 'Python 3.5.7' or above.
pip - Install 'pip' to download and install various Python Libraries directly from the Terminal.
pip install requests - Install the 'requests' Python library using this 'pip' command.
pip install beautifulsoup4 - Install the 'beautifulsoup4' Python library using this 'pip' command.
- The Web-Scraper is build using Python libraries viz. 'requests' & 'bs4'.
- The Scraper scrapes the product details viz. title, price, image, url & specifications.
- Automatically converts & stores the scraped data in JSON format with the .json file extension.
- It has a sleep timer implemented to reduce the load by sending limited no. of request at the given time to the remote server.
- This Web-Scraper project is just for educational/research purposes only!
- Scraping can be helpful in case extracting loads of data for Analysis/Research purpose!
- 'I am not responsible at any cost', if anyone uses it intentionally or unintentionally for any kind of other purposes which violates any Rules/Limitations of any corporation or organization!
- Before scraping any website just ensure that you're not voilating any of the Rules/Limitations or Terms & Conditions mentioned in the /robots.txt of that particular website.
This project is licensed under the Apache-2.0 License - see the LICENSE.md file for details.
Copyrights © - 2020 Aditya Ambre. │ All Rights Reserved.