Thanks to @guibitten03 for the idea and developing the google scholar crawler.

Papers

Crawler to find papers of a given query using Google Scholar.

Install

pip install -r requirements.txt

Execution

There is two ways of executing the script, the first is using environment variables:

export GOOGLE_SCHOLAR_QUERY='("deep learning" OR "neural network") AND \
                             ("image classification" OR "image recognition")'

python3 -m crawler \
             --max 1000 \
             --min-year 2021

or using the argument option:

python3 -m crawler \
             --query '("deep learning" OR "neural network") AND ("image classification" OR "image recognition")'
             --max 1000 \
             --min-year 2021

Maybe you want to pipe the objects to another command:

python3 -m crawler \
             --query '("deep learning" OR "neural network") AND ("image classification" OR "image recognition")' \
             --max 10 \
             --min-year 2021 \
             --stop-pretty-print | grep --ignore-case "ieee"