herbaLook is part of a cross-domain plant identification research involving dried plant specimens (herbarium images) to identify plant photographs in the field with triplet networks.
The proposed triplet network, namely, the Herbarium-Field Triplet Loss (HFTL) Network, is constructed with two Convolutional Neural Networks that map the herbarium and field domains together.
The aim of herbaLook is to demonstrate the trained HFTL model as well as its practicality for taxonomists and fellow naturalists.
In addition, it is also part of the fulfilment of the requirements for my Master Research Degree. More info about the work can be found here.
https://herbalook.neuon-kuching.com/
- Upload a plant image (field domain) and get up to Top-50 prediction results from 997 plant species
- View the similarity scores between the uploaded image and the herbarium dictionary (dataset/herbaria collection) note that classifications by HFTL models are based on similarity scores instead of probabilities
- Analyze the HFTL model's prediction through the Class Activation Map visualizations
- View a sample of the dataset images
- View the list of trained species
- View species sharing the same Genus and Family (identify visually similar species that may be commonly misclassified)
This research is made possible with the support from Swinburne University of Technology Sarawak Campus and NEUON AI Sdn. Bhd., Malaysia.
This application uses the dataset (herbarium and field images) from PlantCLEF 2021 and can be acquired at AIcrowd.
Cross-domain plant classification remains a challenging task and still requires further research as its prediction accuracies are significantly lower than conventional automated plant classification. Nevertheless, the HFTL approach offers a step in alleviating the challenging task of automated plant identification with few field image samples, specifically rare species, which require high-level expertise. More research is welcomed and anticipated for the plant biodiversity community.
Windows
- Install Python 3.7.16
- Set up a virtual environment
- Activate the virtual environment
- Install the dependencies
pip install -r requirements.txt
- Download this repository
- Download the prediction model + dictionary (model.zip), and database images (images.zip) here
- Place both zip files in the herbaLook\media directory
- Unzip each files
- From the virtual environment, go to the downloaded repository
cd path\to\herbaLook\in\your\PC
- Create a new SECRET_KEY by running the following command
python manage.py shell -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())"
- Copy and save the printed key to a .env file
> Right click on the herbaLook root directory (path\to\herbaLook)
> Create a new Text Document
> Type the following into the Text Document
SECRET_KEY=the printed secret key
> Save the Text Document File Name as '.env' and File Type as 'All Files (*)'
- Go back to the virtual environment, make sure you're in the root directory (path\to\herbaLook)
- Link the database (media\images) to your app
python manage.py linkdirectories
- Start the herbaLook web app
python manage.py runserver
- View the app by typing the following in your internet browser
http://127.0.0.1:8000/