Big Data Project : Big Neural Quiz

This project aims to create an online game to test your general knowledge.
Using the microphone to answer the questions, you will be able to play alone or with friends.

Table of contents

• How to play
  • Deployed version
  • Local version
• Repository
• Technical Stack and Architecture
• Data Collection
• Data Preprocessing
• Speech Recognition Model
• Final results
• Quiz Game Application
• Possible improvements
• Authors and acknowledgment

How to play

Deployed version

You can play the game on this link :

https://bigneuralquiz.vercel.app/
(for as long as the server is running, which might not be for long !)

To play the game, you need to create an account and verify it by clicking on the link sent to your email address.
Then you can just start playing !

Local version

If you want to test this in local you need to :

Front-end :

cd ./appbigdataproject 
npm install 
npm run dev

Back-end :

Option 1 (docker):

docker build -t back . 
docker run -p 8080:8080 back

Option 2 (local):

cd Modele
pip install -r requirements.txt
cd ..\server
py .\back.py

You will need to create a .env file in the appbigdataproject folder with the following content:

VITE_SUPABASE_URL = your supabase url
VITE_SUPABASE_KEY = your supabase key
VITE_BACKEND_BASE_URL = http://localhost:8080

Then you can just start playing by going to http://localhost:8080/ in your browser.
Enjoy !

Repository

There are 3 main folders:

1. Models: It contains the python functions for training, the models, the visualisations...

2. appbigdataproject: Frontend application

3. server: Backend REST API

Technical Stack and Architecture

Frontend : ReactJS and Typescript
Backend : FastAPI
Models : Tensorflow, Keras
Database : Supabase
Deployment : Heroku, Vercel
Questions Database : Open Trivia Database

Data Collection

The data used for training the models is from the mozilla common voice dataset, the single word segment dataset.

The dataset is available on this link:

https://commonvoice.mozilla.org/en/datasets

For now we only used the english dataset, but we could use the french dataset as well.

Data Preprocessing

The data is preprocessed in the following way:

• The audio files are converted to wav format, then resampled to 16kHz and mono.
  
• The audio files are then tranformed into spectrograms.
  
• Then the spectrograms are transformed into mel spectrograms.
  
• Finally the mel spectrograms are transformed into mfccs.
  

Speech Recognition Model

The model used for the speech recognition is the following:

Final results

Yes-No: ~90% Accuracy
1 - 2 - 3 - 4: ~75% Accuracy

Quiz Game Application

Possible improvements

• Lightening the docker image (currently ~3.5GB)
• Increasing the size of the dataset
• Fine tuning the model
• Dynamic learning: Using the audio recordings of the users to improve the existing models
• Eventually find and training a more fitting model (instead of the LSTM model)
• Adding a new category for the outputs: "Unrecognized"

Authors and acknowledgment

Modèles:
Simon HUANG
Abdel-Rahim MEZIDI

Backend & Cloud:
Keven BIHAN
Martial GESSEAUME
Kévin TOURNIE

Frontend:
Kévin TOURNIE