Neural Machine Translation on multiple Model

Table of Contents

1. About
2. Architecture
3. Technologies
4. Dataset
5. How to run
6. Acknowledgments

About

• The actually first project AI that I created from scratch (From Data Processing to Create Model, Train & Evalute, ...)
• This project is created to test multiple Model Seq2Seq on the same dataset
• Statuse: Finished. The project is ready, just the code base right now is very messy.

Architecture

• Now avaiable:
  • Encoder (Bi-GRU) -> Decoder (GRU)
  • Encoder (Bi-GRU) -> Attention -> Decoder (GRU)
  • Transformer

Technologies

• Language: Python 3.9.12
• Framework: PyTorch 1.9.0+cu102
• Required Libraries:
  • numpy 1.23.1
  • nltk 3.7
  • matplotlib 3.5.2
  • pandas 1.5.1
  • underthesea 1.3.5

Dataset

• Source: Kaggle
• Already downloaded in data folder

How to run

Run this for details

python main.py -h

Data preparation

python main.py -d

To train the model

python main.py -t -m model_name

Please replace model_name with the following:

• gru: Encoder (Bi-GRU) -> Decoder (GRU)
• attention: Encoder (Bi-GRU) -> Attention -> Decoder (GRU)
• transformer: Transformer -m model_name is not required, if blank, default to gru

To validate the model

python main.py -e -m model_name

Replace model_name like above.

To plot attention

python main.py -a

This only works with attention model

Acknowledgments

• This tutorial on Seq2Seq with Attention
• This tutorial on creating a Chatbot using Seq2Seq
• This tutorial on training with PyTorch
• This blog on NMT
• Code for Transformer
• Code for Attention mechanism