Code Generation using GPT & CodeSearchNet

Overview

This project focuses on developing a code generation model using a GPT-like architecture trained on the CodeSearchNet dataset. The model is built with Python and PyTorch, leveraging the transformers library for NLP tasks. The goal is to generate code snippets based on natural language descriptions.

Features

Uses the CodeSearchNet dataset for training

Implements a GPT-style language model

**Supports fine-tuning for specific programming languages

Implements tokenization, training, and evaluation pipelines

**Future support for deployment via API (FastAPI/Flask)

**: Tentative

Setup Instructions

Prerequisites

Ensure you have Python and pip installed. Then, install the required dependencies:

pip install torch torchvision torchaudio transformers datasets tokenizers sentencepiece tqdm

Clone the Repository

git clone cd

Running the Project

Load and preprocess the dataset

from datasets import load_dataset dataset = load_dataset("code_search_net", "python")

Train the model

python train.py

Evaluate the model

python evaluate.py

Roadmap

Contributing

Currently, this project is closed for contribution.

License

This project is open-source and available under the MIT License.

Acknowledgments

Hugging Face's transformers library

CodeSearchNet dataset

PyTorch community