Automated Data Profiling

A Streamlit web application for automated data profiling and analysis built on top of ydata-profiling. Upload your CSV files and get comprehensive data insights including missing data analysis, statistical summaries, and downloadable reports in HTML and JSON formats.

Features

• 📊 Interactive Data Preview: View your uploaded CSV data
• 🔍 Missing Data Analysis: Identify and quantify missing values
• 📈 Comprehensive Profiling: Generate detailed statistical reports using ydata-profiling
• 📄 Multiple Export Formats: Download reports as HTML (interactive) or JSON (machine-readable)
• 📋 Quick Statistics: View key dataset metrics at a glance

Installation & Setup

Prerequisites

• Python 3.7 or higher
• pip (Python package installer)

Step 1: Clone the Repository

git clone https://github.com/polangco/automated_data_profiling.git
cd automated_data_profiling

Step 2: Create a Virtual Environment

# Create virtual environment
python -m venv venv

# Activate virtual environment
# On macOS/Linux:
source venv/bin/activate
# On Windows:
venv\Scripts\activate

Step 3: Install Dependencies

pip install -r requirements.txt

Step 4: Run the Application

streamlit run app.py

The application will open automatically in your default web browser at http://localhost:8501

Usage

1. Upload CSV File: Use the file uploader to select your CSV file
2. Preview Data: Review the first 20 rows of your dataset
3. Check Missing Data: View summary of missing values by column
4. Generate Reports: The app automatically creates profiling reports using ydata-profiling
5. Download Results:
   • HTML Report: Interactive visualization for manual review
   • JSON Report: Machine-readable format for API integration

Deactivating Virtual Environment

When you're done using the application:

deactivate

Troubleshooting

• Import Errors: Make sure you've activated your virtual environment and installed all requirements
• File Upload Issues: Ensure your CSV file is properly formatted and not corrupted
• Memory Issues: For very large datasets, the profiling process may take longer or require more memory