This project implements an end‑to‑end financial data pipeline for credit card transactions, focusing on fraud detection and business insights. It covers raw data ingestion, ETL, data modeling, exploratory analysis, and visualization using Python and PySpark.
The goal of this project is to build a complete analytics pipeline for a credit card transactions dataset (credit_card_transactions.csv).The pipeline cleans and transforms raw data into an analytical model that supports fraud detection and operational monitoring.
Key objectives:
- Design and implement an ETL workflow for credit card transaction data.
- Create a relational schema and load processed data into dimension and fact tables.
- Perform exploratory data analysis (EDA) to identify fraud patterns and high‑risk segments.
- Visualize trends to support decision‑making for risk and operations teams.
- End‑to‑end ETL pipeline: Extraction, transformation, and loading of credit card transactions into a structured schema.
- Data cleaning and preprocessing: Handling missing values, standardizing formats, deriving surrogate keys, and ensuring referential integrity.
- SQL‑style schema design: Separate tables for customers, cards, merchants, and transactions with appropriate keys and relationships.
- Fraud analytics: Identification of common fraud categories, high‑risk time windows, and geographic hotspots.
- Visualizations: Charts and plots built with Matplotlib and Seaborn to present distributions, trends, and fraud patterns.
-
Languages & Libraries
- Python (Pandas) for data manipulation and EDA.
- PySpark for scalable data processing.
- Matplotlib and Seaborn for visualizations.
-
Data & Analytics
- ETL design and implementation.
- SQL‑oriented schema design and joins.
- Data cleaning and feature preparation for fraud analysis.
notebooks/– Jupyter notebooks for ETL, data cleaning, transformation, and analysis.images/– Exported charts and visualizations used in the analysis.
You can open the notebooks to see step‑by‑step data processing and analytics workflows.
The exploratory analysis and visualizations highlight several important insights about fraudulent behavior and transaction patterns:
- The most common merchant categories or sectors involved in fraudulent activities are identified and ranked.
- Transactions occurring during late night to early morning hours show elevated fraud risk and require increased monitoring.
- Unauthorized transactions tend to cluster around lower‑value payments (often under 500), showing that fraud is not limited to high‑ticket purchases.
- The analysis surfaces top states or regions with the highest volume of suspicious or fraudulent activity.
These outputs are presented through summary tables and visual charts in the notebooks and images directory.
This project demonstrates how to design and implement a complete financial analytics pipeline for credit card fraud detection, from raw CSV to structured tables and actionable insights. By combining ETL best practices, robust schema design, and visual analytics, it provides a practical framework that can be extended to real‑world fraud monitoring and risk management use cases.