SKELAR x mono AI Competition 2026: Anti-Fraud System

Project Overview

This repository contains our machine learning solution for the SKELAR x mono AI Competition. The primary objective is to build a highly accurate payment anti-fraud system for a communication platform.

Payment fraud, such as account takeovers and the use of stolen cards, causes significant financial and reputational damage to both clients and businesses. Our goal is to analyze raw dataset anomalies, build an ideal filtering logic (ML model), and maximize the user-level F1-score without negatively impacting the experience of legitimate customers.

---

Dataset Description

The competition provides 4 primary datasets divided into training and testing sets:

• train_transactions.csv & test_transactions.csv: Contains the transaction history of users. Key fields include transaction timestamps, amounts, success/fail statuses, error groups (e.g., antifraud, 3ds error, insufficient funds), card types, and payment/card countries.
• train_users.csv & test_users.csv: Contains user parameters. Key fields include registration timestamps, email addresses, registration countries, traffic sources, and the target variable is_fraud (binary label, available in the training set only).

---

Solution Architecture

Our solution relies on a heavily optimized LightGBM Classifier combined with rigorous, leak-free feature engineering. Because fraud detection relies heavily on behavioral patterns over time, preventing target leakage between the training and validation folds was our top priority.

1. Feature Engineering Highlights

We extracted 50+ features, focusing on user behavior, velocity, and network associations:

• Time & Velocity Features: Fast-attack flags (transactions occurring within 1 hour of registration) and optimized 1-hour/24-hour rolling transaction counts and sums.
• Geographic Mismatches: Binary flags capturing discrepancies between a user's registration country, card issue country, and actual payment country.
• Leak-Free Aggregations: User-level and card-level statistics (e.g., failure rates, total spend, standard deviation of transaction amounts). These are aggregated strictly within the training fold and mapped to the validation/test folds to prevent future data from leaking into past predictions.
• Leave-One-Out (LOO) Graph Features: To assess network risk without target leakage, we calculated card-level fraud rates. When evaluating a specific user, we subtract that user's own is_fraud label from the card's total fraud count.

2. Validation Strategy

• Stratified 5-Fold Cross Validation: Ensuring the 16.6% fraud rate is balanced across folds.
• Static User-Level Ground Truth: We mapped fraud labels at the user level before the CV loop. This fixes a critical bug where a fraudulent user might be evaluated as "non-fraudulent" in the validation fold if their specific fraudulent transactions were assigned to the training fold.
• Threshold Tuning: Probabilities are output at the transaction level, but the final evaluation requires a binary flag at the user level. We group predictions by id_user, take the max() probability, and tune the threshold (yielding an optimal threshold of ~0.65) to maximize the F1-score.

---

📊 Top 5 Key Features

As per the competition evaluation criteria, here are the top 5 features driving the model's decisions (ranked by LightGBM Gain):

1. user_distinct_cards: The number of unique cards associated with a single user account. Fraudsters frequently test multiple stolen cards (card testing) on a single account.
2. user_fail_count: The absolute number of failed transactions. Normal users might fail once or twice due to typos; fraudsters generate massive failure volumes.
3. user_tx_count: The total number of transactions attempted by the user. Anomalously high volume is a strong indicator of automated attacks.
4. user_fail_rate_type_card_init: The failure rate specifically on initial card binding/tokenization transactions.
5. user_fail_rate: The overall transaction failure rate for the user.

---

Business Integration Strategy

Note: This section addresses the 20% grading criteria regarding business integration.

The business decision comes down to balancing automated blocking versus manual review. Based on our model's output probabilities, we propose a tiered integration approach:

• High Confidence Fraud (Prob >= 0.85): Hard block. These users exhibit textbook fraud behaviors (e.g., massive velocity, high failure rates, multiple distinct cards). Automatically blocking them saves the cost of manual review.
• Grey Zone / Suspicious (0.65 <= Prob < 0.85): Route to manual review or trigger soft-friction (e.g., require 3DS authentication or email verification). This protects the customer experience for potentially legitimate users making unusual purchases.
• Low Risk (Prob < 0.65): Process seamlessly to maximize conversion rates.

By dynamically adjusting these thresholds, the business can scale the manual review team based on available budget and current fraud attack volumes.