This repository contains a fully automated data pipeline built with Apache Airflow to extract, clean, analyze, and report real estate listings from Seloger. It pushes data to MongoDB, Elasticsearch, and Google Sheets, with real-time Slack alerts for monitoring.
Bidirectional data sync engine between Oracle database and external REST API. Updates on either side automatically reflect on the other, preserving data integrity. Built with Apache Spark, Airflow, Python/Flask, and REST API. Features entity relationship handling and hash-based change detection.
A production-style retrieval-augmented generation (RAG) pipeline for arXiv research papers — fetch new papers daily, parse them with a structure-aware PDF model, store metadata in Postgres, chunk + embed each paper, and serve hybrid (BM25 + vector) semantic search via OpenSearch.
Add a description, image, and links to the airlfow topic page so that developers can more easily learn about it.
To associate your repository with the airlfow topic, visit your repo's landing page and select "manage topics."