TASK: Create Inventory of Data Sources and Define Standard Data Contracts

[mvadodariya]

Describe the task
Before implementing the ingestion framework, it's essential to identify and document all existing data sources (mobile apps, portals, legacy systems, APIs). This sub-task focuses on analyzing the structure, frequency (real-time or batch), and data formats used by each source. It also includes defining standardized data contracts or schemas to streamline ingestion and ensure consistency across inputs.

Key Outcomes:

• Create a complete list of all current and upcoming data producers.
• Classify them as real-time or batch sources.
• Document data types, formats, and payload examples.
• Define common interface/contract specifications (e.g., JSON schema, Avro).
• Identify sources needing transformation before ingestion.

[23shivay]

@mvadodariya
@kunjan8794
@harshit-argusoft
@avsethu
hy guys i have tried doing sub tasks mentioned under #93 which are #98 and #99 plzzz guide me am i doing correct and on a right path really hoping for your inputs

[phanisri2230]

@mvadodariya
I will start by listing all existing and upcoming data sources, classify them as real-time or batch, document their formats and sample payloads, and define standard data contracts (JSON/Avro). Please let me know if there are any preferred templates or formats to follow."