Feature: API Re-Publishing POC

[stephenfuqua]

Objective

Perform realtime synchronization from Kafka to a destination Ed-Fi API.

Technical Requirements

• Create a .NET application using Quix Streams
• Put the application in folder path showcase/streaming/api-publisher
• Update the README.md file in showcase/streaming/api-publisher with information on how to run the sample application
• All URLs and configuration parameters must be put into an appsettings.json file
• Call the Discovery API and use the result to
  1. lookup OAuth URL, using JSON Path $.urls.oauth
  2. lookup the base path for the Data Management API, using JSON Path $.urls.dataManagementApi
• Authenticate to the OAuth URL using the client_credentials flow
• Read data from Kafka topic edfi.dms.document
• Transform the data read from Kafka into an HTTP POST request to the matching Data Management API endpoint
• Only one modification is allowed on the JSON documents received from the Kafka stream: delete the $.id element

Example

Here is a message posted to Kafka:

{
 "id": 1,
 "documentpartitionkey": 4,
 "documentuuid": "48de3506-e6bb-1fd8-7fa6-eab6bcebfb74",
 "resourcename": "GradeLevelDescriptor",
 "resourceversion": "5.2.0",
 "isdescriptor": true,
 "projectname": "Ed-Fi",
 "edfidoc": {
  "id": "48de3506-e6bb-1fd8-7fa6-eab6bcebfb74",
  "_etag": "1PTRsk2O6uWoeWmjHp/0Oo/RCQmIC49Csv93f7c72nA=",
  "codeValue": "Ninth Grade",
  "namespace": "uri://ed-fi.org/GradeLevelDescriptor",
  "shortDescription": "9th Grade",
  "_lastModifiedDate": "2025-12-06T20:34:48Z"
 },
 "securityelements": {
  "Staff": [],
  "Contact": [],
  "Student": [],
  "Namespace": [
   "uri://ed-fi.org/GradeLevelDescriptor"
  ],
  "EducationOrganization": []
 },
 "studentschoolauthorizationedorgids": null,
 "studentedorgresponsibilityauthorizationids": null,
 "contactstudentschoolauthorizationedorgids": null,
 "staffeducationorganizationauthorizationedorgids": null,
 "createdat": 1765053289159866,
 "lastmodifiedat": 1765053289159866,
 "lastmodifiedtraceid": "0HNHL0L6OGCAS:00000001",
 "__deleted": "false"
}

From this message, read the following variables:

• projectname
• resourcename
• edfidoc

From this, generate an HTTP Post request with basic authorization token to URL {dataManagementApi}/{projectname}/{resourcename}s and use the edfidoc (with the id element deleted) as the payload.