AiFlow

This project is still a work in progress

OpenAI API someday...?

A minimal project for running local or hosted AI tools, meant to target multiple different types of AI, such as text-text LLMs like ChatGPT and DeepSeek, as well as speech to text and text to speach. Meant to be modularized, built on top of Nvidia's Triton AI Inference Server.

The user can select the different types of media they wish for the model to use, and if such a model is available at the triton endpoint, it can be used

Usage

To start the triton server

# In project root
cd model-repository
docker build -t triton . # "triton" is the container name hardcoded in the compose

# Then, depending on whether you want GPU functionality or not
docker compose -f triton-dev.yml up
# Or
docker compose -f triton-gpu.yml up

Then to start the frontend in dev mode:

npm run dev

Showcase