Pods domain at the same time as the llm domain?
How easy is it to switch llms on the fly? Just thinking about docker compose up and down...
Fine tuning for switching stuff out.
Building intuition for things like latency, throughput, how many users we can server, etc.
opt in to monitoring -- useful for monitoringe and debugging, if you opt in, then you give us permission to use your information for research purproses.
If you have temperature=0.0, then we can give you the same thing back.
Sometimes people want to use the API, but they don't care about privacy, and just want easy start up, and we just automatically give them access to online API stuff (act as open router for the university).
LCD display for the grafana dashboard and LLM usage.
Terms of use for GPUs 2 and 3
File system.
Restrict access to certain users.
Plan for transfer after Accelerate finishes.
Plan for events to bring others together over shared interest.
How to control people leaking API keys -- IP address monitoring to look for location.