Add orcarouter_chat example#323
Conversation
Greptile SummaryThis PR adds a new
Confidence Score: 3/5The app will work for basic use, but the catalog-fetch can freeze a user's UI for up to 10 seconds on every page load and every Refresh click. The orcarouter_chat/orcarouter_chat/orcarouter_chat.py — specifically the Important Files Changed
Reviews (1): Last reviewed commit: "Add orcarouter_chat example" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| @rx.event | ||
| async def load_models(self): | ||
| """Pull the live model catalog from OrcaRouter; fall back to the curated | ||
| flagship list if the network is unavailable so the demo always boots.""" | ||
| try: | ||
| async with httpx.AsyncClient(timeout=10.0) as client: | ||
| resp = await client.get(ORCAROUTER_PRICING_URL) | ||
| resp.raise_for_status() | ||
| payload = resp.json() | ||
| entries = payload.get("data") or [] | ||
| live = [ | ||
| e["model_name"] | ||
| for e in entries | ||
| if "model_name" in e and is_chat_model(e["model_name"], e) | ||
| ] | ||
| if live: | ||
| merged = ["orcarouter/auto", *sorted(m for m in live if m != "orcarouter/auto")] | ||
| self.models = merged | ||
| self.models_source = ( | ||
| f"live ({len(merged)} models from https://www.orcarouter.ai/models)" | ||
| ) | ||
| if self.model not in merged: | ||
| self.model = merged[0] | ||
| return | ||
| except Exception as exc: # noqa: BLE001 - demo path, surface the reason in UI | ||
| self.error = f"Could not load live model catalog: {exc}. Using fallback list." | ||
| self.models = list(FALLBACK_MODELS) | ||
| self.models_source = ( | ||
| f"fallback ({len(FALLBACK_MODELS)} flagship models; " | ||
| "see https://www.orcarouter.ai/models for the full catalog)" | ||
| ) |
There was a problem hiding this comment.
load_models holds the state lock during a 10-second network call
load_models is decorated with plain @rx.event, so Reflex holds the state lock for the entire handler — including the await client.get(ORCAROUTER_PRICING_URL) with a 10-second timeout. While that HTTP call is in flight, no other event (e.g. typing, sending a message) can mutate state for that user. The submit handler correctly uses @rx.event(background=True) and async with self: for exactly this reason; load_models needs the same treatment.
Rule Used: API calls should be made as background tasks to av... (source)
Learned From
reflex-dev/flexgen#2091
| rx.input( | ||
| placeholder="Ask anything...", | ||
| value=State.prompt, | ||
| on_change=State.set_prompt, | ||
| width="100%", | ||
| ), |
There was a problem hiding this comment.
The prompt rx.input has no on_submit (or on_key_down) handler, so pressing Enter in the input box does nothing — the user must click "Send" each time. Adding on_submit wires Enter to the same handler as the button.
| rx.input( | |
| placeholder="Ask anything...", | |
| value=State.prompt, | |
| on_change=State.set_prompt, | |
| width="100%", | |
| ), | |
| rx.input( | |
| placeholder="Ask anything...", | |
| value=State.prompt, | |
| on_change=State.set_prompt, | |
| on_submit=State.submit, | |
| width="100%", | |
| ), |
| if self.model not in merged: | ||
| self.model = merged[0] | ||
| return |
There was a problem hiding this comment.
load_models runs
When a user pins a custom model via apply_custom_model and then clicks Refresh (or the page reloads), load_models overwrites self.model with merged[0] if the custom model is not in the live catalog. The user loses their selection with no feedback. Preserving the selection when it isn't in the new list (or at least skipping the reset) avoids the silent drop.
| if self.model not in merged: | |
| self.model = merged[0] | |
| return | |
| if self.model not in merged: | |
| merged = [self.model, *merged] | |
| return |
- load_models: run as background task (release state lock during HTTP fetch) - Enter key submits via rx.form (was: button-only) - pinned custom models persist across catalog refreshes - assets/favicon.ico: include reflex default
1 participant
Adds an orcarouter_chat example: a minimal Reflex chat app that calls
OrcaRouter, an OpenAI-compatible LLM gateway, and lets you switch the model
from a dropdown populated at startup from OrcaRouter's public pricing
catalog, with a curated fallback list when that endpoint is unreachable.
Disclosure: I'm an engineer on the OrcaRouter team.