backend sparsh#1
Open
tobasummandal wants to merge 17 commits into
Open
Conversation
…world topology map
Introduce background ingest jobs and real-time progress reporting, plus improved error handling and optional VectorAI usage.
Backend: add an async ingest job queue with UUID job IDs, polling endpoint (/ingest/{job_id}), and worker that updates job progress. Enhance /ingest to enqueue jobs (202) and keep backward-compatible ingest logic by adding a ProgressCallback flow in ingest.py that emits percent/phase/messages per chunk. Make VectorAI usage optional and resilient: add db.vectorai_ready(), skip or catch VectorAI operations where unavailable, and log failures instead of failing ingestion. Add Gemini API health/error parsing to surface quota/auth errors and return structured HTTP errors for endpoints using Gemini.
Frontend: update API client to startIngest/getIngestStatus and types for ingest jobs/progress. Dashboard now starts ingest jobs, polls status, displays a progress bar, and surfaces ingest/check errors and summaries. Small UI tweaks for layout and messages.
Misc: minor WorldGraph key handler fix and small refactors/comments. These changes enable reliable long-running ingestion, better user feedback, and graceful degradation when external services (VectorAI/Gemini) are unavailable.
Stop displaying embedding chunk counts in the Dashboard ingest summary and progress line. Updated setIngestSummary and the ingestProgress.totals rendering to only show entities and relationships (frontend/src/components/Dashboard.tsx).
Add render.yaml and DEPLOY_RENDER.md to enable one-click deployment on Render. The blueprint defines a private Docker service (vectoraidb) with a persistent 5GB disk and a Python web service (archivist-backend) that installs dependencies, a local Actian wheel, and exposes a health endpoint. The added docs explain creating the Blueprint, setting the GEMINI_API_KEY secret, and verifying deployment via /health (including vectorai and gemini checks). Environment variables VECTORAI_HOST/PORT and GEMINI_API_KEY are configured in the blueprint.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Introduce a NODE_TYPES list and activeTypes state to allow toggling visible node categories. The graph data is now filtered by selected types (nodes and edges) and the vis Network is rebuilt when activeTypes changes. Adds a toggleable legend UI with hover/active styles and prevents deselecting the last remaining type. Minor UI copy updated to indicate click-to-toggle behavior.
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Add a user-resizable graph panel with a draggable divider: introduce GRAPH_MIN_H / GRAPH_MAX_H / GRAPH_DEFAULT_H, graphHeight state, pointer handlers (capture, move, release) and RAF-throttling to smoothly update height. Pass graphHeight into WorldGraph and redraw the vis.js network on height changes (avoid full rebuild). Adjust WorldGraph to respect the passed height for loading/empty states and the main container, and tweak styles/transitions. Also update FlagPanel placeholders to use full-height centered layouts with a minimum height.
Replace the old _chunk_progress_start with a new _chunk_pct that returns a percent (5-95) for a 0-based chunk index and a sub-phase fraction, giving each chunk an equal slice of the progress range. Update ingest to emit progress using _chunk_pct at several sub-phase checkpoints (start, writing_graph, embedding, chunk_complete) for more consistent and bounded progress values. Update Dashboard UI to derive displayed percent and progress-bar width from chunk_index/total_chunks when available (falling back to the emitted percent), ensuring the frontend reflects chunk-level progress and avoids jitter.
Add root-level dev-up.sh and dev-down.sh to manage the local development stack end-to-end.\n\nWhat this enables:\n- Starts VectorAI Docker dependency (creates container if missing, starts if stopped)\n- Starts backend API on :8000 and frontend Vite app on :5173\n- Writes runtime logs/PIDs under .run for easier diagnostics and lifecycle control\n- Provides a matching down command to stop app processes and VectorAI cleanly\n\nOperational behavior:\n- Guard rails for missing deps (docker/npm/venv/node_modules)\n- Port checks to avoid duplicate starts\n- Health-wait checks for backend and frontend during startup docs(chore): document one-command workflow and ignore runtime artifacts\n\n- Update README with 'One-command dev run' section showing ./dev-up.sh and ./dev-down.sh\n- Add .run/ to .gitignore to keep generated PID/log files out of git status
Add a background mode to dev-up so startup orchestration can be detached from the terminal session.\n\nChanges:\n- Support ./dev-up.sh --background to run bootstrap under nohup and return immediately\n- Route bootstrap output to .run/dev-up.log for visibility\n- Keep existing foreground mode available via --foreground/internal flow\n- Ensure backend/frontend child processes are launched with stdin detached\n- Retain early-fail PID checks so startup fails clearly if services exit immediately\n\nDocs:\n- README now includes explicit background startup command
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.