Nexon (NexonTorrent)

A fully local, Windows-only voice assistant ("Jarvis") built from a Qt desktop shell, a fleet of small FastAPI services, and GPU-accelerated local models LLaMA for reasoning, Whisper for speech-to-text, Piper for speech, YOLO for camera presence, and a FAISS index over your own files.

Nothing leaves the machine except optional web search, which goes to a SearXNG instance you run yourself.

---

Read this first: the two-repo layout

This is the single most confusing thing about the project, so it's up front.

There are two separate codebases that together make up "Nexon":

Thing	Location	What it is
NexonInstaller	D:\Sam\NexonInstaller	The installer suite. Nexon.py (ships as Nexon.exe) creates the venv, installs deps, checks for updates, then starts the payload. Nexon_Torrent_Helper.py downloads the ~3.9 GB model bundle. See its README.
NexonTorrent (this repo)	D:\Sam\NexonTorrent	The actual assistant — the "payload" the launcher runs.

And there is a third location that matters at runtime:

%USERPROFILE%\NexonTorrent\     <- where the payload must physically live to run
%USERPROFILE%\.Nexon\           <- the virtualenv the launcher creates

Guard/Config_Helper.py hardcodes the config path to %USERPROFILE%\NexonTorrent\config\Config.json. The config/Config.json in this source tree is not the file the app reads unless this tree is %USERPROFILE%\NexonTorrent. If you edit config here and nothing changes, that's why.

Boot sequence, end to end

Nexon.exe  (NexonInstaller\Nexon.py, the launcher)
  ├─ reads %USERPROFILE%\NexonTorrent\config\Config.json
  ├─ verifies install_path lock (refuses to run if the exe moved)
  ├─ checks GitHub releases for a newer version
  ├─ creates %USERPROFILE%\.Nexon venv if missing
  ├─ pip install -r requirements.txt
  ├─ pip install torch/torchvision/torchaudio  (cu121 index)
  ├─ pip install llama-cpp-python==0.3.4        (cu121 index)
  └─ launches: .Nexon\Scripts\python.exe  NexonTorrent\Orchestrator.py
                 │
                 └─ Orchestrator.py  (this repo's entry point)
                      └─ Guard/Ai_Launch.py :: UltimatePrecisionToolbar
                           └─ [user flips the toggle]
                                └─ run_guard_stack()  → 5 FastAPI procs + Flask + Qt window

So: Orchestrator.py is the entry point of this repo. Everything else hangs off Guard/.

---

Architecture

graph TD
    O[Orchestrator.py] --> TB["Ai_Launch.py<br/>floating toolbar<br/>(the on/off switch)"]
    TB -->|toggle on| STACK[run_guard_stack]

    STACK --> BE["Backend_Main<br/>:7000 FastAPI"]
    STACK --> WS["ai_search_service<br/>:7100 FastAPI"]
    STACK --> APP["ai_application_service<br/>:7101 FastAPI"]
    STACK --> FS["ai_file_agent<br/>:7102 FastAPI"]
    STACK --> SCH["index_scheduler<br/>:7103 FastAPI"]
    STACK --> FE["Frontend_Main<br/>:5000 Flask/waitress"]
    STACK --> QT["Agent_AI.py<br/>Qt WebView2 window"]

    QT -->|loads| FE
    FE -->|fetch /chat /presence<br/>/notifications /is_speaking| BE

    BE --> WS
    BE --> APP
    BE --> FS
    SCH -->|nightly POST| FS
    SCH -->|nightly POST| APP

    WS -->|HTTP| SX[(SearXNG :8888<br/>you run this)]

Loading

Ports

Every port comes from config/Config.json → ports. All bind 127.0.0.1.

Port	Config key	Service	Module
5000	port_6	Flask UI (served by waitress)	Guard/Guard_AI_Suite/Frontend/Frontend_Main.py
7000	port_1	Assistant core — chat, vision, STT, notifications	Guard/Guard_AI_Suite/Backend/Backend_Main.py
7100	port_2	Web search + page fetch	Guard/Guard_AI_Suite/Tools/Search/WebSearch/ai_search_service.py
7101	port_3	App librarian (Start Menu index + launch)	Guard/Guard_AI_Suite/Tools/App/ai_application_service.py
7102	port_4	File agent (FAISS semantic file search)	Guard/Guard_AI_Suite/Tools/Search/FileSearch/ai_file_agent.py
7103	port_5	Nightly re-index scheduler	Guard/Guard_AI_Suite/Scheduler/index_scheduler.py
8888	(env SEARXNG_BASE)	SearXNG — external, not started by this app	—

run_guard_stack() starts them in this order with a 1s gap between each: file agent → web search → app service → backend → scheduler → Flask → Qt window.

---

How a single utterance flows through the system

This is the core loop, in Backend_Main.py:

1. POST /chat enqueues a ChatJob onto _chat_queue. A single background worker thread (_chat_worker_loop) processes them one at a time, so the LLM is never re-entered concurrently.
2. Fact extraction — FactBrain asks the LLM to pull long-term facts (name, location, occupation, tools, people) out of the message and writes them to memory.sqlite3 via memory_store.add_fact.
3. Intent routing — three signals are combined:
   • IntentRouter — a second, smaller LLaMA instance (intent-k2.gguf) that classifies into chat | vision | system | dev | open | web_search.
   • WebNeedDetector / OpenIntentDetector / FileSearchIntentDetector — all three wrap the same HuggingFace Falconsai/intent_classification model and return a confidence score. Scores >= 0.6 override the router.
4. Dispatch:
   • vision → grabs a webcam frame, runs YOLO, returns a scene description
   • open → POST :7101/open, fuzzy-matches the Start Menu index, launches it
   • file-search intent → POST :7102/search/interactive
   • system / dev → stubs, they return a polite "not wired yet" string
   • anything else → jarvis_with_search_tools()
5. Tool loop — jarvis_with_search_tools gives the LLM a one-shot tool prompt. The model either replies in plain text, or emits a single line of JSON ({"tool": "web_search", "args": {...}}). If it's a tool call, the result is fed back in as a context_hint and the model is asked again. One hop only — there is no multi-turn tool loop.
6. Speech — the answer goes to VoiceAgent.speak(), which shells out to piper.exe, writes agent_response.wav, and plays it through PyAudio.

Meanwhile, independently:

• CameraPresenceWatcher (a daemon thread) polls the webcam with YOLO face detection. 3 consecutive frames with a face → awake; 20 without → asleep, and it cuts off any in-progress speech. The UI polls /presence to reflect this.
• index_scheduler sleeps until midnight, then re-indexes files and apps every 24h.

---

Module map

Entry points

File	Role
Orchestrator.py	Start here. Creates the QApplication, shows the toolbar, docks it bottom-center.
Guard/Ai_Launch.py	The frameless floating toolbar. Custom-painted (animated water-fill "Get Started" button, sliding toggle). The toggle spawns/kills the whole service stack as a child multiprocessing.Process.
Guard/Agent_AI.py	The main assistant window — a qtwebview2 WebView2 host pointed at http://127.0.0.1:5000. Handles minimal mode, Esc-to-collapse, tray, and opens the config editor.

Services

File	Notes
Guard_AI_Suite/Backend/Backend_Main.py	~1350 lines. The brain. LLM wrappers, intent routing, system monitor, chat queue, notifications, /chat /presence /vision_snapshot /notifications /is_speaking /stt.
Guard_AI_Suite/Frontend/Frontend_Main.py	~1600 lines, and all of it is one Python string: a single HTML_TEMPLATE with a Shadcn-Zinc themed UI, a Three.js scene, and the JS polling loops. Served by one Flask route.
Guard_AI_Suite/Tools/Search/WebSearch/ai_search_service.py	Queries SearXNG's JSON API; /fetch_url renders a page with headless Playwright Chromium and strips it to text with BeautifulSoup.
Guard_AI_Suite/Tools/App/ai_application_service.py	Walks the Start Menu, resolves .lnk shortcuts, builds a fuzzy-searchable app index, launches by name.
Guard_AI_Suite/Tools/Search/FileSearch/ai_file_agent.py	Indexes ~/Downloads, ~/Documents, ~/Pictures, ~/Videos, ~/Desktop. Chunks text/code/PDF at 1200 chars with 200 overlap, embeds with a local all-MiniLM-L12-v2, stores vectors in FAISS + metadata in SQLite. Supports open / delete (via Recycle Bin) / RAG-over-file.
Guard_AI_Suite/Scheduler/index_scheduler.py	80 lines. Midnight cron that POSTs to the file and app agents.
Guard_AI_Suite/Backend/presence_watcher.py	YOLO face-presence thread (see above).
Guard_AI_Suite/Backend/voice_agent.py	Piper subprocess + PyAudio playback, with an interruptible stop flag.
Guard_AI_Suite/Memory/memory_store.py	Thin SQLite layer over a facts table. Schema lives in Config.json, not in code.

Support

File	Role
Guard/Config_Helper.py	Loads %USERPROFILE%\NexonTorrent\config\Config.json. Every service calls this at import time.
Guard/find_items.py	Resolves a config-relative path against six candidate roots so the same code works in source and inside a PyInstaller onefile bundle.
Guard/config_editor.py	A Qt config editor with a Pygments-highlighted JSON/Python viewer. Reachable from the assistant window.
Guard/StatusBar.py	RAM bar + network status dot, embedded into the toolbar.
Guard/Style_Qss/qss.py	The Qt stylesheet, as one big string.
Guard/get_local_ip.py	Single helper — UDP-connect trick to find the LAN IP.

Bundled binaries and models (Guard/Main-Engine/)

These are vendored, not downloaded at runtime:

• Model/brain_Model/Llama-3-Groq-8B-Tool-Use-Q4_K_M.gguf — main reasoning model
• Model/intent_Model/intent-k2.gguf — the router model
• Model/all-MiniLM-L12-v2/ — sentence-transformers embeddings for file search
• Model/whisper_Model/ggml-base.bin + STT-Engine/whisper-cli.exe — speech-to-text
• Model/en_onnx-{ryan-high,amy-medium}_Model/ + TTS-Engine/piper-cpu/piper.exe — speech
• Model/yolo_Model/ — yolov8n-face.pt (presence), yolo11n/11s, yolov8n (scene)
• Model/mediapipe_face_Model/face_landmarker.task, Model/face_Model/nate_face_embedding.npy

---

Configuration

Everything tunable lives in config/Config.json (read from %USERPROFILE%\NexonTorrent\config\Config.json). Notable sections:

Key	What it controls
ports	All six service ports
models	Paths to every .gguf / .onnx / .pt, resolved through find_items()
LlamaCppModel_settings	Main model: n_ctx 6096, 512 max tokens, 20 GPU layers
IntentRouter_LlamaCppModel_settings	Router: n_ctx 1024, 128 tokens, 2 GPU layers
file_agent	Which directories to index, extensions, exclusions, chunk size, worker count
memory_db	DB path and the full CREATE TABLE schema as a string
install_path	Written on first launch and then enforced — the launcher refuses to start if Nexon.exe has moved
update_url / github_api_endpoint	Points at github.com/headlessripper/Nexon/releases

Only one setting is an environment variable rather than config: SEARXNG_BASE (default http://127.0.0.1:8888).

---

Requirements

• Windows only. WebView2, .lnk resolution, %USERPROFILE%, CAP_DSHOW webcam capture, and bundled .exe/.dll binaries all assume it.
• Python 3.11 (the checked-in .pyc files are cpython-311).
• An NVIDIA GPU with CUDA 12.1. The launcher installs torch and llama-cpp-python from cu121 indexes. It runs on CPU, but slowly — DEVICE falls back automatically.
• A webcam and a microphone.
• A running SearXNG instance if you want web search.

---

Running it

The intended path

Build/run Nexon.exe from NexonInstaller, with this tree deployed to %USERPROFILE%\NexonTorrent. The launcher does the rest.

Running from source

xcopy /E /I D:\Sam\NexonTorrent %USERPROFILE%\NexonTorrent

Then:

python -m venv %USERPROFILE%\.Nexon

%USERPROFILE%\.Nexon\Scripts\pip install -r %USERPROFILE%\NexonTorrent\requirements.txt

%USERPROFILE%\.Nexon\Scripts\pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

%USERPROFILE%\.Nexon\Scripts\pip install llama-cpp-python==0.3.4 --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121

flask is missing from requirements.txt even though Frontend_Main.py needs it — install it too:

%USERPROFILE%\.Nexon\Scripts\pip install flask

Playwright needs its browser downloaded once:

%USERPROFILE%\.Nexon\Scripts\playwright install chromium

Then run from the NexonTorrent root — cwd matters, because there are no __init__.py files and Guard.* imports resolve as implicit namespace packages relative to the working directory:

cd %USERPROFILE%\NexonTorrent && %USERPROFILE%\.Nexon\Scripts\python Orchestrator.py

A floating black toolbar appears at the bottom of the screen. Click the right side to expand it, then flip the toggle to start the service stack. Expect a slow first start — two GGUF models, YOLO, and a transformers classifier all load up front.

Building

• Guard/BuildConfig/Nexon_Build.json — auto-py-to-exe config for the payload (onefile, console on, --collect-all llama_cpp, every service listed as a hidden import).

The launcher's build configs (Nexon_Orca.json, Nexon.iss) now live with the installer, in NexonInstaller/BuildConfig/. Copies remain here under Guard/BuildConfig/ and can be deleted once you've confirmed the installer builds.

Note: all of these build configs still carry absolute paths to an older project directory (C:\Users\nateg\Documents\Virus_Defense_Engine\). They need rewriting before they'll build from the current layout.

---

Known rough edges

Things that will surprise you, roughly in order of how much time they'll cost:

• The config the app reads is not the config in this tree. See the top section.
• requirements.txt is UTF-16LE with a BOM. pip decodes it fine, but most editors and any grep/cat will show it as spaced-out garbage. Converting it to UTF-8 is safe — the launcher's own parser handles both.
• flask is absent from requirements.txt but is a hard dependency of the Flask frontend.
• /stt doesn't transcribe. The endpoint saves the uploaded blob to incoming_audio.webm and returns an empty string. It also tries to construct STTResponse(text="", error="TTS_ACTIVE_IGNORE"), but STTResponse has no error field, so that branch raises. The working Whisper wrapper lives in Main-Engine/Source/Main/stt_engine.py and is not wired to anything.
• system and dev intents are stubs. The router will classify into them, and the handler returns a "not fully wired yet" sentence.
• Guard/Style_Qss/Head.py is broken and orphaned. It imports Guard.SentinelStorage and Guard.SentinelWeather, but those files live at Guard/Style_Qss/. Nothing imports Head.py, so it never surfaces.
• stt_engine.py imports from find_items import find_items (not Guard.find_items) and needs sounddevice + scipy, neither of which is in requirements.txt. It only works as a standalone script.
• The assistant is hardcoded to one user. Prompts address "Nate" by name, reference "his Lenovo Legion", and memory rows are all written under the literal user key "nate". Search for "nate" to find them all.
• Main-Engine/Source/Tests/ is not a test suite. test.py, test1.py, test2.py, test3.py are ~1000-line snapshots of earlier monolithic versions of the backend. There are no automated tests in this project.
• A stale duplicate source tree still exists at D:\Sam\ZashironSentinel\Nexon_Project\Guard\ (~3.9 GB). It's an older copy of NexonTorrent/Guard/ that has diverged: it ships Brain_M2.gguf where config expects Llama-3-Groq-8B-Tool-Use-Q4_K_M.gguf, and it's missing yolov8n-face.pt (which presence_watcher requires). The installer has been extracted out to D:\Sam\NexonInstaller, so nothing references this tree any more — it's safe to delete once you've confirmed the installer builds. NexonTorrent is canonical.
• Config_Helper.py exists in two versions. This repo's copy has only load_config(); the installer's copy adds save_config() and update_config_field(). Nexon.py needs the superset. If you consolidate, keep the installer's version.
• The install-path lock is strict. Move Nexon.exe after first run and the launcher refuses to start until you clear install_path from the config.
• Startup is time.sleep(1.0) between services, not a health check. On a cold cache the file agent may not be listening when the backend first calls it.

---

Repository layout

NexonTorrent/
├── Orchestrator.py              # entry point
├── requirements.txt             # UTF-16LE
├── test.py                      # fastembed model pre-download probe
├── config/Config.json           # NOT the file read at runtime
├── background/                  # app icon
└── Guard/
    ├── Ai_Launch.py             # floating toolbar + stack launcher
    ├── Agent_AI.py              # WebView2 assistant window
    ├── Config_Helper.py         # config loader
    ├── find_items.py            # dev/frozen path resolver
    ├── config_editor.py         # Qt JSON config editor
    ├── StatusBar.py             # RAM + network widget
    ├── get_local_ip.py
    ├── BuildConfig/             # PyInstaller + Inno Setup configs
    ├── Cache/                   # fastembed cache warm-up scripts
    ├── Style_Qss/               # stylesheet + storage/weather widgets
    ├── background/              # icons, images, tray art
    ├── Guard_AI_Suite/
    │   ├── Backend/             # Backend_Main, presence_watcher, voice_agent
    │   ├── Frontend/            # Flask UI (one giant HTML string)
    │   ├── Memory/              # SQLite fact store
    │   ├── Scheduler/           # nightly re-index
    │   └── Tools/
    │       ├── App/             # Start Menu index + launcher
    │       └── Search/
    │           ├── FileSearch/  # FAISS file agent
    │           └── WebSearch/   # SearXNG + Playwright
    └── Main-Engine/
        ├── Model/               # all .gguf / .onnx / .pt weights
        ├── STT-Engine/          # whisper.cpp binaries
        ├── TTS-Engine/          # piper binaries + espeak-ng data
        ├── Voice_Profiles/      # enrollment WAVs
        └── Source/
            ├── Main/            # stt_engine.py, tts_engine.py (standalone)
            ├── CodeScripts/     # TTS/STT experiments
            └── Tests/           # old monolith snapshots, NOT tests

---

Suggested cleanup order

If you want to make this less confusing, in rough priority:

1. Add __init__.py files (or a pyproject.toml) so Guard.* imports stop depending on the working directory.
2. Make Config_Helper fall back to a config next to the source tree when %USERPROFILE%\NexonTorrent\config\Config.json is absent.
3. Re-encode requirements.txt as UTF-8 and add flask, torch, and llama-cpp-python (with a comment about the cu121 index).
4. Delete Main-Engine/Source/Tests/ and Guard/Style_Qss/Head.py, or move them to an archive/ folder.
5. Replace the time.sleep(1.0) startup gaps with /health polling — every service already exposes one.
6. Pull the "Nate" hardcoding into config so the assistant has a configurable user.

NexonInstaller

The installer and bootstrapper suite for Nexon. This folder does not contain the assistant — it contains the two GUI programs that get the assistant onto a machine and start it.

The assistant itself lives in %USERPROFILE%\NexonTorrent.

---

What's in here

File	Ships as	Toolkit	Role
Nexon.py	Nexon.exe	PySide6	The launcher. Creates the venv, installs dependencies, checks GitHub for updates, then starts the payload.
Nexon_Torrent_Helper.py	separate exe	PyQt6	The payload downloader. Pulls the ~3.9 GB model bundle from HuggingFace with resumable download, extracts it to %USERPROFILE%.
Guard/Config_Helper.py	—	—	Reads/writes %USERPROFILE%\NexonTorrent\config\Config.json. Shared with the payload, but this copy is the superset (see below).
Model-Icon/	bundled asset	—	forward.png / close.png — the start and stop buttons on the downloader.
BuildConfig/Nexon_Orca.json	—	—	auto-py-to-exe config that builds Nexon.exe (onefile, windowed).
BuildConfig/Nexon.iss	—	—	Inno Setup script that wraps the built exe into NexonSetup.exe.

The two scripts use different Qt bindings. Nexon.py is PySide6, Nexon_Torrent_Helper.py is PyQt6. That's why requirements.txt lists both. Don't consolidate them without porting the downloader first.

---

The three-directory model

Nexon spreads itself across three places at runtime. Knowing which is which explains almost every "why isn't my change taking effect" question:

%USERPROFILE%\NexonTorrent\      <- where the payload must physically live to run
%USERPROFILE%\.Nexon\            <- the venv the launcher builds

Guard/Config_Helper.py hardcodes the config path to %USERPROFILE%\NexonTorrent\config\Config.json. Both the installer and the payload read that one file. Editing Config.json in either source tree changes nothing unless that tree is %USERPROFILE%\NexonTorrent.

---

Install flow

1. NexonSetup.exe        (Inno Setup, from BuildConfig/Nexon.iss)
      installs Nexon.exe

2. Nexon_Torrent_Helper  (if the payload isn't present yet)
      downloads Engine-Models.zip from HuggingFace
      extracts to %USERPROFILE%  ->  creates %USERPROFILE%\NexonTorrent\

3. Nexon.exe             (Nexon.py)
      reads %USERPROFILE%\NexonTorrent\config\Config.json
      verifies install_path lock  (refuses to run if the exe moved)
      checks GitHub releases for a newer version
      creates %USERPROFILE%\.Nexon venv if missing
      pip install -r %USERPROFILE%\NexonTorrent\requirements.txt
      pip install torch torchvision torchaudio      (cu121 index)
      pip install llama-cpp-python==0.3.4           (cu121 index)
      launches: .Nexon\Scripts\python.exe  NexonTorrent\Orchestrator.py
                  -> hands off to the payload

Step 3's last line is the handoff point. Everything after it is documented in the payload README.

---

Running from source

python -m venv .venv

.venv\Scripts\pip install -r requirements.txt

Run from this folder — Nexon.py does from Guard.Config_Helper import ..., and with no __init__.py that resolves as an implicit namespace package relative to the working directory:

cd D:\Sam\NexonInstaller && .venv\Scripts\python Nexon.py

Nexon.py will fail immediately with FileNotFoundError if %USERPROFILE%\NexonTorrent\config\Config.json doesn't exist. Either run the payload downloader first, or copy NexonTorrent\config\ there by hand.

To run just the downloader:

cd D:\Sam\NexonInstaller && .venv\Scripts\python Nexon_Torrent_Helper.py

---

Building

Both build configs are auto-py-to-exe format — load them from its GUI, or feed the equivalent flags to PyInstaller directly.

BuildConfig/Nexon_Orca.json still points at an old project directory (C:\Users\nateg\Documents\Virus_Defense_Engine\Nexon.py) and an icon at C:\Users\nateg\Downloads\icons8-ai-96.ico. Update filenames and icon_file to this folder before building.

BuildConfig/Nexon.iss has the same problem: OutputDir, Source, and SetupIconFile all reference Virus_Defense_Engine. It also sets a hardcoded installer password (Password=Nate@360, Encryption=yes) — worth removing or rotating if this ever ships beyond your own machine.

---

Known rough edges

• Config_Helper.py is duplicated and the two copies differ. This folder's copy has save_config() and update_config_field(); the payload's copy (NexonTorrent/Guard/Config_Helper.py) has only load_config(). Nexon.py imports save_config, so it needs this version. If you ever consolidate them, the installer's superset is the one to keep.
• The install-path lock is strict. On first run Nexon.py writes its own directory into config["install_path"]. On later runs a mismatch blocks startup. Moving Nexon.exe means clearing that key by hand.
• The downloader always extracts to %USERPROFILE%, hardcoded. The zip's internal structure is what creates the NexonTorrent folder name — the script doesn't verify the result landed anywhere sensible.
• A stale duplicate of the payload still exists at D:\Sam\ZashironSentinel\Nexon_Project\Guard\ (~3.9 GB). It's an older copy of NexonTorrent/Guard — different Backend_Main.py, ships Brain_M2.gguf instead of the Llama-3-Groq model that Config.json expects, and is missing yolov8n-face.pt that presence_watcher.py requires. Nothing references it. It was left in place deliberately; delete it once you've confirmed this folder builds.