Spoken (fluent-meet-frontend)

"Speak your language, they hear theirs."

Spoken.ai is a real-time, AI-powered voice translation video conferencing platform. It eliminates language barriers in global collaborations by providing instantaneous, natural-sounding voice translation, allowing meeting participants to communicate naturally in their native tongues while others hear them in their own chosen language.

---

Key Features

• Real-Time Voice Translation: Local microphone input is captured, downsampled, and streamed to the backend translation pipeline. Translated speech is played back back-to-back to remote listeners in their target language.
• Direct P2P WebRTC Video & Audio Mesh: Uses direct peer-to-peer WebRTC connections for low-latency sharing of raw video and original audio streams among participants.
• Multilingual Captioning Overlay: Displays real-time transcription and translation captions on top of the meeting interface, separated into interim (real-time updates) and final segments.
• Lobby & Admission Control: Meeting hosts can enable a lobby where guests "knock" to request access, and hosts can approve or deny entry.
• Transcript Downloads: Participants can download a full, time-stamped text transcript of the meeting including original and translated statements at the end of the session.
• Responsive Grid & Custom Controls: An adaptive UI grid adjusting dynamically to participant numbers, featuring options for mic/camera selection, screen sharing, and profile configuration.

---

Technology Stack

• Core Framework: Angular 21 (utilizing Signals for state management, Standalone Components, and OnPush change detection).
• Styling: Tailwind CSS v4 with PostCSS.
• Real-Time Streaming:
  • WebRTC (Mesh Topology): For direct, multi-peer video and original audio streaming.
  • WebSockets: Custom ManagedWebSocket client wrapper with automatic heartbeat, reconnect mechanisms, and exponential backoff. Handles signaling, raw binary audio transmission, and caption JSON streaming.
• Audio Engineering: Web Audio API (with custom inline AudioWorkletNode for real-time downsampling to 16kHz Int16 PCM, and playback context scheduling for gapless audio playout).
• Testing Suite:
  • Unit Testing: Vitest (integrated via native Angular @angular/build:unit-test builder).
  • End-to-End Testing: Playwright for headless browser automation.
• CI/CD & Deployment: Docker, Nginx (with reverse proxy configurations for WebSockets), and Semantic Release.

---

Directory Structure

fluent-meet-frontend/
├── .github/workflows/      # CI/CD configurations
├── nginx/                  # Nginx proxy and HTTPS configuration
│   └── spoken-api.conf     # Reverse proxy settings for WebSocket upgrading
├── public/                 # Static assets (logos, icons, web manifest)
├── scripts/                # Utility scripts
│   └── set-env.mjs         # Build-time environment variable injector
└── src/
    ├── app/
    │   ├── auth/           # JWT & guest session managers, interceptors, guards
    │   ├── core/           # Layout components, error handlers
    │   ├── meeting/        # Meeting state services, room config, REST API wrapper
    │   ├── pages/          # View pages (Login, Pre-join, Lobby, Meeting room, etc.)
    │   ├── realtime/       # WEBRTC peer mesh, Audio capture worklets, WebSocket services
    │   └── shared/         # Reusable UI controls (Video tiles, Captions overlay, Toolbar)
    ├── environments/       # Environment files (local & production targets)
    └── styles.css          # Global stylesheet importing Tailwind CSS

---

Connection Flow & WebRTC Architecture

To learn how Spoken establishes connections, handles low-level WebRTC offers/answers, resolves glare issues, performs linear interpolation audio downsampling, and updates captions, see our comprehensive guide:

👉 Low-Level WebRTC & Audio Meeting Flow Documentation (MEETING_FLOW.md)

---

Getting Started

Prerequisites

• Node.js v22 or higher
• pnpm package manager (installed globally via corepack enable or npm install -g pnpm)

Installation

1. Clone the repository and navigate to the project directory:

   git clone https://github.com/your-org/fluent-meet-frontend.git
   cd fluent-meet-frontend

2. Install dependencies:

   pnpm install

Running Locally

1. Setup the environment configuration in src/environments/environment.ts:

   export const environment = {
     production: false,
     apiBaseUrl: 'http://localhost:8000/api/v1',
     wsBaseUrl: 'ws://localhost:8000/api/v1',
     sentryDsn: '',
     iceServers: [{ urls: 'stun:stun.l.google.com:19302' }],
   };

2. Spin up the development server:

   pnpm start

3. Open your browser and navigate to http://localhost:4200.

---

Production Build & Deployment

Before generating static assets, environment variables are injected dynamically.

Build Configuration

The script scripts/set-env.mjs runs automatically prior to compiler execution to populate src/environments/environment.production.ts. You can configure the following environment variables:

Environment Variable	Description	Default
API_BASE_URL	Production REST API endpoint	https://spoken-api.unraveldocs.xyz/api/v1
WS_BASE_URL	Production WebSocket endpoint	Derived from API_BASE_URL
SENTRY_DSN	Sentry Error Tracking DSN (optional)	"" (Disabled)
METERED_USER	Metered TURN Server username	d0a4445f810607c59d426678
METERED_CREDENTIAL	Metered TURN Server password	lGWo9ICXxlDFdPej

Build Command

Compile the application bundle:

pnpm build

This runs the pre-build config generator and outputs the optimized production files in the dist/ directory.

Nginx Routing Configuration

When hosting the build under a custom server (like Nginx), make sure to configure proxy upgrades for the WebSocket pathways. A reference Nginx site file is provided in nginx/spoken-api.conf. The key directive enables HTTP Upgrade header support:

location /api/v1/ws/ {
    proxy_pass http://fluentmeet_api;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_read_timeout 86400s;
    proxy_send_timeout 86400s;
}

---

Running Tests

Unit Tests (Vitest)

Run the fast unit test suite:

pnpm test

To run tests once without file watch loops (ideal for CI pipelines):

pnpm test-no-watch

End-to-End Tests (Playwright)

Install browser drivers and run the end-to-end integration tests:

pnpm exec playwright install chromium
pnpm e2e

To debug E2E tests interactively via the Playwright UI:

pnpm e2e:ui

To view the generated test coverage reports:

pnpm e2e:report

---

License

This project is licensed under the MIT License - see the LICENSE file for details.