REST in pieces

RestInPieces is a Go framework for building secure, high-performance servers. It is designed to be extended and customized, providing a solid foundation for your own applications while remaining lightweight and focused. The framework uses SQLite as its default database and minimizes reliance on third-party packages, emphasizing simplicity and performance.

To get started, follow the Bootstrapping Guide, which walks you through the initial setup of a new application.

Core Philosophy: One Process Application

This framework is built on the philosophy of One Process Application architectural approach that consolidates an entire application and its core dependencies into a single, self-contained binary.

The central idea is to "absorb" the typical components of a modern web service—such as the database, cache, and job queue—into one application binary. Instead of managing a complex stack of separate services (e.g., a database server, a Redis instance, a reverse proxy), you deploy a single binary. This dramatically simplifies development, deployment, and maintenance.

By running a single Go binary with an embedded SQLite database on one VM, it allows developers to focus on building features rather than managing distributed systems, providing a high-performance foundation that can serve a growing business for years. When the time comes to scale beyond what a single large server can offer, the business will have the resources and clarity to do so effectively.

This approach is heavily inspired by the ideas in One Process Programming Notes.

Content

Framework Key Features

• Data Durability
• Database Drivers
• Router
• Cache
• Authentication
• Security
• Core Infrastructure
• Configuration Management
• Deployment & Operations
• Frontend Integration
• Job Framework
• Performance
• Metrics
• Logger
• Notifications
• Mailer
• Middleware
• Extensibility

Building on the Framework

• Layout Best Practices
• Examples
• Building the Project
• TODO

Key Features

Data Durability

The "one process" paradigm intentionally avoids external dependencies like separate database servers, as they would violate the architectural principle of maintaining a single process per virtual machine. Consequently, the framework relies on an embedded SQLite database for data persistence. This design choice places critical importance on the durability of the single database file.

To address this, the framework provides robust mechanisms for data protection and recovery:

• Local Backups: The framework includes a simple, integrated backup solution for SQLite databases, managed as a background job. This can be configured and activated directly in the application's settings. It operates in two modes:

  • Online Mode: Performs a live backup using SQLite's Online Backup API. This allows the application to continue its operations with minimal interruption, making it ideal for active databases. The backup process copies the database page by page, with configurable pauses to reduce I/O contention.
  • Vacuum Mode: Creates a clean, defragmented, and compact copy of the database using the VACUUM INTO command. This method is thorough but requires more significant locking, making it suitable for maintenance windows or less active databases.

  Backups are saved as compressed .bck.gz archives in a configurable directory, with filenames containing a timestamp and the strategy used. You can pull those gz files from a client available at restinpieces-sqlite-backup.

• Real-Time Replication: For more robust, real-time replication and point-in-time recovery, a Litestream-based integration is available in a separate repository. See restinpieces-litestream for implementation details. This approach ensures that the state of the SQLite database is continuously synchronized to a remote location, providing a strong guarantee against data loss.

Database Drivers

The framework defaults to using zombiezen/go-sqlite, a pure Go SQLite driver that offers excellent performance without relying on CGo. This simplifies the build process and ensures portability. For users who require an alternative, the framework is designed to be modular, and an implementation using the popular crawshaw.io/sqlite driver is also available.

Router

The framework uses Go's standard http.ServeMux as its default router for simplicity and compatibility with the standard library. As of Go 1.22, the standard mux includes support for path parameters. Recognizing that different applications have different routing needs, the router is implemented as a swappable component. For those seeking maximum performance, an alternative implementation using the highly optimized julienschmidt/httprouter is also provided. See restinpieces-httprouter for details.

Cache

For in-memory caching, the framework uses Ristretto, a high-performance, concurrent cache. The caching system is designed around a simple interface, allowing developers to easily swap in their own caching implementation if needed.

Authentication

The framework provides a comprehensive authentication system built around JSON Web Tokens (JWT). Session management is handled via bearer tokens sent in the Authorization header. A key security feature is the use of dynamic JWT signing keys, which are derived from a combination of user-specific credentials (email and password hash) and a global server secret. This ensures that a token's signature is invalidated if a user's password changes.

The system supports multiple authentication and account management workflows through a set of API endpoints:

• Password-based: Includes endpoints for user registration (/register-with-password), login (/auth-with-password), and token refresh (/auth-refresh).
• OAuth2: Provides a generic flow (/auth-with-oauth2) to authenticate users via third-party providers. It handles the token exchange, fetches user information, and creates or links the user account in the local database. An endpoint (/list-oauth2-providers) is available to discover configured providers.
• Account Management: All account management processes, such as email verification, password reset, and email address changes, are handled through secure, multi-step flows. These flows typically involve generating a unique, short-lived JWT that is sent to the user's email via a background job queue, which the user then submits back to a confirmation endpoint.

Security

The "one process" paradigm simplifies deployment by running a single binary on a single VM, but it also means the application is directly exposed to the internet without a reverse proxy like Nginx acting as a first line of defense. This necessitates a defensive approach to security. The framework addresses this with a suite of built-in middleware designed to protect the server from common threats. These include dynamic IP blocking (BlockIp) to mitigate traffic spikes, hostname validation against a whitelist (BlockHost), request body size limitation (BlockRequestBody), and User-Agent filtering (BlockUaList). The framework also helps secure client communications by automatically setting security headers like Strict-Transport-Security.

No CORS support is provided as it contradicts the One Process philosophy. If you need cross-origin requests, you'll need to implement CORS middleware yourself.

Core Infrastructure

The framework is built on standard Go patterns, utilizing middleware and handlers to provide a familiar and robust development experience. It features a set of discoverable API endpoints for essential services, such as token refreshing (/api/refresh-auth) and OAuth2 authentication (/api/auth-with-oauth2), facilitating easy integration and exploration.

Configuration Management

The framework's configuration is securely managed within the SQLite database. The configuration is stored as encrypted, TOML-formatted content in the app_config table, the schema for which is detailed in migrations/schema/app/app_config.sql. Management is performed using the ripc command-line tool, which supports versioning, diffing, and rollbacks. Beyond managing the core application's settings, ripc can be extended to handle custom configuration scopes for your own modules. For more details on the tool, see the ripc documentation.

ripc is though a low-level primitive, whereas ripdep (source) is a high-level orchestrator.

A key feature is support for dynamic updates. The server listens for the SIGHUP signal to trigger a hot-reload of the configuration, allowing most settings to be changed in real-time without service interruption. While the majority of parameters can be updated on-the-fly, critical changes like modifications to TLS certificates require a full server reload to be applied.

Deployment & Operations

The framework provides ripdep, a comprehensive CLI tool designed to manage the full lifecycle of your application. It acts as a high-level wrapper around the ripc binary, orchestrating complex DevOps tasks and remote operations via SSH directly from your local developer machine.

• Remote DevOps: Wraps low-level ripc commands to handle configuration, maintenance modes, and log monitoring without needing manual server access.
• Disaster Recovery: Simplifies the process of bootstrapping new servers and recovering from backups (including Litestream integration) through dedicated commands like build-bootstrap and build-recovery.

This tool encourages a workflow where most configuration and operational decisions are made locally, then securely applied to the remote environment. For detailed usage, see the Deployment Guide.

Frontend Integration

The framework includes a comprehensive JavaScript SDK designed for seamless frontend-backend interaction. The SDK offers full support for all authentication workflows, including password-based and OAuth2 flows, ensuring a consistent integration experience. Beyond authentication, it provides robust utilities for custom error handling, local storage management, and general-purpose request functions to simplify API communication. You can find example usage of the SDK and authentication endpoints at restinpieces-js-sdk.

Job Framework

The framework includes a robust job queue system for handling asynchronous tasks, supporting both one-time and recurrent jobs. This is essential for offloading work from the request-response cycle, such as sending emails, processing data, or performing periodic maintenance.

The system is composed of a scheduler that claims jobs from the job_queue table and an executor that runs the corresponding handler. The framework provides built-in handlers for core functionalities like sending password reset emails, email verifications, and performing local database backups.

You can easily extend the system to run your own custom tasks. This involves two main steps:

1. Write a Job Handler: Create a new handler that implements the JobHandler interface. This is where you define the logic for your task.
2. Insert a Job: Add a new record to the job_queue table in the database. The scheduler will automatically pick it up and execute it using your custom handler.

This design allows for a clean separation of concerns and makes it straightforward to add new background processing capabilities to your application.

Performance

Engineered for high throughput, the framework is capable of handling thousands of requests per second while maintaining a minimal footprint by avoiding unnecessary external dependencies. Production-ready builds are further optimized for size and efficiency, ensuring rapid deployment and execution in resource-constrained environments.

Metrics

The framework provides built-in metrics collection using the prometheus/client_golang library. It includes a middleware that tracks the total number of HTTP requests (http_server_requests_total), a counter labeled by HTTP status code, allowing for detailed monitoring of server responses. Metrics collection can be toggled on or off via configuration without a server restart and is exposed on a configurable endpoint (e.g., /metrics) for a Prometheus server to scrape.

Logger

The framework's logging is built upon the standard slog library for structured logging. It includes a high-performance batching handler that writes logs to the SQLite database, with configurable flush intervals and log levels. For incoming requests, a dedicated middleware logs request details but truncates overly long URI, User-Agent, Referer, and IP values to maintain clean logs. The entire logging implementation can be replaced with a user-defined logger to accommodate custom requirements.

Notifications

The framework's notification system is designed around a Notifier interface, which standardizes how notifications are sent. The primary data structure, Notification, carries a Type (e.g., Alarm, Metric), Source, Message, and a map of Fields for additional structured data.

An official implementation for Discord is included, which sends formatted messages to a configured webhook URL. This notifier operates asynchronously, using goroutines for non-blocking Send calls. It incorporates a rate limiter to prevent API abuse and automatically truncates messages that exceed Discord's 2000-character limit. Developers can create custom notifiers for other services (like Slack or email) by providing their own implementation of the Notifier interface.

Mailer

The framework includes a Mailer component for sending transactional emails over SMTP. It is designed to be flexible and resilient, handling common account management workflows.

• Configuration: The mailer is configured through the application's central configuration provider, allowing for dynamic updates to SMTP settings (host, port, credentials, TLS) without a server restart.
• Protocol Support: It supports standard SMTP authentication methods (PLAIN, CRAM-MD5) and connection security (explicit TLS and STARTTLS).
• Transactional Emails: Pre-built methods are included for common user actions:
  • Email address verification
  • Password reset requests
  • Email change notifications
• Asynchronous Sending: Emails are sent in a non-blocking manner using goroutines, with context-based timeouts to prevent long-running operations from impacting application performance.
• Templating: It uses simple, embedded HTML templates for emails, which can be easily customized.

Middleware

The framework provides a collection of built-in middleware to handle common cross-cutting concerns like security, logging, and metrics.

• ResponseRecorder: A utility middleware that wraps the standard http.ResponseWriter to capture the status code, response size, and timing information. This is used internally by other middleware like Metrics and RequestLog and should typically be the first middleware in the chain.
• RequestLog: Provides structured logging for every incoming HTTP request. It captures details like method, URI, status, duration, remote IP, and user agent, with configurable length limits to keep logs concise.
• Metrics: Collects Prometheus-compatible metrics for HTTP requests, labeled by status code. When activated, metrics are exposed on a configurable endpoint (e.g., /metrics) for scraping.
• BlockIp: Acts as a dynamic IP blocking mechanism to protect the server from traffic spikes and potential denial-of-service attacks. It uses a Top-K sketch algorithm to identify and temporarily block IP addresses that are responsible for a disproportionate amount of traffic, a circuit breaker under heavy load.
• BlockHost: Enforces security by validating the Host header of incoming requests against a configurable whitelist of allowed hostnames. It supports exact matches and wildcard subdomains (e.g., *.example.com).
• BlockRequestBody: Limits the size of incoming request bodies to a configurable maximum. This helps prevent resource exhaustion from excessively large payloads and can be configured to exclude specific URL paths.
• BlockUaList: Filters requests by matching the User-Agent string against a configurable regular expression. This can be used to block scrapers, bots, or other unwanted clients.
• TLSHeaderSTS: Sets the Strict-Transport-Security (HSTS) header for all responses served over a TLS connection, instructing browsers to communicate with the server only over HTTPS.
• Maintenance: When activated via configuration, this middleware puts the server into maintenance mode. It responds to all requests with a 503 Service Unavailable status code, allowing for system updates without shutting down the server.
• Gzip: Serves pre-compressed static assets (.gz files) from a given file system (fs.FS) to clients that support gzip encoding. This reduces bandwidth and improves load times. If a compressed file is not found, it seamlessly falls back to the next handler.

Examples

Detailed examples and integration guides are available to help you build with the framework. You can explore a complete JavaScript SDK Integration at restinpieces-js-sdk to see how to connect your frontend, or review implementations of Custom Routers and DB Drivers at restinpieces-non-default for advanced customization scenarios.

Extensibility

Beyond its core features, the framework is designed to be easily extended to meet diverse application needs. It includes a built-in file server with gzip compression for efficient delivery of static assets and a dedicated asset pipeline for minification and bundling of HTML, CSS, and JavaScript, leveraging scripts available at restinpieces-js-sdk/gen.

Layout Best Practices

Applications built on restinpieces follow this structure:

myapp/
├── cmd/myapp/main.go    # entry point: flags, wiring, daemons, jobs, srv.Run()
├── app.go               # your App wrapper — embeds *core.App, adds your state
├── handlers/            # HTTP handlers as methods on *App
├── middleware/          # custom middleware (App-aware via closure, or plain funcs)
├── routes.go            # register framework + application routes
├── jobs/                # job handler implementations
├── daemons/             # daemon constructors
└── web/src/, web/dist/  # frontend assets, embedded via go:embed

App wrapper. Define your own *App struct that embeds *core.App and adds project-specific state (extra DB pools, third-party clients). Handlers are methods on *App — no global variables, trivial to test.

Handlers and middleware. Handlers use standard http.HandlerFunc signatures. Middleware closes over *App when it needs framework services; stateless middleware stays a plain func(http.Handler) http.Handler.

Routes. All route registration lives in routes.go. Call app.Router().Register() with your chains — the framework's built-in routes are registered internally by restinpieces.New(), yours go on top.

Jobs and daemons. Job handlers implement the framework's handler interface; daemons are long-running background processes. Both are constructed with the state they need and registered in main.go via srv.AddJobHandler / srv.AddDaemon.

Configuration. Store secrets encrypted via age in the shared SQLite database. Use your own config scope (never "application", which the framework reserves). Generations are immutable — save always creates a new record, giving you a full audit trail.

For full detail and code examples, see Layout Best Practices.

Building the Project

Build Server

Builds the example server application.

go build -ldflags="-s -w" -trimpath -o restinpieces_server ./cmd/example/

Build CLI

Builds the ripc command-line tool.

go build -ldflags="-s -w" -trimpath -o ripc ./cmd/ripc/

TODO

Todos.