Hyper-RSS - ⚠️ WARNING 🚧 under construction 👷

Update

The initial goals for Hyper-RSS were completed. It provides basic tooling for creating, seeding, aggregating, and viewing (with audio and video!) HRSS feeds. However, this work highlighted a significant limitation of the Node.js Hypercore implementation: the tooling can only operate within Node.js or similar JavaScript runtime. This limitation severely restricts Hyper-RSS's potential impact.

For any peer-to-peer protocol to be effective, it needs a diverse and large network of peers. Being confined to Node.js means we're limiting potential peers to users of Node.js applications only. We can't ask existing RSS readers, torrent clients, or websites like The Pirate Bay to integrate Hyper-RSS because most aren't built with Node.js.

To address this, I've been working on the datrs project, a Rust implementation of Hypercore stack. When completed, this implementation can be used via FFI (Foreign Function Interface) in any environment that supports C libraries - which is virtually everywhere. You can follow and support this work here.

Description

Peer-to-peer RSS built on Hypercore. This repo provides:

The RSS data structure lives in peer/.

Data Structure

A hyper-rss "feed" is 4 Hypercores (append-only logs) grouped under one Corestore, each wrapping a higher-level data type from the Hypercore stack:

Core name	Wrapper	Purpose
hrss-keys	raw Hypercore (JSON encoding)	bootstrap/discovery indirection
hrss-feed	OrderedHyperbee (custom Hyperbee subclass)	ordered RSS items + feed metadata
hrss-blobKeys	Hyperbee	maps content-hash key → blob id
hrss-blob	Hyperblobs	actual binary blob storage (images, audio/video enclosures)

• keys holds one append at index 0 with the public keys of the other three cores. Its discovery key is the one stable identifier a Reader needs to find a Writer on the swarm; the writer's actual content keys are only revealed after connecting and reading this first block.
• feed (OrderedHyperbee) namespaces one B-tree into key (item hash → JSON item), order (zero-padded insertion index → item key, since a Hyperbee orders by key bytes, not insertion time), and metadata (feed-level fields, written via compare-and-swap).
• blobKeys + blobs (KeyedBlobs) together give a content-addressed blob store: blobKeys (Hyperbee) maps a content key to a blobs (Hyperblobs) id, which locates the actual bytes.

See peer/README.md for the full breakdown, including item ingestion and peer roles.

Goals

• Create a hrss peer for reading writing, available as a library (peer/)
• Create a simple feed reader/aggregator app (aggregator/)
• Be able to easily create a hrss feed from a regular rss/atom feed.
• Updating an hrss feed from a regular rss/atom feed should be easy.
• Standardized format of hrss
• Standardize formats for podcasts and TV shows on top of stand hrss format

Usage

Start the aggregator backend with:

cd aggregator && yarn server

Start the web frontend, which needs aggregator:

cd web && yarn dev