QSOStream is a long-running daemon that tails a live ADIF log file — such as the one produced by WSJT-X — and automatically uploads every new QSO to your QRZ Logbook in real time. It is designed to be left running in the background alongside your logging software and requires no manual intervention once started.
- Features
- Requirements
- Installation
- Configuration
- Usage
- How It Works
- ADIF Field Reference
- Deduplication & Fingerprinting
- Persistence & State Files
- File Rotation & Truncation
- Live Metrics Dashboard
- Running the Tests
- Project Layout
- Troubleshooting
- Incremental file tailing — reads only new bytes on each poll cycle; the byte position is persisted so progress survives restarts.
- Effectively-once uploads — a local SQLite dedupe database prevents double-uploading even if the daemon is restarted mid-run or if QRZ itself returns a duplicate response.
- Automatic retries — each failed upload is retried up to 2 additional times (3 attempts total) before being written to a persistent failure log.
- Failure log — records that fail all attempts are appended to a JSONL file and can be replayed later with the
retry-failurescommand. - Live metrics dashboard — the terminal is refreshed every few seconds with a running tally of QSOs processed, uploaded, deduplicated, and failed.
- File rotation & truncation detection — detects when the log file is replaced or cleared and resets the read pointer automatically, relying on dedupe to avoid re-uploads.
- Zero-configuration deduplication — no manual setup required; the SQLite state database is created automatically on first run.
| Component | Minimum version |
|---|---|
| Python | 3.11 |
| qrz-logbook-client | latest |
| A QRZ Logbook API token | — |
QSOStream does not require any external database server or background service. Everything is stored in ordinary files on disk.
It is strongly recommended to install inside a virtual environment.
# 1. Create and activate a virtual environment
python3 -m venv .venv
source .venv/bin/activate # macOS / Linux
# .venv\Scripts\activate # Windows
# 2. Install QSOStream and its dependencies
pip install -e .After installation the qsostream command is available in your virtual environment. You can also invoke the package directly:
python -m qsostream <command> [options]QSOStream authenticates with the QRZ Logbook API using a personal API token.
Recommended: export it as an environment variable before starting the daemon so you are never prompted:
export QRZ_API_TOKEN="your-qrz-api-token-here"If the variable is not set, QSOStream will prompt you to enter the token interactively at startup (input is hidden). The token is never written to disk.
You can obtain your API token from your QRZ account settings.
QSOStream identifies itself to the QRZ API with a User-Agent header in the format:
QSOStream (YOURCALLSIGN)
The callsign is always taken from the --callsign argument you provide on the command line.
Start the daemon and keep it running while you operate:
qsostream watch \
--adi-path /path/to/wsjtx_log.adi \
--callsign KE9AKI| Flag | Default | Description |
|---|---|---|
--adi-path |
(required) | Absolute or relative path to the ADIF .adi log file to monitor. |
--callsign |
(required) | Your station callsign. Used in the QRZ User-Agent header. |
--state-path |
state/qso_uploader.db |
Path to the SQLite state database. Created automatically if it does not exist. |
--failure-path |
state/failed_qsos.jsonl |
Path to the append-only JSONL failure log. |
--poll-interval |
1.0 |
Seconds between file size checks. |
--metrics-interval |
3.0 |
Seconds between live dashboard refreshes. |
--retry-delay |
15.0 |
Seconds to wait between retry attempts on a failed upload. |
Stop the daemon at any time with Ctrl+C. The terminal will display a final metrics snapshot before exiting. All state is persisted — the daemon will resume from where it left off on the next run.
qsostream watch \
--adi-path ~/Documents/WSJT-X/wsjtx_log.adi \
--callsign KE9AKI \
--state-path ~/.local/share/qsostream/state.db \
--failure-path ~/.local/share/qsostream/failures.jsonl \
--poll-interval 2.0 \
--metrics-interval 5.0 \
--retry-delay 30.0After the watch daemon has been running, any QSOs that exhausted all upload attempts are stored in the failure log. Use this command to attempt to upload them again:
qsostream retry-failures --callsign KE9AKI| Flag | Default | Description |
|---|---|---|
--callsign |
(required) | Your station callsign. |
--state-path |
state/qso_uploader.db |
Path to the SQLite state database. |
--failure-path |
state/failed_qsos.jsonl |
Path to the JSONL failure log to read from. |
--retry-delay |
15.0 |
Seconds to wait between retry attempts. |
When the command completes, the failure log is atomically rewritten (compacted) to contain only the entries that still failed. Successfully recovered records are removed. A summary is printed:
Retry complete: retried=5, recovered=4, remaining=1
ADIF log file ──► ADIStreamParser ──► normalize_identity() ──► SQLite dedupe check
│
already seen ◄───┤
│ new QSO
▼
QRZ Logbook API
(up to 3 attempts)
│ │
ok │ │ all failed
▼ ▼
mark_uploaded append to
in SQLite DB failures.jsonl
- Polling loop — every
--poll-intervalseconds, QSOStream checks the file size. If new bytes are available, they are read from the last known byte offset. - Incremental ADIF parsing — the
ADIStreamParsermaintains a rolling buffer across chunks. Records are only emitted when a complete<eor>delimiter is found, so a partial record at the end of a read is safely held until the next poll. - Normalisation — before any dedupe or upload, each record is normalised: callsigns are uppercased,
TIME_ONinHHMMformat is padded toHHMMSS00,SUBMODEtakes precedence overMODE, andFREQis rounded to six decimal places. - Dedupe check — a SHA-256 fingerprint is computed from the canonical fields and looked up in SQLite. If found, the record is silently skipped.
- Upload & retry — the raw ADIF record is submitted to QRZ. If the API returns an error, QSOStream waits
--retry-delayseconds and retries up to 2 more times. A duplicate response from QRZ is treated as success and the record is persisted in the dedupe database. - State persistence — after each poll cycle, the current byte offset, parser buffer, and file identity (inode + device) are written back to SQLite so the daemon can resume cleanly after a restart.
QSOStream reads standard ADIF .adi files. The following fields are required in each QSO record for the upload to proceed:
| Field | Description |
|---|---|
STATION_CALLSIGN |
Your transmitting callsign (e.g. KE9AKI). |
CALL |
The callsign of the contacted station. Slash calls (e.g. KQ4AYK/AG) are preserved exactly. |
QSO_DATE |
Date of the QSO in YYYYMMDD format. |
TIME_ON |
UTC start time in HHMMSS or HHMM format. WSJT-X uses HHMMSS. |
BAND |
Amateur band (e.g. 20m, 40m). |
MODE or SUBMODE |
Operating mode. If both are present, SUBMODE is used for deduplication. |
The following fields are optional but, if present, are normalised and included in the deduplication fingerprint:
| Field | Description |
|---|---|
FREQ |
Dial frequency in MHz. Normalised to six decimal places (e.g. 14.074000). |
All other fields present in the record are forwarded to QRZ unchanged.
Each QSO is identified by a canonical fingerprint that is a SHA-256 hash of the following pipe-delimited string:
STATION_CALLSIGN|CALL|QSO_DATE|TIME_ON|BAND|MODE_VALUE|FREQ
Where:
MODE_VALUEisSUBMODEif present, otherwiseMODE.FREQis the six-decimal-place normalised frequency, or an empty string if absent.- All string fields are uppercased and trimmed before hashing.
The fingerprint is stored in the uploaded_qsos table of the SQLite database the first time a QSO is successfully uploaded (or confirmed as a duplicate by QRZ). On every subsequent encounter — including across restarts — the record is immediately skipped without making a network request.
By default, QSOStream stores all runtime state in the state/ directory relative to your working directory:
state/
├── qso_uploader.db # SQLite database (WAL mode)
└── failed_qsos.jsonl # Append-only failure log
Both paths are configurable via --state-path and --failure-path.
Contains two tables:
| Table | Purpose |
|---|---|
uploaded_qsos |
Dedupe store — one row per successfully uploaded QSO fingerprint. |
runtime_state |
Key-value store for the current file path, byte offset, and parser buffer. |
The database is opened in WAL mode (PRAGMA journal_mode=WAL) to minimise lock contention and ensure writes are crash-safe.
Each line is a JSON object with the following structure:
{
"timestamp": "2026-03-22T18:30:15.123456+00:00",
"reason": "upload_error: <error description>",
"raw_adif": "<call:5>K1ABC<qso_date:8>20260322...<eor>",
"fields": { "CALL": "K1ABC", "QSO_DATE": "20260322", ... }
}The file is never cleared by the watch daemon — it only ever appends. The retry-failures command rewrites (compacts) it, keeping only entries that still failed.
QSOStream handles two scenarios where the log file changes underneath it:
- Rotation — the file is deleted and recreated (or a new file is moved into place). Detected by a change in the filesystem device number or inode. QSOStream resets its read position to byte 0.
- Truncation — the file's current size is smaller than the last known read position. QSOStream resets its read position to byte 0.
In both cases, deduplication prevents any already-uploaded QSO from being re-uploaded.
While the watch command is running, the terminal is cleared and redrawn every --metrics-interval seconds (default: 3 seconds):
QSOStream Live Metrics
======================
QSOs processed: 142
Uploaded: 138
Skipped (duplicates): 3
Failed: 1
Recent events:
Uploaded: K1ABC
Uploaded: W4XYZ
Skipped duplicate: KQ4AYK/AG
Failed: VE3ABC — upload_error: timeout
Uploaded: N5QQ
The last 10 events are shown. Pressing Ctrl+C prints a final snapshot before exiting.
# Activate your virtual environment first
source .venv/bin/activate
# Run all tests
python -m pytest
# Run with verbose output
python -m pytest -vThe test suite covers ADIF parsing, field normalisation, deduplication fingerprinting, and failure-log retry logic.
ft8-uploader/
├── qsostream/
│ ├── __init__.py # Package version
│ ├── __main__.py # Enables `python -m qsostream`
│ ├── adif.py # Incremental ADIF stream parser
│ ├── cli.py # Argument parsing, watch loop, retry command, metrics
│ ├── normalize.py # Field normalisation and canonical identity / fingerprinting
│ ├── state.py # SQLite-backed state and dedupe store
│ └── uploader.py # QRZ API client wrapper
├── tests/
│ ├── test_adif.py
│ ├── test_normalize.py
│ ├── test_retry_failures.py
│ └── test_state.py
├── pyproject.toml
└── README.md
QRZ API health check failed
The API token is invalid or the QRZ service is unreachable. Verify your token at qrz.com and check your internet connection.
Missing required field: STATION_CALLSIGN
Your logging software is not writing the STATION_CALLSIGN field to the ADIF file. In WSJT-X, ensure your callsign is entered under File → Settings → Station. QSOStream cannot proceed without it.
Records are silently skipped Check the "Skipped (duplicates)" counter in the live dashboard. If it is unexpectedly high, QSOStream may have already uploaded those records in a prior session. This is normal and expected behaviour.
The daemon restarts from the beginning of the file
This happens if the state database is missing, or if the --state-path points to a different file than the one used in the previous session. Make sure you are using the same --state-path value every time.
state/ directory not found
QSOStream creates the state/ directory (and any parent directories specified by --state-path or --failure-path) automatically on first run.
I moved the .adi file
Update --adi-path to the new location. The daemon will detect a new inode and reset its read position; deduplication will prevent re-uploading records it has already seen.