Skip to content

docs: update virtwork-vs-kube-burner comparison with full workload set #181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mrhillsman merged 5 commits into from
Jun 1, 2026
+31 −15
Merged

docs: update virtwork-vs-kube-burner comparison with full workload set #181

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3e42ca7
docs: fix architecture layers in virtwork-vs-kube-burner comparison
mrhillsman Jun 1, 2026
5852d8a
docs: add tps and chaos workloads to comparison workload table
mrhillsman Jun 1, 2026
135cb33
docs: drop stale maturity stats and fix GitHub URL in comparison doc
mrhillsman Jun 1, 2026
1e12fea
docs: update sequence diagram and YAML example with full workload set
mrhillsman Jun 1, 2026
0bc4bbe
docs: add chaos and tps workloads to decision guide and partner valid…
mrhillsman Jun 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 31 additions & 15 deletions docs/virtwork-vs-kube-burner.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,18 +66,22 @@ Workloads deployed inside VMs:
| `memory` | `stress-ng --vm 1 --vm-bytes 80%` | Sustained memory pressure at 80% |
| `database` | PostgreSQL + `pgbench -c 10 -j 2 -T 300` | Realistic OLTP database transactions |
| `network` | `iperf3 --bidir` (server + client VM pairs) | Bidirectional throughput between VMs |
| `tps` | `netperf` + `curl` (server + client VM pairs) | Multi-port HTTP throughput with configurable file size, iterations, and duration |
| `disk` | `fio` with mixed random + sequential profiles | Mixed I/O patterns on a dedicated data disk |
| `chaos-disk` | `fallocate`/`dd` fill-release loop | Sustained disk-pressure events on a data disk |
| `chaos-network` | `tc` + `netem` qdisc | Injected latency and packet loss on VM egress |
| `chaos-process` | shell + `ps`/`kill` | Random process termination inside the VM |

All workloads run as **systemd services** — they survive VM reboots and auto-restart on failure. They produce realistic CPU, memory, database, network, and disk I/O signals for monitoring systems to observe and validate.
All workloads run as **systemd services** — they survive VM reboots and auto-restart on failure. They produce realistic CPU, memory, database, network, and disk I/O signals for monitoring systems to observe and validate. The three chaos workloads extend this by injecting failures — disk pressure, network degradation, process kills — that exercise a partner product's alerting, recovery, and resilience handling.

#### virtwork layered architecture

```mermaid
graph TD
L4["Layer 4 — Orchestration<br/><code>cmd/virtwork · cleanup · audit</code>"]
L3["Layer 3 — Workload Definitions<br/><code>interface · cpu · memory · database · network · disk · registry</code>"]
L4["Layer 4 — Orchestration<br/><code>cmd/virtwork · orchestrator · cleanup</code>"]
L3["Layer 3 — Workload Definitions<br/><code>interface · cpu · memory · disk · database · network · tps · chaos-* · registry</code>"]
L2["Layer 2 — K8s Abstractions<br/><code>vm · resources · wait</code>"]
L1["Layer 1 — Infrastructure<br/><code>config · cluster · cloudinit</code>"]
L1["Layer 1 — Infrastructure<br/><code>config · cluster · cloudinit · logging · audit</code>"]
L0["Layer 0 — Definitions<br/><code>constants</code>"]

L4 --> L3 --> L2 --> L1 --> L0
Expand Down Expand Up @@ -115,7 +119,7 @@ Strict layering: no cross-layer dependencies. Each layer only depends on the lay
| **Monitoring role** | Scrapes cluster metrics, indexes to ES/OpenSearch/local | Produces metrics *for* external monitoring tools to scrape |
| **SSH access** | Via `virtctl ssh` in virt-density tests | First-class: `--ssh-user`, `--ssh-key-file`; `virtctl ssh` or port-forward |
| **Deployment** | Binary or container, runs externally to cluster | Binary or Kustomize-deployed pod running in-cluster |
| **Maturity** | Active, community-maintained, CNCF-adjacent | Beta (0 stars, 58 commits, 2 contributors as of Feb 2026) |
| **Maturity** | Active, community-maintained, CNCF-adjacent | Active development |

---

Expand Down Expand Up @@ -171,9 +175,10 @@ flowchart TD

Q2 -->|Storage — disk I/O| B1["virtwork<br/><b>--workloads disk</b>"]
Q2 -->|Monitoring — CPU/memory metrics| B2["virtwork<br/><b>--workloads cpu,memory</b>"]
Q2 -->|Network — VM-to-VM throughput| B3["virtwork<br/><b>--workloads network</b>"]
Q2 -->|Network — VM-to-VM throughput| B3["virtwork<br/><b>--workloads network,tps</b>"]
Q2 -->|Database — OLTP load| B4["virtwork<br/><b>--workloads database</b>"]
Q2 -->|All of the above on OPL bare metal| B5["virtwork<br/><b>all workloads enabled by default</b>"]
Q2 -->|Chaos / resilience testing| B5["virtwork<br/><b>--workloads chaos-disk,chaos-network,chaos-process</b>"]
Q2 -->|All of the above on OPL bare metal| B6["virtwork<br/><b>all nine workloads enabled by default</b>"]

style A1 fill:#1a3a5c,stroke:#4a9eda,color:#eee
style A2 fill:#1a3a5c,stroke:#4a9eda,color:#eee
Expand All @@ -185,6 +190,7 @@ flowchart TD
style B3 fill:#3a1a1a,stroke:#e94560,color:#eee
style B4 fill:#3a1a1a,stroke:#e94560,color:#eee
style B5 fill:#3a1a1a,stroke:#e94560,color:#eee
style B6 fill:#3a1a1a,stroke:#e94560,color:#eee
style Start fill:#2a2a2a,stroke:#aaa,color:#fff
style Q1 fill:#1e1e1e,stroke:#888,color:#eee
style Q2 fill:#1e1e1e,stroke:#888,color:#eee
Expand Down Expand Up @@ -218,9 +224,9 @@ In the OpenShift Partner Labs (OPL) context, a **partner product** is an ISV's t

| Partner Product Type | What "handles" actually means |
|---|---|
| **Storage** | Does your storage driver correctly serve VM disk I/O? Does it report IOPS and throughput accurately? Does it stay stable under `fio` mixed read/write load? |
| **Monitoring** | Does your monitoring agent correctly scrape CPU, memory, and disk metrics from inside VMs? Do your dashboards reflect what `stress-ng` and `fio` are actually doing? |
| **Network** | Does your CNI or network product correctly route `iperf3 --bidir` traffic between VMs? Does it report the right throughput? Does it hold up under sustained bidirectional load? |
| **Storage** | Does your storage driver correctly serve VM disk I/O? Does it report IOPS and throughput accurately? Does it stay stable under `fio` mixed read/write load? Does it alert correctly when `chaos-disk` fills the volume to 90%? |
| **Monitoring** | Does your monitoring agent correctly scrape CPU, memory, and disk metrics from inside VMs? Do your dashboards reflect what `stress-ng` and `fio` are actually doing? Do your alerts fire when `chaos-process` kills monitored processes? |
| **Network** | Does your CNI or network product correctly route `iperf3 --bidir` and `netperf`/`curl` traffic between VMs? Does it report the right throughput? Does it detect the latency and packet loss injected by `chaos-network`? |
| **Database** | Does your product correctly observe or manage a PostgreSQL instance under `pgbench` OLTP load running inside a VM? |

virtwork exists specifically to generate the signals each of these product categories needs to prove the answer is yes — on OpenShift Partner Labs (OPL) bare metal, against real KubeVirt VMs, with workloads that outlast the tool that created them.
Expand All @@ -241,7 +247,7 @@ sequenceDiagram
Eng->>KB: virt-density run
KB-->>Eng: ✓ Boot latency acceptable<br/>✓ Cluster handles VM density

Eng->>VW: virtwork run --workloads cpu,memory,database,network,disk
Eng->>VW: virtwork run
VW-->>Eng: ✓ VMs created<br/>✓ systemd services started<br/>✓ Run ID logged to SQLite
Note over VW: virtwork exits — systemd<br/>keeps workloads running

Expand Down Expand Up @@ -298,15 +304,25 @@ workloads:
vm_count: 2
cpu_cores: 4
memory: 4Gi
memory:
enabled: true
disk:
enabled: true
database:
enabled: true
cpu_cores: 2
memory: 4Gi
network:
enabled: true # creates N×2 VMs: server + client pairs
disk:
enabled: true # creates server + client VM pairs
tps:
enabled: true
memory:
params:
file-size: "50M"
chaos-disk:
enabled: true
chaos-network:
enabled: true
chaos-process:
enabled: true
```

Expand All @@ -332,7 +348,7 @@ Used together, the two tools cover the full picture: kube-burner establishes tha
- [kube-burner Configuration Reference — KubeVirt](https://kube-burner.github.io/kube-burner/latest/reference/configuration/#kubevirt)
- [kube-burner-ocp GitHub](https://github.com/kube-burner/kube-burner-ocp)
- [kube-burner-ocp Virt Workloads Docs](https://kube-burner.github.io/kube-burner-ocp/latest/)
- [virtwork GitHub](https://github.com/redhat-openshift-partner-labs/virtwork)
- [virtwork GitHub](https://github.com/opdev/virtwork)
- [OpenShift Partner Lab Overview](https://connect.redhat.com/en/blog/the-openshift-partner-lab)
- [Red Hat Virt Density Blog — kube-burner usage](https://developers.redhat.com/blog/2025/11/17/high-scale-performance-testing-virt-density)
- [Use kube-burner to measure OpenShift VM and storage deployment at scale](https://developers.redhat.com/articles/2024/09/04/use-kube-burner-measure-red-hat-openshift-vm-and-storage-deployment-scale)
Loading