Clarification Needed: How to Run KV Cache Workload Across Multiple Nodes

[ekaynar]

I’m trying to understand how to run the KV cache workload in a multi-node setup, but the README does not provide guidance.
Could you please clarify:

• How should KV cache work across multiple nodes?
• What are the rules for closed submissions?

It would be very helpful if you could include a few example run commands for:

• KV cache workload (single-node)
• KV cache workload (multi-node, if applicable)

Thanks!

[dslik]

The mlpstorage program automatically allows you to run multiple instances of the KV cache workload on one or more client systems. The README is in the process of being updated to include details on how to do this. Check back very soon!

[hazemawadalla]

Hey Ughur,
David is handling the mlpsstorage integration but check these:

https://github.com/mlcommons/storage/blob/main/kv_cache_benchmark/docs/MLperf_v3_KV_cache_proposal.md#multi-client-scaling-bypassing-python-gil

Multi-node kv sharing is slated for future development https://github.com/mlcommons/storage/blob/main/kv_cache_benchmark/docs/MLperf_v3_KV_cache_proposal.md#115-distributed-kv-cache-with-redis--valkey

[dslik]

Take a look at https://github.com/dslik/storage/tree/main/kv_cache_benchmark and let me know if this is what you are looking for.