GitHub - wlm64/KVOffload_benchmarking

Documentation can be found here:

Name		Name	Last commit message	Last commit date
Latest commit History 21 Commits
README.md		README.md
bench_KVAD.sh		bench_KVAD.sh
bench_no_offload_serve.sh		bench_no_offload_serve.sh
bench_offload_serve.sh		bench_offload_serve.sh
bench_serve.sh		bench_serve.sh
create_offload_datasets.py		create_offload_datasets.py
create_synthetic_dataset.py		create_synthetic_dataset.py
docfinqa_token_counts_all_splits.csv		docfinqa_token_counts_all_splits.csv
example.yaml		example.yaml
hardware_comparison.py		hardware_comparison.py
narrativeqa_token_counts_all_splits.csv		narrativeqa_token_counts_all_splits.csv
optimized_dissag_sim.py		optimized_dissag_sim.py
pareto_dissage_sim.py		pareto_dissage_sim.py
plot_kv_offload_serve.py		plot_kv_offload_serve.py
plot_relative_ttft.py		plot_relative_ttft.py
process_KVAD_logs.py		process_KVAD_logs.py
process_synthetic_logs.py		process_synthetic_logs.py
server_commands.txt		server_commands.txt
sharegpt_effective_prefill.csv		sharegpt_effective_prefill.csv
synthetic_prompts.zip		synthetic_prompts.zip

Provide feedback