Dear LLMServingSim Team,
Recent research highlights the importance of separating prefill and decode (PD) phases in LLM serving. The motivation for this separation is to more efficiently utilize hardware resources by matching the characteristics of each phase to the most suitable hardware. Specifically, the prefill phase is compute-intensive, while the decode phase is memory-heavy. Supporting different hardware for each phase allows users to optimize performance and cost for heterogeneous and disaggregated serving infrastructure.
Currently, llmservingsim2.0 supports configuring prefill and decode instances on different hardware; however, it appears that only the RTXPRO6000 GPU is officially supported at this time. The ability to officially support additional hardware for either the prefill or decode instance (for example, adding support for a memory-optimized or alternative high-performance GPU) would greatly enhance the applicability and research value of LLMServingSim.
Could you please consider adding support for at least one alternative hardware option? Official support from the maintainers would ensure compatibility and encourage broader adoption of PD separation strategies for heterogeneous deployments.
Thank you for your work and consideration!
Best regards
Dear LLMServingSim Team,
Recent research highlights the importance of separating prefill and decode (PD) phases in LLM serving. The motivation for this separation is to more efficiently utilize hardware resources by matching the characteristics of each phase to the most suitable hardware. Specifically, the prefill phase is compute-intensive, while the decode phase is memory-heavy. Supporting different hardware for each phase allows users to optimize performance and cost for heterogeneous and disaggregated serving infrastructure.
Currently, llmservingsim2.0 supports configuring prefill and decode instances on different hardware; however, it appears that only the RTXPRO6000 GPU is officially supported at this time. The ability to officially support additional hardware for either the prefill or decode instance (for example, adding support for a memory-optimized or alternative high-performance GPU) would greatly enhance the applicability and research value of LLMServingSim.
Could you please consider adding support for at least one alternative hardware option? Official support from the maintainers would ensure compatibility and encourage broader adoption of PD separation strategies for heterogeneous deployments.
Thank you for your work and consideration!
Best regards