Benchmark test nixlbench/kvcache#10
Conversation
Signed-off-by: Jeeja <jeejakp@habana.ai>
|
👋 Hi kpjeeja! Thank you for contributing to intel-staging/nixl. Your PR reviewers will review your contribution then trigger the CI to test your changes. 🚀 |
|
|
||
| matrix: np.ndarray | ||
| mem_type: Literal["cuda", "vram", "cpu", "dram"] | ||
| mem_type: Literal["cuda", "vram", "cpu", "dram","hpu"] |
There was a problem hiding this comment.
are we setting mem_type as hpu in any test/usecase ?
There was a problem hiding this comment.
not sure. Currently it includes both cuda/cpu, so added hpu. KVBench i can't run it as it requires model support
There was a problem hiding this comment.
the test uses the mem_type == device in many places, so i will keep the hpu in mem_type similar to cuda
| ) | ||
|
|
||
| if ( | ||
| not os.environ.get("HABANA_VISIBLE_MODULES") |
There was a problem hiding this comment.
we need to check for devices too as its an exported variable
There was a problem hiding this comment.
next line has a device check - self.traffic_pattern.mem_type == "hpu"
| if synapse_lib.found() and hlthunk_lib.found() | ||
| synapseai_dep = declare_dependency(dependencies: [synapse_lib, hlthunk_lib]) | ||
| elif hlthunk_lib.found() | ||
| # Fallback to just hl-thunk if libSynapse not available |
There was a problem hiding this comment.
i have adopted it from the nixl meason file.
| openmp_dep = dependency('openmp', required: true) | ||
|
|
||
| # Check for etcd-cpp-api - use multiple methods for discovery | ||
| # Try pkg-config first |
There was a problem hiding this comment.
etcd it was not finding the path . so i have to add it to search in the default path
| # Check for etcd-cpp-api - use multiple methods for discovery | ||
| # Try pkg-config first | ||
| etcd_dep = dependency('etcd-cpp-api', required : false) | ||
| if not etcd_dep.found() |
There was a problem hiding this comment.
yes, otherwise it was never finding the etcd lib.
| if not etcd_available | ||
| error('No runtime available or not found') | ||
| endif | ||
| #if not etcd_available |
| static synDeviceId deviceHandle; | ||
| static synStreamHandle stream; | ||
|
|
||
| namespace Synapseaiutils { |
There was a problem hiding this comment.
this will need some clean. will move it as part of another JIRA
| } | ||
|
|
||
| int | ||
| init_synapse_device() { |
There was a problem hiding this comment.
Might be easy to upstream if its class based/ cpp
There was a problem hiding this comment.
will take it as a cleanup activity
| utils_dep = declare_dependency( | ||
| link_with: utils_lib, | ||
| dependencies: utils_deps, | ||
| dependencies: [synapseaiutils_deps, utils_deps], |
There was a problem hiding this comment.
if HAVE_SYNAPSEAI is undefined ? wont this cause a problem?
| nixl_worker_sources, | ||
| include_directories: inc_dir, | ||
| dependencies: nixl_worker_deps, | ||
| dependencies: [synapseaiutils_deps, nixl_worker_deps], |
| getVramDescSynapseai(int devid, size_t buffer_size, uint8_t memset_value) { | ||
| void *host_addr = calloc(1, buffer_size); | ||
| memset(host_addr, memset_value, buffer_size); | ||
| auto device_buffer = Synapseaiutils::allocate_synapse_memory(buffer_size, host_addr); |
There was a problem hiding this comment.
it already taken care in the allocation part. i will cleanup with the directory change
|
|
||
| [tool.meson-python.args] | ||
| setup = ['-Dinstall_headers=false'] | ||
| setup = ["-Dinstall_headers=false"] |
jeromean
left a comment
jeromean
left a comment
There was a problem hiding this comment.
Some cosmetic, err checks, code reorg needed.
3 participants
Enable KVCahe/ Nixl Benchmark test to support Intel Gaudi