Skip to content

fix: speed up test_3136_cuda_reducers.py - #3934

Open
ianna wants to merge 9 commits into
scikit-hep:mainfrom
ianna:ianna/cleanup_test_3136_cuda_reducers
Open

fix: speed up test_3136_cuda_reducers.py#3934
ianna wants to merge 9 commits into
scikit-hep:mainfrom
ianna:ianna/cleanup_test_3136_cuda_reducers

Conversation

@ianna

@ianna ianna commented Mar 25, 2026

Copy link
Copy Markdown
Member

Re-write the tests in a GPU friendly way.

@maxymnaumchyk - FYI

@ianna
ianna requested a review from ariostas March 25, 2026 23:04

@ianna ianna left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ariostas and @maxymnaumchyk - this should speedup the tests a bit.

@github-actions

Copy link
Copy Markdown

The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR3934

@ikrommyd

Copy link
Copy Markdown
Member

It's the tests-cuda-kernels-explicit tests that are slow https://github.com/scikit-hep/awkward/actions/runs/23568619808/job/68625991834?pr=3934. python -m pytest -vv -rs tests-cuda-kernels-explicit took 27 minutes here

@maxymnaumchyk

Copy link
Copy Markdown
Collaborator

yeah, the tests-cuda-kernels-explicit can run the same kernel dozens of times, with different input data every time.

@codecov

codecov Bot commented Mar 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.27%. Comparing base (85c3954) to head (9293d84).

Additional details and impacted files

see 8 files with indirect coverage changes

@ianna

ianna commented Apr 15, 2026

Copy link
Copy Markdown
Member Author

Hmm... there is a discrepancy between CPU and GPU reducers -- the former return a NumPy array while the latter a NumpyArray layout:

prod_expected = array([0, 1, 0, 0])
prod_gpu   = <NumpyArray dtype='int64' len='4'>[0 1 0 0]</NumpyArray>
sum_expected = array([1, 0, 1, 0])
sum_gpu    = <NumpyArray dtype='int64' len='4'>[1 0 1 0]</NumpyArray>

In 2.9.0 I get a high level Array:

>>> import numpy as np
>>> array = np.array([[True, False, False], [True, False, False]])
>>> content = ak.contents.NumpyArray(array.reshape(-1))
>>> offsets = ak.index.Index64(np.array([0, 3, 3, 5, 6], dtype=np.int64))
>>> depth = ak.contents.ListOffsetArray(offsets, content)
>>> depth
<ListOffsetArray len='4'>
    <offsets><Index dtype='int64' len='5'>[0 3 3 5 6]</Index></offsets>
    <content><NumpyArray dtype='bool' len='6'>
        [ True False False  True False False]
    </NumpyArray></content>
</ListOffsetArray>
>>> ak.sum(depth, axis=-1)
<Array [1, 0, 1, 0] type='4 * int64'>
>>> ak.prod(depth, axis=-1)
<Array [0, 1, 0, 0] type='4 * int64'>

@maxymnaumchyk

Copy link
Copy Markdown
Collaborator

It looks like in the tests here, GPU kernels are called with highlevel=False and CPU kernels without that. Could that be the difference? Or do you mean there is a difference somewhere else?

sum_gpu = ak.sum(depth_gpu, axis=-1, highlevel=False)
prod_gpu = ak.prod(depth_gpu, axis=-1, highlevel=False)

sum_expected = cp.asarray(ak.sum(depth, axis=-1))
prod_expected = cp.asarray(ak.prod(depth, axis=-1))

Comment thread tests-cuda/test_3136_cuda_reducers.py Outdated
Comment thread tests-cuda/test_3136_cuda_reducers.py Outdated
@TaiSakuma TaiSakuma added the type/fix PR title type: fix (set automatically) label Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/fix PR title type: fix (set automatically)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants