Provide mechanism to discard data#695
Open
devreal wants to merge 9 commits into
Open
Conversation
devreal
force-pushed
the
parsec-data-discard
branch
2 times, most recently
from
bbd1448 to
1dd2d54
Compare
therault
reviewed
Nov 14, 2024
bosilca
reviewed
Nov 16, 2024
| item = (parsec_list_item_t*)item->list_next; /* conversion needed for volatile */ | ||
| if( 0 == gpu_copy->readers ) { | ||
| if (cpu_copy->flags & PARSEC_DATA_FLAG_DISCARDED) { | ||
| parsec_list_item_ring_chop((parsec_list_item_t*)gpu_copy); |
Contributor
Author
There was a problem hiding this comment.
Same as above. The flag isn't changed and the device copy maintains a reference on the data_t so if we miss the update of the flags here we will evict into the host copy and then release everything.
devreal
force-pushed
the
parsec-data-discard
branch
2 times, most recently
from
dde7129 to
14934a5
Compare
Add a function `parsec_data_discard` that releases the data such that the host copy remains intact but does not prevent destruction of the data once all device copies have been released. This keeps the host copy available for device copies to inspect and avoids potential race conditions in the release process. During an eviction, copies of data with a discarded host copy are not transfered but put directly into the lru. Signed-off-by: Joseph Schuchart <joseph.schuchart@stonybrook.edu>
Otherwise we cannot destroy empty or discarded data. Signed-off-by: Joseph Schuchart <joseph.schuchart@stonybrook.edu>
Signed-off-by: Joseph Schuchart <joseph.schuchart@stonybrook.edu>
Signed-off-by: Joseph Schuchart <joseph.schuchart@stonybrook.edu>
Also OR the flag instead of assigning it. Signed-off-by: Joseph Schuchart <joseph.schuchart@stonybrook.edu>
Discarded data may never be pushed back so don't warn about it still being owned by the device. Signed-off-by: Joseph Schuchart <joseph.schuchart@stonybrook.edu>
Discarded data sit toward the end of the lru while the data to be evicted is at the front. We walk both forward and backward to collect the discarded data from the back, until we either meet the pivot or we found enough data to evict. If we discarded data we don't evict. Signed-off-by: Joseph Schuchart <joseph.schuchart@stonybrook.edu>
devreal
force-pushed
the
parsec-data-discard
branch
from
397ca9b to
53b7721
Compare
therault
previously approved these changes
Jan 30, 2025
therault
left a comment
therault
left a comment
Contributor
There was a problem hiding this comment.
Discussed during the call, issues raised are addressed and this is performance / feature - critical for MRA, so we should merge.
Contributor
|
will review that it doesn't break dplasma and merge |
bosilca
previously approved these changes
Jan 30, 2025
devreal
force-pushed
the
parsec-data-discard
branch
from
be66039 to
4f6b4c8
Compare
Contributor
Author
|
I modified this PR so that |
We only try to find discarded data if we know that there is discarded data. If no one discarded data (e.g., DPLASMA) we don't go look for it. This is also needed to properly clean up discarded data before releasing the zone allocator. Signed-off-by: Joseph Schuchart <joseph.schuchart@stonybrook.edu>
devreal
force-pushed
the
parsec-data-discard
branch
from
4f6b4c8 to
9c7b42b
Compare
abouteiller
reviewed
Feb 20, 2025
| #endif | ||
| }; | ||
|
|
||
| static inline void release_discarded_data(parsec_device_gpu_module_t *gpu_device, parsec_gpu_data_copy_t* gpu_copy) |
Contributor
There was a problem hiding this comment.
compiler says this function is not used.
Contributor
Author
|
This may not be needed anymore if we get |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a function
parsec_data_discardthat releases the data such that the host copy remains intact but does not prevent destruction of the data once all device copies have been released. This keeps the host copy available for device copies to inspect and avoids potential race conditions in the release process. During an eviction, copies of data with a discarded host copy are not transfered but put directly into the lru.Replaces some duplicated code with a call to
parsec_device_release_gpu_copy.