with memory mapping, what bottleneck can occur #65
Answered
by
Eamon2009
eamonsippy
asked this question in
Q&A
|
Lazy OS Loading: Even with memory mapping, what bottleneck can occur when jumping to completely random memory addresses across massive shards during batch generation, and how can an OS-level hint like madvisMADV_SEQUENTIAL) fix it? |
Answered by
Eamon2009
Jul 28, 2026
Replies: 1 comment
|
Because get_batch selects a random start position (pos_dist(thread_rng)), the CPU suddenly requests a memory address that the OS hasn't loaded into RAM yet. This causes a page fault, forcing the thread to freeze and wait for the physical hard drive to fetch the data. Using a memory advice hint (like madvise(MADV_SEQUENTIAL) or madvise(MADV_WILLNEED), though not currently in the code) tells the OS to aggressively pre-load the upcoming pages in the background, eliminating the freeze. |
0 replies
Answer selected by
eamonsippy
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Because get_batch selects a random start position (pos_dist(thread_rng)), the CPU suddenly requests a memory address that the OS hasn't loaded into RAM yet. This causes a page fault, forcing the thread to freeze and wait for the physical hard drive to fetch the data. Using a memory advice hint (like madvise(MADV_SEQUENTIAL) or madvise(MADV_WILLNEED), though not currently in the code) tells the OS to aggressively pre-load the upcoming pages in the background, eliminating the freeze.