Guard against non-positive num_tokens_per_epoch in GPTDataset

[Yifei-Zuo]

What

Fix a possible infinite loop in GPTDataset._get_num_epochs when a data split
is allocated zero tokens.

Why

_get_num_epochs accumulated num_tokens one epoch at a time in a while
loop until it reached the requested token count:

num_epochs = 1
num_tokens = num_tokens_per_epoch
...
while num_tokens < num_tokens_requested:
    num_epochs += 1
    num_tokens += num_tokens_per_epoch

When num_tokens_per_epoch == 0 — which happens when the train/valid/test
split assigns too small a fraction to a split for the dataset size (e.g.
--split 9998,0.002,0.002 on a small corpus) — num_tokens never grows and
the loop spins forever with no diagnostic. This was reported in #957 while
training Mixtral-MoE 8x7B.

What changed

• Raise a RuntimeError when num_tokens_per_epoch <= 0. The message names the
  affected split and explains how to fix the configuration, instead of hanging
  silently.
• Replace the accumulation loop with an equivalent integer ceiling division
  (max(1, -(-num_tokens_requested // num_tokens_per_epoch))). Integer math is
  used rather than math.ceil(a / b) to avoid floating-point drift at exact
  boundaries for large token counts. Verified to return identical values to the
  old loop across 200k randomized cases.

Fixes #957

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.