Fix spurious empty final batch when ntrain is a multiple of batch_size - #3
Open
rezanematpour wants to merge 2 commits into
Open
Fix spurious empty final batch when ntrain is a multiple of batch_size#3rezanematpour wants to merge 2 commits into
rezanematpour wants to merge 2 commits into
Conversation
nbatches = ntrain//batch_size+1 unconditionally adds an extra batch, which becomes an empty (curr_batch_size=0) batch whenever ntrain is an exact multiple of batch_size. Fixed by only adding the extra batch when a remainder actually exists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Across all training loop functions in
Fin-GAN-online.py(e.g.,TrainLoopForGAN, and the variousTrainLoopMain*variants), batch count is computed as:This unconditionally assumes there is always a remainder batch to add. When
ntrainis an exact multiple ofbatch_size, this creates one extra, spurious batch whose size is computed as:which evaluates to 0 for that final batch. This produces empty-tensor batches (e.g.,
condition/realof shape(0, ...), zero-batch LSTM hidden statesh_0/c_0), which can break the forward pass and downstream indexing such asdisc_fake_pred[0][0][0]used for logging.Example
With
ntrain=1000,batch_size=100:nbatches = 1000//100 + 1 = 11i=0..9correctly cover all 1000 rows (size 100 each)i=10getscurr_batch_size = 1000 - 1000 = 0— an empty batch that shouldn't existFix
Only add the extra batch when a remainder actually exists:
This is a minimal, behavior-preserving change:
ntrain % batch_size != 0(the common case) — same batch count and sizes as before.ntrain % batch_size == 0— no more empty final batch.Applied consistently to all occurrences of this pattern in
Fin-GAN-online.py.Testing
Verified the corrected formula against both cases:
ntrain=1050, batch_size=100→ 11 batches, sizes[100]*10 + [50](unchanged from original behavior)ntrain=1000, batch_size=100→ 10 batches, sizes[100]*10, no empty batch (previously would have produced an 11th, empty batch)No other logic was changed.