Skip to content

Commit 8732ce0

Browse files
Copilotanxiangsir
andcommitted
Add patch_positions example showing tensor structure
Co-authored-by: anxiangsir <31175974+anxiangsir@users.noreply.github.com>
1 parent e20e178 commit 8732ce0

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,10 @@ h_positions = (per // patches_per_side).unsqueeze(0).expand(num_frames, -1).resh
193193
w_positions = (per % patches_per_side).unsqueeze(0).expand(num_frames, -1).reshape(1, -1)
194194
# Stack to create patch_positions: [B, num_frames * frame_tokens, 3]
195195
patch_positions = torch.stack([t_positions, h_positions, w_positions], dim=-1)
196+
# patch_positions example (with 256 tokens per frame, 16x16 patch grid):
197+
# patch_positions[0, 0:4, :] -> [[0, 0, 0], [0, 0, 1], [0, 0, 2], [0, 0, 3]] # Frame 0 (t=0), first 4 patches
198+
# patch_positions[0, 256:260, :] -> [[4, 0, 0], [4, 0, 1], [4, 0, 2], [4, 0, 3]] # Frame 1 (t=4, since 16 frames map to 64 positions)
199+
# Each [t, h, w] represents: t=temporal frame index (0-63), h=row in patch grid, w=column in patch grid
196200

197201
with torch.no_grad():
198202
outputs = model(video, patch_positions=patch_positions)

0 commit comments

Comments
 (0)