Motivation
In game-fuzz, we record gameplay as MP4 at a fixed FPS, then pass the file to
get_frame_cov() which decodes it back into frames for perceptual hashing. This is
wasteful in two ways:
- Encode-then-decode roundtrip — we capture screen frames, encode them into MP4,
then gamecov decodes them right back into frames.
- No control over which frames are hashed — the caller has no way to pass
semantically meaningful frames (e.g., captured at action boundaries or during key
game state transitions). All frames at the recording FPS are processed.
Games are continuous simulations — unlike browser/desktop automation, the game world
evolves between player inputs. An ideal sampling strategy would capture frames both at
action boundaries and periodically during idle/wait periods. This requires the ability
to pass pre-captured images directly.
Proposed API
Add a function that accepts a sequence of images instead of an MP4 path:
from gamecov import get_frame_cov_from_images, FrameCoverage
from PIL import Image
from returns.result import Result
def get_frame_cov_from_images(
images: list[Image.Image], # or list[np.ndarray]
) -> Result[FrameCoverage, str]:
"""Compute frame coverage from a list of pre-captured images."""
...
This would complement the existing get_frame_cov(path: str) which remains useful when
an MP4 is already available.
Use Case
# In the fuzzer's action execution loop:
screenshots: list[Image] = []
for action in seed.actions:
execute(action)
screenshots.append(capture_screen(region))
# Compute frame coverage directly from screenshots
frame_cov = get_frame_cov_from_images(screenshots)
Benefits
- Eliminates MP4 encode/decode overhead for coverage computation
- Enables action-boundary and event-driven frame sampling strategies
- Maintains backward compatibility (existing MP4 API unchanged)
Motivation
In game-fuzz, we record gameplay as MP4 at a fixed FPS, then pass the file to
get_frame_cov()which decodes it back into frames for perceptual hashing. This iswasteful in two ways:
then gamecov decodes them right back into frames.
semantically meaningful frames (e.g., captured at action boundaries or during key
game state transitions). All frames at the recording FPS are processed.
Games are continuous simulations — unlike browser/desktop automation, the game world
evolves between player inputs. An ideal sampling strategy would capture frames both at
action boundaries and periodically during idle/wait periods. This requires the ability
to pass pre-captured images directly.
Proposed API
Add a function that accepts a sequence of images instead of an MP4 path:
This would complement the existing
get_frame_cov(path: str)which remains useful whenan MP4 is already available.
Use Case
Benefits