Skip to content

Use ZeroGPU for Space inference#1

Open
vvennela wants to merge 1 commit into
mainfrom
codex/modify-repo-to-use-zerogpu-and-push-to-hugging-face
Open

Use ZeroGPU for Space inference#1
vvennela wants to merge 1 commit into
mainfrom
codex/modify-repo-to-use-zerogpu-and-push-to-hugging-face

Conversation

@vvennela

Copy link
Copy Markdown
Owner

Motivation

  • Ensure model loading and inference run inside Hugging Face ZeroGPU allocations on Spaces so GPU work is limited to allocated calls and local/test runs keep working without the Spaces SDK.
  • Provide a small compatibility layer so spaces.GPU can be applied where available while remaining a no-op in local/test environments.

Description

  • Add zerogpu.py, a compatibility helper that applies spaces.GPU when present and provides an identity decorator otherwise.
  • Wrap Diffusers image generation and pipeline loading with the ZeroGPU decorator by updating art.py to use @gpu for DiffusersImageClient.create_art and DiffusersImageClient.load.
  • Wrap Transformers-backed LLM load/complete paths by adding @gpu to Nemotron and MiniCPM load/complete methods in local_llm.py.
  • Add spaces to requirements.txt and update README.md to suggest zero-a10g hardware and document the ZeroGPU usage.

Testing

  • Running pytest -q in the default environment failed during collection due to module import path resolution errors (automated run reported import errors).
  • Running the test suite with the repo on PYTHONPATH via PYTHONPATH=. pytest -q succeeded and reported 258 passed.

Codex Task

@bolt-new-by-stackblitz

Copy link
Copy Markdown

Review PR in StackBlitz Codeflow Run & review this pull request in StackBlitz Codeflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant