Skip to content

#87 Make latency_ms assertion deterministic by mocking perf_counter #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
deyna256 merged 2 commits into from
Mar 3, 2026
Merged

#87 Make latency_ms assertion deterministic by mocking perf_counter #101

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fba3497
Make latency_ms assertion deterministic by mocking perf_counter
sathvik458 Mar 3, 2026
ec1f4b9
Make latency_ms assertion deterministic using mocked perf_counter
sathvik458 Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions tests/providers/test_openrouter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from typing import Any
from unittest.mock import AsyncMock
from unittest.mock import AsyncMock, patch

import httpx
import pytest
Expand Down Expand Up @@ -45,13 +45,22 @@ async def test_generate_success_returns_content_and_metrics(
successful_api_response: dict[str, Any],
no_retry_config: RetryConfig,
):
async with OpenRouterClient(api_key="k", default_retry_config=no_retry_config) as client:
client._client.post = AsyncMock(return_value=_make_response(200, successful_api_response))
async with OpenRouterClient(
api_key="k", default_retry_config=no_retry_config
) as client:
client._client.post = AsyncMock(
return_value=_make_response(200, successful_api_response)
)

content, metrics = await client.generate(prompt="hello", model="test-model")
with patch("promptum.providers.openrouter.time.perf_counter") as mock_perf:
mock_perf.side_effect = [10.0, 10.123]

content, metrics = await client.generate(
prompt="hello", model="test-model"
)

assert content == "Hello, world!"
assert metrics.latency_ms > 0
assert metrics.latency_ms == pytest.approx(123.0, rel=1e-6)
assert metrics.prompt_tokens == 10
assert metrics.completion_tokens == 20
assert metrics.total_tokens == 30
Expand Down