[NOT-491] Skip live SDK tests without NOTTE_API_KEY#860
Conversation
|
PR author is not in the allowed authors list. |
WalkthroughAdds a Estimated code review effort: 1 (Trivial) | ~5 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/conftest.py`:
- Line 57: Update the NOTTE_API_KEY availability check in the test setup to
treat both missing and empty or whitespace-only values as unavailable, using a
truthiness or stripped-value check so tests skip before making API calls.
- Around line 55-58: Update the require_notte_api_key fixture to load
environment variables from .env before checking NOTTE_API_KEY, using the
existing dotenv utility/import; preserve the current skip behavior when the key
remains unavailable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 4454e2a2-b857-40d0-ab33-17844b595139
📒 Files selected for processing (4)
tests/conftest.pytests/sdk/test_list.pytests/sdk/test_replay_frames.pytests/sdk/test_validator.py
| @pytest.fixture | ||
| def require_notte_api_key(): | ||
| if os.getenv("NOTTE_API_KEY") is None: | ||
| pytest.skip("NOTTE_API_KEY is required for live Notte API tests") |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check whether load_dotenv is already called in conftest.py before the fixture,
# and whether os and dotenv are imported.
# Search for load_dotenv calls in conftest.py
rg -n 'load_dotenv' tests/conftest.py
# Check imports for os and dotenv
rg -n '^import os|^from dotenv|^from notte' tests/conftest.py
# Check if other test files also call load_dotenv inside the test body
rg -n 'load_dotenv' tests/sdk/test_replay_frames.py tests/sdk/test_validator.pyRepository: nottelabs/notte
Length of output: 165
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== tests/conftest.py =="
wc -l tests/conftest.py
cat -n tests/conftest.py | sed -n '1,140p'
echo
echo "== load_dotenv references in tests/ =="
rg -n 'load_dotenv|NOTTE_API_KEY|require_notte_api_key' tests
echo
echo "== files that mention dotenv loading outside tests/ =="
rg -n 'load_dotenv|dotenv' .Repository: nottelabs/notte
Length of output: 21594
Load .env before checking NOTTE_API_KEY
require_notte_api_key runs before any per-test load_dotenv() call, so tests like tests/sdk/test_list.py can be skipped even when the key exists in .env. Call load_dotenv() in the fixture or move dotenv loading to session startup.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/conftest.py` around lines 55 - 58, Update the require_notte_api_key
fixture to load environment variables from .env before checking NOTTE_API_KEY,
using the existing dotenv utility/import; preserve the current skip behavior
when the key remains unavailable.
|
|
||
| @pytest.fixture | ||
| def require_notte_api_key(): | ||
| if os.getenv("NOTTE_API_KEY") is None: |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Consider checking for empty-string API key values.
os.getenv("NOTTE_API_KEY") returns "" (not None) when the variable is set but empty. The test won't skip in that case but will likely fail at the API call. Consider using a truthiness check or strip.
🛡️ Proposed fix
- if os.getenv("NOTTE_API_KEY") is None:
+ if not os.getenv("NOTTE_API_KEY"):
pytest.skip("NOTTE_API_KEY is required for live Notte API tests")📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if os.getenv("NOTTE_API_KEY") is None: | |
| if not os.getenv("NOTTE_API_KEY"): |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/conftest.py` at line 57, Update the NOTTE_API_KEY availability check in
the test setup to treat both missing and empty or whitespace-only values as
unavailable, using a truthiness or stripped-value check so tests skip before
making API calls.
|
2027 auto-runs evals against preview deployments of your docs. To enable this, install one of:
Once a preview is deployed, open a new PR and we'll run the eval automatically. Evaluating agent experience using 2027.dev · View dashboard |
giordano-lucas
changed the title
1 participant
Summary
require_notte_api_keypytest fixture that skips live API tests whenNOTTE_API_KEYis missingNotteClient()and call live Notte API endpoints with that fixtureFixes #859
Test plan
Result without
NOTTE_API_KEYconfigured:Summary by CodeRabbit
Linear: NOT-491 https://linear.app/nottelabsinc/issue/NOT-491/notte-pr-860-skip-live-sdk-tests-without-notte-api-key