Skip to content
This repository was archived by the owner on May 17, 2024. It is now read-only.
This repository was archived by the owner on May 17, 2024. It is now read-only.

RAGAS require OPENAI_API_KEY on import #139

Description

@yujonglee
from ragas import evaluate
Run task test:ci
task: [test:ci] poetry run pytest -m "not todo" -n auto --benchmark-disable --cov=fastrepl --cov-report xml:cov.xml --ignore=tests/integration
Traceback (most recent call last):
  File "/home/runner/work/fastrepl/fastrepl/.venv/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 192, in console_main
    code = main()
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 150, in main
    config = _prepareconfig(args, plugins)
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 331, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 433, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/pluggy/_manager.py", line 112, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/pluggy/_callers.py", line 133, in _multicall
    teardown[0].send(outcome)
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/_pytest/helpconfig.py", line 104, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/pluggy/_result.py", line 108, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/pluggy/_callers.py", line 80, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1075, in pytest_cmdline_parse
    self.parse(args)
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1425, in parse
    self._preparse(args, addopts=addopts)
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1305, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/pluggy/_manager.py", line 364, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/importlib/metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/runner/work/fastrepl/fastrepl/fastrepl/__init__.py", line 6, in <module>
    from fastrepl.eval import *
  File "/home/runner/work/fastrepl/fastrepl/fastrepl/eval/__init__.py", line 2, in <module>
    from fastrepl.eval.model import *
  File "/home/runner/work/fastrepl/fastrepl/fastrepl/eval/model/__init__.py", line 3, in <module>
    from fastrepl.eval.model.others import RAGAS
  File "/home/runner/work/fastrepl/fastrepl/fastrepl/eval/model/others.py", line 9, in <module>
    from ragas import evaluate
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/ragas/__init__.py", line 1, in <module>
    from ragas.evaluation import evaluate
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/ragas/evaluation.py", line 9, in <module>
    from ragas.metrics.base import Metric
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/ragas/metrics/__init__.py", line 1, in <module>
    from ragas.metrics.answer_relevance import AnswerRelevancy, answer_relevancy
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/ragas/metrics/answer_relevance.py", line 115, in <module>
    answer_relevancy = AnswerRelevancy()
  File "<string>", line 4, in __init__
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/ragas/metrics/base.py", line 110, in _llm_factory
    return ChatOpenAI(model_name="gpt-3.5-turbo-16k")  # type: ignore
  File "/home/runner/work/fastrepl/fastrepl/.venv/lib/python3.9/site-packages/langchain/load/serializable.py", line 75, in __init__
    super().__init__(**kwargs)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for ChatOpenAI
__root__
  Did not find openai_api_key, please add an environment variable `OPENAI_API_KEY` which contains it, or pass  `openai_api_key` as a named parameter. (type=value_error)
task: Failed to run task "test:ci": exit status 1
Error: Process completed with exit code 201.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions