Installation of torch with dependency groups leads to AssertionError/OverrideNeededError

[anates]

Description

I tried upgrading an older poetry-based project from python 3.12 to python 3.13, as I no longer have access to python 3.12. For that, I had to upgrade versions for both torch and torchvision, as both packages are required in the project. However, when running poetry install -vvv --only gpu --extras "gpu" or just simply poetry install after deleting the old venv, lock file and cache I received the following error:

OverrideNeededError

({Package('tmp-processing-pipeline', 'v2.0.0'): {'torch': <Dependency torch (>=2.6.0,<3.0.0)>}}, {Package('tmp-processing-pipeline', 'v2.0.0'): {'torch': <Dependency
torch (>=2.6.0,<3.0.0)>}}, {Package('tmp-processing-pipeline', 'v2.0.0'): {'torch': <Dependency torch (>=2.6.0,<3.0.0)>}}, {Package('tmp-processing-pipeline', 'v2.0.0')
: {'torch': <Dependency torch ()>}})

at ~.local\pipx\venvs\poetry\Lib\site-packages\poetry\puzzle\provider.py:699 in complete_package
695│ package_overrides.update({dep.name: dep})
696│ current_overrides.update({package: package_overrides})
697│ overrides.append(current_overrides)
698│
→ 699│ raise OverrideNeededError(*overrides)
700│
701│ # Modifying dependencies as needed
702│ clean_dependencies = []
703│ for dep in dependencies:

The following error occurred when trying to handle this error:

AssertionError

at ~.local\pipx\venvs\poetry\Lib\site-packages\poetry\mixology\version_solver.py:237 in _propagate
233│ # decision level, so we clear [changed] and refill it with the
234│ # newly-propagated assignment.
235│ changed.clear()
236│ result = self._propagate_incompatibility(root_cause)
→ 237│ assert result is not None
238│ assert result != _conflict
239│ assert isinstance(result, str)
240│ changed.add(result)
241│ break

It seems as if the error was introduced by my changes in pyproject.toml, as the previous version worked fine with python 3.12 and poetry 2.1.2.

The original version of pyproject.toml is

[tool.poetry]
name = "tmp-processing-pipeline"
version = "2.0.0a0"
description = ""
authors = ["Roland Axel Richter <roland.richter@empa.ch>"]
readme = "README.md"
license = "MIT"
classifiers = [
    "Development Status :: 4 - Beta",

    "License :: OSI Approved :: MIT License",

    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
]

[tool.poetry.dependencies]
python = ">=3.10,<3.13"
pandas = "^2.2.3"
numpy = "1.26.*"
argparse = "^1.4.0"
openpyxl = "^3.1.5"
pyxlsb = "^1.0.10"
pyserde = "^0.20.1"
watchdog = "^6.0.0"
fonttools = "4.53.1"
torch = { version = "^2.5.1", optional = true }
torchaudio = { version = "^2.1.1", optional = true }
torchvision = { version = "^0.20.0", optional = true }

[tool.poetry.group.cpu]
optional = true

[tool.poetry.group.cpu.dependencies]
torch = { version = "^2.5.1", source = "torch_cpu", markers = "extra=='cpu' and extra!='gpu'" }
torchaudio = { version = "^2.1.1", source = "torch_cpu", markers = "extra=='cpu' and extra!='gpu'" }
torchvision = { version = "^0.20.0", source = "torch_cpu", markers = "extra=='cpu' and extra!='gpu'" }

[tool.poetry.group.gpu]
optional = true

[tool.poetry.group.gpu.dependencies]
torch = { version = "^2.5.1", source = "torch_cuda", markers = "extra=='gpu' and extra!='cpu'" }
torchaudio = { version = "^2.1.1", source = "torch_cuda", markers = "extra=='gpu' and extra!='cpu'" }
torchvision = { version = "^0.20.0", source = "torch_cuda", markers = "extra=='gpu' and extra!='cpu'" }

[tool.poetry.extras]
cpu = ["torch", "torchvision"]
gpu = ["torch", "torchvision"]

[[tool.poetry.source]]
name = "torch_cuda"
url = "https://download.pytorch.org/whl/cu124"
priority = "explicit"

[[tool.poetry.source]]
name = "torch_cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

[[tool.poetry.source]]
name = "PyPI"
priority = "primary"

[tool.poetry.group.dev.dependencies]
nuitka = "^2.4.1"
jupyterlab = "^4.2.5"
pre-commit = "^3.7.1"
pybis = "^1.36.3"
jupyter = "^1.1.1"
pickleshare = "^0.7.5"
pytest = "^8.3.3"
dunamai = "^1.22.0"
pytest-cov = "^5.0.0"
pytest-regtest = "^2.2.1"
sphinx = "^8.1.3"
sphinx-autoapi = "^3.3.3"
seaborn = "^0.13.2"

[tool.poetry-dynamic-versioning]
enable = true
dirty = true
format = "v{base}"

[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning>=1.0.0,<2.0.0"]
build-backend = "poetry_dynamic_versioning.backend"

[tool.pyright]
include = ["tmp_processing_pipeline"]
exclude = ["**/virtualenvs/**"]
 
reportMissingImports = true
reportMissingTypeStubs = false
 
[tool.coverage.report]
exclude_also = ["if TYPE_CHECKING:"]
 
[tool.coverage.run]
omit = ["tmp_processing_pipeline/notebooks/*"]

[tool.pytest.ini_options]
addopts = "-rA --cov=tmp_processing_pipeline --cov-report html --cov-report term --cov-report xml:coverage.xml --junitxml=junit_report.xml"

with the revised version of the toml-file being

[tool.poetry]
name = "tmp-processing-pipeline"
version = "2.0.0a0"
description = ""
authors = ["Roland Axel Richter <roland.richter@empa.ch>"]
readme = "README.md"
license = "MIT"
classifiers = [
    "Development Status :: 4 - Beta",

    "License :: OSI Approved :: MIT License",

    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: 3.13",
]

[tool.poetry.dependencies]
python = ">=3.10,<3.14"
pandas = "^2.2.3"
numpy = "1.26.*"
argparse = "^1.4.0"
openpyxl = "^3.1.5"
pyxlsb = "^1.0.10"
pyserde = "^0.20.1"
watchdog = "^6.0.0"
fonttools = "4.53.1"
torch = { version = "^2.6.0", optional = true }
torchaudio = { version = "^2.1.1", optional = true }
torchvision = { version = "^0.21.0", optional = true }

[tool.poetry.group.cpu]
optional = true

[tool.poetry.group.cpu.dependencies]
torch = { version = "^2.6.0", source = "torch_cpu", markers = "extra=='cpu' and extra!='gpu'" }
torchaudio = { version = "^2.1.1", source = "torch_cpu", markers = "extra=='cpu' and extra!='gpu'" }
torchvision = { version = "^0.21.0", source = "torch_cpu", markers = "extra=='cpu' and extra!='gpu'" }

[tool.poetry.group.gpu]
optional = true

[tool.poetry.group.gpu.dependencies]
torch = { version = "^2.6.0", source = "torch_cuda", markers = "extra=='gpu' and extra!='cpu'" }
torchaudio = { version = "^2.1.1", source = "torch_cuda", markers = "extra=='gpu' and extra!='cpu'" }
torchvision = { version = "^0.21.0", source = "torch_cuda", markers = "extra=='gpu' and extra!='cpu'" }

[tool.poetry.extras]
cpu = ["torch", "torchvision"]
gpu = ["torch", "torchvision"]

[[tool.poetry.source]]
name = "torch_cuda"
url = "https://download.pytorch.org/whl/cu124"
priority = "explicit"
 
[[tool.poetry.source]]
name = "torch_cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

# [[tool.poetry.source]]
# name = "PyPI"
# priority = "primary"
# url = "https://pypi.org/simple"

[tool.poetry.group.dev.dependencies]
nuitka = "^2.4.1"
jupyterlab = "^4.2.5"
pre-commit = "^3.7.1"
pybis = "^1.36.3"
jupyter = "^1.1.1"
pickleshare = "^0.7.5"
pytest = "^8.3.3"
dunamai = "^1.22.0"
pytest-cov = "^5.0.0"
pytest-regtest = "^2.2.1"
sphinx = "^8.1.3"
sphinx-autoapi = "^3.3.3"
seaborn = "^0.13.2"

[tool.poetry-dynamic-versioning]
enable = true
dirty = true
format = "v{base}"
 
[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning>=1.0.0,<2.0.0"]
build-backend = "poetry_dynamic_versioning.backend"

[tool.pyright]
include = ["tmp_processing_pipeline"]
exclude = ["**/virtualenvs/**"]

reportMissingImports = true
reportMissingTypeStubs = false

[tool.coverage.report]
exclude_also = ["if TYPE_CHECKING:"]

[tool.coverage.run]
omit = ["tmp_processing_pipeline/notebooks/*"]

[tool.pytest.ini_options]
addopts = "-rA --cov=tmp_processing_pipeline --cov-report html --cov-report term --cov-report xml:coverage.xml --junitxml=junit_report.xml"

Why do I get the error (extended error below), and what can I do to fix it?

Workarounds

None known

Poetry Installation Method

pipx

Operating System

Windows 11

Poetry Version

2.1.2

Poetry Configuration

cache-dir = "C:\\Users\\riro\\AppData\\Local\\pypoetry\\Cache"
data-dir = "C:\\Users\\riro\\AppData\\Roaming\\pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
python.installation-dir = "{data-dir}\\python"  # C:\Users\riro\AppData\Roaming\pypoetry\python
repositories.torch_cpu.url = "https://download.pytorch.org/whl/cpu"
repositories.torch_cuda.url = "https://download.pytorch.org/whl/cu124"
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs"  # C:\Users\riro\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false

Python Sysconfig

No response

Example pyproject.toml

Poetry Runtime Logs

poetry-runtime.log

Found: C:\Program Files\Python313\python.EXE

Stack trace:

4  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\puzzle\solver.py:170 in _solve
    168│
    169│         try:
  → 170│             result = resolve_version(self._package, self._provider)
    171│
    172│             packages = result.packages

3  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\mixology\__init__.py:18 in resolve_version
     16│     solver = VersionSolver(root, provider)
     17│
  →  18│     return solver.solve()
     19│

2  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\mixology\version_solver.py:192 in solve
    190│             while next is not None:
    191│                 self._propagate(next)
  → 192│                 next = self._choose_package_version()
    193│
    194│             return self._result()

1  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\mixology\version_solver.py:599 in _choose_package_version
    597│             package = locked
    598│
  → 599│         package = self._provider.complete_package(package)
    600│
    601│         conflict = False

OverrideNeededError

({Package('tmp-processing-pipeline', 'v2.0.0'): {'torch': <Dependency torch (>=2.6.0,<3.0.0)>}}, {Package('tmp-processing-pipeline', 'v2.0.0'): {'torch': <Dependency
torch (>=2.6.0,<3.0.0)>}}, {Package('tmp-processing-pipeline', 'v2.0.0'): {'torch': <Dependency torch (>=2.6.0,<3.0.0)>}}, {Package('tmp-processing-pipeline', 'v2.0.0')
: {'torch': <Dependency torch (<empty>)>}})

at ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\puzzle\provider.py:699 in complete_package
     695│                 package_overrides.update({dep.name: dep})
     696│                 current_overrides.update({package: package_overrides})
     697│                 overrides.append(current_overrides)
     698│
  →  699│             raise OverrideNeededError(*overrides)
     700│
     701│         # Modifying dependencies as needed
     702│         clean_dependencies = []
     703│         for dep in dependencies:

The following error occurred when trying to handle this error:


Stack trace:

17  ~\.local\pipx\venvs\poetry\Lib\site-packages\cleo\application.py:327 in run
     325│
     326│             try:
   → 327│                 exit_code = self._run(io)
     328│             except BrokenPipeError:
     329│                 # If we are piped to another process, it may close early and send a

16  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\console\application.py:260 in _run
     258│         with directory(self._working_directory):
     259│             try:
   → 260│                 exit_code = super()._run(io)
     261│             except PoetryRuntimeError as e:
     262│                 io.write_error_line("")

15  ~\.local\pipx\venvs\poetry\Lib\site-packages\cleo\application.py:431 in _run
     429│             io.input.interactive(interactive)
     430│
   → 431│         exit_code = self._run_command(command, io)
     432│         self._running_command = None
     433│

14  ~\.local\pipx\venvs\poetry\Lib\site-packages\cleo\application.py:473 in _run_command
     471│
     472│         if error is not None:
   → 473│             raise error
     474│
     475│         return terminate_event.exit_code

13  ~\.local\pipx\venvs\poetry\Lib\site-packages\cleo\application.py:457 in _run_command
     455│
     456│             if command_event.command_should_run():
   → 457│                 exit_code = command.run(io)
     458│             else:
     459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

12  ~\.local\pipx\venvs\poetry\Lib\site-packages\cleo\commands\base_command.py:117 in run
     115│         io.input.validate()
     116│
   → 117│         return self.execute(io) or 0
     118│
     119│     def merge_application_definition(self, merge_args: bool = True) -> None:

11  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\console\commands\installer_command.py:39 in execute
      37│     def execute(self, io: IO) -> int:
      38│         PoetryKeyring.preflight_check(io, self.poetry.config)
   →  39│         return super().execute(io)
      40│

10  ~\.local\pipx\venvs\poetry\Lib\site-packages\cleo\commands\command.py:61 in execute
      59│
      60│         try:
   →  61│             return self.handle()
      62│         except KeyboardInterrupt:
      63│             return 1

 9  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\console\commands\install.py:173 in handle
     171│         self.installer.verbose(self.io.is_verbose())
     172│
   → 173│         return_code = self.installer.run()
     174│
     175│         if return_code != 0:

 8  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\installation\installer.py:103 in run
     101│             self.verbose(True)
     102│
   → 103│         return self._do_install()
     104│
     105│     def dry_run(self, dry_run: bool = True) -> Installer:

 7  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\installation\installer.py:241 in _do_install
     239│                 source_root=self._env.path.joinpath("src")
     240│             ):
   → 241│                 solved_packages = solver.solve(
     242│                     use_latest=self._whitelist
     243│                 ).get_solved_packages()

 6  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\puzzle\solver.py:87 in solve
      85│         with self._progress(), self._provider.use_latest_for(use_latest or []):
      86│             start = time.time()
   →  87│             packages = self._solve()
      88│             # simplify markers by removing redundant information
      89│             for transitive_info in packages.values():

 5  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\puzzle\solver.py:174 in _solve
     172│             packages = result.packages
     173│         except OverrideNeededError as e:
   → 174│             return self._solve_in_compatibility_mode(e.overrides)
     175│         except SolveFailureError as e:
     176│             raise SolverProblemError(e)

 4  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\puzzle\solver.py:158 in _solve_in_compatibility_mode
     156│             )
     157│             self._provider.set_overrides(override)
   → 158│             new_packages = self._solve()
     159│             override_packages.append((override, new_packages))
     160│

 3  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\puzzle\solver.py:170 in _solve
     168│
     169│         try:
   → 170│             result = resolve_version(self._package, self._provider)
     171│
     172│             packages = result.packages

 2  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\mixology\__init__.py:18 in resolve_version
      16│     solver = VersionSolver(root, provider)
      17│
   →  18│     return solver.solve()
      19│

 1  ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\mixology\version_solver.py:191 in solve
     189│             next: str | None = self._root.name
     190│             while next is not None:
   → 191│                 self._propagate(next)
     192│                 next = self._choose_package_version()
     193│

AssertionError



at ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\mixology\version_solver.py:237 in _propagate
    233│                     # decision level, so we clear [changed] and refill it with the
    234│                     # newly-propagated assignment.
    235│                     changed.clear()
    236│                     result = self._propagate_incompatibility(root_cause)
  → 237│                     assert result is not None
    238│                     assert result != _conflict
    239│                     assert isinstance(result, str)
    240│                     changed.add(result)
    241│                     break

[dimbleby]

duplicate #8196

[mainguyenanhvu]

@anates Do you fix it?

[radoering]

Does not reproduce anymore with Poetry 2.4.1.

[dimbleby]

In fact the second example ("the revised version of the toml-file") still bugs out, but python-poetry/poetry-core#949 and python-poetry/poetry-core#950 between them fix it

[radoering]

My mistake. I probably did not delete the lock file in between trying to reproduce the issue.