Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# pytest-order Release Notes

## Unreleased

### Fixes
- transitive relative chains are resolved as a single globally consistent order

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this merits a patch release after it is merged.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speaking of a patch. This may be a breaking change, because:

  • I found that before, some constraints were dropped silently, and after the chance they produce a warning. I don't have an example in tests now, would you require one?
  • Obviously, ordering for some cases has changed - I had to update tests to match it. Specifically, old versions of tests.test_relative_ordering.test_dependency_loop and tests.test_relative_ordering.test_failed_tests_after_dependency_loop fail on the new code.
  • And of course there are customers who rely on the old, incomplete ordering 😆 😢

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I still think of it as a bug fix. I know you are thinking of Hyrum's Law, but you just made cases possible that before have been handled (incorrectly) as a dependency loop. If somebody relied on that behavior to prevent some implied ordering, I certainly wouldn't bother about it...

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the warning was missing before, so it just improved the visibility of an incorrect marker.

## [Version 1.4.0](https://pypi.org/project/pytest-order/1.4.0/) (2026-04-26)
Allows the plugin to run after `--failed-first` and similar options.

Expand Down
35 changes: 35 additions & 0 deletions docs/source/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,41 @@ ones. This means that relative ordering always takes preference:
In this case, ``test_second`` will be executed before ``test_first``,
regardless of the ordinal markers.

This also applies to a relative marker that references an ordinal-pinned
anchor, and to transitive relative chains, which are resolved as a single
globally consistent order:

.. code:: python

import pytest


def test_setup():
assert True


@pytest.mark.order(after="test_anchor", before="test_last")
def test_middle():
assert True


@pytest.mark.order("second_to_last")
def test_anchor():
assert True


@pytest.mark.order("last")
def test_last():
assert True

Here ``test_middle`` is placed after ``test_anchor`` and before ``test_last``,
relaxing the ``second_to_last`` ordinal so that the relative constraints hold::

test_setup
test_anchor
test_middle
test_last

Several relationships for the same marker
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you need to order a certain test relative to more than one other test, you
Expand Down
156 changes: 140 additions & 16 deletions src/pytest_order/item.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import sys
from typing import Optional, Generic, TypeVar
from collections import defaultdict

from _pytest.python import Function

Expand All @@ -12,11 +13,12 @@
class Item:
"""Represents a single test item."""

def __init__(self, item: Function) -> None:
def __init__(self, item: Function, collection_index: int = 0) -> None:
self.item: Function = item
self.nr_rel_items: int = 0
self.order: Optional[int] = None
self._node_id: Optional[str] = None
self.collection_index: int = collection_index

def inc_rel_marks(self) -> None:
if self.order is None:
Expand Down Expand Up @@ -101,21 +103,51 @@ def sort_numbered_items(self) -> list[Item]:
sorted_list[mid_index:mid_index] = self.unordered_items
return sorted_list

def print_unhandled_items(self) -> None:
failed_items = [mark.item for mark in self.rel_marks] + [
mark.item for mark in self.dep_marks
]
msg = " ".join([item.node_id for item in failed_items])
sys.stdout.write("\nWARNING: cannot execute test relative to others: ")
sys.stdout.write(msg)
if self.settings.error_on_failed_ordering:
sys.stdout.write(" - ignoring the marker.\n")
else:
sys.stdout.write(".\n")
sys.stdout.flush()
if self.settings.error_on_failed_ordering:
for item in failed_items:
item.item.fixturenames.insert(0, "fail_after_cannot_order")
def apply_relative_constraints(self, sorted_list: list[Item]) -> bool:
"""
Reorder sorted_list to satisfy all relative (before/after and
dependency) constraints.

The incoming sorted_list already reflects the absolute ordinal markers,
which act only as a baseline preference: relative markers always take
preference, so an ordinal position is relaxed whenever it conflicts with
a relative constraint (see "Combination of absolute and relative
ordering" in the docs).

Two phases:
1. Iterative move_item passes satisfy the constraints that can be met by
moving a single item, preserving the position of items not involved
in any constraint and the established tie-breaking.
2. A stable topological sort over the full constraint set re-validates
the result. It leaves an already-valid baseline unchanged, but
resolves constraints the iterative pass cannot keep on its own
(transitive chains, or a marker relative to an ordinal-pinned anchor
that a later move re-breaks).

Only a genuine constraint cycle cannot be satisfied; in that case the
caller reports the offending markers.

Returns True if all constraints were satisfied (no cycle).
Comment thread
mrbean-bremen marked this conversation as resolved.
"""
if not self.rel_marks and not self.dep_marks:
return True

rel_marks = list(self.rel_marks)
dep_marks = list(self.dep_marks)
self._apply_iterative(sorted_list)

ordered, had_cycle = self._sort_by_topology(sorted_list, rel_marks, dep_marks)
sorted_list[:] = ordered
return not had_cycle

def _apply_iterative(self, sorted_list: list[Item]) -> None:
still_left = 0
length = self.number_of_rel_groups()
while length and still_left != length:
still_left = length
self.handle_rel_marks(sorted_list)
self.handle_dep_marks(sorted_list)
length = self.number_of_rel_groups()

def number_of_rel_groups(self) -> int:
return len(self.rel_marks) + len(self.dep_marks)
Expand All @@ -137,13 +169,86 @@ def handle_relative_marks(
marks.remove(mark)
all_marks.remove(mark)

def print_unhandled_items(self) -> None:
failed_items = [mark.item for mark in self.rel_marks] + [
mark.item for mark in self.dep_marks
]
msg = " ".join([item.node_id for item in failed_items])
sys.stdout.write("\nWARNING: cannot execute test relative to others: ")
sys.stdout.write(msg)
if self.settings.error_on_failed_ordering:
sys.stdout.write(" - ignoring the marker.\n")
else:
sys.stdout.write(".\n")
sys.stdout.flush()
if self.settings.error_on_failed_ordering:
for item in failed_items:
item.item.fixturenames.insert(0, "fail_after_cannot_order")

def group_order(self) -> Optional[int]:
if self.start_items:
return self.start_items[0][0]
elif self.end_items:
return self.end_items[-1][0]
return None

def _sort_by_topology(
self,
items: list["Item"],
rel_marks: list["RelativeMark[Item]"],
dep_marks: list["RelativeMark[Item]"],
) -> tuple[list["Item"], bool]:
"""
Order items so that all relative constraints are satisfied while staying as
close as possible to the incoming order (the absolute-ordinal baseline).

Relative constraints take preference over the baseline: each item is emitted
right after the items it must follow, so a conflicting ordinal position is
relaxed; items that no constraint orders keep their baseline order. The
incoming order already encodes the absolute ordinals, so no extra ordinal
edges are needed.

Returns (ordered_items, had_cycle). On a constraint cycle the offending edge
is dropped, all items are still emitted, and had_cycle is True.
"""
item_set = set(items)
position = {item: i for i, item in enumerate(items)}
predecessors = _build_predecessors(rel_marks + dep_marks, item_set)
for item in items:
predecessors[item].sort(key=lambda p: position[p])

result: list[Item] = []
placed: set[Item] = set()
on_path: set[Item] = set()
had_cycle = False

# Iterative post-order DFS: emit unplaced predecessors before each item.
for start in items:
if start in placed:
continue
stack: list[tuple[Item, int]] = [(start, 0)]
on_path.add(start)
while stack:
item, next_pred = stack[-1]
preds = predecessors[item]
while next_pred < len(preds) and preds[next_pred] in placed:
next_pred += 1
if next_pred < len(preds):
pred = preds[next_pred]
stack[-1] = (item, next_pred + 1)
if pred in on_path:
had_cycle = True
else:
stack.append((pred, 0))
on_path.add(pred)
continue
stack.pop()
on_path.discard(item)
placed.add(item)
result.append(item)

return result, had_cycle


class ItemGroup:
"""
Expand Down Expand Up @@ -227,3 +332,22 @@ def move_item(mark: RelativeMark[_ItemType], sorted_items: list[_ItemType]) -> b
pos_item -= 1
sorted_items.insert(pos_item + 1, mark.item_to_move)
return True


def _build_predecessors(
marks: list["RelativeMark[Item]"],
item_set: set["Item"],
) -> "defaultdict[Item, list[Item]]":
"""Map each item to the items that must run before it, derived from the
relative marks. A mark either places item_to_move after item (move_after)
or before it."""
predecessors: defaultdict[Item, list[Item]] = defaultdict(list)
for mark in marks:
before, after = (
(mark.item, mark.item_to_move)
if mark.move_after
else (mark.item_to_move, mark.item)
)
if before in item_set and after in item_set and before is not after:
predecessors[after].append(before)
return predecessors
11 changes: 2 additions & 9 deletions src/pytest_order/sorter.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ class Sorter:

def __init__(self, config: Config, items: list[Function]) -> None:
self.settings: Settings = Settings(config)
self.items: list[Item] = [Item(item) for item in items]
self.items: list[Item] = [Item(item, idx) for idx, item in enumerate(items)]
self.node_ids: dict[str, Item] = OrderedDict()
self.node_id_last: dict[str, list[str]] = {}
for item in self.items:
Expand Down Expand Up @@ -458,14 +458,7 @@ def sort_items_in_scope(self, items: list[Item], scope: Scope) -> ItemGroup:

sorted_list = item_list.sort_numbered_items()

still_left = 0
length = item_list.number_of_rel_groups()
while length and still_left != length:
still_left = length
item_list.handle_rel_marks(sorted_list)
item_list.handle_dep_marks(sorted_list)
length = item_list.number_of_rel_groups()
if length:
if not item_list.apply_relative_constraints(sorted_list):
item_list.print_unhandled_items()
return ItemGroup(sorted_list, item_list.group_order())

Expand Down
Loading
Loading