mock-inspector: static analyzer for mock anti-patterns

Author: nunacode

.gitignore

@@ -0,0 +1,18 @@
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+.pytest_cache/
+.venv/
+venv/
+.env
+.coverage
+htmlcov/
+build/
+dist/
+.DS_Store
+.idea/
+.vscode/
+AGENTS.md
+.claude/
+CLAUDE.md

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 python-testing-debugging
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,140 @@
+# mock-inspector
+
+Static analyzer that catches `unittest.mock` / `pytest-mock` anti-patterns that
+**silently pass at runtime** — before they let a broken test go green.
+
+Mocks fail open. A misspelled assertion, a forgotten pair of parentheses, or a
+signature-blind patch does not raise — it quietly asserts nothing and your suite
+stays green. `mock-inspector` reads your test files with the `ast` module (no
+imports, no execution) and points at the exact `file:line`:
+
+```text
+$ python -m mock_inspector tests/
+tests/test_billing.py
+     21:4  HIGH    MI001 silent-assertion
+           'assert_called_once' is accessed but never called - this assertion is a no-op and always passes. Add '()'.
+           see https://python-testing-debugging.com/advanced-mocking-test-doubles-in-python/deep-dive-into-unittestmock/
+     30:4  HIGH    MI002 bogus-assertion-method
+           'assert_called_once_with_args()' is not a real Mock assertion method - it does nothing and never fails. Did you mean a real 'assert_*' method?
+           see https://python-testing-debugging.com/advanced-mocking-test-doubles-in-python/deep-dive-into-unittestmock/
+     34:1  MEDIUM  MI003 patch-without-autospec
+           patch(...) without autospec=True (or spec/new_callable): the mock will accept any arguments and attribute access, hiding real bugs.
+           see https://python-testing-debugging.com/advanced-mocking-test-doubles-in-python/autospec-strict-mocking/
+
+3 findings (2 high, 1 medium)
+```
+
+## Why this exists
+
+Two of these bugs are genuinely invisible. `mock.assert_called_once` **without
+the parentheses** evaluates to a bound method, which is truthy, and the
+statement is discarded — the assertion never runs. `mock.assert_called_onse(...)`
+(a typo) is worse: `Mock` fabricates the attribute on access, so calling it
+returns another mock and never raises. Both make a test that asserts *nothing*
+look like a test that passes.
+
+The rest are design smells that autospec and careful patch targeting prevent. A
+plain `patch()` returns a `MagicMock` that accepts *any* call signature, so a
+test keeps passing after you change (or break) the real function's arguments —
+which is why `autospec=True` exists. And patching a library you don't own
+(`patch("requests.get")`) instead of the seam where your code *uses* it is the
+single most common reason a patch "doesn't take"; more on choosing the right
+target: [where to patch](https://python-testing-debugging.com/advanced-mocking-test-doubles-in-python/patching-strategies-for-complex-codebases/where-to-patch-understanding-mock-patch-targets/).
+
+## The rules
+
+| ID | Name | Severity | What it catches |
+|----|------|----------|-----------------|
+| `MI001` | silent-assertion | HIGH | A `assert_*` assertion accessed as an attribute but never called (no-op). |
+| `MI002` | bogus-assertion-method | HIGH | A called `assert_*` method that Mock does not define (typo → always passes). |
+| `MI003` | patch-without-autospec | MEDIUM | `patch()` / `patch.object()` with no `autospec` / `spec` / `new` / `new_callable`. |
+| `MI004` | mocking-what-you-dont-own | MEDIUM | `patch("<stdlib-or-third-party>.…")` instead of your own seam. |
+| `MI005` | unused-mock | LOW | A mock created (or bound via `as`) but never referenced again in the function. |
+| `MI006` | return_value-and-side_effect | MEDIUM | Both `return_value` and `side_effect` set on one mock (`side_effect` silently wins). |
+
+## Running it
+
+No install required — run it straight from a clone:
+
+```bash
+git clone https://github.com/python-testing-debugging/mock-inspector
+cd mock-inspector
+python -m mock_inspector path/to/your/tests/
+```
+
+Python 3.9+ and the standard library only — no third-party dependencies. On
+Python 3.10+ the stdlib module list (`sys.stdlib_module_names`) powers `MI004`;
+on 3.9 that rule falls back to the curated third-party list only.
+
+You can also install the console script if you prefer (`pip install -e .` gives
+you a `mock-inspector` command), but the documented, canonical way is running
+the module from source.
+
+### Options
+
+| Flag | Effect |
+|------|--------|
+| `--select MI001,MI003` | Run only the listed rules. |
+| `--ignore MI005` | Run everything except the listed rules. |
+| `--format {text,json}` | Output format (default `text`). JSON is machine-readable for CI. |
+| `--stats` | Print over-mock density (mocks per test function) instead of findings. |
+| `--exit-zero` | Report findings but always exit `0` (soft CI gate). |
+| `--third-party mod,mod` | Extra top-level modules to treat as "not yours" for `MI004`. |
+| `--own mod,mod` | Top-level modules to treat as owned (suppress `MI004` for them). |
+| `--no-color` | Disable ANSI color even on a TTY. |
+| `--list-rules` | Print the rule catalog and exit. |
+
+## How it works
+
+1. **Discover** every `*.py` file under the given paths (recursively); files are
+   de-duplicated and processed in a stable order.
+2. **Parse** each file into an AST with `ast.parse`. A file that does not parse
+   is reported on stderr and skipped — the run continues.
+3. **Walk once** for the call/statement rules: a bare `assert_*` attribute
+   statement (`MI001`), a called assert method that is not in the canonical Mock
+   set (`MI002`, matched with a prefix + edit-distance heuristic so real helpers
+   like `assert_frame_equal` are left alone), signature-blind `patch()` calls
+   (`MI003`), library-boundary patch targets (`MI004`), and one call carrying
+   both `return_value` and `side_effect` (`MI006`).
+4. **Walk per function** for the scoped rules: a mock name that is never read
+   again (`MI005`), and `return_value` / `side_effect` split across two
+   attribute assignments on the same object (`MI006`).
+5. **Filter** the findings through `--select` / `--ignore`, then render them as
+   text grouped by file (with a severity summary) or as JSON.
+
+## Exit codes
+
+| Code | Meaning |
+|------|---------|
+| `0` | Clean (or `--exit-zero` was passed). |
+| `1` | At least one finding was reported. |
+| `2` | Usage error, or a file could not be read/parsed and nothing else was found. |
+
+## Demo
+
+```bash
+# Lists a finding for every rule MI001-MI006:
+python -m mock_inspector examples/bad_mocks.py
+
+# The corrected version is clean (exit 0):
+python -m mock_inspector examples/good_mocks.py
+```
+
+`examples/bad_mocks.py` is a commented catalog of each anti-pattern;
+`examples/good_mocks.py` is the minimal fix for each one, side by side.
+
+## Background reading
+
+`mock-inspector` grew out of the mocking material on
+[python-testing-debugging.com](https://python-testing-debugging.com), which digs
+into why these mistakes pass silently and how to structure test doubles so they
+don't. Worth reading alongside the rules above:
+
+- [Advanced mocking & test doubles in Python](https://python-testing-debugging.com/advanced-mocking-test-doubles-in-python/)
+- [A deep dive into unittest.mock](https://python-testing-debugging.com/advanced-mocking-test-doubles-in-python/deep-dive-into-unittestmock/)
+- [Autospec & strict mocking](https://python-testing-debugging.com/advanced-mocking-test-doubles-in-python/autospec-strict-mocking/)
+- [Where to patch: understanding mock.patch targets](https://python-testing-debugging.com/advanced-mocking-test-doubles-in-python/patching-strategies-for-complex-codebases/where-to-patch-understanding-mock-patch-targets/)
+
+## License
+
+MIT — see [LICENSE](LICENSE).

examples/bad_mocks.py

@@ -0,0 +1,64 @@
+"""A teaching artifact: every line below is a mock anti-pattern.
+
+Running ``python -m mock_inspector examples/bad_mocks.py`` should report at
+least one finding for each rule MI001-MI006. None of these mistakes raises an
+error at runtime -- that is precisely why they are dangerous: the tests look
+green while asserting nothing (or asserting the wrong thing).
+
+This file is intentionally *not* meant to be executed as a test suite; it is a
+catalog of bugs for the linter to catch.
+"""
+
+from unittest import mock
+from unittest.mock import MagicMock, Mock, patch
+
+
+def test_mi001_silent_assertion():
+    service = MagicMock()
+    service.charge(42)
+    # BUG (MI001): missing parentheses. This accesses the assertion method as
+    # an attribute and immediately discards it -- it never actually asserts.
+    service.charge.assert_called_once
+    service.charge.assert_called_with
+
+
+def test_mi002_bogus_assertion_method():
+    sender = MagicMock()
+    sender.send("hi")
+    # BUG (MI002): there is no such method. Mock auto-creates it as a child
+    # mock, so the "assertion" happily returns another mock and never fails.
+    sender.send.assert_called_once_with_args("hi")
+    sender.assert_has_call("hi")
+
+
+@patch("myapp.services.mailer")  # BUG (MI003): no autospec -> accepts any call
+def test_mi003_patch_without_autospec(mailer):
+    mailer.deliver(subject="hi", body="there", bogus_kwarg=123)
+    mailer.deliver.assert_called_once()
+
+
+def test_mi004_mocking_what_you_dont_own():
+    # BUG (MI004): patching the stdlib boundary instead of your own seam.
+    with patch("os.getcwd", return_value="/tmp"):
+        pass
+    # BUG (MI004): third-party library boundary.
+    with patch("requests.get") as _resp:
+        _resp.return_value.json.return_value = {}
+
+
+def test_mi005_unused_mock():
+    # BUG (MI005): created and then never referenced again. The assertion the
+    # author meant to write (orphan.something.assert_called()) is missing.
+    orphan = MagicMock()
+    result = 1 + 1
+    assert result == 2
+
+
+def test_mi006_return_value_and_side_effect():
+    # BUG (MI006, call form): side_effect wins; return_value is ignored.
+    client = Mock(return_value=200, side_effect=ConnectionError)
+    # BUG (MI006, attribute form): same footgun spelled out over two lines.
+    fetch = mock.MagicMock()
+    fetch.return_value = {"ok": True}
+    fetch.side_effect = TimeoutError
+    return client, fetch

examples/good_mocks.py

@@ -0,0 +1,55 @@
+"""The corrected version of ``bad_mocks.py``.
+
+Every anti-pattern has been fixed, so ``python -m mock_inspector`` reports
+nothing here. Compare this file with ``bad_mocks.py`` side by side to see the
+minimal change each rule asks for.
+"""
+
+from unittest import mock
+from unittest.mock import MagicMock, patch
+
+
+def test_mi001_silent_assertion():
+    service = MagicMock()
+    service.charge(42)
+    # FIXED (MI001): actually call the assertion.
+    service.charge.assert_called_once()
+    service.charge.assert_called_with(42)
+
+
+def test_mi002_bogus_assertion_method():
+    sender = MagicMock()
+    sender.send("hi")
+    # FIXED (MI002): use the real method names.
+    sender.send.assert_called_once_with("hi")
+    sender.send.assert_has_calls([mock.call("hi")])
+
+
+@patch("myapp.services.mailer", autospec=True)  # FIXED (MI003): autospec on
+def test_mi003_patch_without_autospec(mailer):
+    mailer.deliver(subject="hi", body="there")
+    mailer.deliver.assert_called_once()
+
+
+def test_mi004_mocking_what_you_dont_own():
+    # FIXED (MI004): patch where the object is *used* in your own code, and
+    # give it a spec so it cannot drift from the real signature.
+    with patch("myapp.clients.session", autospec=True) as session:
+        session.get.return_value.json.return_value = {}
+        assert session.get is not None
+
+
+def test_mi005_unused_mock():
+    # FIXED (MI005): the mock is actually exercised and asserted on.
+    dependency = MagicMock()
+    dependency.configure_mock(name="used")
+    dependency.ping()
+    dependency.ping.assert_called_once()
+
+
+def test_mi006_return_value_and_side_effect():
+    # FIXED (MI006): pick exactly one behavior.
+    client = MagicMock(side_effect=ConnectionError)
+    fetch = mock.MagicMock()
+    fetch.return_value = {"ok": True}
+    return client, fetch

mock_inspector/__init__.py

@@ -0,0 +1,9 @@
+"""mock-inspector: a static analyzer for unittest.mock / pytest-mock misuse.
+
+Scans test files with the :mod:`ast` module (no imports, no execution) and
+reports mock anti-patterns that silently pass at runtime.
+"""
+
+from __future__ import annotations
+
+__version__ = "0.1.0"

mock_inspector/__main__.py

@@ -0,0 +1,7 @@
+"""Enable ``python -m mock_inspector``."""
+
+from __future__ import annotations
+
+from mock_inspector.cli import main
+
+raise SystemExit(main())