renovate: fix requirements.txt handling

[chmeliik]

Renovate's pip_requirements manager is very stupid; it updates each package in a vacuum and doesn't respect dependency relationships. This results in completely bogus requirements.txt updates that have to be fixed by re-compiling the requirements file.

Disable it, use the pip-compile manager instead. This one parses the header in the requirements.txt file, identifies the input file, updates packages in the input file and then reruns the compile command.

Note that this will result in fewer updates for transitive dependencies. Specifically, transitive dependencies will only get updated if uv's dependency resolution determines it is necessary, or for security fixes (assuming MintMaker has that set up). That seems like reasonable behavior.

Testing

• chore(deps): update non-rpm dependencies chmeliik/common-runner#22

Note that by defaults, Renovate also wants to refresh the lockfile: chmeliik#23
Disabled lockfile maintenance for pip-compile

[qodo-app-for-konflux-ci]

Review Summary by Qodo

(Agentic_describe updated until commit 989c6af)

Fix Renovate pip requirements handling with pip-compile manager

✨ Enhancement 🐞 Bug fix

Walkthroughs

Description

• Replace pip_requirements with pip-compile manager for better dependency handling
• Pin direct pip dependencies to specific versions in requirements.in
• Update Makefile to use long-form uv flags for Renovate compatibility
• Disable lockfile maintenance to reduce unnecessary transitive dependency updates

Diagram

flowchart LR
  A["pip_requirements manager<br/>disabled"] -->|"replaced by"| B["pip-compile manager"]
  B -->|"parses header and"| C["updates requirements.in"]
  C -->|"reruns compile"| D["regenerates requirements.txt"]
  E["Direct deps pinned<br/>to versions"] -->|"enables"| F["Renovate updates"]
  F -->|"triggers"| D
  G["lockFileMaintenance<br/>disabled"] -->|"prevents"| H["unnecessary transitive<br/>dependency PRs"]

Loading

File Changes

1. devtool/renovate.py ⚙️ Configuration changes +12/-2

Configure pip-compile manager and disable pip_requirements

• Disabled pip_requirements manager with explanatory comment
• Added pip-compile manager configuration with managerFilePatterns pointing to
 requirements.txt
• Disabled lockFileMaintenance to prevent unnecessary refresh PRs
• Added inline documentation explaining the rationale for each configuration

devtool/renovate.py

---

2. devtool/software_list.py 📝 Documentation +1/-1

Update pip requirements regeneration command reference

• Updated error message to reference make pip-requirements instead of raw uv command

devtool/software_list.py

---

3. Makefile ⚙️ Configuration changes +4/-2

Update uv flags to long-form for Renovate compatibility

• Changed uv pip compile flags to long-form syntax (--python-version= instead of
 --python-version)
• Changed output flag to long-form (--output-file= instead of -o)
• Added comment explaining Renovate's fragile flag parsing requirements

Makefile

---

View more (4)

4. README.md 📝 Documentation +1/-1

Document version pinning requirement for pip packages

• Updated example to show pinning awscli to specific version instead of unpinned

README.md

---

5. deps/pip/requirements.in ⚙️ Configuration changes +7/-4

Pin direct dependencies and update documentation

• Updated command reference from raw uv command to make pip-requirements
• Added comment explaining the need to pin direct dependencies to specific versions
• Pinned all three direct dependencies (awscli, huggingface-hub, olot) to specific versions

deps/pip/requirements.in

---

6. deps/pip/requirements.txt Formatting +1/-1

Update autogenerated header with new flag format

• Updated autogenerated header comment to reflect new long-form uv flags

deps/pip/requirements.txt

---

7. renovate.json5 ⚙️ Configuration changes +10/-1

Configure pip-compile manager in Renovate JSON config

• Disabled pip_requirements manager
• Added pip-compile manager configuration with managerFilePatterns and lockFileMaintenance
 disabled
• Maintained existing package rules for grouping dependencies

renovate.json5

---

[qodo-app-for-konflux-ci]

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0)

Context used

✅ Compliance rules (platform): 4 rules

1. RPM Renovate config missing 🐞 Bug ≡ Correctness

Description

renovate.json5 removes the rpm-lockfile manager block, so Renovate will no longer update deps/rpm/**
lockfiles. This also makes renovate.json5 inconsistent with its generator (devtool/renovate.py), so
running devtool gen renovate.json5 will reintroduce the block and cause config churn.

Code

renovate.json5[L140-149]

-  "rpm-lockfile": {
-    "packageRules": [
-      {
-        "matchFileNames": [
-          "deps/rpm/**"
-        ],
-        "groupName": "RPMs"
-      }
-    ]
-  },

Relevance

⭐⭐⭐ High

RPM Renovate handling is intentional/history: rpm-lockfile grouping and deps/rpm/** matcher
added/maintained in PR99/PR114; kept in sync generator.

PR-#99
PR-#114
PR-#139

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The generator clearly writes renovate.json5 from devtool/renovate.py and still includes an
rpm-lockfile manager, but the committed renovate.json5 now skips directly from the gomod block to
pip managers, meaning RPM lockfile updates are disabled and regeneration will reintroduce the
missing block.

devtool/main.py[86-99]
devtool/renovate.py[57-67]
renovate.json5[139-155]
deps/rpm/rpms.lock.yaml[1-4]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`renovate.json5` no longer contains the `rpm-lockfile` manager configuration, which disables Renovate updates for RPM lockfiles under `deps/rpm/**`.

Additionally, `renovate.json5` is explicitly a generated file; the generator (`devtool/renovate.py` via `devtool gen renovate.json5`) still emits an `rpm-lockfile` block. This mismatch means any future regeneration will unexpectedly re-add the removed config.

## Issue Context
- `devtool gen renovate.json5` writes `renovate.json5` from `renovate_json()`.
- `renovate_json()` currently includes an `rpm-lockfile` manager configuration.
- The repo contains RPM lockfiles (e.g., `deps/rpm/rpms.lock.yaml`) that this manager is meant to cover.

## Fix Focus Areas
- renovate.json5[120-156]
- devtool/renovate.py[57-67]
- devtool/__main__.py[86-99]

## Suggested fix
1. Regenerate `renovate.json5` from `devtool/renovate.py` (run `devtool gen renovate.json5`) and commit the resulting file so it includes the `rpm-lockfile` manager again.
2. If RPM updates are intentionally being removed, also remove the `"rpm-lockfile": {...}` block from `devtool/renovate.py` so the generator and generated file stay consistent.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Pip regen hint stale 🐞 Bug ⚙ Maintainability

Description

deps/pip/requirements.in now documents make pip-requirements, but list_pip_packages still
instructs users to regenerate with an old uv pip compile ... -o requirements.txt command when
requirements.txt is stale. This misdirects debugging and references the wrong paths/flags for the
new workflow.

Code

deps/pip/requirements.in[R3-4]

# Add packages here and regenerate requirements.txt with:
-#   uv pip compile --generate-hashes requirements.in -o requirements.txt
+#   make pip-requirements

Relevance

⭐⭐⭐ High

Team updates regen instructions/messages (Makefile workflow in PR69; generated-files error
reproduction text in PR139).

PR-#69
PR-#139

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The PR changes the documented regeneration command to make pip-requirements and updates the
Makefile/requirements header to new uv flag forms, but the devtool error message still points users
to the old direct uv pip compile ... -o requirements.txt invocation, creating an inconsistent and
misleading workflow.

deps/pip/requirements.in[3-5]
Makefile[59-66]
devtool/software_list.py[318-374]
PR-#61

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`deps/pip/requirements.in` and the `Makefile` now establish `make pip-requirements` as the canonical regeneration path, but `devtool/software_list.py:list_pip_packages()` still raises an error telling users to run `uv pip compile ... -o requirements.txt`.

That message is now misleading (wrong working directory/paths and outdated flag usage), so anyone hitting that error will likely run the wrong command.

## Issue Context
- `requirements.in` says to regenerate via `make pip-requirements`.
- `Makefile` shows the actual `uv pip compile` invocation and output path.
- `list_pip_packages()` throws a `ValueError` with regeneration instructions when it can't find a direct dep in the resolved `requirements.txt`.

## Fix Focus Areas
- devtool/software_list.py[318-374]
- deps/pip/requirements.in[3-5]
- Makefile[59-66]

## Suggested fix
Update the `ValueError` message in `list_pip_packages()` to recommend either:
- `make pip-requirements`, or
- the exact equivalent command matching the Makefile (including `deps/pip/...` paths and `--output-file=...`).

Also ensure the message reflects the current file locations (`deps/pip/requirements.in` and `deps/pip/requirements.txt`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Stale build lockfile 🐞 Bug ☼ Reliability ⭐ New

Description

Renovate is configured to manage only deps/pip/requirements.txt via the pip-compile manager, but
requirements-build.txt is generated by a separate pybuild-deps step and won’t be refreshed when
Renovate updates requirements.in/requirements.txt. This can leave requirements-build.txt out of sync
and break hermetic prefetch/build pipelines that consume both files.

Code

renovate.json5[R153-168]

+  "pip-compile": {
+    "managerFilePatterns": [
+      "deps/pip/requirements.txt"
+    ],
    "packageRules": [
      {
        "matchFileNames": [
          "deps/pip/*"
        ],
        "groupName": "Non-RPM dependencies"
      }
-    ]
+    ],
+    "lockFileMaintenance": {
+      "enabled": false
+    }
  },

Relevance

⭐ Low

Similar concern (include requirements-build.txt in hermetic prefetch) was explicitly rejected in PR
#61 review.

PR-#61

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The repo’s canonical workflow generates requirements-build.txt from requirements.txt, and
CI/pipelines consume both. Renovate’s new managerFilePatterns only targets requirements.txt, and the
compile command embedded in requirements.txt regenerates only that file, so Renovate PRs can update
requirements.txt without updating requirements-build.txt.

renovate.json5[150-168]
devtool/renovate.py[68-85]
Makefile[59-69]
deps/pip/requirements.txt[1-2]
deps/pip/requirements-build.txt[1-6]
.tekton/task-runner-pull-request.yaml[49-63]
.tekton/task-runner-push.yaml[39-54]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Renovate’s `pip-compile` manager is only configured to track `deps/pip/requirements.txt`, but your repo’s workflow requires regenerating **both** `deps/pip/requirements.txt` and `deps/pip/requirements-build.txt` together. As a result, Renovate PRs that update Python deps may leave `requirements-build.txt` stale.

## Issue Context
- `make pip-requirements` generates `requirements.txt` via `uv pip compile`, then generates `requirements-build.txt` via `pybuild-deps compile`.
- Tekton hermetic prefetch inputs reference **both** files.

## Fix Focus Areas
- devtool/renovate.py[68-85]
- renovate.json5[150-168]
- Makefile[59-69]

## Suggested fix
Update Renovate config so that whenever `pip-compile` updates the Python dependencies, it also regenerates and commits `deps/pip/requirements-build.txt`.

Practical approaches:
1. Add a `postUpgradeTasks` (or equivalent Renovate hook) for the `pip-compile` manager/package rule to run `make pip-requirements` and include `deps/pip/requirements*.txt` in the commit.
2. Alternatively, configure Renovate to explicitly manage/regenerate `requirements-build.txt` as well (only if Renovate can reliably run the `pybuild-deps compile ...` command shown in that file’s header). Ensure the solution updates both files in the same PR.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Previous review results

Review updated until commit 989c6af

Results up to commit f5e0f10

🐞 Bugs (2) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0)

1. RPM Renovate config missing 🐞 Bug ≡ Correctness

Description

renovate.json5 removes the rpm-lockfile manager block, so Renovate will no longer update deps/rpm/**
lockfiles. This also makes renovate.json5 inconsistent with its generator (devtool/renovate.py), so
running devtool gen renovate.json5 will reintroduce the block and cause config churn.

Code

renovate.json5[L140-149]

-  "rpm-lockfile": {
-    "packageRules": [
-      {
-        "matchFileNames": [
-          "deps/rpm/**"
-        ],
-        "groupName": "RPMs"
-      }
-    ]
-  },

Relevance

⭐⭐⭐ High

RPM Renovate handling is intentional/history: rpm-lockfile grouping and deps/rpm/** matcher
added/maintained in PR99/PR114; kept in sync generator.

PR-#99
PR-#114
PR-#139

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The generator clearly writes renovate.json5 from devtool/renovate.py and still includes an
rpm-lockfile manager, but the committed renovate.json5 now skips directly from the gomod block to
pip managers, meaning RPM lockfile updates are disabled and regeneration will reintroduce the
missing block.

devtool/main.py[86-99]
devtool/renovate.py[57-67]
renovate.json5[139-155]
deps/rpm/rpms.lock.yaml[1-4]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`renovate.json5` no longer contains the `rpm-lockfile` manager configuration, which disables Renovate updates for RPM lockfiles under `deps/rpm/**`.

Additionally, `renovate.json5` is explicitly a generated file; the generator (`devtool/renovate.py` via `devtool gen renovate.json5`) still emits an `rpm-lockfile` block. This mismatch means any future regeneration will unexpectedly re-add the removed config.

## Issue Context
- `devtool gen renovate.json5` writes `renovate.json5` from `renovate_json()`.
- `renovate_json()` currently includes an `rpm-lockfile` manager configuration.
- The repo contains RPM lockfiles (e.g., `deps/rpm/rpms.lock.yaml`) that this manager is meant to cover.

## Fix Focus Areas
- renovate.json5[120-156]
- devtool/renovate.py[57-67]
- devtool/__main__.py[86-99]

## Suggested fix
1. Regenerate `renovate.json5` from `devtool/renovate.py` (run `devtool gen renovate.json5`) and commit the resulting file so it includes the `rpm-lockfile` manager again.
2. If RPM updates are intentionally being removed, also remove the `"rpm-lockfile": {...}` block from `devtool/renovate.py` so the generator and generated file stay consistent.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Pip regen hint stale 🐞 Bug ⚙ Maintainability

Description

deps/pip/requirements.in now documents make pip-requirements, but list_pip_packages still
instructs users to regenerate with an old uv pip compile ... -o requirements.txt command when
requirements.txt is stale. This misdirects debugging and references the wrong paths/flags for the
new workflow.

Code

deps/pip/requirements.in[R3-4]

# Add packages here and regenerate requirements.txt with:
-#   uv pip compile --generate-hashes requirements.in -o requirements.txt
+#   make pip-requirements

Relevance

⭐⭐⭐ High

Team updates regen instructions/messages (Makefile workflow in PR69; generated-files error
reproduction text in PR139).

PR-#69
PR-#139

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The PR changes the documented regeneration command to make pip-requirements and updates the
Makefile/requirements header to new uv flag forms, but the devtool error message still points users
to the old direct uv pip compile ... -o requirements.txt invocation, creating an inconsistent and
misleading workflow.

deps/pip/requirements.in[3-5]
Makefile[59-66]
devtool/software_list.py[318-374]
PR-#61

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`deps/pip/requirements.in` and the `Makefile` now establish `make pip-requirements` as the canonical regeneration path, but `devtool/software_list.py:list_pip_packages()` still raises an error telling users to run `uv pip compile ... -o requirements.txt`.

That message is now misleading (wrong working directory/paths and outdated flag usage), so anyone hitting that error will likely run the wrong command.

## Issue Context
- `requirements.in` says to regenerate via `make pip-requirements`.
- `Makefile` shows the actual `uv pip compile` invocation and output path.
- `list_pip_packages()` throws a `ValueError` with regeneration instructions when it can't find a direct dep in the resolved `requirements.txt`.

## Fix Focus Areas
- devtool/software_list.py[318-374]
- deps/pip/requirements.in[3-5]
- Makefile[59-66]

## Suggested fix
Update the `ValueError` message in `list_pip_packages()` to recommend either:
- `make pip-requirements`, or
- the exact equivalent command matching the Makefile (including `deps/pip/...` paths and `--output-file=...`).

Also ensure the message reflects the current file locations (`deps/pip/requirements.in` and `deps/pip/requirements.txt`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[chmeliik]

Pip regen hint stale

👍 fixed

[qodo-app-for-konflux-ci]

Code review by qodo was updated up to the latest commit 989c6af

[chmeliik]

Stale build lockfile

Indeed, but we've always had that problem and I don't think it's solvable. Mintmaker doesn't allow pybuild-deps as a postUpgradeCommand