feature: check for existing updates with same src url

[ryantm]

fixes #315

@rhendric interested in reviewing this?

[rhendric]

Wouldn't this solution mean that a PR for firefox would prevent PRs for firefox-beta-bin or any other Firefox variation that uses the same wrapper?

Maybe a better check would be to do something like hashing the diffs created?

[ryantm]

@rhendric Good point, how about how it is now where it checks for a duplicate src URL?

[rhendric]

Yeah, that sounds like it would work in principle!

But now there are implementation difficulties; based on my testing, GitHub's search API is too sloppy to match multiword phrases exactly, or even a single URL. Check out how

curl -s -H 'Accept: application/vnd.github.v3+json, application/vnd.github.squirrel-girl-preview+json' "https://api.github.com/search/issues?q=%22https://github.com/grafana/grafana/releases/tag/v3.4.0%22+repo:nixos/nixpkgs"

returns a result in which the URL mentioned is actually https://github.com/grafana/grafana-image-renderer/releases/tag/v3.4.0, even though I used quotes in the search.

That's part of why I suggested a hash; I hope that a string of only alphanumeric characters is safe to search on. We can hash the source URL, and maybe hide it in <!-- --> to minimize clutter for humans?

[rhendric]

This needs a repo:nixos/nixpkgs too, in whatever form it ultimately takes.

[rhendric]

Suggested change

	<> openPRReport searchResult))
	<> openPRReport srcUrlSearchResult))

[rhendric]

This doesn't look right to me—checking for open PRs that target the old src seems useless at best.

I figure we want to do this new check after the rewrites have run in both the hasUpdateScript and not hasUpdateScript cases, so maybe it should be pulled out of checkExistingUpdate and put in its own function.

[zowoq]

Thought of a possible alternative to this, get the src.url of the package when it is read into the supervisor database and then try to drop the duplicates from the update queue?

pseudocode example for getting the src.url:

diff --git a/hosts/build02/supervisor.py b/hosts/build02/supervisor.py
index 68e0c160..ee6f556c 100644
--- a/hosts/build02/supervisor.py
+++ b/hosts/build02/supervisor.py
@@ -692,6 +692,23 @@ class FetcherDataWatcher:
                         return
                 self._storage.delete_fetcher_run(name, run_started)
 
+    def get_src_url(attr_path: str) -> str:
+        try:
+            prefix = "/var/cache/nixpkgs-update/worker/nixpkgs"
+            nix = subprocess.run(
+                [
+                    "nix",
+                    "eval",
+                    "-f",
+                    f"{prefix}",
+                    f"{attr_path}.src.url",
+                ],
+            )
+            output = nix.stdout.strip()
+            return output.removeprefix(prefix)
+        except subprocess.CalledProcessError:
+            return "no_src_url"
+
     def _read_fetcher_lines(
         self, file: TextIO
     ) -> Generator[None, None, list[tuple[str, str]]]:
@@ -716,7 +733,7 @@ class FetcherDataWatcher:
                 continue
             match line.strip().split(" ", maxsplit=1):
                 case [attr_path, payload]:
-                    entries.append((attr_path, payload))
+                    entries.append((attr_path, payload, get_src_url(attr_path)))
                 case _:
                     print(f"Unexpected line in {file.name!r}: {line!r}")