Replace os.path.join with pathlib by berland · Pull Request #13583 · equinor/ert

berland · 2026-05-19T14:24:03Z

Issue
Resolves https://docs.astral.sh/ruff/rules/os-path-join/

Approach
Try limiting the changes to only replacing os.path.join and accepting that there are lots of obvious pathlib improvements that still can be done to the changed lines - leave these for later for easier review. The main motive is to be able to add PTH118 as a ruff rule as soon as possible, and then do more.

PR title captures the intent of the changes, and is fitting for release notes.
Added appropriate release note label
Commit history is consistent and clean, in line with the contribution guidelines.
Make sure unit tests pass locally after every commit (git rebase -i main --exec 'just rapid-tests')

When applicable

When there are user facing changes: Updated documentation
New behavior or changes to existing untested code: Ensured that unit tests are added (See Ground Rules).
Large PR: Prepare changes in small commits for more convenient review
Bug fix: Add regression test for the bug
Bug fix: Add backport label to latest release (format: 'backport release-branch-name')

codecov-commenter · 2026-05-20T09:58:17Z

Codecov Report

❌ Patch coverage is 98.48485% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 89.52%. Comparing base (c81c829) to head (cc8ce1c).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/ert/config/ert_config.py	90.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #13583   +/-   ##
=======================================
  Coverage   89.52%   89.52%           
=======================================
  Files         464      464           
  Lines       32775    32779    +4     
=======================================
+ Hits        29343    29347    +4     
  Misses       3432     3432

Flag	Coverage Δ
cli-tests	`35.76% <34.84%> (-0.01%)`	⬇️
fuzz	`43.94% <34.84%> (-0.01%)`	⬇️
gui-tests	`59.81% <40.90%> (-0.02%)`	⬇️
performance-and-unit-tests	`78.07% <83.33%> (+<0.01%)`	⬆️
test	`45.41% <42.42%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/_ert/forward_model_runner/io/__init__.py	`92.85% <100.00%> (+0.54%)`	⬆️
src/_ert/forward_model_runner/reporting/file.py	`98.58% <100.00%> (ø)`
src/ert/config/_observations.py	`94.90% <100.00%> (+<0.01%)`	⬆️
src/ert/config/observation_config_migrations.py	`94.02% <100.00%> (-0.02%)`	⬇️
src/ert/config/parsing/config_schema_item.py	`96.31% <100.00%> (+0.07%)`	⬆️
src/ert/config/parsing/lark_parser.py	`99.12% <100.00%> (ø)`
src/ert/plugins/hook_implementations/jobs.py	`100.00% <100.00%> (ø)`
src/ert/run_models/everest_run_model.py	`91.68% <100.00%> (ø)`
src/ert/storage/migration/to8.py	`95.45% <100.00%> (ø)`
src/everest/config/everest_config.py	`94.45% <100.00%> (-0.02%)`	⬇️
... and 4 more

Copilot

Pull request overview

This PR replaces os.path.join(...) usages with pathlib.Path-based joining across ERT/Everest runtime code and a broad set of tests, to enable adding Ruff rule PTH118 (discouraging os.path.join) without large follow-up noise.

Changes:

Refactor many path constructions from os.path.join to Path(...) / ... (and related Path(...) conversions) in both source and tests.
Adjust tests to expect string vs Path values where APIs still operate on str paths.
Enable Ruff rule PTH118 in pyproject.toml.

Reviewed changes

Copilot reviewed 41 out of 41 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/everest/utils/utils.py	Switches `relpath` helper to return a `Path`.
tests/everest/test_templating.py	Uses `Path` joining for loading `template_render.py`.
tests/everest/test_res_initialization.py	Updates expected `install_data` arglist values (drops `./` prefix).
tests/everest/test_repo_configs.py	Reworks repo config discovery using `Path.rglob`.
tests/everest/test_logging.py	Converts log/output path handling to `Path`.
tests/everest/test_everest_output.py	Uses `Path` for saved config path and move destination.
tests/everest/test_config_file_loader.py	Aligns path expectations using `Path` joins and string conversions.
tests/ert/unit_tests/run_models/conftest.py	Uses `Path.mkdir(parents=True)` for dummy runpath directory structure.
tests/ert/unit_tests/resources/test_templating.py	Uses `Path` joining for loading `template_render.py`.
tests/ert/unit_tests/resources/test_shell.py	Introduces `SHELL_SCRIPTS` `Path` and updates script imports to use it.
tests/ert/unit_tests/gui/tools/plot/conftest.py	Uses `Path` joining for test-data directory path.
tests/ert/unit_tests/forward_model_runner/test_forward_model_step.py	Converts executable paths to stringified `Path` joins.
tests/ert/unit_tests/config/test_forward_model.py	Updates expected executable path formatting with `Path`.
tests/ert/unit_tests/config/test_ert_config.py	Converts workflow/script temp paths to `Path` usage.
tests/ert/unit_tests/config/test_create_forward_model_json.py	Builds executable paths with `Path` and `str(...)`.
tests/ert/unit_tests/config/config_dict_generator.py	Converts generated config paths to `str(tmp / ...)`.
tests/ert/ui_tests/gui/conftest.py	Uses `Path` joining for copying poly example test-data.
tests/ert/ui_tests/cli/test_update_with_rft_observations.py	Uses `Path` joining for copying rft example test-data.
tests/ert/ui_tests/cli/test_cli.py	Uses `Path` joining for observation config absolute path.
tests/ert/conftest.py	Adds `Path` typing for `source_root` and updates test-data copy paths.
src/everest/config/validation_utils.py	Uses `Path` joining in path validation and absolute path creation.
src/everest/config/server_config.py	Computes session dir using `Path.resolve()` and `Path` joining.
src/everest/config/install_data_config.py	Uses `Path` joining for non-absolute install-data sources.
src/everest/config/everest_config.py	Uses `Path` joining for output/simulation dir computations.
src/ert/storage/migration/to8.py	Uses `Path` joining when listing observation keys directory.
src/ert/run_models/everest_run_model.py	Uses `Path` joining for `.res_runpath_list` path.
src/ert/resources/shell_scripts/symlink.py	Uses `Path` for target existence checks prior to symlink creation.
src/ert/resources/shell_scripts/move_file.py	Accepts `Path` inputs and uses `Path` operations internally.
src/ert/resources/shell_scripts/delete_directory.py	Uses `Path` for existence checks and joined file paths in traversal.
src/ert/resources/shell_scripts/copy_file.py	Uses `Path` joining to build target file paths.
src/ert/resources/shell_scripts/copy_directory.py	Uses `Path` joining to build target directory path when target is a dir.
src/ert/resources/shell_scripts/careful_copy_file.py	Uses `Path` joining to build target file paths.
src/ert/plugins/hook_implementations/jobs.py	Converts rendered job directories to `Path` and iterates with `iterdir()`.
src/ert/config/parsing/lark_parser.py	Uses `Path` for include-directory handling and absolute include resolution.
src/ert/config/parsing/config_schema_item.py	Resolves PATH/EXECUTABLE schema tokens using `Path` operations.
src/ert/config/observation_config_migrations.py	Uses `Path` for resolving OBS/INDEX file paths before storing as string.
src/ert/config/ert_config.py	Uses `Path` for default paths and directory file enumeration; converts to `str` where needed.
src/ert/config/_observations.py	Accepts `Path` for CSV filename and resolves relative paths via `Path`.
src/_ert/forward_model_runner/reporting/file.py	Uses `Path` joining when reporting stderr file location.
src/_ert/forward_model_runner/io/init.py	Reworks executable checks to use `Path` for candidate construction and checks.
pyproject.toml	Enables Ruff rule `PTH118`.

berland · 2026-05-26T08:49:07Z

+                directory / file
+                for file in directory.iterdir()
+                if (directory / file).is_file()


berland · 2026-05-26T08:50:35Z

        elif not path.isabs(content_dict[ConfigKeys.RUNPATH_FILE]):
-            content_dict[ConfigKeys.RUNPATH_FILE] = path.normpath(
-                path.join(config_dir, content_dict[ConfigKeys.RUNPATH_FILE])
+            content_dict[ConfigKeys.RUNPATH_FILE] = str(
+                Path(config_dir) / content_dict[ConfigKeys.RUNPATH_FILE]
            )


berland · 2026-05-26T08:57:20Z

 careful_copy_file = import_from_location(
    "careful_copy",
-    os.path.join(SOURCE_DIR, "src/ert/resources/shell_scripts/careful_copy_file.py"),
+    SHELL_SCRIPTS / "careful_copy_file.py",
 ).careful_copy_file
 mkdir = import_from_location(
    "make_directory",
-    os.path.join(SOURCE_DIR, "src/ert/resources/shell_scripts/make_directory.py"),
+    SHELL_SCRIPTS / "make_directory.py",
 ).mkdir
 careful_copy_file = import_from_location(
    "careful_copy",
-    os.path.join(SOURCE_DIR, "src/ert/resources/shell_scripts/careful_copy_file.py"),
+    SHELL_SCRIPTS / "careful_copy_file.py",
 ).careful_copy_file


removed the duplicate

One change in test directories as we do not need to conserve "./" as a path prefix, this was present in tests just as an artifact of how os.path.join works when joining with ".".

codspeed-hq · 2026-05-26T09:47:10Z

Merging this PR will not alter performance

✅ 36 untouched benchmarks

_{Comparing berland:ruff_pth118 (cc8ce1c) with main (c81c829)}

berland changed the title ~~Use pathlib in forward_model_runner and ert/config~~ Use pathlib in forward_model_runner and src/ert May 19, 2026

berland changed the title ~~Use pathlib in forward_model_runner and src/ert~~ Replace os.path.join with pathlib May 19, 2026

berland force-pushed the ruff_pth118 branch 2 times, most recently from c18f425 to 2a060c4 Compare May 20, 2026 09:47

berland marked this pull request as ready for review May 20, 2026 10:25

berland self-assigned this May 20, 2026

berland added this to SCOUT May 20, 2026

berland moved this to Ready for Review in SCOUT May 20, 2026

berland added the release-notes:maintenance Automatically categorise as maintenance change in release notes label May 20, 2026

berland requested a review from Copilot May 26, 2026 06:52

Copilot started reviewing on behalf of berland May 26, 2026 06:52 View session

Copilot AI reviewed May 26, 2026

View reviewed changes

berland added 5 commits May 26, 2026 10:55

Use pathlib in forward_model_runner and ert/config

baf5603

Use pathlib over os.path.join in src/ert

a7c58bd

Use pathlib over os.path.join in everest

69c8fcf

One change in test directories as we do not need to conserve "./" as a path prefix, this was present in tests just as an artifact of how os.path.join works when joining with ".".

Use pathlib over os.path.join in test files. Enforce via ruff

65959ef

Remove duplicate loading of careful_copy_file

cc8ce1c

berland force-pushed the ruff_pth118 branch from 2a060c4 to cc8ce1c Compare May 26, 2026 08:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace os.path.join with pathlib#13583

Replace os.path.join with pathlib#13583
berland wants to merge 5 commits into
equinor:mainfrom
berland:ruff_pth118

berland commented May 19, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented May 20, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

berland May 26, 2026

Uh oh!

berland May 26, 2026

Uh oh!

berland May 26, 2026

Uh oh!

berland May 26, 2026

Uh oh!

codspeed-hq Bot commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

berland commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

When applicable

Uh oh!

codecov-commenter commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

berland May 26, 2026

Choose a reason for hiding this comment

Uh oh!

berland May 26, 2026

Choose a reason for hiding this comment

Uh oh!

berland May 26, 2026

Choose a reason for hiding this comment

Uh oh!

berland May 26, 2026

Choose a reason for hiding this comment

Uh oh!

codspeed-hq Bot commented May 26, 2026

Merging this PR will not alter performance

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

berland commented May 19, 2026 •

edited

Loading

codecov-commenter commented May 20, 2026 •

edited

Loading