Skip to content

prefect_submitit integration#3202

Open
vineetbansal wants to merge 15 commits into
mainfrom
vb/prefect_submitit
Open

prefect_submitit integration#3202
vineetbansal wants to merge 15 commits into
mainfrom
vb/prefect_submitit

Conversation

@vineetbansal

Copy link
Copy Markdown
Collaborator

Summary of Changes

>> Provide context and a description of your changes here. Make sure to reference any associated issues. <<

Requirements

Note: If you are an external contributor, you will see a comment from @buildbot-princeton. This is solely for the maintainers.

@vineetbansal vineetbansal marked this pull request as draft April 16, 2026 15:50
@codecov

codecov Bot commented Apr 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.55%. Comparing base (9e20655) to head (8642662).

Files with missing lines Patch % Lines
src/quacc/wflow_tools/decorators.py 80.00% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3202      +/-   ##
==========================================
- Coverage   97.64%   97.55%   -0.09%     
==========================================
  Files          98       98              
  Lines        4282     4303      +21     
==========================================
+ Hits         4181     4198      +17     
- Misses        101      105       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vineetbansal vineetbansal marked this pull request as ready for review May 12, 2026 19:57
@vineetbansal

vineetbansal commented May 12, 2026

Copy link
Copy Markdown
Collaborator Author

@Andrew-S-Rosen - on tiger-arrk, I find that the IP address of prefect server as reported on the head node is different from what the compute nodes need to reach it. This is not prefect-submitit specific, but in general when using prefect as the workflow engine on slurm.

So technically, the examples of prefect using slurm that rely on a slurm setup phase should say something like:

    setup=[
        "module load ...",
        "conda activate...",
        f"export QUACC_VASP_PARALLEL_CMD='{vasp_parallel_cmd}'",
        "export PREFECT_API_URL=http://10.36.32.21:4200/api",
        "export PREFECT_SERVER_API_HOST=10.36.32.21",
    ],

@vineetbansal

Copy link
Copy Markdown
Collaborator Author

This PR will keep failing till our PR to prefect-submit is merged:
dexterity-systems/prefect-submitit#17

Comment thread src/quacc/wflow_tools/decorators.py Outdated
return task(_func, namespace=_func.__module__, **kwargs)
elif settings.WORKFLOW_ENGINE == "prefect":
from prefect import task
from prefect_submitit import task

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guard in case user does not have prefect_submitit

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use find_spec instead of a bare try: .. except ImportError.

@vineetbansal vineetbansal changed the title WIP: prefect_submitit prefect_submitit integration May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants