Skip to content

BJQ: initial state restore#28181

Open
mismithhisler wants to merge 6 commits into
f-batch-job-queuefrom
f-bjq-state-restore
Open

BJQ: initial state restore#28181
mismithhisler wants to merge 6 commits into
f-batch-job-queuefrom
f-bjq-state-restore

Conversation

@mismithhisler

Copy link
Copy Markdown
Member

Description

These changes introduce state restore functionality to the DPQ. With these changes, a server can be stopped and restarted, and is able to restore tenant usage and identify batch jobs that a previous queue may have been waiting for placement.

Testing & Reproduction steps

Links

Contributor Checklist

  • Changelog Entry If this PR changes user-facing behavior, please generate and add a
    changelog entry using the make cl command.
  • Testing Please add tests to cover any new functionality or to demonstrate bug fixes and
    ensure regressions will be caught.
  • Documentation If the change impacts user-facing functionality such as the CLI, API, UI,
    and job configuration, please update the Nomad product documentation, which is stored in the
    web-unified-docs repo. Refer to the web-unified-docs contributor guide for docs guidelines.
    Please also consider whether the change requires notes within the upgrade
    guide
    . If you would like help with the docs, tag the nomad-docs team in this PR.
  • LLM Usage If an LLM was used to generate any code, please ensure and confirm you have read
    and followed the AI usage guide.

Reviewer Checklist

  • Backport Labels Please add the correct backport labels as described by the internal
    backporting document.
  • Commit Type Ensure the correct merge method is selected which should be "squash and merge"
    in the majority of situations. The main exceptions are long-lived feature branches or merges where
    history should be preserved.
  • Enterprise PRs If this is an enterprise only PR, please add any required changelog entry
    within the public repository.
  • If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

@mismithhisler mismithhisler self-assigned this Jun 25, 2026
@mismithhisler mismithhisler requested review from a team as code owners June 25, 2026 19:06
@mismithhisler mismithhisler changed the title initial state restore BJQ: initial state restore Jun 25, 2026
Comment thread nomad/queues/priority_queue.go

@allisonlarson allisonlarson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a good start! There are quite a few TODOs that I would be happy to have resolved/wait on a larger PR, but I'm also okay with getting it in so that we can flush things out that way.

Comment thread nomad/queues/dynamic_priority_queue.go Outdated
// When checking for workload placements, we never want to actually block
// in SetEnabled, but it's also entirely possible a queue eval is blocked and
// waiting to be placed from a previous DPQ placement. If that happens
// we should allow it to be processed via the initWorkload chan.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, initWorkload chan doesn't look like it made the cut

Comment thread nomad/queues/dynamic_priority_queue.go Outdated
}

if !placed {
// if this workload has been placed, we will push it to the queue.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if this workload has not been placed?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just going to delete this comment because it's not super helpful 😄

Comment thread nomad/queues/priority_queue.go
Comment thread nomad/queues/dynamic_priority_queue.go Outdated
// Similar to waitForPlacement, isSchedulingComplete will record usage in the event an
// actual placement occurred.
//
// TODO see if can dedup with waitForPlacement

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is a TODO, but yeah the duplication of waitingForPlacement is high. What do you think about trying to address that a bit, even just for the goal of really highlighting the difference between them?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants