Skip to content

TS/runtime-5: align virtual/real time in all types of scheduled task category#10305

Open
feiyang3cat wants to merge 1 commit into
temporalio:mainfrom
feiyang3cat:ts/patch-5
Open

TS/runtime-5: align virtual/real time in all types of scheduled task category#10305
feiyang3cat wants to merge 1 commit into
temporalio:mainfrom
feiyang3cat:ts/patch-5

Conversation

@feiyang3cat
Copy link
Copy Markdown
Contributor

@feiyang3cat feiyang3cat commented May 18, 2026

What changed?

  • mutable_state_impl.go
    • AddTasks: drop-check gate widened to all CategoryTypeScheduled tasks (was timers only); comparison
      uses ms.Now() (virtual) instead of wall.
    • New ToRealTime(virtual) → wall centralizes the conversion.
    • SetSpeculativeWorkflowTaskTimeoutTask converts to wall via ToRealTime before enqueuing.
    • accumulatedSkippedDuration simplified to nil-safe proto getter.
  • timer_queue_{active,standby}_task_executor.go: heartbeat-dedup compares wall-vs-wall via
    ToRealTime(heartbeatTimeoutVis) (was mixed frames), and use ms.Now instead of t.Now for activity timeout/user timer time check
  • task_generator.go: TODO marking ChasmTaskPure / HSM-timer regeneration as not yet supported.
  • interfaces/mutable_state.go: adds ToRealTime; documents virtual-frame contract on AddTasks,
    Now, SetSpeculativeWorkflowTaskTimeoutTask.

Why?

Under time skipping, virtual time inside mutable state can run ahead of wall time, but scheduled-task
firing still runs on wall time. Each crossing needs explicit conversion:

  • Tasks leaving mutable state for the scheduled-task queues must be converted to wall time.
  • Validation of generated/fired tasks must reference virtual (mutable-state) time.

Before this change, virtual and wall values were silently compared — harmless pre-skipping, but now lets
the wrong tasks get dropped/kept and stale heartbeat tasks pass dedup.

How did you test it?

  • built
  • run locally and tested manually
  • covered by existing tests
  • added new unit test(s)
  • added new functional test(s)

@feiyang3cat feiyang3cat requested review from a team as code owners May 18, 2026 01:59
@feiyang3cat feiyang3cat changed the title ts/patch-5: align virtual time and real time in timer task execution ts/patch: align virtual time and real time in timer task execution May 18, 2026
@feiyang3cat feiyang3cat force-pushed the ts/patch-5 branch 3 times, most recently from 3d50bdf to a3f1761 Compare May 18, 2026 03:21
@feiyang3cat feiyang3cat changed the title ts/patch: align virtual time and real time in timer task execution ts/patch: align virtual time and real time in scheduled task execution May 18, 2026
@feiyang3cat feiyang3cat force-pushed the ts/patch-5 branch 2 times, most recently from 4ab596d to cca239e Compare May 18, 2026 05:13
@feiyang3cat feiyang3cat changed the title ts/patch: align virtual time and real time in scheduled task execution TS/patch: align virtual time and real time in scheduled task execution May 18, 2026
@feiyang3cat feiyang3cat changed the title TS/patch: align virtual time and real time in scheduled task execution TS/runtime: align virtual/real time in all types of scheduled task category May 18, 2026
@feiyang3cat feiyang3cat changed the title TS/runtime: align virtual/real time in all types of scheduled task category TS/runtime-5: align virtual/real time in all types of scheduled task category May 18, 2026
@feiyang3cat feiyang3cat force-pushed the ts/patch-5 branch 6 times, most recently from 8c53840 to d848ebf Compare May 18, 2026 20:54
// the overall solution is to attempt to generate a new activity timer task whenever the
// task passed in is safe to be throw away.
referenceTime := t.Now()
actionFn := func(ctx context.Context, wfContext historyi.WorkflowContext, mutableState historyi.MutableState, _ historyi.ReleaseWorkflowContextFunc) (any, error) {
Copy link
Copy Markdown
Contributor Author

@feiyang3cat feiyang3cat May 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs a bit attention

for all executions (the majority won't have timeskipping), the reference time is still delayed to when actionFn runs, and this may be a few us/ms delay depending on the cache miss/hit. right now I judge it as safe and no correctness risks

there is another way to do this is to change the referenceTime only when the mutableState has TimeSkippingInfo set, and I am not taking this path now for simplicity

// real wall-clock, so convert here — callers outside MutableState don't see the virtual
// vs. real distinction. The CategoryTypeScheduled drop-check above runs first so it
// compares virtual-vs-virtual (now is also virtual).
if skip > 0 && category == tasks.CategoryTimer {
Copy link
Copy Markdown
Contributor Author

@feiyang3cat feiyang3cat May 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

though we add other scheduled timer task types in, the executors of archival and speculative tasks don't validate the fire time, so for those tasks the executor files are untouched

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant