0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00

Compare commits

...

16 commits

Author SHA1 Message Date
Leo
909b0e5e73 docs(harness): reword the late-suspend limits to pass Vale
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 22:56:17 -05:00
Leo
372c2da4aa test(libsql): resume a deferred tool that suspends after its run finished; document the limits
Adds a LibSQL case for the durable and evented engines: the deferred tool suspends
only after the stream drained and the run's claim was released, then the task parks
with its payload, resume completes it, and the thread holds the result. Documents
when a late suspend works and when it still fails. (#30182)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 22:51:58 -05:00
Leo
d7c3f7f43c test(core): answer only the approval-resume scenario's own model requests
The suite shares one mock model server. The test before this one ends as
soon as its second deferred tool parks, while its agent is still waking to
report that task to the model. When that request lands after the next test
registered its fixtures, it took the sequenceIndex 0 reply, so this test's
own first turn got "Reply 1." instead of the tool calls, finished without
an approval, and failed on the evented engine in CI.

Scope the fixtures to this test's prompt. aimock counts the sequence per
predicate, so a stray request no longer advances it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 22:35:22 -05:00
Leo
dd5ef57d9e test(core): fail held-save takeover scenarios fast when the save never starts
The three scenarios that hold a memory save until a successor takes over
waited on the held save with no bound. A regression that fails the task
before it saves left them hanging until the 30 s test timeout. They now
fail after 2 s with "save never reached storage".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 21:21:30 -05:00
Leo
adc10a92bf Merge remote-tracking branch 'origin/main' into fix/30182-durable-suspend-after-finish
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 21:03:16 -05:00
Leo
bf00cf0f51 test(core): pin a late background suspend after a same-turn approval resume
On the durable engine the task parks; on the evented engine it fails with
"lost its execution lease", as it does on main. Drop a changeset sentence
about a bug only an earlier version of this PR had, and note the evented
limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 20:47:03 -05:00
Leo
4b12cdf453 fix(core): keep the strict check for lease-backed late background suspends
A lease keeps no record once released, so an empty lease can't tell the
finished run's own release from a recovery that claimed, finished and
released the run since, and no storage fence stands behind the write.
Lease-backed fences now always use verify(), and a late background suspend
on a lease-backed claim uses the strict check once the fence is gone too.
The relaxed post-settlement rule needs a storage generation.

Removes the release counting that rule made dead, pins the strict check for
foreground suspends, and adds the lease-only scenario.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 19:57:42 -05:00
Leo
843ea58f9e fix(core): accept a late background write unless a successor claimed the run
With ownership in storage, verifyNotSuperseded now makes one ownership read
and refuses only when another execution claimed the run (a newer generation
or a missing record). It no longer tracks releases, inspects released
records or waits for a release's reply, so a claim release whose reply never
came no longer leaves a suspending background task running. The storage
fence that runFencedToClaim puts on the write still rejects it once a
successor raised the memory fence.

Lease-backed claims have no generation and no storage fence, so they keep
the strict check until this process starts releasing the claim. Accepting a
lapsed lease there would let a stale write land after a recovery claimed and
released the run.

Adds the held-release-reply scenarios on durable and evented, a two-sibling
deferred task scenario, and the lapsed-lease recovery case; updates the unit
tests that pinned an expired storage claim as refused. (#30182)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 18:35:55 -05:00
Leo
0601c44e0a fix(core): accept a late background write when the heartbeat sees the release mid-check
After two failed ownership reads, the last read of verifyNotSuperseded could
see the claim held while another worker released it and the heartbeat marked
the fence lost. The check then threw that loss though storage showed this
exact claim released. A loss recorded while that read is on its way now
triggers one more read: the same claim released is accepted, a successor is
still refused.

Adds LibSQL-backed cases for a late suspension and a late result save that
reach the database after a successor raised the memory fence, on durable and
evented agents.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 17:11:19 -05:00
Leo
d0f80f1ec0 fix(core): fence late background result writes and count only confirmed releases
A background task's onResult and onExecution memory flushes now run under
the finished run's claim fence, like the late suspend, so storage rejects a
save that lands after a successor raised the memory fence.

The final check of verifyNotSuperseded reads the claim once and classifies a
matching released record as a release instead of falling back to a
release-blind strict check, so another worker's release between two reads no
longer marks the fence lost. Storage release now returns whether the claim
was still held, and only a confirmed release counts as released.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 16:14:05 -05:00
Leo
82d76c5083 fix(core): accept a late background write after a release only storage shows
A late background suspend failed as superseded when the shared claim was
released by another evented worker, or when storage applied this
process's release but its response failed. A post-settlement check that
fails without a confirmed release now reads the ownership record: the
same owner and generation with the lease cleared counts as released and
gets the post-release rule. A successor is refused; a lapsed lease, a
lease-backed claim or a failed read stays strict. Foreground verify() is
unchanged.

Adds the two-worker regression, a lost-release-ack scenario and its
takeover case, and a late onResult during cleanup. (#30182)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 14:38:25 -05:00
Leo
697e8f3dca fix(core): don't mark a late background suspend lost while the finished run's release is in flight
A post-settlement ownership check that read the claim after storage cleared its
lease, but before the release call returned, ran the strict check and marked the
execution lost though nobody took the run. The fence now records a release when
it starts; a check that fails during it waits for the release and then applies
the post-release supersession rule, staying strict if the release failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 13:51:31 -05:00
Leo
d1766814a4 fix(core): fence a late background suspension to the finished run's claim
A late background suspend passed its ownership check and then saved to
memory unfenced, because a run that settled as owner stops fencing its own
writes. A successor that claimed the run and raised the memory fence while
the save was in flight still found the old suspension written and the task
suspended. The suspension's memory writes now carry the finished run's own
claim, so storage rejects them once a newer claim raised the memory fence.

On a shared evented transport, the worker's participant fence stayed
registered between releasing the claim and stopping the participant, so a
late suspend in that gap ran the strict check and failed as superseded.
releaseSharedClaim now marks that participant fence as released.

Adds unit tests for verifyNotSuperseded on the storage and lease backends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 13:23:30 -05:00
Leo
1fc1bf8b07 fix(core): let a late background suspend through while the finished run is still cleaning up
A finished run releases its claim, then retires its memory fence. While that
retirement was pending, the execution was still registered locally, so the
late suspend's check verified the claim it had just released and failed with
DURABLE_AGENT_EXECUTION_SUPERSEDED though no other execution had claimed the
run. Once the claim is released, the local check now accepts the write until
another execution claims the run, matching the check used after the fence is
unregistered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 12:52:33 -05:00
Leo
512adefe2d test(core): prove a late background suspend can't write after a successor claims the run
The #30182 fix lets a background task suspend after its run released its
claim. This scenario covers the other side: once another execution claims
the run, the old execution's suspension is refused with the superseded
error, nothing reaches memory, and the successor's claim is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 12:36:57 -05:00
Leo
9999cc549e fix(core): let durable background tasks with memory suspend after the run finished
A deferred background tool (or a sub-agent reaching a tool that needs
approval) on a durable agent with memory failed with
DURABLE_AGENT_EXECUTION_UNVERIFIED when it suspended after the run that
dispatched it had finished. The background executor runs the origin
run's suspend bookkeeping first, and its memory save used the strict
ownership check, which cannot verify a claim the finished run already
released. The task's own suspend never ran.

The background path now saves that bookkeeping with the post-settlement
check (assertExecutionNotSuperseded), the same one background-task
results already use. It still stops writes once another execution has
claimed the run. Foreground suspends keep the strict check.

Fixes #30182

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 12:23:17 -05:00
9 changed files with 2555 additions and 145 deletions