Steps to reproduce
Title
Retry schedule exits immediately on FAILED_TO_START_DUE_TO_NO_CAPACITY
Description
When a run with a retry schedule fails to start due to no available capacity, it exits immediately with RETRY_LIMIT_EXCEEDED instead of honoring the retry duration.
Steps to Reproduce
- Create a run with a retry schedule (e.g., 2 hours) and non existing hardware specs.
- Schedule the run for a future time (in UTC)
- Job fails with
FAILED_TO_START_DUE_TO_NO_CAPACITY
- Run immediately terminates with
RETRY_LIMIT_EXCEEDED
Expected Behavior
The run should retry for the configured duration (2 hours) when capacity becomes available.
Actual Behavior
Run exits immediately without retrying.
Logs
02/25/2026 07:20 Run main/kimi-k25 Run status changed PENDING -> SUBMITTED
02/25/2026 07:20 Job main/kimi-k25-0-0 Job status changed SUBMITTED -> TERMINATING. Termination reason: FAILED_TO_START_DUE_TO_NO_CAPACITY
02/25/2026 07:20 Job main/kimi-k25-0-0 Job status changed TERMINATING -> FAILED
02/25/2026 07:20 Run main/kimi-k25 Run status changed SUBMITTED -> TERMINATING. Termination reason: RETRY_LIMIT_EXCEEDED
02/25/2026 07:20 Run main/kimi-k25 Run status changed TERMINATING -> PENDING
Environment
- dstack version: 0.20.10
- Deployment: [self-hosted/kubernetes]
Actual behaviour
No response
Expected behaviour
No response
dstack version
0.20.10
Server logs
Additional information
No response
Steps to reproduce
Title
Retry schedule exits immediately on
FAILED_TO_START_DUE_TO_NO_CAPACITYDescription
When a run with a retry schedule fails to start due to no available capacity, it exits immediately with
RETRY_LIMIT_EXCEEDEDinstead of honoring the retry duration.Steps to Reproduce
FAILED_TO_START_DUE_TO_NO_CAPACITYRETRY_LIMIT_EXCEEDEDExpected Behavior
The run should retry for the configured duration (2 hours) when capacity becomes available.
Actual Behavior
Run exits immediately without retrying.
Logs
Environment
Actual behaviour
No response
Expected behaviour
No response
dstack version
0.20.10
Server logs
Additional information
No response