Parser occasionally fails with "The specified key does not exist" due to early startup #3139
Description
🐞 Bug report
Describe the bug
In SecureCodeBox, we observed an intermittent issue where the parser job fails with the following error:
After investigating pod creation timestamps, we found that the parser and scanner pods were created at nearly the same time, indicating that the parser did not wait for the scanner job to complete before starting — which contradicts the expected execution order.
Steps To Reproduce
This is difficult to reproduce consistently, but occurs under the following general scenario:
- Deploy a scanner and a corresponding parser using SecureCodeBox
- Trigger a scan
- Occasionally, parser pod starts and fails with
The specified key does not exist - Use
kubectl get pods -o jsonpath='{range .items[*]}{.metadata.namespace}{"\t"}{.metadata.name}{"\t"}{.status.phase}{"\t"}{.metadata.creationTimestamp}{"\t"}{.status.containerStatuses[0].state.terminated.finishedAt}{"\n"}{end}' - Observe that parser pod and scanner pod were created nearly simultaneously
Expected behavior
Parser job should only start after the scanner job has completed successfully, and after the scan result file is fully uploaded to MinIO.
System (please complete the following information):
- secureCodeBox Version/Release: v4.15.0
- OS: Ubuntu 22.04 (Kubernetes cluster nodes)
- Kubernetes Version: v1.32.0
- Docker Version: 24.0.5
- Browser: N/A
Screenshots / Logs
Parser pod logs:
data: '<?xml version="1.0" encoding="UTF-8"?>\n' +
'<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</
Pod creation timestamps:
kubectl get pods -o jsonpath='{range .items[*]}{.metadata.namespace}{"\t"}{.metadata.name}{"\t"}{.status.phase}{"\t"}{.metadata.creationTimestamp}{"\t"}{.status.containerStatuses[0].state.terminated.finishedAt}{"\n"}{end}'
default parse-test-20250707074054-1752796800-vndjq-ht59g Failed 2025-07-18T00:00:39Z 2025-07-18T00:00:43Z
default parse-test-20250707074054-1752796800-vndjq-lthz7 Failed 2025-07-18T00:00:00Z 2025-07-18T00:00:05Z
default parse-test-20250707074054-1752796800-vndjq-mb88s Failed 2025-07-18T00:01:23Z 2025-07-18T00:01:27Z
default parse-test-20250707074054-1752796800-vndjq-xl27p Failed 2025-07-18T00:00:15Z 2025-07-18T00:00:19Z
default parse-test-20250707074152-1752541200-wqsl4-mbxm5 Succeeded 2025-07-15T01:03:08Z 2025-07-15T01:03:13Z
default parse-test-20250707074152-1752627600-jslms-6g885 Succeeded 2025-07-16T01:03:15Z 2025-07-16T01:03:21Z
default parse-test-20250707074152-1752714000-6vd8z-z42hp Succeeded 2025-07-17T01:03:08Z 2025-07-17T01:03:13Z
default parse-test-20250707074152-1752800400-q6kpn-6sdch Failed 2025-07-18T01:00:14Z 2025-07-18T01:00:18Z
default parse-test-20250707074152-1752800400-q6kpn-cdvpb Failed 2025-07-18T01:00:00Z 2025-07-18T01:00:04Z
default parse-test-20250707074152-1752800400-q6kpn-cvgh6 Failed 2025-07-18T01:00:38Z 2025-07-18T01:00:42Z
default parse-test-20250707074152-1752800400-q6kpn-kswsl Failed 2025-07-18T01:01:22Z 2025-07-18T01:01:26Z
default scan-test-20250707074054-1752624000-vzvc9-rh4qc Succeeded 2025-07-16T00:00:00Z 2025-07-16T00:02:42Z
default scan-test-20250707074054-1752710400-5mcnt-pd8lq Succeeded 2025-07-17T00:00:00Z 2025-07-17T00:02:52Z
default scan-test-20250707074054-1752796800-dm2tm-25rw7 Succeeded 2025-07-18T00:00:00Z 2025-07-18T00:02:39Z
default scan-test-20250707074152-1752627600-7x5zh-6pw7n Succeeded 2025-07-16T01:00:00Z 2025-07-16T01:03:12Z
default scan-test-20250707074152-1752714000-phvgz-4bt4n Succeeded 2025-07-17T01:00:00Z 2025-07-17T01:03:05Z
default scan-test-20250707074152-1752800400-cs8cp-jtjwm Succeeded 2025-07-18T01:00:00Z 2025-07-18T01:03:10Z
Additional context
This suggests that the job controller may incorrectly trigger the parser job before the scanner has completed, or before the result file has been fully uploaded to MinIO.
Possible solutions or mitigation strategies:
- Add additional checks in controller logic to ensure scanner job has
status: Succeeded - Let parser jobs retry access to expected MinIO keys
- Add native file-existence checks within parser startup logic
Let us know if a reproduction setup or further logs are needed. Thank you!