Skip to content

Operator Error messages about concurrent modifications #825

@malexmave

Description

@malexmave

🐞 Bug report

The operator keeps throwing error messages about "the object has been modified; please apply your changes to the latest version and try again". Seems like there is some concurrent modification going on somewhere? We should track this down.

Full error message with a bit of context around it:

2021-11-18T08:33:44.189+0100	INFO	controllers.execution.Scan	Matching ParseDefinition Found	{"scan_parse": "default/nmap-localhost", "ParseDefinition": "nmap-xml"}
2021-11-18T08:33:49.217+0100	INFO	controllers.execution.Scan	Found ScanCompletionHooks	{"ScanCompletionHooks": 1}
2021-11-18T08:33:49.227+0100	INFO	controllers.execution.Scan	Processing hook	{"hook": {"hookName":"cascading-scans","state":"Pending","priority":0,"type":"ReadOnly"}, "jobType": "read-only-hook"}
2021-11-18T08:33:49.231+0100	INFO	controllers.execution.Scan	Created job for hook	{"hook": {"hookName":"cascading-scans","state":"InProgress","jobName":"cascading-scans-nmap-localhost-fbt7r","priority":0,"type":"ReadOnly"}}
2021-11-18T08:33:49.239+0100	INFO	controllers.execution.Scan	Processing hook	{"hook": {"hookName":"cascading-scans","state":"Pending","priority":0,"type":"ReadOnly"}, "jobType": "read-only-hook"}
2021-11-18T08:33:49.252+0100	ERROR	controllers.execution.Scan	Unable to update Scan status	{"error": "Operation cannot be fulfilled on scans.execution.securecodebox.io \"nmap-localhost\": the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/mmaass/go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132
github.com/secureCodeBox/secureCodeBox/operator/controllers/execution/scans.(*ScanReconciler).executeHooks
	/Users/mmaass/Documents/Code/SecureCodeBox/operator/controllers/execution/scans/hook_reconciler.go:147
github.com/secureCodeBox/secureCodeBox/operator/controllers/execution/scans.(*ScanReconciler).Reconcile
	/Users/mmaass/Documents/Code/SecureCodeBox/operator/controllers/execution/scans/scan_controller.go:109
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99
2021-11-18T08:33:49.252+0100	ERROR	controller-runtime.manager.controller.scan	Reconciler error	{"reconciler group": "execution.securecodebox.io", "reconciler kind": "Scan", "name": "nmap-localhost", "namespace": "default", "error": "Operation cannot be fulfilled on scans.execution.securecodebox.io \"nmap-localhost\": the object has been modified; please apply your changes to the latest version and try again"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/mmaass/go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:302
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2
	/Users/mmaass/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/internal/controller/controller.go:216
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185
k8s.io/apimachinery/pkg/util/wait.UntilWithContext
	/Users/mmaass/go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99
2021-11-18T08:33:49.252+0100	INFO	controllers.execution.Scan	Processing hook	{"hook": {"hookName":"cascading-scans","state":"InProgress","jobName":"cascading-scans-nmap-localhost-fbt7r","priority":0,"type":"ReadOnly"}, "jobType": "read-only-hook"}
2021-11-18T08:33:49.263+0100	INFO	controllers.execution.Scan	Processing hook	{"hook": {"hookName":"cascading-scans","state":"InProgress","jobName":"cascading-scans-nmap-localhost-fbt7r","priority":0,"type":"ReadOnly"}, "jobType": "read-only-hook"}

It seems to happen during cascading scan hook execution if I interpret this correctly.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions