Skip to content

Conversation

@psviderski
Copy link
Owner

Fixes: #45
Related PR: #54

Use content.OpenWriter to wait until the reference is unlocked when creating a containerd content store writer.

The following error in unregistry was only possible when unregistry tried to create a new blob writer with the blow writer ID that was previously created:

error resolving upload: create containerd content writer: ref moby/1/upload-35015e47-eac0-4b24-aaaf-ca4329f004bd locked for 17.511331877s (since 2025-10-06 01:54:27.634765743 +0000 UTC m=+4330.405727369): unavailable

And this is possible only when the distribution implementation calls the blobStore.Resume.

I was able to reproduce the issue #45 only when using Docker with disabled containerd image store locally (the one that is used with pussh) and only when pushing to a remote RPi with a relatively slow microSD. Apparently, the new Docker implementation of docker push when using the containerd image store doesn't use the resume requests.

The following errors in containerd logs are still logged when content.OpenWriter attempts to create a writer and waits until the lock is freed:

(*service).Write failed" error="rpc error: code = Unavailable desc = ref moby/1/upload-46990068-fd5a-4598-95e7-ef8967e17d1a locked for 15.733253758s

which I believe is normal as I was able to find similar logs for the writers unrelated to unregistry on some of my machines. There is a comment in containerd to remove this noisy log line: https://github.com/containerd/containerd/blob/2f24aa00a58de442713ea60e4206041e00dcf012/plugins/services/content/contentserver/contentserver.go#L274-L275

@psviderski psviderski merged commit 3710725 into main Dec 16, 2025
1 check passed
@psviderski psviderski deleted the fix/retry-big-layers branch December 16, 2025 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Retrying and error for big layers

2 participants