Docker Layer Corrupt, What Do?

Update 2015-11-10: In the docker registry v2 API (now supported by Google Container Registry) an image digest is checked for each layer. So in theory this problem should no longer be possible.

Attempting a docker deployment, the deploy failed on the target system when docker pull resulted in one layer failing to download with error message:

Untar exit status 1 unexpected EOF

I’m not sure what caused the problem - it may have been from ctrl+c during the docker push step, or maybe something I had less control over. In any case, this one particular layer was corrupt.

This was a bit frustrating because a second docker push of the artifact will fail to resolve the issue - the private registry thinks it already has the layer and there’s no way to force the rewrite of a layer (or if there is, please leave a comment and let me know). And it was a base layer, not a final layer that become corrupt - trivial changes to the build image did not workaround the problem.

Because I’m using Google Cloud’s private registry, the docker images are visible via Google Cloud Storage. I was trivially able to use the cloud storage browser to find the offending layer and delete it manually. This was a bit lucky, because I’d guess not all private registries expose the file system to you.

It would be nice if there was a way to force a docker push to ignore cache and rewrite layers. Or is there a better solution that I’m not thinking of?

I just wanted to share this problem and workaround I had. Hope it helps. Actually, I hope it doesn’t help because nobody else has this problem 😉.