Docker Content Trust and the case of the strange errors e.g. "read/write on closed pipe"

 Whilst trying to build a Docker image on one of my IBM Z build servers yesterday, I hit a weird series of errors, including : -

ERRO[0000] Can't add file /root/dave/hello to tar: io: read/write on closed pipe 

ERRO[0000] Can't close tar writer: io: read/write on closed pipe 

and: -

error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&nocache=1&rm=1&shmsize=0&target=&ulimits=null&version=1: Error: error contacting notary server: unauthorized: The login credentials are not valid, or your IBM Cloud account is not active.

Initially, I thought that the first error only appeared to occur with a Dockerfile that copied a binary object ( actually a compiled binary written in Go ) into the image.

After some fun n' games, I realised what was going on ....

I've reproduced the situation below: -

cd dave

ls -al

total 19016

drwxr-x---  2 root root     4096 Oct  3 13:31 .

drwx------ 15 root root     4096 Oct  3 13:30 ..

-rw-r-----  1 root root       38 Oct  3 13:30 Dockerfile

-rw-r--r--  1 root root       13 Oct  1 12:28 greeting.txt

-rwxr-x---  1 root root 19452575 Oct  3 13:31 hello

cat Dockerfile 

FROM alpine:3.11

COPY greeting.txt .

Note that there's NO mention of the binary file ( hello ) in the Dockerfile and yet .....

docker build --no-cache -f Dockerfile .

Sending build context to Docker daemon 

ERRO[0000] Can't add file /root/dave/hello to tar: io: read/write on closed pipe 

ERRO[0000] Can't close tar writer: io: read/write on closed pipe 

error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&nocache=1&rm=1&shmsize=0&target=&ulimits=null&version=1: Error: error contacting notary server: unauthorized: The login credentials are not valid, or your IBM Cloud account is not active.

Given that I knew that I wasn't logged into Docker Hub or the IBM Cloud Container Registry (ICCR) instance that I've been using for many of my builds .....

I even checked this: -

cat ~/.docker/config.json | json_pp 

{

   "HttpHeaders" : {

      "User-Agent" : "Docker-Client/19.03.6 (linux)"

   },

   "auths" : {}

}

And then it struck me ...... with the force of Mjolnir ....

Note that the message related to my "IBM Cloud account" ?

And also note that the error says "error contacting notary server" ?

And yet I wasn't logged into IBM Cloud .....

However, I had been earlier AND was working with Docker Content Trust (DCT).

And I'd set two environment variables in my current Bash session: -

set | grep DOCKER

DOCKER_CONTENT_TRUST=1

DOCKER_CONTENT_TRUST_SERVER=https://de.icr.io:4443

but I wasn't logged into Docker Hub or, in this case, ICCR .....

So I'd told the Docker Client to set DCT and even told it where the Notary server was ....

And yet .....

So I disabled DCT: -

export DOCKER_CONTENT_TRUST=

and re-ran the build: -

docker build --no-cache -f Dockerfile .

Sending build context to Docker daemon  19.46MB

Step 1/2 : FROM alpine:3.11

3.11: Pulling from library/alpine

7184c046fdf1: Pull complete 

Digest: sha256:9a839e63dad54c3a6d1834e29692c8492d93f90c59c978c1ed79109ea4fb9a54

Status: Downloaded newer image for alpine:3.11

 ---> 4b858171dd2c

Step 2/2 : COPY greeting.txt .

 ---> 164461a814d3

Successfully built 164461a814d3

The moral of the story ? If you're using DCT, remember to log in to your Registry and thus Notary.

Otherwise ... DON'T !!!!!