Tuesday, 19 September 2017

Kubernetes and IBM Bluemix - again with the #HoldingItWrong

So I saw this: -

kubectl get nodes

Unable to connect to the server: could not refresh token: unrecognized error {"errorCode":"BXNIM0408E","errorMessage":"Provided refresh token is expired","context":{"requestId":"4294322993","requestType":"incoming.Kube_Token","startTime":"19.09.2017 11:58:26:739 UTC","endTime":"19.09.2017 11:58:26:741 UTC","elapsedTime":"2","instanceId":"tokenservice_dal06/1","host":"localhost","threadId":"1955e0","clientIp":"","userAgent":"Go-http-client/1.1","locale":"en_US"}}

which made me realise that I had forgotten to set the KUBECONFIG environment variable ( I'm using macOS ).

This I did: -

export KUBECONFIG=/Users/davidhay/.bluemix/plugins/container-service/clusters/DaveHayK8SCluster/kube-config-prod-dal10-DaveHayK8SCluster.yml


but I then started getting this: -

kubectl get nodes

The connection to the server localhost:8080 was refused - did you specify the right host or port?

kubectl proxy

The connection to the server localhost:8080 was refused - did you specify the right host or port?

which made me cuss a bit.

However …..

I love it when my own blog post: -

has the solution :-)

Bottom line, the KUBECONFIG variable was AGAIN wrong :-(

I validated this: -


ls: /Users/davidhay/.bluemix/plugins/container-service/clusters/DaveHayK8SCluster/kube-config-prod-dal10-DaveHayK8SCluster.yml: No such file or directory

So I validated the name of my Kubernetes cluster: -

bx cs clusters

Name                ID                                 State    Created                    Workers   Datacenter   
DaveHayK8SCluster   fff102198c534d0096bacd575488c9dd   normal   2017-08-21T09:59:53+0000   1         par01   

and then searched for the YAML: -

find ~/.bluemix/ -name *.yml


Once i set the variable appropriately: -

export KUBECONFIG=/Users/davidhay/.bluemix//plugins/container-service/clusters/DaveHayK8SCluster/kube-config-par01-DaveHayK8SCluster.yml

things started working again: -

kubectl get nodes

NAME            STATUS    AGE       VERSION   Ready     29d       v1.5.6-4+abe34653415733

kubectl proxy

Starting to serve on

and the proxy now works: -

So again, READ MY (OWN) BLOG :-)

No comments:

Note to self - using the CRI tool - crictl - to clean up unready pods

 Purely 'cos I know I'll need this again: - for i in `crictl pods | grep NotReady | awk '{print $1}'`; do crictl rmp $i; don...