I'm tinkering with IBM Microclimate : -
Microclimate provides an end-to-end, cloud-native solution for creating, building, testing and deploying applications. The solution offers services and tools to help you create and modernize applications in one seamless experience. It covers each step of the process from writing and testing code to building and deployment. The solution enables containerized development, rapid iteration with real-time performance insights, intelligent feedback, diagnostic services, an integrated DevOps pipeline and deployment to the cloud.
also well documented here: -
Microclimate is an end to end development environment that lets you rapidly create, edit, and deploy applications. Applications are run in containers from day one and can be delivered into production on Kubernetes through an automated DevOps pipeline using Jenkins. Microclimate can be installed locally or on IBM Cloud Private, and currently supports Java, Node.js, and Swift.
https://microclimate-dev2ops.github.io/
I've played with this before: -
Playing with Microclimate on IBM Cloud Private
Microclimate on IBM Cloud Private - Permission to write
and will be posting my own build notes, but I'm still following the official documentation here: -
https://github.com/IBM/charts/blob/master/stable/ibm-microclimate/README.md
Having followed all of the pre-requisite steps ( which mainly involve creating lots of artefacts using kubectl ), and having installed the Helm chart, I was following this tutorial: -
Create and deploy Lagom Reactive applications with Microclimate
but found that the resulting Docker container would never start.
I dug around within IBM Cloud Private (ICP) or, to be more accurate, within Kubernetes, upon which ICP is built.
Microclimate comprises a number of Pods, sitting within a dedicated namespace - I'm using micro-climate : -
kubectl get pods -n micro-climate
NAME READY STATUS RESTARTS AGE
mc-adamjava-381f2a40-1ef8-11e9-bb42-adamjava-5c9697c464-dbqdk 1/1 Running 0 27m
mc-yoda-22f5f600-1efb-11e9-964a-yoda-bbb88b5d4-lqzwh 1/1 Running 0 52m
microclimate-ibm-microclimate-67cfd99c7b-bj7p2 1/1 Running 0 55m
microclimate-ibm-microclimate-admin-editor-77ddbdd86-xzlbj 2/2 Running 0 53m
microclimate-ibm-microclimate-admin-filewatcher-6cc6c785cf6lsjx 1/1 Running 0 53m
microclimate-ibm-microclimate-admin-loadrunner-856b4b48b6-jqlmc 1/1 Running 0 53m
microclimate-ibm-microclimate-atrium-7f75d754fd-fp244 1/1 Running 0 1h
microclimate-ibm-microclimate-devops-568c4c5989-kjcqs 1/1 Running 0 1h
microclimate-jenkins-678584959-64jlm 1/1 Running 0 1h
Given that all were running happily, I chose to dive into the logs of the File Watcher pod: -
kubectl logs -f microclimate-ibm-microclimate-admin-filewatcher-6cc6c785cf6lsjx -n micro-climate
[ERROR Tue Jan 22 13:58:43 UTC 2019 | Project: foobar | File Name: null | Function Name: null | Line Number: null] _tickCallback : 189 | unauthorized: authentication required
Pushing Docker images to IBM Cloud Private
but Microclimate still refused to play ball.
It did, however, confirm my suspicion that the problem was with the credentials between Microclimate and the Docker registry.
I looked back at my build notes, and saw that I'd wrongly read this: -
Create the Microclimate registry secret
This secret is used by both Microclimate and Microclimate's pipelines. It allows images to be pushed and pulled from the private registry on your Kubernetes cluster.
Use the following code to create a Docker registry secret:
kubectl create secret docker-registry microclimate-registry-secret \
--docker-server=
--docker-username=
--docker-password=
--docker-email=
Verify that the secret was created successfully and exists in the target namespace for Microclimate before you continue. This secret does not need to be patched to a service account as the Microclimate installation will manage this step.
as meaning that the secret needed to point at DockerHub i.e. the official Docker registry rather than the ICP registry.
After much faffing around, including a full nuke of the Helm chart, I was able to resolve this ... mainly thanks to some awesome support from the IBM Microclimate developer team in Hursley :-)
At one point, I stupidly retraced the very same steps ( using creds for DockerHub ) because my notes were out-of-sync with reality.
However, once I deleted the microclimate-registry-secret : -
kubectl delete secret microclimate-registry-secret -n micro-climate
and recreated it: -
kubectl create secret docker-registry microclimate-registry-secret --docker-server=mycluster.icp:8500 --docker-username=admin --docker-password=admin
pointing at the ICP server ( my cluster.icp:8500 ) and using the ICP credentials, things started to behave.
Just in case, I nuked the main Microclimate pod: -
kubectl delete pod microclimate-ibm-microclimate-67cfd99c7b-zqs8m
which forced the Replica Set to spawn a new pod ( remember, cattle, not pets ).
At that point, I was able to create a new Java / Lagom project, and watch it happily start and work :-)
I also noticed that my other failing-to-start project magically started working.
So, the long story very short ( TL;DR; ), get the darn credentials right.
For the record, I also nuked/recreated the other secret - microclimate-pipeline-secret - as follows: -
kubectl create secret docker-registry microclimate-pipeline-secret \
--docker-server=mycluster.icp:8500 \
--docker-username=admin \
--docker-password=admin \
--namespace=microclimate-pipeline-deployments
So things are now working - now I'm going to rip it all down and REDO FROM START, to see what I've learned :-)
PS For what it's worth, a quick cheat to get the logs of the File Watcher pod, regardless of it's name is: -
kubectl logs -f `kubectl get pods -n micro-climate | grep -i watcher | awk '{print $1}'` -n micro-climate
No comments:
Post a Comment