Thursday, 9 September 2021

And there's more - munging base64 in JSON for etcd

Following on from my earlier post: -

etcd - Today I learned ...

I dug into jq more, and found this: -

base64 decoding function #47

specifically this comment: -

As this is showing up on Google a lot, and good documentation on jq is sparse, here is to everybody who lands here:


echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | jq '.foo | @base64d'

Or even use it when building new objects:

echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | jq '{encoded: .foo, decoded: .foo | @base64d}'

dating back to 2018.

This led me to a neat-o mechanism to encode the key and value of my JSON document: -

cat dave.json

"value": {
"name":"Dave Hay",

jq -r '.[] |= @base64' dave.json

  "key": "MDEyMzQ1",
  "value": "eyJuYW1lIjoiRGF2ZSBIYXkiLCJpZCI6ImRhdmVoYXkxOTY5In0="

I then used this to generate a new JSON document: -

jq -r '.[] |= @base64' dave.json > dave_encoded.json

which I then fed into etcd: -

curl -X POST --silent --cacert /root/ssl/ca-cert.pem --cert /root/ssl/client-cert.pem --key /root/ssl/client-key.pem https://localhost:2379/v3/kv/put -d @dave_encoded.json | jq

  "header": {
    "cluster_id": "14841639068965178418",
    "member_id": "10276657743932975437",
    "revision": "2",
    "raft_term": "2"

and then confirmed that I could pull the data back out: -

curl -X POST --silent --cacert /root/ssl/ca-cert.pem --cert /root/ssl/client-cert.pem --key /root/ssl/client-key.pem https://localhost:2379/v3/kv/range -d '{"key":"MDEyMzQ1"}' | jq -r .kvs[].value | base64 -d | jq


  "name": "Dave Hay",
  "id": "davehay1969"

which is nice

See, I told you I'd find a way .....

No comments:

Today I Learned - useful keyboard shortcut for macOS

Last night, I chose to charge my Apple Magic Mouse - which, you might say, seems like a reasonable thing to do. The thing is that, anyone wh...