Friday, 23 September 2022

More fun with pip

Again with the Python and pip fun, this time on my Mac, where commands such as: 

pip3 list

and: -

pip3 install --upgrade pip

were failing with: -

WARNING: There was an error checking the latest version of pip.

Long story short, the error was actually more obvious than I'd realised ....

In essence, it was actually telling me what was going wrong ...

Looking in indexes:.......

where the location was an internal repository for **ONE** single Python module, which would never have the things I was trying to install e.g. pip itself

I updated the config: -

vi ~/.pip/pip.conf

and commented out the aberrant index ... 

And now we're off to the races ...

Installing pylint on Linux - there's more than one way ...

 I'm running some Travis builds of a project which includes Python modules, and wanted to manually run the pylint linting tool across my GitHub repo, to compare/contrast what Travis was telling me.

I'm running Ubuntu 20.04.5 LTS so just went ahead and installed via the Aptitude package manager: -

apt-get update && apt-get install -y pylint

which seemed fine.

Then I noticed that the results differed - when I checked Travis, I was using pylint 2.15.2 whereas, on Ubuntu, I was way behind the curve: -

pylint --version

pylint 2.4.4
astroid 2.3.3
Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0]

My mistake ? Using Aptitude ...

I uninstalled pylint 

apt-get remove -y pylint

and installed it using the Package Installer (for) Python, aka pip

I did notice that the installation location was different: -

- apt-get install ->>> /usr/bin/pylint

- pip  ->>> /usr/local/bin/pylint

so had to restart my shell to pick up the new version: -

pylint --version

pylint 2.15.3
astroid 2.12.10
Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0]

which is better ( part of my planned change will be to have the Travis job use 2.15.3 as well )

Right, onwards for some linting .....


Saturday, 10 September 2022

Been a while - back with some wget fun

Whilst digging into some Kubernetes testing, validating a fix that I'd made to my cluster, I was deploying both Nginx and Busybox pods.

The Nginx pods were deployed across all my Compute Nodes, to provide a web server tier, and I was using Busybox, again across all the Compute Nodes, to validate that Nginx connectivity was clean and green.

Having obtained the individual IPs of each of the Nginx pods- I'm using Calico Node as my Container Network Interface (CNI) layer - I wanted to hit each Nginx pod from each of the Busybox pods, in turn.

Now, ordinarily, I'd use a command such as: -

curl http://192.168.1.24:80/index.html

to retrieve the sample HTML page that Nginx presents.

However, Busybox is a very cut-down Linux-like environment and, therefore, it doesn't include the curl command.

Thankfully, Busybox does include the wget command, so I was able to use that in a similar manner: -

wget -q --output-document - http://192.168.1.24:80/index.html

which did the trick i.e. dumped the content of nginx.html to the console ( stdout ).

The key parameters are as follows: -

-q == run wget in quiet mode, only returning the required web page, rather than the normal debug

--output-document == tells wget what and where to return the output

- == tells --output-document to write to stdout rather than a specific file/document

Nice !

Friday, 22 July 2022

Grokking grep

A colleague was tinkering with grep and, thanks to him, I discovered a bit more about the trusty little utility.

I had not really explored the -e switch: -

     -e pattern, --regexp=pattern

             Specify a pattern used during the search of the input: an input line is selected if it matches any of the specified patterns.  This option is

             most useful when multiple -e options are used to specify multiple patterns, or when a pattern begins with a dash (‘-’).

but he pointed out that that switch can have problems when parsing, say, a list of strings that are similar but different.

As an example, I have a set of files: -

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 dave
-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 dave_1
-rw-r--r--   1 hayd  wheel    0 22 Jul 17:36 dave_2
-rw-r--r--   1 hayd  wheel    0 22 Jul 17:36 dave_3

If I want to e.g. query a directory for all files containing the word 'dave' I could do this: -

ls -al | grep dave

or, to be more specific: -

ls -al | grep -e Dave

both of which return: -

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 dave
-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 dave_1
-rw-r--r--   1 hayd  wheel    0 22 Jul 17:36 dave_2
-rw-r--r--   1 hayd  wheel    0 22 Jul 17:36 dave_3

However, if I only want to return the file that's named 'dave' and ignore the rest, I'm somewhat stymied.

However, grep -w comes to the rescue: -

     -w, --word-regexp
             The expression is searched for as a word (as if surrounded by ‘[[:<:]]’ and ‘[[:>:]]’; see re_format(7)).  This option has no effect if -x is
             also specified.


so I can run: -

ls -al | grep -w dave

and just get this: -

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 Dave

Isn't that nifty ?

Wednesday, 20 July 2022

To start, press any key .... hey, where's the [Any] key ?

So I was writing some scripts for a demo that I delivered earlier today...

One script will run on my Mac, using the Zsh shell, the other will run on an Ubuntu box, using Bash.

In both cases, I wanted to pause the script and wait for user input before proceeding.

On Ubuntu, I did this: -

read -p "Press [Enter] to continue"

Sadly, however, on macOS 12.4 via zsh, that didn't work ...

read: -p: no coprocess

Thankfully, there's a better / different way: -

read "?Press [Enter] to continue"

which is nice

Today I Learned - Shorter redirection, what's not to like ?

So I write a lot of scrips for Bash and Zsh, and often make use of redirection where, for example, I generate a variable by running another command within parentheses.

As a specific example, I'd query my Floating IP and persist it to a text file for future use: -

ic is floating-ips --output JSON | jq -r '.[].address' > ~/floating_ip.txt

Later on, I'd look to use that Floating IP for, for example, a SSH connection, as follows: -

export floating_ip=$(cat floating_ip.txt)

ssh root@$floating_ip

Authorized uses only. All activity may be monitored and reported.

root@163.61.95.34's password:

However, one of my colleague's did point out that I could reduce the number of characters typed when setting the floating_ip variable: -

export floating_ip=$(< floating_ip.txt)

See, that saves a whole two characters ....

Mind you, I can also reduce things further via: -

ssh root@$(cat floating_ip.txt)

I love shelling !

Friday, 29 April 2022

TIL - read-only variables in Linux

 A co-worker was seeing an exception: -

 line 8: TMOUT: readonly variable

when trying to SCP a file from a remote Linux box.

I did some digging and found a RedHat article: -

Why does it prompt "bash: TMOUT: readonly variable" when sudo'ing or ssh'ing to the system? 

that said, in part: -

The TMOUT variable is usually defined read-only to avoid users from unsetting or modifying its value. Due to this it's not possible to set it twice.

I reproduced the problem on my Ubuntu 18.04 box: -

-bash: TMOUT: readonly variable

with a pair of scripts: -

The first script: -

cat /etc/profile.d/a.sh

readonly TMOUT=500; export TMOUT

sets TMOUT as readonly

The second script: -

cat /etc/profile.d/b.sh

TMOUT=600; export TMOUT

then tries to override it

which I validated: -

fgrep -R TMOUT /etc/profile.d/

/etc/profile.d/a.sh:readonly TMOUT=500; export TMOUT
/etc/profile.d/b.sh:TMOUT=600; export TMOUT

I left my colleague to dig into /etc etc. and see what was going on, but TIL about read-only variables 

Having fun and games with Kubernetes networking

I'd forgotten how much I simply enjoy the opportunities for hacking - in the original naive sense of the word - that Kubernetes (K8s) offers.

Today I've been working to find out why CoreDNS didn't work in my cluster - clue, it was containerd that did it

However, I then started seeing: -

failed to allocate for range 0: no IP addresses available in range set: 10.48.131.1-10.48.131.62

from my CoreDNS pods, having "fixed" containerd.

Thankfully, Google Cloud have a doc for that: -

Pods display failed to allocate for range 0: no IP addresses available in range set error message

In part, it required me to stop containerd and kubelet, and clear out the previously defined IP address range: -

mkdir /var/lib/cni/networks

Once I did this, and restarted containerd and kubelet, we're back in the game !

Wednesday, 27 April 2022

Munging JSON with JQ - without using grep and awk

Further to a previous post: -

Grep AND awk

I wanted to achieve much the same, but only using jq

So here we go: -

ic is images --output JSON | jq -r '.[] | {Name:.name,Architecture:.operating_system.architecture,Status:.status} | select((.Name | contains("ubuntu")) and (.Architecture | startswith("s390x")) and (.Status | startswith("available")))'

which returns: -

{
  "Name": "ibm-ubuntu-18-04-1-minimal-s390x-3",
  "Architecture": "s390x",
  "Status": "available"
}

as opposed to the alternative: -

ic is images | awk '/s390x/ && /ubuntu/ && /available/'

r018-e3d94080-972f-4f18-8a79-60a12d0b61c2   ibm-ubuntu-18-04-1-minimal-s390x-3                 available    s390x   ubuntu-18-04-s390x                   18.04 LTS Bionic Beaver Minimal Install   2               public       provider     none         -   

Tuesday, 26 April 2022

Git and Ubuntu - not branching out

 Whilst trying to make some changes to a GitHub project, using an Ubuntu box, I hit an interesting issue with git switch - namely that it doesn't work 

So I'm running Ubuntu 18.04.6 LTS: -

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic

and had created a new branch: -

git branch

  test_doc_update
* develop

and then tried to switch to it: -

git switch test_doc_update

but instead got: -

git: 'switch' is not a git command. See 'git --help'.

I checked the version of Git: -

git --version

git version 2.17.1

and even tried upgrading it: -

apt-get update && apt-get upgrade -y git

...

The following packages will be upgraded:
  git git-man
2 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.

...

but to no avail: -

git --version

git version 2.17.1

Instead, as per this Git: ‘switch’ is not a git command. See ‘git –help’  I used this instead: -

git checkout test_doc_update

Switched to branch 'test_doc_update'

git branch

* test_doc_update
  develop

Sorted !

For the record, I have a more up-to-date version of git on the Mac, via Homebrew

ls -al `which git`

lrwxr-xr-x  1 hayd  admin  28 19 Apr 08:59 /usr/local/bin/git -> ../Cellar/git/2.36.0/bin/git

git --version

git version 2.36.0

with which git switch DOES work

Monday, 18 April 2022

Tinkering with arrays in ZSH

Someone had asked: -

if you have a command that returns two values, can you assign each value to a separate variable? For example, I have a command that returns two lines, and I want NAME to be set to the first line and TITLE to the second line. I seem to recall doing this in the past, but I can’t find an example or a note on it.

which made me think about how one might achieve this

I'm using zsh, and looked at dumping the output from a command into an array

I started with the sw_vers command: -

ProductName: macOS
ProductVersion: 12.3.1
BuildVersion: 21E258

and found a way to dump the multiline output into an array: -

array_of_lines=("${(@f)$(sw_vers)}")                                 

and checked each of the elements in the array: -

echo $array_of_lines[1]                                                    

ProductName: macOS

echo $array_of_lines[2]

ProductVersion: 12.3.1

echo $array_of_lines[3]

BuildVersion: 21E258

I also used ${#array_of_lines} to get the size of the array: -

echo ${#array_of_lines}

3

to iterate through the array and dump out each element in turn: -

for ((i = 1; i<=${#array_of_lines}; i++)); do echo $array_of_lines[i]; done

ProductName: macOS
ProductVersion: 12.3.1
BuildVersion: 21E258

Whether this helps, we shall see ....

Thursday, 7 April 2022

AirPlay and  TV and macOS Monterey

For those using AirPlay to mirror to an external screen e.g.  TV etc. you can toggle whether the Screen Mirroring tool shows up in the Menu Bar - or not - via System Preferences -> Dock & Menu Bar -> Screen Mirroring





This is especially useful for me now that we're back in the office in meeting rooms equipped with  TV

Wednesday, 30 March 2022

iTerm2 and keyboard navigation - Today I Learned

 I've been using iTerm2 off and on for a few years now, and find it especially useful when running live demos - one nice feature is that I can present multiple terminal windows in the same screen - windowed windows on a Mac 🀣

Right now, I'm using version 3.4.15 of iTerm, but that's not necessarily important right now.

The thing that was making me go "Hmmmm" was the "feature" that meant that I struggled to jump back along a line of commands using the [option] [arrow] keys ...

My muscle memory is that I can use that key combination to jump back and forth within a long list of commands on a single line to, for example, allow me to edit the command, add a switch etc.

By default, in iTerm, hitting [option] and [left-arrow] would, instead of jumping one word to the left ( as in the default macOS Terminal.app ) would instead append [D - which is a pain

So, here's an example: -

echo `date` && echo "Hello World" && echo "Dave"

I want to jump back to the second echo command, and change the greeting to "Hello World!".

Obviously, that's a very very trivial example, but the point remains the same ...

As ever, the internet helped - my Google Fu found this: -

Using Alt/Cmd + Right/Left Arrow in iTerm

which had a bunch of suggested solutions, the most simple of which was to simply append: -

bindkey "\e\e[D" backward-word
bindkey "\e\e[C" forward-word

to ~/.zshrc

Once I did this and restarted my shell ( I'm using Zsh obviously ), I was off to the races ....

For the record, I could've simply sourced in the updated ~/.zshrc via source ~/.zshrc


Grep AND awk

The context for this is that I'm working with the IBM Cloud CLI tool, specifically the Infrastructure Services plugin to find a specific subset of images, from which I can instantiate a Virtual Server Instrance (VSI).

Given this, I wanted to list the images in a specific region and filter by those that: -

- Run on the IBM Z Linux platform, aka s390x **AND**

- Are based upon Ubuntu **AND**

- Are available rather than deprecated

I started with this: -

ic is images

( where ic is an alias for the ibmcloud plugin )

and then filtered using a set of nested grep calls: -

ic is images | grep s390x | grep ubuntu | grep available

r134-0568bd44-e2b7-4876-bcbf-b74b244d3b66   ibm-ubuntu-18-04-1-minimal-s390x-2                 available    s390x   ubuntu-18-04-s390x                   18.04 LTS Bionic Beaver Minimal Install                      2               public       provider     none         -

r134-77d54c6d-5071-4729-ac79-a26c7404866a   ibm-ubuntu-18-04-6-minimal-s390x-3                 available    s390x   ubuntu-18-04-s390x                   18.04 LTS Bionic Beaver Minimal Install                      2               public       provider     none         -

r134-622ffa9c-47e1-450f-9a02-4f0567b7139f   ibm-ubuntu-20-04-2-minimal-s390x-2                 available    s390x   ubuntu-20-04-s390x                   20.04 LTS Focal Fossa Minimal Install                        2               public       provider     none         -

r134-323e4b0e-118b-4199-8d0b-745c05a75194   ibm-ubuntu-20-04-2-minimal-s390x-enclaved-2        available    s390x   ubuntu-20-04-s390x-enclaved          20.04 LTS Focal Fossa Minimal Install for Secure Execution   2               public       provider     none         -

but that's not very elegant

I knew how to use the OR operator in grep e.g.

ic is images | grep "s390x\|ubuntu"

but that's not what I want; I want AND rather that OR

This helped: -

How to run grep with multiple AND patterns?

In other words, use awk rather than grep, as per this: -

ic is images | awk '/s390x/ && /ubuntu/ && /available/'

r134-0568bd44-e2b7-4876-bcbf-b74b244d3b66   ibm-ubuntu-18-04-1-minimal-s390x-2                 available    s390x   ubuntu-18-04-s390x                   18.04 LTS Bionic Beaver Minimal Install                      2               public       provider     none         -

r134-77d54c6d-5071-4729-ac79-a26c7404866a   ibm-ubuntu-18-04-6-minimal-s390x-3                 available    s390x   ubuntu-18-04-s390x                   18.04 LTS Bionic Beaver Minimal Install                      2               public       provider     none         -

r134-622ffa9c-47e1-450f-9a02-4f0567b7139f   ibm-ubuntu-20-04-2-minimal-s390x-2                 available    s390x   ubuntu-20-04-s390x                   20.04 LTS Focal Fossa Minimal Install                        2               public       provider     none         -

r134-323e4b0e-118b-4199-8d0b-745c05a75194   ibm-ubuntu-20-04-2-minimal-s390x-enclaved-2        available    s390x   ubuntu-20-04-s390x-enclaved          20.04 LTS Focal Fossa Minimal Install for Secure Execution   2               public       provider     none         -

which is much better

Friday, 25 March 2022

Fun with Pip and Python on macOS 12.3 Monterey

I'm tinkering with a tool that uses pip and python, and was seeing: -

zsh: /usr/local/bin/pip: bad interpreter: /usr/bin/python: no such file or directory

repeatedly e.g.

pip --version                                                                                                                                                                           

zsh: /usr/local/bin/pip: bad interpreter: /usr/bin/python: no such file or directory

pip 22.0.4 from /Users/hayd/.pyenv/versions/3.10.0/lib/python3.10/site-packages/pip (python 3.10)

I've got Python 3.1 installed via Homebrew, using PyEnv: -

which python                                                                                                                                                                            

/Users/hayd/.pyenv/shims/python

python --version                                                                                                                                                                        

Python 3.10.0

as per an earlier post: -


I checked using brew doctor but nothing jumped off the page at me.

head -n1 /usr/local/bin/pip

#!/usr/bin/python

ls -al /usr/bin/python

ls: /usr/bin/python: No such file or directory

I then moved pip 

mv /usr/local/bin/pip /usr/local/bin/pip.old

and tested: -

which pip

/Users/hayd/.pyenv/shims/pip

and now we're happy: -

pip --version

pip 22.0.4 from /Users/hayd/.pyenv/versions/3.10.0/lib/python3.10/site-packages/pip (python 3.10)

So, in other words, in the pre-PyEnv days, I had pip pointing at the older Apple-supplied Python 2.7, which has long since been deprecated/removed.

Now, post the mv command, it's pointing at pip via PyEnv, which is nice.

Thanks, as ever, to StackOverflow: -


Wednesday, 23 March 2022

Running Podman on Ubuntu 18.04

 I'm leaving this to remind my future self ...

echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_18.04/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list

curl -L "https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_18.04/Release.key" | sudo apt-key add -

sudo apt-get update

sudo apt-get -y upgrade

sudo apt-get install -y podman

podman run docker.io/docker/whalesay:latest cowsay Hello World!

when I want to install and test Podman on Ubuntu 18.04

Remembering that we'll end up with an older version of Podman ...

podman version

Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.15.2
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64


Monday, 21 March 2022

Bash and history - where's it gone ?

I logged into an Ubuntu 18.04.6 LTS box and noticed that my Bash history was completely and utterly gone.

Having typed the command: -

history

I got absolutely nothing back.

Knowing that the command really just outputs the content of the ~/.bash_history file, I checked that out: -

ls -al ~/.bash_history

-rw------- 1 root root 9007 Mar 20 13:39 /home/hayd/.bash_history

Notice that my user name is hayd ....

Also notice the user/group ownership of the file .... root ....

I changed the ownership of the file: -

sudo chown hayd:hayd ~/.bash_history

[sudo] password for hayd: 

and logged out and back in again ...

And now I have my history back ...

...
  348  ~/upgrade.sh 
  349  which podman
  350  apt-get install -y podman
  351  ~/upgrade.sh 
  352  ~/createHelloWorld.sh 
  353  history 
  354  lsb_release -a
...

Also, as mentioned elsewhere, I use an alias - hist - to output history without line numbers: -

alias hist='history | cut -c 8-'

...
~/upgrade.sh 
which podman
apt-get install -y podman
~/upgrade.sh 
~/createHelloWorld.sh 
history 
lsb_release -a
...

Wednesday, 16 March 2022

ZSH and history - going back in time

OK, so this didn't take long to "fix" ...

As an ex-Bash user, I've had an alias setup on all my Unix boxen to allow me to list out my shell's history, without line numbers.

So, therefore, rather than typing: -

history

and seeing line numbers, such as these: -

  989  podman version
  990  cd
  991  hostname
  992  ls -al ~/.ssh
  993  ls -al ~/.ssh
  994  date
  995  cat ~/.ssh/readme.txt

I have an alias setup: -

hist='history | cut -c 8-'

which returns much the same but without line numbers e.g.

hostname
ls -al ~/.ssh
ls -al ~/.ssh
date
cat ~/.ssh/readme.txt

This alias was setup in ~/.bash_profile and is now set up in ~/.zshenv.

However, I'd noticed that hist would only ever return the last 16 commands ...

This was easily solved: -


...
History accepts a range in zsh entries as [first] [last] arguments, so to get them all run history 0.
...

Therefore, I just needed to update my alias: -

alias hist='history 0 | cut -c 8-'

and now I see everything.....

Podman say "No"

Whilst tinkering with podman today, I saw this: -

podman images

Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM

Error: unable to connect to Podman socket: server API version is too old. Client "4.0.0" server "3.4.4"

I tried: -

podman system connection list

which returned: -

Name                         URI                                                          Identity                                 Default

podman-machine-default       ssh://core@localhost:58173/run/user/1000/podman/podman.sock  /Users/hayd/.ssh/podman-machine-default  true

podman-machine-default-root  ssh://root@localhost:58173/run/podman/podman.sock            /Users/hayd/.ssh/podman-machine-default  false

which wasn't terribly useful, and then checked the machine: -

podman machine list

NAME                     VM TYPE     CREATED      LAST UP            CPUS        MEMORY      DISK SIZE

podman-machine-default*  qemu        2 weeks ago  Currently running  1           2.147GB     10.74GB

Having checked versions: -

podman --version

podman version 4.0.2

I tried restarting the machine: -

podman machine stop

Machine "podman-machine-default" stopped successfully

podman machine start

Starting machine "podman-machine-default"

INFO[0000] waiting for clients...                       

INFO[0000] new connection from  to /var/folders/b5/8vqr9tt54v94jxzs0_k2qq2m0000gn/T/podman/qemu_podman-machine-default.sock 

Waiting for VM ...

INFO[0012] Socket forward established: /Users/hayd/.local/share/containers/podman/machine/podman-machine-default/podman.sock to /run/user/0/podman/podman.sock 

ERRO[0013] Couldn't restablish ssh tunnel on path: /run/user/0/podman/podman.sock: ssh: rejected: connect failed (open failed) 

WARN[0013] API socket failed ping test                  


This machine is currently configured in rootless mode. If your containers

require root permissions (e.g. ports < 1024), or if you run into compatibility

issues with non-podman clients, you can switch using the following command: 


podman machine set --rootful


API forwarding listening on: /Users/hayd/.local/share/containers/podman/machine/podman-machine-default/podman.sock


The system helper service is not installed; the default Docker API socket

address can't be used by podman. If you would like to install it run the

following command:


sudo /usr/local/Cellar/podman/4.0.2/bin/podman-mac-helper install


You can still connect Docker API clients by setting DOCKER_HOST using the

following command in your terminal session:


export DOCKER_HOST='unix:///Users/hayd/.local/share/containers/podman/machine/podman-machine-default/podman.sock'


Machine "podman-machine-default" started successfully

but still saw the same issue: -

podman images

Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM

Error: unable to connect to Podman socket: server API version is too old. Client "4.0.0" server "3.4.4"

I've installed podman via Homebrew: -

brew info podman

podman: stable 4.0.2 (bottled), HEAD

Tool for managing OCI containers and pods

https://podman.io/

/usr/local/Cellar/podman/4.0.2 (172 files, 48.7MB) *

  Poured from bottle on 2022-03-15 at 12:23:57

From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb

License: Apache-2.0

==> Dependencies

Build: go ✘, go-md2man ✘

Required: qemu ✔

==> Options

--HEAD

Install HEAD version

==> Caveats

zsh completions have been installed to:

  /usr/local/share/zsh/site-functions

==> Analytics

install: 10,898 (30 days), 36,534 (90 days), 104,199 (365 days)

install-on-request: 10,891 (30 days), 36,516 (90 days), 104,182 (365 days)

build-error: 13 (30 days)

Nothing was terribly informative, so I chose to nuke the machine: -

podman machine rm

The following files will be deleted:


/Users/hayd/.ssh/podman-machine-default

/Users/hayd/.ssh/podman-machine-default.pub

/Users/hayd/.config/containers/podman/machine/qemu/podman-machine-default.ign

/Users/hayd/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-35.20220213.2.0-qemu.x86_64.qcow2

/Users/hayd/.config/containers/podman/machine/qemu/podman-machine-default.json



Are you sure you want to continue? [y/N] y

and create a new one: -

podman machine init

Downloading VM image: fedora-coreos-35.20220305.dev.0-qemu.x86_64.qcow2.xz: done  

Extracting compressed file

Image resized.

Machine init complete

To start your machine run:


podman machine start

podman machine start

Starting machine "podman-machine-default"
INFO[0000] waiting for clients...                       
INFO[0000] new connection from  to /var/folders/b5/8vqr9tt54v94jxzs0_k2qq2m0000gn/T/podman/qemu_podman-machine-default.sock 
Waiting for VM ...
INFO[0018] Socket forward established: /Users/hayd/.local/share/containers/podman/machine/podman-machine-default/podman.sock to /run/user/501/podman/podman.sock 

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command: 

podman machine set --rootful

API forwarding listening on: /Users/hayd/.local/share/containers/podman/machine/podman-machine-default/podman.sock

The system helper service is not installed; the default Docker API socket
address can't be used by podman. If you would like to install it run the
following command:

sudo /usr/local/Cellar/podman/4.0.2/bin/podman-mac-helper install

You can still connect Docker API clients by setting DOCKER_HOST using the
following command in your terminal session:

export DOCKER_HOST='unix:///Users/hayd/.local/share/containers/podman/machine/podman-machine-default/podman.sock'

Machine "podman-machine-default" started successfully

and then: -

podman images

REPOSITORY  TAG         IMAGE ID    CREATED     SIZE

which looked better.

I started a container: -

podman run -it alpine:latest sh

Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3
Copying blob sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3
Copying config sha256:c059bfaa849c4d8e4aecaeb3a10c2d9b3d85f5165c66ad3a4d937758128c4d18
Writing manifest to image destination
Storing signatures
/ # uname -a
Linux c4367a60c3d4 5.15.18-200.fc35.x86_64 #1 SMP Sat Jan 29 13:54:17 UTC 2022 x86_64 Linux
/ # exit

and then re-checked the downloaded images: -

podman images

REPOSITORY                TAG         IMAGE ID      CREATED       SIZE
docker.io/library/alpine  latest      c059bfaa849c  3 months ago  5.87 MB

podman version

Client:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.17.8

Built:      Wed Mar  2 14:04:36 2022
OS/Arch:    darwin/amd64

Server:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.16.14

Built:      Thu Mar  3 14:56:56 2022
OS/Arch:    linux/amd64

More fun with Git and ZSH, this time branch naming ...

Following on from my earlier post: -

ZSH and Git and auto-complete

I wanted to add the capability to display the name of the current GitHub repository branch in my Terminal.

Thankfully, GitHub already allows this, and I had their repository cloned to my Mac.

A quick update: -

cd ~/Documents/GitHub/github.com/git

git fetch origin

git rebase origin/master

and then I copied a pair of scripts: -

cp ~/Documents/GitHub/github.com/git/contrib/completion/git-completion.zsh ~/.zsh/_git

cp ~/Documents/GitHub/github.com/git/contrib/completion/git-prompt.sh ~

noting that git-completion.zsh goes into ~/.zsh/_git as per this: -

# The recommended way to install this script is to make a copy of it as a
# file named '_git' inside any directory in your fpath.
#
# For example, create a directory '~/.zsh/', copy this file to '~/.zsh/_git',
# and then add the following to your ~/.zshrc file:
#
#  fpath=(~/.zsh $fpath)

( source: git-completion.zsh )

This is now my ~/.zshrc file: -

if type brew &>/dev/null; then
    FPATH=$(brew --prefix)/share/zsh-completions:$FPATH

    autoload -Uz compinit
    compinit -u
fi

setopt prompt_subst
. ~/git-prompt.sh
export RPROMPT=$'$(__git_ps1 "%s")'

and, having restarted my Terminal, this is what I see: -


noting that the branch name appears/changes on the right-hand side of the screen

Which is nice

ZSH and Git and auto-complete

Having finally made the switch to ZSH ( zsh ) on my Mac a few weeks back ( yeah, I know, right ? ), I've been tinkering with auto-complete etc.

Lots of advice and guidance online, the simplest solution was to install zsh-completions, as I'm already using Homebrew: -

brew install zsh-completions

This, in part, said: -

To activate these completions, add the following to your .zshrc:

  if type brew &>/dev/null; then
    FPATH=$(brew --prefix)/share/zsh-completions:$FPATH

    autoload -Uz compinit
    compinit
  fi

You may also need to force rebuild `zcompdump`:

  rm -f ~/.zcompdump; compinit

Additionally, if you receive "zsh compinit: insecure directories" warnings when attempting
to load these completions, you may need to run this:

  chmod -R go-w '/usr/local/share/zsh'

Now I didn't read the last part ......

And then I noticed this: -

compinit
   
zsh compinit: insecure directories, run compaudit for list.
Ignore insecure directories and continue [y] or abort compinit [n]? y%                                                                                                                                                                 

A little digging later, I found compaudit, which reported: -

There are insecure directories:
/usr/local/share

and then I re-read the output from the brew command ...

Additionally, if you receive "zsh compinit: insecure directories" warnings when attempting
to load these completions, you may need to run this:

  chmod -R go-w '/usr/local/share/zsh'

but that made no difference - I still saw the same output from compaudit.

Eventually, I found this: -

compaudit | xargs chmod g-w

via this: -


TL;DR; this was the before state: -

ls -al /usr/local/share/zsh

total 0
drwxr-xr-x   3 hayd  admin    96 28 Jan  2019 .
drwxrwxr-x  36 hayd  admin  1152 16 Mar 09:55 ..
drwxr-xr-x  12 hayd  admin   384 15 Mar 12:23 site-functions

and this was the after state: -

ls -al /usr/local/share/zsh

total 0
drwxr-xr-x   3 hayd  admin    96 28 Jan  2019 .
drwxr-xr-x  36 hayd  admin  1152 16 Mar 09:55 ..
drwxr-xr-x  12 hayd  admin   384 15 Mar 12:23 site-functions

In other words, the chmod command removed the group-writeable permissions from the top-level directory - /usr/local/share

This was a good reminder of how one sees permissions using ls: -



Monday, 14 March 2022

Port Stripping

 So, like most things we do with tech, there's probably 73 different ways to achieve this particular requirement ...

However, I had a specific need to parse a JSON document, pull out a specific key, which contained an endpoint URL, and extract only the port number.

Here's an example of my JSON document: -

{

    "endpointURL":"https://www.ibm.com:443"

}

and, specifically, I just want to extract the port number of 443.

This is up with what I ended: -

cat server.json | jq -r '.endpointURL' | sed 's/.*://'

so it's three distinct commands, tied together with pipes: -

1. cat server.json to dump out the content of the document

2. jq -r '.endpointURL'  to only pull the endpointURL key

3. sed 's/.*://' to apply a regular expression ( aka RegExp ) to extract the port number

cat server.json | jq -r '.endpointURL' | sed 's/.*://'

443

Nice!

Monday, 7 March 2022

Ubuntu, where's my stuff ( in /tmp ) ?

At present, I'm tinkering with Kubernetes on Ubuntu 18.04.6 LTS, and had placed some "important" files in /tmp.

And then I rebooted ....

And, of course, /tmp was cleared ... πŸ™€πŸ™€πŸ™€πŸ™€

This is, of course, configurable 

A quick Google took me here: -

How is the /tmp directory cleaned up?

TL;DR; there's a daemon for that: -

man tmpfiles.d

which includes: -

<snip>

       Example 3. Create a directory and prevent its contents from cleanup


       abrt(1), needs a directory created at boot with specific mode and ownership and its content should be preserved from the automatic cleanup applied to the

       contents of /var/tmp:


           # /usr/lib/tmpfiles.d/tmp.conf

           d /var/tmp 1777 root root 30d

</snip>

I tested this by creating the requisite file: -

vi /etc/tmpfiles.d/tmp.conf

containing: -

d /tmp 1777 root root 20d

and placed a dummy file in /tmp: -

touch /tmp/dave

and then rebooted.

And ... /tmp/dave is still there

Which is nice!

Tuesday, 1 March 2022

Go and GitHub and downstream dependencies

 I'm leaving this here to remind the future me ...

Along with: -

url.<base>.insteadOf

 which is better documented in git-config there's also a need to do something similar when using Go with dependencies in GitHub, namely the GOPRIVATE variable, which was added in Go 1.13 as per the release notes here.

This is to mitigate this exception: -

fatal: could not read Username for 'https://github.com': terminal prompts disabled

 Confirm the import path was entered correctly.

It's also well documented from the Go CLI as per the following: -

go help private

The go command defaults to downloading modules from the public Go module
mirror at proxy.golang.org. It also defaults to validating downloaded modules,
regardless of source, against the public Go checksum database at sum.golang.org.
These defaults work well for publicly available source code.
The GOPRIVATE environment variable controls which modules the go command
considers to be private (not available publicly) and should therefore not use
the proxy or checksum database. The variable is a comma-separated list of
glob patterns (in the syntax of Go's path.Match) of module path prefixes.
For example,
 GOPRIVATE=*.corp.example.com,rsc.io/private
causes the go command to treat as private any module with a path prefix
matching either pattern, including git.corp.example.com/xyzzy, rsc.io/private,
and rsc.io/private/quux.
For fine-grained control over module download and validation, the GONOPROXY
and GONOSUMDB environment variables accept the same kind of glob list
and override GOPRIVATE for the specific decision of whether to use the proxy
and checksum database, respectively.
For example, if a company ran a module proxy serving private modules,
users would configure go using:
 GOPRIVATE=*.corp.example.com
 GOPROXY=proxy.example.com
 GONOPROXY=none
The GOPRIVATE variable is also used to define the "public" and "private"
patterns for the GOVCS variable; see 'go help vcs'. For that usage,
GOPRIVATE applies even in GOPATH mode. In that case, it matches import paths
instead of module paths.
The 'go env -w' command (see 'go help env') can be used to set these variables
for future go command invocations.
For more details, see <https://golang.org/ref/mod#private-modules>.

TL;DR; this is what I needed to add: -

go env -w GOPRIVATE=github.com

More fun with variables and spaces, with added jq

Following on from my earlier post: -

Fun with variables and space

I thought I'd have a little fun using jq to achieve a similar outcome, but without the need to munge the output using sed.

This is with what I came up: -

export my_version=$(ic ks versions --output JSON | jq -j '.kubernetes[] | select(.minor | contains(23)) | .major,".",.minor,".",.patch')

which, you have to admit, is definitely NOT shorter than the alternative, but way more fun.

echo $my_version                                                                                                                        

1.23.4

echo $my_version | hexdump

0000000 31 2e 32 33 2e 34 0a                           
0000007

More fun with pip

Again with the Python and pip  fun, this time on my Mac, where commands such as:  pip3 list and: - pip3 install --upgrade pip were failing w...