Getting the architecture right

 I'm building container images using Docker and Podman, and wanted to provide the consuming engineer with a way to specify the architecture e.g. amd64 or s390x etc. of a dependent download from within the Dockerfile itself.

The Dockerfile was hard-coded to pull down the amd64 version; which doesn't work too well on my IBM Z s390x boxes.

Therefore, I set an argument within the Dockerfile, via the ARG parameter, which I left defaulted to amd64 but which allowed the engineer to then over-ride that argument when they run the docker build or podman build command.

This over-ride works via the --build-arg option of the build command.

However, I wanted to be even more lazy and allow the OS upon which the command was being run to set the architecture.

I started with the following: -

docker build --build-arg=$(uname -p) -f Dockerfile .

which worked on my IBM Z box ( running Ubuntu 18.04.06 ).

However, on my x86-64 box, running Ubuntu 20.04.5, this failed because, it transpires, uname -p returns x86_64 rather than amd64.

However, StackExchange came to my rescue : -

On a Debian-based system, the bullet-proof way of determining the architecture, as appropriate for use in a package’s file name, is
dpkg --print-architecture
Note that architecture-independent packages use all there, and you’d have to know that in advance.

So I changed my command to: -

docker build --build-arg ARCH=$(dpkg --print-architecture) -f Dockerfile .

which did the trick.

For the record, docker build and podman build are, in this context, the same

More fun with pip

Again with the Python and pip fun, this time on my Mac, where commands such as: 

pip3 list

and: -

pip3 install --upgrade pip

were failing with: -

WARNING: There was an error checking the latest version of pip.

Long story short, the error was actually more obvious than I'd realised ....

In essence, it was actually telling me what was going wrong ...

Looking in indexes:.......

where the location was an internal repository for **ONE** single Python module, which would never have the things I was trying to install e.g. pip itself

I updated the config: -

vi ~/.pip/pip.conf

and commented out the aberrant index ... 

And now we're off to the races ...

Installing pylint on Linux - there's more than one way ...

 I'm running some Travis builds of a project which includes Python modules, and wanted to manually run the pylint linting tool across my GitHub repo, to compare/contrast what Travis was telling me.

I'm running Ubuntu 20.04.5 LTS so just went ahead and installed via the Aptitude package manager: -

apt-get update && apt-get install -y pylint

which seemed fine.

Then I noticed that the results differed - when I checked Travis, I was using pylint 2.15.2 whereas, on Ubuntu, I was way behind the curve: -

pylint --version

pylint 2.4.4
astroid 2.3.3
Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0]

My mistake ? Using Aptitude ...

I uninstalled pylint 

apt-get remove -y pylint

and installed it using the Package Installer (for) Python, aka pip

I did notice that the installation location was different: -

- apt-get install ->>> /usr/bin/pylint

- pip  ->>> /usr/local/bin/pylint

so had to restart my shell to pick up the new version: -

pylint --version

pylint 2.15.3
astroid 2.12.10
Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0]

which is better ( part of my planned change will be to have the Travis job use 2.15.3 as well )

Right, onwards for some linting .....

Been a while - back with some wget fun

Whilst digging into some Kubernetes testing, validating a fix that I'd made to my cluster, I was deploying both Nginx and Busybox pods.

The Nginx pods were deployed across all my Compute Nodes, to provide a web server tier, and I was using Busybox, again across all the Compute Nodes, to validate that Nginx connectivity was clean and green.

Having obtained the individual IPs of each of the Nginx pods- I'm using Calico Node as my Container Network Interface (CNI) layer - I wanted to hit each Nginx pod from each of the Busybox pods, in turn.

Now, ordinarily, I'd use a command such as: -

curl http://192.168.1.24:80/index.html

to retrieve the sample HTML page that Nginx presents.

However, Busybox is a very cut-down Linux-like environment and, therefore, it doesn't include the curl command.

Thankfully, Busybox does include the wget command, so I was able to use that in a similar manner: -

wget -q --output-document - http://192.168.1.24:80/index.html

which did the trick i.e. dumped the content of nginx.html to the console ( stdout ).

The key parameters are as follows: -

-q == run wget in quiet mode, only returning the required web page, rather than the normal debug

--output-document == tells wget what and where to return the output

- == tells --output-document to write to stdout rather than a specific file/document

Nice !

Grokking grep

A colleague was tinkering with grep and, thanks to him, I discovered a bit more about the trusty little utility.

I had not really explored the -e switch: -

     -e pattern, --regexp=pattern

             Specify a pattern used during the search of the input: an input line is selected if it matches any of the specified patterns.  This option is

             most useful when multiple -e options are used to specify multiple patterns, or when a pattern begins with a dash (‘-’).

but he pointed out that that switch can have problems when parsing, say, a list of strings that are similar but different.

As an example, I have a set of files: -

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 dave

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 dave_1

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:36 dave_2

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:36 dave_3

If I want to e.g. query a directory for all files containing the word 'dave' I could do this: -

ls -al | grep dave

or, to be more specific: -

ls -al | grep -e Dave

both of which return: -

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 dave

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 dave_1

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:36 dave_2

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:36 dave_3

However, if I only want to return the file that's named 'dave' and ignore the rest, I'm somewhat stymied.

However, grep -w comes to the rescue: -

     -w, --word-regexp

             The expression is searched for as a word (as if surrounded by ‘[[:<:]]’ and ‘[[:>:]]’; see re_format(7)).  This option has no effect if -x is

             also specified.

so I can run: -

ls -al | grep -w dave

and just get this: -

-rw-r--r--   1 hayd  wheel    0 22 Jul 17:34 Dave

Isn't that nifty ?

To start, press any key .... hey, where's the [Any] key ?

So I was writing some scripts for a demo that I delivered earlier today...

One script will run on my Mac, using the Zsh shell, the other will run on an Ubuntu box, using Bash.

In both cases, I wanted to pause the script and wait for user input before proceeding.

On Ubuntu, I did this: -

read -p "Press [Enter] to continue"

Sadly, however, on macOS 12.4 via zsh, that didn't work ...

read: -p: no coprocess

Thankfully, there's a better / different way: -

read "?Press [Enter] to continue"

which is nice

Today I Learned - Shorter redirection, what's not to like ?

So I write a lot of scrips for Bash and Zsh, and often make use of redirection where, for example, I generate a variable by running another command within parentheses.

As a specific example, I'd query my Floating IP and persist it to a text file for future use: -

ic is floating-ips --output JSON | jq -r '.[].address' > ~/floating_ip.txt

Later on, I'd look to use that Floating IP for, for example, a SSH connection, as follows: -

export floating_ip=$(cat floating_ip.txt)

ssh root@$floating_ip

Authorized uses only. All activity may be monitored and reported.

root@163.61.95.34's password:

However, one of my colleague's did point out that I could reduce the number of characters typed when setting the floating_ip variable: -

export floating_ip=$(< floating_ip.txt)

See, that saves a whole two characters ....

Mind you, I can also reduce things further via: -

ssh root@$(cat floating_ip.txt)

I love shelling !