Nesting VMs - not quite as cosy as it sounds....

I wrote about this a few months back: -

Kata Containers and Ubuntu Linux - lessons learned - 3/many - a WIP

in the context of VM nesting being a pain ....

For context, I'm trying ( and failing ) to get Kata Containers fully running inside an Ubuntu VM running on VMware Fusion on my Mac.

This is what I currently have: -

Host: macOS 11.5 Big Sur

Virtualisation: VMware Fusion 12.1.2

Guest: Ubuntu 20.04.2 LTS

Kernel: 5.4.0-80-generic #90-Ubuntu SMP

kata-runtime  : 2.1.0

QEMU: 5.2.0

I've been experimenting with various container runtimes here, including containerd and CRI-O 

However, each and every time I'm hitting the same nested virtualisation issue

Most recently, when I try and use crictl and runp to start a container using the Kata 2.0 runtime: -

sudo crictl runp test/testdata/sandbox_config.json

FATA[0002] run pod sandbox: rpc error: code = Unknown desc = CreateContainer failed: failed to launch qemu: exit status 1, error messages from qemu log: qemu-system-x86_64: error: failed to set MSR 0x48d to 0x5600000016

qemu-system-x86_64: ../target/i386/kvm.c:2701: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

: unknown 

I've tried various hacks to mitigate this but I just cannot get past it ...

More digging is required, or this combination is a bust - thankfully I have many other options, including IBM Cloud Virtual Servers.......

More to come ....