Nixpanic's Blog

This is an example of how to configure an environment where you can test CloudStack and Gluster. It uses two machines on the same LAN, one acts as a KVM hypervisor and the other as storage and management server. Because the (virtual) networking in the hypervisor is a little more complex than the networking on the management server, the hypervisor will be setup with an OpenVPN connection so that the local LAN is not affected with 'foreign' network traffic.

Being ill a couple of days and have the need to do something (more or less) productive, I thought of giving Fedora-ARM a go. As I use libvirt and virt-manager for work, running an ARM-emulation this way is my preferred setup.

The Fedora Wiki provides with a nice HowTo a good start. Unfortunately there is already a mentioning that qemu-system-arm gets wrong arguments and a script that functions as a (temporary) workaround should be used. This was a note for Fedora 13, I'm running Fedora 14. So, of course I tried to start my VM without the script first:

qemu-system-arm: -device lsi,id=scsi0,bus=pci.0,addr=0x5: Bus 'pci.0' not found

Ups. This really does not seem to work out of the box :-( Reading the script reveals that bus=pci.0 gets replaced by bus=pci. Manually starting qemu-system-arm with some adjusted bus= parameters seems to start the VM.

Obviously there are loads of others hitting similar issues. For Fedora 14 bug 667345 was filed against libvirt for PPC emulation.

With help from gdb and manually executing qemu-system-arm and comparing it with qemu-kvm, it seems that the virtual hardware is configured differently. Most noticible (for me) seems to be the name of the PCI-bus, on qemu-system-arm it is pci and on qemu-kvm it is pci.0. The function qbus_find_recursive() can be used to breakpoint and to check the names of the available busses (bus->name).

I would assume that at least the busses have the same names for emulated hardware, so either bus=pci or bus=pci.0 should work with any qemu-* command. Unfortunately it is unclear to me how qemu-kvm constructs the virtual hardware, the qemu-system-* are more transparent. Depending on the machine that is being emulated the hardware is 'connected' when the machine is created. The virtual hardware is constructed by qemu-kvm/hw/*.c and depends on the type of machine that is created.

Under qemu-kvm/hw/*.c there are some uses of pci_register_bus() where pci as name for the PCI-bus is passed. It seems easy to rename the PCI-busses to pci.0. This change will break any existing scripts/tools that pass bus=pci on the command line, so the solution is not the best. However, packages are temporary available in case someone wants to try this solution.
A post to the qemu-devel list will show if a patch for qemu, or rather libvirt is preferred.

Fixing the name of the PCI-bus is not the only thing I had to do before I could start my Fedora-ARM VM, though.

The emulated hardware versatileab does not have an ISA-bus, therefore -device isa-serial can not work. The Serial device in the Virtual Machine needs to be removed. An alternative would be a usb-serial, but at the moment, virt-manager does not offer this option (you can probably configure it with virsh edit or virsh define though).


There are more limitations with the emulated versatileab machine. It can obviously not cope with the 512MB of RAM I gave it. The result was shown by virt-manager
libvirtError: operation failed: could not query memory balloon allocation
and in my /var/log/libvirt/fedora.arm.log
qemu: hardware error: pl011_read: Bad offset 101f1018
Giving the machine only 256MB RAM instead of 512MB RAM seems to make this issue go away too.