s390x can't migrate pre qemu 2.8 to qemu4.2/libvirt6.0 (X->F)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu on IBM z Systems |
Fix Released
|
High
|
bugproxy | ||
libvirt |
Fix Released
|
Undecided
|
|||
libvirt (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
I have discussed this with the s390x developers and the consensus was to report that upstream.
I'll do that but also keep a bug open for tracking here.
If migrating a guest that was initially started on Xenial (qemu 2.5) to a new virtualization stack (qemu 4.2 libvirt 6.0) it fails.
The reason is that the old system has defined the machine type of the time e.g.
machine type: s390-ccw-virtio-2.5
The matching commandline is part:
-machine s390-ccw-
-cpu (not present here)
But on the migration to the new stack this becomes:
-machine s390-ccw-
-cpu z13-base,
The problem is that s390x prior to qemu 2.8 had no cpu modelling.
Therefore it fails with:
qemu-
Source XML:
<domain type='kvm' id='3'>
<name>
<uuid>
<memory unit='KiB'
<currentMemory unit='KiB'
<vcpu placement=
<resource>
<partition>
</resource>
<os>
<type arch='s390x' machine=
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='utc'/>
<on_poweroff>
<on_reboot>
<on_crash>
<devices>
<emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/
<backingStore type='file' index='1'>
<format type='qcow2'/>
<source file='/
<
<target dev='vda' bus='virtio'/>
<alias name='virtio-
<address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/
<
<target dev='vdb' bus='virtio'/>
<alias name='virtio-
<address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0001'/>
</disk>
<interface type='network'>
<mac address=
<source network='default' bridge='virbr0'/>
<target dev='vnet1'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0002'/>
</interface>
<console type='pty' tty='/dev/pts/1'>
<source path='/dev/pts/1'/>
<target type='sclp' port='0'/>
<alias name='console0'/>
</console>
<memballoon model='virtio'>
<alias name='balloon0'/>
<address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0003'/>
</memballoon>
</devices>
<seclabel type='dynamic' model='apparmor' relabel='yes'>
<label>
<imagelabel
</seclabel>
</domain>
Source log/commandline:
2020-01-28 13:58:28.719+0000: starting up libvirt version: 1.3.1, package: 1ubuntu10.29 (Matthew Ruffell <email address hidden> Thu, 31 Oct 2019 10:52:41 +1300), qemu version: 2.5.0 (Debian 1:2.5+dfsg-
LC_ALL=C PATH=/usr/
char device redirected to /dev/pts/1 (label charconsole0)
Target log/commandline:
2020-01-28 13:59:04.396+0000: starting up libvirt version: 6.0.0, package: 0ubuntu1~ppa2 (Christian Ehrhardt <email address hidden> Mon, 13 Jan 2020 13:14:14 +0100), qemu version: 4.2.0Debian 1:4.2-1ubuntu1~
LC_ALL=C \
PATH=/usr/
HOME=/var/
XDG_DATA_
XDG_CACHE_
XDG_CONFIG_
QEMU_AUDIO_DRV=none \
/usr/bin/
-name guest=x-
-S \
-object secret,
-machine s390-ccw-
-cpu z13-base,
-m 512 \
-overcommit mem-lock=off \
-smp 1,sockets=
-uuid c4545c20-
-display none \
-no-user-config \
-nodefaults \
-chardev socket,
-mon chardev=
-rtc base=utc \
-no-shutdown \
-boot strict=on \
-blockdev '{"driver"
-blockdev '{"node-
-blockdev '{"driver"
-blockdev '{"node-
-device virtio-
-blockdev '{"driver"
-blockdev '{"node-
-device virtio-
-netdev tap,fd=
-device virtio-
-chardev pty,id=charconsole0 \
-device sclpconsole,
-incoming defer \
-device virtio-
-sandbox on,obsolete=
-msg timestamp=on
char device redirected to /dev/pts/0 (label charconsole0)
2020-01-
2020-01-28 13:59:04.807+0000: shutting down, reason=failed
Migration command:
$ virsh migrate --live x-guest2 qemu+ssh:
After checking the above I found that this isn't even tied to migration which would have been my first assumption.
The following two together in a domain XML lead to the same error:
<cpu mode='host-model' check='partial'/>
<type arch='s390x' machine=
It turns out that
<cpu mode='host-model' check='partial'/>
is the default.
So when I define the following:
$ cat breakme.xml
<domain type='kvm'>
<name>
<memory unit='KiB'
<os>
<type arch='s390x' machine=
</os>
</domain>
$ virsh define breakme.xml
Domain breakme defined from breakme.xml
I get cpu + old-type:
root@testkvm-
<domain type='kvm'>
<name>
<uuid>
<memory unit='KiB'
<currentMemory unit='KiB'
<vcpu placement=
<os>
<type arch='s390x' machine=
<boot dev='hd'/>
</os>
<cpu mode='host-model' check='partial'/>
<clock offset='utc'/>
<on_poweroff>
<on_reboot>
<on_crash>
<devices>
<emulator>
<controller type='pci' index='0' model='pci-root'/>
<memballoon model='virtio'>
<address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/>
</memballoon>
<panic model='s390'/>
</devices>
</domain>
And that breaks on start:
# virsh start breakme
error: Failed to start domain breakme
error: internal error: qemu unexpectedly closed the monitor: 2020-01-
I see no way to remove
<cpu mode='host-model' check='partial'/>
as it is automatically added back.
Adding this section by default (I guess) needs to be avoided when the underlying machine type (anything prior to s390-ccw-virtio-2.8 or even a bit more) is in use.
Then creating guests would not model cpus like
-cpu ...
and that would work with the old type which implicitly did something like -cpu host.
On that note, host-passthrough existed back then.
And making the guest definition to hold
<cpu mode='host-
makes it work.
So older types might want that (instead of nothing)
Works:
$ cat works.xml
<domain type='kvm'>
<name>
<memory unit='KiB'
<cpu mode='host-
<os>
<type arch='s390x' machine=
</os>
</domain>
What do you think about some compat check on machine-type version vs the auto-addition of cpu host-model that didn't work back then?
Related branches
- Rafael David Tinoco (community): Needs Information
- Canonical Server packageset reviewers: Pending requested
- Canonical Server: Pending requested
-
Diff: 655 lines (+603/-0)8 files modifieddebian/changelog (+12/-0)
debian/patches/series (+6/-0)
debian/patches/ubuntu-aa/apparmor-allow-to-call-vhost-user-gpu.patch (+37/-0)
debian/patches/ubuntu-aa/virt-aa-helper-Add-support-for-smartcard-host-certif.patch (+76/-0)
debian/patches/ubuntu/lp-1861125-qemu-Pass-machine-type-to-virQEMUCapsIsCPUModeSuppor.patch (+105/-0)
debian/patches/ubuntu/lp-1861125-qemu_capabilities-Disable-CPU-models-on-old-s390-mac.patch (+104/-0)
debian/patches/ubuntu/lp-1861125-qemuxml2-test-Add-default-CPU-tests-for-s390-ccw-vir.patch (+242/-0)
debian/patches/ubuntu/lp-1861125-ubuntu-models.patch (+21/-0)
Changed in ubuntu-z-systems: | |
status: | New → Triaged |
importance: | Undecided → High |
assignee: | nobody → bugproxy (bugproxy) |
tags: | added: reverse-proxy-bugzilla s390x |
tags: | added: architecture-s39064 bugnameltc-183486 severity-high targetmilestone-inin2004 |
Changed in libvirt: | |
importance: | Unknown → Undecided |
status: | Unknown → Fix Released |
Changed in libvirt (Ubuntu): | |
status: | New → In Progress |
Changed in ubuntu-z-systems: | |
status: | Triaged → In Progress |
Changed in ubuntu-z-systems: | |
status: | In Progress → Fix Released |
Description of problem:
Libvirt defaults to model an s390x CPU but some older machine types are unable to consume that therefore starting old XML guests or migrations to a old->new system fail.
Version-Release number of selected component (if applicable):
- libvirt 6.0
- qemu 4.2
How reproducible:
100%
Steps to Reproduce: breakme< /name> >524288< /memory> 's390-ccw- virtio- 2.5'>hvm< /type>
1. define a simple guest XML with old machine type
$ cat breakme.xml
<domain type='kvm'>
<name>
<memory unit='KiB'
<os>
<type arch='s390x' machine=
</os>
</domain>
$ virsh define breakme.xml
Domain breakme defined from breakme.xml
2. check the cpu model it got focal-to: ~# virsh dumpxml breakme | grep cpu
I get cpu + old-type:
root@testkvm-
<cpu mode='host-model' check='partial'/>
3. That breaks on start: 28T14:37: 27.477232Z qemu-system-s390x: CPU models are not available: KVM doesn't support CPU models
# virsh start breakme
error: Failed to start domain breakme
error: internal error: qemu unexpectedly closed the monitor: 2020-01-
Actual results:
Fails to start the guest:
qemu-system-s390x: CPU models are not available: KVM doesn't support CPU models
Reason:
It creates a commandline that can't work.
Expected results:
Guest with old machine type can be started-on and/or migrated-to new systems.
Additional info:
I found this while testing migration across releases.
If migrating a guest that was initially started on Xenial (qemu 2.5) to a new virtualization stack (qemu 4.2 libvirt 6.0) it fails.
The reason is that the old system has defined the machine type of the time e.g. virtio- 2.5,accel= kvm,usb= off
machine type: s390-ccw-virtio-2.5
The matching commandline is part:
-machine s390-ccw-
-cpu (not present here)
But on the migration to the new stack this becomes: virtio- 2.5,accel= kvm,usb= off (kept from the source of the migration) aen=on, aefsi=on, ... (modelled on the target)
-machine s390-ccw-
-cpu z13-base,
The problem is that s390x prior to qemu 2.8 had no cpu modelling. system- s390x: CPU models are not available: KVM doesn't support CPU models
Therefore it fails with:
qemu-
Source XML: x-guest2< /name> c4545c20- 5415-4f2c- 86c7-df77eab1f6 bb</uuid> >524288< /memory> >524288< /currentMemory> 'static' >1</vcpu> /machine< /partition> 's390-ccw- virtio- 2.5'>hvm< /type> destroy< /on_poweroff> restart< /on_reboot> destroy< /on_crash> /usr/bin/ qemu-system- s390x</ emulator> var/lib/ uvtool/ libvirt/ images/ x-guest2. qcow'/> var/lib/ uvtool/ libvirt/ images/ x-uvt-b64- Y29tLnVidW50dS5 jbG91ZC5kYWlseT pzZXJ2ZXI6MTYuM DQ6czM5MHggMjAy MDAxMjE= '/>
<backingStore/ > /backingStore>
<domain type='kvm' id='3'>
<name>
<uuid>
<memory unit='KiB'
<currentMemory unit='KiB'
<vcpu placement=
<resource>
<partition>
</resource>
<os>
<type arch='s390x' machine=
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='utc'/>
<on_poweroff>
<on_reboot>
<on_crash>
<devices>
<emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/
<backingStore type='file' index='1'>
<format type='qcow2'/>
<source file='/
<
<target dev='vda'...