[kernel-xen] Greetings, Bug, and Broken Link, and Small Kernel Config Change Request

[Gordan Bobic]

On 04/27/2013 05:24 PM, Steven Haigh wrote:
> On 27/04/2013 8:57 PM, Gordan Bobic wrote:
>> On 27/04/2013 11:21, Steven Haigh wrote:
>>
>>>> Finally - I believe I have found a bug. The last version of
>>>> xen-hypervisor where I had PCI/VGA passthrough working was 4.2.1-6.
>>>>
>>>> The later versions result in error 22: invalid argument error when
>>>> starting the VM. So:
>>>>
>>>> Works:
>>>> xen-hypervisor-4.2.1-6.el6.x86_64.rpm
>>>>
>>>> Don't work:
>>>> xen-hypervisor-4.2.1-7.el6.x86_64.rpm
>>>> xen-hypervisor-4.2.2-1.el6.x86_64.rpm
>>>>
>>>> It is this specific package that seems to be responsible
>>>> (/boot/xen.gz).
>>>> I am running the rest of the xen packages of the latest 4.2.2-1
>>>> version.
>>>>
>>>> Any idea what is going wrong here?
>>>
>>> I know many people have problems with pci and vga passthru. We see
>>> people from all distros / versions in ##xen on freenode IRC. It seems to
>>> be very dependent on hardware - and even small things can cause it to
>>> break.
>>>
>>> What I would suggest is to post the config information to the xen-users
>>> list - then if no success to the xen-devel list with as much information
>>> as possible. Maybe even try in the ##xen IRC channel first.
>>
>> I am not convinced this particular issue is hardware related. An issue
>> with identical symptoms has been reported on various Xen versions going
>> back at least 5 years, e.g.:
>> http://old-list-archives.xen.org/xen-users/2007-04/msg00323.html
>>
>> I have never actually seen an answer regarding what causes it or how to
>> debug it further. But the fact that
>>
>> xen-hypervisor-4.2.1-6.el6.x86_64.rpm works
>>
>> and
>>
>> xen-hypervisor-4.2.1-7.el6.x86_64.rpm doesn't
>>
>> might provide some insight into at least this particular specific
>> instance by bisecting the differences.
>>
>> This is completely reproducible on my system. One always works, the
>> other always fails.
>
> This may well be a Xen bug that needs to be lodged upstream. I recommend
> you still file it - as I've started to use the bug tracker to try and
> keep tabs on all these issues.
>
> The changes between 4.2.1-6 and 4.2.1-7 are:
> * Fri Apr 19 2013 Steven Haigh <netwiz at crc.id.au> - 4.2.1-7
> - XSA-44 (CVE-2013-1917) - Xen PV DoS vulnerability with SYSENTER
> - XSA-46 (CVE-2013-1919) - Several access permission issues with IRQs
> for unprivileged guests
>
> So - it may be possible that a fix in these broke something else.

I'll build some custom rpms with only one of these patches at a time, 
and see which one causes things to stop working. I'll try to do that in 
the next day or two.

>>> I'd also like it if you could file it as a bug at:
>>> http://xen.crc.id.au/bugs
>>>
>>> This way, if we can nail down a fix / working solution, we can easily
>>> document and then transfer it to a support article later on.
>>
>> I can do, but I would rather like to know how to debug this further
>> first, or the bug report is going to have very little "meat" on it.
>> Hmm... I might do a diff between the two package versions and see what
>> that yields.
>
> Well, firstly, I'd document the configuration you are using - DomU
> config, system config etc etc. Having as much info as possible about the
> setup helps. Attach logs from /var/log/xen/ for the boot working (using
> the 4.2.1-6 hypervisor), then upgrade and reboot into 4.2.1-7 and
> document the same.
>
> I know I'm going to have to run it up the flag pole to the xen-devel
> guys - however the xen project doesn't really have a bug tracker at all
> - its all done via mailing lists.

I have filed the bug report, as requested, attaching all the logs the 
xen wiki says should be attached.

http://xen.crc.id.au/bugs/view.php?id=5

There are two tar balls attached - one with the logs from a working 
system with /boot/xen.gz-4.2.1-6, and one with logs from a broken system 
withn /boot/xen.gz-4.2.1-7. Everything except /boot/xen.gz is exactly 
the same (the rest of the stack is 4.2.2-1).

>>>> Finally - I would like to request the following change to the kernel
>>>> configuration, if it wouldn't break things for too many people:
>>>>
>>>> < CONFIG_NR_CPUS=8
>>>> > CONFIG_NR_CPUS=32
>>>> < CONFIG_HOTPLUG_PCI_PCIE=y
>>>> > CONFIG_HOTPLUG_PCI_PCIE=m
>
> I've done these two changes in kernel-xen-3.8.10 that I'm going to put
> in the testing directory:
>      http://au1.mirror.crc.id.au/repo/el6/testing/
>
> It includes two patches to try and fix an issue where the networking vif
> would be disabled due to what the Dom0 thinks is a malicious packet. I'm
> running it in production as a test - but I won't push it to the main
> repo until I'm a little more confident that it won't cause any issues.
>
> kernel-xen-3.8.10-1 is currently building and should be there within the
> next hour...

Great, thanks. :)
I'll test the new kernel tomorrow.

Gordan