[kernel-xen] Greetings, Bug, and Broken Link, and Small Kernel Config Change Request

[Steven Haigh]

On 27/04/2013 8:06 PM, Gordan Bobic wrote:
> Hi,
>
> First of all, thank you very much for picking up the ball where RedHat
> dropped it WRT Xen EL6 kernels. I can relate to the frustration (I felt
> the same about the lack of EL6 for ARM and did something about it).
>
> The mailing list link on the support page:
>
> http://xen.crc.id.au/support/
>
> appears to be broken - it is pointing at:
>
> http://xen.crc.id.au/support/mailing_list/

Ahhh - so it is. It is correct in the dropdown navigation menu, but 
incorrect in /support/. Thanks. Fixed.

> Finally - I believe I have found a bug. The last version of
> xen-hypervisor where I had PCI/VGA passthrough working was 4.2.1-6.
>
> The later versions result in error 22: invalid argument error when
> starting the VM. So:
>
> Works:
> xen-hypervisor-4.2.1-6.el6.x86_64.rpm
>
> Don't work:
> xen-hypervisor-4.2.1-7.el6.x86_64.rpm
> xen-hypervisor-4.2.2-1.el6.x86_64.rpm
>
> It is this specific package that seems to be responsible (/boot/xen.gz).
> I am running the rest of the xen packages of the latest 4.2.2-1 version.
>
> Any idea what is going wrong here?

I know many people have problems with pci and vga passthru. We see 
people from all distros / versions in ##xen on freenode IRC. It seems to 
be very dependent on hardware - and even small things can cause it to break.

What I would suggest is to post the config information to the xen-users 
list - then if no success to the xen-devel list with as much information 
as possible. Maybe even try in the ##xen IRC channel first.

I'd also like it if you could file it as a bug at:
	http://xen.crc.id.au/bugs

This way, if we can nail down a fix / working solution, we can easily 
document and then transfer it to a support article later on.

>
> Finally - I would like to request the following change to the kernel
> configuration, if it wouldn't break things for too many people:
>
> < CONFIG_NR_CPUS=8
>  > CONFIG_NR_CPUS=32
>
> This would make dom0 able to use all CPUs on dual 8-core/16-thread
> systems in dom0 if required. If this is deemed undesirable, the nr_cpus
> kernel boot parameter could be used to limit it (but this doesn't appear
> to work to increase the number of available cores past what is set in
> CONFIG_NR_CPUS. This had me scratching my head for a bit figuring out
> why I could only see 8 CPUs on my 24-thread machine.

There is a misconception of what this actually does. The current setting 
is set to 8 CPUs - which is actually 8 vCPUs for the Dom0. Xen can still 
use the number of CPUs detected by the hypervisor (refer xm info) - 
however you can only bind 8 to Dom0. In reality, this is probably WAAAY 
more than required for most configurations - the best practice is to pin 
between 1-4 for exclusive Dom0 use - most people use 1 or 2.

Once again, what you can see in Dom0 is not related to how many vCPUs 
Xen can utilise.

>
> < CONFIG_HOTPLUG_PCI_PCIE=y
>  > CONFIG_HOTPLUG_PCI_PCIE=m
>
> Unfortunately, there is some buggy hardware out there (including the
> EVGA SR-2 I'm running Xen on) that suffers from this bug:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=908023

This gives me a permission denied error.

> https://bugzilla.redhat.com/show_bug.cgi?id=529153
>
> With pciehp built into the kernel, the only workaround I have found that
> works is as listed in the bug report, but it does involve manually
> editing init in the initramfs to drop the offending device out of pciehp
> binding at the earliest possible time.
>
> With pciehp built as a module, it could either be blacklisted or handled
> in a way that ensures that it doesn't just kill the machine as soon as
> it starts, while at the same time still being available to people who
> actually need to use it.

Just trying to check here, do you mean this is on the Dom0 or DomU? What 
is the actual effect? The machine just stops? Fails to boot? something else?

Please lodge a bug at http://xen.crc.id.au/bugs - it'll be easier to 
track through there...

-- 
Steven Haigh

Email: netwiz at crc.id.au
Web: https://www.crc.id.au
Phone: (03) 9001 6090 - 0412 935 897
Fax: (03) 8338 0299