View Issue Details

IDProjectCategoryView StatusLast Update
0000648channel: kernel/el7kernel-mlpublic2016-05-10 08:45
Reporterpjwelsh Assigned Toburakkucat  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionno change required 
Summary0000648: xen disabled in kernel-ml-4.5.3-1.el7.elrepo.x86_64?
DescriptionThere is no /proc/xen when booting kernel-ml-4.5.3-1.el7.elrepo.x86_64. This is working in kernel-ml-4.5.0-1.el7.elrepo.x86_64 at least (as that is what I've fallen back too using). A simple reboot to kernel-ml-4.5.0 gets my new production system xen server running again.
Additional InformationLooks like 4.5.3 was built with support, however:
# grep -i xen /boot/config-4.5.3-1.el7.elrepo.x86_64
CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_512GB=y
CONFIG_XEN_SAVE_RESTORE=y
# CONFIG_XEN_DEBUG_FS is not set
CONFIG_XEN_PVH=y
CONFIG_PCI_XEN=y
CONFIG_XEN_PCIDEV_FRONTEND=m
CONFIG_XEN_BLKDEV_FRONTEND=m
CONFIG_XEN_BLKDEV_BACKEND=m
CONFIG_XEN_SCSI_FRONTEND=m
CONFIG_NETXEN_NIC=m
CONFIG_XEN_NETDEV_FRONTEND=m
CONFIG_XEN_NETDEV_BACKEND=m
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=m
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
CONFIG_TCG_XEN=m
CONFIG_XEN_WDT=m
CONFIG_XEN_FBDEV_FRONTEND=m
# Xen driver support
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SELFBALLOONING=y
CONFIG_XEN_BALLOON_MEMORY_HOTPLUG=y
CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT=512
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=m
CONFIG_XEN_BACKEND=y
CONFIG_XENFS=m
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=m
CONFIG_XEN_GRANT_DEV_ALLOC=m
CONFIG_SWIOTLB_XEN=y
CONFIG_XEN_TMEM=m
CONFIG_XEN_PCIDEV_BACKEND=y
CONFIG_XEN_SCSI_BACKEND=m
CONFIG_XEN_PRIVCMD=m
CONFIG_XEN_ACPI_PROCESSOR=m
CONFIG_XEN_MCE_LOG=y
CONFIG_XEN_HAVE_PVMMU=y
CONFIG_XEN_EFI=y
CONFIG_XEN_AUTO_XLATE=y
CONFIG_XEN_ACPI=y
CONFIG_XEN_SYMS=y
CONFIG_XEN_HAVE_VPMU=y

This is the exact same output as from /boot/config-4.5.0-1.el7.elrepo.x86_64 so I'm not sure why /proc/xen is missing from kernel-ml-4.5.3
TagsNo tags attached.

Activities

burakkucat

2016-05-09 10:07

administrator   ~0004756

Very puzzling. :frown:

Here follows the results from my investigations --

[Build64R7 config-4.5]$ ls -1
config-4.5.0-x86_64
config-4.5.1-x86_64
config-4.5.2-x86_64
config-4.5.3-x86_64
[Build64R7 config-4.5]$ for C in *
> do grep -c XEN $C
> done
47
47
47
47
[Build64R7 config-4.5]$ grep XEN config-4.5.0-x86_64 > list-4.5.0-x86_64
[Build64R7 config-4.5]$ grep XEN config-4.5.1-x86_64 > list-4.5.1-x86_64
[Build64R7 config-4.5]$ grep XEN config-4.5.2-x86_64 > list-4.5.2-x86_64
[Build64R7 config-4.5]$ grep XEN config-4.5.3-x86_64 > list-4.5.3-x86_64
[Build64R7 config-4.5]$ sdiff -s list-4.5.0-x86_64 list-4.5.1-x86_64
[Build64R7 config-4.5]$ sdiff -s list-4.5.1-x86_64 list-4.5.2-x86_64
[Build64R7 config-4.5]$ sdiff -s list-4.5.2-x86_64 list-4.5.3-x86_64
[Build64R7 config-4.5]$

Clearly there is no mishap in our configuration of the kernel packages nor is there any problem in the builds. So let's look and see if there is any hint in the change logs that accompany the source tarballs --

[Build64R7 kernels]$ rpm -qp --changelog kernel-ml-4.5.3-1.el7.elrepo.nosrc.rpm | head -n 15
* Thu May 05 2016 Alan Bartlett <ajb@elrepo.org> - 4.5.3-1
- Updated with the 4.5.3 source tarball.
- [https://www.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.5.3]

* Wed Apr 20 2016 Alan Bartlett <ajb@elrepo.org> - 4.5.2-1
- Updated with the 4.5.2 source tarball.
- [https://www.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.5.2]

* Sat Apr 16 2016 Alan Bartlett <ajb@elrepo.org> - 4.5.1-1
- Updated with the 4.5.1 source tarball.
- [https://www.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.5.1]

* Mon Mar 14 2016 Alan Bartlett <ajb@elrepo.org> - 4.5.0-1
- Updated with the 4.5 source tarball.

[Build64R7 kernels]$ cd /var/tmp
[Build64R7 tmp]$ wget https://www.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.5.1

<snip>

2016-05-09 11:55:08 (2.24 MB/s) - ‘ChangeLog-4.5.1’ saved [270355/270355]

[Build64R7 tmp]$ wget https://www.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.5.2

<snip>

2016-05-09 11:55:28 (1.81 MB/s) - ‘ChangeLog-4.5.2’ saved [156083/156083]

[Build64R7 tmp]$ wget https://www.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.5.3

<snip>

2016-05-09 11:55:51 (726 KB/s) - ‘ChangeLog-4.5.3’ saved [247289/247289]

[Build64R7 tmp]$ ls -1
ChangeLog-4.5.1
ChangeLog-4.5.2
ChangeLog-4.5.3
[Build64R7 tmp]$ for C in ChangeLog*
> do echo $C
> grep -i xen $C
> echo
> done
ChangeLog-4.5.1
      Modules linked in: ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi xen_kbdfront xen_netfront xen_fbfront xen_blkfront
      Hardware name: Xen HVM domU, BIOS 4.3.1OVM 05/14/2014
    x86/iopl: Fix iopl capability check on Xen PV
    unprivileged. This didn't work right on Xen PV. Fix it.
    x86/iopl/64: Properly context-switch IOPL on Xen PV
    On Xen PV, regs->flags doesn't reliably reflect IOPL and the
    specific to Xen PV. After the dust settles, we can merge this with

ChangeLog-4.5.2
    xen/events: Mask a moving irq

ChangeLog-4.5.3
    Acked-by: Sumit Saxena <sumit.saxena@broadcom.com>
    xen kconfig: don't "select INPUT_XEN_KBDDEV_FRONTEND"
    The Xen framebuffer driver selects the xen keyboard driver, so the latter
    will be built-in if XEN_FBDEV_FRONTEND=y. However, when CONFIG_INPUT
    drivers/input/built-in.o: In function `xenkbd_remove':
    xen-kbdfront.c:(.text+0x2f0): undefined reference to `input_unregister_device'
    xen-kbdfront.c:(.text+0x30e): undefined reference to `input_unregister_device'
    XEN_FBDEV_FRONTEND in it but not INPUT_XEN_KBDDEV_FRONTEND. The Kconfig
    fragment we ship in the kernel (kernel/configs/xen.config) however
    Fixes: 36c1132e34bd ("xen kconfig: fix select INPUT_XEN_KBDDEV_FRONTEND")
    x86/mm/xen: Suppress hugetlbfs in PV guests
    propagated out of xen_set_pte_at(), just like for various of its
    Cc: xen-devel <xen-devel@lists.xenproject.org>

[Build64R7 tmp]$

It looks as if this is something you will need to report, upstream, in the Kernel Bug Tracker [1]. Once done, please note the bz number here for our reference.

[1] https://bugzilla.kernel.org/

pjwelsh

2016-05-09 10:48

reporter   ~0004757

Upstream bug filed https://bugzilla.kernel.org/show_bug.cgi?id=117901
Thank you very much for showing the examination path you took! I will try to remember to expand my bug reports with what you have shown.

pjwelsh

2016-05-09 13:20

reporter   ~0004758

I don't think it's a kernel issue now that I've looked at this a little more. I don't yet have a handle on the what happens on a kernel install, but I think that's where the issue is. Running /usr/bin/grub-bootxen.sh manually may have corrected the issue. After kernel-ml-4.5.3 was installed, my /boot/grub2/grub.cfg only showed "xen" results for the older kernel-ml-4.5.0 kernel:
# grep 4\.5 /boot/grub2/grub.cfg |grep xen
    module /vmlinuz-4.5.0-1.el7.elrepo.x86_64 placeholder # root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
        menuentry 'CentOS Linux, with Xen 4.6.1-6.el7 and Linux 4.5.0-1.el7.elrepo.x86_64' --class centos --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.5.0-1.el7.elrepo.x86_64-advanced-ee72ba9a-e2b1-491d-8acc-d0d001f8a731' {
            module /vmlinuz-4.5.0-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
        menuentry 'CentOS Linux, with Xen 4.6 and Linux 4.5.0-1.el7.elrepo.x86_64' --class centos --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.5.0-1.el7.elrepo.x86_64-advanced-ee72ba9a-e2b1-491d-8acc-d0d001f8a731' {
            module /vmlinuz-4.5.0-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
        menuentry 'CentOS Linux, with Xen xen and Linux 4.5.0-1.el7.elrepo.x86_64' --class centos --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.5.0-1.el7.elrepo.x86_64-advanced-ee72ba9a-e2b1-491d-8acc-d0d001f8a731' {
            module /vmlinuz-4.5.0-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
----------------------------------------------------------------


Manually running /usr/bin/grub-bootxen.sh now produces additional results for the 4.5.3 kernel:
# grep 4\.5 /boot/grub2/grub.cfg |grep xen
    module /vmlinuz-4.5.3-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
        menuentry 'CentOS Linux, with Xen 4.6.1-6.el7 and Linux 4.5.3-1.el7.elrepo.x86_64' --class centos --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.5.3-1.el7.elrepo.x86_64-advanced-ee72ba9a-e2b1-491d-8acc-d0d001f8a731' {
            module /vmlinuz-4.5.3-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
        menuentry 'CentOS Linux, with Xen 4.6.1-6.el7 and Linux 4.5.0-1.el7.elrepo.x86_64' --class centos --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.5.0-1.el7.elrepo.x86_64-advanced-ee72ba9a-e2b1-491d-8acc-d0d001f8a731' {
            module /vmlinuz-4.5.0-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
        menuentry 'CentOS Linux, with Xen 4.6 and Linux 4.5.3-1.el7.elrepo.x86_64' --class centos --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.5.3-1.el7.elrepo.x86_64-advanced-ee72ba9a-e2b1-491d-8acc-d0d001f8a731' {
            module /vmlinuz-4.5.3-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
        menuentry 'CentOS Linux, with Xen 4.6 and Linux 4.5.0-1.el7.elrepo.x86_64' --class centos --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.5.0-1.el7.elrepo.x86_64-advanced-ee72ba9a-e2b1-491d-8acc-d0d001f8a731' {
            module /vmlinuz-4.5.0-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
        menuentry 'CentOS Linux, with Xen xen and Linux 4.5.3-1.el7.elrepo.x86_64' --class centos --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.5.3-1.el7.elrepo.x86_64-advanced-ee72ba9a-e2b1-491d-8acc-d0d001f8a731' {
            module /vmlinuz-4.5.3-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodeset
        menuentry 'CentOS Linux, with Xen xen and Linux 4.5.0-1.el7.elrepo.x86_64' --class centos --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-4.5.0-1.el7.elrepo.x86_64-advanced-ee72ba9a-e2b1-491d-8acc-d0d001f8a731' {
            module /vmlinuz-4.5.0-1.el7.elrepo.x86_64 placeholder root=UUID=ee72ba9a-e2b1-491d-8acc-d0d001f8a731 ro rootflags=subvol=root rhgb quiet console=hvc0 earlyprintk=xen nomodese
----------------------------------------------------------------

I am setting up another test server to verify since this one is production and I can not shutdown to test very soon.

If it's a kernel install issue, where should I file a bug report?

toracat

2016-05-09 16:48

administrator   ~0004759

If you had to run /usr/bin/grub-bootxen.sh manually, and if that fixed the issue, then you might want to ask on the centos-virt mailing list.

pjwelsh

2016-05-09 18:50

reporter   ~0004760

Test system up and I can confirm that manually running /usr/bin/grub-bootxen.sh will properly allow for the /proc/xen to be populated. I will pursue this issue with the centos-virt peeps.
Please close.

burakkucat

2016-05-10 08:45

administrator   ~0004761

Closing as "not a bug". (As recommended).

Issue History

Date Modified Username Field Change
2016-05-08 20:25 pjwelsh New Issue
2016-05-08 20:25 pjwelsh Status new => assigned
2016-05-08 20:25 pjwelsh Assigned To => burakkucat
2016-05-09 10:07 burakkucat Note Added: 0004756
2016-05-09 10:48 pjwelsh Note Added: 0004757
2016-05-09 11:18 burakkucat Severity minor => major
2016-05-09 13:20 pjwelsh Note Added: 0004758
2016-05-09 16:48 toracat Note Added: 0004759
2016-05-09 18:50 pjwelsh Note Added: 0004760
2016-05-10 08:45 burakkucat Note Added: 0004761
2016-05-10 08:45 burakkucat Status assigned => closed
2016-05-10 08:45 burakkucat Resolution open => no change required