View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001223 | channel: elrepo/el7 | --elrepo--request-for-enhancement-- | public | 2022-05-19 15:53 | 2022-06-21 19:06 |
| Reporter | svreid | Assigned To | pperry | ||
| Priority | normal | Severity | minor | Reproducibility | N/A |
| Status | assigned | Resolution | open | ||
| Summary | 0001223: Would like to request kmod for NVMe drivers | ||||
| Description | Would it be possible to provide the latest NVMe drivers as a kmod? The version included with EL7 isn't able to detect devices that are visible under EL8. Thanks! | ||||
| Tags | No tags attached. | ||||
| Reported upstream | |||||
|
|
Please can you provide more information. I must admit I don't know much about nvme drivers - I have 2 x NVMe drives on rhel7 and they just work. What is the driver in question - presumably nvme and nvme_core? Or at least that's what's loaded on my rhel7 machine: # lsmod | grep -i nvme nvme 32382 2 nvme_core 63547 5 nvme Also it would be useful to show examples of hardware that is supported in rhel8 but not rhel7. A cursory look at the supported hardware (with modinfo) for rhel7 vs rhel8, there are certainly more devices listed: RHEL7: alias: pci:v*d*sv*sd*bc01sc08i02* alias: pci:v00001E0Fd00000007sv00001028sd*bc*sc*i* alias: pci:v0000144Dd0000A824sv00001028sd*bc*sc*i* alias: pci:v0000144Dd0000A822sv*sd*bc*sc*i* alias: pci:v0000144Dd0000A821sv*sd*bc*sc*i* alias: pci:v00001C5Fd00000540sv*sd*bc*sc*i* alias: pci:v00001C58d00000023sv*sd*bc*sc*i* alias: pci:v00001C58d00000003sv*sd*bc*sc*i* alias: pci:v00001BB1d00000100sv*sd*bc*sc*i* alias: pci:v00008086d00005845sv*sd*bc*sc*i* alias: pci:v00008086d0000F1A6sv*sd*bc*sc*i* alias: pci:v00008086d0000F1A5sv*sd*bc*sc*i* alias: pci:v00008086d00000A55sv*sd*bc*sc*i* alias: pci:v00008086d00000A54sv*sd*bc*sc*i* alias: pci:v00008086d00000A53sv*sd*bc*sc*i* alias: pci:v00008086d00000953sv*sd*bc*sc*i* RHEL8: alias: pci:v*d*sv*sd*bc01sc08i02* alias: pci:v0000106Bd00002005sv*sd*bc*sc*i* alias: pci:v0000106Bd00002003sv*sd*bc*sc*i* alias: pci:v0000106Bd00002001sv*sd*bc*sc*i* alias: pci:v00001D0Fd0000CD02sv*sd*bc*sc*i* alias: pci:v00001D0Fd0000CD01sv*sd*bc*sc*i* alias: pci:v00001D0Fd0000CD00sv*sd*bc*sc*i* alias: pci:v00001D0Fd00008061sv*sd*bc*sc*i* alias: pci:v00001D0Fd00000065sv*sd*bc*sc*i* alias: pci:v00001D0Fd00000061sv*sd*bc*sc*i* alias: pci:v00002646d00002263sv*sd*bc*sc*i* alias: pci:v00002646d00002262sv*sd*bc*sc*i* alias: pci:v00001D97d00002263sv*sd*bc*sc*i* alias: pci:v000015B7d00002001sv*sd*bc*sc*i* alias: pci:v00001C5Cd00001504sv*sd*bc*sc*i* alias: pci:v00001CC1d00008201sv*sd*bc*sc*i* alias: pci:v000010ECd00005762sv*sd*bc*sc*i* alias: pci:v00001D1Dd00002601sv*sd*bc*sc*i* alias: pci:v00001D1Dd00002807sv*sd*bc*sc*i* alias: pci:v00001D1Dd00001F1Fsv*sd*bc*sc*i* alias: pci:v00001B4Bd00001092sv*sd*bc*sc*i* alias: pci:v00001987d00005016sv*sd*bc*sc*i* alias: pci:v0000144Dd0000A822sv*sd*bc*sc*i* alias: pci:v0000144Dd0000A821sv*sd*bc*sc*i* alias: pci:v00001C5Fd00000540sv*sd*bc*sc*i* alias: pci:v00001C58d00000023sv*sd*bc*sc*i* alias: pci:v00001C58d00000003sv*sd*bc*sc*i* alias: pci:v00001BB1d00000100sv*sd*bc*sc*i* alias: pci:v0000126Fd00002263sv*sd*bc*sc*i* alias: pci:v00008086d00005845sv*sd*bc*sc*i* alias: pci:v00008086d0000F1A6sv*sd*bc*sc*i* alias: pci:v00008086d0000F1A5sv*sd*bc*sc*i* alias: pci:v00008086d00000A55sv*sd*bc*sc*i* alias: pci:v00008086d00000A54sv*sd*bc*sc*i* alias: pci:v00008086d00000A53sv*sd*bc*sc*i* alias: pci:v00008086d00000953sv*sd*bc*sc*i* |
|
|
Something I have come across before, but it may be more an issue that the device is not detected (but is supported) so the nvme driver isn't automatically loaded and hence is not in the initramfs image. The solution may be as simple as forcing the nvme driver to load with a dracut conf file containing: add_drivers+=" nvme " Are you able to load the nvme module with modprobe and then see if the device is visible, or is it genuinely a case that some devices are not supported by the nvme driver? A couple of references: https://bugzilla.redhat.com/show_bug.cgi?id=910734 https://bugzilla.redhat.com/show_bug.cgi?id=1368828 |
|
|
Here's what I see when I have CentOS 8 running: # lsmod | grep -i nvm nvme 45056 6 name_core 114688 7 nvme t10_p1 15384 2 sd_mod, nvme_core I don't have the machine at my disposal to try your other suggestions at the moment. My ability to test for SL 7 is limited as I need to be able to see the NVMe disks to install the OS, but have not been able to see them when attempting to install or using a live image earlier than CentOS 8. |
|
|
lspci identities the device as "Raid Bus Controller: Intel Corporation Volume Management Device MNVe Raid Controller (rev 04) Under CentOS 7.9 I loaded NVMe with modprobe, but was still unable to see the device. It does show up in lspci, but the associated volumes aren't visible (they are under CentOS 8). |
|
|
Detailed `lspci -vv` 64:00.5 RAID bus controller: Intel Corporation Volume Management Device NVMe RAID Controller (rev 04) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes NUMA node: 0 Region 0: Memory at 213ffc000000 (64-bit, prefetchable) [size=32M] Region 2: Memory at c2000000 (32-bit, non-prefetchable) [size=32M] Region 4: Memory at 213ffff00000 (64-bit, non-prefetchable) [size=1M] Capabilities: [80] MSI-X: Enable- Count=64 Masked- Vector table: BAR=4 offset=00000000 PBA: BAR=4 offset=00001000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0 ExtTag- RBE- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- |
|
|
With the 5.14-lt kernel: 0000:64:00.5 RAID bus controller: Intel Corporation Volume Management Device NVMe RAID Controller (rev 04) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes NUMA node: 0 Region 0: Memory at 213ffc000000 (64-bit, prefetchable) [size=32M] Region 2: Memory at c2000000 (32-bit, non-prefetchable) [size=32M] Region 4: Memory at 213ffff00000 (64-bit, non-prefetchable) [size=1M] Capabilities: [80] MSI-X: Enable+ Count=64 Masked- Vector table: BAR=4 offset=00000000 PBA: BAR=4 offset=00001000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0 ExtTag- RBE- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: vmd Kernel modules: vmd |
|
|
Based on this, I'm thinking this hardware will need a new `vmd.ko` via `./drivers/pci/controller/vmd.c` |
|
|
Thanks, @jcpunk, for the notes. Looks like vmd.c first appeared in kernel-4.18. Not sure how hard/easy it is to backport to el7 (kernel-3.10). |
|
|
@jcpunk thanks for the extra info. Having reviewed VMD, it would appear VMD is compiled into the kernel for all RHEL releases RHEL7 through to RHEL9. Therefore I do not believe we can backport a vmd.ko module and use that to override what is already built into the RHEL kernel. CONFIG_VMD=y So this doesn't look like something we can fix with a kmod release. The solution would appear to be a custom kernel with the appropriate hardware support. |
|
|
In fact, In el7, vmc.c is in a different location: arch/x86/pci/vmd.c. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2022-05-19 15:53 | svreid | New Issue | |
| 2022-05-19 15:53 | svreid | Status | new => assigned |
| 2022-05-19 15:53 | svreid | Assigned To | => stindall |
| 2022-05-19 15:56 | burakkucat | Assigned To | stindall => toracat |
| 2022-05-19 15:56 | burakkucat | Category | --elrepo--OTHER-- => --elrepo--request-for-enhancement-- |
| 2022-05-19 15:57 | burakkucat | Reproducibility | have not tried => N/A |
| 2022-05-19 18:33 | pperry | Note Added: 0008353 | |
| 2022-05-19 18:48 | pperry | Note Added: 0008354 | |
| 2022-05-19 18:50 | pperry | Assigned To | toracat => pperry |
| 2022-05-19 19:17 | svreid | Note Added: 0008355 | |
| 2022-06-01 15:59 | svreid | Note Added: 0008382 | |
| 2022-06-21 11:01 | jcpunk | Note Added: 0008436 | |
| 2022-06-21 11:52 | jcpunk | Note Added: 0008437 | |
| 2022-06-21 12:01 | jcpunk | Note Added: 0008438 | |
| 2022-06-21 14:02 | toracat | Note Added: 0008439 | |
| 2022-06-21 19:04 | pperry | Note Added: 0008443 | |
| 2022-06-21 19:05 | pperry | Note Edited: 0008443 | |
| 2022-06-21 19:06 | toracat | Note Added: 0008444 |
