View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0001075 | channel: elrepo/el8 | --elrepo--request-for-enhancement-- | public | 2021-02-15 23:19 | 2021-12-17 14:05 |
| Reporter | jfut | Assigned To | toracat | ||
| Priority | normal | Severity | minor | Reproducibility | N/A |
| Status | resolved | Resolution | fixed | ||
| Summary | 0001075: Request for kmod-drbd84 for EL8 | ||||
| Description | Would you be able to build kmod-drbd84 for EL8? kmod-drbd90 is already provided, but Ganeti, a virtual machine cluster management tool, does not yet support DRBD9. I checked and found that kernel-4.18.0-240.el8.src.rpm(linux-4.18.0-240.el8.tar.xz) contains the source code of drbd 8.4.10, and confirmed that it is possible to build it manually using an approach like sata_nv-kmod. # grep REL_VERSION /usr/src/kernels/4.18.0-240.el8.x86_64/include/linux/drbd.h #define REL_VERSION "8.4.10" # make -C /usr/src/kernels/4.18.0-240.el8.x86_64 modules M=$PWD CONFIG_BLK_DEV_DRBD=m | ||||
| Tags | No tags attached. | ||||
|
|
Sure, we will try to build and release drbd84 for el8 (and -utils). |
|
|
Thank you very much! (Ah, we also need -utils). |
|
|
The following packages have been released to the elrepo repository: x86_64 kmod-drbd84-8.4.10-1.2.el8_3.elrepo.x86_64.rpm drbd84-utils-9.15.1-1.el8.elrepo.x86_64.rpm drbd84-utils-sysvinit-9.15.1-1.el8.elrepo.x86_64.rpm src kmod-drbd84-8.4.10-1.2.el8_3.elrepo.src.rpm drbd84-utils-9.15.1-1.el8.elrepo.src.rpm Please note that the kmod package was built against the EL 8.3 GA kernel, 4.18.0-240.el8. It is unlikely to work for newer kernels. Rebuilding against the latest kernel is in the plan. |
|
|
Thanks for the build. I tried it, but I got a WARNING during installation and drbd.ko failed to load. Is there something wrong with my environment? # cat /etc/redhat-release CentOS Linux release 8.3.2011 # uname -a Linux centos8node01.example.org 4.18.0-240.10.1.el8_3.x86_64 #1 SMP Mon Jan 18 17:05:51 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux # dnf --enablerepo=elrepo install kmod-drbd84 drbd84-utils Last metadata expiration check: 0:34:54 ago on Fri 19 Feb 2021 11:56:34 AM JST. Dependencies resolved. =================================================================================================================== Package Architecture Version Repository Size =================================================================================================================== Installing: drbd84-utils x86_64 9.15.1-1.el8.elrepo elrepo 1.4 M kmod-drbd84 x86_64 8.4.10-1.2.el8_3.elrepo elrepo 197 k Transaction Summary =================================================================================================================== Install 2 Packages Total download size: 1.6 M Installed size: 5.7 M Is this ok [y/N]: y Downloading Packages: (1/2): kmod-drbd84-8.4.10-1.2.el8_3.elrepo.x86_64.rpm 81 kB/s | 197 kB 00:02 (2/2): drbd84-utils-9.15.1-1.el8.elrepo.x86_64.rpm 319 kB/s | 1.4 MB 00:04 ------------------------------------------------------------------------------------------------------------------- Total 343 kB/s | 1.6 MB 00:04 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : kmod-drbd84-8.4.10-1.2.el8_3.elrepo.x86_64 1/2 Running scriptlet: kmod-drbd84-8.4.10-1.2.el8_3.elrepo.x86_64 1/2 depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_seq_printf_stats depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_get_cumulative depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_del depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_committed depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_get depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_try_get depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_element_by_index depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_create depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_try_lock depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_destroy depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_reset depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_is_used depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_seq_dump_details depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_put depmod: WARNING: /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko needs unknown symbol lc_find Installing : drbd84-utils-9.15.1-1.el8.elrepo.x86_64 2/2 Running scriptlet: drbd84-utils-9.15.1-1.el8.elrepo.x86_64 2/2 Running scriptlet: kmod-drbd84-8.4.10-1.2.el8_3.elrepo.x86_64 2/2 Running scriptlet: drbd84-utils-9.15.1-1.el8.elrepo.x86_64 2/2 Verifying : drbd84-utils-9.15.1-1.el8.elrepo.x86_64 1/2 Verifying : kmod-drbd84-8.4.10-1.2.el8_3.elrepo.x86_64 2/2 Installed: drbd84-utils-9.15.1-1.el8.elrepo.x86_64 kmod-drbd84-8.4.10-1.2.el8_3.elrepo.x86_64 Complete! # rpm -ql kmod-drbd84 /etc/depmod.d/kmod-drbd84.conf /lib/modules/4.18.0-240.el8.x86_64 /lib/modules/4.18.0-240.el8.x86_64/extra /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84 /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko /usr/share/doc/kmod-drbd84-8.4.10 /usr/share/doc/kmod-drbd84-8.4.10/GPL-v2.0.txt /usr/share/doc/kmod-drbd84-8.4.10/greylist.txt # - kernel-4.18.0-240.10.1.el8_3.x86_64 # modprobe drbd modprobe: FATAL: Module drbd not found in directory /lib/modules/4.18.0-240.10.1.el8_3.x86_64 # ls -al /lib/modules/4.18.0-240.10.1.el8_3.x86_64/weak-updates/ total 4 drwxr-xr-x 2 root root 6 Feb 19 12:13 . drwxr-xr-x. 7 root root 4096 Feb 19 12:13 .. # - Reboot and start with kernel-4.18.0-240.el8.x86_64 # modprobe drbd modprobe: ERROR: could not insert 'drbd': Unknown symbol in module, or unknown parameter (see dmesg) # journalctl -u drbd.service -- Logs begin at Fri 2021-02-19 12:12:45 JST, end at Fri 2021-02-19 12:20:33 JST. -- Feb 19 12:12:51 centos8node01.example.org systemd[1]: Starting DRBD -- please disable. Unless you are NOT using a cluster manager.... Feb 19 12:12:51 centos8node01.example.org drbd[1006]: Can not load the drbd module. Feb 19 12:12:51 centos8node01.example.org systemd[1]: drbd.service: Main process exited, code=exited, status=5/NOTINSTALLED Feb 19 12:12:51 centos8node01.example.org systemd[1]: drbd.service: Failed with result 'exit-code'. Feb 19 12:12:51 centos8node01.example.org systemd[1]: Failed to start DRBD -- please disable. Unless you are NOT using a cluster manager.. |
|
|
As noted earlier, kmod-drbd84-8.4.10-1.2.el8_3.elrepo was built against kernel-4.18.0-240.el8. Since you are running kernel-4.18.0-240.10.1.el8_3, we will rebuild the kmod against that kernel and let you try. |
|
|
Hmm, I rebuilt the kmod against kernel-4.18.0-240.10.1.el8_3 (kmod-drbd84-8.4.10-1.3.el8.x86_64) and installed it on a test system running kernel-4.18.0-240.10.1.el8_3. It installed fine but I saw the same error message as what you reported. $ uname -r 4.18.0-240.10.1.el8_3.x86_64 $ find /lib/modules -name drbd.ko /lib/modules/4.18.0-240.10.1.el8_3.x86_64/extra/drbd84/drbd.ko $ sudo modprobe drbd modprobe: ERROR: could not insert 'drbd': Unknown symbol in module, or unknown parameter (see dmesg) And the dmesg shows: [ 491.130065] drbd: Unknown symbol lc_seq_printf_stats (err 0) [ 491.130110] drbd: Unknown symbol lc_get_cumulative (err 0) [ 491.130198] drbd: Unknown symbol lc_del (err 0) [ 491.130228] drbd: Unknown symbol lc_committed (err 0) [ 491.130236] drbd: Unknown symbol lc_get (err 0) [ 491.130265] drbd: Unknown symbol lc_try_get (err 0) [ 491.130300] drbd: Unknown symbol lc_element_by_index (err 0) [ 491.130345] drbd: Unknown symbol lc_create (err 0) [ 491.130369] drbd: Unknown symbol lc_try_lock (err 0) [ 491.130380] drbd: Unknown symbol lc_destroy (err 0) [ 491.130395] drbd: Unknown symbol lc_reset (err 0) [ 491.130412] drbd: Unknown symbol lc_is_used (err 0) [ 491.130431] drbd: Unknown symbol lc_seq_dump_details (err 0) [ 491.130459] drbd: Unknown symbol lc_put (err 0) [ 491.130471] drbd: Unknown symbol lc_find (err 0) It is possible that drbd84 is not compatible with RHEL-8. |
|
|
However the fact that the code for drbd is in the kernel suggests it should work under this kernel. |
|
|
I added linux-4.18.0-240.el8/lib/lru_cache.c to tar.gz and built it, and it loaded fine! # From https://elrepo.org/linux/elrepo/el8/SRPMS/kmod-drbd84-8.4.10-1.3.el8_3.elrepo.src.rpm # cd /root/rpmbuild/SOURCES # tar zxf drbd-8.4.10-1.tar.gz # From kernel-4.18.0-240.el8.src.rpm(linux-4.18.0-240.el8.tar.xz) # tar xf linux-4.18.0-240.el8.tar.xz # cp -a linux-4.18.0-240.el8/lib/lru_cache.c drbd-8.4.10-1/ # Fix drbd-8.4.10-1/Makefile # vi drbd-8.4.10-1/Makefile 2c2,3 < drbd-y := drbd_bitmap.o drbd_proc.o --- > drbd-y := lru_cache.o > drbd-y += drbd_bitmap.o drbd_proc.o # tar czf drbd-8.4.10-1.tar.gz drbd-8.4.10-1 # rpmbuild -ba /root/rpmbuild/SPECS/drbd84-kmod.spec # dnf install /root/rpmbuild/RPMS/x86_64/kmod-drbd84-8.4.10-1.3.el8.x86_64.rpm # modprobe drbd # cat /proc/drbd version: 8.4.10 (api:1/proto:86-101) srcversion: 81F782E965B67E4F3F4D415 If there is an RPM package that contain lru_cache.o, we may just need to install it instead of the above. |
|
|
Thanks for the great work. It all makes sense. In fact, shortly after my last comment yesterday, I noticed that, when DRBD is enabled in the kernel config file, the following line was added to the config: CONFIG_LRU_CACHE=m So, I think creating a kmod package for lru_cache and making it a Requires: will be the way to go. |
|
|
I have built kmod-lru_cache-0.0-1.el8_3.elrepo.x86_64.rpm. According to my quick test, drbd seems to load without spitting errors. Could you give it a try? I have not added the Requires: to the drbd package yet. |
|
|
Thanks for the details. The lru_cache package is good. |
|
|
OK, good. The next (and final) step is to add the dependency to kmod-drbd84. |
|
|
Sorry, I didn't reload this page before posting my comment and didn't notice kmod-lru_cache-0.0-1.el8_3.elrepo.x86_64.rpm. I tried it and kmod-drbd84 (and with Ganeti), and the general operations of DRBD resources works fine! |
|
|
That's great news. Thanks for the full test. |
|
|
kmod-drbd84-8.4.10-1.4.el8_3.elrepo.x86_64.rpm will show up in our mirrors shortly. This is also built against kernel-4.18.0-240.10.1.el8_3 but now requires kmod-lru_cache. |
|
|
Thanks for the new build and release. Everything works fine. I appreciate your great work. |
|
|
Problem with kmod-drbd84 for el8 is that it seems to require rebuilding for every kernel update. This defeats the purpose of using the kmod approach. Anyway, we need to rebuild this for the latest kernel 4.18.0-240.15.1.el8_3. |
|
|
It is not good. I checked it and it seems that weak-modules cannot detect another extra kernel module, and cannot find the symbol of lru_cache when processing drbd.ko. # Built and tested against EL 8.3 GA kernel 4.18.0-240.el8. # echo /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko | /usr/sbin/weak-modules --verbose --add-modules drbd84 Module drbd.ko from kernel 4.18.0-240.el8.x86_64 is not compatible with kernel 4.18.0-240.10.1.el8_3.x86_64 in symbols: lc_seq_printf_stats lc_get_cumulative lc_del lc_committed lc_get lc_try_get lc_element_by_index lc_create lc_try_lock lc_destroy lc_reset lc_is_used lc_seq_dump_details lc_put lc_find Falling back weak-modules state for kernel 4.18.0-240.10.1.el8_3.x86_64 /sbin/depmod -ae -F /boot/System.map-4.18.0-240.el8.x86_64 4.18.0-240.el8.x86_64 mv /boot/initramfs-4.18.0-240.el8.x86_64.tmp /boot/initramfs-4.18.0-240.el8.x86_64.img # ls -al /lib/modules/4.18.0-240.10.1.el8_3.x86_64/weak-updates/ total 4 drwxr-xr-x 3 root root 23 Feb 21 12:41 . drwxr-xr-x. 7 root root 4096 Feb 21 12:41 .. As an example of a solution, it works fine on kmod-drbd84 built with lru_cache included in the drbd tar.gz(#c7455). https://elrepo.org/bugs/view.php?id=1075#c7455 # echo /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko | /usr/sbin/weak-modules --verbose --add-modules weak module for drbd.ko already exists for kernel 4.18.0-240.10.1.el8_3.x86_64, update case? Module drbd.ko from kernel 4.18.0-240.el8.x86_64 is compatible with kernel 4.18.0-240.10.1.el8_3.x86_64 /sbin/depmod -ae -F /boot/System.map-4.18.0-240.10.1.el8_3.x86_64 4.18.0-240.10.1.el8_3.x86_64 /sbin/depmod -ae -F /boot/System.map-4.18.0-240.el8.x86_64 4.18.0-240.el8.x86_64 mv /boot/initramfs-4.18.0-240.10.1.el8_3.x86_64.tmp /boot/initramfs-4.18.0-240.10.1.el8_3.x86_64.img mv /boot/initramfs-4.18.0-240.el8.x86_64.tmp /boot/initramfs-4.18.0-240.el8.x86_64.img # ls -al /lib/modules/4.18.0-240.10.1.el8_3.x86_64/weak-updates/drbd84/drbd.ko lrwxrwxrwx 1 root root 55 Feb 22 09:21 /lib/modules/4.18.0-240.10.1.el8_3.x86_64/weak-updates/drbd84/drbd.ko -> /lib/modules/4.18.0-240.el8.x86_64/extra/drbd84/drbd.ko DRBD operations also work fine on both 4.18.0-240.el8 and 4.18.0-240.10.1.el8_3. |
|
|
Excellent findings! We will try integrating lru_cache into the drbd code as per your suggestion. |
|
|
kmod-drbd84-8.4.10-1.5.el8_3.elrepo.x86_64.rpm now includes lru_cache and was built against the GA kernel 4.18.0-240.el8. It was confirmed to weak-link to newer kernels 4.18.0-240.10.1.el8_3 and 4.18.0-240.15.1.el8_3. Thanks, @jfut, for all your help. |
|
|
Thank you again for everything you’ve done. It works fine in my environment, too. |
|
|
@jfut You're welcome. This was all possible because of your great help and input. Now closing as 'resolved'. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2021-02-15 23:19 | jfut | New Issue | |
| 2021-02-15 23:19 | jfut | Status | new => assigned |
| 2021-02-15 23:19 | jfut | Assigned To | => toracat |
| 2021-02-16 03:08 | toracat | Note Added: 0007448 | |
| 2021-02-16 04:51 | jfut | Note Added: 0007449 | |
| 2021-02-17 18:38 | toracat | Note Added: 0007450 | |
| 2021-02-18 22:35 | jfut | Note Added: 0007451 | |
| 2021-02-19 00:39 | toracat | Note Added: 0007452 | |
| 2021-02-19 01:38 | toracat | Note Added: 0007453 | |
| 2021-02-19 01:41 | toracat | Note Added: 0007454 | |
| 2021-02-19 02:46 | jfut | Note Added: 0007455 | |
| 2021-02-19 11:31 | toracat | Note Added: 0007456 | |
| 2021-02-19 19:34 | toracat | Note Added: 0007457 | |
| 2021-02-19 20:00 | jfut | Note Added: 0007458 | |
| 2021-02-19 20:10 | toracat | Note Added: 0007459 | |
| 2021-02-19 21:05 | jfut | Note Added: 0007460 | |
| 2021-02-19 21:12 | toracat | Note Added: 0007461 | |
| 2021-02-20 13:31 | toracat | Note Added: 0007462 | |
| 2021-02-20 23:20 | jfut | Note Added: 0007463 | |
| 2021-02-21 12:42 | toracat | Note Added: 0007467 | |
| 2021-02-21 12:42 | toracat | Note Edited: 0007467 | |
| 2021-02-21 20:03 | jfut | Note Added: 0007468 | |
| 2021-02-22 03:33 | toracat | Note Added: 0007469 | |
| 2021-02-22 19:12 | toracat | Note Added: 0007470 | |
| 2021-02-23 22:13 | jfut | Note Added: 0007474 | |
| 2021-02-23 22:21 | toracat | Status | assigned => resolved |
| 2021-02-23 22:21 | toracat | Resolution | open => fixed |
| 2021-02-23 22:21 | toracat | Note Added: 0007475 | |
| 2021-12-17 14:05 | toracat | Relationship added | related to 0001173 |
