View Issue Details

IDProjectCategoryView StatusLast Update
0000905channel: kernel/el7kernel-mlpublic2019-03-06 12:15
ReporterBS86 Assigned Toburakkucat  
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Summary0000905: No schedulers present due to upstream drop of single queue schedulers
DescriptionKernel 5.0 dropped the single queue schedulers and the whole queue.
In the past on kernel-ml and kernel-lt, forcing multiqueue scheduling with scsi_mod.use_blk_mq=y had no effect.

On kernel-ml with 5.0, the effect is that now no schedulers are present at all. It seems like blk_mq or multiqueue schedulers are dropped during build or in the config. They should now be enabled.
Additional Informationcat /sys/block/*/queue/scheduler
none
none
[none]
[none]
[none]
TagsNo tags attached.

Activities

BS86

2019-03-05 00:37

reporter   ~0006238

Addition:
cat /sys/block/*/queue/scheduler
none #dm-0
none #dm-1
[none] #sda
[none] #sdb
[none] #sr0

burakkucat

2019-03-05 10:40

administrator   ~0006239

Acknowledging your report.

A diff of the 5.0.0 and the 4.20.13 configuration files, greping for SCHED, shows the following --

[kernel-ml-config]$ diff config-5.0.0-x86_64 config-4.20.13-x86_64 | grep SCHED
> CONFIG_IOSCHED_NOOP=y
> CONFIG_IOSCHED_DEADLINE=y
> CONFIG_IOSCHED_CFQ=y
> CONFIG_CFQ_GROUP_IOSCHED=y
> CONFIG_DEFAULT_IOSCHED="deadline"
[kernel-ml-config]$

That is quite crazy. I have no idea why those options were disabled.

BS86

2019-03-05 10:48

reporter   ~0006240

Those are the single-queue schedulers that are dropped with 5.0

The correct config paramters are:
cat config.x86_64 |grep -i mq
CONFIG_BLK_WBT_MQ=y
CONFIG_BLK_MQ_PCI=y
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_BLK_MQ_RDMA=y
CONFIG_MQ_IOSCHED_DEADLINE=y
CONFIG_MQ_IOSCHED_KYBER=y

BS86

2019-03-05 10:50

reporter   ~0006241

and not to forget:
CONFIG_IOSCHED_BFQ=y
CONFIG_BFQ_GROUP_IOSCHED=y


For BFQ, the single-queue scheduler was:
CONFIG_IOSCHED_BFQ_SQ=y
CONFIG_BFQ_SQ_GROUP_IOSCHED=y

toracat

2019-03-05 11:18

administrator   ~0006242

@BS86

CONFIG_MQ_IOSCHED_KYBER and CONFIG_IOSCHED_BFQ can be =m. I suppose no problem making them modules?

BS86

2019-03-05 11:47

reporter   ~0006243

I don't think that there is a problem with module-ing them. It just narrows down the choice to None and MQ_deadline.

toracat

2019-03-05 12:04

administrator   ~0006245

An interesting article about I/O scheduler:

https://www.linaro.org/blog/io-bandwidth-management-for-production-quality-services/

(not that this is directly related to the current bug report)

burakkucat

2019-03-05 13:32

administrator   ~0006248

We now have two configuration files based upon your request.

The first has =y for each of the relevant options --

CONFIG_BLK_WBT_MQ=y
CONFIG_BLK_MQ_PCI=y
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_BLK_MQ_RDMA=y
CONFIG_MQ_IOSCHED_DEADLINE=y
CONFIG_MQ_IOSCHED_KYBER=y
CONFIG_IOSCHED_BFQ=y
CONFIG_BFQ_GROUP_IOSCHED=y

The second is identical, with the exception that the three schedulers are now =m --

CONFIG_BLK_WBT_MQ=y
CONFIG_BLK_MQ_PCI=y
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_BLK_MQ_RDMA=y
CONFIG_MQ_IOSCHED_DEADLINE=m
CONFIG_MQ_IOSCHED_KYBER=m
CONFIG_IOSCHED_BFQ=m
CONFIG_BFQ_GROUP_IOSCHED=y

Of the two, I would prefer the latter, as it does not increase the size of the core kernel image. However, I would like to know which of the above you would prefer.

Please let me know and then I will build a new release.

BS86

2019-03-05 14:13

reporter   ~0006249

BFQ has significant better performance on rotational disks for desktop usage, so I would go with the first one. Personally, I use bfq for all rotational and mq_deadline for all non-rotational (even on nvme disks), but loading them during boot is also ok. I don't think that the growth in size is that big with them added.

I want to add that I am not 100% sure if the list of options is complete to fix the problem. I am still in the process of learning my stuff with kernels.

burakkucat

2019-03-05 14:56

administrator   ~0006255

A kernel-ml-5.0.0-2.el7.elrepo package set has now been built and is currently synchronising across our mirror sites.

kernel-ml-5.0.0-2.el7.elrepo.nosrc.rpm
kernel-ml-5.0.0-2.el7.elrepo.x86_64.rpm
kernel-ml-devel-5.0.0-2.el7.elrepo.x86_64.rpm
kernel-ml-doc-5.0.0-2.el7.elrepo.noarch.rpm
kernel-ml-headers-5.0.0-2.el7.elrepo.x86_64.rpm
kernel-ml-tools-5.0.0-2.el7.elrepo.x86_64.rpm
kernel-ml-tools-libs-5.0.0-2.el7.elrepo.x86_64.rpm
kernel-ml-tools-libs-devel-5.0.0-2.el7.elrepo.x86_64.rpm
perf-5.0.0-2.el7.elrepo.x86_64.rpm
python-perf-5.0.0-2.el7.elrepo.x86_64.rpm

[Build64R7 uploads]$ rpm -qp --changelog *.nosrc.rpm | head -n 6
* Tue Mar 05 2019 Alan Bartlett <ajb@elrepo.org> - 5.0.0-2
- CONFIG_BLK_WBT=y, CONFIG_BLK_WBT_MQ=y,
- CONFIG_MQ_IOSCHED_DEADLINE=y, CONFIG_MQ_IOSCHED_KYBER=y,
- CONFIG_IOSCHED_BFQ=y and CONFIG_BFQ_GROUP_IOSCHED=y
- [https://elrepo.org/bugs/view.php?id=905]

[Build64R7 uploads]$

BS86

2019-03-05 23:57

reporter   ~0006258

Thanks. It works now:
cat /sys/block/*/queue/scheduler
none #dm-0
none #dm-1
[mq-deadline] kyber bfq none #sda
mq-deadline kyber [bfq] none #sdb
[mq-deadline] kyber bfq none #sr0

burakkucat

2019-03-06 12:15

administrator   ~0006259

Thank you for confirming the success.

Now closing as "resolved/fixed".

Issue History

Date Modified Username Field Change
2019-03-05 00:31 BS86 New Issue
2019-03-05 00:31 BS86 Status new => assigned
2019-03-05 00:31 BS86 Assigned To => burakkucat
2019-03-05 00:37 BS86 Note Added: 0006238
2019-03-05 10:40 burakkucat Note Added: 0006239
2019-03-05 10:48 BS86 Note Added: 0006240
2019-03-05 10:50 BS86 Note Added: 0006241
2019-03-05 11:18 toracat Note Added: 0006242
2019-03-05 11:47 BS86 Note Added: 0006243
2019-03-05 12:04 toracat Note Added: 0006245
2019-03-05 13:32 burakkucat Note Added: 0006248
2019-03-05 14:13 BS86 Note Added: 0006249
2019-03-05 14:56 burakkucat Note Added: 0006255
2019-03-05 23:57 BS86 Note Added: 0006258
2019-03-06 12:15 burakkucat Note Added: 0006259
2019-03-06 12:15 burakkucat Status assigned => resolved
2019-03-06 12:15 burakkucat Resolution open => fixed