Commit afd41fff authored Aug 30, 2022 by Nathan Huckleberry Committed by Mike Snitzer Oct 18, 2022

dm verity: enable WQ_HIGHPRI on verify_wq



WQ_HIGHPRI increases throughput and decreases disk latency when using
dm-verity. This is important in Android for camera startup speed.

The following tests were run by doing 60 seconds of random reads using
a dm-verity device backed by two ramdisks.

Without WQ_HIGHPRI
lat (usec): min=13, max=3947, avg=69.53, stdev=50.55
READ: bw=51.1MiB/s (53.6MB/s), 51.1MiB/s-51.1MiB/s (53.6MB/s-53.6MB/s)

With WQ_HIGHPRI:
lat (usec): min=13, max=7854, avg=31.15, stdev=30.42
READ: bw=116MiB/s (121MB/s), 116MiB/s-116MiB/s (121MB/s-121MB/s)

Further testing was done by measuring how long it takes to open a
camera on an Android device.

Without WQ_HIGHPRI
Total verity work queue wait times (ms):
880.960, 789.517, 898.852

With WQ_HIGHPRI:
Total verity work queue wait times (ms):
528.824, 439.191, 433.300

The average time to open the camera is reduced by 350ms (or 40-50%).

Signed-off-by: Nathan Huckleberry <nhuck@google.com>
Signed-off-by: Mike Snitzer <snitzer@kernel.org>

parent cea44663

drivers/md/dm-verity-target.c

+10 −8

Original line number	Diff line number	Diff line
		@@ -1401,14 +1401,16 @@ static int verity_ctr(struct dm_target ti, unsigned argc, char *argv)

		/* WQ_UNBOUND greatly improves performance when running on ramdisk */
		wq_flags = WQ_MEM_RECLAIM \| WQ_UNBOUND;
		if (v->use_tasklet) {
		/*
		* Allow verify_wq to preempt softirq since verification in
		* tasklet will fall-back to using it for error handling
		* (or if the bufio cache doesn't have required hashes).
		* Using WQ_HIGHPRI improves throughput and completion latency by
		* reducing wait times when reading from a dm-verity device.
		*
		* Also as required for the "try_verify_in_tasklet" feature: WQ_HIGHPRI
		* allows verify_wq to preempt softirq since verification in tasklet
		* will fall-back to using it for error handling (or if the bufio cache
		* doesn't have required hashes).
		*/
		wq_flags \|= WQ_HIGHPRI;
		}
		v->verify_wq = alloc_workqueue("kverityd", wq_flags, num_online_cpus());
		if (!v->verify_wq) {
		ti->error = "Cannot allocate workqueue";