Commit 650cad56 authored Sep 26, 2023 by Peter Zijlstra

sched/eevdf: Fix avg_vruntime()

The expectation is that placing a task at avg_vruntime() makes it
eligible. Turns out there is a corner case where this is not the case.

Specifically, avg_vruntime() relies on the fact that integer division
is a flooring function (eg. it discards the remainder). By this
property the value returned is slightly left of the true average.

However! when the average is a negative (relative to min_vruntime) the
effect is flipped and it becomes a ceil, with the result that the
returned value is just right of the average and thus not eligible.

Fixes: af4cf404 ("sched/fair: Add cfs_rq::avg_vruntime")
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>

parent 2f2fc17b

kernel/sched/fair.c

+9 −1

Original line number	Diff line number	Diff line
		@@ -664,6 +664,10 @@ void avg_vruntime_update(struct cfs_rq *cfs_rq, s64 delta)
		cfs_rq->avg_vruntime -= cfs_rq->avg_load * delta;
		}

		/*
		* Specifically: avg_runtime() + 0 must result in entity_eligible() := true
		* For this to be so, the result of this function must have a left bias.
		*/
		u64 avg_vruntime(struct cfs_rq *cfs_rq)
		{
		struct sched_entity *curr = cfs_rq->curr;
		@@ -677,8 +681,12 @@ u64 avg_vruntime(struct cfs_rq *cfs_rq)
		load += weight;
		}

		if (load)
		if (load) {
		/* sign flips effective floor / ceil */
		if (avg < 0)
		avg -= (load - 1);
		avg = div_s64(avg, load);
		}

		return cfs_rq->min_vruntime + avg;
		}