sched: Optimize rq_lockp() usage

Browse source

rq_lockp() includes a static_branch(), which is asm-goto, which is
asm volatile which defeats regular CSE. This means that:

	if (!static_branch(&foo))
		return simple;

	if (static_branch(&foo) && cond)
		return complex;

Doesn't fold and we get horrible code. Introduce __rq_lockp() without
the static_branch() on.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Don Hiatt <dhiatt@digitalocean.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Tested-by: Vincent Guittot <vincent.guittot@linaro.org>
Link: https://lkml.kernel.org/r/20210422123308.316696988@infradead.org

...

This commit is contained in:

Peter Zijlstra 2021-03-03 16:45:41 +01:00

parent 9edeaea1bc

commit 9ef7e7e33b

4 changed files with 36 additions and 19 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

16

kernel/sched/core.c

View file

4

kernel/sched/deadline.c

View file

2

kernel/sched/fair.c

View file

33

kernel/sched/sched.h

View file