sched_ext: Fix this_rq() assumptions in dispatch kfuncs

Browse source

Under core scheduling, dispatch runs from within the core-wide pick and can
target a sibling rq, so ops.dispatch() may execute on a CPU different from
the dispatched rq's. Several kfunc paths assumed the two always coincide:

- scx_dsq_move() decided whether an rq lock is held by testing this_rq()'s
  rq flags and lock-danced accordingly. A dispatch for a sibling took the
  unlocked-context branch and acquired the source rq lock on top of the
  already held dispatched rq lock which could deadlock.

- scx_bpf_sub_dispatch() dispatched this_rq() with its stashed
  sub_dispatch_prev, which is NULL when dispatching for a sibling.

- finish_dispatch(), scx_bpf_dsq_reenq() and scx_bpf_dsq_nr_queued()
  resolved SCX_DSQ_LOCAL to this CPU's local DSQ rather than the dispatched
  rq's. The latter two are callable from other rq-locked operations too,
  where SCX_DSQ_LOCAL now likewise resolves to the op's rq. This changes
  behavior also without core scheduling, e.g. for ops.enqueue() running a
  remote wakeup on the waking CPU, and is intended: which CPU happens to
  execute an operation is incidental, the op's rq is what it is operating
  on, and the resolution now matches the insert side where SCX_DSQ_LOCAL
  dispatches land on the task's rq.

Use the rq tracked by scx_locked_rq(), which is set to the dispatched rq
around ops invocations and NULL in unlocked contexts.

Fixes: 4c95380701 ("sched/ext: Fold balance_scx() into pick_task_scx()")
Cc: stable@vger.kernel.org # v6.19+
Signed-off-by: Tejun Heo <tj@kernel.org>

...

This commit is contained in:

Tejun Heo 2026-08-07 11:02:19 -10:00

parent ffaab58d21

commit 3dd52416e4

1 changed files with 35 additions and 23 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

58

kernel/sched/ext/ext.c

View file