mm: avoid unnecessary lru drain for wp_can_reuse_anon_folio()

Browse source

Patch series "mm: drop redundant lru_add_drain in anon folio reuse paths",
v3.

We are doing a large number of redundant lru_add_drain() calls in both
wp_can_reuse_anon_folio() and do_swap_page(), leading to LRU lock
contention and unnecessary overhead.

In wp_can_reuse_anon_folio(), we can check the refcount against the
lru_cache before deciding to drain.  In do_swap_page(), the drain is now
entirely redundant after Kairui's work to route SYNC I/O through the
swapcache in the same way as ASYNC I/O.

Build the kernel within a 1 GB memcg using 20 threads with zRAM swap.  The
number of lru_add_drain() calls is reduced from 276,278 to 226,318, a
reduction of about 18%.

Build the kernel within an 800 MB memcg using 20 threads with zRAM swap. 
The number of lru_add_drain() calls is reduced from 778,950 to 541,149, a
reduction of 30.5%.


This patch (of 4):

There is a case where `folio_ref_count(folio) == 3` and
`!folio_test_swapcache(folio)`. In that case, both
`folio_ref_count(folio) > 3` and
`folio_ref_count(folio) > 1 + folio_test_swapcache(folio)` evaluate
false, causing an unnecessary local LRU drain.

During an Ubuntu boot, I observed over 5,000 redundant local LRU drains. 
For a kernel build with a minimal configuration, I observed more than
20,000 redundant drains.

Fix this by checking against: `1 + in_swapcache + in_lrucache` instead of
hardcoding `folio_ref_count(folio) > 3`.

[baohua@kernel.org: rename in_lru_cache to maybe_in_lru_cache]
  Link: https://lore.kernel.org/20260708145718.82690-1-baohua@kernel.org
Link: https://lore.kernel.org/20260701235955.36126-1-baohua@kernel.org
Link: https://lore.kernel.org/20260701235955.36126-2-baohua@kernel.org
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Kairui Song <kasong@tencent.com>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Baoquan He <baoquan.he@linux.dev>
Cc: Chris Li <chrisl@kernel.org>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Usama Arif <usama.arif@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

...

This commit is contained in:

Barry Song (Xiaomi) 2026-07-02 07:59:52 +08:00 • committed by Andrew Morton

parent 819bfbd47e

commit 8d596d8b81

1 changed files with 7 additions and 3 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

10

mm/memory.c

View file