damon_sysfs_probe_{add,rm}_dirs names a variable for damon_sysf_probe as
'attr'. Probably a trivial copy-pasta error, but it makes the code not
pleasant to read. Fix those.
Link: https://lore.kernel.org/20260630141726.92246-12-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
damon_sysfs_set_probe() is doing not only probe setup but also filters
setup. Split out filters setup for readability.
Link: https://lore.kernel.org/20260630141726.92246-11-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
damon_sysfs_set_probes() function is relatively long. It has two nested
loop for setting two nested entities, namely probe and filter. Split out
the probe level setup for readability.
Link: https://lore.kernel.org/20260630141726.92246-10-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
damon_commit_target_regions() calls damon_set_regions() for updating the
destination target's monitoring target region boundaries. It sets the
boundaries same to source target's monitoring regions, even if they are
adjacent. Meanwhile, damon_set_region() sets the destination target
regions exactly the same to the source, only when the target regions are
empty. When there are existing target regions, only a few regions are
expanded or shrunk to fit on only the boundaries for disjoint regions in
the source. Hence the adjacent source ranges mean nothing in common
cases. When there are many regions, such adjacent range setup is only a
waste of time and space. We recently found [1] it is actually causing
memory overhead. Setup the ranges for only distinct ranges.
Link: https://lore.kernel.org/20260630141726.92246-9-sj@kernel.org
Link: https://lore.kernel.org/20260603112306.58490-1-akinobu.mita@gmail.com [1]
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
DAMON sysfs interface for DAMOS quota has quite extended since its initial
introduction. The test case for that in DAMON sysfs interface essential
file operations test (sysfs.sh) has not accordingly extended, though.
Extend the test case to test all existing files.
Link: https://lore.kernel.org/20260630141726.92246-8-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
DAMON sysfs essential file operations test (sysfs.sh) was extended to test
DAMON probes sysfs directory, by commit 14885da09b
("selftests/damon/sysfs.sh: test probes dir"). Unlike other DAMON sysfs
files, it is testing only a single directory case. Extend it for multiple
directories.
Link: https://lore.kernel.org/20260630141726.92246-5-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Commit 9012c4e647 ("mm/damon: replace damon_rand() with a per-ctx
lockless PRNG") optimized DAMON for better performance. Add a kunit test
for ensuring the bounds of the output.
Link: https://lore.kernel.org/20260630141726.92246-4-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
DAMON ABI document is not updated for the DAMON probe sysfs files.
Update.
Link: https://lore.kernel.org/20260630141726.92246-3-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "mm/damon: update, optimize, and clean up doc, tests, and
code".
Patches 1 and 2 update the design and ABI documents for recently added
DAMON features. Patches 3-7 add or update more unit and self tests for
DAMON to cover recently changed or added functions and sysfs files. Patch
8 optimizes damon_commit_target_regions() to skip unnecessary adjacent
ranges setup. Patches 9-11 clean and fix up recently added DAMON sysfs
interface code for readability.
This patch (of 11):
Commit 9138e27a3b ("mm/damon: add node_eligible_mem_bp goal metric")
introduced DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP but forgot updating the DAMON
design document for that. Update.
Link: https://lore.kernel.org/20260630141726.92246-1-sj@kernel.org
Link: https://lore.kernel.org/20260630141726.92246-2-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: SeongJae Park <sj@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
CONFIG_ARCH_ENABLE_THP_MIGRATION gates PMD-level migration entries.
PMD-level device-private entries use the same migration mechanism and
therefore require the same architecture support.
Upcoming PMD-level swap entries can use the same PMD softleaf helpers
without depending on page migration, so rename the architecture gate to
CONFIG_ARCH_HAS_PMD_SOFTLEAVES. This describes the PMD entry capability
rather than one current user of it.
This is a pure rename: the set of selecting architectures (x86, arm64,
s390, riscv, loongarch, and powerpc on PPC_BOOK3S_64) and the gating
semantics are unchanged.
No functional change intended.
Link: https://lore.kernel.org/20260706114320.1643046-7-usama.arif@linux.dev
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Li <chrisl@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
migrate_vma_collect_pmd() calls softleaf_to_folio() on a non-present PMD
before checking the entry's type. softleaf_to_folio() converts the
entry's offset to a PFN, which is only meaningful for migration or
device-private entries.
A PMD swap entry's offset is a swap offset, not a PFN, so the lookup would
either return a bogus folio pointer or trip pfn_to_page validation on a
debug kernel. In the non-device-private path the returned folio is then
unused (the OR short-circuits to migrate_vma_collect_skip()), but the
lookup itself is already unsafe.
Move the softleaf_to_folio() call inside the device-private branch where
the folio is actually needed, mirroring the equivalent
change_non_present_huge_pmd() fix.
Link: https://lore.kernel.org/20260706114320.1643046-6-usama.arif@linux.dev
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Li <chrisl@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
change_non_present_huge_pmd() calls softleaf_to_folio() unconditionally at
the top of the function. softleaf_to_folio() extracts a PFN from the
entry and converts it to a folio pointer, which is only meaningful for
migration and device_private entries that encode a real PFN.
A swap entry encodes a swap offset instead, so softleaf_to_folio() would
produce a bogus pointer and crash on mprotect() when a PMD swap entry is
present.
Move the call into the migration_write branch where the folio is actually
used, so the function is safe for any non-present PMD type.
Link: https://lore.kernel.org/20260706114320.1643046-5-usama.arif@linux.dev
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Dev Jain <dev.jain@arm.com>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Li <chrisl@kernel.org>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
pagemap_pmd_range_thp() assumes that every non-present PMD is a migration
entry and unconditionally calls softleaf_to_page(). This will crash on
any non-present PMD type that does not encode a PFN, such as the upcoming
PMD-level swap entries.
Guard the page lookup with softleaf_has_pfn(), matching how
pte_to_pagemap_entry() already handles non-present PTEs.
Link: https://lore.kernel.org/20260706114320.1643046-4-usama.arif@linux.dev
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Li <chrisl@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
When a swap entry is installed in a page table, the mm must be added to
init_mm.mmlist so that swapoff can find and unuse its swap entries. This
double-checked locking pattern is currently open-coded in
try_to_unmap_one() and copy_nonpresent_pte().
Move it into mm_prepare_for_swap_entries() in mm/internal.h and convert
both callers so it can be reused by upcoming PMD-level swap entry code
paths that also need to register the mm with swapoff.
copy_nonpresent_pte() previously inserted into &src_mm->mmlist rather than
&init_mm.mmlist, but the insertion point is irrelevant, mmlist is a
circular list and swapoff walks it entirely from init_mm.mmlist, so only
membership matters, not position.
Link: https://lore.kernel.org/20260706114320.1643046-3-usama.arif@linux.dev
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Reviewed-by: Dev Jain <dev.jain@arm.com>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Li <chrisl@kernel.org>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "mm: preparatory patches for PMD level swap entries", v2.
This is the preparatory part of the PMD page table swapin work. The full
PMD swap entry series has been split into two parts:
1. this preparatory series, which contains the first 6 patches. Zi [1]
and Lance [2] suggested to separate this out from the core series.
2. the PMD swap entry core series, which depends on this one. I will
send this once the preparatory series is merged in mm-new as v3
as the combined is currently at v2 [1].
I have not marked this prep series as v3, as its not really adding
support for PMD swap entries.
This series does not introduce PMD swap entries and does not install any
new page-table entry type. It only cleans up existing PMD softleaf
helpers and call sites so the follow-up PMD swap entry series can be
smaller and easier to review.
It should be safe to merge independently. The patches are either helper
additions, refactors of existing open-coded logic, defensive checks that
preserve current migration/device-private behavior, or a mechanical rename
of the PMD softleaf Kconfig gate. The follow-up series depends on these
helpers, but this series does not depend on the follow-up series.
Patch breakdown:
1. mm: add softleaf_to_pmd() and convert existing callers
Add the PMD counterpart to softleaf_to_pte() and convert existing
swp_entry_to_pmd() users that are constructing PMD softleaf
entries.
2. mm: extract mm_prepare_for_swap_entries() helper
Hoist the "register mm with swapoff" double-checked-locking
pattern out of try_to_unmap_one() and copy_nonpresent_pte() so
future PMD-level users do not need another open-coded copy.
3. fs/proc: use softleaf_has_pfn() in pagemap PMD walker
Avoid assuming every non-present PMD softleaf entry encodes a PFN.
Existing migration/device-private behavior is preserved.
4. mm/huge_memory: move softleaf_to_folio() inside migration branch
Keep the folio lookup in change_non_present_huge_pmd() scoped to
the migration-entry branch that actually needs it.
5. mm/migrate_device: move softleaf_to_folio() inside device-private
branch
Apply the same ordering cleanup to migrate_vma_collect_pmd(): only
derive a folio after confirming the PMD entry is device-private.
6. mm: rename ARCH_ENABLE_THP_MIGRATION to ARCH_HAS_PMD_SOFTLEAVES
Rename the architecture gate to describe what it actually enables:
PMD softleaf entries. Migration remains the only current user in
this series; the follow-up series adds PMD swap entries.
This patch (of 6):
Add softleaf_to_pmd() as the PMD counterpart to softleaf_to_pte(),
completing the symmetry of the softleaf abstraction for page table leaf
entries.
The upcoming PMD swap entry support needs to construct PMD entries from
swap entries. Converting existing swp_entry_to_pmd() callers to
softleaf_to_pmd() in a prep patch keeps the feature patches focused on new
functionality rather than mixing refactoring with new code.
Link: https://lore.kernel.org/20260706114320.1643046-1-usama.arif@linux.dev
Link: https://lore.kernel.org/20260706114320.1643046-2-usama.arif@linux.dev
Link: https://lore.kernel.org/all/6E99CC4E-A026-4DE3-8A5A-34216771F521@nvidia.com/ [1]
Link: https://lore.kernel.org/all/b08cafbb-a4b7-4609-84ae-dbb2cfcfc8be@linux.dev/#t [2]
Link: https://lore.kernel.org/all/20260602142537.198755-1-usama.arif@linux.dev/ [3]
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Li <chrisl@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
folio_check_references() will return FOLIOREF_RECLAIM_CLEAN for referenced
file folios, indicating that we can proceed to reclaim clean file folios
or keep them if they are dirty file folios. However, after commit
6b0dfabb35 ("fs: Remove aops->writepage"), we no longer attempt to write
back filesystem folios through reclaim. Instead, we always activate dirty
file folios and wakeup the flush workers to write them back. As a result,
the FOLIOREF_RECLAIM_CLEAN logic is now redundant: for dirty file folios,
we will no longer reach the 'references == FOLIOREF_RECLAIM_CLEAN' branch
in shrink_folio_list().
Additionally, lazyfree folios are also placed on the file LRU list, but if
a lazyfree folio becomes dirty, try_to_unmap() will fail and thus prevent
reclaim of the re-dirtied lazyfree folios.
Therefore, we can drop the FOLIOREF_RECLAIM_CLEAN-related logic.
Link: https://lore.kernel.org/def70a713e10bcbdf3b9fccc2139ecc07b64f2cb.1782715791.git.baolin.wang@linux.alibaba.com
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Barry Song <baohua@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Add a test that exercises kdamond_split_regions() when the total region
count is already above max_nr_regions / 2, asserting that the function
still splits a fraction of the regions (makes progress) and does not
overshoot max_nr_regions.
The region size and min_region_sz are picked so the split arithmetic does
not depend on the page size.
All tests pass:
damon: pass:31 fail:0 skip:0 total:31
Totals: pass:31 fail:0 skip:0 total:31
Link: https://lore.kernel.org/20260626085851.70754-3-jiayuan.chen@linux.dev
Link: https://lore.kernel.org/20260629145630.134891-3-sj@kernel.org
Signed-off-by: Jiayuan Chen <jiayuan.chen@shopee.com>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Cc: Shu Anzai <shu17az@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "mm/damon/core: detect internal variation above
max_nr_regions/2", v3.
kdamond_split_regions() bails out early when nr_regions is already above
max_nr_regions / 2. A large region that picks up new internal variation
after that point never gets split, so we lose visibility into its hot/cold
structure.
We hit this with damon-paddr on hugepage workloads and damon-vaddr on
processes that mmap a large anonymous range.
Example with max_nr_regions == 1500. A target ends up with 799 small
hot/cold regions plus one big region (an earlier merge collapsed a
uniformly-accessed range into a single piece):
H:hot
C:cold
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH..........................|
nr_regions = 800 > max_nr_regions / 2 = 750
Now a cold subarea shows up inside r800:
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH........CCCCCC.............|
The small regions can't merge with each other (their access counts
differ), so budget never frees up. r800 can't be split because nr_regions
> max_nr_regions / 2 returns early. The cold subarea stays invisible.
Patch 1 keeps refining on this path: when nr_regions is above
max_nr_regions / 2 but still under the maximum, it splits a fraction of
the regions instead of returning. The fraction shrinks as the remaining
budget shrinks, so the count approaches max_nr_regions smoothly. A
useless split is undone by the next merge cycle.
Patch 2 adds a KUnit test for the case where nr_regions is already above
max_nr_regions / 2.
Thanks to SJ for the suggestion to drive the split fraction from the
remaining budget rather than an age-based filter.
This patch (of 2):
kdamond_split_regions() returns early when nr_regions is above
max_nr_regions / 2, leaving internal access variation inside a large
region undetected.
Such a layout is common with damon-paddr on hugepage workloads or
damon-vaddr on processes with a large anonymous mmap.
For example, with max_nr_regions == 1500, a target may end up with 799
small alternating-temperature regions plus one large region that absorbed
a uniformly-accessed range during an earlier merge:
H:hot
C:cold
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH..........................|
nr_regions = 800 > max_nr_regions / 2 = 750
If a cold subarea later emerges inside r800:
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH........CCCCCC.............|
The small regions cannot merge with each other (different access counts),
so the budget stays full. r800 cannot be split because nr_regions >
max_nr_regions / 2 causes an early return. The cold subarea is never
discovered.
When nr_regions is above max_nr_regions / 2 but still under the maximum,
split only a fraction of the regions instead of returning. One region in
every 'max_nr_regions / budget' regions is split, where budget is the
remaining room (max_nr_regions - nr_regions), starting from a rotating
offset so different regions get picked over time. The fraction shrinks as
the budget shrinks, so the region count keeps refining while approaching
max_nr_regions smoothly rather than overshooting it. An unnecessary split
is reverted by the next kdamond_merge_regions().
Link: https://lore.kernel.org/20260629145630.134891-1-sj@kernel.org
Link: https://lore.kernel.org/20260626085851.70754-2-jiayuan.chen@linux.dev
Link: https://lore.kernel.org/20260629145630.134891-2-sj@kernel.org
Signed-off-by: Jiayuan Chen <jiayuan.chen@shopee.com>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Cc: Shu Anzai <shu17az@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Add a KUnit test to verify that damos_walk() rejects new requests when
walk_control_obsolete is set.
Commit 33c3f6c2b4 ("mm/damon/core: fix damos_walk() vs kdamond_fn() exit
race") introduced walk_control_obsolete to prevent a race condition where
new requests could be registered during kdamond shutdown and never
handled.
This test simulates the shutdown condition by setting
walk_control_obsolete and verifies that damos_walk() returns -ECANCELED
immediately.
This validates the invariant introduced by the fix and helps prevent
regressions.
Link: https://patch.msgid.link/20260612062337.2459-1-saileshnandanavanam@gmail.com
Link: https://lore.kernel.org/20260629145538.134832-6-sj@kernel.org
Suggested-by: SJ Park <sj@kernel.org>
Signed-off-by: Sailesh Nandanavanam <saileshnandanavanam@gmail.com>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Cc: Akinobu Mita <akinobu.mita@gmail.com>
Cc: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Doehyun Baek <doehyunbaek@gmail.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: "Liam R. Howlett" <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Philippe Laferriere <plafer@proton.me>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
The conversion of a duration expressed in seconds reads as
msecs_to_jiffies(5 * MSEC_PER_SEC), which obscures the intent and
needlessly goes through milliseconds. Use the dedicated secs_to_jiffies()
helper, which expresses the 5-second refresh interval directly. No
functional change.
Found using Coccinelle (scripts/coccinelle/misc/secs_to_jiffies.cocci).
Link: https://lore.kernel.org/20260629145538.134832-3-sj@kernel.org
Signed-off-by: Philippe Laferriere <plafer@proton.me>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Cc: Akinobu Mita <akinobu.mita@gmail.com>
Cc: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Doehyun Baek <doehyunbaek@gmail.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: "Liam R. Howlett" <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Sailesh Nandanavanam <saileshnandanavanam@gmail.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "mm/damon: five misc fixups"
Five patches for miscellaneous DAMON fixups. Use better fit kernel
functions, cleanup/fixup documents, and add unit tests.
The five patches were initially sent and revisioned by different
individuals. Each patch contains changelog on their commentary area. The
patches are curated into this series by SJ, for the convenience in
reposting.
This patch (of 5):
damon_commit_target_regions() temporarily allocates a single contiguous
memory region using kmalloc to store copies of all damon_regions of the
damon_target. However, if the damon_target has a large number of
damon_regions, the total size may exceed KMALLOC_MAX_SIZE.
This problem can be avoided by using kvmalloc instead of kmalloc.
Link: https://lore.kernel.org/20260629145538.134832-1-sj@kernel.org
Link: https://lore.kernel.org/20260629145538.134832-2-sj@kernel.org
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Cc: Doehyun Baek <doehyunbaek@gmail.com>
Cc: Philippe Laferriere <plafer@proton.me>
Cc: Sailesh Nandanavanam <saileshnandanavanam@gmail.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Writing a non-zero value to a kdamond's 'refresh_ms' sysfs file should
make DAMON periodically update the read-only sysfs files on its own,
without the user writing update keywords such as 'update_schemes_stats' to
the 'state' file. This behavior has no test coverage.
Add a test that starts a kdamond with refresh_ms set and a 'stat' scheme
whose default access pattern matches every monitored region, then polls
the scheme's 'nr_tried' stats file directly, without requesting an update.
The value can become non-zero only via the periodic refresh, so the test
confirms refresh_ms works; with refresh_ms disabled the stat stays zero
and the test fails.
Link: https://lore.kernel.org/20260602131217.2210912-3-linuxoid@gmail.com
Link: https://lore.kernel.org/20260629144927.134237-3-sj@kernel.org
Signed-off-by: Ruslan Valiyev <linuxoid@gmail.com>
Reviewed-by: SJ Park <sj@kernel.org>
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "selftests/damon: test kdamond refresh_ms", v2.
The kdamond 'refresh_ms' sysfs file makes DAMON periodically update its
read-only sysfs files (DAMOS stats, tuned monitoring intervals and the
kdamond pid) on its own, so users don't have to write update keywords such
as 'update_schemes_stats' to the 'state' file. It has no selftest
coverage.
The first patch adds refresh_ms support to the _damon_sysfs.py test
control module. The second adds a test that sets refresh_ms and confirms
a scheme's stats are updated under sysfs without an explicit update
request; the test skips on kernels that predate the refresh_ms file.
Tested on current mainline under a DAMON-enabled kernel: the new test
passes and the existing DAMON selftests show no new failures.
This patch (of 2):
The Kdamond class has no way to set the kdamond-level 'refresh_ms' sysfs
file, which makes DAMON periodically update the read-only sysfs files
(DAMOS stats, tuned monitoring intervals and the kdamond pid) on its own.
Add a 'refresh_ms' parameter to Kdamond. When it is set (including to
zero, to disable the periodic update), write it before turning the kdamond
on, so tests can exercise the auto-update behavior. Leaving it unset
keeps the previous behavior of not touching the file, so callers running
against kernels without the feature are unaffected.
Link: https://lore.kernel.org/20260602131217.2210912-2-linuxoid@gmail.com
Link: https://lore.kernel.org/20260629144927.134237-2-sj@kernel.org
Signed-off-by: Ruslan Valiyev <linuxoid@gmail.com>
Reviewed-by: SJ Park <sj@kernel.org>
Signed-off-by: SJ Park <sj@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Add a dedicated test at the end of main() that stages memcg_path via sysfs
and verifies its readback. Configure the memcg filter before start(), do
not call commit(), and ignore start() failures so the test does not depend
on CONFIG_MEMCG or cgroup layout. Call stop() for cleanup without
checking its return value.
Link: https://lore.kernel.org/D2B37130D38E09AC+20260601090634.241864-1-niecheng1@uniontech.com
Link: https://lore.kernel.org/20260629144812.134159-3-sj@kernel.org
Signed-off-by: Cheng Nie <niecheng1@uniontech.com>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "selftests/damon: fix memcg_path staging handling", v5.
Fix a bug in _damon_sysfs.py for damos_filter memcg_path setup, and add
a test case of it in sysfs.py.
This patch (of 2):
DamosFilter stores memcg_path for sysfs staging, but the constructor
assigns it with a trailing comma and therefore turns it into a tuple.
Fix the assignment so memcg_path is stored as the intended string. This
makes memcg filter staging and follow-up validation use the written path
correctly.
Link: https://lore.kernel.org/20260629144812.134159-1-sj@kernel.org
Link: https://lore.kernel.org/464AE12D4BC6B6F4+20260601090519.240482-1-niecheng1@uniontech.com
Link: https://lore.kernel.org/20260629144812.134159-2-sj@kernel.org
Signed-off-by: Cheng Nie <niecheng1@uniontech.com>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
'hugeapge_size' in drgn_dump_damon_status.py was a dead elif branch.
$fail_reason in sysfs.sh was undefined, silently emptying the error
message. 'exit' instead of 'exist' in sysfs.sh skipped a file existence
check. 'nohugeapge' in sysfs.py broke an action dict lookup.
Fix other wrong strings in the same files.
Link: https://lore.kernel.org/20260601032314.424013-4-kunwu.chan@linux.dev
Link: https://lore.kernel.org/20260629144648.134092-4-sj@kernel.org
Co-developed-by: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Cc: Kunwu Chan <chentao@kylinos.cn>
Cc: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
The expectation print has wrong operator precedence: '%' binds before the
conditional expression, so the else branch prints 'not met' without the
prefix 'expectation (>= 14) is'. Add parentheses to fix it.
Also, '\n'.join() on the list of ints raises TypeError; convert to str in
the list comprehension.
Link: https://lore.kernel.org/20260601032314.424013-3-kunwu.chan@linux.dev
Link: https://lore.kernel.org/20260629144648.134092-3-sj@kernel.org
Co-developed-by: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Cc: Kunwu Chan <chentao@kylinos.cn>
Cc: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "selftests/damon: misc fixes for test bugs", v3.
This series fixes several bugs in the DAMON selftests. Most are trivial
but makes test output wrong or even silently pass the one test case for
'avail_operation' file existence check.
Patch 1 fixes mutable default arguments in DamonCtx.__init__() that cause
state to leak between test instances.
Patch 2 fixes wrong operator precedence and join TypeError in
damos_tried_regions.py.
Patch 3 fixes several wrong strings that produce dead elif branches,
skipped file existence checks, and broken dict key lookups.
This patch (of 3):
DamonCtx.__init__() uses mutable default values for monitoring_attrs,
targets, and schemes. In Python these are evaluated once at function
definition time, so multiple DamonCtx instances can unintentionally share
the same lists and DamonAttrs instance.
Replace the mutable defaults with None sentinels and initialize the
objects when needed.
Link: https://lore.kernel.org/20260629144648.134092-1-sj@kernel.org
Link: https://lore.kernel.org/20260601032314.424013-2-kunwu.chan@linux.dev
Link: https://lore.kernel.org/20260629144648.134092-2-sj@kernel.org
Co-developed-by: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: SJ Park <sj@kernel.org>
Cc: Kunwu Chan <chentao@kylinos.cn>
Cc: Wang Lian <lianux.mm@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Let's clean it up a bit:
(1) There is no need to pass "full" anymore.
(2) No architecture overwrites it, and there isn't really a good reason
to do so when dealing with non-present PTEs.
(3) While at it, call it "non-present", similar to copy_nonpresent_pte()
and zap_nonpresent_ptes().
It's a shame that we have clear_non_present_ptes() correspond to
pte_clear() and clear_ptes() correspond to ptep_get_and_clear*().
Link: https://lore.kernel.org/20260629-clear_not_present_full_ptes-v2-3-96089871a1e7@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Oscar Salvador (SUSE) <osalvador@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Andreas Larsson <andreas@gaisler.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
In general, there is no good reason to do anything special when clearing
non-present PTEs.
In theory, HW that does have to invalidate TLBs for non-present PTEs could
benefit from a "full" parameter, but fortunately
pte_clear_not_present_full() is not wired up anymore ... and there would
have to be something very convincing for us to care about that to re-add
it.
So, let's just use pte_clear() directly now. To prevent the compiler
complaining on some configs about "set but not used" addr parameter,
silence that here.
Link: https://lore.kernel.org/20260629-clear_not_present_full_ptes-v2-2-96089871a1e7@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Oscar Salvador (SUSE) <osalvador@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Andreas Larsson <andreas@gaisler.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "mm: cleanup clear_not_present_full_ptes()", v2.
While doing some review, I stumbled over clear_not_present_full_ptes() and
concluded that it needs some love.
Let's remove pte_clear_not_present_full() and cleanup
clear_not_present_full_ptes(), renaming it to clear_non_present_ptes().
This patch (of 3):
On sparc64, pte_clear_not_present_full() nowadays does a simple
__set_pte_at(). In __set_pte_at() -> maybe_tlb_batch_add(), we check
pte_accessible() to see whether to call tlb_batch_add().
However, non-present PTEs are surely not accessible, so tlb_batch_add() is
never called and the "full" parameter is irrelevant.
Let's drop the helper and just let common code do a pte_clear().
pte_clear() on sparc64 maps to set_pte_at()->set_ptes()->__set_pte_at()
... so it ends up calling the same function, just with "full=0".
Given that "full" is irrelevant, there is no change.
We added pte_clear_not_present_full() for sparc64 in commit 90f08e399d
("sparc: mmu_gather rework"), and I suspect that it was already not
required back then.
Link: https://lore.kernel.org/20260629-clear_not_present_full_ptes-v2-0-96089871a1e7@kernel.org
Link: https://lore.kernel.org/20260629-clear_not_present_full_ptes-v2-1-96089871a1e7@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andreas Larsson <andreas@gaisler.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Jann Horn <jannh@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Oscar Salvador (SUSE) <osalvador@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
the issue for tried region directories in the error handling path of
damon_sysfs_populate_region_dir() by adding a kobject_del() call.
Link: https://lore.kernel.org/20260628220121.97360-12-sj@kernel.org
Fixes: b574a82d10 ("mm/damon/sysfs-schemes: implement tried_regions/<r>/probes/")
Signed-off-by: SJ Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
the issue for data attribute probe filter directories in the error
handling path of damon_sysfs_probes_add_dirs() by adding a kobject_del()
call.
Link: https://lore.kernel.org/20260628220121.97360-11-sj@kernel.org
Fixes: af7cb41af9 ("mm/damon/sysfs: implement filters directory")
Signed-off-by: SJ Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
those issues for data attribute probe filter directories by adding
kobject_del() calls.
Link: https://lore.kernel.org/20260628220121.97360-10-sj@kernel.org
Fixes: 82e66aef7714 ("mm/damon/sysfs: implement filter dir")
Signed-off-by: SJ Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
those issues for data attribute probe directories by adding kobject_del()
calls.
Link: https://lore.kernel.org/20260628220121.97360-9-sj@kernel.org
Fixes: bf3ea3d30880 ("mm/damon/sysfs: implement probe dir")
Signed-off-by: SJ Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
those issues for scheme action destination directories by adding
kobject_del() calls.
Link: https://lore.kernel.org/20260628220121.97360-8-sj@kernel.org
Fixes: 2cd0bf85a2 ("mm/damon/sysfs-schemes: implement DAMOS action destinations directory")
Signed-off-by: SJ Park <sj@kernel.org>
Cc: <stable@vger.kernel.org> # 6.17.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
those issues for scheme quota goal directories by adding kobject_del()
calls.
Link: https://lore.kernel.org/20260628220121.97360-7-sj@kernel.org
Fixes: 7f262da0a3 ("mm/damon/sysfs-schemes: implement files for scheme quota goals setup")
Signed-off-by: SJ Park <sj@kernel.org>
Cc: <stable@vger.kernel.org> # 6.8.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
those issues for scheme filter directories by adding kobject_del() calls.
Link: https://lore.kernel.org/20260628220121.97360-6-sj@kernel.org
Fixes: 472e2b70ed ("mm/damon/sysfs-schemes: connect filter directory and filters directory")
Signed-off-by: SJ Park <sj@kernel.org>
Cc: <stable@vger.kernel.org> # 6.3.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
those issues for scheme region directories by adding kobject_del() calls.
This issue was discovered [1] by Sashiko, though its analysis was
partially incorrect.
Link: https://lore.kernel.org/20260628220121.97360-5-sj@kernel.org
Link: https://lore.kernel.org/20260517205828.6204-1-sj@kernel.org [1]
Fixes: 9277d0367b ("mm/damon/sysfs-schemes: implement scheme region directory")
Signed-off-by: SJ Park <sj@kernel.org>
Cc: <stable@vger.kernel.org> # 6.2.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
those issues for scheme directories by adding kobject_del() calls.
Link: https://lore.kernel.org/20260628220121.97360-4-sj@kernel.org
Fixes: 7e84b1f821 ("mm/damon/sysfs: support DAMON-based Operation Schemes")
Signed-off-by: SJ Park <sj@kernel.org>
Cc: <stable@vger.kernel.org> # 5.18.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
those issues for the normal creation path of region directories and the
error path of target directories, by adding kobject_del() calls.
Link: https://lore.kernel.org/20260628220121.97360-3-sj@kernel.org
Fixes: 2031b14ea7 ("mm/damon/sysfs: support the physical address space monitoring")
Signed-off-by: SJ Park <sj@kernel.org>
Cc: <stable@vger.kernel.org> # 5.18.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Patch series "mm/damon/sysfs: kobject_del() directories that users can
create/remove".
DAMON sysfs interface allows users to create and remove arbitrary number
of directories on sysfs, using a few files having 'nr_' prefix. For
example, 'nr_kdamonds'. When the user writes a number 'N' to the files,
directories having name starting from '0' to 'N - 1' are created in the
same directory. The pre-existing number-named directories are removed
before creating the new directories.
For the removal of the existing directories, DAMON sysfs interface use
only kobject_put(). Because DAMON sysfs interface is the only kernel
component that manages the directories, there is no problem in normal
situations. However, if CONFIG_DEBUG_KOBJECT_RELEASE is enabled, the
removal of dirs are delayed. Let's suppose a user writes a non-zero
number to the 'nr_*' files while there are pre-existing number-named
directories, on the config enabled kernel. DAMON sysfs interface
decreases the reference counts of the existing directories and immediately
creates new directories. Because the removal of the sysfs directories is
delayed, it shows some pre-existing directories of the same names when it
tries to create the new directories, and fails.
For example, the issue can be triggered like below:
# grep DEBUG_KOBJECT_RELEASE /boot/config-$(uname -r)
CONFIG_DEBUG_KOBJECT_RELEASE=y
# ls
nr_kdamonds
# echo 1 > nr_kdamonds
# echo 1 > nr_kdamonds
bash: echo: write error: File exists
# dmesg
[...]
[ 300.880458] kobject: kobject_add_internal failed for 0 with -EEXIST, don't try to register things with the same name in the same directory.
[...]
Some of the error handling paths of the directories also lack the
kobject_del() call. If the user uses nr_* file right after the errors,
similar issues can happen.
This doesn't cause catastrophic issues like kernel panics or memory
corruptions. Users can work around by removing all directories first
(write 0 to the nr_* files) and then create new directories after
confirming the old directories are gone. But, this is definitely a bug
that causes a bad user experience.
Fix the issues by calling kobject_del() before creating new directories.
This patch (of 11)
On CONFIG_DEBUG_KOBJECT_RELEASE enabled kernel, lack of kobject_del()
could cause directories creation failures due to the name conflicts. Fix
those issues for normal creation paths of target, context and kdamond
directories, and error paths of context and kdamond directories by adding
kobject_del() calls.
Note that this fix for target directories is not complete since it has a
similar issue in the damon_sysfs_targets_add_dirs() error path. Because
the normal path issue and the error path issue are introduced by different
commits, this commit is fixing only the normal path issue. A commit for
the error path will be added next.
Link: https://lore.kernel.org/20260628220121.97360-1-sj@kernel.org
Link: https://lore.kernel.org/20260628220121.97360-2-sj@kernel.org
Fixes: c951cd3b89 ("mm/damon: implement a minimal stub for sysfs-based DAMON interface")
Signed-off-by: SJ Park <sj@kernel.org>
Cc: <stable@vger.kernel.org> # 5.18.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
damon_sample_prcl_start() calls damon_call() right after damon_start() is
succeeded. The kdamond that has started by the damon_start() could be
terminated by itself before or in the middle of the damon_call()
execution. There could be multiple reasons for such a stop including
monitoring target process termination and kdamond_fn() internal memory
allocation failures. In the case, damon_call() will fail and return an
error without cleaning up the DAMON context object. The
damon_sample_prcl_start() caller assumes it would clean up the object,
though. When the user requests to start DAMON again,
damon_sample_prcl_start() is called again, allocates a new DAMON context
object and overwrites the pointer for the previous object. As a result,
the previous context object is leaked.
Safely stop the kdamond and deallocate the context object when the failure
is returned. Note that the kdamond should be stopped first, because
damon_call() failure means not complete termination of the kdamond but
only the fact that the termination process has started.
The user impact shouldn't be that significant because the race is not easy
to happen, and only up to one DAMON context object can be leaked per race.
The issue was discovered [1] by Sashiko.
Link: https://lore.kernel.org/20260628215447.96166-7-sj@kernel.org
Link: https://lore.kernel.org/20260610035214.4850-1-sj@kernel.org [1]
Fixes: a6c33f1054 ("samples/damon/prcl: use damon_call() repeat mode instead of damon_callback")
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: Zenghui Yu <zenghui.yu@linux.dev>
Cc: <stable@vger.kernel.org> # 6.17.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
damon_sample_wsse_start() calls damon_call() right after damon_start() is
succeeded. The kdamond that has started by the damon_start() could be
terminated by itself before or in the middle of the damon_call()
execution. There could be multiple reasons for such a stop including
monitoring target process termination and kdamond_fn() internal memory
allocation failures. In the case, damon_call() will fail and return an
error without cleaning up the DAMON context object. The
damon_sample_wsse_start() caller assumes it would clean up the object,
though. When the user requests to start DAMON again,
damon_sample_wsse_start() is called again, allocates a new DAMON context
object and overwrites the pointer for the previous object. As a result,
the previous context object is leaked.
Safely stop the kdamond and deallocate the context object when the failure
is returned. Note that the kdamond should be stopped first, because
damon_call() failure means not complete termination of the kdamond but
only the fact that the termination process has started.
The user impact shouldn't be that significant because the race is not easy
to happen, and only up to one DAMON context object can be leaked per race.
The issue was discovered [1] by Sashiko.
Link: https://lore.kernel.org/20260628215447.96166-6-sj@kernel.org
Link: https://lore.kernel.org/20260610034828.4632-1-sj@kernel.org [1]
Fixes: cc9c1b8c20 ("samples/damon/wsse: use damon_call() repeat mode instead of damon_callback")
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: Zenghui Yu <zenghui.yu@linux.dev>
Cc: <stable@vger.kernel.org> # 6.17.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
damon_sample_mtier_stop() assumes its damon_stop() call will always
successfully stops the two DAMON contexts. Hence it deallocates the two
DAMON contexts after the damon_stop() call. However, if a given context
is already stopped, damon_stop() fails and returns an error while letting
the DAMON contexts that have not yet stopped keep running. This kind of
unexpected early DAMON context stops could happen due to memory allocation
failures in kdamond_fn(). Because damon_sample_mtier_stop() just
deallocates all DAMON contexts with damon_target and damon_region objects
that are linked to the contexts, the execution of the unstopped DAMON
context (kdamond) ends up using the memory that freed (use-after-free).
Fix the issue by separating the damon_stop() to be invoked per context.
Note that DAMON_SYSFS also allows multiple DAMON contexts execution. But,
it calls damon_stop() for each context one by one. Hence this issue is
only in mtier.
For the long term, it would be better to refactor damon_stop() to always
ensure stopping all contexts regardless of the failures in the middle.
Make this fix in the current way, though, to keep it simple and easy to
backport. I will do the refactoring later.
The issue was discovered [1] by Sashiko.
Link: https://lore.kernel.org/20260628215447.96166-5-sj@kernel.org
Link: https://lore.kernel.org/20260609014219.3013-1-sj@kernel.org [1]
Fixes: 82a08bde3c ("samples/damon: implement a DAMON module for memory tiering")
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: Zenghui Yu <zenghui.yu@linux.dev>
Cc: <stable@vger.kernel.org> # 6.16.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
damon_sample_mtier_start() callers assume it will clean up resources when
it fails. And the function does the cleanup for context buildup failures.
However, it is not doing the cleanup for damon_start() failure.
As a result, when damon_start() fails, it could leak the memory for DAMON
context. Also, if damon_start() fails for only the second context, the
first context will indefinitely run, and avoid starting other DAMON
contexts since it is running in the exclusive mode. Stop possibly started
DAMON context and free the contexts in case of the failure to fix the
issues.
Note that the issue can reliably be reproduced because the module calls
damon_start() in the exclusive mode. For example,
$ sudo damo start
$ echo Y | sudo tee /sys/module/damon_sample_mtier/parameters/enabled
$ sudo cat /proc/allocinfo | grep damon_new_ctx
Because the first command is running another DAMON instance, the second
command fails the damon_start() call because the new DAMON instance cannot
exclusively run. And without this fix, by repeating the second and the
third commands above, we can show the memory consumption is only
increasing due to the leaks. It requires the sudo permission though.
The issue was discovered [1] by Sashiko.
Link: https://lore.kernel.org/20260628215447.96166-4-sj@kernel.org
Link: https://lore.kernel.org/20260608112455.274231F00893@smtp.kernel.org [1]
Fixes: 82a08bde3c ("samples/damon: implement a DAMON module for memory tiering")
Signed-off-by: SJ Park <sj@kernel.org>
Reviewed-by: Zenghui Yu <zenghui.yu@linux.dev>
Cc: <stable@vger.kernel.org> # 6.16.x
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
