kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-11-13 07:31:07 +00:00
Code Issues Releases Packages Activity
linux/drivers/iommu/iommu-debug-pagealloc.c

164 lines
4 KiB
Raw Normal View History

iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC Add a new config IOMMU_DEBUG_PAGEALLOC, which registers new data to page_ext. This config will be used by the IOMMU API to track pages mapped in the IOMMU to catch drivers trying to free kernel memory that they still map in their domains, causing all types of memory corruption. This behaviour is disabled by default and can be enabled using kernel cmdline iommu.debug_pagealloc. Acked-by: David Hildenbrand (Red Hat) <david@kernel.org> Reviewed-by: Pranjal Shrivastava <praan@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:02 +00:00
iommu: Add calls for IOMMU_DEBUG_PAGEALLOC Add calls for the new iommu debug config IOMMU_DEBUG_PAGEALLOC: - iommu_debug_init: Enable the debug mode if configured by the user. - iommu_debug_map: Track iommu pages mapped, using physical address. - iommu_debug_unmap_begin: Track start of iommu unmap operation, with IOVA and size. - iommu_debug_unmap_end: Track the end of unmap operation, passing the actual unmapped size versus the tracked one at unmap_begin. We have to do the unmap_begin/end as once pages are unmapped we lose the information of the physical address. This is racy, but the API is racy by construction as it uses refcounts and doesn't attempt to lock/synchronize with the IOMMU API as that will be costly, meaning that possibility of false negative exists. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:03 +00:00
iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC Add a new config IOMMU_DEBUG_PAGEALLOC, which registers new data to page_ext. This config will be used by the IOMMU API to track pages mapped in the IOMMU to catch drivers trying to free kernel memory that they still map in their domains, causing all types of memory corruption. This behaviour is disabled by default and can be enabled using kernel cmdline iommu.debug_pagealloc. Acked-by: David Hildenbrand (Red Hat) <david@kernel.org> Reviewed-by: Pranjal Shrivastava <praan@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:02 +00:00
iommu: debug-pagealloc: Check mapped/unmapped kernel memory Now, as the page_ext holds count of IOMMU mappings, we can use it to assert that any page allocated/freed is indeed not in the IOMMU. The sanitizer doesn’t protect against mapping/unmapping during this period. However, that’s less harmful as the page is not used by the kernel. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:05 +00:00
iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC Add a new config IOMMU_DEBUG_PAGEALLOC, which registers new data to page_ext. This config will be used by the IOMMU API to track pages mapped in the IOMMU to catch drivers trying to free kernel memory that they still map in their domains, causing all types of memory corruption. This behaviour is disabled by default and can be enabled using kernel cmdline iommu.debug_pagealloc. Acked-by: David Hildenbrand (Red Hat) <david@kernel.org> Reviewed-by: Pranjal Shrivastava <praan@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:02 +00:00
iommu: Add calls for IOMMU_DEBUG_PAGEALLOC Add calls for the new iommu debug config IOMMU_DEBUG_PAGEALLOC: - iommu_debug_init: Enable the debug mode if configured by the user. - iommu_debug_map: Track iommu pages mapped, using physical address. - iommu_debug_unmap_begin: Track start of iommu unmap operation, with IOVA and size. - iommu_debug_unmap_end: Track the end of unmap operation, passing the actual unmapped size versus the tracked one at unmap_begin. We have to do the unmap_begin/end as once pages are unmapped we lose the information of the physical address. This is racy, but the API is racy by construction as it uses refcounts and doesn't attempt to lock/synchronize with the IOMMU API as that will be costly, meaning that possibility of false negative exists. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:03 +00:00
iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC Add a new config IOMMU_DEBUG_PAGEALLOC, which registers new data to page_ext. This config will be used by the IOMMU API to track pages mapped in the IOMMU to catch drivers trying to free kernel memory that they still map in their domains, causing all types of memory corruption. This behaviour is disabled by default and can be enabled using kernel cmdline iommu.debug_pagealloc. Acked-by: David Hildenbrand (Red Hat) <david@kernel.org> Reviewed-by: Pranjal Shrivastava <praan@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:02 +00:00
iommu: Add calls for IOMMU_DEBUG_PAGEALLOC Add calls for the new iommu debug config IOMMU_DEBUG_PAGEALLOC: - iommu_debug_init: Enable the debug mode if configured by the user. - iommu_debug_map: Track iommu pages mapped, using physical address. - iommu_debug_unmap_begin: Track start of iommu unmap operation, with IOVA and size. - iommu_debug_unmap_end: Track the end of unmap operation, passing the actual unmapped size versus the tracked one at unmap_begin. We have to do the unmap_begin/end as once pages are unmapped we lose the information of the physical address. This is racy, but the API is racy by construction as it uses refcounts and doesn't attempt to lock/synchronize with the IOMMU API as that will be costly, meaning that possibility of false negative exists. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:03 +00:00
iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC Add a new config IOMMU_DEBUG_PAGEALLOC, which registers new data to page_ext. This config will be used by the IOMMU API to track pages mapped in the IOMMU to catch drivers trying to free kernel memory that they still map in their domains, causing all types of memory corruption. This behaviour is disabled by default and can be enabled using kernel cmdline iommu.debug_pagealloc. Acked-by: David Hildenbrand (Red Hat) <david@kernel.org> Reviewed-by: Pranjal Shrivastava <praan@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:02 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: debug-pagealloc: Use page_ext_get_from_phys() Instead of calling pfn_valid() and then getting the page, call the newly added function page_ext_get_from_phys(), which would also check for MMIO and offline memory and return NULL in that case. Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-20 09:19:26 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: debug-pagealloc: Use page_ext_get_from_phys() Instead of calling pfn_valid() and then getting the page, call the newly added function page_ext_get_from_phys(), which would also check for MMIO and offline memory and return NULL in that case. Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-20 09:19:26 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: debug-pagealloc: Use page_ext_get_from_phys() Instead of calling pfn_valid() and then getting the page, call the newly added function page_ext_get_from_phys(), which would also check for MMIO and offline memory and return NULL in that case. Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-20 09:19:26 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: debug-pagealloc: Use page_ext_get_from_phys() Instead of calling pfn_valid() and then getting the page, call the newly added function page_ext_get_from_phys(), which would also check for MMIO and offline memory and return NULL in that case. Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-20 09:19:26 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: debug-pagealloc: Check mapped/unmapped kernel memory Now, as the page_ext holds count of IOMMU mappings, we can use it to assert that any page allocated/freed is indeed not in the IOMMU. The sanitizer doesn’t protect against mapping/unmapping during this period. However, that’s less harmful as the page is not used by the kernel. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:05 +00:00
iommu: Add calls for IOMMU_DEBUG_PAGEALLOC Add calls for the new iommu debug config IOMMU_DEBUG_PAGEALLOC: - iommu_debug_init: Enable the debug mode if configured by the user. - iommu_debug_map: Track iommu pages mapped, using physical address. - iommu_debug_unmap_begin: Track start of iommu unmap operation, with IOVA and size. - iommu_debug_unmap_end: Track the end of unmap operation, passing the actual unmapped size versus the tracked one at unmap_begin. We have to do the unmap_begin/end as once pages are unmapped we lose the information of the physical address. This is racy, but the API is racy by construction as it uses refcounts and doesn't attempt to lock/synchronize with the IOMMU API as that will be costly, meaning that possibility of false negative exists. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:03 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: debug-pagealloc: Use page_ext_get_from_phys() Instead of calling pfn_valid() and then getting the page, call the newly added function page_ext_get_from_phys(), which would also check for MMIO and offline memory and return NULL in that case. Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-20 09:19:26 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: debug-pagealloc: Use page_ext_get_from_phys() Instead of calling pfn_valid() and then getting the page, call the newly added function page_ext_get_from_phys(), which would also check for MMIO and offline memory and return NULL in that case. Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-20 09:19:26 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: Add calls for IOMMU_DEBUG_PAGEALLOC Add calls for the new iommu debug config IOMMU_DEBUG_PAGEALLOC: - iommu_debug_init: Enable the debug mode if configured by the user. - iommu_debug_map: Track iommu pages mapped, using physical address. - iommu_debug_unmap_begin: Track start of iommu unmap operation, with IOVA and size. - iommu_debug_unmap_end: Track the end of unmap operation, passing the actual unmapped size versus the tracked one at unmap_begin. We have to do the unmap_begin/end as once pages are unmapped we lose the information of the physical address. This is racy, but the API is racy by construction as it uses refcounts and doesn't attempt to lock/synchronize with the IOMMU API as that will be costly, meaning that possibility of false negative exists. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:03 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: Add calls for IOMMU_DEBUG_PAGEALLOC Add calls for the new iommu debug config IOMMU_DEBUG_PAGEALLOC: - iommu_debug_init: Enable the debug mode if configured by the user. - iommu_debug_map: Track iommu pages mapped, using physical address. - iommu_debug_unmap_begin: Track start of iommu unmap operation, with IOVA and size. - iommu_debug_unmap_end: Track the end of unmap operation, passing the actual unmapped size versus the tracked one at unmap_begin. We have to do the unmap_begin/end as once pages are unmapped we lose the information of the physical address. This is racy, but the API is racy by construction as it uses refcounts and doesn't attempt to lock/synchronize with the IOMMU API as that will be costly, meaning that possibility of false negative exists. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:03 +00:00
iommu: debug-pagealloc: Track IOMMU pages Using the new calls, use an atomic refcount to track how many times a page is mapped in any of the IOMMUs. For unmap we need to use iova_to_phys() to get the physical address of the pages. We use the smallest supported page size as the granularity of tracking per domain. This is important as it is possible to map pages and unmap them with larger sizes (as in map_sg()) cases. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:04 +00:00
iommu: Add calls for IOMMU_DEBUG_PAGEALLOC Add calls for the new iommu debug config IOMMU_DEBUG_PAGEALLOC: - iommu_debug_init: Enable the debug mode if configured by the user. - iommu_debug_map: Track iommu pages mapped, using physical address. - iommu_debug_unmap_begin: Track start of iommu unmap operation, with IOVA and size. - iommu_debug_unmap_end: Track the end of unmap operation, passing the actual unmapped size versus the tracked one at unmap_begin. We have to do the unmap_begin/end as once pages are unmapped we lose the information of the physical address. This is racy, but the API is racy by construction as it uses refcounts and doesn't attempt to lock/synchronize with the IOMMU API as that will be costly, meaning that possibility of false negative exists. Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Pranjal Shrivastava <praan@google.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:03 +00:00
iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC Add a new config IOMMU_DEBUG_PAGEALLOC, which registers new data to page_ext. This config will be used by the IOMMU API to track pages mapped in the IOMMU to catch drivers trying to free kernel memory that they still map in their domains, causing all types of memory corruption. This behaviour is disabled by default and can be enabled using kernel cmdline iommu.debug_pagealloc. Acked-by: David Hildenbrand (Red Hat) <david@kernel.org> Reviewed-by: Pranjal Shrivastava <praan@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-09 17:18:02 +00:00