linux/tools/testing/cxl/test/mem.c

1897 lines
48 KiB

cxl/test: Remove ret_limit race condition in mock_get_event() Commit 364ee9f3265e ("cxl/test: Enhance event testing") changed the loop iterator in mock_get_event() from a static constant, CXL_TEST_EVENT_CNT, to a dynamic global variable, ret_limit. The intent was to vary the number of events returned per call to simulate events occurring while logs are being read. However, ret_limit is modified without synchronization. When multiple threads call mock_get_event() concurrently, one thread may read ret_limit, another thread may increment it, and the first thread's loop condition and size calculation see and use the updated value. This is visible during cxl_test module load when all memdevs are initializing simultaneously, which includes getting event records. It is not tied to the cxl-events.sh unit test specifically, as that operates on a single memdev. While no actual harm results (the buffer is always large enough and the record count fields correctly reflect what was written), this is a correctness issue. The race creates an inconsistent state within mock_get_event() and adding variability based on a race appears unintended. Make ret_limit a local variable populated from an atomic counter. Each call gets a stable value that won't change during execution. That preserves the intended behavior of varying the return counts across calls while eliminating the race condition. This implementation uses "+ 1" to produce the full range of 1 to CXL_TEST_EVENT_RET_MAX (4) records. Previously only 1, 2, 3 were produced. Signed-off-by: Alison Schofield <alison.schofield@intel.com> Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com>> --- Link: https://patch.msgid.link/20251116013819.1713780-1-alison.schofield@intel.com Signed-off-by: Dave Jiang <dave.jiang@intel.com>
2025-11-15 17:37:49 -08:00
cxl/test: Remove ret_limit race condition in mock_get_event() Commit 364ee9f3265e ("cxl/test: Enhance event testing") changed the loop iterator in mock_get_event() from a static constant, CXL_TEST_EVENT_CNT, to a dynamic global variable, ret_limit. The intent was to vary the number of events returned per call to simulate events occurring while logs are being read. However, ret_limit is modified without synchronization. When multiple threads call mock_get_event() concurrently, one thread may read ret_limit, another thread may increment it, and the first thread's loop condition and size calculation see and use the updated value. This is visible during cxl_test module load when all memdevs are initializing simultaneously, which includes getting event records. It is not tied to the cxl-events.sh unit test specifically, as that operates on a single memdev. While no actual harm results (the buffer is always large enough and the record count fields correctly reflect what was written), this is a correctness issue. The race creates an inconsistent state within mock_get_event() and adding variability based on a race appears unintended. Make ret_limit a local variable populated from an atomic counter. Each call gets a stable value that won't change during execution. That preserves the intended behavior of varying the return counts across calls while eliminating the race condition. This implementation uses "+ 1" to produce the full range of 1 to CXL_TEST_EVENT_RET_MAX (4) records. Previously only 1, 2, 3 were produced. Signed-off-by: Alison Schofield <alison.schofield@intel.com> Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com>> --- Link: https://patch.msgid.link/20251116013819.1713780-1-alison.schofield@intel.com Signed-off-by: Dave Jiang <dave.jiang@intel.com>
2025-11-15 17:37:49 -08:00
cxl/test: Remove ret_limit race condition in mock_get_event() Commit 364ee9f3265e ("cxl/test: Enhance event testing") changed the loop iterator in mock_get_event() from a static constant, CXL_TEST_EVENT_CNT, to a dynamic global variable, ret_limit. The intent was to vary the number of events returned per call to simulate events occurring while logs are being read. However, ret_limit is modified without synchronization. When multiple threads call mock_get_event() concurrently, one thread may read ret_limit, another thread may increment it, and the first thread's loop condition and size calculation see and use the updated value. This is visible during cxl_test module load when all memdevs are initializing simultaneously, which includes getting event records. It is not tied to the cxl-events.sh unit test specifically, as that operates on a single memdev. While no actual harm results (the buffer is always large enough and the record count fields correctly reflect what was written), this is a correctness issue. The race creates an inconsistent state within mock_get_event() and adding variability based on a race appears unintended. Make ret_limit a local variable populated from an atomic counter. Each call gets a stable value that won't change during execution. That preserves the intended behavior of varying the return counts across calls while eliminating the race condition. This implementation uses "+ 1" to produce the full range of 1 to CXL_TEST_EVENT_RET_MAX (4) records. Previously only 1, 2, 3 were produced. Signed-off-by: Alison Schofield <alison.schofield@intel.com> Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com>> --- Link: https://patch.msgid.link/20251116013819.1713780-1-alison.schofield@intel.com Signed-off-by: Dave Jiang <dave.jiang@intel.com>
2025-11-15 17:37:49 -08:00
cxl: Introduce 'struct cxl_dpa_partition' and 'struct cxl_range_info' The pending efforts to add CXL Accelerator (type-2) device [1], and Dynamic Capacity (DCD) support [2], tripped on the no-longer-fit-for-purpose design in the CXL subsystem for tracking device-physical-address (DPA) metadata. Trip hazards include: - CXL Memory Devices need to consider a PMEM partition, but Accelerator devices with CXL.mem likely do not in the common case. - CXL Memory Devices enumerate DPA through Memory Device mailbox commands like Partition Info, Accelerators devices do not. - CXL Memory Devices that support DCD support more than 2 partitions. Some of the driver algorithms are awkward to expand to > 2 partition cases. - DPA performance data is a general capability that can be shared with accelerators, so tracking it in 'struct cxl_memdev_state' is no longer suitable. - Hardcoded assumptions around the PMEM partition always being index-1 if RAM is zero-sized or PMEM is zero sized. - 'enum cxl_decoder_mode' is sometimes a partition id and sometimes a memory property, it should be phased in favor of a partition id and the memory property comes from the partition info. Towards cleaning up those issues and allowing a smoother landing for the aforementioned pending efforts, introduce a 'struct cxl_dpa_partition' array to 'struct cxl_dev_state', and 'struct cxl_range_info' as a shared way for Memory Devices and Accelerators to initialize the DPA information in 'struct cxl_dev_state'. For now, split a new cxl_dpa_setup() from cxl_mem_create_range_info() to get the new data structure initialized, and cleanup some qos_class init. Follow on patches will go further to use the new data structure to cleanup algorithms that are better suited to loop over all possible partitions. cxl_dpa_setup() follows the locking expectations of mutating the device DPA map, and is suitable for Accelerator drivers to use. Accelerators likely only have one hardcoded 'ram' partition to convey to the cxl_core. Link: http://lore.kernel.org/20241230214445.27602-1-alejandro.lucero-palau@amd.com [1] Link: http://lore.kernel.org/20241210-dcd-type2-upstream-v8-0-812852504400@intel.com [2] Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Reviewed-by: Alejandro Lucero <alucerop@amd.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Tested-by: Alejandro Lucero <alucerop@amd.com> Link: https://patch.msgid.link/173864305827.668823.13978794102080021276.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dave Jiang <dave.jiang@intel.com>
2025-02-03 20:24:18 -08:00
cxl: Introduce 'struct cxl_dpa_partition' and 'struct cxl_range_info' The pending efforts to add CXL Accelerator (type-2) device [1], and Dynamic Capacity (DCD) support [2], tripped on the no-longer-fit-for-purpose design in the CXL subsystem for tracking device-physical-address (DPA) metadata. Trip hazards include: - CXL Memory Devices need to consider a PMEM partition, but Accelerator devices with CXL.mem likely do not in the common case. - CXL Memory Devices enumerate DPA through Memory Device mailbox commands like Partition Info, Accelerators devices do not. - CXL Memory Devices that support DCD support more than 2 partitions. Some of the driver algorithms are awkward to expand to > 2 partition cases. - DPA performance data is a general capability that can be shared with accelerators, so tracking it in 'struct cxl_memdev_state' is no longer suitable. - Hardcoded assumptions around the PMEM partition always being index-1 if RAM is zero-sized or PMEM is zero sized. - 'enum cxl_decoder_mode' is sometimes a partition id and sometimes a memory property, it should be phased in favor of a partition id and the memory property comes from the partition info. Towards cleaning up those issues and allowing a smoother landing for the aforementioned pending efforts, introduce a 'struct cxl_dpa_partition' array to 'struct cxl_dev_state', and 'struct cxl_range_info' as a shared way for Memory Devices and Accelerators to initialize the DPA information in 'struct cxl_dev_state'. For now, split a new cxl_dpa_setup() from cxl_mem_create_range_info() to get the new data structure initialized, and cleanup some qos_class init. Follow on patches will go further to use the new data structure to cleanup algorithms that are better suited to loop over all possible partitions. cxl_dpa_setup() follows the locking expectations of mutating the device DPA map, and is suitable for Accelerator drivers to use. Accelerators likely only have one hardcoded 'ram' partition to convey to the cxl_core. Link: http://lore.kernel.org/20241230214445.27602-1-alejandro.lucero-palau@amd.com [1] Link: http://lore.kernel.org/20241210-dcd-type2-upstream-v8-0-812852504400@intel.com [2] Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Reviewed-by: Alejandro Lucero <alucerop@amd.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Tested-by: Alejandro Lucero <alucerop@amd.com> Link: https://patch.msgid.link/173864305827.668823.13978794102080021276.stgit@dwillia2-xfh.jf.intel.com Signed-off-by: Dave Jiang <dave.jiang@intel.com>
2025-02-03 20:24:18 -08:00
cxl/mem: Introduce cxl_memdev_attach for CXL-dependent operation Unlike the cxl_pci class driver that opportunistically enables memory expansion with no other dependent functionality, CXL accelerator drivers have distinct PCIe-only and CXL-enhanced operation states. If CXL is available some additional coherent memory/cache operations can be enabled, otherwise traditional DMA+MMIO over PCIe/CXL.io is a fallback. This constitutes a new mode of operation where the caller of devm_cxl_add_memdev() wants to make a "go/no-go" decision about running in CXL accelerated mode or falling back to PCIe-only operation. Part of that decision making process likely also includes additional CXL-acceleration-specific resource setup. Encapsulate both of those requirements into 'struct cxl_memdev_attach' that provides a ->probe() callback. The probe callback runs in cxl_mem_probe() context, after the port topology is successfully attached for the given memdev. It supports a contract where, upon successful return from devm_cxl_add_memdev(), everything needed for CXL accelerated operation has been enabled. Additionally the presence of @cxlmd->attach indicates that the accelerator driver be detached when CXL operation ends. This conceptually makes a CXL link loss event mirror a PCIe link loss event which results in triggering the ->remove() callback of affected devices+drivers. A driver can re-attach to recover back to PCIe-only operation. Live recovery, i.e. without a ->remove()/->probe() cycle, is left as a future consideration. [ dj: Repalce with updated commit log from Dan ] Cc: Smita Koralahalli <Smita.KoralahalliChannabasappa@amd.com> Reviewed-by: Ben Cheatham <benjamin.cheatham@amd.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Tested-by: Alejandro Lucero <alucerop@amd.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Link: https://patch.msgid.link/20251216005616.3090129-7-dan.j.williams@intel.com Signed-off-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Dave Jiang <dave.jiang@intel.com>
2025-12-15 16:56:16 -08:00