Driver uses monitor destination rings for extended statistics mode and
standalone monitor mode. In extended statistics mode, TLVs are parsed from
the buffer received from the monitor destination ring and assigned to the
ppdu_info structure to update per-packet statistics. In standalone monitor
mode, along with per-packet statistics, the packet data (payload) is
captured, and the driver updates per MSDU to mac80211.
When the AP interface is enabled, only extended statistics mode is
activated. As part of enabling monitor rings for collecting statistics,
the driver subscribes to HAL_RX_MPDU_START TLV in the filter
configuration. This TLV is received from the monitor destination ring, and
kzalloc for the mon_mpdu object occurs, which is not freed, leading to a
memory leak. The kzalloc for the mon_mpdu object is only required while
enabling the standalone monitor interface. This causes a memory leak while
enabling extended statistics mode in the driver.
Fix this memory leak by removing the kzalloc for the mon_mpdu object in
the HAL_RX_MPDU_START TLV handling. Additionally, remove the standalone
monitor mode handlings in the HAL_MON_BUF_ADDR and HAL_RX_MSDU_END TLVs.
These TLV tags will be handled properly when enabling standalone monitor
mode in the future.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-13-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Each PPDU contains numerous TLV tags. HAL_RX_PPDU_START marks the start
of the PPDU, and HAL_RX_PPDU_END_STATUS_DONE marks the end. From the
monitor destination rings, the driver retrieves skb containing these
TLV tags and their corresponding data. Sometimes, one PPDU’s information
spreads across multiple skbs. The current parsing logic uses memset on
struct hal_rx_mon_ppdu_info after parsing each skb, leading to information
loss if a PPDU spans multiple skbs. Fix this by setting the
ppdu_continuation flag when the driver fails to get
HAL_RX_PPDU_END_STATUS_DONE, and do memset on struct
hal_rx_mon_ppdu_info only when the ppdu_continuation flag is not set.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-12-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
In the current implementation, when PPDU spans multiple ring descriptors,
leading to inefficient use of the NAPI budget. The budget counter is
decremented for each ring descriptor, causing rapid depletion of the
budget even though the processing of a single PPDU might not be complete.
To address this issue, modify the code to decrement the budget counter
only when the driver receives HAL_MON_END_OF_PPDU as the end reason. This
change ensures that the budget is decremented only once per PPDU,
resulting in more efficient utilization of the NAPI budget and better
handling of monitor destination ring.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-11-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Currently, the monitor destination ring's descriptor includes a 2-bit
field for the end reason. Out of all the end reason values, hardware uses
HAL_MON_FLUSH_DETECTED and HAL_MON_PPDU_TRUNCATED to indicate buffers that
should not be processed due to system level errors. Driver should not
process entries with these end reasons, as they contain junk values.
However, the current code lacks end reason-specific checks for the monitor
destination ring, leading to the processing of invalid buffers.
Fix this by adding checks for these two end reasons during the reaping
phase. Free the skb if either HAL_MON_FLUSH_DETECTED or
HAL_MON_PPDU_TRUNCATED is detected, preventing the driver from processing
invalid entries.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-10-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
When monitor block in Hardware experiences internal backpressure, a ring
entry with the EMPTY_DESC reason is received in the monitor destination
ring descriptor. The cookie field for this corresponding entry is invalid.
Currently driver attempts to process this cookie resulting in invalid
buf_id warning logs flooding the console. To fix this, skip processing the
destination descriptor when the EMPTY_DESCRIPTOR bit is set to true in ring
descriptor, thereby avoiding the processing of junk cookies.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-9-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
End offset for the monitor destination ring descriptor is defined as
16 bits, while the firmware definition specifies only 12 bits.
The remaining bits (bit 12 to bit 15) are reserved and may contain
junk values, leading to invalid information retrieval. Fix this issue
by updating the correct genmask values.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-8-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Currently, ath12k_dp_mon_srng_process uses ath12k_hal_srng_src_get_next_entry
to fetch the next entry from the destination ring. This is incorrect because
ath12k_hal_srng_src_get_next_entry is intended for source rings, not destination
rings. This leads to invalid entry fetches, causing potential data corruption or
crashes due to accessing incorrect memory locations. This happens because the
source ring and destination ring have different handling mechanisms and using
the wrong function results in incorrect pointer arithmetic and ring management.
To fix this issue, replace the call to ath12k_hal_srng_src_get_next_entry with
ath12k_hal_srng_dst_get_next_entry in ath12k_dp_mon_srng_process. This ensures
that the correct function is used for fetching entries from the destination
ring, preventing invalid memory accesses.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-7-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Currently, monitor ring reaping and processing occur in the same loop,
which requires holding ring locks until skb processing is complete.
However, only the ring reaping part requires the ring lock; the skb
processing part does not need it. This approach is problematic because it
unnecessarily extends the duration for which the ring locks are held,
leading to increased contention and potential backpressure issues.
Fix it by holding ring locks only during the reaping phase, as skb
processing does not require them. First, reap the monitor destination ring
with the ring lock and queue the skbs into an skb list. Then, process the
skbs in this list in a separate loop without holding the ring lock.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-6-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
The current implementation processes the monitor destination ring using
two separate functions, ath12k_dp_mon_srng_process() for standalone
monitor mode and ath12k_dp_mon_rx_process_stats() for statistics.
However, both functions contain same code which performs monitor ring
reaping and skb data processing. To eliminate redundancy, remove the
duplicate code and use a single function to handle both the cases.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-5-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
ath12k_mac_op_configure_filter() gets called multiple times during
interface bringup. Applying filter configuration from this function leads
to writing same filter configurations multiple times.
Resolve this issue by relocating the filter configuration to
ath12k_mac_config_mon_status_default(), which is invoked by both
ath12k_mac_op_start() and ath12k_mac_op_stop(). Additionally, set the
rxmon_disable flag to true when called from ath12k_mac_op_stop() to
disable the monitor destination ring.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-4-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Add provision to configure monitor filter for the destination
ring. These filters are used for requesting statistics or monitor mode
through the monitor destination ring.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-3-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Add source buffer ring and destination buffer ring ID for monitor rings.
These IDs are used for ring configuration during initial ring setup.
Since monitor rings are enabled based on the rxdma1_enable flag, enable it
in the hardware param for the QCN9274 version 2 hardware and increase the
destination ring size to handle MSDU data buffers.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Signed-off-by: P Praneesh <quic_ppranees@quicinc.com>
Link: https://patch.msgid.link/20241223060132.3506372-2-quic_ppranees@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Currently in ath12k_dp_rx_process_wbm_err(), when processing packets
received on the WBM error ring, pdev validation is done based upon the
hw_link_id. But hw_link_id corresponds to link id of a given partner pdev
in a MLO hardware group, and is not the correct index to use to lookup a
pdev in an SoC(ab). As a result, pdev validation fails, and the reaped
packets are dropped instead of being processed.
The correct index to use is the pdev_id, which is already derived in the
function. So update the logic to validate the pdev based upon the pdev_id
instead of the hw_link_id. This matches the logic used in other Rx ring
processing functions.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Fixes: 1a73acb5fb ("wifi: ath12k: move to HW link id based receive handling")
Signed-off-by: Rameshkumar Sundaram <quic_ramess@quicinc.com>
Link: https://patch.msgid.link/20250102043048.2596791-1-quic_ramess@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Kmemleak reported this error:
unreferenced object 0xffff1c165cec3060 (size 32):
comm "insmod", pid 560, jiffies 4296964570 (age 235.596s)
backtrace:
[<000000005434db68>] __kmem_cache_alloc_node+0x1f4/0x2c0
[<000000001203b155>] kmalloc_trace+0x40/0x88
[<0000000028adc9c8>] _request_firmware+0xb8/0x608
[<00000000cad1aef7>] firmware_request_nowarn+0x50/0x80
[<000000005011a682>] local_pci_probe+0x48/0xd0
[<00000000077cd295>] pci_device_probe+0xb4/0x200
[<0000000087184c94>] really_probe+0x150/0x2c0
The firmware memory was allocated in ath12k_pci_probe(), but not
freed in ath12k_pci_remove() in case ATH12K_FLAG_QMI_FAIL bit is
set. So call ath12k_fw_unmap() to free the memory.
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.2.0-02280-QCAHMTSWPL_V1.0_V2.0_SILICONZ-1
Signed-off-by: Miaoqing Pan <quic_miaoqing@quicinc.com>
Reviewed-by: Aditya Kumar Singh <aditya.kumar.singh@oss.qualcomm.com>
Link: https://patch.msgid.link/20250123080226.1116479-1-quic_miaoqing@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Currently, when sending the scan channel list command to the firmware, the
maximum Tx power is not encoded in the reg2 member. This omission causes
the firmware to be unaware of the host's maximum Tx power, leading to
incorrect Tx power derivation at firmware level.
To resolve this issue, encode the maximum Tx power in the scan channel list
command before sending it to firmware.
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.3.1-00173-QCAHKSWPL_SILICONZ-1
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Fixes: d889913205 ("wifi: ath12k: driver for Qualcomm Wi-Fi 7 devices")
Signed-off-by: Sathishkumar Muruganandam <quic_murugana@quicinc.com>
Signed-off-by: Aditya Kumar Singh <quic_adisi@quicinc.com>
Tested-by: Nicolas Escande <nico.escande@gmail.com>
Link: https://patch.msgid.link/20250107-add_max_reg_pwr_in_scan_ch_list_cmd-v1-1-70d9963a21e4@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
The firmware memory was allocated in ath11k_pci_probe() or
ath11k_ahb_probe(), but not freed in ath11k_xxx_remove() in case
ATH11K_FLAG_QMI_FAIL bit is set. So call ath11k_fw_destroy() to
free the memory.
Found while fixing the same problem in ath12k:
https://lore.kernel.org/linux-wireless/20240314012746.2729101-1-quic_miaoqing@quicinc.com
Tested-on: WCN6855 hw2.1 PCI WLAN.HSP.1.1-04546-QCAHSPSWPL_V1_V2_SILICONZ_IOE-1
Signed-off-by: Miaoqing Pan <quic_miaoqing@quicinc.com>
Reviewed-by: Aditya Kumar Singh <aditya.kumar.singh@oss.qualcomm.com>
Link: https://patch.msgid.link/20250123084948.1124357-1-quic_miaoqing@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Currently, the signal strength of "iw dev xxx station dump" always show an
invalid value.
This is because signal strength is only set in ath12k_mgmt_rx_event()
function, and not set for received data packet. So, change to get signal
from firmware and report to mac80211.
After that, "iw dev xxx station dump" show the correct signal strength.
Such as:
Station 00:03:7f:12:03:03 (on wlo1)
inactive time: 36 ms
rx bytes: 61571
rx packets: 336
tx bytes: 28204
tx packets: 205
tx retries: 49
tx failed: 0
beacon loss: 0
beacon rx: 83
rx drop misc: 66
signal: -24 dBm
beacon signal avg: -22 dBm
For WCN7850, the firmware supports db2dbm, so not need to add noise floor.
For QCN9274, the firmware not support db2dbm, so need to add noise floor.
This patch affects the station mode of WCN7850 and QCN9274.
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.1.1-00214-QCAHKSWPL_SILICONZ-1
Signed-off-by: Lingbo Kong <quic_lingbok@quicinc.com>
Link: https://patch.msgid.link/20250115063537.35797-4-quic_lingbok@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Currently, the receive rate of EHT of "iw dev xxx station dump" command
always show an invalid value.
This is because ath12k does not pass information about the rx_status of EHT
to mac80211. So, mac80211 not calculate the receive rate.
To address this issue, add logic for handling rx_status of EHT to the
ath12k_dp_rx_h_rate() function.
After that, "iw dev xxx station dump" show the correct receive rate.
Such as:
Station 00:03:7f:12:03:03 (on wlo1)
inactive time: 48 ms
rx bytes: 59226
rx packets: 320
tx bytes: 26556
tx packets: 191
tx retries: 99
tx failed: 0
beacon loss: 0
beacon rx: 79
rx drop misc: 68
signal: -95 dBm
beacon signal avg: -20 dBm
tx bitrate: 688.2 MBit/s 40MHz EHT-MCS 13 EHT-NSS 2 EHT-GI 0
tx duration: 0 us
rx bitrate: 619.5 MBit/s 40MHz EHT-MCS 8 EHT-NSS 3 EHT-GI 0
This patch affects the station mode of WCN7850 and QCN9274.
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.1.1-00214-QCAHKSWPL_SILICONZ-1
Signed-off-by: Lingbo Kong <quic_lingbok@quicinc.com>
Closes: https://bugzilla.kernel.org/show_bug.cgi?id=219715
Link: https://patch.msgid.link/20250115063537.35797-3-quic_lingbok@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Currently, the transmit rate of "iw dev xxx station dump" command
always show an invalid value.
To address this issue, ath12k parse the info of transmit complete
report from firmware and indicate the transmit rate to mac80211.
This patch affects the station mode of WCN7850 and QCN9274.
After that, "iw dev xxx station dump" show the correct transmit rate.
Such as:
Station 00:03:7f:12:03:03 (on wlo1)
inactive time: 872 ms
rx bytes: 219111
rx packets: 1133
tx bytes: 53767
tx packets: 462
tx retries: 51
tx failed: 0
beacon loss: 0
beacon rx: 403
rx drop misc: 74
signal: -95 dBm
beacon signal avg: -18 dBm
tx bitrate: 1441.1 MBit/s 80MHz EHT-MCS 13 EHT-NSS 2 EHT-GI 0
Tested-on: WCN7850 hw2.0 PCI WLAN.HMT.1.0.c5-00481-QCAHMTSWPL_V1.0_V2.0_SILICONZ-3
Tested-on: QCN9274 hw2.0 PCI WLAN.WBE.1.1.1-00214-QCAHKSWPL_SILICONZ-1
Signed-off-by: Lingbo Kong <quic_lingbok@quicinc.com>
Closes: https://bugzilla.kernel.org/show_bug.cgi?id=219715
Link: https://patch.msgid.link/20250115063537.35797-2-quic_lingbok@quicinc.com
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Use the "Q" instead of "R" constraint to correctly reflect the instruction
format of the tm instruction: the first operand is a memory reference
without index register and short displacement. The "R" constraint indicates
a memory reference with index register instead.
This may lead to compile errors like:
arch/s390/include/asm/bitops.h: Assembler messages:
arch/s390/include/asm/bitops.h:60: Error: operand 1: syntax error; missing ')' after base register
arch/s390/include/asm/bitops.h:60: Error: operand 2: syntax error; ')' not allowed here
arch/s390/include/asm/bitops.h:60: Error: junk at end of line: `,4'
Reported-by: Nathan Chancellor <nathan@kernel.org>
Closes: https://lore.kernel.org/r/20250122-s390-fix-std-for-gcc-15-v1-1-8b00cadee083@kernel.org
Fixes: b2bc1b1a77 ("s390/bitops: Provide optimized arch_test_bit()")
Acked-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Since commit 25f39d3dcb ("s390/pci: Ignore RID for isolated VFs") PFs
which are not initially configured but in standby are considered
isolated. That is they create only a single function PCI domain. Due to
the PCI domains being created on discovery, this means that even if they
are configured later on, sibling PFs and their child VFs will not be
added to their PCI domain breaking SR-IOV expectations.
The reason the referenced commit ignored standby PFs for the creation of
multi-function PCI subhierarchies, was to work around a PCI domain
renumbering scenario on reboot. The renumbering would occur after
removing a previously in standby PF, whose domain number is used for its
configured sibling PFs and their child VFs, but which itself remained in
standby. When this is followed by a reboot, the sibling PF is used
instead to determine the PCI domain number of it and its child VFs.
In principle it is not possible to know which standby PFs will be
configured later and which may be removed. The PCI domain and root bus
are pre-requisites for hotplug slots so the decision of which functions
belong to which domain can not be postponed. With the renumbering
occurring only in rare circumstances and being generally benign, accept
it as an oddity and fix SR-IOV for initially standby PFs simply by
allowing them to create PCI domains.
Cc: stable@vger.kernel.org
Reviewed-by: Gerd Bayer <gbayer@linux.ibm.com>
Fixes: 25f39d3dcb ("s390/pci: Ignore RID for isolated VFs")
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Similar to commit eb6efdfeae ("s390/uaccess: add KMSAN support to
put_user() and get_user()") disable KMSAN instrumention for futex inline
assemblies, which contain dereferenced user pointers. With KMSAN
instrumentation this would lead to accesses of shadows for user pointers,
which should not happen.
Handle the futex operations like they copy a value (old) from user
space to kernel space.
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Rename get_put_user_noinstr_attributes to a more generic
uaccess_kmsan_or_inline name. This allows to use it for other non
put_user()/get_user() uaccess functions withour causing confusion.
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Cleanup the futex_atomic_cmpxchg_inatomic() inline assembly to make it
a bit more readable.
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Cleanup the futex atomic op inline assembly and generate a function for
each futex atomic op. This makes the code hopefully a bit more readable.
Acked-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
The s390 implementations of raw_copy_from_user() and raw_copy_to_user() are
never inlined. However INLINE_COPY_FROM_USER and INLINE_COPY_TO_USER are
still set. This leads to the odd situation that only the error handling
(memset to zero of the not copied bytes) of copy_from_user() is inlined,
while the actual fast path code is out-of-line.
This would make sense if raw_copy_from_user() and raw_copy_to_user() were
implemented in assembler files, where inlining is not possible. But the
current s390 setup does not make any sense.
Address this by moving the raw uaccess copy inline assemblies to the
uaccess header file, and remove INLINE_COPY_FROM_USER and
INLINE_COPY_TO_USER definitions. This way the uaccess code, but now
including error handling, is still out-of-line with the common code
_copy_from_user() and _copy_to_user() variants, which inline the raw
uaccess functions via _inline_copy_from_user() and _inline_copy_to_user().
This reduces the size of the kernel image by ~17kb.
(defconfig, gcc 14.2.0)
Acked-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Use asm goto if available for put_user() and get_user().
This generates slightly better code.
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Remove usage of the operand access control specifier for put_user()
and get_user() (again). Instead hardcode the specifier for both inline
assemblies. This saves one instruction.
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Remove EX_TABLE_UA_LOAD_MEM exception handling and replace it with
EX_TABLE_UA_FAULT. Open code the return code check, and also open code
setting of the destination to zero in case of an error. In almost all cases
the compiler is able to optimize the open coded checks away, since all
users of get_users() must check the return code, and are not supposed to
use the result in case of an error.
In addition this allows to change the get_user() inline assembly so that
the "Q" constraint can be used for the destination, instead of only an "a"
constraint. This generates slightly better code.
Acked-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Remove superfluous underscores, brackets, and early clobber to make
the code a bit more readable.
Acked-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
The __put_user_fn() and __get_user_fn() wrappers are leftovers from the
time where the kernel was compiled with or without mvcos support plus they
were later used to workaround the problems that came with asm register
constructs.
Both reasons do not exist anymore, therefore remove the wrappers and
shorten the code.
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Use asm goto for __mvc_kernel_nofault() if available. This generates
slightly better code, since the error checking happens implicitly with
the goto (aka exception) and the good path comes without any checks and
branches.
Acked-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Use the mvc instruction in order to implement __get_kernel_nofault() and
__put_kernel_nofault(). Both functions have a source and destination
address where the code is supposed to read from and write to. Use the mvc
instruction to copy from source to destination instead of lg/stg like
instructions. This generates slightly better code.
Acked-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Rename EX_TABLE_UA_STORE to a more generic EX_TABLE_UA_FAULT
name. This allows to use the extable type also for uaccess inline
assemblies which read from userspace, without causing confusion.
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Use the more precise CONFIG_CC_HAS_ASM_AOR_FORMAT_FLAGS to tell if the
compiler has support for the A, O, and R inline assembly format flags.
This allows recent Clang compilers to generate better code.
Move code around so the good (aka better) case at the top of each ifdef
construct.
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Introduce CC_HAS_ASM_AOR_FORMAT_FLAGS Kconfig option. Use this option for
inline assemblies where the A, O, or R format flags are used.
Those flags are not available for Clang versions before 19.1.0.
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Config options which can be used to check for compiler bugs and features
have the compiler independent CC prefix in order to avoid duplicating and
having to check config options for multiple compilers. Therefore rename the
config option accordingly.
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Use fpc_sfpc() instead of open-coding.
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
The fixup section was added again by mistake when test_fp_ctl() was
removed. The reason for the removal of the fixup section is described in
commit 484a8ed8b7 ("s390/extable: add dedicated uaccess handler").
Remove it again for the same reason.
Add an exception handler which handles exceptions when the floating point
control register is attempted to be set to invalid values. The exception
handler sets the floating point control register to zero and continues
execution at the specified address.
The new sfpc inline assembly is open-coded to make back porting a bit
easier.
Fixes: 702644249d ("s390/fpu: get rid of test_fp_ctl()")
Cc: stable@vger.kernel.org
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Use a zero identity base when CONFIG_RANDOMIZE_IDENTITY_BASE is off,
slightly optimizing __pa/__va calculations.
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Acked-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Currently, decompression error messages can be very uninformative:
[ 0.029853] startup: read error
[ 0.040507] startup: -- System halted
Improve these messages to make it clear that the error originates from
the decompression code. Additionally, on decompression failures, if
bootdebug is enabled, dump the message ring buffer before halting. This
provides more context for diagnosing startup issues.
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Add boot_debug() calls to log various memory layout decisions and
randomization details during early startup, improving debugging
capabilities.
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Reorder the store_ipl_parmblock(), uv_query_info(), and command line
setup calls to occur earlier. This ensures debug printing covers all
memory tracking activities from the start.
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Introduce boot_debug() calls to track memory detection, online ranges,
reserved areas, and allocations (except for VMEM allocations, which are
too frequent). Instead introduce dump_physmem_reserved() function which
prints out full memory tracking information. This helps in debugging
early boot memory handling.
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
