Commit graph

1,215,304 commits

Author SHA1 Message Date
Peng Fan
927b7d15dc genpd: imx: scu-pd: enlarge PD range
There are 5 LPI2C, 5 LPUART and 32 DMA0 Channel resources per imx_rsrc.h,
and they are in i.MX8QM, so enlarge the PD range for them.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
2023-08-17 11:33:54 +02:00
Peng Fan
b683a36207 genpd: imx: relocate scu-pd under genpd
Move scu-pd driver under genpd directory where the driver
should be.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
2023-08-17 11:33:54 +02:00
Rafael J. Wysocki
5484e31bbb cpuidle: menu: Skip tick_nohz_get_sleep_length() call in some cases
Because the cost of calling tick_nohz_get_sleep_length() may increase
in the future, reorder the code in menu_select() so it first uses the
statistics to determine the expected idle duration.  If that value is
higher than RESIDENCY_THRESHOLD_NS, tick_nohz_get_sleep_length() will
be called to obtain the time till the closest timer and refine the
idle duration prediction if necessary.

This causes the governor to always take the full overhead of
get_typical_interval() with the assumption that the cost will be
amortized by skipping the tick_nohz_get_sleep_length() call in the
cases when the predicted idle duration is relatively very small.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Tested-by: Doug Smythies <dsmythies@telus.net>
2023-08-17 11:28:38 +02:00
Rafael J. Wysocki
4ab4b3b11d ACPI: thermal: Eliminate code duplication from acpi_thermal_notify()
Move the acpi_bus_generate_netlink_event() invocation into
acpi_thermal_trips_update() which allows the code duplication in
acpi_thermal_notify() to be cleaned up, but for this purpose the
event value needs to be passed to acpi_thermal_trips_update() and
from there to acpi_thermal_adjust_thermal_zone() which has to
determine the flag value for __acpi_thermal_trips_update() by
itself.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-08-17 11:25:29 +02:00
Rafael J. Wysocki
2a74c4aca1 ACPI: thermal: Drop unnecessary thermal zone callbacks
Drop the .get_trip_type(), .get_trip_temp() and .get_crit_temp() thermal
zone callbacks that are not necessary any more from the ACPI thermal
driver along with the corresponding callback functions.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-08-17 11:25:22 +02:00
Rafael J. Wysocki
9caaad2ca0 ACPI: thermal: Rework thermal_get_trend()
Rework the ACPI thermal driver's .get_trend() callback function,
thermal_get_trend(), so that it does not call thermal_get_trip_type()
and thermal_get_trip_temp() which are going to be dropped.

This reduces the overhead of the function too, because it will always
carry out a trip point lookup once after the change.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-08-17 11:25:22 +02:00
Rafael J. Wysocki
ec23c1c462 ACPI: thermal: Use trip point table to register thermal zones
Make the ACPI thermal driver use thermal_zone_device_register_with_trips()
to register its thermal zones.

For this purpose, make it create a trip point table that will be passed to
thermal_zone_device_register_with_trips() as an argument.

Also use the thermal_zone_update_trip_temp() helper introduced
previously to update temperatures of the passive and active trip
points after a trip points change notification from the platform
firmware.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-08-17 11:25:22 +02:00
Rafael J. Wysocki
96b8b4365d thermal: core: Rework and rename __for_each_thermal_trip()
Rework the currently unused __for_each_thermal_trip() to pass original
pointers to struct thermal_trip objects to the callback, so it can be
used for updating trip data (e.g. temperatures), rename it to
for_each_thermal_trip() and make it available to modular drivers.

Suggested-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-08-17 11:25:22 +02:00
Rafael J. Wysocki
68b77785a1 ACPI: thermal: Introduce struct acpi_thermal_trip
Add struct acpi_thermal_trip to contain the temperature and valid flag
of each trip point in the driver's local data structures.

This helps to make the subsequent changes more straightforward.

No intentional functional impact.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-08-17 11:25:22 +02:00
Rafael J. Wysocki
68f4f0378c ACPI: thermal: Carry out trip point updates under zone lock
There is a race condition between acpi_thermal_trips_update() and
acpi_thermal_check_fn(), because the trip points may get updated while
the latter is running which in theory may lead to inconsistent results.
For example, if two trips are updated together, using the temperature
value of one of them from before the update and the temperature value
of the other one from after the update may not lead to the expected
outcome.

Moreover, if thermal_get_trend() runs when a trip points update is in
progress, it may end up using stale trip point temperatures.

To address this, make acpi_thermal_trips_update() call
thermal_zone_device_exec() to carry out the trip points update and
use a new  acpi_thermal_adjust_thermal_zone() wrapper around
__acpi_thermal_trips_update() as the callback function for the latter.

While at it, change the acpi_thermal_trips_update() return data type
to void as that function always returns 0 anyway.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-08-17 11:24:27 +02:00
Rafael J. Wysocki
bf07b4a3ba ACPI: thermal: Clean up acpi_thermal_register_thermal_zone()
Rename the trips variable in acpi_thermal_register_thermal_zone() to
trip_count so its name better reflects the purpose, rearrange white
space in the loop over active trips for clarity and reduce code
duplication related to calling thermal_zone_device_register() by
using an extra local variable to store the passive delay value.

No intentional functional impact.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-08-17 11:24:18 +02:00
Rafael J. Wysocki
cba440fab3 thermal: core: Add priv pointer to struct thermal_trip
Add a new field called priv to struct thermal_trip to allow thermal
drivers to store pointers to their local data associated with trip
points.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2023-08-17 11:24:18 +02:00
Randy Dunlap
60046980bf KVM: arm64: nv: Select XARRAY_MULTI to fix build error
populate_nv_trap_config() uses xa_store_range(), which is only built
when XARRAY_MULTI is set, so select that symbol to prevent the build error.

aarch64-linux-ld: arch/arm64/kvm/emulate-nested.o: in function `populate_nv_trap_config':
emulate-nested.c:(.init.text+0x17c): undefined reference to `xa_store_range'

Fixes: e58ec47bf6 ("KVM: arm64: nv: Add trap forwarding infrastructure")
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Oliver Upton <oliver.upton@linux.dev>
Cc: kvmarm@lists.linux.dev
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230816210949.17117-1-rdunlap@infradead.org
2023-08-17 10:24:00 +01:00
Rafael J. Wysocki
9a99a996d1 thermal: core: Introduce thermal_zone_device_exec()
Introduce a new helper function, thermal_zone_device_exec(), that can
be used by drivers to run a given callback routine under the zone lock.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2023-08-17 11:23:32 +02:00
Marc Zyngier
03fb54d0aa KVM: arm64: nv: Add support for HCRX_EL2
HCRX_EL2 has an interesting effect on HFGITR_EL2, as it conditions
the traps of TLBI*nXS.

Expand the FGT support to add a new Fine Grained Filter that will
get checked when the instruction gets trapped, allowing the shadow
register to override the trap as needed.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-29-maz@kernel.org
2023-08-17 10:00:28 +01:00
Marc Zyngier
a63cf31139 KVM: arm64: Move HCRX_EL2 switch to load/put on VHE systems
Although the nVHE behaviour requires HCRX_EL2 to be switched
on each switch between host and guest, there is nothing in
this register that would affect a VHE host.

It is thus possible to save/restore this register on load/put
on VHE systems, avoiding unnecessary sysreg access on the hot
path. Additionally, it avoids unnecessary traps when running
with NV.

To achieve this, simply move the read/writes to the *_common()
helpers, which are called on load/put on VHE, and more eagerly
on nVHE.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-28-maz@kernel.org
2023-08-17 10:00:28 +01:00
Marc Zyngier
0a5d28433a KVM: arm64: nv: Expose FGT to nested guests
Now that we have FGT support, expose the feature to NV guests.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-27-maz@kernel.org
2023-08-17 10:00:28 +01:00
Marc Zyngier
d4d2dacc7c KVM: arm64: nv: Add switching support for HFGxTR/HDFGxTR
Now that we can evaluate the FGT registers, allow them to be merged
with the hypervisor's own configuration (in the case of HFG{RW}TR_EL2)
or simply set for HFGITR_EL2, HDGFRTR_EL2 and HDFGWTR_EL2.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-26-maz@kernel.org
2023-08-17 10:00:28 +01:00
Marc Zyngier
ea3b27d8de KVM: arm64: nv: Expand ERET trap forwarding to handle FGT
We already handle ERET being trapped from a L1 guest in hyp context.
However, with FGT, we can also have ERET being trapped from L2, and
this needs to be reinjected into L1.

Add the required exception routing.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-25-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
a77b31dce4 KVM: arm64: nv: Add SVC trap forwarding
HFGITR_EL2 allows the trap of SVC instructions to EL2. Allow these
traps to be forwarded. Take this opportunity to deny any 32bit activity
when NV is enabled.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-24-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
d0be0b2ede KVM: arm64: nv: Add trap forwarding for HDFGxTR_EL2
... and finally, the Debug version of FGT, with its *enormous*
list of trapped registers.

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-23-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
039f9f12de KVM: arm64: nv: Add trap forwarding for HFGITR_EL2
Similarly, implement the trap forwarding for instructions affected
by HFGITR_EL2.

Note that the TLBI*nXS instructions should be affected by HCRX_EL2,
which will be dealt with down the line. Also, ERET* and SVC traps
are handled separately.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-22-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
5a24ea7869 KVM: arm64: nv: Add trap forwarding for HFGxTR_EL2
Implement the trap forwarding for traps described by HFGxTR_EL2,
reusing the Fine Grained Traps infrastructure previously implemented.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-21-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
15b4d82d69 KVM: arm64: nv: Add fine grained trap forwarding infrastructure
Fine Grained Traps are fun. Not.

Implement the fine grained trap forwarding, reusing the Coarse Grained
Traps infrastructure previously implemented.

Each sysreg/instruction inserted in the xarray gets a FGT group
(vaguely equivalent to a register number), a bit number in that register,
and a polarity.

It is then pretty easy to check the FGT state at handling time, just
like we do for the coarse version (it is just faster).

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-20-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
e880bd3363 KVM: arm64: nv: Add trap forwarding for CNTHCTL_EL2
Describe the CNTHCTL_EL2 register, and associate it with all the sysregs
it allows to trap.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-19-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
cb31632c44 KVM: arm64: nv: Add trap forwarding for MDCR_EL2
Describe the MDCR_EL2 register, and associate it with all the sysregs
it allows to trap.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-18-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
a0b70fb00d KVM: arm64: nv: Expose FEAT_EVT to nested guests
Now that we properly implement FEAT_EVT (as we correctly forward
traps), expose it to guests.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-17-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
d0fc0a2519 KVM: arm64: nv: Add trap forwarding for HCR_EL2
Describe the HCR_EL2 register, and associate it with all the sysregs
it allows to trap.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-16-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
e58ec47bf6 KVM: arm64: nv: Add trap forwarding infrastructure
A significant part of what a NV hypervisor needs to do is to decide
whether a trap from a L2+ guest has to be forwarded to a L1 guest
or handled locally. This is done by checking for the trap bits that
the guest hypervisor has set and acting accordingly, as described by
the architecture.

A previous approach was to sprinkle a bunch of checks in all the
system register accessors, but this is pretty error prone and doesn't
help getting an overview of what is happening.

Instead, implement a set of global tables that describe a trap bit,
combinations of trap bits, behaviours on trap, and what bits must
be evaluated on a system register trap.

Although this is painful to describe, this allows to specify each
and every control bit in a static manner. To make it efficient,
the table is inserted in an xarray that is global to the system,
and checked each time we trap a system register while running
a L2 guest.

Add the basic infrastructure for now, while additional patches will
implement configuration registers.

Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Link: https://lore.kernel.org/r/20230815183903.2735724-15-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
e930694e61 KVM: arm64: Restructure FGT register switching
As we're about to majorly extend the handling of FGT registers,
restructure the code to actually save/restore the registers
as required. This is made easy thanks to the previous addition
of the EL2 registers, allowing us to use the host context for
this purpose.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Oliver Upton <oliver.upton@linux.dev>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-14-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
50d2fe4648 KVM: arm64: nv: Add FGT registers
Add the 5 registers covering FEAT_FGT. The AMU-related registers
are currently left out as we don't have a plan for them. Yet.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-13-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
3ea84b4fe4 KVM: arm64: Add missing HCR_EL2 trap bits
We're still missing a handfull of HCR_EL2 trap bits. Add them.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-12-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
484f86824a KVM: arm64: Correctly handle ACCDATA_EL1 traps
As we blindly reset some HFGxTR_EL2 bits to 0, we also randomly trap
unsuspecting sysregs that have their trap bits with a negative
polarity.

ACCDATA_EL1 is one such register that can be accessed by the guest,
causing a splat on the host as we don't have a proper handler for
it.

Adding such handler addresses the issue, though there are a number
of other registers missing as the current architecture documentation
doesn't describe them yet.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-11-maz@kernel.org
2023-08-17 10:00:27 +01:00
Mark Brown
b206a708cb arm64: Add feature detection for fine grained traps
In order to allow us to have shared code for managing fine grained traps
for KVM guests add it as a detected feature rather than relying on it
being a dependency of other features.

Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Mark Brown <broonie@kernel.org>
[maz: converted to ARM64_CPUID_FIELDS()]
Link: https://lore.kernel.org/r/20230301-kvm-arm64-fgt-v4-1-1bf8d235ac1f@kernel.org
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-10-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
cc24f656f7 arm64: Add HDFGRTR_EL2 and HDFGWTR_EL2 layouts
As we're about to implement full support for FEAT_FGT, add the
full HDFGRTR_EL2 and HDFGWTR_EL2 layouts.

Reviewed-by: Mark Brown <broonie@kernel.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-9-maz@kernel.org
2023-08-17 10:00:27 +01:00
Marc Zyngier
2b062ed483 arm64: Add missing BRB/CFP/DVP/CPP instructions
HFGITR_EL2 traps a bunch of instructions for which we don't have
encodings yet. Add them.

Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-8-maz@kernel.org
2023-08-17 10:00:26 +01:00
Marc Zyngier
57596c8f99 arm64: Add debug registers affected by HDFGxTR_EL2
The HDFGxTR_EL2 registers trap a (huge) set of debug and trace
related registers. Add their encodings (and only that, because
we really don't care about what these registers actually do at
this stage).

Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-7-maz@kernel.org
2023-08-17 10:00:26 +01:00
Marc Zyngier
2b97411fef arm64: Add AT operation encodings
Add the encodings for the AT operation that are usable from NS.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-6-maz@kernel.org
2023-08-17 10:00:26 +01:00
Marc Zyngier
fb1926cccd arm64: Add TLBI operation encodings
Add all the TLBI encodings that are usable from Non-Secure.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-5-maz@kernel.org
2023-08-17 10:00:26 +01:00
Marc Zyngier
6ddea24dfd arm64: Add missing DC ZVA/GVA/GZVA encodings
Add the missing DC *VA encodings.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-4-maz@kernel.org
2023-08-17 10:00:26 +01:00
Marc Zyngier
464f2164da arm64: Add missing ERX*_EL1 encodings
We only describe a few of the ERX*_EL1 registers. Add the missing
ones (ERXPFGF_EL1, ERXPFGCTL_EL1, ERXPFGCDN_EL1, ERXMISC2_EL1 and
ERXMISC3_EL1).

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-3-maz@kernel.org
2023-08-17 10:00:26 +01:00
Marc Zyngier
21f74a5137 arm64: Add missing VA CMO encodings
Add the missing VA-based CMOs encodings.

Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Miguel Luis <miguel.luis@oracle.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Zenghui Yu <yuzenghui@huawei.com>
Reviewed-by: Jing Zhang <jingzhangos@google.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230815183903.2735724-2-maz@kernel.org
2023-08-17 10:00:26 +01:00
Liao Chang
e613d8cff5 cpufreq: cppc: Set fie_disabled to FIE_DISABLED if fails to create kworker_fie
The function cppc_freq_invariance_init() may failed to create
kworker_fie, make it more robust by setting fie_disabled to FIE_DISBALED
to prevent an invalid pointer dereference in kthread_destroy_worker(),
which called from cppc_freq_invariance_exit().

Signed-off-by: Liao Chang <liaochang1@huawei.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
2023-08-17 14:24:58 +05:30
Liao Chang
6a4fec4f6d cpufreq: cppc: cppc_cpufreq_get_rate() returns zero in all error cases.
The cpufreq framework used to use the zero of return value to reflect
the cppc_cpufreq_get_rate() had failed to get current frequecy and treat
all positive integer to be succeed. Since cppc_get_perf_ctrs() returns a
negative integer in error case, so it is better to convert the value to
zero as the return value of cppc_cpufreq_get_rate().

Signed-off-by: Liao Chang <liaochang1@huawei.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
2023-08-17 14:12:11 +05:30
Liao Chang
ba6ea77d0e cpufreq: Prefer to print cpuid in MIN/MAX QoS register error message
When a cpufreq_policy is allocated, the cpus, related_cpus and real_cpus
of policy are still unset. Therefore, it is preferable to print the
passed 'cpu' parameter instead of a empty 'cpus' cpumask in error
message when registering MIN/MAX QoS notifier fails.

Signed-off-by: Liao Chang <liaochang1@huawei.com>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
2023-08-17 14:12:11 +05:30
Raghavendra Rao Ananta
7657ea920c KVM: arm64: Use TLBI range-based instructions for unmap
The current implementation of the stage-2 unmap walker traverses
the given range and, as a part of break-before-make, performs
TLB invalidations with a DSB for every PTE. A multitude of this
combination could cause a performance bottleneck on some systems.

Hence, if the system supports FEAT_TLBIRANGE, defer the TLB
invalidations until the entire walk is finished, and then
use range-based instructions to invalidate the TLBs in one go.
Condition deferred TLB invalidation on the system supporting FWB,
as the optimization is entirely pointless when the unmap walker
needs to perform CMOs.

Rename stage2_put_pte() to stage2_unmap_put_pte() as the function
now serves the stage-2 unmap walker specifically, rather than
acting generic.

Signed-off-by: Raghavendra Rao Ananta <rananta@google.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230811045127.3308641-15-rananta@google.com
2023-08-17 09:40:35 +01:00
Raghavendra Rao Ananta
defc8cc7ab KVM: arm64: Invalidate the table entries upon a range
Currently, during the operations such as a hugepage collapse,
KVM would flush the entire VM's context using 'vmalls12e1is'
TLBI operation. Specifically, if the VM is faulting on many
hugepages (say after dirty-logging), it creates a performance
penalty for the guest whose pages have already been faulted
earlier as they would have to refill their TLBs again.

Instead, leverage kvm_tlb_flush_vmid_range() for table entries.
If the system supports it, only the required range will be
flushed. Else, it'll fallback to the previous mechanism.

Signed-off-by: Raghavendra Rao Ananta <rananta@google.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230811045127.3308641-14-rananta@google.com
2023-08-17 09:40:35 +01:00
Raghavendra Rao Ananta
3756b6f2bb KVM: arm64: Flush only the memslot after write-protect
After write-protecting the region, currently KVM invalidates
the entire TLB entries using kvm_flush_remote_tlbs(). Instead,
scope the invalidation only to the targeted memslot. If
supported, the architecture would use the range-based TLBI
instructions to flush the memslot or else fallback to flushing
all of the TLBs.

Signed-off-by: Raghavendra Rao Ananta <rananta@google.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230811045127.3308641-13-rananta@google.com
2023-08-17 09:40:35 +01:00
Raghavendra Rao Ananta
c42b6f0b1c KVM: arm64: Implement kvm_arch_flush_remote_tlbs_range()
Implement kvm_arch_flush_remote_tlbs_range() for arm64
to invalidate the given range in the TLB.

Signed-off-by: Raghavendra Rao Ananta <rananta@google.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230811045127.3308641-12-rananta@google.com
2023-08-17 09:40:35 +01:00
Raghavendra Rao Ananta
117940aa6e KVM: arm64: Define kvm_tlb_flush_vmid_range()
Implement the helper kvm_tlb_flush_vmid_range() that acts
as a wrapper for range-based TLB invalidations. For the
given VMID, use the range-based TLBI instructions to do
the job or fallback to invalidating all the TLB entries.

Signed-off-by: Raghavendra Rao Ananta <rananta@google.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20230811045127.3308641-11-rananta@google.com
2023-08-17 09:40:35 +01:00