Commit graph

1,031,458 commits

Author SHA1 Message Date
Aaron Liu
9c6c48e623 drm/amdgpu: add GFX Clock Gating support for yellow carp
Add below supports:
GFX Coarse Grain Clock Gating(CGCG)
GFX Coarse grain light sleep/deep sleep(CGLS)
GFX Medium Grain Clock Gating(MGCG)
GFX Medium Grain light sleep/deep sleep(MGLS)
GFX Fine Grain Clock Gating(FGCG)
RLC MGLS
CP  MGLS

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:14 -04:00
Mykola Kostenok
701b54bcb7 platform/mellanox: mlxreg-hotplug: Revert "move to use request_irq by IRQF_NO_AUTOEN flag"
It causes mlxreg-hotplug probing failure: request_threaded_irq()
 returns -EINVAL due to true value of condition:
((irqflags & IRQF_SHARED) && (irqflags & IRQF_NO_AUTOEN))
after flag "IRQF_NO_AUTOEN" has been added to:
	err = devm_request_irq(&pdev->dev, priv->irq,
			       mlxreg_hotplug_irq_handler, IRQF_TRIGGER_FALLING
			       | IRQF_SHARED | IRQF_NO_AUTOEN,
			       "mlxreg-hotplug", priv);

This reverts commit bee3ecfed0 ("platform/mellanox: mlxreg-hotplug: move
to use request_irq by IRQF_NO_AUTOEN flag").

Signed-off-by: Mykola Kostenok <c_mykolak@nvidia.com>
Acked-by: Vadim Pasternak <vadimp@nvidia.com>
Link: https://lore.kernel.org/r/20210603172827.2599908-1-c_mykolak@nvidia.com
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
2021-06-04 22:03:13 +02:00
Aaron Liu
e44510e24e drm/amdgpu/pm: add set_driver_table_location implementation for yellow carp
This patch adds set_driver_table_location implementation for yellow
carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:13 -04:00
Aaron Liu
94adc46fb0 drm/amdgpu/pm: set_pp_feature is unsupport for yellow carp
For yellow carp, SMU firmware just only supports get_pp_feature.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:13 -04:00
Aaron Liu
903bb18bcd drm/amdgpu: enable psp_v13 for yellow carp
This patch enables psp_v13 for yellow carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:13 -04:00
Aaron Liu
04a69d20a0 drm/amdgpu: add psp_v13 support for yellow carp
This patch adds psp_v13 support for yellow carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:12 -04:00
Alex Deucher
1b3869386e drm/amdgpu: add mmhub client support for yellow carp
To help debugging GPUVM page faults.

Acked-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:12 -04:00
Aaron Liu
bea7534994 drm/amdgpu: reserved buffer is not needed with ip discovery enabled
When IP discovery enabled, the reserved buffer has been alloacted.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:12 -04:00
Huang Rui
e15a5fb9b6 drm/amdgpu: introduce a stolen reserved buffer to protect specific buffer region (v2)
Some ASICs such as Yellow Carp needs to reserve a region of video memory
to avoid access from driver. So this patch is to introduce a stolen
reserved buffer to protect specific buffer region.

v2: free this buffer in amdgpu_ttm_fini.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-and-Tested-by: Aaron Liu <aaron.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:12 -04:00
Aaron Liu
cba00ce82d drm/amdgpu: add gfx golden settings for yellow carp (v3)
This patch is to add gfx golden settings for yellow carp post si.

v2: squash in updates (Alex)
v3: squash in LDS update (Alex)

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:11 -04:00
Aaron Liu
120a6db472 drm/amdgpu: add smu ip block for yellow carp(V3)
Yellow carp smu ip version: 13_0_1.
V2: rename smu_v13_0 to smu_v13_0_1.
V3: reuse smu_v13_0 with aldebaran.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:11 -04:00
Aaron Liu
b4bc9f10c7 drm/amd/pm: partially enable swsmu for yellow carp(V2)
This patch is to partially enable swSMU for yellow carp for the moment.
V2: rename smu_v13_0 to smu_v13_0_1.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:11 -04:00
Aaron Liu
f50740be5e drm/amd/pm: add yellow_carp_ppt implementation(V3)
yellow_carp_ppt is swsmu layer 2 code for yellow carp.
V2: rename smu_v13_0 to smu_v13_0_1
V3: cleanup code.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:10 -04:00
Aaron Liu
20761d0a79 drm/amd/pm: add smu13 ip support for moment(V3)
For supporting yellow carp, we need to add smu13 ip
support for the moment.

V2: add smu_v13_0_1.c|h dedicated for apu.
V3: cleanup code.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:10 -04:00
Aaron Liu
e1aeab8a21 drm/amdgpu/pm: add smu v13.0.1 smc header for yellow carp (v2)
This patch is to add smu v13.0.1 smc header for yellow carp.

v2: squash in updates (Alex)

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:10 -04:00
Aaron Liu
52dfd6c47e drm/amdgpu/pm: add smu v13.0.1 firmware header for yellow carp (V4)
This patch is to add smu v13.0.1 firmware header for yellow carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:09 -04:00
Aaron Liu
385bb92fdc drm/admgpu/pm: add smu v13 driver interface header for yellow carp (v3)
This patch is to add smu v13 driver interface header for yellow carp.

v2: squash in updates (Alex)
v3: squash in v69.29.0 update (Alex)

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:09 -04:00
Aaron Liu
011b514fd8 drm/amdgpu: support nbio_7_2_1 for yellow carp
This patch adds nbio_7_2_1 support yellow carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:09 -04:00
Aaron Liu
bf9d4e88c2 drm/amdkfd: add yellow carp KFD support
This patch is to add GFX10 based Yellow Carp KFD support.
We will bypass IOMMU v2.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:09 -04:00
Aaron Liu
5c462ca9a0 drm/amdgpu: set ip blocks for yellow carp
Enable ip blocks for yellow carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:08 -04:00
Aaron Liu
e88d68e106 drm/amdgpu: add sdma support for yellow carp
This patch adds the sdma v5.2 support for yellow carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:08 -04:00
Aaron Liu
bbbdc9739e drm/amdgpu: add gfx support for yellow carp
Add yellow carp checks to gfx10 code.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:08 -04:00
Aaron Liu
531d6e5de8 drm/amdgpu: support fw load type for yellow carp
This patch sets fw load type as direct with fw_load_type=0 for yellow carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:07 -04:00
Aaron Liu
c817cfa313 drm/amdgpu: add gmc v10 supports for yellow carp
Add gfx memory controller support for yellow carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:07 -04:00
Aaron Liu
f82e7e49a6 drm/amdgpu: add yellow carp support for ih block
This patch adds the support for yellow carp ih block.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:07 -04:00
Aaron Liu
e79907216b drm/amdgpu: add nv common ip block support for yellow carp
This patch adds common ip support for yellow carp.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:06 -04:00
Alex Deucher
cdf9979be9 drm/amdgpu: add yellow_carp_reg_base_init function for yellow carp (v2)
This patch adds yellow_carp_reg_base_init function to init the register
base for yellow carp.

v2: squash in updates (Alex)

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:06 -04:00
Aaron Liu
8bf84f60c5 drm/amdgpu: add yellow carp support for gpu_info and ip block setting
This patch adds yellow carp support for gpu_info firmware and ip
block setting.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:06 -04:00
Aaron Liu
90a187d26f drm/amdgpu: add uapi to define yellow carp series
Add a flag to define yellow carp series.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:05 -04:00
Aaron Liu
ee9236b78b drm/amdgpu: add yellow carp asic_type enum
This patch adds yellow carp to amd_asic_type enum and amdgpu_asic_name[].

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:05 -04:00
Aaron Liu
02680c23d7 drm/amdgpu: add yellow carp asic header files (v3)
This patch is to add yellow carp asic header files.

v2: squash in updates (Alex)
v3: squash in DCN updates (Alex)

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:05 -04:00
Chen Li
65e06b7873 radeon: use memcpy_to/fromio for UVD fw upload
I met a gpu addr bug recently and the kernel log
tells me the pc is memcpy/memset and link register is
radeon_uvd_resume.

As we know, in some architectures, optimized memcpy/memset
may not work well on device memory. Trival memcpy_toio/memset_io
can fix this problem.

BTW, amdgpu has already done it in:
commit ba0b2275a6 ("drm/amdgpu: use memcpy_to/fromio for UVD fw upload"),
that's why it has no this issue on the same gpu and platform.

Signed-off-by: Chen Li <chenli@uniontech.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:04 -04:00
Chen Li
c0f2b640e5 radeon: fix coding issues reported from sparse
Also fix some coding issues reported from sparse.

Signed-off-by: Chen Li <chenli@uniontech.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:04 -04:00
Wan Jiabing
48b033098e drm: amdgpu: Remove unneeded semicolon in amdgpu_vm.c
Fix following coccicheck warning:
./drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c:1726:2-3: Unneeded semicolon

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Wan Jiabing <wanjiabing@vivo.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:04 -04:00
Gustavo A. R. Silva
fe9bb984b5 drm/amd/pm: Fix fall-through warning for Clang
In preparation to enable -Wimplicit-fallthrough for Clang, fix a warning
by explicitly adding a break statement instead of letting the code fall
through to the next case.

Link: https://github.com/KSPP/linux/issues/115
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:03:04 -04:00
Rohit Khaire
46ed43e67d drm/amdgpu: Modify GC register access to use _SOC15 macros
In SRIOV environment, KMD should access GC registers
with RLCG if GC indirect access flag enabled.

Using _SOC15 read/write macros ensures that they go
through RLC when flag is enabled.

Signed-off-by: Rohit Khaire <rohit.khaire@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:02:57 -04:00
Rohit Khaire
cec7e80fbf drm/amdgpu: Enable RLCG read/write interface for Sienna Cichlid
Enable this only for Sienna Cichild
since only Navi12 and Sienna Cichlid support SRIOV

Signed-off-by: Rohit Khaire <rohit.khaire@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:02:50 -04:00
Rohit Khaire
18703923a6 drm/amdgpu: Fix incorrect register offsets for Sienna Cichlid
RLC_CP_SCHEDULERS and RLC_SPARE_INT0 have different
offsets for Sienna Cichlid

Signed-off-by: Rohit Khaire <rohit.khaire@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:02:44 -04:00
Eric Huang
810085ddb7 drm/amdgpu: Don't flush/invalidate HDP for APUs and A+A
Integrate two generic functions to determine if HDP
flush is needed for all Asics.

Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:02:38 -04:00
Lijo Lazar
9fd4781b58 drm/amd/pm: Use generic BACO function for smu11 ASICs
Remove ASIC specific functions for BACO support check. Use generic smu11
function instead.

Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Evan Quan <evan.quan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:02:33 -04:00
Lijo Lazar
52a9fd7bc0 drm/amd/pm: Add VF check to BACO support check
For smuv11, check for VF also during BACO check.

Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Evan Quan <evan.quan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:02:27 -04:00
Lijo Lazar
458020dd4f drm/amd/pm: Read BIF STRAP also for BACO check
Avoid reading BIF STRAP each time for BACO capability. Read the STRAP
value while checking BACO capability in PPTable.

Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Evan Quan <evan.quan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:02:22 -04:00
Lijo Lazar
ff05bb18e1 drm/amd/pm: Remove BACO check for aldebaran
BACO/MACO is not applicable for aldebaran. Remove the redundant check.

Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2021-06-04 16:02:13 -04:00
Maximilian Luz
6325ce1542 platform/surface: dtx: Add missing mutex_destroy() call in failure path
When we fail to open the device file due to DTX being shut down, the
mutex is initialized but never destroyed. We are destroying it when
releasing the file, so add the missing call in the failure path as well.

Fixes: 1d60999283 ("platform/surface: Add DTX driver")
Signed-off-by: Maximilian Luz <luzmaximilian@gmail.com>
Link: https://lore.kernel.org/r/20210604132540.533036-1-luzmaximilian@gmail.com
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
2021-06-04 22:00:27 +02:00
Stephen Boyd
51c8b49762 clk: versatile: Depend on HAS_IOMEM
kbuild robot reports that s390 fails to build this driver with
COMPILE_TEST. Let's depend on HAS_IOMEM so that s390 doesn't try to
build it.

Cc: Lee Jones <lee.jones@linaro.org>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Reported-by: kernel test robot <lkp@intel.com>
Fixes: 419b3ab698 ("clk: versatile: remove dependency on ARCH_*")
Link: https://lore.kernel.org/r/20210604192321.2594519-1-sboyd@kernel.org
Signed-off-by: Stephen Boyd <sboyd@kernel.org>
2021-06-04 12:23:28 -07:00
Matti Vaittinen
40e568f9c8 gpio: gpio-regmap: Use devm_add_action_or_reset()
Slightly simplify the devm_gpio_regmap_register() by using the
devm_add_action_or_reset().

Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: Michael Walle <michael@walle.cc>
Signed-off-by: Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com>
2021-06-04 20:53:51 +02:00
Masami Hiramatsu
f4f1c42953 perf probe: Report possible permission error for map__load() failure
Report possible permission error including kptr_restrict setting
for map__load() failure. This can happen when non-superuser runs
perf probe.

With this patch, perf probe shows the following message.

 $ perf probe vfs_read
 Failed to load symbols from /proc/kallsyms
 Please ensure you can read the /proc/kallsyms symbol addresses.
 If the /proc/sys/kernel/kptr_restrict is '2', you can not read
 kernel symbol address even if you are a superuser. Please change
 it to '1'. If kptr_restrict is '1', the superuser can read the
 symbol addresses.
 In that case, please run this command again with sudo.
   Error: Failed to add events.

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Link: http://lore.kernel.org/lkml/162282065877.448336.10047912688119745151.stgit@devnote2
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-06-04 15:43:37 -03:00
Peter Collingbourne
c275c5c6d5 kasan: disable freed user page poisoning with HW tags
Poisoning freed pages protects against kernel use-after-free. The
likelihood of such a bug involving kernel pages is significantly higher
than that for user pages. At the same time, poisoning freed pages can
impose a significant performance cost, which cannot always be justified
for user pages given the lower probability of finding a bug. Therefore,
disable freed user page poisoning when using HW tags. We identify
"user" pages via the flag set GFP_HIGHUSER_MOVABLE, which indicates
a strong likelihood of not being directly accessible to the kernel.

Signed-off-by: Peter Collingbourne <pcc@google.com>
Reviewed-by: Andrey Konovalov <andreyknvl@gmail.com>
Link: https://linux-review.googlesource.com/id/I716846e2de8ef179f44e835770df7e6307be96c9
Link: https://lore.kernel.org/r/20210602235230.3928842-5-pcc@google.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-06-04 19:32:21 +01:00
Peter Collingbourne
013bb59dbb arm64: mte: handle tags zeroing at page allocation time
Currently, on an anonymous page fault, the kernel allocates a zeroed
page and maps it in user space. If the mapping is tagged (PROT_MTE),
set_pte_at() additionally clears the tags. It is, however, more
efficient to clear the tags at the same time as zeroing the data on
allocation. To avoid clearing the tags on any page (which may not be
mapped as tagged), only do this if the vma flags contain VM_MTE. This
requires introducing a new GFP flag that is used to determine whether
to clear the tags.

The DC GZVA instruction with a 0 top byte (and 0 tag) requires
top-byte-ignore. Set the TCR_EL1.{TBI1,TBID1} bits irrespective of
whether KASAN_HW is enabled.

Signed-off-by: Peter Collingbourne <pcc@google.com>
Co-developed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://linux-review.googlesource.com/id/Id46dc94e30fe11474f7e54f5d65e7658dbdddb26
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Andrey Konovalov <andreyknvl@gmail.com>
Link: https://lore.kernel.org/r/20210602235230.3928842-4-pcc@google.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-06-04 19:32:21 +01:00
Peter Collingbourne
7a3b835371 kasan: use separate (un)poison implementation for integrated init
Currently with integrated init page_alloc.c needs to know whether
kasan_alloc_pages() will zero initialize memory, but this will start
becoming more complicated once we start adding tag initialization
support for user pages. To avoid page_alloc.c needing to know more
details of what integrated init will do, move the unpoisoning logic
for integrated init into the HW tags implementation. Currently the
logic is identical but it will diverge in subsequent patches.

For symmetry do the same for poisoning although this logic will
be unaffected by subsequent patches.

Signed-off-by: Peter Collingbourne <pcc@google.com>
Reviewed-by: Andrey Konovalov <andreyknvl@gmail.com>
Link: https://linux-review.googlesource.com/id/I2c550234c6c4a893c48c18ff0c6ce658c7c67056
Link: https://lore.kernel.org/r/20210602235230.3928842-3-pcc@google.com
Signed-off-by: Will Deacon <will@kernel.org>
2021-06-04 19:32:21 +01:00