kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-11-13 07:31:07 +00:00
Code Issues Releases Packages Activity
linux/arch/arm64/include/asm/arch_timer.h

229 lines
4.8 KiB
Raw Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details you should have received a copy of the gnu general public license along with this program if not see http www gnu org licenses extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 503 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Enrico Weigelt <info@metux.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190602204653.811534538@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-06-03 07:44:50 +02:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
clocksource/drivers/arm_arch_timer: Extract elf_hwcap use to arch-helper Different mechanisms are used to test and set elf_hwcaps between ARM and ARM64, this results in the use of ifdeferry in this file when setting/testing for the EVTSTRM hwcap. Let's improve readability by extracting this to an arch helper. Signed-off-by: Andrew Murray <andrew.murray@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2019-06-13 13:51:02 +01:00
arm64: arch_timer: simplify accessors A while back we added {read,write}_sysreg accessors to handle accesses to system registers, without the usual boilerplate asm volatile, temporary variable, etc. This patch makes use of these in the arm64 arch timer accessors to make the code shorter and clearer. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-08 13:55:35 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: fix missing linux/bug.h include in asm/arch_timer.h On next-20150105, defconfig compilation breaks with: ./arch/arm64/include/asm/arch_timer.h:112:2: error: implicit declaration of function ‘BUG’ [-Werror=implicit-function-declaration] Fix by including linux/bug.h, where the BUG macro is defined. This second version incorporates a comment from Mark Rutland <mark.rutland@arm.com> to keep the includes in alphabetical order by filename. Signed-off-by: Paul Walmsley <paul@pwsan.com> Cc: Paul Walmsley <pwalmsley@nvidia.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-01-05 17:38:41 -07:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: arch_timer: Work around QorIQ Erratum A-008585 Erratum A-008585 says that the ARM generic timer counter "has the potential to contain an erroneous value for a small number of core clock cycles every time the timer value changes". Accesses to TVAL (both read and write) are also affected due to the implicit counter read. Accesses to CVAL are not affected. The workaround is to reread TVAL and count registers until successive reads return the same value. Writes to TVAL are replaced with an equivalent write to CVAL. The workaround is to reread TVAL and count registers until successive reads return the same value, and when writing TVAL to retry until counter reads before and after the write return the same value. The workaround is enabled if the fsl,erratum-a008585 property is found in the timer node in the device tree. This can be overridden with the clocksource.arm_arch_timer.fsl-a008585 boot parameter, which allows KVM users to enable the workaround until a mechanism is implemented to automatically communicate this information. This erratum can be found on LS1043A and LS2080A. Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Scott Wood <oss@buserror.net> [will: renamed read macro to reflect that it's not usually unstable] Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-22 03:35:17 -05:00
arm64/arch_timer: include <linux/percpu.h> arch_timer.h includes linux/smp.h since the commit: 6acc71ccac7187fc ("arm64: arch_timer: Allows a CPU-specific erratum to only affect a subset of CPUs") It was included to use DEFINE_PER_CPU(), etc. But It should have included <linux/percpu.h> rather than <linux/smp.h>. It worked because smp.h includes percpu.h. The next commit will remove percpu.h from smp.h and it will break this usage. Explicitly include percpu.h and remove smp.h Signed-off-by: Puranjay Mohan <puranjay@kernel.org> Acked-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Stephen Boyd <swboyd@chromium.org> Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com> Link: https://lore.kernel.org/r/20240503171847.68267-1-puranjay@kernel.org Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2024-05-03 17:18:46 +00:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
clocksource/drivers/arm_arch_timer: Introduce generic errata handling infrastructure Currently we have code inline in the arch timer probe path to cater for Freescale erratum A-008585, complete with ifdeffery. This is a little ugly, and will get worse as we try to add more errata handling. This patch refactors the handling of Freescale erratum A-008585. Now the erratum is described in a generic arch_timer_erratum_workaround structure, and the probe path can iterate over these to detect errata and enable workarounds. This will simplify the addition and maintenance of code handling Hisilicon erratum 161010101. Signed-off-by: Ding Tianhong <dingtianhong@huawei.com> [Mark: split patch, correct Kconfig, reword commit message] Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2017-02-06 16:47:41 +00:00
clocksource/arm_arch_timer: Direcly assign set_next_event workaround When a given timer is affected by an erratum and requires an alternative implementation of set_next_event, we do a rather complicated dance to detect and call the workaround on each set_next_event call. This is clearly idiotic, as we can perfectly detect whether this CPU requires a workaround while setting up the clock event device. This only requires the CPU-specific detection to be done a bit earlier, and we can then safely override the set_next_event pointer if we have a workaround associated to that CPU. Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by; Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:04 +01:00
clocksource/drivers/arch_arm_timer: Move workaround synchronisation around We currently handle synchronisation when workarounds are enabled by having an ISB in the __arch_counter_get_cnt?ct_stable() helpers. While this works, this prevents us from relaxing this synchronisation. Instead, move it closer to the point where the synchronisation is actually needed. Further patches will subsequently relax this. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-14-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:21 +01:00
clocksource/arm_arch_timer: Direcly assign set_next_event workaround When a given timer is affected by an erratum and requires an alternative implementation of set_next_event, we do a rather complicated dance to detect and call the workaround on each set_next_event call. This is clearly idiotic, as we can perfectly detect whether this CPU requires a workaround while setting up the clock event device. This only requires the CPU-specific detection to be done a bit earlier, and we can then safely override the set_next_event pointer if we have a workaround associated to that CPU. Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by; Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:04 +01:00
arm64: arch_timer: Work around QorIQ Erratum A-008585 Erratum A-008585 says that the ARM generic timer counter "has the potential to contain an erroneous value for a small number of core clock cycles every time the timer value changes". Accesses to TVAL (both read and write) are also affected due to the implicit counter read. Accesses to CVAL are not affected. The workaround is to reread TVAL and count registers until successive reads return the same value. Writes to TVAL are replaced with an equivalent write to CVAL. The workaround is to reread TVAL and count registers until successive reads return the same value, and when writing TVAL to retry until counter reads before and after the write return the same value. The workaround is enabled if the fsl,erratum-a008585 property is found in the timer node in the device tree. This can be overridden with the clocksource.arm_arch_timer.fsl-a008585 boot parameter, which allows KVM users to enable the workaround until a mechanism is implemented to automatically communicate this information. This erratum can be found on LS1043A and LS2080A. Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Scott Wood <oss@buserror.net> [will: renamed read macro to reflect that it's not usually unstable] Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-22 03:35:17 -05:00
clocksource/arm_arch_timer: Direcly assign set_next_event workaround When a given timer is affected by an erratum and requires an alternative implementation of set_next_event, we do a rather complicated dance to detect and call the workaround on each set_next_event call. This is clearly idiotic, as we can perfectly detect whether this CPU requires a workaround while setting up the clock event device. This only requires the CPU-specific detection to be done a bit earlier, and we can then safely override the set_next_event pointer if we have a workaround associated to that CPU. Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by; Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:04 +01:00
arm64: arch_timer: Work around QorIQ Erratum A-008585 Erratum A-008585 says that the ARM generic timer counter "has the potential to contain an erroneous value for a small number of core clock cycles every time the timer value changes". Accesses to TVAL (both read and write) are also affected due to the implicit counter read. Accesses to CVAL are not affected. The workaround is to reread TVAL and count registers until successive reads return the same value. Writes to TVAL are replaced with an equivalent write to CVAL. The workaround is to reread TVAL and count registers until successive reads return the same value, and when writing TVAL to retry until counter reads before and after the write return the same value. The workaround is enabled if the fsl,erratum-a008585 property is found in the timer node in the device tree. This can be overridden with the clocksource.arm_arch_timer.fsl-a008585 boot parameter, which allows KVM users to enable the workaround until a mechanism is implemented to automatically communicate this information. This erratum can be found on LS1043A and LS2080A. Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Scott Wood <oss@buserror.net> [will: renamed read macro to reflect that it's not usually unstable] Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-22 03:35:17 -05:00
arm64: arch_timer: Add infrastructure for multiple erratum detection methods We're currently stuck with DT when it comes to handling errata, which is pretty restrictive. In order to make things more flexible, let's introduce an infrastructure that could support alternative discovery methods. No change in functionality. Acked-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2017-01-19 17:20:59 +00:00
arm64: arch_timer: Add erratum handler for CPU-specific capability Should we ever have a workaround for an erratum that is detected using a capability and affecting a particular CPU, it'd be nice to have a way to probe them directly. Acked-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2017-03-20 16:47:59 +00:00
arm64: arch_timer: Allow erratum matching with ACPI OEM information Just as we're able to identify a broken platform using some DT information, let's enable a way to spot the offenders with ACPI. The difference is that we can only match on some OEM info instead of implementation-specific properties. So in order to avoid the insane multiplication of errata structures, we allow an array of OEM descriptions to be attached to an erratum structure. Acked-by: Thomas Gleixner <tglx@linutronix.de> Tested-by: dann frazier <dann.frazier@canonical.com> Tested-by: Hanjun Guo <hanjun.guo@linaro.org> Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2017-02-21 14:37:30 +00:00
arm64: arch_timer: Add infrastructure for multiple erratum detection methods We're currently stuck with DT when it comes to handling errata, which is pretty restrictive. In order to make things more flexible, let's introduce an infrastructure that could support alternative discovery methods. No change in functionality. Acked-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2017-01-19 17:20:59 +00:00
arm64: arch_timer: Work around QorIQ Erratum A-008585 Erratum A-008585 says that the ARM generic timer counter "has the potential to contain an erroneous value for a small number of core clock cycles every time the timer value changes". Accesses to TVAL (both read and write) are also affected due to the implicit counter read. Accesses to CVAL are not affected. The workaround is to reread TVAL and count registers until successive reads return the same value. Writes to TVAL are replaced with an equivalent write to CVAL. The workaround is to reread TVAL and count registers until successive reads return the same value, and when writing TVAL to retry until counter reads before and after the write return the same value. The workaround is enabled if the fsl,erratum-a008585 property is found in the timer node in the device tree. This can be overridden with the clocksource.arm_arch_timer.fsl-a008585 boot parameter, which allows KVM users to enable the workaround until a mechanism is implemented to automatically communicate this information. This erratum can be found on LS1043A and LS2080A. Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Scott Wood <oss@buserror.net> [will: renamed read macro to reflect that it's not usually unstable] Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-22 03:35:17 -05:00
arm64: arch_timer: Rework the set_next_event workarounds The way we work around errata affecting set_next_event is not very nice, at it imposes this workaround on errata that do not need it. Add new workaround hooks and let the existing workarounds use them. Acked-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2017-01-27 10:27:09 +00:00
clocksource/drivers/arm_arch_timer: Introduce generic errata handling infrastructure Currently we have code inline in the arch timer probe path to cater for Freescale erratum A-008585, complete with ifdeffery. This is a little ugly, and will get worse as we try to add more errata handling. This patch refactors the handling of Freescale erratum A-008585. Now the erratum is described in a generic arch_timer_erratum_workaround structure, and the probe path can iterate over these to detect errata and enable workarounds. This will simplify the addition and maintenance of code handling Hisilicon erratum 161010101. Signed-off-by: Ding Tianhong <dingtianhong@huawei.com> [Mark: split patch, correct Kconfig, reword commit message] Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2017-02-06 16:47:41 +00:00
arm64: arch_timer: Add infrastructure for multiple erratum detection methods We're currently stuck with DT when it comes to handling errata, which is pretty restrictive. In order to make things more flexible, let's introduce an infrastructure that could support alternative discovery methods. No change in functionality. Acked-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2017-01-19 17:20:59 +00:00
arm64: Implement arch_counter_get_cntpct to read the physical counter As we are about to use the physical counter on arm64 systems that have KVM support, implement arch_counter_get_cntpct() and the associated errata workaround functionality for stable timer reads. Cc: Will Deacon <will.deacon@arm.com> Cc: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
2017-10-18 13:06:25 +02:00
clocksource/drivers/arm_arch_timer: Introduce generic errata handling infrastructure Currently we have code inline in the arch timer probe path to cater for Freescale erratum A-008585, complete with ifdeffery. This is a little ugly, and will get worse as we try to add more errata handling. This patch refactors the handling of Freescale erratum A-008585. Now the erratum is described in a generic arch_timer_erratum_workaround structure, and the probe path can iterate over these to detect errata and enable workarounds. This will simplify the addition and maintenance of code handling Hisilicon erratum 161010101. Signed-off-by: Ding Tianhong <dingtianhong@huawei.com> [Mark: split patch, correct Kconfig, reword commit message] Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2017-02-06 16:47:41 +00:00
arm64: arch_timer: Rework the set_next_event workarounds The way we work around errata affecting set_next_event is not very nice, at it imposes this workaround on errata that do not need it. Add new workaround hooks and let the existing workarounds use them. Acked-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2017-01-27 10:27:09 +00:00
arm64: arch_timer: Allow an workaround descriptor to disable compat vdso As we are about to disable the vdso for compat tasks in some circumstances, let's allow a workaround descriptor to express exactly that. Signed-off-by: Marc Zyngier <maz@kernel.org> Acked-by: Mark Rutland <mark.rutland@arm.com> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20200706163802.1836732-3-maz@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
2020-07-06 17:38:00 +01:00
clocksource/drivers/arm_arch_timer: Introduce generic errata handling infrastructure Currently we have code inline in the arch timer probe path to cater for Freescale erratum A-008585, complete with ifdeffery. This is a little ugly, and will get worse as we try to add more errata handling. This patch refactors the handling of Freescale erratum A-008585. Now the erratum is described in a generic arch_timer_erratum_workaround structure, and the probe path can iterate over these to detect errata and enable workarounds. This will simplify the addition and maintenance of code handling Hisilicon erratum 161010101. Signed-off-by: Ding Tianhong <dingtianhong@huawei.com> [Mark: split patch, correct Kconfig, reword commit message] Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2017-02-06 16:47:41 +00:00
arm64: arch_timer: Allows a CPU-specific erratum to only affect a subset of CPUs Instead of applying a CPU-specific workaround to all CPUs in the system, allow it to only affect a subset of them (typical big-little case). This is done by turning the erratum pointer into a per-CPU variable. Acked-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2017-02-20 18:34:48 +00:00
arm64: arch_timer: Work around QorIQ Erratum A-008585 Erratum A-008585 says that the ARM generic timer counter "has the potential to contain an erroneous value for a small number of core clock cycles every time the timer value changes". Accesses to TVAL (both read and write) are also affected due to the implicit counter read. Accesses to CVAL are not affected. The workaround is to reread TVAL and count registers until successive reads return the same value. Writes to TVAL are replaced with an equivalent write to CVAL. The workaround is to reread TVAL and count registers until successive reads return the same value, and when writing TVAL to retry until counter reads before and after the write return the same value. The workaround is enabled if the fsl,erratum-a008585 property is found in the timer node in the device tree. This can be overridden with the clocksource.arm_arch_timer.fsl-a008585 boot parameter, which allows KVM users to enable the workaround until a mechanism is implemented to automatically communicate this information. This erratum can be found on LS1043A and LS2080A. Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Scott Wood <oss@buserror.net> [will: renamed read macro to reflect that it's not usually unstable] Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-22 03:35:17 -05:00
clocksource/arm_arch_timer: Drop use of static key in arch_timer_reg_read_stable Let's start with the removal of the arch_timer_read_ool_enabled static key in arch_timer_reg_read_stable. It is not a fast path, and we can simplify things a bit. Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:05 +01:00
arm64: Add CNT{P,V}CTSS_EL0 alternatives to cnt{p,v}ct_el0 CNTPCTSS_EL0 and CNTVCTSS_EL0 are alternatives to the usual CNTPCT_EL0 and CNTVCT_EL0 that do not require a previous ISB to be synchronised (SS stands for Self-Synchronising). Use the ARM64_HAS_ECV capability to control alternative sequences that switch to these low(er)-cost primitives. Note that the counter access in the VDSO is for now left alone until we decide whether we want to allow this. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-16-maz@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
2021-10-17 13:42:23 +01:00
clocksource/arm_arch_timer: Drop use of static key in arch_timer_reg_read_stable Let's start with the removal of the arch_timer_read_ool_enabled static key in arch_timer_reg_read_stable. It is not a fast path, and we can simplify things a bit. Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:05 +01:00
arm64: Add CNT{P,V}CTSS_EL0 alternatives to cnt{p,v}ct_el0 CNTPCTSS_EL0 and CNTVCTSS_EL0 are alternatives to the usual CNTPCT_EL0 and CNTVCT_EL0 that do not require a previous ISB to be synchronised (SS stands for Self-Synchronising). Use the ARM64_HAS_ECV capability to control alternative sequences that switch to these low(er)-cost primitives. Note that the counter access in the VDSO is for now left alone until we decide whether we want to allow this. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-16-maz@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
2021-10-17 13:42:23 +01:00
clocksource/arm_arch_timer: Drop use of static key in arch_timer_reg_read_stable Let's start with the removal of the arch_timer_read_ool_enabled static key in arch_timer_reg_read_stable. It is not a fast path, and we can simplify things a bit. Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:05 +01:00
arm64: arch_timer: Allows a CPU-specific erratum to only affect a subset of CPUs Instead of applying a CPU-specific workaround to all CPUs in the system, allow it to only affect a subset of them (typical big-little case). This is done by turning the erratum pointer into a per-CPU variable. Acked-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
2017-02-20 18:34:48 +00:00
clocksource/arm_arch_timer: Drop use of static key in arch_timer_reg_read_stable Let's start with the removal of the arch_timer_read_ool_enabled static key in arch_timer_reg_read_stable. It is not a fast path, and we can simplify things a bit. Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:05 +01:00
arm64/arch_timer: Provide noinstr sched_clock_read() functions With the intent to provide local_clock_noinstr(), a variant of local_clock() that's safe to be called from noinstr code (with the assumption that any such code will already be non-preemptible), prepare for things by providing a noinstr sched_clock_read() function. Specifically, preempt_enable_*() calls out to schedule(), which upsets noinstr validation efforts. Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Tested-by: Michael Kelley <mikelley@microsoft.com> # Hyper-V Link: https://lore.kernel.org/r/20230519102715.435618812@infradead.org
2023-05-19 12:21:02 +02:00
clocksource/arm_arch_timer: Drop use of static key in arch_timer_reg_read_stable Let's start with the removal of the arch_timer_read_ool_enabled static key in arch_timer_reg_read_stable. It is not a fast path, and we can simplify things a bit. Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:05 +01:00
arm64: arch_timer: Work around QorIQ Erratum A-008585 Erratum A-008585 says that the ARM generic timer counter "has the potential to contain an erroneous value for a small number of core clock cycles every time the timer value changes". Accesses to TVAL (both read and write) are also affected due to the implicit counter read. Accesses to CVAL are not affected. The workaround is to reread TVAL and count registers until successive reads return the same value. Writes to TVAL are replaced with an equivalent write to CVAL. The workaround is to reread TVAL and count registers until successive reads return the same value, and when writing TVAL to retry until counter reads before and after the write return the same value. The workaround is enabled if the fsl,erratum-a008585 property is found in the timer node in the device tree. This can be overridden with the clocksource.arm_arch_timer.fsl-a008585 boot parameter, which allows KVM users to enable the workaround until a mechanism is implemented to automatically communicate this information. This erratum can be found on LS1043A and LS2080A. Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Scott Wood <oss@buserror.net> [will: renamed read macro to reflect that it's not usually unstable] Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-22 03:35:17 -05:00
clocksource: arch_timer: Make register accessors less error-prone Using an enum for the register we wish to access allows newer compilers to determine if we've forgotten a case in our switch statement. This allows us to remove the BUILD_BUG() instances in the arm64 port, avoiding problems where optimizations may not happen. To try and force better code generation we're currently marking the accessor functions as inline, but newer compilers can ignore the inline keyword unless it's marked __always_inline. Luckily on arm and arm64 inline is __always_inline, but let's make everything __always_inline to be explicit. Suggested-by: Thomas Gleixner <tglx@linutronix.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Marc Zyngier <Marc.Zyngier@arm.com> Signed-off-by: Stephen Boyd <sboyd@codeaurora.org> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Acked-by: Mark Rutland <mark.rutland@arm.com>
2013-07-18 16:59:28 -07:00
clocksource/drivers/arm_arch_timer: Extend write side of timer register accessors to u64 The various accessors for the timer sysreg and MMIO registers are currently hardwired to 32bit. However, we are about to introduce the use of the CVAL registers, which require a 64bit access. Upgrade the write side of the accessors to take a 64bit value (the read side is left untouched as we don't plan to ever read back any of these registers). No functional change expected. Reviewed-by: Oliver Upton <oupton@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Tested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-4-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:11 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: arch_timer: simplify accessors A while back we added {read,write}_sysreg accessors to handle accesses to system registers, without the usual boilerplate asm volatile, temporary variable, etc. This patch makes use of these in the arm64 arch timer accessors to make the code shorter and clearer. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-08 13:55:35 +01:00
clocksource/drivers/arm_arch_timer: Drop unnecessary ISB on CVAL programming Switching from TVAL to CVAL has a small drawback: we need an ISB before reading the counter. We cannot get rid of it, but we can instead remove the one that comes just after writing to CVAL. This reduces the number of ISBs from 3 to 2 when programming the timer. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-12-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:19 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
clocksource/drivers/arm_arch_timer: Move system register timer programming over to CVAL In order to cope better with high frequency counters, move the programming of the timers from the countdown timer (TVAL) over to the comparator (CVAL). The programming model is slightly different, as we now need to read the current counter value to have an absolute deadline instead of a relative one. There is a small overhead to this change, which we will address in the following patches. Reviewed-by: Oliver Upton <oupton@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Tested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-5-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:12 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
clocksource/arm_arch_timer: Add build-time guards for unhandled register accesses As we are about to change the registers that are used by the driver, start by adding build-time checks to ensure that we always handle all registers and access modes. Suggested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-2-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:09 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: arch_timer: simplify accessors A while back we added {read,write}_sysreg accessors to handle accesses to system registers, without the usual boilerplate asm volatile, temporary variable, etc. This patch makes use of these in the arm64 arch timer accessors to make the code shorter and clearer. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-08 13:55:35 +01:00
clocksource/drivers/arm_arch_timer: Drop unnecessary ISB on CVAL programming Switching from TVAL to CVAL has a small drawback: we need an ISB before reading the counter. We cannot get rid of it, but we can instead remove the one that comes just after writing to CVAL. This reduces the number of ISBs from 3 to 2 when programming the timer. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-12-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:19 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
clocksource/drivers/arm_arch_timer: Move system register timer programming over to CVAL In order to cope better with high frequency counters, move the programming of the timers from the countdown timer (TVAL) over to the comparator (CVAL). The programming model is slightly different, as we now need to read the current counter value to have an absolute deadline instead of a relative one. There is a small overhead to this change, which we will address in the following patches. Reviewed-by: Oliver Upton <oupton@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Tested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-5-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:12 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
clocksource/arm_arch_timer: Add build-time guards for unhandled register accesses As we are about to change the registers that are used by the driver, start by adding build-time checks to ensure that we always handle all registers and access modes. Suggested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-2-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:09 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
clocksource/arm_arch_timer: Add build-time guards for unhandled register accesses As we are about to change the registers that are used by the driver, start by adding build-time checks to ensure that we always handle all registers and access modes. Suggested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-2-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:09 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
clocksource: arch_timer: Make register accessors less error-prone Using an enum for the register we wish to access allows newer compilers to determine if we've forgotten a case in our switch statement. This allows us to remove the BUILD_BUG() instances in the arm64 port, avoiding problems where optimizations may not happen. To try and force better code generation we're currently marking the accessor functions as inline, but newer compilers can ignore the inline keyword unless it's marked __always_inline. Luckily on arm and arm64 inline is __always_inline, but let's make everything __always_inline to be explicit. Suggested-by: Thomas Gleixner <tglx@linutronix.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Marc Zyngier <Marc.Zyngier@arm.com> Signed-off-by: Stephen Boyd <sboyd@codeaurora.org> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org> Acked-by: Mark Rutland <mark.rutland@arm.com>
2013-07-18 16:59:28 -07:00
clocksource/drivers/arm_arch_timer: Move system register timer programming over to CVAL In order to cope better with high frequency counters, move the programming of the timers from the countdown timer (TVAL) over to the comparator (CVAL). The programming model is slightly different, as we now need to read the current counter value to have an absolute deadline instead of a relative one. There is a small overhead to this change, which we will address in the following patches. Reviewed-by: Oliver Upton <oupton@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Tested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-5-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:12 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: arch_timer: simplify accessors A while back we added {read,write}_sysreg accessors to handle accesses to system registers, without the usual boilerplate asm volatile, temporary variable, etc. This patch makes use of these in the arm64 arch timer accessors to make the code shorter and clearer. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-08 13:55:35 +01:00
clocksource/arm_arch_timer: Add build-time guards for unhandled register accesses As we are about to change the registers that are used by the driver, start by adding build-time checks to ensure that we always handle all registers and access modes. Suggested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-2-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:09 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: arch_timer: simplify accessors A while back we added {read,write}_sysreg accessors to handle accesses to system registers, without the usual boilerplate asm volatile, temporary variable, etc. This patch makes use of these in the arm64 arch timer accessors to make the code shorter and clearer. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-08 13:55:35 +01:00
clocksource/arm_arch_timer: Add build-time guards for unhandled register accesses As we are about to change the registers that are used by the driver, start by adding build-time checks to ensure that we always handle all registers and access modes. Suggested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-2-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:09 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
clocksource/arm_arch_timer: Add build-time guards for unhandled register accesses As we are about to change the registers that are used by the driver, start by adding build-time checks to ensure that we always handle all registers and access modes. Suggested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-2-maz@kernel.org Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2021-10-17 13:42:09 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: arch_timer: simplify accessors A while back we added {read,write}_sysreg accessors to handle accesses to system registers, without the usual boilerplate asm volatile, temporary variable, etc. This patch makes use of these in the arm64 arch timer accessors to make the code shorter and clearer. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-08 13:55:35 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
ARM64: arch_timer: add support to configure and enable event stream This patch adds support for configuring the event stream frequency and enabling it. It also adds the hwcaps as well as compat-specific definitions to the user to detect this event stream feature. Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: Will Deacon <will.deacon@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Olof Johansson <olof@lixom.net> Signed-off-by: Sudeep KarkadaNagesha <sudeep.karkadanagesha@arm.com>
2013-08-13 15:57:53 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: arch_timer: simplify accessors A while back we added {read,write}_sysreg accessors to handle accesses to system registers, without the usual boilerplate asm volatile, temporary variable, etc. This patch makes use of these in the arm64 arch timer accessors to make the code shorter and clearer. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-08 13:55:35 +01:00
ARM64: arch_timer: add support to configure and enable event stream This patch adds support for configuring the event stream frequency and enabling it. It also adds the hwcaps as well as compat-specific definitions to the user to detect this event stream feature. Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: Will Deacon <will.deacon@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Olof Johansson <olof@lixom.net> Signed-off-by: Sudeep KarkadaNagesha <sudeep.karkadanagesha@arm.com>
2013-08-13 15:57:53 +01:00
arm64: arch_timer: simplify accessors A while back we added {read,write}_sysreg accessors to handle accesses to system registers, without the usual boilerplate asm volatile, temporary variable, etc. This patch makes use of these in the arm64 arch timer accessors to make the code shorter and clearer. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2016-09-08 13:55:35 +01:00
arm_arch_timer: Expose event stream status The arch timer configuration for a CPU might get reset after suspending said CPU. In order to reliably use the event stream in the kernel (e.g. for delays), we keep track of the state where we can safely consider the event stream as properly configured. After writing to cntkctl, we issue an ISB to ensure that subsequent delay loops can rely on the event stream being enabled. Signed-off-by: Julien Thierry <julien.thierry@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2017-10-13 14:32:55 +01:00
ARM64: arch_timer: add support to configure and enable event stream This patch adds support for configuring the event stream frequency and enabling it. It also adds the hwcaps as well as compat-specific definitions to the user to detect this event stream feature. Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: Will Deacon <will.deacon@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Olof Johansson <olof@lixom.net> Signed-off-by: Sudeep KarkadaNagesha <sudeep.karkadanagesha@arm.com>
2013-08-13 15:57:53 +01:00
arm64: arch_timer: mark functions as __always_inline If CONFIG_FUNCTION_GRAPH_TRACER is enabled function arch_counter_get_cntvct() is marked as notrace. However, function __arch_counter_get_cntvct is marked as inline. If CONFIG_OPTIMIZE_INLINING is set that will make the two functions tracable which they shouldn't. Rework so that functions __arch_counter_get_* are marked with __always_inline so they will be inlined even if CONFIG_OPTIMIZE_INLINING is turned on. Fixes: 0ea415390cd3 ("clocksource/arm_arch_timer: Use arch_timer_read_counter to access stable counters") Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Anders Roxell <anders.roxell@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-06-03 11:14:02 +02:00
clocksource: arch_timer: Fix code to use physical timers when requested This is a bug fix for using physical arch timers when the arch_timer_use_virtual boolean is false. It restores the arch_counter_get_cntpct() function after removal in 0d651e4e "clocksource: arch_timer: use virtual counters" We need this on certain ARMv7 systems which are architected like this: * The firmware doesn't know and doesn't care about hypervisor mode and we don't want to add the complexity of hypervisor there. * The firmware isn't involved in SMP bringup or resume. * The ARCH timer come up with an uninitialized offset between the virtual and physical counters. Each core gets a different random offset. * The device boots in "Secure SVC" mode. * Nothing has touched the reset value of CNTHCTL.PL1PCEN or CNTHCTL.PL1PCTEN (both default to 1 at reset) One example of such as system is RK3288 where it is much simpler to use the physical counter since there's nobody managing the offset and each time a core goes down and comes back up it will get reinitialized to some other random value. Fixes: 0d651e4e65e9 ("clocksource: arch_timer: use virtual counters") Cc: stable@vger.kernel.org Signed-off-by: Sonny Rao <sonnyrao@chromium.org> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Olof Johansson <olof@lixom.net>
2014-11-23 23:02:44 -08:00
arm64: arch_timer: Ensure counter register reads occur with seqlock held When executing clock_gettime(), either in the vDSO or via a system call, we need to ensure that the read of the counter register occurs within the seqlock reader critical section. This ensures that updates to the clocksource parameters (e.g. the multiplier) are consistent with the counter value and therefore avoids the situation where time appears to go backwards across multiple reads. Extend the vDSO logic so that the seqlock critical section covers the read of the counter register as well as accesses to the data page. Since reads of the counter system registers are not ordered by memory barrier instructions, introduce dependency ordering from the counter read to a subsequent memory access so that the seqlock memory barriers apply to the counter access in both the vDSO and the system call paths. Cc: <stable@vger.kernel.org> Cc: Marc Zyngier <marc.zyngier@arm.com> Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Link: https://lore.kernel.org/linux-arm-kernel/alpine.DEB.2.21.1902081950260.1662@nanos.tec.linutronix.de/ Reported-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-29 17:26:22 +01:00
clocksource: arch_timer: Fix code to use physical timers when requested This is a bug fix for using physical arch timers when the arch_timer_use_virtual boolean is false. It restores the arch_counter_get_cntpct() function after removal in 0d651e4e "clocksource: arch_timer: use virtual counters" We need this on certain ARMv7 systems which are architected like this: * The firmware doesn't know and doesn't care about hypervisor mode and we don't want to add the complexity of hypervisor there. * The firmware isn't involved in SMP bringup or resume. * The ARCH timer come up with an uninitialized offset between the virtual and physical counters. Each core gets a different random offset. * The device boots in "Secure SVC" mode. * Nothing has touched the reset value of CNTHCTL.PL1PCEN or CNTHCTL.PL1PCTEN (both default to 1 at reset) One example of such as system is RK3288 where it is much simpler to use the physical counter since there's nobody managing the offset and each time a core goes down and comes back up it will get reinitialized to some other random value. Fixes: 0d651e4e65e9 ("clocksource: arch_timer: use virtual counters") Cc: stable@vger.kernel.org Signed-off-by: Sonny Rao <sonnyrao@chromium.org> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Olof Johansson <olof@lixom.net>
2014-11-23 23:02:44 -08:00
arm64: arch_timer: mark functions as __always_inline If CONFIG_FUNCTION_GRAPH_TRACER is enabled function arch_counter_get_cntvct() is marked as notrace. However, function __arch_counter_get_cntvct is marked as inline. If CONFIG_OPTIMIZE_INLINING is set that will make the two functions tracable which they shouldn't. Rework so that functions __arch_counter_get_* are marked with __always_inline so they will be inlined even if CONFIG_OPTIMIZE_INLINING is turned on. Fixes: 0ea415390cd3 ("clocksource/arm_arch_timer: Use arch_timer_read_counter to access stable counters") Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Anders Roxell <anders.roxell@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-06-03 11:14:02 +02:00
clocksource/arm_arch_timer: Use arch_timer_read_counter to access stable counters Instead of always going via arch_counter_get_cntvct_stable to access the counter workaround, let's have arch_timer_read_counter point to the right method. For that, we need to track whether any CPU in the system has a workaround for the counter. This is done by having an atomic variable tracking this. Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:07 +01:00
Merge branch 'for-next/timers' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux into for-next/core Conflicts: arch/arm64/Kconfig arch/arm64/include/asm/arch_timer.h
2019-05-01 15:45:36 +01:00
arm64: Add CNT{P,V}CTSS_EL0 alternatives to cnt{p,v}ct_el0 CNTPCTSS_EL0 and CNTVCTSS_EL0 are alternatives to the usual CNTPCT_EL0 and CNTVCT_EL0 that do not require a previous ISB to be synchronised (SS stands for Self-Synchronising). Use the ARM64_HAS_ECV capability to control alternative sequences that switch to these low(er)-cost primitives. Note that the counter access in the VDSO is for now left alone until we decide whether we want to allow this. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-16-maz@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
2021-10-17 13:42:23 +01:00
Merge branch 'for-next/timers' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux into for-next/core Conflicts: arch/arm64/Kconfig arch/arm64/include/asm/arch_timer.h
2019-05-01 15:45:36 +01:00
clocksource/arm_arch_timer: Use arch_timer_read_counter to access stable counters Instead of always going via arch_counter_get_cntvct_stable to access the counter workaround, let's have arch_timer_read_counter point to the right method. For that, we need to track whether any CPU in the system has a workaround for the counter. This is done by having an atomic variable tracking this. Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:07 +01:00
arm64: arch_timer: mark functions as __always_inline If CONFIG_FUNCTION_GRAPH_TRACER is enabled function arch_counter_get_cntvct() is marked as notrace. However, function __arch_counter_get_cntvct is marked as inline. If CONFIG_OPTIMIZE_INLINING is set that will make the two functions tracable which they shouldn't. Rework so that functions __arch_counter_get_* are marked with __always_inline so they will be inlined even if CONFIG_OPTIMIZE_INLINING is turned on. Fixes: 0ea415390cd3 ("clocksource/arm_arch_timer: Use arch_timer_read_counter to access stable counters") Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Anders Roxell <anders.roxell@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-06-03 11:14:02 +02:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: arch_timer: Ensure counter register reads occur with seqlock held When executing clock_gettime(), either in the vDSO or via a system call, we need to ensure that the read of the counter register occurs within the seqlock reader critical section. This ensures that updates to the clocksource parameters (e.g. the multiplier) are consistent with the counter value and therefore avoids the situation where time appears to go backwards across multiple reads. Extend the vDSO logic so that the seqlock critical section covers the read of the counter register as well as accesses to the data page. Since reads of the counter system registers are not ordered by memory barrier instructions, introduce dependency ordering from the counter read to a subsequent memory access so that the seqlock memory barriers apply to the counter access in both the vDSO and the system call paths. Cc: <stable@vger.kernel.org> Cc: Marc Zyngier <marc.zyngier@arm.com> Tested-by: Vincenzo Frascino <vincenzo.frascino@arm.com> Link: https://lore.kernel.org/linux-arm-kernel/alpine.DEB.2.21.1902081950260.1662@nanos.tec.linutronix.de/ Reported-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-29 17:26:22 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00
arm64: arch_timer: mark functions as __always_inline If CONFIG_FUNCTION_GRAPH_TRACER is enabled function arch_counter_get_cntvct() is marked as notrace. However, function __arch_counter_get_cntvct is marked as inline. If CONFIG_OPTIMIZE_INLINING is set that will make the two functions tracable which they shouldn't. Rework so that functions __arch_counter_get_* are marked with __always_inline so they will be inlined even if CONFIG_OPTIMIZE_INLINING is turned on. Fixes: 0ea415390cd3 ("clocksource/arm_arch_timer: Use arch_timer_read_counter to access stable counters") Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Anders Roxell <anders.roxell@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-06-03 11:14:02 +02:00
clocksource/arm_arch_timer: Use arch_timer_read_counter to access stable counters Instead of always going via arch_counter_get_cntvct_stable to access the counter workaround, let's have arch_timer_read_counter point to the right method. For that, we need to track whether any CPU in the system has a workaround for the counter. This is done by having an atomic variable tracking this. Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:07 +01:00
Merge branch 'for-next/timers' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux into for-next/core Conflicts: arch/arm64/Kconfig arch/arm64/include/asm/arch_timer.h
2019-05-01 15:45:36 +01:00
arm64: Add CNT{P,V}CTSS_EL0 alternatives to cnt{p,v}ct_el0 CNTPCTSS_EL0 and CNTVCTSS_EL0 are alternatives to the usual CNTPCT_EL0 and CNTVCT_EL0 that do not require a previous ISB to be synchronised (SS stands for Self-Synchronising). Use the ARM64_HAS_ECV capability to control alternative sequences that switch to these low(er)-cost primitives. Note that the counter access in the VDSO is for now left alone until we decide whether we want to allow this. Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20211017124225.3018098-16-maz@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
2021-10-17 13:42:23 +01:00
Merge branch 'for-next/timers' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux into for-next/core Conflicts: arch/arm64/Kconfig arch/arm64/include/asm/arch_timer.h
2019-05-01 15:45:36 +01:00
clocksource/arm_arch_timer: Use arch_timer_read_counter to access stable counters Instead of always going via arch_counter_get_cntvct_stable to access the counter workaround, let's have arch_timer_read_counter point to the right method. For that, we need to track whether any CPU in the system has a workaround for the counter. This is done by having an atomic variable tracking this. Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2019-04-08 16:49:07 +01:00
ARM: convert arm/arm64 arch timer to use CLKSRC_OF init This converts arm and arm64 to use CLKSRC_OF DT based initialization for the arch timer. A new function arch_timer_arch_init is added to allow for arch specific setup. This has a side effect of enabling sched_clock on omap5 and exynos5. There should not be any reason not to use the arch timers for sched_clock. Signed-off-by: Rob Herring <rob.herring@calxeda.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Kukjin Kim <kgene.kim@samsung.com> Cc: Tony Lindgren <tony@atomide.com> Cc: Simon Horman <horms@verge.net.au> Cc: Magnus Damm <magnus.damm@gmail.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: John Stultz <john.stultz@linaro.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-samsung-soc@vger.kernel.org Cc: linux-omap@vger.kernel.org Cc: linux-sh@vger.kernel.org Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2013-04-10 18:27:51 -05:00
clocksource/drivers/arm_arch_timer: Extract elf_hwcap use to arch-helper Different mechanisms are used to test and set elf_hwcaps between ARM and ARM64, this results in the use of ifdeferry in this file when setting/testing for the EVTSTRM hwcap. Let's improve readability by extracting this to an arch helper. Signed-off-by: Andrew Murray <andrew.murray@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
2019-06-13 13:51:02 +01:00
arm64: move from arm_generic to arm_arch_timer The arch_timer driver supports a superset of the functionality of the arm_generic driver, and is not tied to a particular arch. This patch moves arm64 to use the arch_timer driver, gaining additional functionality in doing so, and removes the (now unused) arm_generic driver. Timer-related hooks specific to arm64 are moved into arch/arm64/kernel/time.c. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-11-20 10:06:00 +00:00