linux/arch/arm/include/asm/io.h

429 lines
14 KiB

ARM: mm: Make virt_to_pfn() a static inline Making virt_to_pfn() a static inline taking a strongly typed (const void *) makes the contract of a passing a pointer of that type to the function explicit and exposes any misuse of the macro virt_to_pfn() acting polymorphic and accepting many types such as (void *), (unitptr_t) or (unsigned long) as arguments without warnings. Doing this is a bit intrusive: virt_to_pfn() requires PHYS_PFN_OFFSET and PAGE_SHIFT to be defined, and this is defined in <asm/page.h>, so this must be included *before* <asm/memory.h>. The use of macros were obscuring the unclear inclusion order here, as the macros would eventually be resolved, but a static inline like this cannot be compiled with unresolved macros. The naive solution to include <asm/page.h> at the top of <asm/memory.h> does not work, because <asm/memory.h> sometimes includes <asm/page.h> at the end of itself, which would create a confusing inclusion loop. So instead, take the approach to always unconditionally include <asm/page.h> at the end of <asm/memory.h> arch/arm uses <asm/memory.h> explicitly in a lot of places, however it turns out that if we just unconditionally include <asm/memory.h> into <asm/page.h> and switch all inclusions of <asm/memory.h> to <asm/page.h> instead, we enforce the right order and <asm/memory.h> will always have access to the definitions. Put an inclusion guard in place making it impossible to include <asm/memory.h> explicitly. Link: https://lore.kernel.org/linux-mm/20220701160004.2ffff4e5ab59a55499f4c736@linux-foundation.org/ Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
2022-06-02 10:18:32 +02:00
ARM: 8341/1: io: Unpessimize relaxed io accessors commit 195bbcac2e5c12f7fb ("ARM: 7500/1: io: avoid writeback addressing modes for __raw_ accessors") disables writeback addressing modes for raw i/o. However, the "+Q" output constraint forces the compiler to disable load hoist optimizations (because the output constraint informs the compiler of memory stores which the compiler assumes may alias other memory). Since the relaxed accessors only guarantee ordering wrt i/o accesses to the same device and not to main memory, there's never a possibility of an accessor invalidating a hoisted load (because only non-i/o loads would have been hoisted). The effect is especially noticable with complex address inputs in loops. For example, the following code: #include <linux/kernel.h> #include <linux/io.h> static const int *remap; void wr_loop(void __iomem *base, int c, int val) { int i; for (i = 0; i < c; i++) writew_relaxed(val, base + remap[c >> 2]); } generates current master | this patch 0: e3510000 cmp r1, #0 | 0: e3510000 cmp r1, #0 4: d12fff1e bxle lr | 4: d12fff1e bxle lr 8: e3003000 movw r3, #0 | 8: e3c1c003 bic ip, r1, #3 c: e3403000 movt r3, #0 | c: e6ff2072 uxth r2, r2 10: e92d4010 push {r4, lr} | 10: e3a03000 mov r3, #0 14: e6ff2072 uxth r2, r2 | 14: e59cc000 ldr ip, [ip] 18: e3c14003 bic r4, r1, #3 | 18: e080000c add r0, r0, ip 1c: e593e000 ldr lr, [r3] | 20: e3a03000 mov r3, #0 | 1c: e1c020b0 strh r2, [r0] | 20: e2833001 add r3, r3, #1 24: e79ec004 ldr ip, [lr, r4] | 24: e1530001 cmp r3, r1 28: e080c00c add ip, r0, ip | 28: 1afffffb bne 1c 2c: e1cc20b0 strh r2, [ip] | 2c: e12fff1e bx lr 30: e2833001 add r3, r3, #1 | 34: e1530001 cmp r3, r1 | 38: 1afffff9 bne 24 | | 3c: e8bd8010 pop {r4, pc} | Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Peter Hurley <peter@hurleysoftware.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-04-13 14:18:50 +01:00
ARM: 8341/1: io: Unpessimize relaxed io accessors commit 195bbcac2e5c12f7fb ("ARM: 7500/1: io: avoid writeback addressing modes for __raw_ accessors") disables writeback addressing modes for raw i/o. However, the "+Q" output constraint forces the compiler to disable load hoist optimizations (because the output constraint informs the compiler of memory stores which the compiler assumes may alias other memory). Since the relaxed accessors only guarantee ordering wrt i/o accesses to the same device and not to main memory, there's never a possibility of an accessor invalidating a hoisted load (because only non-i/o loads would have been hoisted). The effect is especially noticable with complex address inputs in loops. For example, the following code: #include <linux/kernel.h> #include <linux/io.h> static const int *remap; void wr_loop(void __iomem *base, int c, int val) { int i; for (i = 0; i < c; i++) writew_relaxed(val, base + remap[c >> 2]); } generates current master | this patch 0: e3510000 cmp r1, #0 | 0: e3510000 cmp r1, #0 4: d12fff1e bxle lr | 4: d12fff1e bxle lr 8: e3003000 movw r3, #0 | 8: e3c1c003 bic ip, r1, #3 c: e3403000 movt r3, #0 | c: e6ff2072 uxth r2, r2 10: e92d4010 push {r4, lr} | 10: e3a03000 mov r3, #0 14: e6ff2072 uxth r2, r2 | 14: e59cc000 ldr ip, [ip] 18: e3c14003 bic r4, r1, #3 | 18: e080000c add r0, r0, ip 1c: e593e000 ldr lr, [r3] | 20: e3a03000 mov r3, #0 | 1c: e1c020b0 strh r2, [r0] | 20: e2833001 add r3, r3, #1 24: e79ec004 ldr ip, [lr, r4] | 24: e1530001 cmp r3, r1 28: e080c00c add ip, r0, ip | 28: 1afffffb bne 1c 2c: e1cc20b0 strh r2, [ip] | 2c: e12fff1e bx lr 30: e2833001 add r3, r3, #1 | 34: e1530001 cmp r3, r1 | 38: 1afffff9 bne 24 | | 3c: e8bd8010 pop {r4, pc} | Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Peter Hurley <peter@hurleysoftware.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-04-13 14:18:50 +01:00
ARM: 8341/1: io: Unpessimize relaxed io accessors commit 195bbcac2e5c12f7fb ("ARM: 7500/1: io: avoid writeback addressing modes for __raw_ accessors") disables writeback addressing modes for raw i/o. However, the "+Q" output constraint forces the compiler to disable load hoist optimizations (because the output constraint informs the compiler of memory stores which the compiler assumes may alias other memory). Since the relaxed accessors only guarantee ordering wrt i/o accesses to the same device and not to main memory, there's never a possibility of an accessor invalidating a hoisted load (because only non-i/o loads would have been hoisted). The effect is especially noticable with complex address inputs in loops. For example, the following code: #include <linux/kernel.h> #include <linux/io.h> static const int *remap; void wr_loop(void __iomem *base, int c, int val) { int i; for (i = 0; i < c; i++) writew_relaxed(val, base + remap[c >> 2]); } generates current master | this patch 0: e3510000 cmp r1, #0 | 0: e3510000 cmp r1, #0 4: d12fff1e bxle lr | 4: d12fff1e bxle lr 8: e3003000 movw r3, #0 | 8: e3c1c003 bic ip, r1, #3 c: e3403000 movt r3, #0 | c: e6ff2072 uxth r2, r2 10: e92d4010 push {r4, lr} | 10: e3a03000 mov r3, #0 14: e6ff2072 uxth r2, r2 | 14: e59cc000 ldr ip, [ip] 18: e3c14003 bic r4, r1, #3 | 18: e080000c add r0, r0, ip 1c: e593e000 ldr lr, [r3] | 20: e3a03000 mov r3, #0 | 1c: e1c020b0 strh r2, [r0] | 20: e2833001 add r3, r3, #1 24: e79ec004 ldr ip, [lr, r4] | 24: e1530001 cmp r3, r1 28: e080c00c add ip, r0, ip | 28: 1afffffb bne 1c 2c: e1cc20b0 strh r2, [ip] | 2c: e12fff1e bx lr 30: e2833001 add r3, r3, #1 | 34: e1530001 cmp r3, r1 | 38: 1afffff9 bne 24 | | 3c: e8bd8010 pop {r4, pc} | Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Peter Hurley <peter@hurleysoftware.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-04-13 14:18:50 +01:00
ARM: 8341/1: io: Unpessimize relaxed io accessors commit 195bbcac2e5c12f7fb ("ARM: 7500/1: io: avoid writeback addressing modes for __raw_ accessors") disables writeback addressing modes for raw i/o. However, the "+Q" output constraint forces the compiler to disable load hoist optimizations (because the output constraint informs the compiler of memory stores which the compiler assumes may alias other memory). Since the relaxed accessors only guarantee ordering wrt i/o accesses to the same device and not to main memory, there's never a possibility of an accessor invalidating a hoisted load (because only non-i/o loads would have been hoisted). The effect is especially noticable with complex address inputs in loops. For example, the following code: #include <linux/kernel.h> #include <linux/io.h> static const int *remap; void wr_loop(void __iomem *base, int c, int val) { int i; for (i = 0; i < c; i++) writew_relaxed(val, base + remap[c >> 2]); } generates current master | this patch 0: e3510000 cmp r1, #0 | 0: e3510000 cmp r1, #0 4: d12fff1e bxle lr | 4: d12fff1e bxle lr 8: e3003000 movw r3, #0 | 8: e3c1c003 bic ip, r1, #3 c: e3403000 movt r3, #0 | c: e6ff2072 uxth r2, r2 10: e92d4010 push {r4, lr} | 10: e3a03000 mov r3, #0 14: e6ff2072 uxth r2, r2 | 14: e59cc000 ldr ip, [ip] 18: e3c14003 bic r4, r1, #3 | 18: e080000c add r0, r0, ip 1c: e593e000 ldr lr, [r3] | 20: e3a03000 mov r3, #0 | 1c: e1c020b0 strh r2, [r0] | 20: e2833001 add r3, r3, #1 24: e79ec004 ldr ip, [lr, r4] | 24: e1530001 cmp r3, r1 28: e080c00c add ip, r0, ip | 28: 1afffffb bne 1c 2c: e1cc20b0 strh r2, [ip] | 2c: e12fff1e bx lr 30: e2833001 add r3, r3, #1 | 34: e1530001 cmp r3, r1 | 38: 1afffff9 bne 24 | | 3c: e8bd8010 pop {r4, pc} | Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Peter Hurley <peter@hurleysoftware.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-04-13 14:18:50 +01:00
ARM: 8341/1: io: Unpessimize relaxed io accessors commit 195bbcac2e5c12f7fb ("ARM: 7500/1: io: avoid writeback addressing modes for __raw_ accessors") disables writeback addressing modes for raw i/o. However, the "+Q" output constraint forces the compiler to disable load hoist optimizations (because the output constraint informs the compiler of memory stores which the compiler assumes may alias other memory). Since the relaxed accessors only guarantee ordering wrt i/o accesses to the same device and not to main memory, there's never a possibility of an accessor invalidating a hoisted load (because only non-i/o loads would have been hoisted). The effect is especially noticable with complex address inputs in loops. For example, the following code: #include <linux/kernel.h> #include <linux/io.h> static const int *remap; void wr_loop(void __iomem *base, int c, int val) { int i; for (i = 0; i < c; i++) writew_relaxed(val, base + remap[c >> 2]); } generates current master | this patch 0: e3510000 cmp r1, #0 | 0: e3510000 cmp r1, #0 4: d12fff1e bxle lr | 4: d12fff1e bxle lr 8: e3003000 movw r3, #0 | 8: e3c1c003 bic ip, r1, #3 c: e3403000 movt r3, #0 | c: e6ff2072 uxth r2, r2 10: e92d4010 push {r4, lr} | 10: e3a03000 mov r3, #0 14: e6ff2072 uxth r2, r2 | 14: e59cc000 ldr ip, [ip] 18: e3c14003 bic r4, r1, #3 | 18: e080000c add r0, r0, ip 1c: e593e000 ldr lr, [r3] | 20: e3a03000 mov r3, #0 | 1c: e1c020b0 strh r2, [r0] | 20: e2833001 add r3, r3, #1 24: e79ec004 ldr ip, [lr, r4] | 24: e1530001 cmp r3, r1 28: e080c00c add ip, r0, ip | 28: 1afffffb bne 1c 2c: e1cc20b0 strh r2, [ip] | 2c: e12fff1e bx lr 30: e2833001 add r3, r3, #1 | 34: e1530001 cmp r3, r1 | 38: 1afffff9 bne 24 | | 3c: e8bd8010 pop {r4, pc} | Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Peter Hurley <peter@hurleysoftware.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-04-13 14:18:50 +01:00
ARM: 8341/1: io: Unpessimize relaxed io accessors commit 195bbcac2e5c12f7fb ("ARM: 7500/1: io: avoid writeback addressing modes for __raw_ accessors") disables writeback addressing modes for raw i/o. However, the "+Q" output constraint forces the compiler to disable load hoist optimizations (because the output constraint informs the compiler of memory stores which the compiler assumes may alias other memory). Since the relaxed accessors only guarantee ordering wrt i/o accesses to the same device and not to main memory, there's never a possibility of an accessor invalidating a hoisted load (because only non-i/o loads would have been hoisted). The effect is especially noticable with complex address inputs in loops. For example, the following code: #include <linux/kernel.h> #include <linux/io.h> static const int *remap; void wr_loop(void __iomem *base, int c, int val) { int i; for (i = 0; i < c; i++) writew_relaxed(val, base + remap[c >> 2]); } generates current master | this patch 0: e3510000 cmp r1, #0 | 0: e3510000 cmp r1, #0 4: d12fff1e bxle lr | 4: d12fff1e bxle lr 8: e3003000 movw r3, #0 | 8: e3c1c003 bic ip, r1, #3 c: e3403000 movt r3, #0 | c: e6ff2072 uxth r2, r2 10: e92d4010 push {r4, lr} | 10: e3a03000 mov r3, #0 14: e6ff2072 uxth r2, r2 | 14: e59cc000 ldr ip, [ip] 18: e3c14003 bic r4, r1, #3 | 18: e080000c add r0, r0, ip 1c: e593e000 ldr lr, [r3] | 20: e3a03000 mov r3, #0 | 1c: e1c020b0 strh r2, [r0] | 20: e2833001 add r3, r3, #1 24: e79ec004 ldr ip, [lr, r4] | 24: e1530001 cmp r3, r1 28: e080c00c add ip, r0, ip | 28: 1afffffb bne 1c 2c: e1cc20b0 strh r2, [ip] | 2c: e12fff1e bx lr 30: e2833001 add r3, r3, #1 | 34: e1530001 cmp r3, r1 | 38: 1afffff9 bne 24 | | 3c: e8bd8010 pop {r4, pc} | Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Peter Hurley <peter@hurleysoftware.com> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2015-04-13 14:18:50 +01:00
ARM: pxa/sa1100: move I/O space to PCI_IOBASE PXA and StrongARM1100 traditionally map their I/O space 1:1 into virtual memory, using a per-bus io_offset that matches the base address of the ioremap mapping. In order for PXA to work in a multiplatform config, this needs to change so I/O space starts at PCI_IOBASE (0xfee00000). Since the pcmcia soc_common support is shared with StrongARM1100, both have to change at the same time. The affected machines are: - Anything with a PCMCIA slot now uses pci_remap_iospace, which is made available to PCMCIA configurations as well, rather than just PCI. The first PCMCIA slot now starts at port number 0x10000. - The Zeus and Viper platforms have PC/104-style ISA buses, which have a static mapping for both I/O and memory space at 0xf1000000, which can no longer work. It does not appear to have any in-tree users, so moving it to port number 0 makes them behave like a traditional PC. - SA1100 does support ISA slots in theory, but all machines that originally enabled this appear to have been removed from the tree ages ago, and the I/O space is never mapped anywhere. - The Nanoengine machine has support for PCI slots, but looks like this never included I/O space, the resources only define the location for memory and config space. With this, the definitions of __io() and IO_SPACE_LIMIT can be simplified, as the only remaining cases are the generic PCI_IOBASE and the custom inb()/outb() macros on RiscPC. S3C24xx still has a custom inb()/outb() in this here, but this is already removed in another branch. Signed-off-by: Arnd Bergmann <arnd@arndb.de>
2022-04-24 14:36:35 +02:00
ARM: pxa/sa1100: move I/O space to PCI_IOBASE PXA and StrongARM1100 traditionally map their I/O space 1:1 into virtual memory, using a per-bus io_offset that matches the base address of the ioremap mapping. In order for PXA to work in a multiplatform config, this needs to change so I/O space starts at PCI_IOBASE (0xfee00000). Since the pcmcia soc_common support is shared with StrongARM1100, both have to change at the same time. The affected machines are: - Anything with a PCMCIA slot now uses pci_remap_iospace, which is made available to PCMCIA configurations as well, rather than just PCI. The first PCMCIA slot now starts at port number 0x10000. - The Zeus and Viper platforms have PC/104-style ISA buses, which have a static mapping for both I/O and memory space at 0xf1000000, which can no longer work. It does not appear to have any in-tree users, so moving it to port number 0 makes them behave like a traditional PC. - SA1100 does support ISA slots in theory, but all machines that originally enabled this appear to have been removed from the tree ages ago, and the I/O space is never mapped anywhere. - The Nanoengine machine has support for PCI slots, but looks like this never included I/O space, the resources only define the location for memory and config space. With this, the definitions of __io() and IO_SPACE_LIMIT can be simplified, as the only remaining cases are the generic PCI_IOBASE and the custom inb()/outb() macros on RiscPC. S3C24xx still has a custom inb()/outb() in this here, but this is already removed in another branch. Signed-off-by: Arnd Bergmann <arnd@arndb.de>
2022-04-24 14:36:35 +02:00
ARM: pxa/sa1100: move I/O space to PCI_IOBASE PXA and StrongARM1100 traditionally map their I/O space 1:1 into virtual memory, using a per-bus io_offset that matches the base address of the ioremap mapping. In order for PXA to work in a multiplatform config, this needs to change so I/O space starts at PCI_IOBASE (0xfee00000). Since the pcmcia soc_common support is shared with StrongARM1100, both have to change at the same time. The affected machines are: - Anything with a PCMCIA slot now uses pci_remap_iospace, which is made available to PCMCIA configurations as well, rather than just PCI. The first PCMCIA slot now starts at port number 0x10000. - The Zeus and Viper platforms have PC/104-style ISA buses, which have a static mapping for both I/O and memory space at 0xf1000000, which can no longer work. It does not appear to have any in-tree users, so moving it to port number 0 makes them behave like a traditional PC. - SA1100 does support ISA slots in theory, but all machines that originally enabled this appear to have been removed from the tree ages ago, and the I/O space is never mapped anywhere. - The Nanoengine machine has support for PCI slots, but looks like this never included I/O space, the resources only define the location for memory and config space. With this, the definitions of __io() and IO_SPACE_LIMIT can be simplified, as the only remaining cases are the generic PCI_IOBASE and the custom inb()/outb() macros on RiscPC. S3C24xx still has a custom inb()/outb() in this here, but this is already removed in another branch. Signed-off-by: Arnd Bergmann <arnd@arndb.de>
2022-04-24 14:36:35 +02:00
arm[64]/memremap: don't abuse pfn_valid() to ensure presence of linear map The semantics of pfn_valid() is to check presence of the memory map for a PFN and not whether a PFN is covered by the linear map. The memory map may be present for NOMAP memory regions, but they won't be mapped in the linear mapping. Accessing such regions via __va() when they are memremap()'ed will cause a crash. On v5.4.y the crash happens on qemu-arm with UEFI [1]: <1>[ 0.084476] 8<--- cut here --- <1>[ 0.084595] Unable to handle kernel paging request at virtual address dfb76000 <1>[ 0.084938] pgd = (ptrval) <1>[ 0.085038] [dfb76000] *pgd=5f7fe801, *pte=00000000, *ppte=00000000 ... <4>[ 0.093923] [<c0ed6ce8>] (memcpy) from [<c16a06f8>] (dmi_setup+0x60/0x418) <4>[ 0.094204] [<c16a06f8>] (dmi_setup) from [<c16a38d4>] (arm_dmi_init+0x8/0x10) <4>[ 0.094408] [<c16a38d4>] (arm_dmi_init) from [<c0302e9c>] (do_one_initcall+0x50/0x228) <4>[ 0.094619] [<c0302e9c>] (do_one_initcall) from [<c16011e4>] (kernel_init_freeable+0x15c/0x1f8) <4>[ 0.094841] [<c16011e4>] (kernel_init_freeable) from [<c0f028cc>] (kernel_init+0x8/0x10c) <4>[ 0.095057] [<c0f028cc>] (kernel_init) from [<c03010e8>] (ret_from_fork+0x14/0x2c) On kernels v5.10.y and newer the same crash won't reproduce on ARM because commit b10d6bca8720 ("arch, drivers: replace for_each_membock() with for_each_mem_range()") changed the way memory regions are registered in the resource tree, but that merely covers up the problem. On ARM64 memory resources registered in yet another way and there the issue of wrong usage of pfn_valid() to ensure availability of the linear map is also covered. Implement arch_memremap_can_ram_remap() on ARM and ARM64 to prevent access to NOMAP regions via the linear mapping in memremap(). Link: https://lore.kernel.org/all/Yl65zxGgFzF1Okac@sirena.org.uk Link: https://lkml.kernel.org/r/20220426060107.7618-1-rppt@kernel.org Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> Reported-by: "kernelci.org bot" <bot@kernelci.org> Tested-by: Mark Brown <broonie@kernel.org> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Mark Brown <broonie@kernel.org> Cc: Mark-PK Tsai <mark-pk.tsai@mediatek.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Tony Lindgren <tony@atomide.com> Cc: Will Deacon <will@kernel.org> Cc: <stable@vger.kernel.org> [5.4+] Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-05-09 17:34:28 -07:00