Now that the GVA->GPA page walker is initialized independently,
init_kvm_softmmu() does not do anything more than calling
kvm_init_shadow_mmu() so eliminate it from the call chain.
At the same time, rename kvm_init_shadow_mmu() to
init_kvm_shadow_mmu() for consistency with init_kvm_tdp_mmu().
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
At this point, vcpu->arch.ngva_walk and vcpu->arch.root_gva_walk contain
the same information; compare init_kvm_page_walk() on one side with
init_kvm_softmmu() + shadow_mmu_init_context() on the other. They only
differ in when each is active, and root_gva_walk is also used by shadow
paging, via FNAME(walk_addr) and its callers.
Always use the same instance of kvm_pagewalk to do GVA->GPA translations,
for both guest emulation and shadow paging, instead of flipping the
gva_walk pointer back and forth. After all the page walking does behave
the same no matter if you are in guest mode or not; the difference lies
in the behavior of kvm_translate_gpa and thus in vcpu->arch.mmu, not in
the page walker itself.
This completes the transition from walk_mmu/nested_mmu as the page
walking entry points to gva_walk/ngpa_walk, and removes duplicated
code between the initialization of root_mmu.w and ngva_walk (The
Struct Formerly Known As nested_mmu).
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Replace kvm_mmu's w field with a pointer to an external instance of
struct kvm_pagewalk. This is the first step towards using a single
kvm_pagewalk struct for all GVA walks, whether nested or not.
With this patch, non-MMU code basically does not use kvm_mmu anymore:
it does care about page walks, but it funnels (almost) all interactions
with the TLB to mmu.c.
kvm_mmu_invalidate_addr() still needs to go from kvm_pagewalk to kvm_mmu,
and it cannot anymore use container_of() to do it; the mapping is hardcoded
based on the provided kvm_pagewalk, since there are just three of them.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
nested_mmu is now only used for its w member. While there is still a
single container_of() going from (possibly) gva_walk to its containing
struct kvm_mmu, it is never reached for nested_mmu and therefore it is
safe to strip nested_mmu.w out of its containing struct kvm_mmu.
So do it, and rename it following the model of gva_walk itself.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Now that walk_mmu is only accessed for its "w" member, store
directly the pointer to it. Since it is only used to convert
guest GVAs or nGVAs, call it gva_walk.
Note that there is still one container_of() going from (possibly)
walk_mmu.w to its containing struct kvm_mmu, but for now all instances
of struct kvm_pagewalk do live within a kvm_mmu.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
kvm_mmu_invalidate_addr()'s callers only want to tell it whether to
invalidate a GVA or GPA. This will ultimately be represented by two
different kvm_pagewalk structs, so adjust the type of the parameter.
As of this patch, the GVA case is represented by both root_mmu.w and
nested_mmu.w. Since nested_mmu never has a sync_spte callback, it would
exit at its check, but really nested_mmu should not be a kvm_mmu in the
first place: it is only used as the walk_mmu, and walk_mmu is only used
for its struct kvm_pagewalk member.
Since calling container_of() on the nested_mmu would be bogus after it is
turned into a struct kvm_pagewalk, introduce a separate check to check
if no work is needed beyond kvm_x86_call(flush_tlb_gva). Implement it
so that it is as similar as possible to what was happening until now; that
is, do nothing if the invalidation is happening for a nested GVA.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The no_hash_pointers boot parameter is now handled as an alias for
hash_pointers=never. However, hash_pointers=never only records the
selected mode during early parameter parsing, and no_hash_pointers is
not updated until hash_pointers_finalize() runs later from SLUB init.
This leaves a window during very early boot where %p output is still
hashed even though the user explicitly requested unhashed pointers with
no_hash_pointers or hash_pointers=never.
Set no_hash_pointers as soon as the "never" mode is parsed. The later
hash_pointers_finalize() call still keeps the final policy decision in
one place, but explicit requests to disable pointer hashing now take
effect for early boot users too.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Link: https://patch.msgid.link/20260618110640.82749-1-kaitao.cheng@linux.dev
Signed-off-by: Petr Mladek <pmladek@suse.com>
Fix the unit-address of cpu1 node to match its reg property value.
Fixes: f6d1890e5f4d ("arm64: tegra: Add device tree for Tegra264")
Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
The Tegra234 SoC uses Cortex-A78AE cores, not Cortex-A78. Update the
compatible string for all CPU nodes to match the actual hardware.
Tegra234 hardware reports:
# head /proc/cpuinfo | egrep 'implementer|part'
CPU implementer : 0x41
CPU part : 0xd42
Which maps to (from arch/arm64/include/asm/cputype.h):
#define ARM_CPU_IMP_ARM 0x41
#define ARM_CPU_PART_CORTEX_A78AE 0xD42
Fixes: a12cf5c339 ("arm64: tegra: Describe Tegra234 CPU hierarchy")
Signed-off-by: Sumit Gupta <sumitg@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
As promised, this removes the remaining instances of
container_of(w, struct kvm_mmu, w), meaning that struct
kvm_pagewalk's definition is pretty much complete.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
With this change, walk_addr_generic and its callees do not need to use
container_of() anymore. There are only two remaining occurrences of
container_of, in permission_fault() and kvm_mmu_refresh_passthrough_bits().
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
struct kvm_pagewalk's behavior depends on the CPU state and its
page format. Move related fields so that walk_mmu remains
self contained.
Note that for now, some of the accessors still use kvm_mmu
to split the churn.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Injection of page faults is also part of accesses to guest
page tables; in particular, __kvm_inject_emulated_page_fault()
calls it on walk_mmu. Move it to struct kvm_pagewalk as
part of converting walk_mmu to a struct kvm_pagewalk.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Continue with yet another callback used in FNAME(walk_addr_generic),
as another step towards removing container_of() from there.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
gva_to_gpa is the main entry point into walk_mmu, which
is only used for guest page table walking (as opposed to building
the page tables). Moving gva_to_gpa to struct kvm_pagewalk
is a step towards making walk_mmu a struct kvm_pagewalk, and
removes several uses of struct kvm_mmu in x86.c.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Start moving page walking functionality out of kvm_mmu; the easiest
target is the callbacks.
Change the kvm_mmu_get_guest_pgd() wrapper to take a struct kvm_pagewalk
too, avoiding the MMU indirection (and associated container_of) whenever
the caller already has one. All container_of uses need to go before
nested_mmu can be changed to a struct kvm_pagewalk, signifying that it
is not used to build page tables.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
In preparation for separating walking and building of page tables,
introduce a dummy struct kvm_pagewalk and pass it around instead of
its containing kvm_mmu to functions that do not build the page tables.
Outermost functions retrieve the mmu via container_of, while internal
functions can pass around the struct kvm_pagewalk pointer.
x86.c is still mostly oblivious to the existence of struct kvm_pagewalk,
with are only a couple exceptions for now, but the plan is for it to
use struct kvm_pagewalk whenever dealing with guest page tables and have
only limited knowledge of struct kvm_mmu.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Just always go through kvm_translate_gpa(), which will either invoke
the vendor check or just return hc->ingpa back.
This is a better way to fix the issue of commit 464af6fc2b ("KVM:
x86: check for nEPT/nNPT in slow flush hypercalls", 2026-05-03).
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Add the testcase for the jited inline of bpf_get_smp_processor_id(),
only for LoongArch currently.
Here is the test result on LoongArch:
$ sudo ./test_progs -t verifier_jit_inline
#604/1 verifier_jit_inline/inline_bpf_get_current_task:OK
#604/2 verifier_jit_inline/inline_bpf_get_smp_processor_id:OK
#604 verifier_jit_inline:OK
Summary: 1/2 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Make it possible to limit certain tests to LoongArch, just like it is
already done for x86_64, arm64, riscv64, and s390x.
This is a follow up patch of:
commit ee7fe84468 ("selftests/bpf: __arch_* macro to limit test cases to specific archs")
commit 1e4e6b9e26 ("selftests/bpf: Add __arch_s390x macro")
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
There is no LoongArch support for get_preempt_count() currently and its
fallback path always returns 0, just add it so that bpf_in_interrupt(),
bpf_in_nmi(), bpf_in_hardirq(), bpf_in_serving_softirq(), bpf_in_task()
work for LoongArch as well.
The latest kernels select CONFIG_THREAD_INFO_IN_TASK, it can just read
preempt_count from the thread_info which is embedded within task_struct.
With this patch, "./test_progs -t exe_ctx" passes on LoongArch.
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Add missing clocks and clock-frequency properties to i2c nodes for LS2K2000.
Signed-off-by: Hongliang Wang <wanghongliang@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Add missing clocks and clock-frequency properties to i2c nodes for LS2K1000.
Signed-off-by: Hongliang Wang <wanghongliang@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Add missing clocks and clock-frequency properties to i2c nodes for LS2K0500.
Signed-off-by: Hongliang Wang <wanghongliang@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The pointer to thread_info is always available in the $tp register, so
the call to bpf_get_smp_processor_id() can be inlined into a single load
instruction.
(1) Here is the sample test.c:
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
SEC("raw_tp/sys_enter")
int test_cpuid(void *ctx)
{
return bpf_get_smp_processor_id();
}
char _license[] SEC("license") = "GPL";
(2) Here are the test steps:
sudo yum install libbpf-devel kernel-devel bpftool
clang -target bpf -O2 -c test.c -o test.o
sudo sysctl -w net.core.bpf_jit_enable=1
sudo bpftool prog show name test_cpuid
sudo rm -f /sys/fs/bpf/test_cpuid
sudo bpftool prog load test.o /sys/fs/bpf/test_cpuid
ID=$(sudo bpftool prog show pinned /sys/fs/bpf/test_cpuid | grep -oE '^[0-9]+')
sudo bpftool prog dump jited id $ID
(3) Here are the test results:
Before: 6 instructions
...
64: lu12i.w $t1, 1213
68: ori $t1, $t1, 1680
6c: lu32i.d $t1, 0
70: lu52i.d $t1, $t1, -1792
74: jirl $ra, $t1, 0
78: move $a5, $a0
...
After: 1 instruction
...
64: ld.wu $a5, $tp, 16
...
This is similar with commit 2ddec2c80b ("riscv, bpf: inline
bpf_get_smp_processor_id()").
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The pointer to task_struct is always available in the $tp register, so
the calls to bpf_get_current_task() and bpf_get_current_task_btf() can
be inlined into a single move instruction.
(1) Here is the sample test.c:
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
SEC("raw_tp/sys_enter")
long test_task(void *ctx)
{
return (long)bpf_get_current_task();
}
char _license[] SEC("license") = "GPL";
(2) Here are the test steps:
sudo yum install libbpf-devel kernel-devel bpftool
clang -target bpf -O2 -c test.c -o test.o
sudo sysctl -w net.core.bpf_jit_enable=1
sudo bpftool prog show name test_task
sudo rm -f /sys/fs/bpf/test_task
sudo bpftool prog load test.o /sys/fs/bpf/test_task
ID=$(sudo bpftool prog show pinned /sys/fs/bpf/test_task | grep -oE '^[0-9]+')
sudo bpftool prog dump jited id $ID
(3) Here are the test results:
Before: 6 instructions
...
64: lu12i.w $t1, 1093
68: ori $t1, $t1, 3320
6c: lu32i.d $t1, 0
70: lu52i.d $t1, $t1, -1792
74: jirl $ra, $t1, 0
78: move $a5, $a0
...
After: 1 instruction
...
64: move $a5, $tp
...
This is similar with commit 2bb138cb20 ("bpf, arm64: Inline
bpf_get_current_task/_btf() helpers").
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The current code updates the tail call counter (TCC) using a pre-increment
approach, it stores the incremented value back to memory before performing
any boundary or target validation checks.
This causes two major issues:
1. When a tail call fails because the target program is NULL, the TCC is
incorrectly incremented and saved in memory anyway.
2. This dummy increment implicitly consumes one slot of the allowed tail
call budget. As a result, the subsequent loop reaches the maximum limit
prematurely, leading to a test failure where the actual loop count is
32 instead of the expected 33.
Fix this by deferring the counter update. Change the branch condition to
BPF_JSGE (greater or equal) so that we check the boundary first. The TCC
is only incremented and stored back to memory after the boundary check
and the NULL-target check both pass.
Before:
$ sudo ./test_progs -t tailcalls/tailcall_3
...
test_tailcall_count:FAIL:tailcall count unexpected tailcall count: actual 32 != expected 33
...
#465/3 tailcalls/tailcall_3:FAIL
#465 tailcalls:FAIL
After:
$ sudo ./test_progs -t tailcalls/tailcall_3
#465/3 tailcalls/tailcall_3:OK
#465 tailcalls:OK
Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
Fixes: c0fcc955ff ("LoongArch: BPF: Fix the tailcall hierarchy")
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The current LoongArch BPF JIT implementation hardcodes the number of
prologue instructions skipped during a tail call as a magic number '7'
in the jirl instruction. However, the accompanying comment explaining
this offset is completely outdated. It inaccurately states that only
a single TCC initialization instruction is bypassed, but in reality,
multiple setup slots are skipped, so fix these outdated comments in
__build_epilogue().
While at it, refine the comments in build_prologue() to describe the
skipped setup slots (RA saving, fentry nops, and the TCC register slot)
using proper dynamic tracing context. Also, remove the magic number '7'
by introducing descriptive macros to formally define the prologue layout
and make the tail call jump offset self-documenting.
Fixes: 61319d15a5 ("LoongArch: BPF: Adjust the jump offset of tail calls")
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The vDSO needs to have a unique build id in a similar manner to the
kernel and modules. Use the build salt macro.
Signed-off-by: Bastian Blank <waldi@debian.org>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
set_direct_map_valid_noflush() incorrectly passes 1 to __set_memory()
instead of nr. This causes only the first page's attr to be updated when
nr > 1.
Other architectures all pass nr correctly.
Cc: stable@vger.kernel.org
Fixes: 0c6378a715 ("arch: introduce set_direct_map_valid_noflush()")
Signed-off-by: Xuewen Wang <wangxuewen@kylinos.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
When hardware page table walker (PTW) is enabled on LoongArch, the CPU
may set _PAGE_DIRTY directly in the page table entry during a write TLB
miss, without going through the software TLB store handler. The software
TLB store handler (tlbex.S:254) sets both _PAGE_DIRTY and_PAGE_MODIFIED
together:
ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
Since hardware PTW only sets _PAGE_DIRTY, the software-only bit, i.e.
_PAGE_MODIFIED is left unchanged. This creates a window where a PTE has
_PAGE_DIRTY set (hardware knows the page is dirty) but _PAGE_MODIFIED
clear (software is unaware).
When fork()/clone() triggers copy-on-write, __copy_present_ptes() calls
pte_wrprotect(), which unconditionally clears both the _PAGE_WRITE and
_PAGE_DIRTY bits:
pte_val(pte) &= ~(_PAGE_WRITE | _PAGE_DIRTY);
Since _PAGE_MODIFIED was never set, the dirtiness information is lost
completely. Subsequently, when memory pressure triggers page reclaim,
page_mkclean() / try_to_unmap() sees the page as clean (i.e. pte_dirty()
returns false) and the page may be freed without writeback, causing data
corruption.
Fix this by propagating the _PAGE_DIRTY bit to the _PAGE_MODIFIED bit in
both pte_wrprotect() and pmd_wrprotect() before clearing writeable bits:
if (pte_val(pte) & _PAGE_DIRTY)
pte_val(pte) |= _PAGE_MODIFIED;
The pmd_wrprotect() fix handles the CONFIG_TRANSPARENT_HUGEPAGE case,
where pmd entries need the same treatment.
This ensures the software dirty tracking bit (checked by pte_dirty() and
pmd_dirty(), which read both the _PAGE_DIRTY and _PAGE_MODIFIED bits) is
preserved across fork COW write-protection.
The issue was found by the LTP madvise09 test case, which exercises page
reclaim after "madvise(MADV_FREE), write and fork" operation sequence on
private anonymous mappings.
Cc: stable@vger.kernel.org
Fixes: 09cfefb7fa ("LoongArch: Add memory management")
Co-developed-by: Tianyang Zhang <zhangtianyang@loongson.cn>
Signed-off-by: Tianyang Zhang <zhangtianyang@loongson.cn>
Signed-off-by: Hongchen Zhang <zhanghongchen@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
arch_kimage_file_post_load_cleanup() and load_other_segments(), both
inside the CONFIG_KEXEC_FILE block, take a struct kimage pointer before
the forward declaration appears. Move the forward declaration above so
it precedes its first use instead of relying on a transitive include.
Fixes: d162feec6b ("LoongArch: Add preparatory infrastructure for kexec_file")
Signed-off-by: 谢致邦 (XIE Zhibang) <Yeking@Red54.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
This is a port of MIPS commit 9f3f3bdc6d ("MIPS: smp: report dying
CPU to RCU in stop_this_cpu()"). smp_send_stop() parks all secondary
CPUs in stop_this_cpu(). And the function marks the CPU offline for the
scheduler via set_cpu_online(false) but never informs RCU, so RCU keeps
expecting a quiescent state from CPUs that are now spinning forever with
interrupts disabled.
As long as nothing waits for an RCU grace period after smp_send_stop()
this is harmless, which is why it went unnoticed. However, since commit
91840be8f7 ("irq_work: Fix use-after-free in irq_work_single() on
PREEMPT_RT"), irq_work_sync() calls synchronize_rcu() on architectures
without an irq_work self-IPI, i.e. where arch_irq_work_has_interrupt()
returns false. Any irq_work_sync() issued in the reboot/shutdown/halt
path after smp_send_stop() then blocks on a grace period that can never
complete, hanging the reboot:
WARNING: CPU: 0 PID: 15 at kernel/irq_work.c:144 irq_work_queue_on
...
rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
rcu: Offline CPU 1 blocking current GP.
rcu: Offline CPU 2 blocking current GP.
rcu: Offline CPU 3 blocking current GP.
This issue needs some hacks to reproduce, and it was not noticed on
LoongArch because arch_irq_work_has_interrupt() usually returns true.
Call rcutree_report_cpu_dead() once interrupts are disabled, mirroring
the generic CPU-hotplug offline path, so RCU stops waiting on the parked
CPUs and grace periods can still complete. LoongArch shuts down all CPUs
here without going through the CPU-hotplug mechanism, so this report is
not otherwise issued.
Cc: <stable@vger.kernel.org>
Fixes: 91840be8f7 ("irq_work: Fix use-after-free in irq_work_single() on PREEMPT_RT")
Reviewed-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
For ACPI system we suppose the ISA/LPC PIO range is registered together
with PCI root bridge. But the fact is there may be some early access to
the ISA/LPC PIO range before ACPI PCI root register (most of them are
due to abnormal BIOS). Unconditionally register the ISA/LPC PIO range
usually causes ACPI PCI root register fail because of the address range
confliction. So we add a pair of helpers: acpi_add_early_pio() to add
PIO for early access, and acpi_remove_early_pio() to remove PIO before
PCI root register. Since acpi_remove_early_pio() may be called multiple
times, we add an acpi_pio flag to ensure PIO be removed only once.
Cc: <stable@vger.kernel.org>
Tested-by: Yuanzhen Gan <elysia-best@simplelinux.cn.eu.org>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Like other architectures such as x86, arm64, riscv, powerpc and s390,
select THREAD_INFO_IN_TASK for LoongArch to move thread_info off the
stack into task_struct. This follows modern kernel standards and also
makes the system more secure.
With this patch, thread_info is included in task_struct at an offset
of 0 instead of being placed at the bottom of the kernel stack. Thus,
the $tp register points to both thread_info and task_struct.
To support this, introduce a per-CPU variable cpu_tasks to store the
pointer to the current task_struct. This decouples the recovery of the
$tp register from the stack pointer during exception entry.
Then initialize cpu_tasks for the primary and secondary CPUs during
arch-specific setup and SMP boot paths. To eliminate the dangerous
windows during the early initialization where the cpu_tasks remains
uninitialized, set_current() is invoked as early as possible in both
setup_arch() and start_secondary(). This ensures the $tp recovery
barrier is armed in case any early boot exceptions or kernel panics
occur.
Modify SAVE_SOME and handle_syscall to restore the $tp register from
cpu_tasks, and also use the la_abs absolute addressing for cpu_tasks
access in assembly to bypass the relocation limits within exception
handling sections. By advancing the preservation of u0 in SAVE_SOME,
we reuse the PERCPU_BASE_KS value in u0 for the cpu_tasks calculation,
effectively eliminating a duplicate csrrd instruction execution on SMP
platforms.
Update <asm/switch_to.h> and <kernel/switch.S> to fully support the
CONFIG_THREAD_INFO_IN_TASK feature.
Remove the obsolete next_ti argument from __switch_to(), which shifts
the remaining arguments ahead in the calling convention (sched_ra from
a3 to a2, and sched_cfa from a4 to a3). Under the new configuration,
__switch_to() now directly derives the thread pointer ($tp) from the
next task_struct pointer in a1.
To preserve the optimal and clean "move tp, a1" path for 64-bit kernels,
the thread pointer ($tp) is assigned directly from a1 in the core path.
For 32-bit kernels, where a1 carries a 2000-byte structural pointer bias
at entry, an explicit adjustment "PTR_ADDI tp, tp, -TASK_STRUCT_OFFSET"
is introduced at the function exit.
In the context of __switch_to(), local interrupts are disabled, and the
kernel is in a critical switching phase where handling any synchronous
exception is practically impossible and prohibited.
If any synchronous exception or watchpoint does trigger in this narrow
window, it constitutes a fatal double fault and the kernel is expected
to die/panic immediately anyway. Therefore, the temporary biased value
in $tp is safe and acceptable here.
Additionally, evaluate the stack lookup as a single load instruction
"LONG_LPTR t0, a1, (TASK_STACK - TASK_STRUCT_OFFSET)", this perfectly
satisfies both 32-bit and 64-bit kernels. Using the "next" pointer in
a1 as the base register, rather than $tp, effectively unchains the data
dependency (RAW hazard) from the preceding move instruction, maximizing
the instruction-level parallelism and superscalar execution efficiency
while naturally adapting the structural shift.
With CONFIG_THREAD_INFO_IN_TASK enabled, the kernel stack life cycle is
decoupled from task_struct and can be freed concurrently.
Currently, show_stacktrace() reads raw stack data via __get_addr() and
subsequently calls show_backtrace() to unwind the frame, without holding
any reference to the target task's stack. If show_stacktrace() is called
on a concurrently exiting task, it could attempt to read from a freed or
reallocated kernel stack. This introduces a severe use-after-free (UAF)
read risk or kernel panics.
Wrap the entire stack inspection process inside show_stacktrace() with
a try_get_task_stack() and put_task_stack() pair. This ensures the task
stack remains pinned safely during both the raw stack data dump loop and
the subsequent stack unwinding phase.
Also, ensure that the task pointer is initialized to "current" early if
it is NULL, so that try_get_task_stack() always operates on a valid task
reference.
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEjF9xRqF1emXiQiqU1w0aZmrPKyEFAmo7AFcACgkQ1w0aZmrP
KyGwJQ/+IbMU5lwz7IY3Xdp6TTJyRYj6IF47tRpygts73bmFAfgxpkHw8Kjly2gu
outN2xyYJKZ/yeIbDZDCC7FVNFS/LUjV0+AeIj78LIfmaf/a33T6gg92vfiQdB2I
rhNEx23rho3aVN5seRpkdATwNAflt3hG+GLAToRMDf1CgQnN8w+wvyOXdg1tAJXP
ArmgSS2ILhe8shrLp51csg9rFyPYiF4P9IQnHNVYanVjfIseHz2YIkR4ynHKMhts
eThPDHDN6k60K+gVMGPIA6n0OFaRdVH/BRqRvlvYJW4Y35YxDLUa6OmN0MErrsWa
Fd2isoD6MZYD5MTeYFc8Mz/zxyVj1/AZwYsqYDqGUhrW1bKhsjK4BtNkffIkFjP1
xTrEDnP0+8qKMRbN1kXTlkf4FYM+8gml+mOP47usJlO3aG5iiSXYa9yFy6mfo7vX
B8rjANFfM42eMQNrMZw5xYER8AXbMAOcMZoWnhvYbHrlDY8QIpGhMHnvfn1MykmC
jTZ3mQUBYxFniHP8kL+XstCIqr5rmb6Me6obOWYRfZOEWGhpxcZOG070uDgcRbdN
A906pag1SLPQAD997/7pxdjiN9PyLoFi+16Lkguzpk/dVdDBbCJhaXTGTVT8oleu
Nu+tj6sXh8Dy1m2BXOCFglDRde/7QTlFwOytrpyAg7gM9Y29Nqw=
=BXU0
-----END PGP SIGNATURE-----
Merge tag 'nf-26-06-23' of git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf
Pablo Neira Ayuso says:
====================
Netfilter fixes for net
The following patchset contains Netfilter fixes for net:
1) Add a workaround to avoid a possible crash if nf_nat and nft_chain_nat are
compiled built-in and nf_nat fails to register, allowing nft_chain_nat to
access the incorrect pernetns area. This is crash specific of all built-in
compilation. From Matias Krause.
2) Revisit conncount GC optimization for confirmed conntracks, skip GC round
if IPS_ASSURED is set on. This is addressing an issue for corner case
use case scenario involving locally generated traffic. No crash, just a
functionality fix. From Fernando F. Mancera.
3) Validate iph->ihl in flowtable IPIP tunnel support, from Lorenzo Bianconi.
This a sanity check to bounces back malformed IPIP packets to classic
forwarding path.
4) Kdoc fixes for x_tables.h, from Randy Dunlap.
5) Use info->options so nft_synproxy_tcp_options() stays on the same local
snapshot, otherwise eval path can observe inconsistent mix of mss and
timestamps. From Runyu Xiao.
6) Add conntrack_sctp_collision.sh to cover for SCTP INIT collisions.
From Yi Chen.
7) Do not allow NFPROTO_UNSPEC targets if family is NFPROTO_BRIDGE in
nft_compat. This allows to use non-sense targets such as xt_nat leading
to crash. From Florian Westphal.
8) Add a selftest queueing from bridge family. From Florian Westphal.
9) Do not allow to reset a conntrack helper via ctnetlink. This feature
antedates the creation of the conntrack-tools, and it is not used
I don't have a usecase for it, I prefer to remove than fixing it.
10) Add deprecation warning for IPv4 only conntrack helpers for PPTP
and IRC. From Florian Westphal.
11) Store the master tuple in the expectation object and use it,
otherwise SLAB_TYPESAFE_RCU rules allow to display incorrect
master tuple information through ctnetlink.
12) Run expectation eviction when inserting an expectation with no
helper, this is a fix for the nft_ct custom expectation support.
13) Fix nft_ct custom expectation timeouts, userspace provides a
timeout in milliseconds but kernel assumes this comes in seconds.
From Florian Westphal.
14) Cap maximum number of expectations per class to 255 expectations
per master conntrack at helper registration. This is a fix to
restrict the maximum number of expectations per master conntrack
which can be a issue for the new lazy GC expectation approach.
* tag 'nf-26-06-23' of git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf:
netfilter: nf_conntrack_helper: cap maximum number of expectation at helper registration
netfilter: nft_ct: expectation timeouts are passed in milliseconds
netfilter: nf_conntrack_expect: run expectation eviction with no helper
netfilter: nf_conntrack_expect: store master_tuple in expectation
netfilter: conntrack: add deprecation warnings for irc and pptp trackers
netfilter: ctnetlink: do not allow to reset helper on existing conntrack
selftests: nft_queue.sh: add a bridge queue test
netfilter: nft_compat: ebtables emulation must reject non-bridge targets
selftests: netfilter: conntrack_sctp_collision.sh: Introduce SCTP INIT collision test
netfilter: nft_synproxy: stop bypassing the priv->info snapshot
netfilter: x_tables.h: fix all kernel-doc warnings
netfilter: flowtable: Validate iph->ihl in nf_flow_ip4_tunnel_proto()
netfilter: nf_conncount: prevent connlimit drops for early confirmed ct
netfilter: nf_nat: avoid invalid nat_net pointer use on failed nf_nat_init()
====================
Link: https://patch.msgid.link/20260623221548.701545-1-pablo@netfilter.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Tony Nguyen says:
====================
Intel Wired LAN Driver Updates 2026-06-22 (ice, i40e, e1000e)
For ice:
Dawid changes call to release control VSI during reset to prevent
leaking it.
Lukasz fixes flow control error check to check value rather than treat
is as bitmap values.
Paul makes link related errors non-fatal to probe to allow for recovery
in certain NVM update situations.
Marcin moves netif_keep_dst() to only be called once when entering
switchdev mode.
ZhaoJinming adds a cleanup path for ice_dpll_init_info() to prevent
memory leaks on error path.
For i40e:
Mohamed Khalfella corrects argument passed in macro to match the
one provided to the macro.
For e1000e:
Dima resolves power state issues by adjusting value of PLL clock gate
and re-enabling K1; a quirk table is added to keep it off for known bad
systems.
* '100GbE' of git://git.kernel.org/pub/scm/linux/kernel/git/tnguy/net-queue:
e1000e: Reconfigure PLL clock gate timeout and re-enable K1 on Meteor Lake
i40e: Fix i40e_debug() to use struct i40e_hw argument
ice: dpll: fix memory leak in ice_dpll_init_info error paths
ice: dpll: set pointers to NULL after kfree in ice_dpll_deinit_info
ice: call netif_keep_dst() once when entering switchdev mode
ice: fix ice_init_link() error return preventing probe
ice: fix AQ error code comparison in ice_set_pauseparam()
ice: fix FDIR CTRL VSI resource leak in ice_reset_all_vfs()
====================
Link: https://patch.msgid.link/20260622220059.2471844-1-anthony.l.nguyen@intel.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The current irq definition of the wake irq and the lpi irq
is wrong, replace them with the right number and name.
Fixes: 30f0ba420e ("net: stmmac: Add glue layer for Spacemit K3 SoC")
Signed-off-by: Inochi Amaoto <inochiama@gmail.com>
Reviewed-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
Link: https://patch.msgid.link/20260623074637.503864-3-inochiama@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The current MII interface register definition from the vendor is wrong,
use the right number for the macro. Also, correct the interface mask
in spacemit_set_phy_intf_sel() so it can update the register with the
right number
Fixes: 30f0ba420e ("net: stmmac: Add glue layer for Spacemit K3 SoC")
Signed-off-by: Inochi Amaoto <inochiama@gmail.com>
Reviewed-by: Maxime Chevallier <maxime.chevallier@bootlin.com>
Link: https://patch.msgid.link/20260623074637.503864-2-inochiama@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
mac->phy_node is acquired via of_parse_phandle() in spl2sw_probe() and
stored in the mac private data, transferring ownership of the
device_node reference to mac. On driver removal, spl2sw_phy_remove()
disconnects the PHY but never drops that reference, so each
probe-then-remove cycle leaks one of_node refcount per port permanently.
Drop the reference after phy_disconnect(). While at it, remove the
redundant inner "if (ndev)" check; comm->ndev[i] was just verified
non-NULL on the line above.
Compile-tested only; no SP7021 hardware available.
Fixes: fd3040b939 ("net: ethernet: Add driver for Sunplus SP7021")
Signed-off-by: Shitalkumar Gandhi <shitalkumar.gandhi@cambiumnetworks.com>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Link: https://patch.msgid.link/f3bdd4c91f3e2269b4e256075f9dc70808b1b8e9.1782195965.git.shitalkumar.gandhi@cambiumnetworks.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
drm_ras includes drm/drm_ras.h, which is a relatively new header not yet
shipped in most distro kernel-header packages. Without the explicit
entry, the build might fail with a message like this:
drm_ras-user.c:19:10: error: ‘DRM_RAS_CMD_CLEAR_ERROR_COUNTER’ \
undeclared here (not in a function); did you mean \
‘DRM_RAS_CMD_GET_ERROR_COUNTER’
Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
Link: https://patch.msgid.link/20260623070818.2161810-1-linux@leemhuis.info
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
gem_init_one() calls gem_remove_one() when register_netdev() fails.
gem_remove_one() unregisters and frees resources owned by the net_device,
including the DMA block, MMIO mapping, PCI regions, and the net_device
itself. gem_init_one() then falls through to its own cleanup labels and
frees the same resources again.
Keep the register_netdev() error path in gem_init_one(): clear drvdata so
PM/remove paths do not see a half-registered device, remove the NAPI
instance added during probe, and let the existing cleanup labels release
the resources once.
The issue was found by a local static-analysis checker for probe error
paths. The reported path was manually inspected before sending this fix.
Compile-tested with CONFIG_SUNGEM=y. Runtime testing was not performed
because no sungem hardware is available.
Fixes: 1da177e4c3 ("Linux-2.6.12-rc2")
Signed-off-by: Ruoyu Wang <ruoyuw560@gmail.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20260623025759.3468566-1-ruoyuw560@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
In tcp_ao_delete_key(), the del_async path skips the current_key
and rnext_key validity checks present in the synchronous path,
assuming these pointers are always NULL on LISTEN sockets. However,
if a key was added with set_current=1/set_rnext=1 while the socket
was in CLOSE state, current_key and rnext_key will be non-NULL
after listen() transitions the socket to LISTEN.
When such a key is deleted with del_async=1, hlist_del_rcu() and
call_rcu() free the key without clearing the dangling pointers.
After the RCU grace period, getsockopt(TCP_AO_INFO) dereferences
current_key->sndid and rnext_key->rcvid from freed slab memory.
Clear current_key and rnext_key in the del_async path when they
reference the key being deleted.
Fixes: d6732b95b6 ("net/tcp: Allow asynchronous delete for TCP-AO keys (MKTs)")
Signed-off-by: HanQuan <eilaimemedsnaimel@gmail.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20260623015208.1191687-1-eilaimemedsnaimel@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Use $(AR) to allow build system to override the archiver tool (e.g.,
when cross-compiling for a different architecture) by setting the AR
environment variable.
GNU Make defaults AR to ar, so this change will not break existing build
environments that do not explicitly set AR.
Fixes: 07c3cc51a0 ("tools: net: package libynl for use in selftests")
Fixes: 86878f14d7 ("tools: ynl: user space helpers")
Signed-off-by: Greg Thelen <gthelen@google.com>
Link: https://patch.msgid.link/20260622161659.145047-1-gthelen@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Configurations with mlx5 built-in but macsec=m fail to link:
x86_64-linux-ld: drivers/infiniband/hw/mlx5/macsec.o: in function `mlx5r_add_gid_macsec_operations':
macsec.c:(.text+0x77d): undefined reference to `macsec_netdev_is_offloaded'
x86_64-linux-ld: drivers/infiniband/hw/mlx5/macsec.o: in function `mlx5r_del_gid_macsec_operations':
macsec.c:(.text+0xe81): undefined reference to `macsec_netdev_is_offloaded'
Fix the dependency so this configuration cannot happen.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Sabrina Dubroca <sd@queasysnail.net>
Link: https://patch.msgid.link/20260622124229.2444502-1-arnd@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
kalmia_rx_fixup() computes usb_packet_length = skb->len - (2 *
KALMIA_HEADER_LENGTH) as a u16, guarded only by a pre-loop check that
skb->len is at least KALMIA_HEADER_LENGTH, which is 6. A device can
deliver a short bulk-IN frame with skb->len in the 6 to 11 range, or
leave a short trailing remainder on a later loop iteration. Either case
underflows usb_packet_length to about 65530.
That bypasses the usb_packet_length < ether_packet_length truncation path.
The device-supplied ether_packet_length, a le16 up to 65535 read from
header_start[2], then drives a memcmp() and the following skb_trim() and
skb_pull() past the end of the rx buffer. The rx buffer is hard_mtu * 10,
which is 14000 bytes. That is an out of bounds read.
Require both the start and end framing headers to be present before
subtracting them, on every loop iteration.
Fixes: d40261236e ("net/usb: Add Samsung Kalmia driver for Samsung GT-B3730")
Cc: stable@vger.kernel.org
Signed-off-by: Maoyi Xie <maoyixie.tju@gmail.com>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Link: https://patch.msgid.link/178211531778.2216480.12637613349790980750@maoyixie.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Even with both paths gated on gs->gro_hint, geneve_gro_complete()
re-derives the inner dispatch type and length from the packet and the
current gs->gro_hint, independently of geneve_gro_receive(). The two can
disagree if gs->gro_hint flips under a concurrent geneve_quiesce()/
geneve_unquiesce() (sk_user_data is NULL across a synchronize_net()), or if
the re-read option bytes differ from the ones receive parsed.
geneve_gro_receive() already records the inner network header position in
NAPI_GRO_CB()->inner_network_offset. Have geneve_gro_complete() compute the
offset it is about to dispatch at, adding ETH_HLEN in the ETH_P_TEB case
where eth_gro_complete() steps over the inner MAC header, and bail out if
it lands past inner_network_offset.
Use a lower bound rather than exact equality: between gh_len and the inner
L3 header, geneve_gro_receive() may also have pulled an inner VLAN tag
(vlan_gro_receive() advances the recorded offset past it), which only moves
inner_network_offset further out. A valid frame therefore always satisfies
inner_nh <= inner_network_offset, while a gh_len inflated by a hint
gro_receive() did not honour dispatches past the validated inner header,
i.e. the out-of-bounds completion. Only the latter is rejected.
Fixes: fd0dd79657 ("geneve: use GRO hint option in the RX path")
Suggested-by: Paolo Abeni <pabeni@redhat.com>
Co-developed-by: Weiming Shi <bestswngs@gmail.com>
Signed-off-by: Xiang Mei <xmei5@asu.edu>
Link: https://patch.msgid.link/20260618032622.484720-2-xmei5@asu.edu
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
