0
Fork 0
mirror of https://codeberg.org/ziglang/zig synced 2026-11-18 07:05:05 +00:00
zig/lib/std/debug.zig

1874 lines
75 KiB

2019-05-26 13:17:34 -04:00
2019-05-26 13:17:34 -04:00
2016-08-17 20:11:04 -07:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
2026-04-13 00:43:34 -07:00
2025-10-08 20:35:34 -07:00
2025-09-02 15:54:36 +01:00
2019-05-26 13:17:34 -04:00
2025-09-17 23:30:04 +01:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-09-08 14:31:45 +01:00
2025-12-09 18:44:39 -08:00
2025-12-17 15:47:33 -08:00
2025-12-09 18:44:39 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 17:00:41 -08:00
2025-12-17 17:00:41 -08:00
2025-12-17 15:47:33 -08:00
2025-12-09 18:44:39 -08:00
2025-12-09 18:44:39 -08:00
2025-12-17 15:47:33 -08:00
2025-12-09 18:44:39 -08:00
2025-12-09 18:44:39 -08:00
2025-09-01 16:50:39 +01:00
2025-09-01 16:50:39 +01:00
2025-12-17 17:00:41 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
rework fuzz testing to be smith based -- On the standard library side: The `input: []const u8` parameter of functions passed to `testing.fuzz` has changed to `smith: *testing.Smith`. `Smith` is used to generate values from libfuzzer or input bytes generated by libfuzzer. `Smith` contains the following base methods: * `value` as a generic method for generating any type * `eos` for generating end-of-stream markers. Provides the additional guarantee `true` will eventually by provided. * `bytes` for filling a byte array. * `slice` for filling part of a buffer and providing the length. `Smith.Weight` is used for giving value ranges a higher probability of being selected. By default, every value has a weight of zero (i.e. they will not be selected). Weights can only apply to values that fit within a u64. The above functions have corresponding ones that accept weights. Additionally, the following functions are provided: * `baselineWeights` which provides a set of weights containing every possible value of a type. * `eosSimpleWeighted` for unique weights for `true` and `false` * `valueRangeAtMost` and `valueRangeLessThan` for weighing only a range of values. -- On the libfuzzer and abi side: --- Uids These are u32s which are used to classify requested values. This solves the problem of a mutation causing a new value to be requested and shifting all future values; for example: 1. An initial input contains the values 1, 2, 3 which are interpreted as a, b, and c respectively by the test. 2. The 1 is mutated to a 4 which causes the test to request an extra value interpreted as d. The input is now 4, 2, 3, 5 (new value) which the test corresponds to a, d, b, c; however, b and c no longer correspond to their original values. Uids contain a hash component and type component. The hash component is currently determined in `Smith` by taking a hash of the calling `@returnAddress()` or via an argument in the corresponding `WithHash` functions. The type component is used extensively in libfuzzer with its hashmaps. --- Mutations At the start of a cycle (a run), a random number of values to mutate is selected with less being exponentially more likely. The indexes of the values are selected from a selected uid with a logarithmic bias to uids with more values. Mutations may change a single values, several consecutive values in a uid, or several consecutive values in the uid-independent order they were requested. They may generate random values, mutate from previous ones, or copy from other values in the same uid from the same input or spliced from another. For integers, mutations from previous ones currently only generates random values. For bytes, mutations from previous mix new random data and previous bytes with a set number of mutations. --- Passive Minimization A different approach has been taken for minimizing inputs: instead of trying a fixed set of mutations when a fresh input is found, the input is instead simply added to the corpus and removed when it is no longer valuable. The quality of an input is measured based off how many unique pcs it hit and how many values it needed from the fuzzer. It is tracked which inputs hold the best qualities for each pc for hitting the minimum and maximum unique pcs while needing the least values. Once all an input's qualities have been superseded for the pcs it hit, it is removed from the corpus. -- Comparison to byte-based smith A byte-based smith would be much more inefficient and complex than this solution. It would be unable to solve the shifting problem that Uids do. It is unable to provide values from the fuzzer past end-of-stream. Even with feedback, it would be unable to act on dynamic weights which have proven essential with the updated tests (e.g. to constrain values to a range). -- Test updates All the standard library tests have been updated to use the new smith interface. For `Deque`, an ad hoc allocator was written to improve performance and remove reliance on heap allocation. `TokenSmith` has been added to aid in testing Ast and help inform decisions on the smith interface.
2026-02-13 17:48:57 -05:00
2025-07-18 10:42:04 -07:00
2025-07-18 10:42:04 -07:00
2025-09-08 14:31:45 +01:00
2025-12-21 17:06:02 -08:00
std.os.uefi.tables: ziggify boot and runtime services (#23441) * std.os.uefi.tables: ziggify boot and runtime services * avoid T{} syntax Co-authored-by: linusg <mail@linusgroh.de> * misc fixes * work * self-review quickfixes * dont make MemoryMapSlice generic * more review fixes, work * more work * more work * review fixes * update boot/runtime services references throughout codebase * self-review fixes * couple of fixes i forgot to commit earlier * fixes from integrating in my own project * fixes from refAllDeclsRecursive * Apply suggestions from code review Co-authored-by: truemedian <truemedian@gmail.com> * more fixes from review * fixes from project integration * make natural alignment of Guid align-8 * EventRegistration is a new opaque type * fix getNextHighMonotonicCount * fix locateProtocol * fix exit * partly revert 7372d65 * oops exit data_len is num of bytes * fixes from project integration * MapInfo consistency, MemoryType update per review * turn EventRegistration back into a pointer * forgot to finish updating MemoryType methods * fix IntFittingRange calls * set uefi.Page nat alignment * Back out "set uefi.Page nat alignment" This backs out commit cdd9bd6f7f5fb763f994b8fbe3e1a1c2996a2393. * get rid of some error.NotFound-s * fix .exit call in panic * review comments, add format method * fix resetSystem data alignment * oops, didnt do a final refAllDeclsRecursive i guess * review comments * writergate update MemoryType.format * fix rename --------- Co-authored-by: linusg <mail@linusgroh.de> Co-authored-by: truemedian <truemedian@gmail.com>
2025-07-12 10:18:53 -07:00
2025-12-17 17:00:41 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 17:00:41 -08:00
2025-12-17 15:47:33 -08:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
2025-12-17 15:47:33 -08:00
2026-04-13 00:43:34 -07:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-10-08 20:35:34 -07:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2026-04-13 00:43:34 -07:00
2025-12-17 17:00:41 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2026-04-13 00:43:34 -07:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2026-04-13 00:43:34 -07:00
2025-12-17 17:00:41 -08:00
2025-12-17 15:47:33 -08:00
2018-12-02 18:54:04 -05:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
2018-12-02 18:54:04 -05:00
2018-07-29 23:25:40 -04:00
2018-12-02 18:54:04 -05:00
2025-09-13 10:29:20 +01:00
2018-12-02 18:54:04 -05:00
2018-07-29 23:25:40 -04:00
2026-04-13 00:43:34 -07:00
2026-04-13 00:43:34 -07:00
2026-04-13 00:43:34 -07:00
2026-04-13 00:43:34 -07:00
2026-04-13 00:43:34 -07:00
2026-04-13 00:43:34 -07:00
2025-12-17 15:47:33 -08:00
2026-04-13 00:43:34 -07:00
2026-04-13 00:43:34 -07:00
2026-04-13 00:43:34 -07:00
2026-04-13 00:43:34 -07:00
2025-09-08 14:31:45 +01:00
2017-04-24 12:01:19 -04:00
2017-04-24 12:01:19 -04:00
2017-04-24 12:01:19 -04:00
2025-09-01 16:50:39 +01:00
2025-12-08 23:14:34 -08:00
2025-06-30 15:54:41 -07:00
2025-12-08 17:14:31 -08:00
2025-06-30 15:54:41 -07:00
2025-12-08 17:14:31 -08:00
2025-12-08 17:14:31 -08:00
2025-12-08 23:14:34 -08:00
2025-12-08 17:14:31 -08:00
2017-04-24 12:01:19 -04:00
2025-09-08 14:31:45 +01:00
2025-09-01 16:50:39 +01:00
2025-09-08 14:31:45 +01:00
2025-09-08 14:31:45 +01:00
std.heap.ArenaAllocator: make it threadsafe Modifies the `Allocator` implementation provided by `ArenaAllocator` to be threadsafe using only atomics and no synchronization primitives locked behind an `Io` implementation. At its core this is a lock-free singly linked list which uses CAS loops to exchange the head node. A nice property of `ArenaAllocator` is that the only functions that can ever remove nodes from its linked list are `reset` and `deinit`, both of which are not part of the `Allocator` interface and thus aren't threadsafe, so node-related ABA problems are impossible. There *are* some trade-offs: end index tracking is now per node instead of per allocator instance. It's not possible to publish a head node and its end index at the same time if the latter isn't part of the former. Another compromise had to be made in regards to resizing existing nodes. Annoyingly, `rawResize` of an arbitrary thread-safe child allocator can of course never be guaranteed to be an atomic operation, so only one `alloc` call can ever resize at the same time, other threads have to consider any resizes they attempt during that time failed. This causes slightly less optimal behavior than what could be achieved with a mutex. The LSB of `Node.size` is used to signal that a node is being resized. This means that all nodes have to have an even size. Calls to `alloc` have to allocate new nodes optimistically as they can only know whether any CAS on a head node will succeed after attempting it, and to attempt the CAS they of course already need to know the address of the freshly allocated node they are trying to make the new head. The simplest solution to this would be to just free the new node again if a CAS fails, however this can be expensive and would mean that in practice arenas could only really be used with a GPA as their child allocator. To work around this, this implementation keeps its own free list of nodes which didn't make their CAS to be reused by a later `alloc` invocation. To keep things simple and avoid ABA problems the free list is only ever be accessed beyond its head by 'stealing' the head node (and thus the entire list) with an atomic swap. This makes iteration and removal trivial since there's only ever one thread doing it at a time which also owns all nodes it's holding. When the thread is done it can just push its list onto the free list again. This implementation offers comparable performance to the previous one when only being accessed by a single thread and a slight speedup compared to the previous implementation wrapped into a `ThreadSafeAllocator` up to ~7 threads performing operations on it concurrently. (measured on a base model MacBook Pro M1)
2026-02-15 17:05:00 +01:00
2025-09-08 14:31:45 +01:00
2025-09-08 14:31:45 +01:00
std.debug: detect general protection faults on x86_64-linux ```zig const std = @import("std"); pub fn main() !void { var addr: *u8 = @ptrFromInt(0xaaaaaaaaaaaaaaaa); addr.* = 1; } ``` On x86_64-linux: Before: ``` $ zig run x.zig Segmentation fault at address 0x0 /home/wooster/Desktop/zig/x.zig:5:5: 0x21d887 in main (x) addr.* = 1; ^ /home/wooster/Desktop/zig-linux-x86_64/lib/std/start.zig:583:37: 0x21d847 in posixCallMainAndExit (x) const result = root.main() catch |err| { ^ /home/wooster/Desktop/zig-linux-x86_64/lib/std/start.zig:251:5: 0x21d371 in _start (x) asm volatile (switch (native_arch) { ^ ???:?:?: 0x0 in ??? (???) Aborted (core dumped) ``` After: ``` $ zig run x.zig --zig-lib-dir lib General protection exception /home/wooster/Desktop/zig/x.zig:5:5: 0x21d907 in main (x) addr.* = 1; ^ /home/wooster/Desktop/zig/lib/std/start.zig:583:37: 0x21d8c7 in posixCallMainAndExit (x) const result = root.main() catch |err| { ^ /home/wooster/Desktop/zig/lib/std/start.zig:251:5: 0x21d3f1 in _start (x) asm volatile (switch (native_arch) { ^ ???:?:?: 0x0 in ??? (???) Aborted (core dumped) ``` As @IntegratedQuantum pointed out in <https://github.com/ziglang/zig/issues/17745#issuecomment-1783815386>, it seems that if `code` of the `siginfo_t` instance is a certain value (128), you are able to distinguish between a general protection exception and a segmentation fault. This does not seem to be documented on `man sigaction`: ``` The following values can be placed in si_code for a SIGSEGV signal: SEGV_MAPERR Address not mapped to object. SEGV_ACCERR Invalid permissions for mapped object. SEGV_BNDERR (since Linux 3.19) Failed address bound checks. SEGV_PKUERR (since Linux 4.6) Access was denied by memory protection keys. See pkeys(7). The protection key which applied to this access is available via si_pkey. ``` (those constants are 1, 2, 3, and 4; none of them are the 128) I can't find a lot of documentation about this but it seems to work consistently for me on x86_64-linux. Here is a gist which provides additional evidence that this is a reliable way of checking for a general protection fault: https://gist.github.com/ytoshima/5682393 (read comment in first line) See also: https://stackoverflow.com/questions/64309366/why-is-the-segfault-address-null-when-accessing-memory-that-has-any-of-the-16-mo This only seems to affect x86_64 and on 32-bit x86 this does not seem to be a problem. Helps with #17745 but doesn't close it because the issue still exists on Windows and other POSIX OSs. I also limited this to x86_64-linux for now because that's the only platform where I tested it. Might work on more POSIX OSs.
2023-10-29 19:21:32 +01:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
improvements targeted at improving async functions * Reuse bytes of async function frames when non-async functions make `noasync` calls. This prevents explosive stack growth. * Zig now passes a stack size argument to the linker when linking ELF binaries. Linux ignores this value, but it is available as a program header called GNU_STACK. I prototyped some code that memory maps extra space to the stack using this program header, but there was still a problem when accessing stack memory very far down. Stack probing is needed or not working or something. I also prototyped using `@newStackCall` to call main and that does work around the issue but it also brings its own issues. That code is commented out for now in std/special/start.zig. I'm on a plane with no Internet, but I plan to consult with the musl community for advice when I get a chance. * Added `noasync` to a bunch of function calls in std.debug. It's very messy but it's a workaround that makes stack traces functional with evented I/O enabled. Eventually these will be cleaned up as the root bugs are found and fixed. Programs built in blocking mode are unaffected. * Lowered the default stack size of std.io.InStream (for the async version) to 1 MiB instead of 4. Until we figure out how to get choosing a stack size working (see 2nd bullet point above), 4 MiB tends to cause segfaults due to stack size running out, or usage of stack memory too far apart, or something like that. * Default thread stack size is bumped from 8 MiB to 16 to match the size we give for the main thread. It's planned to eventually remove this hard coded value and have Zig able to determine this value during semantic analysis, with call graph analysis and function pointer annotations and extern function annotations.
2019-09-11 20:22:49 -04:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
2025-09-08 14:31:45 +01:00
std: rework/remove ucontext_t Our usage of `ucontext_t` in the standard library was kind of problematic. We unnecessarily mimiced libc-specific structures, and our `getcontext` implementation was overkill for our use case of stack tracing. This commit introduces a new namespace, `std.debug.cpu_context`, which contains "context" types for various architectures (currently x86, x86_64, ARM, and AARCH64) containing the general-purpose CPU registers; the ones needed in practice for stack unwinding. Each implementation has a function `current` which populates the structure using inline assembly. The structure is user-overrideable, though that should only be necessary if the standard library does not have an implementation for the *architecture*: that is to say, none of this is OS-dependent. Of course, in POSIX signal handlers, we get a `ucontext_t` from the kernel. The function `std.debug.cpu_context.fromPosixSignalContext` converts this to a `std.debug.cpu_context.Native` with a big ol' target switch. This functionality is not exposed from `std.c` or `std.posix`, and neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale is that these types and functions do not conform to a specific ABI, and in fact tend to get updated over time based on CPU features and extensions; in addition, different libcs use different structures which are "partially compatible" with the kernel structure. Overall, it's a mess, but all we need is the kernel context, so we can just define a kernel-compatible structure as long as we don't claim C compatibility by putting it in `std.c` or `std.posix`. This change resulted in a few nice `std.debug` simplifications, but nothing too noteworthy. However, the main benefit of this change is that DWARF unwinding---sometimes necessary for collecting stack traces reliably---now requires far less target-specific integration. Also fix a bug I noticed in `PageAllocator` (I found this due to a bug in my distro's QEMU distribution; thanks, broken QEMU patch!) and I think a couple of minor bugs in `std.debug`. Resolves: #23801 Resolves: #23802
2025-09-17 18:38:11 +01:00
2025-12-17 17:00:41 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 17:00:41 -08:00
2025-12-17 15:47:33 -08:00
improvements targeted at improving async functions * Reuse bytes of async function frames when non-async functions make `noasync` calls. This prevents explosive stack growth. * Zig now passes a stack size argument to the linker when linking ELF binaries. Linux ignores this value, but it is available as a program header called GNU_STACK. I prototyped some code that memory maps extra space to the stack using this program header, but there was still a problem when accessing stack memory very far down. Stack probing is needed or not working or something. I also prototyped using `@newStackCall` to call main and that does work around the issue but it also brings its own issues. That code is commented out for now in std/special/start.zig. I'm on a plane with no Internet, but I plan to consult with the musl community for advice when I get a chance. * Added `noasync` to a bunch of function calls in std.debug. It's very messy but it's a workaround that makes stack traces functional with evented I/O enabled. Eventually these will be cleaned up as the root bugs are found and fixed. Programs built in blocking mode are unaffected. * Lowered the default stack size of std.io.InStream (for the async version) to 1 MiB instead of 4. Until we figure out how to get choosing a stack size working (see 2nd bullet point above), 4 MiB tends to cause segfaults due to stack size running out, or usage of stack memory too far apart, or something like that. * Default thread stack size is bumped from 8 MiB to 16 to match the size we give for the main thread. It's planned to eventually remove this hard coded value and have Zig able to determine this value during semantic analysis, with call graph analysis and function pointer annotations and extern function annotations.
2019-09-11 20:22:49 -04:00
2019-09-25 18:17:29 +03:00
2019-09-25 18:17:29 +03:00
improvements targeted at improving async functions * Reuse bytes of async function frames when non-async functions make `noasync` calls. This prevents explosive stack growth. * Zig now passes a stack size argument to the linker when linking ELF binaries. Linux ignores this value, but it is available as a program header called GNU_STACK. I prototyped some code that memory maps extra space to the stack using this program header, but there was still a problem when accessing stack memory very far down. Stack probing is needed or not working or something. I also prototyped using `@newStackCall` to call main and that does work around the issue but it also brings its own issues. That code is commented out for now in std/special/start.zig. I'm on a plane with no Internet, but I plan to consult with the musl community for advice when I get a chance. * Added `noasync` to a bunch of function calls in std.debug. It's very messy but it's a workaround that makes stack traces functional with evented I/O enabled. Eventually these will be cleaned up as the root bugs are found and fixed. Programs built in blocking mode are unaffected. * Lowered the default stack size of std.io.InStream (for the async version) to 1 MiB instead of 4. Until we figure out how to get choosing a stack size working (see 2nd bullet point above), 4 MiB tends to cause segfaults due to stack size running out, or usage of stack memory too far apart, or something like that. * Default thread stack size is bumped from 8 MiB to 16 to match the size we give for the main thread. It's planned to eventually remove this hard coded value and have Zig able to determine this value during semantic analysis, with call graph analysis and function pointer annotations and extern function annotations.
2019-09-11 20:22:49 -04:00
2022-04-21 10:14:23 +02:00
2025-12-09 18:44:39 -08:00
2025-09-13 10:29:20 +01:00
2026-04-13 00:43:34 -07:00
2026-04-11 22:27:23 -07:00
2022-04-21 10:14:23 +02:00
2025-12-17 17:00:41 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00
2025-12-17 15:47:33 -08:00