0
Fork 0
mirror of https://codeberg.org/ziglang/zig synced 2026-11-18 07:05:05 +00:00
Commit graph

39,602 commits

Author SHA1 Message Date
Pavel Verigo
6716bf52e7 Merge pull request 'wasm: enable incremental tests for selfhosted' (#36534) from pavelverigo/zig:wasm-incr-tests-enable into master
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/36534
2026-08-17 11:08:23 +02:00
mlugg
7b02ab7588 Merge pull request 'Elf2: many enhancements' (#36528) from mlugg/elf2-again into master
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/36528
2026-08-17 10:23:45 +02:00
Pavel Verigo
46af37a1ad wasm: fix and test when dead function emitted after incremental update 2026-08-16 23:23:57 +02:00
Jan Procházka
75044cb04c llvm: synchronize target initialization 2026-08-16 22:19:28 +02:00
rpkak
00af3bfc40
remove RunTranslatedC.zig and TranslateC.zig
Unused since 1bdcdbd996 of #24497
2026-08-16 19:28:35 +02:00
Pavel Verigo
23541774d6 wasm: enable incremental tests for selfhosted
Closes #31810
2026-08-16 19:01:56 +02:00
Matthew Lugg
a402fdc813
Elf2: fix 32-bit build 2026-08-16 11:26:24 +01:00
Alex Rønne Petersen
99e540dc39 Merge pull request 'std.zig.system.x86: deal with lying hypervisors' (#36527) from alexrp/zig:x86-cpu-detect-nonsense into master
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/36527
2026-08-16 12:14:17 +02:00
Alex Rønne Petersen
77bb7adc5c std.Target: require libc for emscripten
closes https://codeberg.org/ziglang/zig/issues/36498
2026-08-16 12:11:36 +02:00
Matthew Lugg
77f7d01a69
tests: run some module tests with Elf2
It cannot yet handle the behavior tests or compiler_rt tests due to them
using `@export` with `.internal`, but we can at least pass the standard
library and libc tests on x86_64.
2026-08-16 10:38:52 +01:00
Matthew Lugg
e9e0ce6c60
Elf2: fix exporting UAVs which are otherwise dead 2026-08-16 10:37:20 +01:00
Matthew Lugg
29bcd2d543
std.posix.test: fix dl_iterate_phdr test
The old test was relying on the following properties:

* There is a segment containing the ehdr, i.e. whose file offset is 0

* That segment is the first phdr with type `PT_LOAD`; because `PT_LOAD`
  segments are sorted on `p_vaddr`, this means its virtual address is
  the lowest of all loadable segments

The first assumption, although not standard, is somewhat reasonable, and
the commit prior to this one makes `Elf2` obey it. However, the second
assumption is not based on anything, and to my knowledge the linker is
entirely welcome to lay out the virtual address space as it sees fit.
Therefore, refactor this test to eliminate that bad assumption.
2026-08-16 10:37:20 +01:00
Matthew Lugg
ede0017eb9
Elf2: include the ehdr in the rodata segment
Although to my knowledge this is not strictly required by the format or
by any OS, it is highly conventional, and not doing so can definitely
break things in practice---including one of the standard library tests!

To achieve this, we add all of the segment nodes *before* the ehdr,
initializing rodata as a fixed "header" node in the `.elf` node, and
then the `.ehdr` node goes within that rodata segment node.
2026-08-16 10:37:20 +01:00
Matthew Lugg
507f2d0ab1
MappedFile: fix bug in growing fixed nodes 2026-08-16 10:37:20 +01:00
Alex Rønne Petersen
8a539f903b
Elf2: support the 64-bit hash table format
This is used on s390x and alpha. While `Elf2` doesn't yet support either
of those targets, we'll probably work on s390x soon, so it's worth
getting this support in now.
2026-08-16 10:37:20 +01:00
Alex Rønne Petersen
4f15073e4f
std.elf: update hash structures for the 64-bit variant 2026-08-16 10:37:20 +01:00
Alex Rønne Petersen
45ad4aab95
std.os.linux: Elf_Symndx is u64 on alpha like s390x 2026-08-16 10:37:20 +01:00
Matthew Lugg
a2ddff74f8
Elf2: fix incorrect validation in SPARC relocation
It turns out that SPARC has a strange definition of "PC-relative" where
the offsets are... unsigned? This results in some annoying restrictions
when linking: specifically, the GOT needs to appear after all code
segments, because these PC-relative relocations are used to access it.
When violating that, we should at least get relocation errors, but
because we were doing the wrong validation here, we were just triggering
miscompilations instead.
2026-08-16 10:37:19 +01:00
Matthew Lugg
58c84c59c0
Elf2: add segment for PLT on SPARC
On SPARC, because JUMP_SLOT relocations write directly to the PLT, the
PLT is required to be RWX. Conventionally, this is achieved by placing
it in the "data" segment and making that RWX instead of RW, but that
just seems unnecessarily dangerous. Instead, let's try making a segment
specifically for the PLT.
2026-08-16 10:37:19 +01:00
Matthew Lugg
99e54fc420
Elf2: many alignment fixes
Notably, this ensures that loadable segments are well-aligned in all
cases, and does so *without* requiring segments to have well-aligned
file offsets (which would result in needless bloat in the ELF file). It
also fixes a bunch of small alignment bugs across the backend, and adds
correct handling for the case of an input section / NAV / UAV forcing a
section, and possibly even segment, to increase its alignment.

Resolves: https://codeberg.org/ziglang/zig/issues/32135
2026-08-16 10:37:19 +01:00
Matthew Lugg
bd805e82b9
Elf2: populate fixed bits in SPARC relocations 2026-08-16 10:37:19 +01:00
Matthew Lugg
594a6d5665
Elf2: fix endianness bug in GOT 2026-08-16 10:37:19 +01:00
Matthew Lugg
52dd990f80
Elf2: PLT fixes 2026-08-16 10:37:19 +01:00
Matthew Lugg
67e525d546
Elf2: fix typo in relocation logic 2026-08-16 10:37:19 +01:00
Matthew Lugg
73a65674e4
link.MappedFile: clarify some logic
No functional changes, just a small refactor so I can properly
understand what's happening in this logic.
2026-08-16 10:37:19 +01:00
Matthew Lugg
c7fed8d0c6
Elf2: implement .hash
This section, of type `SHT_HASH`, is a hash table used for looking up
symbol names in DSOs. It is referenced in `.dynamic` by the `DT_HASH`
entry. Without this, glibc assumes we export no symbols, and musl
refuses to load our DSOs at all!

Modern ELF systems broadly consider `SHT_HASH`/`DT_HASH` deprecated in
favour of `SHT_GNU_HASH`/`DT_GNU_HASH`, another hash table with a
different format which is more efficient for misses. However, as with so
many things in ELF, GNU's hash table format seems to be carefully
engineered to make incremental linking as difficult as possible. Every
dynamic linker out there continues to support `DT_HASH`, which actually
plays pretty nicely with incremental compilation, so I think we'll stick
with `DT_HASH` when doing incremental links.

I would now like to take a moment to complain about these hash tables.

These tables exist in order to make symbol lookups efficient when
applying relocations. The ldso implementations for both glibc and musl
libc work by iterating all relocations, and, for each one, looking up
the target symbol in each candidate DSO based on their `DT_HASH` or
`DT_GNU_HASH` table. The `DT_GNU_HASH` format came about to speed up
this lookup, since it's happening very often, so a small speedup can
lead to major improvements in load time.

However, I don't understand why nobody first solved the really obvious
inefficiency: every single symbol lookup is doing N different hash table
lookups where N is the number of loaded DSOs! A clearly better approach
for a dynamic linker to take is to have a single global hash table, and
every time a DSO is loaded, to add its symbols to that table. Then each
symbol lookup requires only *one* hash map lookup instead of N! Plus,
this change doesn't affect ELF files at all---it could be made in libc
implementations tomorrow. Yes, you need to iterate each DSO's dynamic
symbol table once in full, but that shouldn't be a big deal when you're
already iterating their *relocations* (those are far more numerous than
dynamic symbols!).

Here's another idea for free---after building the hash map, iterate
everyone's dynamic symbol table a second time, and build a lookup table
from their dynamic symbol index to the resolved symbol in the hash map.
Now relocation application doesn't even need a hash map; just a lookup
table access! Maybe this one isn't worthwhile, since the difference
between one LUT access and one hash map lookup isn't enough to warrant
the extra allocation and iteration, but it seems worth a try...
computers are fast, y'know!

What drives me a little crazy is that when someone ran into performance
problems with relocation application, instead of just spending an hour
speeding up lookups in ldso, they chose to invent lazy PLT binding. In
doing so, they opted to inflict needless complexity, unpredictable
runtime performance characteristics, unreportable failures, and security
risks in the form of a mutable jump table, upon every ELF system, for
ever. But I guess at least no poor soul had to implement a hash map.

But fine, let's look at status quo, and compare the standard `SHT_HASH`
format with `SHT_GNU_HASH`. As we all know, GNU's track record terms of
high-quality contributions to the ELF ecosystem is flawless, so there
must have been some good reasons for the changes.

Some small things which seem fine: they changed the hash function, which
I'll assume was with fair reason, and they added a bloom filter to allow
lookups to fail early (important if you accept the design of looking up
symbol names in individual DSOs).

The next change was to add a header field called `symoffset`, so that
the hash table can avoid wasting space on chains for the first N symbol
table entries which aren't actually global symbols. Unfortunately, it
seems that someone missed a memo, because this field, in a format which
debuted in 2006, has been redundant since the mid-90s. ELF files pretty
much universally use a separate `SHT_DYNSYM` section for the dynamic
symbol table, with the express purpose of *omitting* all the `STB_LOCAL`
symbols to save space. If a symbol doesn't appear in the hash table, it
shouldn't be in the dynamic symbol table to begin with. This quirk of
the `SHT_GNU_HASH` section isn't particularly *offensive*, it's just...
not useful.

Then the actual hash table representation. The "chains" array no longer
forms a linked list; instead, `chains[sym_idx]` now holds the *hash* for
the symbol's name. The linked list is eliminated in favour of an
assumption that symbols in the same bucket are contiguous in the symbol
table, and the least-significant bit of `chains[sym_idx]` is repurposed
to indicate the last symbol in a bucket (so you know when to stop
iterating).

In general I'm all for replacing linked lists with arrays, but this
particular change is really annoying. Setting aside the fact that it's
kinda odd for metadata *about* the symbol table to mandate a specific
ordering *within* the symbol table, this rigid ordering requirement is
also terrible for incremental linkers! It means that adding or removing
a symbol requires shifting potentially the entire symbol table up or
down to make sure the new symbol is in the right place, unless you're
happy having gaps all over the table (which I've been trying to avoid in
`Elf2`). I'm sure the contiguous-symbols assumption and the caching of
the hashes does improve lookup performance, but it seems like a far more
complicated solution than just doing some kind of caching in ldso
implementations. How this became the accepted solution to slow symbol
lookups is truly beyond me.
2026-08-16 10:19:44 +01:00
Matthew Lugg
c7a169280d
std.elf: add definitions for SHT_HASH header and hash function 2026-08-16 09:30:30 +01:00
Matthew Lugg
44449730c7
Elf2: fix crashes when emitting dynamic libraries 2026-08-16 09:30:29 +01:00
Matthew Lugg
c6d7ab8a23
Elf2: delete some dead code 2026-08-16 09:30:27 +01:00
Matthew Lugg
31500242f0 link: rework export API
The old API fundamentally did not make sense, because the frontend does
not track exports incrementally: it only knows the set of referenced
exports at the end of each incremental update. Attempting to represent
this as an incremental add/remove-style API was just a lie. It was also
buggy; `deleteExport` was called from the main thread, leading to races
in incremental compilation if `deleteExport` was actually implemented!

We also exposed data in an unnecessarily complicated way which did not
benefit most linkers. The only one which actually cared about knowing
all exports for a given nav/uav upfront was the C linker (which here I
have written to reconstruct that data), and even it can probably be
refactored at some point to avoid that.

So, replace the export API with a simpler one, where all the frontend
does is tell the linker about every `Zcu.Export.Index` which is alive,
just before calling `flush`. In theory this information could actually
be passed into `flush`, but using a separate function made for a simpler
implementation in practice.

Resolves: https://github.com/ziglang/zig/issues/23616
2026-08-16 10:29:59 +02:00
Marcel W. Wysocki
e25015bdce
cc: add --wrap=, --nmagic, --fatal-warnings, and -m linker args
Needed for linking the Linux kernel.
2026-08-16 08:41:27 +02:00
Alex Rønne Petersen
5ee0c64223
std.zig.system.x86: deal with lying hypervisors
closes https://codeberg.org/ziglang/zig/issues/36515
2026-08-16 07:25:03 +02:00
Alex Rønne Petersen
18c87599b1
std.zig.system.x86: simplify detectIntelProcessor() code a bit
mutable state begone
2026-08-16 07:15:19 +02:00
lg
767d252691 llvm: don't scalarize bitcast vectors on LE (#36512)
`scalarize_bit_cast_vector_non_elementwise` was previously unconditionally
enabled on both big-endian and little-endian targets, which led to
suboptimal emitting of vector-to-integer bitcasts (https://zig.godbolt.org/z/qEG8zqfqn). This patch only
enables it on big-endian targets as the little-endian semantics match
the zig semantics.

Reviewed-on: https://codeberg.org/ziglang/zig/pulls/36512
2026-08-16 04:06:29 +02:00
Richalsu
f506a49142 std.Io: add error.ConnectionTimedOut for net read/write/send/receive 2026-08-15 10:46:35 +02:00
Guy Fischman
a72236ae8e compiler: lower array-to-vector coercion in the backend 2026-08-15 10:38:52 +02:00
Lukas Lalinsky
4987b62a90 Fix two bugs in RwLock
Bug 1: `tryLock` can acquire a lock while another reader still holds
shared access to the lock. This one if fairly easy to reproduce, so the
test always catches it.

Bug 2: Cancelled writer can leave a stale semaphore permit, essentially
leaving the lock in an unusable state. This one is hard to reproduce
deterministically, the test catches it most of the time, but not always.

Let me know if you are OK with probabilistic tests like this, or I
should remove them.

Fixes https://codeberg.org/ziglang/zig/issues/36217
Fixes https://codeberg.org/ziglang/zig/issues/36178
2026-08-15 10:34:54 +02:00
Alex Rønne Petersen
cbd8f97cb0
test: disable some failing self-hosted riscv64 tests 2026-08-15 08:50:12 +02:00
Ben Burkert
434c9fe4d7 asn1: replace use of deprecated lang.Type.Enum API
The API for std.lang.Type changed in PR #35234, but the ASN.1 tag
encoder is using the old API. Update the check for an exhaustive enum
for the new API, and add a test that exercises this prong of the
encoder.
2026-08-15 06:19:09 +02:00
Matthew Lugg
ff24d73096
Maker: fix hidden merge conflict
There was a silent merge conflict in
https://codeberg.org/ziglang/zig/pulls/36454 leading to build failures
in Maker.

Fixes: 5d7cf3f34d
2026-08-14 12:09:38 +01:00
pentuppup
e78dfccb56 Maker: restore error message when combining --watch and --webui 2026-08-14 12:46:38 +02:00
Andrew Kelley
5d7cf3f34d zig build: revert package related changes to Configuration
partial revert of 8e48a5c524
partial revert of 8380531aec

fixes regression in the build system causing crash when trying to build
ffmpeg project from the ecosystem.

upon closer inspection, I don't think this is the right direction for
adding this data to Configuration. Let's try again and be a little more
careful about it.
2026-08-14 12:40:19 +02:00
Ryan Liptak
e61d804142 flate.Decompress: Fix potential infinite loop when using limits
Previously, the code relied on being able to write an entire match at once. When combined with limited streaming, this could result in an infinite loop with e.g. streamExact.

The fix here is to support writing partial matches up to the limit, and then continue writing the rest of the match on the next stream.

Fixes https://github.com/ziglang/zig/issues/25032
2026-08-14 12:26:14 +02:00
Ryan Liptak
7b890ff978 Fix Compilation.Path checks when one path is a root path
Previously, it was assumed that the byte after the cwd/outer path had to be a path separator, but that's not true when the cwd/outer path is a root with no components (e.g. `/` or `C:\`)

Fixes #32005
2026-08-14 12:25:13 +02:00
Andrew Kelley
63cfe88f0c Merge pull request 'build system: rework user input options' (#36503) from fix-build-system into master
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/36503
2026-08-14 10:30:15 +02:00
Andrew Kelley
e0f7e43a54 build system: rework user input options
followups from 8e48a5c524 along with
other related changes.

- user input options should not be serialized into Configuration.
- more helpful log messages when invalid -D arguments are provided
- use array hash maps in various places, simplifying iteration and
  sorting.
- when creating a package instance, sort the user input options once,
  before the lookup, rather than creating a fresh hash map and sorting
  it on every call to hash (wtf!)
  - when sorting, don't use ascii case insensitivity (wtf!!)
- simplify the detection of invalid arguments passed to zig build
- use package hash rather than file system path in package instance key
2026-08-13 23:40:03 -07:00
Andrew Kelley
896945d8dd std.Build.Serialize: better array awareness 2026-08-13 23:40:03 -07:00
Andrew Kelley
396ddd4443 std.Build.Serialize: use block to limit var scope 2026-08-13 23:40:03 -07:00
Andrew Kelley
e82505d5dc std.Build.Configuration: rename PackageInstance to Package.Instance 2026-08-13 23:40:03 -07:00
Andrew Kelley
4e5b535609 Merge pull request 'Maker: detect modifications to configurer and recompile it' (#36485) from reconfigure into master
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/36485
2026-08-14 08:39:05 +02:00