0
Fork 0
mirror of https://codeberg.org/ziglang/zig synced 2026-11-18 07:05:05 +00:00
Commit graph

116 commits

Author SHA1 Message Date
Andrew Kelley
a68b5ee372 std.Build.Cache: introduce API for taking the files
ability to take ownership of the set of input files prior to
deinitializing a Manifest
2026-08-13 11:32:46 +02:00
Techatrix
59451cce93
std.Build.Cache: skip over prefixes that are the cwd
Any of the added cache prefixes may be the cwd itself. Skip them like
the null prefix.
2026-07-03 19:33:58 +02:00
Andrew Kelley
15f00d1756 std.Build.Cache: make Progress.Node for hits
Occasionally, two different Zig processes are waiting on the same
compilation artifacts. For example, if the user runs `zig build` for the
first time in two different terminals with the same version of Zig, one
of them will win the race on compiler_rt and start building it, while
the other process waits. Then they both use the same cache artifacts.

Now, the fact that this is happening is communicated to the user by
displaying it in a progress node during the checking for cache hit.
2026-07-01 10:16:22 +02:00
Andrew Kelley
779b6cc63c Maker: partially implement configurer cache invalidation
progress towards #35473
2026-06-29 23:50:20 -07:00
Andrew Kelley
b4b92c71e5 std.zig.buildExeSubprocess: fix memory management of file system inputs 2026-06-29 23:50:19 -07:00
Andrew Kelley
c8dad46ced Maker: add configure file system inputs to configuration cache
also wire up progress node to child
2026-06-29 23:50:19 -07:00
Andrew Kelley
20e7cc8d88 std.Build.Cache: bump max number of prefixes 2026-06-29 23:50:19 -07:00
Andrew Kelley
0c978ba957 WIP: migrate build and fetch commands to Maker process 2026-06-29 23:50:18 -07:00
Andrew Kelley
a04f90c35b introduce std.Build API for making configure script depend on fs 2026-06-29 23:50:18 -07:00
David Senoner
c91727108e
Replace usages of ArrayHashMapUnmanaged with array_hash_map.Custom 2026-06-10 16:14:58 +02:00
Krzysztof Wolicki
9e80795623
all: update to use new std.lang.Type definitions 2026-05-27 10:03:51 +01:00
pentuppup
e4512531d5 fix some missed errors from build system rework 2026-05-27 02:18:25 +02:00
Andrew Kelley
7c718fc72e fix compilation errors from rebase conflicts 2026-05-25 18:54:34 -07:00
Andrew Kelley
a399d37886 maker: upgrade some of the run step logic 2026-05-25 18:54:34 -07:00
Andrew Kelley
e10cbf08ee configure/make phase process separation sketch
`zig build` CLI kicks off async task to compile optimized make runner
executable, does fetch, compiles configure process in debug mode, then
checks cache for the CLI options that affect configuration only. On hit,
skips building/running the configure script. On miss, runs it, saves
result in cache.

The cached artifact is a "configuration" file - a serialized build step
graph, which also includes unlazy package dependencies and additional
file system dependencies.

Next, awaits task for compiling optimized make runner executable, passes
configuration file to it. Make runner is responsible for the CLI after
that point.

For the use case of detecting when `git describe` needs to be rerun, we
can allow the configure process to manually add a file system mtime
dependencies, in this case it would be on `.git/index` and `.git/HEAD`.

This will enable two optimizations:

1. The bulk of the build system will not be rebuilt when user changes
   their configure script.

2. The user logic can be completely bypassed when the CLI options
   provided do not affect the configure phase - even if they affect the
   make phase.

Remaining tasks in the branch:

* some stuff in `zig build` CLI is `@panic("TODO")`.
* configure runner needs to implement serialization of build graph using
  std.zig.Configuration
* build runner needs to be transformed into make runner, consuming
  configuration file as input and deserializing the step graph.
* introduce depending only on a file's metadata and *not* its contents
  into the cache system, and add a std.Build API for using it.
2026-05-25 18:54:33 -07:00
Ryan Liptak
3252a05531 Prefer <err> => |e| return e over <err> => return <err>
Avoids the potential for a typo on the `return <err>` side of the prong
2026-04-20 18:03:14 -07:00
Andrew Kelley
b1d1806fef std.process: currentDir -> currentPath
In Zig standard library, Dir means an open directory handle. path
represents a file system identifier string. This function is better
named after "current path" than "current dir". "get" and "working" are
superfluous.
2026-01-29 18:47:58 -08:00
Andrew Kelley
649aaf4814 std: migrate getcwd to Io
progress towards #30150
2026-01-29 18:40:55 -08:00
Andrew Kelley
08447ca47e std.fs.path: make relative a pure function
Instead of querying the operating system for current working directory
and environment variables, this function now accepts those things as
inputs.
2026-01-04 00:27:08 -08:00
Ryan Liptak
51a6f3a525 Update a few more callsites for std.Io changes 2025-12-23 22:15:11 -08:00
Andrew Kelley
a8088306f6 std: rename other Dir "make" functions to "create" 2025-12-23 22:15:11 -08:00
Andrew Kelley
7ce5ee2e92 std: update remaining unit tests for std.Io API changes 2025-12-23 22:15:10 -08:00
Andrew Kelley
16f8af1b9a compiler: update various code to new fs API 2025-12-23 22:15:09 -08:00
Andrew Kelley
4a53e5b0b4 fix a handful of compilation errors related to std.fs migration 2025-12-23 22:15:08 -08:00
Andrew Kelley
1dcfc8787e update all readFileAlloc() to accept Io instance 2025-12-23 22:15:08 -08:00
Andrew Kelley
4be8be1d2b update all rename() to rename(io) 2025-12-23 22:15:08 -08:00
Andrew Kelley
4218344dd3 std.Build.Cache: remove readSmallFile and writeSmallFile
These were to support optimizations involving detecting when to avoid
calling into LLD, which are no longer implemented.
2025-12-23 22:15:08 -08:00
Andrew Kelley
6f46570958 link.MachO: update parallel hasher to std.Io 2025-12-23 22:15:08 -08:00
Andrew Kelley
9f4d40b1f9 update all stat() to stat(io) 2025-12-23 22:15:08 -08:00
Andrew Kelley
f53248a409 update all std.fs.cwd() to std.Io.Dir.cwd() 2025-12-23 22:15:08 -08:00
Andrew Kelley
8328de24f1 update all occurrences of openFile to receive an io instance 2025-12-23 22:15:08 -08:00
Andrew Kelley
d1d2c37af2 std: all Dir functions moved to std.Io 2025-12-23 22:15:07 -08:00
Alex Rønne Petersen
d22231c039
Compilation: track indirect file system inputs from clang's depfile
Co-authored-by: Matthew Lugg <mlugg@mlugg.co.uk>
2025-12-05 11:22:10 +01:00
Andrew Kelley
5816646aa0
Merge pull request #26037 from ziglang/build-runner-std.Io
build runner: update from std.Thread.Pool to std.Io
2025-11-25 02:49:37 -08:00
Ryan Liptak
53e615b920
Merge pull request #25993 from squeek502/windows-paths
Teach `std.fs.path` about the wonderful world of Windows paths
2025-11-24 15:27:24 -08:00
Andrew Kelley
3f34f5e433 build runner: update Mutex and Condition usage to std.Io 2025-11-24 14:34:18 -08:00
Ryan Liptak
59b8bed222 Teach fs.path about the wonderful world of Windows paths
Previously, fs.path handled a few of the Windows path types, but not all of them, and only a few of them correctly/consistently. This commit aims to make `std.fs.path` correct and consistent in handling all possible Win32 path types.

This commit also slightly nudges the codebase towards a separation of Win32 paths and NT paths, as NT paths are not actually distinguishable from Win32 paths from looking at their contents alone (i.e. `\Device\Foo` could be an NT path or a Win32 rooted path, no way to tell without external context). This commit formalizes `std.fs.path` being fully concerned with Win32 paths, and having no special detection/handling of NT paths.

Resources on Windows path types, and Win32 vs NT paths:

- https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html
- https://chrisdenton.github.io/omnipath/Overview.html
- https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file

API additions/changes/deprecations

- `std.os.windows.getWin32PathType` was added (it is analogous to `RtlDetermineDosPathNameType_U`), while `std.os.windows.getNamespacePrefix` and `std.os.windows.getUnprefixedPathType` were deleted. `getWin32PathType` forms the basis on which the updated `std.fs.path` functions operate.
- `std.fs.path.parsePath`, `std.fs.path.parsePathPosix`, and `std.fs.path.parsePathWindows` were added, while `std.fs.path.windowsParsePath` was deprecated. The new `parsePath` functions provide the "root" and the "kind" of a path, which is platform-specific. The now-deprecated `windowsParsePath` did not handle all possible path types, while the new `parsePathWindows` does.
- `std.fs.path.diskDesignator` has been deprecated in favor of `std.fs.path.parsePath`, and same deal with `diskDesignatorWindows` -> `parsePathWindows`
- `relativeWindows` is now a compile error when *not* targeting Windows, while `relativePosix` is now a compile error when targeting Windows. This is because those functions read/use the CWD path which will behave improperly when used from a system with different path semantics (e.g. calling `relativePosix` from a Windows system with a CWD like `C:\foo\bar` will give you a bogus result since that'd be treated as a single relative component when using POSIX semantics). This also allows `relativeWindows` to use Windows-specific APIs for getting the CWD and environment variables to cut down on allocations.
- `componentIterator`/`ComponentIterator.init` have been made infallible. These functions used to be able to error on UNC paths with an empty server component, and on paths that were assumed to be NT paths, but now:
  + We follow the lead of `RtlDetermineDosPathNameType_U`/`RtlGetFullPathName_U` in how it treats a UNC path with an empty server name (e.g. `\\\share`) and allow it, even if it'll be invalid at the time of usage
  + Now that `std.fs.path` assumes paths are Win32 paths and not NT paths, we don't have to worry about NT paths

Behavior changes

- `std.fs.path` generally: any combinations of mixed path separators for UNC paths are universally supported, e.g. `\/server/share`, `/\server\share`, `/\server/\\//share` are all seen as equivalent UNC paths
- `resolveWindows` handles all path types more appropriately/consistently.
  + `//` and `//foo` used to be treated as a relative path, but are now seen as UNC paths
  + If a rooted/drive-relative path cannot be resolved against anything more definite, the result will remain a rooted/drive-relative path.
  + I've created [a script to generate the results of a huge number of permutations of different path types](https://gist.github.com/squeek502/9eba7f19cad0d0d970ccafbc30f463bf) (the result of running the script is also included for anyone that'd like to vet the behavior).
- `dirnameWindows` now treats the drive-relative root as the dirname of a drive-relative path with a component, e.g. `dirname("C:foo")` is now `C:`, whereas before it would return null. `dirnameWindows` also handles local device paths appropriately now.
- `basenameWindows` now handles all path types more appropriately. The most notable change here is `//a` being treated as a partial UNC path now and therefore `basename` will return `""` for it, whereas before it would return `"a"`
- `relativeWindows` will now do its best to resolve against the most appropriate CWD for each path, e.g. relative for `D:foo` will look at the CWD to check if the drive letter matches, and if not, look at the special environment variable `=D:` to get the shell-defined CWD for that drive, and if that doesn't exist, then it'll resolve against `D:\`.

Implementation details

- `resolveWindows` previously looped through the paths twice to build up the relevant info before doing the actual resolution. Now, `resolveWindows` iterates backwards once and keeps track of which paths are actually relevant using a bit set, which also allows it to break from the loop when it's no longer possible for earlier paths to matter.
- A standalone test was added to test parts of `relativeWindows` since the CWD resolution logic depends on CWD information from the PEB and environment variables

Edge cases worth noting

- A strange piece of trivia that I found out while working on this is that it's technically possible to have a drive letter that it outside the intended A-Z range, or even outside the ASCII range entirely. Since we deal with both WTF-8 and WTF-16 paths, `path[0]`/`path[1]`/`path[2]` will not always refer to the same bits of information, so to get consistent behavior, some decision about how to deal with this edge case had to be made. I've made the choice to conform with how `RtlDetermineDosPathNameType_U` works, i.e. treat the first WTF-16 code unit as the drive letter. This means that when working with WTF-8, checking for drive-relative/drive-absolute paths is a bit more complicated. For more details, see the lengthy comment in `std.os.windows.getWin32PathType`
- `relativeWindows` will now almost always be able to return either a fully-qualified absolute path or a relative path, but there's one scenario where it may return a rooted path: when the CWD gotten from the PEB is not a drive-absolute or UNC path (if that's actually feasible/possible?). An alternative approach to this scenario might be to resolve against the `HOMEDRIVE` env var if available, and/or default to `C:\` as a last resort in order to guarantee the result of `relative` is never a rooted path.
- Partial UNC paths (e.g. `\\server` instead of `\\server\share`) are a bit awkward to handle, generally. Not entirely sure how best to handle them, so there may need to be another pass in the future to iron out any issues that arise. As of now the behavior is:
  + For `relative`, any part of a UNC disk designator is treated as the "root" and therefore isn't applicable for relative paths, e.g. calling `relative` with `\\server` and `\\server\share` will result in `\\server\share` rather than just `share` and if `relative` is called with `\\server\foo` and `\\server\bar` the result will be `\\server\bar` rather than `..\bar`
  + For `resolve`, any part of a UNC disk designator is also treated as the "root", but relative and rooted paths are still elligable for filling in missing portions of the disk designator, e.g. `resolve` with `\\server` and `foo` or `\foo` will result in `\\server\foo`

Fixes #25703
Closes #25702
2025-11-21 00:03:44 -08:00
Benjamin Jurk
4b5351bc0d
update deprecated ArrayListUnmanaged usage (#25958) 2025-11-20 14:46:23 -08:00
Alex Rønne Petersen
9ab7eec23e represent Mac Catalyst as aarch64-maccatalyst-none rather than aarch64-ios-macabi
Apple's own headers and tbd files prefer to think of Mac Catalyst as a distinct
OS target. Earlier, when DriverKit support was added to LLVM, it was represented
a distinct OS. So why Apple decided to only represent Mac Catalyst as an ABI in
the target triple is beyond me. But this isn't the first time they've ignored
established target triple norms (see: armv7k and aarch64_32) and it probably
won't be the last.

While doing this, I also audited all Darwin OS prongs throughout the codebase
and made sure they cover all the tags.
2025-11-14 11:33:35 +01:00
Andrew Kelley
8a1e6c8c39 std.Io: implement dirStatPath 2025-10-29 06:20:49 -07:00
Andrew Kelley
ebcc6f166c std.Io: bring back Timestamp but also keep Clock.Timestamp
this feels better
2025-10-29 06:20:49 -07:00
Andrew Kelley
47aa5a70a5 std: updating to std.Io interface
got the build runner compiling
2025-10-29 06:20:48 -07:00
Andrew Kelley
774df26835 WIP: hack at std.Io on a plane 2025-10-29 06:20:48 -07:00
Andrew Kelley
ea169e6ccf std.Build.Cache: clarify parameter is sub path, not basename 2025-09-24 20:01:19 -07:00
Andrew Kelley
9a0970a12b rework std.Io.Writer.Allocating to support runtime-known alignment
Also, breaking API changes to:
* std.fs.Dir.readFileAlloc
* std.fs.Dir.readFileAllocOptions
2025-08-30 00:48:50 -07:00
Andrew Kelley
79f267f6b9 std.Io: delete GenericReader
and delete deprecated alias std.io
2025-08-29 17:14:26 -07:00
Alex Kladov
7d63e777a4 fix memory leak
closes #24421
2025-07-16 18:34:34 +02:00
Andrew Kelley
5496901e71 std.Io.Reader.appendRemaining: add missing assert 2025-07-14 00:14:21 -07:00
Linus Groh
eb37552536 Remove numerous things deprecated during the 0.14 release cycle
Basically everything that has a direct replacement or no uses left.

Notable omissions:

- std.ArrayHashMap: Too much fallout, needs a separate cleanup.
- std.debug.runtime_safety: Too much fallout.
- std.heap.GeneralPurposeAllocator: Lots of references to it remain, not
  a simple find and replace as "debug allocator" is not equivalent to
  "general purpose allocator".
- std.io.Reader: Is being reworked at the moment.
- std.unicode.utf8Decode(): No replacement, needs a new API first.
- Manifest backwards compat options: Removal would break test data used
  by TestFetchBuilder.
- panic handler needs to be a namespace: Many tests still rely on it
  being a function, needs a separate cleanup.
2025-07-11 08:17:43 +02:00
Andrew Kelley
dce08d9a57 std.Build.Cache: remove debugging remnants
oops!
2025-07-09 20:01:15 -07:00