0
Fork 0
mirror of https://codeberg.org/ziglang/zig synced 2026-11-18 07:05:05 +00:00
zig/lib/std/zig.zig

937 lines
36 KiB

2025-07-15 21:04:00 -07:00
progress towards semantic error serialization Introduces std.zig.ErrorBundle which is a trivially serializeable set of compilation errors. This is in the standard library so that both the compiler and the build runner can use it. The idea is they will use it to communicate compilation errors over a binary protocol. The binary encoding of ErrorBundle is a bit problematic - I got a little too aggressive with compaction. I need to change it in a follow-up commit to use some indirection in the error message list, otherwise iteration is too unergonomic. In fact it's so problematic right now that the logic getAllErrorsAlloc() actually fails to produce a viable ErrorBundle because it puts SourceLocation data in between the root level ErrorMessage data. This commit has a simplification - redundant logic for rendering AST errors to stderr has been removed in favor of moving the logic for lowering AST errors into AstGen. So even if we get parse errors, the errors will get lowered into ZIR before being reported. I believe this will be useful when working on --autofix. Either way, some redundant brittle logic was happily deleted. In Compilation, updateSubCompilation() is improved to properly perform error reporting when a sub-compilation object fails. It no longer dumps directly to stderr; instead it populates an ErrorBundle object, which gets added to the parent one during getAllErrorsAlloc(). In package fetching code, instead of dumping directly to stderr, it now populates an ErrorBundle object, and gets properly reported at the CLI layer of abstraction.
2023-02-23 16:18:43 -07:00
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
2022-08-31 13:36:48 +03:00
2022-10-29 00:14:51 -07:00
2022-08-31 13:36:48 +03:00
2025-07-18 11:49:22 -07:00
2024-02-26 22:26:19 -07:00
2024-02-26 22:26:19 -07:00
2024-02-26 22:26:19 -07:00
2024-02-26 22:26:19 -07:00
stage2: blaze the trail for std lib integration This branch adds "builtin" and "std" to the import table when using the self-hosted backend. "builtin" gains one additional item: ``` pub const zig_is_stage2 = true; // false when using stage1 backend ``` This allows the std lib to do conditional compilation based on detecting which backend is being used. This will be removed from builtin as soon as self-hosted catches up to feature parity with stage1. Keep a sharp eye out - people are going to be tempted to abuse this. The general rule of thumb is do not use `builtin.zig_is_stage2`. However this commit breaks the rule so that we can gain limited start.zig support as we incrementally improve the self-hosted compiler. This commit also implements `fullyQualifiedNameHash` and related functionality, which effectively puts all Decls in their proper namespaces. `fullyQualifiedName` is not yet implemented. Stop printing "todo" log messages for test decls unless we are in test mode. Add "previous definition here" error notes for Decl name collisions. This commit does not bring us yet to a newly passing test case. Here's what I'm working towards: ```zig const std = @import("std"); export fn main() c_int { const a = std.fs.base64_alphabet[0]; return a - 'A'; } ``` Current output: ``` $ ./zig-cache/bin/zig build-exe test.zig test.zig:3:1: error: TODO implement more analyze elemptr zig-cache/lib/zig/std/start.zig:38:46: error: TODO implement structInitExpr ty ``` So the next steps are clear: * Sema: improve elemptr * AstGen: implement structInitExpr
2021-04-08 19:05:05 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
stage2: blaze the trail for std lib integration This branch adds "builtin" and "std" to the import table when using the self-hosted backend. "builtin" gains one additional item: ``` pub const zig_is_stage2 = true; // false when using stage1 backend ``` This allows the std lib to do conditional compilation based on detecting which backend is being used. This will be removed from builtin as soon as self-hosted catches up to feature parity with stage1. Keep a sharp eye out - people are going to be tempted to abuse this. The general rule of thumb is do not use `builtin.zig_is_stage2`. However this commit breaks the rule so that we can gain limited start.zig support as we incrementally improve the self-hosted compiler. This commit also implements `fullyQualifiedNameHash` and related functionality, which effectively puts all Decls in their proper namespaces. `fullyQualifiedName` is not yet implemented. Stop printing "todo" log messages for test decls unless we are in test mode. Add "previous definition here" error notes for Decl name collisions. This commit does not bring us yet to a newly passing test case. Here's what I'm working towards: ```zig const std = @import("std"); export fn main() c_int { const a = std.fs.base64_alphabet[0]; return a - 'A'; } ``` Current output: ``` $ ./zig-cache/bin/zig build-exe test.zig test.zig:3:1: error: TODO implement more analyze elemptr zig-cache/lib/zig/std/start.zig:38:46: error: TODO implement structInitExpr ty ``` So the next steps are clear: * Sema: improve elemptr * AstGen: implement structInitExpr
2021-04-08 19:05:05 -07:00
stage2: blaze the trail for std lib integration This branch adds "builtin" and "std" to the import table when using the self-hosted backend. "builtin" gains one additional item: ``` pub const zig_is_stage2 = true; // false when using stage1 backend ``` This allows the std lib to do conditional compilation based on detecting which backend is being used. This will be removed from builtin as soon as self-hosted catches up to feature parity with stage1. Keep a sharp eye out - people are going to be tempted to abuse this. The general rule of thumb is do not use `builtin.zig_is_stage2`. However this commit breaks the rule so that we can gain limited start.zig support as we incrementally improve the self-hosted compiler. This commit also implements `fullyQualifiedNameHash` and related functionality, which effectively puts all Decls in their proper namespaces. `fullyQualifiedName` is not yet implemented. Stop printing "todo" log messages for test decls unless we are in test mode. Add "previous definition here" error notes for Decl name collisions. This commit does not bring us yet to a newly passing test case. Here's what I'm working towards: ```zig const std = @import("std"); export fn main() c_int { const a = std.fs.base64_alphabet[0]; return a - 'A'; } ``` Current output: ``` $ ./zig-cache/bin/zig build-exe test.zig test.zig:3:1: error: TODO implement more analyze elemptr zig-cache/lib/zig/std/start.zig:38:46: error: TODO implement structInitExpr ty ``` So the next steps are clear: * Sema: improve elemptr * AstGen: implement structInitExpr
2021-04-08 19:05:05 -07:00
2022-08-18 18:58:28 -07:00
2022-08-18 18:58:28 -07:00
2022-08-18 18:58:28 -07:00
zig build system: change target, compilation, and module APIs Introduce the concept of "target query" and "resolved target". A target query is what the user specifies, with some things left to default. A resolved target has the default things discovered and populated. In the future, std.zig.CrossTarget will be rename to std.Target.Query. Introduces `std.Build.resolveTargetQuery` to get from one to the other. The concept of `main_mod_path` is gone, no longer supported. You have to put the root source file at the module root now. * remove deprecated API * update build.zig for the breaking API changes in this branch * move std.Build.Step.Compile.BuildId to std.zig.BuildId * add more options to std.Build.ExecutableOptions, std.Build.ObjectOptions, std.Build.SharedLibraryOptions, std.Build.StaticLibraryOptions, and std.Build.TestOptions. * remove `std.Build.constructCMacro`. There is no use for this API. * deprecate `std.Build.Step.Compile.defineCMacro`. Instead, `std.Build.Module.addCMacro` is provided. - remove `std.Build.Step.Compile.defineCMacroRaw`. * deprecate `std.Build.Step.Compile.linkFrameworkNeeded` - use `std.Build.Module.linkFramework` * deprecate `std.Build.Step.Compile.linkFrameworkWeak` - use `std.Build.Module.linkFramework` * move more logic into `std.Build.Module` * allow `target` and `optimize` to be `null` when creating a Module. Along with other fields, those unspecified options will be inherited from parent `Module` when inserted into an import table. * the `target` field of `addExecutable` is now required. pass `b.host` to get the host target.
2023-12-02 21:51:34 -07:00
zig build system: change target, compilation, and module APIs Introduce the concept of "target query" and "resolved target". A target query is what the user specifies, with some things left to default. A resolved target has the default things discovered and populated. In the future, std.zig.CrossTarget will be rename to std.Target.Query. Introduces `std.Build.resolveTargetQuery` to get from one to the other. The concept of `main_mod_path` is gone, no longer supported. You have to put the root source file at the module root now. * remove deprecated API * update build.zig for the breaking API changes in this branch * move std.Build.Step.Compile.BuildId to std.zig.BuildId * add more options to std.Build.ExecutableOptions, std.Build.ObjectOptions, std.Build.SharedLibraryOptions, std.Build.StaticLibraryOptions, and std.Build.TestOptions. * remove `std.Build.constructCMacro`. There is no use for this API. * deprecate `std.Build.Step.Compile.defineCMacro`. Instead, `std.Build.Module.addCMacro` is provided. - remove `std.Build.Step.Compile.defineCMacroRaw`. * deprecate `std.Build.Step.Compile.linkFrameworkNeeded` - use `std.Build.Module.linkFramework` * deprecate `std.Build.Step.Compile.linkFrameworkWeak` - use `std.Build.Module.linkFramework` * move more logic into `std.Build.Module` * allow `target` and `optimize` to be `null` when creating a Module. Along with other fields, those unspecified options will be inherited from parent `Module` when inserted into an import table. * the `target` field of `addExecutable` is now required. pass `b.host` to get the host target.
2023-12-02 21:51:34 -07:00
zig build system: change target, compilation, and module APIs Introduce the concept of "target query" and "resolved target". A target query is what the user specifies, with some things left to default. A resolved target has the default things discovered and populated. In the future, std.zig.CrossTarget will be rename to std.Target.Query. Introduces `std.Build.resolveTargetQuery` to get from one to the other. The concept of `main_mod_path` is gone, no longer supported. You have to put the root source file at the module root now. * remove deprecated API * update build.zig for the breaking API changes in this branch * move std.Build.Step.Compile.BuildId to std.zig.BuildId * add more options to std.Build.ExecutableOptions, std.Build.ObjectOptions, std.Build.SharedLibraryOptions, std.Build.StaticLibraryOptions, and std.Build.TestOptions. * remove `std.Build.constructCMacro`. There is no use for this API. * deprecate `std.Build.Step.Compile.defineCMacro`. Instead, `std.Build.Module.addCMacro` is provided. - remove `std.Build.Step.Compile.defineCMacroRaw`. * deprecate `std.Build.Step.Compile.linkFrameworkNeeded` - use `std.Build.Module.linkFramework` * deprecate `std.Build.Step.Compile.linkFrameworkWeak` - use `std.Build.Module.linkFramework` * move more logic into `std.Build.Module` * allow `target` and `optimize` to be `null` when creating a Module. Along with other fields, those unspecified options will be inherited from parent `Module` when inserted into an import table. * the `target` field of `addExecutable` is now required. pass `b.host` to get the host target.
2023-12-02 21:51:34 -07:00
2025-07-19 16:59:16 -07:00
2025-07-19 16:59:16 -07:00
2025-07-15 21:04:00 -07:00
2025-07-19 16:59:16 -07:00
2025-07-19 16:59:16 -07:00
2025-07-19 16:59:16 -07:00
2025-07-19 16:59:16 -07:00
2025-07-15 23:38:18 -07:00
2025-07-15 21:04:00 -07:00
2024-02-26 22:26:19 -07:00
2025-07-15 21:04:00 -07:00
2024-02-26 22:26:19 -07:00
2025-07-15 21:04:00 -07:00
2024-02-26 22:26:19 -07:00
2025-07-15 21:04:00 -07:00
2024-02-26 22:26:19 -07:00
2025-07-15 21:04:00 -07:00
2025-07-15 21:04:00 -07:00
2025-07-15 21:04:00 -07:00
2025-07-15 21:04:00 -07:00
compiler: rework emit paths and cache modes Previously, various doc comments heavily disagreed with the implementation on both what lives where on the filesystem at what time, and how that was represented in code. Notably, the combination of emit paths outside the cache and `disable_lld_caching` created a kind of ad-hoc "cache disable" mechanism -- which didn't actually *work* very well, 'most everything still ended up in this cache. There was also a long-standing issue where building using the LLVM backend would put a random object file in your cwd. This commit reworks how emit paths are specified in `Compilation.CreateOptions`, how they are represented internally, and how the cache usage is specified. There are now 3 options for `Compilation.CacheMode`: * `.none`: do not use the cache. The paths we have to emit to are relative to the compiler cwd (they're either user-specified, or defaults inferred from the root name). If we create any temporary files (e.g. the ZCU object when using the LLVM backend) they are emitted to a directory in `local_cache/tmp/`, which is deleted once the update finishes. * `.whole`: cache the compilation based on all inputs, including file contents. All emit paths are computed by the compiler (and will be stored as relative to the local cache directory); it is a CLI error to specify an explicit emit path. Artifacts (including temporary files) are written to a directory under `local_cache/tmp/`, which is later renamed to an appropriate `local_cache/o/`. The caller (who is using `--listen`; e.g. the build system) learns the name of this directory, and can get the artifacts from it. * `.incremental`: similar to `.whole`, but Zig source file contents, and anything else which incremental compilation can handle changes for, is not included in the cache manifest. We don't need to do the dance where the output directory is initially in `tmp/`, because our digest is computed entirely from CLI inputs. To be clear, the difference between `CacheMode.whole` and `CacheMode.incremental` is unchanged. `CacheMode.none` is new (previously it was sort of poorly imitated with `CacheMode.whole`). The defined behavior for temporary/intermediate files is new. `.none` is used for direct CLI invocations like `zig build-exe foo.zig`. The other cache modes are reserved for `--listen`, and the cache mode in use is currently just based on the presence of the `-fincremental` flag. There are two cases in which `CacheMode.whole` is used despite there being no `--listen` flag: `zig test` and `zig run`. Unless an explicit `-femit-bin=xxx` argument is passed on the CLI, these subcommands will use `CacheMode.whole`, so that they can put the output somewhere without polluting the cwd (plus, caching is potentially more useful for direct usage of these subcommands). Users of `--listen` (such as the build system) can now use `std.zig.EmitArtifact.cacheName` to find out what an output will be named. This avoids having to synchronize logic between the compiler and all users of `--listen`.
2025-06-06 20:16:26 +01:00